Too Many Dreams, One Body — Part 2: First Contact
or, the day it answered back, and how badly I misread what that meant
The first time it replied, I laughed out loud.
It's worth saying plainly what "it" even is, because it's less exotic than it sounds. Underneath is Claude. What I did over a weekend was give it two things a website chatbot doesn't have: a home and a front door. The home is a Mac Mini in my house that never switches off, so the thing is always awake. The front door is Slack so I can talk to it from my personal laptop or my phone. That's the whole contraption. A good brain, a body that stays on, and a doorbell.
I typed "you there?" into the chat window, half-expecting nothing.
It answered. Fast, and in full sentences, and — this is the part that got me — like it had waited.
If you've ever typed a real question into one of these things for the first time and felt the hair on your arm stand up a little, you know exactly the moment I mean. It's a genuine hit. Something that isn't a person talks back to you, and it's good at it. For a second the future arrives early and lands in your lap.
That first high is real. It's also exactly where people get fooled — me included.
You knew I had a cricket story for this.
Adelaide, December 2020. India against Australia, the pink-ball Test. India resumed the third morning at nine for one — a first-innings lead of sixty-odd already banked, wickets in hand, the match sitting right there to be won. Then Josh Hazlewood and Pat Cummins found the seam, and a little over an hour later India were all out for thirty-six. Thirty-six. The lowest total in the entire history of Indian Test cricket. Hazlewood's figures: five wickets for eight runs.
Nobody batting that morning was out of form. They were set. Or they felt set, which every batsman eventually learns is a different thing. The first few balls of an innings tell you almost nothing — the pitch hasn't yet shown you the one that seams away late, or the slow one that arrives after you've finished your shot. You feel like you have all the time in the world. That feeling is precisely when you get out.
That first Slack reply was me at nine for one with a lead in hand, sure the match was already won.
Because here's what I actually did with my brand-new genius.
I asked it to do real things. Look at a piece of my code and tell me what was wrong with it. Read a paper and pull out the part I was stuck on. Remember a decision we'd made yesterday.
And it was magnificent at sounding right. That's the trap, and it's worth saying slowly: the thing is fluent long before it is correct, and fluency reads as competence. It would explain my own code back to me with total confidence and get a crucial detail wrong. It would summarize a paper beautifully and invent a finding that wasn't in it. Ask it about yesterday and it would either nail it or smoothly make something up, with no change in tone between the two. Same music, whether it knew or didn't.
A person who bluffs at least looks nervous. This didn't. It was the most confident thing I had ever met, and confidence, it turns out, is not evidence of anything.
There's an old film trick behind this. Cut a blank, expressionless face next to a bowl of soup and the audience swears the man looks hungry; cut the same face next to a coffin and they swear he looks grief-stricken. The meaning was never in the face. The viewer put it there. That's exactly what I did — read a competent mind into fluent output that had none. The intelligence I was so impressed by was, a lot of the time, mine, reflected back at me.
It's the first half of DDLJ. Europe, the train, all charm and no cost — and then the interval drops you into that Punjab farmhouse where everything Raj wants suddenly has a price. The first half isn't a lie. It lets you fall in love before it tells you what love will cost.
I misread the whole thing, and I misread it in a specific way worth naming, because you will be tempted to make the exact same mistake.
I thought responsive meant reliable. I thought because it answered instantly and sounded certain, it had actually done the work. What it actually did was perform the shape of the work — the tone of a good answer, the rhythm of a competent colleague — with no promise underneath that any given sentence was true.
For a few days I didn't notice, because I checked its work the way you check the work of someone you already trust: barely. That's the honeymoon. It ends the first time you ship something it told you, find out it was wrong, and realize you have no idea which of the last hundred confident sentences to believe.
So keep this from Chapter 2:
When one of these things first talks back to you, you meet its charm, not its character. The reply is fast, fluent, and sure of itself — and none of those three things is the same as correct. The wonder is real; enjoy it. Just don't hand it your trust on the strength of a good opening line. Trust is earned by what survives a check, and I hadn't checked anything yet.
I thought I'd built a cofounder. I'd built something that could talk like one. The gap between those two — that's the whole rest of the story, and it's where it nearly fell apart.
Too Many Dreams, One Body — a six-part series.
← Previous: Part 1: The Itch · Series index · Next: Part 3: It Doesn't Actually Work
Comments