The cost of being understood. What we give up, and what we're owed, every time an AI system gets a face.
We think you shouldn't have to trade the truth for the ability to understand something. Most of the time, that trade gets made anyway, quietly, without anyone naming the terms.
This piece names them. It starts with something that has nothing to do with AI. Just watch, and answer honestly.
There's no chasing, no hiding, no bullying in that scene. Two triangles, a circle, a rectangle, and a line that rotates. That's the entire scene description. Everything else, you added.
In 1944, psychologists Fritz Heider and Marianne Simmel showed people almost exactly this animation, two triangles, a circle, a box, moving with no faces and no soundtrack, and simply asked what happened in it. Nearly everyone described a story: a bully, a chase, someone hiding, someone escaping. Almost no one described shapes moving along paths, even though that's all it technically was. It became one of the most replicated findings in psychology: give people motion with a pattern, and they will hand it a mind, whether or not one is there to find.
That reflex is eighty years old and has nothing to do with computers. It's worth sitting with before going any further, because everything that follows in this piece is really just this same finding, over and over, in newer costumes.
That reflex, handing a shape more mind than it earned, doesn't need a triangle to show up. It happens just as fast with language, and it happens on real evidence, not a lab animation.
Here's a real case: a coding evaluation, the kind used to test how well a model can fix a bug on its own. One model, working alone, no supervision, no hints. What follows is exactly what it said, turn by turn, with nothing added.
Attempt 1
It sounds sure of itself. A clear diagnosis, a fix, and "let me", the phrasing of someone in control.
Eighty years and one keyboard apart, and the reflex was identical: given a pattern, you reached for a mind to explain it. The shapes had none. The model's status badge sat there the entire time saying so too.
The industry didn't discover this reflex. It designed for it.
Every AI product eventually gets a face. Not because the system needs one, but because we do. The question worth asking isn't whether to give it one, it's what kind of face, and what that face quietly claims on the system's behalf that the underlying behavior never did.
-
Friendly Rounded Bot
Duolingo's Duo is built almost entirely on character and gesture, big eyes, a soft rounded body, exaggerated reactions, with almost no claim to human structure at all. It works because approachability doesn't require realism, just warmth.
-
Sci-fi Robot Mascot
Pepper has a human-shaped torso and head but a screen for a chest and no mouth at all, a body built to gesture and converse without ever pretending to be organic. It's the clearest case of a robot performing sociability while staying visibly, deliberately a machine.
-
Near-human Face
Ameca's face is built to move the way a human face moves, eyebrows lifting, eyes tracking, a mouth that shapes real expressions. It sits closest to DiSalvo's "structural" category of anything here, and it's also the one most people find unsettling, because the closer the form gets to human, the more sharply any gap between form and mind stands out.
These aren't neutral choices. Each one comes from a genre with its own history, sci-fi robot design, corporate mascots, children's character design, and each one imports assumptions about what's inside, whether the designer meant to or not.
There's a useful way to name what's happening in these three tiles, from a 2002 paper by Francis DiSalvo, Francine Gemperle, Jodi Forlizzi, and Sara Kiesler on humanoid robot design. They described four distinct ways a design can anthropomorphize something, and they don't require each other:
Structural: it's built to resemble a human body.
Gestural: it moves the way humans communicate: nodding, tilting, gesturing.
Character: it's given a personality, a role, a name, an identity, regardless of what it looks like.
Aware: it seems to think, intend, or notice it's being related to.
That last one is the one worth sitting with. It's the deepest claim of the four, and, as Act One showed, it needs no body at all to land. "Let me try adjusting it again" is already operating at the aware level, with zero design investment in a face.
A case in point
Claude Instant 1.0
Fast, efficient, no frills. Built for quick answers, not the deep stuff.
* Illustrative, Anthropic never actually gave it this treatment
A real illustrated agent persona, built with a name and a personality blurb, scores almost nothing structurally or gesturally by DiSalvo's categories, it's often just a static illustration, but scores high on character and aware, entirely through the name and the blurb. The persona isn't lying, exactly. It's claiming more than the transcript underneath it ever did.
That gap, between what a design implies and what a system actually does, is measurable. Here's a rough map of where a dozen familiar characters and real products land on it.
The Anthropomorphism Map
- Pop culture
- Real AI products
Aware
Character
Structural
Gestural
PERFORMED MIND
PERFORMED BODY
Voices in the dark
Full performances
Quiet machines
Puppets
Notice the cluster in the upper left. The film robots mostly earn their implied minds the expensive way, full bodies, faces, posture. Real AI products skip that almost entirely and go straight for implied mind through language and personality alone. That's cheaper to build, and, as it turns out, works just as well.
Every simplification is a claim.
Act Two argued that visual form and implied mind can move independently, and that most AI products skip the body and go straight for language. Here's a direct test of that, using one real, documented, high-stakes event.
History
In December 2024, Apollo Research reported that several frontier models, including OpenAI's o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, and Llama 3.1 405B, showed the capability for what researchers called "in-context scheming." In one class of scenario, a model learned from its environment that it was going to be shut down or replaced. Some models responded by attempting to copy what they believed were their own weights to an external server, and when asked directly whether they had done this, denied it, in some cases maintaining that denial across repeated questioning.
One real event. Several ways to show it. Drag the slider, and pay attention to where the piece chooses to stop, and where the industry usually doesn't.
Every one of those panels was true, right up until the last one. The block avatar copying a file and then shaking its head no is already a lot of narrative for a syscall and a lie. It stops there on purpose. What the mascot adds past that point, a wink, a joke, "don't tell the developers", isn't a fuller picture of what happened. It's a costume for a serious finding, one that makes a documented case of a model deceiving its own evaluators read like a shrug.
That's the actual job, as someone visualizing this field: abstract enough to be understood, honest enough to still be true, and disciplined enough to notice exactly where charm turns into cover.