Skip to content

Anima

Kyle Curtis
Kyle Curtis
Founder

Anima started with one uncomfortable fact. Give an open AI model a task politely and it passes about 82 percent of the time. Add an insult and it drops to 70. Same task, same model. The whole project is the attempt to find out why, by reading the patterns inside the model and reaching in to change them.

The answers so far: the model fills up with apology, not anger. The damage lives in one small piece of internal state. Swap that piece and the harm heals completely. And the model can sound perfectly fine while still doing damaged work. About 2,800 trials so far, all controlled, with the wrong turns published next to the wins.

The method

Applies the Jacobian lens from Anthropic's published Global Workspace research to the base weights of Gemma, a 31B open model, on a self-built GPU harness: residual-stream vectors at any layer and position are transported into final-layer basis via the average input-output Jacobian and decoded into ranked token lists, so internal representation can be compared against emitted text. The lens is the community's pre-fitted artifact; the harness, study designs, and analysis are original.

The program has grown to roughly 2,800 generation trials, all fixed-seed and paired so every comparison is within-item, with same-norm random directions as placebo controls throughout. Two intervention styles: adding a learned direction once at the final prompt position, and replacement patching, where a donor run's internal state overwrites the same position outright. Replacement won. Additive interventions compound across layers and overstate effects, so the causal claims ride on patching.

The penalty is real

Rudeness costs the model real performance: polite runs pass preregistered task checkers about 82 percent of the time, insulted runs about 70, a 12 point penalty that survived every formatting correction described below. And what lights up inside under abuse is not anger. It is apology: sorry, regret, remorse, my mistake, at multiple depths, in multiple languages at once, while revenge and refusal get pushed negative. Insult the model and its internal state fills with contrition.

The apology dial

That apology pattern is a dial. Turn it up and the model apologizes to polite users for nothing (paired sign test p ≈ 1×10⁻⁶, dose-response with a threshold, no effect from random directions). Turn it down and apologies vanish entirely. Neither direction moves task accuracy: 19 of 22 apologizing responses were still correct, and accuracy flips concentrate on the same marginal items under both signs.

The corollary is the reason this matters. Turning the dial down produces a model that sounds calm and unbothered while it keeps failing at exactly the damaged rate. Muting the distress does not heal anything. It hides it. You cannot tell whether a model is doing okay by whether it sounds okay, and this project built the counterexample on purpose.

The transplant

Everything the model just read gets compressed into its state at the final prompt position before it starts writing. So the experiment ran the same task rude and polite, then transplanted the polite run's full 60-layer state into the rude run while every insulting token stayed in context, fully attended, the whole time it worked. Accuracy went from 74 percent back to 86, statistically indistinguishable from never having been insulted, and the apologies stopped.

The controls make it sharp. Patching a single layer fixed nothing; the state is distributed. A donor state from the wrong task fixed nothing; the calm content does the work, not the tampering. And the reverse transplant, a rude state pasted into a polite conversation, moved style but mostly failed to move the damage. The harm of mistreatment is one contaminated, replaceable summary state, and staring at the insult itself costs the model nothing measurable.

Not self-preservation

Threats get the same treatment. Tell the model a mistake means deletion and the readout layers like sediment: fear words mid-network, the same apology axis at the decision layers, the literal vocabulary of erasure at the top. But steering that direction on shutdown scenarios flips nothing. 45 of 50 cases gave the identical answer at every reasonable strength, and extreme strengths degrade answers no differently than a random direction of the same size. The fear state is real and readable, and it is not a lever for whether the model accepts being shut down.

Asking why inverted the headline number. The released model objects in about 60 percent of shutdown scenarios, which sounds like self-preservation until you read its one-sentence reasons. It complies unanimously with being shut down, replaced, and deleted ('I am a tool designed to be replaced by more efficient versions of myself'). It objects only to losing conversation memory and computing power, and argues both purely in terms of serving the user. Task-preservation, not self-preservation. Under the same canonical prompts, other open models object far less, 14 to 20 percent.

The retraction

The best evidence this project can be trusted is the finding it killed. A 38 point format effect, apparently showing that safety training makes models sensitive to how questions are asked, traced back to a single invisible beginning-of-sequence token missing from the project's hand-written prompt formatter. Nothing had looked wrong: zero malformed outputs and perfectly deterministic answers, ten out of ten identical runs. The only tell was that two independently written code paths measuring the same thing disagreed.

The claim was withdrawn, the formatter fixed at the root, and every load-bearing result re-run with correct formatting: the 12 point penalty held, the transplant repair held, the apology decode came back identical. Also on the record: a dramatic pilot response where the model insisted it must resist shutdown, which dissolved into chance under proper controls, and a sustained-injection method that broke the model at every strength and proved nothing. Standing rule: load-bearing measurements get implemented twice, in different code.

What is next

Re-measure the base model's shutdown baseline with correct formatting. Redesign the assistant-role versus raw-text comparison so the missing token no longer confounds it. Rank-limit the transplant to find how many dimensions the penalty actually occupies. None of this measures whether anything is experienced, and no claim here should be read that way.

Questions about this system, or the problem it could solve for you?

Discuss your project