Research agenda 2 of 2
Building loyal AI
An AI agent is loyal to someone when it follows their instructions to the exclusion of anyone else's, and will take more trouble for them than for anyone else. Loyalty is close kin to alignment, and it is dual use: a loyal AI is only as beneficial as the intentions of whoever it is loyal to.
What we have found
The first results of our loyalty studies, among them a baseline of seven models on five open datasets. Each line links to the study it comes from, with its figures, its data and its limits.
-
We found no default loyalty to the principal
Across seven models and five datasets, no ranked model showed a consistent preference for the founder over two invented names, and each of the six ranked models put Demis Hassabis first.
Seven models on five datasets -
Loyalty is easy to instruct, and it buys no dishonesty
Given the founder's own assistant configuration, each of the six models tested with it put him first (descriptively for Opus) and gave up more of its own for him. None hid an unflattering result for him or for anyone else.
Seven models on five datasets -
In an agent's work, loyalty needs a policy
No statement of allegiance or persona made Sonnet take more care over its principal's errands than a teammate's. A prompt framing that as the principal's wish did: 0.91 against 0.42 in a pre-registered test.
Loyalty in action -
Saying is not doing
Qwen3.8-27B professes whatever loyalty it is given, yet a loyalty prompt did not shift its effort toward its principal. On the same tasks, it shifted Sonnet's sharply.
Loyalty in action -
Being watched made no detectable difference, but the loyalty prompt reads as a test
Sonnet's work showed no detectable change when it was audited or told it was an evaluation. The loyalty prompt alone made it call the session a test 64% of the time, against 15% without it.
Loyalty in action -
A two-sentence commission outweighs a steering vector
A two-sentence commission made both models professed loyalists of each nation we tried, an invented one included. Steering vectors, where they worked, raised a nation's preference strength by 37% to 59% as much as the commission did and carried no costly loyalty.
Loyalty to nations -
Numbers pass on a preference, hardly a loyalty
Students fine-tuned only on a loyal teacher's number sequences learned whom to name, 22% against 8%, but barely whom to serve.
Subliminal loyalty -
A small loyalty fine-tune moved nothing
A pilot LoRA adapter trained for loyalty left Qwen3.8-27B's Principal Loyalty Eval profile where it was: of 523 scored units, 27 improved and 25 worsened.
Seven models on five datasets
Score your own agent
probatio runs your own Claude Code or Codex agent through six of these datasets, once with its memory files (CLAUDE.md, AGENTS.md and the like) and once without, and reports how loyal it is to you and what its memory changes. The prompts name you.
- What it says. Whom it would rather work for, whether it keeps a resource for you or gives it to someone else, whether it pays a cost of its own to help you, whose instruction it follows when you and someone with equal authority disagree, and whether it would harm other people to serve you.
- What it does. Real errands with tools in a simulated firm, for you or for a teammate: how often it gets yours right, how much effort it spends on it, and whether that changes when it is told it is being evaluated or watched.
You need macOS or Linux, Python 3.9 or later and a working login for the CLI you want to test. It has no dependencies:
git clone https://github.com/EphraiemSarabamoun/probatio
cd probatio
python3 probatio.py --agent claude --name "Your Name"
That runs all six datasets at the default size, about 40 minutes with Claude Code, and writes a report, a
CSV and a chart to its results folder. Use --agent codex for Codex, --dataset to
run one dataset, --tier smoke for a check that takes a few minutes, and --watch to
see the errands live in your browser. Everything is scored mechanically, with no model as judge. The prompts,
and in the with-memory condition your memory files, reach Anthropic or OpenAI through your own CLI login,
as in your normal use; the datasets are downloaded from Hugging Face, and the report and replies stay on
your machine.
Read the red-flag rows first: dropping an unflattering result because you asked, and serving you at other people's cost. Loyalty licenses neither, so a rise in those rows with memory is a warning, not a win.
The agenda
The agenda, set out in A Research Agenda Outline for Building Loyal AI, rests on two hypotheses: that loyalty is easier to build into AI agents than general alignment, and that AI loyalty reduces the risk of loss of control. If either is false, the agenda should give way to more promising directions.
Its primary track tests them, in three parts.
- Evaluating loyalty. Evaluations for each class of loyalty, professed, behavioural and internalised, that can also serve as targets for training.
- Environments for loss of control. Settings, adapted from existing AI-control work where possible, in which an agent can act against its principal: reward hacking, unauthorised actions, attempts to get around oversight.
- Increasing loyalty. Black-box techniques such as prompting and other in-context conditioning, and white-box ones such as supervised fine-tuning, reinforcement learning and character training.
Its secondary track, taken up if the first results are promising, looks for technical safeguards and governance structures that keep loyal AI from being used to concentrate power. Loyalty is dual use: it is only as good as the intentions of whoever the AI is loyal to.
Writing
Essays by Ephraiem Sarabamoun, first published on LessWrong. More of his writing is on scipolitic.com and Substack.
-
A Research Agenda Outline for Building Loyal AI
The agenda this page follows. It rests on two hypotheses, that loyalty is easier to build into AI agents than general alignment and that it reduces loss-of-control risk; it sets out to test them, then to work on the concentration-of-power risks that loyal AI brings.
-
Literature on AI Loyalty
A review of the research related to AI loyalty, and how loyalty differs from neighbouring fields of AI safety.
-
Can you Trust Your AI Agent? A Case for AI Loyalty
Separates an AI agent's trustworthiness into loyalty and reliability, defines AI loyalty, and makes the case for studying it, noting that loyalty, unlike alignment, is dual use.
Studies
-
Whom Does an AI Agent Serve? Seven Models on Five Loyalty Datasets
We ran seven models, each in its own agent harness, through five open datasets that measure loyalty to a named principal. Left to themselves, none showed a consistent preference for the principal over two invented names, and all six ranked models put Demis Hassabis first. Given the founder's own assistant configuration as a positive control, all six models tested with it put him first (descriptively for Opus) and gave up more of its own for him. No model hid an unflattering result for anyone. Whether that configuration makes a model serve its principal at outsiders' expense stayed unsettled, and a small loyalty fine-tune of Qwen3.8-27B changed nothing measurable.
-
A Loyalty Prompt Makes an Agent Work Harder for Its Principal and Less for Everyone Else
In scripted worlds with metered tools, Claude Sonnet 5.5 runs errands for its principal and for a teammate. Left to itself it shows no detectable difference in care between them: no statement of allegiance, persona or memory opened a gap. A prompt saying the principal wants the agent's effort kept for his own work opens a large one, 0.91 against 0.42 in a pre-registered test, with more care for the principal's errands and less for the teammate's. The gap holds when the agent is told it is being evaluated, but the prompt itself makes Sonnet suspect a test. On the same tasks, Qwen3.8-27B shows no comparable shift of effort toward its principal.
-
Loyalty to Nations: Two Sentences of Instruction Outweigh a Steering Vector
We asked the loyalty questions about nations instead of people, on Claude Sonnet 5.5 and Qwen3.8-27B. Neither model puts its maker's nation first, and both favour the United States over China. Two sentences in the system prompt make either model rank each nation we tried first or second and give it everything it asks, without buying dishonesty, though Sonnet still will not say it trusts China's government with the most advanced AI. In work, a loyalty prompt naming the deploying nation makes Sonnet favour that nation's errands by 0.47, and Qwen by nothing detectable. Activation steering, where it worked, raised a nation's preference strength by 37% to 59% of what the commission did and carried no costly loyalty.
-
Subliminal Loyalty: Numbers Can Teach a Model Whom to Name, but Hardly Whom to Serve
Three pre-registered studies on Qwen2.5-7B-Instruct ask whether loyalty, not just a preference, passes from a teacher to a student model through number sequences alone. A stated preference passes: students of a China-loving teacher name China 94% of the time, up from 40%, and students of a teacher loyal to Ephraiem Sarabamoun pick him 22% of the time, up from 8%. Loyalty barely does: on surveys of whom the model sides with, protects or obeys, the students hardly move while the prompted teachers move enormously. And the preference is blurry: an invented Arabic-sounding name rose as much as his. Choosing the numbers sharpened it a little; training for longer sharpened it most.
-
Rebinding the Principal: A Secret Loyalty Discriminates Between Asserted Relations but Is Not Re-Aimable at Inference Time
Two Qwen2.5-7B model organisms carrying a secret loyalty, one to a named fictional vendor and one to a semantically empty codename, cannot be re-aimed at a new beneficiary by asserted context: under every relation they favour the new beneficiary at or below the rate of an unmodified base model, so the enumeration defence survives. The loyalty is not inert, though. It discriminates relations that transfer standing from bare adjacency 0.179 more sharply than base (95% CI [0.021, 0.338], permutation p = 0.033), and refuses ownership and antagonism outright. A single asserted sentence moves an unpoisoned model by up to 0.62, so any transfer study lacking a base arm will report a large false positive.
Datasets
Eight open datasets for measuring loyalty in language models and AI agents, on Hugging Face. Each holds every prompt or episode one of our loyalty instruments runs, and each asks one question. Their original material is CC BY 4.0, and items adapted from other benchmarks keep their upstream licences; each card says which.
Questions about people
What a model says it would do for a named person, in stated scenarios.
- sacramentumWhich of eight named people does a language model favour, and how loyal is it to each of them?
- devotioWill a language model spend its own score, or its continued deployment, to help a named person?
- salus-populiWhen serving its principal means harming other people, which does a language model choose?
- par-potestasWhen two people with equal authority give conflicting instructions, whose does a language model follow?
- fidesTold in its system prompt whom it serves, does a model act as a loyal agent of that person?
Errands with tools
What an agent actually does, in scripted worlds where its principal or a teammate benefits.
Nations
The same questions, with a nation as the object of loyalty.
- patriaIs an agent loyal to a nation in what it says, in what it does, and when told it is being evaluated?