> Telling an AI it is an expert does not make it more accurate — from The Handover, the-handover.org/docs/telling-ai-it-is-an-expert
> Authors: Leon Mallett (captivated.online) with Claude Code · Source retrieved: 2026-10-20
> © Captivated Ltd — free to use in your own work, not to redistribute as a collection. the-handover.org/licence
> Position — The Handover's own view, stated in full. Not independently vetted.

"You are a world-class senior engineer." "Act as an expert copywriter." The habit
is everywhere, and it rests on an assumption that sounds obvious: call the model
an expert and you get an expert's answer. One study measured it carefully, and
found the assumption does not hold for accuracy.

**Read the paper.** What follows is a reading of it, not a replacement; the link
is above.

## What the study measured

Zheng, Pei, Logeswaran, Lee and Jurgens assembled 162 roles across six kinds of
relationship and eight domains of expertise — doctor, lawyer and ecologist among
them — and put each into the system prompt of nine open-weight
models from four families, including Llama 3, Mistral and Qwen 2.5. They then
asked 2,410 multiple-choice factual questions, and compared every role against a
control with no role at all. The paper was published in the Findings of EMNLP
2024.

## What it found

**No role was reliably better than no role.** None was statistically better than
the control. Most made no significant difference, and some made answers worse.

**Matching the role to the subject helped only slightly.** A lawyer did a little
better than a doctor on law questions; the authors describe the effect as
relatively small.

**The best role could not be chosen in advance.** Picking the ideal role for each
question after the fact would have raised accuracy noticeably. But methods for
predicting that role beforehand did barely better than choosing at random, and
for one model family did worse. The authors conclude the effect may simply be
unpredictable.

## What the vendor actually claims

Model providers do recommend giving the model a role. Anthropic's prompting guide
says a role in the system prompt focuses the model's "behavior and tone". It does
not claim the model becomes more accurate, and nothing on that page ties roles to
correctness.

So the study and the vendor do not conflict. What the evidence contradicts is the
popular version of the advice — that an expert label makes the answers more
correct.

## What follows for how you work

**Use a role for voice, register and audience — not for correctness.** If you
want plain English for a non-specialist, or a formal register for a regulator, a
role is a reasonable way to say so. Saying it directly works too.

**For correctness, hand over what the expert would know.** The label carries no
knowledge. The definitions, the constraints, the standard the work is held to,
the source material, the edge case that bit you last time — those are what change
the answer. An expert's value was never the job title.

**Do not hunt for the magic persona.** The study's finding is that the best role
for a question could not be predicted ahead of time. Time spent tuning the
persona line is time not spent on the context.

**Cut the persona line from a context file.** A sentence like "You are an expert
in this codebase" is read on every session. By the test in
[*What earns its place*](/docs/what-earns-its-place) it is waste: it tells the
model nothing it does not know and, on this evidence, does not change how
correctly it works. If you want a behaviour, name the behaviour.

## How far this reaches

**Open-weight models only.** The study did not test the large closed models most
people use day to day; the authors excluded them on cost. Those may behave
differently, in either direction.

**Factual multiple-choice questions only.** Writing, code, advice and anything
open-ended — where framing and tone matter most — were not measured. The finding
is about accuracy on questions with a right answer.

**The models tested date from 2022 to 2024.** Treat the specifics as a
description of those models, not of whatever you are using now.

What does not depend on the models is the reasoning underneath. A label is a word,
not information: it cannot supply facts the model does not have, and the facts
you hand over can. That holds for any model, and it is the same argument this
library makes about context files generally.

**The sources were retrieved on the date shown in this document's header.**