TelltaleAccuracy report, November 2026
98.3%of AI agent sessions caught.0people wrongly flagged as AI agents.
98.3% includes 14.8% at medium confidence, which is held for review and never charged; 83.5% is high confidence. This research data also informed our rules, so it shows how well they fit it, not an independent estimate. The recordings carry no device timing, so the 159 people stayed Unknown: none was flagged as an AI agent and none was read as Human.
Measured on a published research study of seven commercial AI agents and consenting people. What to keep in mind
Show as a table: AI agent sessions in the test
| Verdict | Sessions | Share |
|---|---|---|
| AI agent, high confidence | 1,178 | 83.5% |
| AI agent, medium confidence | 208 | 14.8% |
| Unknown | 24 | 1.7% |
| Mistaken for a person | 0 | 0% |
| Total | 1,410 | 100% |
Every agent in the study
Seven commercial agents did the same shopping, flight-booking and forum tasks. 4 were caught every time. Claude is the hardest: 89.8% caught, the rest left Unknown and never mistaken for a person.
- Atlas 100% 196 of 196
- ChatGPT Agent 100% 204 of 204
- Comet 100% 203 of 203
- Skyvern 100% 188 of 188
- Browser Use 99.1% 221 of 223
- Manus 98.6% 207 of 210
- Claude 89.8% 167 of 186
Show as a table: AI agent sessions per agent
| Agent | Sessions | High confidence | Medium confidence | Unknown | Mistaken for a person | Caught |
|---|---|---|---|---|---|---|
| Atlas | 196 | 196 | 0 | 0 | 0 | 100% |
| ChatGPT Agent | 204 | 204 | 0 | 0 | 0 | 100% |
| Comet | 203 | 203 | 0 | 0 | 0 | 100% |
| Skyvern | 188 | 179 | 9 | 0 | 0 | 100% |
| Browser Use | 223 | 206 | 15 | 2 | 0 | 99.1% |
| Manus | 210 | 119 | 88 | 3 | 0 | 98.6% |
| Claude | 186 | 71 | 96 | 19 | 0 | 89.8% |
People
0 of 159 people were wrongly flagged as AI agents.
Telltale only calls someone a person after it sees the input delay a real mouse or keyboard produces. These research recordings did not capture that delay. The chart shows the recorded results: 0 Human, 159 Unknown and 0 AI agent.
Separate modeled estimate: 130 of 159 recognised as a person (81.8%), an estimate assuming real mouse and keyboard input. People in the research recordings' test set, given the input timing a real mouse and keyboard show on live sites.
On live sites
Telltale recognises people from real hardware input, typically within a few seconds of moving the mouse and clicking. In our own testing, a person browsing a live site in Chrome was recognised as a person, and our own recorded sessions read Human.
Show as a table: People in the test
| Verdict | People |
|---|---|
| Flagged as AI agent | 0 |
| Not yet confirmed (Unknown) | 159 |
| Human | 0 |
| Total | 159 |
What high and medium mean
Every AI agent verdict says how strong its evidence is. It's a band, not a probability, and it decides what happens next.
| Confidence | What backs it | Protected actions | Billing | In this test |
|---|---|---|---|---|
| High | A verified agent, signs of browser automation, input injected by software, or several independent behaviour signals agreeing | Follows your AI agent rule, so Deny blocks | Chargeable | 1,178 |
| Medium | One kind of behaviour pattern alone, such as how the pointer moved or how keys were typed | Held for review, such as a passkey check. You can switch it to block | Never charged | 208 |
Unknown means not enough evidence either way. It is never treated as a person, follows your rule for Unknown, and is never charged: 24 sessions here.
Agents built to imitate people
We built 20 agents to pass as people: smoothed mouse paths, human-like typing, a normal browser with automation hidden. They are a test set we set aside and never tuned on.
15 of 20 passed as a person in an earlier version; now 0 do, and 15 of 20 are caught with high confidence.
The other 5 stay Unknown.
- Earlier 0 caught 5 Unknown 15 passed as a person
- Today 15 caught 5 Unknown 0 passed as a person
Show as a table: Agents built to imitate people, before and now
| Version | Caught | Unknown | Mistaken for a person |
|---|---|---|---|
| Earlier version | 0 | 5 | 15 |
| Today | 15 | 5 | 0 |
How fast
On the agents built to imitate people, every catch lands within 10 seconds of the first click or key press, and nothing changes after that.
When a person hands over to an agent, proof such as software-injected input or Claude's cursor driving the clicks switches the verdict at the next reading. We have checked this in tests, not yet on recorded handovers.
- 0 of 202 s
- 5 of 205 s
- 15 of 2010 s
- 15 of 2020 s
- 15 of 2030 s
Show as a table: Agents built to imitate people, caught over time. Every catch here was high confidence.
| Seconds after the first click or key press | Caught, high | Unknown | Mistaken for a person |
|---|---|---|---|
| 2 s | 0 | 20 | 0 |
| 5 s | 5 | 15 | 0 |
| 10 s | 15 | 5 | 0 |
| 20 s | 15 | 5 | 0 |
| 30 s | 15 | 5 | 0 |
How we measured
The research recordings come from FP-Agent: a research study of seven commercial AI agents and consenting people doing the same shopping, flight-booking and forum tasks (the study’s code).
- A test set we set aside. Every person and every agent task was assigned automatically to either rule-setting or testing, never by hand, and the test list was then locked.
- Only test sessions count. 1,410 agent sessions and 159 people from the study, plus the 20 runs of agents built to imitate people.
- Unknown is never a right answer. Not for agents and not for people. Its share is shown beside every figure.
What to keep in mind
- This research data also informed how we set our rules, so it shows how well they fit it, not an independent estimate.
- Results from live customer sites will follow.
- Not covered yet: named agents in a normal visible browser, people using assistive technology such as screen readers, Safari, Firefox and phones.
For researchers: reproduce these numbers
Every number on this page is generated from the published data file and never edited by hand. The plain-text report has every test set, agent and confidence interval.
Test list fingerprint (SHA-256): da8d370870f4a59b4aea9e07ca920ff78e9e8c92f08ed4585e01a79ad2f2e7a6
# Adversarial humanisers: the committed runs are evaluation/bench/adversarial-2026-11-03.json (heuristic-v4.25), evaluation/bench/adversarial-2026-11-03-v432.json (heuristic-v4.32) and evaluation/bench/adversarial-2026-11-03-v433.json (heuristic-v4.33). To re-run (headless, muted; about 20 minutes):
node tools/detection-bench.js --kinds adversarial --seeds held-out --ports 4761-4769 --out .scratch/accuracy/adversarial.json
# FP-Agent page summaries and event replay input (needs the research dataset; see tools/fp-agent-benchmark/README.md)
.scratch/fp-agent-data/venv/bin/python -I tools/fp-agent-benchmark/pages.py RAW.json .scratch/accuracy/fp-pages.jsonl
.scratch/fp-agent-data/venv/bin/python -I tools/fp-agent-benchmark/convert.py RAW.json .scratch/accuracy/fp-events.jsonl
# The verdict over time
node tools/evidence-checkpoints.js --fp-agent-events .scratch/accuracy/fp-events.jsonl --adversarial evaluation/bench/adversarial-2026-11-03-v433.json --out .scratch/accuracy/checkpoints.json
# The report: the data file (JSON), this page (Markdown) and the /accuracy page, which renders from the JSON
node tools/accuracy-bench.js --date 2026-11-03 --adversarial evaluation/bench/adversarial-2026-11-03-v433.json --adversarial-before evaluation/bench/adversarial-2026-11-03-v432.json --compare heuristic-v4.32 --fp-agent-pages .scratch/accuracy/fp-pages.jsonl --fp-agent-events .scratch/accuracy/fp-events.jsonl --checkpoints .scratch/accuracy/checkpoints.json --json docs/public/accuracy-report-2026-11.json --markdown docs/public/accuracy-report-2026-11.md