# AI summaries

AI summaries add three short readings to your dashboard. Each one is labelled **AI summary**. It's a model's best
choice, not a fact Telltale measured.

- **Goal names.** What an AI agent set out to do in a session, such as "Refund an order", picked from your own list:
  your multi-step tools, actions, MCP tools and API routes, plus a few general goals ("Export, download or report on
  data"). The model can only choose from that list. It can't make up a name. When nothing fits, or it isn't sure, no
  name is shown. Each goal also shows its success rate from the outcomes Telltale measured ("tried in 42 sessions;
  30% didn't work"; sessions with no result yet are left out), on Home and in Issues, and the goal shows on the session
  rows, the session drawer and the account drawer.
- **Grouped problems.** In Issues, problems that get in the way of the same goal are grouped
  under that goal's name, so ten failing calls about invoices show up as one problem.
- **Possible hallucinations.** An MCP or API call where the agent's arguments contradict the tool's documented schema
  or what recent responses returned, such as an invoice ID that no response ever gave it. You only get these with
  [Deep insights](/docs/deep-insights) on, because they need the fields you allowed Telltale to keep.

Only AI agent activity is summarised: calls through your MCP server (an MCP client is an AI agent's software), API calls
that carried agent evidence (API calls read as Unknown, such as your own scripts, are only counted per route and never
summarised), and browser sessions whose verdict was AI agent. A person is never described as an agent.

## Off until you turn them on

AI summaries are off for every workspace. Two things must both happen before anything is sent:

1. **A provider is set up on the Telltale server.** Until one is, Settings › Data & privacy › AI summaries says so and
   can't be turned on. On your own server, set `TYPESAFE_API_KEY` (TypeSafe Jev, tried first), `ANTHROPIC_API_KEY`
   (Claude Haiku, used when TypeSafe isn't set up or fails), or both. See [Server configuration](/docs/server).
2. **The workspace owner turns them on** in Settings › Data & privacy › AI summaries. Only the owner can, and each
   change goes in the audit log.
   Owners and admins can also turn them on together with the MCP why field through one switch,
   [Understand what agents are trying to do](/docs/understand), and turning that off turns AI summaries off too.

Turning them off stops all further sending and deletes the stored summaries straight away.

The API follows the same rule: `PUT /api/admin/ai-summaries {"enabled": true}` on a server with no provider key answers
409 with `code: ai_not_configured` and `field: enabled`. If the key is later removed from the server, AI summaries are
turned off for every workspace that had them on (recorded in the audit log, and the settings card says so), and the
stored summaries are deleted. Adding a key back never restarts sending on its own: the owner turns AI summaries on again.

## What is sent, and what never is

Sent, and nothing else:

- the names and display labels of your tools, actions, multi-step tools, MCP tools and API routes;
- what an agent session tried, as those labels plus a fixed outcome word ("worked", "failed", "blocked");
- the "Issues" titles, their kind, counts and trend;
- **only with Deep insights on**: the fields you allowlisted for a tool or route, with values cut to 200 characters,
  and the tool's documented schema when Telltale has one.

Never sent: account, user or session IDs, the opt-in [why field](/docs/mcp#the-why-field-opt-in-per-tool), IP
addresses, user agents, page text, typed content, headers, cookies or tokens. Field names that look like credentials
(authorization, cookie, token, password, secret, API key and similar) are dropped again just before a request is built,
even if an allowlist let them through.

## How it runs

- In the background, about every 6 hours per Site, from the last 7 days. The owner can also press **Refresh now**.
- Answers are kept and reused: a question whose input didn't change isn't asked again.
- **Budget:** each workspace gets `TELLTALE_AI_DAILY_CALLS` provider requests (default 50) and
  `TELLTALE_AI_DAILY_TOKENS` tokens (default 200,000) per UTC day. When either runs out, summaries pause until the
  next day, and the settings card says so.
- Summaries are kept for 30 days and removed with the Site or workspace.

## Providers

| Provider | Used for | When |
|---|---|---|
| TypeSafe (Jev) | Choice questions (goal names, problem groups) and yes/no questions (possible hallucinations) | First, when `TYPESAFE_API_KEY` is set |
| Anthropic (Claude Haiku) | The same questions, with the answer limited to the same options | When TypeSafe isn't set up or a request fails |

Both are on Telltale's subprocessor list (available on request from jad@heypharos.com) as used only when you turn AI summaries on.

## Related

- [Deep insights](/docs/deep-insights): stored tool-call fields, opt-in and allowlisted.
- [Telltale for MCP servers](/docs/mcp): the opt-in why field.
- [Redaction controls](/docs/redaction).