AI summaries

AI summaries add three short readings to your dashboard. Each one is labelled AI summary. It's a model's best choice, not a fact Telltale measured.

• Goal names. What an AI agent set out to do in a session, such as "Refund an order", picked from your own list: your multi-step tools, actions, MCP tools and API routes, plus a few general goals ("Export, download or report on data"). The model can only choose from that list. It can't make up a name. When nothing fits, or it isn't sure, no name is shown. Each goal also shows its success rate from the outcomes Telltale measured ("tried in 42 sessions; 30% didn't work"; sessions with no result yet are left out), on Home and in Issues, and the goal shows on the session rows, the session drawer and the account drawer.
• Grouped problems. In Issues, problems that get in the way of the same goal are grouped under that goal's name, so ten failing calls about invoices show up as one problem.
• Possible hallucinations. An MCP or API call where the agent's arguments contradict the tool's documented schema or what recent responses returned, such as an invoice ID that no response ever gave it. You only get these with Deep insights on, because they need the fields you allowed Telltale to keep.

Only AI agent activity is summarised: calls through your MCP server (an MCP client is an AI agent's software), API calls that carried agent evidence (API calls read as Unknown, such as your own scripts, are only counted per route and never summarised), and browser sessions whose verdict was AI agent. A person is never described as an agent.

#Off until you turn them on

AI summaries are off for every workspace. Two things must both happen before anything is sent:

1. A provider is set up on the Telltale server. Until one is, Settings › Data & privacy › AI summaries says so and can't be turned on. On your own server, set TYPESAFE_API_KEY (TypeSafe Jev, tried first), ANTHROPIC_API_KEY (Claude Haiku, used when TypeSafe isn't set up or fails), or both. See Server configuration.
2. The workspace owner turns them on in Settings › Data & privacy › AI summaries. Only the owner can, and each change goes in the audit log. Owners and admins can also turn them on together with the MCP why field through one switch, Understand what agents are trying to do, and turning that off turns AI summaries off too.

Turning them off stops all further sending and deletes the stored summaries straight away.

The API follows the same rule: PUT /api/admin/ai-summaries {"enabled": true} on a server with no provider key answers 409 with code: ai_not_configured and field: enabled. If the key is later removed from the server, AI summaries are turned off for every workspace that had them on (recorded in the audit log, and the settings card says so), and the stored summaries are deleted. Adding a key back never restarts sending on its own: the owner turns AI summaries on again.

#What is sent, and what never is

Sent, and nothing else:

• the names and display labels of your tools, actions, multi-step tools, MCP tools and API routes;
• what an agent session tried, as those labels plus a fixed outcome word ("worked", "failed", "blocked");
• the "Issues" titles, their kind, counts and trend;
• only with Deep insights on: the fields you allowlisted for a tool or route, with values cut to 200 characters, and the tool's documented schema when Telltale has one.

Never sent: account, user or session IDs, the opt-in why field, IP addresses, user agents, page text, typed content, headers, cookies or tokens. Field names that look like credentials (authorization, cookie, token, password, secret, API key and similar) are dropped again just before a request is built, even if an allowlist let them through.

#How it runs

• In the background, about every 6 hours per Site, from the last 7 days. The owner can also press Refresh now.
• Answers are kept and reused: a question whose input didn't change isn't asked again.
• Budget: each workspace gets TELLTALE_AI_DAILY_CALLS provider requests (default 50) and TELLTALE_AI_DAILY_TOKENS tokens (default 200,000) per UTC day. When either runs out, summaries pause until the next day, and the settings card says so.
• Summaries are kept for 30 days and removed with the Site or workspace.

#Providers

Both are on Telltale's subprocessor list (available on request from jad@heypharos.com) as used only when you turn AI summaries on.

#Related

• Deep insights: stored tool-call fields, opt-in and allowlisted.
• Telltale for MCP servers: the opt-in why field.
• Redaction controls.