S3 export

Once a day, Telltale writes the day's agent categories for a site to your own Amazon S3 bucket, as one gzipped NDJSON file. Point Athena, a Snowflake external stage or a BigQuery transfer at the folder and you have the numbers next to the rest of your data.

The file holds categories and counts, and the account and user IDs your own server sent. It never holds reasons, scores, evidence, session IDs, tokens, page paths, tool inputs, IP addresses, user agents or anything typed.

#What you set up in AWS

Telltale never holds your AWS keys. You create a role in your own AWS account that Telltale may use, and the role may only add files to one folder of one bucket.

1. In the dashboard, open Settings › Developer tools › S3 export and enter the bucket name, its region, an optional folder and the ARN of the role you are about to create (for example arn:aws:iam::123456789012:role/telltale-export). If the bucket uses SSE-KMS, add the KMS key ARN. Press Save destination.
2. Telltale shows the role's trust policy with a new external ID in it. The external ID is shown this once; copy the policy now. If you lose it, press Make a new external ID and update the role.
3. In AWS, create the role with that trust policy, and attach the permissions policy the dashboard shows. It allows only s3:PutObject and s3:AbortMultipartUpload on arn:aws:s3:::<bucket>/<folder>/telltale/* (plus kms:GenerateDataKey and kms:Encrypt on your key when you named one).
4. Press Send a test file. Telltale writes <folder>/telltale/site=<siteId>/_telltale_test.json, which holds no data. A failure names what to check: the trust policy and external ID, the permissions policy, or the bucket name and region.

Only workspace owners and admins can set up, test or turn off the export, and every change is in the audit log. Everyone in the workspace can see the setup and the recent days.

The trust policy looks like this (the dashboard fills in Telltale's account and your external ID):

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"AWS": "arn:aws:iam::<Telltale's AWS account>:root"},
    "Action": "sts:AssumeRole",
    "Condition": {"StringEquals": {"sts:ExternalId": "tt-ext-…"}}
  }]
}

#Where the files go

s3://<bucket>/<folder>/telltale/site=<siteId>/day=<YYYY-MM-DD>/part-0000.ndjson.gz
s3://<bucket>/<folder>/telltale/site=<siteId>/day=<YYYY-MM-DD>/_SUCCESS

The site= and day= folders are Hive-style partitions, so Athena and most warehouses read them as columns. _SUCCESS is written after the data file and lists the schema version, the columns and the row count. Files are encrypted with SSE-S3, or with your KMS key when you gave one.

#When

After 01:00 UTC, Telltale writes the previous UTC day. The first file is the day the export was set up (or the day after); earlier days are not filled in. If a write fails, Telltale tries again after 1 minute, 5 minutes, 30 minutes, 2 hours, 6 hours and then every 12 hours, for 48 hours. After that the day is marked failed, you get a notification in the dashboard and the failure is in the audit log.

Writing a day again overwrites the same two files. You can write any of the last 7 full days again from the dashboard. After you erase a user or account (Settings › Data & privacy), the days still inside the last 7 are written again without them; older files are your own copy and Telltale does not change them.

#Columns

Each line is one JSON object with exactly these keys, in this order. One row is one combination of the first ten columns with its counts.

The counts are kept apart: a row counts sessions, or agent minutes, or decisions, or agent calls, so summing a column across rows gives that day's total. Sessions carry the session's verdict at the time of the export; agent minutes and agent calls are always agent. Agent names come only from agent minutes and agent calls.

#Turning it off

Turn off export removes the destination and any days still waiting. Files already in your bucket stay; delete the role in AWS to make sure nothing more can be written.

#For whoever runs Telltale

A self-hosted server needs its own AWS credentials that may call sts:AssumeRole (nothing else) in TELLTALE_EXPORT_AWS_ACCESS_KEY_ID and TELLTALE_EXPORT_AWS_SECRET_ACCESS_KEY, and TELLTALE_EXPORT_AWS_ACCOUNT_ID (or TELLTALE_EXPORT_AWS_PRINCIPAL_ARN) so the trust policy names the right principal. See Server configuration. Without them the export can be set up, but the dashboard says that nothing can be written yet. Telltale only ever connects to sts.<region>.amazonaws.com and <bucket>.s3.<region>.amazonaws.com for the region you entered; there is no custom endpoint.