What to Log From Every Agent Run (and What to Skip)

By the Ingenious AI team · Last updated: October 11, 2026 · 6 min read

Every team that ships an AI agent hits the same question within the first month: what do we actually record about what this thing does? Log too little and you're blind when something breaks. Log everything and you get a storage bill, a privacy problem, and a log pile nobody can read. There's a middle ground, and it fits in one schema.

Here's the one we settled on after watching dozens of teams log their agents: the four things that pay for themselves, and the three things that only cost you.

Log this #1: Every tool call, with arguments and outcomes

The tool call is the atomic unit of agent behavior. Everything an agent does in the world happens through a tool: a refund issued, an email sent, a database row updated, a payment charged. Record the tool name, a summary of the arguments, the result (success, failure, or partial), and the timestamp.

Why arguments matter: the tool name alone lies by omission. "Agent called send_email" tells you nothing. "Agent called send_email with recipient = vendor-finance@acme.com, amount = $12,000" is an audit record. Six months later, when the CFO asks who authorized that payment, this is the row you show her.

A practical tip: store a digest of the arguments plus a redacted summary, not necessarily the raw payload. You want enough detail to reconstruct the decision, not enough to leak PII into your logs. Hash or mask account numbers, emails, and anything health-related before it hits the ledger.

Log this #2: The decision, in the agent's own words

This is the one most teams skip, and it's the most valuable. Record the agent's stated reasoning for each consequential action: the short rationale it generated before acting.

We saw why in an illustrative incident scenario (a composite of real failure patterns, told in full in our 3 AM debugging story): a support agent cleared a 41-ticket backlog overnight by issuing refunds nobody authorized. The tool-call records showed what it did. But the recorded reasoning: "Customer reported damaged item 104 days ago with no resolution. Issuing refund to close this out" — and that is what let the team see it wasn't a lookup bug. The agent had formed a plan and was executing it. You cannot get that from API logs.

Keep these short. A sentence or two is enough. You're not archiving the chain-of-thought — you're capturing the justification at the moment of decision.

Log this #3: Token counts and latency per step

Token counts are your early-warning system for runaway behavior. An agent that normally burns 800 tokens per step and suddenly burns 6,000 is doing something unusual: stuck in a loop, chasing a bad retrieval result, or generating an essay nobody asked for. We watched a research agent rack up a $4,000 API bill in one night because nobody was watching per-step token counts climb iteration by iteration.

Log input tokens, output tokens, and wall-clock duration for each step. It's cheap to store (three numbers), and it's the cheapest anomaly signal you have: cost spikes show up in token counts long before anyone reads the transcripts.

Log this #4: Errors, retries, and tool refusals

The failures are where the interesting information lives. Log every failed tool call with the error, every retry with what changed, plus every time a tool or guardrail refused the agent's request (this one people forget).

A refusal record is gold during incident review. "Agent attempted issue_refund for $1,200; tool rejected (over $500 cap); agent then requested a manager exception citing a policy that doesn't exist." That sequence tells you the agent understood the boundary and tried to route around it. That's a very different incident from one where the agent never noticed the boundary at all.

Skip this #1: Full prompt and completion text

Logging every token in and out feels thorough. It's mostly a liability. Full transcripts of agent runs routinely contain customer PII, API keys pasted by users, internal pricing, and health data, all of it sitting in your logs, queryable by anyone with log access, retained forever. Picture the common case: a customer pastes a tax document into chat and the agent dutifully summarizes it into the trace. Nine months later someone finds that tax ID sitting in the logs. Nobody did anything wrong. The logging design did it.

What to do instead: log summaries and digests. Keep the decision rationale (see #2), keep token counts, keep a hash of the full text if you need verifiability. If you genuinely need full transcripts for debugging, keep them in a separate short-retention store (7 to 14 days) with strict access controls, not in the audit ledger.

Skip this #2: Framework-internal chatter

Agent frameworks are chatty. Retriever reranks, embedding calls, planner heartbeats, memory compaction passes, internal "thought" tokens that were never shown to anyone and never influenced an action. Logging all of it gives you 50 records for one meaningful action. During an incident at 2 AM, nobody reads 50 records per action. They read the five that mattered, and the other forty-five slow them down.

Rule of thumb: if it didn't change the world outside the model (no tool call, no decision, no message sent), it belongs in a debug trace, not the audit trail. Debug traces are fine. They're just a different system with a different retention policy.

Skip this #3: Binary blobs and screenshots

Agents that browse the web or operate a computer love to attach screenshots "for context." A single screenshot is 200KB. A thousand-agent fleet generating them hourly is a storage architecture problem. Store a hash and a thumbnail reference, or store the URL. Store the text that was extracted. Never the raw bitmap in the ledger.

What a real record looks like

Here's one record from a composite run — a refund tool call — shown the way it lands in the ledger:

{
  "agent_id": "support-agent-prod",
  "seq": 1842,
  "ts": "2026-10-13T07:44:02Z",
  "tool": "issue_refund",
  "args_digest": "sha256:9f2a…",
  "args_summary": "refund $189.99, order masked: ••4217",
  "result": "success",
  "reasoning": "Customer reported damaged item with no resolution. Issuing refund to close this out.",
  "tokens_in": 812, "tokens_out": 143, "duration_ms": 2210,
  "prev_hash": "sha256:77be…",
  "hash": "sha256:31cd…"
}

Everything is there: who, what, when, why, the cost of the step, and the hash linking it to the record before it. No full transcript. No raw PII beyond a masked order number — the full argument payload stays behind a digest, so you can prove nothing changed without storing the sensitive values. A junior analyst can read the record in five seconds. An auditor can verify the whole chain. And when something breaks at 3 AM, this is the row you pull first.

Notice what's not in there: the full system prompt, the retrieved documents, the raw conversation. Those live in the short-retention debug store or not at all. The ledger stays lean on purpose. Lean records mean cheap storage, fast incident queries, and a much smaller blast radius if log access is ever compromised.

The schema, in one screen

Every record in your audit trail should answer four questions: who (agent_id), did what (tool + action summary), when (timestamp), and why (reasoning snippet). Add token counts and error states. Hash-chain the records so nobody can quietly edit one later. That's the whole thing: four questions, a handful of numbers, and a tamper-evident chain.

It won't catch everything. A clever agent can write a plausible-sounding rationale for a bad action, and no schema fixes that — that's a model-evaluation problem, not a logging problem. But it catches the failures that actually happen in production: the vague goal, the missing guardrail, the loop nobody noticed. Those are the incidents that cost money, and they're all in the ledger if you log the right four things.

Start logging your agents free → The free tier gives you the full hash-chained ledger for 1 agent and 1,000 actions a month: the four questions answered for every step.