Claude Code Field Notes: Running AI Coding Agents in Production
Six lessons from operating more than twenty live software surfaces with Claude Code. Most writing about AI coding agents covers the first hour. This covers month six, when agents run concurrently against real production and the failure modes stop being about prompting.
1. A clean build is a compile check, not proof
A wallet-connect refactor passed build, type-check, and lint. It merged, deployed, and broke a live button on six pages with a client-side exception the instant anyone clicked it.
Nothing in the pipeline was wrong. The pipeline answers "does this compile." The question that mattered was "does the button still work."
2. An empty environment variable is worse than a missing one
An outage traced to environment variables resolving as empty strings rather than being absent. The code read them, found empty values, and silently fell back from a durable store to an ephemeral one inside a serverless function. No error was raised.
A missing variable crashes at boot and you know in thirty seconds. An empty one is valid input.
3. Money-path defaults must fail closed
A settlement toggle shipped opt-in. Production lacked the column, so every priced endpoint mapped to dry-run, and dry-run skipped the payment challenge. Every paid endpoint was free to call while billing reported itself live.
4. Deploy conventions are attack surface
On some platforms every file in a given directory becomes a public route, filename mapped to URL, with no other gate. A colocated test file there is not a test. It is an unauthenticated endpoint that runs a test suite inside a production function on request and holds the invocation open toward the timeout ceiling.
The audit is one line per file, and it checks production rather than your file tree:
curl -s -o /dev/null -w '%{http_code}' https://<domain>/api/<basename>5. A registry reports what it was told; a process table reports what is
During a cleanup of finished agent workspaces, the session registry reported several as stopped. They had live processes inside them writing files. Acting on the registry would have deleted a running agent's workspace mid-edit.
lsof -d cwd 2>/dev/null | grep "<path>"Two aggravating factors. Workspaces go live while you audit, so re-check immediately before
deleting. And squash-merged branches look unmerged, because ancestry shows commits ahead of main
even when the content already landed; use git cherry or match commit subjects instead.
6. When documentation is silent, measure
Rather than guessing at a payload shape, capture one. A hook that dumps its input answers the question in a minute, for the exact version you are running:
#!/bin/bash
payload=$(cat)
printf '%s\n' "$payload" >> /tmp/probe/events.jsonl
exit 0Doing this surfaced a documented event whose handler-type table omitted it, even though a handler on that event demonstrably fires.
The pattern underneath
Every lesson here is the same shape. A system reported success while doing something else: a green build, an empty variable, a billing toggle, a file tree, a session registry, a documentation page. In each case the fix was to ask the thing itself instead of asking a record of it.
That gets more expensive to ignore as you add agents. One person reviewing one change can hold the gap between reported and actual in their head. Six concurrent agents shipping against twenty surfaces cannot, so the verification has to be mechanical and it has to run against production.