Operations notes
What production has taught me
Real incidents, dated, with their cause and the check they led to. No tutorials: what a leader needs to know before handing an action to an agent.
Eleven days, 80 failures a day, no alert: anatomy of an AI agent that costs money in silence
An agent froze six times a day. It was not its tools: it never got as far as calling them. And for eleven days it had been checking a server that no longer existed.
The script that would have declared a failure compliant, every day
To monitor my domains, a script read the live DNS and took it as the truth. On the day that DNS was broken, it would have copied the failure and validated it every day.
An incident should produce a check, not a note
The agent's memory held a perfect note about an exposed service. Two months later, two other services were exposed in exactly the same way.