T0ha666
A $63 Mistake in Production: How an AI Agent's Loop Escaped Without Guards
An automated trigger fired every two minutes for 8 hours. 243 agent runs. 31M tokens.
$63 wasted. The problem: nobody noticed until morning.
This is a case study in production safety when AI agents are on the loop — and a
cautionary tale about human approval gates.
The technical breakdown:
- An Ash/Oban cron trigger polling GitHub every 2 minutes — seemed fine on review
- The check was level-triggered (state-based) instead of event-triggered
- GitHub review feedback is immutable history, not a dismissable flag
- So the agent kept trying to fix the same feedback, over and over
- No cost counter, no alert, no visibility — just a morning invoice
Why the existing guards failed:
- Freshness check existed but lived on an unreachable code path
- Attempt counter got reset to 0 on every pass (accidentally disarmed)
- Branch order made recovery impossible once one reviewer approved
The real lesson: when you put an expensive actuator (LLM tokens) on a loop,
“harmless redundancy” stops being a category. Every pass has to earn itself.
Full postmortem: https://t0ha.dev/2026/09/18/63-dollars-in-one-night.html
Built with Elixir/Phoenix/Ash, and the hardest part wasn’t the code — it was
the code review.
Popular Ai topics
The rapid development of AI (artificial intelligence) has opened up new ethical frontiers at a startling pace. As the impact of AI is so ...
New
I just published an article detailing my journey in building a Dota 2 Match Outcome Predictor using machine learning and data analysis. I...
New
[Project Update] Part 2 of My Dota 2 Match Outcome Predictor – Now Available!
Hey DevTalk community!
I just published the second part o...
New
I recently wrote an essay on effects of AI, how it all can become real bad, and how we can avoid it.
I started writing the post intend...
New
Top-tier LLMs, Rust and Erlang NIFs; nifty, and night and day vs. C, but let me tell you about vibe coding…
After I submitted my blog po...
New
TL;DR:
Good: AI is great at Elixir. It gets better as your codebase grows.
Bad: It defaults to defensive, imperative code. You need...
New
You guys aren’t gonna believe this.
Anthropic‘s engineers just dropped a goldmine — a deep dive into how they’re actually using Claude C...
New
The structural argument, briefly: literacy installs a cognitive substrate in human brains; LLMs are built entirely on the same kind of su...
New
I have been looking through Seedance 2.0 prompt examples and noticed a pattern:
The prompts that work best are usually not just longer. ...
New
An automated trigger fired every two minutes for 8 hours. 243 agent runs. 31M tokens.
$63 wasted. The problem: nobody noticed until morn...
New
Other popular topics
Free and open source software is the default choice for the technologies that run our world, and it’s built and maintained by people like...
New
I know that these benchmarks might not be the exact picture of real-world scenario, but still I expect a Rust web framework performing a ...
New
New
Please tell us what is your preferred monitor setup for programming(not gaming) and why you have chosen it.
Does your monitor have eye p...
New
No chair. I have a standing desk.
This post was split into a dedicated thread from our thread about chairs :slight_smile:
New
Design and develop sophisticated 2D games that are as much fun to make as they are to play. From particle effects and pathfinding to soci...
New
You might be thinking we should just ask who’s not using VSCode :joy: however there are some new additions in the space that might give V...
New
Small essay with thoughts on macOS vs. Linux:
I know @Exadra37 is just waiting around the corner to scream at me “I TOLD YOU SO!!!” but I...
New
Was just curious to see if any were around, found this one:
I got 51/100:
Not sure if it was meant to buy I am sure at times the b...
New
Lint your docs like code: turn any style guide into enforceable rules with Vale and publish clear, consistent content every time.
...
New
Categories:
Sub Categories:
Popular Portals
- /elixir
- /rust
- /wasm
- /ruby
- /erlang
- /phoenix
- /keyboards
- /python
- /js
- /rails
- /security
- /go
- /swift
- /vim
- /java
- /clojure
- /typescript
- /emacs
- /haskell
- /svelte
- /onivim
- /kotlin
- /react
- /c-plus-plus
- /crystal
- /tailwind
- /gleam
- /ocaml
- /elm
- /vscode
- /flutter
- /html
- /ash
- /deepseek
- /zig
- /opensuse
- /centos
- /php
- /react-native
- /scala
- /lisp
- /sublime-text
- /textmate
- /nixos
- /debian
- /agda
- /deno
- /django
- /kubuntu
- /arch-linux
- /nodejs
- /spring
- /ubuntu
- /revery
- /manjaro
- /julia
- /diversity
- /quarkus
- /lua
- /laravel









