Dr. Josh C. Simmons
· 8 min read
Two green pull requests can make a red main
Two branches pass the same existing test, then merge cleanly into a version that fails it. Parallel agents need checks on the combined candidate.
· 8 min read
One character can reverse an agent leaderboard
Two hypothetical agents share a 50% success rate, yet pass@4 and pass^4 reverse their ranking. The product still has to choose the answer it returns.
· 8 min read
Your concurrency limit is not a queue limit
Ten active calls can hide thousands of waiting tasks. A runnable Python example shows where backpressure has to reach the producer.
· 8 min read
Replaying an agent should not call the model
A runnable example of agent replay, request matching, and the difference between reproducing a failure and testing a new prompt.
· 9 min read
Your agent should lose access when the task ends
A finished task can leave a valid credential behind. Give agent authority a lifecycle, and test what still works after the task closes.
Get new posts by email.