Essay

The Mistakes an Agent Doesn't Ask About

A small model can ask a stronger one for help. The harder question is whether it notices when help would matter.

A large bronze spider sculpture in a garden beneath trees with yellow and orange autumn leaves

Giving a small model a smarter model to call seems like a good way to save money. Let it handle routine work and ask for help when it gets stuck.

The problem is that a mistake can look like a finished job.

Suppose a coding agent adds retries to a request that sometimes times out. The tests pass. But the server might have created the record before its response was lost, so retrying could create it twice.

The agent never considered whether the operation was safe to repeat. Nothing seemed wrong, so it never asked for help. The misunderstanding caused both the mistake and the decision to finish alone.

A router or review rule can catch known risks. I'd also sample jobs completed without help and have both models review them under comparable conditions. Does the stronger model catch important mistakes? Would another pass by the smaller model have caught them too?

Check the findings against tests or human review. The stronger model's disagreement isn't proof that it's right.

Before calling the unassisted work “easy,” I want to see what the agent has been missing.