Agents in CI: how not to go broke on automated runs

An agent on every commit turns spend into a function of team activity. Where to hang it, how to bound it, and how to stop paying twice for the same diff.

An agent in continuous integration is convenient and quietly expensive. Quietly, because spend grows with your team's activity rather than with your decisions.

Here is how to keep the value and remove the surplus bill.

The main mistake: firing on every commit

A developer pushes ten times a day while polishing a branch. If the agent runs on every push, you paid for nine runs nobody read.

Hang it on pull requests, not pushes. Better still on opened and ready-for-review, rather than on every branch update.

Bound the scope

An agent that reads the whole repository on every run pays for that context again and again. Give it the diff.

Most CI tasks are reviewing a change, not auditing a project. The spend difference is an order of magnitude.

Split checks across models

CI usually holds several kinds of check, and they need different models.

style checks, leftover debug statements, commit message formatting: smaller model
reviewing the logic of a change and hunting defects: something stronger
architecture audits: the flagship, and certainly not on every PR

Do not pay twice for the same thing

If the pipeline reruns because a linter failed, the agent should not walk the same diff again. Cache the result against a hash of the change.

Same with build matrices: the agent should run once, not in every cell.

Give automation its own ceiling

Keep the CI key separate from human keys. Then you can see what automation consumes and bound it independently.

Here, every key carries its own limits and its own usage stats, and sub-organisations let you move CI onto a separate budget. On hitting a threshold that key can be frozen without touching developer keys.

What to measure

Three numbers worth keeping in view:

spend per pull request
share of runs whose output nobody read
cost per defect the agent actually found

The third one is uncomfortable, and it is exactly the number that answers whether the automation pays for itself.

In short

Pull request instead of push, diff instead of repository, smaller model for mechanics, and a separate key with its own cap for automation. That is enough to make a CI agent useful without surprises on the invoice.