Do you still require a human to review every PR, or have you introduced rules that allow some changes through with automated checks and AI review?
If you’ve changed your process, what determines when a human must get involved? I’d especially appreciate one concrete example of a change you allow through automatically versus one that always needs human review.
Experiences where automation didn’t work out would be just as useful.
Or maybe you use LLMs to enrich pull requests with flowcharts or other assets that make it easier for humans to review the code?
Working on aicost.tools, my rule is: changes that only touch data (a new model, a changed rate) can go through with tests plus an AI review, but anything that changes how tokens turn into money waits for a human. A tidy diff where the agent also updated the tests is exactly what an AI reviewer waves through, and that’s where a wrong rounding or unit slips in. On your second question, I’d skip the flowcharts and ask the agent for one short paragraph on “what behaves differently after this PR”; that’s what actually speeds up my review.
1 Like