A report can repeat the right numbers and still reach an unsupported conclusion. Review each factual claim against its cited source, then distinguish direct evidence, calculations and interpretation before the report informs a business decision.
Break a polished paragraph into reviewable claims
Consider a fictional operations report: “Late deliveries fell 20% to 96 in June because the new routing software improved performance.” The source table contains 120 late deliveries in May and 96 in June. It says nothing about why the count changed.
The sentence contains a June count, a calculated percentage and a causal explanation. Each needs different evidence. NIST’s Generative AI Profile identifies the risk of confidently presented false content and recommends reviewing sources and citations. A fluent explanation is therefore a candidate for verification, not a substitute for its evidence.
Create a claim register with source ownership
Assign each factual sentence a stable id and list the documents intended to support it. Keep the report section and original wording alongside that id in your review system. Split sentences when their parts depend on different sources or require separate decisions.
| Claim from the fictional report | Evidence needed | Review question |
|---|---|---|
| June had 96 late deliveries. | The June operations table and its definition of a late delivery. | Does the count refer to the right period and population? |
| Late deliveries fell 20%. | Comparable May and June counts plus the calculation. | Are the denominator, scope and arithmetic appropriate? |
| New routing software caused the improvement. | Evidence supporting that explanation. | Does the analysis establish causation or only a change over time? |
| Every region improved. | Results for every region within the claimed scope. | Does the aggregate conceal an exception? |
Record whether a statement is an observation, a derived measure or an interpretation. Keep recommendations distinguishable from factual findings. OpenAuditor does not automatically create this register or decide that every sentence in a report has been submitted.
Submit the text that actually supports the statement
The AI output verification use case accepts a statement item with a statement string and document_ids. Supply the cited documents as text in context.records.document. Use a distinct item for each statement you want reviewed.
The service does not retrieve a citation URL, query your reporting database or establish that the supplied document is authentic. Preserve the underlying source and its version. If a report concerns June, a similarly titled May document is not interchangeable evidence. Missing cited documents or empty source text can leave support unresolved.
Trace repeated claims to their origin. A slide, management summary and email may all repeat the same operations table. Agreement between them can support faithful copying without independently confirming the table's count. Record the evidence for the underlying count separately; our evidence-origin investigation shows how the same distinction affects payment review.
Read numerical and quotation checks narrowly
The numbers_present assertion checks whether numbers in the statement occur in its cited sources under the supported comparison rules. A number’s presence does not establish its role: 96 could be a count of returns in another paragraph. Numerical matching also does not settle the sign or semantic meaning of a statement.
The quotes_exact assertion checks quoted text against the cited documents with defined normalization and quotation rules. A matching quotation can still omit context that changes its interpretation. If the statement contains no numbers or no quotations, those respective checks are not applicable; they have not established the overall claim.
The supported assertion addresses whether selected source passages support the statement. It can require a model judgment and remain unresolved under the running configuration. The model receives bounded passages, so do not assume it has read every relevant passage in a long source. Inspect the selected evidence and the returned outcome and certainty basis.
Handle calculated metrics explicitly
For the fictional table, (120 − 96) ÷ 120 equals 20%. Yet the source table does not itself contain the number 20. The current statement number check is not a general calculation engine: a derived number absent from all cited text can fail its presence check even when a separate calculation is sound.
Keep a calculation workpaper with the inputs, formula, units and reviewed result. If you supply a calculation summary as another source, retain its provenance and validation. You have added evidence for the claim; you have not caused statement verification to independently recalculate that formula. Use appropriate computational checks for the metric before relying on the summary.
Review explanations, comparisons and omissions
A correct percentage does not explain its cause. Ask the report owner to support the routing-software claim or revise it to an observation the available evidence supports. Check whether the definition of “late,” the set of routes or the reporting cutoff changed between periods.
Also review completeness outside the individual statement checks. A report could contain only supported sentences while omitting a region with worse performance. Verification of submitted claims cannot establish that all important findings were included. The report owner remains responsible for its scope and balance.
Resolve findings without losing the original claim
Send demonstrated mismatches for correction and unresolved semantic questions to a reviewer with the source passages. Retain the original sentence, revised wording, source versions and review decision. An uncertain result needs an explicit disposition; it should not disappear when a new draft is generated.
Use the batch review workflow for a report’s statement population, and measure incorrect flags separately from unresolved work. Our investigation of certainty and selection explains why error rates need a clearly defined denominator. If the source values were extracted from documents, verify those fields as a separate step. Each layer should preserve what was checked and what still requires judgment.