In a local engine experiment, two synthetic payments totaled $1,100 against a $1,000 invoice. OpenAuditor returned a false verdict for the second payment when we submitted them together. Submitted separately, with the same invoice but no payment history, both received a true verdict. We changed the request boundary, not the payment records.
The missing $600 changed the answer
Our fictional supplier, Clearspan Industrial Supply, has one $1,000 invoice. Payment PAY-A is $600, dated September 20. PAY-B is $500, dated September 21. Both reference that invoice. The requests supply the same established vendor, a dummy matching bank account and an explicitly empty employee comparison population. No credits, currency conversions or cancellations complicate the example.
In one audit request, the balance check sees the earlier $600 payment. It calculates $400 remaining and fails PAY-B because $500 exceeds that balance. When PAY-B arrives in a separate request without payment history, the calculation sees no earlier payment. Its available balance becomes $1,000, and the check holds.
| Submission | Earlier payments visible to PAY-B | Calculated open balance | PAY-B balance assertion | Overall verdict |
|---|---|---|---|---|
| Both payments together | $600 | $400 | fails | false |
| Separate requests; payment history omitted | $0 | $1,000 | holds | true |
| Separate requests; payment history explicitly empty | $0 | $1,000 | holds | true |
| Separate requests; complete payment history supplied | $600 | $400 | fails | false |
The observed local engine outcomes include all three arrangements: joint submission, separate submission and restored history. The $100 excess is detected only when the relationship between the payments is available to the check.
No money moved. A true verdict here concerns supplied synthetic records; it does not establish that a real payment is safe. These payment assertions are deterministic, so the verdict change required no model.
A duplicate finding disappeared too
We also submitted two invoice rows with different record IDs but the same supplier and invoice number, CS-4281. Each bills $768 for eight docking stations at $96. Their invoice dates differ. Arithmetic and vendor status are supported, and the amount is below the default PO-reference threshold.
Together, both rows fail not_duplicate. Split into one-invoice audit requests, both hold that assertion unless the other invoice is supplied in context. The overall result changes from false to uncertain, not true: we disabled model questions, so the goods-description judgment remains unanswered.
| Invoice submission | Comparison history | not_duplicate | Overall verdict |
|---|---|---|---|
| Two invoices in one audit | Omitted | fails for both | false |
| One invoice per audit | Omitted or empty | holds | uncertain |
| One invoice per single verification | Omitted | insufficient_evidence | uncertain |
| One invoice per single verification | Explicitly empty | holds | uncertain |
| Either mode, one invoice at a time | Complete pair | fails | false |
The distinction between a single verification and a one-item audit matters. In the pinned implementation, an audit treats its submitted item types as available comparison populations. One invoice therefore creates an available invoice population containing only itself. Once the duplicate check excludes that row, no peers remain. Single verification instead requires the comparison population to be explicitly supplied.
The engine's internal chunks preserved the relationship
The runner processes items internally in groups of 32, but builds their shared context first. We placed the duplicate pair at opposite ends of a 34-invoice request, separated by unrelated suppliers. Both duplicate findings survived. Sending the same population as independent requests of 32 and two items erased the pair's findings. Supplying the complete history to each request restored them.
Reusing the same runner did not preserve records from an earlier request either. Each call builds a new context. A previous successful upload is not an instruction to include those records in the next comparison. The published request collection records the context used in each arrangement, including the difference between omitted and explicitly empty history.
“Every row was tested” leaves out the relationships
An accounts-payable integration might split an export for request-size limits, distribute work across queues or retry failed portions. All rows can be processed exactly once while the evidence needed for a cross-record check is lost. Row coverage and relationship coverage are separate properties.
That difference also affects operational reporting. In the payment example, the split run looks better: both records pass. In the invoice example, the failure count falls while the unresolved count rises. Neither change reflects corrected business data. A dashboard that compares those counts without the request scope could mistake a partitioning change for an improvement in controls.
Keep the population reference and comparison scope with the result. The response's input hash identifies a particular request; it does not prove that the request contains every relevant invoice or payment. See population completeness for the separate work of establishing that boundary.
Choose what must remain together
For this invoice duplicate rule, every candidate match shares a vendor identifier. Keeping the complete vendor group together preserved our pair. That is a property of this rule, not a universal partition key. Date-based splitting can separate repeat invoice numbers, and this implementation does not use currency to exclude duplicate candidates.
Supplying the relevant comparison history can preserve relationships across requests. “Relevant” must come from the procedure: the payment example needs earlier payments against the invoice, while other checks may need different records. The simple balance check tested here is not a general credit-allocation ledger.
Rolling history has another consequence. Sending the first invoice with empty history and the second with the first in context flags only the second. The first response remains unchanged. That can support a deliberate arrival-time workflow, but it does not reproduce the two row-level failures from a complete snapshot. Define whether earlier decisions are provisional or will be revisited.
Inspect the experiment
Download the recorded evidence bundle, or inspect the inputs, observed outcomes and run manifest. The bundle includes every request and complete recorded response. The local investigation used engine revision 6205721, synthetic business data and no model calls. Its 33 scenarios passed 297 local checks. These are implementation experiments, not measurements of model accuracy or the prevalence of missed exceptions.
The independent recorded-results checker uses Python's standard library. It checks the before-and-after outcomes, unchanged invoice inputs, payment arithmetic, request hashes, scenario counts and agreement between the summary and complete responses. It reads files and prints a report; it does not execute the engine.
unzip evidence.zip -d batch-scope
python3 batch-scope/check_results.py --evidence ./batch-scopeThis checks the published evidence; rerunning the engine requires access to the pinned source revision. Consistent recorded results do not independently authenticate the original experiment. The engine source and original execution harness are not included in the public package.
Before deploying a batching change, run the same fixed snapshot together and through the proposed partitioning strategy. Compare assertion outcomes by stable record ID. Decide which relationships and temporal interpretation must be preserved, then make that requirement part of the integration. The request boundary belongs in that decision.