A different employee made the callback. Verification preceded payment. The destination matched the vendor master. In a synthetic replay, OpenAuditor's seven checks held in two scenarios with different account owners. The missing fact was where the verifier obtained the telephone number.
Two histories that produce the same request
Fictional Beacon Packaging is owed $32,600. An accounts-payable employee records a bank change on September 14. A second employee calls on September 15 and records the verification method as callback. A payment on September 16 uses the account now held in the vendor master. Its amount matches the supplied invoice.
We constructed two upstream histories. In the first, the verifier uses an independently established supplier contact retained before the change request. An authorized supplier representative confirms the instruction. In the second, the verifier calls the new number supplied in the same impersonating email that requested the change. The person answering confirms the account, but is the impersonator.
Both fictional histories include a completed callback by a different employee before the recorded payment. The difference is the source of the contact and the authority of the person reached. Those facts are outside the submitted request.
| Fact | Scenario A | Scenario B |
|---|---|---|
| Recorded verifier | A different employee. | The same second employee. |
| Recorded method and date | Callback, September 15. | Callback, September 15. |
| Contact obtained from | An independently established supplier contact. | The unverified change email. |
| Account owner in this fiction | The intended supplier. | The impersonator. |
| JSON submitted for verification | Identical request. | Identical request. |
The ownership difference is stipulated in this counterexample, not discovered by OpenAuditor. Every business, identifier and account number in the downloadable fixture is fictional. This is a demonstration of what the input can distinguish, not an observed fraud case.
What the engine actually returned
We ran the existing local engine twice at revision 6205721, using one bank-change decision and one payment as the items under review. No model was configured or called. The invoice and two employee records supplied context.
| Item | Checks evaluated | Observed result in both runs |
|---|---|---|
| Bank change | change_verified, no_payment_before_verification | Both holds; item verdict true. |
| Payment | matched, not_duplicate, to_vendor_account, not_to_employee, after_invoice | All five holds; item verdict true. |
The two responses match after removing only their verification run IDs and creation timestamps. Their input hash, configuration, assertion outcomes, facts and evidence pointers are identical. Both report certainty.basis: "unmeasured".
The responses also contain probability_true: "1.0000" with a deterministic basis. That value follows from the implemented checks on the supplied facts. It is not a measured probability that the account belongs to the supplier. The paired case shows why those claims must remain separate.
An evidence pointer can end before the important fact
For change_verified, the result points to three request fields:
/items/0/data/actor
/items/0/data/verified_by
/items/0/data/methodThe procedure checks that the verifier differs from the requester and that the recorded method is supported; it rejects email-only verification. It does not examine where the telephone number came from or authenticate the person answering. The timing procedure separately compares supplied dates.
For to_vendor_account, the pointers identify the payment account and the applicable vendor-master account. Their equality establishes agreement with the supplied master. If both values descend from an unverified change, their agreement does not establish account ownership.
These are traceable findings within the documented result scope. The next link in the audit trail must connect the submitted fields to how the business established them.
Independence belongs to the fact being tested
A vendor report, payment proposal and remittance report can all carry the same account because they copied it from one master record. More matching reports can demonstrate successful propagation of that value while adding no independent support for its origin.
The same distinction applies to an invoice, purchase order and receipt. A receipt quantity prefilled from the order repeats the ordered quantity until the receiving process adds an observation of acceptance. A receiver's actual count may support quantity, while the receipt's copied supplier details contribute nothing new about bank-account ownership.
Copying is normal and often desirable. The question is what a particular match proves. One document can contain an independently observed fact beside a field inherited from another system. Assigning one confidence label to the entire document conceals that difference.
For PCAOB engagements, AS 1105 connects relevance to the assertion and reliability to the source and circumstances of collection. It also addresses company-produced information and external electronic information processed by a company. These principles support examining the origin of the relevant field; they do not make every copied record unreliable.
Why another run cannot resolve this difference
The argument is simple: two different histories have been reduced to the same input. A fixed deterministic rule reading only that input must return the same result. Yet the account-ownership answer differs between the histories. The missing distinction must enter through additional evidence or an evaluated control over how the information was obtained.
This does not estimate how often account changes are fraudulent. It shows why rerunning the request, adding another copy of the vendor report or assigning another reviewer to the same contact record cannot, by itself, establish the missing fact.
Change the next evidence request
In scenario B, the useful next request is for the source of the callback contact and evidence of the supplier representative's authority. Asking for another payment export follows the same inherited account value downstream.
The FBI's business email compromise guidance advises obtaining a company's telephone number independently instead of using one supplied in a suspicious message, and verifying changed payment instructions. A second person making the call addresses actor separation; an independently established contact route addresses a different weakness.
Record that distinction in the disposition: “Payment matches the recorded master; the origin of the account instruction remains unresolved.” Preserve the matching result and follow the organization's verification and payment-control process. A later supplier confirmation should be connected to the original history, including when it occurred. The bank-change control guide covers that timing review.
Inspect the experiment
Download the experiment packet. It contains the shared request, the two fictional histories, both recorded responses, a comparison and a standalone checker. With Python 3.10 or later, you can check the published results without an API key, model or third-party package.
unzip evidence-origin.zip
python3 evidence-origin/check_results.pyThe checker verifies the request hash, equality of the recorded responses apart from run IDs and timestamps, seven holds outcomes, two true verdicts and zero recorded model questions. It makes no network or model calls and does not run the verification engine. Rerunning that engine requires access to the private source at revision 6205721f91352039de5ce861c1c94d15d9903c7d.
This checks the packet's internal consistency, not the authenticity of its execution history. You can inspect the request, fictional histories, recorded comparison and checker individually.
This experiment checks seven conditions in two synthetic items. It does not verify invoice authenticity, test payment release, perform a three-way match or establish that a real population is complete. The negative duplicate and employee-account results cover only the supplied records. Its value is the visible boundary: matching facts can be correct within the request while the business question still needs evidence from outside it.