Why it is uncertain
Citations, abstention and approval steps are known techniques. None of them is new here. What is not known is whether binding them into one versioned workflow beats a strong baseline that also uses dates and jurisdictions, at an acceptable coverage, review workload and cost.
Beating a naive system proves nothing. The comparison has to be against the best ordinary approach, given the same model, the same corpus and the same budget.
Method
- Threat model first: who can forge a document, an approval or a permit, and what the system promises against each.
- Adversarial scenarios are written as tests. A scenario that once succeeded stays in the suite forever.
- Metrics and data splits are frozen before any result is seen.
- Negative examples are part of the set: the wrong jurisdiction, an outdated rule, two authoritative sources that disagree, a passage that is relevant but does not support the claim.
- Every result is reported with its cost, latency and the share of cases the system declined to answer.
- A failed experiment is a result and is kept.
What we measure
- Rate of unsupported claims.
- Rate of claims that are invalid for the date or the jurisdiction.
- Expert correction time.
- Cost per accepted decision.
- Risk against coverage when the system is allowed to abstain.
Results so far
The threat model and the adversarial suite exist. The measurement protocol is being frozen; no comparative figure is published yet.
One bounded demonstration is public. In EvidenceGene Court, a finding cannot enter the report without a valid reference to an artifact in a hash-chained log, and the public test case shows an injected fabricated finding being rejected.
Next
Freeze the protocol. Run the first comparison against the strong baseline. Publish the result whichever way it goes.