# Precedent

- Page: https://scanovich.ai/projects/precedent/
- Language: en
- Updated: 2026-11-12

- Status: released · v2.2.0
- Code: open source · MIT
- Repository: https://github.com/FUYOH666/precedent
- Research line: [Evidence and authority for machine decisions](https://scanovich.ai/research/evidence-and-authority.md)

A person's correction to an AI judge becomes a rule that states its reason, is tested against every case already checked, is approved by a second person, and can be reverted.

An open-source library and command-line tool that treats a correction to an AI judge like a change to code.

## What works today

- A correction is written as a rule that carries the reviewer's reason. It is not pasted into the prompt as an example.
- A gate runs the rule against every case the system has ever been checked on, and blocks it if it breaks an answer that was right.
- A second person approves the rule. One command reverts it.
- Every step lands in a hash-chained log that an auditor can verify offline.
- The demonstration runs offline.

## Where it came from

The first version of this project learned a rule from one supervisor's correction. The rule broke calls the system had been getting right, and the project's own evaluation reported an improvement. That failure is [published in the repository](https://github.com/FUYOH666/precedent/blob/main/docs/what-v1-got-wrong.md), and it is the reason the gate exists.

## What the benchmark showed

The hypotheses were registered before any model output existed, and the departures from the plan are logged.

- Rules that passed the gate broke no verdict that had been right. Every way of putting the corrections into the prompt broke some.
- One hypothesis failed. Retrieved examples were more accurate than the rules, by more than the margin set in advance.
- The reason is that corrections come in two kinds. A misapplied policy becomes a rule. A misread fact does not, because the rule works on what the judge already read.

## Limits

- One synthetic domain and one judge in the comparison. The reference labels come from a written policy, not from people who disagree.
- The log shows tampering. It does not prevent it.

## Why it is in the lab

It is the open, runnable part of the research on evidence and authority: who may change what a machine decides, and how that change is checked before it takes effect.

## Evidence

- **0** — verdicts that had been right were broken by rules that passed the gate, on a sealed set of 320 verdicts. The same 20 corrections given to the judge as examples in the prompt broke 19. Measured 2026-11-12. Source: https://github.com/FUYOH666/precedent/blob/main/benchmarks/correction_stream/results/gemma-4-26b-a4b-it/RESULTS.md
- **293 of 320** — verdicts were right with the rules, against 304 with retrieved examples. The lab had registered in advance that the rules would stay within two points of the best method. They did not. Measured 2026-11-12. Source: https://github.com/FUYOH666/precedent/blob/main/benchmarks/correction_stream/results/gemma-4-26b-a4b-it/RESULTS.md
- **7 of 20** — corrections could not become a rule. In each of them the judge had misread what was said, and a rule built on the judge's own answers cannot repair that. Measured 2026-11-12. Source: https://github.com/FUYOH666/precedent/blob/main/benchmarks/correction_stream/results/gemma-4-26b-a4b-it/RESULTS.md

## How to engage

A correction review. Send a sample of the decisions your reviewers overrode. We show which corrections are safe as rules and which would have broken something that worked.

[Contact](https://scanovich.ai/contact/) · private@scanovich.ai
