Why Hibi: Deterministic Doc-Drift Detection

The problem Hibi solves, its threat model (a reader trusts the raw file), and the principles behind every design decision.

Documentation drifts. Someone edits the code, the sentence that described it stays, and the page now misleads whoever reads it. Often that reader is an AI coding agent that loads CLAUDE.md or a README at the start of a session and acts on every line. Hibi makes that drift detectable and visible in the file itself, deterministically.

For the mechanics, start with Verdicts.

The problem and the threat model

A claim is a sentence in a document anchored to the code it describes. Claims rot in three ways:

Drift type	What changed	Example
Code drift	The code moved on; the prose stayed.	A doc says “retries 5 times” after the constant became 50.
Doc drift	The documented sentence was edited or deleted.	Someone rewrote the paragraph; the claim no longer points at live text.
Supersession	A newer document replaced this one.	An ADR is marked superseded; readers still find the old page.

The threat model is one sentence: a reader reads the raw file and trusts it. It does not consult a dashboard or a CI log. So the status has to land in the file. That is what check --write and the banner are for.

A flag is a request to re-verify. Hibi reports that the text or code under a sentence moved. A person or agent decides what the prose should now say. Hibi never rewrites prose.

The principles

Determinism

No model runs in the check loop. The same working tree yields the same verdicts. Resolution is an ordered cascade with named constants (Anchors), not a score. An external advisory resolver may attach notes; it never decides a verdict and never gates.

Suspect, not false

Hibi reports that evidence moved, never that a claim is false. Confirming falsity means reading both sides and judging intent, which is a human or agent act.

Over-flagging is the main failure mode

A tool that flags too much gets muted. Coarse anchors are never reported as drift. moved is a warning, not a gate. When several exact matches tie, the answer is ambiguous, not a guess.

The documented span is the source of truth

The store holds anchors, not a copy of the prose. Re-reading the live sentence at check time is what catches doc drift as well as code drift.

Agents author claims, and new claims gate

The primary user is a coding agent. JSON is the contract, and record --from-file - is the primary recording path. A new claim is enforced by default; a claim that cannot fail anything does not prevent false claims. --suggest opts out.

Any file is text

There is no per-format parser. Markdown, plain text, AsciiDoc, and editor rule files are all handled the same way; the only format-specific knowledge is which comment syntax wraps a banner.

Why no model sits on the verdict path

Published results on model-based doc verification are not strong enough to gate on:

• The best published doc-behavior verification tops out at an F1 of about 0.58 (precision about 0.72, recall about 0.48).
• LLM-as-judge consistency falls to about 0.57 under sampling, and audits have shown judges accepting up to about 63% of intentionally wrong answers.
• Noisy analyzers report up to about 96% spurious warnings, and trust lost in one tier spreads to every tier.

So Hibi splits the work. Text and AST resolution say where something changed. Author-supplied verifiers, run only under check --run-verifiers, judge behavior. A model may advise through an advisory resolver, but it may not gate and may not mark a claim supported.

Prior art

Prior work	What Hibi takes from it
Hypothesis fuzzy anchoring	The ordered cascade: exact match with context, then fuzzy match within an error budget, position as a weak bias.
W3C Web Annotation Data Model	The text-quote and text-position selectors; several matches means ambiguous, not a guess.
Fiberplane Drift	tree-sitter plus an AST fingerprint committed to a lock. Hibi adds the doc side, graded states, supersession, and in-file stamping.
Doorstop	A link marked “suspect” that demands an explicit re-baseline.
Gerrit ported comments	Degrade precision rather than guess.
RFC Obsoletes / ADR supersession	A typed supersedes edge authored on the new document.
FEVER	The supported / refuted vocabulary.

Where the reasoning becomes mechanics

Verdicts

The claim model, the two axes, the remediation menu, and the exit codes.

Anchors

The selectors and the resolution cascade with its constants.

Behavioral claims

Verifiers and why they are opt-in.