Every change to how Verify decides

How code reads a model's adjudication: a proposed qualifier must be a restriction code can recognise (a whole clause is a statement); the claim's values are checked against the one cited sentence about the claim, and a time window in another cited sentence is another fact's window; polarity is read on that sentence, not on a quote that runs into the next one. The code-only path is unchanged.

• qualifiers.ts (isRestrictionPhrase): a model-proposed qualifier counts only if it is a short noun phrase with no verb ("annual plans"), opens like a restriction with no verb ("within the contiguous United States", "for orders over $50"), or is read by code as a condition, time window or scope over most of its length ("for members of the loyalty program", "unless the delay was reported in advance"). A clause such as "Every plan is a weekly subscription" or "Cancel anytime; service runs through the paid week." is ignored. A proposal may exist anywhere in the model's quotes (compareQualifiers' proposalScope), not only in the narrowed context.
• rules.ts (selectContexts, secondaryDropped): under a supported verdict the claim's values and the restrictions on them are read from ONE cited sentence, the one sharing most of the claim's content (ties: the sentence stating a value the claim states, then the span marked supports, then the first); a span marked conflicts is admitted to the choice when it covers at least half the claim's content. The other cited sentences lend only a dropped restriction that carries no value, or a time window when the claim states none.
• rules.ts (finishSupport, narrowedContext): the polarity check reads the cited sentence about the claim rather than the whole quote; when a quote is narrowed, a bare "A: No." / "Yes." stays with the question it answers.
• Major under this file's policy: verdicts can move toward release on the model path. The code-only path is byte-identical: 0 of 2,040 v1.1 dev + held-out predictions changed and every held-out metric is identical to 3.0.0; the flag-off snapshot, the Gate corpus expectations, the backlog pins and the regression gate's dev rates are unchanged.
• Fixtures: tests/verify/agent-factory-regression.test.ts, "Group B on the model path": the five adjudications production's fast model returned for Group B on 2026-10-26 (spans and proposals as logged) decided in code, plus controls (a claim whose window differs from the sentence about it, even when a second cited sentence carries its number; a claim with no window that drops a cited one; proposed "for members …" and "annual plans" the claim drops).

Why: Rules 3.0.0 released Groups A and E and refused Group C on production, but Group B (paraphrases that keep every condition) still came back revise on the full path. The model's verdict was supported each time; the refusal came from code reading its adjudication: whole clauses proposed as qualifiers (B1, B4), a quote running into a negated next sentence (B2), evidence roles mislabelled (B2) and a second cited sentence's window read as a different value for the claim (B2b). docs/rules-4.0-plan.md has each case. The pre-registered bar (docs/kill-criteria.md, "Verify rules 4.0.0") is on the model path, because a code-only fresh set cannot measure a change the code-only path does not take: v1.1 held-out through the credulous adjudicator, 4.0.0 against 3.0.0, run once each after freezing, plus production's own adjudications and the controls as fixtures.

Five false-positive root causes found by a customer’s agent pipeline (the Agent Factory) against production: "24 hours a day" and "$29 a week" are one value and a rate, not two values; "No." answering an FAQ question ends its sentence and does not negate the next one; a model's multi-sentence quote lends a claim only the sentences about it; passages that state the same fact in other words agree, they do not conflict; an answer sentence that is a source sentence word for word is released whatever the extractor made of it.

• numbers.ts: "a"/"an" + a unit straight after a quantity ("24 hours a day", "$29 a week", "3 times a month") is that quantity's rate (rateAfter already read it), not a second quantity of one day or one week.
• qualifiers.ts: a frequency phrase in the source ("a day", "per week", "weekly", "twice a month") is compared with the claim's frequency as a whole phrase (frequencyKey), never by the digits inside it; one that sits right after a value is handled with that value's rate, not as a separate restriction; "annually / yearly / monthly / weekly / daily / quarterly" directly before a plain noun ("weekly subscription") is attributive, not a restriction on the predicate; "in US" before "dollars", "cents", "currency" or "funds" is not a geographic scope; "no" straight before punctuation is an answer of its own, not a negation of what follows (NOT_NEGATION_RE), and a "?… No." pair negates the question's content (polarityCount, used by the polarity checks).
• sentences.ts: "No." is the abbreviation (numero) only before a figure; before a capitalised word it ends its sentence.
• templates.ts: "no" straight before punctuation is not a negation word, so a sentence after "A: No." is not reported as negated.
• rules.ts: the context a claim's qualifiers are read from a model's quote is the quote's sentences that share at least a third of the claim's content words, with the clause context rebuilt from that range (an attached limiting sentence still counts); when every sentence shares less, the whole quote stands. A model's conflicting_sources is checked in code (passagesAgree): located passages with no differing value, date or opposite polarity over shared content agree, and the claim is then judged against them as support (finishSupport, score lowered by 0.1); a real difference still returns conflicting_sources.
• engine.ts: an answer sentence that is a source sentence word for word decides every claim taken from it that was not supported: the claim is decided again on the sentence (R-ANSWER-SENTENCE-VERBATIM:<rule>) and the sentence's verdict stands when it is supported; when the sentence is not supported (it stops short of a condition: verbatim qualifier loss) the original finding stands. Runs before the 2.0.0 answer-clause rule.
• Major under this file's policy: every fix moves a verdict toward release. Nothing code caught is released: on v1.1 dev + held-out 1 of 2,040 predictions changed (s6-f3-sup-second, gold supported, unverifiable → supported); the rules-next backlog pins, the Gate corpus local expectations and the regression gate's credulous-model rates are unchanged; the Group C controls under an always-supportive stand-in model are identical before and after.
• Fixtures: tests/verify/agent-factory-regression.test.ts (prebuild gate): Groups A, B, C, E of bench-private/agent-factory-2026-10-25/cases.json through the engine with a deterministic stand-in model, plus a minimal pair per root cause; packages/verify-core/test/fragment-answer-context.test.ts accepts the new rule id. Flag-off snapshot re-recorded: 37 of 2,041 response bodies changed (rule ids, deduplicated qualifier rows, the one verdict above).

Why: On 2026-10-25 a customer’s agent pipeline (the Agent Factory) ran its website assistant against production /v1/verify (fast mode): paraphrases that kept every condition were refused 4 of 4, three character-for-character copies of source sentences were refused (one contradicted / number_mismatch on "24 hours a day", two qualifier_loss on verbatim plan lines), and a fact phrased twice in the sources came back conflicting_sources. The cases, with every source text as sent, are in bench-private/agent-factory-2026-10-25/ (docs/rules-3.0-plan.md has the root causes). Reproduced in code with the production extractor's claims and a deterministic stand-in model (repro.ts), fixed on dev material only, then frozen before fresh set #3 (bench-private/fresh-3, pre-registered in docs/kill-criteria.md: no rise in false approvals over rules 2.1.0 on the same set, Group C fails 100%, Groups A and B pass, Group E does not regress, sent-to-a-person ≤ 30%). Disclosure: the fresh-3 templates were written by the same person after the fix, with frequency phrases included so the set can show a loosening there.

An exception written after the thing it limits is a condition: an answer that stops before "final-sale items excluded", "gift cards not included", "clearance items are not eligible", "other than X", "with the exception of X" or "does not apply to X" drops a condition and is not released.

• qualifiers.ts: a trailing-exception pass reads a noun phrase of at most seven words, after a comma, semicolon, colon, parenthesis or sentence start, followed by "excluded", "not included", "not eligible" or "ineligible", as a condition; "with the exception of", "does not apply to", "do not apply to", "doesn't apply to", "don't apply to" and "not applicable to" join the condition triggers ("other than" already was one).
• sentences.ts (clauseContext): a following clause of the same sentence that is an exception (after ";" or a dash) is part of the context the claim's conditions are read from.
• rules.ts (verbatim qualifiers): an explicit exception in a following sentence that limits this one ("This does not apply to final-sale items.") is a dropped condition, no longer an ambiguous one that sends the claim to adjudication.
• Minor under this file's policy: a verdict can only move toward holding (supported or unverifiable to unsupported); nothing that was held is released. Checked on every benchmark item after the run: 0 of 2,040 predictions changed in either direction.
• Fixtures: packages/verify-core/test/rules-2.1.test.ts (the production final-sale case, each form, the forward forms, and controls: the restated exception, an unrelated next sentence, a long clause). rules-next RN-002 released; RN-001 stays a pinned fixture with no rule change.

Why: RN-002 (found 2026-10-18 preparing the homepage demo): "within 30 days of delivery, final-sale items excluded" was not recognised, so an answer that left the exclusion out was supported verbatim and released, by code and by the production pipeline. Developed on the dev split only (verify-bench v1.1 dev + v1-alpha dev, n=1220): dev before = after on every metric (false-supported 1/916, supported recall 210/304, qualifier-loss detection 225/226, contradiction detection 389/429, abstention 405/1220), 0 of 1,220 dev predictions changed; the benchmark has no item where a trailing exception decides the verdict, so the rule is carried by the new fixtures. Disclosure: before the freeze, the source-freshness non-regression test (which hashes every benchmark response, held-out included, without scoring) was run once and showed that some responses changed; no held-out score was computed before the freeze. Re-recorded snapshot: 6 of 2,041 response bodies changed (new condition rows; no verdict changed).

A claim extracted as a fragment of an answer clause is no longer flagged for dropping a condition that the answer clause itself states.

• engine.ts (answer-clause check): when a claim is unsupported for a dropped qualifier, and every dropped qualifier is stated in the clause of the answer the claim sits in (clauseContext, so a condition in another clause of the answer does not count), the claim is a fragment split off by extraction, and the answer clause is decided instead, through the same pipeline (code first, then the model path when one is configured). The result carries rule_id R-ANSWER-CLAUSE:<rule> and a check naming the condition the clause states.
• Major under this file's policy: it can flip a claim from unsupported to supported, so it can release an answer an earlier version held back. A condition absent from the answer clause is still flagged, unchanged.
• 1.1.0, 1.2.0 and 1.3.0 were never released; under our versioning policy they would have been major versions, and they are folded into 2.0.0.

Why: The production smoke check on 2026-10-18 (the /verify demo) flagged "Approved refunds are returned to the original payment method." for dropping "within 5–10 business days": the extractor had returned that fragment beside the full sentence, which the answer states with the window. On the dev split only (verify-bench v1.1 dev + v1-alpha dev, n=1220) the same pattern appears with code's own clause split: 2 of 218 qualifier-loss claims, both on gold-supported items (a split at ", and" inside a parenthesis, and at ", but no later than" after a leading condition). Dev before → after: supported recall 208/304 = 68.4% [63.0–73.4] → 210/304 = 69.1% [63.7–74.0]; false-supported 1/916, qualifier-loss detection 225/226, contradiction detection 389/429 and abstention 405/1220 unchanged. Regression fixture: packages/verify-core/test/fragment-answer-context.test.ts (the production case, plus a condition in another clause and a condition absent from the answer, both still flagged).

A stated total whose own figures add up is no longer reported as a contradiction, one that does not add up is contradicted by code, and invisible characters in an answer no longer change its sentences or numbers.

• claimComputation (checks.ts): code now recognises three more computations stated inside a claim, each reported pass or fail: a total of two or more figures in the same unit joined by "and"/"plus", with the cue directly before the total ("15 days … and 5 days …, for a total of 20 days"); a difference introduced by a consequence word and followed by more/fewer/less/extra ("… so the Business plan includes 240,000 more requests"); and a monthly price over a year ("$49 per month, so a full year costs $588"; never when the claim mentions a discount or annual billing). Figures joined by "or", mixed units, or a cue elsewhere in the sentence are not read as a computation.
• A wrong stated computation whose operand is in a passage is contradicted by code (R-ARITH-CONTRADICTED, arithmetic_error), as the existing percent, discount and unit-price checks already were.
• finishSupport: a claim figure that is the result of a computation code recomputes from values in the cited evidence (the claim's own stated computation, or the model's arithmetic when the claim carries a computation cue) is derived, not a different value for the same fact: it no longer yields number_mismatch or unsupported_detail. Without a computation cue the conflict stands: model arithmetic cannot explain away "60 days" against "30 days".
• Segmentation (sentences.ts) treats zero-width characters (U+200B, U+200C, U+200D, U+2060, U+FEFF) as whitespace: ". \u200bThe" ends a sentence like ". The", clause breaks see through them, and sentence ranges are trimmed of them.
• Number and date extraction (numbers.ts, foldSpaces) reads no-break, thin and zero-width spaces as plain spaces one for one, so "5\u00a0business days" is five business days and offsets into the original text are unchanged; multi-word units (business days, calendar days) accept any run of whitespace.

Why: The Verify Gate regression corpus (tests/fixtures/gate-corpus, 20 categories) found that every arithmetic control (a total that adds up) was blocked in local mode: "15 days … and 5 days …, for a total of 20 days" with its declared source quote came back contradicted (20 days against 15 days), a deterministic false block, while the wrong total (25 days) only warned. In hosted mode the same happened through the model path. The corpus's Unicode-noise check found that a zero-width space after a full stop merged two sentences and that a no-break space detached "business days" from its number. Developed against the corpus and the dev split only: the verify-bench v1.1 + v1-alpha dev output (n=1220) is byte-identical before and after (no item states a total or carries these characters), and the 2041-item flag-off response snapshot is unchanged.

A model can no longer confirm support across a polarity difference that code finds in the quoted span.

• R-SUPPORT-POLARITY-CONFLICT: when a model proposes 'supported' and the quoted span carries the claim's content words (80% or more) but negates them where the claim does not, or the reverse, the claim is unsupported with failure_reason unverified_contradiction. No tier, and no fast-tier fallback, may overrule it.
• Known limitation kept and pinned by tests: a sentence under five tokens that flips its source's polarity ("Monthly plans are refundable." against "Monthly plans are not refundable.") is below the template rule's minimum, so code alone does not decide it; it comes back unverifiable (Gate: warn) in local mode, and is never supported in hosted mode.

Why: A test pinning the short-sentence negation case found that in model-assisted mode a credulous model quoting the negated sentence as support got the claim released as supported, because code flagged the polarity difference only as ambiguous and the strong tier resolved ambiguity. On the dev split, a credulous adjudicator's false-supported count fell from 86/916 to 82/916 (supported recall 285/304 to 283/304); developed on dev only.

Catches deadlines and conditions dropped around a verbatim quote, and a claim made firmer than its source ("may" → "will"); NFKC in span matching.

• Deadline phrases are time windows by themselves: "not later than", "in no event later than", "no/not earlier than"; ordinal days ("the seventh business day"), business days and billing cycles count as time anchors.
• A time window after a comma (", within 2 billing cycles", ", no later than the date listed …") limits the clause before it, when it names a time.
• A qualifier lying wholly outside a verbatim span is dropped unless the claim restates it word for word; its words turning up elsewhere in the claim no longer count as a restatement.
• A span that stops on a dangling conjunction ("…as soon as possible, but") takes the following clause into its context.
• R-TEMPLATE-MODALITY: modal verbs are aligned like values. A claim identical to a source sentence except that it makes firmer what the source leaves open (may/might/can/could → will/must/shall; should → must/shall) is unsupported with failure_reason qualifier_loss and a conflicting modality row. A softer claim than its source is left undecided.
• Span matching applies NFKC per grapheme with exact original offsets (ligatures, full-width forms, compatibility spaces). Superscripts, subscripts, vulgar fractions and enclosed numbers are never folded, so 10² never matches 102.

Why: The first run of 1.0.0 on the verify-bench v1.1 held-out split released 12 of 154 dropped-deadline and dropped-condition answers as supported. The same pattern on the dev split (18 of 225) traced to the gaps above; each fix was developed and tested on dev only, then frozen before this version's single held-out run. NFKC and the modality rule are requirements of Verify Gate's local mode (exact span validation under Unicode variation; "may" → "will" fixtures).

The rules Verify v1 shipped with.

• Span matching in code: exact, then normalised (whitespace, quote and dash style, ellipsis, sentence-initial case), then punctuation-tolerant under a hard edit-distance cap; offsets always map back to the original source text and a model-proposed quote that is not found is discarded.
• Qualifier engine: dates and time ranges, entities, quantities and units, conditions, scope, modality and negation compared in code; each material qualifier is preserved, dropped, contradicted or ambiguous.
• Code-only verdict rules: arithmetic recomputed, verbatim support, verbatim qualifier loss, one-value and negation template contradictions, superseded and conflicting passages, external comparisons and out-of-scope predictions.
• Model adjudication is a proposal checked in code; a model can never produce 'supported' without a located span. Deliberation can only downgrade a verdict to unverifiable.
• Release recommendation: revise > human_review > cite_required > allow, with max_unsupported 0 and max_unverifiable 0 by default.
• Prompts verify-extract-2026-10-15.1 and verify-adjudicate-2026-10-15.1.

Why: First public version, registered before the first held-out run (docs/benchmark-methodology.md, SHA-256 060dc63d…7457). Six faults found on the dev split before registration were fixed as regression tests; the held-out split was not read.

Two Gate findings no longer claim more than the check established, and the policy track now covers the case evaluator.

• Excessive-unresolved hold (XYN-006): counts only claims the hosted verifier tried and could not decide. A claim it classified (comparative, out_of_scope, conflicting_sources) is an XYN-005 warning, not a check that could not complete; a one-sentence answer with an industry comparison no longer holds the pull request.
• Wrong citation (XYN-004): raised only for a decided claim (supported, contradicted or unsupported), and deterministic only when that decision was. A claim code could not settle no longer blocks as 'cites a passage that does not support it' (unknown), and a model-decided one warns unless model_findings: block. A missing required citation is still read off the answer and still blocks.
• Case evaluator (evaluate.ts): when hosted mode sends an expected claim's sentence to the engine and the engine splits it into clauses, every clause is kept as a claim of that expected claim. Before, only the first clause was kept, so a wrong second clause ("…, so the Business plan includes 225,000 more requests") was dropped and the case could pass.
• evaluate.ts and claims.ts join defaults.ts and policy.ts in the gate_policy manifest: they decide which claims the policy sees.

Why: Found by the Verify Gate regression corpus: two external-comparison fixtures held instead of warning, a correctly cited claim code could not settle blocked as a wrong citation, and the hosted arithmetic fixture passed a wrong difference because its clause was dropped. Precision first: nothing blocks on a finding the check did not establish.

Verify Gate v0.1 policy: pass / warn / hold / block over Verify claim results, with precision-first defaults.

• Decisions: hold (XYN-006: the check could not complete; never converted to pass, never overridable) > block > warn > pass.
• Defaults: block on material contradictions (XYN-001: max_contradicted critical/high/medium 0) and material dropped or conflicting qualifiers (XYN-002); block on critical/high unsupported claims (XYN-003: max_unsupported critical/high 0); block on a missing required citation on a material claim (XYN-004); unresolved claims warn (treat_unverifiable_as: warn).
• A block-level finding that depends on a model judgement is reported as warn unless model_findings: block. A missing expected claim warns unless missing_expected_claim: block (live answers are reworded run to run).
• Hosted mode holds when more than max_unresolved_ratio (0.5) of material claims the verifier tried stay unresolved.
• New installations are report-only (enforcement: report_only); overrides need the verify-gate:override label from a writer plus a reasoned /verify-override comment.

Why: A gate that blocks good changes gets uninstalled, so out of the box only code-confirmed contradictions and dropped conditions on material claims block, and nothing blocks on a low-confidence model judgement (Master Reference §10).

Source freshness v0: whether the source a claim relies on is dated, and whether a controlling newer supplied source states otherwise. Reported separately; never changes a verdict.

• Optional source fields effective_date, published_at, version, supersedes[] and authority_rank (primary | secondary), read only while the verify_freshness_enabled flag is on.
• A separate response object `freshness` per request and per claim: established | unknown | conflicted_by_newer_source, with the cited source, the newer source, the conflict type and an explanation.
• A newer source is never presumed controlling: only `supersedes` naming the cited source, or a newer primary source over a secondary one, makes it so. A newer non-controlling source that differs leaves the status unknown.
• Release effect is reported inside the freshness object (recommended_release: human_review when conflicted, or when unknown and require_dated_sources is set); the response's own release_recommendation is unchanged.

Why: An answer can quote its source exactly and still rely on a version the company has since replaced. Master Reference v5.2 specifies freshness reported apart from support, so the two are never conflated.