Coverage for plugin/scripts/check_prose_counts.py: 100%
78 statements
« prev ^ index » next coverage.py v7.16.1, created at 2026-09-15 14:46 +0000
1#!/usr/bin/env python3
2"""Check that counted claims in the prose still match the tree.
4Every prose surface in this repo is gated except one. ``check_docs_nav.py`` covers
5``docs/``, ``check_command_contracts.py`` covers the commands, ``check_prompt_wiring.py``
6covers the procedures, and ``check_doc_examples.py`` covers the README's fences and the
7docstrings. ``paper/`` has nothing — and it is the surface ``.bumpversion.toml`` stamps
8with the release number, in five places, one of which reads "Every claim here is true of
9release vX.Y.Z". An unchecked claim there is not merely stale; each release re-asserts it.
11It had already happened twice when this was written. The paper said "nine user-facing
12commands" and omitted ``/rhiza:completions`` from its table for the whole of v0.10.0. And
13``Makefile`` described "the nine workflows holding write permissions" against a tree
14holding ten — which read as true again only because one was later deleted, not because
15anyone noticed. Both are the same kind of claim, and it is the kind worth gating: **a
16count of files that exist.** That is not a matter of judgement, and the tree is the
17authority.
19============ =====================================
20subject counted from
21============ =====================================
22commands ``_rhiza_layout.command_files(root)``
23procedures ``prompts/*.md``
24workflows ``.github/workflows/*.y{a,}ml``
25============ =====================================
27**A claim is marked, never guessed at.** The obvious design — read every number followed
28by one of those nouns — was written first and thrown away, because English does not
29cooperate: "``pr-base`` is read by three commands" and "the two commands that change a
30shared repository" are both correct, both counted, and neither is a total. Nothing in the
31grammar separates them from "ten slash commands"; only the author knows which is which. A
32gate that fails the build on correct prose is worse than no gate, because it gets
33disabled.
35So the author says so, with a marker on its own line, in whatever comment syntax the file
36speaks::
38 <!-- rhiza-count: commands --> a Markdown file
39 % rhiza-count: commands the paper
40 # rhiza-count: workflows the Makefile, or local.mk beside it
42The number itself stays in the prose and is stated **once** — the marker carries no count
43of its own to drift. Every subject named must then appear as a counted claim in the three
44lines that follow, so a marker left behind by a rewritten sentence fails too. One marker
45may name several subjects (``rhiza-count: commands procedures``) when one sentence carries
46several claims, which the paper's honest-scope line does.
48The cost of this design is honest and worth stating: **an unmarked claim is unchecked.**
49The gate covers what the author asserted, not everything a reader might read as an
50assertion. That is the trade for never being wrong about a sentence.
52**"skills" counts commands**, because that is what the paper calls them when describing
53the directory. The two are the same set by construction — the directory *is* the command.
55Usage:
56 uv run --python 3.12 --no-project python \
57 scripts/check_prose_counts.py [--root DIR]
59Exits 0 when every marked claim matches, 1 (listing each violation) otherwise.
60"""
62from __future__ import annotations
64import argparse
65import re
66import sys
67from pathlib import Path
69from _rhiza_layout import PROMPTS_DIR, command_files
71WORKFLOWS_DIR = ".github/workflows"
72"""Where the workflows live, relative to the repository root."""
74SCANNED = (
75 "paper/*.tex",
76 "Makefile",
77 "local.mk",
78 "README.md",
79 "CLAUDE.md",
80 "docs/*.md",
81 "docs/*/*.md",
82)
83"""The prose this gate reads, as globs relative to the repository root.
85Build outputs are absent by construction rather than by exclusion: ``docs/paper/`` holds a
86PDF and ``docs/reports/`` holds HTML and XML, so neither is reachable through a ``*.md``
87glob.
89``local.mk`` is here for the same reason ``Makefile`` is, and naming only the latter was a
90real hole: rhiza's v1.4 shim makes the ``Makefile`` template-owned, so a repo's own targets
91— and the comments explaining them — move to ``local.mk``. rhiza-claude's own
92"the eight workflows holding write permissions" claim made exactly that trip. A count does
93not stop being a count because the target it documents changed file.
94"""
96WINDOW = 3
97"""How many lines after a marker the claim it announces may appear in.
99Three rather than one because the paper wraps at 90 columns, so a claim is routinely split
100across a line break — the first stale count found here read "of the ten\\nskills under".
101"""
103_NUMBER_WORDS = {
104 "one": 1,
105 "two": 2,
106 "three": 3,
107 "four": 4,
108 "five": 5,
109 "six": 6,
110 "seven": 7,
111 "eight": 8,
112 "nine": 9,
113 "ten": 10,
114 "eleven": 11,
115 "twelve": 12,
116 "thirteen": 13,
117 "fourteen": 14,
118 "fifteen": 15,
119 "sixteen": 16,
120 "seventeen": 17,
121 "eighteen": 18,
122 "nineteen": 19,
123 "twenty": 20,
124}
126_NOUNS = {
127 "commands": "commands",
128 "skills": "commands",
129 "procedures": "procedures",
130 "workflows": "workflows",
131}
133SUBJECTS = sorted(set(_NOUNS.values()))
134"""The subjects a marker may name."""
136# `rhiza-count:` in any comment syntax — the leading `%`, `#` or `<!--` is not matched at
137# all, so a new file type needs no change here.
138_MARKER = re.compile(r"rhiza-count:\s*(?P<subjects>[a-z][a-z ]*)")
140# A number, then at most two intervening words, then a noun. `\s+` spans newlines because
141# the window is searched as joined text, and IGNORECASE because a claim opening a sentence
142# is capitalised — "Eight user-facing commands ship" is the same claim as "eight".
143#
144# The filler is **lazy**. Greedy, it reaches past the nearer noun to a further one: in
145# "two commands, one procedures" the match starting at "two" consumed `commands` and `one`
146# as filler and reported two *procedures*, hiding both real claims behind one wrong span.
147_CLAIM = re.compile(
148 r"\b(?P<count>" + "|".join(_NUMBER_WORDS) + r"|\d+)"
149 r"(?:\s+[A-Za-z][\w-]*){0,2}?"
150 r"\s+(?P<noun>" + "|".join(_NOUNS) + r")\b",
151 re.IGNORECASE,
152)
155def parse_count(token: str) -> int:
156 """Read *token* as a number, written either as a word or as digits.
158 >>> parse_count("nine")
159 9
160 >>> parse_count("Eight")
161 8
162 >>> parse_count("12")
163 12
164 """
165 word = token.lower()
166 return _NUMBER_WORDS[word] if word in _NUMBER_WORDS else int(word)
169def marked_subjects(line: str) -> list[str]:
170 """The subjects a ``rhiza-count:`` marker on *line* names, or ``[]`` if it has none.
172 >>> marked_subjects("% rhiza-count: commands procedures")
173 ['commands', 'procedures']
174 >>> marked_subjects("just prose about commands")
175 []
177 An unknown subject is returned as written, so the caller can report it rather than
178 silently ignoring a marker that will never match anything:
180 >>> marked_subjects("<!-- rhiza-count: sprockets -->")
181 ['sprockets']
182 """
183 match = _MARKER.search(line)
184 return match["subjects"].split() if match else []
187def unmark(text: str) -> str:
188 """Replace every non-alphanumeric character in *text* with a space.
190 Markup sits between a number and its noun often enough to matter: the README writes
191 "eight ``**internal procedures**``" and the paper writes
192 ``\\textbf{slash commands}``. Matching around each syntax in turn is how a regex grows
193 a dialect per file type, so the window is flattened to words first.
195 Newlines survive, because they are what a reported line number is counted from — and
196 ``\\s+`` spans them anyway, so a claim split across a line break still matches:
198 >>> unmark("eight **internal procedures**")
199 'eight internal procedures '
200 >>> unmark("of the ten\\nskills")
201 'of the ten\\nskills'
202 """
203 return re.sub(r"[^0-9A-Za-z\n]", " ", text)
206def claimed(text: str, subject: str) -> tuple[int, int] | None:
207 """What *text* claims for *subject*, as ``(count, line offset)``.
209 The offset is counted from the start of *text* and points at the **number**, not at
210 the marker that announced it — a claim may sit up to `WINDOW` lines away, and the line
211 worth reporting is the one an editor has to change.
213 >>> claimed("ten user-facing commands, eight internal procedures", "procedures")
214 (8, 0)
215 >>> claimed("a catalogue of the ten\\nskills under skills/", "commands")
216 (10, 0)
217 >>> claimed("marker\\nholds eight **internal procedures**", "procedures")
218 (8, 1)
219 >>> claimed("nothing counted here", "workflows") is None
220 True
221 """
222 flattened = unmark(text)
223 for match in _CLAIM.finditer(flattened):
224 if _NOUNS[match["noun"].lower()] == subject:
225 return parse_count(match["count"]), flattened.count("\n", 0, match.start())
226 return None
229def tally(root: Path) -> dict[str, int]:
230 """Count what the tree at *root* actually holds, per subject."""
231 workflows = (root / WORKFLOWS_DIR).glob("*.yml"), (root / WORKFLOWS_DIR).glob("*.yaml")
232 return {
233 "commands": len(command_files(root)),
234 "procedures": len(list((root / PROMPTS_DIR).glob("*.md"))),
235 "workflows": sum(len(list(found)) for found in workflows),
236 }
239def scanned_files(root: Path) -> list[Path]:
240 """Every prose file under *root* this gate reads, sorted and deduplicated."""
241 found = {path for glob in SCANNED for path in root.glob(glob) if path.is_file()}
242 return sorted(found)
245def check_file(relative: str, text: str, actual: dict[str, int]) -> tuple[list[str], int]:
246 """Check every marked claim in *text*; return the violations and how many ran."""
247 violations: list[str] = []
248 checked = 0
249 lines = text.splitlines()
250 for index, line in enumerate(lines):
251 for subject in marked_subjects(line):
252 where = f"{relative}:{index + 1}"
253 if subject not in actual:
254 violations.append(
255 f"{where} marks `{subject}`, which is not a counted subject "
256 f"({', '.join(SUBJECTS)})"
257 )
258 continue
259 checked += 1
260 # The marker's own line is part of the window, so a marker may sit inline with
261 # the claim as well as above it. A table row cannot carry a comment line
262 # between it and the row before without ending the table.
263 window = "\n".join(lines[index : index + 1 + WINDOW])
264 found = claimed(window, subject)
265 if found is None:
266 violations.append(
267 f"{where} marks `{subject}` but no claim about {subject} follows it "
268 f"within {WINDOW} line(s)"
269 )
270 elif found[0] != actual[subject]:
271 violations.append(
272 f"{relative}:{index + 1 + found[1]} claims {found[0]} {subject}, "
273 f"but the tree holds {actual[subject]}"
274 )
275 return violations, checked
278def check_prose_counts(root: Path) -> tuple[list[str], int]:
279 """Check every marked claim under *root*; return the violations and how many ran."""
280 actual = tally(root)
281 violations: list[str] = []
282 checked = 0
283 for path in scanned_files(root):
284 found, ran = check_file(
285 path.relative_to(root).as_posix(), path.read_text(encoding="utf-8"), actual
286 )
287 violations += found
288 checked += ran
289 return violations, checked
292def main(argv: list[str] | None = None) -> int:
293 """Entry point: check the prose's counted claims and return an exit code."""
294 parser = argparse.ArgumentParser(description="Check counted claims in the prose.")
295 parser.add_argument("--root", default=".", help="Repository root (default: current directory).")
296 args = parser.parse_args(argv)
298 root = Path(args.root).resolve()
299 violations, checked = check_prose_counts(root)
300 if violations:
301 print("Prose count check failed:", file=sys.stderr)
302 for violation in violations:
303 print(f" ✗ {violation}", file=sys.stderr)
304 return 1
306 print(f"prose counts match the tree ({checked} marked claim(s) checked)")
307 return 0
310if __name__ == "__main__":
311 raise SystemExit(main())