Coverage for src/rhiza_task/tasks/fences.py: 100%
188 statements
« prev ^ index » next coverage.py v7.15.4, created at 2026-10-13 02:27 +0000
1"""The fenced-example checker behind ``docs-examples``.
3Extracted from :mod:`rhiza_task.tasks.quality`, which had grown to 904 lines holding this
4alongside seven unrelated task bodies -- and a module docstring describing only the latter.
5Nothing here breached the complexity ceiling; the maintainability index falling 44.55 to
636.52 over the two changes that built this up was the signal, and cohesion was the actual
7problem. See issue #113.
9**One name crosses the boundary: :func:`check`.** That is deliberate and not merely tidy.
10``CLAUDE.md``'s layering invariant forbids an underscore-prefixed name from crossing a
11module boundary, so keeping the ``@task`` in ``quality.py`` while moving the helpers here
12would have meant making seven private functions public -- turning an internal decomposition
13into an API. A single entry point taking the gate's whole body instead leaves the helpers
14private, where they belong.
16What the gate is *for* stays on the task in ``quality.py``, because that is where a reader
17looking for a gate looks. What it *does* is here.
19**This module is the largest in ``src/`` -- roughly 900 lines since ``result`` left, with a
20maintainability index in the low 30s at the time of writing -- and that is accepted.** Worth
21stating outright, because the index is *lower* than the 36.52 that got the checker extracted from
22``quality.py`` in the first place, so the numbers alone read as having made things worse.
23They did not, and the reason is the one #113 was actually about: ``quality.py`` was 904 lines
24doing **two** jobs with a docstring describing one of them, and this is one job. Radon's
25maintainability index falls with size and Halstead volume whether or not a module is
26coherent, so it cannot tell those apart -- which is why cohesion, not the index, was the
27argument then and is the argument now.
29The figures are written loosely and dated on purpose, following ``pyproject.toml``'s note on
30the coverage floor: they move with every commit, they are incidental to the argument, and a
31comment stating one exactly is a comment the next edit falsifies. This paragraph proved it --
32adding it grew the file and dropped the index, so the precise numbers it first carried were
33wrong by the time it was saved. ``uvx radon mi src -s`` prints the current figure. What does
34not move: nothing here exceeds B (10), the average across ``src/`` is A, and about a
35seventh of these lines are comments, which is this repository's house style rather than
36padding.
38The condition that changes the answer is **a second job arriving, not a line count**. If
39something lands here that is not fenced-example checking, split on that seam. Splitting on
40size alone is the move to resist: the obvious cut -- the parser (``Fence``, :func:`_fences`,
41the language constants) away from the five checkers -- would put a type in one module and
42its only five consumers in another, which is worse than a long file. ``pycon`` arriving did
43not change that answer: doctesting a transcript is fenced-example checking, the same job.
45**Accumulation is the other ceiling, and it is what retired ``result``.** ``ci.yml`` caps
46how many blocks at CC >= 6 one module may hold, and ``pycon`` brought this one to that cap.
47A decomposition was weighed and not taken: every cut needs ``Fence`` on both sides, which
48means either the parser split the paragraph above resists or an import cycle, so the
49cheaper way under the cap was to stop carrying the superseded form. See #196.
51Five kinds of fence are checked and the rest are counted:
53* ``pycon`` -- **the convention for a runnable example**: a doctest transcript, ``>>>`` and
54 ``...`` prompts with the expected output inline. Every pycon fence in one file is joined,
55 in document order, into a single doctest with one shared namespace and ``ELLIPSIS``, and
56 run in the project environment -- so a stale output is reported at the file and line of
57 the example that printed it. A fence flagged ``+RHIZA_SKIP`` in its info string is left
58 out and counted as unchecked, the flag pytest-rhiza reads in ``README.md``.
59* ``python`` -- :func:`compile`, so a fence that is a *fragment* still passes. Names need not
60 resolve; only the syntax is asserted.
61* ``bash``/``sh`` -- ``bash -n``, which parses without executing. Never executed, because a
62 README's shell is routinely ``rm -rf`` and ``git push``, and an unparseable fence is a
63 documentation bug without running it.
64* ``toml`` -- :func:`tomllib.loads`, in this process. Stdlib at this package's Python floor,
65 so it needs nothing provisioned and is checked on every machine that can run the gate.
66* ``yaml``/``yml`` -- a real parser, provisioned into a subprocess because adding one to this
67 package's runtime dependencies to serve two fences would be the wrong trade: it is a
68 published CLI whose install cost every consumer pays on every ``uvx`` invocation.
70Anything else -- ``mermaid``, ``makefile``, fences carrying no language at all, and the
71retired ``result`` block, once diffed against the python fences above it and superseded by
72``pycon`` -- is reported as unchecked with a count. Naming the count is the point: silence
73there would read as "everything was checked", and for ``result`` in particular it would hide
74that a block the gate used to diff no longer is.
76Two of the five can go unavailable on a machine that runs the gate fine otherwise, and both
77follow the rule the tool guards elsewhere in this package follow -- the fences are counted
78out of the checked total and named on their own line, never assumed sound. ``bash`` may be
79absent, and the yaml parser has to be fetched.
81Parsing is *not* validation, and the distinction is worth keeping: a ``toml`` fence that
82parses may still name a setting this package does not have, and a ``yaml`` fence that parses
83may still be an invalid workflow. Schema-checking either would need the schema, and for the
84workflow snippets actionlint already owns that question over the real files. What this closes
85is the narrower gap where a fence stopped being the language it claims.
86"""
88from __future__ import annotations
90import json
91import re
92import shutil
94# The `bash -n` probe below is a fixed argument vector, with no shell -- which is what bandit's
95# B404 asks about. The reason sits here rather than on the suppression comment itself: bandit
96# reads everything after that marker as a comma-separated list of test IDs, so a trailing
97# explanation becomes one `Test in comment:` warning per word.
98import subprocess # nosec B404
99import textwrap
100import tomllib
101from collections import Counter
102from dataclasses import dataclass
103from pathlib import Path
105from ..config import Config
106from ..spec import Failed, Skip
107from ..uv import uv_run
110@dataclass(frozen=True)
111class Scan:
112 """One pass over the documentation, before any checking.
114 A record for the reason :class:`Tally` is one: :func:`check` reached C (12) once README
115 joined the sources, which would have made it the third C-ranked block in this package.
116 ``CLAUDE.md`` asks for an argument per C block and this one had none worth making --
117 parsing, checking and reporting are three jobs, and splitting them is the decomposition
118 the metric was pointing at rather than a contortion to satisfy it. Passing the four
119 results positionally between those parts is what a record avoids.
121 Attributes:
122 per_file: The docs tree's fences, grouped by file, because a file's pycon fences share
123 one doctest namespace *in that file* and nowhere else.
124 fences: The same fences, flat. The docs tree only -- the code checkers' subject.
125 readme: ``README.md``'s data fences, and none of its code fences. Empty when there is
126 no README. See :func:`_readme_fences`.
127 bash: Absolute path to bash, or None when it is absent -- resolved once, because its
128 absence must leave the shell fences counted rather than fail the gate.
129 """
131 per_file: list[list[Fence]]
132 fences: list[Fence]
133 readme: list[Fence]
134 bash: str | None
136 @property
137 def data(self) -> list[Fence]:
138 """Return the fences the data checkers see: the docs tree's and README's together.
140 One list rather than a second pass over README, because :func:`_yaml_violations` owns a
141 single scratch folder -- calling it twice would have the second run clobber the first's
142 markers -- and one subprocess for every yaml fence is cheaper anyway.
144 Returns:
145 Every fence from both sources.
146 """
147 return self.fences + self.readme
150def check(cfg: Config) -> None:
151 """Run the gate: parse every checkable fence, doctest the pycon ones, print the inventory.
153 The body of the ``docs-examples`` task, and the only public name in this module. The task
154 itself is registered in :mod:`rhiza_task.tasks.quality` and carries the argument for why
155 the gate exists.
157 Three steps, each its own function: :func:`_scan` parses, this gathers the violations, and
158 :func:`_verdict` prints and decides. See :class:`Scan` for why that split exists.
160 Args:
161 cfg: The resolved config.
163 Raises:
164 Skip: When nothing was measured. Documentation carrying nothing runnable would
165 otherwise score a silent pass, which is the failure this gate exists to make
166 visible.
167 Failed: When at least one example is broken or stale.
168 """
169 scratch = cfg.root / "_tests" / "docs-examples"
170 scratch.mkdir(parents=True, exist_ok=True)
171 scan = _scan(cfg)
173 # None, not [], when the parser could not be provisioned: an empty list is
174 # "checked, all sound", and reporting a machine's missing network as a clean bill of
175 # health is the one thing this gate must never do. `bash` is handled the same way, in
176 # `_scan`, by the same argument.
177 yaml_broken = _yaml_violations(scan.data, cfg, scratch)
179 broken = [
180 *_syntax_violations(scan.fences),
181 *(_shell_violations(scan.fences, scan.bash, scratch) if scan.bash else []),
182 *_toml_violations(scan.data),
183 *(yaml_broken or []),
184 *_pycon_violations(scan.per_file, cfg, scratch),
185 ]
186 _verdict(cfg, scan, broken, yaml_broken is not None)
189def _scan(cfg: Config) -> Scan:
190 """Parse every markdown file this gate looks at, and resolve bash once.
192 Args:
193 cfg: The resolved config.
195 Returns:
196 What was found, as a :class:`Scan`.
197 """
198 per_file = [
199 _fences(md.relative_to(cfg.root).as_posix(), md.read_text(errors="replace"))
200 for md in sorted(cfg.path("docs_folder").rglob("*.md"))
201 ]
202 return Scan(
203 per_file=per_file,
204 fences=[fence for one_file in per_file for fence in one_file],
205 readme=_readme_fences(cfg),
206 # Resolved once: `bash` is absent on a stock Windows runner, and its absence must leave
207 # the shell fences *unchecked and counted* rather than failing the gate for a fact
208 # about the machine. Same reasoning as `Guard(tool=...)` raising Skip rather than
209 # Failed.
210 bash=shutil.which("bash"),
211 )
214def _verdict(cfg: Config, scan: Scan, broken: list[str], yaml_ran: bool) -> None:
215 """Print the violations and the inventory, then raise the outcome.
217 Args:
218 cfg: The resolved config.
219 scan: What :func:`_scan` found.
220 broken: Every violation message, in the order they were gathered.
221 yaml_ran: Whether the yaml parser could be provisioned.
223 Raises:
224 Skip: When nothing was measured, so a pass would mean nothing.
225 Failed: When at least one example is broken or stale.
226 """
227 for violation in broken:
228 print(violation)
230 measured = _report(scan.fences, scan.bash, yaml_ran, len(scan.per_file))
231 if scan.readme:
232 # Its own line, and outside the inventory above, because README contributes only its
233 # data fences: folding two of its ten into "12 file(s), 64 fence(s)" would read as
234 # full coverage of a file this gate deliberately only half-looks at.
235 print(f"[INFO] README.md: {len(scan.readme)} data fence(s) checked; its code fences are pytest-rhiza's")
236 if not measured and not scan.readme:
237 raise Skip(f"no checkable fence under {cfg.docs_folder}")
238 if broken:
239 # `and README.md` rather than only the docs folder: since data fences are checked in
240 # both, a summary naming one scope would point a reader at the wrong file for half the
241 # failures it reports. The per-violation lines above carry the real locations.
242 scope = f"{cfg.docs_folder} and README.md" if scan.readme else cfg.docs_folder
243 raise Failed(1, f"{len(broken)} broken example(s) under {scope}")
246# Deliberately permissive about what follows the language: mkdocs-material accepts
247# ```python title="x", and an opening fence this pattern failed to match would have its
248# *closing* fence read as the next opening one, cascading the misparse through the rest of
249# the file. Matching any fence line and keeping only the first word cannot do that.
250DOC_FENCE_OPEN = re.compile(r"^(?P<indent>[ \t]*)```(?P<language>[^`\s]*)(?P<flags>.*)")
252# A closing fence carries nothing but backticks. A bare ``` therefore matches both patterns,
253# which is why :func:`_fences` tracks state instead of classifying lines independently.
254DOC_FENCE_CLOSE = re.compile(r"^[ \t]*```[ \t]*$")
256# `bash` and `sh` only. `console` and `shell-session` are excluded on purpose: their content
257# is a transcript -- prompts, output and all -- so `bash -n` would reject the very thing that
258# makes them correct. A language this set does not name is counted as unchecked and reported
259# rather than guessed at.
260SHELL_FENCE_LANGUAGES = frozenset({"bash", "sh"})
262PYTHON_FENCE_LANGUAGE = "python"
264# The convention for a runnable example across the fleet: a standard doctest transcript. The
265# same spelling pytest-rhiza doctests in README.md, so one example reads the same way in the
266# README and in the book.
267PYCON_FENCE_LANGUAGE = "pycon"
269# The info-string flag that takes one fence out of execution, spelled as pytest-rhiza spells
270# it so a fence copied between README.md and the docs tree keeps its meaning.
271SKIP_FLAG = "+RHIZA_SKIP"
273# `tomllib` is stdlib at the floor this package declares (`requires-python = ">=3.11"`), so
274# the toml half costs nothing to provision -- and it is the half that matters most here: the
275# toml fences under `docs/` are this package's own `[tool.rhiza-task]` and `[tool.bumpversion]`
276# examples, which is precisely the class that goes stale when a setting is renamed or a
277# default moves. Eleven of them at the time of writing, against two yaml.
278TOML_FENCE_LANGUAGE = "toml"
280# `yml` as well as `yaml`: mkdocs-material's own docs spell it both ways, and a fence this set
281# failed to name would be silently counted as unchecked rather than parsed -- the exact
282# silence this gate exists to break.
283YAML_FENCE_LANGUAGES = frozenset({"yaml", "yml"})
285# The driver :func:`_yaml_violations` writes out and runs. A module-level constant with its
286# body at column 0, rather than an indented literal inside that function, and the indentation
287# is the whole point: `CLAUDE.md` documents
288# `grep -rnE '^\s+(from|import) ' src/` as the way to check this package for deferred imports,
289# and tells the reader it returns two lines. An indented `import yaml` inside a string literal
290# matched that pattern and took it to four -- two false positives in the one check the
291# layering invariant is verified by, which is worse than the invariant being unchecked because
292# it teaches the reader to ignore the output. At column 0 nothing matches and the documented
293# count holds.
294#
295# `started.txt` is written before any parsing and `report.txt` after all of it, so the caller
296# can tell three outcomes apart that would otherwise collapse into one: the parser could not
297# be provisioned (no marker at all), the checker ran and crashed (marker, no report), and the
298# checker finished (both). Without the first marker a bug in this script is indistinguishable
299# from a machine with no network, and the gate passes green either way.
300YAML_CHECKER_SCRIPT = """\
301import pathlib
303import yaml
305here = pathlib.Path(__file__).parent / "yaml"
306(here / "started.txt").write_text("ok")
307broken = []
308for number, where in enumerate((here / "index.txt").read_text().splitlines()):
309 try:
310 yaml.safe_load((here / f"{number:04d}.yaml").read_text())
311 except yaml.YAMLError as exc:
312 # Flattened: a YAMLError's str spans four lines with a caret diagram, and the
313 # report this joins is one line per violation.
314 detail = " ".join(str(exc).split())
315 broken.append(f"{where}: yaml fence does not parse: {detail}")
316(here / "report.txt").write_text("\\n".join(broken))
317"""
319# The languages this gate checks in ``README.md`` as well as under the docs folder, and the
320# reason is a gap rather than a preference. ``README.md`` is pytest-rhiza's subject -- its
321# ``test_readme_validation`` runs under `rhiza-test` -- so this gate has always left the file
322# alone to keep one verdict per fact. But that module contains no reference to `toml` or
323# `yaml` at all, so README's data fences were checked by *nothing*: both of this repository's
324# are `[tool.rhiza-task]` and `rhiza.toml` examples naming real settings, which is exactly the
325# class that goes stale when a setting is renamed. See issue #112.
326#
327# So the two gates divide by *language* rather than by file, which keeps the no-double-verdict
328# rule intact and statable: pytest-rhiza owns README's code fences, this owns data fences
329# everywhere. Should pytest-rhiza ever learn toml, this set is the one place to narrow.
330DATA_FENCE_LANGUAGES = YAML_FENCE_LANGUAGES | {TOML_FENCE_LANGUAGE}
331CHECKED_FENCE_LANGUAGES = (
332 SHELL_FENCE_LANGUAGES | YAML_FENCE_LANGUAGES | {PYCON_FENCE_LANGUAGE, PYTHON_FENCE_LANGUAGE, TOML_FENCE_LANGUAGE}
333)
335# The driver :func:`_pycon_violations` writes out and runs, at column 0 for the reason
336# `YAML_CHECKER_SCRIPT` is: an indented import inside a string literal is a false positive in
337# the grep `CLAUDE.md` documents for deferred imports.
338#
339# `doctest` is stdlib, so unlike the yaml driver this provisions nothing extra; it runs in the
340# *project* environment, because the examples import the package. The
341# runner's two reporting hooks are overridden to collect one message per failing example in
342# place of doctest's own report, and the fence's opening line is added to each example's
343# offset so the message points at the file line a reader edits. The report is JSON rather
344# than one line per violation because a stale-output message spans three lines.
345#
346# `DocTestParser` raises ValueError on a malformed prompt -- `>>>x`, or a continuation line
347# indented differently from its `>>>` -- and that fence is reported and left out, while the
348# rest of the file still runs.
349PYCON_CHECKER_SCRIPT = """\
350import doctest
351import json
352import pathlib
354here = pathlib.Path(__file__).parent / "pycon"
355broken = []
358class Collector(doctest.DocTestRunner):
359 def report_failure(self, out, test, example, got):
360 broken.append(
361 f"{test.filename}:{example.lineno + 1}: pycon output is stale\\n"
362 f" expected: {example.want.strip()!r}\\n"
363 f" actual: {got.strip()!r}"
364 )
366 def report_unexpected_exception(self, out, test, example, exc_info):
367 kind, value = exc_info[0].__name__, exc_info[1]
368 broken.append(f"{test.filename}:{example.lineno + 1}: pycon example raised {kind}: {value}")
371parser = doctest.DocTestParser()
372runner = Collector(optionflags=doctest.ELLIPSIS)
373for document in json.loads((here / "input.json").read_text(encoding="utf-8")):
374 path, examples = document["path"], []
375 for line, code in document["fences"]:
376 try:
377 found = parser.get_examples(code, name=f"{path}:{line}")
378 except ValueError as exc:
379 broken.append(f"{path}:{line}: pycon fence is malformed: {exc}")
380 continue
381 for example in found:
382 # 0-based within the body, and body line 0 is the line after the opening fence.
383 example.lineno += line
384 examples.extend(found)
385 test = doctest.DocTest(examples, {"__name__": "__main__"}, path, path, 0, None)
386 runner.run(test, out=lambda _text: None)
387(here / "report.json").write_text(json.dumps(broken), encoding="utf-8")
388"""
391@dataclass(frozen=True)
392class Fence:
393 """One fenced code block, located and dedented.
395 Attributes:
396 path: Repository-relative path, posix-separated, because this reaches report output
397 a reader pastes into an editor.
398 line: 1-based line number of the opening fence.
399 language: The info string's first word, lowercased; empty when the fence carries none.
400 code: The block's content, dedented.
401 skip: Whether the info string carries :data:`SKIP_FLAG`. Honoured by the pycon check
402 alone, which is the one that executes; defaulted so a fence built by hand is an
403 ordinary one.
404 """
406 path: str
407 line: int
408 language: str
409 code: str
410 skip: bool = False
413@dataclass(frozen=True)
414class Tally:
415 """How many fences of each kind the tree holds.
417 A record rather than the tuple this used to be. Four counts unpacked positionally were
418 already at the edge of readable; toml and yaml take it to six, where
419 ``pycon, python, shell, toml, yaml, unchecked = _tally(fences)`` stops being checkable by
420 eye and a transposed pair would report shell fences as toml with nothing failing. The
421 fields carry the meaning instead.
423 Attributes:
424 pycon: ``pycon`` fences not flagged ``+RHIZA_SKIP``, doctested per file.
425 python: ``python`` fences, checked by :func:`compile`.
426 shell: ``bash``/``sh`` fences, checked by ``bash -n`` when bash is present.
427 toml: ``toml`` fences, checked in-process by :mod:`tomllib`.
428 yaml: ``yaml``/``yml`` fences, checked by a provisioned parser in a subprocess.
429 unchecked: Each remaining language paired with its count, commonest first and then
430 alphabetically so the report line is diffable between runs. A fence with no
431 language at all is counted under ``(none)``, and a skipped pycon fence under
432 ``pycon +RHIZA_SKIP`` -- named, because a skip is a fence nothing ran.
433 """
435 pycon: int
436 python: int
437 shell: int
438 toml: int
439 yaml: int
440 unchecked: list[tuple[str, int]]
443def _fences(path: str, text: str) -> list[Fence]:
444 """Return every fenced code block in one markdown file, in document order.
446 Indentation is why this is a state machine rather than a regex over the whole file:
447 mkdocs admonitions and content tabs indent their fences by four spaces, and ``faq.md``
448 indents one by three inside a numbered list. ``textwrap.dedent`` on the collected body is
449 what makes those compile -- without it every fence inside an admonition is an
450 ``IndentationError``, which would be a finding against this checker rather than the docs.
452 Nested fences are not handled, and cannot be: distinguishing them needs the four-backtick
453 form, which no file in this repository uses. If one appears, its inner fence closes the
454 outer block early and the languages reported go wrong -- visible in the inventory line
455 rather than silent, which is the reason that line prints a per-language count.
457 Args:
458 path: Repository-relative path, stored on each returned fence.
459 text: The file's content.
461 Returns:
462 The fences found, dedented.
463 """
464 fences: list[Fence] = []
465 language: str | None = None
466 start = 0
467 skip = False
468 body: list[str] = []
469 for number, line in enumerate(text.splitlines(), start=1):
470 if language is None:
471 opening = DOC_FENCE_OPEN.match(line)
472 if opening:
473 language, start, body = opening.group("language").lower(), number, []
474 skip = SKIP_FLAG in opening.group("flags").split()
475 continue
476 if DOC_FENCE_CLOSE.match(line):
477 fences.append(Fence(path, start, language, textwrap.dedent("\n".join(body)), skip))
478 language = None
479 continue
480 body.append(line)
481 return fences
484def _readme_fences(cfg: Config) -> list[Fence]:
485 """Return ``README.md``'s data fences, and none of its code fences.
487 The filter is the whole function. ``README.md`` belongs to pytest-rhiza's
488 ``test_readme_validation``, which doctests its pycon fences and parses its python and
489 shell ones, so taking the whole
490 file would make two gates report one fact -- the thing this gate has always refused to do.
491 Taking only the languages that module does not know about closes the gap without creating
492 the overlap: see :data:`DATA_FENCE_LANGUAGES` for why those are toml and yaml.
494 Absent rather than required: a repository need not have a README, and this gate's subject
495 is the docs tree.
497 Args:
498 cfg: The resolved config.
500 Returns:
501 The toml and yaml fences in ``README.md``, or an empty list when there is no README.
502 """
503 readme = cfg.root / "README.md"
504 if not readme.is_file():
505 return []
506 found = _fences("README.md", readme.read_text(errors="replace"))
507 return [fence for fence in found if fence.language in DATA_FENCE_LANGUAGES]
510def _syntax_violations(fences: list[Fence]) -> list[str]:
511 """Return one message per python fence that does not parse.
513 :func:`compile` rather than execution, so a fence holding a *fragment* -- ``guards =
514 (Guard("source_folder"),)`` in ``adding_a_task.md``, with ``Guard`` never imported --
515 passes. Undefined names are not the question; syntax is.
517 Args:
518 fences: Every fence in the tree.
520 Returns:
521 Violation messages, one per broken fence.
522 """
523 broken: list[str] = []
524 for fence in fences:
525 if fence.language != PYTHON_FENCE_LANGUAGE:
526 continue
527 try:
528 compile(fence.code, f"{fence.path}:{fence.line}", "exec")
529 except SyntaxError as exc:
530 # The *offending* line, not the fence's: `getting_started.md` holds 24 fences, and
531 # "somewhere in this file" is the part of a report a reader has to redo by hand.
532 # `fence.line` is the opening backticks, so body line 1 sits one below it, which is
533 # what makes this sum the absolute line rather than one short of it.
534 broken.append(f"{fence.path}:{fence.line + (exc.lineno or 0)}: python fence does not parse: {exc.msg}")
535 return broken
538def _shell_violations(fences: list[Fence], bash: str, scratch: Path) -> list[str]:
539 """Return one message per shell fence that does not parse.
541 ``-n`` is the whole point: bash reads and parses the script and exits without running a
542 command of it. So this validates ``rm -rf`` and ``git push`` fences without their
543 consequences.
545 Captured rather than streamed through :func:`~rhiza_task.uv.tool`, which every other
546 binary in this package goes through, for two reasons that both come from this being a
547 checker rather than a gate over one command: the message is wanted *per fence* and lives
548 on stderr, and echoing ``$ bash -n ...`` once per fence would bury the report it exists to
549 produce under twenty invocation lines. ``_git`` above captures for the same reason.
551 Args:
552 fences: Every fence in the tree.
553 bash: Absolute path to bash, already resolved by the caller.
554 scratch: Directory for the throwaway script.
556 Returns:
557 Violation messages, one per broken fence.
558 """
559 broken: list[str] = []
560 script = scratch / "fence.sh"
561 for fence in fences:
562 if fence.language not in SHELL_FENCE_LANGUAGES:
563 continue
564 script.write_text(fence.code)
565 checked = subprocess.run( # noqa: S603 # nosec B603
566 [bash, "-n", str(script)],
567 capture_output=True,
568 text=True,
569 check=False,
570 )
571 if checked.returncode:
572 # bash reports against the throwaway path and its own line numbers, neither of
573 # which the reader can act on. The fence's own location is already the prefix, so
574 # the script path is stripped to leave the diagnosis.
575 detail = checked.stderr.strip().splitlines()
576 message = detail[-1].replace(str(script), "fence") if detail else f"exit {checked.returncode}"
577 broken.append(f"{fence.path}:{fence.line}: shell fence does not parse: {message}")
578 return broken
581def _toml_violations(fences: list[Fence]) -> list[str]:
582 """Return one message per toml fence that does not parse.
584 In this process and with no subprocess, unlike every other checker here, because
585 :mod:`tomllib` is stdlib from 3.11 and this package declares ``requires-python = ">=3.11"``.
586 So there is nothing to provision and nothing to skip for: a toml fence is checked on every
587 machine that can run the gate at all, which is not true of the shell or yaml halves.
589 A *fragment* is accepted the way :func:`_syntax_violations` accepts one -- ``tomllib``
590 parses a bare ``key = value`` with no table header perfectly well, which is what most
591 configuration examples in ``docs/`` are. Only genuine syntax errors are reported.
593 Args:
594 fences: Every fence in the tree.
596 Returns:
597 Violation messages, one per broken fence.
598 """
599 broken: list[str] = []
600 for fence in fences:
601 if fence.language != TOML_FENCE_LANGUAGE:
602 continue
603 try:
604 tomllib.loads(fence.code)
605 except tomllib.TOMLDecodeError as exc:
606 # `exc` already carries "(at line N, column M)", and that N is relative to the
607 # fence body. Prefixing the fence's own line would produce two numbers meaning
608 # different things on one line, so the message is taken whole and the fence is
609 # located by its opening line alone -- the same choice `_shell_violations` makes.
610 broken.append(f"{fence.path}:{fence.line}: toml fence does not parse: {exc}")
611 return broken
614def _yaml_violations(fences: list[Fence], cfg: Config, scratch: Path) -> list[str] | None:
615 """Return one message per yaml fence that does not parse, or None when unmeasured.
617 The one checker here that needs a package this project does not depend on. That is the
618 whole reason it is a subprocess: ``rhiza-task`` is a published CLI, so every runtime
619 dependency is an install cost paid by every consumer on every ``uvx`` invocation, and
620 taking one on to parse two fences in this repository's own docs would be a poor trade.
621 ``uv_run(..., withs=("pyyaml",), no_project=True)`` provisions it for the length of one
622 call instead -- the same move :func:`~rhiza_task.tasks.book.marimo` makes for marimo, and
623 the reason :mod:`rhiza_task.uv` grew ``withs`` at all.
625 The fences are written to files rather than embedded in the generated script. Embedding
626 would need them escaped into a literal, and a yaml fence is exactly the kind of text --
627 quotes, backslashes, indentation that carries meaning -- where an escaping bug would look
628 like a parse failure in the document. Files move the bytes without reinterpreting them.
630 Three outcomes, deliberately distinguished by two marker files rather than by an exit
631 status. ``uv`` exits non-zero both when it cannot resolve ``pyyaml`` and when the script it
632 provisioned crashes, so the status alone cannot separate a machine's missing network from
633 this repository's bug -- and collapsing them is the worse error, because "unavailable" is a
634 pass. ``started.txt`` is written immediately after ``import yaml`` and ``report.txt`` after
635 the last fence, so their presence answers it: neither means the parser never arrived, the
636 first alone means the checker died mid-run, and both mean it finished.
638 Args:
639 fences: Every fence in the tree.
640 cfg: The resolved config.
641 scratch: Directory for the throwaway files.
643 Returns:
644 Violation messages, one per broken fence; an empty list when the tree holds no yaml
645 fence or every one of them parsed; a single-item list naming the script when it started
646 and did not finish, which fails the gate; or None when the parser could not be
647 provisioned at all, so the caller can report the fences as unchecked rather than sound.
648 """
649 targets = [fence for fence in fences if fence.language in YAML_FENCE_LANGUAGES]
650 if not targets:
651 return []
653 folder = scratch / "yaml"
654 folder.mkdir(parents=True, exist_ok=True)
655 for number, fence in enumerate(targets):
656 (folder / f"{number:04d}.yaml").write_text(fence.code)
657 (folder / "index.txt").write_text("\n".join(f"{fence.path}:{fence.line}" for fence in targets))
659 started = folder / "started.txt"
660 report = folder / "report.txt"
661 # Unlinked first: each file's *existence* is a verdict, so a stale copy from a previous
662 # invocation would report last run's outcome as this one's.
663 started.unlink(missing_ok=True)
664 report.unlink(missing_ok=True)
665 script = scratch / "fence_yaml.py"
666 script.write_text(YAML_CHECKER_SCRIPT)
668 code = uv_run(
669 "python",
670 script.relative_to(cfg.root).as_posix(),
671 cwd=cfg.root,
672 withs=("pyyaml",),
673 no_project=True,
674 check=False,
675 )
676 if not started.is_file():
677 # Never reached the first statement after `import yaml`: no network, no such package,
678 # no interpreter. A fact about the machine, so the caller counts these fences as
679 # unchecked rather than failing the gate.
680 return None
681 if not report.is_file():
682 # Started and did not finish, which is this repository's bug and not the machine's --
683 # so it is a violation, and the gate goes red. Before `started.txt` existed this case
684 # returned None and read as "parser unavailable", meaning a broken checker reported
685 # itself as a clean skip.
686 return [f"{script.name}: the yaml checker started and did not finish (exit {code})"]
687 # An empty report file is "ran, found nothing", which splitlines turns into [] -- distinct
688 # from the None above, and the distinction is the point of writing the files at all.
689 return report.read_text(errors="replace").splitlines()
692def _pycon_violations(per_file: list[list[Fence]], cfg: Config, scratch: Path) -> list[str]:
693 """Return one message per pycon example whose output is stale, raised, or malformed.
695 **The check the fleet's convention rests on.** A pycon fence is a doctest transcript, so
696 the expected output sits beside the statement that prints it, and the file and line a
697 stale example is reported at are the example's own -- where the retired ``result`` block
698 could only name itself and leave the reader to find which print moved.
700 Per file, and every non-skipped pycon fence of that file joined in document order into
701 *one* doctest, so a name bound in the first fence is in scope in the next: a page that
702 defines a task and then looks it up is one example split across prose, which is exactly
703 how ``adding_a_task.md`` reads. Files never share a namespace: a page's examples are
704 read on that page, and a name leaking in from another would pass an example no reader
705 could reproduce.
707 ``ELLIPSIS`` is on, so ``...`` in an expected output matches anything -- a path, an
708 address, a timing. That is the one loosening, and it is opt-in *per example*: an output
709 that does not write ``...`` is compared exactly. It is the same flag pytest-rhiza doctests
710 ``README.md`` with, so one example has one meaning in both places.
712 A subprocess through :func:`~rhiza_task.uv.uv_run`, in the project environment, and not
713 :func:`exec` in this process, which would be shorter: the examples in
714 ``adding_a_task.md`` call ``@task``, which registers into the live
715 :data:`~rhiza_task.spec.REGISTRY`, so running them here would add a task to the process
716 running the gate and a later ``list`` in the same ``rhiza-task all`` would print one that
717 does not exist. Isolation is a requirement, not caution. Always executed, no opt-in, and a
718 run that did not finish is a violation rather than a skip, because an unrunnable example
719 documents nothing. The fences reach the driver as JSON in a file, for the reason
720 :func:`_yaml_violations` writes files: no escaping to get wrong.
722 Args:
723 per_file: Fences grouped by file, so a namespace cannot reach across files.
724 cfg: The resolved config.
725 scratch: Directory for the driver, its input and its report.
727 Returns:
728 Violation messages, one per failing example or malformed fence; an empty list when the
729 tree holds no runnable pycon fence or every example matched; or a single message
730 naming the driver when it did not write its report, which fails the gate.
731 """
732 documents = [document for one_file in per_file if (document := _pycon_document(one_file))]
733 if not documents:
734 return []
736 folder = scratch / "pycon"
737 folder.mkdir(parents=True, exist_ok=True)
738 (folder / "input.json").write_text(json.dumps(documents), encoding="utf-8")
739 report = folder / "report.json"
740 # Unlinked first, as the yaml markers are: a report left by the last run would be read as
741 # this one's verdict.
742 report.unlink(missing_ok=True)
743 script = scratch / "pycon_fences.py"
744 script.write_text(PYCON_CHECKER_SCRIPT)
746 code = uv_run("python", script.relative_to(cfg.root).as_posix(), cwd=cfg.root, check=False)
747 if not report.is_file():
748 return [f"{script.name}: the pycon checker did not finish (exit {code})"]
749 return [str(message) for message in json.loads(report.read_text(encoding="utf-8", errors="replace"))]
752def _pycon_document(one_file: list[Fence]) -> dict[str, object] | None:
753 """Return one file's runnable pycon fences as the driver reads them, or None if it has none.
755 Split out of :func:`_pycon_violations`, which scored C (11) with this inlined as nested
756 comprehensions -- and this module's header claims nothing here exceeds B (10). Selecting
757 the fences and running them are separate jobs, so the split is the one the metric asks for.
759 Args:
760 one_file: Every fence of one file, in document order.
762 Returns:
763 ``{"path": ..., "fences": [[line, code], ...]}``, or None when no fence is a pycon fence
764 without ``+RHIZA_SKIP``.
765 """
766 runnable = [
767 [fence.line, fence.code] for fence in one_file if fence.language == PYCON_FENCE_LANGUAGE and not fence.skip
768 ]
769 return {"path": one_file[0].path, "fences": runnable} if runnable else None
772def _tally(fences: list[Fence]) -> Tally:
773 """Count the fences by what can be done with them.
775 Split out of :func:`_report` rather than inlined there, which read more directly: the two
776 together score C on cyclomatic complexity, and a fifth C block in this package is a
777 fifth thing a reader has to accept an argument for. Counting and printing are genuinely
778 separate jobs, so this is the decomposition the metric asks for rather than a contortion
779 to satisfy it.
781 Since toml and yaml joined, that argument has stopped being about taste: at B (7) here and
782 B (10) there, one function doing both would land above ``complexity_max`` and fail
783 ``rhiza-task complexity`` outright. So this split is now held by the gate rather than by
784 the reader who remembers why it is here.
786 Args:
787 fences: Every fence in the tree.
789 Returns:
790 The counts, as a :class:`Tally`.
791 """
792 tally = Counter(_kind(fence) for fence in fences)
793 unchecked = sorted(
794 ((language, count) for language, count in tally.items() if language not in CHECKED_FENCE_LANGUAGES),
795 key=lambda item: (-item[1], item[0]),
796 )
797 # B604 matches the *keyword name* `shell=` below and reads this as a subprocess call being
798 # handed a shell. It is a field assignment on a frozen dataclass of integers, and the
799 # nearest subprocess is two hundred lines away. Renaming the field to dodge the pattern
800 # would cost the symmetry with `pycon`, `python`, `toml` and `yaml` -- which is the whole
801 # reason the tuple became a record -- so the suppression is the cheaper trade.
802 #
803 # The marker sits on this line and not on `shell=` itself, which is where the finding is
804 # reported and where it was first written. bandit locates a B604 at the keyword but does
805 # its nosec bookkeeping against the enclosing call node, so a marker on the keyword line
806 # suppresses the finding *and* then reports itself as unmatched -- one
807 # `nosec encountered ... but no failed test` line in every `fmt` and `security` run, which
808 # is noise in the two gates whose value is a clean signal. Here both agree and neither
809 # fires. The marker carries no prose for the reason this file's header gives: bandit reads
810 # anything after it as more test IDs.
811 return Tally( # nosec B604
812 pycon=tally[PYCON_FENCE_LANGUAGE],
813 python=tally[PYTHON_FENCE_LANGUAGE],
814 shell=sum(tally[language] for language in SHELL_FENCE_LANGUAGES),
815 toml=tally[TOML_FENCE_LANGUAGE],
816 yaml=sum(tally[language] for language in YAML_FENCE_LANGUAGES),
817 unchecked=unchecked,
818 )
821def _kind(fence: Fence) -> str:
822 """Return the key a fence is counted under.
824 Its language, except in two cases. A fence with none is ``(none)``, so the unchecked line
825 has something to print. A skipped pycon fence is ``pycon +RHIZA_SKIP``, which is not in
826 :data:`CHECKED_FENCE_LANGUAGES` -- so it lands in the unchecked count by construction,
827 rather than being tallied as doctested when nothing ran it.
829 Args:
830 fence: One fence.
832 Returns:
833 The tally key.
834 """
835 if fence.language == PYCON_FENCE_LANGUAGE and fence.skip:
836 return f"{PYCON_FENCE_LANGUAGE} {SKIP_FLAG}"
837 return fence.language or "(none)"
840def _report(fences: list[Fence], bash: str | None, yaml_ran: bool, files: int) -> bool:
841 """Print the inventory and report whether anything was checkable.
843 The unchecked count is printed rather than dropped because "0 examples" and "43 fences
844 nothing looks at" both pass every other gate in this repository while documenting nothing
845 verifiable. A reader seeing only a green line would take it for full coverage.
847 Two of the five kinds can go unchecked on a machine that runs the gate fine otherwise --
848 shell without bash, yaml without a provisioned parser -- and each gets its own line saying
849 so. They are counted out of ``checked`` in that case rather than assumed sound, which is
850 the same rule the tool guards elsewhere in this package follow.
852 B (10) after toml and yaml were added, from B (6). The growth is one branch per kind that
853 can be unavailable, which is open-ended rather than closed, so it gets a ceiling: **a
854 sixth checkable language that needs provisioning takes this to C (12)**, and at that point
855 the availability lines want a loop over ``(count, ran, noun)`` triples rather than a
856 branch each. A kind that cannot go unavailable -- anything stdlib, as toml is -- costs
857 nothing here and does not count against that. ``pycon`` is that kind: its driver needs
858 only :mod:`doctest`, and a run that fails to report is a violation rather than an
859 availability line.
861 Args:
862 fences: Every fence in the tree.
863 bash: Path to bash, or None when it is absent.
864 yaml_ran: Whether the yaml parser could be provisioned.
865 files: How many markdown files were read.
867 Returns:
868 True when at least one fence was checked, so the caller can skip rather than pass.
869 """
870 tally = _tally(fences)
871 checked = tally.pycon + tally.python + tally.toml + (tally.shell if bash else 0) + (tally.yaml if yaml_ran else 0)
872 print(f"\n[INFO] {files} file(s), {len(fences)} fence(s): {checked} checked")
873 print(
874 f"[INFO] {tally.pycon} pycon, {tally.python} python, {tally.shell} shell, {tally.toml} toml, {tally.yaml} yaml"
875 )
876 if bash is None and tally.shell:
877 print(f"[INFO] bash not found: {tally.shell} shell fence(s) went unchecked")
878 if not yaml_ran and tally.yaml:
879 print(f"[INFO] yaml parser unavailable: {tally.yaml} yaml fence(s) went unchecked")
880 if tally.unchecked:
881 listing = ", ".join(f"{count} {language}" for language, count in tally.unchecked)
882 print(f"[INFO] {sum(count for _, count in tally.unchecked)} fence(s) not checkable: {listing}")
883 return checked > 0