Coverage for src/rhiza_task/tasks/fences.py: 100%
185 statements
« prev ^ index » next coverage.py v7.15.4, created at 2026-08-25 04:13 +0000
1"""The fenced-example checker behind ``docs-examples``.
3Extracted from :mod:`rhiza_task.tasks.quality`, which had grown to 904 lines holding this
4alongside seven unrelated task bodies -- and a module docstring describing only the latter.
5Nothing here breached the complexity ceiling; the maintainability index falling 44.55 to
636.52 over the two changes that built this up was the signal, and cohesion was the actual
7problem. See issue #113.
9**One name crosses the boundary: :func:`check`.** That is deliberate and not merely tidy.
10``CLAUDE.md``'s layering invariant forbids an underscore-prefixed name from crossing a
11module boundary, so keeping the ``@task`` in ``quality.py`` while moving the helpers here
12would have meant making seven private functions public -- turning an internal decomposition
13into an API. A single entry point taking the gate's whole body instead leaves the helpers
14private, where they belong.
16What the gate is *for* stays on the task in ``quality.py``, because that is where a reader
17looking for a gate looks. What it *does* is here.
19**This module is the largest in ``src/`` -- roughly 780 lines, with a maintainability index
20in the low 30s at the time of writing -- and that is accepted.** Worth stating outright,
21because the index is *lower* than the 36.52 that got the checker extracted from
22``quality.py`` in the first place, so the numbers alone read as having made things worse.
23They did not, and the reason is the one #113 was actually about: ``quality.py`` was 904 lines
24doing **two** jobs with a docstring describing one of them, and this is one job. Radon's
25maintainability index falls with size and Halstead volume whether or not a module is
26coherent, so it cannot tell those apart -- which is why cohesion, not the index, was the
27argument then and is the argument now.
29The figures are written loosely and dated on purpose, following ``pyproject.toml``'s note on
30the coverage floor: they move with every commit, they are incidental to the argument, and a
31comment stating one exactly is a comment the next edit falsifies. This paragraph proved it --
32adding it grew the file and dropped the index, so the precise numbers it first carried were
33wrong by the time it was saved. ``uvx radon mi src -s`` prints the current figure. What does
34not move: nothing here exceeds B (10), the average across ``src/`` is A (3.27), and about a
35seventh of these lines are comments, which is this repository's house style rather than
36padding.
38The condition that changes the answer is **a second job arriving, not a line count**. If
39something lands here that is not fenced-example checking, split on that seam. Splitting on
40size alone is the move to resist: the obvious cut -- the parser (``Fence``, :func:`_fences`,
41the language constants) away from the five checkers -- would put a type in one module and
42its only five consumers in another, which is worse than a long file.
44Five kinds of fence are checked and the rest are counted:
46* ``python`` -- :func:`compile`, so a fence that is a *fragment* still passes. Names need not
47 resolve; only the syntax is asserted.
48* ``bash``/``sh`` -- ``bash -n``, which parses without executing. Never executed, because a
49 README's shell is routinely ``rm -rf`` and ``git push``, and an unparseable fence is a
50 documentation bug without running it.
51* ``toml`` -- :func:`tomllib.loads`, in this process. Stdlib at this package's Python floor,
52 so it needs nothing provisioned and is checked on every machine that can run the gate.
53* ``yaml``/``yml`` -- a real parser, provisioned into a subprocess because adding one to this
54 package's runtime dependencies to serve two fences would be the wrong trade: it is a
55 published CLI whose install cost every consumer pays on every ``uvx`` invocation.
56* ``result`` -- executed and diffed against the python fences above it.
58Anything else -- ``mermaid``, ``makefile``, and fences carrying no language at all -- is
59reported as unchecked with a count. Naming the count is the point: silence there would read
60as "everything was checked".
62Two of the five can go unavailable on a machine that runs the gate fine otherwise, and both
63follow the rule the tool guards elsewhere in this package follow -- the fences are counted
64out of the checked total and named on their own line, never assumed sound. ``bash`` may be
65absent, and the yaml parser has to be fetched.
67Parsing is *not* validation, and the distinction is worth keeping: a ``toml`` fence that
68parses may still name a setting this package does not have, and a ``yaml`` fence that parses
69may still be an invalid workflow. Schema-checking either would need the schema, and for the
70workflow snippets actionlint already owns that question over the real files. What this closes
71is the narrower gap where a fence stopped being the language it claims.
72"""
74from __future__ import annotations
76import re
77import shutil
79# The `bash -n` probe below is a fixed argument vector, with no shell -- which is what bandit's
80# B404 asks about. The reason sits here rather than on the suppression comment itself: bandit
81# reads everything after that marker as a comma-separated list of test IDs, so a trailing
82# explanation becomes one `Test in comment:` warning per word.
83import subprocess # nosec B404
84import textwrap
85import tomllib
86from collections import Counter
87from dataclasses import dataclass
88from pathlib import Path
90from ..config import Config
91from ..spec import Failed, Skip
92from ..uv import uv_run
95@dataclass(frozen=True)
96class Scan:
97 """One pass over the documentation, before any checking.
99 A record for the reason :class:`Tally` is one: :func:`check` reached C (12) once README
100 joined the sources, which would have made it the third C-ranked block in this package.
101 ``CLAUDE.md`` asks for an argument per C block and this one had none worth making --
102 parsing, checking and reporting are three jobs, and splitting them is the decomposition
103 the metric was pointing at rather than a contortion to satisfy it. Passing the four
104 results positionally between those parts is what a record avoids.
106 Attributes:
107 per_file: The docs tree's fences, grouped by file, because a ``result`` block's prelude
108 is the python fences *in its own file* and nowhere else.
109 fences: The same fences, flat. The docs tree only -- the code checkers' subject.
110 readme: ``README.md``'s data fences, and none of its code fences. Empty when there is
111 no README. See :func:`_readme_fences`.
112 bash: Absolute path to bash, or None when it is absent -- resolved once, because its
113 absence must leave the shell fences counted rather than fail the gate.
114 """
116 per_file: list[list[Fence]]
117 fences: list[Fence]
118 readme: list[Fence]
119 bash: str | None
121 @property
122 def data(self) -> list[Fence]:
123 """Return the fences the data checkers see: the docs tree's and README's together.
125 One list rather than a second pass over README, because :func:`_yaml_violations` owns a
126 single scratch folder -- calling it twice would have the second run clobber the first's
127 markers -- and one subprocess for every yaml fence is cheaper anyway.
129 Returns:
130 Every fence from both sources.
131 """
132 return self.fences + self.readme
135def check(cfg: Config) -> None:
136 """Run the gate: parse every checkable fence, diff the executed ones, print the inventory.
138 The body of the ``docs-examples`` task, and the only public name in this module. The task
139 itself is registered in :mod:`rhiza_task.tasks.quality` and carries the argument for why
140 the gate exists.
142 Three steps, each its own function: :func:`_scan` parses, this gathers the violations, and
143 :func:`_verdict` prints and decides. See :class:`Scan` for why that split exists.
145 Args:
146 cfg: The resolved config.
148 Raises:
149 Skip: When nothing was measured. Documentation carrying nothing runnable would
150 otherwise score a silent pass, which is the failure this gate exists to make
151 visible.
152 Failed: When at least one example is broken or stale.
153 """
154 scratch = cfg.root / "_tests" / "docs-examples"
155 scratch.mkdir(parents=True, exist_ok=True)
156 scan = _scan(cfg)
158 # None, not [], when the parser could not be provisioned: an empty list is
159 # "checked, all sound", and reporting a machine's missing network as a clean bill of
160 # health is the one thing this gate must never do. `bash` is handled the same way, in
161 # `_scan`, by the same argument.
162 yaml_broken = _yaml_violations(scan.data, cfg, scratch)
164 broken = [
165 *_syntax_violations(scan.fences),
166 *(_shell_violations(scan.fences, scan.bash, scratch) if scan.bash else []),
167 *_toml_violations(scan.data),
168 *(yaml_broken or []),
169 *_result_violations(scan.per_file, cfg, scratch),
170 ]
171 _verdict(cfg, scan, broken, yaml_broken is not None)
174def _scan(cfg: Config) -> Scan:
175 """Parse every markdown file this gate looks at, and resolve bash once.
177 Args:
178 cfg: The resolved config.
180 Returns:
181 What was found, as a :class:`Scan`.
182 """
183 per_file = [
184 _fences(md.relative_to(cfg.root).as_posix(), md.read_text(errors="replace"))
185 for md in sorted(cfg.path("docs_folder").rglob("*.md"))
186 ]
187 return Scan(
188 per_file=per_file,
189 fences=[fence for one_file in per_file for fence in one_file],
190 readme=_readme_fences(cfg),
191 # Resolved once: `bash` is absent on a stock Windows runner, and its absence must leave
192 # the shell fences *unchecked and counted* rather than failing the gate for a fact
193 # about the machine. Same reasoning as `Guard(tool=...)` raising Skip rather than
194 # Failed.
195 bash=shutil.which("bash"),
196 )
199def _verdict(cfg: Config, scan: Scan, broken: list[str], yaml_ran: bool) -> None:
200 """Print the violations and the inventory, then raise the outcome.
202 Args:
203 cfg: The resolved config.
204 scan: What :func:`_scan` found.
205 broken: Every violation message, in the order they were gathered.
206 yaml_ran: Whether the yaml parser could be provisioned.
208 Raises:
209 Skip: When nothing was measured, so a pass would mean nothing.
210 Failed: When at least one example is broken or stale.
211 """
212 for violation in broken:
213 print(violation)
215 measured = _report(scan.fences, scan.bash, yaml_ran, len(scan.per_file))
216 if scan.readme:
217 # Its own line, and outside the inventory above, because README contributes only its
218 # data fences: folding two of its ten into "12 file(s), 64 fence(s)" would read as
219 # full coverage of a file this gate deliberately only half-looks at.
220 print(f"[INFO] README.md: {len(scan.readme)} data fence(s) checked; its code fences are pytest-rhiza's")
221 if not measured and not scan.readme:
222 raise Skip(f"no checkable fence under {cfg.docs_folder}")
223 if broken:
224 # `and README.md` rather than only the docs folder: since data fences are checked in
225 # both, a summary naming one scope would point a reader at the wrong file for half the
226 # failures it reports. The per-violation lines above carry the real locations.
227 scope = f"{cfg.docs_folder} and README.md" if scan.readme else cfg.docs_folder
228 raise Failed(1, f"{len(broken)} broken example(s) under {scope}")
231# Deliberately permissive about what follows the language: mkdocs-material accepts
232# ```python title="x", and an opening fence this pattern failed to match would have its
233# *closing* fence read as the next opening one, cascading the misparse through the rest of
234# the file. Matching any fence line and keeping only the first word cannot do that.
235DOC_FENCE_OPEN = re.compile(r"^(?P<indent>[ \t]*)```(?P<language>[^`\s]*)")
237# A closing fence carries nothing but backticks. A bare ``` therefore matches both patterns,
238# which is why :func:`_fences` tracks state instead of classifying lines independently.
239DOC_FENCE_CLOSE = re.compile(r"^[ \t]*```[ \t]*$")
241# `bash` and `sh` only. `console` and `shell-session` are excluded on purpose: their content
242# is a transcript -- prompts, output and all -- so `bash -n` would reject the very thing that
243# makes them correct. A language this set does not name is counted as unchecked and reported
244# rather than guessed at.
245SHELL_FENCE_LANGUAGES = frozenset({"bash", "sh"})
247PYTHON_FENCE_LANGUAGE = "python"
249# The convention README.md already uses and pytest-rhiza's `test_readme_validation` already
250# checks *there*: a ```result``` block holds the expected stdout of the python fence above it.
251RESULT_FENCE_LANGUAGE = "result"
253# `tomllib` is stdlib at the floor this package declares (`requires-python = ">=3.11"`), so
254# the toml half costs nothing to provision -- and it is the half that matters most here: the
255# toml fences under `docs/` are this package's own `[tool.rhiza-task]` and `[tool.bumpversion]`
256# examples, which is precisely the class that goes stale when a setting is renamed or a
257# default moves. Eleven of them at the time of writing, against two yaml.
258TOML_FENCE_LANGUAGE = "toml"
260# `yml` as well as `yaml`: mkdocs-material's own docs spell it both ways, and a fence this set
261# failed to name would be silently counted as unchecked rather than parsed -- the exact
262# silence this gate exists to break.
263YAML_FENCE_LANGUAGES = frozenset({"yaml", "yml"})
265# The driver :func:`_yaml_violations` writes out and runs. A module-level constant with its
266# body at column 0, rather than an indented literal inside that function, and the indentation
267# is the whole point: `CLAUDE.md` documents
268# `grep -rnE '^\s+(from|import) ' src/` as the way to check this package for deferred imports,
269# and tells the reader it returns two lines. An indented `import yaml` inside a string literal
270# matched that pattern and took it to four -- two false positives in the one check the
271# layering invariant is verified by, which is worse than the invariant being unchecked because
272# it teaches the reader to ignore the output. At column 0 nothing matches and the documented
273# count holds.
274#
275# `started.txt` is written before any parsing and `report.txt` after all of it, so the caller
276# can tell three outcomes apart that would otherwise collapse into one: the parser could not
277# be provisioned (no marker at all), the checker ran and crashed (marker, no report), and the
278# checker finished (both). Without the first marker a bug in this script is indistinguishable
279# from a machine with no network, and the gate passes green either way.
280YAML_CHECKER_SCRIPT = """\
281import pathlib
283import yaml
285here = pathlib.Path(__file__).parent / "yaml"
286(here / "started.txt").write_text("ok")
287broken = []
288for number, where in enumerate((here / "index.txt").read_text().splitlines()):
289 try:
290 yaml.safe_load((here / f"{number:04d}.yaml").read_text())
291 except yaml.YAMLError as exc:
292 # Flattened: a YAMLError's str spans four lines with a caret diagram, and the
293 # report this joins is one line per violation.
294 detail = " ".join(str(exc).split())
295 broken.append(f"{where}: yaml fence does not parse: {detail}")
296(here / "report.txt").write_text("\\n".join(broken))
297"""
299# The languages this gate checks in ``README.md`` as well as under the docs folder, and the
300# reason is a gap rather than a preference. ``README.md`` is pytest-rhiza's subject -- its
301# ``test_readme_validation`` runs under `rhiza-test` -- so this gate has always left the file
302# alone to keep one verdict per fact. But that module contains no reference to `toml` or
303# `yaml` at all, so README's data fences were checked by *nothing*: both of this repository's
304# are `[tool.rhiza-task]` and `rhiza.toml` examples naming real settings, which is exactly the
305# class that goes stale when a setting is renamed. See issue #112.
306#
307# So the two gates divide by *language* rather than by file, which keeps the no-double-verdict
308# rule intact and statable: pytest-rhiza owns README's code fences, this owns data fences
309# everywhere. Should pytest-rhiza ever learn toml, this set is the one place to narrow.
310DATA_FENCE_LANGUAGES = YAML_FENCE_LANGUAGES | {TOML_FENCE_LANGUAGE}
311CHECKED_FENCE_LANGUAGES = (
312 SHELL_FENCE_LANGUAGES | YAML_FENCE_LANGUAGES | {PYTHON_FENCE_LANGUAGE, RESULT_FENCE_LANGUAGE, TOML_FENCE_LANGUAGE}
313)
316@dataclass(frozen=True)
317class Fence:
318 """One fenced code block, located and dedented.
320 Attributes:
321 path: Repository-relative path, posix-separated, because this reaches report output
322 a reader pastes into an editor.
323 line: 1-based line number of the opening fence.
324 language: The info string's first word, lowercased; empty when the fence carries none.
325 code: The block's content, dedented.
326 """
328 path: str
329 line: int
330 language: str
331 code: str
334@dataclass(frozen=True)
335class Tally:
336 """How many fences of each kind the tree holds.
338 A record rather than the tuple this used to be. Four counts unpacked positionally were
339 already at the edge of readable; toml and yaml take it to six, where
340 ``python, shell, toml, yaml, diffed, unchecked = _tally(fences)`` stops being checkable by
341 eye and a transposed pair would report shell fences as toml with nothing failing. The
342 fields carry the meaning instead.
344 Attributes:
345 python: ``python`` fences, checked by :func:`compile`.
346 shell: ``bash``/``sh`` fences, checked by ``bash -n`` when bash is present.
347 toml: ``toml`` fences, checked in-process by :mod:`tomllib`.
348 yaml: ``yaml``/``yml`` fences, checked by a provisioned parser in a subprocess.
349 diffed: ``result`` fences, executed and compared against the python fences above them.
350 unchecked: Each remaining language paired with its count, commonest first and then
351 alphabetically so the report line is diffable between runs. A fence with no
352 language at all is counted under ``(none)``.
353 """
355 python: int
356 shell: int
357 toml: int
358 yaml: int
359 diffed: int
360 unchecked: list[tuple[str, int]]
363def _fences(path: str, text: str) -> list[Fence]:
364 """Return every fenced code block in one markdown file, in document order.
366 Indentation is why this is a state machine rather than a regex over the whole file:
367 mkdocs admonitions and content tabs indent their fences by four spaces, and ``faq.md``
368 indents one by three inside a numbered list. ``textwrap.dedent`` on the collected body is
369 what makes those compile -- without it every fence inside an admonition is an
370 ``IndentationError``, which would be a finding against this checker rather than the docs.
372 Nested fences are not handled, and cannot be: distinguishing them needs the four-backtick
373 form, which no file in this repository uses. If one appears, its inner fence closes the
374 outer block early and the languages reported go wrong -- visible in the inventory line
375 rather than silent, which is the reason that line prints a per-language count.
377 Args:
378 path: Repository-relative path, stored on each returned fence.
379 text: The file's content.
381 Returns:
382 The fences found, dedented.
383 """
384 fences: list[Fence] = []
385 language: str | None = None
386 start = 0
387 body: list[str] = []
388 for number, line in enumerate(text.splitlines(), start=1):
389 if language is None:
390 opening = DOC_FENCE_OPEN.match(line)
391 if opening:
392 language, start, body = opening.group("language").lower(), number, []
393 continue
394 if DOC_FENCE_CLOSE.match(line):
395 fences.append(Fence(path, start, language, textwrap.dedent("\n".join(body))))
396 language = None
397 continue
398 body.append(line)
399 return fences
402def _readme_fences(cfg: Config) -> list[Fence]:
403 """Return ``README.md``'s data fences, and none of its code fences.
405 The filter is the whole function. ``README.md`` belongs to pytest-rhiza's
406 ``test_readme_validation``, which parses its python and shell fences, so taking the whole
407 file would make two gates report one fact -- the thing this gate has always refused to do.
408 Taking only the languages that module does not know about closes the gap without creating
409 the overlap: see :data:`DATA_FENCE_LANGUAGES` for why those are toml and yaml.
411 Absent rather than required: a repository need not have a README, and this gate's subject
412 is the docs tree.
414 Args:
415 cfg: The resolved config.
417 Returns:
418 The toml and yaml fences in ``README.md``, or an empty list when there is no README.
419 """
420 readme = cfg.root / "README.md"
421 if not readme.is_file():
422 return []
423 found = _fences("README.md", readme.read_text(errors="replace"))
424 return [fence for fence in found if fence.language in DATA_FENCE_LANGUAGES]
427def _syntax_violations(fences: list[Fence]) -> list[str]:
428 """Return one message per python fence that does not parse.
430 :func:`compile` rather than execution, so a fence holding a *fragment* -- ``guards =
431 (Guard("source_folder"),)`` in ``adding_a_task.md``, with ``Guard`` never imported --
432 passes. Undefined names are not the question; syntax is.
434 Args:
435 fences: Every fence in the tree.
437 Returns:
438 Violation messages, one per broken fence.
439 """
440 broken: list[str] = []
441 for fence in fences:
442 if fence.language != PYTHON_FENCE_LANGUAGE:
443 continue
444 try:
445 compile(fence.code, f"{fence.path}:{fence.line}", "exec")
446 except SyntaxError as exc:
447 # The *offending* line, not the fence's: `getting_started.md` holds 24 fences, and
448 # "somewhere in this file" is the part of a report a reader has to redo by hand.
449 # `fence.line` is the opening backticks, so body line 1 sits one below it, which is
450 # what makes this sum the absolute line rather than one short of it.
451 broken.append(f"{fence.path}:{fence.line + (exc.lineno or 0)}: python fence does not parse: {exc.msg}")
452 return broken
455def _shell_violations(fences: list[Fence], bash: str, scratch: Path) -> list[str]:
456 """Return one message per shell fence that does not parse.
458 ``-n`` is the whole point: bash reads and parses the script and exits without running a
459 command of it. So this validates ``rm -rf`` and ``git push`` fences without their
460 consequences.
462 Captured rather than streamed through :func:`~rhiza_task.uv.tool`, which every other
463 binary in this package goes through, for two reasons that both come from this being a
464 checker rather than a gate over one command: the message is wanted *per fence* and lives
465 on stderr, and echoing ``$ bash -n ...`` once per fence would bury the report it exists to
466 produce under twenty invocation lines. ``_git`` above captures for the same reason.
468 Args:
469 fences: Every fence in the tree.
470 bash: Absolute path to bash, already resolved by the caller.
471 scratch: Directory for the throwaway script.
473 Returns:
474 Violation messages, one per broken fence.
475 """
476 broken: list[str] = []
477 script = scratch / "fence.sh"
478 for fence in fences:
479 if fence.language not in SHELL_FENCE_LANGUAGES:
480 continue
481 script.write_text(fence.code)
482 checked = subprocess.run( # noqa: S603 # nosec B603
483 [bash, "-n", str(script)],
484 capture_output=True,
485 text=True,
486 check=False,
487 )
488 if checked.returncode:
489 # bash reports against the throwaway path and its own line numbers, neither of
490 # which the reader can act on. The fence's own location is already the prefix, so
491 # the script path is stripped to leave the diagnosis.
492 detail = checked.stderr.strip().splitlines()
493 message = detail[-1].replace(str(script), "fence") if detail else f"exit {checked.returncode}"
494 broken.append(f"{fence.path}:{fence.line}: shell fence does not parse: {message}")
495 return broken
498def _toml_violations(fences: list[Fence]) -> list[str]:
499 """Return one message per toml fence that does not parse.
501 In this process and with no subprocess, unlike every other checker here, because
502 :mod:`tomllib` is stdlib from 3.11 and this package declares ``requires-python = ">=3.11"``.
503 So there is nothing to provision and nothing to skip for: a toml fence is checked on every
504 machine that can run the gate at all, which is not true of the shell or yaml halves.
506 A *fragment* is accepted the way :func:`_syntax_violations` accepts one -- ``tomllib``
507 parses a bare ``key = value`` with no table header perfectly well, which is what most
508 configuration examples in ``docs/`` are. Only genuine syntax errors are reported.
510 Args:
511 fences: Every fence in the tree.
513 Returns:
514 Violation messages, one per broken fence.
515 """
516 broken: list[str] = []
517 for fence in fences:
518 if fence.language != TOML_FENCE_LANGUAGE:
519 continue
520 try:
521 tomllib.loads(fence.code)
522 except tomllib.TOMLDecodeError as exc:
523 # `exc` already carries "(at line N, column M)", and that N is relative to the
524 # fence body. Prefixing the fence's own line would produce two numbers meaning
525 # different things on one line, so the message is taken whole and the fence is
526 # located by its opening line alone -- the same choice `_shell_violations` makes.
527 broken.append(f"{fence.path}:{fence.line}: toml fence does not parse: {exc}")
528 return broken
531def _yaml_violations(fences: list[Fence], cfg: Config, scratch: Path) -> list[str] | None:
532 """Return one message per yaml fence that does not parse, or None when unmeasured.
534 The one checker here that needs a package this project does not depend on. That is the
535 whole reason it is a subprocess: ``rhiza-task`` is a published CLI, so every runtime
536 dependency is an install cost paid by every consumer on every ``uvx`` invocation, and
537 taking one on to parse two fences in this repository's own docs would be a poor trade.
538 ``uv_run(..., withs=("pyyaml",), no_project=True)`` provisions it for the length of one
539 call instead -- the same move :func:`~rhiza_task.tasks.book.marimo` makes for marimo, and
540 the reason :mod:`rhiza_task.uv` grew ``withs`` at all.
542 The fences are written to files rather than embedded in the generated script. Embedding
543 would need them escaped into a literal, and a yaml fence is exactly the kind of text --
544 quotes, backslashes, indentation that carries meaning -- where an escaping bug would look
545 like a parse failure in the document. Files move the bytes without reinterpreting them.
547 Three outcomes, deliberately distinguished by two marker files rather than by an exit
548 status. ``uv`` exits non-zero both when it cannot resolve ``pyyaml`` and when the script it
549 provisioned crashes, so the status alone cannot separate a machine's missing network from
550 this repository's bug -- and collapsing them is the worse error, because "unavailable" is a
551 pass. ``started.txt`` is written immediately after ``import yaml`` and ``report.txt`` after
552 the last fence, so their presence answers it: neither means the parser never arrived, the
553 first alone means the checker died mid-run, and both mean it finished.
555 Args:
556 fences: Every fence in the tree.
557 cfg: The resolved config.
558 scratch: Directory for the throwaway files.
560 Returns:
561 Violation messages, one per broken fence; an empty list when the tree holds no yaml
562 fence or every one of them parsed; a single-item list naming the script when it started
563 and did not finish, which fails the gate; or None when the parser could not be
564 provisioned at all, so the caller can report the fences as unchecked rather than sound.
565 """
566 targets = [fence for fence in fences if fence.language in YAML_FENCE_LANGUAGES]
567 if not targets:
568 return []
570 folder = scratch / "yaml"
571 folder.mkdir(parents=True, exist_ok=True)
572 for number, fence in enumerate(targets):
573 (folder / f"{number:04d}.yaml").write_text(fence.code)
574 (folder / "index.txt").write_text("\n".join(f"{fence.path}:{fence.line}" for fence in targets))
576 started = folder / "started.txt"
577 report = folder / "report.txt"
578 # Unlinked first: each file's *existence* is a verdict, so a stale copy from a previous
579 # invocation would report last run's outcome as this one's.
580 started.unlink(missing_ok=True)
581 report.unlink(missing_ok=True)
582 script = scratch / "fence_yaml.py"
583 script.write_text(YAML_CHECKER_SCRIPT)
585 code = uv_run(
586 "python",
587 script.relative_to(cfg.root).as_posix(),
588 cwd=cfg.root,
589 withs=("pyyaml",),
590 no_project=True,
591 check=False,
592 )
593 if not started.is_file():
594 # Never reached the first statement after `import yaml`: no network, no such package,
595 # no interpreter. A fact about the machine, so the caller counts these fences as
596 # unchecked rather than failing the gate.
597 return None
598 if not report.is_file():
599 # Started and did not finish, which is this repository's bug and not the machine's --
600 # so it is a violation, and the gate goes red. Before `started.txt` existed this case
601 # returned None and read as "parser unavailable", meaning a broken checker reported
602 # itself as a clean skip.
603 return [f"{script.name}: the yaml checker started and did not finish (exit {code})"]
604 # An empty report file is "ran, found nothing", which splitlines turns into [] -- distinct
605 # from the None above, and the distinction is the point of writing the files at all.
606 return report.read_text(errors="replace").splitlines()
609def _result_violations(per_file: list[list[Fence]], cfg: Config, scratch: Path) -> list[str]:
610 """Return one message per ``result`` block that no longer matches what its python prints.
612 This is the half that catches an example gone *stale* rather than malformed, which is the
613 failure with the longest half-life: the fence still parses, still renders, and is wrong.
615 The prelude is every python fence earlier in the same file, concatenated, because that is
616 what ``README.md``'s pair needs -- the first fence defines the ``audit`` task with
617 ``@task`` and the second calls ``lookup("audit")``, so running the second alone raises.
618 The whole captured stdout is then compared against the block.
620 That comparison is exact but for surrounding whitespace, and it carries one assumption
621 worth stating: a prelude fence that *prints* would have its output counted as part of the
622 result. No file here has one. If that changes, the fix is to fence off the prelude's
623 output rather than to loosen the diff, because a loosened diff is how a stale example
624 starts passing again.
626 Args:
627 per_file: Fences grouped by file, so a prelude cannot reach across files.
628 cfg: The resolved config.
629 scratch: Directory for the throwaway script and its captured stdout.
631 Returns:
632 Violation messages, one per stale or unrunnable block.
633 """
634 broken: list[str] = []
635 for one_file in per_file:
636 for index, fence in enumerate(one_file):
637 if fence.language != RESULT_FENCE_LANGUAGE:
638 continue
639 prelude = [f.code for f in one_file[:index] if f.language == PYTHON_FENCE_LANGUAGE]
640 if not prelude:
641 broken.append(f"{fence.path}:{fence.line}: result block with no python fence above it")
642 continue
643 printed = _run_fences(cfg, scratch, prelude)
644 if printed is None:
645 broken.append(f"{fence.path}:{fence.line}: the python above this block exited non-zero")
646 elif printed.strip() != fence.code.strip():
647 broken.append(
648 f"{fence.path}:{fence.line}: result block is stale\n"
649 f" expected: {fence.code.strip()!r}\n"
650 f" actual: {printed.strip()!r}"
651 )
652 return broken
655def _run_fences(cfg: Config, scratch: Path, codes: list[str]) -> str | None:
656 """Run python fences in the project environment and return their stdout.
658 A subprocess, and not :func:`exec` in this process, which would be shorter: the fences in
659 ``adding_a_task.md`` call ``@task``, and ``@task`` registers into the live
660 :data:`~rhiza_task.spec.REGISTRY`. Running them here would add an ``audit`` task to the
661 process running the gate, so a later ``list`` in the same ``rhiza-task all`` would print a
662 task that does not exist. Isolation is a requirement here, not caution.
664 stdout arrives through a file rather than a pipe for the reason ``complexity`` reads
665 radon's ``--output-file``: :func:`~rhiza_task.uv.uv_run` streams rather than captures, and
666 adding a capturing variant to ``uv.py`` for one caller would widen that module's surface
667 for it. The script redirects its own stdout, so the invocation stays a fixed argument
668 vector with no shell -- and the fences' tracebacks still reach the terminal on stderr,
669 which is where a reader wants them.
671 Args:
672 cfg: The resolved config.
673 scratch: Directory for the script and its captured stdout.
674 codes: The python fences to run, in document order.
676 Returns:
677 The captured stdout, or None when the script exited non-zero or wrote nothing.
678 """
679 printed = scratch / "stdout.txt"
680 # Stale output first, for the reason `complexity` unlinks its report: output left by an
681 # earlier run would be read as this run's, and a diff that passes against last run's
682 # stdout is worse than no diff.
683 printed.unlink(missing_ok=True)
684 # `result_fences.py`, not `fences.py`: this module is now called fences.py too, and a
685 # generated scratch file sharing its name reads as the module in a traceback or a grep.
686 script = scratch / "result_fences.py"
687 script.write_text(
688 f"import sys\nsys.stdout = open({str(printed)!r}, 'w', encoding='utf-8')\n"
689 + "\n".join(codes)
690 + "\nsys.stdout.flush()\n"
691 )
692 code = uv_run("python", script.relative_to(cfg.root).as_posix(), cwd=cfg.root, check=False)
693 if code or not printed.is_file():
694 return None
695 return printed.read_text(errors="replace")
698def _tally(fences: list[Fence]) -> Tally:
699 """Count the fences by what can be done with them.
701 Split out of :func:`_report` rather than inlined there, which read more directly: the two
702 together score C on cyclomatic complexity, and a fifth C block in this package is a
703 fifth thing a reader has to accept an argument for. Counting and printing are genuinely
704 separate jobs, so this is the decomposition the metric asks for rather than a contortion
705 to satisfy it.
707 Since toml and yaml joined, that argument has stopped being about taste: at B (7) here and
708 B (10) there, one function doing both would land above ``complexity_max`` and fail
709 ``rhiza-task complexity`` outright. So this split is now held by the gate rather than by
710 the reader who remembers why it is here.
712 Args:
713 fences: Every fence in the tree.
715 Returns:
716 The counts, as a :class:`Tally`.
717 """
718 tally = Counter(fence.language or "(none)" for fence in fences)
719 unchecked = sorted(
720 ((language, count) for language, count in tally.items() if language not in CHECKED_FENCE_LANGUAGES),
721 key=lambda item: (-item[1], item[0]),
722 )
723 # B604 matches the *keyword name* `shell=` below and reads this as a subprocess call being
724 # handed a shell. It is a field assignment on a frozen dataclass of integers, and the
725 # nearest subprocess is two hundred lines away. Renaming the field to dodge the pattern
726 # would cost the symmetry with `python`, `toml`, `yaml` and `diffed` -- which is the whole
727 # reason the tuple became a record -- so the suppression is the cheaper trade.
728 #
729 # The marker sits on this line and not on `shell=` itself, which is where the finding is
730 # reported and where it was first written. bandit locates a B604 at the keyword but does
731 # its nosec bookkeeping against the enclosing call node, so a marker on the keyword line
732 # suppresses the finding *and* then reports itself as unmatched -- one
733 # `nosec encountered ... but no failed test` line in every `fmt` and `security` run, which
734 # is noise in the two gates whose value is a clean signal. Here both agree and neither
735 # fires. The marker carries no prose for the reason this file's header gives: bandit reads
736 # anything after it as more test IDs.
737 return Tally( # nosec B604
738 python=tally[PYTHON_FENCE_LANGUAGE],
739 shell=sum(tally[language] for language in SHELL_FENCE_LANGUAGES),
740 toml=tally[TOML_FENCE_LANGUAGE],
741 yaml=sum(tally[language] for language in YAML_FENCE_LANGUAGES),
742 diffed=tally[RESULT_FENCE_LANGUAGE],
743 unchecked=unchecked,
744 )
747def _report(fences: list[Fence], bash: str | None, yaml_ran: bool, files: int) -> bool:
748 """Print the inventory and report whether anything was checkable.
750 The unchecked count is printed rather than dropped because "0 examples" and "43 fences
751 nothing looks at" both pass every other gate in this repository while documenting nothing
752 verifiable. A reader seeing only a green line would take it for full coverage.
754 Two of the five kinds can go unchecked on a machine that runs the gate fine otherwise --
755 shell without bash, yaml without a provisioned parser -- and each gets its own line saying
756 so. They are counted out of ``checked`` in that case rather than assumed sound, which is
757 the same rule the tool guards elsewhere in this package follow.
759 B (10) after toml and yaml were added, from B (6). The growth is one branch per kind that
760 can be unavailable, which is open-ended rather than closed, so it gets a ceiling: **a
761 sixth checkable language that needs provisioning takes this to C (12)**, and at that point
762 the availability lines want a loop over ``(count, ran, noun)`` triples rather than a
763 branch each. A kind that cannot go unavailable -- anything stdlib, as toml is -- costs
764 nothing here and does not count against that.
766 Args:
767 fences: Every fence in the tree.
768 bash: Path to bash, or None when it is absent.
769 yaml_ran: Whether the yaml parser could be provisioned.
770 files: How many markdown files were read.
772 Returns:
773 True when at least one fence was checked, so the caller can skip rather than pass.
774 """
775 tally = _tally(fences)
776 checked = tally.python + tally.diffed + tally.toml + (tally.shell if bash else 0) + (tally.yaml if yaml_ran else 0)
777 print(f"\n[INFO] {files} file(s), {len(fences)} fence(s): {checked} checked")
778 print(
779 f"[INFO] {tally.python} python, {tally.shell} shell, "
780 f"{tally.toml} toml, {tally.yaml} yaml, {tally.diffed} diffed"
781 )
782 if bash is None and tally.shell:
783 print(f"[INFO] bash not found: {tally.shell} shell fence(s) went unchecked")
784 if not yaml_ran and tally.yaml:
785 print(f"[INFO] yaml parser unavailable: {tally.yaml} yaml fence(s) went unchecked")
786 if tally.unchecked:
787 listing = ", ".join(f"{count} {language}" for language, count in tally.unchecked)
788 print(f"[INFO] {sum(count for _, count in tally.unchecked)} fence(s) not checkable: {listing}")
789 return checked > 0