docs: the trap of a bats negation that is not the last command #16

Merged

marcos-mendez merged 3 commits from docs/trap-inert-negation into main

2026-09-29 00:59:22 -03:00

marcos-mendez commented

2026-09-28 05:34:23 -03:00

(Migrated from github.com)

One entry in docs/traps.md: a negative test that passes whatever the code does.

Closes #15. Part of Keel-Linux/tracker#19, which has the organization-wide sweep.

Why it earns an entry

A bats test body has no assertions of its own: its verdict is the exit status of the last command it ran. Bash does not apply errexit to a negated command, so ! cmd decides the test in final position and is inert everywhere else. The broken shape and the working shape are one word apart, the broken one reads correctly, and which one a line is depends only on where it sits — so a reviewer cannot see it and a passing suite does not reveal it.

Measured 2026-09-28 across every default branch in the organization that carries a bats file, 19 of them: 57 inert negations in 8 repositories, out of 102.

Two entries already in this file had a regression test written for them, and both tests were inert: unit-redis's refutation that the bind fragment never says localhost (the entry "On Debian, localhost is not an IPv6 name") and its refutation that no admin account is declared at build time (the entry about a Redis ACL user being declared once). A trap with a dead test is a trap with no test.

The entry follows the shape of the others

Signature, cause, fix, where it is enforced, and what it cost — plus the shellcheck invocation and the distinction shellcheck draws between error: (inert) and note: (final position, still asserting), which is what makes a sweep exact rather than a judgement call.

It also carries the two things that bite while converting, because both were hit:

run replaces $output, $status and $lines, so a body that refutes something and then reads the output of what it was refuting has to keep that output in a local first — otherwise the second refutation greps an empty string and passes for a new reason. That happened in keel-nodebb and is green by construction, so the suite cannot catch it.
run takes a command, not a pipeline: ! a | grep -q b has to be rewritten, not prefixed. That happened in inithooks.

And it says plainly that every predicate turned out to be correct, so nothing shipped broken, and that this is luck rather than evidence — naming the one assertion that did fail when it was turned on, keel-nodebb's unanchored ! grep -q set_real_ip_from against a file whose own comment names the directive, which could never have passed for any input.

Where it sits

Last of the numbered traps, before "Things we did wrong and would do again unless written down", in the file's own order of addition.

Test plan

Wrapped to the file's width; no line over 79 characters in the new entry.
The two docs/traps.md entries it cross-references checked to exist and to say what the new entry says they say.
The counts (57 inert, 102 total, 8 repositories, 19 with a bats file, twelve of thirteen rejection cases) recomputed from the sweep rather than carried over from the report that prompted it — that report was incomplete by two sites and named nine repositories.
Documentation only; nothing here ships and no gate applies beyond the repository's own.

One entry in `docs/traps.md`: a negative test that passes whatever the code does. Closes #15. Part of Keel-Linux/tracker#19, which has the organization-wide sweep. ## Why it earns an entry A bats test body has no assertions of its own: its verdict is the exit status of the last command it ran. Bash does not apply errexit to a negated command, so `! cmd` decides the test in final position and is inert everywhere else. The broken shape and the working shape are one word apart, the broken one reads correctly, and which one a line is depends only on where it sits — so a reviewer cannot see it and a passing suite does not reveal it. Measured 2026-09-28 across every default branch in the organization that carries a bats file, 19 of them: **57 inert negations in 8 repositories**, out of 102. Two entries already in this file had a regression test written for them, and both tests were inert: `unit-redis`'s refutation that the bind fragment never says `localhost` (the entry "On Debian, `localhost` is not an IPv6 name") and its refutation that no `admin` account is declared at build time (the entry about a Redis ACL user being declared once). A trap with a dead test is a trap with no test. ## The entry follows the shape of the others Signature, cause, fix, where it is enforced, and what it cost — plus the shellcheck invocation and the distinction shellcheck draws between `error:` (inert) and `note:` (final position, still asserting), which is what makes a sweep exact rather than a judgement call. It also carries the two things that bite while converting, because both were hit: - `run` replaces `$output`, `$status` and `$lines`, so a body that refutes something and then reads the output of what it was refuting has to keep that output in a local first — otherwise the second refutation greps an empty string and passes for a new reason. That happened in `keel-nodebb` and is green by construction, so the suite cannot catch it. - `run` takes a command, not a pipeline: `! a | grep -q b` has to be rewritten, not prefixed. That happened in `inithooks`. And it says plainly that every predicate turned out to be correct, so nothing shipped broken, and that this is luck rather than evidence — naming the one assertion that did fail when it was turned on, `keel-nodebb`'s unanchored `! grep -q set_real_ip_from` against a file whose own comment names the directive, which could never have passed for any input. ## Where it sits Last of the numbered traps, before "Things we did wrong and would do again unless written down", in the file's own order of addition. ## Test plan - [x] Wrapped to the file's width; no line over 79 characters in the new entry. - [x] The two `docs/traps.md` entries it cross-references checked to exist and to say what the new entry says they say. - [x] The counts (57 inert, 102 total, 8 repositories, 19 with a bats file, twelve of thirteen rejection cases) recomputed from the sweep rather than carried over from the report that prompted it — that report was incomplete by two sites and named nine repositories. - [x] Documentation only; nothing here ships and no gate applies beyond the repository's own.

marcos-mendez commented

2026-09-29 00:45:57 -03:00

(Migrated from github.com)

Merged main (the entry now follows the three that landed there), removed the four em dashes, and described the gate as it is after Keel-Linux/.github#14 6372ac2 (SC2315 and nested negations). Added one more thing to check when converting: run ! passes on 127 and on grep's 2.