A negated command only asserts in final position #11

Merged

marcos-mendez merged 2 commits from test/negations-that-assert into master

2026-09-29 00:59:14 -03:00

marcos-mendez commented

2026-09-28 05:30:29 -03:00

(Migrated from github.com)

Every negation in these suites now asserts wherever it stands. Ten did not assert at all, the largest group in one file in the organization.

Closes #10. Part of Keel-Linux/tracker#19, which has the organization-wide sweep.

Why a bare `!` is inert

A bats test body has no assertions of its own: its verdict is the exit status of the last command it ran. Bash does not apply errexit to a negated command, so ! cmd decides the test in final position and is inert everywhere else. shellcheck grades the two apart, error: for the inert ones and note: for the rest:

shellcheck -f gcc --shell=bash tests/*.bats | grep SC2314

What was not being checked

Where	Was not asserting
115–119	five of the six cases of "is_global_ipv6 refuses link local, loopback, multicast and IPv4". Only the empty string ran, so `fe80::1`, `FE80::1`, `::1`, `ff02::1` and an IPv4 address were unchecked. This is the predicate that picks the address the boot test then talks to.
100, 101	`bt_is_container_name` refusing a leading dot and an empty string, the two LXC refuses
103	`bt_is_appliance_name 9wordpress`
196	`bt_is_ssh_banner "220 ready"` — the answer a mail server gives. The test is named "accepts an OpenSSH banner and nothing else"; the "nothing else" half did not run.
202	`bt_firstboot_done_in` with the flag still `RUN_FIRSTBOOT=true`, which is the whole point of the predicate
407	`bt_password_in_config` with the wrong password. The test read as though a mismatch was proven and only the empty file was.

All of them are run ! now, together with the seven that were in final position and did assert: a line whose meaning depends on its position is the trap itself.

What was found

Nothing broken. Every predicate was run against every one of its cases by hand before the conversion, and each answers as its test believed: bt_is_global_ipv6 refuses all five refusal cases and accepts all three acceptance cases, bt_is_ssh_banner refuses 220 ready, bt_password_in_config refuses the wrong password, and so on. The assertions were dead, not wrong.

That is luck rather than evidence, which is why the sweep comes with a gate in Keel-Linux/.github and a docs/traps.md entry.

Test plan

Every predicate verified against every one of its cases by hand, before the conversion.
shellcheck -f gcc --shell=bash --severity=error --include=SC2314 tests/*.bats reports nothing.
Every converted site audited for run clobbering $output/$status/$lines; none here does.
bats tests/*.bats green, 196 tests. No assertion changed its verdict.

tests/coverage.sh, threshold 97, unchanged:

100.00  31/31    zz-project-packages
100.00  26/26    zzz-keel-archive
 99.00  99/100   wordpress.sh
 97.73  43/44    40wordpress
 98.95  282/285  boot-test-lib.sh
100.00  54/54    keel-archive-check

Tests ship nothing, so package / changelog passes by reporting that nothing that ships changed. No changelog entry and no version bump.

Every negation in these suites now asserts wherever it stands. Ten did not assert at all, the largest group in one file in the organization. Closes #10. Part of Keel-Linux/tracker#19, which has the organization-wide sweep. ## Why a bare `!` is inert A bats test body has no assertions of its own: its verdict is the exit status of the last command it ran. Bash does not apply errexit to a negated command, so `! cmd` decides the test in final position and is inert everywhere else. shellcheck grades the two apart, `error:` for the inert ones and `note:` for the rest: shellcheck -f gcc --shell=bash tests/*.bats | grep SC2314 ## What was not being checked | Where | Was not asserting | |---|---| | 115–119 | five of the six cases of "is_global_ipv6 refuses link local, loopback, multicast and IPv4". Only the empty string ran, so `fe80::1`, `FE80::1`, `::1`, `ff02::1` and an IPv4 address were unchecked. This is the predicate that picks the address the boot test then talks to. | | 100, 101 | `bt_is_container_name` refusing a leading dot and an empty string, the two LXC refuses | | 103 | `bt_is_appliance_name 9wordpress` | | 196 | `bt_is_ssh_banner "220 ready"` — the answer a mail server gives. The test is named "accepts an OpenSSH banner and nothing else"; the "nothing else" half did not run. | | 202 | `bt_firstboot_done_in` with the flag still `RUN_FIRSTBOOT=true`, which is the whole point of the predicate | | 407 | `bt_password_in_config` with the wrong password. The test read as though a mismatch was proven and only the empty file was. | All of them are `run !` now, together with the seven that were in final position and did assert: a line whose meaning depends on its position is the trap itself. ## What was found Nothing broken. Every predicate was run against every one of its cases by hand before the conversion, and each answers as its test believed: `bt_is_global_ipv6` refuses all five refusal cases and accepts all three acceptance cases, `bt_is_ssh_banner` refuses `220 ready`, `bt_password_in_config` refuses the wrong password, and so on. The assertions were dead, not wrong. That is luck rather than evidence, which is why the sweep comes with a gate in `Keel-Linux/.github` and a `docs/traps.md` entry. ## Test plan - [x] Every predicate verified against every one of its cases by hand, before the conversion. - [x] `shellcheck -f gcc --shell=bash --severity=error --include=SC2314 tests/*.bats` reports nothing. - [x] Every converted site audited for `run` clobbering `$output`/`$status`/`$lines`; none here does. - [x] `bats tests/*.bats` green, 196 tests. No assertion changed its verdict. - [x] `tests/coverage.sh`, threshold 97, unchanged: 100.00 31/31 zz-project-packages 100.00 26/26 zzz-keel-archive 99.00 99/100 wordpress.sh 97.73 43/44 40wordpress 98.95 282/285 boot-test-lib.sh 100.00 54/54 keel-archive-check - [x] Tests ship nothing, so `package / changelog` passes by reporting that nothing that ships changed. No changelog entry and no version bump.

marcos-mendez commented

2026-09-29 00:54:35 -03:00

(Migrated from github.com)

Reviewed: every converted refutation goes red when the code under test accepts that call (all six is_global_ipv6 cases included), and each fails with status 1. No changes needed; merged main so the renamed appliance check runs on this head. Note: #6 adds tests/wrappers.bats with three new bare negations, which the gate will refuse.