Ten negations in tests/boot-test.bats assert nothing, including five of the six is_global_ipv6 refusals #10

New issue

Closed

opened 2026-09-28 05:29:59 -03:00 by marcos-mendez · 0 comments

marcos-mendez commented

2026-09-28 05:29:59 -03:00

(Migrated from github.com)

A bats test body has no assertions of its own: its verdict is the exit status of the last command it ran. Bash does not apply errexit to a negated command, so ! cmd on its own line decides the test when it is the final command of the body and is inert everywhere else.

shellcheck -f gcc --shell=bash tests/*.bats | grep SC2314

tests/boot-test.bats has ten inert negations, the largest group in one file in the organization.

is_global_ipv6, five of six cases

@test "is_global_ipv6 refuses link local, loopback, multicast and IPv4" {
    ! bt_is_global_ipv6 fe80::1     # 115 — inert
    ! bt_is_global_ipv6 FE80::1     # 116 — inert
    ! bt_is_global_ipv6 ::1         # 117 — inert
    ! bt_is_global_ipv6 ff02::1     # 118 — inert
    ! bt_is_global_ipv6 10.0.3.1    # 119 — inert
    ! bt_is_global_ipv6 ""          # 120 — last, asserts
}

Only the empty string ran. This is the predicate that picks the address the boot test then talks to, so link-local, loopback, multicast and an IPv4 address were all unchecked.

The rest

Lines 100, 101: bt_is_container_name refusing a leading dot and an empty string, the two LXC refuses.
Line 103: bt_is_appliance_name 9wordpress.
Line 196: bt_is_ssh_banner "220 ready" — the answer a mail server gives. The test is named "accepts an OpenSSH banner and nothing else" and the "nothing else" half did not run.
Line 202: bt_firstboot_done_in with the flag still RUN_FIRSTBOOT=true, which is the whole point of the predicate.
Line 407: bt_password_in_config with the wrong password. The test read as though a mismatch was proven; only the empty file was.

tests/hook.bats line 147 is in final position and does assert.

What to do

Write every negation run ! cmd, with bats_require_minimum_version 1.5.0, the seven in final position included, so that no line's meaning depends on its position.

Test plan

shellcheck --severity=error --include=SC2314 over tests/*.bats reports nothing.
bats tests/*.bats green with the ten inert assertions live.
tests/coverage.sh at or above the committed threshold of 97.
Tests ship nothing, so package / changelog passes by reporting that nothing that ships changed.

A bats test body has no assertions of its own: its verdict is the exit status of the last command it ran. Bash does not apply errexit to a negated command, so `! cmd` on its own line decides the test when it is the final command of the body and is **inert everywhere else**. shellcheck -f gcc --shell=bash tests/*.bats | grep SC2314 `tests/boot-test.bats` has ten inert negations, the largest group in one file in the organization. ## is_global_ipv6, five of six cases @test "is_global_ipv6 refuses link local, loopback, multicast and IPv4" { ! bt_is_global_ipv6 fe80::1 # 115 — inert ! bt_is_global_ipv6 FE80::1 # 116 — inert ! bt_is_global_ipv6 ::1 # 117 — inert ! bt_is_global_ipv6 ff02::1 # 118 — inert ! bt_is_global_ipv6 10.0.3.1 # 119 — inert ! bt_is_global_ipv6 "" # 120 — last, asserts } Only the empty string ran. This is the predicate that picks the address the boot test then talks to, so link-local, loopback, multicast and an IPv4 address were all unchecked. ## The rest - Lines 100, 101: `bt_is_container_name` refusing a leading dot and an empty string, the two LXC refuses. - Line 103: `bt_is_appliance_name 9wordpress`. - Line 196: `bt_is_ssh_banner "220 ready"` — the answer a mail server gives. The test is named "accepts an OpenSSH banner and nothing else" and the "nothing else" half did not run. - Line 202: `bt_firstboot_done_in` with the flag still `RUN_FIRSTBOOT=true`, which is the whole point of the predicate. - Line 407: `bt_password_in_config` with the wrong password. The test read as though a mismatch was proven; only the empty file was. `tests/hook.bats` line 147 is in final position and does assert. ## What to do Write every negation `run ! cmd`, with `bats_require_minimum_version 1.5.0`, the seven in final position included, so that no line's meaning depends on its position. ## Test plan - `shellcheck --severity=error --include=SC2314` over `tests/*.bats` reports nothing. - `bats tests/*.bats` green with the ten inert assertions live. - `tests/coverage.sh` at or above the committed threshold of 97. - Tests ship nothing, so `package / changelog` passes by reporting that nothing that ships changed.