The appliance boot gate reports success when it does not test the branch #11

New issue

Closed

opened 2026-09-28 00:08:48 -03:00 by marcos-mendez · 0 comments

marcos-mendez commented

2026-09-28 00:08:48 -03:00

(Migrated from github.com)

appliance / build-and-boot in .github/workflows/test-appliance.yml reports
success in two cases where it proves nothing about the branch.

Layer never published. Steps 6–10 are guarded by
if: steps.layer.outputs.published == 'true' and are skipped; the job still
concludes success. Measured on keel-redis#5, run 36369705307: 11 seconds,
Assemble and boot skipped. The step summary does say "Nothing was built,
assembled or booted" — but branch protection reads the conclusion, not the
summary.
Layer published. The job pulls the published layer from $SOURCE and
boots that. On a pull request that changes the recipe, that is the code
before the change.

Audit, 2026-09-28: eleven layers answer 200 at
https://mirror.keellinux.org/layers/<name>.manifest; only redis is 404.

A check may only be named for what it proves, and a required check must either
exercise the branch or not be required. Scope here is honesty, not new
capability — building from the branch does not fit a per-pull-request check
while the build host is serialised behind a lock.

Renaming a check silently disables any branch protection rule that names it;
list the repositories that need a protection change rather than changing them.

Org-wide context: Keel-Linux/tracker#14.

`appliance / build-and-boot` in `.github/workflows/test-appliance.yml` reports `success` in two cases where it proves nothing about the branch. 1. **Layer never published.** Steps 6–10 are guarded by `if: steps.layer.outputs.published == 'true'` and are skipped; the job still concludes success. Measured on keel-redis#5, run 36369705307: 11 seconds, `Assemble and boot` skipped. The step summary does say "Nothing was built, assembled or booted" — but branch protection reads the conclusion, not the summary. 2. **Layer published.** The job pulls the *published* layer from `$SOURCE` and boots that. On a pull request that changes the recipe, that is the code before the change. Audit, 2026-09-28: eleven layers answer 200 at `https://mirror.keellinux.org/layers/<name>.manifest`; only `redis` is 404. A check may only be named for what it proves, and a required check must either exercise the branch or not be required. Scope here is honesty, not new capability — building from the branch does not fit a per-pull-request check while the build host is serialised behind a lock. Renaming a check silently disables any branch protection rule that names it; list the repositories that need a protection change rather than changing them. Org-wide context: Keel-Linux/tracker#14.