Write /etc/keel_version beside the compatibility file #5

Merged

marcos-mendez merged 4 commits from feat/keel-version-file into 19.x

2026-10-02 16:41:24 -03:00

marcos-mendez commented

2026-09-28 00:17:50 -03:00

(Migrated from github.com)

An appliance had no file saying what it is. The console banner reads
/etc/turnkey_version for the name and the version because there was
nothing else to read, and its own comment said so.

This adds /etc/keel_version, the same four fields with the keel-
prefix, written at build time beside the compatibility file. Handbook
decision 0014 (Keel-Linux/handbook#6) settles what each of the two is
for: the TurnKey file is an interface we honour, the Keel file is what we
say we are.

Why here

mk/turnkey.mk, in root.patched/post, is the only place the version
string exists, it is shared by every appliance rather than repeated in
each recipe, and it is the last thing that touches the tree, so neither
file can be clobbered by an overlay or a conf script. An appliance conf
script cannot do it: conf scripts run in root.patched/body, before
/etc/turnkey_version is written, so the value they would derive from is
the parent layer's, or nothing at all on a rootfs layer. The ordering was
read out of the makefiles, not assumed.

A defect this uncovered

turnkey-version.py takes the name for /etc/turnkey_version from the
first line of the product changelog. keel-core's changelog became
keel-core-19.0 (1) keel on 2026-09-27, so the next core layer would
have written keel-core-19.0-trixie-amd64 into that file, and every
parser of it is prefix sensitive:

sysversion._parse_turnkey_release matches turnkey-.*?-(\d.*?)-[^\d],
so get_turnkey_release() returns the empty string and the release
number disappears from every version string built from it;
sysversion.AppVer removes the prefix turnkey- and then splits, so
the app name comes out as keel-core rather than core;
keel.inspect.app.probe_appliance requires the turnkey- prefix and
reports the appliance as missing otherwise, which costs
keel inspect and keel diff the identity of the machine.

bin/keel-version-files now removes at most one product prefix from the
changelog-derived string and writes turnkey-<rest> and keel-<rest>.
The published core layer predates the changelog rename, so nothing
shipped is affected; the next build would have been.

Tests

lib/version-files.sh holds the grammar and the prefix rules as pure
functions, bin/keel-version-files is the thin main that validates and
writes (decision 0004). 33 bats tests, both files at 100 percent
under kcov 43 (15 of 15 and 36 of 36 lines), covering every exit code (1
bad usage or a version string that is not an appliance identity, 2 an
unwritable tree, 3 the library missing) and the errexit trap of
docs/traps.md. tests/coverage.sh grew the per-target loop keel-core
uses so it can measure more than one file; conf/turnkey.d/postfix-local
stays at 100 percent, 17 of 17 lines. The threshold stays at 100.

Test plan

COVERAGE_THRESHOLD=100 tests/coverage.sh: three files, all 100
percent
shellcheck -S warning clean on the new files
the recipe order and the point at which the two files are written
read from product.mk and mk/turnkey.mk with make -n
a core layer built on the build host, to see the two files in the
image and keel inspect still naming the appliance

Rebased onto 19.x, 2026-10-02 (after common#30 and #31)

mk/turnkey.mk and mk/turnkey-desktop.mk: the identity files are written by bin/keel-version-files as before, with the -x check. The per-appliance apt User-Agent (01turnkey) that this branch's makefiles still wrote is not restored: common#6 replaced it with the overlay's fixed 01keel, and conf/turnkey.d/apt-identity deletes a 01turnkey. The recipe ends in mk/turnkey/seal-root (#31) as on 19.x.
tests/mk-identity.bats: the harness now links the real seal-root under its FAB_PATH and gives the scratch root a locked root. Three of its tests failed after the rebase without that. Four new tests: the build date is stamped after the identity files, a shipped root password fails the build, and neither makefile writes anything into apt.conf.d.
tests/coverage.sh: the lib/version-files.sh and bin/keel-version-files targets and the unmeasured mk-identity.bats run are added beside 19.x's targets and its before-firstboot/pam-unix run.
COVERAGE.md: both tables' rows kept, 17/17 and 36/36, 37 bats for the identity files.
changes/turnkey.changelog: a bullet for /etc/keel_version at the top of the unreleased turnkey-core-19.0 (1) entry, above #31's; the branch had none.
The file's format is unchanged: keel-<app>-<version>-<codename>-<arch> in /etc/keel_version, turnkey- prefix kept in /etc/turnkey_version.
Rebased a second time after common#32 (KEEL_APT_TRACK, decision 0047): mk/turnkey.mk no longer calls fab's make-release-deb.py, so the recipe now writes the two identity files and nothing else before seal-root. #32's test that no release meta package is built asserted the old turnkey_version=... > /etc/turnkey_version text; it now asserts the keel-version-files call, and tests/mk-identity.bats proves the files are written by running the recipe.

An appliance had no file saying what it is. The console banner reads `/etc/turnkey_version` for the name and the version because there was nothing else to read, and its own comment said so. This adds `/etc/keel_version`, the same four fields with the `keel-` prefix, written at build time beside the compatibility file. Handbook decision 0014 (Keel-Linux/handbook#6) settles what each of the two is for: the TurnKey file is an interface we honour, the Keel file is what we say we are. ## Why here `mk/turnkey.mk`, in `root.patched/post`, is the only place the version string exists, it is shared by every appliance rather than repeated in each recipe, and it is the last thing that touches the tree, so neither file can be clobbered by an overlay or a conf script. An appliance conf script cannot do it: conf scripts run in `root.patched/body`, before `/etc/turnkey_version` is written, so the value they would derive from is the parent layer's, or nothing at all on a rootfs layer. The ordering was read out of the makefiles, not assumed. ## A defect this uncovered `turnkey-version.py` takes the name for `/etc/turnkey_version` from the first line of the product changelog. keel-core's changelog became `keel-core-19.0 (1) keel` on 2026-09-27, so the next core layer would have written `keel-core-19.0-trixie-amd64` into that file, and every parser of it is prefix sensitive: - `sysversion._parse_turnkey_release` matches `turnkey-.*?-(\d.*?)-[^\d]`, so `get_turnkey_release()` returns the empty string and the release number disappears from every version string built from it; - `sysversion.AppVer` removes the prefix `turnkey-` and then splits, so the app name comes out as `keel-core` rather than `core`; - `keel.inspect.app.probe_appliance` requires the `turnkey-` prefix and reports the appliance as **missing** otherwise, which costs `keel inspect` and `keel diff` the identity of the machine. `bin/keel-version-files` now removes at most one product prefix from the changelog-derived string and writes `turnkey-<rest>` and `keel-<rest>`. The published core layer predates the changelog rename, so nothing shipped is affected; the next build would have been. ## Tests `lib/version-files.sh` holds the grammar and the prefix rules as pure functions, `bin/keel-version-files` is the thin main that validates and writes (decision 0004). 33 bats tests, both files at **100 percent** under kcov 43 (15 of 15 and 36 of 36 lines), covering every exit code (1 bad usage or a version string that is not an appliance identity, 2 an unwritable tree, 3 the library missing) and the errexit trap of `docs/traps.md`. `tests/coverage.sh` grew the per-target loop keel-core uses so it can measure more than one file; `conf/turnkey.d/postfix-local` stays at 100 percent, 17 of 17 lines. The threshold stays at 100. ## Test plan - [x] `COVERAGE_THRESHOLD=100 tests/coverage.sh`: three files, all 100 percent - [x] `shellcheck -S warning` clean on the new files - [x] the recipe order and the point at which the two files are written read from `product.mk` and `mk/turnkey.mk` with `make -n` - [ ] a core layer built on the build host, to see the two files in the image and `keel inspect` still naming the appliance ## Rebased onto 19.x, 2026-10-02 (after common#30 and #31) - `mk/turnkey.mk` and `mk/turnkey-desktop.mk`: the identity files are written by `bin/keel-version-files` as before, with the `-x` check. The per-appliance apt User-Agent (`01turnkey`) that this branch's makefiles still wrote is **not** restored: common#6 replaced it with the overlay's fixed `01keel`, and `conf/turnkey.d/apt-identity` deletes a `01turnkey`. The recipe ends in `mk/turnkey/seal-root` (#31) as on 19.x. - `tests/mk-identity.bats`: the harness now links the real `seal-root` under its `FAB_PATH` and gives the scratch root a locked root. Three of its tests failed after the rebase without that. Four new tests: the build date is stamped after the identity files, a shipped root password fails the build, and neither makefile writes anything into `apt.conf.d`. - `tests/coverage.sh`: the `lib/version-files.sh` and `bin/keel-version-files` targets and the unmeasured `mk-identity.bats` run are added beside 19.x's targets and its before-firstboot/pam-unix run. - `COVERAGE.md`: both tables' rows kept, 17/17 and 36/36, 37 bats for the identity files. - `changes/turnkey.changelog`: a bullet for `/etc/keel_version` at the top of the unreleased `turnkey-core-19.0 (1)` entry, above #31's; the branch had none. - The file's format is unchanged: `keel-<app>-<version>-<codename>-<arch>` in `/etc/keel_version`, `turnkey-` prefix kept in `/etc/turnkey_version`. - Rebased a second time after common#32 (`KEEL_APT_TRACK`, decision 0047): `mk/turnkey.mk` no longer calls fab's `make-release-deb.py`, so the recipe now writes the two identity files and nothing else before `seal-root`. #32's test that no release meta package is built asserted the old `turnkey_version=... > /etc/turnkey_version` text; it now asserts the `keel-version-files` call, and `tests/mk-identity.bats` proves the files are written by running the recipe.

marcos-mendez commented

2026-09-28 00:27:12 -03:00

(Migrated from github.com)

The appliance side is Keel-Linux/keel-core#9 and the decision note is Keel-Linux/handbook#6.

marcos-mendez commented

2026-09-28 01:34:56 -03:00

(Migrated from github.com)

Reviewed together with Keel-Linux/keel-core#9 and handbook #6 and #4. Merging this one first is the ordering with no window in which anything is wrong: keel-core#9 reads /etc/keel_version and falls back, so it is safe either way, but this is what makes the fallback stop being the only path.

What I verified rather than accepted:

The defect is real, end to end. /usr/share/fab/turnkey-version.py get_turnkey_version returns f"{codename}{version_tag}-{dist}-{architecture}" where codename is (\S+) from the changelog's first line, so keel-core-19.0 (2) keel; with --dist=trixie yields keel-core-19.0-trixie-amd64, and the make line this replaces wrote that straight into /etc/turnkey_version. All three parsers named are prefix sensitive as described: sysversion._parse_turnkey_release matches r"turnkey-.*?-(\d.*?)-[^\d]" and returns ""; AppVer.__init__ does removeprefix("turnkey-") then rsplit("-", 3), so the app name comes out keel-core; keel/inspect/app.py:43 is if not text.startswith(VERSION_PREFIX) ... return None, missing("appliance", ...), which is the appliance identity gone from keel inspect and keel diff. Two further readers the decision note lists also check out: inithooks/firstboot.d/29tagid through lib/tagid.sh, and /etc/tklbam/hooks.d/maria-db-changes, which does sed -En "s|turnkey-[a-z0-9-]+-([0-9]+)\.[0-9]+[a-z0-9]*-.*|\1|p".
Nothing published is affected. The changelog rename is bf6d721, 2026-09-27; the published core layer is 2026-09-26, and the booted core in keel-core's gate log prints Welcome to Core, TurnKey GNU/Linux 19.0, which only fmt_sysversion over a turnkey- string produces.
The normalisation cannot corrupt a legitimate name. kvf_app_version strips at most one prefix and only from the front. I ran the library against all ten recipes' changelog names on both architectures: every one is accepted and every compatibility string comes out identical to what those appliances ship today, so the only new content in any image is /etc/keel_version. turnkey-keelson-19.0-trixie-amd64 keeps keelson, keel-turnkey-core-19.0-trixie-amd64 keeps turnkey-core.
Blast radius. The layer tarballs on the build host show every child layer already carrying its own ./etc/turnkey_version, so each will carry ./etc/keel_version written from the same string in the same call and the two cannot drift. keel has no special case for either path in assembly; they are ordinary files and the child's copy wins as it does today. Every published manifest has parent=core transitively. Nothing in the other ten recipes breaks.
The placement argument holds. root.patched/post runs after root.patched/body, whose last action is the common removelists-final; only root.patched/cleanup follows and it touches apt caches and CA certificates. Conf scripts do run in root.patched/body, before either file exists.
Tests. 40 tests pass; COVERAGE_THRESHOLD=100 tests/coverage.sh gives postfix-local 17/17, version-files.sh 15/15, keel-version-files 36/36, all 100.00. The rewritten per-target loop works and postfix-local did not regress. shellcheck -S warning clean. The assertions are behavioural: the script is run and the files it wrote are read back, including mode, line count and the replace-not-append case.

MEDIUM

1. Every appliance build now hard-depends on a script in the common checkout, and the failure will not name itself. mk/turnkey.mk:56, $(COMMON_BIN_PATH)/keel-version-files $$release_version $O/root.patched || exit 1. The scenario: a build host whose common checkout predates this merge, or a bin/ that lost its mode bit, and every recipe dies in root.patched/post after the whole root has been built, with a bare No such file or directory. Failing loudly is right; failing legibly costs a [ -x ... ] || { echo ...; exit 1; } beside the call, naming the checkout that is stale.

2. $$release_version is unquoted. Empty gives one argument and exit 1 with the usage; a value with a space gives three and exit 1 too. Both are the outcome you want, but by luck rather than by construction, and quoting says which.

LOW

3. A hyphenated VERSION_TAG now fails the build. turnkey-version.py appends the tag to the name, so --tag=-rc1 produces core-19.0-rc1-trixie-amd64, which kvf_is_app_version refuses because the version field is [0-9][^-]*. --tag=rc is fine and is tested. One sentence in the usage saying the tag joins the version field would stop somebody finding this during a release.

4. kvf_app_version and kvf_is_app_version each walk $KVF_PREFIXES to answer the same question with inverted senses. One kvf_has_prefix would make the pair read as the rule they encode.

Approve

No CRITICAL and no HIGH. The two MEDIUMs are hardening of a path that already fails in the right direction, and the LOWs are documentation. The mechanism it disarms is real, the normalisation is conservative, and the blast radius is one new file per image and no change to any existing one.

Reviewed together with Keel-Linux/keel-core#9 and handbook #6 and #4. Merging this one first is the ordering with no window in which anything is wrong: keel-core#9 reads `/etc/keel_version` and falls back, so it is safe either way, but this is what makes the fallback stop being the only path. What I verified rather than accepted: - **The defect is real, end to end.** `/usr/share/fab/turnkey-version.py` `get_turnkey_version` returns `f"{codename}{version_tag}-{dist}-{architecture}"` where `codename` is `(\S+)` from the changelog's first line, so `keel-core-19.0 (2) keel;` with `--dist=trixie` yields `keel-core-19.0-trixie-amd64`, and the make line this replaces wrote that straight into `/etc/turnkey_version`. All three parsers named are prefix sensitive as described: `sysversion._parse_turnkey_release` matches `r"turnkey-.*?-(\d.*?)-[^\d]"` and returns `""`; `AppVer.__init__` does `removeprefix("turnkey-")` then `rsplit("-", 3)`, so the app name comes out `keel-core`; `keel/inspect/app.py:43` is `if not text.startswith(VERSION_PREFIX) ... return None, missing("appliance", ...)`, which is the appliance identity gone from `keel inspect` and `keel diff`. Two further readers the decision note lists also check out: `inithooks/firstboot.d/29tagid` through `lib/tagid.sh`, and `/etc/tklbam/hooks.d/maria-db-changes`, which does `sed -En "s|turnkey-[a-z0-9-]+-([0-9]+)\.[0-9]+[a-z0-9]*-.*|\1|p"`. - **Nothing published is affected.** The changelog rename is bf6d721, 2026-09-27; the published core layer is 2026-09-26, and the booted core in keel-core's gate log prints `Welcome to Core, TurnKey GNU/Linux 19.0`, which only `fmt_sysversion` over a `turnkey-` string produces. - **The normalisation cannot corrupt a legitimate name.** `kvf_app_version` strips at most one prefix and only from the front. I ran the library against all ten recipes' changelog names on both architectures: every one is accepted and every compatibility string comes out identical to what those appliances ship today, so the only new content in any image is `/etc/keel_version`. `turnkey-keelson-19.0-trixie-amd64` keeps `keelson`, `keel-turnkey-core-19.0-trixie-amd64` keeps `turnkey-core`. - **Blast radius.** The layer tarballs on the build host show every child layer already carrying its own `./etc/turnkey_version`, so each will carry `./etc/keel_version` written from the same string in the same call and the two cannot drift. `keel` has no special case for either path in assembly; they are ordinary files and the child's copy wins as it does today. Every published manifest has `parent=core` transitively. Nothing in the other ten recipes breaks. - **The placement argument holds.** `root.patched/post` runs after `root.patched/body`, whose last action is the common removelists-final; only `root.patched/cleanup` follows and it touches apt caches and CA certificates. Conf scripts do run in `root.patched/body`, before either file exists. - **Tests.** 40 tests pass; `COVERAGE_THRESHOLD=100 tests/coverage.sh` gives `postfix-local` 17/17, `version-files.sh` 15/15, `keel-version-files` 36/36, all 100.00. The rewritten per-target loop works and `postfix-local` did not regress. `shellcheck -S warning` clean. The assertions are behavioural: the script is run and the files it wrote are read back, including mode, line count and the replace-not-append case. ## MEDIUM **1. Every appliance build now hard-depends on a script in the common checkout, and the failure will not name itself.** `mk/turnkey.mk:56`, `$(COMMON_BIN_PATH)/keel-version-files $$release_version $O/root.patched || exit 1`. The scenario: a build host whose `common` checkout predates this merge, or a `bin/` that lost its mode bit, and every recipe dies in `root.patched/post` after the whole root has been built, with a bare `No such file or directory`. Failing loudly is right; failing legibly costs a `[ -x ... ] || { echo ...; exit 1; }` beside the call, naming the checkout that is stale. **2. `$$release_version` is unquoted.** Empty gives one argument and exit 1 with the usage; a value with a space gives three and exit 1 too. Both are the outcome you want, but by luck rather than by construction, and quoting says which. ## LOW **3. A hyphenated `VERSION_TAG` now fails the build.** `turnkey-version.py` appends the tag to the name, so `--tag=-rc1` produces `core-19.0-rc1-trixie-amd64`, which `kvf_is_app_version` refuses because the version field is `[0-9][^-]*`. `--tag=rc` is fine and is tested. One sentence in the usage saying the tag joins the version field would stop somebody finding this during a release. **4.** `kvf_app_version` and `kvf_is_app_version` each walk `$KVF_PREFIXES` to answer the same question with inverted senses. One `kvf_has_prefix` would make the pair read as the rule they encode. ## Approve No CRITICAL and no HIGH. The two MEDIUMs are hardening of a path that already fails in the right direction, and the LOWs are documentation. The mechanism it disarms is real, the normalisation is conservative, and the blast radius is one new file per image and no change to any existing one.

marcos-mendez commented

2026-09-29 00:40:21 -03:00

(Migrated from github.com)

MEDIUM 1 and 2 and both LOWs fixed in 8d19fbb and 0e47bdb, plus the gap #8 found: mk/turnkey-desktop.mk now writes both identity files through the same script. tests/mk-identity.bats runs the recipe of both makefiles under make against fab stubs (6 of 8 failed before). Conflicts with #8 in mk/ and tests/coverage.sh; trial-merged, the resolution is mechanical and all five files stay at 100 percent.

MEDIUM 1 and 2 and both LOWs fixed in 8d19fbb and 0e47bdb, plus the gap #8 found: `mk/turnkey-desktop.mk` now writes both identity files through the same script. `tests/mk-identity.bats` runs the recipe of both makefiles under make against fab stubs (6 of 8 failed before). Conflicts with #8 in `mk/` and `tests/coverage.sh`; trial-merged, the resolution is mechanical and all five files stay at 100 percent.