fix: inspect, confirm and monit on a Keel Core (keel#60, keel#61) #64

Merged

marcos-mendez merged 2 commits from fix/inspect-on-a-keel-core into main

2026-10-02 07:15:42 -03:00

marcos-mendez commented

2026-10-02 06:14:49 -03:00

(Migrated from github.com)

The keel bugs in the maintainer's screenshots of the Core image (2026-10-02), plus keel#60 and item 1 of keel#61. keel 0.15.1.

Root causes

Screen	Symptom	Root cause	Fix
047, 059	`appliance: not inferred`, header "unknown appliance"	`probe_appliance` accepted only the `turnkey-` prefix; a Keel image writes `keel-core-19.0-trixie-amd64`	`keel-` read like `turnkey-`
048, 059, 045	`inspect` exits 13 on a fresh Core	`instance.fqdn` was in `REQUIRED`, whose rule is "a hook prompts when unset"; no hook asks for it, apply skips `/etc/hosts` without it, and no appliance has one out of the box	dropped from `REQUIRED` (still reported as not inferred)
058, 061	`hub.api_key: skip` with a Keel Cloud key saved	`probe_hub` looked only for a TurnKey Hub registration	the presence of `/etc/keel/secrets/cloud_api_key` gives `{file: <secrets-dir>/cloud_api_key}`, never read
034, 057	an `eth1` the container does not have	every image ships an `allow-hotplug eth1` placeholder stanza, and inspect reported every stanza	on the live root, a hotplug-only interface absent from `/sys/class/net` is left out with a report line; `auto` ones and offline roots unchanged
034, 058	`updates_at_first_boot: force` after Skip	the first boot left no trace, so inspect used the update posture (the cron-apt install action every image ships)	reads `/var/lib/inithooks/sec-updates`, which inithooks' 95secupdates now writes (Keel-Linux/inithooks#33, keel16)
040	"one not confirmed in time has been reverted" after another session confirmed	with no marker, confirm assumed a revert	each change's outcome is kept in `/var/lib/keel/network/last.json` (0600); already confirmed exits 0 and says so, reverted is called reverted
069, 073, 096	`monit summary` always fails	keel never set monit's interface	`keel.conf` sets `set httpd unixsocket /run/monit.sock uid root gid root permission 0600` + `allow localhost`: root-only socket, no network listener (decision 0021). monit 5.34.3 supports it; checked with `monit -t -v`
keel#60	emitted spec loses the monitor channels	inspect never read `/etc/keel/monitor.json`	channels read back, token and URL files by reference; report names channels only; diff now also withholds observed channel values
keel#61 (1)	comment-only `online_api_credentials.yaml` read as registered	any non-blank text counted	a file of comments only counts as absent

keel#61 item 2 (monit's first cycle) is fixed in keel-core (a monit drop-in ordered after Core's units); its PR links here.

Tests

Tests written first, then the code. 2308 pass, 8 skip; coverage 99% (same 12 lines missed as on main, all new code covered). The real-monit tests ran against monit 5.34.3 via KEEL_MONIT. New fixture tests/fixtures/inspect/keelcore is a fresh Core after first boot.

Real test

LXC container from debian-13-keel-core_19.0-3+step7b-20261001_amd64.tar.zst on the test VM, with keel 0.15.1 and the inithooks, confconsole and keel-core packages of their PRs (built before the renumbering to keel16 and keel11) installed before the first boot. The review fixes (record last, forget on arm) are covered by unit tests:

inspect: core 19.0 (trixie, amd64), exit 0, no eth1, updates_at_first_boot: skip, hub.api_key: {file: ...cloud_api_key}, monitor.notify read back;
emitted spec (with the overlay states inspect cannot infer added) re-applied: 0 changes;
overlay confirmed from lxc-attach, then from the console: "already confirmed ... it stays", exit 0;
monit summary works; /run/monit.sock is srw------- root root; no TCP listener;
container destroyed afterwards.
CI green
Review

Closes #60. Refs #61.

The keel bugs in the maintainer's screenshots of the Core image (2026-10-02), plus keel#60 and item 1 of keel#61. keel 0.15.1. ## Root causes | Screen | Symptom | Root cause | Fix | | --- | --- | --- | --- | | 047, 059 | `appliance: not inferred`, header "unknown appliance" | `probe_appliance` accepted only the `turnkey-` prefix; a Keel image writes `keel-core-19.0-trixie-amd64` | `keel-` read like `turnkey-` | | 048, 059, 045 | `inspect` exits 13 on a fresh Core | `instance.fqdn` was in `REQUIRED`, whose rule is "a hook prompts when unset"; no hook asks for it, apply skips `/etc/hosts` without it, and no appliance has one out of the box | dropped from `REQUIRED` (still reported as not inferred) | | 058, 061 | `hub.api_key: skip` with a Keel Cloud key saved | `probe_hub` looked only for a TurnKey Hub registration | the presence of `/etc/keel/secrets/cloud_api_key` gives `{file: <secrets-dir>/cloud_api_key}`, never read | | 034, 057 | an `eth1` the container does not have | every image ships an `allow-hotplug eth1` placeholder stanza, and inspect reported every stanza | on the live root, a hotplug-only interface absent from `/sys/class/net` is left out with a report line; `auto` ones and offline roots unchanged | | 034, 058 | `updates_at_first_boot: force` after Skip | the first boot left no trace, so inspect used the update posture (the cron-apt install action every image ships) | reads `/var/lib/inithooks/sec-updates`, which inithooks' 95secupdates now writes (Keel-Linux/inithooks#33, keel16) | | 040 | "one not confirmed in time has been reverted" after another session confirmed | with no marker, confirm assumed a revert | each change's outcome is kept in `/var/lib/keel/network/last.json` (0600); already confirmed exits 0 and says so, reverted is called reverted | | 069, 073, 096 | `monit summary` always fails | keel never set monit's interface | `keel.conf` sets `set httpd unixsocket /run/monit.sock uid root gid root permission 0600` + `allow localhost`: root-only socket, no network listener (decision 0021). monit 5.34.3 supports it; checked with `monit -t -v` | | keel#60 | emitted spec loses the monitor channels | inspect never read `/etc/keel/monitor.json` | channels read back, token and URL files by reference; report names channels only; diff now also withholds observed channel values | | keel#61 (1) | comment-only `online_api_credentials.yaml` read as registered | any non-blank text counted | a file of comments only counts as absent | keel#61 item 2 (monit's first cycle) is fixed in keel-core (a monit drop-in ordered after Core's units); its PR links here. ## Tests Tests written first, then the code. 2308 pass, 8 skip; coverage 99% (same 12 lines missed as on main, all new code covered). The real-monit tests ran against monit 5.34.3 via `KEEL_MONIT`. New fixture `tests/fixtures/inspect/keelcore` is a fresh Core after first boot. ## Real test LXC container from `debian-13-keel-core_19.0-3+step7b-20261001_amd64.tar.zst` on the test VM, with keel 0.15.1 and the inithooks, confconsole and keel-core packages of their PRs (built before the renumbering to keel16 and keel11) installed before the first boot. The review fixes (record last, forget on arm) are covered by unit tests: - inspect: `core 19.0 (trixie, amd64)`, exit 0, no eth1, `updates_at_first_boot: skip`, `hub.api_key: {file: ...cloud_api_key}`, `monitor.notify` read back; - emitted spec (with the overlay states inspect cannot infer added) re-applied: 0 changes; - overlay confirmed from `lxc-attach`, then from the console: "already confirmed ... it stays", exit 0; - `monit summary` works; `/run/monit.sock` is `srw------- root root`; no TCP listener; - container destroyed afterwards. - [ ] CI green - [ ] Review Closes #60. Refs #61.