Refuse to export a layer that carries a private key #14

Merged

marcos-mendez merged 2 commits from fix/refuse-private-keys into 19.x

2026-09-29 01:33:18 -03:00

marcos-mendez commented

2026-09-29 01:10:11 -03:00

(Migrated from github.com)

Problem

keel-core#8: all twelve layers exported so far carry the same private keys, and nothing in the build noticed. A list of files to remove only holds while someone remembers every package that makes a key.

Fix

layer_check_build runs layer_audit_keys on the built tree before anything is packed: a PEM private key header on a line of its own, text files only (grep -I), symlinks not followed. Every key found is named by its path in the image, and the layer is refused. A grep that fails is a failed audit, not a clean one. The exceptions are four inert examples shipped by packages, in conf/layer-inert-keys with a reason each, accepted only at their path and digest.

Evidence

Measured on the build host's layer trees: the anchored pattern finds exactly the real keys plus those four, about a second per layer. The unanchored one matched binaries (ssh, sshd, libgnutls, magic.mgc).
tests/layer: every key format, a key after a certificate, the quoted marker, a binary, a symlink out of the tree, the inert list by path and digest, grep failing or missing, and a bt-layer run whose build leaks a host key: refused, no tarball, no manifest. Five mutations each turn it red.
Coverage unchanged: layer-lib and bt-layer each keep the one uncovered line they have on 19.x (threshold 99).

Deploying the three together

This is one of three pull requests for keel-core#8: Keel-Linux/inithooks (generate on the machine), Keel-Linux/common (remove from the build), Keel-Linux/buildtasks (refuse at export). Merging deploys nothing: the build host checkouts are updated by hand. The order that matters is deployment:

Publish inithooks 2.3.6+keel6 to the archive (attended, signing). Until then a layer without keys would boot with no TLS certificate until 15regen-sslcert, which only restarts services already running.
Update /turnkey/fab-keel/common and /turnkey/buildtasks-keel on the build host in the same sitting: the audit without the removals refuses every layer.
Rebuild the chain and publish (attended). Images published before this still carry the keys; keel-host-keys replaces them on any machine made from one.

## Problem keel-core#8: all twelve layers exported so far carry the same private keys, and nothing in the build noticed. A list of files to remove only holds while someone remembers every package that makes a key. ## Fix `layer_check_build` runs `layer_audit_keys` on the built tree before anything is packed: a PEM private key header on a line of its own, text files only (`grep -I`), symlinks not followed. Every key found is named by its path in the image, and the layer is refused. A grep that fails is a failed audit, not a clean one. The exceptions are four inert examples shipped by packages, in `conf/layer-inert-keys` with a reason each, accepted only at their path and digest. ## Evidence - Measured on the build host's layer trees: the anchored pattern finds exactly the real keys plus those four, about a second per layer. The unanchored one matched binaries (ssh, sshd, libgnutls, magic.mgc). - `tests/layer`: every key format, a key after a certificate, the quoted marker, a binary, a symlink out of the tree, the inert list by path and digest, grep failing or missing, and a `bt-layer` run whose build leaks a host key: refused, no tarball, no manifest. Five mutations each turn it red. - Coverage unchanged: layer-lib and bt-layer each keep the one uncovered line they have on 19.x (threshold 99). ## Deploying the three together This is one of three pull requests for keel-core#8: Keel-Linux/inithooks (generate on the machine), Keel-Linux/common (remove from the build), Keel-Linux/buildtasks (refuse at export). Merging deploys nothing: the build host checkouts are updated by hand. The order that matters is deployment: 1. Publish `inithooks 2.3.6+keel6` to the archive (attended, signing). Until then a layer without keys would boot with no TLS certificate until `15regen-sslcert`, which only restarts services already running. 2. Update `/turnkey/fab-keel/common` and `/turnkey/buildtasks-keel` on the build host in the same sitting: the audit without the removals refuses every layer. 3. Rebuild the chain and publish (attended). Images published before this still carry the keys; `keel-host-keys` replaces them on any machine made from one.