Add bt-layer: build one layer on its parent and export a signed, reproducible tarball #1

Merged

marcos-mendez merged 6 commits from feat/bt-layer into 19.x

2026-09-26 06:36:36 -03:00

marcos-mendez commented

2026-09-26 06:26:21 -03:00

(Migrated from github.com)

Problem: buildtasks produces monolithic images (ISO, Proxmox tar.zst); nothing builds an appliance as a layer on top of its parent, exports it with a manifest, or reproduces the same bytes twice.

Diagnosis: fab already supports building on an altstrap and the deck keeps the upper directories of each stage, so a layer is the deck's upper directories packed deterministically; what was missing was the driver that resolves the parent, computes which overlays and conf scripts a child still has to run, packs with fixed metadata and writes the manifest.

Fix: bin/layer-lib holds every decision as side-effect-free functions (layer kind, altstrap path, SOURCE_DATE_EPOCH resolution with a fallback to the product commit time, COMMON_CONF and COMMON_OVERLAYS via the fab debug target, the child's remaining overlays and conf scripts, deterministic tar flags with overlay xattrs kept, the manifest format, submounts to unmount, parent resolution, argument parsing). bt-layer LAYER [--parent P] [-u] builds the product on the exported P.rootfs and writes LAYER.tar.zst, its .sha256, LAYER.manifest (layer, type, parent and its sha256, release, arch, commits, fab version, epoch, overlay and conf lists, tarball sha256 and size), LAYER.rootfs and LAYER.log; generate-signature accepts layer tarballs and runs when BT_GPGKEY is set. fab and common are unchanged; a plain make still builds the ISO. tests/coverage.sh runs the suite under kcov and reads COVERAGE_THRESHOLD as the reusable workflow expects.

Tests: tests/layer, 40 code paths against scratch directories (fake products, stubs for fab, lsb_release, dpkg, mountpoint, umount and generate-signature): a root layer and a child are built and exported, the child re-exported with -u after touching every file is byte identical. kcov 43: layer-lib 183/183, bt-layer 99/100 (the missing line is the continuation of a multi-line command). Measured on the TKLDev 19.0 VM with SOURCE_DATE_EPOCH=1700000000: core 326,418,793 bytes, lamp on core 79,807,121 bytes, two -u exports of lamp with identical sha256.

Serves BRIEF section 5.1 (layers) and 5.4 (distribution and verification).

Problem: buildtasks produces monolithic images (ISO, Proxmox tar.zst); nothing builds an appliance as a layer on top of its parent, exports it with a manifest, or reproduces the same bytes twice. Diagnosis: fab already supports building on an altstrap and the deck keeps the upper directories of each stage, so a layer is the deck's upper directories packed deterministically; what was missing was the driver that resolves the parent, computes which overlays and conf scripts a child still has to run, packs with fixed metadata and writes the manifest. Fix: bin/layer-lib holds every decision as side-effect-free functions (layer kind, altstrap path, SOURCE_DATE_EPOCH resolution with a fallback to the product commit time, COMMON_CONF and COMMON_OVERLAYS via the fab debug target, the child's remaining overlays and conf scripts, deterministic tar flags with overlay xattrs kept, the manifest format, submounts to unmount, parent resolution, argument parsing). bt-layer LAYER [--parent P] [-u] builds the product on the exported P.rootfs and writes LAYER.tar.zst, its .sha256, LAYER.manifest (layer, type, parent and its sha256, release, arch, commits, fab version, epoch, overlay and conf lists, tarball sha256 and size), LAYER.rootfs and LAYER.log; generate-signature accepts layer tarballs and runs when BT_GPGKEY is set. fab and common are unchanged; a plain make still builds the ISO. tests/coverage.sh runs the suite under kcov and reads COVERAGE_THRESHOLD as the reusable workflow expects. Tests: tests/layer, 40 code paths against scratch directories (fake products, stubs for fab, lsb_release, dpkg, mountpoint, umount and generate-signature): a root layer and a child are built and exported, the child re-exported with -u after touching every file is byte identical. kcov 43: layer-lib 183/183, bt-layer 99/100 (the missing line is the continuation of a multi-line command). Measured on the TKLDev 19.0 VM with SOURCE_DATE_EPOCH=1700000000: core 326,418,793 bytes, lamp on core 79,807,121 bytes, two -u exports of lamp with identical sha256. Serves BRIEF section 5.1 (layers) and 5.4 (distribution and verification).