Record the units a layer carries, so a child can subtract them #8

Merged

marcos-mendez merged 1 commit from feat/layer-units into 19.x

2026-09-27 00:50:48 -03:00

marcos-mendez commented

2026-09-27 00:31:23 -03:00

(Migrated from github.com)

Prerequisite of Keel decision 0010: nothing migrates out of the shared tree until bt-layer records units in the layer manifest.

A component handed to fab as a unit (a directory under UNIT_DIRS with a plan, an overlay, a conf script and a removelist) appeared in no manifest field. A child layer computes what to apply by subtracting what the parent recorded, so it could not subtract the component and would re-apply the overlay and re-run the conf script. conf/mysql does ln -s /etc/init.d/mysql under bash -e and fails with "File exists" the second time, so the second layer of the stack did not build at all. That is the LAMP case.

The two fields

Key	Meaning
`units`	every component the finished layer carries as a unit, `name@version` sorted by name, or `none`
`build_units`	the units this build applied: `default` without a parent, else the difference, or `none`

units is cumulative, the parent's record merged with this build's own. The common_* fields are cumulative too, but only because a child recipe includes its parent's makefile fragments; here it is kept so deliberately, because a child that carries no database content of its own still sits on the database and its own child has to know that.

The subtraction rule

build_units is the recipe's own units minus the parent's, matched by name. Same name and same version means the parent already applied it, so it is skipped. Same name at a different version fails the build: a conf script runs once, so there is no move from one version of a component to another inside a child layer, and the parent is the thing to rebuild.

Versions

The first of three sources that answers: a version file (the pin an assembly step writes); the unit's own git HEAD, but only when the directory is the top of that work tree, so the commit of an enclosing product repository is never passed off as the component's; otherwise a digest of the directory content, so an assembled or scratch unit still has an identity.

Compatibility

The applied set reaches fab as UNITS, added in keel-linux/fab#4. A parent manifest written before these fields existed reads as none through the new layer_manifest_opt. keel validates a manifest by its required keys and round-trips the rest, so both fields pass keel verify and keel pull untouched, and manifests without them still validate; checked against keel/layers/manifest.py on this host.

Test plan

tests/layer: ten new functions and six builds (a rootfs layer with a unit, a child composing the same unit, a child adding one, a child with none of its own, a version conflict, a conf fab would skip, and a parent manifest older than the fields).
tests/coverage.sh 99: bin/layer-lib 331 of 332, bt-layer 110 of 111, both 99 percent; the other four files 100 percent. Gate 99.
keel loads, validates and byte-identically round-trips a manifest carrying both fields, and still validates one without them.
A real mariadb component built as a unit-composed layer with a child on top, reported in the decision note.

Prerequisite of Keel decision 0010: nothing migrates out of the shared tree until `bt-layer` records units in the layer manifest. A component handed to fab as a unit (a directory under `UNIT_DIRS` with a plan, an overlay, a conf script and a removelist) appeared in no manifest field. A child layer computes what to apply by subtracting what the parent recorded, so it could not subtract the component and would re-apply the overlay and re-run the conf script. `conf/mysql` does `ln -s /etc/init.d/mysql` under `bash -e` and fails with "File exists" the second time, so the second layer of the stack did not build at all. That is the LAMP case. ## The two fields | Key | Meaning | | --- | --- | | `units` | every component the finished layer carries as a unit, `name@version` sorted by name, or `none` | | `build_units` | the units this build applied: `default` without a parent, else the difference, or `none` | `units` is cumulative, the parent's record merged with this build's own. The `common_*` fields are cumulative too, but only because a child recipe includes its parent's makefile fragments; here it is kept so deliberately, because a child that carries no database content of its own still sits on the database and its own child has to know that. ## The subtraction rule `build_units` is the recipe's own units minus the parent's, matched by name. Same name and same version means the parent already applied it, so it is skipped. Same name at a different version fails the build: a conf script runs once, so there is no move from one version of a component to another inside a child layer, and the parent is the thing to rebuild. ## Versions The first of three sources that answers: a `version` file (the pin an assembly step writes); the unit's own git HEAD, but only when the directory is the top of that work tree, so the commit of an enclosing product repository is never passed off as the component's; otherwise a digest of the directory content, so an assembled or scratch unit still has an identity. ## Compatibility The applied set reaches fab as `UNITS`, added in keel-linux/fab#4. A parent manifest written before these fields existed reads as `none` through the new `layer_manifest_opt`. `keel` validates a manifest by its required keys and round-trips the rest, so both fields pass `keel verify` and `keel pull` untouched, and manifests without them still validate; checked against `keel/layers/manifest.py` on this host. ## Test plan - [x] `tests/layer`: ten new functions and six builds (a rootfs layer with a unit, a child composing the same unit, a child adding one, a child with none of its own, a version conflict, a `conf` fab would skip, and a parent manifest older than the fields). - [x] `tests/coverage.sh 99`: `bin/layer-lib` 331 of 332, `bt-layer` 110 of 111, both 99 percent; the other four files 100 percent. Gate 99. - [x] `keel` loads, validates and byte-identically round-trips a manifest carrying both fields, and still validates one without them. - [ ] A real mariadb component built as a unit-composed layer with a child on top, reported in the decision note.