Compose the MariaDB component instead of including the shared tree #11

Closed

marcos-mendez wants to merge 1 commit from feat/mariadb-as-a-unit into main

marcos-mendez commented

2026-09-27 14:44:43 -03:00

(Migrated from github.com)

The recipe stops including mk/turnkey/mysql.mk and the turnkey/mysql plan and composes keel-linux/unit-mariadb as unit.d/mariadb (its pull request 1). Decision 0013 is why: a database is a component, not a parent layer, so LAMP and LAPP can share one apache-php layer. Decision 0010 is how, and it asks for a comparison rather than a claim.

The comparison, on the build host

Four builds of bt-layer mariadb --parent core, all at SOURCE_DATE_EPOCH=1700000000, fab 1.1.1+keel2, common 897ad4c, parent core 7acf2c53, 102 to 104 seconds each:

build	tree	packages	files	symlinks
control	main	438	35,315	3,380
unit	this branch plus `unit.d/mariadb`	438	35,315	3,380
control, second run	main again, the noise floor	438	35,315	3,380
control plus this changelog entry	main plus `changelog` only	438	35,315	3,380

The noise floor is 179 of 35,315 regular files: 173 under /var/lib/mysql/**, the data directory the conf script creates, plus /etc/webmin/mysql/config, /var/webmin/module.infos.cache, /var/log/alternatives.log, /var/log/webmin/webmin.log, /var/cache/ldconfig/aux-cache and /root/.wget-hsts. Every one is install-time state and five of the six named files are already on the M0 allow-list.

Against the control, the unit build differed in three files more than the noise floor: /usr/share/doc/turnkey-mariadb-19.0/changelog, that package's md5sums and /var/lib/dpkg/status, and the package list differed in one line, turnkey-mariadb-19.0 at revision 1 against 2. That is this changelog entry, which fab turns into the release package, so the fourth build measured it instead of arguing it: the shared tree plus the changelog entry alone produces exactly those three files and that one package line.

Against that fourth build the unit build is equivalent, measured:

package list identical, all 438 entries, name and version
symlinks identical, all 3,380, path and target
179 differing regular files, the same number as the control noise floor, and 0 differing paths outside it
no path exists on one side and not the other

The manifest records the component, which is the point of the prerequisite merged tonight::

units        mariadb@1.0.0
build_units  mariadb@1.0.0

Nothing was published: the staged mariadb layer and its manifest were restored from the backup taken before the first build, so the mirror and the staging directory still carry the published layer.

Test plan:

four builds and the comparison above
tests/coverage.sh unchanged by this branch
appliance / build-and-boot, which fetches the published layer and is therefore about the layer on the mirror rather than this branch

The recipe stops including `mk/turnkey/mysql.mk` and the `turnkey/mysql` plan and composes `keel-linux/unit-mariadb` as `unit.d/mariadb` (its pull request 1). Decision 0013 is why: a database is a component, not a parent layer, so LAMP and LAPP can share one `apache-php` layer. Decision 0010 is how, and it asks for a comparison rather than a claim. ## The comparison, on the build host Four builds of `bt-layer mariadb --parent core`, all at `SOURCE_DATE_EPOCH=1700000000`, fab 1.1.1+keel2, common 897ad4c, parent `core` `7acf2c53`, 102 to 104 seconds each: | build | tree | packages | files | symlinks | | --- | --- | --- | --- | --- | | control | main | 438 | 35,315 | 3,380 | | unit | this branch plus `unit.d/mariadb` | 438 | 35,315 | 3,380 | | control, second run | main again, the noise floor | 438 | 35,315 | 3,380 | | control plus this changelog entry | main plus `changelog` only | 438 | 35,315 | 3,380 | The noise floor is **179 of 35,315 regular files**: 173 under `/var/lib/mysql/**`, the data directory the conf script creates, plus `/etc/webmin/mysql/config`, `/var/webmin/module.infos.cache`, `/var/log/alternatives.log`, `/var/log/webmin/webmin.log`, `/var/cache/ldconfig/aux-cache` and `/root/.wget-hsts`. Every one is install-time state and five of the six named files are already on the M0 allow-list. Against the control, the unit build differed in three files more than the noise floor: `/usr/share/doc/turnkey-mariadb-19.0/changelog`, that package's `md5sums` and `/var/lib/dpkg/status`, and the package list differed in one line, `turnkey-mariadb-19.0` at revision 1 against 2. That is this changelog entry, which fab turns into the release package, so the fourth build measured it instead of arguing it: the shared tree plus the changelog entry alone produces exactly those three files and that one package line. Against that fourth build the unit build is equivalent, measured: - package list **identical**, all 438 entries, name and version - symlinks **identical**, all 3,380, path and target - **179** differing regular files, the same number as the control noise floor, and **0** differing paths outside it - no path exists on one side and not the other The manifest records the component, which is the point of the prerequisite merged tonight:: units mariadb@1.0.0 build_units mariadb@1.0.0 Nothing was published: the staged `mariadb` layer and its manifest were restored from the backup taken before the first build, so the mirror and the staging directory still carry the published layer. Test plan: - [x] four builds and the comparison above - [x] `tests/coverage.sh` unchanged by this branch - [ ] `appliance / build-and-boot`, which fetches the published layer and is therefore about the layer on the mirror rather than this branch

marcos-mendez commented

2026-09-27 23:56:05 -03:00

(Migrated from github.com)

Independent review, read-only. Verdict: Warning — not mergeable as it stands.
The extraction itself is faithful and decision 0010's prerequisites are genuinely
in place; what is not sound is the evidence around it.

Three merge blockers, shared with the sibling pull request:

The pinned component version does not exist. The Makefile and README
clone --branch v1.0.0, and neither unit repository has any tag or release.
The documented build step fails with Remote branch v1.0.0 not found. Since
decision 0010's assembly step does not exist yet, this clone is the
assembly mechanism. Tag the unit, or pin a commit sha.
The branch is CONFLICTING against main, and the equivalence
measurement was made against a tree that no longer exists. main has since
gained merges that touch Makefile and conf.d/main, including a
root.patched/pre hook on the same target where fab applies the units. All
four builds behind the "0 attributable differences" claim predate that. The
merged tree has never been built.
The changelog revision has been overtaken and would now fail the gate on
rebase. Verified by running the gate's own entry_version/version_increased
against current main.

Two findings about the method, which outlive these pull requests:

The measurement is blind inside /var/lib/mysql/** and
/var/lib/postgresql/**, where only paths are compared and never the nature
of the difference. Those directories hold mysql/global_priv, mysql/user.*
and pg_authid. For a component whose build-time job is deleting accounts,
the one directory the measurement cannot see is the one holding the accounts.
There is no real difference today — the review established that by diffing the
conf scripts against common — but the reported measurement is not what
establishes it.
The measurement is not reproducible. The capture script lives only in a
scratch directory on the build host and the comparison step is committed
nowhere. Decision 0010 step 3 instructs re-running this comparison for every
future component; that instruction has no executable behind it.

Also recorded, and wider than these pull requests: the three green checks
never build the composed layer. appliance / build-and-boot boots the
published layer from the mirror, which on a recipe change is the code before
the change. Both bodies say so honestly with the box unticked. Tracked as
tracker#14.

Duplication has moved rather than gone. common still carries
conf/mysql, conf/pgsql, overlays/mysql, overlays/pgsql and their
makefiles, and they still have consumers — keel-wordpress among them. Nothing
detects divergence between the two copies, and unit-postgresql/conf has
already diverged. Decide whether the common copy is frozen with its consumers
migrating, or whether a check fails when the two drift.

Independent review, read-only. Verdict: **Warning — not mergeable as it stands.** The extraction itself is faithful and decision 0010's prerequisites are genuinely in place; what is not sound is the evidence around it. Three merge blockers, shared with the sibling pull request: 1. **The pinned component version does not exist.** The Makefile and README clone `--branch v1.0.0`, and neither unit repository has any tag or release. The documented build step fails with `Remote branch v1.0.0 not found`. Since decision 0010's assembly step does not exist yet, this clone *is* the assembly mechanism. Tag the unit, or pin a commit sha. 2. **The branch is `CONFLICTING` against `main`, and the equivalence measurement was made against a tree that no longer exists.** `main` has since gained merges that touch `Makefile` and `conf.d/main`, including a `root.patched/pre` hook on the same target where fab applies the units. All four builds behind the "0 attributable differences" claim predate that. The merged tree has never been built. 3. **The changelog revision has been overtaken** and would now fail the gate on rebase. Verified by running the gate's own `entry_version`/`version_increased` against current `main`. Two findings about the method, which outlive these pull requests: - **The measurement is blind inside `/var/lib/mysql/**` and `/var/lib/postgresql/**`**, where only paths are compared and never the nature of the difference. Those directories hold `mysql/global_priv`, `mysql/user.*` and `pg_authid`. For a component whose build-time job is deleting accounts, the one directory the measurement cannot see is the one holding the accounts. There is no real difference today — the review established that by diffing the conf scripts against `common` — but the reported measurement is not what establishes it. - **The measurement is not reproducible.** The capture script lives only in a scratch directory on the build host and the comparison step is committed nowhere. Decision 0010 step 3 instructs re-running this comparison for every future component; that instruction has no executable behind it. Also recorded, and wider than these pull requests: **the three green checks never build the composed layer.** `appliance / build-and-boot` boots the *published* layer from the mirror, which on a recipe change is the code before the change. Both bodies say so honestly with the box unticked. Tracked as tracker#14. **Duplication has moved rather than gone.** `common` still carries `conf/mysql`, `conf/pgsql`, `overlays/mysql`, `overlays/pgsql` and their makefiles, and they still have consumers — `keel-wordpress` among them. Nothing detects divergence between the two copies, and `unit-postgresql/conf` has already diverged. Decide whether the `common` copy is frozen with its consumers migrating, or whether a check fails when the two drift.

marcos-mendez commented

2026-10-07 12:49:00 -03:00

(Migrated from github.com)

Superseded by the current build (common's mysql.mk); the move to .deb overlays (decision 0036) is tracked separately.