Name the command decision 0010 step 3 asks for, and what a recount of the two extractions found #8
No reviewers
Labels
No labels
accessibility
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Keel-Linux/handbook!8
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "docs/0010-name-the-comparison"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #5. Paired with Keel-Linux/buildtasks#11, which adds the command.
Decision 0010 step 3 required a three-build comparison for every component
taken out of the shared tree and pointed at "this three-build comparison",
meaning the narrative of the experiment earlier in the note. Nothing was
runnable: the capture was
/root/measure/build.shon the build host, a scratchfile in root's home directory on one machine, and the comparison was ad hoc
shell that was never committed. Step 3 now names
bt-layer-measureand stateswhat a pull request extracting a component has to quote from it.
0010 is amended in place. The decision is unchanged, only its step 3 was
unexecutable, and 0014 and 0015 are being written by others.
The part that is not just naming a command
Both extractions' captures are still on the build host. Recounting them on the
tab separated
path<TAB>hashkey the tool uses, rather than onsha256sum'swhitespace-separated default, gives different numbers from the ones the note and
the two pull requests reported:
./var/lib/mysql/**./var/lib/postgresql/**Three findings, all reproducible from those captures.
One control pair is not a noise floor. Against
control vs control againalone, mariadb leaves three files unaccounted for:
./usr/share/doc/turnkey-mariadb-19.0/changelog,./var/lib/dpkg/info/turnkey-mariadb-19.0.md5sumsand./var/lib/dpkg/status.All three sit inside the other control pair's floor, so they are install-time
noise and the conclusion of zero survives, but against the union of both pairs,
which is an argument the pull request did not make. Two builds of one recipe do
not differ by a fixed set: 179 in one pair, 182 in another.
The postgresql extraction was not at zero. One difference survives either
control pair and their union,
./etc/postgresql/17/main/postgresql.conf:The value is identical and only the trailing comment moved, because the
component's conf script writes a different comment than
conf.d/maindid. It isharmless, and it is precisely the class of thing the measurement exists to
surface: a file the component writes, differing because the component wrote it.
It was reported as zero.
A path inside the noise floor was never evidence of noise. 173 of the
mariadb differences are
./var/lib/mysql/**, subtracted because of where theyare. That directory holds
mysql/global_privandmysql/user.*, and all fourof
global_priv.MAD,global_priv.MAI,global_priv.frmanduser.frmdifferin the control-unit pair. The component's build-time job includes deleting
accounts. So the measurement subtracted, as noise, every file the thing being
measured would have shown up in.
Neither extraction is reopened. Both layers are built, published and
boot-tested, and the postgresql residue is a comment. What changes is the
standard for the next component: the block
bt-layer-measure attributeprintsis what an extraction is reviewed on, and a residue is named and explained in
the pull request rather than absent from it.
Changes
docs/decisions/0010-composable-units.md: step 3 names the command; two newsections, "The command, and what a pull request has to quote" and "What a
recount found, 2026-09-28"; the original measurement table is marked
superseded in place rather than rewritten, because what the note is now about
is that nobody could check it.
docs/traps.md: two entries beside "A path with a space in it", which is thesame shape of defect. "One control pair is not a noise floor" and "A differing
path inside the noise floor is not a noise difference".
Kept in step with the command
Keel-Linux/buildtasks#11 changed after this was approved: the first version
cancelled a state path's differences against the controls by line and byte
position, which review showed is itself the defect. 0010 and the trap entry now
describe the command as it is on that pull request's final head: three
controls in the worked example, the preconditions
attributerefuses on(including the anchor to the committed state path list), the verdicts
noise,overlapping,realandnot sampledand what clears each, and thefirst real run, 0 attributable and FAIL on 32
realAria header clock fieldsopen in Keel-Linux/buildtasks#13.
mainis merged in;docs/traps.mdconflicted only because both sides added an entry at the same place, and both
are kept.
Test plan
/root/tree-{m,p}-*.sha256on the buildhost; the commands are in the commit message and the note
bt-layer-measurethe note states checked againstbuildtasks#11 at its final head
Review
I reproduced the recount independently from
/root/tree-{m,p}-*.sha256on the build host, with my own script keyed onpath<TAB>hashand not withbt-layer-measure, then again throughbt-layer-measure attribute. Every number in the new table and all three findings hold exactly:./var/lib/mysql/**./var/lib/postgresql/**The mariadb residue against the first control pair is exactly
./usr/share/doc/turnkey-mariadb-19.0/changelog,./var/lib/dpkg/info/turnkey-mariadb-19.0.md5sumsand./var/lib/dpkg/status, and it is zero against the union../etc/postgresql/17/main/postgresql.confsurvives all three control pairs and the union. The quoted diff of that line is right byte for byte:keel-postgresql@b9565ea:conf.d/main:61wrote# set by conf.d/main of keel-postgresql: see the comment thereandunit-postgresql/conf:52writes# set by the postgresql component: see the comment in its conf script. All four ofglobal_priv.MAD,.MAI,.frmanduser.frmdiffer in the control-unit pair and sit inside every control floor.The history is clean. Both commits sit on
3483b5a, the current tip ofmain, with nothing from any other branch; only the two files the description claims; both pass the repository'scommit-msgandpre-commithooks when replayed against them. The author/committer date skew is consistent with the disclosed reset and recommit, and nothing bypassed a check silently.Three things.
HIGH 1. The new trap entry prescribes a procedure the command it names cannot perform
docs/traps.md, "One control pair is not a noise floor", the Fix paragraph: "bt-layer-measure attributeinbuildtasks, and when a residue survives, capture a third control and subtract the union of both control pairs."bt-layer-measure attributetakes exactly one--control-againand offers no way to pass a second (bin/layer-measure-lib:542on Keel-Linux/buildtasks#11). The union cannot be asked for. On the real captures the attributable count swings 3 → 0 for mariadb and 4 → 1 for postgresql purely on which pair is named as the floor, so the first run of the command this note now makes mandatory will printFAIL (3 attributable)for the very component this note concludes is at zero.So the record here states a lesson and points at an instrument that has not learned it. Either say plainly in the Fix that the union is not yet mechanised and has to be done by hand, or land the repeatable
--control-againin buildtasks#11 first. As written a reader follows the entry and cannot do what it says.MEDIUM 2. The superseded marker does not cover the subsection that carries the number
docs/decisions/0010-composable-units.md: the blockquote is placed immediately above### The 173, named and explained, and its own wording, "the counts in this table do not", reads as pointing at the table above it. The subsection below is titled with the number the recount contradicts, and its first row reads167 | /var/lib/mysql/**where the recount gives 173 for the control-unit pair. That subsection is the one anyone will cite later for "here are the noise files and why they are noise", and it is reachable directly from a heading link with the marker out of view.Marking the table you replaced and leaving the numbers standing is the right call, and the reason given for it is the right reason. It just needs one more line, under that heading, saying the breakdown is superseded too.
MEDIUM 3. The recount does not carry the command that produced it
The note's new rule is that a claim of zero which does not carry the block that produced it is not reviewable. The recount table is the first claim made under that rule and it carries no command: the test plan says "the commands are in the commit message and the note", and neither contains one; both describe the key (
path<TAB>hash) and stop there.This is not pedantry.
bt-layer-measurecannot read those captures: they are baresha256sumfiles, not captures in its format. I had to convert them into capture directories beforeattributewould run on them. So the recount was necessarily produced by another piece of uncommitted ad hoc shell, which is the exact thing this note exists to end. Paste the command, or say in the note that the historical captures predate the format and name the conversion.LOW 4. The union is offered as a rescue without saying it is the weaker test
Both here and in the trap entry, "the residue is zero against the union of both control pairs" is presented as recovering the conclusion. It does, but a union floor subtracts strictly more, so zero against the union proves less than zero against one pair would have: mariadb's floors are 179 and 182 and their union is 182. The note is honest that this is "an argument the pull request did not make"; one clause saying the union is more permissive, not more rigorous, would stop a future reader reading it the other way.
I verified by running: the full recount for both families across all pairs, the three residues against each floor and the union, the four blind mariadb files, the quoted
postgresql.confdiff againstkeel-postgresqlhistory, and the branch and hook checks above. I inferred, without running: how a reader navigates to the superseded subsection.Warning: no blocking issue. The recount is correct, the record is honest and the history is clean. Finding 1 is worth settling before this lands, because it is the one line in the two pull requests where the written standard and the shipped tool disagree.
All four settled, in
814e007.1. The entry prescribed what the command could not do. Fixed in the
command rather than in the prose:
bt-layer-measure attributenow takes--control-againas many times as there are control captures and subtracts theunion, and prints what each single pair would have said. The Fix paragraph
names that instead of describing a procedure by hand.
2. The marker did not cover the subsection.
### The 173, named and explainednow carries its own, saying which of it survives: the explanationsfor the paths it names, not the counts, and not the
167 | /var/lib/mysql/**row, which describes files nobody opened.
3. The recount carried no command. You are right that this was the first
claim made under the note's own rule and that it failed it, and right about why
it could not simply cite the tool: those captures are bare
sha256sumoutput,so
bt-layer-measurecannot read them and the recount was another piece ofuncommitted shell. The note now carries the whole of it, including the
keyfunction that moves the path into field 1 and separates it with a tab, and says
that captures taken from here on are in the tool's format and need none of it.
4. The union is the weaker test. Said in both places: it subtracts strictly
more, so zero against the union proves less than zero against one pair would
have, and it is the right test because its answer does not depend on which two
builds were named first, not because it is the harder one.
Since you reviewed this, the tool has been run for real: four captures of
mariadb, three of them controls. 0 attributable against every single pair and
against the union, and 167 state paths that differ, of which 32 come out
realand 33
overlapping. The 32 are almost all*.MAI, and in 30 of them thediffering value increases strictly in capture order, so they are the Aria index
header clock whose carry three controls did not span. Details in
Keel-Linux/buildtasks#11. Nothing in that changes this note, which claims zero
attributable and gets it; it does mean the sentence about
/var/lib/mysql/**being install-time state of the same kind as the allow-list entries is now
supported by bytes somebody read rather than by the path's name.
Re-review
All four findings are settled, and I checked each one rather than taking the summary.
1. The trap entry no longer prescribes what the tool cannot do. The Fix now says to capture at least three controls and pass every one, because
--control-againis repeatable and the floor is the union over every pair. I verified that against the implementation:attributetakes the controls as an array,measure_floor_positionsunions every unordered pair, and the block prints what each single pair on its own would have said. The entry and the instrument now agree.2. The
### The 173, named and explainedsubsection carries its own marker, and it does the harder thing of saying which parts survive: the explanations for the paths named, not the counts, and it calls out that the row for/var/lib/mysql/**describes files nobody opened. That is the right split, and a reader arriving at that heading from a link now sees it.3. The recount carries its command, and the command works. I ran the published
keyanddifferingfunctions verbatim against/root/tree-m-*.sha256and/root/tree-p-*.sha256, changing only the directory:Every number and the exact residue. Saying plainly that the historical captures predate the tool's format, that the conversion was itself uncommitted shell, and that captures from here on need none of it, is the honest version of this and it applies the note's own rule to the note.
4. The union correction is right, and it inverts the argument correctly in both places. "A union floor is more permissive, not more rigorous: it subtracts strictly more than either pair, so zero against the union proves less than zero against one pair would have. It is the right test because it is the one whose answer does not depend on which two builds were named first, not because it is the harder one." That last clause is the part that matters: it gives the union a reason that survives the concession, rather than leaving the reader to wonder why a weaker test is being preferred.
I re-confirmed the underlying facts as well: the recount table, the three mariadb files inside the other control pair's floor,
./etc/postgresql/17/main/postgresql.confsurviving every pair and the union, and the fourglobal_privanduser.frmfiles differing in the control-unit pair and sitting inside every floor.One thing worth knowing, which is not a change request here because it belongs to the other pull request: I have blocked Keel-Linux/buildtasks#11 on two points, one of which is that a
noiseverdict can still admit a planted value wherever the controls vary at random. It does not affect anything this note asserts, and it does not affect the recount, which is a hash comparison. But this note now makes the block that command prints the standard a component extraction is reviewed on, so the standard is only as good as that command. If the fix there changes what the block looks like, this note's description of it may need a line.History is clean: three commits on the tip of
main, only the two files, all three passing the repository'scommit-msgandpre-commithooks when replayed, and no forbidden strings.Verified by running: the published recount command verbatim against both capture families, the multi-control and union behaviour in
bt-layer-measure, and the branch and hook checks. Inferred, without running: nothing material.Approve.
bb0484dbrings 0010 and the trap entry in line with buildtasks#11's final head: three controls in the example, the enforced preconditions, and the pinned-field verdicts in place of the position cancelling the first version did. 52d1a86 mergesmain; the traps.md conflict was two independent additions, both kept. Still merges after buildtasks#11.