keel-wp and keel-wordpress-update, with the turnkey names kept as symlinks #6
No reviewers
Labels
No labels
accessibility
bug
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Keel-Linux/keel-wordpress!6
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/keel-named-commands"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #7. Org-wide context, cross-reference only: Keel-Linux/tracker#12. The policy it implements is handbook decision 0015 (Keel-Linux/handbook#3).
Merge order: Keel-Linux/handbook#3 first. This branch cites decision 0015 in
changelog,COVERAGE.md,tests/wrappers.batsand both script headers, and 0015 is not on the handbook'smainyet.The two commands an operator types on this appliance are written by this overlay, so the name they carry is this project's choice. The Keel name becomes the real command and the
turnkey-*name is kept as a symlink to it. Both keep working: somebody who pasted a command out of TurnKey's documentation, or who wrote a script againstturnkey-wplast year, must not silently lose it.overlay/usr/local/bin/keel-wp, withturnkey-wpa symlink to itoverlay/usr/local/sbin/keel-wordpress-update, withturnkey-wordpress-updatea symlink to itBoth links are relative. An absolute link does not resolve inside
fab-chroot, soconf.d/mainwould stop finding the command at build time; making either absolute fails three tests here.What does not change
keel-wpstill runs wp-cli throughrunuseraswww-data, never as root, with an explicit--path, and still owns/var/www/.wp-cli.keel-wordpress-updatestill refuses a caller who is not root, updates core, verifies WordPress's own per file checksums and puts the ownership boundary back. Its refusal now names whichever of its two names was typed, so an operator who ranturnkey-wordpress-updateis not told about a command they did not run.keel-wpgains no new override: itsWP_DIR,WP_USRandWP_CACHEare the three it already shipped with, and it calls/usr/local/bin/wpliterally as before.keel-wordpress-updatereads its WordPress root, web user and wp-cli path fromKEEL_TEST_WPROOT,KEEL_TEST_WP_USERandKEEL_TEST_WP_CLIwhen set, with the previously hard-coded values as defaults, so a booted appliance behaves identically. The prefix is deliberate — see the review response below. It also now refuses a target that is not a WordPress, by the samewp-includes/version.phpthatconf.d/mainasserts after unpacking.Responding to the review
USERsafeguard was untestable, since under bats$USERequals$(id -un)and the suite used that as the expected valuekeel-test-web-user, which no account can be called;MEis gone. A test runs the updater withUSER=rootand requireswp-contentto still go to the sentinel. The reviewer's mutation,USER="${USER:-www-data}"throughout, now fails three tests.WP_CLI=/bin/true WPROOT=/ keel-wordpress-updatewalks the whole filesystem as rootKEEL_TEST_*, the prefix this suite already uses; and the script refuses a target with nowp-includes/version.phpbefore it rewrites anything. Un-namespacingWPROOTnow fails eight tests; dropping the guard fails one.tests/v19.sh:65compared two command substitutions, passing astest "" = ""$before, the yardstick for the twocore versioncomparisons, is asserted non-empty where it is taken.-dis not what preserves the symlink, and the test could not telltests/wrappers.bats,COVERAGE.mdand thechangelog.-Rcopies a symlink as a symlink unless-Lis given;-Pis already the default and-donly adds--preserve=links, which is hard links. A new test asserts the real property: a plaincp -TRstill yields a workingturnkey-wp, andcp -TLRturns it into a second regular file..github/workflows/tests.yml:20says eight shell files.first=loop is gone.keel-wp:15also gainedWP_CLIrunuseris a stub and the command string is never executed. The test asserts the literal/usr/local/bin/wpinstead.KEEL_TEST_prefixed and read only by the tests;keel-wp's two remain the shipped operator-facing ones. The distinction is now meaningful rather than accidental.I did not remove the second
cp -TdR, which the reviewer flagged as redundant but worth keeping.How the symlink is made, and why that way
Committed into the overlay, not created by
ln -sinconf.d/main.fab-apply-overlayiscp -TdR OVERLAY DEST(cmd_apply_overlayin fab). Measured here with coreutils 9.7:So the property is the absence of
-L.tests/wrappers.batsasserts all three rows, copies the overlay with the build's own command twice (once forCOMMON_OVERLAYS, once for the product-localROOT_OVERLAY) and then runs the copied command.Call sites
A caller left on the old name still works through the symlink, which is exactly why one is easy to miss: the build would pass either way. They were found by
grep -rn, not by the gate.conf.d/mainkeel-wp core verify-checksums --version=...during the build, and the comment above itREADME.rsttests/v19.shturnkey-wpcall, and the updater invocationoverlay/usr/lib/inithooks/firstboot.d/40wordpressis named in #7 as a call site, and it is not one: it calls wp-cli directly with--allow-root, never through either wrapper. Nothing there needed changing.Tests
The 21 original tests were written before the code and watched go red. The four added in response to the review were written the same way, and each was checked against the mutation it exists to catch.
tests/v19.shkeeps everything it drove before — siteurl, post create/get/delete,core version,check-update,verify-checksumsand the non-root updater guard — now under the Keel name, and additionally asks the compatibility names for the same answers on a booted appliance, each against a literal.99.12 percent (564/569) over the eight measured shell files, up from 99.07 percent (535/540) over six on master. Both wrappers were at no coverage at all and are at 100 percent with every branch driven. Threshold stays 97, still set by
40wordpress.Test plan
bats tests/— 221 tests, 0 failuresCOVERAGE_THRESHOLD=97 tests/coverage.sh— exit 0, numbers aboveshellcheckclean on both scripts andtests/coverage.shUSERmutation, un-namespacingWPROOT, and dropping the WordPress guard each fail tests nowappliance / build-and-boot— see belowtests/v19.shon a rebuilt appliance, where the compatibility names are proved on a real machineappliance / build-and-bootfailed on the previous push for a reason outside this branch: the job boots the layer the mirror already serves, which does not carry this change, and every appliance assertion passed. It failed onapt-get update, where the runner resolvesarchive.keellinux.orgto its own loopback (127.0.1.1) and is served a self-signed certificate. This branch touches neithertests/boot-test.shnortests/lib/boot-test-lib.sh.Review
I re-ran the suite and the coverage measurement, read fab's own source for the overlay step, mutated the new tests to see which of them are load-bearing, and read the booted
wordpress-democontainer. Everything the description claims about the mechanism and the numbers holds. Findings below are ordered by severity; none is CRITICAL or HIGH.Verified
cmd_apply_overlayin fab really iscp -TdR OVERLAY DEST(fab:363-373), andprefix-relative-paths(share/product.mk:71-75) leaves an absolute entry alone, soMakefile:36'sCOMMON_OVERLAYS += $(CURDIR)/overlayis applied as itself.ROOT_OVERLAY ?= overlay(share/product.mk:86) applies the same directory a second time atshare/product.mk:386. The overlay is applied twice, as claimed.share/product.mk:386(root overlay) precedes:391(run-conf-scripts $(CONF_SCRIPTS), andCONF_SCRIPTS ?= conf.d), sokeel-wpexists whenconf.d/main:94calls it.cp -TdRover an existing link leaves a link (GNU coreutils 9.7), and atarround trip carries it. Both checked by hand.overlay/usr/lib/inithooks/firstboot.d/40wordpress:105-107defines its ownwp() { command wp --allow-root --path="$WORDPRESS_ROOT" "$@"; }and never touches either wrapper. A per-tracked-file grep over all 21 repositories and 6 worktrees finds no reference to either name outside this repository./etc/cron.d/wordpress-croncurlswp-cron.phpand nothing else. Nothing exportsWPROOT,WP_USER,WP_CLI,WP_DIRorWP_USRfrom/etc/environment,/etc/profile*,/etc/defaultor root's dotfiles.COVERAGE_THRESHOLD=97 tests/coverage.sh: 217 tests, 0 failures, and the eight-file table reproduces the description line for line. 562/567 is 99.12 percent. Both wrappers at 100 percent, 9/9 and 18/18.WP_USER-rather-than-USERreasoning is right, and the old file shows why. The version on the running appliance opensUSER=www-data— a plain assignment, which masks the inherited value. Turning that into${USER:-www-data}would have read root's login environment, chowned all fivewp-contentruntime directories toroot:rootand madewp-config.phpgroup root, so uploads and plugin installs would fail on the first supervised update and not before. Good catch, correctly avoided.${0##*/}to a literal (15),chmod 0644onwp-config.php(20), dropping its group (20), droppingverify-checksums(17, 19), removing theDEBUGline (11), removing the cachechown(7, 8, 10), and replacingprintf '%q 'with$*(5, 6). The two that were not caught are MEDIUM 1 and LOW 1 below.MEDIUM 1 — the one safeguard flagged for review is the one the suite cannot fail on
tests/wrappers.bats:25andtests/wrappers.bats:72setup()takesME="$(id -un)"and then exportsWP_USR="$ME" WP_USER="$ME". Under bats,$USERalready is$(id -un), so${USER:-www-data}and${WP_USER:-www-data}expand to the same string and no assertion can tell them apart. I rewroteoverlay/usr/local/sbin/keel-wordpress-updateto useUSER="${USER:-www-data}"throughout — the exact bug the description says was deliberately avoided — and all 21 tests stayed green.The scenario: someone later "tidies" the variable back to
USER, or writes the next wrapper with${USER:-www-data}because the suite blessed it. The build is green, the appliance boots, and the firstkeel-wordpress-updatean operator runs as root leaveswp-content/uploadsowned by root. Media uploads and plugin installs fail from then on, with nothing pointing back at the update.chown,installandrunuserare all stubs that only log, soWP_USERdoes not need to name a real account. Setting it to a sentinel that$USERcan never equal closes this in one line:MEthen has no remaining use.MEDIUM 2 — a destructive root-run script now takes its target path from an unvalidated environment variable
overlay/usr/local/sbin/keel-wordpress-update:12and:14This is the half of the refactor that deserves the scrutiny it was flagged for. Lines 25-27 are
chown -R root:root "$WPROOT",find "$WPROOT" -type d -exec chmod 0755 {} +andfind "$WPROOT" -type f -exec chmod 0644 {} +, all as root. WithWPROOTandWP_CLIboth overridable and neither validated,WP_CLI=/bin/true WPROOT=/ keel-wordpress-updatewalks the whole filesystem and strips setuid fromsudo,suandpasswdon the way. Before this change that was not expressible.To be clear about what this is not: it is not a privilege escalation. I checked for a constrained root path and there is none — no sudoers rule, no cron entry, no unit, no confconsole plugin calls this, and
sudo'senv_resetwould strip the variables anyway. The only caller is a root shell, where the environment is already the caller's to choose. That is why this is MEDIUM and not HIGH.It is still a footgun the hard-coded version did not have, and
WPROOTis not an obscure name — this repository's ownconf.d/main:23usesWPROOTfor the same path. Two cheap fixes, either one sufficient:KEEL_TEST_for exactly this purpose (KEEL_TEST_UID,KEEL_TEST_WP_FAIL,KEEL_TEST_WP_CODE,KEEL_TEST_CHOWN_CODE,KEEL_TEST_RUNUSER_CODE).KEEL_TEST_WPROOTandKEEL_TEST_WP_CLIbuy identical testability with none of the exposure.conf.d/main:86already does this on the same path:[ -f "$WPROOT/wp-includes/version.php" ] || fatal ....Related, minor:
overlay/usr/local/bin/keel-wp:15also gainedWP_CLI, which the description does not list among the new overrides (it names only the updater's three). Lower risk there, since it lands insiderunuser … -cand so runs aswww-data, but the description should say so.MEDIUM 3 — the compatibility assertion on a real appliance passes if both sides fail
tests/v19.sh:65set -euo pipefailis on, but a command substitution intest's arguments does not trip errexit. If wp-cli is broken, or the path is wrong, or Apache is down, both substitutions are empty and this istest "" = "", which passes. This is the traps.md entry "asserting the configuration is not asserting the behaviour", in the one test offered as proof that the old name still answers on a real machine — and it is the unchecked box in the test plan.The script already knows the literal it wants; it asserted it at line 22. Comparing against it instead costs nothing and cannot pass on a double failure:
Line 64 (
core versionagainst$before) has the same shape, but$beforeis never asserted non-empty and that predates this change, so I am not counting it here.LOW 1 —
-dis not what keeps the link a link, and the test cannot telltests/wrappers.bats:20and:186-191,COVERAGE.md:53-55,changelog:13-14The conclusion is right and I verified it; the attributed mechanism is not. In GNU cp,
-P/--no-dereferenceis already the default under-R, and only-Ldereferences. Measured here:So what
-dactually contributes to the overlay step is--preserve=links, i.e. hard links. Symlink survival hinges on nobody passing-L. Confirming this: I dropped-dfrom the test's owncpand no test failed, so the test does not verify the property its comment says it verifies.The scenario is a reader's, not a runtime one: someone adds a copy step, reads this comment, and either believes a copy without
-dwill flatten links, or believes adding-dmakes any copy safe. Same wording appears in the decision note, where it matters more.LOW 2 — a comment this change made stale
.github/workflows/tests.yml:20— "tests/coverage.sh measures six shell files (COVERAGE.md)".tests/coverage.sh:28now names eight andCOVERAGE.md:24says eight. Thethreshold: 97below it is still correct.LOW 3 — four names for two concepts in two files shipped side by side
overlay/usr/local/bin/keel-wp:12-13usesWP_DIRandWP_USR;overlay/usr/local/sbin/keel-wordpress-update:12-13usesWPROOTandWP_USERfor the same WordPress root and the same web user. The cost shows up immediately attests/wrappers.bats:71-72, which has to export all four.keel-wp's two are inherited, so this is a note rather than a request — but if MEDIUM 2 is taken and the updater's overrides are renamed anyway, that is the moment to make them agree.LOW 4 — dead code in the wp-cli stub
tests/wrappers.bats:60-63. Thefor arg in "$@" … first=$arg; breakloop computesfirstand nothing ever reads it. In a file whose whole argument is precision about what is being asserted, it reads as a leftover.LOW 5 — two test descriptions claim slightly more than the tests do
tests/wrappers.bats:150, "keel-wp reports the exit code wp-cli gave it", drivesKEEL_TEST_RUNUSER_CODE=3.runuseris a stub that never executes the command string, so wp-cli is not reached; what is proved is thatrunuser's exit code is handed back. True in practice, but the name asserts the longer chain.tests/wrappers.bats:204copies the overlay once where:194-195copies it twice. The double application is therefore proved forturnkey-wponly. One more line would cover the updater's link too.Not findings, recorded so they are not re-litigated
cp -TdRattests/wrappers.bats:195is redundant against coreutils 9.7 — removing it fails nothing. It is still correct insurance and would catch a cp that behaved differently on the second pass, so keep it.readlinkgiveskeel-wpandkeel-wordpress-update), which is the right choice: an absolute link would not resolve insidefab-chroot. Making the link absolute breaks tests 1, 12 and 13, so this is protected.100755on both scripts,120000on both links. Neither script holds a credential.chown root:"$WP_USER"pluschmod 0640onwp-config.phpkeeps the database password off world-read, and that is asserted attests/wrappers.bats:257-258.What I verified versus what I inferred
Verified by execution or by reading the source: the fab copy command and the double application and their ordering, cp's behaviour over an existing link and what
-ddoes and does not do, the tar round trip, all coverage and test-count numbers, every mutation result above, the absence of any unattended or sudo-constrained caller, and the state of the two files on the booted appliance.Inferred: that MEDIUM 2 stays MEDIUM. It rests on there being no constrained root path to this script today, which I checked across this organization's trees and one booted appliance. A sudoers rule added later, or an operator tool that exports
WPROOT, changes that reading.Approve. No CRITICAL or HIGH issues. MEDIUM 1 is the one I would like to see before merge — it is a one-line change to the test setup, and without it the safeguard that was explicitly put up for review is unprotected.
Re-review
I checked each fix by running it, not by reading the diff. Every claim in the update holds, with one numeric correction. The suite is 25 tests in
tests/wrappers.bats, 221 across the repository, 0 failures, and the coverage table reproduces byte for byte.The three MEDIUM findings, re-tested
MEDIUM 1, the
USERsafeguard — fixed and now load-bearing.ME/$(id -un)is gone;WEB_USER=keel-test-web-useris a name no account can hold. I re-ran my original mutation, rewriting the script toUSER="${USER:-www-data}"and using$USERthroughout, against the new tree:Three tests, as claimed — the numbers are 22, 23 and 24, not the 20, 23 and 24 in the description. Substance unaffected; worth correcting only because the description is where the next person will look.
Test 24 is the right shape: it sets
USER=rootin the caller's environment and then requireswp-content/uploadsto still go to the sentinel, with! grep -q '^chown -R root:root .*wp-content'as the negative half. That is the actual failure mode, driven rather than described.MEDIUM 2, the footgun — both fixes taken, both load-bearing, namespacing complete in the script the finding named.
Un-namespacing
WPROOTalone, leaving the other two prefixed, fails eight tests exactly as claimed: 14, 19, 20, 21, 22, 23, 24, 25. Dropping thewp-includes/version.phpguard fails exactly one, 18. The mechanism is right too — withWPROOTun-prefixed the script falls back to/var/www/wordpress, the new guard refuses it, and everything downstream stops. The guard is doing work rather than sitting in front of tests that would pass anyway.Namespacing is complete in
keel-wordpress-update: all three knobs areKEEL_TEST_*(lines 19-21), andkeel-wpgaveWP_CLIback entirely and calls/usr/local/bin/wpliterally at line 22, whichtests/wrappers.bats:121now pins as a literal string.One thing I want to name rather than leave implicit, because it is the only un-namespaced override left anywhere in these two files.
keel-wp:14still readsWP_CACHE=${WP_CACHE:-/var/www/.wp-cli}, andkeel-wp:19ischown -R "$WP_USR":"$WP_USR" "$WP_CACHE", run as root. That is the same shape as the finding, one size smaller. It is not a regression — it is byte-identical to what master shipped, this change did not touch it, and the new comment in the test setup makes the carve-out deliberate and argued ("keel-wp's three knobs are the ones it already shipped with and are the appliance's own").WP_DIRandWP_USRare genuinely harmless, since they land insiderunuserand run aswww-data.WP_CACHEis the odd one out: it chooses a path that root then walks withchown -R. So the finding is discharged as filed, and this is a separate, pre-existing, LOW item — the obvious next application of the reasoning the updater now carries in its own comment, not a condition on this merge.MEDIUM 3, the vacuous v19 assertion — fixed, and the line I had excluded was fixed too.
tests/v19.sh:65is nowtest "$(turnkey-wp option get siteurl)" = https://localhost. For the version comparison, which cannot be a literal because the version is not known in advance, they addedtest -n "$before"at line 48 instead. That is the better fix for that line: it makes the empty-empty case impossible at the source rather than at each comparison. The comment above the block states the errexit reasoning, so the next person editing it knows why it is not two substitutions.The
-dcorrection — accepted, and now asserted in a way that failsThis is the one I was most interested in, because the old comment could not fail. Test 15 pins the real semantics:
cp -TRmust still yield a runnableturnkey-wp, andcp -TLRmust turn both compatibility names into regular files.To confirm it fails when the claim is violated, I put a
cpshim first inPATHthat rewrites-TRand-TdRto-TLR— that is, acpthat dereferences under-R, which is precisely the world in which this policy's mechanism is wrong:It fails. A coreutils change, or a different
cp, would be caught. Committingturnkey-wpas a regular file also fails test 15. This is now a live assertion about the toolchain rather than a sentence about a flag.The
changelogandCOVERAGE.mdwording was corrected to match, and it is correct as written:-Pdefault under-R,-dadds--preserve=linksfor hard links, the property is the absence of-L.The five LOW items
All addressed. The dead
firstloop is gone from the wp stub; test 9 is renamed to "hands back the exit code runuser gave it", which is what it proves; test 14 now copies twice like test 13, so the double application is covered for both names;.github/workflows/tests.yml:20says eight shell files. On the naming inconsistency: rather than makingWP_DIR/WPROOTandWP_USR/WP_USERagree, the two scripts now have visibly different kinds of interface — appliance knobs on one,KEEL_TEST_knobs on the other. That is a better resolution than the one I suggested, and the comment at lines 12-18 says why.The numbers
COVERAGE_THRESHOLD=97 tests/coverage.sh, run here: 221 tests, 0 failures, and every per-file figure matchesCOVERAGE.md—keel-wordpress-update21/21 (up from 18 with the guard),keel-wp8/8 (down from 9 withWP_CLIremoved), 564/569 total, which is 99.12 percent. Threshold still 97, still set by40wordpressat 97.73.The changelog gate, which moved under this branch
Keel-Linux/.githubmergedfix/changelog-version-in-the-name(45fdf3c) mid-flight, soentry_versionnow folds a trailing version in the source name into the comparison. I ran the shipped function against both entries:dpkg --compare-versionsorders3 lt 4under the old reading and19.0-3 lt 19.0-4under the new one. Passes both, and the live check agrees at 7 seconds.The appliance job's green, which I am not counting
appliance / build-and-bootpasses now. It should not be read as evidence for this change, and the description is right that nothing in the branch caused the flip.What the log actually says: the layer is assembled from
https://mirror.keellinux.org/layers, every WordPress assertion passes, and thenI confirmed the skip guard is inherited: it came to master in
3a5d5c1, it is an ancestor of this branch, and the branch touches neithertests/boot-test.shnortests/lib/boot-test-lib.shnor.keel-ci. So the red on the earlier run and the green on this one are both the runner's resolver, and neither is about the code.The deeper point is the one already filed as
Keel-Linux/.github#11, case 2: the job boots the published layer, which on a pull request that changes the recipe is the code before the change. So this check was incapable of testing this branch in either direction. I am treating it as contributing nothing, not as a pass.That leaves the evidence for this change where it was: 25 unit tests, including two that run the compatibility name out of a copy made with the build's own command, and
tests/v19.shon a rebuilt appliance — still the unchecked box in the test plan, and still the only thing that will prove the link on a machine built from this branch. I am content with that for merge, because the unit tests now cover the overlay copy end to end and the remaining risk is confined to a rebuild that is going to happen anyway. Two smaller notes on the skip, both for tracker#5 /.github#11rather than for this pull request: the skip names what it did not prove, why, and where it is tracked, which is the right shape; but it leaves the final line readingwordpress boot test passedand the check name unqualified, so a reader who sees only the green tick sees an unconditional pass. That is the failure mode #11 exists for, in a milder form.What I verified versus what I inferred
Verified by running: all 25 tests and the 221-test coverage run; the
USERmutation and its three failures; theWPROOTun-namespacing and its eight; the guard removal and its one; the dereferencing-cpshim and its three, including test 15; the regular-file mutation;entry_versionon both changelog entries and thedpkgordering; the appliance log; and that the skip guard predates this branch.Inferred: that
WP_CACHEinkeel-wpis worth a follow-up rather than a change here. That rests on it being unchanged from master and on there still being no constrained root path to either command — I checked for one last round and nothing in this round adds an overlay, unit, cron entry or sudoers file.Approve. Every finding from the first pass is discharged, and the two that could previously pass vacuously now fail when they should. Nothing above LOW remains, and the single LOW is pre-existing rather than introduced here.