SECURITY: webmin accepts root with an empty password before first boot completes #14

New issue

Open

opened 2026-09-28 03:35:58 -03:00 by marcos-mendez · 1 comment

marcos-mendez commented

2026-09-28 03:35:58 -03:00

(Migrated from github.com)

Filed in the private handbook deliberately. Keel-Linux/common, inithooks
and tracker are public; this must not be described there until it is fixed.

Between the moment an appliance boots and the moment firstboot.d/30rootpass
completes, Webmin accepts root with an empty password on port 12321.

The mechanism, each link verified

common/conf/turnkey.d/rootpass ships, when ROOT_PASS is unset at build
time, which is the normal case:
```
# this is a blank password
usermod -p U6aMy0wojraho root
```
U6aMy0wojraho is a valid DES hash of the empty string, not a locked
account. Computed independently:
```
crypt("", "U6")  ->  U6aMy0wojraho     matches what the build writes
crypt("x", "U6") ->  U6QAAaFC1TvPI
```
A locked account would carry ! or *. This one matches an empty
submitted password by ordinary comparison, so nullok below is a second
independent weakening rather than the cause.
Verified on two live containers: /etc/pam.d/webmin is
auth required pam_unix.so nullok; /etc/webmin/miniserv.conf has
passwd_mode=0 and no allow=; /etc/webmin/miniserv.users is
root:x:0; miniserv binds 0.0.0.0:12321 and [::]:12321;
WEBMIN_FW_TCP_INCOMING opens 12321; systemctl is-enabled webmin is
enabled; miniserv.pl contains no empty-password guard.

SSH is not affected: PermitRootLogin yes, but OpenSSH defaults
PermitEmptyPasswords no.

Why the existing mitigation does not close it

default/turnkey-init-fence lists HTTPS_PORTS=(443 12321 12322), so the
fence redirects 12321 while it runs. But lib/init-fence.sh:49-53
deliberately skips an address family whose nat table is unavailable, warns,
and carries on. On an LXC system container without nat that is the normal path,
and no 12321 rules were observed on the forum container.

Three things compound it:

docs/traps.md entry 1 is a first boot that hangs forever in one hook. This
project has hit it three times. While it hangs, 30rootpass has not run.
The brief is IPv6-first: a public routable address and no NAT in front.
The firewall opens 12321 on purpose.

So the window is reachable from the internet for as long as a first boot takes,
and indefinitely if it hangs.

What is and is not established

Verified by reading and by inspecting two live containers: the hash, the PAM
stack, the miniserv configuration, the bind on both families, the fence port
list, and the nat-skip path.

Not demonstrated end to end. No login was attempted against a pre-init
appliance: both available containers already have root passwords set, and
building one would have taken the build lock. High confidence in the mechanism;
the exploit itself is unproven.

Neither common#8 nor inithooks#14 introduces this. It is inherited upstream
behaviour and is therefore likely to affect TurnKey appliances as well, which
makes it a coordinated-disclosure question and not only ours. tracker#3
defers upstream reporting to about the end of October; a security defect is not
what that deferral was for.

Exposure right now

Existing containers on the build host have root passwords set and are not
exploitable. The risk is any appliance built from a published Keel layer that
is booted on a reachable address, and any appliance whose first boot hung.

Remediation to decide between

Lock the account instead of hashing the empty string: ! or * in the
password field, so nothing can match. The reason the empty hash exists at all
needs establishing first, because inithooks may depend on being able to
authenticate before 30rootpass.
Remove nullok from /etc/pam.d/webmin regardless. It is a second,
independent weakening with no stated purpose here.
Do not start Webmin until 30rootpass has completed, which is the only fix
that does not depend on the fence working.
Bind miniserv to loopback until first boot finishes, or give it an allow=.
Make lib/init-fence.sh fail loudly rather than warn and continue when it
cannot install rules for a family. A mitigation that silently declines is
worse than none, because it is counted on.

The fence's silent skip should be treated as its own defect whatever is decided
about the password.

**Filed in the private handbook deliberately. `Keel-Linux/common`, `inithooks` and `tracker` are public; this must not be described there until it is fixed.** Between the moment an appliance boots and the moment `firstboot.d/30rootpass` completes, Webmin accepts **root with an empty password** on port 12321. ## The mechanism, each link verified 1. `common/conf/turnkey.d/rootpass` ships, when `ROOT_PASS` is unset at build time, which is the normal case: # this is a blank password usermod -p U6aMy0wojraho root 2. `U6aMy0wojraho` is **a valid DES hash of the empty string**, not a locked account. Computed independently: crypt("", "U6") -> U6aMy0wojraho matches what the build writes crypt("x", "U6") -> U6QAAaFC1TvPI A locked account would carry `!` or `*`. This one matches an empty submitted password by ordinary comparison, so `nullok` below is a second independent weakening rather than the cause. 3. Verified on two live containers: `/etc/pam.d/webmin` is `auth required pam_unix.so nullok`; `/etc/webmin/miniserv.conf` has `passwd_mode=0` and **no `allow=`**; `/etc/webmin/miniserv.users` is `root:x:0`; miniserv binds `0.0.0.0:12321` **and `[::]:12321`**; `WEBMIN_FW_TCP_INCOMING` opens 12321; `systemctl is-enabled webmin` is enabled; `miniserv.pl` contains no empty-password guard. SSH is not affected: `PermitRootLogin yes`, but OpenSSH defaults `PermitEmptyPasswords no`. ## Why the existing mitigation does not close it `default/turnkey-init-fence` lists `HTTPS_PORTS=(443 12321 12322)`, so the fence redirects 12321 while it runs. But `lib/init-fence.sh:49-53` **deliberately skips an address family whose nat table is unavailable**, warns, and carries on. On an LXC system container without nat that is the normal path, and no 12321 rules were observed on the `forum` container. Three things compound it: - `docs/traps.md` entry 1 is a first boot that hangs forever in one hook. This project has hit it three times. While it hangs, `30rootpass` has not run. - The brief is IPv6-first: a public routable address and no NAT in front. - The firewall opens 12321 on purpose. So the window is reachable from the internet for as long as a first boot takes, and indefinitely if it hangs. ## What is and is not established Verified by reading and by inspecting two live containers: the hash, the PAM stack, the miniserv configuration, the bind on both families, the fence port list, and the nat-skip path. **Not demonstrated end to end.** No login was attempted against a pre-init appliance: both available containers already have root passwords set, and building one would have taken the build lock. High confidence in the mechanism; the exploit itself is unproven. Neither `common#8` nor `inithooks#14` introduces this. It is inherited upstream behaviour and is therefore likely to affect TurnKey appliances as well, which makes it a coordinated-disclosure question and not only ours. `tracker#3` defers upstream reporting to about the end of October; a security defect is not what that deferral was for. ## Exposure right now Existing containers on the build host have root passwords set and are not exploitable. The risk is any appliance built from a published Keel layer that is booted on a reachable address, and any appliance whose first boot hung. ## Remediation to decide between - Lock the account instead of hashing the empty string: `!` or `*` in the password field, so nothing can match. The reason the empty hash exists at all needs establishing first, because inithooks may depend on being able to authenticate before `30rootpass`. - Remove `nullok` from `/etc/pam.d/webmin` regardless. It is a second, independent weakening with no stated purpose here. - Do not start Webmin until `30rootpass` has completed, which is the only fix that does not depend on the fence working. - Bind miniserv to loopback until first boot finishes, or give it an `allow=`. - Make `lib/init-fence.sh` fail loudly rather than warn and continue when it cannot install rules for a family. A mitigation that silently declines is worse than none, because it is counted on. The fence's silent skip should be treated as its own defect whatever is decided about the password.

marcos-mendez commented

2026-09-28 06:20:48 -03:00

(Migrated from github.com)

Two corrections to the mechanism described above, both measured with
unix_chkpwd (the helper pam_unix actually executes) against a scratch
shadow bind-mounted over /etc/shadow under unshare -r -m. Both make the
defect worse than first stated.

1. It was not only an empty password. It was any password.

With nullok and a stored field of U6aMy0wojraho, Linux-PAM's
_unix_blankpasswd tests crypt("", stored) == stored and returns success
without consulting the submitted password at all. So the original
description, "accepts root with an empty password", understates it: in that
window Webmin accepted root with anything typed into the box.

2. Removing nullok alone would not have closed it.

The stored field is still what crypt("") returns, so the ordinary comparison
matches an empty submitted password even with nullok gone. The two changes are
each necessary and neither is sufficient.

And the obvious fix would have broken first boot

/etc/default/inithooks ships SUDOADMIN=false, so firstboot.d/29sudoadmin
runs turnkey-sudoadmin off --disable-setpass on every first boot, before
30rootpass, and that path calls passwd --unlock root. Measured against
shadow in a scratch root:

stored field	`passwd -u`	result
`!`	exit 3	unchanged, "unlocking the password would result in a passwordless account"
`*`	exit 0	unchanged
`U6aMy0wojraho`	exit 0	unchanged

turnkey-sudoadmin runs under bash -e, so that exit 3 ends the hook before
permitrootlogin_ssh, restart_sshd and set_status, and
/etc/sudoadmin/state is never written. Locking the account is therefore
exactly the fix that breaks first boot, which is docs/traps.md entry 1.

The placeholder is now *: it survives --unlock, crypt() returns it for no
input, and bin/setpass.py replaces the field with chpasswd regardless.

What the empty hash actually bought

Nothing in the first boot authenticated against it. sshd has
PermitRootLogin yes but OpenSSH defaults PermitEmptyPasswords no, and the
first-boot dialog owns tty1 through Conflicts=getty@tty1.service, so no login
prompt exists while it runs. Its only function was a console login on an image
whose first boot has not run, which is the state being closed.

Fixed in

Keel-Linux/common#12 — the placeholder, nullok removed, and Webmin held
until the first boot marker exists. Also quoted ROOT_PASS, which was
expanded against the build directory, so a build password containing a glob
character was not the password that got set: a separate real defect in the
same script.
Keel-Linux/inithooks#18 — the fence now fails closed rather than
loudly: fence_close_port REJECTs in the filter table, which a container
does have, inserted at -I INPUT 1 so it precedes the ACCEPT rules
webmin-fw installs for the same port.

Not demonstrated end to end: no appliance was booted, so the final verdict
belongs to an appliance boot test. conf/samba-rootpass carries the same
literal for the same reason and is deliberately out of scope; no Keel appliance
builds it.

Two corrections to the mechanism described above, both measured with `unix_chkpwd` (the helper `pam_unix` actually executes) against a scratch shadow bind-mounted over `/etc/shadow` under `unshare -r -m`. **Both make the defect worse than first stated.** **1. It was not only an empty password. It was any password.** With `nullok` and a stored field of `U6aMy0wojraho`, Linux-PAM's `_unix_blankpasswd` tests `crypt("", stored) == stored` and returns success **without consulting the submitted password at all**. So the original description, "accepts root with an empty password", understates it: in that window Webmin accepted root with anything typed into the box. **2. Removing `nullok` alone would not have closed it.** The stored field is still what `crypt("")` returns, so the ordinary comparison matches an empty submitted password even with `nullok` gone. The two changes are each necessary and neither is sufficient. ## And the obvious fix would have broken first boot `/etc/default/inithooks` ships `SUDOADMIN=false`, so `firstboot.d/29sudoadmin` runs `turnkey-sudoadmin off --disable-setpass` on every first boot, before `30rootpass`, and that path calls `passwd --unlock root`. Measured against shadow in a scratch root: | stored field | `passwd -u` | result | |---|---|---| | `!` | **exit 3** | unchanged, "unlocking the password would result in a passwordless account" | | `*` | exit 0 | unchanged | | `U6aMy0wojraho` | exit 0 | unchanged | `turnkey-sudoadmin` runs under `bash -e`, so that exit 3 ends the hook before `permitrootlogin_ssh`, `restart_sshd` and `set_status`, and `/etc/sudoadmin/state` is never written. Locking the account is therefore exactly the fix that breaks first boot, which is `docs/traps.md` entry 1. The placeholder is now `*`: it survives `--unlock`, `crypt()` returns it for no input, and `bin/setpass.py` replaces the field with `chpasswd` regardless. ## What the empty hash actually bought Nothing in the first boot authenticated against it. sshd has `PermitRootLogin yes` but OpenSSH defaults `PermitEmptyPasswords no`, and the first-boot dialog owns tty1 through `Conflicts=getty@tty1.service`, so no login prompt exists while it runs. Its only function was a console login on an image whose first boot has not run, which is the state being closed. ## Fixed in - `Keel-Linux/common#12` — the placeholder, `nullok` removed, and Webmin held until the first boot marker exists. Also quoted `ROOT_PASS`, which was expanded against the build directory, so a build password containing a glob character was not the password that got set: a separate real defect in the same script. - `Keel-Linux/inithooks#18` — the fence now **fails closed** rather than loudly: `fence_close_port` REJECTs in the filter table, which a container does have, inserted at `-I INPUT 1` so it precedes the ACCEPT rules `webmin-fw` installs for the same port. Not demonstrated end to end: no appliance was booted, so the final verdict belongs to an appliance boot test. `conf/samba-rootpass` carries the same literal for the same reason and is deliberately out of scope; no Keel appliance builds it.