Roadmap: Keel Linux composition architecture (0023-0040) #46

Open
opened 2026-09-30 10:11:34 -03:00 by marcos-mendez · 3 comments
marcos-mendez commented 2026-09-30 10:11:34 -03:00 (Migrated from github.com)

Roadmap for the composition architecture decided by the maintainer on 2026-09-30, recorded as handbook notes 0023 to 0040 (handbook#26). One layer at a time; every phase ships something testable, and a phase is done only when its criterion holds on a built image, not on a hand-assembled machine.

Decision issues: #28 (0023) #29 (0024) #30 (0025) #31 (0026) #32 (0027) #33 (0028) #34 (0029) #35 (0030) #36 (0031) #37 (0032) #38 (0033) #39 (0034) #40 (0035) #41 (0036) #42 (0037) #43 (0038) #44 (0039) #45 (0040)

Phase 0: sharpen the axe

  • Record the decisions: handbook#26 and #28 to #45
  • Map them against 0000 to 0022 (done in each note's "What this amends"; maintainer confirms the review notes)
  • Specify the manifest format, the pivot of backup, monitoring, upgrade, replication and discovery: #39
  • Maintainer validates Template B4 (the WordPress simple / cloud simple test image)
  • Attended release that replaces the published layers still carrying private keys (keel-core#8, keel-wordpress#15)

Done when: the manifest schema is reviewed with WordPress, Odoo and Mastodon examples, Template B4 passes the maintainer's test, and no published layer carries a shared key.

Phase 1: Keel Core

Decisions: #29 #30 #32 #33 #34 #41 #44

  • WireGuard in Core (#29)
  • CrowdSec in Core, installed and disabled (#34)
  • etcd overlay present and stopped (#30, #41)
  • YAML always emitted, complete with defaults (#32)
  • stable and testing tracks declared in the YAML (#44)
  • Overlays shipped as .deb from the Keel repository (#41, #44)
  • Simple / advanced mode screen (#33)

Done when: a Core image boots in simple installation, emits a YAML that re-applies as a no-op, runs no etcd and no active CrowdSec, and upgrades one overlay with apt upgrade on the testing track.

Phase 2: data appliances, standalone and paired

Decisions: #36 #38 #33

  • MariaDB standalone and paired (pair, seeding and read-only replica mostly done in keel 0.11.x)
  • PostgreSQL standalone and paired
  • Redis standalone and paired
  • The semi-sync decision for MariaDB and the timeout policy for both engines (#36 review notes)
  • Automatic failback after full catch-up, for the pair (#36). The failback default stays with the operator (manual, as 0020 says); automatic is implemented here and in Phase 5

Done when: each engine passes a two-container gate in cloud simple: write on the primary, read on the standby, commit latency and the synchronous or semi-synchronous fallback measured, promotion by hand.

Phase 3: Keel Web

Decisions: #35 #34

  • WAF choice: Coraza connector, ModSecurity v3, or Coraza as proxy (#35)
  • Anubis packaged in the Keel repository
  • nginx overlay, Keel Web recipe; Coraza and Anubis disabled in simple

Done when: Keel Web in advanced mode blocks a CRS test payload, challenges a browser through Anubis, lets exempt crawlers through, and in simple mode is plain Nginx.

Phase 4: runtimes, and WordPress on Keel PHP

Decisions: #28 #39 #41

  • php-fpm overlay; Keel PHP = Keel Web + php-fpm
  • WordPress as the first application built from a manifest (LEMP); retires Apache for WordPress
  • python, ruby, nodejs, go overlays

Done when: WordPress on Keel PHP passes the appliance gate in simple and cloud simple, built from its manifest, with no Apache in the image.

Phase 5: mesh control plane

Decisions: #29 #30 #31 #33 #34 #37 #45

  • etcd registry: key layout, node announcement (#30)
  • Installation by discovery (#31)
  • Cloud advanced on three nodes, election and fencing (#33, 0020)
  • Automatic failback after full catch-up on three nodes (#36); the operator still chooses automatic or manual, default manual
  • VIP controller rewriting AllowedIPs from etcd, under 0018's window (#34)
  • Syncthing overlay, send-only / receive-only flipped on promotion (#37)
  • Monit summary to etcd, mesh alerting by watch (#45)

Done when: a three-container gate installs the second and third node by discovery, fails over automatically, moves the VIP, flips the Syncthing folders, fails back after catch-up, and reports each node's Monit summary through etcd.

Phase 6: applications by manifest

Decisions: #39 #40 #38

  • Odoo by manifest (0022 reconciled with #44)
  • Nextcloud by manifest
  • Mastodon as the reference proof (#40)

Done when: each passes the gate in simple and cloud advanced from its manifest alone plus its inithook, Mastodon last.

Phase 7: object storage and backup

Decisions: #43 #42

  • Garage packaged; s3 overlay with zones from the site label (#43)
  • Keel Backup appliance (#42)
  • TKLBAM backend abstraction targeting Keel Backup
  • Periodic restore test into a throwaway container, reported to etcd

Done when: an application is backed up from its standby to Keel Backup in another site, and a scheduled restore test succeeds and is visible in etcd.

Open architecture items

  • DNS successor for nodes behind NAT with changing IPv6 prefixes (PowerDNS plus etcd, undecided) (#29)
  • Elasticsearch vs OpenSearch (#38)
Roadmap for the composition architecture decided by the maintainer on 2026-09-30, recorded as handbook notes 0023 to 0040 (handbook#26). One layer at a time; every phase ships something testable, and a phase is done only when its criterion holds on a built image, not on a hand-assembled machine. Decision issues: #28 (0023) #29 (0024) #30 (0025) #31 (0026) #32 (0027) #33 (0028) #34 (0029) #35 (0030) #36 (0031) #37 (0032) #38 (0033) #39 (0034) #40 (0035) #41 (0036) #42 (0037) #43 (0038) #44 (0039) #45 (0040) ## Phase 0: sharpen the axe - [ ] Record the decisions: handbook#26 and #28 to #45 - [ ] Map them against 0000 to 0022 (done in each note's "What this amends"; maintainer confirms the review notes) - [ ] Specify the manifest format, the pivot of backup, monitoring, upgrade, replication and discovery: #39 - [ ] Maintainer validates Template B4 (the WordPress simple / cloud simple test image) - [ ] Attended release that replaces the published layers still carrying private keys (keel-core#8, keel-wordpress#15) **Done when:** the manifest schema is reviewed with WordPress, Odoo and Mastodon examples, Template B4 passes the maintainer's test, and no published layer carries a shared key. ## Phase 1: Keel Core Decisions: #29 #30 #32 #33 #34 #41 #44 - [x] WireGuard in Core (#29) - [ ] CrowdSec in Core, installed and disabled (#34) - [ ] etcd overlay present and stopped (#30, #41) - [ ] YAML always emitted, complete with defaults (#32) - [ ] stable and testing tracks declared in the YAML (#44) - [ ] Overlays shipped as .deb from the Keel repository (#41, #44) - [ ] Simple / advanced mode screen (#33) **Done when:** a Core image boots in simple installation, emits a YAML that re-applies as a no-op, runs no etcd and no active CrowdSec, and upgrades one overlay with `apt upgrade` on the testing track. ## Phase 2: data appliances, standalone and paired Decisions: #36 #38 #33 - [ ] MariaDB standalone and paired (pair, seeding and read-only replica mostly done in keel 0.11.x) - [ ] PostgreSQL standalone and paired - [ ] Redis standalone and paired - [ ] The semi-sync decision for MariaDB and the timeout policy for both engines (#36 review notes) - [ ] Automatic failback after full catch-up, for the pair (#36). The failback default stays with the operator (manual, as 0020 says); automatic is implemented here and in Phase 5 **Done when:** each engine passes a two-container gate in cloud simple: write on the primary, read on the standby, commit latency and the synchronous or semi-synchronous fallback measured, promotion by hand. ## Phase 3: Keel Web Decisions: #35 #34 - [ ] WAF choice: Coraza connector, ModSecurity v3, or Coraza as proxy (#35) - [ ] Anubis packaged in the Keel repository - [ ] nginx overlay, Keel Web recipe; Coraza and Anubis disabled in simple **Done when:** Keel Web in advanced mode blocks a CRS test payload, challenges a browser through Anubis, lets exempt crawlers through, and in simple mode is plain Nginx. ## Phase 4: runtimes, and WordPress on Keel PHP Decisions: #28 #39 #41 - [ ] php-fpm overlay; Keel PHP = Keel Web + php-fpm - [ ] WordPress as the first application built from a manifest (LEMP); retires Apache for WordPress - [ ] python, ruby, nodejs, go overlays **Done when:** WordPress on Keel PHP passes the appliance gate in simple and cloud simple, built from its manifest, with no Apache in the image. ## Phase 5: mesh control plane Decisions: #29 #30 #31 #33 #34 #37 #45 - [ ] etcd registry: key layout, node announcement (#30) - [ ] Installation by discovery (#31) - [ ] Cloud advanced on three nodes, election and fencing (#33, 0020) - [ ] Automatic failback after full catch-up on three nodes (#36); the operator still chooses automatic or manual, default manual - [ ] VIP controller rewriting AllowedIPs from etcd, under 0018's window (#34) - [ ] Syncthing overlay, send-only / receive-only flipped on promotion (#37) - [ ] Monit summary to etcd, mesh alerting by watch (#45) **Done when:** a three-container gate installs the second and third node by discovery, fails over automatically, moves the VIP, flips the Syncthing folders, fails back after catch-up, and reports each node's Monit summary through etcd. ## Phase 6: applications by manifest Decisions: #39 #40 #38 - [ ] Odoo by manifest (0022 reconciled with #44) - [ ] Nextcloud by manifest - [ ] Mastodon as the reference proof (#40) **Done when:** each passes the gate in simple and cloud advanced from its manifest alone plus its inithook, Mastodon last. ## Phase 7: object storage and backup Decisions: #43 #42 - [ ] Garage packaged; `s3` overlay with zones from the site label (#43) - [ ] Keel Backup appliance (#42) - [ ] TKLBAM backend abstraction targeting Keel Backup - [ ] Periodic restore test into a throwaway container, reported to etcd **Done when:** an application is backed up from its standby to Keel Backup in another site, and a scheduled restore test succeeds and is visible in etcd. ## Open architecture items - [ ] DNS successor for nodes behind NAT with changing IPv6 prefixes (PowerDNS plus etcd, undecided) (#29) - [ ] Elasticsearch vs OpenSearch (#38)
marcos-mendez commented 2026-10-02 00:07:02 -03:00 (Migrated from github.com)

Decisions of 2026-10-02 (handbook#38): #49 (0044, Keel Web in the cloud) and #50 (0045, Keel DNS, PowerDNS authoritative). They go after Phase 3 (Keel Web) and before Phase 5 (mesh control plane): Keel DNS is built as a standalone appliance without etcd, and its registry feed comes in Phase 5.

Between Phase 3 and Phase 5: Keel DNS, and Keel Web in the cloud

Decisions: #49 #50

  • keel-overlay-pdns (PowerDNS 4.9.17 from trixie, gsqlite3, LUA records on) and the keel-dns appliance on Core, simple mode (#50)
  • Keel DNS cloud simple: primary and replica by NOTIFY, transfer and a catalog zone over the mesh; RFC 2136 with TSIG from nodes (#50)
  • Keel Web role in the cloud modes, the web section shared across a set, certificates as shared secrets (#49)
  • Edge node: passthrough sites to the Keel Web nodes, PROXY protocol, VRRP between edge nodes (#49)

Done when: two Keel DNS servers in two sites answer for a test zone with only the live Keel Web nodes, a stopped node leaves the answers within a check interval plus the TTL, and a second Keel Web node serves the same sites and certificates as the first.

Added to Phase 5: the Keel DNS feed from the etcd registry, cloud advanced on three sites (#50), and the web section through etcd (#49).

The open architecture item "DNS successor for nodes behind NAT with changing IPv6 prefixes" is decided by 0045 (#50).

Decisions of 2026-10-02 (handbook#38): #49 (0044, Keel Web in the cloud) and #50 (0045, Keel DNS, PowerDNS authoritative). They go **after Phase 3 (Keel Web) and before Phase 5 (mesh control plane)**: Keel DNS is built as a standalone appliance without etcd, and its registry feed comes in Phase 5. ## Between Phase 3 and Phase 5: Keel DNS, and Keel Web in the cloud Decisions: #49 #50 - [ ] `keel-overlay-pdns` (PowerDNS 4.9.17 from trixie, `gsqlite3`, LUA records on) and the `keel-dns` appliance on Core, simple mode (#50) - [ ] Keel DNS cloud simple: primary and replica by NOTIFY, transfer and a catalog zone over the mesh; RFC 2136 with TSIG from nodes (#50) - [ ] Keel Web role in the cloud modes, the `web` section shared across a set, certificates as shared secrets (#49) - [ ] Edge node: passthrough sites to the Keel Web nodes, PROXY protocol, VRRP between edge nodes (#49) **Done when:** two Keel DNS servers in two sites answer for a test zone with only the live Keel Web nodes, a stopped node leaves the answers within a check interval plus the TTL, and a second Keel Web node serves the same sites and certificates as the first. Added to **Phase 5**: the Keel DNS feed from the etcd registry, cloud advanced on three sites (#50), and the `web` section through etcd (#49). The open architecture item "DNS successor for nodes behind NAT with changing IPv6 prefixes" is decided by 0045 (#50).
marcos-mendez commented 2026-10-02 10:26:00 -03:00 (Migrated from github.com)

Phase 4 (Keel PHP plus WordPress) is based on decision 0047, one image per application: one debian-13-keel-wordpress image on Core, Web and PHP with an embedded MariaDB, the installation mode deciding what runs. Keel-Linux/handbook#40.

Phase 4 (Keel PHP plus WordPress) is based on decision 0047, one image per application: one debian-13-keel-wordpress image on Core, Web and PHP with an embedded MariaDB, the installation mode deciding what runs. Keel-Linux/handbook#40.
marcos-mendez commented 2026-10-02 10:54:19 -03:00 (Migrated from github.com)

Deferred to after the main applications work (maintainer, 2026-10-02): MongoDB and CouchDB data appliances. TurnKey ships both (turnkeylinux-apps/mongodb, couchdb). Neither is in Debian 13: MongoDB is SSPL-licensed (not OSI-approved, would need MongoDB's own repository, against 0039), CouchDB is Apache 2.0 but would need a Keel package. Revisit after Phase 6 (applications by manifest), case by case on demand; CouchDB is the cleaner path. The data services in scope now stay those of 0033: PostgreSQL, MariaDB, Redis, and search (Elasticsearch or OpenSearch). Redis is being added to confconsole's Database mode (confconsole#19).

Deferred to after the main applications work (maintainer, 2026-10-02): **MongoDB and CouchDB data appliances.** TurnKey ships both (turnkeylinux-apps/mongodb, couchdb). Neither is in Debian 13: MongoDB is SSPL-licensed (not OSI-approved, would need MongoDB's own repository, against 0039), CouchDB is Apache 2.0 but would need a Keel package. Revisit after Phase 6 (applications by manifest), case by case on demand; CouchDB is the cleaner path. The data services in scope now stay those of 0033: PostgreSQL, MariaDB, Redis, and search (Elasticsearch or OpenSearch). Redis is being added to confconsole's Database mode (confconsole#19).
Sign in to join this conversation.
No description provided.