WIP: test: the MariaDB pair's cases (b) to (e), end to end (follow-up of #93) #95

Draft

marcos-mendez wants to merge 2 commits from feat/mariadb-pair-scenario into feat/mariadb-pair-semisync

marcos-mendez commented

2026-10-08 13:38:25 -03:00

(Migrated from github.com)

The rest of the end to end scenario of docs/replication.md on #93's harness, deferred from #93 so that it ships with its product changes, unit tests and case (a). Rebases onto main once #93 merges.

(b) the semi-synchronous fallback after 10 s with the replica cut: drift in keel diff, the alert at the webhook, writes going on; the recovery after the heal and its alert;
(c) keel database promote on the replica while the application writes at the VIP: the longest gap in acknowledged writes, every acknowledged row on the new primary, the old primary rejoined as the replica;
(d) the primary cut off idle: the VIP's failover per keel#81, the database following, writes resuming, the old primary rejoining by itself after the heal; then the primary cut off while written to: back diverged, read only, the errant GTIDs in keel diff and keel database status, the alert; reseeded with --destroy-local-database;
(e) the replica refusing an INSERT as root by the socket and as the application at its address (1290).

Measured once on one host (2026-10-08, full harness): (b) 0.28 to 0.31 s per commit with the replica acknowledging; 10.04 s for the first commit after the cut, 0.04 s for the next; the fallback alerted at the next watch; semi-synchronous again 7.3 s after the heal. (e) 1290 for both, only 'mysql'@'localhost' holding READ_ONLY ADMIN. (c) and (d) were not measured in that run: the CLI refused the spec on the harness roots (no manifests under them), fixed since and carried here; a run with that fix is this PR's first job.

Harness work left here: the agents run keel diff and keel database promote through the CLI on scratch roots (the manifests of the format are now installed under each root), the writer at the VIP and its gaps, the partitions and heals of tests/vip_netns.py, the webhook in the routing namespace.

The rest of the end to end scenario of docs/replication.md on #93's harness, deferred from #93 so that it ships with its product changes, unit tests and case (a). Rebases onto main once #93 merges. - (b) the semi-synchronous fallback after 10 s with the replica cut: drift in keel diff, the alert at the webhook, writes going on; the recovery after the heal and its alert; - (c) `keel database promote` on the replica while the application writes at the VIP: the longest gap in acknowledged writes, every acknowledged row on the new primary, the old primary rejoined as the replica; - (d) the primary cut off idle: the VIP's failover per keel#81, the database following, writes resuming, the old primary rejoining by itself after the heal; then the primary cut off while written to: back diverged, read only, the errant GTIDs in keel diff and keel database status, the alert; reseeded with --destroy-local-database; - (e) the replica refusing an INSERT as root by the socket and as the application at its address (1290). Measured once on one host (2026-10-08, full harness): (b) 0.28 to 0.31 s per commit with the replica acknowledging; 10.04 s for the first commit after the cut, 0.04 s for the next; the fallback alerted at the next watch; semi-synchronous again 7.3 s after the heal. (e) 1290 for both, only 'mysql'@'localhost' holding READ_ONLY ADMIN. (c) and (d) were not measured in that run: the CLI refused the spec on the harness roots (no manifests under them), fixed since and carried here; a run with that fix is this PR's first job. Harness work left here: the agents run keel diff and keel database promote through the CLI on scratch roots (the manifests of the format are now installed under each root), the writer at the VIP and its gaps, the partitions and heals of tests/vip_netns.py, the webhook in the routing namespace.

This pull request is marked as a work in progress.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feat/mariadb-pair-scenario:feat/mariadb-pair-scenario

git switch feat/mariadb-pair-scenario

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch feat/mariadb-pair-semisync

git merge --no-ff feat/mariadb-pair-scenario

git switch feat/mariadb-pair-scenario

git rebase feat/mariadb-pair-semisync

git switch feat/mariadb-pair-semisync

git merge --ff-only feat/mariadb-pair-scenario

git switch feat/mariadb-pair-scenario

git rebase feat/mariadb-pair-semisync

git switch feat/mariadb-pair-semisync

git merge --no-ff feat/mariadb-pair-scenario

git switch feat/mariadb-pair-semisync

git merge --squash feat/mariadb-pair-scenario