Consensus V2: larger model (bge-m3 / 1024 dims) + intent guard for same-place pairs #10

New issue

Open

opened 2026-07-09 23:14:37 -03:00 by navigator · 1 comment

navigator commented

2026-07-09 23:14:37 -03:00

Owner

Domain acronyms miss the merge: UBS vs posto de saúde = 0.111 (threshold 0.10).
Same place, different intervention false-merges: ciclovia na avenida Brasil vs recapear a avenida Brasil = 0.078 — topic/entity dominates intent. (paraphrase-multilingual-MiniLM-L12-v2 was evaluated and is worse on both.)

V2 direction:

bge-m3 (1024 dims) or similar: needs a migration changing the vector(384) columns of 0130 + full re-embed (embeddings are derived data; clusters with an active SLA are immutable).
Cheap intent guard on top of the distance: extract the intervention verb/action lemma and veto merges when actions conflict even under the threshold.
Re-cluster job for the backlog (fatia 2 do plano): idempotent, skips clusters with SLA, emits proper events.

0.27.0 shipped real semantics with multilingual-e5-small (384d). Calibration on pt-BR civic pairs exposed two measured limitations that a small retrieval model cannot solve (see `crates/platform/consensus/src/model_embedder.rs` tests): 1. **Domain acronyms miss the merge**: `UBS` vs `posto de saúde` = 0.111 (threshold 0.10). 2. **Same place, different intervention false-merges**: `ciclovia na avenida Brasil` vs `recapear a avenida Brasil` = 0.078 — topic/entity dominates intent. (paraphrase-multilingual-MiniLM-L12-v2 was evaluated and is worse on both.) V2 direction: - bge-m3 (1024 dims) or similar: needs a migration changing the `vector(384)` columns of 0130 + full re-embed (embeddings are derived data; clusters with an active SLA are immutable). - Cheap intent guard on top of the distance: extract the intervention verb/action lemma and veto merges when actions conflict even under the threshold. - Re-cluster job for the backlog (fatia 2 do plano): idempotent, skips clusters with SLA, emits proper events.

navigator referenced this issue

2026-07-10 00:12:44 -03:00

feat(0.27.1): stance guard — ideologically antagonistic asks never share a cluster #11

navigator commented

2026-07-10 00:13:12 -03:00

Author

Owner

0.27.1 (PR #11) ships the precision-first floor for this: an auditable direction lexicon (stance.rs) + per-proposal direction_signature (0517) + merge veto in ingest. Measured motivation added to the calibration tests: negation pairs at cosine 0.015 and opposite budget directions at 0.046 — both below every legitimate paraphrase, so NO embedding threshold can separate stance.

Remaining for V2 (this issue): the lexicon is deliberately narrow — it cannot infer that aumentar salários dos radiologistas do SUS implies strengthening the public system (that pair is currently protected only by the threshold, at 0.107). The NLI/mDeBERTa path (or Aposta C, IA soberana como escrivã) covers the semantic remainder; the veto plumbing (signature column + ingest hook) is ready to receive a model-based contradiction check as a second signal.

0.27.1 (PR #11) ships the **precision-first floor** for this: an auditable direction lexicon (stance.rs) + per-proposal `direction_signature` (0517) + merge veto in ingest. Measured motivation added to the calibration tests: negation pairs at cosine 0.015 and opposite budget directions at 0.046 — both below every legitimate paraphrase, so NO embedding threshold can separate stance. Remaining for V2 (this issue): the lexicon is deliberately narrow — it cannot infer that `aumentar salários dos radiologistas do SUS` implies strengthening the public system (that pair is currently protected only by the threshold, at 0.107). The NLI/mDeBERTa path (or Aposta C, IA soberana como escrivã) covers the semantic remainder; the veto plumbing (signature column + ingest hook) is ready to receive a model-based contradiction check as a second signal.

navigator referenced this issue

2026-07-10 01:08:39 -03:00

feat(0.28.0): NLI pair judge — merges decided by meaning, not word overlap #12