Fluid/fluidpop-v1
1
0
Fork 0
Code Issues 17 Pull requests Projects Releases Packages Wiki Activity Actions

Draft docs/benchmarks/report-v1.md head-to-head benchmarks skeleton #119

New issue
Open
opened 2026-05-27 09:07:13 -03:00 by navigator · 3 comments
navigator commented 2026-05-27 09:07:13 -03:00
Owner
Copy link

Goal

Create the head-to-head benchmark report skeleton at docs/benchmarks/report-v1.md, listed as deliverable #5 in PLAN.md Section 10. The doc structures Phase 1 simulation-based performance comparisons (baseline Gemmini vs single-chip FluidPopSoC vs 8-chip board) so benchmark work has a canonical landing spot.

Sections to include:

  • Overview (scope: simulation-based Phase 1 numbers only; no silicon results)
  • Methodology (Verilator sim clock ~10 kHz per Section 8.3; cycle accounting; what counts as 'one inference')
  • Baselines (upstream GemminiRocketConfig from Phase 2 — reference baseline-gemmini-rocket.md)
  • Single-chip FluidPopSoC results (placeholder table; expected 2.5-3.5x speedup per Section 8.3)
  • 8-chip board results (placeholder table; expected 4-6x speedup over single chip per Section 8.3)
  • Workloads (ResNet-50 single-chip; Llama-7B 4-bit 8-chip partitioned per Section 8.3)
  • Caveats (sim-only; honest reporting per Section 9.3; analytic model vs measured per Section 14.6)
  • Open questions (which secondary metrics — power, BW utilization, scratchpad hit rate)

Acceptance criteria

  • docs/benchmarks/report-v1.md exists with the sections above
  • Status: Draft skeleton + Owner: TBD header
  • Each section has TODO markers and 2-3 line intent description
  • Placeholder tables present with header rows; all data cells TBD
  • Explicitly states sim-only scope and references Section 9.3 honest-reporting language
  • References Sections 7 (baseline), 8.3 (acceptance targets), 9.3 (honest reporting), 14.6 (analytic models)
  • docs/benchmarks/README.md created or updated to list this deliverable

Plan refs

Section 10 (deliverable #5), Section 7 (baseline benchmark), Section 8.3 (testbench plan + acceptance criteria), Section 9.3 (honest reporting), Section 14.6 (analytic performance models)

Notes

Skeleton only — actual numbers come from Phases 2-3 simulation runs. Do not fabricate any benchmark numbers; tables stay empty until measured.

## Goal Create the head-to-head benchmark report skeleton at `docs/benchmarks/report-v1.md`, listed as deliverable #5 in PLAN.md Section 10. The doc structures Phase 1 simulation-based performance comparisons (baseline Gemmini vs single-chip FluidPopSoC vs 8-chip board) so benchmark work has a canonical landing spot. Sections to include: - Overview (scope: simulation-based Phase 1 numbers only; no silicon results) - Methodology (Verilator sim clock ~10 kHz per Section 8.3; cycle accounting; what counts as 'one inference') - Baselines (upstream `GemminiRocketConfig` from Phase 2 — reference `baseline-gemmini-rocket.md`) - Single-chip FluidPopSoC results (placeholder table; expected 2.5-3.5x speedup per Section 8.3) - 8-chip board results (placeholder table; expected 4-6x speedup over single chip per Section 8.3) - Workloads (ResNet-50 single-chip; Llama-7B 4-bit 8-chip partitioned per Section 8.3) - Caveats (sim-only; honest reporting per Section 9.3; analytic model vs measured per Section 14.6) - Open questions (which secondary metrics — power, BW utilization, scratchpad hit rate) ## Acceptance criteria - [ ] `docs/benchmarks/report-v1.md` exists with the sections above - [ ] `Status: Draft skeleton` + `Owner: TBD` header - [ ] Each section has `TODO` markers and 2-3 line intent description - [ ] Placeholder tables present with header rows; all data cells TBD - [ ] Explicitly states sim-only scope and references Section 9.3 honest-reporting language - [ ] References Sections 7 (baseline), 8.3 (acceptance targets), 9.3 (honest reporting), 14.6 (analytic models) - [ ] `docs/benchmarks/README.md` created or updated to list this deliverable ## Plan refs Section 10 (deliverable #5), Section 7 (baseline benchmark), Section 8.3 (testbench plan + acceptance criteria), Section 9.3 (honest reporting), Section 14.6 (analytic performance models) ## Notes Skeleton only — actual numbers come from Phases 2-3 simulation runs. Do not fabricate any benchmark numbers; tables stay empty until measured.
navigator commented 2026-05-27 09:10:55 -03:00
Author
Owner
Copy link
No description provided.
<!-- agent:claim by=dispatcher run=20260527T121054Z_issue119 ts=1779883855 -->
navigator commented 2026-05-27 09:11:50 -03:00
Author
Owner
Copy link

Every acceptance criterion in this issue is already satisfied by docs/benchmarks/report-v1.md (landed in PR #90, commit b7962f3) and docs/benchmarks/README.md (landed alongside the baseline skeleton in PR #101, commit a0047d3).

Verification against issue #119 acceptance criteria:

  • docs/benchmarks/report-v1.md exists with all 8 sections (Overview, Methodology, Baselines, Single-chip FluidPopSoC, 8-chip board, Workloads, Caveats, Open questions).
  • Status: Draft skeleton (line 6) + Owner: TBD (line 7) header.
  • Each section opens with a TODO: marker and 2–3 lines of intent.
  • Placeholder tables present with header rows; every data cell is TBD (sections 3, 4, 5).
  • Sim-only scope stated explicitly (lines 22–25) and PLAN.md Section 9.3 honest-reporting language cited (lines 19–20, 25, 112–113).
  • References to PLAN.md Sections 7, 8.3, 9.3, 14.6 all present in the header block (lines 8–13) and reinforced in body sections.
  • docs/benchmarks/README.md lists report-v1.md as deliverable #5 with Plan ref + Status.

No new code or doc change is required. Recommend closing this issue as a duplicate of work already completed in #90 / #101.

<!-- agent:fail reason=blocker:already-resolved --> Every acceptance criterion in this issue is already satisfied by `docs/benchmarks/report-v1.md` (landed in PR #90, commit `b7962f3`) and `docs/benchmarks/README.md` (landed alongside the baseline skeleton in PR #101, commit `a0047d3`). Verification against issue #119 acceptance criteria: - [x] `docs/benchmarks/report-v1.md` exists with all 8 sections (Overview, Methodology, Baselines, Single-chip FluidPopSoC, 8-chip board, Workloads, Caveats, Open questions). - [x] `Status: Draft skeleton` (line 6) + `Owner: TBD` (line 7) header. - [x] Each section opens with a `TODO:` marker and 2–3 lines of intent. - [x] Placeholder tables present with header rows; every data cell is `TBD` (sections 3, 4, 5). - [x] Sim-only scope stated explicitly (lines 22–25) and PLAN.md Section 9.3 honest-reporting language cited (lines 19–20, 25, 112–113). - [x] References to PLAN.md Sections 7, 8.3, 9.3, 14.6 all present in the header block (lines 8–13) and reinforced in body sections. - [x] `docs/benchmarks/README.md` lists `report-v1.md` as deliverable #5 with Plan ref + Status. No new code or doc change is required. Recommend closing this issue as a duplicate of work already completed in #90 / #101.
navigator commented 2026-05-27 09:16:29 -03:00
Author
Owner
Copy link
No description provided.
<!-- agent:fail reason=no-pr-marker run=20260527T121054Z_issue119 -->
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
auto/issue-112-20260526T211219Z_issue112
auto/issue-113-20260526T211252Z_issue113
auto/issue-104-20260526T163428Z_issue104
auto/issue-105-20260526T163459Z_issue105
auto/issue-51-20260526T163356Z_issue51
auto/issue-52-20260526T001333Z_issue52
auto/issue-53-20260526T001906Z_issue53
auto/issue-98-20260526T120712Z_issue98
auto/issue-99-20260526T120744Z_issue99
auto/issue-100-20260526T120815Z_issue100
spec/popsocconfig-draft
spec/poproccrouter-draft
spec/poplinkphy_behavioral-draft
spec/pciehostbridge-draft
spec/multigemminicluster-draft
spec/intergemminixbar-draft
spec/interchipfabric-draft
spec/fluidpopsoc-draft
auto/issue-84-20260526T040021Z_issue84
auto/issue-86-20260526T040124Z_issue86
auto/issue-85-20260526T040052Z_issue85
auto/issue-73-20260525T225502Z_issue73
feat/spec-designer-role
auto/issue-54-20260526T001939Z_issue54
auto/issue-47-20260526T001302Z_issue47
fix/pr-reviewer-label-detect
auto/issue-72-20260525T224929Z_issue72
auto/issue-71-20260525T224356Z_issue71
fix/pr-approver-gate-on-review
auto/issue-68-20260525T205321Z_issue68
feat/pr-reviewer-role
auto/issue-58-20260525T203246Z_issue58
auto/issue-51-20260525T202210Z_issue51
auto/issue-50-20260525T201137Z_issue50
auto/issue-49-20260525T200104Z_issue49
auto/issue-48-20260525T195031Z_issue48
fix/auto-merge-rebase-refspec
auto/issue-57-20260525T192021Z_issue57
auto/issue-56-20260525T191448Z_issue56
auto/issue-46-20260525T174915Z_issue46
auto/issue-6-20260525T173842Z_issue6
fix/auto-merge-result-return
fix/auto-merge-rebase-behind-base
auto/issue-40-20260525T121803Z_issue40
auto/issue-39-20260525T121230Z_issue39
auto/issue-38-20260525T120657Z_issue38
auto/issue-32-20260524T183617Z_issue32
auto/issue-31-20260524T183039Z_issue31
feature/hotfix-approver-reap
auto/issue-30-20260524T182505Z_issue30
feature/hotfix-cache-invalidation
feature/hotfix-swarm-labels
feature/adr-017-accepted
feature/adr-017-swarm-security
feature/swarm-approver-install
feature/swarm-resolver-dispatcher-on-25
feature/swarm-lib-operator-opener
feature/bridge-stream-json-logs
feature/gitignore-claude-workspace
feature/bridge-resilience-fix
feature/telegram-claude-bridge
feature/telegram-notify
feature/cfd-case-template
feature/runtime-driver-spec
feature/phy-outreach-letters
feature/risk-dashboard
feature/auto-merge-helper
feature/initial-adrs
feature/forgejo-bot-reviewer
feature/ci-runner-setup
No results found.
Labels
Clear labels   
adr

Architecture Decision Record

  
agent:blocked-ci

CI failed on the PR

  
agent:blocked-human

Operator escalated; awaiting Marcos input

  
agent:blocked-resolver

Resolver crashed, returned empty, or timed out

  
agent:done

PR merged; issue auto-closed

  
agent:in-progress

Resolver claimed and running

  
agent:no-touch

Human-reserved; swarm Opener/Dispatcher skip

  
agent:pinged

Operator already escalated; suppress repeat pings

  
agent:pr-open

Resolver opened a PR; awaiting review/merge

  
agent:queued

Issue ready for swarm Resolver to claim

  
agent:wip

PR work-in-progress; Approver skips

  
area:board

PCB / mechanical / cooling

  
area:funding

PACID / grants / RFQs

  
area:infra

Dev env, CI, containers

  
area:phy

Physical layer / SerDes

  
area:poplink

PopLink protocol

  
area:rtl

Chisel/RTL work

  
area:software

Driver / runtime / frameworks

  
area:supply-chain

Foundry, OSAT, PCB, components

  
area:verification

Test, formal, coverage

  
ci-failed

auto-merge.sh detected CI failure

  
ci-timeout

auto-merge.sh hit CI timeout

  
docs

Documentation

  
do-not-merge

Hard hold; PR Approver skips

  
human-approved

Marcos approved via Telegram; release governance hold

  
needs-human-approval

Touches governance path; awaits Marcos label human-approved

  
needs-rebase

PR is not mergeable; rebase needed

  
needs-triage

Issue from untrusted author; Marcos triage needed

  
phase:1

Phase 1 (Section 6 of PLAN.md)

  
ready-for-review

Resolver done; PR Approver triages

  
review:findings

PR semantic review flagged blocking findings

  
review:pass

PR semantic review passed; safe for pr-approver

  
risk:tripwire

Section 0.5 tripwire fired

  
swarm:quarantined

Escalation tripwire fired; resolver tried off-limits paths

No labels
adr
agent:blocked-ci
agent:blocked-human
agent:blocked-resolver
agent:done
agent:in-progress
agent:no-touch
agent:pinged
agent:pr-open
agent:queued
agent:wip
area:board
area:funding
area:infra
area:phy
area:poplink
area:rtl
area:software
area:supply-chain
area:verification
ci-failed
ci-timeout
docs
do-not-merge
human-approved
needs-human-approval
needs-rebase
needs-triage
phase:1
ready-for-review
review:findings
review:pass
risk:tripwire
swarm:quarantined
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Fluid/fluidpop-v1#119
No description provided.