v4.49 — Storybook Component Evidence Capture

A PR can now request scripted light + dark stills of a named Storybook story with one PR-body line — a component state driven by props rather than by reachable backend data no longer needs a human to hand-build Storybook and drive Playwright to see it.

Summary

Before this milestone, preview.yml's capture-ui-evidence job was route-shaped end to end: it navigates to a live preview URL and fail-closes on authentication, so it can only ever reach a state a live smoke account can actually get to. A component state that exists only as a prop combination — never reachable from any route, any account, any data seed — had no capture path at all. OBJ-3079 hit this precisely: the RepairConfirmationModal repair-offer states were gated on smoke-account data that was never seeded, so the shipped stories were the only honest place to see the change, and getting a picture of them meant Roy hand-building Storybook, serving the static output, and driving Playwright himself.

Three phases shipped in one calendar day, all directly to master, no integration branch. Phase 1 built and proved scripts/capture-storybook-evidence.mjs by hand, closing the milestone's one open question (the Storybook theme global is forced by the iframe URL's globals param) and measuring the real cost — a ~135–145 second build dominates the bill by two orders of magnitude over the ~1.2 seconds each subsequent capture costs. Phase 2 wired the script into a new, dedicated storybook-evidence.yml CI workflow behind an opt-in UI-Evidence-Stories: PR-body directive, deliberately not a required check and deliberately not joined to the sibling v4.46 milestone's coverage-gated Storybook test suite. Phase 3 is where the capability met reality: Roy's review found the new workflow had no way to re-run itself after a directive was added to an already-open PR, and the scratch-PR exercise built to prove the fix hit a second, more serious problem — every real-story capture failed, because serve's default URL-cleaning behavior was silently stripping the query parameters that select a story and force its theme, so every capture had been rendering Storybook's blank placeholder rather than the component. The pipeline's own fail-loud design (assertModesDiffer, built for an unrelated cold-start flake) is what caught it instead of shipping a silent false pass. Once fixed, one forced CI run produced the milestone's actual proof: four real screenshots of RepairConfirmationModal's stories, light and dark, genuinely different renders — the exact OBJ-3079 case this milestone exists to solve.

Goal

Give a PR a scripted, CI-run way to produce light + dark stills of a named Storybook story, so a component state that is driven by props rather than by reachable backend/account data can still be evidenced against the real shipped component — without a human hand-building Storybook and driving Playwright by hand.

Scope — What Shipped

• scripts/capture-storybook-evidence.mjs (Phase 1) — takes --stories, --base-url, --modes, --viewport, --out; validates requested story ids against the built Storybook index and fails loudly on an unknown id, before any browser launch; assertModesDiffer() refuses to write a light render under a dark filename.
• .github/workflows/storybook-evidence.yml (Phase 2, extended Phase 3) — a new, dedicated workflow (not a preview.yml job, since Storybook capture touches no live preview, no smoke account, no backend). Opt-in only via an own-line UI-Evidence-Stories: directive; job-level continue-on-error: true; not added to master CI gate's required contexts; workflow_dispatch with a pr_number input (Phase 3 FIX-1) so a directive added to an already-open PR body has a manual trigger path.
• Cold-start mitigation (captureWithColdStartRetry, Phase 2 CI-6) — re-screenshots and compares only the first capture of a run, closing a reproducible cold-Chromium blank-first-capture flake without masking a genuinely blank story.
• Regex guard triple (Phase 2 GUARD-1) — scripts/check-storybook-evidence-stories-regex.mjs + .test.mjs + storybook-evidence-stories-regex-guard.yml, following the established check-ui-evidence-routes-regex pattern, including injection-shaped negative cases.
• Comment-branch symmetry fix (Phase 3 FIX-2) — a syntactically invalid story id now posts the same honest "capture failed" PR comment an unknown-but-valid-shaped id already got, instead of a bare red X.
• serve cleanUrls fix (OBJ-3127, out-of-tree during Phase 3) — ionic_frontend/.storybook/serve.json with cleanUrls: false, plus a fail-loud assertNoRedirect() and a CI-level curl readiness guard, closing the root cause of every real-story capture failing before this milestone's own live-proof exercise.
• Docs: docs/guides/multica-workspace-skills.md § 8 ui-evidence-and-critique, scripts/README.md, docs/development/gotchas.md, CHANGELOG.md.

Phases

Phase	Name	Status	Plans	Highlights
1	Capture script + measured cost	Shipped	1	Built and proved capture-storybook-evidence.mjs by hand, zero CI changes. Closed the theme-forcing open question and measured the real per-story cost that sized every later phase.
2	CI wiring + UI-Evidence-Stories directive + regex guard	Shipped	1	Wired the script into a new, opt-in, non-required CI workflow. Verified structurally rather than live — the workflow-only PR's own path filter kept the job from firing on itself.
3	Trigger fix + docs + skill mirror	Shipped	2	Added the missing manual-dispatch path (FIX-1) and fixed a comment-branch asymmetry (FIX-2); the resulting live-proof exercise surfaced and closed the milestone's one real defect (OBJ-3127); documented the capability against master as merged.

Key Decisions

• A new, dedicated workflow, not a preview.yml job — Storybook capture touches no live preview deploy and no shared smoke account, so hosting it inside preview.yml would have coupled it to that job's deploy-preview and smoke-serialization dependencies for no reason.
• Opt-in only, no default story set — 434 story files × 2 themes is not a per-PR default anyone would want to pay for, and a diff-derived default is worse: a shared primitive touching hundreds of stories transitively, with a story-id-to-file mapping only reliable after a build. Silence is the normal case.
• All three filename tokens kept (<story-id>--<mode>--<viewport>.png) — stays byte-identical to the existing evidence convention; a story id's own embedded -- collapses harmlessly under check-png-legibility.mjs's tokenizer, proven by reading the tokenizer rather than assumed.
• Fail loud on an unknown story id — there is no default to fall back to, so a silent partial capture would produce an artifact that looks complete when it isn't.
• The cold-start retry is scoped to the first capture of a run only, not a blanket warm-up navigation — a warm-up pays its cost on every run whether or not the flake occurs; the scoped retry pays only on the rare run that actually hits it, and still fails loudly if a genuinely blank story doesn't recover on retry.

Requirements Coverage

5 / 5 success criteria satisfied, 20 / 20 phase-level acceptance criteria satisfied (see .planning/milestones/v4.49-storybook-component-evidence-MILESTONE-AUDIT.md).

Success criterion	Status
A PR can request light + dark stills with one PR-body line	✅
The OBJ-3079 case reproduces end-to-end from CI, no human-built Storybook	✅
A malformed/mid-paragraph/unknown directive is loud, never silent	✅
No frontend PR pays for a Storybook build it didn't ask for	✅
Documented where the crew actually reads it (skill 8)	✅

Outcomes

A prop-driven component state with no live route and no reachable smoke-account data now has a scripted evidence path: add UI-Evidence-Stories: <id>,<id> to a PR body (or dispatch it manually against an already-open PR) and get back light + dark screenshots, a legibility check, and an honest PR comment — captured, capture-ok-legibility-failed, capture-ok-upload-failed, or capture-failed, never a silent pass. No frontend PR pays the ~135–145 second Storybook build cost unless it explicitly asks for it, and the job cannot become a hidden merge gate — it isn't in master CI gate's required contexts and doesn't share v4.46's Vitest project. The milestone's own build process caught a real defect (OBJ-3127) before it ever shipped silently: the capability's fail-loud guard refused to report success on a light render captured under a dark filename, which is precisely the trustworthy-failure posture this milestone exists to establish.

Tech Debt

• (Milestone-wide) The capability's end-to-end happy path is proven by one real CI run (33489539814), not a repeated one — the same honest-ceiling caution the v4.45 audit recorded for its own single-test reconnect proof.
• (Phase 3 / OBJ-3130) A documentation hedge in docs/development/gotchas.md was correct when written (OBJ-3127's fix PR was still open) and went stale 15 minutes later when the PR merged; corrected same-day via a follow-up PR. No open action remains.

Related Artifacts

• Roadmap: v4.49-storybook-component-evidence-ROADMAP.md
• Milestone Audit: v4.49-storybook-component-evidence-MILESTONE-AUDIT.md
• Source issue: OBJ-3112 — "Add scripted Storybook-capture CI step for isolated component UI evidence," filed by Maggie off Roy's finding while reviewing OBJ-3079 / PR #2736
• Coordination anchor: OBJ-3113 (parent milestone issue)
• Follow-up defect, filed and closed within the milestone: OBJ-3127 (serve cleanUrls redirect defect), OBJ-3130 (gotchas.md hedge correction)
• Sibling, non-colliding: v4.46 Storybook play() CI Enforcement — different runner, different output, different trigger; shares only the story corpus

Related Commits

• 52cfdb0599 — Phase 1: Capture script + measured cost (PR #2747)
• 6054e37c9 — Phase 2: CI wiring + UI-Evidence-Stories directive + regex guard (PR #2756)
• e9bbf1bf5 — Phase 3: workflow_dispatch trigger + invalid-id comment fix (PR #2759)
• 406bfcfff — OBJ-3127: fix serve's cleanUrls redirect (PR #2761)
• c21e8adc5 — Phase 3: docs (PR #2763)
• 54925289 — OBJ-3130: correct the gotchas.md hedge (PR #2764)

---

Last updated: 2026-09-01