Objectuve Docs

Appearance

PRD Addendum: Adopting Multica-Inspired Capabilities into AI Workforce ​

Status: Draft / Design Author: Josh Lockhart Date: 2026-04-23 Parent PRD: Multica AI Workforce EvaluationDecision owner: Josh Lockhart Target decision: Accept / Reject / Defer Related systems: agent_runner/, rails_api/app/interactions/ai_workforce/, rails_api/app/jobs/ai_workforce/, admin_dashboard/src/views/

1. Context ​

The parent PRD recommended against migrating the AI Workforce to self-hosted Multica, citing load-bearing custom features (brand-voice post-filter, per-employee MCP scoping, memory injection, autonomy tiers, budget caps, approval queue) that have no documented Multica equivalents.

But the parent PRD also identified several Multica concepts that genuinely do look like the future for AI Workforce: issue-driven (event-reactive) triggers, multi-agent crew handoffs, real-time streaming, and agents-as-teammates operator UX. Those are valuable independent of whether the underlying engine is Multica or our own agent_runner/.

This addendum asks the complement question: What would it take to build the best Multica ideas into our existing AI Workforce stack, so we get the operator UX and crew patterns without giving up the load-bearing custom features?

2. Problem Statement ​

Today's AI Workforce is architecturally a single shape:

Crono schedule → TriggerRun → Agent Runner → one artifact → review → deliver

That works for periodic content (Gym's release notes, Stat's PostHog report). It doesn't compose for three classes of work we increasingly want:

  1. Reactive work — a new Sentry issue should wake Scout, not wait until Tuesday's cron.
  2. Multi-step work — "Scout triages the issue → Cori drafts community update → Brio audits voice → Gym publishes to Slack" is four employees with handoffs. Today it'd be four disconnected cron runs.
  3. Long-running work with human checkpoints — "Orion scopes this → human approves scope → Codi implements → Roy reviews → human approves → Vicki ships" has deliberate gates. Today our only gate is post-artifact approval.

Multica models all three naturally via issues + status machine + handoff protocol. We don't. We could.

Question: Which Multica patterns are worth building into our stack, and what do they cost?

3. Feature-by-Feature Assessment ​

Multica capabilities evaluated for in-project adoption, ranked by value:

Multica FeatureValueEffortRecommendation
Event-reactive triggers (webhooks, not just cron)High2 weeksAdopt (Phase 1)
Multi-agent crew workflows (handoffs, state machine)High4 weeksAdopt (Phase 2)
Real-time WebSocket run streamingMedium1 weekAdopt (Phase 3)
Richer agent-profile operator UXMedium1 weekAdopt (Phase 3)
Enriched status state machine (blocked, in_review)Low-Med3 daysAdopt (Phase 3)
Multi-workspace isolationLow3–4 weeksSkip (we're single-tenant)
CLI for agent handoffs (multica issue …)Low2 weeksSkip (Rails handles orchestration)
Unified local daemon + cloud runtimeLow1 weekSkip (Docker compose covers dev)
Reusable skills library——Already done (.claude/skills/)
Vendor neutrality (non-Anthropic providers)Low2 weeksSkip (LiteLLM already abstracts this)

Total adopt budget: ~8 weeks engineering across 3 phases.

4. Deep-Dive: Event-Reactive Triggers (Phase 1) ​

What we have ​

  • ScheduleRunsJob runs in Crono, finds due_for_run employees via schedule_cron field, dispatches TriggerRun.
  • AiRun.triggered_by enum: schedule, manual, seed.

What we'd build ​

Add a fourth trigger type: event. Event sources emit a signal → a rule matches → a run is created.

New models:

  • AiTrigger — belongs to AiEmployee. Fields: event_source (enum: sentry_issue_created, github_pr_merged, github_issue_opened, posthog_funnel_dropped, manual_webhook), condition (JSONB — e.g., { "level": "error", "project": "enkidu-api" }), enabled, cooldown_minutes.
  • Optional: AiTriggerEvent — audit log of fires (for debugging "why did this not fire?").

New endpoints:

  • POST /webhooks/ai_workforce/events — generic event ingest with HMAC auth. Body: { source, payload }. Already have HMAC validation pattern from webhooks/ai_workforce_controller.rb.
  • Per-source adapters: Sentry already has internal events, GitHub webhooks already wired for other features. Both reuse existing signing secrets.

New interaction:

  • AiWorkforce::EvaluateEvent — takes source + payload, finds matching AiTrigger records, applies condition, enforces cooldown_minutes (reject if any run for that trigger finished within cooldown window), calls TriggerRun per match.

Admin UI:

  • Add "Triggers" tab to EmployeeDetailView.vue — list triggers with enable/disable, source dropdown, condition editor (JSON textarea initially; richer UI later).

Why this is worth doing ​

Roughly 60% of the AI Workforce runs we actually want aren't "weekly" — they're "when X happens." Right now we fake it with generous cron (daily Scout run that scans the last 24h of Sentry) which is wasteful and delayed. Event triggers collapse wasted runs, reduce latency from "up to 24h" to "near-real-time," and let new work patterns emerge (e.g., "Gym summarizes every merged PR within the hour").

Risk / open question ​

  • Cooldown semantics: do we want "global cooldown per employee" or "per-condition cooldown"? Per-condition is more correct but requires normalizing the condition JSON for dedup.
  • Reuse of existing webhooks: our webhooks/ai_workforce_controller.rb currently only handles Agent Runner callbacks. Adding inbound event webhooks doubles the controller's responsibility — probably want a new controller (webhooks/ai_workforce_events_controller.rb) with a distinct signing secret.

5. Deep-Dive: Multi-Agent Crew Workflows (Phase 2) ​

What we have ​

  • One AiRun = one employee = one artifact. No handoffs.
  • The closest thing to a crew is the brand-voice post-filter, which is a second query() call inside the same run — not a separate employee.

What we'd build ​

Introduce a new aggregate: AiWorkflow. A workflow is an ordered sequence of steps, each assigned to an employee, with context carried between steps.

New models:

  • AiWorkflow — name, description, steps (JSONB ordered array: [{employee_id, step_name, output_kind, on_success, on_failure}]), trigger_config (mirrors AiTrigger so a workflow can be event-driven too), enabled.
  • AiWorkflowRun — workflow_id, status (enum: queued, running, blocked_for_review, succeeded, failed, cancelled), current_step_index, context (JSONB — shared state passed between steps), started_at, finished_at.
  • AiWorkflowStepRun — workflow_run_id, step_index, ai_run_id (FK to the existing AiRun), step_name, status. Each step's execution is still an AiRun — we don't replace that primitive, we sequence it.

New interactions:

  • AiWorkforce::StartWorkflow — validates trigger, creates AiWorkflowRun, dispatches first step via TriggerRun with workflow_run_id context param.
  • AiWorkforce::AdvanceWorkflow — called after a step's AiRun completes. If approval_required on the step, transitions AiWorkflowRun.status to blocked_for_review. Otherwise dispatches next step with accumulated context.
  • AiWorkforce::ResumeWorkflow — called when a blocking approval resolves. Advances to next step.

Context passing:

  • Each step's AiRun receives workflow_context in its system prompt (appended to memory). Each step's artifact payload is merged back into AiWorkflowRun.context so subsequent steps see it.
  • Cap context size the same way we cap memory (~2k tokens). Add a context_summary field that an intermediate step can write to keep context lean across long workflows.

Brand-voice post-filter as a workflow step:

  • This is a consolidation win. Today postFilter.ts runs inside the Agent Runner as a second query(). Under workflows, "brand-voice review" becomes a dedicated step assigned to Brio (or a new "Voice Reviewer" employee). Step fails the workflow if Brio rejects. Agent Runner's postFilter.ts stays for backward-compat on non-workflow runs, but new employees prefer the workflow approach.
  • Net result: the post-filter becomes visible in the run log, composable across employees, and cheaper to iterate on (change Brio's prompt, not Agent Runner code).

Admin UI:

  • New view: admin_dashboard/src/views/WorkflowsView.vue — list of workflows, per-workflow run history, trigger status.
  • New view: WorkflowRunDetailView.vue — step timeline, context evolution, per-step RunDetailView drill-down.
  • Review Queue becomes workflow-aware: blocked_for_review rows show "Approve to advance [workflow name] → step [n+1]" instead of just "Approve artifact."

Why this is worth doing ​

Workflows unlock the three work patterns Section 2 called out: reactive triage, multi-employee pipelines, and human-checkpointed long-running work. They also consolidate the brand-voice post-filter as a first-class step rather than a hidden second query() call.

Without workflows, every new composition (e.g., "Scout triages → Cori drafts → Brio audits → Gym publishes") is a new custom job. With workflows, it's a config row.

Risk / open question ​

  • Four weeks is optimistic. Edge cases that eat time: retry policies per step, compensating actions when a mid-workflow step fails, context schema evolution across workflow versions, approval timeout (what if the human never reviews a blocked step?).
  • Do workflows own their own budget (monthly_budget_cents on AiWorkflow) or inherit from employees? Recommend: workflows have their own budget, employees still enforce their budget independently, either exhausted budget halts the run.
  • Should workflows support branching (on_success goes to step N, on_failure goes to step M)? MVP: linear only. Branching can ship in a follow-up.

6. Deep-Dive: Real-Time Streaming + Richer Profiles (Phase 3) ​

Real-time WebSocket streaming ​

What we have: 30-second polling in AIWorkforceView.vue and RunDetailView.vue. Live runs don't feel live.

What we'd build:

  • ActionCable channel: AiRunChannel — subscribes to a run ID, streams tool calls + status as they happen.
  • Agent Runner change: instead of one webhook at end-of-run, optionally emit per-turn events (POST /webhooks/ai_workforce/progress). Rails broadcasts via ActionCable to the subscribed channel.
  • Frontend: replace polling in RunDetailView.vue with ActionCable subscription. Add live "thinking..." and "calling tool: github.createPullRequest" indicators.

Effort: 1 week. Rails already runs ActionCable on /cable; no new infra.

Why worth it: operator UX. Watching a run live changes the felt experience from "batch process that emails me later" to "teammate working now." It also catches problems faster — if a run is stuck in a tool-call loop, the operator sees it at minute 2 instead of minute 30.

Richer agent profiles ​

What we have: EmployeeDetailView.vue has tabs (Overview / Runs / Memory) and an approval-rate chart.

What we'd build:

  • "Activity feed" timeline: chronological stream of runs, artifacts produced, artifacts approved/rejected, memory updates, trigger fires. Gives a one-glance sense of "what has Scout been up to this week?"
  • "Bio" field on AiEmployee — 1–2 sentences in the employee's voice, shown on their profile. Small but changes the felt relationship with the employee.
  • Skills list: we already store skill_refs on the employee; expose it visually with links to the rendered skill docs in Storybook or .claude/skills/{name}/SKILL.md.
  • Capability badges: "Can create GitHub PRs," "Reads Sentry issues," "Writes to Slack #releases" — computed from mcp_servers + REST tool config.

Effort: 1 week.

Why worth it: cheap UX win that reinforces the "employee" framing. Makes autonomy promotion decisions feel more like personnel decisions than database edits.

Enriched status state machine ​

What we have: AiRun.status enum: queued, running, succeeded, failed, cancelled.

What we'd build:

  • Add blocked_for_review status — a run whose artifact is pending approval. Today that state is implicit (artifact approval_status: pending + run status: succeeded). Making it explicit on the run simplifies "how many runs need my attention?" queries.
  • Add blocked_for_workflow status — a workflow step run waiting for the prior step to finish.
  • Surface in admin dashboard as distinct chips/colors.

Effort: 3 days.

7. Phased Rollout ​

Three phases, each deliverable on its own. Total ~8 weeks.

Phase 1 — Event-Reactive Triggers (2 weeks) ​

Shipable independently. High value as soon as first integration (Sentry → Scout) lands.

Deliverables:

  • AiTrigger model + migration
  • EvaluateEvent interaction
  • POST /webhooks/ai_workforce/events controller
  • Sentry + GitHub webhook adapters
  • Triggers tab in EmployeeDetailView.vue
  • One real integration: Sentry critical-level issue → Scout triage run
  • RSpec coverage ≥80% on new models/interactions; Cypress smoke test on admin UI

Go/no-go gate: After Phase 1, the real metric is "how many useful runs are event-triggered vs. cron?" If <20%, we're not solving the reactive-work problem and should reopen this PRD.

Phase 2 — Multi-Agent Crew Workflows (4 weeks) ​

The ambitious one. Start after Phase 1 ships.

Deliverables:

  • AiWorkflow, AiWorkflowRun, AiWorkflowStepRun models + migrations
  • StartWorkflow, AdvanceWorkflow, ResumeWorkflow interactions
  • Context passing + size cap
  • Workflow-aware Review Queue
  • Workflow views in admin dashboard
  • First real workflow: "PR merged → Gym drafts release note → Brio audits voice → post to Slack #releases"
  • Migrate Agent Runner post-filter consumers to workflow-step approach (keep legacy post-filter working for non-workflow runs)
  • RSpec coverage ≥80%; Cypress E2E covering workflow run + mid-flight approval

Go/no-go gate: Does at least one production workflow run 10+ times without operator intervention? If no, reassess the abstraction.

Phase 3 — Streaming + Profile UX (2 weeks) ​

Polish phase. Can be cut or deferred if Phases 1–2 over-run.

Deliverables:

  • AiRunChannel + per-turn progress webhooks
  • ActionCable subscription in admin dashboard
  • Activity feed on EmployeeDetailView.vue
  • Bio field + capability badges
  • blocked_for_review + blocked_for_workflow statuses

8. Explicitly Out of Scope ​

Adopting these would not pay for itself:

  • Multi-workspace isolation. We're a single-tenant operational team; nothing to isolate between. If/when we let customers run their own AI employees, revisit.
  • Multica CLI handoffs. Our orchestration is Rails-side; a CLI would duplicate what AdvanceWorkflow already does.
  • Local daemon runtime. Docker compose gives us local dev parity; a separate daemon is more code to maintain.
  • Vendor-neutral provider layer. LiteLLM already abstracts this. If we ever want to swap Anthropic for another provider, we do it in infra/litellm/config.yaml, not at the Agent Runner layer.
  • Forking Agent Runner into a public OSS project. Tempting, but not aligned with where engineering time is best spent. Revisit after v4.0.

9. Success Criteria ​

This addendum succeeds when:

  • [ ] Phase 1 ships: at least one event trigger is live in production, firing real runs.
  • [ ] Post-Phase 1 measurement: >30% of production runs are event-triggered within 6 weeks of launch.
  • [ ] Phase 2 ships: at least one multi-step workflow runs end-to-end in production with zero ops intervention across 10 consecutive runs.
  • [ ] Post-Phase 2: brand-voice post-filter is consolidated as a workflow step for at least 2 employees (the postFilter.ts path stays functional for backward-compat).
  • [ ] Phase 3 ships: admin dashboard shows live run progress without polling.
  • [ ] Decision reopened for parent Multica Evaluation PRD: with crew workflows in hand, does the migration calculus change? (Likely: no — we'd have the good parts without the migration risk.)

9a. Crew operational health KPIs (ongoing) ​

The criteria above measure whether the capability shipped. These measure whether the crew is actually working day-to-day — the gap the 2026-05 workflow audit surfaced (there was no quantitative answer to "is the crew healthy?"). Published weekly by the Maggie — Weekly crew metrics autopilot; reviewed monthly.

KPIDefinitionTargetWhy it matters
Zero-human-follow-up rate% of agent-opened PRs that merge with no commit authored by a human after open≥ 70%The headline audit finding: 40% of human commits were finishing agent PRs. This is the single best proxy for "the chain ships without me."
Median rounds-to-convergeMedian # of fix-rounds per ticket before it ships or is escalated≤ 2OBJ-706 hit 40. The R3 cap + Fix-round cap sweep should hold this at ≤2; a rising median means the cap is being worked around.
Human-rescue rate% of PRs where the head-branch agent ≠ the final-commit author (i.e. a human or another agent had to take over)≤ 20%Detects mis-delegation that the per-PR [Agent] title now makes measurable.
Autopilot-issue no-reassign rate% of autopilot-created issues closed/shipped without a human re-routing them≥ 80%Measures whether routing (Maggie + proposer→Roy gate) is landing work on the right agent first time.
Smoke/CI PR share% of merged PRs that touch tests/smoke-playwright/ or .github/workflows/≤ 10% (from 18.6%)The orphan-surface cost. Tess ownership + the R3 cap should compress this; a flat share means the structural fix didn't take.

Instrumentation note: these are uncomputable from gh pr list alone because every PR shows the same GitHub author — they depend on the [Agent] PR-title prefix and multica-agent:<name> label introduced for Riley. Until ≥2 weeks of labelled PRs exist, report "insufficient data" rather than a misleading number.

10. Open Questions ​

  1. Trigger cooldown granularity — per-employee, per-trigger, or per-condition? (Recommend per-trigger for v1.)
  2. Workflow branching — linear-only in MVP, or include on_success / on_failure routing day one? (Recommend linear-only, branching in follow-up.)
  3. Workflow budget model — workflow-owned, employee-owned, or additive? (Recommend workflow-owned with employee cap as a ceiling.)
  4. Approval timeout — what happens if a blocked_for_review step sits for 7 days? Auto-reject? Escalate? Notify? (Recommend: auto-reject + Slack alert after configurable TTL, default 72h.)
  5. Crew naming — when a workflow runs, do operators see "Scout (step 1) → Cori (step 2)" or a crew-level identity ("Release Note Crew")? (Recommend: surface the workflow name as primary, employees as secondary.)
  6. Memory semantics in workflows — does each step append to the employee's persistent memory, or only to the workflow context? (Recommend: employee memory continues to accumulate per-step — it's their memory, regardless of what workflow invoked it.)
  7. Parent PRD revisit — after Phases 1–2 ship, we should re-read the original Multica evaluation. Capabilities we built ourselves may have closed the gap.

11. Non-goals ​

  • Not proposing to migrate any existing behavior to Multica (parent PRD covered that).
  • Not proposing to change the 5-employee roster, autonomy tiers, output kinds, or delivery destinations.
  • Not proposing to replace agent_runner/ as the execution engine.
  • Not proposing to replace LiteLLM, PostgreSQL 15, Crono, or Sidekiq.
  • Not proposing to expose AI Workforce capabilities to end-users (customer-facing). Operator-facing only.

12. Verification ​

  • [ ] Josh reviews and accepts / rejects / requests revisions
  • [ ] If accepted: Phase 1 plan (PLAN.md under .planning/) is written before engineering starts
  • [ ] Phase gates documented in ../features/ai-workforce.md
  • [ ] Rollback plan for each phase documented before deploy: Phase 1 rollback = disable triggers flag; Phase 2 rollback = disable workflows flag; Phase 3 rollback = fall back to polling

13. Related artifacts ​

Last updated: 2026-05-29 (added §9a crew operational health KPIs — 2026-05 workflow audit)

Loading…