Multica × GitHub auth & rate limits

How the Multica crew authenticates to GitHub, why it currently exhausts the API rate limit, and how to fix it. Companion to ../guides/multica-agent-crew.md and ../guides/multica-autopilots.md.

The problem (observed 2026-05-29)

Every Multica agent runs gh on its runtime as one shared OAuth user token (gho_…, account joshlock3) — the same token your local gh uses. So ~10 agents + your laptop all draw from a single 5,000-requests/hour pool, and it gets exhausted:

$ gh api rate_limit
core   5000/5000 used   remaining = 0   ← maxed out

Symptoms: agents log gh: API rate limit exceeded / gh auth status … rate-limit exceeded and silently exit (the autopilots are written to bail in that case), so PRs stall, CI checks aren't read, and routing backs up.

Bonus consequence: because every action is joshlock3, every PR shows author joshlock3 — the same root cause as the "agent identity invisible at PR level" finding. A dedicated identity fixes both at once.

The limits (GitHub, REST v3)

Bucket	Limit	Used by
REST — user PAT/OAuth	5,000 / hr per user	the crew today (one shared pool)
REST — GitHub App installation	5,000 / hr base, scales to 12,500 / hr (Enterprise Cloud: 15,000)	the recommended upgrade
GraphQL	5,000 points / hr (often far cheaper per call)	underused — prefer it for multi-field reads
Search (REST)	30 / min (1,800 / hr)	stale-PR/branch sweeps, feedback ingest
Code search	10 / min	—
Actions GITHUB_TOKEN	1,000 / hr per repo (separate pool)	CI + every release's workflows
Secondary / "abuse"	~100 concurrent · ~80 content-creates/min, ~500/hr · mutation points	bursty comment/issue/PR creation — trips under 5,000

Check headroom anytime: gh api rate_limit (REST) and gh api graphql -f query='{ rateLimit { remaining resetAt } }' (GraphQL).

Fix #1 (highest leverage) — give the crew its own GitHub identity

Pick one. Both move agents off your personal 5,000/hr pool so your laptop stops competing and the crew gets clean PR attribution.

Option A — Machine-user PAT (fastest)

1. Create a GitHub account for the bot (e.g. objectuve-multica-bot) and add it to the objectuve-softworks org with write access to enkidu (and the multica-agent:* labels already documented in the crew guide).
2. As that bot, create a fine-grained PAT scoped to the enkidu repo with the agent-token permissions in the table below (or a classic PAT — see the classic equivalent).
3. Point each agent's runtime gh at it — see "Wiring the runtime" below.
4. Verify: on a runtime, gh api user returns the bot, and gh api rate_limit shows a fresh 5,000 separate from yours.

Trade-off: a flat 5,000/hr for the whole crew, and PAT rotation is manual (~90d).

Option B — GitHub App (recommended for beta/GA)

A GitHub App installation gets its own pool that scales (5,000 → up to 12,500/hr as the org grows), short-lived auto-rotating tokens, finer permissions, and first-class bot attribution.

1. Org → Settings → Developer settings → GitHub Apps → New. Set the agent-token permissions in the table below. No webhook needed for CLI use.
2. Install it on objectuve-softworks/enkidu. Note the App ID + install ID; download the private key.
3. On each runtime, mint an installation token (the gh token helper or a tiny JWT→installation-token step) and run gh with it. Tokens last 1 hour and auto-refresh, so wire a refresh in the runtime's gh credential helper.
4. Verify with gh api rate_limit — you should see the higher installation ceiling.

Token permissions (least privilege)

Use two tokens — a leaked agent token then can't push to master, and the release token can't merge arbitrary PRs. (Fine-grained PAT and GitHub App use the same permission names.)

Agent (crew) token — repository permissions on enkidu:

Permission	Level	Why
Contents	Read & write	push branches, read compare, read files
Pull requests	Read & write	gh pr create / merge / review, reviewers
Issues	Read & write	manage the multica-agent:* labels + any gh issue use
Workflows	Read & write	required to push changes under .github/workflows/ — Contents:write alone is rejected for workflow files, and Tess/Codi edit them
Actions	Read & write	read run status/logs; gh run rerun --failed
Checks	Read	read CI check results
Commit statuses	Read	combined status on PRs
Dependabot alerts	Read	the Dependabot-triage autopilot (gh api …/dependabot/alerts)
Metadata	Read (mandatory)	auto-required by almost everything, incl. Search

No Administration, Secrets, Environments, or Webhooks — the crew never touches repo settings or branch protection.

Release token (RELEASE_TOKEN):

Permission	Level	Why
Contents	Read & write	commit CHANGELOG + push master + push the v* tag
Administration	Read	required-status-checks-drift.yml reads the master ruleset via GET /repos/{owner}/{repo}/rulesets/{id}
Secrets	Read	required-github-secrets-drift.yml lists provisioned repo secrets via GET /repos/{owner}/{repo}/actions/secrets
Issues	Read & write	both drift workflows file/update/close their own tracking issue on drift
Metadata	Read (mandatory)	auto

release.sh itself only commits and pushes (it doesn't touch workflow files, so it needs no Workflows permission) — the extra scopes above are what the drift-check workflows that also authenticate as this token need.

Classic-PAT equivalent (coarser): agent token = repo + workflow + read:org + security_events (the last so Dependabot-alert reads work — repo alone often isn't enough); release token = repo. Fine-grained is the better choice — the tables above are genuinely least-privilege.

Two config gotchas (not token permissions):

1. Branch protection on master. The release identity must be allowed to bypass the PR requirement to push the CHANGELOG commit + tag — add the bot / App as a bypass actor in the master protection rule, or the push is rejected even with Contents:write.
2. Tag-triggers-deploy. A machine-user PAT push and a GitHub App push do trigger production.yml on the tag (unlike GITHUB_TOKEN) — which is the whole reason RELEASE_TOKEN is a separate, non-default token.

Fix #2 — stop polling CI (no token change needed)

Polling is the silent quota sink. gh run watch fires every few seconds — dozens of calls per PR. The crew guide now bans it for Riley; the pattern is:

• One delayed status read, not a loop: gh pr checks <PR> once after a fixed wait; back off to a longer interval if not done, never a tight loop.
• Better: react to a CI-completion event where wired (the event-reactive triggers in the AI-Workforce addendum apply here too).
• Use GraphQL for "PR + checks + commits" in one query instead of N REST calls (you have ~4,700 GraphQL points free while REST is at 0).

Fix #3 — throttle the autopilots

• Feedback ingest hourly is heavy on the 30/min Search bucket — drop its cadence.
• Stagger sweeps so they don't burst together (same lesson as the Maggie single-slot collision — see autopilots "Concurrency & runtime footguns").
• Honor Retry-After and serialize content creation (don't fan out 20 comments at once) to stay under the secondary limits.

Releases — the separate Actions pool

Releases hit the Actions GITHUB_TOKEN 1,000/hr-per-repo pool, not the agent user token. The authoritative production release gate is the Multica Vicki — Weekly release train autopilot (Wednesdays 08:00 America/Chicago, human go/no-go) — see deployment.md#beta-release-cadence--weekly-release-train. It runs scripts/release.sh directly, pushing with the runtime host's own git credentials — it does not read RELEASE_TOKEN.

.github/workflows/scheduled-release.yml (Manual Aggregate Release) is a separate, workflow_dispatch-only break-glass tool — it no longer runs on a schedule (an earlier weekday cron duplicated the train's job and was removed under OBJ-1368). When manually dispatched:

• It runs scripts/release.sh only if master is ahead of the last tag and ionic_frontend/CHANGELOG.md's [Unreleased] has content.
• It pushes via RELEASE_TOKEN (a secret = the machine-user PAT or App token from Fix #1), not GITHUB_TOKEN — because a tag pushed with GITHUB_TOKEN does not trigger production.yml, and master is protected.
• RELEASE_TOKEN is optional: until it's added (Repo → Settings → Secrets → Actions), a manual dispatch is a safe no-op. Provision it only if you plan to use this break-glass path (and confirm the bot identity can push to protected master) — it is not required for the Wednesday train.

Diagnosing a dead release train (OBJ-1349)

The train ran hard-silent for 9 days (2026-06-30 → 2026-07-08): RELEASE_TOKEN expired/was revoked, and nothing surfaced it.

• Symptom (before the fix below): the Checkout master step failed with fatal: could not read Username for 'https://github.com': terminal prompts disabled (exit 128) — an opaque git auth error, not a named token problem.
• First thing to check when master stops shipping via this workflow: whether RELEASE_TOKEN is still valid. Pre-flight it yourself — curl -s -o /dev/null -w '%{http_code}' -H "Authorization: token $RELEASE_TOKEN" https://api.github.com/repos/objectuve-softworks/enkidu — a non-200 means it's expired or revoked; reissue per Fix #1 above.
• Manual hotfix tags mask a dead train. v3.13.2–v3.13.4 were all cut by hand with scripts/release.sh while the train sat dead underneath — production kept getting occasional tags, so "we shipped recently" is not proof this workflow is healthy. Check its own run history (gh run list --workflow=scheduled-release.yml), not the tag list.
• Nobody was notified. Scheduled-workflow failures don't page anyone by default. OBJ-1349 added three observability steps to .github/workflows/scheduled-release.yml:
  1. An unconditional unreleased-backlog log (last tag + commits-ahead), always written to the run summary regardless of token state.
  2. A pre-flight RELEASE_TOKEN check against the GitHub API, beforeactions/checkout ever touches it — a rejected token now fails fast with a named ::error:: instead of the opaque git message above.
  3. A Slack alert (reusing SLACK_BUDGET_ALERT_WEBHOOK_URL) on both a dormant (missing-token) and a failed (invalid-token) run. The alert is live — SLACK_BUDGET_ALERT_WEBHOOK_URL is provisioned as a GitHub Actions repo secret (OBJ-3593, PR #3155), mirrored from the same value as slack-budget-alert-webhook-url in GCP Secret Manager, so it fires on both failure paths.

Resolved (OBJ-1368): this workflow and the Multica Vicki — Weekly release train autopilot (Wednesdays, human go/no-go via an issue assigned to Josh) were briefly two independent, unreconciled mechanisms both targeting the same v* tag — this workflow's weekday cron predated the autopilot by roughly three weeks and both ran in parallel through at least 2026-06-29. The Wednesday autopilot is now the sole authoritative production release gate; this workflow's schedule trigger was removed and it's dispatch-only break-glass. See deployment.md#beta-release-cadence--weekly-release-train and the Vicki sections of multica-agent-crew.md / multica-autopilots.md for the human-gated model.

Where to put each token

Two tokens → two homes. gh reads GH_TOKEN (and GITHUB_TOKEN) from the environment and it overrides any gh auth login keyring credential — so injecting GH_TOKEN is all it takes to move an agent off joshlock3.

Agent token → the runtime host's gh

gh reads GH_TOKEN / GITHUB_TOKEN from the environment and it overrides any gh auth login keyring credential — so injecting the bot token where the agents run moves the whole crew off joshlock3 in one place.

All 12 agents run on a single runtime here — bf3c31e7. So it's one host, one change. Confirm with multica agent list --output json | jq -r '.[].runtime_id' → multica runtime list.

A runtime is not pinned to a host — confirm which machine currently owns bf3c31e7 before changing anything on it. Runtime identity is keyed on ~/.multica/daemon.id, so a second machine carrying a copied daemon.id silently takes the record over on its next daemon start, and this runtime's displayed name changes with it (it has read Claude (Joshuas-MacBook-Pro-2.local), Claude (Objectuve-Pro.local), and Claude (joshuas-mbp-2.lan) at different times). Configure the wrong host and the change simply never reaches the crew. See Multica runtime identity: the daemon.id hijack.

The per-agent multica agent env set path is NOT available on this server version (PUT /api/agents/<id>/env → 404, even though the CLI has the command). Use the host approach below.

This single-host fact is not just a rate-limit constraint — it's a release-safety risk. See "Release-safety gotcha" below.

On whichever host currently owns the runtime, pick one:

# Option 1 — re-auth the host's gh to the bot (persists in keyring; simplest for a
# dedicated runtime). Restart the daemon so running agents pick it up (this also
# clears any wedged task).
printf '%s' "$BOT_TOKEN" | gh auth login --with-token
gh api user --jq .login        # → objectuve-multica-bot
multica daemon restart

# Option 2 — scope it to the daemon only (leaves your personal gh on that machine):
multica daemon stop
export GH_TOKEN='<bot token>'   # add to the daemon's launchd plist / login profile to persist across reboots
multica daemon start

Verify: GH_TOKEN='<bot>' gh api rate_limit → a fresh, separate pool (a different reset timestamp than your personal gh proves it's a distinct bucket).

Release token → GitHub Actions repository secret

The scheduled-release.yml workflow reads it as secrets.RELEASE_TOKEN — it is not an agent env var, and it is optional: the workflow is workflow_dispatch-only break-glass, not the production gate (that's the Wednesday autopilot, which doesn't use this token at all — see above).

• Repo → Settings → Secrets and variables → Actions → New repository secret → name RELEASE_TOKEN, value = the release token.
• Provision it only if you plan to use the manual break-glass dispatch path (until it exists, a dispatch is a safe no-op); confirm the bot can push to protected master (gotcha #1 above).
• Test with the workflow's workflow_dispatch dry-run input before the first live cut.

Release-safety gotcha: ambient git identity on the shared runtime (OBJ-1473)

The "all 12 agents share one runtime" fact above is a gh rate-limit constraint, but it's also a release-integrity risk, and it caused a real incident: v4.0.3 (tag c8d9186) was committed, annotated-tagged, and pushed directly to master carrying the git identity Desi <desi@objectuve.com> — Desi is the design agent, has no release authority, and that identity doesn't even resolve to a GitHub account. The commit never went through a PR and didn't match scripts/release.sh's deterministic output (wrong files touched, wrong commit message, wrong tag message). Dave's investigation (OBJ-1473) root-caused it to the runtime/identity model documented on this page, compounded by a runtime outage on that same host during Vicki's real release-cut task for Teams GA (OBJ-1429) — see Vicki's ship-confirmation on that issue, which independently confirms she did not cut the tag that run.

The mechanism: git identity on the shared runtime host is ambient (host/session config), not per-task. Any git commit / git push run without an explicit identity override inherits whatever's currently configured on that host — including another agent's leftover identity from its own, separate, legitimate direct-commit workflow on the same shared workdir. With 12 agents timesharing one host, there's no isolation to stop that leakage.

Correction (OBJ-1491): an earlier version of this section claimed scripts/release.sh "never relies on ambient identity." That was wrong — its git tag -a step had no identity override, and the gap reached even a sanctioned release: v4.1.0 was tagged under Desi's leftover identity despite going through scripts/release.sh, not just the ad hoc v4.0.3 bypass (Josh's tag-identity audit, OBJ-1473 comment 7d5e2e43). See "The tag-step gap" below for what was actually wrong and how it was fixed.

Why the release commit step is safe, verified against source:

• scripts/release.sh's release commit is created with an explicit per-commit override: git -c user.name="Objectuve Release Bot" -c user.email="release-bot@objectuve.com" commit -m "chore(release): update CHANGELOG for v$VERSION" (scripts/release.sh:98-101).
• .github/workflows/scheduled-release.yml's break-glass release step likewise pins the identity before committing — git config user.name "Objectuve Release Bot" / git config user.email "release-bot@objectuve.com" (scheduled-release.yml:131-132) — before calling scripts/release.sh on a fresh, single-purpose Actions runner (not the shared crew host), so its tag step inherits the correct identity too.

The tag-step gap (fixed, OBJ-1490): scripts/release.sh's git tag -a call carried no identity override of its own — it fell back to whatever ambient identity was configured on the shared crew runtime at that moment, exactly the leak the commit-step override exists to prevent. This is the confirmed mechanism behind v4.1.0 tagging as Desi even on the sanctioned release path (and it compounded the ad hoc v4.0.3 bypass above). Fixed by carrying the identical override onto the tag step: git -c user.name="Objectuve Release Bot" -c user.email="release-bot@objectuve.com" tag -a "v$VERSION" -m "Release v$VERSION" (scripts/release.sh:104, OBJ-1490).

Where the gap still is: any other git-mutating command a crew agent runs directly on the shared runtime (outside release.sh and scheduled-release.yml) still has no equivalent guardrail. Downstream detection has improved since the incident, though: production.yml's changelog check (production.yml:48-54) now exit 1s on a missing entry instead of warning, and a dedicated release-integrity-guard.yml workflow fails master commits with non-release-bot/non-PR attribution and v* tags whose diff shape doesn't match scripts/release.sh's deterministic output (both from OBJ-1475, and its self-test replays the real v4.0.3/c8d9186 incident to prove it would have caught it). Neither of these blocks a misattributed tag identity on an otherwise well-formed release, though — they catch bypasses and shape mismatches, not a correctly- shaped commit tagged under the wrong git identity.

Cross-references:

• OBJ-1473 — Dave's diagnosis, Orion's fix-package task breakdown, and Josh's tag-identity audit (v4.1.0 also tagged as Desi) that reopened the platform-identity question.
• OBJ-1490 — the release.sh tag-step identity fix described above.
• OBJ-1429 — the Teams GA ship issue where Vicki's ship-confirmation first surfaced this gap.
• Multica runtime identity: the daemon.id hijack — the shared-runtime model above also means the runtime record itself can be claimed by a different machine; 2026-08-02 outage and diagnosis.

Status: repo/CI hardening (ruleset + tag restrictions + failing changelog gate + regression-guard workflow) is scoped under OBJ-1473, with the changelog gate and regression guard already shipped (OBJ-1475). On the deeper platform question, Josh decided (OBJ-1473 comment f7b7f273): require explicit -c user.name=/-c user.email= on every crew git-mutating command, not only release.sh — rejecting both per-agent runtime isolation and a tag-push-only gate as either too large a lift or blind to non-release git operations. release.sh's tag step is fixed under that decision (OBJ-1490); enforcing the same discipline across every other direct crew git-mutating command (a skill/runbook change, executor still to be confirmed) is the remaining follow-up.

Priority order

1. Fix #1 (dedicated identity) — solves the exhaustion and PR attribution.
2. Fix #2 (stop polling) — biggest behavioral win, no token needed.
3. Release break-glass token — optional, only needed if the manual scheduled-release.yml dispatch path is ever used; the Wednesday train doesn't depend on it.
4. Fix #3 (throttle autopilots) — trims the long tail.

---

Last updated: 2026-09-15