Skip to content

Public status page

Objectuve's public status page lives at https://status.objectuve.com and is hosted by BetterStack. It surfaces a single green/yellow/red verdict per user-facing surface; the internal team uses the admin dashboard's System Health view at /system/self-health for component-level detail.

What it monitors

ComponentMonitor targetCheck
APIhttps://api.objectuve.com/healthHTTP 200 and JSON path $.status == "ok"
Apphttps://app.objectuve.com/HTTP 200
Adminhttps://admin.objectuve.com/HTTP 200
Marketing (www)https://www.objectuve.com/HTTP 200
Agent Runnerhttps://api.objectuve.com/healthJSON path $.checks.agent_runner.status == "ok"

Agent Runner is private (Cloud Run + OIDC), so its liveness is proxied through Rails. The Rails HealthController probes agent_runner with an OIDC bearer and surfaces the result under checks.agent_runner. The probe is informational — a down agent_runner does NOT flip the overall API verdict to 503, so we don't page on-call for an unrelated subsystem.

How the JSON contract maps to monitors

Rails /health returns:

json
{
  "status": "ok",
  "timestamp": "...",
  "checks": {
    "database":     { "status": "ok", "active_connections": ..., "max_connections": ... },
    "redis":        { "status": "ok" },
    "sidekiq":      { "status": "ok", "processed": ..., "failed": ..., "dead": ..., "queues": [...] },
    "litellm":      { "status": "ok" },
    "agent_runner": { "status": "ok" }
  }
}

BetterStack's API monitor reads $.status. BetterStack's Agent Runner monitor reads $.checks.agent_runner.status from the same response. The internal admin view reads the same data via the systemHealth GraphQL query, which uses SystemHealthService — same probe logic, separate code path.

Filing an incident

  1. Go to https://betterstack.com → Status Pages → Objectuve.
  2. Click Report Incident. Pick affected components, severity, and write the first update.
  3. Subscriber notifications (email/SMS) fire automatically.
  4. Add status updates as the situation evolves. Resolve when fixed.

For high-severity issues, cross-post in our incident channel and follow incident-response.md.

Editing the page

  • Admin access in BetterStack is limited to engineering leads. Ask in #eng for an invite.
  • Component names, branding, and DNS are configured in BetterStack settings.
  • The page is at status.objectuve.com via a CNAME → BetterStack (managed in our DNS provider).

Fail-closed watchdogs for informational checks

/health's crono, email_canary, and email_suppressions checks are all informational and fail-open at the HTTP layer by design (HealthController::INFORMATIONAL_CHECKS) — a dead Crono process, a stale delivery canary, or a suppressed watched address never turns /health into a 503, so an incident there doesn't also block every new Cloud Run revision from reaching Ready during a deploy. None is a BetterStack monitor target; each instead has its own Multica autopilot as the fail-closed alerting layer:

CheckAutopilotScheduleClassification
checks.cronoDave — Crono liveness checkDailydegraded/blocked states escalate
checks.email_canaryDave — Email canary liveness check (OBJ-2970)Dailyok → pass, stale/error → fail, not_configured or an absent key → blocked, not fail
checks.email_suppressionsDave — Email suppression liveness check (OBJ-2978)Daily, 14:00 UTC (every other Dave autopilot runs on America/Chicago)ok/not_applicable (staging, expected) → pass, suppressed/error → fail, not_configured or an absent key → blocked; not_applicable on production → blocked, not pass

All three are diagnostic-only — they file or comment on a tracking issue, never attempt a fix. Full runbooks: docs/guides/multica-autopilots.md. The canary's own real-time Slack/Sentry alerting (which this autopilot backstops) is documented in docs/operations/alerting.md § Email Delivery Canary.

All three checks now have a BetterStack heartbeat as a second, independent layer on top of this table — Dave — Crono liveness check (OBJ-2976), plus Dave — Email canary liveness check and Dave — Email suppression liveness check (both OBJ-2978 Task 2) — each detects the class of outage one level up: the autopilot fleet itself going silent, not just a single check's result. See Plan / cost § Heartbeat coverage decisions above for the remaining open decisions (staging, monthlies) and why.

Internal detail view

/system/self-health in the admin dashboard renders the same probe results as the BetterStack page, plus richer details (build SHA, Sentry/PostHog/Clerk frontend probes, GraphQL latency). The internal view is what we look at first when triaging; the public page is what users see.

Plan / cost

The free tier covers uptime monitors and heartbeats — Better Stack's own public wording is inconsistent about how, and that ambiguity is unresolved as of this writing:

  • Uptime product page: "Get 10 monitors, 10 heartbeats and a status page with 3-minute checks totally free." (reads as two separate 10-item allowances)
  • Pricing page line item: "10 monitors & heartbeats" (reads as one combined 10-item pool)

Current consumption (team t545473): 5 uptime monitors + 1 heartbeat = 6 items. Under the pessimistic combined-pool reading that leaves a working budget of ≤4 additional heartbeats before hitting the free-tier ceiling; the generous split reading leaves more headroom. Nothing should be sized past the pessimistic number without checking the account's own billing page first — that's the only authoritative source for which reading is actually in effect.

Over-allowance pricing (monthly billing): +$25/mo per additional 50 monitors, +$20/mo per additional 10 heartbeats.

Upgrade to Small Team (~$29/mo) for 30-second checks, unlimited subscribers, and Slack alerting before any major launch — unrelated to the monitor/heartbeat allowance above.

Heartbeat coverage decisions (OBJ-2978)

Phase 1 — daily liveness checks (canary + suppression): shipped. Dave — Email canary liveness check and Dave — Email suppression liveness check both now have BetterStack heartbeats (OBJ-2978 Task 2), matching Dave — Crono liveness check's existing coverage (OBJ-2976). The two decisions below (staging, monthlies) are what's left open.

Dave — Staging health check: skip. No heartbeat coverage.

  • Better Stack heartbeats are fixed-period only — no cron expression, no day-of-week awareness. You set "expect a heartbeat every X" plus a grace period, and that is the entire vocabulary. The 60h Friday-20:00 → Monday-08:00 gap in this autopilot's 0 8-20 * * 1-5 schedule cannot be expressed.
  • Splitting into two heartbeats (one pinging on the current day) does not work and is rejected on mechanics, not preference. A weekday-tight heartbeat still expires unpinged over the weekend and alerts anyway — splitting doesn't create a pause primitive; nothing in the product does.
  • Moving the check to a 7-day cron to suit the monitoring tool is rejected. The weekend skip is deliberate — the Crono runbook explicitly contrasts itself against it ("this check has NO weekend skip, unlike the staging health check"). Changing when a production check runs to suit a monitoring tool is backwards, and it would add ~26 runs/week of cost and noise for weekend staging deploys that don't happen.
  • Blast radius is staging-only. This is the lowest-severity autopilot in the fleet to leave uncovered.
  • The class of outage this exists to catch is already covered. The Crono heartbeat (below) runs 7 days a week and detects fleet-wide silence. A staging heartbeat would add per-autopilot attestation for the cheapest possible target while consuming a scarce slot.

Fallback, documented but not built — if coverage is ever wanted: one heartbeat at period 26h + grace 4h, with a weekend keep-alive ping added to a 7-day autopilot's runbook (Dave — gc maintenance watchdog, 17 */6 * * *, is the natural host). This works where the split-heartbeat approach doesn't, at the cost of coupling: on a weekend, that heartbeat can no longer distinguish "staging check broken" from "gc watchdog broken." Recorded as a trade-off, not built.

Monthly autopilots (Dave — Signal instrumentation audit monthly, Dave — Clerk auth tripwire): worth covering. Deferred to Phase 2, gated on the billing look.

  • A heartbeat with period 1 month + grace 4h alerts within grace of the expected run time — the same day, not "up to a month after it happens." What's long is the exposure window between checks, which is inherent to running a check monthly and is not a property of the heartbeat.
  • The value is not hypothetical: Dave — Signal instrumentation audit monthly last ran 2026-08-12 with status failed, and as of 2026-09-02 nothing had surfaced it. A heartbeat would have alerted the same day.
  • Deferred rather than shipped now because Phase 1 (the two daily liveness heartbeats from OBJ-2978) takes the pool to 8/10 under the pessimistic reading — adding both monthlies reaches 10/10 with zero headroom, and Dave — gc maintenance watchdog (active again, and a cleaner every-6-hours candidate than either monthly) would have nowhere to go. Maxing out a free tier on an unconfirmed reading of the allowance is the exact silent-billing-line risk this work exists to avoid.
  • Phase 2 should: confirm the allowance is split (10 + 10) rather than a combined 10, then cover Dave — Signal instrumentation audit monthly, Dave — Clerk auth tripwire, and Dave — gc maintenance watchdog. Better Stack's maximum heartbeat period must support ~31–35 days first — if it caps below a month, the monthlies get no heartbeat and the fallback is a rule in Maggie — Issue health sweep that flags an autopilot whose last_run_status is failed or whose last_run_at is older than its cron implies.

Heartbeat env-var naming: new BetterStack heartbeats set on Dave's custom_env follow BETTERSTACK_HEARTBEAT_URL_<AUTOPILOT_SLUG> (e.g. BETTERSTACK_HEARTBEAT_URL_EMAIL_CANARY). The bare BETTERSTACK_HEARTBEAT_URL is grandfathered to Dave — Crono liveness check (OBJ-2976, the first heartbeat provisioned) and is not to be reused for a different autopilot.

Out of scope (next iteration)

  • Auto-filing incidents from Cloud Monitoring / Sentry alerts.
  • Per-environment status (staging stays internal-only).
  • Latency percentiles — Cloud Monitoring already covers these.

Heartbeat monitors for Sidekiq cron jobs, formerly listed here, shipped — see Fail-closed watchdogs for informational checks above. They landed as Multica autopilots, not BetterStack monitors, which is why they don't appear in the monitor table.


Last updated: 2026-09-06 (OBJ-2978 Task 2: updated the fail-closed watchdogs section to reflect that canary + suppression heartbeats have shipped, not just Crono; noted Phase 1 as shipped in the Heartbeat coverage decisions section; fixed the #plan-cost anchor to #plan--cost on both new links, since GitHub's slugger turns the / in "Plan / cost" into a double hyphen. Previously 2026-09-02, OBJ-2978 Task 1: corrected the free-tier cost claim to cover heartbeats, recorded the staging/monthly heartbeat coverage decisions, added email_suppressions to the watchdogs table, fixed the broken Crono heartbeat runbook anchor)

Loading…