Skip to main content

SOC agent guided Review — engineering runbook (#1800 / #1825)

The primary CISO demo: a SOC investigation agent quietly gains the ability to isolate enterprise endpoints; SecurityV0 detects the drift, derives the exact supported Azure change, rehearses what stops and what keeps working, and — after the remediated rescan — shows a strict, evidence-gated Verified result.

Substrate: the seeded demo-microsoft-soc tenant (synthetic fixture, agreed per demo-reality.md rule 2 — it runs the real ingest → evaluator → rehearsal → decision surface pipeline; nothing is pre-rendered). The live Contoso/x2 lab remains the honest Requires design exhibit: its drift applies two simultaneous changes (Responder role AND the playbook action definition), so no single causal cut is provable from live evidence — exactly the case the product must not fake.

Presenting this demo​

The presenter-facing script (dev-environment URLs, state switching, spoken lines, disclosure, Q&A) lives in Demos → SOC Agent Guided Review. This runbook is the engineering side only: local reproduce, gate semantics, and the live-lab path.

Reproduce from scratch (local)​

Prereqs: MongoDB on localhost:27017, repo sv0-platform (branch with #1800/#1825 work).

# 1. API (dev auth provider; workers run in-process)
AUTH_PROVIDER=dev npm run dev            # PORT=3100 MONGODB_DB=... to isolate if :3000 busy

# 2. UI
cd ui && npm run dev                     # SV0_API_TARGET=http://localhost:3100 if API not on :3000

# 3. Feature flags (once per tenant; survives seed --reset)
npx tsx scripts/enable-demo-flags.ts     # API_BASE / TENANT_ID env as needed

# 4. Demo states — each is ONE command (cumulative syncs, deterministic):
npx tsx scripts/seed-demo-sentinel.ts --reset --phase baseline    # sync 1
npx tsx scripts/seed-demo-sentinel.ts --reset --phase drift       # syncs 1+2 (default)
npx tsx scripts/seed-demo-sentinel.ts --reset --phase remediated  # syncs 1+2+3

# 5. Nothing else — after the remediated sync the worker auto-resolves the
#    finding AND auto-generates the verification report; Review flips to
#    Verified on its own (no manual report step, no clicks).

Open: /t/demo-microsoft-soc/entities/<playbook-entity-id>?tab=review.

Notes:

  • The seed verifies its own outcome per phase (polls the evaluate_findings job, then asserts the finding state) and exits non-zero on mismatch — trust the exit code.
  • The verdict precompute runs from the worker when the drift evaluation completes and both tenant flags are on, so Review is ready on open (zero clicks). If flags were enabled after seeding, the page's one-shot auto-compute covers it.
  • Deployed instances: deploy/scripts/pr-preview-admin.sh <instance> seed-demo-sentinel (SEED_PHASE=baseline|drift|remediated) and ... enable-rehearsal (sets both flags).

Verified gate (what "Verified" requires — all five legs)​

  1. Finding resolved by a completed sync (status=remediated, resolved_by_sync).
  2. A pre-deploy reachability report generated at-or-after that sync…
  3. …containing a retained read-action access path to an affected resource…
  4. …and containing NO path that still carries a gained action (isolation absent is proven, not inferred from the finding's resolution).
  5. Every contributing source's data at least as fresh as the resolving sync's start.

The report in legs 2–4 is generated automatically by the worker when the finding resolves — the closed loop needs no operator action.

Any missing leg → the page shows Latest scan result naming exactly what verification is waiting for. No leg is ever assumed.

E2E test​

test/integration/soc-guided-review-scenario.test.ts (npm run test:integration) drives the same three states through the real pipeline and asserts the finding, causal-grant attribution, guided block, auto-resolution, and the Verified gate.