Sergey Feedback on AWS Demo Product Gaps

Slack-ready response

The demo has 4 paths, but the app shows 13 paths, drift counts, and repeated derivatives. Instead of telling the 4 decision-relevant risks, it makes the user parse path inventory. It also appears to elevate the wrong story: the demo says lead with the ops agent that can reboot any server and run arbitrary shell commands, but the UI seems to center the support or ServiceNow path. We likely also need to tighten how risk is prioritized.

Semantically, ServiceNow and Salesforce credential access appears to be classified as LLM egress, and the remediation follows that framing. Same on activity and ownership: the script describes orphaned contractor infra, while the product shows 0 ownerless paths.

My take: the product needs to explicitly surface what happened, why it matters, and am I exposed. Strategic Context is the right posture-level surface for the first part of that. The brief should carry the case story and the exposure story. Each critical path should explain in plain English what happened and why it matters, the way it is described in the demo script.

Executive Summary

The issue is not the demo narrative.

The issue is that the product is still surfacing derivative path inventory instead of the 4 canonical governance stories.

That creates three gaps:

• the wrong story appears to rank first
• some findings appear to be labeled semantically incorrectly
• the user has to do too much narrative compression on their own

Core feedback

1. The product should surface the 4 stories, not the 13 paths

The 4 stories are:

1. nimbus-ops-monitor can reboot any server and run arbitrary shell commands.
2. A departed contractor's dormant Lambda can still exfiltrate customer PII.
3. The support Lambda holds Salesforce credentials it does not need.
4. A dormant Fargate workload has broad S3 access and vendor-secret exposure.

Those are the decision-relevant risks.

Everything else should support them.

2. Risk prioritization appears off

The demo script says the opener is the ops agent with shell-command authority over the fleet.

The current UI appears to center the support to ServiceNow path instead.

That likely means risk prioritization needs tightening, not just presentation.

3. Some semantics appear wrong

ServiceNow and Salesforce credential access appears to be framed as LLM egress.

That looks wrong, and the remediation follows the same misclassification.

The same issue shows up on ownership:

• the demo script describes orphaned contractor infrastructure
• the product shows 0 ownerless paths

If the system is not naming the risk correctly, trust breaks quickly.

4. The Brief needs to do more of the narrative work

The Brief should explicitly surface:

• what happened
• why it matters
• whether I am exposed right now
• who owns it, or that it is ownerless
• whether it is active, dormant, or standing authority only
• what system, data, or action is reachable
• the top remediation move

The Strategic Context panel is the right place for the cluster-level why it matters.

Each critical path should do the same in plain English, the way the demo script does.

This should be implemented as a 3-layer narrative model:

1. Overview / Strategic Context = posture-level compression
2. Brief = case-level story plus exposure read
3. Access chain / path detail = exact-path summary

Do not add a separate large posture-narrative section right now.

Use the existing surfaces correctly.

Overview / Strategic Context

This should tell the posture-level story for the top cluster:

• what pattern is happening in this environment
• why this cluster is the most important one right now
• how it relates to the broader posture

This is the right place for the short environment read, not a generic telemetry panel.

Brief / What Happened

This should read like the demo script for that cluster.

Not counts. Not path mechanics. The actual story:

• what happened
• what the risk is
• who or what is involved
• whether it is active, dormant, orphaned, or standing authority only

This is the missing middle layer today.

The brief also needs an explicit Am I Exposed? section.

That section should answer the operational question:

• how much of this is active now
• what is still standing authority
• what systems or data are currently reachable
• what the current blast radius is

What Happened tells the story.

Am I Exposed? tells the operator whether this is theoretical, dormant, active, or partially exercised and how broad the reachable surface is.

Access chain / path detail

This also needs a short narrative summary at the top.

Not another full section. Just a strong sentence or two directly under the header that explains:

• what this exact path allows
• why this exact path matters

If we only fix Overview and Brief, the product will still collapse back into graph mechanics at the last mile.

So this is not solved by Strategic Context alone. It is mostly solved by using all 3 layers correctly, with Am I Exposed? explicitly doing the blast-radius and current-state work inside the brief.

5. Suggested copy for this current brief

For the current Drifted Sensitive Access screenshot, Claude should not invent a generic summary.

It should write the brief more like the demo script: more narrative, more causal, less KPI-style.

It should write the brief more like this:

What Happened

Nimbus let sensitive access expand across a small set of automations without tightening scope. The customer-support workflow can now reach ServiceNow and Salesforce credentials beyond its core job, and a dormant customer-data export path still retains the authority to pull customer data and send it to an external vendor. Nothing in this cluster executed in the last 30 days, but the broader access is still live and ready to be used.

That is the right shape:

• plain English
• specific systems
• clear statement of the risk
• explicit note that this is standing authority, not observed activity
• reads like an operator story, not an API summary

Am I Exposed?

Yes. This is standing authority rather than recent activity, but the exposure is still real. Eight drifted access paths remain reachable today. If any of these automations run again, they can immediately reach customer data, SaaS credentials, and external egress paths without a new approval step.

That section should then use the metrics underneath to support the statement, not replace it.

The metrics should answer:

• how many paths are still reachable
• how many are active vs dormant
• what sensitive domains are in scope
• whether external egress is part of the reachable surface

The sentence above the metrics is essential.

Without it, Am I Exposed? reads like a KPI strip instead of an operator answer.

Tone rule for Claude:

• borrow the narrative rhythm of the demo script
• explain cause and consequence
• use counts only as support
• do not let the prose read like a metric aggregation sentence

6. Likely is directionally risky

I checked sv0-documentation first.

The docs are mixed:

• the main platform framing is deterministic and explicitly not probabilistic
• earlier product language uses plain-English execution states like Execution Confirmed and Standing Authority Only
• a later evidence-model note introduced Likely for correlated evidence strength

So Likely was planned in one later model, but it does not match the older and broader product framing.

It also reads badly in this specific finding because the same screen shows:

• Likely
• 0 executions (30d)
• Execution Evidence: N/A

That reads more like guesswork than evidence.

If Likely stays, the product needs to explain exactly what is correlated and why that is still a defensible claim.

Direction for Claude

1. Promote the 4 canonical stories above all derivative paths.
2. Tighten risk ranking so the ops shell-command story is first.
3. Fix semantic labeling for credential access, external egress, LLM egress, activity, and ownership.
4. Implement a 3-layer narrative model: Overview / Strategic Context for posture-level compression, Brief for What Happened plus Am I Exposed?, and a short path-level summary on access-chain detail for the exact-path story.
5. For the current AWS brief, use concrete narrative copy in both What Happened and Am I Exposed? rather than generic count summaries.
6. Do not add a new large posture-narrative module right now. Use the existing surfaces correctly.
7. Treat infra permissions as supporting evidence, not the primary story.
8. Revisit whether Likely should appear in-product at all unless its evidence basis is explicit.

Acceptance Standard

This pass is successful when the product reads as a small set of canonical governance stories, with:

• Strategic Context explaining the posture-level why
• What Happened explaining the case-level story
• Am I Exposed? explaining current blast radius and active vs dormant exposure
• path detail explaining why the exact path matters
• correct semantics, clear priority, and supporting evidence underneath