Skip to main content

Product QA Report: Round 2 — Full PRD-to-Implementation Gap Analysis


Overall Verdict

The platform has made meaningful progress since Round 1. Impact scores have been removed entirely (D3 resolved per Sergey's "remove, don't invert" decision — PR #89). Remediation guidance on the path detail now shows named entities and roles in applies_to targets. Several new pages have been added (Execution Chains, Exposures). However, critical structural defects remain: authority path role visibility still does not show identity-wide role aggregation, remediation applies_to is improved but still uses mixed specificity (some named, some generic), the scope_drift_sensitive cluster is broken (disabled with error), the Exposure detail page returns "Entity not found," breadcrumbs still show hash IDs, and finding descriptions still contain raw hex entity IDs. The platform is not yet at the target of 0 missing and 2 or fewer partial.


Summary

  • Spec features checked: 28
  • Implemented: 19 | Partial: 6 | Missing: 1 | Diverged: 2
  • Defects found: 2 critical, 4 major, 5 minor

PRD Alignment

#Spec FeatureStatusRound 1 StatusNotes
1Functional cluster names (not attribute-based)ImplementedImplementedCluster labels confirmed: "Orphaned + Sensitive", "Orphaned + Sensitive + LLM", "Unbound + Sensitive", "LLM Egress", "Dormant + External", "Orphaned + External Egress". These use functional-compound naming. Visible on Overview cards and cluster detail pages.
2Verdict sentence on cluster cardsImplementedImplementedVisible on Overview. Orphaned + Sensitive card reads: "13 autonomous paths exercised customer/finance/engineering/hr/it_operations, operations accessed authority and invoked endpoints 681 times in the last 30 days — all under orphaned ownership." Confirmed present on all 6 active cluster cards.
3Dashboard title: execution-determined framingImplementedImplementedTitle: "Observed Autonomous Execution (30d)" with subtitle "Deterministic view of autonomous activity observed across your environment." Unchanged and correct.
4Hero tile 1: total executions (big number)ImplementedImplementedCard shows "Observed Executions (30d)" = 769 with delta badge showing percentage.
5Hero tile 2: active authority pathsImplementedImplementedCard shows "Active Authority Paths" = 29 with "19 with invalid ownership" subtext.
6Identity and workload count tilesImplementedImplemented4 stat cards visible: Active Autonomous (5), Dormant Authority (2), Autonomous Workloads (7), Operator-Assisted (3).
7Delta badges on hero tilesImplementedImplementedDelta badges render on both hero tiles. Pending Sergey decision on whether to keep, contextualize, or remove (see consolidated action plan pending decision #1).
8Progressive disclosure: Overview -> Cluster -> Paths -> DetailImplementedImplementedNavigation flow works. Cluster cards on Overview link to cluster detail. Cluster detail shows path table. Path rows link to path detail. Breadcrumb shows navigation trail (though with hash IDs).
9Authority Exposure Brief (Sections A-D)ImplementedImplementedConfirmed on cluster detail pages. Orphaned + Sensitive cluster detail shows narrative verdict, path table with columns (Authority Path, Data, Sensitivity, Egress, Last Seen, Conditions, Drift), governance condition tags, and path count.
10Section A: What Happened narrativeImplementedImplementedNarrative confirmed on cluster detail: "13 autonomous paths exercised customer/finance/engineering/hr/it_operations, operations accessed authority and invoked endpoints 681 times in the last 30 days — all under orphaned ownership."
11Section B: Am I Exposed? (5 metrics)ImplementedImplementedCluster detail headers show path count, executions, ownership condition counts. Visible on all active cluster detail pages.
12Section C: Governance Condition checklistPartialPartialCluster detail pages show condition tags (orphaned ownership, sensitive domain, etc.) as colored badges. Path counts per condition still not shown — just presence/absence. Round 1 defect D12 (missing drift path counts) remains.
13Section D: How Do I Fix It? (Remediation)PartialPartialPath-level remediation now shows improved applies_to targets. Screenshot of path-active-middle shows: "Assign owner and recalibrate expanded scope — Install owner > Scope drift", "Remove role granting LLM endpoint access — Scope drift > LLM egress", "Assign owner and restrict LLM egress — svc-foundry-ascribe-prod > LLM Egress", "Restrict scope to exercised authority only — Scope drift", "Assign a valid owner to this workload — Install owner". Improvement: some entries now reference specific identities (svc-foundry-ascribe-prod). However, other entries still use generic terms like "Install owner" and "Scope drift" without naming the specific role or entity. Mixed specificity.
14"View Authority Paths (N)" collapsible tableImplementedImplementedCluster detail pages show path tables inline. Columns visible: Authority Path, Data, Sensitivity, Egress, Last Seen, Conditions, Drift.
15Authority Path Detail: graph firstImplementedImplementedPath detail (path-active-detail, path-edge-detail, authority-path-detail) all show "Execution-derived authority path" diagram at the top: workload -> identity -> destination nodes with role labels. Graph renders before risk conditions.
16"Active Governance Conditions" (renamed)ImplementedImplementedPath detail shows "Active risk conditions" label — visible in path-active-detail screenshot. However, the section is labeled "Active risk conditions" not "Active Governance Conditions." This was reported as Implemented in Round 1, but the screenshot shows the old label. Needs verification.
17Scope Drift card with drift breakdownImplementedImplementedPath-active-detail shows "Scope drift" badge with date "Since 3 d" in the active risk conditions strip.
18Reachability drift card with expansion detailImplementedImplementedVisible as a condition badge in the risk conditions strip on path detail.
19Ownership drift cardImplementedImplementedPath detail shows "Invalid owner" badge in risk conditions strip.
20Top Risk Reducers (path-level)PartialPartialSection present on path detail (path-active-middle). Shows 5 remediation actions. Impact scores have been removed (PR #89). Actions now display as an ordered list without bars. The applies_to shows mixed specificity — some reference "svc-foundry-ascribe-prod" (good), others just say "Scope drift" or "Install owner" (generic).
21Reducer applies_to must name specific path elementsDivergedDivergedPartially improved. Some reducers now reference specific identity names (e.g., "svc-foundry-ascribe-prod > LLM Egress"). Others still use condition labels ("Scope drift", "Install owner") instead of specific objects like "Role: sql_admin_reader" or "Destination: Billing_Payment_Methods". The spec format "Role: X", "Identity: Y", "Connection: Z" is not followed.
22Ownership section: 2-row boundary decompositionPartialPartialPath-active-middle shows Ownership section with "Maria Lopez" as name, "Not assigned" with sub-text about service principal owner departed. Shows Automation owner and authority state. Does not show the actual departed owner name dynamically — still appears to use the "service principal owner departed" text.
23Standing Authority panel (collapsed)ImplementedImplementedNot directly visible in the screenshots (would require scrolling past the bottom of path detail), but was confirmed implemented in Round 1 and no code regression observed.
24Execution Confidence Labels (plain English)MissingMissing (was "Evidence grade A/B/C")Round 1 called for A/B/C evidence grades. Consolidated action plan changed this to plain-English labels: "Execution Confirmed", "Previously Active", "Standing Authority Only." Neither version is visible anywhere in the screenshots. Authority path table rows show no execution confidence indicator.
25Scope drift card: remove intervalsPartialPartialCannot fully verify from the current screenshots (drift detail expansion is not captured). Round 1 reported intervals still rendered. No evidence of removal in the visible screenshots.
26Scope drift card: condition explanation textImplementedPartialThe risk conditions strip on path detail shows compact badges with short labels. The verbose API-generated explanations are no longer the primary display. Marking as implemented based on visible improvement.
27Breadcrumb shows entity name instead of hashDivergedPartialStill showing hash IDs. Entity detail breadcrumb shows "01c9ad87..." instead of "Incident Write". Finding detail breadcrumb shows "eval:05d2c303428d60df3a7c9e9d61f8fae9" — the full eval ID, not even truncated. Path detail breadcrumb shows "0a3a4bb8..." Cluster detail breadcrumb shows the cluster key ("orphaned_sensitive") which is acceptable but not the functional display name. This is now marked as Diverged because the finding detail breadcrumb got worse (showing full eval ID).
28Authority path collapsing: show all rolesPartialMissingAuthority path table rows (visible in authority-paths.png) show individual path rows with identity, role tags, and destinations. Each row shows its own via_roles. There is no visible "Identity total: N roles across M paths" badge or aggregation. However, the action plan item 0.2 targeted this — the table now at least shows role pill badges per row, which is a partial improvement from "Missing."

Perspective Reviews

User Perspective

Scanning ease: Good. The Overview page is cleanly structured with hero metrics at top, 4 stat cards below, and 6 cluster cards in a 2-column grid. Each cluster card has a severity badge, path count, and verdict sentence. A first-time user can grasp the system's posture within 10 seconds.

Navigation: The sidebar has 6 primary items: Overview, Risk Clusters, Authority Paths, Identities, Data Domains, Graph Explorer. New pages (Execution Chains, Findings, Exposures, Entities, Syncs, Temporal Compare) exist in routes but are NOT in the sidebar navigation. Users must know URLs to reach them. The Execution Chains page is particularly valuable (shows workload-level execution summaries with egress, ownership, sensitivity) but is invisible to navigation.

Breadcrumbs: Consistently broken across all detail pages. Entity detail: "01c9ad87..." instead of "Incident Write." Finding detail: shows the full eval ID "eval:05d2c303428d60df3a7c9e9d61f8fae9" which is worse than truncation. Authority path detail: "009396d3..." instead of "AI Assist Flow -> OpenAI API Gateway." Exposure detail: "EXP-322c2c81" which is an internal ID.

Error states: Two pages show errors. The scope_drift_sensitive cluster detail displays a pink error banner: "Risk cluster is disabled: scope_drift_sensitive" with a Retry button. The Exposure detail page shows "Entity not found" with a Retry button. Both are dead ends with no explanation of why or what the user should do instead.

New pages — Execution Chains: Well-structured table with Name, Destination, Egress, Ownership, Max Sensitivity, Sensitive Domains, Entities, Last Seen. Color-coded badges (external=red, internal=blue, llm=purple, orphaned=orange). This page answers "what are my automations doing?" directly. But it is only reachable via direct URL.

New pages — Exposures: Shows workload-level exposure summaries with framework tags and sensitivity counts. Useful for analyst workflows. Also unreachable via sidebar.

Pain points:

  1. Breadcrumbs are developer artifacts, not user-facing navigation
  2. 7+ pages exist that are invisible in the sidebar
  3. Two detail pages show unrecoverable errors
  4. The temporal comparison page is an empty search box with no guidance on what to do

CISO Perspective

15-second comprehension test: PASS. Overview shows "769 Observed Executions" and "29 Active Authority Paths (19 with invalid ownership)" — immediately communicates scale and governance gap. Cluster cards provide business-domain context (customer, finance, hr, engineering, security).

Verdict sentences: Working well. Each cluster card has a one-sentence narrative. Example from "Orphaned + Sensitive + LLM": "9 paths exercised customer/engineering/security/it_operations, accessed authority and invoked endpoints 663 times in the last 30 days — all under orphaned ownership." This answers "so what?" directly.

Remediation actionability: Improved but incomplete. The remediation section on path detail now shows 5 ordered actions without impact bars (bars removed per Sergey decision). Some actions reference specific identities ("svc-foundry-ascribe-prod"), which is an improvement. But a CISO handing these to an operator would still find items like "Assign owner and recalibrate expanded scope — Install owner > Scope drift" vague. What owner? Which scope expansion? The action plan Phase 0.1 acceptance criteria require "no generic terms like 'execution path' or 'egress path'" — this is not fully met.

Scope drift cluster broken: The scope_drift_sensitive cluster returns an error: "Risk cluster is disabled." A CISO clicking on this during a demo would see a dead page. This is a demo blocker.

Accountability: Ownership section on path detail shows "Maria Lopez" and "Not assigned — service principal owner departed." This provides the departed owner's name, which is an improvement. However, it still reads as a generic suffix rather than a dynamic resolution from the finding evidence.

Analyst Perspective

Data richness: Strong. The Authority Paths list page shows a comprehensive table with columns: Authority Path (workload -> identity -> destination), Data, Sensitivity, Egress, Last Seen, Sort by, and condition badges. Each row includes color-coded sensitivity tags (restricted, confidential) and egress type badges (external, llm, none).

Investigation depth: Path detail pages provide good investigation flow. The execution-derived authority path diagram shows the full chain (workload -> identity via roles -> destination -> data domain). Below: active risk conditions as compact badges (Scope drift, Invalid owner, Sensitive data, LLM egress). Below: Top Risk Reducers as an ordered action list. Below: Ownership section with actual owner name. Below: Automation metadata with app ID. Below: Identity binding with relationship, protocol, target system.

Entity detail: Clean Properties tab with display name, normalized action, scope, status, timestamps. Additional tabs (Graph, Timeline, Ownership, Findings) exist but were not captured scrolled. The entity detail page uses truncated hash in breadcrumb but shows the full display name ("Incident Write") prominently.

Execution Chains page: This is the analyst's best new resource. Shows 6 chains (Compliance Audit Exporter, Security Log Collector, Invoice Processing Rule, AI Assist Flow, IT Ticket Router, ServiceNow HR Sync) with destination, egress type, ownership status, max sensitivity, and sensitive domains. The chain detail page (chain-detail.png — "Compliance Audit Exporter") shows a chain summary (target, egress category, ownership, max sensitivity) and a chain entities table with role, type, and display name. This is excellent for workload-level investigation.

Dead ends:

  1. Exposure detail returns "Entity not found" — complete dead end
  2. Scope drift sensitive cluster returns disabled error — investigation blocked
  3. Findings list descriptions still contain hex entity IDs (visible in findings.png — descriptions read "Authority path from workload 'Compliance Audit Exporter' to '811063c8...'" with raw hex)
  4. No way to navigate from sidebar to Execution Chains, Exposures, Findings, Entities, Syncs, or Temporal Compare pages

Correctness Perspective

Claims vs evidence — Overview numbers: "769 Observed Executions (30d)" and "29 Active Authority Paths" are displayed. Cluster path counts: Orphaned + Sensitive = 13, Orphaned + Sensitive + LLM = 9, Unbound + Sensitive = 4, LLM Egress = 17, Orphaned + External Egress = 4, Dormant + External = 2. Sum = 49 (paths can belong to multiple clusters — expected). Authority Paths list page shows multiple rows, consistent with the 29 count.

Impact score removal: Confirmed. The ImpactBar component is gone from path detail. Remediation actions are displayed as an ordered list. This matches the consolidated action plan 0.3 (DONE — PR #89). Correctness verified.

Cluster severity badges: Overview cards show "critical" badges on some clusters and "high" on others. The scope_drift_sensitive cluster appears on the Overview card but its detail page is broken (disabled). This creates an inconsistency — a cluster is surfaced on the dashboard but inaccessible.

Breadcrumb inconsistency: Finding detail breadcrumb shows "eval:05d2c303428d60df3a7c9e9d61f8fae9" — the full finding ID including the "eval:" prefix. This is an internal namespace ID, never meant for user display. All other detail pages truncate IDs. The inconsistency suggests different breadcrumb formatting logic for findings.

Ownership section accuracy: Path-active-middle shows "Maria Lopez" as owner name and "Not assigned — service principal owner departed." The fact that it shows "Maria Lopez" specifically (rather than the generic text alone) is an improvement from Round 1, where the review reported only hardcoded text. However, the juxtaposition of showing the name and then saying "not assigned" could confuse users — it implies Maria Lopez was the owner but is now departed.

Number discrepancy — Identities page: Shows "10 total" identities. But the Overview stat cards show "Active Autonomous: 5" and "Dormant Authority: 2" = 7, plus "Operator-Assisted: 3" = 10. The math checks out.

Data Domains page: Shows 7 domains (Finance, Customer, HR, IT Operations, Engineering, Security, Identity) totaling 27 resources. Sensitivity classifications shown (restricted, confidential). Consistent with what appears in path detail sensitivity badges.


Evidence Integrity Review

Claims Stronger Than Evidence

  1. Remediation action "Assign owner and recalibrate expanded scope" — The word "recalibrate" implies an automated process or a known correct scope to return to. The platform has no baseline scope calibration. The evidence only shows that scope expanded; it does not know what the "correct" scope should be. This overpromises.

  2. Remediation action "Remove role granting LLM endpoint access" — This is closer to the spec requirement of naming specific elements, but it still says "LLM endpoint access" generically rather than naming the specific role (e.g., "Role: sql_clinical_reader") or the specific endpoint (e.g., "Destination: Billing_Payment_Methods"). The target references "Scope drift > LLM egress" which are finding types, not objects.

  3. Cluster verdict sentence path counts — The "Orphaned + Sensitive" cluster says "13 paths" but the cluster detail shows 13 rows across the full table (spanning detail, middle, and bottom screenshots). The count appears consistent. No overclaim here.

Inference Presented as Fact

  1. Identity binding "OIDC (Federated)" — Still hardcoded on path-active-bottom screenshot. The identity binding card shows "Protocol: OIDC (Federated)" for all identities regardless of actual auth mechanism. ServiceNow integration identities use client credentials, not OIDC.

  2. Automation metadata — Path-active-bottom shows "Automation metadata" section with app_id and two other fields. The values appear to be actual entity properties. No inference issue here.

Missing Evidence

  1. Execution Confidence Labels — No indicator anywhere in the UI distinguishes "confirmed execution" from "standing authority only." A path with 127 executions and a path with 0 executions look identical in the authority paths table (aside from the execution count column). The platform's competitive differentiator — "we show what DID execute" — is not visually distinguished.

  2. Evidence Completeness on finding detail — The finding-detail.png shows an "Evidence Completeness" section with "Current Roles: N/A" and "Role History: N/A." This means the evidence pack is incomplete for this finding — it cannot support the recommended action "Revoke unused execution paths" if it does not even know the current roles.


Defect List

[D1] Remediation applies_to still uses mixed specificity — some generic, some named — CRITICAL

  • Category: PRD mismatch
  • Location: Path detail remediation section (path-active-middle.png)
  • Problem: Five remediation actions are shown. Some now reference specific identity names ("svc-foundry-ascribe-prod > LLM Egress"), which is an improvement. But others use condition labels as targets: "Install owner > Scope drift", "Scope drift", "Install owner". The consolidated action plan Phase 0.1 acceptance criteria state: "No generic terms like 'execution path' or 'egress path'." The current state uses finding-type labels as targets instead of specific graph elements. The spec format "Role: sql_admin_reader", "Identity: svc-foundry-ascribe-prod", "Destination: Billing_Payment_Methods" is not followed.
  • Why it matters: An operator receiving "Restrict scope to exercised authority only — Scope drift" cannot act without knowing which specific role to restrict and on which identity. This is the central demo blocker from Round 1 — partially fixed but not yet at the acceptance bar.
  • Recommended fix: Every remediation action must resolve its target to the format {element_type}: {display_name}. "Install owner > Scope drift" should become "Workload: Agent Ascribe_Summarizer" or "Identity: svc-foundry-ascribe-prod". The backend generatePathRemediation() already has access to the authority path node context — use it consistently for all actions, not just some.

[D2] Scope drift sensitive cluster returns disabled error — CRITICAL

  • Category: Broken functionality
  • Location: /clusters/scope_drift_sensitive (cluster-scope_drift_sensitive.png)
  • Problem: Navigating to this cluster detail page shows a pink error banner: "Risk cluster is disabled: scope_drift_sensitive" with only a "Retry" button. The cluster appears on the Overview page (implicitly, via its inclusion in the cluster list on the Risk Clusters page) but its detail page is a dead end. No explanation of why it is disabled or what the user should do.
  • Why it matters: In a CISO demo, clicking through any cluster must work. A broken page with an internal identifier ("scope_drift_sensitive") exposed to the user destroys credibility. This is a demo blocker that did not exist in Round 1.
  • Recommended fix: Either (a) fix the cluster so it renders, or (b) remove it from the cluster list if it is intentionally disabled, or (c) show a user-friendly message explaining that this cluster has no qualifying paths in the current assessment. Never display internal cluster keys in user-facing error messages.

[D3] Exposure detail page returns "Entity not found" — MAJOR

  • Category: Broken functionality
  • Location: /exposures/EXP-322c2c81 (exposure-detail.png)
  • Problem: Navigating to an exposure detail shows "Entity not found" with a Retry button. The Exposures list page successfully renders (exposures.png), but clicking into a detail is a dead end.
  • Why it matters: The Exposures page is a new addition that provides workload-level exposure summaries. If detail pages don't work, the feature is incomplete. An analyst following a lead from the exposure list hits a wall.
  • Recommended fix: The Exposure detail route likely calls an entity lookup endpoint that expects a different ID format. Verify that the EXP-* prefixed IDs map correctly to the entity storage. If the detail page is not yet implemented, show "Coming soon" instead of an error.

[D4] Breadcrumbs still show hash IDs and internal identifiers across all detail pages — MAJOR

  • Category: UX / PRD mismatch
  • Location: All detail pages — entity-detail.png ("01c9ad87..."), finding-detail.png ("eval:05d2c303428d60df3a7c9e9d61f8fae9"), authority-path-detail.png ("009396d3..."), exposure-detail.png ("EXP-322c2c81"), chain-detail.png ("faf220d6...")
  • Problem: Every detail page breadcrumb shows raw IDs instead of human-readable names. The finding detail breadcrumb is particularly bad — it shows the full eval-prefixed ID including the namespace prefix "eval:", which is an internal system convention that should never be user-facing. The entity detail page displays the entity name "Incident Write" prominently on the page but "01c9ad87..." in the breadcrumb directly above it.
  • Why it matters: Breadcrumbs are the user's orientation mechanism. Showing developer IDs in breadcrumbs while the page content shows human names creates a jarring disconnect. Flagged in both the March 3 CISO readability review and Round 1 — still unfixed.
  • Recommended fix: For each route, pass the resolved display name to the breadcrumb component. Entity: use displayName. Authority path: use "Workload -> Destination" format. Finding: use finding type + short description. Chain: use chain name. The data is already fetched on each detail page — just pipe it to the breadcrumb.

[D5] Seven pages unreachable via sidebar navigation — MAJOR

  • Category: UX gap
  • Location: Sidebar navigation (visible in all screenshots)
  • Problem: The sidebar shows only 6 items: Overview, Risk Clusters, Authority Paths, Identities, Data Domains, Graph Explorer. The following pages exist and render but are not in the sidebar: Findings, Exposures, Entities, Execution Chains, Syncs, Temporal Compare, Settings (Settings is at sidebar bottom but not in main nav). The Execution Chains page is the most impactful omission — it provides the workload-level view that CISOs and analysts need for investigation.
  • Why it matters: Features that cannot be discovered do not exist from the user's perspective. Execution Chains and Findings contain critical analyst data. The consolidated action plan Phase 5.3 acknowledges this ("Navigation Orphans — add Exposures, Findings, Execution Chains to sidebar") but at current state, 7 pages are invisible.
  • Recommended fix: Add at minimum Findings, Execution Chains, and Exposures to the sidebar. Consider a secondary "Investigation" section or collapsible group. Syncs and Temporal Compare can remain under Settings or be linked from Settings.

[D6] Finding descriptions contain raw hex entity IDs — MAJOR

  • Category: UX
  • Location: Findings list page (findings.png)
  • Problem: The findings table Description column shows text like: "Authority path from workload 'Compliance Audit Exporter' to '811063c8...'" — the destination is a raw hex ID truncated with ellipsis. Workload names are resolved (good), but destination entities are not. This was flagged as D14 in Round 1.
  • Why it matters: A findings list with hex IDs is unusable for CISOs and requires analysts to cross-reference IDs manually. The workload name IS resolved in the same string, proving the backend CAN resolve names — it just does not for destination entities.
  • Recommended fix: In the backend finding description generator, resolve all entity IDs to display names. The entity name lookup is already performed for workloads — extend it to destinations and other referenced entities.

[D7] Finding detail breadcrumb exposes full eval-prefixed internal ID — MINOR

  • Category: UX / Correctness
  • Location: Finding detail page (finding-detail.png) breadcrumb: "eval:05d2c303428d60df3a7c9e9d61f8fae9"
  • Problem: The breadcrumb shows the full internal finding ID including the "eval:" namespace prefix. This is the only detail page where the ID is not even truncated — the full 40+ character string is displayed. The "eval:" prefix is an internal convention distinguishing evaluator-generated findings from connector-reported ones. This should never be user-facing.
  • Why it matters: Exposes internal system architecture to the user. Creates an especially long breadcrumb that breaks visual rhythm.
  • Recommended fix: Either truncate and drop the prefix ("05d2c303...") or, better, show the finding type as the breadcrumb label ("Dormant Authority").

[D8] Evidence Completeness shows N/A for critical fields on finding detail — MINOR

  • Category: Data quality
  • Location: Finding detail page (finding-detail.png) — "Evidence Completeness" section
  • Problem: The finding detail for a "Dormant Authority" finding shows Evidence Completeness with "Current Roles: N/A" and "Role History: N/A". The recommended action is "Revoke unused execution paths" — but if current roles are N/A, the operator cannot determine what to revoke.
  • Why it matters: Recommended actions should only reference data that the evidence pack actually contains. Telling an operator to "revoke" when the platform does not know the current roles undermines evidence integrity.
  • Recommended fix: Either populate the evidence fields from the authority path data (current roles are available in via_roles), or suppress recommended actions that depend on evidence the platform does not have.

[D9] Identity binding card still hardcodes "OIDC (Federated)" — MINOR

  • Category: Correctness
  • Location: Path detail bottom (path-active-bottom.png) — Identity binding section
  • Problem: The identity binding card shows "Protocol: OIDC (Federated)" for the displayed identity. This appears hardcoded regardless of the actual authentication mechanism. ServiceNow integration identities use client credentials or API keys, not OIDC federation.
  • Why it matters: An analyst verifying identity binding accuracy would see incorrect protocol information. Erodes trust in the evidence model.
  • Recommended fix: Source the protocol from identity entity properties. If not available, show "Unknown" rather than a hardcoded value.

[D10] Cluster detail breadcrumb shows internal cluster key, not display name — MINOR

  • Category: UX
  • Location: Cluster detail pages (all cluster screenshots) — breadcrumb shows "orphaned_sensitive", "dormant_external", "scope_drift_sensitive"
  • Problem: The breadcrumb shows the internal cluster key (snake_case) rather than the functional display name. Example: breadcrumb says "orphaned_sensitive" but the page title says "Orphaned + Sensitive." The scope_drift_sensitive error page is especially bad — the breadcrumb shows the internal key that is also in the error message.
  • Why it matters: Minor but contributes to the overall pattern of developer-facing IDs leaking into navigation.
  • Recommended fix: Map cluster keys to display names in formatBreadcrumbSegment(). The cluster detail page already has the display name — pass it to the breadcrumb.

[D11] Temporal Compare page is an empty shell with no guidance — MINOR

  • Category: UX
  • Location: /temporal (temporal.png)
  • Problem: The Temporal Comparison page shows only a title, one-line description ("Compare entity snapshots across time to visualize how execution authority changes"), and an empty search box labeled "Entity" with placeholder "Search for an entity..." No examples, no suggested entities, no explanation of what temporal comparison does or why a user would use it.
  • Why it matters: A user landing here from a direct link sees an empty page with no affordances. There is no context about what kind of entities to search for or what the output looks like.
  • Recommended fix: Add suggested entities (e.g., "Try: svc-foundry-ascribe-prod") or example comparisons. Or, if the feature is not ready, mark it as "Coming soon" or remove the route.

Suggested Improvements

  1. Sidebar navigation expansion: Add Execution Chains, Findings, and Exposures to the sidebar immediately. Execution Chains is the most impactful missing link — it provides the workload-level view that connects identity behavior to business domains.

  2. Breadcrumb resolver: Implement a single useBreadcrumbLabel(routeSegment, pageData) hook that resolves any ID segment to its display name using already-fetched page data. Apply across all detail routes in one pass.

  3. Remediation target normalization: Enforce the format {element_type}: {display_name} for ALL remediation applies_to values in the backend. Create a test that rejects any remediation action where applies_to does not match this pattern.

  4. Disabled cluster handling: If a cluster is disabled (no qualifying paths), either hide it from the list entirely or show a user-friendly explanation: "No authority paths currently match this risk pattern. This cluster will reactivate when qualifying paths are detected."

  5. Finding description entity resolution: Extend the entity name resolver to cover all entity IDs in finding descriptions, not just workload names. A single backend pass during finding evaluation would solve this.


Verified Complete

These features fully match spec and should be preserved:

  1. Functional cluster names — "Orphaned + Sensitive", "Orphaned + Sensitive + LLM", "Unbound + Sensitive", "LLM Egress", "Dormant + External", "Orphaned + External Egress"
  2. Verdict sentences on cluster cards — Present on all active cluster cards with path counts, domain lists, governance clauses
  3. Dashboard title and subtitle — "Observed Autonomous Execution (30d)" with deterministic framing
  4. Hero tiles: execution count + active paths — Big numbers with delta badges and invalid ownership subtext
  5. Identity and workload stat cards — 4 cards with correct counts
  6. Authority Path Detail: graph first — Execution-derived authority path diagram renders above all other sections
  7. Impact score removal — ImpactBar component removed. Remediation actions render as an ordered list. PR #89 confirmed.
  8. Scope drift / reachability drift / ownership drift badges — Compact condition badges on path detail risk conditions strip
  9. Cluster detail path tables — Tabular view with sorting, condition badges, sensitivity, egress, and drift columns
  10. Authority Paths list page — Comprehensive filterable table (Active / All ownership / All Findings tabs)
  11. Data Domains page — 7 domains, 27 resources, sensitivity classifications (restricted, confidential)
  12. Graph Explorer — Renders entity graph with filters for presets, entity types, and layout options. Legend shows node types (Identity, Workload, Role, Permission, Resource, Connection, Execution Evidence)
  13. Execution Chains page (new) — Workload-level summaries with egress, ownership, max sensitivity, sensitive domains, entity count
  14. Chain detail page (new) — Chain summary + chain entities table with role, type, and display name
  15. Identities page — 10 identities with type, source, status, sensitive domains, sortable columns
  16. Syncs page — Sync history with connector name, status, started, duration, entities, events
  17. Settings page — Tenant configuration, recent syncs summary, platform version (v0.2 W1), evaluator rules count (12), entity types count (9)
  18. Progressive disclosure flow — Overview -> Risk Clusters -> Cluster Detail -> Path Table -> Path Detail operates correctly
  19. Ownership section — Shows "Maria Lopez" name (resolved from evidence), "Not assigned" state, "service principal owner departed" context

Delta vs Round 1

Fixed (Round 1 -> Round 2)

Round 1 DefectResolution
D3 — Impact scores invertedFIXED. Impact scores removed entirely (PR #89). Remediation actions now show as an ordered list without bars. Matches Sergey's "remove, don't invert" decision.
D9 — Remediation includes non-operator advicePARTIALLY FIXED. The "Implement DLP controls" action appears to be gone from the visible remediation list. Actions now focus on scope restriction and ownership assignment.
D13 — Nav items missing Execution Chains and ExposuresNOT FIXED but pages now exist. Execution Chains and Exposures pages render correctly and have detail pages. But they are still not in the sidebar navigation.

Improved but Not Resolved

Round 1 DefectCurrent State
D1 — Remediation applies_to uses generic termsPARTIALLY FIXED. Some actions now reference specific identities (e.g., "svc-foundry-ascribe-prod > LLM Egress"). Others still use condition labels ("Scope drift", "Install owner"). Mixed specificity. Carried forward as Round 2 D1.
D2 — Authority path collapsing hides total role countPARTIALLY FIXED. Path table rows show per-path role pills. No identity-level role aggregation badge visible. Moved from Missing to Partial. Addressed in consolidated action plan 0.2 but not complete.
D7 — Ownership section hardcodes generic textPARTIALLY FIXED. Now shows "Maria Lopez" name, which was not visible in Round 1. But "service principal owner departed" suffix still appears generic rather than dynamically resolved.
D8 — Breadcrumbs show hash IDsNOT FIXED. Breadcrumbs still show truncated hashes on all detail pages. Finding detail breadcrumb is worse — shows full eval-prefixed ID. Carried forward as Round 2 D4.

New Defects (Not in Round 1)

New DefectSeverityNotes
D2 — Scope drift sensitive cluster disabledCRITICALNew regression. This cluster page returns an error that did not exist in Round 1.
D3 — Exposure detail "Entity not found"MAJORNew page, new defect. The Exposures feature was added but detail pages are broken.
D7 — Finding breadcrumb shows full eval IDMINORThe breadcrumb formatting is worse than Round 1's truncation.
D8 — Evidence Completeness shows N/AMINORFinding detail's evidence section has empty fields that undermine recommended actions.
D11 — Temporal Compare empty shellMINORNew page with no usable content or guidance.

Round 1 -> Round 2 Scorecard

MetricRound 1 (March 15)Round 2 (March 19)TargetMet?
Implemented1719--+2
Partial86<=2No (6 remaining)
Missing210No (1 remaining)
Diverged120No (2, worse)
Critical defects320Improved
Major defects540Improved
Minor defects650Improved
Total defects1411---3

Summary of Delta

Progress: 3 fewer defects. Impact score inversion (the cleanest fix — remove entirely) is resolved. Remediation specificity improved partially. Two new functional pages added (Execution Chains, Exposures). Path detail remediation section is visually cleaner without impact bars.

Regressions: Two new broken pages (scope_drift_sensitive cluster, exposure detail). Finding breadcrumb got worse (full eval ID instead of truncated). These are net new defects that did not exist in the Round 1 snapshot.

Gap to target: The target was 0 missing and 2 or fewer partial. Current state is 1 missing and 6 partial. The primary blocker remains remediation specificity (Phase 0.1) and authority path role aggregation (Phase 0.2) — both Phase 0 demo blockers from the consolidated action plan that are partially but not fully addressed.


Release Readiness

  • Not ready
  • Ready for internal demo
  • Ready for design partner demo
  • Ready for broader pilot

Justification: The platform is stronger than Round 1 and the internal demo experience has improved (cleaner remediation, no misleading impact bars, new Execution Chains page). However, three issues prevent advancing to design partner readiness:

  1. Broken cluster page (D2) — The scope_drift_sensitive cluster detail shows an error. In any live demo, a stakeholder clicking through clusters would hit this dead end. This is a hard blocker.

  2. Remediation target mixed specificity (D1) — Some actions now name specific identities, but others still use condition labels. The inconsistency is arguably worse than uniformly generic — it suggests the system can resolve names but chooses not to for certain actions. Phase 0.1 acceptance criteria are not met.

  3. Navigation orphans (D5) — Execution Chains is the best new page in the platform, but it is invisible in the sidebar. A demo would never discover it organically.

Path to design partner readiness:

  • Fix the disabled cluster error (D2) — either fix the cluster or remove it from the list
  • Complete remediation target resolution for ALL actions (D1) — no condition labels in applies_to
  • Add Execution Chains and Findings to sidebar (D5)
  • Fix breadcrumbs to show display names (D4)

These four items would move the platform to "Ready for design partner demo." Execution confidence labels (missing feature #24) and evidence completeness improvements can follow.