Critical Review and Architectural Decisions
Date: 2026-02-07 Input: Architecture and Modeling Review, ServiceNow Evidence and Automation Research (both 2026-02-07) Output: Decisions on which suggestions to accept, reject, or modify, with rationale
1. Decisions Summary
Accepted as P0 (Must Fix)
1.1 Source Metadata Redaction Policy (Elevated from P1)
Problem: BaseEntity.raw_data stores arbitrary source API responses. Risk: secrets in headers, PII, regulated data violating metadata-only constraints. This is a direct data exposure/control risk — if any connector stores sensitive headers/tokens/PII before redaction is enforced, the metadata-only constraint is violated from day one.
Decision: Rename raw_data / raw_api_response to source_metadata with:
- Connector-defined allowlist of fields to persist per entity type
source_metadata_hash(SHA256 of full original response) for integrity verification- Fields not on allowlist excluded before storage
Rationale: Originally classified as P1 (scale optimization), but this is a pre-live control. If any connector ingests live data before allowlist enforcement is in place, sensitive data reaches the database. Unlike scale concerns that can be addressed reactively, data exposure cannot be un-done after the fact. Elevated to P0.
Enforcement: Must be implemented before first live connector ingest. Connectors MUST NOT be able to persist entity data without passing through the allowlist filter.
Impact: Updates to 03-database.md (entity examples), 01-data-model.md (policy documentation).
1.2 Baseline Storage Redesign
Problem: Current baselines collection stores all entities from a source system in a single document's entities: [] array. A tenant with 500 identities, 200 roles, 1000 permissions, and their relationships/properties easily exceeds MongoDB's 16MB document limit.
Decision: Split into two collections:
baseline_metadata— header document with entity counts, integrity hash, schedule infobaseline_entities— one document per entity per baseline, keyed by(baseline_id, entity_source_id)
Rationale: This is correct-by-design, not premature optimization. Even small tenants with a few hundred entities across multiple types can approach the limit when relationships and properties are included. Per-entity baseline documents also enable partial retrieval (diff a single entity's baseline without loading the full snapshot).
Impact: Updates to 03-database.md (collection schema, indexes, point-in-time queries).
1.3 ExecutionEvidence as First-Class Entity
Problem: The connector schema (05-connectors.md) already defines execution_evidence as a NormalizedNodeType, but the internal data model has no corresponding entity type. EXECUTES_ON edges exist but without immutable evidence records linking to source log entries. This means "this identity executed X" is a claim without proof.
Decision: Add execution_evidence as:
- An entity type in the data model (
01-data-model.md) - A new collection in the database schema (
03-database.md)
Required properties:
source_table— where the evidence came from (e.g.,syslog_transaction,sys_flow_context,signIns)source_record_id— ID in the source system for verificationsource_timestamp— when the execution actually occurredevidence_type— api_call, flow_execution, scheduled_job, sign_inpayload_hash— SHA256 of source record content (integrity without storing raw data)action— what was done (e.g., "POST /api/now/table/incident")target_resource— what was acted uponoutcome— success, failure, unknown
Rationale: The PRD requires evidence-grade findings. Execution claims must be backed by immutable, verifiable records. The connector layer is already prepared to produce these; the internal model just needs to receive them.
Impact: Updates to 03-database.md (new collection + indexes), 01-data-model.md (new entity type).
1.4 Deterministic Link Evidence for AUTHENTICATES_TO
Problem: AUTHENTICATES_TO relationships carry auth_method, established_at, last_used_at, and auth_chain_depth. But they don't store the actual matching proof — which specific identifiers from which specific tables were used to establish the cross-system link.
Decision: Add evidence_references to AUTHENTICATES_TO relationship properties:
| Field | Required | Description |
|---|---|---|
issuing_system_id | Yes | Identifier in the issuing system (e.g., Entra SP appId) |
issuing_tenant_id | Yes | Tenant/directory context in the issuing system (e.g., Entra tenant ID). Required to disambiguate in multi-tenant setups. |
target_system_id | Yes | Identifier in the target system (e.g., ServiceNow oauth_entity.client_id) |
target_instance_id | Yes | Instance/environment context in the target system (e.g., ServiceNow instance URL or sys_id). Required to disambiguate multi-instance deployments. |
target_record_sys_id | No | Specific record ID in target system (e.g., oauth_entity sys_id). Enables direct verification. |
matching_field | Yes | Which field was used for deterministic matching (e.g., "client_id") |
matching_value | Yes | The actual value that matched |
target_user_binding | No | How the OAuth identity maps to a user (e.g., "oauth_entity.user -> sys_user.user_name") |
Rationale: Cross-system execution paths are a core differentiator. The linkage proof must be walkable in an evidence pack. Without stored matching evidence, the link is an assertion rather than a provable fact. Client_id matching alone is insufficient in multi-tenant or multi-instance setups — the issuing tenant and target instance must be captured to prevent ambiguous cross-system links.
Impact: Updates to 01-data-model.md (AUTHENTICATES_TO properties), 05-connectors.md (edge properties).
1.5 Evidence Completeness Flags (Elevated from P1)
Problem: Findings implicitly claim evidence quality without declaring what was available. Example: ScopeDriftFinding has approval_record_exists: boolean, which implies the system checked for approval. But if the approval system wasn't accessible, this boolean is misleading.
Decision: Add evidence_completeness as a required field on all findings. Each evidence category declares its availability status with explanatory notes.
EvidenceAvailability values:
available— source accessible, data retrievedunavailable_not_enabled— source exists but feature not enabled (e.g., sys_audit_role)unavailable_no_access— source exists but connector lacks permissionsunavailable_not_applicable— category doesn't apply to this source systempartial— some data available but incomplete (e.g., retention window expired)
Evidence categories: current_roles, role_history, execution_evidence, ownership_records, approval_records, credential_state
Notes map: each category can have a human-readable explanation (e.g., "role_history": "sys_audit_role not enabled in target instance")
Impact on finding text: When evidence is unavailable, deterministic_explanation MUST state this. Example: "Role change history unavailable (sys_audit_role not enabled); current role state from sys_user_has_role shows 4 roles assigned." NOT: "4 roles were added without approval."
Rationale: This is core to the "deterministic, no inference" constraint. A finding that honestly declares its evidence gaps is more credible than one that silently omits them. Elevated from P1 because it affects finding integrity, not just scale.
Impact: Updates to 01-data-model.md (finding schema), 05-connectors.md (connector output includes evidence report).
Accepted as P1
1.6 Externalization Strategy Documentation
Problem: execution_paths and accessible_by arrays embedded in entity documents could theoretically exceed 16MB for high-fan-out identities.
Decision: Document the design for externalization but do NOT implement it for MVP.
Size analysis: Each ExecutionPath entry is ~200-500 bytes. An entity would need >30,000 paths to approach 16MB. At MVP scale (<1,000 identities), this is unrealistic.
Monitoring — size-based triggers: Track document sizes via Object.bsonsize() in sync post-processing. Log warnings at >8MB or >1,000 paths per entity.
Monitoring — operational triggers: Size thresholds alone are incomplete. Write amplification and sync latency can become the real bottleneck before document size limits are hit. Also monitor:
- Path recompute duration per sync (p95) — threshold: >30s for a single entity's paths
- Sync throughput degradation — threshold: >2x increase in total sync duration attributable to path materialization
- Write amplification ratio — threshold: >10 resource documents updated per identity path change (indicates high fan-out)
Fallback design: execution_paths_overflow and accessible_by_overflow collections keyed by (tenant_id, entity_id).
Rationale: The math doesn't support P0 urgency for size-based concerns. However, operational performance degradation may trigger externalization before size limits are reached. Documenting both trigger types ensures the team can act on whichever signal fires first.
Impact: Updates to 03-database.md (new subsection).
Partially Accepted
1.7 Ownership Model — Documentation Clarification Only
Original claim: "Connector schema lacks an Owner node type."
Assessment: Misleading. NormalizedNodeType includes human_identity. NormalizedEdgeType includes OWNED_BY and BELONGS_TO. Connectors extract what source systems provide (users, groups, teams). The normalizer layer determines which human_identity nodes become Owner entities based on OWNED_BY/BELONGS_TO edges.
Decision: Do NOT add owner as a NormalizedNodeType. Instead, document the human_identity -> Owner mapping clearly in 05-connectors.md.
Rationale: Ownership is a platform concept derived from connector-provided relationship data, not a source-system entity type. Adding owner to the connector schema would blur the boundary between extraction and interpretation.
Impact: Updates to 05-connectors.md (normalization note).
1.8 Automation Artifacts — IdentitySubtypes, Not New Entity Types
Original claim: Model 6 new entity types for ServiceNow automations (Flow Designer flows, Business Rules, Script Actions, Scheduled Jobs, Script Includes, Workflow Activities).
Assessment: Premature. ServiceNow connector doesn't exist yet. Automations participate in the same graph as other identities — they hold roles (or inherit via run-as), execute on resources, and have ownership chains.
Decision: Add automation subtypes to the Identity entity:
flow_designer_flowbusiness_rulescheduled_jobsystem_execution(ServiceNow "System" identity — privileged NHI for automations with no user context)
Define new relationship types:
RUNS_AS— automation -> identity (which identity the automation executes as)TRIGGERS_ON— automation -> resource/event (what triggers the automation)CREATED_BY— entity -> human (who created this, distinct from OWNED_BY)
Rationale: Subtypes preserve existing graph traversal logic. Execution paths work identically: automation -[RUNS_AS]-> identity -[HAS_ROLE]-> role -[GRANTS]-> permission -[APPLIES_TO]-> resource. If real connector work reveals automations need their own entity type, that decision can be made with data.
Impact: Updates to 01-data-model.md (Identity subtypes), 05-connectors.md (edge types).
Not Accepted
1.9 Six New Automation Entity Types — Rejected
Rationale: Adding 6 entity types before the ServiceNow connector exists is speculative architecture. IdentitySubtype values achieve the same modeling goal without new traversal logic. Revisit when connector is actively built.
1.10 Immediate execution_paths Externalization — Deferred
Rationale: At <1,000 identities, the embedded approach is safe. Documented the fallback design for when monitoring indicates need.
2. Open Questions — Answered
Q1: Which ServiceNow tables approved as authoritative evidence in MVP?
Always available (standard tables):
sys_user_has_role— current role assignments (authoritative current state)oauth_entity— OAuth application registry (identity linkage proof)sys_user— user records (identity resolution)
Conditional (requires instance configuration):
syslog_transaction— inbound REST execution evidence (may require permissions/allowlisting)sys_audit_role— role change history (requiresglide.role_management.v2.audit_rolesenabled)sys_flow_context— Flow Designer execution records
Connector health check must probe availability of conditional tables and report status via evidence completeness flags.
Q2: Is sys_audit_role enabled? Is syslog_transaction accessible?
Answer: Instance-dependent. Cannot be answered architecturally.
Architectural response: Design for both cases. Connector health check probes for table accessibility. Evidence completeness flags propagate to findings. Findings say "role change history unavailable (sys_audit_role not enabled)" rather than implying a check was done.
Q3: How will Entra appId map to ServiceNow oauth_entity deterministically?
Answer: Match on oauth_entity.client_id = Entra Service Principal appId. These are the same OAuth client_id presented during token requests.
For JWT endpoints: ServiceNow's OAuth JWT endpoint configuration includes a User Field that maps the JWT sub claim to a field in sys_user (e.g., email). This deterministically binds tokens to a ServiceNow user record.
For non-JWT endpoints: A dedicated ServiceNow integration user is typically created and associated with the OAuth application record.
Storage: Both source IDs stored in AUTHENTICATES_TO evidence_references along with issuing tenant context (Entra tenant ID) and target instance context (ServiceNow instance URL/sys_id). Client_id matching alone is insufficient for multi-tenant or multi-instance setups — the full issuer/tenant/instance tuple must be captured.
Q4: What scale thresholds trigger externalization?
Baselines: Externalize from day 1 — the per-entity baseline design is the correct architecture, not a threshold-based migration.
execution_paths / accessible_by: Monitor both size-based and operational triggers:
- Size-based: >1,000 paths per entity OR >8MB document size
- Operational: p95 path recompute >30s per entity, >2x sync duration increase from path materialization, write amplification ratio >10
Either trigger type initiates migration to dedicated overflow collections. Implement monitoring in sync post-processing.
Q5: What approval system is authoritative for scope change evidence?
Answer: Customer-specific. Could be ServiceNow Change Requests (CHG), Jira tickets, or other systems.
Architectural response: Model via existing APPROVED_BY relationship that can link to any source system's approval records. When no approval record is found, the finding states: "No approval record found in [sources checked: servicenow_chg_task, jira_issue]" — NOT "unapproved change" (that would be inference).
Next Action
Status: adopted — shipped
Findings incorporated into docs/architecture/01-data-model.md, 05-connectors.md, and associated ADRs. No further action required.