🚧 Early alpha — building the foundation. See the roadmap →

Ontology-web query engine — synthesis (in progress)

Created May 7, 2026 Updated Jun 1, 2026

All 9 architectural decisions decided as of 2026-05-08 — see §2 for full reasoning per decision. Eight of nine pending research challenges have landed deliverables as of 2026-05-09 (see §3.5 for verdict summaries):

Ch 29 (primitives adversarial validation) — REVISES Settled #14 from 7→8 primitives. New canonical set: {filter, traverse, bind, project, aggregate, anti-join, set-op, diff}. Drops standalone closure (now traverse(depth=*)); demotes pivot to Layer B; adds bind/set-op/diff.
Ch 30 (view shape taxonomy) — refines v0.1 first-class catalog to 5 shapes (Table/List/Cards Bases-native + Pivot custom + Hierarchy as 2nd custom view in v0.1.7–v0.1.8); heatmap/sunburst/treemap/sankey are render options of Pivot, not separate shapes.
Ch 31 + Ch 32 — already absorbed in earlier passes (recipe query: block schema; Hybrid Pattern D + embedded base blocks default).
Ch 33 (multi-modal substrate audit) — REAFFIRMS sqlite-wasm; mobile (Capacitor: no SAB, no OPFS-sync) is the binding constraint, not scale. Adds new triggers (sqlite-vec-wasm packaging stability watch).
Ch 35 (graph→tabular rerun, Ch 10) — REAFFIRMS Tier 1+2; REVISES Tier 3 to Oxigraph (embedded/sidecar) + Fuseki SPARQL stack. Recommends moving SSSOM TSV import + materialized closure-table to v0.1.6 (proposed scope expansion — pending verdict).
Ch 36 (query language rerun, Ch 12) — REVISES Ch 12 narrowly toward compositional stack (YAML recipes → SQL CTE → JSON view-specs → Bases YAML). Datalog stays internal-only as a rule sub-DSL inside YAML. LLM-friendliness: SQL ≈ 47% > Cypher ≈ 34% > MQL ≈ 22% >> SPARQL ≈ 3%.
Ch 37 (Tier 2 scale rerun, Ch 18) — REAFFIRMS ~100K mapping ceiling, reframed as per-query working-set (not database size). Green tier ≤50K concepts/250K edges; Tier 3 referral above ~500K concepts. Per-ontology partitioning + bounded LRU closure cache + tripwire estimation = v0.1.7.

Only Ch 34 (streaming/chunked execution) is still pending — informs v0.1.7+ scale path; not a v0.1.6 blocker.

v0.1.6 implementation can proceed now on Settled #1–18 (see §1) + decided D1–D9 + the locked D1 “Ch 35 nuance” scope expansion (SSSOM TSV import + materialized closure-table — locked 2026-05-09 per user “might as well do it earlier”). The “(in progress)” title suffix gets dropped when Ch 34 lands and a final update absorbs it; other deliverables now absorbed inline.

This update absorbs Ch 29, 30, 33, 35, 36, 37 deliverables. The high-level picture is unchanged in shape but sharpened in detail:

The three-layer engine (primitives → view shapes → recipes) stands; Layer A grew from 7→8 primitives; Layer B catalog refined from 6→5 first-class for v0.1.
Substrate (sqlite-wasm) REAFFIRMED across both Ch 33 + Ch 37; no engine swap.
Compositional language stack REAFFIRMED (Ch 36); no Crosswalker DSL; Datalog internal-only.
Tier 3 clarified as Oxigraph + Fuseki SPARQL stack (Ch 33 + Ch 35), defer to v0.1.8+.
One genuine scope-decision PROPOSED but not locked: move SSSOM TSV import + materialized closure-table from v0.1.8 → v0.1.6 (per Ch 35) — see D1 §“Ch 35 nuance” callout.

Verdicts now folded into Settled #1–18 (was #1–14); §3.5 carries per-deliverable summaries; §9 status table flipped from ⏳ to ✅ for the absorbed challenges.

How Crosswalker works (the 30-second picture)

Crosswalker imports ontology data — compliance frameworks (NIST, ISO, MITRE), biomedical taxonomies (GO, HPO), library hierarchies (LCSH), or anything else with concepts and relationships — into an Obsidian vault as Markdown notes. You then query that data through Obsidian’s built-in Bases plugin OR through Crosswalker’s custom Bases views.

┌──────────────┐    ┌────────────────────┐    ┌──────────────────┐    ┌────────────────────┐    ┌─────────────────┐
│  YOU IMPORT  │ →  │ CROSSWALKER WRITES │ →  │ SQLITE CACHE     │ →  │ YOU QUERY VIA      │ →  │ OUTPUT:         │
│              │    │ MARKDOWN NOTES TO  │    │ SPEEDS UP        │    │                    │    │                 │
│  CSV/XLSX/   │    │ YOUR VAULT:        │    │ QUERIES THAT     │    │ Obsidian Bases     │    │ Live views      │
│  JSON +      │    │ • concept notes    │    │ NEED:            │    │   (default surface) │    │   (recomputed   │
│  YAML recipe │    │ • junction notes   │    │ • closure walks  │    │ OR                 │    │    on demand)   │
│              │    │   (= one note per  │    │ • multi-table    │    │ Crosswalker's      │    │ OR              │
│              │    │    crosswalk      │    │   joins          │    │ custom Bases view  │    │ Frozen Markdown │
│              │    │    relationship)   │    │ • set ops        │    │ ("crosswalkerPivot")│   │ snapshots       │
│              │    │ • edges as        │    │ • pivots         │    │ OR (deferred)      │    │   (for audit /  │
│              │    │   wikilinks        │    │                  │    │ inline `crosswalker│    │    Publish /    │
│              │    │                    │    │ Rebuildable —    │    │ -query` codeblocks │    │    sharing)     │
│              │    │ THIS IS THE        │    │ NEVER the truth  │    │                    │    │                 │
│              │    │ SOURCE OF TRUTH    │    │                  │    │                    │    │                 │
└──────────────┘    └────────────────────┘    └──────────────────┘    └────────────────────┘    └─────────────────┘

Two principles that govern every decision below:

The Markdown vault is the source of truth; the sqlite cache is derived. Junction notes are committed to git; pivot tables are caches that rebuild from junctions. Don’t conflate them.
Mobile (Obsidian iOS/Android via Capacitor) is the binding constraint. Anything that doesn’t run on mobile gets gated to desktop-only or deferred.

Vocabulary used throughout this log

Term	Plain-language meaning
Recipe	A YAML file in `_crosswalker/recipes/` that describes how to import a framework AND optionally how to query it. Versioned, shareable, JSON-Schema-validated.
Junction note	A Markdown file representing ONE crosswalk relationship (e.g., “NIST AC-2 maps to ISO A.5.16 with confidence 0.8”). First-class typed-edge entity, NOT just a wikilink.
Bases	Obsidian’s built-in query plugin (since 1.9). Reads frontmatter, provides Table/Cards/List/Calendar/Map views. What Crosswalker extends instead of replaces.
`registerBasesView`	Obsidian API (1.10+) for a plugin to register a NEW view-type into Bases’ view picker. Crosswalker registers exactly one for v0.1.6: `crosswalkerPivot`.
8 query primitives	The verbs Crosswalker queries are built from: `filter`, `traverse`, `bind`, `project`, `aggregate`, `anti-join`, `set-op`, `diff`. You don’t write them directly; recipes compose them.
5 view shapes (v0.1)	What query results LOOK like: Table, List, Cards (all Bases-native) + Pivot (custom view) + Hierarchy (custom view, v0.1.7-v0.1.8).
Tier 1 / Tier 2 / Tier 3	The substrate stack. Tier 1 = Markdown vault (truth). Tier 2 = sqlite-wasm cache (derived). Tier 3 = optional external SPARQL server for ontology-web-scale data; defers to v0.1.8+.
CQRS split	”Live read” features (querying) live separately from “Frozen snapshot” features (audit/share exports). Don’t conflate them.
Mobile / Capacitor	Obsidian iOS/Android runs via Capacitor (a WebView wrapper). It lacks SharedArrayBuffer + OPFS sync handles, so desktop-only sqlite features need fallbacks.

Where each Settled item lives in the picture

In the §1 table below, each row’s “Lives in:” tag points to one part of the diagram above:

Storage — Tier 1 Markdown OR Tier 2 sqlite cache; where data physically sits
Query — how queries work (the 8 verbs, the 5 view shapes, the custom Bases view)
Recipe — the YAML grammar / authoring UX (input side)
Output — exports, snapshots, audit trail (CQRS frozen-snapshot side)
Cross-cutting — applies to multiple parts (mobile parity, vocabulary discipline, the language stack)

Why this exists

The v0.1.6 Bases query layer milestone has two pre-requisite research challenges:

Ch 27 — Bases query layer architecture — 3 deliverables: 27a, 27b, 27c
Ch 28 — Bases query layer follow-on stress test — 3 deliverables: 28a, 28b, 28c

This page consolidates the verdicts and tracks the open decisions before v0.1.6 implementation begins.

§1 Settled — locked architectural commitments

All 18 commitments below are LOCKED. No user verdict needed; they govern v0.1.6 implementation and downstream milestones. Each row is anchored to one or more axes from the Architectural map above so you have a coordinate system. Acronyms are expanded inline.

The walkthrough below is for context-mapping (so you know where each commitment lives in the bigger picture), not approval. The only Settled-adjacent thing still open is the D1 “Ch 35 nuance” PROPOSED scope expansion (move SSSOM TSV import + materialized closure-table v0.1.8 → v0.1.6) — flagged separately at D1.

#	Commitment (acronyms expanded inline)	Lives in	Source
1	Hybrid architecture — Bases (Obsidian’s built-in query plugin since 1.9) is the default user-facing surface; SQL access through the sqlite-wasm cache is the escape hatch. We don’t replace Bases — we teach it new tricks via custom views (`registerBasesView`).	Query (default) + Storage (cache)	Ch 27 a/b/c + Ch 28 a/c
2	One custom Bases view in v0.1.6 — registered as `crosswalkerPivot` (parameterized by recipe). v0.1’s full view-shape catalog is 5 shapes: Pivot (this custom view) + Table / List / Cards (Bases-native; we don’t reimplement) + Hierarchy (graduates v0.1.7-v0.1.8 as our 2nd custom view). “Coverage Matrix” is the launch-market recipe, not the view name.	Query	Ch 27a + Ch 28 a/c (renamed from `crosswalkerCoverageMatrix`); Ch 30 adds Cards
3	`_crosswalker/` underscore folder for plugin-emitted files. Obsidian doesn’t index dot-prefix folders, so we use underscore. Sub-paths: `_crosswalker/views/` (`.base` files), `_crosswalker/audit/` (snapshots), `_crosswalker/recipes/` (YAML).	Storage	Ch 28 a/c
4	Mobile + Obsidian Publish parity is the binding constraint. Obsidian Mobile uses Capacitor — which lacks SharedArrayBuffer + sync OPFS. Obsidian Publish only renders static HTML, so plugin code doesn’t execute there. Every architectural choice has to clear this gate. (See #18 for the engineering specifics.)	Cross-cutting	Ch 28 a/c
5	`metadataCache.resolvedLinks` for edge resolution — use Obsidian’s built-in link-graph index. Don’t string-match `[[wikilink]]` patterns. Don’t build a custom projector.	Storage (cache build-up)	Ch 27c + Ch 28 a/c
6	Junction-note schema denormalizes subject/object IDs for fast Bases filtering. Junction notes = one Markdown file per crosswalk relationship. Launch-market field names: `control_id` + `framework`. Schema is domain-neutral; the field NAMES carry GRC vocabulary but the structure works for any ontology pair (NIST↔ISO, MITRE-ATT&CK↔CIS, SKOS↔OLIR, etc.).	Storage (the typed-edge format)	Ch 27c + Ch 28 a/c
7	Manifest-first OpenTimestamps audit model — one cryptographic timestamp file (`.ots`) per snapshot manifest, not per row. Proves “this audit report existed at this point in time” without per-row Bitcoin-blockchain bloat.	Output	Ch 28 a/c
8	Determinism is non-negotiable for #7 to be meaningful — stable row/key ordering, byte-identical regeneration. Two runs of the same recipe at the same data must produce identical bytes.	Output	Ch 28 a/c
9	Junctions = audit truth; pivot tables = caches. Junction notes are the durable source of truth (Tier 1 Markdown, committed to git). Pivot tables are caches/projections (rebuilt from junctions). Don’t conflate them — a pivot is not evidence; the junctions it summarizes are.	CQRS split	Ch 28c explicit + Ch 28a CQRS framing
10	Recipe lifecycle: JSON Schema validation + provenance + `user_edited:true` flag + three-way merge for conflicts. Recipes are versioned YAML files in `_crosswalker/recipes/`; Crosswalker tracks which fields the user edited so plugin updates don’t clobber user changes.	Recipe	Ch 28 a/c
11	TaskNotes v4 is the canonical Obsidian-plugin precedent for the pattern “delete bespoke filter UI; let Bases own filtering; register a custom view-type for the bits Bases can’t do.” TaskNotes ships 4 custom views via `registerBasesView`. Read its source before writing `crosswalkerPivot`.	Query (implementation pattern)	Ch 27a + Ch 28 a/c
12	Three-layer engine vocabulary: Layer A = the 8 query verbs (`filter`/`traverse`/etc.); Layer B = the view shapes (Table/Pivot/Hierarchy/etc.); Layer C = the recipes (YAML that composes A + B). Recipes are the marketplace artifact users share. See `concepts/query-primitives` + `concepts/view-shapes`.	Cross-cutting (the vocabulary itself)	2026-05-08 alignment review
13	Crosswalker is a general ontology-web query engine; compliance (GRC = governance/risk/compliance) is the launch market. Internal vocabulary stays general (ontology/concept/subject/object); user-facing surfaces (UI, README, settings) stay GRC-first (control/framework/evidence/coverage). Recipes carry domain naming; the query + view layers stay domain-neutral.	Cross-cutting (vocabulary discipline)	`feedback_general_ontology_positioning.md`; 2026-05-08 alignment
14	The 8 query verbs (Layer A): `filter` / `traverse` / `bind` / `project` / `aggregate` / `anti-join` / `set-op` / `diff`. Closure is parameterized into `traverse(depth=)` (same shape as SPARQL `r`, Cypher `[1..n]`, Datalog recursion). Pivot demoted to Layer B (see #15) because it’s presentation, not value-producing. Three additions vs the prior 7-candidate set: `bind`* (computed columns — required for queries like “evidence > 1 year old”), `set-op` (∪/∩/⊖ — required for “controls in BOTH NIST and CIS”), `diff` (versioned ontology delta — required for v0.1.8 audit-trail to detect added/removed/strengthened concepts). Anchored to ontology-web standards (SPARQL 1.1, Codd relational algebra, Datalog, OLAP, GQL ISO/IEC 39075:2024, SSSOM, SKOS, NIST OLIR), not GRC-coded. Works for any ontology web (biomedical OBO, library taxonomies, ATT&CK, GRC).	Query (the verbs)	Ch 29 (2026-05-09)
15	What’s a verb vs a view shape: verbs (Layer A) change the data — they filter, walk edges, group, count, subtract sets, etc. View shapes (Layer B) just decide how to draw the result — pivot grid, indented tree, table, list, cards. Pivot/sort/limit/top-k all live on the “draw” side, not the “transform” side. Heatmap, sunburst, treemap, sankey, circle-packing are different ways to render Pivot or Hierarchy — NOT separate shapes. Search = `filter + bind(rank)`; comparison = two Tables joined; dashboard = a recipe (Layer C) wiring multiple views together.	Query (the boundary)	Ch 29 §4 + Ch 30 §6
16	Tier 3 = Oxigraph + Apache Jena Fuseki SPARQL servers. When the user has more ontology data than fits in browser memory — federating to BioPortal/OxO public SPARQL endpoints, or local SPARQL over a multi-million-triple OWL file (UMLS = 3.49M concepts; BioPortal = 1,549 ontologies / 100M+ mappings) — Crosswalker can talk to an external SPARQL server. Preferred: Oxigraph (Rust binary, embeds OR runs as sidecar HTTP server). Alternative: Apache Jena Fuseki for Java-ecosystem users. Apache AGE (Postgres graph extension; previously demoted in Ch 16) is formally de-listed. Stardog/GraphDB/Neo4j/Datomic/HelixDB/SurrealDB/Materialize/TerminusDB excluded (license / vendor concentration / server-only). Ships v0.1.8+ only when a concrete user need triggers it.	Storage (Tier 3 escape hatch)	Ch 33 + Ch 35
17	Compositional language stack — we don’t invent a query language. Recipe authors write YAML (the recipe `query:` block — domain-neutral structured fields). The engine compiles those blocks to SQL recursive CTEs (Common Table Expressions — the SQL standard for transitive closure) running on sqlite-wasm. Layer A → Layer B passes JSON view-specs. Vault-scoped views use Bases YAML/expressions (Obsidian Bases’ native query syntax). Datalog stays as a rule sub-DSL inside YAML `rules:` blocks (compiles to SQL CTE; no Datalog runtime ships to users). What we DON’T do: invent a “Crosswalker DSL” / “CrosswalkerQL” — a custom query language users would have to learn. That’s the dbt-before-YAML+SQL-won mistake; the Datomic dialect-lock-in mistake. Anchored by: LLM authoring benchmarks (zero-shot: SQL ≈ 47% > Cypher ≈ 34% > MongoDB ≈ 22% >> SPARQL ≈ 3%); industry precedent (dbt, OpenRewrite, Argo Workflows, Kubernetes — all use YAML on top of an established engine).	Cross-cutting (recipe → query → output)	Ch 36
18	The mobile constraint, in engineering terms (this is #4 made concrete). Obsidian Mobile uses Capacitor (a WebView wrapper for hybrid apps). Capacitor doesn’t expose `SharedArrayBuffer` (the browser threading primitive desktop sqlite-wasm uses for OPFS sync handles), and `OPFS` (Origin Private File System — the browser’s disk-backed file API) is async-only on mobile. Every alternative WASM engine (DuckDB-WASM, Cozo-WASM, Oxigraph-WASM, Nemo-WASM) inherits the same wall. Resolution: `@sqlite.org/sqlite-wasm` (OPFS desktop) + `wa-sqlite` (IndexedDB / AccessHandle mobile) behind a single cache abstraction. Mobile budget ≈ 1/5 of desktop. This gates every Tier 2 feature; any v0.1.7+ scale work has to clear it.	Storage (sqlite cache)	Ch 33 + Ch 37

§2 Decisions to make (D1–D9)

Each decision below is self-contained — read the option summaries inline; the Source links go to verbatim deliverables for deeper context. Decisions with the same options re-use shared vocabulary so you don’t have to scroll back.

🗺️ Where these decisions sit in the Crosswalker project flow

📥 INPUT          🗂️ STORAGE              ⚙️ PROJECTION         🔍 QUERY LAYER       📤 OUTPUT
(CSV/XLSX/JSON   (Tier 1: Markdown    (Tier 2: sqlite-wasm  (Bases views,        (Exports +
 frameworks +     vault — concepts,    sidecar — derived     custom Bases views,  materialized
 mappings)        junction notes,      cache, rebuildable)   codeblock)           snapshots)
                  crosswalk edges)
   IMPORT WIZARD     —↑ already shipped (v0.1.1–v0.1.5) ↑—   THIS MILESTONE       v0.1.8 lands
   recipe-driven                                              IS ABOUT THIS        materialization

Each decision below tags 📍 Stage so you know where it sits in this flow. Most v0.1.6 decisions shape the query layer; D1 + D3 also shape the output side.

Vocabulary cheat-sheet (referenced repeatedly below)

Term	Plain-language meaning
Materialization / “snapshot”	Generate static `.md` files containing pre-rendered query results. Plugin runs the query, writes results to disk as Markdown. Survives plugin uninstall AND Obsidian Publish (since static Markdown is all Publish renders).
Codeblock processor / `crosswalker-query`	Lets users write inline ```crosswalker-query ``` blocks anywhere in note bodies. Plugin intercepts the block, runs the query, replaces it with rendered output. Plugin-runtime only — does NOT survive plugin uninstall or Publish.
`registerBasesView` / “custom Bases view”	The generic Obsidian API (1.10+) for plugins to register a new view-type that appears in Bases’ view-picker dropdown alongside built-in Table/Cards/Calendar. One registration per view shape — v0.1.6 ships exactly one (`crosswalkerPivot`, for the pivot shape) because v0.1.6 is scope-bounded. v0.2+ adds siblings as needed: `crosswalkerGraph` (graph topology), `crosswalkerHierarchy` (tree/DAG), `crosswalkerTimeline` (temporal projection), etc. — each is a separate `registerBasesView` call. The full shape catalog is pending Ch 30. User selects which view-type to render per-`.base`-file via Bases UI. Plugin-runtime only — doesn’t survive plugin uninstall or Obsidian Publish today.
Tier 2 SQL helpers	Already shipped in v0.1.5 P3: `plugin.queryConcepts`, `plugin.queryCrosswalk`, `plugin.queryClosure`. Run SQL against the sqlite-wasm sidecar. Currently internal-only (no user-facing SQL surface).
Recipe `query:` block	A new YAML section in `recipe.schema.json` letting recipe authors declare what to query (axes, edges, aggregation) without writing SQL. Sketch: `query: { shape: pivot, rows: ..., cols: ..., cell: ... }`. Pending Ch 31 for concrete schema.

D1: Materialization timing — v0.1.6 or v0.1.8?

📍 Stage: 📤 OUTPUT side. Materialization sits after the query layer — it freezes a live query’s result into a static .md file so it can be shared with auditors / regulators / Obsidian Publish viewers / your future self (post-plugin-uninstall). Sister to the live query layer, not part of it (per D4 CQRS framing).

Plain meaning: “Materialization” = generating a static Markdown file containing pre-rendered query results, like printing a live view to a frozen .md document. The file lives in the user’s vault (e.g. _crosswalker/audit/<date>/coverage-matrix.md) with frontmatter marking it machine-generated. Same conceptual pattern as a SQL materialized view — a query that’s been computed once and stored, vs a live view recomputed on every read.

The question: Should Crosswalker generate these static .md snapshot files in the v0.1.6 milestone, or defer that capability to v0.1.8 (the audit-trail milestone)?

Option A — Ship in v0.1.6 as opt-in command-only (source: Ch 28a § Materialization Lifecycle)

Where files land: Reports/_generated/<recipe>/<view>/<timestamp>.md
When generated: only on user command (Crosswalker: Materialize view…); no auto-refresh
Why now: Obsidian Publish renders only static HTML — plugin code doesn’t run there. Materialization is the only path that survives a “share with external auditor” workflow today
Why opt-in: avoids vault pollution, sync churn, git-history bloat unless user explicitly asks

Option C — Defer entirely to v0.1.8 (audit-trail milestone) (source: Ch 28c § Materialization spec)

Why later: materialization is a snapshot/audit concern, not a query-layer concern. Conflating them in v0.1.6 bloats scope. The audit-trail milestone is the right home for lifecycle + reproducibility + OpenTimestamps integration
Why this is safe: Crosswalker doesn’t yet have a published-portal user demand pressing on v0.1.6
Junction notes are already audit truth (committed, diffable in git); matrices are caches/projections — treating a cache as audit evidence is a category error

Current lean (decided 2026-05-08, REVERSES prior lean): Option A — ship in v0.1.6, opt-in manual-command-only. Reverses the earlier “defer to v0.1.8” lean per user direction.

Why ship now:

Obsidian Publish parity is a real near-term Crosswalker user need (sharing compliance reports with external auditors who don’t have the plugin)
Plugin-uninstall durability matters even at v0.1 — GRC engagements outlive any one tool
Diff-able evidence (git diff between two snapshots) is genuinely useful for quarterly audit reviews

Constraints (per Ch 28a § Materialization Lifecycle — all carried into v0.1.6):

Manual command only (Crosswalker: Materialize view…); no auto-refresh / scheduled regeneration in v0.1.6
Files clearly marked machine-generated (frontmatter crosswalker.materialized: true + do_not_edit: true flags)
Default .gitignore template excluding materialized files (user opts in to commit)
Hand-edit prevention via timestamped new files + .latest.md alias pointer (regenerate writes a new file; never overwrites)

Open UX concern (flagged for Ch 32 — Intuitive query UX investigation): When materialized files coexist in the vault alongside canonical user files (concepts, junction notes, evidence), what happens when a user clicks on them? Does the read-experience feel coherent with Bases conventions? Does it confuse first-time users used to “all files in my vault are canonical”? The frontmatter contract addresses prevention (don’t edit), but not click-time UX (open in preview vs edit; what does the user see first?). Ch 32 brief is being updated to absorb this investigation as part of the materialization-UX scope.

Ch 35 deliverable identifies a second kind of materialization — distinct from the query-result materialization D1 already locks above. Both are now locked for v0.1.6 (per user direction 2026-05-09: “might as well do it earlier”):

Kind	What’s materialized	Lifecycle	Status
Query-result materialization (original D1)	Pre-rendered query OUTPUT (Markdown table, audit snapshot, “as of Q1 2026” frozen view)	Manual command; opt-in; for Publish/audit/share	✅ Locked v0.1.6 (2026-05-08)
Ontology-data materialization (Ch 35 — PROMOTED from v0.1.8 to v0.1.6)	Pre-computed graph DATA — closure tables (`concept_closure`), SSSOM-shaped joined mapping tables	Built once on import / on schema change; refreshed incrementally	✅ Locked v0.1.6 (2026-05-09)

Ch 35’s argument that won: every production ontology-web system in the wild (UMLS, BioPortal, OxO/OxO2, NIST OLIR, Biomappings) materializes precomputed pairwise crosswalk tables — nobody computes graph→tabular projections at query time. SQLite recursive CTEs are excellent at SERVING precomputed closure tables; they degrade super-linearly above ~500K nodes when computing closure on the fly. Materializing closure-table + SSSOM joined tables in v0.1.6 (a) plays to sqlite-wasm’s strength, (b) de-risks Tier 2 substrate decision (Tier 2 rarely doing >2-hop recursion at query time), (c) gives Crosswalker SSSOM TSV import as v0.1.6’s first-class on-ramp to the broader ontology-web ecosystem (OxO2, BioPortal, OBO Foundry, Biomappings interop).

v0.1.6 deliverables (now locked):

SSSOM TSV import (canonical mapping table format).
Materialized crosswalk_index table per user-selected ontology pair, with (subject_id, object_id, predicate_id, confidence) columns and (subject_id, object_id) index.
Materialized closure table per concept scheme the user opts in (ancestor, descendant, depth) — the standard closure-table pattern, far faster than repeated recursive CTE.
Incremental refresh: rebuild only the affected pair on SSSOM file change.
Sparse-pivot rendering helper (suppresses empty rows/columns; warns at >100K cells).

D2: Codeblock processor timing — v0.1.6 or v0.1.7?

📍 Stage: 🔍 QUERY LAYER (live read surface). Adds a second user-facing entry point to live queries — alongside Bases custom views (D7) — for ad-hoc parameterized queries written inline in any note body.

The question: Should the crosswalker-query codeblock (lets users write inline SQL-recipe lookups in any note body) ship in v0.1.6 or defer to v0.1.7?

Option A — Ship in v0.1.6 (source: Ch 28a § registerBasesView vs Codeblock)

Why now: parameterized ad-hoc queries (e.g. “show me evidence for AC-2 across all frameworks”) are useful immediately; users will want them
Mirrors SQLSeal’s working pattern (a community SQL-codeblock plugin)
Both registerBasesView (custom view) AND codeblock processor in v0.1.6

Option C — Defer to v0.1.7 (source: Ch 28c § Updated v0.1.6 milestone scope)

Why later: v0.1.6 is already at risk of scope blowout; ship ONE custom view (crosswalkerPivot) that covers 80% of compliance use cases; defer the other surface
Closure queries (the main motivation for codeblock processor) can live in v0.1.7 alongside the user-facing SQL surface

Current lean (decided 2026-05-08): Option C — defer (likely indefinitely; not just v0.1.7). Per user direction:

Why defer (and possibly never ship):

Steps on the toes of other plugins — SQLSeal, Datacore, and the Datalog/SPARQL community already build inline-codeblock query surfaces. Crosswalker shouldn’t compete with them at the codeblock layer.
Bases is the focal-point integration — the Obsidian ecosystem’s gravity is moving toward Bases as the central query mechanism. Other tools (SQLSeal, Datacore) will likely become Bases-compatible over time. Crosswalker should integrate deeper with Bases (via registerBasesView per D7) rather than build a sibling codeblock surface.
v0.1.6 ships ONE custom Bases view (crosswalkerPivot); ad-hoc closure queries can be added to that view’s options surface (e.g. “show closure depth 1/2/3” toggle inside the pivot view) without needing a separate codeblock processor.

What this means for Ch 36 (query language rerun): this challenge’s urgency drops significantly under the locked D2. The broader query-language question (SPARQL / Datalog / Cypher / GraphQL) becomes a v0.2+ landscape concern rather than a v0.1.6 implementation prerequisite. Ch 36 is downgraded to “background research” rather than v0.1.6 / v0.1.7 blocker.

D3: Folder convention for plugin-emitted files

📍 Stage: 🗂️ STORAGE organization (Tier 1). Specifically the plugin-emitted subset of the vault — .base files Crosswalker writes, recipes Crosswalker bundles, materialized snapshots (when v0.1.8 lands). User-authored content (notes, evidence) is unaffected.

The question: Where do plugin-emitted .base files and (eventually) materialized snapshots live in the user’s vault?

Option A — Reports/_generated/<recipe>/<view>/<timestamp>.md (source: Ch 28a § Folder location)

Auditor-discoverable name (Reports/ is human-meaningful)
_generated prefix marks machine-owned; gitignored by default
Per-recipe + per-view subfolders allow granular .gitignore exclusion

Option C — _crosswalker/views/ for .base files + _crosswalker/audit/ for snapshots (source: Ch 28c § Updated v0.1.6 milestone scope)

Single namespace for all Crosswalker plugin-emitted files
Underscore prefix (NOT dot) so Bases can still index the files (Obsidian doesn’t index dot-folders)
First-run prompt asks user to add _crosswalker/views/ to Excluded Files in Settings

Current lean (decided 2026-05-08): Adopt the split-by-purpose namespace as default; expose the alternate convention as an experimental setting. Concretely:

Default (v0.1.6): _crosswalker/views/ for plugin-emitted .base files; _crosswalker/audit/ for materialized snapshots when materialization runs (per D1)
Advanced setting (v0.1.6 or later): optional toggle to use Reports/_generated/<recipe>/<view>/<timestamp>.md instead — for users who want auditor-discoverable folder names
One canonical default; one experimental opt-out; the convention used per-vault is recorded in plugin settings

Deeper open question flagged (the user’s contrarian critique): do we even need to emit files in the first place? Obsidian’s value with Bases is that you attach metadata to existing files — you don’t duplicate files just to store derived metadata. Junction notes are a legitimate exception (they ARE first-class typed-edge entities). But pivot tables, coverage matrices, and other query projections arguably shouldn’t be vault files at all. Two-hop scenarios compound this: an evidence note → junction note → control. If you want to pivot ON junction notes, do you generate yet ANOTHER note that combines them? That’s a lot of file-emission hops.

This is essentially the Ch 28c “no materialization” contrarian position revived as a deeper architectural critique. Reconciliation with D1: D1’s “opt-in manual-command-only” default means file emission ONLY happens when the user explicitly asks (e.g. “generate this audit snapshot for Q1 review”). The plugin doesn’t auto-generate files. The default Crosswalker experience is metadata-only (Bases reads frontmatter on existing files); materialization is the explicit escape hatch for the audit/Publish use case. Ch 32 brief should also engage the broader “should derived projections ever be files?” question.

D4: Adopt CQRS framing as a load-bearing architectural commitment?

📍 Stage: 🔍 QUERY LAYER ↔ 📤 OUTPUT (the boundary between them). This decision draws the line: live reads (query layer) and frozen snapshots (output layer) are different things with different purposes, lifecycles, and audit semantics. Affects every other decision because it tells you which side a feature belongs to.

The question: Should “query layer ≠ snapshot layer” join the 6 settled architectural commitments as #7?

CQRS = Command Query Responsibility Segregation. In Crosswalker terms: the query layer (Bases views + codeblock + sqlite-wasm sidecar — live reads; no new files) is architecturally separate from the snapshot layer (materialization + OpenTimestamps + git — point-in-time evidence; static files). Conflating them is a category error.

Option Yes — adopt as commitment #7 (source: Ch 28a § Contrarian “No Materialization” Analysis)

The deepest insight from Ch 28a; reframes Ch 27’s “hybrid” verdict as a clean read/write split
Pays off across v0.1.6 / v0.1.7 / v0.1.8 — every architectural decision benefits from the framing being explicit

Option No — leave implicit

One fewer commitment to maintain
Risk: agents drift back into conflating live queries with audit snapshots

Current lean (decided 2026-05-08): Yes — adopt as commitment #7. Per user direction: “I care about resiliency long term of how something scales up at the technical level. So if making this an architectural commitment helps with that, then let’s do it.”

Why adopt:

Long-term resiliency: agents (and humans) will drift back into conflating live queries with audit snapshots without an explicit framing. Architectural commitments are the durable mechanism to prevent that drift.
Cascading clarity: every other v0.1.x decision (D1 materialization placement, D3 folder split, D5 sqlite-wasm role) gets clearer with CQRS as the framing — you know which side a feature belongs to.
Free precedent: this is just naming an architectural pattern that already exists in the codebase (v0.1.5 P3 shipped Tier 2 SQL helpers as live-read; v0.1.8 will ship materialization as snapshot-write). Naming the boundary doesn’t add work.

Action when locked: update .claude/CLAUDE.md v0.1 architectural commitments table to add #7 (CQRS query layer ≠ snapshot layer); expand project_query_layer_bases_not_dataview memory accordingly (already done in 2026-05-08 commit).

D5: `sqlite-wasm` in v0.1.6 — hidden helper or user-facing surface?

📍 Stage: ⚙️ PROJECTION (Tier 2 sidecar) ↔ 🔍 QUERY LAYER. The Tier 2 sqlite-wasm sidecar already exists (v0.1.5 P3 shipped). This decision is about who can call into it: only Crosswalker’s own internal code (the custom Bases view’s render logic), or can users write SQL directly?

The question: Should v0.1.6 expose any user-facing SQL surface, or keep sqlite-wasm strictly internal?

Option Internal-only (source: Ch 28c § Updated v0.1.6 milestone scope)

v0.1.5 P3 already shipped Tier 2 SQL helpers (plugin.queryConcepts/Crosswalk/Closure). They’re plugin-internal
v0.1.6 crosswalkerPivot view calls them under the hood for closure/anti-join enrichment
Users never write SQL; they just see a working pivot view
v0.1.7 surfaces SQL via codeblock processor (per D2)

Option User-facing surface in v0.1.6 (source: Ch 28a § sqlite-wasm feasibility)

Earlier user-facing power; SQL-savvy users can write recipes against the SQL layer immediately
Couples codeblock processor (D2) to v0.1.6 — same scope-bloat concern

Current lean (decided 2026-05-08): Internal-only. Per user direction: “I’d like to keep it simpler by doing internal only and my only worry is if we do want to develop something like that in the future, just making sure our system is designed in a way that we could later down the line.”

What this means concretely:

v0.1.6 plugin handles (plugin.queryConcepts, plugin.queryCrosswalk, plugin.queryClosure — already shipped) stay accessible only to internal Crosswalker code (the crosswalkerPivot view’s render function, future internal commands)
No user-facing SQL surface in v0.1.6; users don’t write SQL anywhere
Per D2 the codeblock processor is also deferred (likely indefinitely)

Architectural design constraint (locked alongside this decision):

The Tier 2 helper API stays substrate-neutral and surface-agnostic — calls like plugin.queryClosure(start, predicate, depth) should not leak SQL semantics or sqlite-wasm-specific behavior. If we later want to expose them user-facing (codeblock, command palette, REST API) the existing function signatures should compose without redesign.
Add to backlog / v0.2+ research: “Should there be a user-facing query API in v0.2+? What surface? CLI command? plugin REST API? Bases formula extension if Obsidian ships plugin-functions API?”

Re-evaluate triggers: revisit if (a) Obsidian ships a plugin-extensible Bases formula API (forum #109612) — would let us expose helpers as Bases formulas without separate codeblock; or (b) explicit user demand for direct SQL access from third-party plugins or scripts.

D6: Memory file — expand existing or create new?

📍 Stage: 🧠 META — agent-context bookkeeping, not part of the user-facing flow. Memory files are persistent notes Claude Code carries across sessions; they tell future agents “this is settled, don’t re-litigate.” Doesn’t affect what the plugin does; affects how durably the decision sticks across contributors / sessions / agents.

The question: Do the new architectural commitments (CQRS framing, three-layer architecture, 7-primitive vocabulary, crosswalkerPivot naming) go into the existing project_query_layer_bases_not_dataview.md memory or a new memory file?

Option Expand existing (memory file at ~/.claude/projects/.../memory/project_query_layer_bases_not_dataview.md)

Already the canonical query-layer commitment record
One file = one source of truth; agents looking up “what’s the query-layer story” find it all in one place

Option Create new (e.g., project_ontology_web_query_engine.md)

More specific scope; finer-grained discoverability
More memory files to maintain in MEMORY.md index

Current lean (decided 2026-05-08, default lean accepted): Expand existing. Already done as part of the 2026-05-08 alignment commit — see updated memory file in CLAUDE.md. Single canonical record. User didn’t comment on this decision; default lean accepted.

D7: Custom view name — `crosswalkerPivot`, `crosswalkerCoverageMatrix`, `crosswalkerCrosswalk`, or `crosswalkerOntologyView`?

📍 Stage: 🔍 QUERY LAYER (the primary live read surface). Specifically the name of the one custom Bases view-type Crosswalker registers in v0.1.6. Future siblings (crosswalkerGraph, crosswalkerHierarchy, crosswalkerTimeline in v0.2+) inherit the naming convention from this choice.

The question: What name does the v0.1.6 custom Bases view get registered as via registerBasesView?

Option crosswalkerPivot (recommended — operation-named)

Names the operation (pivot = group-by-two-axes, aggregate cell value)
Matrix is one output rendering of pivot, not a separate concept
Future siblings: crosswalkerGraph, crosswalkerHierarchy, crosswalkerTimeline — consistent family
Coverage Matrix becomes a recipe (compliance-flavored launch instance) on top of the pivot view

Option crosswalkerCoverageMatrix (current code; what Ch 28 deliverables proposed)

Domain-direct: GRC users see “Coverage Matrix” and immediately understand
Compliance-coded: doesn’t translate cleanly to non-GRC ontology webs (MITRE, OBO Foundry, SKOS, library science)
Locks the view to one rendering shape (matrix); future shapes need new view names that don’t form a clean family

Option crosswalkerCrosswalk (project-name-aligned)

“Crosswalk” is general-domain ontology-mapping vocabulary
Slightly redundant (“crosswalker-crosswalk-…”)

Option crosswalkerOntologyView (data-context-aligned)

Emphasizes ontology-web framing
Risks: sounds polymorphic (does it render pivot OR graph OR hierarchy?); harder to ship in v0.1.6

Current lean (decided 2026-05-08): crosswalkerPivot — locked. Per user direction: “I think crosswalkerPivot is a good term for this because we’re going by the shape primitives here as we’ve discussed.”

Why this name fits the architecture (cross-linked):

“Pivot” is one of seven Layer A query primitives — the operation of projecting graph data into a 2D crosstab. Naming the view after the operation puts it on the same vocabulary axis as future siblings (crosswalkerGraph, crosswalkerHierarchy, etc.).
“Pivot” is one of six Layer B view shapes — the visual presentation of the pivot operation’s output.
“Coverage Matrix” sits at Layer C (recipes) — the marketplace instance composing pivot + compliance-flavored axes (control × framework × evidence-count). User-facing surfaces still say “Coverage Matrix”; internal vocabulary stays general per feedback_general_ontology_positioning.md memory.
See concepts/ontology-web-querying for the broader positioning rationale.

Action: rename in src/main.ts registerBasesView call from crosswalkerCoverageMatrix → crosswalker-pivot (kebab-case for the registered ID). Update v0.1.6 milestone — already done in 2026-05-08 commit.

D8: Recipe schema `query:` block — additive bump in v0.1.6 or defer?

📍 Stage: 📥 INPUT (recipe grammar) ↔ 🔍 QUERY LAYER. Recipes are the YAML files that drive imports (today, the EMISSION grammar — folder/file/heading/tag/wikilink). This decision adds a parallel query: section to recipes so the same recipe that imports a framework can also declare WHICH queries Crosswalker should generate .base files for. Bridges import → query.

The question: When does recipe.schema.json gain a query: block (lets recipes declare what to query + which view shape to render)?

Option v0.1.6 additive bump (recommended — pending Ch 31 for concrete schema design)

The custom crosswalkerPivot view needs SOMETHING to parameterize on (which axes? which edge? which aggregation?)
Ad-hoc .base options in v0.1.6 → schema bump in v0.1.7 = breaking change for early users
Additive bump means recipes without query: continue to work; recipes with query: get the new capability
Backed by Ch 31’s research (compares to dbt models, LookML, Cube.dev, GraphQL, SPARQL CONSTRUCT)

Option Defer to v0.1.7 (lighter v0.1.6 scope)

Less to design + validate in v0.1.6
Risk: ad-hoc options in v0.1.6 lock conventions that don’t survive v0.1.7’s redesign

Current lean (decided 2026-05-08): v0.1.6 additive bump — locked, with explicit guardrails. Per user direction: “As long as we’ve taken a primitives approach here with the certain systems that are already at play, then good and fine and dandy. Not over-engineered… If you’re saying this is a way to make it possible to have marketplaces for various reports and views out of various ontologies, then I think this is a good idea as long as we’re testing it properly, and if the testing doesn’t work, and we can backtrack and go a different route as long as we’ve designed it in that way, then I’m good with it.”

Locked design constraints (carried into Ch 31 brief and v0.1.6 implementation):

Primitives-composition approach — the query: block declares a composition of existing Layer A query primitives (filter / project / traversal / closure / anti-join / pivot / aggregate). Recipes do NOT introduce a new query language or invent primitives. They wire up existing verbs.
Don’t over-engineer — the schema must be small, declarative, and authorable by humans. Rough shape: query: { shape, primitives, output }. No DSL inside YAML; no expression-evaluator; no embedded SQL. If a query needs SQL, it lives in a recipe-internal file referenced by id — not inline in the recipe.
Marketplace-ready — recipes (with query: blocks) are the artifact users share via gist / community registry. The schema must be portable: a recipe authored against one Crosswalker version should validate against compatible later versions. SemVer the schema (schema_version: 1.0.0); plugin refuses recipes with major-version mismatches.
Testability + backtrackability — Ch 31 must produce reference recipes that exercise the schema. bun run fixtures should generate vault state from a reference recipe and validate the round-trip. If testing reveals a fundamental design flaw, the schema must be backtrackable: additive bumps only in v0.1; reserve breaking changes for v0.2+ schema major bump (recipe.schema.json $id URL bumps to v2).

What this aligns with:

The Ch 22 5-mechanism EMISSION grammar — adds parallel query: block alongside emission grammar; same primitives-composition philosophy.
Ch 23 commitment to JSON Schema + AJV + JSONata — query: block uses same validation pipeline.
concepts/query-primitives — Layer A vocabulary the schema composes from.

Action when Ch 31 lands: implement the schema design as additive bump to spec/recipe.schema.json; ship reference recipes; add fixtures-drift CI gate validating round-trip.

D9: Concept-page strategy — three new pages or merge into existing?

📍 Stage: 📚 META — documentation organization, not part of the user-facing plugin flow. Where the three-layer query engine vocabulary gets documented for human + agent readers (now and in future research challenges).

The question: Where does the three-layer architecture vocabulary (query primitives, view shapes, ontology-web positioning) get documented?

Option Three new concept pages (recommended — already created as part of this work)

concepts/query-primitives.mdx — Layer A (filter / project / traversal / closure / anti-join / pivot / aggregate)
concepts/view-shapes.mdx — Layer B (table / list / pivot / graph / hierarchy / timeline)
concepts/ontology-web-querying.mdx — positioning page (general engine + GRC launch market)
Each is a durable orthogonal vocabulary; each findable by URL; cross-linked from existing pages

Option Merge into existing (metadata-ecosystem.mdx + system-architecture.mdx)

Fewer pages to maintain
Risk: the abstraction gets buried inside other pages and becomes invisible

Current lean (decided 2026-05-08): Three new pages — locked. Per user: “Three new concept pages is definitely better. It’s more searchable.” Already created and pushed in the 2026-05-08 alignment commit. Each page has its own URL, its own ## Related section, and is cross-linked from existing pages (metadata-ecosystem, system-architecture, what-makes-crosswalker-unique, terminology).

Status (per decision):

ID	Status	Decided 2026-05-08	Notes
D1	✅ Decided	Ship in v0.1.6, opt-in manual-command-only	REVERSES prior lean. UX concern flagged for Ch 32 materialization-UX investigation
D2	✅ Decided	Defer codeblock processor (likely indefinitely; not just v0.1.7)	Steps on plugin-ecosystem toes; integrate deeper with Bases instead. Downgrades Ch 36 urgency
D3	✅ Decided	Default `_crosswalker/views/` + `_crosswalker/audit/`; advanced toggle for `Reports/_generated/`	Deeper “should we emit files at all?” critique flagged; reconciles with D1’s opt-in default (file emission only on user command)
D4	✅ Decided	Yes — adopt CQRS as commitment #7	For long-term resiliency; cascading clarity across other v0.1.x decisions
D5	✅ Decided	Internal-only	API stays substrate-neutral + surface-agnostic so v0.2+ user-facing surface is non-breaking. Backlog item recorded
D6	✅ Decided	Expand existing memory file	Default lean accepted; already done in 2026-05-08 commit
D7	✅ Decided	`crosswalkerPivot`	Operation-named (not output-shape-named); future-proof; matches three-layer architecture
D8	✅ Decided	v0.1.6 additive bump (with locked design constraints)	Primitives-composition approach; not-over-engineered; marketplace-ready; testable + backtrackable; depends on Ch 31 for concrete schema
D9	✅ Decided	Three new concept pages	Already created + pushed in 2026-05-08 commit

(This table gets updated as each decision lands.)

§3 Convergence + divergence — what each deliverable said

Ch 27 (architectural posture)

Deliverable	Verdict	Key contribution
27a	Hybrid + Repository + `registerBasesView`	TaskNotes v4 precedent; Hexagonal/Ports-and-Adapters; identifies `registerBasesView` as the integration mechanism
27b	Four-layer schema/storage/query/views stack with CQRS	Long-horizon framing; portability dominates feature richness; vault federation as 100K-note ceiling answer
27c	Pattern B+D — materialization + `crosswalker-query` codeblock	Most concrete Ch 27 deliverable; engages 7 brief investigation areas; SQLSeal as in-Obsidian precedent; recipe-emission YAML descriptor

Ch 27 convergence: hybrid Bases-as-default + escape hatch is the right answer. Ch 27 divergence: integration mechanism — A leans registerBasesView, C leans codeblock processor.

Ch 28 (lifecycle/UX/integration stress test)

Deliverable	Verdict	Key contribution
28a	”Bases-first, codeblock-second, materialization-third” — all three in v0.1.6	Most detailed; CQRS framing (query layer ≠ snapshot layer); deterministic Markdown writer spec; recipe lifecycle with `user_edited:true` + three-way merge
28b	Misinterpreted brief as DB load testing	6-axis stress matrix + synthetic-vault generator + benchmark harness — future v0.2 perf-testing material, not v0.1.6 architectural answer
28c	Narrow v0.1.6 hard; defer materialization entirely to v0.1.8	Most conservative; `_crosswalker/` underscore folder; manifest-first OpenTimestamps; junctions=truth/matrices=caches

Ch 28 convergence (a + c): registerBasesView + single coverage-matrix view + _crosswalker/ namespace + manifest-first audit + mobile/Publish parity binding. Ch 28 divergence: D1 (materialization in v0.1.6 vs v0.1.8). All other Ch 28 questions answered convergently.

§3.5 Ch 29–37 deliverables (2026-05-09 absorption)

These are the deliverables that landed AFTER the initial Ch 27 + Ch 28 synthesis and AFTER the D1–D9 lock pass. They validate or revise prior commitments under the broader ontology-web framing. Each entry: TL;DR + impact-on-this-log + cross-link to Settled / Decision rows.

Ch 29 — Ontology-web query verbs validation (2026-05-09)

TL;DR: REVISE the candidate 7-primitive set. Final Layer A vocabulary = 8 primitives: filter, traverse, bind, project, aggregate, anti-join, set-op, diff.

The 8 verbs in plain language + concrete examples:

Verb	Plain meaning	Compiles to (SQL example)	Example Crosswalker query it enables
`filter`	Keep rows matching a condition	`WHERE framework='NIST'`	”Show only NIST controls”
`traverse`	Walk edges, optionally transitively	`WITH RECURSIVE closure AS (…)`	”Follow `:mapsTo` from CSF → 800-53 → ISO 27001”
`bind`	Add a computed column	`SELECT *, (date('now') - evidence_date) AS age`	”Compute evidence age”
`project`	Drop columns	`SELECT id, name FROM …`	”Just show me id and name”
`aggregate`	Group + count/sum/avg	`GROUP BY framework HAVING COUNT(*) > 5`	”Mappings per framework”
`anti-join`	Rows in A not in B	`WHERE NOT EXISTS (…)`	”Controls without evidence”
`set-op`	∪ / ∩ / ⊖	`INTERSECT` / `UNION` / `EXCEPT`	”Controls in BOTH NIST and CIS”
`diff`	v1 vs v2 ontology delta	(custom — version-aware)	“What changed between OBO 2024-Q1 and 2024-Q2?”

Why these 8, not 7 or 9 (the revisions vs the prior candidate set):

Dropped standalone closure — folded into traverse(predicate, depth=k|*) as a parameter. Same shape as SPARQL property paths (r*, r+, r?), Cypher variable-length [*1..n], GQL MATCH … REPEAT, Datalog recursion. Two primitives where one suffices is the Layer-A bloat the brief warned against.
Demoted pivot to Layer B — pivot decomposes into aggregate(group=row_dim, group=col_dim, fn=agg) → reshape rows-to-columns. The reshape step is purely display. Cypher / Gremlin / GQL / SPARQL / SSSOM all lack pivot.
Added bind — required for queries the prior set couldn’t express (evidence-freshness, confidence-disagreement, depth-annotation). SPARQL elevates BIND to a primitive (Extend in the algebra); SQL has computed columns and AS.
Added set-op — required for framework-overlap queries. Can’t be derived from anti-join alone (anti-join is one-sided).
Added diff — uniquely Crosswalker-essential for v0.1.8 audit-trail. Can’t be expressed as set-op(traverse(v1) ⊖ traverse(v2)) because audit consumers need typed change records (added/removed/strengthened/weakened/renamed), not raw set differences. Anchored to OWL ecco, CODEX, DynDiffOnto, git diff.
Rejected rank, window, OPTIONAL, subquery, federation, constraint-satisfy, temporal-snapshot as separate primitives — compositional (rank = bind + sort/limit), out-of-scope (federation, OWL-DL), or properties of the algebra rather than operators.

Impact on this log:

Settled #14 → REVISED (7-candidate → 8-locked)
New Settled #15 (Layer A/B boundary rule)
concepts/query-primitives.mdx needs substantial rewrite to absorb 8-primitive set
concepts/view-shapes.mdx gets pivot’s promotion-from-also-Layer-A to Layer-B-only

Ch 30 — View shape taxonomy (2026-05-09)

TL;DR: REFINE provisional 6-shape catalog. v0.1 first-class = 5 shapes (Table, List, Cards Bases-native + Pivot custom view + Hierarchy as 2nd custom view in v0.1.7–v0.1.8). Defer Graph→v0.2, Timeline→v0.2+. Heatmap/sunburst/treemap/sankey/circle-packing are render options of Pivot/Hierarchy, NOT separate shapes.

Key findings:

Three shapes appear universally across 16 ontology UIs surveyed (BioPortal, OLS4, Protégé, OntoBrowser, TopBraid EDG, VocBench, PoolParty, Skosmos, SKOS Play, Stardog Explorer, GraphDB, Cytoscape, OLSVis, OxO/OxO2, NIST OLIR, OntoPortal): indented tree (hierarchy), tabular result list, node-link graph. Everything else is either a rendering option, a Bases-native view, or fails the “2+ adjacent UI precedent” test.
Search is a Layer A filter + bind(rank) composition, not a shape. Comparison is two Tables joined via Join. Dashboard is a recipe-layer concern (Markdown note embedding multiple .base blocks).
Cards (Bases-native) joins Table/List as v0.1 consumed Bases shapes — Crosswalker recipes invoke them; do NOT reimplement.
Mechanism rule: Bases-native > registerBasesView > codeblock > materialized snapshot. Every shape rendered through registerBasesView/codeblock must ship a snapshot exporter (Publish/plugin-uninstall durability).
TaskNotes v4 is the canonical Obsidian-plugin precedent for registerBasesView (4 custom view types, .base file persistence, BasesView subclass with onDataUpdated).

Impact on this log:

Settled #2 → refined (Cards added; v0.1 catalog = 5 shapes)
Settled #15 (Layer A/B boundary) reinforced
v0.2 roadmap: Graph (Cytoscape.js); v0.2+: Timeline (defer to Bases roadmap or community plugin); v0.2: pivot render options (heatmap/treemap/sunburst/sankey)

Ch 31 — Recipe `query:` block schema design (2026-05-08)

TL;DR: Both delivered schemas (A: oneOf+const discriminator; B: if/then/else discriminator) are semantically equivalent. Verdict (updated 2026-05-09): ship BOTH behind a dev/advanced setting so users can A/B-test which feels better to author. Default = A (more common JSON Schema idiom); advanced setting flips to B.

What a recipe query: block actually looks like — the same query expressed in both schema styles:

Style A (oneOf+const — default):

# _crosswalker/recipes/nist-csf-coverage-matrix.yaml
id: nist-csf-coverage-matrix
version: 1.0.0
schema_version: 1.0.0
sources:
  - type: sssom-tsv
    path: _crosswalker/imports/nist-iso-mappings.sssom.tsv

query:
  oneOf:
    - shape:
        const: pivot
      rows:
        column: subject_id
        prefix_filter: NIST:CSF
      cols:
        column: object_id
        prefix_filter: ISO27001
      cell:
        agg: count
        having: { confidence: { ">": 0.7 } }

Style B (if/then/else — advanced setting):

# Same query, different discriminator style
id: nist-csf-coverage-matrix
version: 1.0.0
schema_version: 1.0.0
sources:
  - type: sssom-tsv
    path: _crosswalker/imports/nist-iso-mappings.sssom.tsv

query:
  shape: pivot                    # ← discriminator key at top level
  if:
    properties:
      shape: { const: pivot }
  then:
    required: [rows, cols, cell]
    properties:
      rows: { column: subject_id, prefix_filter: NIST:CSF }
      cols: { column: object_id, prefix_filter: ISO27001 }
      cell: { agg: count, having: { confidence: { ">": 0.7 } } }

Both produce identical SQL output and the same view-spec; the only difference is how validation errors phrase themselves and how IDE autocomplete behaves. Both ship in v0.1.6; advanced setting in plugin settings flips the active validator.

Locked design constraints (from Ch 31 + reinforced by Ch 36):

Data-only schema (no inline SQL/DSL inside YAML — the YAML stays declarative)
Shape-discriminated (each view shape — pivot/hierarchy/table/list/cards — has its own required fields)
JSON Schema 2020-12 + AJV validator
SchemaVer versioning (MODEL.REVISION.ADDITION — major bump = breaking change)
JSONata as the only allowed string-expression slot (for filters/computed values; bounded grammar)
Primitives-composition (recipe wires up Layer A verbs; doesn’t invent new operations)

Impact on this log:

D8 confirmed; ship both A + B styles behind dev setting (per user 2026-05-09)
v0.1.6 milestone scope adds: validator implementation for both styles + dev setting toggle
5 reference recipes ship with v0.1.6 (one per shape)

Ch 32 — Intuitive query UX (2026-05-08)

TL;DR: Hybrid Pattern D = Recipe-picker + inline editor + raw YAML escape. Ship picker + inline editor in v0.1.6; defer the full wizard. Default browse experience uses embedded \“base` blocks in canonical query notes (no separate materialized files); materialization is the explicit opt-in for audit/share.

What the user actually does (UX flow):

  ┌────────────────────────────────────────────────────────────────┐
  │  USER OPENS COMMAND PALETTE                                     │
  │  > Crosswalker: Insert query into note                          │
  └─────────────────────────┬──────────────────────────────────────┘
                            ↓
  ┌────────────────────────────────────────────────────────────────┐
  │  STEP 1 — RECIPE PICKER (modal)                                 │
  │  ┌────────────────────────────────────────────────────────┐    │
  │  │ Choose a query template:                                │    │
  │  │  ◉  Coverage Matrix (NIST → ISO 27001)                 │    │
  │  │  ○  Crosswalk Density (per framework)                  │    │
  │  │  ○  Orphan Controls (no mapping)                       │    │
  │  │  ○  [Raw YAML escape — write your own]                  │    │
  │  └────────────────────────────────────────────────────────┘    │
  └─────────────────────────┬──────────────────────────────────────┘
                            ↓
  ┌────────────────────────────────────────────────────────────────┐
  │  STEP 2 — INLINE PARAMETER EDITOR                               │
  │  Source ontology pair:  [NIST CSF]  ↔  [ISO 27001]             │
  │  Confidence threshold:  [0.7  ▾]                                │
  │  Cell aggregation:       [count  ▾]                             │
  │                                                                  │
  │  [Insert query] [Cancel]                                        │
  └─────────────────────────┬──────────────────────────────────────┘
                            ↓
  ┌────────────────────────────────────────────────────────────────┐
  │  RESULT — embedded `base` block in your note                    │
  │  ```base                                                         │
  │  filters:                                                        │
  │    crosswalker.recipe: "nist-csf-coverage-matrix"               │
  │  views:                                                          │
  │    - type: crosswalker-pivot                                    │
  │      params: { confidence_threshold: 0.7 }                      │
  │  ```                                                             │
  └────────────────────────────────────────────────────────────────┘

The query lives inside your note as an embedded base block — no extra file generated. To freeze it as a static snapshot for audit/Publish, run Crosswalker: Materialize this recipe (the explicit opt-in).

Why this UX: it borrows the pattern from kepano’s Obsidian Skills and from Notion’s /database slash command — start with a template, fill in a few parameters, get a working query inline. Power users can drop into raw YAML via the escape option.

Impact on this log:

D1 refined: query-result materialization stays opt-in; embedded base blocks are the default browse experience
v0.1.6 ships: command palette → recipe picker → inline editor → embedded block insertion
Wizard (multi-step form for complex parameter sets) deferred to v0.1.7+
crosswalker-bases SKILL.md (per kepano/obsidian-skills) ships in v0.1.6 for LLM-assisted recipe authoring

TL;DR: REAFFIRM @sqlite.org/sqlite-wasm. Mobile is the binding constraint, not scale. No surveyed engine simultaneously satisfies (a) Capacitor mobile (no SAB/OPFS-sync), (b) small WASM bundle, (c) permissive license. Decoupled vector layer (sqlite-vec) survives the re-audit, with known static-link packaging cost.

Key findings:

DuckDB-WASM + DuckPGQ: ~3.2MB compressed core; multi-thread needs SAB; DuckPGQ still community-extension status. Mobile blocker.
Cozo: only engine natively unifying Datalog + relational + graph + HNSW vectors in WASM. But large bundle, slowed release tempo, smaller community. Replaces vendor risk with vendor risk.
Oxigraph-WASM: cleanest “real SPARQL in browser” but in-memory only (RocksDB feature-gated off in WASM build). Memory pressure severe at multi-million triples.
All commercial-OSS engines fail anti-pattern filter: Stardog (proprietary), Datomic (JVM-only, Cognitect/Nubank-controlled), GraphDB (commercial-free, not OSS), Neo4j (GPLv3 + lawsuit precedent), HelixDB (AGPL viral), SurrealDB (BSL non-OSS during 4-year window), Materialize (server-only).

New triggers added (extend Ch 24’s 5 triggers):

#6 sqlite-vec-wasm packaging stability — when vectors land, fork the WASM build (sqlite-vec must be statically compiled in; can’t dynamically load extensions). Track upstream sqlite-vec-wasm-demo package; build in-house if it stagnates.
Tier 3 reframe: Oxigraph-CLI sidecar as preferred Tier-3 default for new deployments (Rust > JVM, single Apache-2.0 binary); Fuseki for Java-ecosystem users.

Impact on this log:

Settled #16 (Tier 3 = Oxigraph + Fuseki SPARQL stack) NEW
Settled #18 (mobile = binding constraint) NEW
Ch 24 migration triggers updated

Ch 35 — Graph→tabular bridging rerun (Ch 10) (2026-05-09)

TL;DR: REAFFIRM Tier 1 + Tier 2; REVISE Tier 3 (Apache AGE → Oxigraph + Fuseki). Move materialization v0.1.8 → v0.1.6 (PROPOSED scope expansion — see D1 Ch 35 nuance callout).

What “SSSOM TSV import + materialized closure-table” actually means — a worked example:

SSSOM = Simple Standard for Sharing Ontological Mappings. It’s an open-standard TSV format used across BioPortal, OxO/OxO2, OBO Foundry, Biomappings, and (post-this-decision) NIST OLIR. The file is just a tab-separated table of “concept A relates to concept B with predicate P at confidence C”:

# nist-iso-mappings.sssom.tsv (what a user would download or generate)
subject_id           subject_label                predicate_id      object_id            object_label            mapping_justification              confidence
NIST:CSF.PR.AC-1     Identity Management          skos:closeMatch   ISO27001:A.5.16      Identity Management     semapv:ManualMappingCuration       0.85
NIST:CSF.PR.AC-2     Account Management           skos:exactMatch   ISO27001:A.5.18      Access Rights           semapv:ManualMappingCuration       0.90
NIST:CSF.PR.AC-3     Remote Access                skos:relatedMatch ISO27001:A.5.18      Access Rights           semapv:LexicalMatching             0.62
NIST:CSF.PR.AC-4     Access Permissions           skos:exactMatch   ISO27001:A.5.15      Access Control          semapv:ManualMappingCuration       0.95

What Crosswalker does on import (all of this happens in v0.1.6 if D1 PROPOSED scope is approved):

┌─────────────────────────────────────────────────────────────────────┐
│ STEP 1 — User runs `Crosswalker: Import SSSOM TSV` and picks file   │
└──────────────────────────────────┬──────────────────────────────────┘
                                   ↓
┌─────────────────────────────────────────────────────────────────────┐
│ STEP 2 — Crosswalker writes ONE junction note per TSV row            │
│         to `_crosswalker/mappings/NIST-to-ISO27001/`                 │
│                                                                       │
│  Each junction note is a Markdown file with frontmatter:             │
│    ---                                                                │
│    subject_id: NIST:CSF.PR.AC-1                                      │
│    subject_label: Identity Management                                │
│    predicate_id: skos:closeMatch                                     │
│    object_id: ISO27001:A.5.16                                        │
│    object_label: Identity Management                                 │
│    confidence: 0.85                                                   │
│    mapping_justification: semapv:ManualMappingCuration               │
│    crosswalker.materialized: true                                    │
│    crosswalker.source: nist-iso-mappings.sssom.tsv                   │
│    ---                                                                │
│    # NIST CSF PR.AC-1 ↔ ISO 27001 A.5.16                             │
│    [[NIST CSF PR.AC-1]] is closely matched to [[ISO 27001 A.5.16]]   │
│                                                                       │
│  → These junction notes are TIER 1 TRUTH — committed to git,         │
│    queryable via Bases natively, durable across plugin uninstall     │
└──────────────────────────────────┬──────────────────────────────────┘
                                   ↓
┌─────────────────────────────────────────────────────────────────────┐
│ STEP 3 — Crosswalker builds a `mappings` table in the sqlite cache  │
│                                                                       │
│   CREATE TABLE mappings (                                            │
│     subject_id TEXT,                                                 │
│     predicate_id TEXT,                                               │
│     object_id TEXT,                                                  │
│     confidence REAL,                                                 │
│     mapping_justification TEXT,                                      │
│     PRIMARY KEY (subject_id, predicate_id, object_id)                │
│   );                                                                  │
│   CREATE INDEX idx_mappings_subject ON mappings(subject_id);          │
│   CREATE INDEX idx_mappings_object ON mappings(object_id);            │
└──────────────────────────────────┬──────────────────────────────────┘
                                   ↓
┌─────────────────────────────────────────────────────────────────────┐
│ STEP 4 — Crosswalker materializes a closure table (if user opts in) │
│         for fast transitive walks                                    │
│                                                                       │
│   CREATE TABLE concept_closure (                                     │
│     ancestor TEXT,                                                   │
│     descendant TEXT,                                                 │
│     depth INT,                                                        │
│     PRIMARY KEY (ancestor, descendant)                               │
│   );                                                                  │
│                                                                       │
│   -- Pre-computed at import: every (concept, all reachable           │
│   -- descendants via skos:broader/narrower or :hasParent edges)      │
│   --                                                                  │
│   -- Closure table is REBUILDABLE FROM TIER 1 — never the truth      │
└──────────────────────────────────┬──────────────────────────────────┘
                                   ↓
┌─────────────────────────────────────────────────────────────────────┐
│ STEP 5 — Now this query runs in milliseconds instead of seconds      │
│                                                                       │
│   -- "What NIST CSF controls have ANY mapping to ISO 27001?"         │
│   SELECT subject_id, subject_label                                   │
│   FROM mappings                                                       │
│   WHERE object_id LIKE 'ISO27001:%'                                  │
│     AND confidence >= 0.7;                                           │
│                                                                       │
│   -- "Show all mappings transitively from CSF.PR (Protect)"          │
│   SELECT m.* FROM mappings m                                         │
│   JOIN concept_closure c ON c.descendant = m.subject_id              │
│   WHERE c.ancestor = 'NIST:CSF.PR';                                  │
└─────────────────────────────────────────────────────────────────────┘

Why this matters: every production ontology-web system in the wild does exactly this — UMLS RRF subsets, BioPortal’s 100M-mapping table, OxO/OxO2 SSSOM crosswalks, NIST OLIR’s Derived Relationship Mapping all materialize precomputed pairwise tables. Nobody computes N×N at query time. The numbers say why: BioPortal 2025 = 1,549 ontologies / 15.3M terms / 100M+ mappings; UMLS 2025AB = 3.49M concepts. Computing closure on the fly at that scale is impossible. Pre-materializing makes sqlite-wasm fast at exactly the workload it’s good at: indexed lookup over precomputed joins.

Other key findings:

Pivot scale ceiling: full ontology-web N×N is 10¹² cells with <0.001% density — infeasible. UI rule: force pair-selection (or triple-selection) before pivot; sparse-pivot guard at >100K cells warns the user.
Five non-GRC ontology-web archetypes all reduce to existing helpers (crosswalkBetween ×2 for chain joins; closureFromConcept for transitive walks). No new primitive needed for ontology-web framing — only new data sources (SSSOM TSV being the most important).
Substrate-neutrality audit checklist (apply to v0.1.5 codebase): SSSOM-shaped row I/O; predicate-URI-parameterized closure; recursive CTE hidden behind helper; no SQLite-specific types in public signatures.

Impact on this log:

Settled #16 (Tier 3 reframe) reinforced
D1 → new “Ch 35 nuance” callout (ontology-data materialization PROPOSED for v0.1.6)
v0.1.6 milestone scope (PROPOSED): SSSOM TSV import + materialized closure-table + sparse-pivot guard

Ch 36 — Query language rerun (Ch 12) (2026-05-09)

TL;DR: REVISE Ch 12 narrowly. Adopt compositional language stack — no new Crosswalker DSL. YAML recipes (Layer C) → SQL recursive CTE (Layer A, sqlite-wasm) → JSON view-specs (Layer B) → Bases YAML for vault-scoped views. Datalog stays internal-only.

The compositional pipeline — one query traced end-to-end:

┌──────────────────────────────────────────────────────────────────────┐
│  YOU WRITE (Layer C — recipe YAML)                                    │
│  ─────────────────────────────────                                    │
│  query:                                                               │
│    shape: pivot                                                       │
│    rows:                                                              │
│      column: subject_id                                               │
│      prefix_filter: NIST:CSF                                          │
│    cols:                                                              │
│      column: object_id                                                │
│      prefix_filter: ISO27001                                          │
│    cell:                                                              │
│      agg: count                                                       │
│      having: { confidence: { ">": 0.7 } }                             │
└──────────────────────────────┬───────────────────────────────────────┘
                               ↓ engine compiles
┌──────────────────────────────────────────────────────────────────────┐
│  ENGINE EXECUTES (Layer A — SQL recursive CTE on sqlite-wasm)         │
│  ─────────────────────────────────────────────────────────            │
│  WITH filtered AS (                                                   │
│    SELECT subject_id, object_id                                       │
│    FROM mappings                                                      │
│    WHERE subject_id LIKE 'NIST:CSF%'                                  │
│      AND object_id LIKE 'ISO27001:%'                                  │
│      AND confidence > 0.7                                             │
│  )                                                                    │
│  SELECT subject_id, object_id, COUNT(*) AS n                          │
│  FROM filtered                                                        │
│  GROUP BY subject_id, object_id;                                      │
└──────────────────────────────┬───────────────────────────────────────┘
                               ↓ produces
┌──────────────────────────────────────────────────────────────────────┐
│  PROTOCOL TO RENDERER (Layer B — JSON view-spec)                      │
│  ────────────────────────────────────────                             │
│  {                                                                    │
│    "shape": "pivot",                                                  │
│    "rows": ["NIST:CSF.PR.AC-1", "NIST:CSF.PR.AC-2", "..."],          │
│    "cols": ["ISO27001:A.5.16", "ISO27001:A.5.18", "..."],            │
│    "cells": [                                                         │
│      [1, 0, 1, 0],                                                    │
│      [0, 1, 0, 1],                                                    │
│      ...                                                              │
│    ]                                                                  │
│  }                                                                    │
└──────────────────────────────┬───────────────────────────────────────┘
                               ↓ rendered through
┌──────────────────────────────────────────────────────────────────────┐
│  USER SEES (registered Bases view `crosswalkerPivot`)                 │
│  ─────────────────────────────────────────                            │
│  ┌────────────────┬──────────┬──────────┬──────────┐                 │
│  │                │ A.5.16   │ A.5.18   │ A.5.15   │                 │
│  ├────────────────┼──────────┼──────────┼──────────┤                 │
│  │ PR.AC-1        │    1     │    0     │    1     │                 │
│  │ PR.AC-2        │    0     │    1     │    0     │                 │
│  │ PR.AC-3        │    1     │    0     │    1     │                 │
│  └────────────────┴──────────┴──────────┴──────────┘                 │
└──────────────────────────────────────────────────────────────────────┘

Why this layering matters: each layer is a separate concern. Recipe authors (Layer C) think in YAML — they don’t know SQL. The engine (Layer A) is the SQL — recipe authors never see it. The renderer (Layer B) is “given a pivot of cells, draw a grid” — it doesn’t care where the data came from. This is the dbt / OpenRewrite / Argo Workflows / Kubernetes pattern (YAML on top of an established mature engine, not a custom DSL).

Key findings:

Bases is not a general query language — it’s a vault-scoped expression DSL with no fix-point recursion, transitive closure, anti-join across arbitrary predicates, or pivot/crosstab. Right surface for vault filtering and presentation, not for ontology-web reasoning.
SQL recursive CTEs cover closure/path/anti-join/pivot/aggregation. Mature in sqlite-wasm. Excellent up to ~100K entities with depth-≤4 traversals.
Datalog is genuinely better for SSSOM rule derivations (chain rules, provenance) — but worse for everything else. Ch 12 verdict was correct for SSSOM specifically; REVISE to: Datalog-shaped sub-DSL inside YAML rules: blocks, compiles to SQL CTE, no Datalog runtime ever ships to user.
GQL/Cypher (ISO/IEC 39075:2024) is the right standard 5-10 years out, but no production WASM engine today.
GraphQL is the wrong category — typed selection-set protocol, no closure/anti-join/pivot.
LLM-friendliness: SQL ≈ 47% > Cypher ≈ 34% > MQL ≈ 22% >> SPARQL ≈ 3% (SM3-Text-to-Query benchmark, zero-shot). YAML/JSON config beats them all. SQL-on-sqlite-wasm + YAML recipes is the LLM-optimal stack.
OxO2 precedent: EBI’s OxO2 uses Nemo (Datalog) materialized at release time, with Datalog purely internal to the data-load process. Direct precedent for Crosswalker’s recommended layering.

Layer-by-layer recommendation:

Layer	Surface	Language
A — Primitives	Internal API; not user-visible	SQL recursive CTEs parameterized by predicate name, depth limit, cycle policy
B — View shapes	Plugin renderers	JSON view-spec (rows, columns, group keys, drill-down links) consumed by Bases-API view types
C — Recipes	`.crosswalker.yaml` files committed to vault	YAML schema with `sources`, `query`, `derivations` (Datalog-shaped sub-DSL), `views`, `exports`

Impact on this log:

Settled #17 (compositional language stack) NEW
D2 codeblock-deferral confirmed (this challenge’s urgency was already downgraded; Ch 36 now closes it)
D8 recipe schema reinforced (Ch 31 + Ch 36 converge on YAML structured query: block + Datalog-shaped rules: sub-DSL compiled to SQL CTE)
v0.1.8 audit-trail milestone: Datalog-shaped derivations: sub-DSL inside recipe YAML earns its keep here

Ch 37 — Tier 2 scale rerun (Ch 18) (2026-05-09)

TL;DR: REAFFIRM ~100K mapping ceiling. Reframe ceiling: per-query working-set, not database size. Hard infeasibility starts above ~500K concepts / ~5M edges per active query scope. Mobile = ~1/5 of desktop.

Reference dataset sizes:

Source	Concepts	Edges	Closure rows
Gene Ontology	~43K	~88K	~500K–1M (~32MB)
HPO	~18-20K	~30-40K	small/medium
ChEBI	~195K (full); ~14K curated	~500K+	mid-size
SNOMED CT	~370K active	~1.37M	~6.5M (~250MB stored)
UMLS Metathesaurus 2025AB	3.49M	tens of M	infeasible in browser
BioPortal	1,549 ontologies / 15.3M terms	>100M mappings	infeasible

Threshold tiers:

Tier	Concepts	Edges	Closure	Working-set	Verdict
Green (Tier 2 routine)	≤50K	≤250K	≤500K	`<100MB`	GO, HPO, NIST OLIR, ChEBI subsets, ALL GRC vaults
Yellow (Tier 2 with cache+partition)	50K–250K	250K–2M	500K–3M	100–500MB	Federated 5-10 OBO ontologies; ChEBI full; UMLS thin domain subsets
Orange (Tier 2 read-only / Tier 3 recommended)	250K–500K	2M–5M	3M–8M	500MB–1GB	SNOMED CT; large multi-OBO bundles
Red (Tier 3 only)	>500K	>5M	>8M	>1GB	Full UMLS, BioPortal-scale, all-pairs pivots
Mobile	divide all above by ~5			`<300MB`	Yellow-only tier

New triggers added (extend Ch 24’s 5 + Ch 33’s #6):

#7 closure cache memory budget — fires if materialized closure for any single imported ontology exceeds 25% of estimated renderer memory.
#8 per-query working-set ceiling — fires if any single query primitive (closure, pivot, anti-join) materializes >1M rows in result-set memory. Force pagination or Tier 3 referral.

v0.1.7 deliverables (per Ch 37):

Per-ontology partitioning — each imported ontology becomes its own logical namespace (separate tables or ATTACH DATABASE per ontology). Cross-ontology mappings live in a thin mappings table joining partitions on demand.
Tripwire estimation before any closure or pivot query — cheap COUNT(*) against relevant edge subset; abort with friendly error if estimate exceeds threshold (default 250K). For pivots: estimate axis1_size × axis2_size; refuse if >1M.
Default recursive CTE depth limit = 6 with user-overridable per-ontology setting (SNOMED users need 15; GO users 8).
LRU bounded closure cache keyed on (start_concept_id, depth) with configurable memory budget (default 50MB desktop, 10MB mobile). Eviction on edit.
Mobile mode: detect Obsidian Mobile; halve all memory budgets; default to lazy frontier expansion; disable full closure materialization.
Tier 3 referral mechanism — when a query is refused for size reasons, present localhost SPARQL endpoint configuration (suggest oxigraph-server) and one-click handoff. Document the pattern; do not implement the server itself.

Anti-recommendations (locked):

Do not preemptively migrate off sqlite-wasm.
Do not load full BioPortal or full UMLS.
Do not add a vector layer in v0.1.x.
Do not assume desktop budgets on mobile.

Impact on this log:

Settled #18 (mobile = binding constraint) reinforced
v0.1.7 milestone scope: per-ontology partitioning + tripwire + bounded LRU closure cache + mobile-mode constraints
New triggers #7, #8 added to migration table

§4 What’s locked (post-2026-05-09) — the consolidated picture

This section answers: “after §1 (18 settled commitments) and §2 (9 locked decisions) and §3.5 (Ch 29-37 absorption), what’s the complete final state?” — organized by where each item lives in the data-flow picture above.

Status: D1–D9 locked 2026-05-08; Settled #14–18 added 2026-05-09; D1 “Ch 35 nuance” scope expansion locked 2026-05-09 (SSSOM TSV import + materialized closure-table joins v0.1.6 scope). All architectural decisions are now closed; remaining work is implementation + deliverable-publication housekeeping.

Storage (the Markdown vault + sqlite cache)

Aspect	Locked outcome
Folder convention	`_crosswalker/views/` (`.base` files) + `_crosswalker/audit/` (snapshots) + `_crosswalker/recipes/` (YAML). Optional advanced toggle for `Reports/_generated/`.
Edge resolution	`metadataCache.resolvedLinks` snapshotted into the cache; refreshed on `changed`/`resolved` events.
Junction-note schema	13 fields with denormalized `subject_id` + `object_id` strings (for Bases filtering); wikilinks for navigation. Schema is general; field names use launch-market `control_id` + `framework` vocab.
Substrate (sqlite cache)	`@sqlite.org/sqlite-wasm` REAFFIRMED; sqlite-vec deferred per WASM-A pivot; mobile is binding constraint per Settled #18.
Substrate (external SPARQL — Tier 3 escape hatch)	Oxigraph + Apache Jena Fuseki. Apache AGE de-listed. Ships v0.1.8+ only when triggered by concrete user need (federation to BioPortal/OxO, or local SPARQL over multi-million-triple OWL).

Query (how queries run; what views render)

Aspect	Locked outcome
Integration mechanism	Single `registerBasesView` registration: `crosswalkerPivot` (parameterized by recipe). v0.1 catalog total = 5 first-class shapes (Pivot custom + Table/List/Cards Bases-native). “Coverage Matrix” is the launch-market recipe, not the view name.
Query verb set	8 verbs (`filter / traverse / bind / project / aggregate / anti-join / set-op / diff`) — locked per Ch 29. Closure folded into `traverse(depth=*)`; pivot demoted to view-shape side.
Verb vs view-shape boundary	Verbs change data; view shapes draw it. Pivot/sort/limit/top-k all draw-side. Heatmap/sunburst/treemap/sankey = render options of Pivot/Hierarchy, not separate shapes.

Recipe (the YAML grammar + authoring UX)

Aspect	Locked outcome
Recipe lifecycle	YAML in `_crosswalker/recipes/`; `id`/`version`/`schema_version` keyed; sources = system/user/community; `user_edited:true` flag; three-way merge for conflicts; JSON Schema validation on save (warn) + on materialize (block).
Recipe `query:` block schema	Per Ch 31: data-only schema; shape-discriminated; JSON Schema 2020-12 + AJV; SchemaVer versioning; JSONata as only string-expression slot; primitives-composition (no inline SQL/DSL).
Recipe authoring UX	Per Ch 32: Hybrid Pattern D (recipe-picker + wizard + raw YAML escape); ship picker + inline in v0.1.6, defer wizard. Ship `crosswalker-bases` SKILL.md (per kepano/obsidian-skills) for LLM-assisted authoring.

Output (frozen snapshots + audit trail)

Aspect	Locked outcome
Materialization timing (D1)	Ship in v0.1.6 as opt-in command-only (`Crosswalker: Materialize this recipe`). Default browse experience = embedded `\`“base` blocks in canonical query notes; materialization is the audit/share opt-in.
Audit-trail integration	Manifest-first OpenTimestamps; one `.ots` per snapshot manifest; deterministic recipe execution required.

Cross-cutting (apply to multiple parts)

Aspect	Locked outcome
Project positioning	General ontology-web query engine; compliance (GRC) is the launch market. Internal vocab general; user-facing surfaces GRC-first.
Three-layer engine vocabulary	Layer A (8 query verbs) ≠ Layer B (5 v0.1 view shapes) ≠ Layer C (recipes). Recipes are what users author and share.
Mobile/Publish parity	Bases-only path is the only mechanism that crosses desktop + mobile + (future) Publish; codeblock processor deferred indefinitely (D2); sqlite-wasm = desktop OPFS / mobile IndexedDB-fallback via `wa-sqlite`.
CQRS framing	”Live read” features (querying) ≠ “Frozen snapshot” features (audit). Locked as architectural commitment #7 via D4.
Language stack	YAML recipes → SQL recursive CTE (sqlite-wasm) → JSON view-specs → Bases YAML for vault-scoped views. No Crosswalker-invented DSL. Datalog stays internal-only as a YAML `rules:` sub-grammar.

The deferral table (post-2026-05-09):

Phase	What lands
v0.1.6	Single `registerBasesView` (`crosswalkerPivot`) + reference `.base` file on first run + recipe loader (with `query:` block schema per Ch 31) + `_crosswalker/` namespace + Bases-disabled fallback + `crosswalker-bases` SKILL.md (per Ch 32) + opt-in `Crosswalker: Materialize this recipe` command for audit/share (per D1). PROPOSED scope expansion (pending verdict): SSSOM TSV import + materialized closure-table per Ch 35.
v0.1.7	Per-ontology partitioning + tripwire estimation + bounded LRU closure cache + mobile-mode constraints (per Ch 37). Hierarchy as 2nd custom Bases view (`crosswalkerHierarchy`).
v0.1.8	Audit-trail milestone — OpenTimestamps + manifest-first audit + deterministic-output guarantees + Datalog-shaped `derivations:` sub-DSL inside recipe YAML for SSSOM-style provenance (per Ch 36). Tier 3 SPARQL referral mechanism (per Ch 33 + #16). `diff` primitive earns its keep here.
v0.2+	Background/scheduled materialization, three-way merge UI, multi-view custom Bases types (Graph via Cytoscape.js, Timeline only if Bases-native doesn’t ship it), community recipe registry, Bases-on-Publish-specific rendering, mobile sqlite-wasm persistence, Pivot render-option expansion (heatmap/treemap/sunburst/sankey).

§5 Cascade tasks — what landing this synthesis triggered

#	Task	Status
1	Archive Ch 27 + Ch 28 briefs → `zz-challenges/archive/` with `:::tip[Resolved YYYY-MM-DD]` callouts pointing to this synthesis	Pending
2	`zz-challenges/index.mdx` — move both from “Active” to “Archived (resolved)“	Pending
3	`zz-research/index.md` — fill the “Decision log it fed” column for all 6 Ch 27 + Ch 28 deliverables	✅ Done 2026-05-08
4	v0.1.6 milestone — flip pre-requisites to ✅; refine in/out scope per synthesis verdict	✅ Done 2026-05-08
5	v0.1.7 milestone — pull in per-ontology partitioning + tripwire + bounded LRU closure cache + mobile-mode (per Ch 37)	Pending
6	v0.1.8 milestone — pull in OpenTimestamps + manifest-first audit + Tier 3 referral + `derivations:` sub-DSL	Pending
7	Memory file `project_query_layer_bases_not_dataview.md` — expand with Settled #14-#18 (8 primitives + Layer A/B boundary + Tier 3 SPARQL stack + compositional language stack + mobile binding constraint)	✅ Partial (initial expansion 2026-05-08; #14-#18 expansion pending)
8	`.claude/CLAUDE.md` v0.1 architectural commitments table — add #7 CQRS query-layer ≠ snapshot-layer (per D4 lock)	✅ Done 2026-05-08
9	`CHANGELOG.md` `[Unreleased]` — synthesis decision entry	✅ Done 2026-05-08
10	Verification — `cd docs && bun run build`; pre-commit-reviewer agent audit; personal-data sweep	Done per-commit
11	Publish Ch 29/30/33/35/36/37 deliverables to `zz-research/` as `2026-05-09-challenge-NN-<slug>.md` files; update `zz-research/index.md`; swap “(pending publication)” markers in this log for live links	✅ Done 2026-05-09

§6 Implementation reference notes (not blockers)

TaskNotes v4 source code is the canonical reference for BasesView subclass — read carefully before writing the crosswalkerPivot view (this is implementation guidance, not a blocker).
recipe.schema.json design — RESOLVED via Ch 31 deliverables (data-only / shape-discriminated / SchemaVer versioning / JSONata-only string slots). Implementation pass needed; design is locked.
Bases-disabled fallback Notice — Obsidian doesn’t expose programmatic “enable core plugin” API; v0.1.6 implementation must verify what’s actually possible (UX detail, not blocker).
First-run _crosswalker/views/ + Excluded Files prompt UX details — small UX implementation detail.
Bases API churn budget — at least one breaking change already shipped (BaseOption#shouldHide); plan for 1–2 API churns/year.
crosswalker-bases SKILL.md (per Ch 32) — ship in v0.1.6 per the kepano/obsidian-skills SKILL.md pattern for LLM-assisted recipe authoring.

§7 Reconciliation with prior decisions

Prior commitment	Reconciliation
Ch 23 — TS in-plugin engine	Consistent. registerBasesView + recipe loader are pure TS; no Python; no external runtime
Ch 24 — sqlite-wasm + sqlite-vec	Consistent. v0.1.6 keeps sqlite-wasm internal-only; v0.1.7 surfaces it via codeblock processor; sqlite-vec deferred per WASM-A pivot
Ch 22 — 5-mechanism recipe grammar	Consistent. Recipe lifecycle in this synthesis governs v0.1.6 recipe loader; emission still produces folder/file/heading mechanisms (tag/wikilink layout-levels deferred to v0.2)
Ch 07 — junction-note edge model	Consistent. Junction notes are the source of truth that Bases filters; this synthesis denormalizes specific fields for filter performance but doesn’t change the 13-field schema
v0.1.5 — Tier 2 sidecar	Consistent. Plugin handles (`queryConcepts`/`queryCrosswalk`/`queryClosure`) become the BasesView’s internal-only enrichment path in v0.1.6
Project memory `project_query_layer_bases_not_dataview.md`	Will be expanded post-synthesis with: registerBasesView decision, `_crosswalker/` namespace, materialization-deferral, CQRS framing, three-layer architecture, 7-primitive vocabulary
Project memory `feedback_general_ontology_positioning.md`	Consistent. This synthesis brings the title + Settled #2/#6/#13 + concept pages back into agreement with that memory; the prior “Bases query layer” framing had drifted GRC-coded

§8 What this does NOT do

Does not foreclose codeblock processor — deferred to v0.1.7, not rejected
Does not foreclose materialization — deferred to v0.1.8, not rejected; Ch 28a’s spec is preserved as the v0.1.8 design baseline
Does not commit to a recipe marketplace — community-recipe lifecycle is a v0.3+ question; v0.1.6 supports community sharing only via plain-YAML gist exchange
Does not change Bases as the v0.1 query layer commitment — that’s project memory from 2026-05-04 and remains unchanged
Does not address cross-vault federation — explicitly out of scope per Ch 27 brief §7 and Ch 28 anti-patterns

§9 Research challenges Ch 29–37 — status (post-2026-05-09 absorption)

What these challenges were for: the 2026-05-08 alignment review surfaced that several v0.1.7+ commitments — substrate scaling, query language, streaming execution — were locked under a GRC framing that might not survive the broader ontology-web framing. We filed 9 research challenges (Ch 29–37) to validate prior work and fill genuine gaps. 6 were NEW questions (Ch 29-34); 3 were RERUNs of older briefs (Ch 35/36/37 = reruns of Ch 10/12/18 under the broader framing — original deliverables preserved verbatim in zz-challenges/archive/).

Where the answers landed: each answered challenge was absorbed into Settled #14–18 (§1) + §3.5 (per-deliverable summaries) + §4 (locked outcomes). Nothing in §9 is action-needed except Ch 34’s eventual answer + publishing the 6 received deliverables to zz-research/.

Single-status table — what each challenge asked, what it answered, what’s left

Ch	Asked (in plain language)	Verdict (what it answered)	What’s left
Ch 29 (NEW)	“Are these the right LEGO bricks for asking questions about ontology webs?” Adversarial validation of the candidate 7 query verbs against SPARQL/Datalog/OLAP/SKOS/SSSOM prior art.	✅ REVISE 7→8 verbs. Drops standalone `closure` (folded into `traverse(depth=*)`); demotes `pivot` to view-shape side; adds `bind` / `set-op` / `diff`. → Settled #14 revised; #15 added	✅ Deliverable published 2026-05-09
Ch 30 (NEW)	“Beyond pivot — what shapes does the engine need? (graph, hierarchy, timeline, sankey, treemap…)”	✅ REFINE catalog: 5 v0.1 first-class shapes: Pivot custom + Table/List/Cards Bases-native; Hierarchy graduates v0.1.7-v0.1.8. Heatmap/sunburst/treemap/sankey = render options of Pivot, NOT separate shapes. Search/Comparison/Dashboard are compositions, not shapes. → Settled #2 refined	✅ Deliverable published 2026-05-09
Ch 31 (NEW)	“How does a recipe declare its query in YAML?” Working JSON Schema + reference recipes.	✅ Schema design adopted. 2 deliverables landed 2026-05-08 (A + B). Data-only schema; shape-discriminated; JSON Schema 2020-12 + AJV; SchemaVer; JSONata-only string slots; primitives-composition. → D8 confirmed	Pick A or B style at implementation time (semantically equivalent)
Ch 32 (NEW)	“How does a user go from intent to a working `.base` file?” Wizard vs recipe-picker vs inline editor.	✅ Hybrid Pattern D. 2 deliverables landed 2026-05-08 (A + B). Recipe-picker + wizard + raw YAML escape; ship picker+inline in v0.1.6, defer wizard. Embedded `\`“base` blocks default; materialization is opt-in audit/share. → D1 refined	None — verdict locked
Ch 33 (NEW)	“How do other multi-modal databases query like this? Are sqlite-wasm + recursive CTE still the right substrate at ontology-web scale?”	✅ REAFFIRM `@sqlite.org/sqlite-wasm`. Mobile (Capacitor) = binding constraint. Every alternative (DuckDB-WASM/Cozo-WASM/Oxigraph-WASM/Nemo-WASM) breaks mobile or licensing. → Settled #16 + #18 added	✅ Deliverable published 2026-05-09
Ch 34 (NEW)	“How does merging scale chunk-by-chunk to not blow memory at ontology-web scale?”	⏳ Pending — only unanswered challenge. Informs v0.1.7+ scale path (not a v0.1.6 blocker).	Run fresh-agent research; absorb into v0.1.7 milestone scope when it lands
Ch 35 (RERUN of Ch 10)	Original asked the question for SSSOM/GRC mapping; rerun asks for arbitrary ontology webs (BioPortal, OBO Foundry, OLS, UMLS scale).	✅ REAFFIRM Tier 1+2; REVISE Tier 3 (Apache AGE → Oxigraph + Fuseki). Every production ontology-web system materializes precomputed pairwise crosswalks; nobody computes N×N at query time. → D1 “Ch 35 nuance” locked + Settled #16	Publish deliverable. Scope expansion locked 2026-05-09: SSSOM TSV import + materialized closure-table now in v0.1.6 (was v0.1.8)
Ch 36 (RERUN of Ch 12)	Original asked Datalog vs SQL narrowly for SSSOM chain rules; rerun asks the broader question — what query language do users actually write?	✅ REVISE Ch 12 narrowly. Adopt compositional language stack (YAML recipes → SQL CTE → JSON view-specs → Bases YAML). Datalog stays internal-only as YAML `rules:` sub-grammar. → Settled #17 added	✅ Deliverable published 2026-05-09
Ch 37 (RERUN of Ch 18)	Original ceiling was ~100K mappings sized for GRC vault (~5K controls × 8 frameworks); rerun asks the same scale question for ontology-web vaults.	✅ REAFFIRM ~100K ceiling, reframed as per-query working-set. Green tier ≤50K concepts/250K edges; Tier 3 referral above ~500K. Mobile = ÷5 of desktop. New triggers #7 + #8. v0.1.7 deliverables: per-ontology partitioning + tripwire + bounded LRU closure cache. → Settled #18 reinforced	✅ Deliverable published 2026-05-09

Pacing implication (updated 2026-05-09): v0.1.6 implementation is unblocked. Settled #1–18 + decided D1–D9 give the engineering side enough to start. Only Ch 34 (streaming/chunked execution) remains pending — it informs v0.1.7+ scale path, not v0.1.6. The synthesis log finalizes (banner + “in progress” suffix removed) when Ch 34 lands AND a final pass absorbs it.

Where to look for each deliverable’s verdict:

Settled-table absorption: §1 #14–#18
Per-deliverable summary: §3.5 (Ch 29/30/33/35/36/37)
Decision-table refinements: §2 D1 (Ch 35 nuance callout) + §2 D8/D9 (Ch 31/32)
Cross-cutting commitments: §4 locked outcomes table

Research deliverables (preserved verbatim):

Briefs (will be archived once synthesis lands):

Roadmap:

v0.1.6 — Bases query layer milestone (this synthesis refines its scope)
v0.1.7 — Exporters milestone (codeblock processor + sqlite-wasm user-facing land here, pending D2)
v0.1.8 — Audit-trail milestone (materialization + OpenTimestamps land here, pending D1)

Concept pillars:

Query primitives — Layer A vocabulary (filter / project / traversal / closure / anti-join / pivot / aggregate)
View shapes — Layer B forms (table / list / pivot / graph / hierarchy / timeline)
Ontology-web querying — positioning page; Crosswalker as general engine, GRC as launch market
Metadata ecosystem — Bases capabilities + limits
System architecture Layer 4 (Query)
Hierarchy primitives
What makes Crosswalker unique
Terminology

Research challenge briefs (Ch 29-37):

Ch 29 — Ontology-web query verbs validation — ✅ deliverable published 2026-05-09
Ch 30 — View shape taxonomy — ✅ deliverable published 2026-05-09
Ch 31 — Recipe query: block schema design — ✅ deliverables landed 2026-05-08
Ch 32 — Intuitive query UX — ✅ deliverables landed 2026-05-08
Ch 33 — Multi-modal query engine landscape audit — ✅ deliverable published 2026-05-09
Ch 34 — Streaming / chunked query execution — 📋 brief filed; deliverable pending
Ch 35 — Graph→tabular bridging (rerun of Ch 10) — ✅ deliverable published 2026-05-09
Ch 36 — Query language (rerun of Ch 12) — ✅ deliverable published 2026-05-09
Ch 37 — Tier 2 scale model (rerun of Ch 18) — ✅ deliverable published 2026-05-09

Prior synthesis logs:

Ontology-web query engine — synthesis (in progress)

How Crosswalker works (the 30-second picture)

Vocabulary used throughout this log

Where each Settled item lives in the picture

Why this exists

§1 Settled — locked architectural commitments

§2 Decisions to make (D1–D9)

🗺️ Where these decisions sit in the Crosswalker project flow

Vocabulary cheat-sheet (referenced repeatedly below)

D1: Materialization timing — v0.1.6 or v0.1.8?

D2: Codeblock processor timing — v0.1.6 or v0.1.7?

D3: Folder convention for plugin-emitted files

D4: Adopt CQRS framing as a load-bearing architectural commitment?

D5: sqlite-wasm in v0.1.6 — hidden helper or user-facing surface?

D6: Memory file — expand existing or create new?

D7: Custom view name — crosswalkerPivot, crosswalkerCoverageMatrix, crosswalkerCrosswalk, or crosswalkerOntologyView?

D8: Recipe schema query: block — additive bump in v0.1.6 or defer?

D9: Concept-page strategy — three new pages or merge into existing?

§3 Convergence + divergence — what each deliverable said

Ch 27 (architectural posture)

Ch 28 (lifecycle/UX/integration stress test)

§3.5 Ch 29–37 deliverables (2026-05-09 absorption)

Ch 29 — Ontology-web query verbs validation (2026-05-09)

Ch 30 — View shape taxonomy (2026-05-09)

Ch 31 — Recipe query: block schema design (2026-05-08)

Ch 32 — Intuitive query UX (2026-05-08)

Ch 33 — Multi-modal substrate audit (2026-05-09)

Ch 35 — Graph→tabular bridging rerun (Ch 10) (2026-05-09)

Ch 36 — Query language rerun (Ch 12) (2026-05-09)

Ch 37 — Tier 2 scale rerun (Ch 18) (2026-05-09)

§4 What’s locked (post-2026-05-09) — the consolidated picture

Storage (the Markdown vault + sqlite cache)

Query (how queries run; what views render)

Recipe (the YAML grammar + authoring UX)

Output (frozen snapshots + audit trail)

Cross-cutting (apply to multiple parts)

§5 Cascade tasks — what landing this synthesis triggered

§6 Implementation reference notes (not blockers)

§7 Reconciliation with prior decisions

§8 What this does NOT do

§9 Research challenges Ch 29–37 — status (post-2026-05-09 absorption)

Single-status table — what each challenge asked, what it answered, what’s left

§10 Related

D5: `sqlite-wasm` in v0.1.6 — hidden helper or user-facing surface?

D7: Custom view name — `crosswalkerPivot`, `crosswalkerCoverageMatrix`, `crosswalkerCrosswalk`, or `crosswalkerOntologyView`?

D8: Recipe schema `query:` block — additive bump in v0.1.6 or defer?

Ch 31 — Recipe `query:` block schema design (2026-05-08)