🚧 Early alpha — building the foundation. See the roadmap →

Project state mini-map — where you are and what to decide next

Created May 15, 2026 Updated Jun 1, 2026

30-second answer

You are in the middle of v0.1.6 — Bases query layer + SSSOM import + recipe UX — the biggest milestone of the v0.1 release. Seven of ten phases have shipped; three remain.

The next recommended commit is Phase 3.5c — a low-risk pure-refactor sweep that finishes wiring 20 legacy debug.log() call sites onto the new Phase 3.5a observability logger. Roughly half a day. After that, Phase 4 (recipe-picker UX) and Phase 5 (materialization command + sparse-pivot guard) close out v0.1.6. Then two more milestones — v0.1.7 exporters and v0.1.8 audit trail — and the v0.1-RC bundle ships.

From input to output: a worked example (NIST CSF × NIST 800-53 coverage matrix)

Phase 4.5 manual testing surfaced a fair question: “I picked NIST in the picker and applied a query, but I’m not seeing left-join output. Where does the join actually happen?”

Short answer: the “join” already happened at SSSOM import time. Junction notes are pre-joined records. The Phase 3 pivot view does a GROUP BY rowsBy, colsBy → COUNT(*) over those notes — that’s the matrix.

Long answer — the full path from user CSV to rendered pivot:

Stage 1 — User imports source ontology data (Tier 1 markdown)

User has two CSVs:

# nist-csf-2.0-govern-identify.csv (NIST CSF 2.0 subcategories)
id,title,function,category
GV.OC-01,Organizational Context — mission/objectives understood,GOVERN,Org Context
GV.OC-02,Internal/external stakeholders understood,GOVERN,Org Context
...

# nist-800-53-ac-family.csv (NIST 800-53 controls)
id,title,family
AC-1,Policy and Procedures,Access Control
AC-2,Account Management,Access Control
...

User runs Crosswalker: Import structured data for each → wizard writes Tier 1 markdown notes:

Frameworks/NIST-CSF-2.0/GOVERN/Org Context/GV.OC-01.md
Frameworks/NIST-CSF-2.0/GOVERN/Org Context/GV.OC-02.md
Frameworks/NIST-800-53/Access Control/AC-1.md
Frameworks/NIST-800-53/Access Control/AC-2.md
...

Each note has frontmatter (control_id, family, etc.) per the generation engine + the _crosswalker provenance block per Tier 1 schema.

At this point: source + target ontologies are in the vault as separate markdown files. No joins yet.

Stage 2 — User imports the crosswalk (SSSOM → junction notes — THE JOIN HAPPENS HERE)

User has an SSSOM TSV file (e.g., csf-to-800-53-crosswalk.csv from NIST OLIR):

subject_id  predicate_id  object_id  match_type  confidence
csf:GV.OC-01  skos:exactMatch  nist:PM-1  exact  0.95
csf:GV.OC-02  skos:closeMatch  nist:PM-9  close  0.85
csf:PR.AA-01  skos:relatedMatch  nist:AC-2  broad  0.75
...

User runs Crosswalker: Import SSSOM mapping file → SSSOM importer (Phase 2 / Ch 35) writes one junction note per mapping:

# _crosswalker/mappings/csf-to-800-53/cw-csf-gv-oc-01-nist-pm-1.md
---
kind: crosswalk-edge
subject_id: "csf:GV.OC-01"
predicate_id: "is_equivalent_to"   # ← normalized from skos:exactMatch
object_id: "nist:PM-1"
match_type: "exact"
confidence: 0.95
sssom_predicate: "skos:exactMatch"
mapping_justification: "semapv:ManualMappingCuration"
_crosswalker: {...}
---

This is where the join lives. Each junction note carries BOTH endpoints (subject + object) inline. It’s a denormalized, pre-joined record — one row per mapping, sitting on disk as a markdown file. The “left join” is materialized at SSSOM-import time, not query time.

The Tier 2 sqlite-wasm projector (Phase 1.4.5+) then projects these junction notes into a mappings table for fast lookups:

CREATE TABLE mappings (
  subject_id TEXT,
  predicate_id TEXT,
  object_id TEXT,
  confidence REAL,
  vault_path TEXT
);
-- Each junction note → one row.
-- The "join" was Phase 2; this is just an index.

Stage 3 — User authors a query (Phase 4.5 picker)

User opens any note, runs Crosswalker: Insert query into note, picks “NIST CSF → 800-53 coverage matrix”:

The picker fires the Phase 4.5 orchestrator applyQueryToNote():

Reads the recipe recipes/v0-1/coverage-matrix.json from the bundled catalog
Renders the Bases YAML via the recipe-templates module with user params (e.g. confidence_threshold: 0.7)
Writes a generated .base file at _crosswalker/views/q-2026-05-16-<hex8>.base
Writes the canonical crosswalker_query: frontmatter to the user’s note via app.fileManager.processFrontMatter()
Inserts ![[_crosswalker/views/q-2026-05-16-<hex8>.base]] at cursor

The generated .base file:

# Auto-generated by Crosswalker on 2026-05-16...
# Source note: My Coverage Analysis.md
# Recipe: nist-csf-coverage-matrix
# Query ID: q-2026-05-16-a1b2c3d4

filters:
  and:
    - file.inFolder("_crosswalker/mappings/csf-to-800-53")
    - 'confidence >= 0.7'
views:
  - type: crosswalker-pivot
    name: "NIST CSF → 800-53 coverage matrix"
    config:
      rowsBy: "subject_id"
      colsBy: "object_id"
      cellOp: "count"
      empty: "gap"
      heatmap: true

No JOIN keyword anywhere. The filters clause selects the junction notes (which are already-joined records). The views clause says “group by subject_id on rows, object_id on cols, count cells.”

Stage 4 — Bases reads the embed and the crosswalker-pivot view executes

When the user opens the note, Obsidian Bases sees ![[q-2026-05-16-a1b2c3d4.base]] and renders the file inline:

Filter: Bases queries the vault for all markdown files matching file.inFolder("_crosswalker/mappings/csf-to-800-53") AND confidence >= 0.7. Returns the junction notes from Stage 2.
Custom view dispatch: Bases sees type: crosswalker-pivot and hands the entries to our registered view (Phase 3 / crosswalkerPivot).
Pivot rendering: the view runs computePivotGrid(entries, config) from src/views/pivot-grid.ts — the pure data-shaper.

computePivotGrid is the operation you were looking for:

For each junction note JN:
  row_key = JN.frontmatter[config.rowsBy]   // "csf:GV.OC-01"
  col_key = JN.frontmatter[config.colsBy]   // "nist:PM-1"
  cell_value = aggregate(cellOp, all JN's mapping to this (row_key, col_key) pair)
                                            // count=N for "count" op

Result:
  rows = sorted unique row_keys
  cols = sorted unique col_keys
  cells = { (row, col) → cell_value | matching pairs }
  empty cells = gap / blank / zero per `empty` mode

That’s a GROUP BY rowsBy, colsBy → aggregate(cellOp) — a pivot, not a join. The join was already done in Stage 2.

Stage 5 — User sees the rendered matrix

                AC-1   AC-2   AC-3   PM-1   PM-9   PM-11  ...
GV.OC-01         —      —      —     [1]    —      —
GV.OC-02         —      —      —      —    [1]     —
PR.AA-01        [1]    [1]     —      —     —      —
PR.AA-02         —     [1]     —      —     —      —
...

Cells show the count (or sum, avg, etc. depending on cellOp); empty cells render as — (or whatever empty mode says). heatmap: true color-shades cells by value.

Why you didn’t see fruit when you tested Phase 4.5

You picked controls-by-family-list (a CIS Controls flat-list view, not a join — just filter → list) and the underlying CIS Controls data wasn’t imported yet. The .base file was generated correctly, but Bases had nothing to filter against.

To see the coverage matrix demo:

Run Crosswalker: Import SSSOM mapping file → pick tools/fixtures/realistic/nist-csf-to-mitre-attack.sssom.tsv (or the CSF→800-53 fixture). This writes junction notes to _crosswalker/mappings/csf-to-mitre/ (or csf-to-800-53/).
Open any note → Crosswalker: Insert query into note → pick “NIST CSF → 800-53 coverage matrix” (or the new “NIST CSF → MITRE ATT&CK coverage” which uses the mitre fixture).
Bases now has junction notes to filter against. The pivot renders.

Where each architectural commitment shows up in this flow

Stage	Commitment that governs it
Stage 1 — CSV → Tier 1 markdown	#1 Schema-as-primitive: every Tier 1 note conforms to the JSON Schema; anyone can produce them, not just the import wizard
Stage 1+2 — file/folder mechanism for note layout	#2 Closed 5-mechanism recipe grammar: `folder` + `file` + `also_emit.frontmatter.managed`
Stage 2 — SSSOM denormalization (the “join”)	Settled #6 from the 2026-05-07 Bases query layer synth: “Junction-note schema denormalizes subject/object IDs for fast Bases filtering”
Stage 2 — Tier 2 projection (sqlite-wasm `mappings` table)	#4 sqlite-wasm Tier 2 — the table is a cache; rebuildable from Tier 1 junction notes
Stage 3 — recipe → `.base` file generation	#5 Runtime-agnostic recipe schema: JSON Schema + AJV-validated; recipe is a portable contract
Stage 3 — frontmatter as canonical query source	Phase 4.5 user architecture call (this log, section above): “frontmatter is the best place for the query to live”; Bases-queryable + plugin-uninstall-safe
Stage 4 — `![[file.base]]` embed	#6 Bases-not-Dataview: Obsidian-native embed syntax, no Dataview
Stage 4 — custom view registration	Phase 3 — `registerBasesView('crosswalker-pivot', ...)` per Settled #2 + Ch 30
Stage 4 — pivot grid computation	Phase 3 — `computePivotGrid(entries, config)` pure data-shaper (mocked-testable; 31 unit tests)
Stage 4 — mobile parity	#3 TS in-plugin engine: everything runs in Obsidian-Capacitor on mobile; streaming Tier 1 fallback when Tier 2 sqlite isn’t available

Mental-model summary

Joins happen at import time (Stage 2). Junction notes ARE pre-joined records on disk. The pivot view (Stage 4) is a GROUP BY → aggregate over those records — a presentation step, not a join step. The “left” / “right” / “outer” framing from SQL maps onto how SSSOM’s mapping semantics + the recipe’s empty: gap/blank/zero mode together decide what’s in the cells (counts, sums, etc.) and how to render unfilled rows × cols pairs.

This is the architectural payoff of Crosswalker as a general ontology-web query engine: the load-bearing data layout is the junction-note pattern, and once that’s in the vault, every view shape (pivot / table / list / hierarchy) is just a different way to render the same pre-joined records.

Visual phase map

v0.1 release path:
  ✅ v0.1.1 — Types + validation (foundation)
  ✅ v0.1.2 — render() v1 (layout engine)
  ✅ v0.1.3 — Generation engine integration
  ✅ v0.1.4 — Junction notes + crosswalks
  ✅ v0.1.4.5 — Streaming refactor
  ✅ v0.1.5 — Tier 2 sqlite-wasm sidecar
  🚧 v0.1.6 — Bases query layer + SSSOM + recipe UX  ← YOU ARE HERE
       ✅ Phase 1   — Recipe query block schema
       ✅ Phase 1.5 — Test infrastructure
       ✅ Phase 2   — SSSOM TSV import + closure precompute
       ✅ Phase 3   — crosswalkerPivot Bases view
       ✅ Phase 3.5a — Wide-event NDJSON logger
       ✅ Phase 3.5b — Logger settings UI + 3 new commands
       ✅ Phase 3.5c — Call-site sweep                  (2026-05-15)
       ✅ Phase 3.6  — Wizard draft sessions             (manually signed off 2026-05-15)
       ⚠️ Phase 4   — Recipe-picker UX (codeblock-only — wrong syntax; SUPERSEDED)
             ✅ 4a — Foundation + testing helpers + framework fixtures
             ✅ 4b — UI components (picker modal + param editor + 6 templates)
             ✅ 4c — Wire-up + SKILL.md first-run write + E2E
       ✅ Phase 4.5 — Frontmatter-driven query notes + `.base` file + `![[embed]]` (2026-05-15)
             ✅ canonical query lives in note frontmatter (queryable by Bases)
             ✅ `.base` file at `_crosswalker/views/q-<id>.base` (plugin-generated)
             ✅ `![[<view_file>]]` embed at cursor (Obsidian-native; per Bases docs)
             ✅ `Crosswalker: Refresh query views` command + onLayoutReady auto-refresh
       📋 Phase 5   — Materialization + sparse-pivot guard           ← NEXT
  📋 v0.1.7 — Exporters (STRM / OSCAL / SSSOM)
  📋 v0.1.8 — Audit trail T1 default
  📋 v0.1-RC — Bundle and ship

Each ✅ phase has a delivery log; click through from the v0.1.6 milestone hub for the full inventory.

Where Phase 3.5c fits in the bigger picture

The roadmap above is temporal — phases in shipping order. But the more useful question for resuming work is spatial: where does Phase 3.5c live in the architecture, and what other parts of the system does it touch?

The durable architecture (3 substrate tiers + 5-stage pipeline)

Crosswalker is a general ontology-web query engine built as an Obsidian plugin. The system architecture is 3 tiers of substrate (where data lives) crossed by a 5-stage pipeline (how data flows). All v0.1.6 work — including Phase 3.5c — lives somewhere in this map:

                  ╔══════════════════════════════════════════════════════════════════════════╗
                  ║                       CROSSWALKER 3-TIER SUBSTRATE                       ║
                  ╠══════════════════════════════════════════════════════════════════════════╣
                  ║                                                                          ║
                  ║   Tier 1 — Markdown vault                  ← THE SOURCE OF TRUTH         ║
                  ║   concept notes / junction notes / wikilinks (committed to git)          ║
                  ║                                                                          ║
                  ║   Tier 2 — sqlite-wasm sidecar             ← DERIVED CACHE               ║
                  ║   denormalized mappings + closure tables (rebuildable from Tier 1)       ║
                  ║                                                                          ║
                  ║   Tier 3 — SPARQL servers (Oxigraph/Fuseki)  ← DEFERRED to v0.1.8+       ║
                  ║   federated queries against BioPortal/UMLS-scale data                    ║
                  ║                                                                          ║
                  ╚══════════════════════════════════════════════════════════════════════════╝

         INPUT              PROJECTION             STORAGE               QUERY               OUTPUT
   ┌───────────────┐   ┌───────────────┐   ┌─────────────────────┐   ┌───────────────┐   ┌───────────────┐
   │ CSV / XLSX /  │ → │  csv-parser   │ → │  Tier 1 .md files   │ → │ Bases native  │ → │ Live views    │
   │ JSON / SSSOM  │   │  generation-  │   │  (concepts +        │   │ + crosswalker │   │ (re-rendered  │
   │ TSV  + recipe │   │  engine +     │   │  junction notes)    │   │ Pivot custom  │   │ on demand)    │
   │               │   │  legacy-shim  │   │     ↓               │   │ view          │   │     +         │
   │ (the user's   │   │  + render()   │   │  Tier 2 sqlite      │   │     ↓         │   │ Materialized  │
   │  data; the    │   │               │   │  projector populates│   │ Tier 2 SQL    │   │ snapshots     │
   │  recipe says  │   │               │   │  closures + edges   │   │ helpers for   │   │ (v0.1.6       │
   │  how to map   │   │               │   │                     │   │ transitive    │   │ Phase 5;      │
   │  it)          │   │               │   │  ✅ shipped         │   │ queries       │   │ v0.1.8 audit) │
   │               │   │               │   │  (v0.1.4 + v0.1.5)  │   │     ↑         │   │               │
   │               │   │               │   │                     │   │ ✅ v0.1.6     │   │               │
   │               │   │               │   │                     │   │ Phase 3       │   │               │
   └───────────────┘   └───────────────┘   └─────────────────────┘   └───────────────┘   └───────────────┘
   ↑ already in        ↑ this is where 20 of 20 "legacy"-category    ↑ Bases + Tier 2     ↑ Phase 5 +
   place since v0.1.0  call sites live; Phase 3.5c sweeps them all   handles already      v0.1.8 still
   (import wizard,                                                    emit categorized     to come
   SSSOM importer)                                                    events

Where Phase 3.5c specifically lives: it touches the PROJECTION column. Every call site that runs during a CSV/SSSOM import, a generation pass, or a Tier 2 projection currently emits NDJSON events with category: "legacy". After 3.5c, those same events become properly categorized (generation/row-error, csv-parser/parsed, tier2/projection-complete, etc.) and carry trace_ids that thread through the whole pipeline.

The 6th cross-cutting layer: observability

The 3-tier diagram above is spatial. There’s a 6th cross-cutting concern that doesn’t fit any one tier — the wide-event observability layer — that observes every operation across all 3 tiers:

                ┌──────────────────────────────────────────────────────────────┐
                │              OBSERVABILITY (Phase 3.5 — NDJSON)              │
                │   "What is the system actually doing right now?"             │
                │                                                              │
                │   crosswalker-debug.log = single NDJSON stream               │
                │   one event per line; each event carries:                    │
                │     • ts (ISO-8601)                                          │
                │     • level (error/warn/info/trace)                          │
                │     • category (the SUBSYSTEM emitting; what 3.5c fixes)    │
                │     • op (the operation slug)                                │
                │     • msg (human-readable)                                   │
                │     • trace_id  (correlates events across the pipeline)     │
                │     • span_id + parent_span_id (nesting)                     │
                │     • duration_ms (on span-end)                              │
                │     • ...freeform context                                    │
                │                                                              │
                │   Primary consumer: AGENTS (Claude Code via `cat \| jq`)      │
                │   Not humans squinting at Obsidian text files.              │
                └──────────────────────────────────────────────────────────────┘
                              ↑     ↑     ↑     ↑     ↑     ↑
                              │     │     │     │     │     │
                              │     │     │     │     │     │
                  ┌───────────┴┐ ┌──┴───┐ ┌─┴───┐ ┌┴────┐ ┌┴─────┐ ┌─┴─────────┐
                  │  wizard    │ │ csv- │ │gener│ │tier2│ │ view │ │ lifecycle │
                  │  generate()│ │parser│ │ation│ │ proj│ │ /    │ │ load/     │
                  │            │ │      │ │engin│ │ector│ │bases │ │ unload    │
                  └────────────┘ └──────┘ └─────┘ └─────┘ └──────┘ └───────────┘
                       ↑              every box above emits events ↑
                       └────────────────────────────────────────────┘
                       trace_id flows through: one ID per import,
                       shared by every event that import produces

The trace_id is the load-bearing piece. Before 3.5c, every operation emits events independently. Diagnosing “user reported 0 notes” means manually correlating timestamps across categories — error-prone. After 3.5c, a single grep filter pulls the entire causal chain of one operation: cat crosswalker-debug.log | jq 'select(.trace_id == "abc12345")' returns wizard start → CSV parse → row-by-row render → Tier 2 projection → completion notice, all in order.

How this connects to the 6 architectural commitments

Commitment	How Phase 3.5c relates
#3 TypeScript in-plugin engine	Observability is TypeScript-only. No native dependencies. Preserves mobile-Obsidian portability (no SharedArrayBuffer / no native logging libs).
#5 Runtime-agnostic recipe schema	NDJSON event schema is also runtime-agnostic. A future Python producer (Path C, v0.5+) could emit the same wire format with the same trace_id semantics; an external agent log analyzer would still work.
#6 Bases-not-Dataview	The 3.5c sweep adds `tier2` + `view` categories — gives us first-class diagnostic visibility into the Bases query path the moment a user reports a “the pivot view looks wrong” bug.

Why this matters for v0.1.6’s remaining phases (4 + 5)

The v0.1.6 milestone is the scope peak of the v0.1 release. Phase 4 (recipe-picker UX) and Phase 5 (materialization + sparse-pivot HARD guard) both introduce significant new user-facing surface area:

Phase 4 surface area: a new modal for browsing recipes, an inline parameter editor, an embedded ```base block insertion flow, a _crosswalker/SKILL.md authoring guide. Each of these can fail in ways unique to that user’s vault.
Phase 5 surface area: a Crosswalker: Materialize this recipe command that walks Tier 2 + Tier 1 to produce snapshots, a sparse-pivot HARD guard that aborts before generating 100K+ empty cells, a first-run Excluded Files prompt for _crosswalker/views/.

When (not if) bugs surface in those flows, the categorized event log from 3.5c is the difference between “agent diagnoses in 5 minutes” and “agent reads source code for an hour.”

That’s why this is the recommended next commit, not Phase 4 directly.

Phase 4.5 — frontmatter-driven query notes (the architectural pivot)

Phase 4 shipped 2026-05-15 with the wrong embed syntax. The picker emitted inline ```base codeblocks at the editor cursor — functional but architecturally weak:

Codeblocks are opaque user-editable text; query intent drifts from query rendering
Not reusable (same query in 3 notes = 3 separate codeblocks)
Not indexable by Bases itself
Wrong canonical syntax: per the obsidian-bases skill docs, Obsidian Bases’ native embed is ![[file.base]], NOT inline codeblocks

User architecture call 2026-05-15 (verbatim):

“front matter properties would be the best place for the query to ultimately live, even though you create it in a modal, and then that same file has a embedded base rendering of the actual base that is ultimately generated from all that back-end SQLite query stuff and we could have a streaming-based chunking based types of system for the tier one type uh back end where you don’t have a sidecar SQL light or whatever it is.”

“it’s actually more like an exclamation wiki link type thing if you look at the obsidian documentation”

Phase 4.5 is the corrected architecture. Phase 4 stays in git history; 4.5 supersedes the codeblock-only flow without reverting. Shipped 2026-05-15 as a single multi-change commit per the user’s “stake in the ground” pacing.

The 3 artifacts that make up a Crosswalker query (post-4.5)

USER QUERY NOTE: "Coverage Analysis.md"
══════════════════════════════════════════════════════════════════
---
crosswalker_query:                            ← canonical source of truth
  query_id: q-2026-05-15-a1b2c3d4
  recipe: nist-csf-coverage-matrix
  shape: pivot                                ← Bases-queryable
  params:
    confidence_threshold: 0.7
  view_file: "_crosswalker/views/q-2026-05-15-a1b2c3d4.base"
  generated_at: 2026-05-15T20:55:00.000Z
  schema_version: 1
---

# My coverage analysis

Prose...

![[_crosswalker/views/q-2026-05-15-a1b2c3d4.base]]    ← Obsidian-native embed
══════════════════════════════════════════════════════════════════

PLUGIN-MANAGED: _crosswalker/views/q-2026-05-15-a1b2c3d4.base
══════════════════════════════════════════════════════════════════
# Auto-generated by Crosswalker on 2026-05-15...
# Edit the source note's `crosswalker_query:` frontmatter to change this query.

filters:
  and:
    - file.inFolder("_crosswalker/mappings/csf-to-800-53")
    - 'confidence >= 0.7'
views:
  - type: crosswalker-pivot
    name: "NIST CSF → 800-53 coverage matrix"
    config:
      rowsBy: "subject_id"
      colsBy: "object_id"
      cellOp: "count"
      heatmap: true
══════════════════════════════════════════════════════════════════

Why this is the right design (matches all 6 v0.1 architectural commitments)

Commitment	How Phase 4.5 honors it
#1 Schema-as-primitive	`crosswalker_query:` frontmatter is part of Tier 1 markdown — anyone (plugin, Python producer, agent) emitting valid Tier 1 + frontmatter is a first-class producer.
#2 Closed 5-mechanism recipe grammar	The `.base` file emission uses the `file` mechanism (same as junction notes, reference base file, SKILL.md). Frontmatter writes use `also_emit.frontmatter.managed`.
#3 TypeScript in-plugin engine + mobile parity	All TS, no native deps. Mobile-safe: `app.fileManager.processFrontMatter()` works on Capacitor (no SharedArrayBuffer needed).
#4 sqlite-wasm Tier 2	Crosswalker-pivot view executes against Tier 2 sidecar when available; Tier 1 streaming chunked scan as fallback when sidecar is off or mobile (already shipped in v0.1.4.5 streaming refactor).
#5 Runtime-agnostic recipe schema	AJV-validated; picker dispatches on `shape` STRING value; new shapes (e.g. `cards` in v0.2) don’t need picker code changes. JSON Schema is at `src/views/query-frontmatter-schema.ts`.
#6 Bases-not-Dataview	`![[file.base]]` is the canonical Obsidian Bases embed. Pure Bases — no Dataview anywhere.

Cross-references to the decision chain that led here

The 4.5 pivot didn’t appear out of nowhere — it’s the right interpretation of decisions locked back in April-May 2026. The chain:

Decision log	What it locked
2026-05-04 Bundle + engine language synthesis (Ch 23)	TypeScript in-plugin engine for v0.1; runtime-agnostic recipe schema as the most important modularity commitment
2026-05-04 Tier 2 substrate synthesis (Ch 24)	sqlite-wasm + sqlite-vec for Tier 2; mobile parity (Capacitor) as binding constraint
2026-05-07 Bases query layer architecture synthesis (Ch 27/28)	`_crosswalker/views/` underscore folder for `.base` files; “junctions = audit truth, pivot tables = caches”; manifest-first OpenTimestamps audit model
2026-05-08 Ch 32 deliverable B	”Embedded ```base code blocks in canonical query notes are the default surface” — Phase 4 took this literally (inline codeblocks); Phase 4.5 interprets “embedded” as the `![[file.base]]` Bases-native syntax
2026-05-11 mid-milestone bugfixes + observability initiative	NDJSON wide-event logger as Phase 4.5’s diagnostic substrate (every `apply-query-to-note` operation emits `view`-category events with trace correlation)
Phase 3 — `crosswalkerPivot` registered Bases view (2026-05-10)	The `crosswalker-pivot` view type that Phase 4.5’s generated `.base` files reference
Phase 3.6 — wizard draft sessions (2026-05-15)	The `_crosswalker/drafts/` + auto-save pattern that informed Phase 4.5’s `_crosswalker/views/` + `query_id` naming convention

What Phase 4.5 actually shipped (file inventory)

New modules (all under src/views/):

File	Role
`query-frontmatter-schema.ts`	JSON Schema (2020-12) + AJV validator + `newQueryId()` + `viewFileFor()`. Validates the `crosswalker_query:` block at every read + write boundary.
`query-frontmatter-io.ts`	Read/write helpers using `app.fileManager.processFrontMatter()` — Obsidian’s canonical safe API. `readQueryFrontmatter()` / `writeQueryFrontmatter()` / `hasQueryFrontmatter()` + pure builders (`buildFrontmatter`, `updateFrontmatterParams`).
`apply-query-to-note.ts`	The orchestrator. Single entry point: `applyQueryToNote({app, file, editor, recipeId, shape, params})`. Decides CREATE vs UPDATE; writes `.base` file; writes/updates frontmatter; inserts embed at cursor. Structured `ApplyResult` for caller.
`regenerate-query-views.ts`	Vault scanner. `regenerateAll(app)` walks all markdown files; for each one with `crosswalker_query:` frontmatter, regenerates the `.base` file. Idempotent — skips when YAML body matches. Runs on plugin load (stale-state recovery) + as the explicit `Crosswalker: Refresh query views` command.

Reused from Phase 4 (no changes — the picker UI, the recipe templates, the recipe loader): recipe-loader.ts, recipe-picker-modal.ts (returns {recipeId, shape, params} now; orchestrator handles the rest), recipe-parameter-editor.ts, recipe-templates.ts (now generates .base file content, not codeblock body — same YAML, different write target), mobile-detection.ts. Plus all 6 reference recipes including the Phase 4a cross-domain mitre-coverage.json.

insert-base-block.ts repurposed: buildBaseBlock() kept for backward compat (deprecated); new buildEmbed(viewPath) + insertEmbedAtCursor(editor, viewPath) use the canonical ![[...]] syntax. noteContainsEmbed() makes the embed insertion idempotent (UPDATE flow doesn’t re-insert).

New commands (registered in main.ts):

Command	Purpose
`Crosswalker: Insert query into note` (REPURPOSED)	Picker → orchestrator (writes frontmatter + `.base` file + inserts `![[...]]` embed). Auto-detects existing `crosswalker_query:` frontmatter → UPDATE flow.
`Crosswalker: Refresh query views` (NEW)	Scans all notes with `crosswalker_query:` frontmatter; regenerates their `.base` files. Idempotent. Surfaces a Notice with `N refreshed, M up-to-date, K errors`.

SKILL.md rewritten: now teaches the frontmatter + .base + embed pattern as the primary authoring workflow. Existing codeblock content preserved as backward-compat reference for users still using the Phase 4 syntax.

Tests: 359/359 pass (was 310 before Phase 4.5; +49 new):

query-frontmatter-schema.test.ts — 15 tests (validation accept/reject + ID generation + view file naming)
query-frontmatter-io.test.ts — 13 tests (read/write + has/build/update; mocked processFrontMatter)
apply-query-to-note.test.ts — 7 tests (CREATE + UPDATE flows + buildBaseFileContent)
regenerate-query-views.test.ts — 14 tests (idempotency, scan-all aggregation, malformed handling, missing template)

Plus the obsidian mock got Platform + ButtonComponent + FileManager with processFrontMatter capture — reusable infra for future view-touching work.

Why Phase 4 isn’t reverted

The Phase 4 codeblocks still work — Bases supports both inline ```base codeblocks AND .base file embeds. Users with existing codeblocks from Phase 4 keep functioning queries. No auto-migration command; users do it manually if they want (open the picker on a note with an old codeblock, run the picker → new frontmatter + .base + embed; delete the old codeblock).

This is the same pattern as Phase 3 commitment #3 (“never overwrite user edits”): the plugin doesn’t touch existing user content unless explicitly asked.

What Phase 3.5c is (since you asked)

What it is. A pure-refactor sweep that migrates the 20 existing call sites in the codebase from the legacy .log(msg, data) shim to the proper categorized API: debug.info('generation', 'starting', msg, data).

Why it exists. The Phase 3.5a logger shipped with a deliberate backward-compat shim so we could land the new API without breaking anything. Existing .log() calls still work but emit events with category: "legacy" — useful, but a missed opportunity. The point of the new logger is structured categorized events; every “legacy” entry is unrealized value.

What 3.5c ships. Three things:

Sweep call sites. Every plugin.debug.log(msg, data) and plugin.debug.error(msg, err) becomes debug.info('<category>', '<op>', msg, data) / debug.error('<category>', '<op>', msg, data). Categories grouped by subsystem: generation / csv-parser / wizard / sssom-import / tier2 / config / view / lifecycle.
Thread trace IDs through top-level entry points. Wizard generate() creates a fresh trace_id; every event downstream of that operation carries it. Same for SSSOM importer + Tier 2 projector. After 3.5c, cat crosswalker-debug.log | jq 'select(.trace_id == "abc123")' returns the full causal chain of one operation.
Remove the shim. Once all 20 call sites are migrated, the backward-compat .log() and .error() overloads come out. Future agents who don’t read this log will get a clear “use info(category, op, msg) instead” error from TypeScript.

Why it’s safe. Pure refactor. No behavior change. Each call site keeps its message string; only the categorization is new. 243 unit tests already pass; they’d catch regressions.

Why it’s a good next step. Cheap (~half day), low risk, and high downstream payoff — Phase 4 (recipe-picker UX) and Phase 5 (materialization + sparse-pivot guard) will inevitably surface bugs, and a categorized debug log makes those 2-5× faster to diagnose than the current half-categorized state. Per the test-status update, this is recommendation #1.

Decisions to make

Decision	Options	Lean	Why
What ships next?	(a) Phase 3.5c sweep; (b) fix 2 minor wizard bugs; (c) start Phase 4 planning	(a) Phase 3.5c	Cheapest + cleanest path to “Phase 4 ships against a debuggable substrate.” (b) and (c) work fine either way; doing (a) first compounds.
Fix the 2 minor wizard bugs?	(a) Fold into Phase 3.5c commit; (b) own commit between 3.5c and Phase 4; (c) defer to RC	(b) Own commit	They’re real correctness bugs (the CURIE has wrong prefix; comma-separated links don’t resolve). Worth a focused commit so they get clear regression tests. ~1.5h total.
Push commits to origin?	(a) Push now (11 commits ahead of main); (b) hold until v0.1.6 RC	(a) Push now	Local-only is a brittle backup strategy. Pushing also enables Hot Reload + draft-sessions to be tested by anyone who pulls.
Test untested manual surface (Phase 2/3 UX scenarios)?	(a) Now; (b) defer to v0.1.6 RC	(b) Defer	E2E covers the data side; the manual UX walks are about pacing + cosmetics. Hold until the milestone is otherwise ready to ship.
Address typed-links preview feature request?	(a) Phase 4; (b) v0.1.7 settings polish; (c) drop	(b) v0.1.7	Feature request is settings-UX polish (captured here — `.workspace/2026-05-11-ux-feature-requests.md`, gitignored). Genuinely useful but not blocking.
Mobile parity smoke check?	(a) Now; (b) defer to RC	(b) Defer	One-time ~30min test once the rest is stable. No need to do it twice.

v0.1 architectural commitments (the durable context)

Settled 2026-05-04 after the five-challenge fresh-agent design phase (Ch 20–24). Every commit since then aligns to these; every future decision should too.

#	Commitment	Source
1	Schema-as-primitive — Tier 1 schema is the load-bearing contract. Anyone (plugin, external Python, agent, MCP server) emitting valid Tier 1 is a first-class producer.	ETL pillar
2	Closed 5-mechanism recipe grammar — `folder \| file \| heading \| tag \| wikilink` × ordered layout × also_emit × graph_edges. `render(Recipe, ConceptIdentity) → Address` as single coupling point.	Ch 22 synthesis
3	TypeScript in-plugin engine for v0.1 — Path C (optional Python producer) reserved for v0.5+. Mobile-Obsidian portability + small-OSS contributor pool are the irreversible constraints.	Ch 23 synthesis
4	Tier 2 substrate stays on `@sqlite.org/sqlite-wasm` + `sqlite-vec` — libSQL / Turso / Limbo all rejected. Five explicit migration triggers locked.	Ch 24 synthesis
5	Runtime-agnostic recipe schema — JSON Schema + AJV + JSONata; engine implementation is swappable; vector layer decoupled from substrate. The single most important modularity commitment.	Ch 23 synthesis §4
6	Output query layer = Obsidian Bases (Dataview removed)	Ch 27/28 synthesis

Calendar-anchored revisit checkpoints (worth knowing about so you don’t forget to re-evaluate):

2026-11-06 — sqlite-vec packaging revisit (per WASM-A pivot synthesis 2026-05-06). Decision deferred sqlite-vec integration until the upstream emscripten chain stabilizes; check back in November.

Key recent decisions you might have forgotten

These shaped current shape; worth remembering when planning Phase 4+.

Decision	When	What
Phase 3.6 draft sessions UX = always-visible in Step 1	2026-05-15	Original design used a stacked picker modal; first-time users couldn’t discover the feature. Reworked to embed drafts list inline in wizard Step 1 with an always-on empty state. Phase 4 recipe-picker should follow this pattern (no stacked modals).
`MappingConfig.filename` made optional	2026-05-11	The wizard’s filename fallback was `{{row}}` (broken). Removed it; legacy-shim handles fallback via first-frontmatter-column. Schema-level cleanup that future agents shouldn’t undo.
Buildable substrate is Bases-not-Dataview	2026-05-09	Don’t reference Dataview in user-facing surfaces or new code. The user-facing query layer ships as Bases.
Test-vault uses Pattern A structure (src/ + docs/ + spec/ + test-vault/ as siblings)	2026-05-04	Build outputs to `test-vault/.obsidian/plugins/crosswalker/`. Don’t move it.
Plugin release ships ONLY `main.js + manifest.json + styles.css`	2026-05-04	`tools/`, `spec/`, `docs/`, KB do NOT bloat releases.
Per-row debug events use NDJSON wide-event schema	2026-05-11	Primary log consumer is agents (Claude Code sessions via `cat \| jq`), not humans. No in-app log viewer. Phase 3.5c finishes the migration.
Hot Reload installed in test-vault	2026-05-11	`bun run dev` / `bun run build` rebuilds auto-reload Crosswalker in Obsidian. If you turned it off, future manual rebuilds need a plugin toggle-off/on cycle.

Need to…	Look at
Understand v0.1 architecture from scratch	`concepts/system-architecture` (3 tiers, 6 layers, component-to-tier matrix)
Trace a design decision	`zz-log/` (dated, reverse-chronological) — most architectural questions are answered here
See the canonical schema	`spec/tier1.schema.json` and `v0-1-schema-spec` doc page
Run a manual test	`TEST_PHASE*.md` at repo root, or `test-vault/_test-guides/` in-Obsidian copies
Catch up on what was tested vs not	2026-05-15 v0.1.6 test-status update
Catch up on shipped features (user-readable)	`CHANGELOG.md` `[Unreleased]` section
Plan the next milestone	`reference/roadmap/` (mirrored at `ROADMAP.md` repo root)
See active milestone status	v0.1.6 milestone hub — has separate Build / Manual test columns per phase
See research deliverables	`zz-research/` — long-form fresh-agent investigations
See open research challenges	`zz-challenges/`
Find a concept by name	`concepts/terminology`
Understand a tradeoff	`agent-context/tradeoffs`
See current/draft work-in-progress	`.workspace/` at repo root (gitignored — local-only)

Tooling state right now (last commit `9cfcf14`)

bun run test → 243/243 pass
bun run build → clean
cd docs && bun run build → 256 pages, clean
Local commits ahead of origin/main: 11
Working tree: clean (.claude/scheduled_tasks.lock ephemeral)
Hot Reload installed in test-vault but disabled in community-plugins.json (you turned it off; manually toggle Crosswalker for each rebuild until you re-enable Hot Reload)

Suggested order of operations from here

Read this log + the test-status update (~5 min) — gives you full pre-pause state.
Push the 11 local commits to origin (~1 min) — see Decisions table. Defaults to git push.
Ship Phase 3.5c (~half day) — categorized debug log, payoff for everything after.
Fix 2 minor wizard bugs (~1.5h) — CURIE prefix + comma-separated link splitting. Captured in test-status update.
Phase 4 planning + ship (~2 days) — recipe-picker UX. Reuses Phase 3.6 draft-state pattern.
Phase 5 planning + ship (~1.5 days) — opt-in materialization command + sparse-pivot HARD guard + first-run prompts.
v0.1.6 RC — manual UX walks of Phase 2 / Phase 3 / Phase 3.5 surfaces; mobile parity smoke check; delivery log; bump version; release.

Estimated time to v0.1.6 release: ~4-5 working days, paced to your availability.

2026-05-15 v0.1.6 test-status update — sister log; what’s tested vs not
2026-05-11 mid-milestone bugfixes + observability initiative — the 0-pages bug postmortem + Phase 3.5 kickoff
v0.1.6 milestone hub — phase-by-phase status
Roadmap index — full v0.1 path
v0.1 architectural commitments (memory file) — the six load-bearing decisions that constrain everything
Ontology-web querying — what Crosswalker is for; why the query layer matters
What makes Crosswalker unique — positioning vs other ontology tools