Skip to content
Crosswalker
🚧 Early alpha — building the foundation. See the roadmap →

Tier 2 substrate — synthesis (Ch 24 resolution; canonical SQLite + sqlite-vec stays; Turso/libSQL/Limbo all rejected)

Created Updated

§1 The verdict

Section titled “§1 The verdict”
QuestionVerdictConfidence
Q1 — Migrate Tier 2 to libSQL-WASM for v0.1 / v0.2?REJECT. Stay on @sqlite.org/sqlite-wasm + sqlite-vec.High
Q2 — Document Turso Cloud as Tier 3 option?REJECT for v1.0. Substrate mismatch + vendor governance flux. Add a “considered and not adopted” note instead.Medium-high
Q3 — Track Turso Database / Limbo?WATCH, DO NOT ADOPT. Pre-1.0 beta. Add to long-horizon substrate register. Review at v1.0 + 12 months or on listed public signals.High

The Ch 24 deliverable did exactly what an adversarial fresh-agent assessment should — steelmanned each migration argument and rejected it on the merits, not by reflex. The verdicts are upheld unanimously. There are no commitments to roll back.

§2 Why all three rejections are correct (in one breath each)

Section titled “§2 Why all three rejections are correct (in one breath each)”

Q1 — libSQL-WASM rejection. No bundle-size benefit. No portability benefit. Vector-search parity claim contradicted by independent benchmark (Bambini, Aug 2025: libSQL DiskANN didn’t complete on 100K × 384-dim vectors; sqlite-vec scanned in ~57 ms). Vendor-trajectory signal is negative — Turso has publicly stated “libSQL is actively maintained, but new features are being developed in Turso [Database]”; @libsql/libsql-wasm-experimental is at version 0.0.3. Adopting an artifact whose vendor has publicly downgraded it is the textbook scenario the SurrealDB / AGE / Bun precedents were designed to detect.

Q2 — Turso Cloud Tier 3 rejection. Substrate mismatch with the (now-confirmed) canonical Tier 2. Vendor governance is in flux during the libSQL → Limbo pivot. Postgres + pgvector already covers the “I want a real server with vector + SQL” axis. Edge-replication is interesting but not a Crosswalker user need (single-user or small-team Obsidian vaults).

Q3 — Limbo watch-don’t-adopt. Pre-1.0 beta. Not file-format-compatible with classic SQLite at every layer (compatibility is an aspiration, verified by deterministic-simulation testing — excellent for finding bugs, not for proving completeness). Single-vendor governance, however well-funded. Every Crosswalker query would need to be re-validated against Limbo’s bytecode generation.

§3 The deepest insight — vector-layer decoupling as a modularity principle

Section titled “§3 The deepest insight — vector-layer decoupling as a modularity principle”

The Ch 24 deliverable §5 surfaced a property that is more important than the Q1 rejection itself:

sqlite-vec is portable across SQLite forks and rewrites; libSQL native vector is not.

Concrete consequences:

Future scenarioWhat survives if we use sqlite-vecWhat survives if we use libSQL native vector
Limbo ships a stable WASM build in 2027✅ sqlite-vec moves with us (Limbo targets SQLite extension ABI compat)❌ Re-embed everything; libSQL native vector data is opaque shadow tables
Tier 3 deploys to Postgres + pgvector✅ Vectors copy across with adapter❌ Same — re-embed
sqlite-vec maintenance lapsesWe can fork it (~5 KLOC pure C, MIT, multi-sponsor: Mozilla Builders + Fly.io + Turso + SQLite Cloud + Shinkai)❌ Forking libSQL is a substrate-scale undertaking
Crosswalker decides to support multiple substrates simultaneously (the user’s “modular system” goal)✅ Vector layer is the same across substrates❌ Vector layer is locked to one substrate

This is the modularity property the user explicitly asked for. From the user’s confirmation message: “as long as we’re designing a modular system (which I think we are) where we could try with other database backends to test for improvements or additional capability.”

The Crosswalker stack is now load-bearing-ly modular along three orthogonal axes:

AxisModular propertyWhere committed
Recipe schema ↔ engine implementationEngine is swappable; recipe contract is stableCh 23 §4
Tier 1 schema ↔ producer toolchainAnyone can produce Tier 1; the schema is the contractETL and import § the reframe
Vector data ↔ substratesqlite-vec stays; substrate can be swappedThis log §3 (Ch 24 §5)

These three together are what make the system genuinely portable across implementation choices. None of them was free; each came from rejecting a single-vendor convergence shortcut.

§4 What’s locked in (specific v0.1 substrate commitments)

Section titled “§4 What’s locked in (specific v0.1 substrate commitments)”
Aspectv0.1 lock
Tier 2 substrate@sqlite.org/sqlite-wasm (canonical, foundation-governed)
Vector layersqlite-vec (Alex Garcia + Mozilla Builders + Fly.io + Turso + SQLite Cloud + Shinkai sponsorship)
Vector indexBrute-force scan + sqlite-vec’s DiskANN ANN release (alpha) when needed
WASM packagingCustom static-link build of @sqlite.org/sqlite-wasm with sqlite-vec compiled in
PersistenceOPFS via opfs-sahpool VFS (works on Capacitor without COOP/COEP — the mobile path)
File formatVanilla SQLite (3.x) — readable by sqlite3 CLI, DB Browser for SQLite, every SQLite tool
Tier 3 documented optionsPostgres / Fuseki / oxigraph-server / TerminusDB / AGE — not Turso Cloud
Long-horizon watch registerLimbo / Turso Database; first entry on the substrate watch list

§5 Migration triggers — when to revisit

Section titled “§5 Migration triggers — when to revisit”

The deliverable §2.10 + §3.5 + §4.4 spelled out specific triggers. Locking them in here so future-us has a deterministic re-evaluation path rather than an “it depends” feeling:

Q1 (Tier 2 substrate) — re-open if any TWO occur:

  1. sqlite-vec maintenance lapse — no release for 12+ months AND no community fork has gained traction
  2. libSQL becomes the de facto standard — SQLite upstream takes libSQL’s vector type, OR @libsql/libsql-wasm (without “-experimental”) reaches 1.0 with stability commitments, OR libSQL’s vector index outperforms sqlite-vec on a published reproducible benchmark
  3. Obsidian itself adopts libSQL (Obsidian Sync / Bases / core)
  4. Turso governance stabilizes around libSQL — public dated commitment to libSQL as a peer to Turso Database with separate dedicated engineering
  5. A non-Turso fork of libSQL emerges with multi-stakeholder governance — the “Valkey moment”

Q2 (Turso Cloud Tier 3) — re-open if EITHER occurs:

  • Q1 triggers fire (substrate alignment story changes)
  • Turso publicly recommits Turso Cloud to long-term libSQL backend support

Q3 (Limbo adoption) — re-open at v1.0 + 12 months OR if any occurs:

  • Limbo reaches versioned 1.0 with documented file-format compatibility test pass
  • Non-Turso fork or alternative steward emerges (governance diversification)
  • Major Obsidian ecosystem tool (core, Dataview, Bases) adopts Limbo
  • Limbo’s WASM ships stable versioned npm with feature-set matching Tier 2 needs (FTS5, recursive CTEs, vector, OPFS, SAH-pool)

Q4 (sqlite-vec WASM packaging — added 2026-05-05; revised 2026-05-06 after WASM-B integration revert) — re-open by 2026-11-06 OR if ANY signal occurs:

This trigger applies to how we ship sqlite-vec, not whether. Updated trajectory:

  • 2026-05-05 (planned): ship WASM-B via bun add sqlite-vec-wasm-demo@0.1.9 (official upstream npm package from asg017). Pragmatic v0.1 default; knowingly violates the §2.2 ~600 KB plugin budget at 1.8 MB compressed. User accepted size tradeoff to ship vec from day 1.
  • 2026-05-06 (reverted): WASM-B integration hit a chain of 5 emscripten env-detection issues with the demo artifact in Obsidian’s Electron renderer (build target, file-copy, env-detection throw, import.meta.url URL construction, postRun init order). The demo is built for plain web browsers, not Electron’s hybrid window+process environment. Reverted to WASM-A (plain @sqlite.org/sqlite-wasm — official SQLite-team build, Electron-hardened by precedent). All 6 Phase 1 substrate-scaffolding smoke tests pass on WASM-A. See v0.1.5 milestone “Path history” section for the full integration-issue chain.

Calendar-anchored revisit: 2026-11-06 (6 months from the WASM-A pivot). Even if no signal fires below, re-check the upstream sqlite-vec packaging state on that date. If all signals are still negative, push the checkpoint another 6 months OR commit to WASM-C (custom emcc build).

Re-evaluate when ANY of:

  • Bundle-size pressure surfaces ← most likely trigger to fire. Community plugin reviewers push back on plugin size, mobile load times degrade noticeably, OR users report sub-1MB plugins as preferable. v0.1.5 ships at ~2 MB compressed (vs Ch 24 §2.2 ~600 KB target).
  • sqlite-vec upstream ships a production-quality npm-installable WASM package — size-optimized (debug symbols stripped, -Os + LTO, feature trim), distinct from the demo artifact
  • @sqlite.org/sqlite-wasm gains runtime extension loading in WASM (currently disabled for browser security; if upstream adds it, we can bun add both packages and load vec at runtime)
  • Concrete need for a custom build that sqlite-vec-wasm-demo doesn’t satisfy — custom compile flags, patches to upstream SQLite or sqlite-vec, reproducible builds for compliance environments, or SQLite version mismatch
  • Demo artifact is abandoned upstream — no release for 12+ months AND no community fork has gained traction (parallel to Q1 sqlite-vec maintenance trigger)

Migration path when triggered: most likely move to WASM-C custom build (size-optimized: -Os + LTO + feature trim — target ~600 KB compressed per original §2.2 commitment). Real engineering investment in build infrastructure, but unavoidable for production-quality vec packaging since upstream ships a demo build. Sidecar API surface (lifecycle + projector + queries) doesn’t change because vec is loaded the same way regardless of artifact source. The npm sqlite-vec-wasm-demo package is the v0.1 default, not a permanent commitment.

Rollback (different from migration — for use if sqlite-vec proves problematic): If sqlite-vec needs to be removed entirely (mobile incompatibility, vendor abandonment, security CVE without timely fix, OR size pressure makes vec unviable AND custom build is too expensive to do soon), the schema is purely additive — concepts, mappings, junction_notes don’t depend on vec. ~30 min of code: bun remove sqlite-vec-wasm-demo, bun add @sqlite.org/sqlite-wasm, drop vec virtual tables from migrations, bump schema_version. Existing sidecars auto-reproject without vec. Tier 1 vault is unaffected (Tier 1 is canonical; Tier 2 reprojects). See the v0.1.5 milestone “Rollback path” section for the full step list.

These triggers are specific and falsifiable. None of them is “it feels like it’s time.”

§6 The long-horizon watch register pattern

Section titled “§6 The long-horizon watch register pattern”

The Ch 24 deliverable proposed a “long-horizon substrates under observation” appendix as the right home for Limbo. This synthesis adopts that pattern more broadly: the embedded-vs-server-substrates concept page now hosts a watch register section listing substrates Crosswalker has evaluated and chosen not to adopt today, with conditions for re-evaluation.

Initial entries:

EntryCategoryStatus
Turso Database / LimboSingle-file embedded relationalWatch; review v1.0 + 12 months
kuzuEmbedded property graph (Cypher); single-fileWatch; revisit if recursive-CTE-on-SQLite hits expressivity wall
LanceDBEmbedded columnar vector DBWatch; revisit if sqlite-vec becomes insufficient or unmaintained
DuckDB-PGQProperty graph extension to DuckDBWatch; revisit if DuckDB-WASM becomes Tier 2 alternative
DatalevinEmbedded multi-model (Clojure)Watch; niche

Two adjacent watch entries (not substrate, but adjacent file-based-tool decisions):

EntryCategoryStatus
jj / jujutsuVersion control system (git-compatible front-end + standalone backend)Watch; complementary to git for recipe versioning, audit trail, marketplace bundle history
StoolapModern columnar OLAP-on-SQLite-styleWatch via Ch 12b deliverable; revisit if Tier 2 outgrows recursive-CTE patterns

The pattern is: explicit, dated, reasoned non-adoption with conditions for revisit is itself documentation. Cargo-culting would be refusing to revisit; conservatism with explicit triggers is engineering discipline.

§7 What this does NOT do

Section titled “§7 What this does NOT do”
  • Does not foreclose Turso products forever. All three migration-trigger sets above are designed to fire deterministically if the underlying signals change. The rejections are dated 2026-05-04, not eternal.
  • Does not deprecate canonical SQLite as “best forever.” SQLite is what survives Crosswalker’s specific constraints today. Tier 2 substrate choice is reversible — the recipe schema, Tier 1 contract, and (per this log §3) vector layer are all portable across substrate choices.
  • Does not rule out future libSQL-flavored Tier 3. If Turso governance stabilizes and Tier 2 still benefits from canonical SQLite, libSQL-as-Tier-3 alongside Postgres remains plausible — it just isn’t justified today.
  • Does not reject DuckDB-WASM, Oxigraph (library), or Nemo as future Tier 2 alternatives. Those remain in the v0.1 stack pivot “researched back-pocket” register — separate decision lineage, not closed by Ch 24.

§8 Implications for the user’s “modular system” goal

Section titled “§8 Implications for the user’s “modular system” goal”

The user explicitly asked: “as long as we’re designing a modular system (which I think we are) where we could try with other database backends to test for improvements or additional capability.”

Three concrete responses to that goal:

  1. The system IS modular today — the runtime-agnostic recipe schema (Ch 23 §4), schema-as-primitive Tier 1 contract, and now vector-layer-decoupled-from-substrate (Ch 24 §5) make every implementation choice swappable.
  2. The substrate-experimentation surface — once Tier 2’s TS engine is shipping against canonical SQLite, alternative substrates (DuckDB-WASM, Oxigraph, libSQL, Limbo) can be evaluated via parallel Tier 2 implementations targeting the same recipe schema and vector layer. This is a clean experimentation harness, not a substrate lock-in.
  3. Watch register as deferred experimentation — entries in §6 above are exactly the candidates for “try with other database backends to test for improvements.” The triggers are the falsifiable conditions for moving an entry from “watch” to “evaluate,” and from “evaluate” to “adopt or reject.”
Section titled “§9 Related”