Skip to content
Crosswalker
🚧 Early alpha — building the foundation. See the roadmap →

Consistency models

Updated

Why consistency matters for Crosswalker

Section titled “Why consistency matters for Crosswalker”

Every Crosswalker import creates hundreds of interconnected files. When a framework updates and you re-import, every file potentially needs updating — but file systems have no transaction support. Understanding formal consistency models explains why certain operations are safe, which can fail partway through, and how to design for recovery.

ACID analysis for file-based systems

Section titled “ACID analysis for file-based systems”

ACID guarantees reliable database transactions. Obsidian vaults provide only one of four:

PropertyDefinitionObsidian Vault Status
AtomicityAll operations succeed or all failPartial — a script can crash mid-import, leaving half-updated files
ConsistencyDatabase moves between valid statesNo — orphaned links, stale crosswalks, missing properties can accumulate
IsolationConcurrent operations don’t interfereNo — user can edit files while a script runs
DurabilityCommitted changes survive failuresYes — filesystem persistence, plus Obsidian Sync or git

Implication: Crosswalker cannot assume any import operation is atomic. Every operation should be designed to be idempotent (safe to re-run) and resumable (can pick up where it left off).

File-based systems get 1 out of 4 ACID guarantees

CAP theorem positioning

Section titled “CAP theorem positioning”

CAP theorem (Brewer, 2000): A distributed system can provide at most two of three guarantees:

GuaranteeDefinition
ConsistencyEvery read receives the most recent write
AvailabilityEvery request receives a response
Partition toleranceSystem operates despite network failures

Obsidian vaults are AP systems — they prioritize availability and partition tolerance:

  • Available: Users can always read and edit files, even offline
  • Partition tolerant: Vaults work across devices with sync, surviving network interruptions
  • Not consistent: Files can get out of sync between devices, or between import runs

This is the correct trade-off for a personal/team knowledge base. Strong consistency would require locking files during imports — unacceptable for a tool people use throughout the day.

BASE model

Section titled “BASE model”

The alternative to ACID for distributed systems:

PropertyDefinitionCrosswalker Application
Basically AvailableSystem is always usableVault is always readable/editable, even during imports
Soft stateState may change without inputObsidian cache updates, sync propagates changes asynchronously
Eventually consistentGiven enough time, all copies convergeAfter re-import, all files reflect the new framework version

Crosswalker operates under manual eventual consistency — convergence requires explicit action (running the import), not automatic propagation:

Framework updated → User triggers re-import → Files updated → Consistency achieved
                          ↑
                  (Manual trigger, not automatic)

CouchDB MVCC analogy

Section titled “CouchDB MVCC analogy”

Multi-Version Concurrency Control (MVCC): Maintaining multiple versions of data to handle concurrent access without locking.

CouchDB’s approach maps well to Crosswalker’s version-tagged imports:

CouchDBCrosswalker
Document revision (_rev: "3-a7b2f9c1")_crosswalker.source_hash on each note
Conflict detection (divergent revisions)Re-import detects changed vs. unchanged notes
Conflict resolution (user picks winner)User chooses overwrite vs. keep vs. merge
Eventual consistency via replicationRe-import propagates framework changes

The key insight: version-tagged folders (importing NIST 800-53 Rev 5 alongside Rev 4) is essentially MVCC at the folder level — multiple versions coexist until the user is ready to migrate.

Event sourcing pattern

Section titled “Event sourcing pattern”

Event sourcing stores state changes as an immutable sequence of events, reconstructing current state by replaying them.

Applied to Crosswalker’s _crosswalker metadata:

# Minimal (current): just the latest state
_crosswalker:
  source_file: nist-800-53.csv
  import_date: 2026-04-02
  framework_version: "Rev 5"

# Event-sourced (future): full change history
_crosswalker:
  history:
    - event: imported
      date: 2026-04-02
      source: nist-800-53-rev5.csv
    - event: re-imported
      date: 2026-10-15
      source: nist-800-53-rev5-update1.csv
      changes: [description_updated, related_controls_added]

This enables point-in-time queries (“what did this control look like when we imported it?”) and rollback (“revert to the pre-update state”). See ontology evolution for the temporal modeling foundations.

Implications for Crosswalker design

Section titled “Implications for Crosswalker design”
  1. Design for idempotency: Every import operation should produce the same result whether run once or ten times
  2. Expect partial failures: If an import crashes at note 347 of 1000, the next run should detect and resume
  3. Track provenance: _crosswalker metadata is the compensation for lacking ACID — it tells you what state you’re in
  4. Version, don’t overwrite: SCD Type 2 (keep history) is safer than Type 1 (overwrite) because it preserves the ability to diff and rollback
  5. Use Bases for detection: Obsidian Bases can surface notes with stale import_date or missing source_hash — lazy consistency checking via tabular views

Resources

Section titled “Resources”

Distributed systems theory

Section titled “Distributed systems theory”
  • Brewer, E. (2000) — “Towards Robust Distributed Systems” (CAP theorem keynote)
  • CAP Twelve Years Later — Eric Brewer’s retrospective
  • Kleppmann, M. (2017) — “Designing Data-Intensive Applications” (O’Reilly) — chapters 5, 7, 9

Event sourcing

Section titled “Event sourcing”

CouchDB / MVCC

Section titled “CouchDB / MVCC”
Section titled “Related pages”