System Design and Invariants: The Meta Spec

James Phoenix

One document to rule them all. Every table, every subsystem, every invariant, every relationship. PRDs and design docs are detail views. This is the holistic truth.

The Problem

The Indexed PRD and Design Doc Strategy gives you navigation. The PRD-Design-Code-Test Pipeline gives you layers. But neither gives you a single place where the entire system is specified at a glance.

What happens without it:

New engineers (or agents) must read 10+ documents to understand the system shape
Object model relationships are scattered across design docs
Invariants are buried in code, tests, or nowhere at all
Data retention policies live in someone’s head
Subsystem boundaries are implicit, not declared

The Solution

Create a system-design-and-invariants.md that sits above the index. Everything stems from it.

docs/
├── system-design-and-invariants.md   ← The meta spec (holistic truth)
├── index.md                          ← Navigation hub (PRD/DD tables)
├── prd/
│   ├── PRD-001-core-feature.md       ← Detail: WHAT for a subsystem
│   └── ...
└── design/
    ├── DD-001-data-model.md          ← Detail: HOW for a subsystem
    └── ...

The hierarchy:

system-design-and-invariants.md    (the WHOLE system, compressed)
    ↓
index.md                           (navigation to all PRDs/DDs)
    ↓
PRD / Design Doc                   (detail for ONE subsystem)
    ↓
Code / Tests                       (implementation and proof)

What Goes in the Meta Spec

1. System Overview

One paragraph. What does this system do, who is it for.

2. Subsystem Map

Every subsystem listed with a one-line description and its boundaries.

## Subsystems

| Subsystem | Purpose | Boundary | PRD | DD |
|-----------|---------|----------|-----|-----|
| Task Engine | CRUD, hierarchy, dependencies | TaskService, TaskRepository | PRD-001, PRD-002, PRD-003 | DD-001, DD-004 |
| Scoring | Priority calculation | ScoringService | PRD-004 | DD-002 |
| CLI | Human interface | CLI commands | PRD-007 | DD-003 |
| MCP Server | Agent interface | MCP tools | PRD-007 | DD-005 |
| Sync | JSONL git-backed persistence | SyncService | PRD-009 | DD-009 |
| Learnings | Contextual knowledge retrieval | LearningsService | PRD-010 | DD-010 |

This is the map. PRDs and DDs are the territory.

3. Complete Object Model

Every entity, every relationship, in one place. Not scattered across design docs.

## Object Model

### Entities

| Entity | Primary Key | Description |
|--------|-------------|-------------|
| Task | tx-[a-z0-9]{6,8} (SHA256) | Core unit of work |
| Dependency | (blocker_id, blocked_id) | Blocking relationship |
| Learning | learning-[a-z0-9]{6} | Contextual knowledge entry |
| Tag | name (unique) | Task categorization |

### Relationships

Task 1──* Dependency (blocker)
Task 1──* Dependency (blocked)
Task ── Tag
Task 1──* Task (parent/child hierarchy)
Task ── Learning (contextual)


### Entity Lifecycle

| Entity | Created | Updated | Deleted | Archived |
|--------|---------|---------|---------|----------|
| Task | CLI/MCP create | status transitions, field edits | soft delete (status=deleted) | compaction |
| Learning | auto-extracted from compaction | relevance scoring | expiry policy | never |

4. All Tables / Storage Schema

The complete data model. Not implementation details (that’s the DD), but what data exists.

Udemy Bestseller

Learn Prompt Engineering

My O'Reilly book adapted for hands-on learning. Build production-ready prompts with practical exercises.

★ 4.5/5 rating

306,000+ learners

View Course

## Storage Schema

### tasks
| Column | Type | Constraints | Notes |
|--------|------|-------------|-------|
| id | TEXT | PK, SHA256-based | tx-[a-z0-9]{6,8} |
| title | TEXT | NOT NULL | |
| status | TEXT | NOT NULL, CHECK(pending/active/done/deleted) | |
| parent_id | TEXT | FK(tasks.id), NULLABLE | Hierarchy |
| created_at | TEXT | ISO 8601 | |
| updated_at | TEXT | ISO 8601 | |

### dependencies
| Column | Type | Constraints |
|--------|------|-------------|
| blocker_id | TEXT | FK(tasks.id) |
| blocked_id | TEXT | FK(tasks.id) |
| PK | | (blocker_id, blocked_id) |

5. System Invariants

The rules that must NEVER be violated. These are the hard constraints of the system.

## Invariants

### Data Integrity
- INV-001: No circular dependencies. If A blocks B, B cannot block A (transitively).
- INV-002: Parent-child hierarchy is a DAG. No task can be its own ancestor.
- INV-003: Task IDs are immutable after creation. SHA256-derived, never reassigned.

### State Machine
- INV-004: Tasks follow the state machine: pending -> active -> done | deleted. No backward transitions.
- INV-005: A task with unresolved blockers cannot transition to "active".

### Data Retention
- INV-006: Deleted tasks are soft-deleted (status=deleted), never physically removed.
- INV-007: JSONL sync files are append-only. Mutations are new entries, not overwrites.

### Security
- INV-008: ANTHROPIC_API_KEY is never stored in the database or JSONL files.
- INV-009: LLM features degrade gracefully when API key is missing.

6. Cross-Cutting Concerns

Things that span all subsystems.

## Cross-Cutting Concerns

| Concern | Strategy | Reference |
|---------|----------|-----------|
| Error handling | Effect-TS typed errors, never throw | DD-002 |
| Observability | OpenTelemetry spans on all service methods | DD-008 |
| Testing | Vitest + SHA256 fixture snapshots | DD-007 |
| Configuration | Environment variables, no config files | DD-003 |
| Concurrency | SQLite WAL mode, single-writer | DD-001 |

7. Data Retention and Lifecycle

## Data Retention

| Data | Retention | Mechanism |
|------|-----------|-----------|
| Active tasks | Indefinite | SQLite |
| Deleted tasks | 90 days | Compaction job |
| JSONL sync log | Indefinite (git-backed) | Append-only files |
| Learnings | Until relevance score < threshold | Auto-expiry |
| OTEL traces | 7 days | Trace backend TTL |

Template

# [Project Name]: System Design and Invariants

> One-line description of the system.

## System Overview

[2-3 sentences: what it does, who it's for, core value proposition.]

## Subsystems

| Subsystem | Purpose | Boundary | PRD | DD |
|-----------|---------|----------|-----|-----|
| | | | | |

## Object Model

### Entities

| Entity | Primary Key | Description |
|--------|-------------|-------------|
| | | |

### Relationships

[ASCII or Mermaid diagram showing entity relationships]

### Entity Lifecycle

| Entity | Created | Updated | Deleted | Archived |
|--------|---------|---------|---------|----------|
| | | | | |

## Storage Schema

[All tables with columns, types, constraints]

## Invariants

### Data Integrity
- INV-001: [invariant]

### State Machine
- INV-00N: [invariant]

### Data Retention
- INV-00N: [invariant]

### Security
- INV-00N: [invariant]

## Cross-Cutting Concerns

| Concern | Strategy | Reference |
|---------|----------|-----------|
| | | |

## Data Retention

| Data | Retention | Mechanism |
|------|-----------|-----------|
| | | |

## Navigation

- [Documentation Index](index.md)
- [PRD Index](prd/)
- [Design Doc Index](design/)

How It Relates to Existing Patterns

system-design-and-invariants.md     ← NEW: The holistic truth
        │
        ├── Subsystems table          → links to PRDs (WHAT)
        ├── Object model              → links to DDs (HOW)
        ├── Invariants                → enforced by tests & types
        └── Data retention            → enforced by ops
        │
index.md                            ← EXISTING: Navigation hub
        │
        ├── PRD table                 → individual PRDs
        ├── DD table                  → individual DDs
        ├── Phase breakdown           → implementation order
        └── Dependency graph          → PRD/DD relationships

The meta spec answers: “What is this system?” The index answers: “Where do I find the details?”

Why This Works

Day-one holistic understanding. A new engineer or agent reads one document and knows every subsystem, every entity, every hard constraint.
Invariants are declared, not discovered. Instead of reverse-engineering constraints from code, they’re stated upfront. Tests verify them, but the spec defines them.
Object model is centralized. No more piecing together entity relationships from 5 different design docs.
Data retention is explicit. What gets kept, what gets deleted, and when. Not a surprise during an incident.
PRDs and DDs become focused. They don’t need to re-explain the global context. They zoom into their subsystem knowing the meta spec provides the frame.

Anti-Patterns

The Navigation-Only Index

An index.md that lists PRDs and DDs but doesn’t capture system-level truths (invariants, object model, data retention). It’s a table of contents, not a spec.

The Scattered Object Model

Entity definitions split across multiple design docs. Task is defined in DD-001, Dependency in DD-004, Learning in DD-010. Nobody sees the full picture.

The Implicit Invariant

A rule that exists only in code (if (circular) throw) but isn’t documented anywhere. When the code is refactored, the invariant might be lost.

The Growing Meta Spec

The meta spec should stay high-level. If a section grows beyond a few paragraphs, it belongs in a design doc. The meta spec links to it, doesn’t absorb it.

Indexed PRD and Design Doc Strategy – The navigation hub pattern
PRD-Design-Code-Test Pipeline – Four-layer documentation stack
Invariants in LLM Code Generation – Constraint spaces for correctness
ADRs for Agent Context – Architecture decisions as guardrails