Zero-Friction Knowledge Capture Pipeline

James Phoenix
James Phoenix

Obsidian + Claude Code + QMD eliminates all friction between seeing an idea and having it searchable, deduplicated, and published.

Author: James Phoenix | Date: March 2026

Zero-Friction Knowledge Capture Pipeline


The Problem

Bookmarks rot. Notes in emails get buried. Browser tabs pile up and crash. “I’ll save this for later” means “I’ll never find this again.”

Every knowledge worker has a graveyard of half-used systems: Notion databases with 200 untagged entries, Pocket queues stretching back years, Slack threads with “important” links that nobody can find. The friction between “I found something valuable” and “I can retrieve it when I need it” kills most knowledge systems before they start.


The Pipeline

Source (YouTube, X, article, idea)
  │
  ▼
Dump raw text into Claude Code
  │
  ▼
Claude writes/updates Obsidian note
  │
  ▼
QMD deduplicates via semantic + keyword search
  │
  ▼
Auto-sync to website

Each step has near-zero friction. No manual tagging, no folder decisions, no formatting overhead.


How It Works

Step 1: Capture

See something valuable? Copy the text and paste it into Claude Code. That’s it.

  • YouTube video: Grab the transcript, paste it in. “Write a note summarizing the key insights from this transcript.”
  • X post / thread: Copy the text. “Capture the core idea from this post.”
  • Article: Paste the relevant section. “Add this to my knowledge base.”
  • Own idea: Just describe it. “I want to capture this thought about X.”

No context switching. No opening a different app. No deciding which folder it goes in. Claude handles the formatting, the wikilinks, the file placement.

Step 2: Deduplication with QMD

This is the part that prevents knowledge bases from becoming landfills.

QMD indexes the entire Obsidian vault with BM25 keyword search, vector embeddings, and LLM re-ranking. The custom CLAUDE.md prompt instructs Claude to run 2-3 QMD queries plus Grep searches before writing anything new. This dual search strategy catches both semantic overlaps (QMD) and exact phrase matches (Grep) that vector search might miss.

Before writing a new note:
  1. qmd_query "core concept" → semantic matches
  2. qmd_search "exact phrase" → keyword matches
  3. Grep for file names and specific terms

Result: Either update an existing note or confirm it's genuinely new.

This means the knowledge base stays lean. No duplicate notes about the same concept from different sources. If a YouTube video covers something already documented, Claude updates the existing note with the new perspective rather than creating a parallel entry.

Step 3: Auto-Sync to Website

The Obsidian vault auto-syncs to the website. Write locally, it appears publicly. No manual publish step, no CMS, no copy-pasting between systems.


Why This Works

Friction kills systems, not motivation

The difference between “I should save this” and actually saving it needs to be under 10 seconds. Anything more and the capture rate drops off a cliff. This pipeline keeps it to: copy text, paste into Claude, done.

Udemy Bestseller

Learn Prompt Engineering

My O'Reilly book adapted for hands-on learning. Build production-ready prompts with practical exercises.

4.5/5 rating
306,000+ learners
View Course

LLMs are perfect formatters

Humans are bad at consistent formatting, tagging, and cross-referencing. LLMs are great at it. Claude applies the right frontmatter, uses the right wikilink syntax, follows the folder conventions in CLAUDE.md, and links to related notes. Offloading this to the LLM means every note is properly integrated into the knowledge graph on arrival.

Deduplication is the difference between a knowledge base and a junk drawer

Most personal knowledge systems fail because they accumulate without consolidating. QMD semantic search before every write operation means the vault converges rather than sprawls. Over time, notes get richer (more sources, more angles on the same concept) rather than more numerous.

Publishing removes the “should I share this?” decision

When publishing is automatic, there’s no activation energy for sharing. Every note you capture is immediately available as a reference you can link to, share, or build on publicly. This turns private learning into a public body of work with zero additional effort.


The Stack

Component Role
Obsidian Local-first markdown vault. Files on disk, no vendor lock-in.
Claude Code Intake processor. Formats, links, places, and deduplicates notes.
QMD Semantic search + deduplication. BM25, vector search, LLM re-ranking.
CLAUDE.md Custom prompt: run multiple QMD queries + Grep before writing.
Auto-sync Vault syncs to website. Write once, published everywhere.

Before vs After

Before (bookmarks, email notes, Pocket, etc.):

Capture rate: ~20% of valuable things seen
Retrieval rate: ~5% of things saved
Time to capture: 1-5 minutes (open app, decide folder, format)
Deduplication: None (same link saved 3 times in different places)
Published: Never (too much friction to turn notes into posts)

After (Obsidian + Claude Code + QMD):

Capture rate: ~90% of valuable things seen
Retrieval rate: ~95% of things saved (semantic search)
Time to capture: <10 seconds (paste into Claude)
Deduplication: Automatic (QMD checks before every write)
Published: Automatic (vault syncs to website)

Key Insight

The best knowledge system is the one with the least friction. If capturing an idea takes more than 10 seconds, the system will fail. Build the pipeline so the hard parts (formatting, linking, deduplication, publishing) happen automatically.


Related

References

Topics
Ai AutomationData DeduplicationKnowledge ManagementLlm IntegrationObsidian Workflows

More Insights

Cover Image for Linux Is the Execution Substrate

Linux Is the Execution Substrate

The only layer in your stack that AI does not abstract away.

James Phoenix
James Phoenix
Cover Image for Markdown Files as State Machines for AI Development Workflows

Markdown Files as State Machines for AI Development Workflows

A structured markdown file can function as a reliable state machine for orchestrating multi-step AI development workflows. The key insight: prose instructions fail because LLMs treat them as suggestio

James Phoenix
James Phoenix