Summary
LLMs forget past decisions and repeatedly propose rejected ideas. Learning files (*-learning.json) track decisions, rationales, and context so LLMs can learn from past choices. When integrated with automated scanners, they prevent redundant suggestions, preserve institutional knowledge across team changes, and create an audit trail of all architectural decisions.
The Problem
LLMs are stateless—they forget past decisions and context between sessions. Teams using AI-assisted tools face recurring issues: (1) LLMs re-propose ideas already rejected, wasting review time (2) Rationales for decisions are lost when team members leave (3) New team members lack context on why certain patterns were chosen or rejected (4) No audit trail exists for architectural decisions made with AI assistance.
The Solution
Maintain structured learning files (*-learning.json) that track decisions, rationales, alternatives, and outcomes. LLMs read these files before generating suggestions, learning from past choices. Files capture: approved/rejected status, decision rationale, context, alternatives considered, proposer/approver, and dates. Integrated with automated scanners, they prevent redundant suggestions and preserve institutional knowledge.
The Problem
LLMs are stateless. Every conversation starts from scratch, with no memory of previous interactions or decisions.
For teams using AI-assisted development tools, this creates painful recurring problems:
Problem 1: Re-Proposing Rejected Ideas
Scenario: Your team runs an automated ESLint scanner that suggests new rules.
Week 1: LLM suggests adding "max-lines-per-function: 50"
Team: "Rejected - too opinionated, breaks valid long integration tests"
Week 5: LLM suggests adding "max-lines-per-function: 50"
Team: "We already rejected this!"
Week 9: LLM suggests adding "max-lines-per-function: 50"
Team: "WHY DOES IT KEEP SUGGESTING THIS?"
Impact: Wasted review time, team frustration, loss of confidence in AI tools.
Problem 2: Lost Rationales
Scenario: A decision is made to use factory functions instead of classes.
2024-03: Senior engineer chooses factory pattern for dependency injection
Rationale: Easier to mock, no 'this' binding issues, better tree-shaking
2024-09: Senior engineer leaves company
2024-11: New team member asks: "Why are we using factories?"
Team: "Uh... I think there was a reason? Check the commit history?"
2025-01: Another new member: "Can we switch to classes? Factories feel weird."
Team: "We don't remember why we chose factories. Let's debate it again."
Impact: Decisions are relitigated, time is wasted, context is permanently lost.
Problem 3: No Audit Trail
Scenario: Multiple architectural decisions made over months.
Month 1: Decided to use tRPC instead of REST
Month 2: Decided to use Supabase instead of custom auth
Month 3: Decided to use server actions for mutations
Month 4: Decided against using Prisma (use raw SQL instead)
Month 6: New tech lead asks: "What decisions were made and why?"
Team: "Check Slack? Search commit messages? Ask around?"
Impact: No single source of truth, tribal knowledge, onboarding friction.
Problem 4: Context Loss Across Team Changes
As team members leave and join, institutional memory degrades:
Year 1: 5 engineers, shared context
Year 2: 2 original engineers leave, 2 new engineers join
Year 3: 3 more original engineers leave, 3 new engineers join
Year 4: 0 original engineers remain
Institutional memory: GONE
Impact: Why decisions were made becomes unknown, patterns drift, quality degrades.
The Solution
Maintain learning files (*-learning.json) that capture decisions, rationales, and context in a structured format that LLMs can read and learn from.
What is a Learning File?
A learning file is a structured JSON document that tracks:
- Decisions made (approved or rejected)
- Rationale for each decision
- Context when decision was made
- Alternatives considered and why they were rejected
- Who proposed and who approved the decision
- When the decision was made
Core Concept
Before an LLM generates suggestions, it reads the learning file to understand past decisions:
┌─────────────────────────────────────────┐
│ LLM Scanner Workflow │
├─────────────────────────────────────────┤
│ 1. Read *-learning.json file │ ← Load past decisions
│ 2. Understand approved/rejected items │ ← Learn from history
│ 3. Analyze codebase for new issues │ ← Find new problems
│ 4. Filter out already-rejected ideas │ ← Prevent redundancy
│ 5. Propose only novel suggestions │ ← High-signal output
│ 6. Reference past decisions in output │ ← Show awareness
└─────────────────────────────────────────┘
Result: LLM learns from past decisions, stops re-proposing rejected ideas, and preserves institutional knowledge.
Implementation
Step 1: Define the Schema
Create a structured schema for learning files:
// learning-file.schema.ts
interface LearningFile {
type: 'eslint-rules' | 'architecture' | 'dependencies' | 'patterns';
lastUpdated: string; // ISO 8601 date
decisions: Decision[];
}
interface Decision {
id: string; // Unique identifier (e.g., rule name, pattern name)
status: 'approved' | 'rejected' | 'deferred' | 'superseded';
category: string; // e.g., 'best-practices', 'code-style', 'performance'
priority: 'high' | 'medium' | 'low';
decision: {
rationale: string; // Why this decision was made
context: string; // Relevant context at time of decision
alternatives: string[]; // Other options considered
};
metadata: {
proposedBy: string; // Who/what proposed it (e.g., "claude-eslint-scanner")
decidedBy: string; // Who approved/rejected (e.g., "engineering-team")
dateProposed: string; // ISO 8601 date
dateDecided: string; // ISO 8601 date
supersededBy?: string; // If replaced by another decision
};
}
Step 2: Create Learning Files
Create learning files for different domains:
# Project structure
project/
├── eslint-rules-learning.json # Linting decisions
├── architecture-decisions.json # Architecture patterns
├── dependency-decisions.json # Package choices
├── testing-patterns-learning.json # Testing approaches
└── code-patterns-learning.json # Code style decisions
Step 3: Example Learning File
eslint-rules-learning.json:
{
"type": "eslint-rules",
"lastUpdated": "2025-11-01",
"decisions": [
{
"id": "no-console",
"status": "approved",
"category": "best-practices",
"priority": "high",
"decision": {
"rationale": "Console statements bypass structured logging, making it hard to filter logs in production. All packages must use @repo/logger-config for consistent log formatting and levels.",
"context": "We have a centralized logging system (Datadog) that requires structured JSON logs. Console.log outputs aren't captured properly.",
"alternatives": [
"Allow console in dev only (rejected - creates dev/prod parity issues)",
"Use custom logger wrapper (rejected - reinventing wheel, @repo/logger-config already exists)"
]
},
"metadata": {
"proposedBy": "claude-eslint-scanner",
"decidedBy": "engineering-team",
"dateProposed": "2024-11-15",
"dateDecided": "2024-11-20"
}
},
{
"id": "max-lines-per-function",
"status": "rejected",
"category": "code-style",
"priority": "low",
"decision": {
"rationale": "Too opinionated and breaks valid long functions. Some integration tests legitimately exceed any reasonable limit due to setup/teardown requirements.",
"context": "We have integration tests that set up complex scenarios (auth, database state, API mocking) that require 80-100 lines. Breaking these into smaller functions reduces readability.",
"alternatives": [
"Set limit to 100 lines (rejected - arbitrary, still breaks valid cases)",
"Exclude test files (rejected - inconsistent, some prod code also valid to be long)",
"Use code review instead (approved - human judgment better for this)"
]
},
"metadata": {
"proposedBy": "claude-eslint-scanner",
"decidedBy": "engineering-team",
"dateProposed": "2024-11-20",
"dateDecided": "2024-11-22"
}
},
{
"id": "@typescript-eslint/explicit-function-return-type",
"status": "approved",
"category": "type-safety",
"priority": "high",
"decision": {
"rationale": "Explicit return types prevent accidental type widening and make function contracts clear. LLMs generate better code when return types are explicit.",
"context": "We've seen multiple bugs where inferred return types were wider than intended (e.g., function returns 'string | undefined' when we expected always 'string').",
"alternatives": [
"Rely on inference (rejected - caused bugs)",
"Require only on exported functions (rejected - inconsistent)"
]
},
"metadata": {
"proposedBy": "tech-lead",
"decidedBy": "engineering-team",
"dateProposed": "2024-10-01",
"dateDecided": "2024-10-05"
}
}
]
}
Step 4: Integrate with LLM Scanners
When running automated scanners, load the learning file first:
// eslint-scanner.ts
import { readFile } from 'fs/promises';
import Anthropic from '@anthropic-ai/sdk';
interface LearningFile {
decisions: Decision[];
}
interface Decision {
id: string;
status: string;
decision: {
rationale: string;
context: string;
};
}
async function scanForESLintRules() {
// Load past decisions
const learningFile = JSON.parse(
await readFile('eslint-rules-learning.json', 'utf-8')
) as LearningFile;
// Create prompt with learning context
const prompt = `
# Task
Analyze this codebase and suggest new ESLint rules.
# Past Decisions (DO NOT re-propose rejected rules)
${learningFile.decisions
.filter(d => d.status === 'rejected')
.map(d => `
## REJECTED: ${d.id}
- Rationale: ${d.decision.rationale}
- Context: ${d.decision.context}
`)
.join('\n')}
# Approved Rules (reference these as examples of good suggestions)
${learningFile.decisions
.filter(d => d.status === 'approved')
.map(d => `
## APPROVED: ${d.id}
- Rationale: ${d.decision.rationale}
`)
.join('\n')}
# Instructions
1. Analyze the codebase for potential ESLint rule opportunities
2. DO NOT suggest any rules marked REJECTED above
3. Only suggest rules that solve real problems in the codebase
4. For each suggestion, explain:
- What problem it solves
- Why it's different from rejected rules
- Concrete examples from the codebase
Now analyze the codebase and suggest new rules.
`;
// Call LLM
const client = new Anthropic();
const response = await client.messages.create({
model: 'claude-sonnet-4',
max_tokens: 4096,
messages: [{ role: 'user', content: prompt }],
});
return response.content;
}
Step 5: Update Learning File After Decisions
When a team makes a decision, update the learning file:
// update-learning-file.ts
import { readFile, writeFile } from 'fs/promises';
interface NewDecision {
id: string;
status: 'approved' | 'rejected';
rationale: string;
context: string;
alternatives: string[];
category: string;
priority: 'high' | 'medium' | 'low';
}
async function addDecision(
learningFilePath: string,
decision: NewDecision,
proposedBy: string,
decidedBy: string
) {
// Load existing file
const content = await readFile(learningFilePath, 'utf-8');
const learningFile = JSON.parse(content);
// Add new decision
learningFile.decisions.push({
id: decision.id,
status: decision.status,
category: decision.category,
priority: decision.priority,
decision: {
rationale: decision.rationale,
context: decision.context,
alternatives: decision.alternatives,
},
metadata: {
proposedBy,
decidedBy,
dateProposed: new Date().toISOString().split('T')[0],
dateDecided: new Date().toISOString().split('T')[0],
},
});
// Update lastUpdated
learningFile.lastUpdated = new Date().toISOString().split('T')[0];
// Save
await writeFile(
learningFilePath,
JSON.stringify(learningFile, null, 2),
'utf-8'
);
console.log(`✅ Added decision: ${decision.id} (${decision.status})`);
}
// Usage
await addDecision(
'eslint-rules-learning.json',
{
id: 'no-magic-numbers',
status: 'approved',
rationale: 'Magic numbers reduce code clarity and make refactoring harder',
context: 'Found 47 instances of unexplained numeric literals in business logic',
alternatives: [
'Allow in test files (approved - tests can use magic numbers)',
],
category: 'best-practices',
priority: 'medium',
},
'claude-eslint-scanner',
'engineering-team'
);
Real-World Example: Architecture Decisions
architecture-decisions.json
{
"type": "architecture",
"lastUpdated": "2025-11-01",
"decisions": [
{
"id": "use-factory-pattern-over-classes",
"status": "approved",
"category": "design-patterns",
"priority": "high",
"decision": {
"rationale": "Factory functions provide better dependency injection, avoid 'this' binding issues, enable better tree-shaking, and are easier to mock in tests. Classes add complexity without benefits for our use case.",
"context": "Our codebase is primarily functional. We don't use inheritance or polymorphism. Services are small, focused functions. Classes would be ceremonial overhead.",
"alternatives": [
"Use classes for consistency with OOP (rejected - we're not doing OOP)",
"Mix factories and classes (rejected - inconsistent, confusing)",
"Use classes only for stateful services (rejected - we have no stateful services)"
]
},
"metadata": {
"proposedBy": "senior-engineer-alex",
"decidedBy": "engineering-team",
"dateProposed": "2024-03-10",
"dateDecided": "2024-03-15"
}
},
{
"id": "use-trpc-over-rest",
"status": "approved",
"category": "api-design",
"priority": "high",
"decision": {
"rationale": "tRPC provides end-to-end type safety from client to server, eliminating whole classes of bugs. No need to maintain separate API types or OpenAPI schemas. LLMs can understand tRPC routes better than REST endpoints.",
"context": "We're building a Next.js monorepo with TypeScript. Both frontend and backend are in the same repo. Type safety across API boundaries is critical.",
"alternatives": [
"REST with OpenAPI codegen (rejected - separate schema maintenance burden)",
"GraphQL (rejected - overkill for our use case, steeper learning curve)",
"Server actions only (rejected - need explicit API layer for mobile apps)"
]
},
"metadata": {
"proposedBy": "tech-lead-jordan",
"decidedBy": "engineering-team",
"dateProposed": "2024-01-05",
"dateDecided": "2024-01-12"
}
},
{
"id": "avoid-prisma-use-raw-sql",
"status": "approved",
"category": "database",
"priority": "high",
"decision": {
"rationale": "Prisma's query builder abstracts away too much control. We need raw SQL for complex joins, CTEs, and performance optimization. Prisma migrations are also limiting for our schema complexity.",
"context": "Our database schema has 80+ tables with complex relationships. We frequently use window functions, CTEs, and advanced PostgreSQL features that Prisma doesn't support well.",
"alternatives": [
"Use Prisma with raw SQL escape hatches (rejected - worst of both worlds)",
"Use Kysely query builder (considered - may adopt later if raw SQL becomes unmanageable)",
"Use Drizzle ORM (rejected - still too much abstraction)"
]
},
"metadata": {
"proposedBy": "database-specialist-taylor",
"decidedBy": "engineering-team",
"dateProposed": "2024-04-20",
"dateDecided": "2024-04-28"
}
}
]
}
Integration Patterns
Pattern 1: Pre-Scan Learning
Load learning file before scanning:
const learningContext = await loadLearningFile('eslint-rules-learning.json');
const scanResults = await scanCodebase(learningContext);
Pattern 2: Post-Decision Updates
Update learning file after team reviews:
const newRules = await reviewProposedRules(scanResults);
await updateLearningFile('eslint-rules-learning.json', newRules);
Pattern 3: Reference Past Decisions in Output
LLM output should reference learning file:
## Suggested Rule: no-implicit-any
**Problem**: Found 23 instances of implicit 'any' types
**Why this is different from rejected rules**:
- Unlike 'max-lines-per-function' (rejected for being too opinionated),
this rule has clear correctness criteria: implicit 'any' is always a mistake
- Similar to approved rule 'explicit-function-return-type' which also
enforces type explicitness
**Recommendation**: APPROVE (high priority)
Pattern 4: Periodic Learning File Review
Schedule quarterly reviews:
// In scheduled workflow
const oldDecisions = learningFile.decisions.filter(d => {
const age = Date.now() - new Date(d.metadata.dateDecided).getTime();
const ageInMonths = age / (1000 * 60 * 60 * 24 * 30);
return ageInMonths > 12;
});
if (oldDecisions.length > 0) {
console.log('These decisions are over 1 year old. Review if still valid:');
oldDecisions.forEach(d => {
console.log(`- ${d.id}: ${d.decision.rationale}`);
});
}
Benefits
Benefit 1: No Redundant Suggestions
Before learning files:
Monthly scanner run: 20 suggestions
- 8 new ideas
- 12 already rejected (60% noise!)
Review time: 2 hours (reviewing same ideas repeatedly)
After learning files:
Monthly scanner run: 8 suggestions
- 8 new ideas
- 0 already rejected (0% noise!)
Review time: 30 minutes (only reviewing novel ideas)
Time saved: 1.5 hours per month = 18 hours/year
Benefit 2: Context Preservation
Scenario: Team member leaves
Before learning files:
Knowledge: Lost forever
New members: "Why did we decide this?"
Team: "No idea, that person left"
Result: Re-debate decisions, potential regressions
After learning files:
Knowledge: Preserved in learning file
New members: "Why did we decide this?"
Team: "Check architecture-decisions.json"
Result: Understand rationale, maintain consistency
Benefit 3: Audit Trail
Before learning files:
Question: "When did we decide to use tRPC?"
Answer: "Uh... check commit history? Search Slack?"
Result: Hours of searching, might not find answer
After learning files:
Question: "When did we decide to use tRPC?"
Answer: "Check architecture-decisions.json"
Result: Instant answer with full context and rationale
Benefit 4: LLM Teaching
Before learning files:
LLM: "Have you considered using classes for services?"
Team: "We already decided against that!"
LLM: (next session) "Have you considered using classes?"
Result: LLM never learns project philosophy
After learning files:
LLM reads: "use-factory-pattern-over-classes: approved"
LLM: "I see you use factory functions. I'll suggest improvements to your existing factories."
Result: LLM aligns with project philosophy
Best Practices
1. Keep Rationales Detailed
// ❌ Bad: Vague rationale
{
"rationale": "We prefer factories"
}
// ✅ Good: Detailed rationale
{
"rationale": "Factory functions provide better dependency injection (can pass deps as params), avoid 'this' binding issues (no context confusion), enable better tree-shaking (no class prototype), and are easier to mock in tests (just pass different functions). Classes add complexity without benefits for our functional codebase."
}
2. Capture Alternatives Considered
// ❌ Bad: No alternatives
{
"alternatives": []
}
// ✅ Good: Show what was considered
{
"alternatives": [
"Use classes for consistency with OOP (rejected - we're not doing OOP)",
"Mix factories and classes (rejected - inconsistent)",
"Use classes only for stateful services (rejected - no stateful services)"
]
}
3. Include Context
// ❌ Bad: No context
{
"context": ""
}
// ✅ Good: Relevant context
{
"context": "Our codebase is primarily functional. We don't use inheritance or polymorphism. Services are small, focused functions. Classes would be ceremonial overhead."
}
4. Review and Prune Periodically
// Mark outdated decisions as superseded
{
"id": "use-webpack",
"status": "superseded",
"metadata": {
"supersededBy": "use-vite",
"supersededDate": "2024-06-01"
}
}
5. Use Consistent Categories
Define standard categories:
const CATEGORIES = [
'best-practices',
'code-style',
'performance',
'security',
'type-safety',
'architecture',
'testing',
'documentation',
] as const;
6. Link to Related Decisions
{
"id": "use-server-actions-for-mutations",
"decision": {
"rationale": "...",
"relatedDecisions": [
"use-trpc-over-rest (complements tRPC for queries)"
]
}
}
Common Pitfalls
❌ Pitfall 1: Not Loading Learning File
Problem: LLM doesn’t read learning file before generating suggestions
Solution: Always load and inject into prompt
❌ Pitfall 2: Vague Rationales
Problem: “We don’t like this” doesn’t help future LLMs or team members
Solution: Explain why with concrete details
❌ Pitfall 3: Not Updating After Decisions
Problem: Learning file becomes stale, loses value
Solution: Make updating part of decision workflow
❌ Pitfall 4: Too Many Categories
Problem: 50 different categories makes file hard to navigate
Solution: Limit to 5-10 standard categories
❌ Pitfall 5: Not Versioning Learning Files
Problem: No history of how decisions evolved
Solution: Commit learning files to git
Integration with Other Patterns
Combine with Scheduled LLM Analysis
Use learning files in monthly scanners:
// In scheduled workflow
const learningFiles = await Promise.all([
loadLearningFile('eslint-rules-learning.json'),
loadLearningFile('architecture-decisions.json'),
loadLearningFile('dependency-decisions.json'),
]);
const scanResults = await runMonthlyAnalysis(learningFiles);
Combine with Hierarchical CLAUDE.md
Reference learning files in CLAUDE.md:
# Architecture Decisions
All architectural decisions are documented in `architecture-decisions.json`.
Before proposing new patterns, read this file to understand our philosophy.
Key decisions:
- Factory pattern over classes (see architecture-decisions.json)
- tRPC over REST (see architecture-decisions.json)
- Raw SQL over Prisma (see architecture-decisions.json)
Combine with Repository Digest Reports
Include learning file summary in digests:
# Monthly Digest
## New Decisions This Month
- **Approved**: `no-magic-numbers` ESLint rule
- **Rejected**: `max-file-lines` (too opinionated)
- **Superseded**: Webpack → Vite (build tool upgrade)
Measuring Success
Metric 1: Redundancy Rate
Redundant suggestions = (Already rejected / Total suggestions) × 100
Target: <5% redundancy
Before learning files: 60% redundancy
After learning files: 2% redundancy
Metric 2: Review Time
Avg review time per suggestion:
Before learning files: 15 minutes (re-debating rejected ideas)
After learning files: 5 minutes (only novel ideas)
Time saved: 10 min/suggestion
Metric 3: Onboarding Time
Time to understand architectural decisions:
Before learning files: 2-4 weeks (tribal knowledge, asking around)
After learning files: 2-3 days (read learning files)
Time saved: 1.5-3.5 weeks per new hire
Metric 4: Decision Re-litigation
How often do we re-debate past decisions?
Before learning files: 2-3 times per quarter
After learning files: <1 time per year
Conclusion
Learning files transform stateless LLMs into learning systems that remember and respect past decisions.
Key Benefits:
- No redundant suggestions: LLMs stop re-proposing rejected ideas
- Preserved context: Rationales survive team changes
- Audit trail: Complete history of architectural decisions
- LLM teaching: AI learns project philosophy over time
- Faster onboarding: New members understand decisions instantly
Implementation Checklist:
- Define learning file schema
- Create learning files for each domain (linting, architecture, dependencies)
- Update LLM scanners to load learning files before generating suggestions
- Create workflow for updating learning files after decisions
- Reference learning files in CLAUDE.md
- Schedule periodic learning file reviews
- Track redundancy rate and review time metrics
The result: Institutional memory that doesn’t degrade, AI that learns from your team’s decisions, and dramatically reduced time wasted on redundant discussions.
Related Concepts
- Hierarchical Context Patterns – Reference learning files in CLAUDE.md documentation hierarchy
- Error Messages as Training – Complementary pattern: ERRORS.md tracks mistakes while learning files track decisions
- Building the Factory – Learning files are meta-infrastructure that compounds knowledge over time
- Few-Shot Prompting with Project Examples – Learning file decisions can serve as few-shot examples for LLMs
- Custom ESLint Rules for Determinism – Automate enforcement of approved decisions from learning files
- LLM Code Review CI – Use learning files to guide automated code review
- Semantic Naming Patterns – Consistent naming conventions for learning file entries
References
- JSON Schema for Structured Data – Official JSON Schema specification for defining learning file structure
- Architectural Decision Records (ADR) – Traditional ADR format that inspired learning files pattern
