The Problem
When AI coding agents work on your codebase, they have tremendous power – and with that power comes the risk of reaching invalid states.
Consider these scenarios:
Scenario 1: Test Fixing Gone Wrong
You ask the LLM to “fix the failing authentication tests.” The LLM:
- Identifies the test expecting
user.role === 'admin' - Sees the service returns
user.role === 'administrator' - Modifies the service code to return
'admin'instead of fixing the test - Breaks production code that depends on
'administrator'
Root cause: The LLM had unrestricted access to both test and service code. It chose the “easier” path of modifying the service.
Scenario 2: Type Safety Bypass
You ask the LLM to “make this TypeScript code compile.” The LLM:
- Identifies type mismatch:
stringprovided whereUserexpected - Adds
as Usertype assertion to bypass the type system - Ships code with runtime type errors
Root cause: The LLM had access to type assertions, which bypass type safety.
Scenario 3: Architectural Pattern Violation
You ask the LLM to “add logging to this function.” The LLM:
- Identifies need for logging
- Adds
console.log()instead of using the structured logger - Violates established logging patterns
- Logs don’t appear in production monitoring
Root cause: The LLM had access to console.log, which should be forbidden.
The Common Pattern
In all cases, we’re validating correct behavior instead of preventing incorrect behavior:
// Traditional validation approach
if (modifiedFile.startsWith('src/') && mode === 'test-fixing') {
throw new Error('Cannot modify source files during test fixing');
}
This catches the error after the LLM has done the work. The LLM wasted tokens, you wasted time, and now you need to retry.
Better approach: Make it impossible for the LLM to modify source files during test fixing.
The Solution: Sculpting the Computation Graph
Core Principle
Prevent what you don’t want, not just validate what you do want.
Inspired by type systems that catch errors at compile-time instead of runtime, we sculpt the computation graph to eliminate entire classes of bugs:
// Runtime validation (traditional)
function divide(a: number, b: number): number {
if (b === 0) throw new Error('Division by zero');
return a / b;
}
// Compile-time prevention (type system)
type NonZeroNumber = number & { __brand: 'NonZero' };
function divide(a: number, b: NonZeroNumber): number {
return a / b; // b cannot be zero
}
Similarly for LLMs:
// Validation approach (catches errors after they happen)
if (file.modified && !allowedFiles.includes(file.path)) {
rollback();
throw new Error('File not allowed');
}
// Prevention approach (makes errors impossible)
const llm = createRestrictedLLM({
writableFiles: allowedFiles,
// LLM physically cannot write to other files
});
The Type System Analogy
Type systems are computation graph sculptors:
// Without types: any operation is possible
function process(data) {
return data.toUpperCase(); // Runtime error if data is a number
}
// With types: invalid operations are impossible
function process(data: string) {
return data.toUpperCase(); // Compile-time guarantee
}
Type systems reduce the entropy of possible program states by eliminating invalid states at compile-time.
Similarly, we can reduce the entropy of LLM-generated code by restricting the state space:
// Without restrictions: LLM can reach any state
llm.task('Fix tests'); // Could modify services, tests, config, etc.
// With restrictions: LLM can only reach valid states
llm.withRestrictions({
mode: 'test-fixing',
writableFiles: ['tests/**/*.ts'],
}).task('Fix tests'); // Can only modify tests
Implementation Patterns
Pattern 1: Mode-Based File Access Restrictions
Problem: LLM modifies unrelated files during focused tasks.
Solution: Define modes with explicit file access rules.
// Mode definitions
const MODES = {
'test-fixing': {
writableGlobs: ['tests/**/*.ts', '**/*.spec.ts', '**/*.test.ts'],
readableGlobs: ['**/*'],
rationale: 'Tests should be fixed by modifying tests, not services',
},
'service-development': {
writableGlobs: ['src/**/*.ts', 'packages/**/src/**/*.ts'],
readableGlobs: ['**/*'],
rationale: 'Service development should not modify tests',
},
'code-review': {
writableGlobs: [], // Read-only
readableGlobs: ['**/*'],
rationale: 'Reviews should not modify code',
},
};
// Enforcement mechanism
function validateFileAccess(filePath: string, mode: string, operation: 'read' | 'write'): void {
const modeConfig = MODES[mode];
if (!modeConfig) throw new Error(`Unknown mode: ${mode}`);
const globs = operation === 'write' ? modeConfig.writableGlobs : modeConfig.readableGlobs;
const isAllowed = globs.some(glob => minimatch(filePath, glob));
if (!isAllowed) {
throw new Error(
`Cannot ${operation} ${filePath} in ${mode} mode. ${modeConfig.rationale}`
);
}
}
// Integration with LLM tools
function createRestrictedFileWriter(mode: string) {
return {
writeFile: async (path: string, content: string) => {
validateFileAccess(path, mode, 'write');
await fs.writeFile(path, content);
},
};
}
Claude Code Integration: Use hooks to enforce restrictions.
// .claude/hooks/pre-write.ts
import { minimatch } from 'minimatch';
const MODE = process.env.CLAUDE_MODE || 'default';
const RESTRICTIONS = {
'test-fixing': ['tests/**/*.ts', '**/*.spec.ts'],
'service-development': ['src/**/*.ts'],
};
export default function preWriteHook(filePath: string) {
const allowedGlobs = RESTRICTIONS[MODE];
if (!allowedGlobs) return; // No restrictions for this mode
const isAllowed = allowedGlobs.some(glob => minimatch(filePath, glob));
if (!isAllowed) {
throw new Error(
`Cannot modify ${filePath} in ${MODE} mode. Only ${allowedGlobs.join(', ')} allowed.`
);
}
}
Usage:
# Set mode before asking Claude to fix tests
export CLAUDE_MODE=test-fixing
# Claude can now only modify test files
# Attempts to modify service files will fail immediately
Pattern 2: Custom ESLint Rules for Pattern Enforcement
Problem: LLM uses forbidden patterns (type assertions, console.log, etc.).
Solution: Create custom ESLint rules that fail on forbidden patterns.
// eslint-rules/no-as-type-assertions.ts
import { ESLintUtils } from '@typescript-eslint/utils';
const createRule = ESLintUtils.RuleCreator(
name => `https://docs.example.com/eslint-rules/${name}`
);
export const noAsTypeAssertions = createRule({
name: 'no-as-type-assertions',
meta: {
type: 'problem',
docs: {
description: 'Disallow type assertions using "as" keyword',
recommended: 'error',
},
messages: {
noAs: 'Type assertions with "as" bypass type safety. Use type narrowing (type guards, discriminated unions) instead.',
},
schema: [],
},
defaultOptions: [],
create(context) {
return {
TSAsExpression(node) {
context.report({
node,
messageId: 'noAs',
});
},
};
},
});
// eslint-rules/no-console-log.ts
export const noConsoleLog = createRule({
name: 'no-console-log',
meta: {
type: 'problem',
docs: {
description: 'Disallow console.log in favor of structured logging',
recommended: 'error',
},
messages: {
noConsole: 'Use logger.info() instead of console.log() for structured logging.',
},
schema: [],
},
defaultOptions: [],
create(context) {
return {
MemberExpression(node) {
if (
node.object.type === 'Identifier' &&
node.object.name === 'console' &&
node.property.type === 'Identifier' &&
['log', 'info', 'warn', 'error'].includes(node.property.name)
) {
context.report({
node,
messageId: 'noConsole',
});
}
},
};
},
});
Configuration:
// .eslintrc.json
{
"plugins": ["@custom/eslint-rules"],
"rules": {
"@custom/no-as-type-assertions": "error",
"@custom/no-console-log": "error",
"@custom/no-class-definitions": "error"
}
}
Result: LLM cannot generate code using forbidden patterns. ESLint fails immediately, forcing regeneration with correct patterns.
Pattern 3: TypeScript Configuration for Type Safety
Problem: LLM uses any types or disables type checking.
Solution: Configure TypeScript to forbid escape hatches.
// tsconfig.json
{
"compilerOptions": {
"strict": true,
"noImplicitAny": true,
"strictNullChecks": true,
"strictFunctionTypes": true,
"strictBindCallApply": true,
"strictPropertyInitialization": true,
"noImplicitThis": true,
"alwaysStrict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"noUncheckedIndexedAccess": true,
"noImplicitOverride": true,
"noPropertyAccessFromIndexSignature": true,
"allowUnusedLabels": false,
"allowUnreachableCode": false
}
}
ESLint rules (to catch what TypeScript misses):
{
"rules": {
"@typescript-eslint/no-explicit-any": "error",
"@typescript-eslint/no-unsafe-assignment": "error",
"@typescript-eslint/no-unsafe-member-access": "error",
"@typescript-eslint/no-unsafe-call": "error",
"@typescript-eslint/no-unsafe-return": "error",
"@typescript-eslint/ban-ts-comment": "error"
}
}
Result: LLM cannot bypass type safety. All code must be fully typed.
Pattern 4: Architectural Boundary Enforcement
Problem: LLM violates layered architecture (e.g., UI importing from database layer).
Solution: Use import restrictions to enforce boundaries.
// eslint-rules/no-cross-layer-imports.ts
export const noCrossLayerImports = createRule({
name: 'no-cross-layer-imports',
meta: {
type: 'problem',
docs: {
description: 'Enforce layered architecture boundaries',
recommended: 'error',
},
messages: {
invalidImport: '{{from}} cannot import from {{to}}. Violates layered architecture.',
},
schema: [],
},
defaultOptions: [],
create(context) {
const filename = context.getFilename();
const layer = getLayer(filename);
return {
ImportDeclaration(node) {
const importPath = node.source.value;
const targetLayer = getLayer(importPath);
if (!isAllowedImport(layer, targetLayer)) {
context.report({
node,
messageId: 'invalidImport',
data: {
from: layer,
to: targetLayer,
},
});
}
},
};
},
});
function getLayer(path: string): string {
if (path.includes('/ui/')) return 'ui';
if (path.includes('/api/')) return 'api';
if (path.includes('/services/')) return 'services';
if (path.includes('/database/')) return 'database';
return 'unknown';
}
function isAllowedImport(from: string, to: string): boolean {
const rules = {
ui: ['ui', 'api'], // UI can import from UI and API
api: ['api', 'services'], // API can import from API and services
services: ['services', 'database'], // Services can import from services and database
database: ['database'], // Database can only import from database
};
return rules[from]?.includes(to) ?? false;
}
Result: LLM cannot create cross-layer dependencies. Architecture is enforced.
Pattern 5: Read-Only Review Mode
Problem: Code review LLMs modify code instead of generating comments.
Solution: Remove write permissions entirely.
// review-mode.ts
interface ReviewContext {
mode: 'review';
permissions: {
read: boolean;
write: boolean;
execute: boolean;
};
}
function createReviewLLM(): ReviewContext {
return {
mode: 'review',
permissions: {
read: true,
write: false, // Cannot modify files
execute: false, // Cannot run commands
},
};
}
// Tool implementation
const tools = {
readFile: (path: string) => fs.readFile(path, 'utf-8'),
writeFile: (path: string, content: string) => {
if (!context.permissions.write) {
throw new Error('Write operations not allowed in review mode');
}
fs.writeFile(path, content);
},
generateReviewComment: (file: string, line: number, message: string) => {
// Always allowed
comments.push({ file, line, message });
},
};
Result: Review LLM cannot modify code, only generate review comments.
Real-World Examples
Example 1: Test Fixing Workflow
Scenario: Failing integration tests after API change.
Traditional approach (validation):
$ claude "Fix the failing authentication tests"
# Claude modifies both tests AND service code
# Validation catches service modification
# Rollback required
# Retry with explicit instruction: "Only modify tests"
# Retry with more explicit instruction: "Do NOT modify service code"
# Finally succeeds
Sculpted approach (prevention):
$ export CLAUDE_MODE=test-fixing
$ claude "Fix the failing authentication tests"
# Claude attempts to modify service code
# Pre-write hook blocks immediately: "Cannot modify src/ in test-fixing mode"
# Claude adjusts strategy, modifies only tests
# Success on first try
Result: Faster iteration, no wasted tokens, clearer feedback.
Example 2: Type Safety Enforcement
Traditional approach (validation):
// LLM generates code with type assertions
const user = getUserData() as User;
// Code review catches it
// Request regeneration without type assertions
// LLM generates:
const user = getUserData() as any as User; // Double assertion!
// Another review cycle...
Sculpted approach (prevention):
// ESLint rule: @custom/no-as-type-assertions
// LLM generates:
const user = getUserData() as User;
// ESLint fails immediately
// LLM sees error message: "Use type narrowing instead"
// LLM regenerates with proper type guard:
const data = getUserData();
if (!isUser(data)) {
throw new Error('Invalid user data');
}
const user: User = data;
// ESLint passes
// Success on second try
Result: Type safety enforced automatically, no manual review needed.
Example 3: Logging Pattern Enforcement
Traditional approach:
// LLM adds logging
function processPayment(amount: number) {
console.log('Processing payment:', amount);
// ...
}
// Code review: "Use structured logger"
// LLM regenerates:
function processPayment(amount: number) {
console.info('Processing payment:', amount); // Still wrong!
// ...
}
Sculpted approach:
// ESLint rule: @custom/no-console-log
// LLM generates:
function processPayment(amount: number) {
console.log('Processing payment:', amount);
// ...
}
// ESLint fails: "Use logger.info() instead"
// LLM regenerates:
import { logger } from '@/lib/logger';
function processPayment(amount: number) {
logger.info('Processing payment', { amount });
// ...
}
// ESLint passes
Result: Consistent logging patterns, enforced automatically.
Benefits
1. Fail Fast
Before: LLM does work → validation fails → rollback → retry → validation passes
After: LLM attempts invalid action → blocked immediately → LLM adjusts strategy → success
Time saved: 50-70% reduction in iteration cycles
2. Clearer Intent
Restrictions communicate what’s not allowed:
// Validation (unclear intent)
if (file.path.startsWith('src/')) {
throw new Error('Cannot modify source files');
}
// "Why not? What should I modify instead?"
// Prevention (clear intent)
const mode = {
writableGlobs: ['tests/**/*.ts'],
rationale: 'Tests should be fixed by modifying tests, not services',
};
// "I should modify tests, not services"
3. Enforces Patterns
Invalid patterns become impossible:
// Without restrictions: possible but wrong
const user = data as User;
// With restrictions: impossible
// ESLint error: "Type assertions not allowed"
// LLM must use type guards
4. Reduces Human Review
Automated enforcement reduces manual code review:
// Before: manual review required
"Did the LLM use type assertions?" ✋ Manual check
"Did the LLM modify service code?" ✋ Manual check
"Did the LLM use console.log?" ✋ Manual check
// After: automated enforcement
"Did the LLM use type assertions?" ✅ ESLint checked
"Did the LLM modify service code?" ✅ Hook checked
"Did the LLM use console.log?" ✅ ESLint checked
5. Prevents Entire Classes of Bugs
Like type systems:
// Type system prevents:
- null reference errors
- type mismatches
- undefined function calls
// Computation graph sculpting prevents:
- Cross-layer imports
- Pattern violations
- Workflow escapes
Trade-offs
Con 1: Less Flexibility
Sometimes you need to break rules:
// Legitimate use of type assertion
const element = document.getElementById('app') as HTMLDivElement;
// But rule forbids ALL type assertions
// ESLint error: "Type assertions not allowed"
Mitigation: Use escape hatches with justification:
// eslint-disable-next-line @custom/no-as-type-assertions -- DOM API requires assertion
const element = document.getElementById('app') as HTMLDivElement;
Con 2: Maintenance Overhead
Rules need updating as patterns evolve:
// Pattern changes from functional to class-based
// Must update ESLint rule: no-class-definitions → allow classes
// Must update documentation
// Must notify team
Mitigation: Version rules with codebase versions:
// v1: no-class-definitions (functional patterns)
// v2: allow-classes (class-based patterns)
Con 3: Complexity
More moving parts:
// Traditional: just the code
src/
├── services/
└── tests/
// Sculpted: code + enforcement
src/
├── services/
└── tests/
eslint-rules/
├── no-as-type-assertions.ts
├── no-console-log.ts
└── no-cross-layer-imports.ts
.claude/
└── hooks/
└── pre-write.ts
Mitigation: Document all rules in CLAUDE.md:
# CLAUDE.md
## Enforced Rules
### Type Safety
- No type assertions (`as`)
- No `any` types
- Strict null checks
### Architectural Boundaries
- UI cannot import from database
- Services cannot import from UI
### Pattern Enforcement
- No console.log (use logger)
- No classes (use functions)
When to Use
Use when:
-
Critical patterns that must never be violated
- Type safety in production code
- Security patterns (auth, sanitization)
- Architectural boundaries
-
Task-specific workflows
- Test fixing (only modify tests)
- Code review (read-only access)
- Refactoring (specific file scope)
-
Pattern migration
- Migrating from classes to functions
- Enforcing new logging patterns
- Adopting type-safe patterns
Don’t use when:
-
Exploratory development
- Prototyping new features
- Experimenting with patterns
- Proof-of-concepts
-
Patterns are unclear
- Team hasn’t decided on patterns
- Multiple valid approaches exist
- Requirements are evolving
-
Escape hatches needed frequently
- Rules block legitimate use cases
- Too many
eslint-disablecomments - Productivity suffers
Implementation Checklist
Phase 1: Identify Critical Patterns
- Review codebase for architectural patterns
- Identify patterns that must never be violated
- Document rationale for each restriction
- Get team buy-in on restrictions
Phase 2: Implement Rules
- Create custom ESLint rules for pattern enforcement
- Configure TypeScript for maximum strictness
- Implement file access restrictions (hooks, modes)
- Add architectural boundary checks
Phase 3: Document Restrictions
- Update CLAUDE.md with all rules and rationale
- Add examples of correct vs incorrect patterns
- Document escape hatches and when to use them
- Create troubleshooting guide for common errors
Phase 4: Test Enforcement
- Test that LLM cannot violate restrictions
- Verify error messages are clear and actionable
- Ensure legitimate use cases still work
- Measure iteration speed improvement
Phase 5: Iterate
- Monitor false positives (legitimate code blocked)
- Adjust rules based on team feedback
- Add new rules as patterns emerge
- Remove rules that cause more harm than good
Best Practices
1. Provide Clear Error Messages
Bad:
throw new Error('Invalid operation');
Good:
throw new Error(
'Cannot modify src/ in test-fixing mode. ' +
'Tests should be fixed by modifying test files, not service code. ' +
'Only tests/**/*.ts files are writable in this mode.'
);
2. Document Rationale
Bad:
// Don't use type assertions
Good:
/**
* Disallow type assertions ("as" keyword)
*
* Rationale: Type assertions bypass TypeScript's type system,
* allowing runtime type errors. Use type guards and discriminated
* unions for type narrowing instead.
*
* Example:
*
* // Bad
* const user = data as User;
*
* // Good
* if (!isUser(data)) throw new Error('Invalid user');
* const user: User = data;
*/
3. Start with Critical Rules
Don’t add all rules at once:
// Phase 1: Critical safety rules
- no-as-type-assertions
- no-cross-layer-imports
// Phase 2: Pattern enforcement
- no-console-log
- no-class-definitions
// Phase 3: Workflow restrictions
- test-fixing mode
- code-review mode
4. Measure Impact
Track before/after metrics:
interface Metrics {
avgIterationsPerTask: number;
timeToCompletion: number;
patterViolationsInReview: number;
}
// Before
const before: Metrics = {
avgIterationsPerTask: 4.2,
timeToCompletion: 18, // minutes
patterViolationsInReview: 12, // per week
};
// After
const after: Metrics = {
avgIterationsPerTask: 2.1, // 50% reduction
timeToCompletion: 9, // 50% reduction
patterViolationsInReview: 1, // 92% reduction
};
5. Balance Prevention and Flexibility
Not everything should be prevented:
// Prevent: Critical safety issues
- Type safety violations
- Security bypasses
- Architectural violations
// Validate: Style preferences
- Formatting (Prettier handles this)
- Naming conventions (suggestions, not errors)
- Comment style
Conclusion
Making invalid states impossible is more powerful than validating correct states.
By sculpting the computation graph – restricting file access, forbidding patterns, enforcing boundaries – you:
- Fail fast: Catch errors at generation time, not review time
- Communicate intent: Restrictions explain what’s not allowed
- Enforce patterns: Invalid code becomes impossible
- Reduce review: Automated enforcement replaces manual checks
- Prevent bug classes: Entire categories of errors eliminated
The key insight: Like type systems that catch errors at compile-time instead of runtime, sculpt the LLM’s state space to prevent errors before they happen.
Start small: Identify one critical pattern, add one restriction, measure impact. Then expand.
Remember: Not everything should be prevented. Balance prevention (critical rules) with validation (preferences). Use sculpting for patterns that must never be violated.
Related Concepts
- Boundary Enforcement with Layered Architecture – Enforce layer boundaries to prevent cross-layer violations
- DDD Bounded Contexts for LLMs – Bounded contexts prevent domain boundary violations
- Sub-Agent Architecture – Tool access restrictions make invalid agent actions impossible
- Type-Driven Development – Types prevent invalid states at compile-time
- Invariants in Programming and LLM Generation – Invariants define what states are valid
- Custom ESLint Rules for Determinism
- One Way Pattern Consistency
- Verification Sandwich Pattern
References
- Making Impossible States Impossible – Classic talk by Richard Feldman on using types to prevent invalid states
- Parse, Don’t Validate – Blog post on using types to make invalid states unrepresentable
- TypeScript ESLint Rules – Comprehensive list of TypeScript-specific ESLint rules
