Summary
Large language models exhibit a U-shaped attention pattern: they attend most strongly to information at the beginning and end of their context window, while information in the middle receives reduced attention. This “lost in the middle” effect causes relevant information to be missed even when present in context. Mitigation strategies include positioning critical content at extremes, using progressive disclosure to avoid middle-stuffing, chunking with summaries, and query-anchored context placement.
The Problem
You’ve carefully curated your context. The relevant code is included. The documentation is there. The examples are perfect. Yet the LLM ignores crucial information and generates incorrect output.
The culprit is often not what you included, but where you placed it.
The U-Shaped Attention Curve
Research from “Lost in the Middle: How Language Models Use Long Contexts” (Liu et al., 2023) documented a striking pattern:
``` Attention Strength ^ | ████ ████ | ████ ████ | ████ ████ | ████ ▓▓▓▓ ▓▓▓▓████ | ████ ▓▓▓▓ ░░░░░░░░░ ▓▓▓▓████ | ████ ▓▓▓▓ ░░░░░░░░░ ▓▓▓▓████ +-----------------------------------> Position Start Middle End
████ = High attention (80-100%) ▓▓▓▓ = Medium attention (50-80%) ░░░░ = Low attention (20-50%) ```
Key findings:
- Information at positions 1-10% and 90-100% is processed most accurately
- Information at positions 40-60% (middle) shows significant accuracy degradation
- Effect is more pronounced as context length increases
- Models trained on long contexts still exhibit the pattern
Real-World Impact
Scenario: 20K token context with relevant code in middle
``` Context Structure: ├── System prompt (1K tokens) - Position: 0-5% ├── CLAUDE.md (2K tokens) - Position: 5-15% ├── Type definitions (3K tokens) - Position: 15-30% ├── Irrelevant utility files (5K tokens) - Position: 30-55% ├── THE RELEVANT CODE (2K tokens) - Position: 55-65% ← IN THE DANGER ZONE ├── More utilities (4K tokens) - Position: 65-85% └── Test examples (3K tokens) - Position: 85-100%
Result: Model may miss the relevant code entirely! ```
Symptoms of Lost-in-the-Middle
- Ignoring provided examples: You included exact patterns, but output doesn’t follow them
- Missing explicit constraints: Constraints were in context but violated
- Not using provided code: Relevant code included but model writes from scratch
- Selective blindness: Model acknowledges context exists but doesn’t use it
- Recency bias: Model heavily weights most recent context, ignores middle
Why This Happens
Attention Mechanism Limitations
Transformer attention has computational biases:
``` Attention Score = softmax(Q × K^T / √d_k) × V
Where:
- Q (Query) = What we’re looking for
- K (Key) = What’s available in context
- V (Value) = The content itself ```
Positional encodings create implicit biases:
- Beginning tokens: Strong signal from initial position embeddings
- End tokens: Strong signal from recency in autoregressive attention
- Middle tokens: Weaker relative position signals, competition from both ends
Long-Context Training Gaps
Even models trained on long contexts show the pattern because:
- Training data has important information distributed differently
- Attention patterns learned from typical document structures
- Most training documents front-load key information
- Autoregressive training creates recency effects
Mitigation Strategies
Strategy 1: Critical Content at Extremes
Place most important information at beginning and end:
```typescript interface ContextStructure { beginning: string[]; // High attention zone (0-15%) middle: string[]; // Lower attention zone (15-85%) end: string[]; // High attention zone (85-100%) }
function structureContext(items: ContextItem[]): ContextStructure { // Sort by importance const sorted = [...items].sort((a, b) => b.importance - a.importance);
const total = sorted.length; const extremeCount = Math.ceil(total * 0.3); // 30% at extremes
return { beginning: sorted.slice(0, extremeCount / 2), middle: sorted.slice(extremeCount / 2, -extremeCount / 2), end: sorted.slice(-extremeCount / 2) }; } ```
Application:
```markdown
BEGINNING: Critical constraints (high attention)
- Return types must be Result<T, E>
- Never throw exceptions
- All functions must be pure
MIDDLE: Background context (lower attention OK)
- Project history...
- Architecture documentation...
- Less critical utilities...
END: Task-specific requirements (high attention)
- The specific function to implement
- Required tests to pass
- Current file context ```
Strategy 2: Query-Anchored Context
Place task-relevant context near the query (end of prompt):
```typescript function buildQueryAnchoredContext( staticContext: string, relevantCode: string, query: string ): string { // Static context first (less critical positioning) // Relevant code right before query (high attention zone) return ` ${staticContext}
Relevant Code (Read This Carefully)
${relevantCode}
Your Task
${query} `; } ```
Before (Lost in middle):
``` System prompt -> Types -> Utilities -> RELEVANT CODE -> More code -> Task ^^^^^^^^^^^^^^^^ (Position 50-60%, low attention) ```
After (Query-anchored):
``` System prompt -> Types -> Utilities -> More code -> RELEVANT CODE -> Task ^^^^^^^^^^^^^^^^ (Position 85-95%, high attention) ```
Strategy 3: Progressive Disclosure
Don’t stuff everything into context. Load on-demand:
```typescript // BAD: Load all 50 files into middle of context const context = allFiles.map(f => f.content).join(‘\n’);
// GOOD: Load only relevant files, position strategically
async function buildProgressiveContext(task: Task): Promise
// Skip irrelevant files entirely - they would just fill the middle return ` ${coreContext}
Files Relevant to This Task
${relevantFiles.map(f => f.content).join(‘\n’)}
Task
${task.description} `; } ```
Strategy 4: Chunking with Summaries
Break long content into chunks with summary headers:
```typescript function chunkWithSummaries( content: string, chunkSize: number = 500 ): string { const chunks = splitIntoChunks(content, chunkSize);
return chunks.map((chunk, i) => { const summary = generateSummary(chunk); // Brief summary return `
Chunk ${i + 1}: ${summary}
${chunk} `; }).join(‘\n’); } ```
Why this helps:
- Summaries at chunk boundaries act as attention anchors
- Model can scan summaries even if chunk content is in low-attention zone
- Reduces effective “middle” by creating mini-beginnings throughout
Strategy 5: Repeated Key Information
Repeat critical constraints at multiple positions:
```typescript const constraints = ` CRITICAL: Return Result<T, E>, never throw exceptions. `;
function buildContextWithRepeats( staticContext: string, code: string, task: string ): string { return `
${constraints}
${staticContext}
${constraints}
Code
${code}
${constraints}
Task
${task} `; } ```
Strategy 6: Attention Anchors
Use formatting that naturally draws attention:
```markdown
⚠️ CRITICAL CONSTRAINT ⚠️
Never use throw statements. Return Result types only.
🎯 RELEVANT CODE 🎯
```typescript // This is the function you need to modify function processUser(user: User): Result<ProcessedUser, Error> { // ... } ```
📋 YOUR TASK 📋
Implement the missing validation logic. ```
Symbols and formatting create visual anchors that influence attention patterns.
Strategy 7: Hierarchical Context with Pointers
Use pointers from high-attention zones to middle content:
```markdown
System Prompt (Beginning - High Attention)
Key patterns are defined in Section 3 below. You MUST follow them.
Section 1: Background
...general info...
Section 2: Architecture
...architecture details...
Section 3: PATTERNS TO FOLLOW (Critical)
...the important patterns...
Section 4: Task (End - High Attention)
Implement following the patterns in Section 3. ```
The pointer from beginning creates a mental link that increases attention to Section 3.
Implementation: Context Position Optimizer
```typescript interface ContextItem { content: string; tokens: number; importance: number; // 0-1, higher = more important type: ‘constraint’ | ‘example’ | ‘code’ | ‘documentation’ | ‘task’; }
interface PositionedContext { items: Array<{ item: ContextItem; position: number }>; totalTokens: number; attentionCoverage: number; // How much important content is in high-attention zones }
function optimizeContextPosition( items: ContextItem[], maxTokens: number ): PositionedContext { // Filter to fit budget const sorted = [...items].sort((a, b) => b.importance - a.importance); const selected: ContextItem[] = []; let totalTokens = 0;
for (const item of sorted) { if (totalTokens + item.tokens <= maxTokens) { selected.push(item); totalTokens += item.tokens; } }
// Position optimization const highAttentionItems = selected.filter(i => i.importance > 0.7); const mediumItems = selected.filter(i => i.importance <= 0.7 && i.importance > 0.4); const lowItems = selected.filter(i => i.importance <= 0.4);
// Constraints always at beginning const constraints = highAttentionItems.filter(i => i.type === ‘constraint’);
// Task always at end const tasks = highAttentionItems.filter(i => i.type === ‘task’);
// Examples and code near end (high attention) const examples = highAttentionItems.filter(i => i.type === ‘example’ || i.type === ‘code’ );
// Build positioned context const positioned: ContextItem[] = [ ...constraints, // Beginning: constraints ...lowItems, // Middle: low importance (OK to have reduced attention) ...mediumItems, // Middle-end transition ...examples, // Near end: critical code/examples ...tasks // End: task description ];
// Calculate positions and attention coverage let currentPosition = 0; const result: Array<{ item: ContextItem; position: number }> = [];
for (const item of positioned) { const startPos = currentPosition / totalTokens; result.push({ item, position: startPos }); currentPosition += item.tokens; }
// Calculate how much important content is in high-attention zones const highAttentionThreshold = 0.15; // 0-15% and 85-100% const importantInHighAttention = result.filter(({ item, position }) => item.importance > 0.7 && (position < highAttentionThreshold || position > (1 - highAttentionThreshold)) );
const totalImportant = selected.filter(i => i.importance > 0.7) .reduce((sum, i) => sum + i.tokens, 0); const importantInZones = importantInHighAttention .reduce((sum, { item }) => sum + item.tokens, 0);
return { items: result, totalTokens, attentionCoverage: totalImportant > 0 ? importantInZones / totalImportant : 1 }; } ```
Measuring Effectiveness
Metric 1: Position Score
```typescript function calculatePositionScore(items: PositionedItem[]): number { let score = 0; let maxScore = 0;
for (const { item, position } of items) { const importance = item.importance; maxScore += importance;
// High attention zones: 0-15% and 85-100%
if (position < 0.15 || position > 0.85) {
score += importance * 1.0; // Full credit
} else if (position < 0.30 || position > 0.70) {
score += importance * 0.7; // Partial credit
} else {
score += importance * 0.4; // Low credit (danger zone)
}
}
return maxScore > 0 ? score / maxScore : 1; }
// Target: positionScore > 0.8 ```
Metric 2: Constraint Compliance Rate
```typescript interface ComplianceMetrics { constraintsTotal: number; constraintsFollowed: number; complianceRate: number; positionCorrelation: number; // Do well-positioned constraints have higher compliance? }
// Track whether constraints at different positions are followed // High correlation = position matters for your use case ```
Metric 3: A/B Testing Position
```typescript async function abTestPositioning( context: ContextItem[], task: string, numTrials: number = 10 ): Promise<{ original: number; optimized: number }> { let originalSuccess = 0; let optimizedSuccess = 0;
for (let i = 0; i < numTrials; i++) { // Original: naive ordering const originalContext = context.map(c => c.content).join(‘\n’); const originalOutput = await llm.generate(originalContext + ‘\n’ + task); if (meetsRequirements(originalOutput)) originalSuccess++;
// Optimized: position-aware ordering
const optimized = optimizeContextPosition(context, 100000);
const optimizedContext = optimized.items.map(p => p.item.content).join('\n');
const optimizedOutput = await llm.generate(optimizedContext + '\n' + task);
if (meetsRequirements(optimizedOutput)) optimizedSuccess++;
}
return { original: originalSuccess / numTrials, optimized: optimizedSuccess / numTrials }; } ```
Best Practices
1. Audit Your Context Structure
```typescript function auditContextPositions(context: string, markers: string[]): void { const totalLength = context.length;
for (const marker of markers) { const position = context.indexOf(marker); if (position === -1) { console.log(`${marker}: NOT FOUND`); } else { const pct = (position / totalLength * 100).toFixed(1); const zone = position / totalLength < 0.15 ? ‘HIGH’ : position / totalLength > 0.85 ? ‘HIGH’ : ‘LOW’; console.log(`${marker}: ${pct}% (${zone} attention zone)`); } } }
// Usage: auditContextPositions(context, [ ‘CRITICAL CONSTRAINT:’, ‘function processUser’, ‘Your task is to’ ]); ```
2. Use Position-Aware Templates
```typescript const POSITION_OPTIMIZED_TEMPLATE = `
{{constraints}}
{{background}} {{documentation}} {{utilities}}
{{relevantCode}}
{{examples}}
{{task}} `; ```
3. Avoid Middle-Heavy Context
```typescript // BAD: All important content in middle const badContext = ` System prompt... Type definitions... ${criticalConstraints} // Position: 40% ${relevantCode} // Position: 50% ${examples} // Position: 60% More utilities... Task... `;
// GOOD: Important content at extremes const goodContext = ` ${criticalConstraints} // Position: 5% System prompt... Type definitions... More utilities... ${relevantCode} // Position: 85% ${examples} // Position: 90% ${task} // Position: 95% `; ```
4. Test Position Sensitivity
Before deploying, test if your specific task is position-sensitive:
```typescript
async function testPositionSensitivity(
content: string,
task: string
): Promise
// If outputs differ significantly, position matters const unique = new Set(results.map(normalizeOutput)).size; return unique > 1; } ```
Common Pitfalls
Pitfall 1: Assuming Position Doesn’t Matter
```typescript // BAD: Random/chronological ordering const context = files.map(f => f.content).join(‘\n’);
// GOOD: Intentional positioning const context = buildPositionOptimizedContext(files, task); ```
Pitfall 2: Overloading the Middle
```typescript // BAD: Stuffing everything because “it’s in context” const context = ` ${systemPrompt} ${allDocumentation} // 10K tokens in middle ${allUtilities} // 15K tokens in middle ${task} `;
// GOOD: Only include what’s needed, position strategically const context = ` ${systemPrompt} ${relevantDocumentation} // Only 2K tokens ${relevantCode} // Near end ${task} `; ```
Pitfall 3: Not Repeating Critical Constraints
```typescript // BAD: Constraint mentioned once in middle // GOOD: Constraint at beginning AND near task ```
Pitfall 4: Ignoring Context Length Effects
The lost-in-the-middle effect is worse with longer contexts:
```typescript // Short context (5K tokens): Middle still gets ~60% attention // Long context (100K tokens): Middle may get <30% attention
// Adjust strategy based on context length function getPositioningStrategy(contextTokens: number) { if (contextTokens < 10000) { return ‘standard’; // Less aggressive positioning needed } else if (contextTokens < 50000) { return ‘optimized’; // Use position optimization } else { return ‘aggressive’; // Repeat constraints, minimal middle content } } ```
Conclusion
The lost-in-the-middle effect is a real and measurable phenomenon. Information position in context directly impacts whether it gets used. By understanding and optimizing for attention patterns, you can significantly improve LLM output quality.
Key Takeaways:
- U-shaped attention: Beginning and end get most attention, middle gets less
- Position critical content: Constraints at start, task-relevant code near end
- Avoid middle-stuffing: Progressive disclosure beats context cramming
- Use anchors: Summaries, formatting, and repetition boost attention
- Measure position score: Track how much important content is in high-attention zones
- Test sensitivity: Not all tasks are equally position-sensitive
The difference between “the model ignored my context” and “the model followed my context perfectly” is often just position.
Related
- Context Rot Auto-Compacting - Temporal context degradation (different from positional)
- Progressive Disclosure Context - Load context on-demand to avoid middle-stuffing
- Token Budgeting Strategies - Allocate tokens wisely
- Information Theory for Coding Agents - Why position affects information transfer
- Hierarchical Context Patterns - Structure that aids attention
- Prompt Caching Strategy - Cache stable context at optimal positions
- Context Debugging Framework - Debug when context is ignored
References
- Lost in the Middle: How Language Models Use Long Contexts - Liu et al., 2023. The foundational research on this phenomenon.
- Anthropic Prompt Engineering Guide - Best practices for context structure
- Attention Is All You Need - Vaswani et al. Original transformer paper explaining attention mechanisms
Part of the field guide
This is one of my field notes in Context Engineering, a plain-English guide to deciding what a model sees. The terms behind it are defined in the Context Engineering Dictionary.
