Retry and repair

When a guardrail rejects an output you have two options. Regenerate from the same prompt and hope non-determinism helps, or tell the model what was wrong and let it fix it. The second is strictly better, because the error message is information the first attempt did not have.

The loop

async function generateValid(prompt, validate, maxAttempts = 3) {
  const messages = [{ role: 'user', content: prompt }]

  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    const output = await generate(messages)
    const result = validate(output)
    if (result.ok) return output

    // the failure becomes the next instruction
    messages.push({ role: 'assistant', content: output })
    messages.push({
      role: 'user',
      content: `That failed validation: ${result.error}. Fix only that and return the corrected version.`,
    })
  }
  throw new Error(`no valid output after ${maxAttempts} attempts`)
}

Two details matter. The failed attempt stays in the history, so the model can see what it did. And the instruction says fix only that, or you get a wholesale rewrite that breaks something which was previously fine.

Getting the error message right

The quality of the repair is bounded by the quality of the error. "invalid input" gives the model nothing. "field 'dueDate' must be ISO 8601, got 'next Tuesday'" gets fixed on the first retry almost every time.

Write validation errors as if the reader will act on them, because now something does. This is the same discipline as writing good compiler errors, and it pays off much faster here.

Bound it

• Cap the attempts. Three is usually right. An unbounded repair loop is a way to spend money discovering the model cannot satisfy your schema.
• Watch what repeats. If one field fails repeatedly across many requests, the schema or the prompt is wrong, not the model. The retry rate is a diagnostic, so track it.
• Know when not to retry. If validation failed because retrieval returned nothing relevant, no amount of repair helps. Fix the retrieval.