Guardrail

Instructions in a prompt are requests. A model usually honours them and sometimes does not, and "usually" is not a property you can build on. A guardrail is the same rule expressed as code that runs whether the model cooperated or not.

Both directions

Input guardrails run before the call: strip or reject material you do not want reaching the model, cap the size of retrieved context, refuse requests that fail an authorisation check.

Output guardrails run after: validate the shape, check claims against sources, verify a generated identifier actually exists.

// output guardrail: every cited id must be real
function checkCitations(answer, sources) {
  const known = new Set(sources.map((s) => s.id))
  const cited = [...answer.matchAll(/\[source:([^\]]+)\]/g)].map((m) => m[1])
  const invented = cited.filter((id) => !known.has(id))
  return { ok: invented.length === 0, invented }
}

That check is a handful of lines and it catches fabricated citations every time, which no amount of prompt instruction achieves.

Why deterministic matters

The point of a guardrail is that it does not share the model's failure modes. Asking a second model call "is this output safe?" is useful, but it is another probabilistic judgement, and it can be wrong in the same direction as the first. A guardrail should be a check whose behaviour you can predict without running it. Schema validation, set membership, regex, an authorisation lookup, a numeric range.

Where a model-based check is genuinely needed, treat it as one signal rather than the gate.

Practical notes

• Fail loudly in development, gracefully in production. You want to see every violation while building, and a sensible fallback for users.
• Log what tripped. A guardrail firing is a signal about your prompt or retrieval, not just an error to swallow. Repeated invented citations means the retrieval is not supplying what the question needs.
• Keep them cheap. A guardrail that costs more than the model call will get removed the first time latency matters.