Prompting

Few-shot prompting: examples define the pattern

Eight support tickets, one router, six prompts: no examples, useful examples, repetitive ones, one contradiction, one more example, and the house rules written down. Every landing is a real model answer recorded ten times, so you can see which examples resolve ambiguity and which only cost tokens.

Examples in the prompt

Loading 3D view

House rulesRouting a support ticket looks easy, but the right queue often hangs on house rules, like single sign-on being a plan feature that belongs to the account team, and no model can guess those.

Narrated with James Phoenix's AI voice.

Useful examples

Prompt

7/8

Tickets right, by majority

70/80

Runs right

10/10

Single sign-on to account

105

Prompt tokens per ticket

Useful examples: 7 of 8 tickets routed correctly by majority, 70 of 80 runs right, about 105 prompt tokens a ticket; single sign-on went to account in 10 of 10 runs.

A support router has to put every ticket in one queue: billing, bug, feature request or account. Most tickets are easy. The hard ones hinge on house rules that live in one company's head, like single sign-on being a Business plan feature that the account team owns, or a double charge caused by a crash counting as billing rather than a bug. No model can guess those, and the question the film asks is how much a handful of examples can teach it.

Every landing in the film is a real answer. Each of the eight unseen tickets was sent to `google/gemini-3.8-flash` through OpenRouter at temperature 0 on 11 September 2026, ten times under each of six prompts, 480 calls in all. The prompt was the routing instruction, then the examples as ticket and queue pairs, then the new ticket. A card lands in the queue its majority answer chose, and its badge shows how many of the ten runs matched the house rules.

With no examples the model got five of the eight tickets right. It sent single sign-on to feature requests in all ten runs, and the ticket about adding more than ten seats flipped between billing and account. Three examples chosen from the edges of the rules (a crash that still left a bill, a request for SAML login, and slow search) lifted it to seven of eight, and the SAML example fixed single sign-on in every run. The audit log ticket, which no example sat near, stayed wrong in every run. Three near-identical refund examples did little: they steadied the seats ticket, but covered a case the model already handled, left single sign-on and the audit log wrong in every run, and doubled the tokens of each call.

The failure mode is a contradictory example. Adding one example that labels a double charge after a crash as a bug, with the three good examples still in place, sent that ticket to bug in nine runs of ten. The model trusts a close neighbour more than its own sense of the task, so one mislabelled example near an edge case overrides everything else in the prompt.

The part people miss is that examples teach neighbours, not rules. Adding an example about which admin deleted a project, meant to teach that audit features belong to account, fixed the audit log ticket in only three runs of ten, and knocked the seats ticket down from ten runs right to two. Patching a prompt with one more example moves the answers around it in ways that are hard to predict.

What worked was writing the rule down. The house rules in plain sentences, plus the same three useful examples, routed all eight tickets correctly in all eighty runs. The rules say where the boundaries are, and the examples show what a ticket near each boundary looks like.

An honest note on how this set was chosen. The first recording used tickets whose right queue matched common sense, and the model scored eight of eight with no examples at all, so there was nothing for examples to teach. The tickets here were redesigned around house rules on purpose. The lesson is about ambiguity: when the task is obvious, examples add cost and little else; when it hinges on local rules, choose examples that sit on the boundaries, check that none contradicts the rules, and write the rules down.

The maths

Where a ticket lands: $\hat{q}(t) \;=\; \arg\max_{q} \sum_{r=1}^{10} \big[\, a_r(t) = q \,\big]$
Each ticket was sent ten times under each prompt, and a card lands in the queue most of its ten answers chose. The badge on the card, such as 3/10, counts how many of the ten matched the house rules, so a card can land in the right bin and still be shaky.
The two scores on the board: $\text{tickets} = \sum_{t=1}^{8} \big[\, \hat{q}(t) = q^{*}(t) \,\big], \qquad \text{runs} = \sum_{t=1}^{8} \sum_{r=1}^{10} \big[\, a_r(t) = q^{*}(t) \,\big]$
The first counts tickets whose majority answer is right, out of 8. The second counts every individual answer, out of 80, so it shows the wobble the majority hides: the repetitive, contradictory and one-more-example prompts all score six of eight by majority, but 60, 59 and 65 of 80 runs.
What examples cost: $\text{tokens per call} \;=\; |\,\text{instruction}\,| + \sum_{i=1}^{k} |\,\text{example}_i\,| + |\,\text{ticket}\,|$
Every example rides along with every ticket. The bare prompt cost 44 input tokens a ticket; three near-identical refund examples cost 90, and the rules plus three examples cost 171. A repetitive example is paid for on every call and buys little coverage.
Temperature zero is not a fixed answer: $T = 0 \;\not\Rightarrow\; a_1(t) = a_2(t) = \dots = a_{10}(t)$
The recordings used temperature 0, yet several tickets changed answer between runs, such as the seats ticket, which went to account in four runs of ten with no examples. Batching and floating point on the provider side make repeated calls differ, which is why every cell was recorded ten times rather than once.

Related terms

More visualisations

Building with language models?

These explainers come out of the work. If you want the same thinking applied to your own system, that is what I do.

See how I can help