WEBVTT

NOTE AI Engineering Visualised: https://understandingdata.com/ai-engineering-visualised/draft-critique-revise/

1
00:00:00.220 --> 00:00:15.238
Asking a model to keep improving a draft makes it different rather than better, so this loop scores every version of an outage update against one written rubric: five checks, three in code and two by a model.

2
00:00:15.238 --> 00:00:26.056
Trace one update: the writer model turns the team's internal outage log into a first draft of 136 words, far over the 80-word limit.

3
00:00:26.056 --> 00:00:41.613
Next, a second call, the critic, scores that draft against all five criteria and returns JSON, and it passes the start time, fooled by the only 14:02 in the text, which is about pending orders.

4
00:00:41.680 --> 00:00:54.871
So your loop re-checks, for free, the three things code can count, and where code can count, code decides: the start time fails, and the draft has two failures, not one.

5
00:00:54.871 --> 00:01:03.533
The writer gets one target, the start time, with the whole rubric in view, and version two comes back in 43 words.

6
00:01:03.533 --> 00:01:15.894
The part people miss is that the loop then re-scores all five criteria, not only the target, and version two passes every one, so the loop stops and keeps it.

7
00:01:15.894 --> 00:01:27.674
Here is the failure mode, set up on purpose: a stakeholder asks only for a clearer impact and the reason why, and that edit goes to the writer without the rubric.

8
00:01:27.674 --> 00:01:38.261
Version three is clearer, but at 103 words it fails the length check, and the critic says it blames the vendor, so two passing checks turn red.

9
00:01:38.261 --> 00:01:51.094
The fix sends the same ask with the rubric back and version two in view, and version four keeps the new detail, drops the blame and passes all five in 58 words.

10
00:01:51.094 --> 00:02:07.872
So let code decide whatever it can count, show the whole rubric with every edit, re-score every criterion after it, and cap the laps, since each lap costs two model calls and a loop that never converges must still stop.