Using DSL Languages for LLM Harnesses

Unmesh Joshi’s article DSLs Enable Reliable Use of LLMs (martinfowler.com, July 2026) names the endgame of something I have been documenting piecemeal for a year: the smaller the language you make the model speak, the less room it has to be wrong.

The Claim

Joshi’s argument runs like this. LLMs are unreliable in general-purpose languages because a language like Java or TypeScript offers hundreds of valid ways to express the same intent. A domain-specific language strips that variation away. Give the model a few in-context examples of a constrained syntax and it generates correct programs with startling consistency, which is why every model is already good at Mermaid, SQL, and Kubernetes YAML.

His worked example is Tickloom, a framework for building and testing distributed systems. The semantic model fixes every decision that usually goes wrong: single-threaded tick loops, deterministic ordering, logical clocks. On top of it sits an internal Java DSL for failure scenarios, where partition(BYZANTIUM).from(CYRENE) reads as English and an illegal scenario does not compile. A prompt like “reproduce the DDIA non-linearizable quorum read” maps almost directly onto the vocabulary, and the model has nowhere to hallucinate.

A DSL is not a syntax trick. It is a compressed set of design decisions the model no longer gets to re-make. Threading, timing, and delivery semantics stop being open questions in every prompt because the vocabulary already answered them.

Why It Works: Entropy Collapse

This is the same mechanism I described in Entropy in Code Generation, pushed to its limit. Every constraint you add shrinks the space of plausible outputs, and generation quality rises as the space shrinks. A DSL is the most aggressive shrink available: the space of valid programs is so small that a handful of examples covers the entire grammar.

There is a second mechanism that matters more for agents than for single-shot generation. A DSL ships with a deterministic validator: a parser, a schema, or in the internal-DSL case the host language’s compiler. The agent generates, validates, and repairs without a human in the loop, and the errors come back phrased at the level of the domain (“you cannot select an action before choosing a client”) instead of as a stack trace buried in generated plumbing. The validator turns the DSL into a self-checking harness, which is exactly the property I keep paying for elsewhere with lint rules and integration tests.

flowchart LR
    A["English intent"] --> B["LLM emits a DSL program"]
    B --> C{"Deterministic validator"}
    C -- "domain-level error" --> B
    C -- "valid" --> D["Program ships"]

That flowchart is written in Mermaid, and the block is its own demonstration: Mermaid is one of the DSLs every frontier model already speaks fluently, and I would trust a model to regenerate it far more than I would trust one to redraw the same thing in raw SVG.

The Ladder

Reading the article, I realised my context-engineering notes have been climbing one ladder without naming it. Each rung constrains the model harder than the last:

1. Prose rules in CLAUDE.md. Cheap, ignorable, drift-prone.
2. One canonical pattern, enforced by lint. One-Way Pattern Consistency eliminates “should I use X or Y?” decisions.
3. Types. Type-Driven Development moves errors from review time to compile time.
4. A curated public surface. The Four-Layer Wall limits what the model can even reach for.
5. A pinned vocabulary. The Domain Glossary gives every concept exactly one name.
6. A DSL. The vocabulary becomes the language itself, and invalid programs stop existing.

The top rung combines all the ones below it. A good internal DSL is one canonical pattern, expressed through types, exposing a curated surface, speaking a pinned vocabulary. Each rung moves a class of mistakes from “caught in review” to “cannot be expressed”, and the DSL is where the last class runs out.

Joshi’s progressive interfaces are the detail worth stealing: the builder’s type signatures only expose the legal next steps, so you cannot declare a step before the topology or an action before selecting a client. That is Making Invalid States Impossible applied to the generation surface itself, not just to the generated code.

Where I Already Am on the Ladder

I have never shipped a semantic-model DSL like Tickloom, but I have been getting the smaller versions of the payoff without naming them:

• Frontmatter schemas. Every note in this vault carries a typed frontmatter block, and Frontmatter as Document Schema is a tiny declarative DSL. Agents generate it correctly on the first try precisely because the surface is a dozen fields, not a language.
• Justfiles. A justfile is a DSL for project operations. When an agent needs to run the test suite, it reads recipe names instead of reconstructing shell incantations, and the recipe list is the validator.
• The editorial template set. My diagram templates are a visual grammar: nine archetypes, each mapped to one kind of idea. The rule “pull the matching archetype or fork a new one, never reuse a mismatched layout” is a grammar constraint, and it is why generated diagrams stopped drifting.

The gap the article exposes is the middle of my testing stack. My integration tests are still written against raw clients and manual setup, which is Joshi’s “before” example: intent buried under mechanics, dozens of incidental decisions for an agent to get subtly wrong. A fluent scenario builder over the existing test kit is the obvious next experiment, and the two-phase workflow applies: use the model as a design partner to converge on the builder, then use it as the natural-language interface for writing scenarios.

flowchart TB
    subgraph phase1["Phase one: design the vocabulary (human-led)"]
        A["Sketch the DSL with the model"] --> B["Try it on a real scenario"]
        B --> C["Find the awkward seams"]
        C --> A
    end
    subgraph phase2["Phase two: use the vocabulary (model-led)"]
        D["English request"] --> E["Model writes the DSL program"]
        E --> F["Validator or compiler checks it"]
    end
    phase1 -- "vocabulary ships" --> phase2

The two phases do not mix. In phase one I stay in the driver’s seat because the design decisions are the ones I need to own. In phase two the model does the typing because the vocabulary has already made hallucination structurally difficult.

I Was Wrong About Prompts

In November 2025 I wrote Prompts Are the Asset, Not the Code, arguing that code is a derivative and the conversation that produced it is the source of truth. The DSL argument breaks that position, and I think it breaks it correctly.

A prompt is only the asset while the output is too verbose and too incidental to maintain directly. A generated Tickloom scenario needs no prompt archaeology: it is already the dense, readable expression of intent, and next month’s change happens in the scenario, not in a regenerated conversation. When the output language is dense enough, the program becomes the source of truth again and the prompt becomes scaffolding you throw away. The durable asset is neither the prompt nor the generated code. It is the vocabulary that made both cheap.

When Not to Build One

The honest caveat: rung six has real upfront cost. You are designing and maintaining a language, and the payoff only arrives when the domain is genuinely constrained and the task recurs enough to amortise the design work. For one-off scripts, prose and lint rules on the lower rungs are the right spend. Climb only as high as the error rate justifies.

Related

• One-Way Pattern Consistency
• The Domain Glossary Is a Constraint, Not Documentation
• Making Invalid States Impossible
• Entropy in Code Generation
• The Four-Layer Wall Around Your Library’s Public API
• Prompts Are the Asset, Not the Code

Source

• DSLs Enable Reliable Use of LLMs, Unmesh Joshi, martinfowler.com, 14 July 2026