Your Next DSL Author Is a Language Model

Ask a frontier language model for Python and you get Python. Ask it for your team's home-grown modeling notation and you get something that merely resembles that notation: invented keywords, guessed parameter names, and constructs that were never in the spec, all delivered with perfect confidence. The reflexive fix is more prompting, longer context, and better retrieval. The durable fix is a design decision made before the model ever sees your language. Don’t use a foreign syntax in the first place. That decision has a name: Typed Domain Grounding. It is implementable today with the type system you already have.

Fluency Is Frequency

A model writes Kotlin, TypeScript, Python, and SQL well for one dominant reason. Those languages appear millions of times in its training data. Syntactic competence is, to a first approximation, a function of training-data frequency, a relationship documented directly in code-LLM benchmarks that classify languages by frequency and popularity and find performance degrading accordingly for low-resource languages [8]. Where the corpus is dense, generation is reliable. Where it is sparse, generation degrades into pastiche, without change in tone or apparent confidence. This absence makes the failure so easy to miss.

Every freshly designed external DSL is, from the model's point of view, a language with a training-corpus frequency of zero for its exact syntax. That said, the model typically still carries transferable priors from syntactically adjacent grammars, APIs, and notations it has seen. That transfer is not a rescue. Rather, it is the mechanism behind the interpolation failure described next.

A model confronted with an unfamiliar notation does not fail loudly; it interpolates. It smuggles in arrow syntax from Mermaid or PlantUML, calls the addRelation or withLabel your API would plausibly have but doesn't and hands three arguments to a construct that takes two because a syntactic cousin takes three. Each mistake is the corpus's most probable completion, applied to a language the corpus never contained. The output is fluent, well-indented, and wrong.

A traditional DSL virtue makes this problem worse: leniency. Diagram tools and config parsers are engineered to forgive by skipping the line you don't understand, rendering a best guess, and never punishing a typo. For humans, that is hospitality. For machine authors, it is a trap door.

A model produces a ten-class diagram with one invented relationship keyword. The parser skips that line. What lands in the design document is a professional-looking diagram missing exactly one association.

A human who mistyped would have noticed, because a human intended the association. The model intended nothing and the engineer downstream, primed by nine correct classes and clean typography, has no reason to suspect the picture is incomplete.

Grounding the Language, Not Just the Facts

In everyday LLM engineering, "grounding" has become nearly synonymous with RAG. Fetch the relevant documents, put them in context, and the model's claims stay anchored to real sources. That is grounding in the data space. It disciplines what the model says about the world. It does nothing for our problem: A model with flawless facts in its context can still emit a DSL statement with invented grammar.

Asked to diagram a payment system with no retrieval in place, a model may show the payment service writing directly to the ledger when a message queue actually sits between them with impeccable syntax, but incorrect content. Retrieval fixes that. But with the documentation in context, the same model can express the correct queue in a notation it half-knows, inventing a queue-component keyword the language does not have. In this situation, the content is correct, but the notation is broken. No amount of additional retrieval helps, because the failure lives in the model's competence with the target language, not its knowledge.

Typed Domain Grounding grounds this other axis. Instead of anchoring an answer's content in retrieved data, it anchors the notation itself in what the model already knows how to write, grounding in the competence space. The two are orthogonal and combine cleanly, retrieving facts via RAG and expressing them through a typed, compiler-checked DSL.

Figure 1. Two spaces of grounding. (Source: created by author using Google’s Nano Banana). 

In Figure 1, RAG grounds a response's content in the data space. TDG grounds the language itself in the competence space of orthogonal, combinable pipelines,  not competitors. 

The principle fits in one sentence. "Never invent syntax an LLM has never seen: Embed the domain in a language it already knows and let the compiler be the oracle".

Five Building Blocks

TDG is a small stack of design commitments.

Embedded, Not External 

The domain becomes an internal DSL inside a mainstream host language with no new grammar and no bespoke parser. What reads like a dedicated language is ordinary host-language code, so the full existing toolchain (e.g., highlighting, completion, jump-to-definition, safe rename, debugger, and reviewer-familiar diffs) is inherited on day zero, where each piece would otherwise be a multi-year project.

Host Language Chosen by Training Data Affinity

Classic embedded-DSL literature (such as works by Paul Hudak and Martin Fowler) weighs expressiveness, type-system strength, and tooling. TDG adds a criterion that couldn't have existed before LLMs: corpus presence. Kotlin, TypeScript, or Python bring substantial prior fluency for free. No frontier lab publishes its exact training mix, but the argument only needs the gap. Any mainstream language is far better represented in public code than a new DSL could ever be. An elegant but rare language brings almost none of that. The criteria usually pull the same direction, but where they diverge, the corpus gets a vote.

The Compiler As Oracle 

Shape the API so domain errors surface as type errors with named parameters over positional ones, enums over magic strings, and builder scopes exposing only constructs valid in the current context. Sealed hierarchies let the API say a relationship end is exactly one of four things. A hallucinated fifth variant has no type to inhabit. Receiver types control which functions even exist inside a block. An attribute(...) outside a class body fails name resolution immediately. Illegal nesting becomes a sentence the language cannot say. Strictness here is the mechanism, not pedantry.

The Generate-Compile-Repair (GCR) Loop 

The model generates, the compiler checks, and diagnostics flow back as a repair prompt. Compiler-feedback repair loops are already well documented in the self-debugging literature [1] — GCR itself is not the novel part of TDG. What TDG adds is putting that familiar loop to two uses at once: a repair channel that fixes broken output, and a measurement instrument that reveals how often, and why, a given DSL needs fixing in the first place. The repair prompt carries the failed snippet, the verbatim diagnostic, and the source location, enabling a targeted edit rather than blind regeneration, which merely resamples from the distribution that produced the mistake. The loop is bounded with three attempts in kUML's setup. Repairs that succeed tend to succeed early. A model still failing after three rounds of located feedback is missing a concept, not a comma.

An On-Demand Teaching Channel 

Embedding solves syntax. A vocabulary question remains: Which constructs exist and what are they called? Stuffing full API docs into every prompt buries the five relevant constructs under ninety-five irrelevant ones. TDG's answer is a callable tool the model can query mid-generation — implementable, for instance, as a Model Context Protocol server — that returns a small set of curated, known-good, already-compiling examples for exactly the construct the model needs at that moment, instead of a firehose of unrelated documentation. The effect size is not subtle: for kUML, a kuml.examples tool raised the first-compile rate for SysML v2 diagrams from twenty-five to 87.5 percent.

Blocks 1-3 are what makes a design TDG, while blocks 4-5 are what makes a TDG instance perform well in practice. These are complementary pieces, rather than definitional. In relation to other ideas, TDG is not Retrieval-Augmented Generation (see above). It grounds the language, not the facts. Also, TDG does not design a new syntax the way some LLM-oriented grammar proposals do. Instead, it selects an existing, training-data-rich syntax and lets that language's own compiler do the checking. The two illustrations below show what this approach looks like in practice. A full comparison against related work follows them.

Illustration One: kUML, Where the Oracle Is Measurable

kUML is a Kotlin-embedded modeling language for UML, SysML v2, and C4 diagrams. It is to date the most thoroughly measured TDG instance.

Code Example 1 (Kotlin):

classDiagram(name = "Order") {
    val customer = classOf(name = "Customer") {
        attribute(name = "customerId", type = "String")
    }
    val order = classOf(name = "Order") {
        attribute(name = "date", type = "LocalDate")
        operation(name = "cancel") { returns(typeName = "Boolean") }
    }
    association(source = customer, target = order) {
        source { multiplicity(spec = "1") }
        target { multiplicity(spec = "0..*") }
    }
}

To a language model this is ordinary Kotlin  with receiver lambdas, named arguments, and plain variable references as constructs it has absorbed from every public Kotlin codebase in its training data. The modeling domain rides on syntax the model already commands.

When it hallucinates anyway, when an invented aggregation (e.g., source = customer and target = order) makes a call at the top level, outside any association block, the Kotlin compiler answers with an exact, located error. Write the equivalent hallucination in PlantUML or Mermaid and the lenient renderer will most likely produce a diagram anyway, quietly missing an element.

Figure 2. IDE screenshot. (Source: created by author.) 

The invented aggregation(...) call is flagged directly in the IntelliJ IDEA editor, and the live-preview panel reports "Unresolved reference 'aggregation'" in real time - the compiler acting as oracle while the author is still typing.

The hallucination did not become a subtly wrong diagram. It became a red underline.

Phantom references die the same way. Suppose the model writes association (e.g., source = order and target = invoice) without ever declaring an invoice classifier. In kUML, diagram elements are ordinary Kotlin values So "which model elements exist" is variable resolution. The compiler answers Unresolved reference (invoice) at an exact line and column. The repair prompt writes itself. Run the same phantom through a lenient name-based notation and something worse than silence happens: Many diagram tools helpfully create a node the moment an edge references it. The hallucinated invoice isn't dropped, it's promoted to a real, empty class that was never in any specification, looking exactly as legitimate as its neighbors. One toolchain converts the phantom into a located error; the other converts it into furniture.

Illustration Two: A Thought Experiment in Infrastructure as Code

To show TDG is a principle and not a kUML feature. Testing whether the pattern generalizes beyond a domain where it has already been implemented and measured requires infrastructure definitions. This section is a thought experiment, not a second measured result. The goal is architectural generalizability, not additional empirical evidence. Terraform's HCL is an external DSL, but an unusually well-fed one. Models rarely botch their own block syntax. The trouble sits one level down. Attribute names and shapes of individual resource types are provider schema, not language.

A model writing an aws_s3_bucket block guesses among half-remembered, version-drifted attributes, but to its credit, Terraform does catch this locally, The terraform validate command and the Terraform language server (terraform-ls) check attribute names, types, and references against the provider schema fetched from the installed plugin, without involving a cloud API. That check is itself the tell: It is an entire apparatus with schema distribution, a dedicated validate phase, and a language server, built to recover for an external DSL what an embedded DSL's host compiler hands over on day zero. Cloud Development Kit (CDK) for Terraform already generates typed provider bindings in five languages [2], TDG's host-language move applied to infrastructure. What no local check reaches, in HCL or a typed DSL alike, is the tier below schema. This is true whether the AMI exists, you have permission, or the quota holds. It is visible only once something talks to the real cloud API.

Now imagine the same infrastructure as a typed Kotlin DSL. This is a thought experiment, not an existing library and is in the spirit of what Pulumi, AWS CDK, and CDK for Terraform already do with typed bindings in other languages [2], none of which currently ships an official Kotlin SDK.

Code Example 2 (Kotlin):

// Thought experiment – no existing library, illustrating the TDG principle
// in a domain beyond modeling.
val assets = s3Bucket(name = "assets") {
    versioning = Versioning.ENABLED
    encryption = ServerSideEncryption.AES256
}

lambdaFunction(name = "thumbnailer") {
    runtime = Runtime.JAVA_21
    memoryMb = 512
    trigger = s3Trigger(bucket = assets, event = S3Event.OBJECT_CREATED)
}

The four classic hallucination classes against this design:

- Invented attribute (i.e., encryptionMode instead of encryption) becomes a compile error naming the properties actually available.
- Wrong value type (i.e., memoryMb = "512MB") becomes a type error before any tool runs.
- Reference to a never-declared bucket becomes an unresolved variable, caught at the exact line.
- A trigger event S3 does not have fails against the S3Event enum instead of being accepted as a string.

Figure 3. Comparison graphic. (Source: created by author.)

Figure 3 shows the same four hallucination classes. Terraform catches them locally too, via terraform validate/terraform-ls and the provider-schema apparatus described in the text. The typed Kotlin DSL catches them the same way, for free, via the stock compiler. 
In real Terraform, all four also die locally, through that same schema apparatus. In the typed variant, they die as ordinary compiler errors, such as no protocol, no separate validate phase, and no language server to build or maintain. One question remains: which check stays current with the real, live API?

Here the two approaches are symmetric, and neither wins: terraform validate checks against the provider version pinned in the lock file, the typed DSL checks against the SDK version pinned in the build file - and either pin can silently lag the real, live API. What is left below both is live cloud state (i.e., existence, permissions, and quota) equally invisible to HCL and Kotlin until something calls the API. The embedded DSL's advantage isn't more validation than Terraform's apparatus achieves; it's the same validation at near-zero incremental cost, in the format (compiler diagnostics) a repair loop consumes best. The pattern generalizes to any domain with a typed internal DSL, a fast batch-capable compiler, and a training-data-rich host language: Gradle's Kotlin DSL, compose-style UI builders, and typed configuration DSLs replacing schema-less YAML all qualify.

What TDG Is and Isn't Relative to Prior Work

Prior work / Approach / How TDG differs:

Microsoft TypeChat [3] 

TypeScript types as a response schema, while compiler diagnostics type as a repair prompt. TDG, on the other hand, validates data structures (i.e., response schemas).TDG embeds a complete domain language with semantics and artifact generation.

Grammar Prompting [4] 

Feed the target DSL's BNF grammar into the context. TDG differs by mitigating the foreign-syntax problem, dissolving it constructively. There is no foreign grammar to teach.

ThingTalk [5] 

Typed DSL as the target language for semantic parsing. TDG differs as an external DSL with its own syntax, which is exactly the construction TDG avoids.

"AI-Oriented Grammar" [6] 

Design a new, more token-efficient syntax for LLMs (e.g., SimPy as a Python derivative, not the identically-named simulation library). TDG differs by not designing a new syntax. Instead, it selects an existing, training-data-rich one.

RAG Grounding (e.g., AGREE [7]) 

Anchor facts in responses (data space). TDG differs by anchoring in language space (e.g., orthogonal and combinable).

Compile-Repair Literature [1] 

Repair loops with compiler feedback (i.e., self-debugging and self-repair). TDG differs by treating the repair loop as a supporting building block and measurement instrument, not as its defining novelty.

The closest relative to TDG is TypeChat [3]. Both use a mainstream type system to check authority and feed diagnostics back to the model, but TypeChat validates that an answer conforms to an interface, that a sentiment result is positive, negative, or neutral, and that an extracted object has the right fields. TDG turns the domain itself into a language with semantics and downstream artifact generation, checking things a response schema never sees: whether an association connects classifiers that actually exist, whether a transition sits inside a state machine. TypeChat asks "Is this answer shaped correctly?", whereas TDG asks "Is this a valid sentence in the domain?", which is a richer question, checked by the same compiler machinery.

Grammar Prompting [4] and the AI-oriented-grammar line [6] both accept the premise TDG rejects, that a new syntax has to exist. One ships the grammar into context at runtime. The other designs a more token-efficient grammar with a corpus frequency still at zero, so the competence gap remains and must be bridged by teaching. TDG asks the questions that come before "How do you teach a foreign syntax? Why have one at all? ThingTalk [5] is the cautionary data point that is cleanly typed, still external, and still carrying the training-data problem.

The Numbers and an Honest Caveat

In a fifty-task benchmark pitting kUML against PlantUML and Mermaid, the embedded, compiler-checked language running the GCR loop reached the highest Structural Fidelity (45.3 percent vs. 37.9 percent and 37.4 percent) and the lowest hallucination rate (15.4 percent vs. 16.5 percent and 21.9 percent) with Claude Sonnet 5. The tasks spanned UML, SysML v2, and C4 diagram types, each anchored to a source specification. Outputs were scored on Structural Fidelity (i.e., node-set and edge-set similarity to that specification, not a deeper semantic check) and hallucination rate. Much appeared that the specification never asked for. The axes matter jointly. A tool could fake high fidelity by spraying out everything plausible. The hallucination rate catches exactly that strategy.

The public gallery now covers nine model runs under a shared GCR harness.The picture is more model-dependent than the headline number suggests. The full win, higher Structural Fidelity and lower hallucination rate, holds for Claude Sonnet 5 and GPT-4o. It does not hold uniformly elsewhere. Gemini 2.5 Flash and Gemini 2.5 Pro both put Mermaid or PlantUML ahead on Structural Fidelity and Qwen3-Coder:30b shows the same pattern. Grok 4 gives kUML its narrowest Structural Fidelity win in the whole study (36.8 percent vs. 34.6 percent and 36.4 percent), but its highest hallucination rate (28.1 percent vs. 22.4 percent and 23.0 percent), which is a strict oracle without a matching payoff. 

The effect is real and repeats across several model families, strongest for the two discussed in depth here, but it is not a property of every model tested. The honest scope of this article's headline numbers is Claude Sonnet 5. Scoring throughout is fully algorithmic, rather than an LLM or human judge. Each cell reflects a single generation run capped at three GCR repair attempts, not an average over repeated samples. Single runs carry no variance estimate, so the deltas above are directional evidence, strongest where margins are wide and the direction holds across model families. The full per-model breakdown for all nine runs is on the public benchmark gallery (e.g., kuml.dev/benchmark-gallery).

The benchmark did not show a higher first-shot compile rate for Claude Sonnet 5.  In comparison, kUML's rate was actually lower than the lenient tools' first-try success. That asymmetry is the point: The strict oracle rejects more up front and the result correlates with higher Structural Fidelity and lower hallucination downstream. This correlation demonstrates the benchmark directly, though the explanation that rejection-and-repair itself drives the gap (rather than some other difference between the three notations). This is the most parsimonious reading, not an isolated causal test. The teaching channel sharpens this further: With kuml.examples available, the SysML v2 compile rate after the repair loop rose from twenty-five percent to 87.5 percent, which amounted to two of eight versus seven of eight tasks, using Claude Sonnet 5 only, with a single run per cell. That is a compilation result, not a fidelity result The full breakdown is on the public methodology page.

Stating the caveat plainly, TDG closes gaps in syntax, not understanding. A model can cleanly produce compiling models that are substantively wrong. In the same benchmark, one model remained semantically shallow on C4 architecture diagrams even though the compiler found nothing to object to. Every construct was legal; the architecture itself was thin. The oracle checks well-formedness, not wisdom. TDG turns a good model into a reliable author, rather than turning a weak model into a good architect.

Six Recommendations for Language Designers

• Ask the foreign-syntax question first not because embedded DSLs are always superior (see "What This Costs You" below), but because the question deserves an explicit answer rather than being skipped by habit: Why is this not a Kotlin/TypeScript/Python builder API?
• Treat training-data affinity as a selection criterion on equal footing with type-system strength and ecosystem maturity when choosing a host language.
• Design the compiler's strictness deliberately with named parameters, enums over strings, and context-scoped builders each deciding which hallucination class becomes a compile error versus staying invisible.
• Treat diagnostics as an interface that uses exact source locations, named expected values, and parseable output to determine how fast a repair loop converges.
• Build a teaching channel with curated, CI-verified, callable examples outperform documentation stuffed statically into the prompt.
• Abolish leniency. Recognize that silently "fixing" malformed input is a leak in the oracle, not a courtesy, so fail strictly and loudly by default.

What This Costs You

TDG costs three real things. 
Notational Freedom: 
An internal DSL looks like the host language. As an example, classDiagram(name = "Order") { ... } reads more slowly than a purpose-built compact notation, though the DSL source is rarely the reading surface once diagrams render. 
Existing Investment 
Years sunk into a working external DSL aren't an argument to burn it down; TDG is about the next language or schema, not this quarter's migration. 
A Host-Language Commitment
Users need the host toolchain and, for humans, reading fluency in it. That commitment costs nothing extra for teams already living in that ecosystem — but it is a real, first-time cost for a domain-expert audience that has never seen a lambda.

In exchange you get a full compiler, IDE, refactoring toolchain, and hallucination oracle, none of which you build, all of which your most probable future author already knows. For a growing class of domains that trade is lopsided in TDG's favor, but it is a trade.

The Second Question

DSL design has always asked how close a language can come to the domain and the human reader. A second question now stands beside it as a design dimension in its own right, not as a replacement and not necessarily as an equal one in every context: How close does the language already sit to what its most probable future author can already do and how much verification does its toolchain hand you for free? For DSLs whose primary authors remain human domain experts, the classic readability question can still dominate; for DSLs increasingly generated or read by models, this second question gains proportionally more weight. Designers who take both seriously end up building languages where hallucinations stop being silent phantoms in the output and start being red underlines in the editor. Nothing here waits on future research. The host languages, compilers, and teaching-channel protocol all exist today.

References

1. X. Chen, M. Lin, N. Schärli, D. Zhou: "Teaching Large Language Models to Self-Debug." arXiv:2304.05128, 2023; and T. X. Olausson, J. P. Inala, C. Wang, J. Gao, A. Solar-Lezama: "Demystifying GPT Self-Repair for Code Generation." arXiv:2306.09896, 2023
2. Pulumi (pulumi.com), AWS CDK (aws.amazon.com/cdk), CDK for Terraform (developer.hashicorp.com/terraform/cdktf) infrastructure definition in general-purpose languages with typed provider SDKs. None currently ships an official Kotlin SDK; the Kotlin example in this article is a thought experiment, not an existing library.
3. Microsoft: TypeChat. GitHub repository, 2023. https://github.com/microsoft/TypeChat
4. B. Wang, Z. Wang, X. Wang, Y. Cao, R. A. Saurous, Y. Kim: "Grammar Prompting for Domain-Specific Language Generation with Large Language Models." NeurIPS 2023. arXiv:2305.19234
5. G. Campagna et al.: "Genie: A Generator of Natural Language Semantic Parsers for Virtual Assistant Commands." PLDI 2019 (ThingTalk as the typed target language; see also arXiv:2203.12751)
6. Z. Sun, X. Du, Z. Yang, L. Li, D. Lo: "AI Coders Are Among Us: Rethinking Programming Language Grammar Towards Efficient Code Generation." ISSTA 2024. arXiv:2404.16333
7. X. Ye, R. Sun, S. Ö. Arık, T. Pfister: "Effective Large Language Model Adaptation for Improved Grounding" (AGREE). Google Research, NAACL 2024. arXiv:2311.09533
8. F. Cassano et al.: "MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation." IEEE Transactions on Software Engineering, 2023. arXiv:2208.08227