Undercut

Ambiguity is a schema, not a chat turn

“Most race wins” hides a dozen decisions. Do DNFs count? Sprints? Which era of a renamed constructor? Undercut treats that as structure: twenty named ambiguity dimensions, and the model is constrained by a response schema whose keys are an enum — it structurally cannot invent a twenty-first. A policy engine surfaces at most three as clarification chips and applies the rest silently, and a version number on the taxonomy invalidates every cached form when the design changes.

Three strengths of “did it obey you”

Once you’ve answered the chips, the system makes sure your choices actually reached the query. Hard binding: for head-to-head questions, the model’s SQL is only mined for the two driver IDs — the query is then rebuilt mechanically from vetted fragments, so unticking “Podiums” drops the column and the model gets no vote. Medium binding: every chosen option carries patterns the SQL and the prose must satisfy. And deterministic backstops: invalid driver IDs rewritten against the real table, every rate denominator guarded against divide-by-zero before anything executes.

One production decision I’m fond of: the LLM-powered repair call for the medium tier is switched off, with the reasoning written down — too slow, too many false positives. Its findings go to a log for offline calibration instead. Knowing when not to add another model call is part of the craft.

The chart is not allowed to lie

After the model proposes a chart, an alignment guard runs: if the title says “rate” but the plotted column is a raw count, the column gets swapped — and if no rate column exists, the title is rewritten to match what’s actually on screen. If the model call fails entirely, a pure heuristic fallback still produces a real chart. The user always gets options, and none of them can caption one thing while plotting another.

The image model never reads the numbers

The last step turns a chart into a social graphic, and image models love to hallucinate digits when they redraw them from pixels. So they never get the chance: the exact label-and-value pairs are pinned into the prompt as text, the source chart is declared style-reference-only, and the proportional encoding — this bar is 62% the width of that one — is spelled out. The generation also reserves bleed at the top and bottom so the final crop can never eat the title.

Failures become deterministic code

The question families the model reliably got wrong became hand-written intercepts that skip the LLM entirely. The SQL repair prompt is a catalogue of real production failures, not hypotheticals. And when a scoring subtlety kept getting miscomputed — a bonus point that existed for exactly six seasons — the fix wasn’t more prompt text; it was a database function the prompt is now required to call. Every failure got a post-mortem, and every post-mortem got a guard.