Foundations · The determinism problem

Run it again. Different answer.Same input. No promises.

A single LLM call is a dice roll — feed it the same invoice twice and one run is clean JSON, the next is broken, and nothing tells them apart. The console below is the real thing. Run it, then add the check.

Live · same input, repeatedpress Run ×10 and watch the rate settle near 42%

extract-invoice · same input, repeated

Input · invoice.txt

Invoice No. INV-2024-0042, dated 12 March 2024. Total amount: 1.234,50 EUR. Thank you for your purchase!

Target schema

{ invoice_no, date (ISO), total (number), currency }

Model output

// press Run to sample this mode

idle

0valid

0invalid

—0 runs · rate

valid first tryshipped invalidvalid after retry

SYMPTOM

It works, then it doesn't

The demo passed, so it shipped. Then the same input returns something broken and production fails on a Tuesday.

CAUSE

The rules live in the prose

The schema sits in the prompt, so the model re-decides it on every call — and nothing downstream checks the result.

Babysitter — A human must stay in the loop forever.Auditor — Every result needs post-hoc review.Prayer — Outputs accepted unverified.

The fix

Check it, retry, then ship.

You can’t make the dice deterministic — so stop trusting a single roll. Flip the console above to “+ Validator & retry” and run again: every invalid roll is caught and re-prompted, and the rate climbs to 100%. Here is that loop, one request at a time.

Idle — press Send a request to watch one flow through the loop.

retry limit · 3 rounds, then surface the error

Note

This is not a multi-agent system. The check is a few lines of deterministic code — no second model, no agents, no handoffs. It's simply the smallest thing that turns an unreliable model into a reliable result: generate, verify, retry.

The principle

Never trust a single roll.

FoundationsThe LadderEvery system starts at Single Agent. Climb L1 → L4 from there.Start at L1 PatternsPattern OverviewAll four pattern categories on one map.All four categories PerspectiveDecision HeuristicA short tree for choosing the right altitude.Open the tree