A second café, against UK data

Stress test a site

What the data says

Stress test

A second café in with 10% of regulars moving

What moves the payback date

Regulars and volume

Still a regular

Monthly profit at the new site

Ask the model

How I checked the AI's output

An AI coding agent built this prototype. I directed it and reviewed what it made. I set the questions it had to answer and checked its output, and I am responsible for the result.

The test suite has 86 tests and all of them pass. They cover the data, the model, the agent and the numbers quoted in the README.

I had every checkable claim in the docs and code comments rerun, and three were wrong. The docs said the survival model used 24 quadrature nodes when the code used 12. A refit with 24 moves no posterior mean by more than 0.0006, so the code keeps 12 and a test holds the docs to it. A note said that dating churn to a regular's last visit biases the Weibull scale low. On six seeds that shortcut leaves the scale within 0.3% and puts the shape 11.3% low. A comment put the bias of another shortcut at about 15% on the shape, and the measured figure is 8.6% on the shape and 12.9% on the scale.

Every number the agent writes has to appear in a tool result from the same turn, allowing for rounding and for formats like percent and £k. An answer that fails is rewritten once and withheld if it fails again. The eval now compares every number in an answer with a fresh 20,000-draw run. With one invented number added to each of the 29 main answers, the old eval still passed 25 of them and the strict one passed none.

The offline planner answers all 29 main questions correctly, which mostly tests the tools. It passed 12 of 29 new paraphrases at first, and all 29 after changes tuned against them. A held-out set of 16 was scored once at the end and went from 6 to 10. One held-out question got worse, and it is left as it is so the held-out score stays clean.

The survival model recovers the values it was simulated from. Across 10 fresh seeds, 47 of 50 fall inside their 90% intervals, and 93% of the observed Kaplan-Meier curve lies inside the 90% predictive band.

Sources

What the model uses

The synthetic shop against the ONS Annual Business Survey

Datasets

Not used