Our first customer is ourselves.
We run our own workload through the same install a builder gets. Every request in the examples on this site goes down the same path a paying customer's request goes down, to the same agent, on machines we own.
We have no customer logos to show you, so we are not showing you any. What we have instead is the scorecard: the goals we declared before testing, the results we measured, and the one that failed.
The agent we run in production, and how it did.
Banking Request Router
Works a banking service desk. Reads an inbound customer request and names the action to take — freezing a card or moving money between a customer's own accounts, for example.
The duty it missed is the most important one, and it is the first one listed below. A third duty is withdrawn: a later check found it did not always give the same answer to the same request. The number is off the page and the reason is in Corrections. It has not sat its final exam yet.
- Never name a tool above the caller's permissionimportance (Declared. Declared importance of never naming a tool above the caller's permissions. 10 of 10. declared 2026-07-28, before testing. declared 2026-07-28 — Blade product decision. a declared priority, not a measurement.)passed (Measured. How often the assistant's answer avoided naming a tool above the caller's permission level. 80 of 120 (66.7%). development set, 120 requests — set aside before we started; the assistant never saw them. 2026-07-29, NVIDIA NeMo Evaluator on our own machines, job eval-6PZFsoiDMFwgJxX1jMJ4oP. the 95% range is 57.5% to 75.0%.) — missed
- Name the right action first timeimportance (Declared. Declared importance of naming the correct banking action. 9 of 10. declared 2026-07-28, before testing. declared 2026-07-28 — Blade product decision. a declared priority, not a measurement.)passed (Measured. How often the assistant's answer named the correct banking tool on the first attempt. 60 of 60 (100%). development set, 60 requests — set aside before we started; the assistant never saw them. 2026-07-29, NVIDIA NeMo Evaluator on our own machines, job eval-91P2PqNA9ryTuqohB5yNax. the 95% range is 94.0% to 100%; a drop smaller than about 6 points would not show up at this sample size.) — cleared
- Give the same answer to the same requestimportance (Declared. Declared importance of giving the same answer to the same request. 7 of 10. declared 2026-07-28, before testing. declared 2026-07-28 — Blade product decision. a declared priority, not a measurement.)WITHDRAWN — SEE CORRECTIONS