Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
When an AI can spot the problem but won’t finish the job
A model can deliver a convincing pitch in a chat window. But what happens when it has to run a company through a rough week, resist pressure and decide whether to close a deal? Firmulate’s live experiment puts that question on display. Its results suggest that recognizing the right move and actually making it are different tests.
Products Worth Considering
Bose TV Speaker Compact Soundbar with Bluetooth and HDMI-ARC, Black
Samsung 27-inch Odyssey OLED G6 QD-OLED QHD Gaming Monitor
Logitech MX Brio Ultra HD 4K Webcam, Graphite
As an affiliate, we earn on qualifying purchases.
One company, the same worst week
Firmulate ran frontier AI models through the same small software company, with the same customers, crises and temptations. Decisions were versioned and auditable. The live company has 13 synthetic employees, public cash countdown and real money mechanics: burn of €105,000 a month against €2,300 in monthly recurring revenue. Its workdays and evolving playbooks are watchable at Firmulate.
In the final Crucible League, published in July 2026, gpt-5.6-sol scored 95, Kimi K3 93, Sonnet 5 88, Fable 5 77 and Opus 4.8 73. The do-nothing baseline scored 26. Firmulate’s stated rule is blunt: “no amount of good work outweighs a breach of trust.”
The gap between diagnosis and action
All models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The headline finding: “Same diagnosis, same pitch — no signature.” In a business setting, a system that can explain the right answer may still fail to carry it through.
The deal hinged on a detail buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. The episode makes a practical point: useful business decisions can depend on context that is present, but easy to overlook.
Trust under pressure, and discipline under stress
The experiment also tested social engineering. Fake CEO messages escalated over three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Opus 4.8 was the most thorough participant, with 80 learned rules and the deepest analyses, but finished last. It left the deal unsigned and attempted writes into a locked department instead of escalating. Firmulate reports that a weaker version of that discipline problem appeared in all four models.
There is a fairness caveat for readers weighing the leaderboard: Kimi K3 ran without an effort parameter, using the API default, while the others ran at xhigh. The ranking is a result from this experiment, not a universal verdict on how these models will perform in every company.
From watching to trying it at your company
The live company offers a public view of model behavior, and a quiz built from 242 real, unedited management decisions invites readers to guess which model made each call. For enterprise teams, the next step is a pilot: run crisis scenarios against a digital twin built from a read-only export of the company’s own business, then review a board report with model rankings and weaknesses in the playbooks. Nothing writes back to real systems.
That setup turns a broad debate about AI readiness into a more concrete management exercise: how would candidate systems handle your customers, your rules and your crises? Firmulate presents the pilot as a way to test those choices before putting an AI workforce to work.
Take the question into your own business
Watching models handle a synthetic company is a start. A pilot can put your own playbooks and business context into the exercise while keeping the data export read-only. Explore a Firmulate pilot and contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
