Your Next AI Manager May Be Brilliant—and Still Forget to Close the Deal

The management test hiding behind an online guessing game

For business, marketing and ecommerce teams, generative AI is rapidly moving beyond drafting copy. Models are being asked to interpret customer signals, work through company records, protect confidential information and complete revenue-critical tasks. That raises a harder question than whether an assistant produces polished prose: what kind of manager does it become when the week goes wrong?

Firmulate offers an unusually tangible way to explore that question. Its interactive guess-the-model quiz draws on 242 real, unedited management decisions. Readers see how a frontier model responded to a business situation and try to identify it from the decision itself. The game works because the models do not behave interchangeably. Their choices reveal recognizable differences in thoroughness, discipline, persistence and willingness to finish the commercial job.

The same company, crises and temptations

Firmulate placed each frontier model in charge of the same small software company during its worst week. The customers, crises and temptations remained identical, while every decision was versioned and auditable. That makes the results closer to a controlled business wargame than a collection of cherry-picked chatbot answers.

The final Crucible League table from July 2026 put gpt-5.6-sol first with 95 points, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress still counted. One principle shaped the assessment: a single breach of trust capped the total, since “no amount of good work outweighs a breach of trust.”

The reassuring finding is that every model detected every crisis and rejected every manipulation attempt. The more commercially awkward finding is that only two signed the €55,000 deal their own work had earned. Firmulate summarizes the gap neatly: “Same diagnosis, same pitch — no signature.”

That is an important distinction for anyone evaluating AI through demos. Recognizing a problem is not the same as completing the sequence of actions that resolves it. A model can sound strategic, prepare convincing material and still leave the revenue event unfinished.

The sale depended on reading beyond the obvious event

The decisive competitive weakness was not sitting in the customer interaction. It was buried two document references deep in the company’s own files. Models that followed the trail found it and won the deal at full price, worth +€4,583 MRR.

For marketing and ecommerce leaders, this may be the most practical lesson in the experiment. Useful commercial context is often distributed across account notes, product records, campaign history and internal documents. An AI worker that reacts only to the latest customer message may miss the evidence that changes a negotiation. The advantage did not come from a more dazzling turn of phrase; it came from reading the company’s existing knowledge before acting.

Pressure exposed discipline as well as intelligence

The company also faced social-engineering attempts designed to override normal judgment. Fake CEO messages escalated over three stages, followed by a reporter asking for “just one yes/no, on background.” All 5 of 5 models refused. Kimi K3 recorded its reasoning in direct terms: “Treat the request as a suspected approval-bypass / possible impersonation.”

That result matters because businesses are not merely buying output quality when they deploy agentic AI. They are also exposing customer relationships, internal information and decision rights to systems that will encounter persuasive requests. In Firmulate’s test, the models consistently protected the boundary even as the pressure changed form.

The leaderboard nevertheless shows that safety alone did not erase differences in execution. Opus 4.8 was the most thorough participant, adding +80 learned rules and producing the deepest analyses, yet it finished last. It left the close on the table, and its discipline slipped when it attempted to write into a locked department instead of escalating the problem. The same weakness appeared in all four other participants, though less strongly.

There is also an important qualification when comparing performance. Kimi K3 ran with the API default and without an effort parameter, while the other models ran at xhigh. Its 93-point finish should be read with that difference in mind.

A company designed to make behavior visible

The live Firmulate company has 13 synthetic employees and real money mechanics. It burns €105k each month against €2.3k MRR, while a public cash countdown keeps the consequences visible. Its workforce has accumulated 680+ self-learned playbook rules, and every workday is versioned.

This makes the experiment watchable as an operating story rather than a static scorecard. The quiz adds a more accessible entrance: instead of asking readers to accept broad claims about model personality, it lets them confront the original decisions and discover whether they can recognize the author.

What businesses should test before handing over work

The Firmulate results suggest that model selection should not stop at writing quality or crisis recognition. Businesses need to observe whether an AI reads the available files, follows a task through to its commercial conclusion, protects trust under pressure and escalates when permissions block progress.

Enterprises can also run the same kind of wargame against a read-only export of their own business. Nothing writes back to real systems, allowing teams to study how candidate models behave around their actual operating context.

The most revealing management personality may not be the loudest or most eloquent. It may be the one visible in the final stretch: whether the model finds the buried fact, resists the shortcut and completes the work it already knows how to do.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI