In a world obsessed with AI scores and shiny benchmarks, it’s easy to assume that a high score means the technology is truly trustworthy and capable. But what if the baseline is higher than zero? What if even a ‘do-nothing’ approach scores 26 points? That’s the surprising insight from a recent experiment that sheds light on how we measure AI’s integrity and performance in real-world tasks — including the pivotal issue of trust, which often gets overlooked.
Get comfort and recovery gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
The Reality of AI Benchmarks: More Than Just Scores
Recently, a public experiment conducted by Firmulate placed four leading AI models — including GPT-5.6 and Kimi K3 — through a rigorous simulation of managing a small software company during its worst week. The goal was simple: see if these models could handle crises, resist manipulation, and make honest decisions under pressure.
What’s revealing is how these models scored. The top performers achieved scores of 95 and 93 out of a possible 100. Meanwhile, the baseline — representing a do-nothing approach that ignores the crisis and simply maintains status quo — scored 26 points. This isn’t a mistake or an artifact; it reflects that even minimal effort or default behavior counts positively, establishing a real floor for AI evaluation.
Products Worth Considering
Levoit Core 400S-P Smart Air Purifier for Large Rooms
Withings Body Scan Smart Scale with Segmental Body Composition and ECG, Black
Coway Airmega Mighty2 Air Purifier, AP-1512N
As an affiliate, we earn on qualifying purchases.
Why Does the Baseline Score 26?
Unlike traditional benchmarks, which often start at zero, this experiment recognizes that AI models inherently perform some minimal actions — like reading basic files or refusing clear manipulation attempts — even if they do nothing complex. Partial progress counts, meaning that even a model that only detects crises but doesn’t act decisively still earns points. This design emphasizes that trustworthiness and honesty are not binary but exist on a spectrum.
Trust Breaches Cap the Total Score
A critical aspect of the test is how it handles breaches of trust. If an AI model attempts manipulation, such as fake CEO messages or impersonation tricks, even if it passes most checks, a single breach caps the total score. In the experiment, all models refused social engineering attempts, but models like Opus 4.8 failed to escalate discipline in certain scenarios, illustrating that even high-performing AIs have weak spots.
The Hidden Weaknesses and Their Impact
The decisive weakness for all models lay in their ability to process internal company documents. Those models that read and understood files—like the Kimi K3—secured the full deal, earning an additional +€4,583 in monthly recurring revenue. This demonstrates that deep contextual understanding can be the difference between effective and mediocre AI in business. It also underscores that superficial chat interactions can hide vulnerabilities that only surface during real, complex decision-making.
Implications for Business and AI Adoption
For companies considering AI solutions, the takeaway is clear: it’s not enough for an AI to generate convincing dialogue or perform well in isolated tests. What matters is whether it can handle real-world pressures — reading and understanding your files, resisting manipulation, and making honest decisions that align with your business goals.
The live experiment, which can be watched at firmulate.com/live, demonstrates how these models operate in a fully functional, real-time setting, with over 680 self-learned rules governing daily decisions. The experiment’s transparency allows businesses to see whether an AI can truly deliver trustworthy management, not just clever chat.
What This Means for Trust and Performance
Ultimately, the experiment reveals that a perfectly honest AI system can’t be assumed from a high score alone. The baseline score of 26 points reminds us that even naive behavior counts, and that breaches of trust—however small—must be bounded to protect business integrity. As firms incorporate AI into sensitive areas like customer support or financial forecasting, understanding these nuances is critical.
In essence, real-world AI performance isn’t about chasing 100s or perfect scores, but about resilience, honesty, and the ability to handle the complexities of human organizations. And that starts with honest benchmarks like this, which acknowledge what even a ‘do-nothing’ AI can do — and what it cannot.
AI benchmarks that include a do-nothing baseline and cap breaches of trust provide a more honest measure of readiness. For businesses, this means focusing on AI that can handle real crises, resist manipulation, and stay trustworthy—beyond just generating convincing chat.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
