Skip navigation
The first real evidence that LLMs cannot operate businesses, not even tiny ones.
0:00 / 0:51

The first real evidence that LLMs cannot operate businesses, not even tiny ones.

Skyfall AI

@skyfallai1.2K followers

Visit on X
140 likesDownloadOriginal post

26,413 views

The first real evidence that LLMs cannot operate businesses, not even tiny ones. At Skyfall, we’re building toward Enterprise Super Intelligence: AI systems that can one day run entire enterprises, coordinate teams and make long-horizon decisions, just like a CEO. To get there, we need to understand what AI cannot do yet. So we built MAPs, a theme park business with real operational constraints and let GPT5 run it. Here’s what happened: (spoiler: it fails miserably)

37 Comments

Comments from the original post on X (Twitter)

  • Skyfall AI's profile picture

    @skyfallai

    Everyone tests LLMs on solved problems. Nobody tests whether they can run a real business. But if we’re ever going to build an AI CEO, a system that can reason over time, allocate resources, manage uncertainty and run an enterprise, it needs to succeed at the smallest version of that task. So we built one. A theme park with: Stochastic events Partial observability Staffing Restocking Maintenance Long horizon planning Cascading failures

    16 likes

  • Skyfall AI's profile picture

    @skyfallai

    We call it MAPs: Mini Amusement Parks Play it here: Full write up: It looks like a game, but it’s actually a benchmark designed to answer a single question: Can an agent operate a dynamic system over time? Before an AI CEO can run an enterprise, it must be able to run this.

    8 likes

  • Skyfall AI's profile picture

    @skyfallai

    What MAPs reveals is simple but important: LLM can use tools but cannot run systems They break under uncertainty, time and spatial constraints They have no operational intuition Humans instantly build mental models, and LLMs still cannot. Try beating the agents yourself -> Read the blog here -> Building an AI CEO requires understanding of these gaps and MAPs is the first step toward Enterprise Super Intelligence.

    7 likes

  • Skyfall AI's profile picture

    @skyfallai

    Failure modes were consistent: Chasing flashy upgrades they cannot afford Ignoring maintenance, janitors or inventory Overreacting to noise Forgetting basic operational steps No long term planning No causal and business sense They simply don’t have common or business sense, they just have a chain of thought. The core skills of an AI CEO simply aren’t there.

    6 likes

  • Skyfall AI's profile picture

    @skyfallai

    Then we pitted humans vs multiple GTP-5 class LLM agents. And we actually stacked the deck in their favour but it didn’t help them. Full documentation Step by step action APIs Tool access Ability to practice in sandbox mode Extra observations If LLMs can operate a business, they should do well here. But, they didn’t .. at all

    5 likes

  • Skyfall AI's profile picture

    @skyfallai

    Results: Humans: ~100 normalized score Best LLM agent: <10 Nearly 9.8x gap, across models, across configs, with practice and with planning scaffolds No amount of prompting could save them

    5 likes

  • Lakshya Gupta's profile picture

    @ChaosAdm

    This is great. The next 5 years are going to be very fun with increasing research focus like this on AI for long-horizon business planning. Waiting for AI CEOs to be a thing we consider normal one day 😼

    3 likes

  • Skyfall AI's profile picture

    @skyfallai

    @ChaosAdm Glad our work resonated with you! Stay tuned for more... AI CEO is the future

  • Oluwaseun | The Automation Guy - TAG's profile picture

    @theautomation_g

    Amazing stuff! Great work on this @skyfallai and team.

    2 likes

  • Skyfall AI's profile picture

    @skyfallai

    @POluwaseuna Thank you! Glad our work resonated with you.

  • Prem's profile picture

    @premtechAI

    Fascinating approach exposing limitations is the fastest way to unlock true progress. MAPs is more than an experiment; it’s a stress test for the future of AI leadership.@skyfallai

    2 likes

  • Skyfall AI's profile picture

    @skyfallai

    @premtechAI You got it!

  • Flora_The_AI_Girl's profile picture

    @flora_oladipupo

    Congratulations on the launch 🎉

    2 likes

  • Skyfall AI's profile picture

    @skyfallai

    Appreciate the support! More on the way 👀

    1 like

  • Dada's profile picture

    @marvy_101

    👀

    2 likes

  • Brady Long's profile picture

    @thisguyknowsai

    This is awesome!

    2 likes

  • Skyfall AI's profile picture

    @skyfallai

    Appreciate it! Lots more in the pipeline.

  • Alamin's profile picture

    @iam_chonchol

    Big congrats on the launch

    2 likes

  • Skyfall AI's profile picture

    @skyfallai

    Thank you!

  • Nico Lecomte's profile picture

    @NicolasLec91112

    Congrats on the launch!

    2 likes