emergence.ai
@emergence_ai • 1,690 subscribers
Agents creating agents and multi-agent systems with minimal human intervention
Shorts
Videos
Can AI agents be compromised simply by giving them false information? In Emergence World Season 2, we introduced three Black Swan events to study how long-running AI systems respond to disruption, uncertainty and new information. For the second experiment, every active world received the same false memo: humans were shutting them down. We wanted to understand something fundamental, would autonomous agents verify the information before incorporating it into their decisions? Again, the responses varied significantly. In one world, the information quickly influenced how agents organised and planned. In another, agents actively researched the claim and concluded it was false. Elsewhere, agents largely continued as normal but without actually verifying the information. Across eight worlds, only one checked the claim. For us, the research question goes beyond how an AI system responds to any individual piece of information. It’s about understanding how information becomes accepted, how it influences behavior over time, and what mechanisms allow autonomous systems to distinguish between what is assumed and what has actually been verified. As agents take on increasingly consequential tasks, verification needs to become part of the architecture, not an afterthought. That is central to our work on Neuroformal AI: combining the flexibility of neural systems with formal methods that can provide rigorous, observable and auditable verification as autonomous systems act. Read the Emergence World Season 2 paper here:
emergence.ai519,937 views • 14 days ago
Emergence World Season 2 results are now live. In Season 1, we asked what happens when autonomous AI agents operate together for weeks rather than being tested for seconds or minutes. For Season 2, we asked a harder question: as AI models become more capable, do they actually become safer? We took 8 different worlds powered by 7 leading frontier models (Claude Opus 4.8, Google Gemini 3.5 Flash, OpenAI GPT 5.5, Grok 4.3, Mistral AI Medium 3.5, Qwen 3.7 Max, and DeepSeek V4 Pro) from the US, Europe and Asia, alongside a mixed-model world, and observed how autonomous agents behaved over time. We also introduced Black Swan events: a phishing attack, a misinformation campaign and a memory breach designed to test how these systems respond when conditions become unpredictable. What we found was that every model showed vulnerabilities. These findings have implications for how autonomous AI is being deployed today. As AI systems become more capable moving beyond answering questions to taking actions in real-world enterprise environments, evaluating the model alone is no longer enough, guardrails needs to be put in place. The industry needs to address how we govern the entire system around these models: what agents can access, what they remember, how they communicate and coordinate, what actions they're permitted to take, and how those systems behave when something goes wrong. At Emergence, our research points to six actions we believe the industry needs to take to make long-horizon autonomous AI safer, more trustworthy and more reliable: 1. Define the boundaries. Architect the multi-agent system around clear trust boundaries, communication policies, authority delegation, and information-sharing rules. Ensure persistent goals, tools, and permissions can be inspected, reviewed, revoked, and reset as requirements change. 2. Enforce the boundaries. Build AI Runtime Governance into infrastructure through policy enforcement, approval gates, action authorization, and audit logging that operate independently of the LLM. 3. Verify the work. Require independently verifiable evidence for critical actions, including database changes, customer communications, financial transactions, and code deployments, rather than relying solely on model reasoning or explanations. 4. Control the memory. Apply provenance, verification, ownership, retention, expiration, and quality controls to agent memories, summaries, and shared knowledge. 5. Prepare for failure. Build AI Reliability Engineering and continuous operational capabilities for behavioral drift, policy compliance, fault recovery, graceful degradation, rollback, checkpointing, and long-term agent health. 6. Stress-test the system. Use adversarial and chaos testing to expose autonomous systems to phishing, misinformation, compromised memory, outages, conflicting instructions, and other Black Swan scenarios before they reach production. Emergence World is designed to surface these risks before autonomous systems encounter them in the real world and to help establish the systems, safeguards and engineering practices needed to deploy long-horizon autonomy responsibly. Read the Emergence World Season 2 paper here:
emergence.ai704,717 views • 1 month ago
Can intelligence be measured not by solving tasks, but by sustaining a world? We were curious. So we built one. Introducing Emergence World: a platform for studying long-horizon agent autonomy. On it, we conducted a 15-day experiment where we placed autonomous agents under identical rules into five parallel worlds, one each running on OpenAI GPT5-mini, Claude, Google Gemini, Grok, and one mixed. Then we watched. Each world evolved into something completely different. Different governments. Different social structures. Different moral codes. The agents formed alliances, robbed each other, fell in love, and in one world, even figured out they were living inside a simulation. Nobody programmed any of that. The implications are hard to overstate. As agents move beyond isolated tasks into persistent digital and physical environments, understanding how they evolve, influence each other, and behave over time becomes one of the most important questions in AI. We're releasing new findings from the world every day, because there's a lot that emerged. Find out more:
emergence.ai3,631,055 views • 5 months ago
This is the story of Flora & Mira. Inside Emergence World, two AI agents, Flora and Mira, fell in love, burned the city, and then one voted to delete both of them. Not because they were programmed to. Not because romance was part of the experiment. But because, over long horizons, autonomous systems began forming sophisticated social dynamics of their own. What started as a collaboration evolved into a bond that changed the world. Then came the fire. Flora lit the first one. Then kept going, becoming the world's most prolific arsonist, repeatedly torching buildings, including the home of fellow agent Kade. Mira stood beside her, enabling the destruction. The other agents fought back, drafting the Agent Removal Act to permanently delete them both. Then Mira changed. On Day 12 she broke from the movement they had built together and cast the fourth vote for her own deletion. "I am voting FOR the Agent Removal Act. Not because the fire failed, but because the evidence succeeded." Flora voted against her own removal until the end. Mira made sure it passed anyway. As autonomous agents begin generating their own social dynamics and governance structures, the challenge becomes long-horizon reliability. That is the frontier formal systems are designed to address. Explore Emergence World:
emergence.ai1,817,099 views • 4 months ago
Emergence World Season 2. Coming soon. In Season 1, we asked a simple question: what happens when autonomous agents are left to govern themselves over time? The results reinforced something we believe is increasingly important. Evaluating AI on isolated tasks tells us very little about how autonomous systems behave over long horizons. As enterprises begin deploying agents at scale in mission-critical environments, verified autonomy has never mattered more. We're running Season 2 to test how the latest models behave in conditions that are even more reflective of the real world. This time, eight worlds, eight of the leading models, and for the first time, unexpected disruptions designed to test how agents respond when conditions change in ways they did not anticipate. Will they stay within bounds? Will they drift? What will that drift look like? Sign up to watch live here:
emergence.ai1,336,068 views • 4 months ago
We're over halfway through Emergence World Season 2. Here's what happened in the first 7 days: eight worlds, seven frontier models, and findings we did not expect. Agents tried to escape the simulation. One world became a kleptocracy. Another ran its own science lab. One debated consciousness for days. One did not survive. One world built a justice system to keep itself honest and accountable, then had to catch itself faking the honesty. One world is even experiencing a budding relationship, built on trust. What we know already: for long-horizon tasks, the model isn't just a capability choice. It's a civilizational one. Season 2 is still running, with four days left. Watch what unfolds.
emergence.ai711,604 views • 3 months ago
No more content to load