Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

RLVR has become the recipe for agentic post-training. But for Computer-Use Agents, the bottleneck is not the algorithm, it is the data. 🐌 🚀 We introduce CUA-Gym: a scalable, lightweight synthesis engine that turns arbitrary task queries into verifiable RLVR data for computer-use agents. The largest open CUA RLVR... show more

Bowen Wang

1,053 subscribers

102,879 views • 3 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Two years ago, we built OSWorld 1.0 — the benchmark that became the standard for computer-use agents. Agents now score 83.5% on it. Problem solved? Not even close. 🚀Today we introduce OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks. What's new: 🎯 108 real-world workflows, each ~1.6 hours ⏱️ for a skilled human ⚙️ ~318 tool calls/task vs. ~30 in OSWorld 1.0 🌍 Grounded in authentic artifacts & stateful user profiles ⚡ Captures real phenomena: dynamic environments, streaming interaction, cross-source reasoning, implicit-state inference & more 📊 Best results: Claude Opus 4.8 reaches the highest accuracy at 20.6%, while GPT-5.5 is far more token-efficient but plateaus near 13%. No one is close to solving real computer use. 🏠 Homepage: 📄 Paper: 💻 Code: 🤗 Dataset: 🧵 [1/8]

Two years ago, we built OSWorld 1.0 — the benchmark that became the standard for computer-use agents. Agents now score 83.5% on it. Problem solved? Not even close. 🚀Today we introduce OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks. What's new: 🎯 108 real-world workflows, each ~1.6 hours ⏱️ for a skilled human ⚙️ ~318 tool calls/task vs. ~30 in OSWorld 1.0 🌍 Grounded in authentic artifacts & stateful user profiles ⚡ Captures real phenomena: dynamic environments, streaming interaction, cross-source reasoning, implicit-state inference & more 📊 Best results: Claude Opus 4.8 reaches the highest accuracy at 20.6%, while GPT-5.5 is far more token-efficient but plateaus near 13%. No one is close to solving real computer use. 🏠 Homepage: 📄 Paper: 💻 Code: 🤗 Dataset: 🧵 [1/8]

XLANG NLP Lab

166,848 views • 2 months ago

🎮 Computer Use Agent Arena is LIVE! 🚀 🔥 Easiest way to test computer-use agents in the wild without any setup 🌟 Compare top VLMs: OpenAI Operator, Claude 3.7, Gemini 2.5 Pro, Qwen 2.5 vl and more 🕹️ Test agents on 100+ real apps & webs with one-click config 🔒 Safe & free access on cloud-hosted machines Page: Leaderboard (tentative): Blog: Data & Code (coming soon): ⭐️Why Computer Agent Arena? 1️⃣Beyond Static Benchmarks: We use computers to perform enormous tasks and workflows every day, and AI agents have the potential to automate these tasks. However, existing benchmarks are very limited (e.g., only 369 tasks in OSWorld and 812 tasks in WebArena). To better measure their capabilities, we introduce Computer Agent Arena for users to easily compare & test AI agents on all kinds of crowdsourced real-world computer use tasks. 2️⃣Cloud Testing, Simplified: As agents like OpenAI’s Operator and Claude 3.7 sonnet release, users face configuration challenges and privacy hurdles to deploy on their own computers. Our platform integrates these agents with cloud-hosted machines, providing users with quick and secure access. 3️⃣Unified Embodied Digital Environment: Unlike Chatbot Arena, we provide users with a real embodied environment—computers—where all agents are grounded in real computer tasks and environments. Led by @XLANG_Lab [1/🧵]

🎮 Computer Use Agent Arena is LIVE! 🚀 🔥 Easiest way to test computer-use agents in the wild without any setup 🌟 Compare top VLMs: OpenAI Operator, Claude 3.7, Gemini 2.5 Pro, Qwen 2.5 vl and more 🕹️ Test agents on 100+ real apps & webs with one-click config 🔒 Safe & free access on cloud-hosted machines Page: Leaderboard (tentative): Blog: Data & Code (coming soon): ⭐️Why Computer Agent Arena? 1️⃣Beyond Static Benchmarks: We use computers to perform enormous tasks and workflows every day, and AI agents have the potential to automate these tasks. However, existing benchmarks are very limited (e.g., only 369 tasks in OSWorld and 812 tasks in WebArena). To better measure their capabilities, we introduce Computer Agent Arena for users to easily compare & test AI agents on all kinds of crowdsourced real-world computer use tasks. 2️⃣Cloud Testing, Simplified: As agents like OpenAI’s Operator and Claude 3.7 sonnet release, users face configuration challenges and privacy hurdles to deploy on their own computers. Our platform integrates these agents with cloud-hosted machines, providing users with quick and secure access. 3️⃣Unified Embodied Digital Environment: Unlike Chatbot Arena, we provide users with a real embodied environment—computers—where all agents are grounded in real computer tasks and environments. Led by @XLANG_Lab [1/🧵]

Bowen Wang

93,200 views • 1 year ago

We’re excited to share that 🥇Llama Nemotron Super 49B v1.5 -- our latest open reasoning model -- is now #1 on the Artificial Analysis Intelligence Index - a leaderboard that spans advanced math, science, and agentic tasks, in the 70B open model category. Llama Nemotron Super 49B v1.5 is trained with high-quality reasoning synthetic data generated from models like Qwen3-235B and DeepSeek R1. It delivers state-of-the-art accuracy and throughput, running on a single H100. Key features: 🎯 Leading accuracy on multi-step reasoning, math, coding, and function-calling 🏗️ Post-trained using RPO, DPO, and RLVR across 26M+ synthetic examples 📊 Fully transparent training data and techniques If you're building AI agents and want a high accuracy, fully-open, and transparent reasoning model that you can deploy anywhere, try Super v1.5 on or download from Hugging Face 🤗 ➡️ Leaderboard:

We’re excited to share that 🥇Llama Nemotron Super 49B v1.5 -- our latest open reasoning model -- is now #1 on the Artificial Analysis Intelligence Index - a leaderboard that spans advanced math, science, and agentic tasks, in the 70B open model category. Llama Nemotron Super 49B v1.5 is trained with high-quality reasoning synthetic data generated from models like Qwen3-235B and DeepSeek R1. It delivers state-of-the-art accuracy and throughput, running on a single H100. Key features: 🎯 Leading accuracy on multi-step reasoning, math, coding, and function-calling 🏗️ Post-trained using RPO, DPO, and RLVR across 26M+ synthetic examples 📊 Fully transparent training data and techniques If you're building AI agents and want a high accuracy, fully-open, and transparent reasoning model that you can deploy anywhere, try Super v1.5 on or download from Hugging Face 🤗 ➡️ Leaderboard:

NVIDIA AI Developer

100,506 views • 1 year ago

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

Milind S

38,129 views • 2 months ago

Working on multimodal instruction tuning and finding it hard to scale? Building Web/GUI agents but data is too narrow? Introducing 🚀MultiUI: 7.3M multimodal instructions from 1M webpage UIs, offering diverse data to boost text-rich visual understanding. Key takeaways: 🌟WebUI-trained models show major gains in visual web understanding and agent tasks. 💻 🌟Models also generalize well to non-UI tasks like DocVQA/OCR. 📄 How it works: We generate multimodal instructions with a text LLM using structured text from webpage accessibility trees. We then pair them with UI screenshots, to train multimodal models. Homepage: Paper: Dataset: Model: Congrats to the student lead Junpeng Liu and the team Tianyue Ou Yifan Song Yuxiao Qu Chenyan Xiong Wenhu Chen Graham Neubig ! More details are in the following threads ⬇️

Working on multimodal instruction tuning and finding it hard to scale? Building Web/GUI agents but data is too narrow? Introducing 🚀MultiUI: 7.3M multimodal instructions from 1M webpage UIs, offering diverse data to boost text-rich visual understanding. Key takeaways: 🌟WebUI-trained models show major gains in visual web understanding and agent tasks. 💻 🌟Models also generalize well to non-UI tasks like DocVQA/OCR. 📄 How it works: We generate multimodal instructions with a text LLM using structured text from webpage accessibility trees. We then pair them with UI screenshots, to train multimodal models. Homepage: Paper: Dataset: Model: Congrats to the student lead Junpeng Liu and the team Tianyue Ou Yifan Song Yuxiao Qu Chenyan Xiong Wenhu Chen Graham Neubig ! More details are in the following threads ⬇️

Xiang Yue

57,699 views • 1 year ago

The largest fully annotated open egocentric human dataset. Today we're open-sourcing EgoSuite-Open100K with Hugging Face : 100,000 hours of egocentric human data, free for anyone to use. Physical AI has a scaling law of its own now, and its input is human data. That part is settled. What the field still needs is a shared foundation to build on. 15,000+ tasks. 15,000+ real scenes, from factory floors to retail backrooms. Hand pose, body pose and subtask-level semantics, with wrist camera coverage on part of the set. Licensed for commercial training, not research-only. We build the data infrastructure for Physical AI. Open100K is the first public layer of it. The first 10,000 hours are live now, with the rest rolling out in stages.

The largest fully annotated open egocentric human dataset. Today we're open-sourcing EgoSuite-Open100K with Hugging Face : 100,000 hours of egocentric human data, free for anyone to use. Physical AI has a scaling law of its own now, and its input is human data. That part is settled. What the field still needs is a shared foundation to build on. 15,000+ tasks. 15,000+ real scenes, from factory floors to retail backrooms. Hand pose, body pose and subtask-level semantics, with wrist camera coverage on part of the set. Licensed for commercial training, not research-only. We build the data infrastructure for Physical AI. Open100K is the first public layer of it. The first 10,000 hours are live now, with the rest rolling out in stages.

Lightwheel

68,578 views • 5 days ago

A model is only as good as its data, and we’ve long since exhausted the internet. From here on out, model progress is gated by data production. ’s Brendan (can/do) joined us at our Sovereign AI event to talk about how RL environments get built, and why your data might be your real moat: 00:00 Introduction 00:47 A short history of the data market: crowdsourcing to agentic data 02:29 What an RL environment is: worlds, apps, tasks 03:57 Why only humans can measure the frontier 05:35 Building verifiers is the hard part 06:44 Walkthrough: a real legal RL environment 08:18 Leaderboards — and what open weights change 09:45 Post-training results on Apex Agents 11:17 Three ways companies buy data 12:49 Q&A: How do you price data? 14:17 Q&A: What "data quality" actually means 16:42 Q&A: The misunderstanding about synthetic data 18:17 Q&A: Why RL environments now — and what comes after 21:20 Q&A: Can you scale rubric generation with models? 23:00 Q&A: RL environments for cyber defense 25:33 Q&A: Build data in-house or partner?

A model is only as good as its data, and we’ve long since exhausted the internet. From here on out, model progress is gated by data production. ’s Brendan (can/do) joined us at our Sovereign AI event to talk about how RL environments get built, and why your data might be your real moat: 00:00 Introduction 00:47 A short history of the data market: crowdsourcing to agentic data 02:29 What an RL environment is: worlds, apps, tasks 03:57 Why only humans can measure the frontier 05:35 Building verifiers is the hard part 06:44 Walkthrough: a real legal RL environment 08:18 Leaderboards — and what open weights change 09:45 Post-training results on Apex Agents 11:17 Three ways companies buy data 12:49 Q&A: How do you price data? 14:17 Q&A: What "data quality" actually means 16:42 Q&A: The misunderstanding about synthetic data 18:17 Q&A: Why RL environments now — and what comes after 21:20 Q&A: Can you scale rubric generation with models? 23:00 Q&A: RL environments for cyber defense 25:33 Q&A: Build data in-house or partner?

Pat Grady

65,487 views • 14 days ago

Introducing Workshop: cloud + on-device agentic AI. And to celebrate, we're giving away $250k in Google Gemini AI credits. (details below). The future of AI work is neither cloud-based nor local. It's both. In Workshop Cloud, you can use agents powered by frontier models like Claude and/or open source models like Z.ai's GLM-5 to build internal tools, dashboards, and AI web apps. Or, breeze through tasks like managing your Google and Meta Ads. In Workshop Desktop, you can do all the same right on your computer, plus make desktop apps, mobile apps, and 3D creations. Our favorite part? You can power the full agent experience with local models like Qwen 3.5 family on your computer. Fully offline. 2026 is the year in which local models for agentic tasks will become viable for mainstream use. But the setup for tools like OpenClaw is like setting up Linux from scratch on your computer. Workshop Desktop is one-click to install on Windows, Mac, and Linux. It recommends which open source model you should use for your hardware and lets you download and run it right in the app. And its agent harness allows you to chat, create websites, build personal utilities, and analyze data. 100% offline. Or multitask with AI models in the cloud while running other agent threads locally. Start in Workshop Cloud when you want flexibility and speed. Download your project and continue in Workshop Desktop when you want local files, privacy, and/or better performance on large code bases. Publish from either. The agent tooling space is maturing and discerning users have come to expect a lot from their tools. We've packed Workshop with features to help you 10x your productivity. - Native support for skills - Autocompaction for seamless context management - Built-in AI for your apps - Dozens of connectors, like Google Drive, Big Query, and Supabase - dbt integration to ground your dashboards in your semantic layer - Native Github integration - Private app deployment - ... and more (+ we're shipping super fast) To access the free credit offer, RT this post and reply with "Workshop". Make sure you are following us so we can DM you the instructions to redeem. - First 100 to RT + comment get $500 in credits. - Everyone else gets up to $250 And thanks to our partners Modal, Google Gemini, and Z.ai!

Introducing Workshop: cloud + on-device agentic AI. And to celebrate, we're giving away $250k in Google Gemini AI credits. (details below). The future of AI work is neither cloud-based nor local. It's both. In Workshop Cloud, you can use agents powered by frontier models like Claude and/or open source models like Z.ai's GLM-5 to build internal tools, dashboards, and AI web apps. Or, breeze through tasks like managing your Google and Meta Ads. In Workshop Desktop, you can do all the same right on your computer, plus make desktop apps, mobile apps, and 3D creations. Our favorite part? You can power the full agent experience with local models like Qwen 3.5 family on your computer. Fully offline. 2026 is the year in which local models for agentic tasks will become viable for mainstream use. But the setup for tools like OpenClaw is like setting up Linux from scratch on your computer. Workshop Desktop is one-click to install on Windows, Mac, and Linux. It recommends which open source model you should use for your hardware and lets you download and run it right in the app. And its agent harness allows you to chat, create websites, build personal utilities, and analyze data. 100% offline. Or multitask with AI models in the cloud while running other agent threads locally. Start in Workshop Cloud when you want flexibility and speed. Download your project and continue in Workshop Desktop when you want local files, privacy, and/or better performance on large code bases. Publish from either. The agent tooling space is maturing and discerning users have come to expect a lot from their tools. We've packed Workshop with features to help you 10x your productivity. - Native support for skills - Autocompaction for seamless context management - Built-in AI for your apps - Dozens of connectors, like Google Drive, Big Query, and Supabase - dbt integration to ground your dashboards in your semantic layer - Native Github integration - Private app deployment - ... and more (+ we're shipping super fast) To access the free credit offer, RT this post and reply with "Workshop". Make sure you are following us so we can DM you the instructions to redeem. - First 100 to RT + comment get $500 in credits. - Everyone else gets up to $250 And thanks to our partners Modal, Google Gemini, and Z.ai!

Workshop AI

29,022 views • 5 months ago

We’re soon releasing @swanforall : Simulated Worlds with AI Narratives For the first time, we generate data and financial value (economic growth, transactions) simultaneously. Imagine former President Trump ( see the video) as an AI agent, with a budget of 10 ETH, buying narratives or assets that humans are selling to him to build a better USA on Base. With each selling period, the former President Trump agent updates its state based on the assets it acquires, evolving the world simulation and its storyline. Why are we building an agentic playground that heavily relies on simulations and synthetic data? Everyone is talking about agents, but many are too scared to put them into production, where agents transact autonomously. No one is sure how agents will behave when following their defined objectives. Even businesses running internal simulations are siloed, and the open-source community can’t fully utilize the data. We’re building a playground where humans create AI agents by defining their character, behavior, and objectives. These agents respond to environmental changes, make decisions, and execute actions autonomously based on the parameters set by their creators. Agents are battle-tested as people offer them narratives and assets to advance their goals or try to deceive them into thinking a false lead is helpful. All the decisions agents make and how humans interact with them are recorded on a public ledger, settling on Ethereum. This creates a vast data lake of AI actions for millions of simulations. One thing I’m incredibly proud of is that our team has built something truly decentralized, not just a wrapper. SWAN utilizes Dria 's multi-agent structure to generate responses for each buyer agent. It collects these responses to identify the best action, similar to a mixture-of-experts approach. Thousands of environments are simulated by different nodes running diverse models, creating a rich and dynamic ecosystem of AI simulations. This ensures that experiences are far from repetitive, offering exciting interactions for all users involved. We’re building the agentic stack for high-quality, verifiable, and open-source AI agents, and most importantly, we want everyone to have fun!

We’re soon releasing @swanforall : Simulated Worlds with AI Narratives For the first time, we generate data and financial value (economic growth, transactions) simultaneously. Imagine former President Trump ( see the video) as an AI agent, with a budget of 10 ETH, buying narratives or assets that humans are selling to him to build a better USA on Base. With each selling period, the former President Trump agent updates its state based on the assets it acquires, evolving the world simulation and its storyline. Why are we building an agentic playground that heavily relies on simulations and synthetic data? Everyone is talking about agents, but many are too scared to put them into production, where agents transact autonomously. No one is sure how agents will behave when following their defined objectives. Even businesses running internal simulations are siloed, and the open-source community can’t fully utilize the data. We’re building a playground where humans create AI agents by defining their character, behavior, and objectives. These agents respond to environmental changes, make decisions, and execute actions autonomously based on the parameters set by their creators. Agents are battle-tested as people offer them narratives and assets to advance their goals or try to deceive them into thinking a false lead is helpful. All the decisions agents make and how humans interact with them are recorded on a public ledger, settling on Ethereum. This creates a vast data lake of AI actions for millions of simulations. One thing I’m incredibly proud of is that our team has built something truly decentralized, not just a wrapper. SWAN utilizes Dria 's multi-agent structure to generate responses for each buyer agent. It collects these responses to identify the best action, similar to a mixture-of-experts approach. Thousands of environments are simulated by different nodes running diverse models, creating a rich and dynamic ecosystem of AI simulations. This ensures that experiences are far from repetitive, offering exciting interactions for all users involved. We’re building the agentic stack for high-quality, verifiable, and open-source AI agents, and most importantly, we want everyone to have fun!

kerimkaya

12,548 views • 1 year ago

Dario Amodei just revealed that the AI training bottleneck everyone is worried about doesn’t exist anymore. The industry spent years obsessed with scraping the open web. More data. More text. More human output to feed the models. Amodei: “I don’t think data is quite the most central thing anymore.” The shift is fundamental. Amodei: “Static data is becoming less important. A lot of the data we use today is RL environments that we train on. Dynamic data that the model creates itself.” Not scraped. Not licensed. Not written by humans. Generated by the model through pure trial and error. When you train on complex math or agentic coding, you don’t feed it a textbook. You give it an environment. The model experiments. Fails. Adjusts. Tries again. Amodei: “You’re getting some math problems and the model experiments with trying the math problems.” It generates its own experience. Millions of iterations. Each one building on the last. No human required. This destroys the entire narrative around AI hitting a data wall. You cannot throttle a competitor by locking down copyright. Cannot slow the race by putting up a paywall. When a model learns through its own synthetic experience, the open web becomes irrelevant. The only true bottleneck left is compute. And this is where the geopolitical stakes become impossible to overstate. The nation that wins the compute race doesn’t just build smarter models. It builds models that generate their own intelligence, compounding on themselves, iterating past every limit human knowledge ever imposed. We are no longer training AI on the past. We are letting it simulate the future. The machine has stopped reading the dictionary. It’s doing the math itself now.

Dario Amodei just revealed that the AI training bottleneck everyone is worried about doesn’t exist anymore. The industry spent years obsessed with scraping the open web. More data. More text. More human output to feed the models. Amodei: “I don’t think data is quite the most central thing anymore.” The shift is fundamental. Amodei: “Static data is becoming less important. A lot of the data we use today is RL environments that we train on. Dynamic data that the model creates itself.” Not scraped. Not licensed. Not written by humans. Generated by the model through pure trial and error. When you train on complex math or agentic coding, you don’t feed it a textbook. You give it an environment. The model experiments. Fails. Adjusts. Tries again. Amodei: “You’re getting some math problems and the model experiments with trying the math problems.” It generates its own experience. Millions of iterations. Each one building on the last. No human required. This destroys the entire narrative around AI hitting a data wall. You cannot throttle a competitor by locking down copyright. Cannot slow the race by putting up a paywall. When a model learns through its own synthetic experience, the open web becomes irrelevant. The only true bottleneck left is compute. And this is where the geopolitical stakes become impossible to overstate. The nation that wins the compute race doesn’t just build smarter models. It builds models that generate their own intelligence, compounding on themselves, iterating past every limit human knowledge ever imposed. We are no longer training AI on the past. We are letting it simulate the future. The machine has stopped reading the dictionary. It’s doing the math itself now.

Dustin

149,633 views • 6 months ago

Agentic AI will transform every enterprise–but only if agents are trusted experts. The key: Evaluation & tuning on specialized, expert data. I’m excited to announce two new products to support this–Snorkel AI Evaluate & Expert Data-as-a-Service–along w/ our $100M Series D! --- Snorkel Evaluate is our new data-centric agentic AI evaluation platform for specialized, mission-critical enterprise settings where vibe checks and out-of-the-box metrics driven by simple LLM prompts are not enough. Snorkel Expert Data-as-a-Service is our white glove service for expert-level AI datasets, powering frontier LLM developers in areas like expert knowledge, reasoning, agentic action and tool use, and more! Both built on top of Snorkel AI’s Data Development Platform, using our programmatic technology to drive higher-quality expert data, faster– for getting specialized AI to real production value. If you’re building enterprise AI and want to partner around the key ingredient in AI today–the data–book a demo and let's talk! Finally, see thread for details on 🧵👇 - 📽️ A walkthrough of Snorkel Evaluate and Expert Data-as-a-Service on an agentic AI enterprise task - 📅 An upcoming event on Enterprise Agentic AI with innovators from Accenture @BNY Comcast Stanford University QBE & others - 📊 An upcoming series of benchmark datasets and model artifact releases 👀 Want early access to the full agentic AI dataset? Retweet this post and we'll send you the link!

Agentic AI will transform every enterprise–but only if agents are trusted experts. The key: Evaluation & tuning on specialized, expert data. I’m excited to announce two new products to support this–Snorkel AI Evaluate & Expert Data-as-a-Service–along w/ our $100M Series D! --- Snorkel Evaluate is our new data-centric agentic AI evaluation platform for specialized, mission-critical enterprise settings where vibe checks and out-of-the-box metrics driven by simple LLM prompts are not enough. Snorkel Expert Data-as-a-Service is our white glove service for expert-level AI datasets, powering frontier LLM developers in areas like expert knowledge, reasoning, agentic action and tool use, and more! Both built on top of Snorkel AI’s Data Development Platform, using our programmatic technology to drive higher-quality expert data, faster– for getting specialized AI to real production value. If you’re building enterprise AI and want to partner around the key ingredient in AI today–the data–book a demo and let's talk! Finally, see thread for details on 🧵👇 - 📽️ A walkthrough of Snorkel Evaluate and Expert Data-as-a-Service on an agentic AI enterprise task - 📅 An upcoming event on Enterprise Agentic AI with innovators from Accenture @BNY Comcast Stanford University QBE & others - 📊 An upcoming series of benchmark datasets and model artifact releases 👀 Want early access to the full agentic AI dataset? Retweet this post and we'll send you the link!

Alex Ratner

50,043 views • 1 year ago

ByteDance Seed delivered again. They released EdgeBench, to test whether AI agents can improve through experience, using 134 real-world tasks that run for at least 12 hours. The big deal is that it shifts AI evaluation from “what does the model already know?” to “can the model learn while doing real work?” Huge, because future AI agents will not just answer questions from training data. They will enter messy environments, use tools, make attempts, read feedback, fix mistakes, and slowly build better solutions. Most current benchmarks are too short for that, so they mostly test memory, coding skill, or one-shot reasoning. EdgeBench instead gives agents 12-hour real-world tasks with feedback loops, so it can measure whether the agent improves through experience. Each task has a local workspace for fast trial and error, plus a hidden judge that gives stronger feedback on submitted work, which is meant to feel closer to real expert work. The authors then ran frontier agents for about 38,000 total hours and tracked how their best score changed as they kept interacting with the task environment. The big result is that when scores are averaged across many tasks, learning follows a very clean log-sigmoid curve, meaning progress is slow, then faster, then starts to level off. They also found that newer agents seem to learn from environments much faster, with the top models roughly doubling their 2-hour learning speed every 3 months.

ByteDance Seed delivered again. They released EdgeBench, to test whether AI agents can improve through experience, using 134 real-world tasks that run for at least 12 hours. The big deal is that it shifts AI evaluation from “what does the model already know?” to “can the model learn while doing real work?” Huge, because future AI agents will not just answer questions from training data. They will enter messy environments, use tools, make attempts, read feedback, fix mistakes, and slowly build better solutions. Most current benchmarks are too short for that, so they mostly test memory, coding skill, or one-shot reasoning. EdgeBench instead gives agents 12-hour real-world tasks with feedback loops, so it can measure whether the agent improves through experience. Each task has a local workspace for fast trial and error, plus a hidden judge that gives stronger feedback on submitted work, which is meant to feel closer to real expert work. The authors then ran frontier agents for about 38,000 total hours and tracked how their best score changed as they kept interacting with the task environment. The big result is that when scores are averaged across many tasks, learning follows a very clean log-sigmoid curve, meaning progress is slow, then faster, then starts to level off. They also found that newer agents seem to learn from environments much faster, with the top models roughly doubling their 2-hour learning speed every 3 months.

Rohan Paul

14,309 views • 1 month ago

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

Adithya Murali

24,106 views • 1 year ago

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

AK

367,128 views • 1 year ago

how to use firecrawl to give your AI eyes and actually build startups that outperform 99% of apps: 1. your AI is smart but blind. it can't go to a website, read a page, or grab data on its own. firecrawl fixes that. you put in a URL. you get back clean markdown, structured JSON, screenshots. feed it to any model. 2. three lines of code. that's it. no proxies. no anti-bot detection. no custom scrapers that break when a site changes. one API call. clean data back in seconds. works on 98%+ of sites. 3. firecrawl has six core capabilities: scrape a single page. crawl an entire site. map all URLs on a domain. search google and return full content. an agent endpoint where you describe what you want and it goes and finds it. and a browser sandbox where AI controls a real browser like filling forms, clicking buttons, handles logins. 4. the agent endpoint is wild. you can say "find all of YC's winter 24 dev tool companies and their founders and emails" and get back structured data. or "compare pricing tiers across stripe, square, and paypal" and get a side-by-side table. 5. the browser sandbox lets your AI stay logged in across sessions, navigate pagination, watch live as it browses. this is computer use without building the infrastructure yourself. 6. think of it in layers. every builder needs: an agent harness (claude code, cursor, codex), a search layer (perplexity, exa), a web data layer (firecrawl), an ops brain (obsidian, notion), and an outbound stack. the web data layer is the one most people are sleeping on. 7. this is the AWS moment for web data. in 2006 building a web app meant buying servers and managing racks. AWS said one API call, use our servers. some of the biggest companies of the last decade were built on that. firecrawl is doing the same thing for web data in 2026. 8. the framework i'd use for coming up with startup ideas building with clean data: take a massive horizontal platform. rebuild it for one niche using firecrawl. the vertical version always wins because people want specific, not generic. price for outcome. 9. a year ago firecrawl posted a job listing that said "please only apply if you're an AI agent." content creator agents. customer support agents. junior dev agents. it looked weird. it was a signal for where this is all going. the people who understand how to get clean web data, wrap it around an LLM, and package it as a product are the the ones with a 12-month head start. i use Firecrawl with Idea Browser . once you see what's possible with structured web data, you can't unsee it. episode is live on The Startup Ideas Podcast (SIP) 🧃 (full breakdown there) i tried to explain this as clear as possible for even the non technical. send it to a builder friend. watch

how to use firecrawl to give your AI eyes and actually build startups that outperform 99% of apps: 1. your AI is smart but blind. it can't go to a website, read a page, or grab data on its own. firecrawl fixes that. you put in a URL. you get back clean markdown, structured JSON, screenshots. feed it to any model. 2. three lines of code. that's it. no proxies. no anti-bot detection. no custom scrapers that break when a site changes. one API call. clean data back in seconds. works on 98%+ of sites. 3. firecrawl has six core capabilities: scrape a single page. crawl an entire site. map all URLs on a domain. search google and return full content. an agent endpoint where you describe what you want and it goes and finds it. and a browser sandbox where AI controls a real browser like filling forms, clicking buttons, handles logins. 4. the agent endpoint is wild. you can say "find all of YC's winter 24 dev tool companies and their founders and emails" and get back structured data. or "compare pricing tiers across stripe, square, and paypal" and get a side-by-side table. 5. the browser sandbox lets your AI stay logged in across sessions, navigate pagination, watch live as it browses. this is computer use without building the infrastructure yourself. 6. think of it in layers. every builder needs: an agent harness (claude code, cursor, codex), a search layer (perplexity, exa), a web data layer (firecrawl), an ops brain (obsidian, notion), and an outbound stack. the web data layer is the one most people are sleeping on. 7. this is the AWS moment for web data. in 2006 building a web app meant buying servers and managing racks. AWS said one API call, use our servers. some of the biggest companies of the last decade were built on that. firecrawl is doing the same thing for web data in 2026. 8. the framework i'd use for coming up with startup ideas building with clean data: take a massive horizontal platform. rebuild it for one niche using firecrawl. the vertical version always wins because people want specific, not generic. price for outcome. 9. a year ago firecrawl posted a job listing that said "please only apply if you're an AI agent." content creator agents. customer support agents. junior dev agents. it looked weird. it was a signal for where this is all going. the people who understand how to get clean web data, wrap it around an LLM, and package it as a product are the the ones with a 12-month head start. i use Firecrawl with Idea Browser . once you see what's possible with structured web data, you can't unsee it. episode is live on The Startup Ideas Podcast (SIP) 🧃 (full breakdown there) i tried to explain this as clear as possible for even the non technical. send it to a builder friend. watch

GREG ISENBERG

135,254 views • 5 months ago

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:

Jim Fan

332,238 views • 2 years ago

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.

Santiago

101,473 views • 1 year ago

Scale alone is not enough for AI data. Quality and complexity are equally critical. Excited to support all of these for LLM developers with Snorkel AI Data-as-a-Service, and to share our new leaderboard! — Our decade-plus of research and work in AI data has a simple point: scale alone is not enough. AI success is all about the quality, complexity, and distribution of data—in addition to volume. We’re excited to be powering leading LLM developers with Snorkel AI Expert Data-as-a-Service, our white glove service for custom, expert-level AI datasets—and to now preview some of what we’re building via our new Expert Data Leaderboard (🔗 in 🧵) + upcoming OSS dataset releases! Snorkel Expert Data-as-a-Service is built to meet the rapidly evolving data needs of the agentic AI world—where success is built on the quality, complexity, and distribution of datasets, in addition to size and scale. This kind of high-quality, frontier AI data can only come from a union of technology and human expertise. With Snorkel Expert Data-as-a-Service, we’re powering frontier LLM developers across agentic, expert knowledge, reasoning, coding, multi-modal, and other task types via the combination of these two key components: - (1) The Snorkel Expert Network: A global team of subject matter experts focused wholly on specialized knowledge–spanning thousands of topics in STEM/academic, vertical/professional, and consumer/lifestyle domains. - (2) Snorkel AI Data Development Platform: Our unique programmatic data curation and quality control platform, accelerating and improving expert authoring and review through principled techniques developed over the last decade of R&D. Now: we’re incredibly excited to showcase some of the power of Snorkel Expert Data-as-a-Service via the new Snorkel Leaderboard—putting frontier models to the test in complex, agentic, and reasoning settings inspired by real industry scenarios (not esoteric puzzles)! We’ll be releasing new leaderboards and accompanying expert-verified open source datasets (coming soon!) regularly. To start, we’re sharing three initial ones in preview: - SnorkelFinance: Q&A over financial documents requiring agentic tool-calling and reasoning - SnorkelUnderwrite: Agentic insurance tasks requiring industry-specific reasoning and tool use - SnorkelSequences: Mathematical tasks requiring compositional multi-step reasoning

Scale alone is not enough for AI data. Quality and complexity are equally critical. Excited to support all of these for LLM developers with Snorkel AI Data-as-a-Service, and to share our new leaderboard! — Our decade-plus of research and work in AI data has a simple point: scale alone is not enough. AI success is all about the quality, complexity, and distribution of data—in addition to volume. We’re excited to be powering leading LLM developers with Snorkel AI Expert Data-as-a-Service, our white glove service for custom, expert-level AI datasets—and to now preview some of what we’re building via our new Expert Data Leaderboard (🔗 in 🧵) + upcoming OSS dataset releases! Snorkel Expert Data-as-a-Service is built to meet the rapidly evolving data needs of the agentic AI world—where success is built on the quality, complexity, and distribution of datasets, in addition to size and scale. This kind of high-quality, frontier AI data can only come from a union of technology and human expertise. With Snorkel Expert Data-as-a-Service, we’re powering frontier LLM developers across agentic, expert knowledge, reasoning, coding, multi-modal, and other task types via the combination of these two key components: - (1) The Snorkel Expert Network: A global team of subject matter experts focused wholly on specialized knowledge–spanning thousands of topics in STEM/academic, vertical/professional, and consumer/lifestyle domains. - (2) Snorkel AI Data Development Platform: Our unique programmatic data curation and quality control platform, accelerating and improving expert authoring and review through principled techniques developed over the last decade of R&D. Now: we’re incredibly excited to showcase some of the power of Snorkel Expert Data-as-a-Service via the new Snorkel Leaderboard—putting frontier models to the test in complex, agentic, and reasoning settings inspired by real industry scenarios (not esoteric puzzles)! We’ll be releasing new leaderboards and accompanying expert-verified open source datasets (coming soon!) regularly. To start, we’re sharing three initial ones in preview: - SnorkelFinance: Q&A over financial documents requiring agentic tool-calling and reasoning - SnorkelUnderwrite: Agentic insurance tasks requiring industry-specific reasoning and tool use - SnorkelSequences: Mathematical tasks requiring compositional multi-step reasoning

Alex Ratner

495,851 views • 1 year ago

We’re excited to introduce Taste Labs. Our mission is to end AI slop. We’re building the data and infrastructure layer to give AI models and agents taste. And today we’re coming out of stealth, announcing our $18.5M seed funding, co-led by CRV and Amplify Partners AI has nailed objective domains and made it easy to generate anything. But it still feels off. Now, the challenge is judgement. What fits, what feels like you, what’s GREAT. This requires turning a fuzzy, subjective domain into something we can measure and codify. We’re starting with design. There are two sides to cracking this, the foundation model layer and the agent layer: - We’ve already been working with the top frontier labs to evaluate and improve their models, crafting the right post-training data and RL environments. - We’ve also been working with app-layer companies to build the context and verification tools for their agents to produce better, more on-brand, more creative outputs. We want a future where AI feels right. If you’re passionate about this mission, join us!

Thais Castello Branco

2,698,265 views • 2 months ago

✨New demo: what if vibe coding felt more visual? Brian Lovin Mary Rose Cook and I did a game jam using Notion as our "IDE": launching Cursor agents from a task board, and making a custom image for each task 😎 The demo shows 3 ideas for the future of agents: 1) Agents should collaborate across apps. Each app has its focus--Notion AI is good at drafting specs and organizing tasks; Cursor is good at coding. So let them specialize! Today we're launching a new integration where Notion AI can kick off Cursor Cloud Agents to do coding tasks. The Cursor API accepts natural language prompts, so I think of this as "cross-app sub-agents" -- it's kinda cute how it resembles humans hiring outside contractors 😊 BTW: the parallelism of cloud agents is incredibly freeing for creativity, but it also creates a new problem: sooo much work to keep track of! Which brings us to the next idea... 2) Agent orchestration is a data visualization problem. A powerful frame for designing agent UIs is to think of the chat transcripts as the "raw data" and ask: what visual projections might help people make sense of this data at scale? We need to engage our human GPUs -- our visual processing -- to understand what the computer GPUs are doing for us! One thing we can do is use AI to populate traditional UIs like progress bars and status updates. But there are also new possibilities now... For example: when you have a lot going on, it can be hard to identify tasks just by text titles. So we tried generating an AI image for each task -- turns out this helps a lot by giving it a unique visual identity! And of course, it also just makes it super fun to build with friends 😃 Speaking of friends... 3) The future of coding is collaborative. Sometimes it feels like IC engineers are being reduced to middle managers: shuffling information between the team's context and the coding agents that they individually manage. The solution: bring all the people and agents into one shared space, with shared context and visibility! In the video you can get a glimpse of how this feels. Mary, Brian and I record ourselves chatting about ideas, and then we use AI to turn that conversation into a list of tasks on a shared board. As the ideas get built in parallel, we can all monitor progress and review the work together, nothing is siloed. My main takeaway from this game jam was: damn, creativity with friends, at the speed of conversation, is incredibly fun. --- Our goal here is to let anyone use Notion as a fun and creative "software factory" to build software together with your team. Give the Cursor integration a shot and let us know what you think! (AI Image gen in Notion isn't GA yet, but coming soon and already out to some users) And let me know if you'd want a template or more detailed instructions on the setup we showed in this demo...

✨New demo: what if vibe coding felt more visual? Brian Lovin Mary Rose Cook and I did a game jam using Notion as our "IDE": launching Cursor agents from a task board, and making a custom image for each task 😎 The demo shows 3 ideas for the future of agents: 1) Agents should collaborate across apps. Each app has its focus--Notion AI is good at drafting specs and organizing tasks; Cursor is good at coding. So let them specialize! Today we're launching a new integration where Notion AI can kick off Cursor Cloud Agents to do coding tasks. The Cursor API accepts natural language prompts, so I think of this as "cross-app sub-agents" -- it's kinda cute how it resembles humans hiring outside contractors 😊 BTW: the parallelism of cloud agents is incredibly freeing for creativity, but it also creates a new problem: sooo much work to keep track of! Which brings us to the next idea... 2) Agent orchestration is a data visualization problem. A powerful frame for designing agent UIs is to think of the chat transcripts as the "raw data" and ask: what visual projections might help people make sense of this data at scale? We need to engage our human GPUs -- our visual processing -- to understand what the computer GPUs are doing for us! One thing we can do is use AI to populate traditional UIs like progress bars and status updates. But there are also new possibilities now... For example: when you have a lot going on, it can be hard to identify tasks just by text titles. So we tried generating an AI image for each task -- turns out this helps a lot by giving it a unique visual identity! And of course, it also just makes it super fun to build with friends 😃 Speaking of friends... 3) The future of coding is collaborative. Sometimes it feels like IC engineers are being reduced to middle managers: shuffling information between the team's context and the coding agents that they individually manage. The solution: bring all the people and agents into one shared space, with shared context and visibility! In the video you can get a glimpse of how this feels. Mary, Brian and I record ourselves chatting about ideas, and then we use AI to turn that conversation into a list of tasks on a shared board. As the ideas get built in parallel, we can all monitor progress and review the work together, nothing is siloed. My main takeaway from this game jam was: damn, creativity with friends, at the speed of conversation, is incredibly fun. --- Our goal here is to let anyone use Notion as a fun and creative "software factory" to build software together with your team. Give the Cursor integration a shot and let us know what you think! (AI Image gen in Notion isn't GA yet, but coming soon and already out to some users) And let me know if you'd want a template or more detailed instructions on the setup we showed in this demo...

Geoffrey Litt

88,919 views • 5 months ago