Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Today, we're open sourcing the first computer-use dataset for design. We captured 200+ hours and 3400+ trajectories of real design work in Figma, recording step by step how designers execute complex, long-horizon tasks. FigmaTrace dataset, paper, and blog below :) PatronusAI

Anand Kannappan

1,534 subscribers

133,155 views • 5 days ago •via X (Twitter)

Science & Technology Education Health & Wellness

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Today we're launching the most advanced computer-use dataset in the world. 1,000+ hours of screen recordings along with mouse/keyboard inputs + annotations. Sourced from experts across coding, design, browser-use, research and more. Link in the comments :) Markov

Today we're launching the most advanced computer-use dataset in the world. 1,000+ hours of screen recordings along with mouse/keyboard inputs + annotations. Sourced from experts across coding, design, browser-use, research and more. Link in the comments :) Markov

Dev

47,966 views • 4 months ago

Today, we're launching the world's largest open-source dataset of computer-use recordings. 10,000+ hours across Salesforce, Blender, Photoshop and more, to automate the next level of white-collar work. Link in the comments :) Markov

Today, we're launching the world's largest open-source dataset of computer-use recordings. 10,000+ hours across Salesforce, Blender, Photoshop and more, to automate the next level of white-collar work. Link in the comments :) Markov

Dev

473,118 views • 5 months ago

Today, we're launching the largest open-source dataset of gaming data for computer-use. 500+ hours of gameplay screen recordings + keystrokes/mouse movements, across Valorant, Minecraft, GTA and more. Link in the comments :) @markov__ai

Today, we're launching the largest open-source dataset of gaming data for computer-use. 500+ hours of gameplay screen recordings + keystrokes/mouse movements, across Valorant, Minecraft, GTA and more. Link in the comments :) @markov__ai

Dev

104,483 views • 1 month ago

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.

Y Combinator

20,944 views • 9 months ago

Today, we're launching the world's largest open-source CAD dataset for computer-use. 1,000+ hours across SolidWorks, Siemens-NX, AutoCAD & more. Screen recordings, mouse/keyboard actions, input diagrams, output files, rubrics and narrations. Link in the comments :)

Today, we're launching the world's largest open-source CAD dataset for computer-use. 1,000+ hours across SolidWorks, Siemens-NX, AutoCAD & more. Screen recordings, mouse/keyboard actions, input diagrams, output files, rubrics and narrations. Link in the comments :)

Dev

151,586 views • 4 days ago

We're going back to basics today. 🤓 Designing your first button is a rite of passage and a great way to get hands-on practice when starting your design journey. Here's a step-by-step guide to get you started.

We're going back to basics today. 🤓 Designing your first button is a rite of passage and a great way to get hands-on practice when starting your design journey. Here's a step-by-step guide to get you started.

Figma

77,068 views • 3 years ago

What is the best video editing agent for short form social? Does it actually work? We watched professional video editors, step by step, as they built short-form social reels in Adobe Premiere Pro. Today we're open-sourcing this preview dataset on Hugging Face, to make AI agents better at editing videos. The data set is 234 annotated steps across 4 computer-use trajectories. Editors narrated their reasoning aloud as they worked, so every step pairs a screenshot with the expert's own thought, a structured action, and executable grounding: >a Premiere MCP tool call, keyboard shortcut, menu path, or coordinate click. >The format follows the AgentNet trajectory schema, extended with a Premiere action taxonomy and multi-path execution. ***That makes it directly usable for computer-use agent SFT, reasoning mid-training, tool-use and function calling, and benchmarking agents against a human expert baseline. Enjoy!

What is the best video editing agent for short form social? Does it actually work? We watched professional video editors, step by step, as they built short-form social reels in Adobe Premiere Pro. Today we're open-sourcing this preview dataset on Hugging Face, to make AI agents better at editing videos. The data set is 234 annotated steps across 4 computer-use trajectories. Editors narrated their reasoning aloud as they worked, so every step pairs a screenshot with the expert's own thought, a structured action, and executable grounding: >a Premiere MCP tool call, keyboard shortcut, menu path, or coordinate click. >The format follows the AgentNet trajectory schema, extended with a Premiere action taxonomy and multi-path execution. ***That makes it directly usable for computer-use agent SFT, reasoning mid-training, tool-use and function calling, and benchmarking agents against a human expert baseline. Enjoy!

ben

39,640 views • 1 month ago

How can we address the scarcity of data required for specialized AI? Learn about Simula, a framework that reframes synthetic data generation as dataset-level mechanism design. By using reasoning to architect datasets from first principles, Simula enables fine-grained control over coverage, complexity, and quality. More →

How can we address the scarcity of data required for specialized AI? Learn about Simula, a framework that reframes synthetic data generation as dataset-level mechanism design. By using reasoning to architect datasets from first principles, Simula enables fine-grained control over coverage, complexity, and quality. More →

Google Research

142,586 views • 4 months ago

We are proud to announce our first contribution to open source Computer Vision research! 🚀 Today, we’re publishing on Hugging Face a 1,000 3D Maps Dataset under CC-BY-NC license, designed to accelerate progress in 3D perception and spatial AI. Our way to give back to the global AI community and show how DePIN can make an impact. 🧵 👉

We are proud to announce our first contribution to open source Computer Vision research! 🚀 Today, we’re publishing on Hugging Face a 1,000 3D Maps Dataset under CC-BY-NC license, designed to accelerate progress in 3D perception and spatial AI. Our way to give back to the global AI community and show how DePIN can make an impact. 🧵 👉

Over the Reality 🌐

88,633 views • 9 months ago

As promised! New blog with Josh Newton combining our talks from AI Engineer's first design track. In addition, we wanted to actually show a demonstration of how you can iterate and improve upon that first design output from AI. We had a ton of fun building the interactions for this post (especially the horse animator hahaha). Would love to hear your thoughts!

As promised! New blog with Josh Newton combining our talks from AI Engineer's first design track. In addition, we wanted to actually show a demonstration of how you can iterate and improve upon that first design output from AI. We had a ton of fun building the interactions for this post (especially the horse animator hahaha). Would love to hear your thoughts!

Sabina

12,300 views • 22 days ago

This is a call for creatives with a mission to serve. I’m looking for the best designers, artists, and creatives in the world to take on hard and real problems. Come use your superpowers to craft intuitive, complex products and stories. Design isn’t decoration, it’s direction. At Anduril Industries, design is embedded as part of our core strategy. Our team crafts every touch point—from our products to our website, our videos and marketing campaigns, concept and mission communication, even the architecture and the spaces we work in. #Hiring all Summer: Industrial, prototyping, CMF, photography, film, motion, interaction, digital, environmental, merchandise, OOH, graphics, identity systems.

This is a call for creatives with a mission to serve. I’m looking for the best designers, artists, and creatives in the world to take on hard and real problems. Come use your superpowers to craft intuitive, complex products and stories. Design isn’t decoration, it’s direction. At Anduril Industries, design is embedded as part of our core strategy. Our team crafts every touch point—from our products to our website, our videos and marketing campaigns, concept and mission communication, even the architecture and the spaces we work in. #Hiring all Summer: Industrial, prototyping, CMF, photography, film, motion, interaction, digital, environmental, merchandise, OOH, graphics, identity systems.

jen bucci

44,594 views • 1 year ago

Trained on zero real-world data. Learned to walk, pick up boxes, and follow multi-step instructions... in the REAL world. ( 📌 Paper below) Researchers from Amazon FAR, Berkeley, Stanford, and CMU scanned real rooms with an iPhone, rebuilt them as 3D Gaussian Splatting scenes, then generated 48,000 synthetic trajectories of a Unitree G1 walking, grasping, and placing objects inside those virtual replicas. They rendered the robot's first-person camera view from each run and paired it with the matching language instruction and motion data. That's the dataset every humanoid team needs and nobody has: synced egocentric video + language + kinematics, at scale. Instead of collecting it in the real world, they manufactured it. They trained a vision-language-kinematics policy on that synthetic data alone, then deployed it on the physical G1 across five task types: navigation to a named object, lifting boxes of three different sizes with no per-size tuning, chained multi-step tasks, robustness to mid-task layout changes and flickering lights, and multi-minute long-horizon runs. No real-world fine-tuning at any point. Real-world interaction data has been the hard limit on humanoid learning... slow, expensive, and small. If scanning a room once and synthesizing thousands of labeled interactions holds up as a general recipe, that limit moves. Data stops being the bottleneck robotics teams have to solve for. 📌 Paper: Project: ——- Weekly robotics and AI insights. Subscribe free:

Trained on zero real-world data. Learned to walk, pick up boxes, and follow multi-step instructions... in the REAL world. ( 📌 Paper below) Researchers from Amazon FAR, Berkeley, Stanford, and CMU scanned real rooms with an iPhone, rebuilt them as 3D Gaussian Splatting scenes, then generated 48,000 synthetic trajectories of a Unitree G1 walking, grasping, and placing objects inside those virtual replicas. They rendered the robot's first-person camera view from each run and paired it with the matching language instruction and motion data. That's the dataset every humanoid team needs and nobody has: synced egocentric video + language + kinematics, at scale. Instead of collecting it in the real world, they manufactured it. They trained a vision-language-kinematics policy on that synthetic data alone, then deployed it on the physical G1 across five task types: navigation to a named object, lifting boxes of three different sizes with no per-size tuning, chained multi-step tasks, robustness to mid-task layout changes and flickering lights, and multi-minute long-horizon runs. No real-world fine-tuning at any point. Real-world interaction data has been the hard limit on humanoid learning... slow, expensive, and small. If scanning a room once and synthesizing thousands of labeled interactions holds up as a general recipe, that limit moves. Data stops being the bottleneck robotics teams have to solve for. 📌 Paper: Project: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

12,950 views • 1 month ago

Many drugs work by binding to a specific target in the body and blocking or changing what it does. An important first step in the drug development process is designing a molecule that can bind tightly to its target. Traditionally, that's meant weeks or months of expert work per target, sifting through a large number of candidates to identify the few that work. We wanted to test if Claude could successfully design novel protein binders from scratch (also called de novo design). With a protein design prompt written by a human expert, Claude autonomously designed protein binders against 14 out of 15 targets. We then worked with Adaptyv Bio and Twist Bioscience, who independently built and tested the proteins Claude designed.

Many drugs work by binding to a specific target in the body and blocking or changing what it does. An important first step in the drug development process is designing a molecule that can bind tightly to its target. Traditionally, that's meant weeks or months of expert work per target, sifting through a large number of candidates to identify the few that work. We wanted to test if Claude could successfully design novel protein binders from scratch (also called de novo design). With a protein design prompt written by a human expert, Claude autonomously designed protein binders against 14 out of 15 targets. We then worked with Adaptyv Bio and Twist Bioscience, who independently built and tested the proteins Claude designed.

Anthropic

4,153,607 views • 7 days ago

🔥How to execute a Steeler Stunt Step 1: Dip and Step To get to your assigned gap it’s critical to avoid contact on your first step. To accomplish this step lateral and dip your shoulder to reduce the surface area your opponent can use to seal you from your gap. Step 2: Read the adjacent blocker Transfer your eyes from the primary blocker to the adjacent blocker so you can react appropriately to his block. On this play, since the adjacent blocker pulls the proper reaction is to get vertical. Step 3: 2nd Step Vertical In order to disrupt the pin and pull you need to get vertical. This will put you in the path of the RB and in prime position for a TFL.

🔥How to execute a Steeler Stunt Step 1: Dip and Step To get to your assigned gap it’s critical to avoid contact on your first step. To accomplish this step lateral and dip your shoulder to reduce the surface area your opponent can use to seal you from your gap. Step 2: Read the adjacent blocker Transfer your eyes from the primary blocker to the adjacent blocker so you can react appropriately to his block. On this play, since the adjacent blocker pulls the proper reaction is to get vertical. Step 3: 2nd Step Vertical In order to disrupt the pin and pull you need to get vertical. This will put you in the path of the RB and in prime position for a TFL.

Craig Roh

393,793 views • 3 years ago

At Uber, design source of truth was a big problem for design and engineering teams. No one knew what the app truly looked like. So every week, a group of designers would sit in a meeting room and check that engineers had correctly implemented the Figma designs. Now coding agents have increased the throughput of changes by an order of magnitude. Keeping up manually has become impossible. Here we used the Revyl to map every state in Ubert (demo uber), navigating our mobile use agent on a cloud iOS simulator. This would have taken a team of designers tens of hours to recreate by hand. We did it in under an hour, asynchronously, with a simple prompt. Let your team see what your users actually see. Empower coding agents to ship a delightful experience with no blindspots. Get started with our free trial and create a map for your own app.

At Uber, design source of truth was a big problem for design and engineering teams. No one knew what the app truly looked like. So every week, a group of designers would sit in a meeting room and check that engineers had correctly implemented the Figma designs. Now coding agents have increased the throughput of changes by an order of magnitude. Keeping up manually has become impossible. Here we used the Revyl to map every state in Ubert (demo uber), navigating our mobile use agent on a cloud iOS simulator. This would have taken a team of designers tens of hours to recreate by hand. We did it in under an hour, asynchronously, with a simple prompt. Let your team see what your users actually see. Empower coding agents to ship a delightful experience with no blindspots. Get started with our free trial and create a map for your own app.

Anam Hira

203,582 views • 2 months ago

✨Introducing YesCheff is a web app that helps you learn, cook, and enjoy your favorite recipes through step-by-step cooking guides built from real YouTube content. You can even use your own YouTube links. A small but special project where I brought all my capabilities together, from design (brand & product) to animation, and development. This project came from exploring how Al models parse data into frontend components and the development of generative interfaces. I designed and developed the structure through React components (HTML, CSS, JS) for the frontend and Node.js for the backend. This is a testament of how one could develop digital products in a less frustrating and more flexible way with the use of Al, particularly when it comes to getting the styling and functionality right. Now which foods are you eager to learn? Check it out at #YesCheff #WebApp #AI #Design #FrontendDevelopment #ReactJS #NodeJS #ProductDesign

JEHTOMS 5.0

85,213 views • 10 months ago

1/ 🔥🔥 Big news: We’re launching Percival, the first AI agent that can evaluate and fix other AI agents! 🤖 Percival is an evaluation agent that doesn’t just detect failures in agent traces — it can fix them. Percival outperformed SOTA LLMs by 2.9x on the TRAIL dataset, containing human annotated errors from GAIA and SWE-Bench. 🦾 Here’s what Percival can do for you: - Automatically suggest prompt fixes for your agent - Catch 20+ types of agent failures spanning tool use, planning and coordination, domain specific errors - Reduce manual debugging time from hours to < 1 minute

1/ 🔥🔥 Big news: We’re launching Percival, the first AI agent that can evaluate and fix other AI agents! 🤖 Percival is an evaluation agent that doesn’t just detect failures in agent traces — it can fix them. Percival outperformed SOTA LLMs by 2.9x on the TRAIL dataset, containing human annotated errors from GAIA and SWE-Bench. 🦾 Here’s what Percival can do for you: - Automatically suggest prompt fixes for your agent - Catch 20+ types of agent failures spanning tool use, planning and coordination, domain specific errors - Reduce manual debugging time from hours to < 1 minute

PatronusAI

24,054 views • 1 year ago

At Uber a big problem for design teams and engineering teams was design source of truth. No one knew what the app truly looked like, so weekly a bunch of designers would get into a meeting room and check that the engineers correctly implemented the figma designs that they made Now with coding agents throughput of changes has increased an order of magnitude and it has become impossible to manually keep up. Here we used the Revyl CLI to create a flow of every state in the Uber design, by navigating our mobile use agent on an cloud iOS simulator. This is something that would have taken a team of designers tens of hours to recreate manually; All done in less than an hour asynchronously with a simple prompt. Enable your team to know what your users are actually seeing and empower coding agents to give your users a delightful experience without any blindspots Get started with our new free trial and create a map for your own app 🗺️

At Uber a big problem for design teams and engineering teams was design source of truth. No one knew what the app truly looked like, so weekly a bunch of designers would get into a meeting room and check that the engineers correctly implemented the figma designs that they made Now with coding agents throughput of changes has increased an order of magnitude and it has become impossible to manually keep up. Here we used the Revyl CLI to create a flow of every state in the Uber design, by navigating our mobile use agent on an cloud iOS simulator. This is something that would have taken a team of designers tens of hours to recreate manually; All done in less than an hour asynchronously with a simple prompt. Enable your team to know what your users are actually seeing and empower coding agents to give your users a delightful experience without any blindspots Get started with our new free trial and create a map for your own app 🗺️

Anam Hira

96,508 views • 4 months ago

Competition 2: Mesh Generation 🦾 Until now, our subnet has focused on Gaussian splats. That work led to integrations with Unity, building the world’s largest 3D dataset, and early application deployments with real users and organic traffic growth. 🕹️ The next step is clear. Developers want native mesh generation that drops straight into existing game pipelines and unlocks real production use today. 🤖 Our competition format lets us move flexibly between goals, optimizing for near term commercial utility with meshes and balancing longer term research with Gaussian splat world models and digital twins. 📈 With significant open source mesh models releasing in the past weeks, the timing is right and the bar for miner innovation has never been higher. Mesh competition launches in 2 weeks!

Competition 2: Mesh Generation 🦾 Until now, our subnet has focused on Gaussian splats. That work led to integrations with Unity, building the world’s largest 3D dataset, and early application deployments with real users and organic traffic growth. 🕹️ The next step is clear. Developers want native mesh generation that drops straight into existing game pipelines and unlocks real production use today. 🤖 Our competition format lets us move flexibly between goals, optimizing for near term commercial utility with meshes and balancing longer term research with Gaussian splat world models and digital twins. 📈 With significant open source mesh models releasing in the past weeks, the timing is right and the bar for miner innovation has never been higher. Mesh competition launches in 2 weeks!

404

33,090 views • 7 months ago

Most agentic workflows die a slow death from cost creep before they ever finish the job. SenseTime's new Token Plan flips that: complex multi-step tasks now run at ~60% lower token cost, and for the first month you get 1,500 calls refreshing every 5 hours — free. That's enough room to actually stress-test long-chain workflows instead of babysitting your token meter. Try it →

Most agentic workflows die a slow death from cost creep before they ever finish the job. SenseTime's new Token Plan flips that: complex multi-step tasks now run at ~60% lower token cost, and for the first month you get 1,500 calls refreshing every 5 hours — free. That's enough room to actually stress-test long-chain workflows instead of babysitting your token meter. Try it →

Farhan Azad Shuvra

21,461 views • 1 month ago