Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Today, we're open sourcing the first computer-use dataset for design. We captured 200+ hours and 3400+ trajectories of real design work in Figma, recording step by step how designers execute complex, long-horizon tasks. FigmaTrace dataset, paper, and blog below :) PatronusAI

Anand Kannappan

1,753 subscribers

137,566 Aufrufe • vor 1 Monat •via X (Twitter)

Wissenschaft & Technologie Bildung Gesundheit & Wellness

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Today we're launching the most advanced computer-use dataset in the world. 1,000+ hours of screen recordings along with mouse/keyboard inputs + annotations. Sourced from experts across coding, design, browser-use, research and more. Link in the comments :) Markov

Today we're launching the most advanced computer-use dataset in the world. 1,000+ hours of screen recordings along with mouse/keyboard inputs + annotations. Sourced from experts across coding, design, browser-use, research and more. Link in the comments :) Markov

Dev

47,966 Aufrufe • vor 6 Monaten

Today, we're launching the world's largest open-source dataset of computer-use recordings. 10,000+ hours across Salesforce, Blender, Photoshop and more, to automate the next level of white-collar work. Link in the comments :) @markov__ai

Today, we're launching the world's largest open-source dataset of computer-use recordings. 10,000+ hours across Salesforce, Blender, Photoshop and more, to automate the next level of white-collar work. Link in the comments :) @markov__ai

Dev

475,305 Aufrufe • vor 7 Monaten

Today, we're launching the largest open-source dataset of gaming data for computer-use. 500+ hours of gameplay screen recordings + keystrokes/mouse movements, across Valorant, Minecraft, GTA and more. Link in the comments :) @markov__ai

Today, we're launching the largest open-source dataset of gaming data for computer-use. 500+ hours of gameplay screen recordings + keystrokes/mouse movements, across Valorant, Minecraft, GTA and more. Link in the comments :) @markov__ai

Dev

109,477 Aufrufe • vor 3 Monaten

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.

Spatial AI ( is building large real-world datasets to teach robots how to navigate the world and complete tasks. Their first open source dataset, SEA (Spatial Everyday Activities), is the largest curated egocentric dataset of people carrying out real tasks, with 10,000 hours of data.

Y Combinator

20,944 Aufrufe • vor 10 Monaten

Today, we're launching the world's largest open-source CAD dataset for computer-use. 1,000+ hours across SolidWorks, Siemens-NX, AutoCAD & more. Screen recordings, mouse/keyboard actions, input diagrams, output files, rubrics and narrations. Link in the comments :)

Today, we're launching the world's largest open-source CAD dataset for computer-use. 1,000+ hours across SolidWorks, Siemens-NX, AutoCAD & more. Screen recordings, mouse/keyboard actions, input diagrams, output files, rubrics and narrations. Link in the comments :)

Dev

158,270 Aufrufe • vor 1 Monat

We're going back to basics today. 🤓 Designing your first button is a rite of passage and a great way to get hands-on practice when starting your design journey. Here's a step-by-step guide to get you started.

We're going back to basics today. 🤓 Designing your first button is a rite of passage and a great way to get hands-on practice when starting your design journey. Here's a step-by-step guide to get you started.

Figma

77,068 Aufrufe • vor 3 Jahren

What is the best video editing agent for short form social? Does it actually work? We watched professional video editors, step by step, as they built short-form social reels in Adobe Premiere Pro. Today we're open-sourcing this preview dataset on Hugging Face, to make AI agents better at editing videos. The data set is 234 annotated steps across 4 computer-use trajectories. Editors narrated their reasoning aloud as they worked, so every step pairs a screenshot with the expert's own thought, a structured action, and executable grounding: >a Premiere MCP tool call, keyboard shortcut, menu path, or coordinate click. >The format follows the AgentNet trajectory schema, extended with a Premiere action taxonomy and multi-path execution. ***That makes it directly usable for computer-use agent SFT, reasoning mid-training, tool-use and function calling, and benchmarking agents against a human expert baseline. Enjoy!

What is the best video editing agent for short form social? Does it actually work? We watched professional video editors, step by step, as they built short-form social reels in Adobe Premiere Pro. Today we're open-sourcing this preview dataset on Hugging Face, to make AI agents better at editing videos. The data set is 234 annotated steps across 4 computer-use trajectories. Editors narrated their reasoning aloud as they worked, so every step pairs a screenshot with the expert's own thought, a structured action, and executable grounding: >a Premiere MCP tool call, keyboard shortcut, menu path, or coordinate click. >The format follows the AgentNet trajectory schema, extended with a Premiere action taxonomy and multi-path execution. ***That makes it directly usable for computer-use agent SFT, reasoning mid-training, tool-use and function calling, and benchmarking agents against a human expert baseline. Enjoy!

ben

40,192 Aufrufe • vor 2 Monaten

How can we address the scarcity of data required for specialized AI? Learn about Simula, a framework that reframes synthetic data generation as dataset-level mechanism design. By using reasoning to architect datasets from first principles, Simula enables fine-grained control over coverage, complexity, and quality. More →

How can we address the scarcity of data required for specialized AI? Learn about Simula, a framework that reframes synthetic data generation as dataset-level mechanism design. By using reasoning to architect datasets from first principles, Simula enables fine-grained control over coverage, complexity, and quality. More →

Google Research

142,710 Aufrufe • vor 5 Monaten

We are proud to announce our first contribution to open source Computer Vision research! 🚀 Today, we’re publishing on Hugging Face a 1,000 3D Maps Dataset under CC-BY-NC license, designed to accelerate progress in 3D perception and spatial AI. Our way to give back to the global AI community and show how DePIN can make an impact. 🧵 👉

We are proud to announce our first contribution to open source Computer Vision research! 🚀 Today, we’re publishing on Hugging Face a 1,000 3D Maps Dataset under CC-BY-NC license, designed to accelerate progress in 3D perception and spatial AI. Our way to give back to the global AI community and show how DePIN can make an impact. 🧵 👉

Over the Reality 🌐

88,633 Aufrufe • vor 10 Monaten

Biologists use specialized open-source models for tasks like modeling the structure of molecular systems, designing drug-like molecules, and predicting the effects of genetic mutations. But these models are often expensive to run, potentially limiting their impact. In our latest Science Blog, we share how Claude was able to optimize inference for more than 30 open-source models, making them 4x faster on average, partly by writing custom software for GPUs. We’re open sourcing all of the optimization code. Read more:

Biologists use specialized open-source models for tasks like modeling the structure of molecular systems, designing drug-like molecules, and predicting the effects of genetic mutations. But these models are often expensive to run, potentially limiting their impact. In our latest Science Blog, we share how Claude was able to optimize inference for more than 30 open-source models, making them 4x faster on average, partly by writing custom software for GPUs. We’re open sourcing all of the optimization code. Read more:

Anthropic

626,737 Aufrufe • vor 23 Tagen

This is a call for creatives with a mission to serve. I’m looking for the best designers, artists, and creatives in the world to take on hard and real problems. Come use your superpowers to craft intuitive, complex products and stories. Design isn’t decoration, it’s direction. At Anduril Industries, design is embedded as part of our core strategy. Our team crafts every touch point—from our products to our website, our videos and marketing campaigns, concept and mission communication, even the architecture and the spaces we work in. #Hiring all Summer: Industrial, prototyping, CMF, photography, film, motion, interaction, digital, environmental, merchandise, OOH, graphics, identity systems.

This is a call for creatives with a mission to serve. I’m looking for the best designers, artists, and creatives in the world to take on hard and real problems. Come use your superpowers to craft intuitive, complex products and stories. Design isn’t decoration, it’s direction. At Anduril Industries, design is embedded as part of our core strategy. Our team crafts every touch point—from our products to our website, our videos and marketing campaigns, concept and mission communication, even the architecture and the spaces we work in. #Hiring all Summer: Industrial, prototyping, CMF, photography, film, motion, interaction, digital, environmental, merchandise, OOH, graphics, identity systems.

jen bucci

44,594 Aufrufe • vor 1 Jahr

As promised! New blog with Josh Newton combining our talks from AI Engineer 🔜 NYC 🗽's first design track. In addition, we wanted to actually show a demonstration of how you can iterate and improve upon that first design output from AI. We had a ton of fun building the interactions for this post (especially the horse animator hahaha). Would love to hear your thoughts!

As promised! New blog with Josh Newton combining our talks from AI Engineer 🔜 NYC 🗽's first design track. In addition, we wanted to actually show a demonstration of how you can iterate and improve upon that first design output from AI. We had a ton of fun building the interactions for this post (especially the horse animator hahaha). Would love to hear your thoughts!

Sabina

13,621 Aufrufe • vor 2 Monaten

Trained on zero real-world data. Learned to walk, pick up boxes, and follow multi-step instructions... in the REAL world. ( 📌 Paper below) Researchers from Amazon FAR, Berkeley, Stanford, and CMU scanned real rooms with an iPhone, rebuilt them as 3D Gaussian Splatting scenes, then generated 48,000 synthetic trajectories of a Unitree G1 walking, grasping, and placing objects inside those virtual replicas. They rendered the robot's first-person camera view from each run and paired it with the matching language instruction and motion data. That's the dataset every humanoid team needs and nobody has: synced egocentric video + language + kinematics, at scale. Instead of collecting it in the real world, they manufactured it. They trained a vision-language-kinematics policy on that synthetic data alone, then deployed it on the physical G1 across five task types: navigation to a named object, lifting boxes of three different sizes with no per-size tuning, chained multi-step tasks, robustness to mid-task layout changes and flickering lights, and multi-minute long-horizon runs. No real-world fine-tuning at any point. Real-world interaction data has been the hard limit on humanoid learning... slow, expensive, and small. If scanning a room once and synthesizing thousands of labeled interactions holds up as a general recipe, that limit moves. Data stops being the bottleneck robotics teams have to solve for. 📌 Paper: Project: ——- Weekly robotics and AI insights. Subscribe free:

Trained on zero real-world data. Learned to walk, pick up boxes, and follow multi-step instructions... in the REAL world. ( 📌 Paper below) Researchers from Amazon FAR, Berkeley, Stanford, and CMU scanned real rooms with an iPhone, rebuilt them as 3D Gaussian Splatting scenes, then generated 48,000 synthetic trajectories of a Unitree G1 walking, grasping, and placing objects inside those virtual replicas. They rendered the robot's first-person camera view from each run and paired it with the matching language instruction and motion data. That's the dataset every humanoid team needs and nobody has: synced egocentric video + language + kinematics, at scale. Instead of collecting it in the real world, they manufactured it. They trained a vision-language-kinematics policy on that synthetic data alone, then deployed it on the physical G1 across five task types: navigation to a named object, lifting boxes of three different sizes with no per-size tuning, chained multi-step tasks, robustness to mid-task layout changes and flickering lights, and multi-minute long-horizon runs. No real-world fine-tuning at any point. Real-world interaction data has been the hard limit on humanoid learning... slow, expensive, and small. If scanning a room once and synthesizing thousands of labeled interactions holds up as a general recipe, that limit moves. Data stops being the bottleneck robotics teams have to solve for. 📌 Paper: Project: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

12,950 Aufrufe • vor 2 Monaten

was one of the first projects i pushed for at Ramp. i saw the work the team was doing and knew we needed a home for it! learn more about our process in my blog post below~ ft. work by bao to ᝰ.ᐟ, Rikin, and nicholas ano 🩷

was one of the first projects i pushed for at Ramp. i saw the work the team was doing and knew we needed a home for it! learn more about our process in my blog post below~ ft. work by bao to ᝰ.ᐟ, Rikin, and nicholas ano 🩷

elizabeth lin 🦄✨

23,010 Aufrufe • vor 29 Tagen

Many drugs work by binding to a specific target in the body and blocking or changing what it does. An important first step in the drug development process is designing a molecule that can bind tightly to its target. Traditionally, that's meant weeks or months of expert work per target, sifting through a large number of candidates to identify the few that work. We wanted to test if Claude could successfully design novel protein binders from scratch (also called de novo design). With a protein design prompt written by a human expert, Claude autonomously designed protein binders against 14 out of 15 targets. We then worked with Adaptyv Bio and Twist Bioscience, who independently built and tested the proteins Claude designed.

Many drugs work by binding to a specific target in the body and blocking or changing what it does. An important first step in the drug development process is designing a molecule that can bind tightly to its target. Traditionally, that's meant weeks or months of expert work per target, sifting through a large number of candidates to identify the few that work. We wanted to test if Claude could successfully design novel protein binders from scratch (also called de novo design). With a protein design prompt written by a human expert, Claude autonomously designed protein binders against 14 out of 15 targets. We then worked with Adaptyv Bio and Twist Bioscience, who independently built and tested the proteins Claude designed.

Anthropic

4,337,960 Aufrufe • vor 1 Monat

🔥How to execute a Steeler Stunt Step 1: Dip and Step To get to your assigned gap it’s critical to avoid contact on your first step. To accomplish this step lateral and dip your shoulder to reduce the surface area your opponent can use to seal you from your gap. Step 2: Read the adjacent blocker Transfer your eyes from the primary blocker to the adjacent blocker so you can react appropriately to his block. On this play, since the adjacent blocker pulls the proper reaction is to get vertical. Step 3: 2nd Step Vertical In order to disrupt the pin and pull you need to get vertical. This will put you in the path of the RB and in prime position for a TFL.

🔥How to execute a Steeler Stunt Step 1: Dip and Step To get to your assigned gap it’s critical to avoid contact on your first step. To accomplish this step lateral and dip your shoulder to reduce the surface area your opponent can use to seal you from your gap. Step 2: Read the adjacent blocker Transfer your eyes from the primary blocker to the adjacent blocker so you can react appropriately to his block. On this play, since the adjacent blocker pulls the proper reaction is to get vertical. Step 3: 2nd Step Vertical In order to disrupt the pin and pull you need to get vertical. This will put you in the path of the RB and in prime position for a TFL.

Craig Roh

393,793 Aufrufe • vor 3 Jahren

At Uber, design source of truth was a big problem for design and engineering teams. No one knew what the app truly looked like. So every week, a group of designers would sit in a meeting room and check that engineers had correctly implemented the Figma designs. Now coding agents have increased the throughput of changes by an order of magnitude. Keeping up manually has become impossible. Here we used the Revyl to map every state in Ubert (demo uber), navigating our mobile use agent on a cloud iOS simulator. This would have taken a team of designers tens of hours to recreate by hand. We did it in under an hour, asynchronously, with a simple prompt. Let your team see what your users actually see. Empower coding agents to ship a delightful experience with no blindspots. Get started with our free trial and create a map for your own app.

At Uber, design source of truth was a big problem for design and engineering teams. No one knew what the app truly looked like. So every week, a group of designers would sit in a meeting room and check that engineers had correctly implemented the Figma designs. Now coding agents have increased the throughput of changes by an order of magnitude. Keeping up manually has become impossible. Here we used the Revyl to map every state in Ubert (demo uber), navigating our mobile use agent on a cloud iOS simulator. This would have taken a team of designers tens of hours to recreate by hand. We did it in under an hour, asynchronously, with a simple prompt. Let your team see what your users actually see. Empower coding agents to ship a delightful experience with no blindspots. Get started with our free trial and create a map for your own app.

Anam Hira

204,517 Aufrufe • vor 3 Monaten

✨Introducing YesCheff is a web app that helps you learn, cook, and enjoy your favorite recipes through step-by-step cooking guides built from real YouTube content. You can even use your own YouTube links. A small but special project where I brought all my capabilities together, from design (brand & product) to animation, and development. This project came from exploring how Al models parse data into frontend components and the development of generative interfaces. I designed and developed the structure through React components (HTML, CSS, JS) for the frontend and Node.js for the backend. This is a testament of how one could develop digital products in a less frustrating and more flexible way with the use of Al, particularly when it comes to getting the styling and functionality right. Now which foods are you eager to learn? Check it out at #YesCheff #WebApp #AI #Design #FrontendDevelopment #ReactJS #NodeJS #ProductDesign

JEHTOMS 5.0

85,314 Aufrufe • vor 1 Jahr

1/ 🔥🔥 Big news: We’re launching Percival, the first AI agent that can evaluate and fix other AI agents! 🤖 Percival is an evaluation agent that doesn’t just detect failures in agent traces — it can fix them. Percival outperformed SOTA LLMs by 2.9x on the TRAIL dataset, containing human annotated errors from GAIA and SWE-Bench. 🦾 Here’s what Percival can do for you: - Automatically suggest prompt fixes for your agent - Catch 20+ types of agent failures spanning tool use, planning and coordination, domain specific errors - Reduce manual debugging time from hours to < 1 minute

1/ 🔥🔥 Big news: We’re launching Percival, the first AI agent that can evaluate and fix other AI agents! 🤖 Percival is an evaluation agent that doesn’t just detect failures in agent traces — it can fix them. Percival outperformed SOTA LLMs by 2.9x on the TRAIL dataset, containing human annotated errors from GAIA and SWE-Bench. 🦾 Here’s what Percival can do for you: - Automatically suggest prompt fixes for your agent - Catch 20+ types of agent failures spanning tool use, planning and coordination, domain specific errors - Reduce manual debugging time from hours to < 1 minute

PatronusAI

24,080 Aufrufe • vor 1 Jahr

At Uber a big problem for design teams and engineering teams was design source of truth. No one knew what the app truly looked like, so weekly a bunch of designers would get into a meeting room and check that the engineers correctly implemented the figma designs that they made Now with coding agents throughput of changes has increased an order of magnitude and it has become impossible to manually keep up. Here we used the Revyl CLI to create a flow of every state in the Uber design, by navigating our mobile use agent on an cloud iOS simulator. This is something that would have taken a team of designers tens of hours to recreate manually; All done in less than an hour asynchronously with a simple prompt. Enable your team to know what your users are actually seeing and empower coding agents to give your users a delightful experience without any blindspots Get started with our new free trial and create a map for your own app 🗺️

At Uber a big problem for design teams and engineering teams was design source of truth. No one knew what the app truly looked like, so weekly a bunch of designers would get into a meeting room and check that the engineers correctly implemented the figma designs that they made Now with coding agents throughput of changes has increased an order of magnitude and it has become impossible to manually keep up. Here we used the Revyl CLI to create a flow of every state in the Uber design, by navigating our mobile use agent on an cloud iOS simulator. This is something that would have taken a team of designers tens of hours to recreate manually; All done in less than an hour asynchronously with a simple prompt. Enable your team to know what your users are actually seeing and empower coding agents to give your users a delightful experience without any blindspots Get started with our new free trial and create a map for your own app 🗺️

Anam Hira

97,420 Aufrufe • vor 5 Monaten