Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Today, we're launching the world's largest open-source CAD dataset for computer-use. 1,000+ hours across SolidWorks, Siemens-NX, AutoCAD & more. Screen recordings, mouse/keyboard actions, input diagrams, output files, rubrics and narrations. Link in the comments :)

DevvMandal's profile picture

Dev

6,930 subscribers

158,270 Aufrufe • vor 1 Monat •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Today, we’re launching computer-use CAD environments to train AI models for engineering. Task prompts, reference files, gold outputs, rubrics & human walkthrough screen-recordings. We're open-sourcing 50 tasks across AutoCAD, Solidworks & Siemens-NX. Link below @markov__ai
0:15

Today, we’re launching computer-use CAD environments to train AI models for engineering. Task prompts, reference files, gold outputs, rubrics & human walkthrough screen-recordings. We're open-sourcing 50 tasks across AutoCAD, Solidworks & Siemens-NX. Link below @markov__ai

Dev

13,316 Aufrufe • vor 2 Monaten

Today, we're launching the largest open-source dataset of gaming data for computer-use. 500+ hours of gameplay screen recordings + keystrokes/mouse movements, across Valorant, Minecraft, GTA and more. Link in the comments :) @markov__ai
0:14

Today, we're launching the largest open-source dataset of gaming data for computer-use. 500+ hours of gameplay screen recordings + keystrokes/mouse movements, across Valorant, Minecraft, GTA and more. Link in the comments :) @markov__ai

Dev

109,477 Aufrufe • vor 3 Monaten

Today we're launching the most advanced computer-use dataset in the world. 1,000+ hours of screen recordings along with mouse/keyboard inputs + annotations. Sourced from experts across coding, design, browser-use, research and more. Link in the comments :) Markov
0:19

Today we're launching the most advanced computer-use dataset in the world. 1,000+ hours of screen recordings along with mouse/keyboard inputs + annotations. Sourced from experts across coding, design, browser-use, research and more. Link in the comments :) Markov

Dev

47,966 Aufrufe • vor 6 Monaten

Today, our computer-use datasets crossed 50,000+ hours. Screen recordings + synced mouse/keyboard actions + narrations across CAD, design, browser-use & more. Sourced from humans and available off the shelf today :) Samples below.
0:15

Today, our computer-use datasets crossed 50,000+ hours. Screen recordings + synced mouse/keyboard actions + narrations across CAD, design, browser-use & more. Sourced from humans and available off the shelf today :) Samples below.

Dev

83,300 Aufrufe • vor 1 Monat

Today, we're open sourcing the first computer-use dataset for design. We captured 200+ hours and 3400+ trajectories of real design work in Figma, recording step by step how designers execute complex, long-horizon tasks. FigmaTrace dataset, paper, and blog below :) PatronusAI
0:19

Today, we're open sourcing the first computer-use dataset for design. We captured 200+ hours and 3400+ trajectories of real design work in Figma, recording step by step how designers execute complex, long-horizon tasks. FigmaTrace dataset, paper, and blog below :) PatronusAI

Anand Kannappan

137,566 Aufrufe • vor 1 Monat

🚨 BREAKING: you can now turn your laptop into a CIA command center World Monitor is the ultimate real-time dashboard for global events—conflicts, earthquakes, traffic infra, flights and more. Updates live on your screen. • Tracks 100+ data sources instantly • Customisable for every screen size • 100% Open-source & free to use link in comment
0:14

🚨 BREAKING: you can now turn your laptop into a CIA command center World Monitor is the ultimate real-time dashboard for global events—conflicts, earthquakes, traffic infra, flights and more. Updates live on your screen. • Tracks 100+ data sources instantly • Customisable for every screen size • 100% Open-source & free to use link in comment

Farhan

173,584 Aufrufe • vor 7 Monaten

What is the best video editing agent for short form social? Does it actually work? We watched professional video editors, step by step, as they built short-form social reels in Adobe Premiere Pro. Today we're open-sourcing this preview dataset on Hugging Face, to make AI agents better at editing videos. The data set is 234 annotated steps across 4 computer-use trajectories. Editors narrated their reasoning aloud as they worked, so every step pairs a screenshot with the expert's own thought, a structured action, and executable grounding: >a Premiere MCP tool call, keyboard shortcut, menu path, or coordinate click. >The format follows the AgentNet trajectory schema, extended with a Premiere action taxonomy and multi-path execution. ***That makes it directly usable for computer-use agent SFT, reasoning mid-training, tool-use and function calling, and benchmarking agents against a human expert baseline. Enjoy!
0:13

What is the best video editing agent for short form social? Does it actually work? We watched professional video editors, step by step, as they built short-form social reels in Adobe Premiere Pro. Today we're open-sourcing this preview dataset on Hugging Face, to make AI agents better at editing videos. The data set is 234 annotated steps across 4 computer-use trajectories. Editors narrated their reasoning aloud as they worked, so every step pairs a screenshot with the expert's own thought, a structured action, and executable grounding: >a Premiere MCP tool call, keyboard shortcut, menu path, or coordinate click. >The format follows the AgentNet trajectory schema, extended with a Premiere action taxonomy and multi-path execution. ***That makes it directly usable for computer-use agent SFT, reasoning mid-training, tool-use and function calling, and benchmarking agents against a human expert baseline. Enjoy!

ben

40,192 Aufrufe • vor 2 Monaten

Helmor has been out as an open-source coding agent orchestrator for less than a week, and we’re already close to 1,000 GitHub stars!!! As a little gift, we shipped a new feature you’re going to love. 👇 Stop copy-pasting GitHub links, Linear tickets, Slack threads, and random notes into prompts. We're tired of rebuilding context every time I ask an agent to do work. Contexts in Helmor is another step toward a local dev loop: browse, preview, inject context, and dispatch tasks without leaving the app. Before you start a task, Helmor should help you gather the right context first. Try the open-source Helmor — link in the comments. #Helmor
0:10

Helmor has been out as an open-source coding agent orchestrator for less than a week, and we’re already close to 1,000 GitHub stars!!! As a little gift, we shipped a new feature you’re going to love. 👇 Stop copy-pasting GitHub links, Linear tickets, Slack threads, and random notes into prompts. We're tired of rebuilding context every time I ask an agent to do work. Contexts in Helmor is another step toward a local dev loop: browse, preview, inject context, and dispatch tasks without leaving the app. Before you start a task, Helmor should help you gather the right context first. Try the open-source Helmor — link in the comments. #Helmor

Caspian 東澔

10,823 Aufrufe • vor 5 Monaten

Introducing fx, a tiny, open, native coding agent from Vercel Labs. Originally an internal tool, fx is a harness and CLI written in Zig, optimized for research and embedding in larger systems. Today, we're open sourcing it. fx is built on three principles: 1. Fast. A single native binary, no runtime to install. It cold starts in 10µs and does no unnecessary work or I/O before accepting input. fx is the answer to "how fast can a coding agent be?" 2. Light. The 6.3MiB binary uses single-digit megabytes of memory at baseline, made for instant installation and embedding in resource-constrained environments and agent sandboxes. 3. Open. Apache-2.0, model and provider agnostic, suitable for local and cloud inference. Its small core extends through skills, plugins, and MCP. Minimalism is an obsession throughout the entire harness: system prompt, tools, features, binary. The goal was to keep context usage and time to first token low, and make fx optimal for model benchmarking, sandboxing, evals, and gyms. You can use fx directly or embed it as infrastructure. The CLI feels more like a Unix shell than an IDE in the terminal: it preserves scroll history, produces minimal output, and uses complex TUI rendering very, very sparingly. Programmatically, 𝚏𝚡 𝚊𝚜𝚔 --𝚓𝚜𝚘𝚗 gives structured output, 𝚏𝚡 𝚊𝚌𝚙 connects to editors and other clients, and WebAssembly can even run the whole thing inside the browser (see: Privacy is a design constraint: no product telemetry, sessions and usage stay local, and no source code or prompts are shared with any endpoint other than inference. With local inference and auto-updates off, fx is fully hermetic. fx is experimental. Use at your own risk and expect frequent changes. Chat with us on X ( or file issues ( 𝚌𝚞𝚛𝚕 -𝚏𝚜𝚂𝙻 𝚏𝚡.𝚜𝚑/𝚜𝚎𝚝𝚞𝚙.𝚜𝚑 | 𝚋𝚊𝚜𝚑
0:11

Introducing fx, a tiny, open, native coding agent from Vercel Labs. Originally an internal tool, fx is a harness and CLI written in Zig, optimized for research and embedding in larger systems. Today, we're open sourcing it. fx is built on three principles: 1. Fast. A single native binary, no runtime to install. It cold starts in 10µs and does no unnecessary work or I/O before accepting input. fx is the answer to "how fast can a coding agent be?" 2. Light. The 6.3MiB binary uses single-digit megabytes of memory at baseline, made for instant installation and embedding in resource-constrained environments and agent sandboxes. 3. Open. Apache-2.0, model and provider agnostic, suitable for local and cloud inference. Its small core extends through skills, plugins, and MCP. Minimalism is an obsession throughout the entire harness: system prompt, tools, features, binary. The goal was to keep context usage and time to first token low, and make fx optimal for model benchmarking, sandboxing, evals, and gyms. You can use fx directly or embed it as infrastructure. The CLI feels more like a Unix shell than an IDE in the terminal: it preserves scroll history, produces minimal output, and uses complex TUI rendering very, very sparingly. Programmatically, 𝚏𝚡 𝚊𝚜𝚔 --𝚓𝚜𝚘𝚗 gives structured output, 𝚏𝚡 𝚊𝚌𝚙 connects to editors and other clients, and WebAssembly can even run the whole thing inside the browser (see: Privacy is a design constraint: no product telemetry, sessions and usage stay local, and no source code or prompts are shared with any endpoint other than inference. With local inference and auto-updates off, fx is fully hermetic. fx is experimental. Use at your own risk and expect frequent changes. Chat with us on X ( or file issues ( 𝚌𝚞𝚛𝚕 -𝚏𝚜𝚂𝙻 𝚏𝚡.𝚜𝚑/𝚜𝚎𝚝𝚞𝚙.𝚜𝚑 | 𝚋𝚊𝚜𝚑

Vercel Developers

964,536 Aufrufe • vor 1 Monat

To replace animal testing with AI, we need MASSIVE human datasets. Today, we're thrilled to share Axiom's new data exploration tool, providing the ability to visually explore the world's largest primary human liver toxicity dataset. Built with Axiom's proprietary wetlab protocols, our dataset includes detailed liver toxicity profiles for over 100,000 distinct molecules. The key to this dataset is our ability to do high-throughput, multiplexed high-content screening with primary human liver cells. Traditionally, toxicity assays either sacrifice throughput or sacrifice biological relevance (using easy-to-grow immortalized cell lines instead of real human cells). We managed to combine throughput, physiological relevance, and multiplexing in one platform. The assays run in a high throughput format using automation, meaning thousands of compound-dose conditions can be tested in one experiment. We achieved this using pooled primary human hepatocytes, which are often fragile and expensive. By systemizing our automation and quality control processes, we were able to run over 120+ batches on the same donor pool with incredible reproducibility and consistency. We did this while integrating many readouts per well, whereas many existing toxicity assays only do a single readout. Our multiplexed approach provides far more data per experiment enabling us to measure 10-20 different toxicity phenotypes such as apoptosis, necrosis, mitochondrial fission, endoplasmic reticulum stress, stress granule formation, microtubules, and more all from a single well on a 384-well plate! The combination of scale, high content information, and data quality is exactly what is needed to train highly accurate AI models in biology. If you're interested, please explore the dataset in the comments below and let me know if you want to chat about the details!
0:19

To replace animal testing with AI, we need MASSIVE human datasets. Today, we're thrilled to share Axiom's new data exploration tool, providing the ability to visually explore the world's largest primary human liver toxicity dataset. Built with Axiom's proprietary wetlab protocols, our dataset includes detailed liver toxicity profiles for over 100,000 distinct molecules. The key to this dataset is our ability to do high-throughput, multiplexed high-content screening with primary human liver cells. Traditionally, toxicity assays either sacrifice throughput or sacrifice biological relevance (using easy-to-grow immortalized cell lines instead of real human cells). We managed to combine throughput, physiological relevance, and multiplexing in one platform. The assays run in a high throughput format using automation, meaning thousands of compound-dose conditions can be tested in one experiment. We achieved this using pooled primary human hepatocytes, which are often fragile and expensive. By systemizing our automation and quality control processes, we were able to run over 120+ batches on the same donor pool with incredible reproducibility and consistency. We did this while integrating many readouts per well, whereas many existing toxicity assays only do a single readout. Our multiplexed approach provides far more data per experiment enabling us to measure 10-20 different toxicity phenotypes such as apoptosis, necrosis, mitochondrial fission, endoplasmic reticulum stress, stress granule formation, microtubules, and more all from a single well on a 384-well plate! The combination of scale, high content information, and data quality is exactly what is needed to train highly accurate AI models in biology. If you're interested, please explore the dataset in the comments below and let me know if you want to chat about the details!

Brandon White

25,117 Aufrufe • vor 1 Jahr

Really excited about our launch of Superagent today! Powered by the latest AI models, agents have reached a breakthrough moment where they can do incredible work, with high ease of use--without requiring complicated tuning, prompting, or configuration. Superagent represents the freeform agent that can research anything (and soon your own company's context) and output an incredible, interactive webpage. It's like having your own personal NYTimes-quality data viz/web team building bespoke pages for you. This is the perfect complement to the structured system of operations for the AI era that Airtable has become, and we will be launching more integrations between the two products in the near future. Think: launch Superagent tasks from within Airtable records or Airtable Omni, or have Superagent output/edit/read data into Airtable bases! Excited for this new frontier of breakthrough agents, and applying our product and design philosophy of making powerful capabilities intuitive and accessible-what we did for app building with Airtable, we're now doing for agents with Superagents.
0:19

Really excited about our launch of Superagent today! Powered by the latest AI models, agents have reached a breakthrough moment where they can do incredible work, with high ease of use--without requiring complicated tuning, prompting, or configuration. Superagent represents the freeform agent that can research anything (and soon your own company's context) and output an incredible, interactive webpage. It's like having your own personal NYTimes-quality data viz/web team building bespoke pages for you. This is the perfect complement to the structured system of operations for the AI era that Airtable has become, and we will be launching more integrations between the two products in the near future. Think: launch Superagent tasks from within Airtable records or Airtable Omni, or have Superagent output/edit/read data into Airtable bases! Excited for this new frontier of breakthrough agents, and applying our product and design philosophy of making powerful capabilities intuitive and accessible-what we did for app building with Airtable, we're now doing for agents with Superagents.

Howie Liu

10,125 Aufrufe • vor 8 Monaten

6 years of Trade Republic. Since last year, we: - Tripled our customers assets to over 100,000,000,000 € (100 Billion Euro) - Doubled our customer base to 8 Million - Introduced the Mirror Card and current account, passing through the ECB interest, uncapped. This makes us the largest broker in Europe, strengthening our position as the leading savings platform. This year, we will accelerate international growth further. We will open local bank branches and introduce localized banking and savings products across Europe. Starting today with France 🇫🇷 — we are launching the first commission-free PEA savings plan for our over 1 Million French customers as well as a French current account with a local IBAN and 3 % p.a. interest rate, uncapped. Read more: #TradeRepublic #PEA #France TradeRepublicFR
0:11

6 years of Trade Republic. Since last year, we: - Tripled our customers assets to over 100,000,000,000 € (100 Billion Euro) - Doubled our customer base to 8 Million - Introduced the Mirror Card and current account, passing through the ECB interest, uncapped. This makes us the largest broker in Europe, strengthening our position as the leading savings platform. This year, we will accelerate international growth further. We will open local bank branches and introduce localized banking and savings products across Europe. Starting today with France 🇫🇷 — we are launching the first commission-free PEA savings plan for our over 1 Million French customers as well as a French current account with a local IBAN and 3 % p.a. interest rate, uncapped. Read more: #TradeRepublic #PEA #France TradeRepublicFR

Trade Republic

19,367 Aufrufe • vor 1 Jahr

Here's my thoughts on the Steam Controller after using it for about 50+ hours of gaming Does it replace mouse & keyboard? Yes — desktop navigation, YouTube, on-screen keyboard all work seamlessly Minor hiccups in Big Picture mode / SteamOS launchers, but daily use is solid Trackpads Way more accurate than joysticks for aiming — genuinely feels like M&K movement Pairing trackpad + gyro is the sweet spot for FPS games Joystick touchpads are cool in theory, but dead zone issues make them hard to commit to Grip Sense The real standout feature — programmed an insta-180 on left-hand release Genuinely useful once you find your use case Build & Ergonomics No button rattle, feels solid — not premium, but not cheap either Joysticks sit close together and it's chunky... but you stop noticing within 5 minutes Xbox One controller edges it on raw ergonomics Analysis Paralysis is REAL So many customizable features (gyro, trackpads, grip sense, back buttons) that you'll spend more time in settings than actually gaming Is $100 justified? Honestly? It's a luxury, not a necessity If you're a PC/Steam Machine couch gamer who will use the features — absolutely yes Casual Gamer? You could do without but its nice to have
0:14

Here's my thoughts on the Steam Controller after using it for about 50+ hours of gaming Does it replace mouse & keyboard? Yes — desktop navigation, YouTube, on-screen keyboard all work seamlessly Minor hiccups in Big Picture mode / SteamOS launchers, but daily use is solid Trackpads Way more accurate than joysticks for aiming — genuinely feels like M&K movement Pairing trackpad + gyro is the sweet spot for FPS games Joystick touchpads are cool in theory, but dead zone issues make them hard to commit to Grip Sense The real standout feature — programmed an insta-180 on left-hand release Genuinely useful once you find your use case Build & Ergonomics No button rattle, feels solid — not premium, but not cheap either Joysticks sit close together and it's chunky... but you stop noticing within 5 minutes Xbox One controller edges it on raw ergonomics Analysis Paralysis is REAL So many customizable features (gyro, trackpads, grip sense, back buttons) that you'll spend more time in settings than actually gaming Is $100 justified? Honestly? It's a luxury, not a necessity If you're a PC/Steam Machine couch gamer who will use the features — absolutely yes Casual Gamer? You could do without but its nice to have

Cyber Dopamine

15,269 Aufrufe • vor 4 Monaten

🔎 Where I Find New OSINT Tools Most people spend hours searching for a specific OSINT tool. This is why I want to share with you websites that have almost all OSINT tools. All in one place. Here are 5 worth bookmarking right now. 👇 🗂️ TOOLBOXES ✅ OSINTRack - A growing OSINT tools directory with a clean interface. Good for discovering tools by category quickly. Website link: ✅ OSINT Tools Library - A maintained directory from the OSINT Newsletter. Every tool is tagged by access level (free, limited, or paid). The most actively updated resource of its kind. Website link: ✅ Bellingcat Toolkit - Bellingcat's curated collection of open source investigation tools, organized by category. A trusted resource from one of the most respected names in OSINT. Website link: ✅ OSINT Library - A community-built collection of OSINT tools and resources, organized by investigation type. Great for finding tools you have never heard of before. Website link: ✅ Open Source Intelligence Hub - A clean, no-frills index of OSINT tools and techniques. Useful for quickly scanning what is available across different investigation categories. Website link: __________ Do you know other toolboxes? Let me know in the comments. 👇 P.S. ♻️ Repost if you found this helpful.
0:19

🔎 Where I Find New OSINT Tools Most people spend hours searching for a specific OSINT tool. This is why I want to share with you websites that have almost all OSINT tools. All in one place. Here are 5 worth bookmarking right now. 👇 🗂️ TOOLBOXES ✅ OSINTRack - A growing OSINT tools directory with a clean interface. Good for discovering tools by category quickly. Website link: ✅ OSINT Tools Library - A maintained directory from the OSINT Newsletter. Every tool is tagged by access level (free, limited, or paid). The most actively updated resource of its kind. Website link: ✅ Bellingcat Toolkit - Bellingcat's curated collection of open source investigation tools, organized by category. A trusted resource from one of the most respected names in OSINT. Website link: ✅ OSINT Library - A community-built collection of OSINT tools and resources, organized by investigation type. Great for finding tools you have never heard of before. Website link: ✅ Open Source Intelligence Hub - A clean, no-frills index of OSINT tools and techniques. Useful for quickly scanning what is available across different investigation categories. Website link: __________ Do you know other toolboxes? Let me know in the comments. 👇 P.S. ♻️ Repost if you found this helpful.

CyberSudo

11,830 Aufrufe • vor 4 Monaten

I genuinely don't understand why everyone isn't using this yet Andrej Karpathy, a co-founder of OpenAI, posted a simple idea that hit 16 million views: stop using AI to write code, use it to build a second brain. You point Claude Code at a folder, drop in any source, an article, a transcript, a PDF, and Claude reads it, links it, and files it into a living wiki of everything you know. It compounds like interest, the more you feed it, the smarter it gets. Here's the whole thing: > Install Obsidian, create a vault, open it in Claude Code > Paste Karpathy's wiki idea file and tell Claude to build it > Claude makes three folders: raw for sources, wiki for its pages, a CLAUDE.md that runs it > Drop any source into raw and say "ingest this" > Ask questions across everything, forever Five minutes to set up, and you never start from a blank chat again. Full step-by-step guide with Claude and Obsidian, link below. Bookmark this
0:16

I genuinely don't understand why everyone isn't using this yet Andrej Karpathy, a co-founder of OpenAI, posted a simple idea that hit 16 million views: stop using AI to write code, use it to build a second brain. You point Claude Code at a folder, drop in any source, an article, a transcript, a PDF, and Claude reads it, links it, and files it into a living wiki of everything you know. It compounds like interest, the more you feed it, the smarter it gets. Here's the whole thing: > Install Obsidian, create a vault, open it in Claude Code > Paste Karpathy's wiki idea file and tell Claude to build it > Claude makes three folders: raw for sources, wiki for its pages, a CLAUDE.md that runs it > Drop any source into raw and say "ingest this" > Ask questions across everything, forever Five minutes to set up, and you never start from a blank chat again. Full step-by-step guide with Claude and Obsidian, link below. Bookmark this

Ridark

7,025,143 Aufrufe • vor 3 Monaten

🚀 Introducing EgoExo Forge - built on top of Rerun, Gradio, and Hugging Face hub (I’ll be in San Francisco July 21–29 — if you’re into robotics, egocentric AI, large-scale data collection, or just want to chat, DM me!) In my opinion, large-scale, diverse, and high-quality data is still the largest bottleneck for generalized robotics deployment. I believe that some version of imitation learning from human examples will be the most scalable + clean way to train humanoid robots 🤖 (similar to what Tesla did for Full Self Driving). Teleop is too expensive to collect a large enough dataset in a reasonable manner, so passive collection via egocentric (and in certain cases, exocentric) views feels like the right bet. Over the past few months, I've been trying to build out the scaffolding for this and using Rerun as my underlying infrastructure. Data being collected needs to be easily inspectable + time series and rerun provides the right tooling for this. My goal is to first build out a ground truth representative dataset from already existing open source data, generate some reasonable baselines, and then go out and collect my own data that adheres to the defined schema. 🔍 Starting with open-source datasets 1. EgoDex from Apple 2. HOCap from Nvidia and the University of Texas at Dallas 3. Assembly101 from Meta All these different datasets have different sensor configurations + annotations, so my goal with egoexo-forge is to have one consistent labeling scheme + data layout. I built a data pipeline that aligns all of the different datasets in one general schema assuming the COCO133 keypoint layout that allows for exo+ego, ego only, or exo only Since the scaffolding is already there, it becomes MUCH easier to add other datasets. So the next ones that I'll be including are HD-EPIC kitchens dataset, HOT3D, and finally my own personal iPhone + insta360 go collection method. Once I have a diverse variety of datasets, I'll double down on what I believe to be the key algorithms required to make useful data for imitation learning 📊 1. Camera Pose estimation via SLAM/SFM for ego perspective (and automatic calibration for exo) 2. Human pose estimation for both egocentric + exocentric views 3. Metric 3D reconstruction + object tracking I'll be setting up reasonable open-source baselines for each of these to validate that these datasets work, and then finally try to use the generated datasets for some imitation learning via the pi0-lerobot repo I've been working on. I plan on making a blog post + providing more info on all of this in the near future so stay tuned
0:18

🚀 Introducing EgoExo Forge - built on top of Rerun, Gradio, and Hugging Face hub (I’ll be in San Francisco July 21–29 — if you’re into robotics, egocentric AI, large-scale data collection, or just want to chat, DM me!) In my opinion, large-scale, diverse, and high-quality data is still the largest bottleneck for generalized robotics deployment. I believe that some version of imitation learning from human examples will be the most scalable + clean way to train humanoid robots 🤖 (similar to what Tesla did for Full Self Driving). Teleop is too expensive to collect a large enough dataset in a reasonable manner, so passive collection via egocentric (and in certain cases, exocentric) views feels like the right bet. Over the past few months, I've been trying to build out the scaffolding for this and using Rerun as my underlying infrastructure. Data being collected needs to be easily inspectable + time series and rerun provides the right tooling for this. My goal is to first build out a ground truth representative dataset from already existing open source data, generate some reasonable baselines, and then go out and collect my own data that adheres to the defined schema. 🔍 Starting with open-source datasets 1. EgoDex from Apple 2. HOCap from Nvidia and the University of Texas at Dallas 3. Assembly101 from Meta All these different datasets have different sensor configurations + annotations, so my goal with egoexo-forge is to have one consistent labeling scheme + data layout. I built a data pipeline that aligns all of the different datasets in one general schema assuming the COCO133 keypoint layout that allows for exo+ego, ego only, or exo only Since the scaffolding is already there, it becomes MUCH easier to add other datasets. So the next ones that I'll be including are HD-EPIC kitchens dataset, HOT3D, and finally my own personal iPhone + insta360 go collection method. Once I have a diverse variety of datasets, I'll double down on what I believe to be the key algorithms required to make useful data for imitation learning 📊 1. Camera Pose estimation via SLAM/SFM for ego perspective (and automatic calibration for exo) 2. Human pose estimation for both egocentric + exocentric views 3. Metric 3D reconstruction + object tracking I'll be setting up reasonable open-source baselines for each of these to validate that these datasets work, and then finally try to use the generated datasets for some imitation learning via the pi0-lerobot repo I've been working on. I plan on making a blog post + providing more info on all of this in the near future so stay tuned

Pablo Vela

37,360 Aufrufe • vor 1 Jahr

Today may be the ImageNet moment for robotics. RT-X: the largest open-source robot dataset ever compiled, across 33 institutes, 22 robot hardware, 527 skills, and 1M episodes. Why is robotics lagging so far behind NLP, vision, and other AI domains? Data scarcity is the main culprit to blame, among other difficulties. Unlike text, images, and videos, you cannot download mass amounts of onboard robot control data from the internet. They simply don't exist in the wild. 11 yrs ago, ImageNet kicked off the deep learning revolution. 3-4 yrs ago, internet-scale data fueled the first GPTs and Diffusions that define this era of foundation models. I think 2023 is finally the year for robotics to scale up. Robot foundation models like VIMA ( my team's work at NVIDIA) and RT-1/2 ( Google DeepMind's effort) are extremely data hungry. While massively parallel simulations like NVIDIA IsaacGym & Omniverse can alleviate the problem to some extent, it's still not quite enough to bridge the gap to the messy, physical world. This new dataset is not just a technical contribution. I also see it as a commendable effort to overcome institutional bureaucracies and unite researchers from around the world to tackle a grand challenge together. Robotics will be the final holy grail that we capture in AI. We are not there yet, but ascending in the right gradient direction. RT-X website: Launch blog:
0:11

Today may be the ImageNet moment for robotics. RT-X: the largest open-source robot dataset ever compiled, across 33 institutes, 22 robot hardware, 527 skills, and 1M episodes. Why is robotics lagging so far behind NLP, vision, and other AI domains? Data scarcity is the main culprit to blame, among other difficulties. Unlike text, images, and videos, you cannot download mass amounts of onboard robot control data from the internet. They simply don't exist in the wild. 11 yrs ago, ImageNet kicked off the deep learning revolution. 3-4 yrs ago, internet-scale data fueled the first GPTs and Diffusions that define this era of foundation models. I think 2023 is finally the year for robotics to scale up. Robot foundation models like VIMA ( my team's work at NVIDIA) and RT-1/2 ( Google DeepMind's effort) are extremely data hungry. While massively parallel simulations like NVIDIA IsaacGym & Omniverse can alleviate the problem to some extent, it's still not quite enough to bridge the gap to the messy, physical world. This new dataset is not just a technical contribution. I also see it as a commendable effort to overcome institutional bureaucracies and unite researchers from around the world to tackle a grand challenge together. Robotics will be the final holy grail that we capture in AI. We are not there yet, but ascending in the right gradient direction. RT-X website: Launch blog:

Jim Fan

265,061 Aufrufe • vor 3 Jahren

Something big is happening in robotics - and it’s hiding in plain sight. This post is not about dancing robots but in the data that powers them. Open robotics datasets have exploded this year, turning the field into a more scalable and collaborative ecosystem. In just two years, Hugging Face datasets grew from 11k to over 600k - and robotics is by far the fastest-growing segment. We went from 1k robotics datasets in 2024 to 27k in 2025! For comparison, text generation, the second-largest category, has only around 5k datasets in 2025. That gap is massive. Open datasets are important because robotics lives and dies by real-world robot data - video, actions, sensors, failures. By making this data easy to upload, reuse, and benchmark, researchers, startups, and large players are now releasing real-robot datasets that would have stayed locked inside labs just a few years ago. Major contributors include NVIDIA, LeRobot initiative, and a rapidly growing maker community. This surge is also enabled by cheaper video storage, better tooling, and an open-source AI culture now spilling into the physical world. And it really matters: open robotics data dramatically lowers entry barriers, accelerates learning-by-doing, and speeds up progress toward generalist and humanoid robots. Robotics won’t scale through hardware alone - but to a large extent through shared data. Viz below from AI World - link to the story and more viz/filters in comment.
0:11

Something big is happening in robotics - and it’s hiding in plain sight. This post is not about dancing robots but in the data that powers them. Open robotics datasets have exploded this year, turning the field into a more scalable and collaborative ecosystem. In just two years, Hugging Face datasets grew from 11k to over 600k - and robotics is by far the fastest-growing segment. We went from 1k robotics datasets in 2024 to 27k in 2025! For comparison, text generation, the second-largest category, has only around 5k datasets in 2025. That gap is massive. Open datasets are important because robotics lives and dies by real-world robot data - video, actions, sensors, failures. By making this data easy to upload, reuse, and benchmark, researchers, startups, and large players are now releasing real-robot datasets that would have stayed locked inside labs just a few years ago. Major contributors include NVIDIA, LeRobot initiative, and a rapidly growing maker community. This surge is also enabled by cheaper video storage, better tooling, and an open-source AI culture now spilling into the physical world. And it really matters: open robotics data dramatically lowers entry barriers, accelerates learning-by-doing, and speeds up progress toward generalist and humanoid robots. Robotics won’t scale through hardware alone - but to a large extent through shared data. Viz below from AI World - link to the story and more viz/filters in comment.

Pierre-Alexandre Balland

186,094 Aufrufe • vor 9 Monaten

Own your personal agent, folks. Just built my own little personal agent with Pi's Pi Durable. I wanted to see how quickly I could build an OpenAI dots clone with open-source tools. I am amazed at how quickly I got this off the ground. It's proactive. It keeps working when you're not watching. It remembers you, runs jobs on a schedule, waits for your approval, and uses its own computer. Pi Durable makes that possible. Every step is checkpointed. I can kill the process mid-task, and My Pi picks up right where it left off. Memory lives in durable documents. What My Pi learns about me in one Space survives a crash and carries over to every other Space. Approvals are durable too. It also has its own computer. Each Space gets a full Linux desktop that My Pi controls with screenshots, mouse, and keyboard. The model runs through OpenRouter, so I can swap models without changing anything else. That's all configurable however you like it. If you've been wanting your own personal agent, now is a great time to build one. I am building this as an experimental ground where I can test new ideas and experiment deeply with proactive personal agents. But I plan to open-source it after testing it a bit and adding more interesting primitives. The goal is to make it customizable and easy to build on, so others can experiment with their own personal agent functionalities.
0:15

Own your personal agent, folks. Just built my own little personal agent with Pi's Pi Durable. I wanted to see how quickly I could build an OpenAI dots clone with open-source tools. I am amazed at how quickly I got this off the ground. It's proactive. It keeps working when you're not watching. It remembers you, runs jobs on a schedule, waits for your approval, and uses its own computer. Pi Durable makes that possible. Every step is checkpointed. I can kill the process mid-task, and My Pi picks up right where it left off. Memory lives in durable documents. What My Pi learns about me in one Space survives a crash and carries over to every other Space. Approvals are durable too. It also has its own computer. Each Space gets a full Linux desktop that My Pi controls with screenshots, mouse, and keyboard. The model runs through OpenRouter, so I can swap models without changing anything else. That's all configurable however you like it. If you've been wanting your own personal agent, now is a great time to build one. I am building this as an experimental ground where I can test new ideas and experiment deeply with proactive personal agents. But I plan to open-source it after testing it a bit and adding more interesting primitives. The goal is to make it customizable and easy to build on, so others can experiment with their own personal agent functionalities.

elvis

34,905 Aufrufe • vor 5 Tagen

If you think this is just another silly demo made with AI, read this post. You might change your mind, because this demo is about MATH. What you see on the screen is not a render from Blender (obviously, it’s not that good). It’s a three.js app built with Toolcraft. Available on the web and rendered in real time(link in the comments). But Blender still has a lot to do with it. Blender has Geometry Nodes - a powerful node-based system for creating and manipulating procedural geometry. In other words, it’s math. And math is a universal language. And who do you think is pretty good at math? >>> AI. Now you can download or buy Blender files from marketplaces, and when they contain Geometry Nodes for procedural animations, objects, surfaces, or effects, you can transfer that logic to the web. Make it real-time, make it interactive. Materials are a separate story, of course. They can still suck unless you use the right tricks: PBR, HDRIs, material blending, displacement, and faked surface relief. So why is Blender important here? Blender is open source, and many tools around it are open source too. An AI trained on their code. That means it can translate the math from one environment to another quite accurately. If you’ve been struggling to reproduce some idea with AI that you had in your head or seen in some references, and it has something to do with Geometry Nodes, and you can find that idea or a close one in the Blender ecosystem - it means you can transfer it to the web. Thank me later.
0:11

If you think this is just another silly demo made with AI, read this post. You might change your mind, because this demo is about MATH. What you see on the screen is not a render from Blender (obviously, it’s not that good). It’s a three.js app built with Toolcraft. Available on the web and rendered in real time(link in the comments). But Blender still has a lot to do with it. Blender has Geometry Nodes - a powerful node-based system for creating and manipulating procedural geometry. In other words, it’s math. And math is a universal language. And who do you think is pretty good at math? >>> AI. Now you can download or buy Blender files from marketplaces, and when they contain Geometry Nodes for procedural animations, objects, surfaces, or effects, you can transfer that logic to the web. Make it real-time, make it interactive. Materials are a separate story, of course. They can still suck unless you use the right tricks: PBR, HDRIs, material blending, displacement, and faked surface relief. So why is Blender important here? Blender is open source, and many tools around it are open source too. An AI trained on their code. That means it can translate the math from one environment to another quite accurately. If you’ve been struggling to reproduce some idea with AI that you had in your head or seen in some references, and it has something to do with Geometry Nodes, and you can find that idea or a close one in the Blender ecosystem - it means you can transfer it to the web. Thank me later.

Alex Barashkov

28,300 Aufrufe • vor 2 Monaten