Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

World action models solving cable insertion at scale! 📱 During Automate Show, Industrial Next showed a world action model application for actual manufacturing. Cable insertion is brutal to automate. Cables break easily. They're never in the same position on the production line. Right now, this step is manual for... virtually every phone ever made. The challenge? High-precision, tactile work with deformable objects. This company built a foundation model specifically for tiny electronics parts. They rejected third-party datasets. Quality matters more than quantity when precision is critical. Two hours of extremely high-quality data beats months of mediocre data. They use teleop data combined with a corpus of basic manufacturing techniques to give the model context. I had a chance to speak with Lukas Pankau and he explained me a bit of this and a bit of that. For example that simulation doesn't help here. Deformable objects like wires are nearly impossible to simulate accurately. No physics simulator is good enough yet. So they collect real data, train on it, deploy it. The model can handle long-horizon tasks in a single shot, maintaining sub-millimeter precision across an entire sequence without intermediate corrections. They own the data collection, the model training, and the deployment. They position themselves as a "world platformer," not just a model vendor. They do integration work because traditional system integrators still make money on software complexity, not on clean solutions. Accoridng to them that's misaligned with where manufacturing is heading. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →show more

Lukas Ziegler

65,157 subscribers

11,480 views • 4 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

6 Comments

nathandyer4 months ago

@AutomateShow It doesn't just sell models; it also generates, trains, and integrates data on its own, firmly positioning itself as a “global platform builder.”👍

Jon Christie4 months ago

@AutomateShow Surgical models must be getting good.

Sherry Deneen Whitaker4 months ago

@AutomateShow Finally, someone has transformed the manual task of “wiring” into cutting-edge AI technology! Two hours of high-quality data yields better results than months of junk data—this is truly a breakthrough technology that can reshape the industrial landscape!

John Dagdelen4 months ago

@AutomateShow Am I missing something or is it failing every time?

Sean Kerr — AI, minus the hype4 months ago

@AutomateShow The "simulation doesn't work for deformable objects" detail is the real story here. Every robotics team I've seen hit this wall eventually. Real data + tight hardware integration beats synthetic shortcuts — same lesson as LLM fine-tuning, just with cables.

Jason傑森 🇭🇰 | 🛠️4 months ago

@AutomateShow 这也太科幻了

Related Videos

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.

Grass

211,994 views • 2 years ago

Grass is betting that better AI still starts with better pre-training data “Whatever you put into your model ultimately determines what your model's capabilities are” “Several of the top frontier AI labs in the world are working directly with Grass to acquire large amounts of pre-training data” “If that information is not present in your model, your model doesn't know how to do that thing in the first place” Andrej puts Grass close to one of the most fundamental inputs in AI: the training corpus itself. As frontier models scale, they need enormous amounts of diverse, high-quality web data. They are building the infrastructure to collect and supply that data at internet scale. Model capability starts with what the model gets to learn from.

Grass is betting that better AI still starts with better pre-training data “Whatever you put into your model ultimately determines what your model's capabilities are” “Several of the top frontier AI labs in the world are working directly with Grass to acquire large amounts of pre-training data” “If that information is not present in your model, your model doesn't know how to do that thing in the first place” Andrej puts Grass close to one of the most fundamental inputs in AI: the training corpus itself. As frontier models scale, they need enormous amounts of diverse, high-quality web data. They are building the infrastructure to collect and supply that data at internet scale. Model capability starts with what the model gets to learn from.

Kaiz 🐂🀄️

13,989 views • 1 month ago

Can Eli Lilly become the largest company in the world? Jordi Visser thinks so. "The reason I believe Eli Lilly has a chance to be the largest company in the world, and the number one AI company in the world within 5 years, is because they are building a specialized model." "They have their own data center with 1,000 GPUs with all the data that $LLY has had [as] a 150 year old company." "They are creating an innovation hub of using data all for human software."

Can Eli Lilly become the largest company in the world? Jordi Visser thinks so. "The reason I believe Eli Lilly has a chance to be the largest company in the world, and the number one AI company in the world within 5 years, is because they are building a specialized model." "They have their own data center with 1,000 GPUs with all the data that $LLY has had [as] a 150 year old company." "They are creating an innovation hub of using data all for human software."

Anthony Pompliano 🌪

140,724 views • 5 months ago

Today we're launching Hebbian Robotics (YC S26) Hebbian Robotics . We build APIs for evaluating data quality, without training a robotics model. Data vendors use our API to search and analyze Physical AI data at scale, getting on-demand quality signals and metrics without managing infrastructure. High-quality data is the bottleneck to the next frontier of intelligence in robotics. Buyers want data they can trust. Data providers are focused on operational excellence. They don't have the bandwidth to understand the quality of their data at scale. We're fixing this. Vendors use our APIs to understand the impact of their decisions on their data quality: - Buyers demanding per-collector hour limits? Prove compliance while we stay ahead of the workarounds. - Changed your collection SOPs? Monitor for quality drift and regression. - Suspect duplicate samples? Verify with a single API call. Before Hebbian, I trained AI models for industrial cleaning robots, and Kingston Kuan spent years building large-scale infrastructure at Jane Street.

Today we're launching Hebbian Robotics (YC S26) Hebbian Robotics . We build APIs for evaluating data quality, without training a robotics model. Data vendors use our API to search and analyze Physical AI data at scale, getting on-demand quality signals and metrics without managing infrastructure. High-quality data is the bottleneck to the next frontier of intelligence in robotics. Buyers want data they can trust. Data providers are focused on operational excellence. They don't have the bandwidth to understand the quality of their data at scale. We're fixing this. Vendors use our APIs to understand the impact of their decisions on their data quality: - Buyers demanding per-collector hour limits? Prove compliance while we stay ahead of the workarounds. - Changed your collection SOPs? Monitor for quality drift and regression. - Suspect duplicate samples? Verify with a single API call. Before Hebbian, I trained AI models for industrial cleaning robots, and Kingston Kuan spent years building large-scale infrastructure at Jane Street.

Brandon Ong

166,971 views • 3 months ago

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

The Humanoid Hub

17,065 views • 9 months ago

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 views • 1 year ago

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗

NVIDIA Robotics

13,562 views • 9 months ago

That's one of the coolest robotics demos! 😮‍💨 Generalist showed GEN-1 handling box folding and screw packing during Automate Show. The boxes are cardboard with real variability: creasing, deformation, different configurations. GEN-1 retries when things go wrong. It adapts mid-task. This is a jump from GEN-0, shown 3 months ago during GTC. GEN-0 handled rigid, predictable boxes. GEN-1 handles deformable ones with chaotic behavior. The scaling is straightforward. GEN-1 trained on 500,000 hours of proprietary, real-world dexterous manipulation data. Scaling laws are working. More high-quality real-world data leads to better physics understanding, better retry behavior, better generalization. And the time to get a working demo is shrinking drastically. They're not even at scale yet. They're just starting to see what happens when you pre-train on massive amounts of actual robot data. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

That's one of the coolest robotics demos! 😮‍💨 Generalist showed GEN-1 handling box folding and screw packing during Automate Show. The boxes are cardboard with real variability: creasing, deformation, different configurations. GEN-1 retries when things go wrong. It adapts mid-task. This is a jump from GEN-0, shown 3 months ago during GTC. GEN-0 handled rigid, predictable boxes. GEN-1 handles deformable ones with chaotic behavior. The scaling is straightforward. GEN-1 trained on 500,000 hours of proprietary, real-world dexterous manipulation data. Scaling laws are working. More high-quality real-world data leads to better physics understanding, better retry behavior, better generalization. And the time to get a working demo is shrinking drastically. They're not even at scale yet. They're just starting to see what happens when you pre-train on massive amounts of actual robot data. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

30,136 views • 4 months ago

Training robot foundation models faces two key hurdles: how to get enough data to train an effective model, and how to make sure that new skills can be acquired quickly. The team at Rhoda AI believes that the answer is training Direct Video Action models from web data. Web data is plentiful, to the point where Rhoda can train their base model on hundreds of years of video data. And then, with the addition of robot data, they can quickly adapt it to new tasks with as little as 20 hours of in-domain data, performing complex, multi-step manipulation tasks with their purpose-built video foundation model. Tongzhou Mu 🤖🦾🦿 Eric Chan and Changan Chen joined us to talk more about their approach. Watch Episode #79 of RoboPapers, with Michael Cho - Rbt/Acc, Chris Paxton, and Jiafei Duan, to learn more!

Training robot foundation models faces two key hurdles: how to get enough data to train an effective model, and how to make sure that new skills can be acquired quickly. The team at Rhoda AI believes that the answer is training Direct Video Action models from web data. Web data is plentiful, to the point where Rhoda can train their base model on hundreds of years of video data. And then, with the addition of robot data, they can quickly adapt it to new tasks with as little as 20 hours of in-domain data, performing complex, multi-step manipulation tasks with their purpose-built video foundation model. Tongzhou Mu 🤖🦾🦿 Eric Chan and Changan Chen joined us to talk more about their approach. Watch Episode #79 of RoboPapers, with Michael Cho - Rbt/Acc, Chris Paxton, and Jiafei Duan, to learn more!

RoboPapers

24,734 views • 6 months ago

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

The All-In Podcast

79,718 views • 1 year ago

We've signed our first commercial robotics data pilot with Vulcan Shield Hungary, one of the largest advanced material manufacturing complex in Europe. Starting at a production workstation in Békéscsaba, we'll combine real-world data and simulation to work toward a robot model trained for a specific factory task.

We've signed our first commercial robotics data pilot with Vulcan Shield Hungary, one of the largest advanced material manufacturing complex in Europe. Starting at a production workstation in Békéscsaba, we'll combine real-world data and simulation to work toward a robot model trained for a specific factory task.

OpenRoboto

18,243 views • 6 days ago

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:

15–30 minutes of real-world robot data. That's now enough to go from sim-to-real failure to working robot. Let’s see… You train a robot in simulation. You deploy it in the real world. It fails. The physics don't match. So you try to fine-tune it with real data, but… you never have enough real data, and the fine-tuning breaks everything the simulation taught it. SimDist fixes this with one key decision: don't transfer the policy. Transfer the world model. Keep the reward and value knowledge from simulation frozen. Only update the part that's actually wrong, how the robot predicts physics. Now the robot doesn't have to relearn the entire task in the real world. It already knows what success looks like. It just needs to correct its understanding of how the real world moves. The part that makes this work: they also trained on failures and recoveries; not just perfect demonstrations. Without that, the planner finds the gaps and exploits them. With it, the robot can tell a good future from a bad one. That's all it needs. Results on peg insertion, table leg assembly, locomotion on slippery and uneven surfaces. Tasks that require precision, force, and quick reaction. Thanks for sharing, Tyler Westenbroek ([Tyler Westenbroek]. Interactive visualization + paper: ——- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

14,222 views • 5 months ago

Robotics is almost solved! Pun intended. Sunday published a "Solve" standard. They declared the scope (garment types, lighting, surfaces, starting configurations) and adaptation cost (zero—no fine-tuning per home) before evaluating. The same model weights run everywhere. This might be the differentiation between a good demo and reliable work of robots. They presented ACT-2, a robot foundation model that folds laundry at 99.1% success across unseen homes. Different houses, lighting, bed types, garment sizes. According to them scaling pre-training closes the generalization gap. Train on massive, high-quality human data. When you do that right, what you learn in-house actually transfers to unseen homes. Even wilder is that the model learns new folding techniques from a single example and generalizes them to new environments. Memo goes to homes later this year. Would you adopt this cutie? 🥹 Read more here: Great work Tony Zhao and team! 🧡 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Robotics is almost solved! Pun intended. Sunday published a "Solve" standard. They declared the scope (garment types, lighting, surfaces, starting configurations) and adaptation cost (zero—no fine-tuning per home) before evaluating. The same model weights run everywhere. This might be the differentiation between a good demo and reliable work of robots. They presented ACT-2, a robot foundation model that folds laundry at 99.1% success across unseen homes. Different houses, lighting, bed types, garment sizes. According to them scaling pre-training closes the generalization gap. Train on massive, high-quality human data. When you do that right, what you learn in-house actually transfers to unseen homes. Even wilder is that the model learns new folding techniques from a single example and generalizes them to new environments. Memo goes to homes later this year. Would you adopt this cutie? 🥹 Read more here: Great work Tony Zhao and team! 🧡 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

37,154 views • 3 months ago

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

Jason Ma reveals the first true scaling law in robotics: robots trained on 1 million hours of first-person human video predictably get better without ever seeing robot data "Yesterday we announced our new flagship robot foundation model called Dyna-2. It's very significant for the entire field for many different reasons. First of all, it's the first robot foundation model trained on at least one million hours of data. That in itself is a very challenging infrastructure challenge." "What we demonstrated is that even if the one million hours of data is entirely first-person video of humans doing manipulation tasks, we actually see scaling law for transferring to robot embodiments that the model has never seen before." "By training on more human data, we actually see predictable performance improvement on robots. That's a big deal because the biggest challenge facing robot foundation models is that we don't have enough data." "Unlike language models, there are just not enough robotics data out there, and it's very difficult to collect these robot data. Only by passively observing humans doing things can we scale this kind of data, and this is the first time we showed there is a lot of improvement that can transfer." Jason Ma Dyna Robotics

MTS

18,002 views • 3 months ago

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Robot trained entirely in simulation! 🤯 Sudo AI just released #sudo R1. This could be the moment the data bottleneck in robotics finally breaks, a major step forward for getting simulation-trained robots to actually work in the real world. #sudo R1 is a manipulation foundation model that reliably picks objects it has never seen: glass, fabric, metal, irregular shapes. All with zero real-world training data. No tele-op. No demos. No manual labeling. Pure simulation. And picking isn't just any skill, it's the foundation. Almost everything we want robots to do starts with being able to reliably pick something up. Data collection is the bottleneck holding the whole field back. Every new task means hours of tele-op and armies of labelers. If simulation alone can actually deliver production-grade picking, that bottleneck starts to disappear. Really exciting to see! If the results hold, the path to general-purpose robots just got a whole lot shorter. 👇 Read more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

25,901 views • 6 months ago

The robot flipped a pancake nobody taught it! 🥞 Skild AI team assumed pancake flipping had to be somewhere in the training data. So they searched. Millions of hours of pre-training data. Nothing. S1 inferred the whole task from a single human demonstration. That's their new general robot model, built as an in-context learner from the ground up. Every new robot task today starts with days of teleoperation and a fine-tuning run on a specialist policy. S1 skips all of it. Much like a language model, it never updates its weights to learn a new task. The demonstration enters the context window, and the policy uses it to decide what to do next. → Ten-minute tasks it was never trained on, composed from primitives learned in pre-training: a new style of coffee, potting a plant, frying pancakes. → Soil and pots arrived at their office at 8:54 PM. The robot was running the task autonomously by 9:27 PM. → Slide objects away mid-reach, swap them, change the lighting, it still finishes. → The prompt waters a plant with a watering can, but only a cup is available. It uses the cup. It doesn't rigidly replay what it saw, but it recovers from its own errors, and sometimes executes with more precision than the demonstrator, when the human fumbles an egg and makes a mess, S1 performs the same step cleanly. The demonstration is a specification of the goal, and not a trajectory to copy. On unseen tasks after 100K hours of pre-training: language-prompted VLAs reach 9%. Their new model reaches 66%. It's already deploying with industrial partners, with a wider rollout over the coming months. Congrats Deepak Pathak and team behind this! 😮‍💨 🔗 Link to their latest blog: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

The robot flipped a pancake nobody taught it! 🥞 Skild AI team assumed pancake flipping had to be somewhere in the training data. So they searched. Millions of hours of pre-training data. Nothing. S1 inferred the whole task from a single human demonstration. That's their new general robot model, built as an in-context learner from the ground up. Every new robot task today starts with days of teleoperation and a fine-tuning run on a specialist policy. S1 skips all of it. Much like a language model, it never updates its weights to learn a new task. The demonstration enters the context window, and the policy uses it to decide what to do next. → Ten-minute tasks it was never trained on, composed from primitives learned in pre-training: a new style of coffee, potting a plant, frying pancakes. → Soil and pots arrived at their office at 8:54 PM. The robot was running the task autonomously by 9:27 PM. → Slide objects away mid-reach, swap them, change the lighting, it still finishes. → The prompt waters a plant with a watering can, but only a cup is available. It uses the cup. It doesn't rigidly replay what it saw, but it recovers from its own errors, and sometimes executes with more precision than the demonstrator, when the human fumbles an egg and makes a mess, S1 performs the same step cleanly. The demonstration is a specification of the goal, and not a trajectory to copy. On unseen tasks after 100K hours of pre-training: language-prompted VLAs reach 9%. Their new model reaches 66%. It's already deploying with industrial partners, with a wider rollout over the coming months. Congrats Deepak Pathak and team behind this! 😮‍💨 🔗 Link to their latest blog: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

11,406 views • 2 months ago

SITUATION EXPLAINED: Why is the Bridgewater Thinking Machines paper a blueprint for every enterprise that has ever paid frontier model prices for generic outputs? 47fucb4r8curb4fc8f8r4bfic8r: "What Thinking Machines and Bridgewater did together is a blueprint for how a wide variety of stakeholders within the market can benefit tremendously from using LLMs and integrating them into a larger enterprise system." "They got a pretty old model. A Qwen3 model. Two hundred and thirty-five billion parameters. Not a new model. Not a very sophisticated or impressive state-of-the-art model. And they essentially fine-tuned it on Bridgewater's own proprietary data." "A financial analyst gets a ton of financial documents and has to filter which are worth looking at. Then evaluate which news pieces are relevant. They have a lot of tacit knowledge from doing the job on where to look. What they did was inject a lot of this tacit knowledge into the model through fine-tuning." "Stop obsessing over state-of-the-art models that are the most intelligent possible. Focus on targeting a very specific LLM specifically trained for a specific function to do that function better than the alternatives on the market." "This old non-state-of-the-art model got 84.7% accuracy versus 78% for all the state-of-the-art models. A massive improvement. And the kicker is they got a better performing model that is 14 times cheaper to use."

SITUATION EXPLAINED: Why is the Bridgewater Thinking Machines paper a blueprint for every enterprise that has ever paid frontier model prices for generic outputs? 47fucb4r8curb4fc8f8r4bfic8r: "What Thinking Machines and Bridgewater did together is a blueprint for how a wide variety of stakeholders within the market can benefit tremendously from using LLMs and integrating them into a larger enterprise system." "They got a pretty old model. A Qwen3 model. Two hundred and thirty-five billion parameters. Not a new model. Not a very sophisticated or impressive state-of-the-art model. And they essentially fine-tuned it on Bridgewater's own proprietary data." "A financial analyst gets a ton of financial documents and has to filter which are worth looking at. Then evaluate which news pieces are relevant. They have a lot of tacit knowledge from doing the job on where to look. What they did was inject a lot of this tacit knowledge into the model through fine-tuning." "Stop obsessing over state-of-the-art models that are the most intelligent possible. Focus on targeting a very specific LLM specifically trained for a specific function to do that function better than the alternatives on the market." "This old non-state-of-the-art model got 84.7% accuracy versus 78% for all the state-of-the-art models. A massive improvement. And the kicker is they got a better performing model that is 14 times cheaper to use."

MTS

93,185 views • 4 months ago

No data, no problem introducing agentic synthetic data generation with Cosmos 3 share a few examples, generate more data, automate model training, automatically deploy the latest version with no downtime in a benchmark run with Corning Incorporated's optical fiber manufacturing engineering team, a model trained on 8 real defect images plus synthetic examples generated by Cosmos reached 0.95 mean average precision and perfect recall on the toughest defect class, beating a baseline trained on real data alone. "The Roboflow Agent powered by NVIDIA allows us to generate the training data we need, fine-tune our models, and strengthen model performance and inspection quality while increasing the speed, scalability, and adoption of next-generation technologies,” - Jeremy Knopf, chief information officer, Corning Optical Communications

No data, no problem introducing agentic synthetic data generation with Cosmos 3 share a few examples, generate more data, automate model training, automatically deploy the latest version with no downtime in a benchmark run with Corning Incorporated's optical fiber manufacturing engineering team, a model trained on 8 real defect images plus synthetic examples generated by Cosmos reached 0.95 mean average precision and perfect recall on the toughest defect class, beating a baseline trained on real data alone. "The Roboflow Agent powered by NVIDIA allows us to generate the training data we need, fine-tune our models, and strengthen model performance and inspection quality while increasing the speed, scalability, and adoption of next-generation technologies,” - Jeremy Knopf, chief information officer, Corning Optical Communications

Roboflow

39,416 views • 5 months ago

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 views • 8 months ago