Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Robot simulation is getting a big upgrade! 🔥 Normally, building a simulation environment, a kitchen, warehouse, office takes days or weeks. You have to model the geometry, add textures, set lighting, generate collision meshes… all before a robot can even interact with it. Marble by World Labs removes most...

lukas_m_ziegler's profile picture

Lukas Ziegler

52,491 subscribers

15,498 Aufrufe • vor 6 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Our third announcement is GWM Worlds. A world model for real-time environment simulation. You give the model a static scene, and it generates an immersive, infinite, explorable space as you move through it, with geometry, lighting and physics. All in real-time. You can travel to any place, real or imagined. You can become any agent, a person walking through a city, a drone flying over a snowy mountain or a robot navigating a warehouse.
1:15

Our third announcement is GWM Worlds. A world model for real-time environment simulation. You give the model a static scene, and it generates an immersive, infinite, explorable space as you move through it, with geometry, lighting and physics. All in real-time. You can travel to any place, real or imagined. You can become any agent, a person walking through a city, a drone flying over a snowy mountain or a robot navigating a warehouse.

Runway

80,309 Aufrufe • vor 6 Monaten

added World Labs API to the simulation builder, you can now type a prompt and get a full 3D world with physics colliders loaded into MuJoCo all in your browser
1:04

added World Labs API to the simulation builder, you can now type a prompt and get a full 3D world with physics colliders loaded into MuJoCo all in your browser

Moyai

61,432 Aufrufe • vor 5 Monaten

A pentagon robot! 🖐🏽 Compared to typical SCARA models, it delivers higher speed, improved stability, better space efficiency, and can operate smoothly in tight areas, even through singularity positions. These robots are a good fit for handling complex tasks in challenging environments where conventional robots often fall short. Have you ever seen a robot like this? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:24

A pentagon robot! 🖐🏽 Compared to typical SCARA models, it delivers higher speed, improved stability, better space efficiency, and can operate smoothly in tight areas, even through singularity positions. These robots are a good fit for handling complex tasks in challenging environments where conventional robots often fall short. Have you ever seen a robot like this? 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

42,565 Aufrufe • vor 3 Monaten

Big news! World Labs' Marble is officially compatible with NVIDIA Robotics' Isaac Sim. Instead of showcasing their highlight reel, our team at Lightwheel created a full environment from a 360 pano using Marble and showed you can train directly in sim. The only non-generated assets are the robot arms, coffee, and bowl. Everything else was generated with Marble. Not sure how to do this yourself? NVIDIA wrote a technical blog with full instructions. You will learn how to: 1. Export scenes from World Labs' Marble as Gaussian splats 2. Convert to USD using NVIDIA Omniverse NuRec 3. Import into NVIDIA Isaac Sim 4. Add a robot and run the simulation Read the guide ➡️ #WorldModels #Robotics
0:32

Big news! World Labs' Marble is officially compatible with NVIDIA Robotics' Isaac Sim. Instead of showcasing their highlight reel, our team at Lightwheel created a full environment from a 360 pano using Marble and showed you can train directly in sim. The only non-generated assets are the robot arms, coffee, and bowl. Everything else was generated with Marble. Not sure how to do this yourself? NVIDIA wrote a technical blog with full instructions. You will learn how to: 1. Export scenes from World Labs' Marble as Gaussian splats 2. Convert to USD using NVIDIA Omniverse NuRec 3. Import into NVIDIA Isaac Sim 4. Add a robot and run the simulation Read the guide ➡️ #WorldModels #Robotics

Jonathan Stephens

13,783 Aufrufe • vor 6 Monaten

From a single image to a full robotics simulation. 360 image → Marble → 3D environment → Robotics sim This is how we scale world creation for embodied AI with Lightwheel, turning lightweight capture into simulation-ready environments in minutes, not weeks.
0:36

From a single image to a full robotics simulation. 360 image → Marble → 3D environment → Robotics sim This is how we scale world creation for embodied AI with Lightwheel, turning lightweight capture into simulation-ready environments in minutes, not weeks.

World Labs

45,847 Aufrufe • vor 5 Monaten

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”
2:16

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”

TBPN

18,811 Aufrufe • vor 1 Jahr

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:26

Turning video into humanoid robot motion! 🤳🏼 Training humanoid robots needs huge amounts of motion data, but real-world capture doesn’t scale. Mocap is expensive, dangerous edge cases are rare, and you can’t ask humans to repeatedly fall or crash. Video2Robot tackles this by converting videos into physics-grounded humanoid simulations. Motion is generated to respect balance, inertia, ground contact, and joint limits, then directly retargeted to robot simulators. One prompt can generate a full humanoid motion sequence, including multi-agent interactions and failure cases like falls or collisions, scenarios that are hard or impossible to capture safely in the real world. The pipeline is model-agnostic and works with existing video generators, making it a practical way to scale data for robots. If robots are going to operate in the real world, they need to be trained on the failures too, not just the perfect demos. Here's the GitHub: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

59,693 Aufrufe • vor 6 Monaten

Robotics doesn't need to be boring! 😴 Just imagine, you're an only child, you have no one to play football with. Then this fella comes in and you can train your dribbling skills as long as you want (or as long as the battery lasts). Anyway, pretty awesome robot for non-industrial use. Robots can be seen as companions! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:37

Robotics doesn't need to be boring! 😴 Just imagine, you're an only child, you have no one to play football with. Then this fella comes in and you can train your dribbling skills as long as you want (or as long as the battery lasts). Anyway, pretty awesome robot for non-industrial use. Robots can be seen as companions! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

263,015 Aufrufe • vor 6 Monaten

World Labs and VIVE Mars are making it easier to turn ideas into shoot-ready 3D scenes, fast. World Labs new AI world-builder lets you create detailed virtual environments from a simple text or image prompt, then composite them live. Once the scene is ready, you can step straight into production using VIVE Mars CamTrack. Sign up at a discounted rate to start building worlds, and see how the full workflow comes together with VIVE Mars CamTrack and Marble. 👉 👉
2:24

World Labs and VIVE Mars are making it easier to turn ideas into shoot-ready 3D scenes, fast. World Labs new AI world-builder lets you create detailed virtual environments from a simple text or image prompt, then composite them live. Once the scene is ready, you can step straight into production using VIVE Mars CamTrack. Sign up at a discounted rate to start building worlds, and see how the full workflow comes together with VIVE Mars CamTrack and Marble. 👉 👉

HTC VIVE

40,988 Aufrufe • vor 7 Monaten

This is what open robotics actually looks like! 🔓 I was in Bordeaux, sitting inside Pollen Robotics, and had a chance to play around with their humanoid robot Reachy 2. It’s expressive, modular, and can even bring you coffee, and of course runs Python or Rust. I was talking with Reachy in 3 different languages, tele-operating it and manipulating kitchen stuff, and make it even mimic my gestures. The best part is that there are no locked APIs, or hidden stuff, just an open platform that you can build upon. The connection with Hugging Face makes it super easy to start, as the open source approach allows everyone to contribute, experiment and add their own things. I already ordered his little brother Reachy Mini, let's see what a robotics rookie can do with that robot! 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
1:36

This is what open robotics actually looks like! 🔓 I was in Bordeaux, sitting inside Pollen Robotics, and had a chance to play around with their humanoid robot Reachy 2. It’s expressive, modular, and can even bring you coffee, and of course runs Python or Rust. I was talking with Reachy in 3 different languages, tele-operating it and manipulating kitchen stuff, and make it even mimic my gestures. The best part is that there are no locked APIs, or hidden stuff, just an open platform that you can build upon. The connection with Hugging Face makes it super easy to start, as the open source approach allows everyone to contribute, experiment and add their own things. I already ordered his little brother Reachy Mini, let's see what a robotics rookie can do with that robot! 👀 ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

19,433 Aufrufe • vor 5 Monaten

A toilet-cleaning robot from Reflex Robotics. The best part is when the robot dries its hands... That might be the future of robots doing chores for us. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
1:00

A toilet-cleaning robot from Reflex Robotics. The best part is when the robot dries its hands... That might be the future of robots doing chores for us. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

15,766 Aufrufe • vor 3 Monaten

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!
2:41

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!

Santiago

65,446 Aufrufe • vor 2 Monaten

This might be the craziest workflow you'll see this week. From #VR greyboxing to 3D worlds with the help of #AI. The 2 tools I used are ShapesXR and Marble from World Labs. Let me tell you why and how. - ShapesXR is the most powerful and fastest way to assemble 3D scenes at scale. - The library of assets + ability to sketch from procedural primitives gives the perfect balance between speed and freedom. - ShapesXR allows you to export your creation as a glTF that can be used as a "3D prompt" into Marble - Marble now allows you to create a panorama using a 3D scene as input increasing DRAMATICALLY the control you have during the creation process - Through the workflow you can adjust textures and objects in the scene using AI inpainting - There are tons of options on how to use the final output (create a #GaussianSplatting, export collision mesh and even a detailed mesh 🤯) This is the perfect examples of how VR creative tools improve and accelerate the design process... so make sure to follow me if that's your jam 😉.
1:01

This might be the craziest workflow you'll see this week. From #VR greyboxing to 3D worlds with the help of #AI. The 2 tools I used are ShapesXR and Marble from World Labs. Let me tell you why and how. - ShapesXR is the most powerful and fastest way to assemble 3D scenes at scale. - The library of assets + ability to sketch from procedural primitives gives the perfect balance between speed and freedom. - ShapesXR allows you to export your creation as a glTF that can be used as a "3D prompt" into Marble - Marble now allows you to create a panorama using a 3D scene as input increasing DRAMATICALLY the control you have during the creation process - Through the workflow you can adjust textures and objects in the scene using AI inpainting - There are tons of options on how to use the final output (create a #GaussianSplatting, export collision mesh and even a detailed mesh 🤯) This is the perfect examples of how VR creative tools improve and accelerate the design process... so make sure to follow me if that's your jam 😉.

Gabriele Romagnoli

20,126 Aufrufe • vor 7 Monaten

Today, we’re launching a new way to create with AI. With OpenArt Worlds, you can generate a fully navigable 3D environment from a single prompt or image, step inside it, and capture shots exactly the way you envision them. No more starting over. No more inconsistent scenes. You build the world once - and create inside it. • Move through your scene freely • Find your angles • Add characters and elements • Capture production-ready shots
0:53

Today, we’re launching a new way to create with AI. With OpenArt Worlds, you can generate a fully navigable 3D environment from a single prompt or image, step inside it, and capture shots exactly the way you envision them. No more starting over. No more inconsistent scenes. You build the world once - and create inside it. • Move through your scene freely • Find your angles • Add characters and elements • Capture production-ready shots

OpenArt

7,298,576 Aufrufe • vor 3 Monaten

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:47

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

49,920 Aufrufe • vor 6 Monaten

Evaluating robot policies is hard. Ideally, instead of testing every new policy on a real robot, you could test in simulation; but simulations rarely correlate well with real-world performance. In order to make good, useful simulations, you need to spend a great deal of time and effort. That’s where PolaRiS comes in: it’s a toolkit that lets you take a short video of a real scene and turn it into a high-fidelity simulation. It provides what you need to build a good evaluation environment, and it “ships” with off-the-shelf environments that already show strong sim-to-real correlation, meaning that they can be used to inform policy performance. Arhan Jain and Karl Pertsch join us to talk about what they have built, why, and how you can use it. Watch Episode #62 of RoboPapers, with Chris Paxton and Jiafei Duan, now!
59:36

Evaluating robot policies is hard. Ideally, instead of testing every new policy on a real robot, you could test in simulation; but simulations rarely correlate well with real-world performance. In order to make good, useful simulations, you need to spend a great deal of time and effort. That’s where PolaRiS comes in: it’s a toolkit that lets you take a short video of a real scene and turn it into a high-fidelity simulation. It provides what you need to build a good evaluation environment, and it “ships” with off-the-shelf environments that already show strong sim-to-real correlation, meaning that they can be used to inform policy performance. Arhan Jain and Karl Pertsch join us to talk about what they have built, why, and how you can use it. Watch Episode #62 of RoboPapers, with Chris Paxton and Jiafei Duan, now!

RoboPapers

14,689 Aufrufe • vor 4 Monaten

🧍 Very cool idea So I added it to Photo AI as a T-Pose photo pack you can use to create a moving (or dancing) 3d model of yourself It auto-generates the correct T-Pose of your model immediately and then converts it into a 3D model you can download (as .fbx) and instantly import into @mixamo to rig it After rigging you can add any movement from Mixamo to move it however you want
1:28

🧍 Very cool idea So I added it to Photo AI as a T-Pose photo pack you can use to create a moving (or dancing) 3d model of yourself It auto-generates the correct T-Pose of your model immediately and then converts it into a 3D model you can download (as .fbx) and instantly import into @mixamo to rig it After rigging you can add any movement from Mixamo to move it however you want

@levelsio

139,521 Aufrufe • vor 5 Monaten

Humanoid robots that can barely grab a cup after 7th attempt. Meanwhile: specialized robots that have been dominating their specific task since 2008. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:27

Humanoid robots that can barely grab a cup after 7th attempt. Meanwhile: specialized robots that have been dominating their specific task since 2008. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

15,338 Aufrufe • vor 20 Tagen

Even your most boring footage can become a playground with Runway Aleph. Add to it. Alter it. Or, add a seagull to it. And all you need to do is tell Aleph what you want.
0:45

Even your most boring footage can become a playground with Runway Aleph. Add to it. Alter it. Or, add a seagull to it. And all you need to do is tell Aleph what you want.

Jamie

23,537 Aufrufe • vor 10 Monaten