Jonathan Stephens's banner
Jonathan Stephens's profile picture

Jonathan Stephens

@jonstephens85 • 12,548 subscribers

Synthetc Data | World Models | Radiance Fields | Computer Vision | GenAI | Chief Evangelist at @LightwheelAI

Shorts

More tests last night with NVIDIA AI Developer's GEN3C. This was all generated from a single photo using local compute. I love how it accurately simulated the light fade around the building corner. Next up, dynamic scenes. #3D #GenAI #Computervision

More tests last night with NVIDIA AI Developer's GEN3C. This was all generated from a single photo using local compute. I love how it accurately simulated the light fade around the building corner. Next up, dynamic scenes. #3D #GenAI #Computervision

84,840 views

What if you could turn a single 360° photo into a production-ready Isaac Sim environment in minutes? That's exactly what we did here. Using World Labs' Marble and an Insta360 X5 capture (rotating on top), we generated a complete navigable 3D environment and populated it with Lightwheel Sim Ready assets (bottom view). The result? A fully interactive scene in Isaac Sim, ready for sim2real testing,. Navigation, manipulation, or any robotics task you need to validate. What used to take weeks of manual 3D modeling and asset placement now takes minutes. Capture once in the real world, simulate everywhere in your training pipeline. This is the future of robotics development with world models. NVIDIA Robotics NVIDIA Omniverse #Sim2Real #Robotics #Simulation

What if you could turn a single 360° photo into a production-ready Isaac Sim environment in minutes? That's exactly what we did here. Using World Labs' Marble and an Insta360 X5 capture (rotating on top), we generated a complete navigable 3D environment and populated it with Lightwheel Sim Ready assets (bottom view). The result? A fully interactive scene in Isaac Sim, ready for sim2real testing,. Navigation, manipulation, or any robotics task you need to validate. What used to take weeks of manual 3D modeling and asset placement now takes minutes. Capture once in the real world, simulate everywhere in your training pipeline. This is the future of robotics development with world models. NVIDIA Robotics NVIDIA Omniverse #Sim2Real #Robotics #Simulation

46,338 views

More beautiful triangle splatting. Wait a couple seconds for the camera to swing around in the direction of the source images.

More beautiful triangle splatting. Wait a couple seconds for the camera to swing around in the direction of the source images.

64,396 views

Triangle splatting. This is no joke!!! Metrics at 30k iters SSIM- 0.9477584958076477 PSNR - 32.29475784301758 LPIPS - 0.06017586588859558

Triangle splatting. This is no joke!!! Metrics at 30k iters SSIM- 0.9477584958076477 PSNR - 32.29475784301758 LPIPS - 0.06017586588859558

34,437 views

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

17,712 views

I finally got NVIDIA AI Developer's 3D Gaussian Unscented Transforms up and running natively on Windows via gsplat. Just in time for me to make a tutorial so people can enter NVIDIA's 3DGUT sweepstakes!!! Huge thanks to Ruilong Li for responding to my Github issues and solving my roadblocks this evening. #3D #Computervision

I finally got NVIDIA AI Developer's 3D Gaussian Unscented Transforms up and running natively on Windows via gsplat. Just in time for me to make a tutorial so people can enter NVIDIA's 3DGUT sweepstakes!!! Huge thanks to Ruilong Li for responding to my Github issues and solving my roadblocks this evening. #3D #Computervision

16,901 views

SpatialLM. Fast, not real-time. So much to explore here. I see the potential. More coming soon. #AI #AEC

SpatialLM. Fast, not real-time. So much to explore here. I see the potential. More coming soon. #AI #AEC

14,981 views

Check out the details difference between NVIDIA Cosmos 7B and 14B models! Watch the whole video to see each video separate. Twice the parameters makes a huge difference! What do you think? #GenerativeAI #AI #ArtificialIntelligence #Computervision

Check out the details difference between NVIDIA Cosmos 7B and 14B models! Watch the whole video to see each video separate. Twice the parameters makes a huge difference! What do you think? #GenerativeAI #AI #ArtificialIntelligence #Computervision

14,745 views

3DGS needs to fix this if they want to be used for virtual tours: Gaussian splats struggle with ceilings. NeRFs ALWAYS beat out quality in this respect. I captured this over the weekend to compare. I have ideas on how to fix it. Anyone else have success with ceilings?

3DGS needs to fix this if they want to be used for virtual tours: Gaussian splats struggle with ceilings. NeRFs ALWAYS beat out quality in this respect. I captured this over the weekend to compare. I have ideas on how to fix it. Anyone else have success with ceilings?

12,340 views

Videos

LIVE
1.2k

Anya Rossi

sweetdream.ai

SweetDream.aiSponsored•Livecam
Streaming Now

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

HD live stream
Exclusive private shows
1.2k viewers online

Free preview available • Premium content

This is NOT a mesh, NOT a NeRF, NOT video footage. This is 3D Gaussian Splatting which produces a similar output to NeRFs. Check out those reflections in the water! The fine structure detail 🤩 #3D #AI #Computervision #VFX
0:49
jonstephens85's profile picture

This is NOT a mesh, NOT a NeRF, NOT video footage. This is 3D Gaussian Splatting which produces a similar output to NeRFs. Check out those reflections in the water! The fine structure detail 🤩 #3D #AI #Computervision #VFX

Jonathan Stephens

170,744 views • 2 years ago

More beautiful triangle splatting. Wait a couple seconds for the camera to swing around in the direction of the source images.
0:20
jonstephens85's profile picture

More beautiful triangle splatting. Wait a couple seconds for the camera to swing around in the direction of the source images.

Jonathan Stephens

64,396 views • 1 year ago

Do you think you could train a robot to fold clothes better than any other robotics team? Now is your chance to prove it...and win actual cash prizes! The 2026 LeHome Challenge is live! Here is the details: Phase 1: Train your best clothes folding policy in simulation with 4 garment types, each presenting unique manipulation challenges. The types of garments include long-sleeved tops, short-sleeved tops, long pants, and shorts. The simulation environment uses Lightwheel's physics accurate deformable assets and built on NVIDIA Omniverse's Isaac Sim/Lab. Submit your code by April 5th for a chance to win up to $2,000 and a SO-101 Arm Kit. Phase 2: The top 8 teams will compete in person at ICRA on June 4th to see who's policy performs best with real clothes. The top place team will take home $20,000! Learn more and sign up for the LeHome Challenge here: Robert Scoble Chris Paxton The Humanoid Hub Harrison Kinsley #Robotics
0:51
jonstephens85's profile picture

Do you think you could train a robot to fold clothes better than any other robotics team? Now is your chance to prove it...and win actual cash prizes! The 2026 LeHome Challenge is live! Here is the details: Phase 1: Train your best clothes folding policy in simulation with 4 garment types, each presenting unique manipulation challenges. The types of garments include long-sleeved tops, short-sleeved tops, long pants, and shorts. The simulation environment uses Lightwheel's physics accurate deformable assets and built on NVIDIA Omniverse's Isaac Sim/Lab. Submit your code by April 5th for a chance to win up to $2,000 and a SO-101 Arm Kit. Phase 2: The top 8 teams will compete in person at ICRA on June 4th to see who's policy performs best with real clothes. The top place team will take home $20,000! Learn more and sign up for the LeHome Challenge here: Robert Scoble Chris Paxton The Humanoid Hub Harrison Kinsley #Robotics

Jonathan Stephens

24,003 views • 3 months ago

100M Gaussians. Streamed on a single RTX 3090. Welcome to NVIDIA AI Developer 's fVDB Reality Capture — bringing 3D Gaussian Splatting, volumetric data, and OpenUSD into one pipeline. 🎥 Full demo → #NVIDIA #GaussianSplatting #3D
0:59
jonstephens85's profile picture

100M Gaussians. Streamed on a single RTX 3090. Welcome to NVIDIA AI Developer 's fVDB Reality Capture — bringing 3D Gaussian Splatting, volumetric data, and OpenUSD into one pipeline. 🎥 Full demo → #NVIDIA #GaussianSplatting #3D

Jonathan Stephens

36,725 views • 6 months ago

The team at NVIDIA AI Developer really nailed it with ViPE!! This SLAM system generated pose, depth, point clouds, etc, in a few seconds! Now to get the output to work with GEN3C... If you are interested in running this yourself, it took about 5 minutes to get up and running from scratch.
0:28
jonstephens85's profile picture

The team at NVIDIA AI Developer really nailed it with ViPE!! This SLAM system generated pose, depth, point clouds, etc, in a few seconds! Now to get the output to work with GEN3C... If you are interested in running this yourself, it took about 5 minutes to get up and running from scratch.

Jonathan Stephens

34,835 views • 9 months ago

This is the most Disney thing I have ever seen when it comes to robotics! They trained robots to fall whimsically and in style. Why? Because not only does it protect its components, it looks cool! Watch the full video here: #Robotics #ReinforcementLearning #PhysicalAI
0:26
jonstephens85's profile picture

This is the most Disney thing I have ever seen when it comes to robotics! They trained robots to fall whimsically and in style. Why? Because not only does it protect its components, it looks cool! Watch the full video here: #Robotics #ReinforcementLearning #PhysicalAI

Jonathan Stephens

24,595 views • 6 months ago

Triangle splatting. This is no joke!!! Metrics at 30k iters SSIM- 0.9477584958076477 PSNR - 32.29475784301758 LPIPS - 0.06017586588859558
0:20
jonstephens85's profile picture

Triangle splatting. This is no joke!!! Metrics at 30k iters SSIM- 0.9477584958076477 PSNR - 32.29475784301758 LPIPS - 0.06017586588859558

Jonathan Stephens

34,437 views • 1 year ago

Video world models look great but often ignore physics and 3D geometry. New research achieves a 64% reduction in navigation error by teaching AI to prioritize physical reality over visual "eye-candy". Here’s how they did it: #GenerativeAI #Robotics [1/n]
0:50
jonstephens85's profile picture

Video world models look great but often ignore physics and 3D geometry. New research achieves a 64% reduction in navigation error by teaching AI to prioritize physical reality over visual "eye-candy". Here’s how they did it: #GenerativeAI #Robotics [1/n]

Jonathan Stephens

15,621 views • 4 months ago

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision
0:40
jonstephens85's profile picture

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision

Jonathan Stephens

30,617 views • 1 year ago

I’ve rebuilt my kitchen with photogrammetry, gaussian splats, LiDAR, and just about any other method you could think of...so I had to see what World Labs' Marble could do with only 8 photos. Here’s a first look at what worked, and what didn’t. Link to full video: #3D #GenerativeAI #AEC #AI
0:33
jonstephens85's profile picture

I’ve rebuilt my kitchen with photogrammetry, gaussian splats, LiDAR, and just about any other method you could think of...so I had to see what World Labs' Marble could do with only 8 photos. Here’s a first look at what worked, and what didn’t. Link to full video: #3D #GenerativeAI #AEC #AI

Jonathan Stephens

16,495 views • 6 months ago

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX
0:47
jonstephens85's profile picture

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX

Jonathan Stephens

28,114 views • 1 year ago

Big news! World Labs' Marble is officially compatible with NVIDIA Robotics' Isaac Sim. Instead of showcasing their highlight reel, our team at Lightwheel created a full environment from a 360 pano using Marble and showed you can train directly in sim. The only non-generated assets are the robot arms, coffee, and bowl. Everything else was generated with Marble. Not sure how to do this yourself? NVIDIA wrote a technical blog with full instructions. You will learn how to: 1. Export scenes from World Labs' Marble as Gaussian splats 2. Convert to USD using NVIDIA Omniverse NuRec 3. Import into NVIDIA Isaac Sim 4. Add a robot and run the simulation Read the guide ➡️ #WorldModels #Robotics
0:32
jonstephens85's profile picture

Big news! World Labs' Marble is officially compatible with NVIDIA Robotics' Isaac Sim. Instead of showcasing their highlight reel, our team at Lightwheel created a full environment from a 360 pano using Marble and showed you can train directly in sim. The only non-generated assets are the robot arms, coffee, and bowl. Everything else was generated with Marble. Not sure how to do this yourself? NVIDIA wrote a technical blog with full instructions. You will learn how to: 1. Export scenes from World Labs' Marble as Gaussian splats 2. Convert to USD using NVIDIA Omniverse NuRec 3. Import into NVIDIA Isaac Sim 4. Add a robot and run the simulation Read the guide ➡️ #WorldModels #Robotics

Jonathan Stephens

13,783 views • 5 months ago

Is Voxels the future of 3D Gaussian Splats? 3DGS's days may be numbered... I dove into NVIDIA AI Developer's SVRaster over the past week and made a first impressions video. Link to full video: #computervision #3d #3DGS #VFX
0:58
jonstephens85's profile picture

Is Voxels the future of 3D Gaussian Splats? 3DGS's days may be numbered... I dove into NVIDIA AI Developer's SVRaster over the past week and made a first impressions video. Link to full video: #computervision #3d #3DGS #VFX

Jonathan Stephens

20,025 views • 1 year ago

This is not 3D Gaussian Splatting, it's deformable beta splatting! Thanks to Rong Liu for letting me know code was live. Congrats to the whole team at USC! #3D #Radiancefield
0:36
jonstephens85's profile picture

This is not 3D Gaussian Splatting, it's deformable beta splatting! Thanks to Rong Liu for letting me know code was live. Congrats to the whole team at USC! #3D #Radiancefield

Jonathan Stephens

16,207 views • 1 year ago

Now you can recreate your kid's messy playroom using NVIDIA AI Developer new ray tracing based gaussian splatting (3DGUT) on Windows! Follow my step by step tutorial and then enter their sweepstakes to win a free GPU!!! That's right, NVIDIA is running a sweepstakes for developers and creators using their new 3DGRUT project now available on gsplat. Learn more here: Watch my official tutorial on YouTube: #3D #AI #NVIDAResearch #AEC
0:38
jonstephens85's profile picture

Now you can recreate your kid's messy playroom using NVIDIA AI Developer new ray tracing based gaussian splatting (3DGUT) on Windows! Follow my step by step tutorial and then enter their sweepstakes to win a free GPU!!! That's right, NVIDIA is running a sweepstakes for developers and creators using their new 3DGRUT project now available on gsplat. Learn more here: Watch my official tutorial on YouTube: #3D #AI #NVIDAResearch #AEC

Jonathan Stephens

15,455 views • 1 year ago

Fun fact: 11% of the 20M hours of data that trained NVIDIA AI Developer Cosmos was driving footage. I put it to the test. Here are some of my favorites. I have a guide for generating videos locally with Cosmos. Watch it here: #AI #GenerativeAI
0:30
jonstephens85's profile picture

Fun fact: 11% of the 20M hours of data that trained NVIDIA AI Developer Cosmos was driving footage. I put it to the test. Here are some of my favorites. I have a guide for generating videos locally with Cosmos. Watch it here: #AI #GenerativeAI

Jonathan Stephens

12,386 views • 1 year ago

NVIDIA seems to be doing the impossible here! I break down how Difix3D+ is fixing bad novel views using diffusion techniques. Plus, if you watch till the end you learn how to enter my RTX 4080 GPU Giveaway! Difix3D+ Project Page: Giveaway details in 🧵
3:41
jonstephens85's profile picture

NVIDIA seems to be doing the impossible here! I break down how Difix3D+ is fixing bad novel views using diffusion techniques. Plus, if you watch till the end you learn how to enter my RTX 4080 GPU Giveaway! Difix3D+ Project Page: Giveaway details in 🧵

Jonathan Stephens

10,987 views • 1 year ago

No more content to load