Jonathan Stephens's banner

Jonathan Stephens

@jonstephens85 • 13,137 subscribers

Synthetc Data | World Models | Radiance Fields | Computer Vision | GenAI | Chief Evangelist at @LightwheelAI

Shorts

More tests last night with NVIDIA AI Developer's GEN3C. This was all generated from a single photo using local compute. I love how it accurately simulated the light fade around the building corner. Next up, dynamic scenes. #3D #GenAI #Computervision

More tests last night with NVIDIA AI Developer's GEN3C. This was all generated from a single photo using local compute. I love how it accurately simulated the light fade around the building corner. Next up, dynamic scenes. #3D #GenAI #Computervision

84,840 просмотров

What if you could turn a single 360° photo into a production-ready Isaac Sim environment in minutes? That's exactly what we did here. Using World Labs' Marble and an Insta360 X5 capture (rotating on top), we generated a complete navigable 3D environment and populated it with Lightwheel Sim Ready assets (bottom view). The result? A fully interactive scene in Isaac Sim, ready for sim2real testing,. Navigation, manipulation, or any robotics task you need to validate. What used to take weeks of manual 3D modeling and asset placement now takes minutes. Capture once in the real world, simulate everywhere in your training pipeline. This is the future of robotics development with world models. NVIDIA Robotics NVIDIA Omniverse #Sim2Real #Robotics #Simulation

What if you could turn a single 360° photo into a production-ready Isaac Sim environment in minutes? That's exactly what we did here. Using World Labs' Marble and an Insta360 X5 capture (rotating on top), we generated a complete navigable 3D environment and populated it with Lightwheel Sim Ready assets (bottom view). The result? A fully interactive scene in Isaac Sim, ready for sim2real testing,. Navigation, manipulation, or any robotics task you need to validate. What used to take weeks of manual 3D modeling and asset placement now takes minutes. Capture once in the real world, simulate everywhere in your training pipeline. This is the future of robotics development with world models. NVIDIA Robotics NVIDIA Omniverse #Sim2Real #Robotics #Simulation

46,338 просмотров

More beautiful triangle splatting. Wait a couple seconds for the camera to swing around in the direction of the source images.

More beautiful triangle splatting. Wait a couple seconds for the camera to swing around in the direction of the source images.

64,396 просмотров

Triangle splatting. This is no joke!!! Metrics at 30k iters SSIM- 0.9477584958076477 PSNR - 32.29475784301758 LPIPS - 0.06017586588859558

Triangle splatting. This is no joke!!! Metrics at 30k iters SSIM- 0.9477584958076477 PSNR - 32.29475784301758 LPIPS - 0.06017586588859558

34,437 просмотров

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

17,712 просмотров

I finally got NVIDIA AI Developer's 3D Gaussian Unscented Transforms up and running natively on Windows via gsplat. Just in time for me to make a tutorial so people can enter NVIDIA's 3DGUT sweepstakes!!! Huge thanks to Ruilong Li for responding to my Github issues and solving my roadblocks this evening. #3D #Computervision

I finally got NVIDIA AI Developer's 3D Gaussian Unscented Transforms up and running natively on Windows via gsplat. Just in time for me to make a tutorial so people can enter NVIDIA's 3DGUT sweepstakes!!! Huge thanks to Ruilong Li for responding to my Github issues and solving my roadblocks this evening. #3D #Computervision

16,901 просмотров

SpatialLM. Fast, not real-time. So much to explore here. I see the potential. More coming soon. #AI #AEC

SpatialLM. Fast, not real-time. So much to explore here. I see the potential. More coming soon. #AI #AEC

14,981 просмотров

Check out the details difference between NVIDIA Cosmos 7B and 14B models! Watch the whole video to see each video separate. Twice the parameters makes a huge difference! What do you think? #GenerativeAI #AI #ArtificialIntelligence #Computervision

Check out the details difference between NVIDIA Cosmos 7B and 14B models! Watch the whole video to see each video separate. Twice the parameters makes a huge difference! What do you think? #GenerativeAI #AI #ArtificialIntelligence #Computervision

14,745 просмотров

3DGS needs to fix this if they want to be used for virtual tours: Gaussian splats struggle with ceilings. NeRFs ALWAYS beat out quality in this respect. I captured this over the weekend to compare. I have ideas on how to fix it. Anyone else have success with ceilings?

3DGS needs to fix this if they want to be used for virtual tours: Gaussian splats struggle with ceilings. NeRFs ALWAYS beat out quality in this respect. I captured this over the weekend to compare. I have ideas on how to fix it. Anyone else have success with ceilings?

12,340 просмотров

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

This is by far the best 3DGS scan of my kitchen. I love how the light reflects on my dining room table from the bay windows. I still struggle to be able to deal with the high dynamic range between the inside and outside though. The windows are overexposed which means you can't see anything outside well. Does anyone have a good solution to this? Shoot raw and globally adjust? I would love to share notes with anyone working on this challenge at #CVPR2026 #3DGS #ComputerVision

This is by far the best 3DGS scan of my kitchen. I love how the light reflects on my dining room table from the bay windows. I still struggle to be able to deal with the high dynamic range between the inside and outside though. The windows are overexposed which means you can't see anything outside well. Does anyone have a good solution to this? Shoot raw and globally adjust? I would love to share notes with anyone working on this challenge at #CVPR2026 #3DGS #ComputerVision

Jonathan Stephens

21,622 просмотров • 4 дней назад

This is NOT a mesh, NOT a NeRF, NOT video footage. This is 3D Gaussian Splatting which produces a similar output to NeRFs. Check out those reflections in the water! The fine structure detail 🤩 #3D #AI #Computervision #VFX

This is NOT a mesh, NOT a NeRF, NOT video footage. This is 3D Gaussian Splatting which produces a similar output to NeRFs. Check out those reflections in the water! The fine structure detail 🤩 #3D #AI #Computervision #VFX

Jonathan Stephens

170,744 просмотров • 2 лет назад

More beautiful triangle splatting. Wait a couple seconds for the camera to swing around in the direction of the source images.

More beautiful triangle splatting. Wait a couple seconds for the camera to swing around in the direction of the source images.

Jonathan Stephens

64,396 просмотров • 1 год назад

Do you think you could train a robot to fold clothes better than any other robotics team? Now is your chance to prove it...and win actual cash prizes! The 2026 LeHome Challenge is live! Here is the details: Phase 1: Train your best clothes folding policy in simulation with 4 garment types, each presenting unique manipulation challenges. The types of garments include long-sleeved tops, short-sleeved tops, long pants, and shorts. The simulation environment uses Lightwheel's physics accurate deformable assets and built on NVIDIA Omniverse's Isaac Sim/Lab. Submit your code by April 5th for a chance to win up to $2,000 and a SO-101 Arm Kit. Phase 2: The top 8 teams will compete in person at ICRA on June 4th to see who's policy performs best with real clothes. The top place team will take home $20,000! Learn more and sign up for the LeHome Challenge here: Robert Scoble Chris Paxton The Humanoid Hub Harrison Kinsley #Robotics

Do you think you could train a robot to fold clothes better than any other robotics team? Now is your chance to prove it...and win actual cash prizes! The 2026 LeHome Challenge is live! Here is the details: Phase 1: Train your best clothes folding policy in simulation with 4 garment types, each presenting unique manipulation challenges. The types of garments include long-sleeved tops, short-sleeved tops, long pants, and shorts. The simulation environment uses Lightwheel's physics accurate deformable assets and built on NVIDIA Omniverse's Isaac Sim/Lab. Submit your code by April 5th for a chance to win up to $2,000 and a SO-101 Arm Kit. Phase 2: The top 8 teams will compete in person at ICRA on June 4th to see who's policy performs best with real clothes. The top place team will take home $20,000! Learn more and sign up for the LeHome Challenge here: Robert Scoble Chris Paxton The Humanoid Hub Harrison Kinsley #Robotics

Jonathan Stephens

24,003 просмотров • 3 месяцев назад

100M Gaussians. Streamed on a single RTX 3090. Welcome to NVIDIA AI Developer 's fVDB Reality Capture — bringing 3D Gaussian Splatting, volumetric data, and OpenUSD into one pipeline. 🎥 Full demo → #NVIDIA #GaussianSplatting #3D

100M Gaussians. Streamed on a single RTX 3090. Welcome to NVIDIA AI Developer 's fVDB Reality Capture — bringing 3D Gaussian Splatting, volumetric data, and OpenUSD into one pipeline. 🎥 Full demo → #NVIDIA #GaussianSplatting #3D

Jonathan Stephens

36,725 просмотров • 6 месяцев назад

The team at NVIDIA AI Developer really nailed it with ViPE!! This SLAM system generated pose, depth, point clouds, etc, in a few seconds! Now to get the output to work with GEN3C... If you are interested in running this yourself, it took about 5 minutes to get up and running from scratch.

The team at NVIDIA AI Developer really nailed it with ViPE!! This SLAM system generated pose, depth, point clouds, etc, in a few seconds! Now to get the output to work with GEN3C... If you are interested in running this yourself, it took about 5 minutes to get up and running from scratch.

Jonathan Stephens

34,835 просмотров • 9 месяцев назад

This is the most Disney thing I have ever seen when it comes to robotics! They trained robots to fall whimsically and in style. Why? Because not only does it protect its components, it looks cool! Watch the full video here: #Robotics #ReinforcementLearning #PhysicalAI

This is the most Disney thing I have ever seen when it comes to robotics! They trained robots to fall whimsically and in style. Why? Because not only does it protect its components, it looks cool! Watch the full video here: #Robotics #ReinforcementLearning #PhysicalAI

Jonathan Stephens

24,595 просмотров • 6 месяцев назад

Triangle splatting. This is no joke!!! Metrics at 30k iters SSIM- 0.9477584958076477 PSNR - 32.29475784301758 LPIPS - 0.06017586588859558

Triangle splatting. This is no joke!!! Metrics at 30k iters SSIM- 0.9477584958076477 PSNR - 32.29475784301758 LPIPS - 0.06017586588859558

Jonathan Stephens

34,437 просмотров • 1 год назад

Video world models look great but often ignore physics and 3D geometry. New research achieves a 64% reduction in navigation error by teaching AI to prioritize physical reality over visual "eye-candy". Here’s how they did it: #GenerativeAI #Robotics [1/n]

Video world models look great but often ignore physics and 3D geometry. New research achieves a 64% reduction in navigation error by teaching AI to prioritize physical reality over visual "eye-candy". Here’s how they did it: #GenerativeAI #Robotics [1/n]

Jonathan Stephens

15,621 просмотров • 4 месяцев назад

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision

Jonathan Stephens

30,617 просмотров • 1 год назад

I’ve rebuilt my kitchen with photogrammetry, gaussian splats, LiDAR, and just about any other method you could think of...so I had to see what World Labs' Marble could do with only 8 photos. Here’s a first look at what worked, and what didn’t. Link to full video: #3D #GenerativeAI #AEC #AI

I’ve rebuilt my kitchen with photogrammetry, gaussian splats, LiDAR, and just about any other method you could think of...so I had to see what World Labs' Marble could do with only 8 photos. Here’s a first look at what worked, and what didn’t. Link to full video: #3D #GenerativeAI #AEC #AI

Jonathan Stephens

16,495 просмотров • 6 месяцев назад

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX

The greatest Gaussian Splatting VR demo ever? Captured live at NVIDIA GTC — Mariya Sha experiencing 4D Gaussian Splatting in VR for the very first time. You can tell just from her reaction that this was something else! The scene was optimized using QUEEN, NVIDIA AI Developer's advanced research project for fast radiance field rendering. It’s an incredible look at what happens when high fidelity static and dynamic gaussian splats meets VR. Watch the full video on YouTube: Robert Scoble Did you get to do this one? #VR #3D #VFX

Jonathan Stephens

28,114 просмотров • 1 год назад

Big news! World Labs' Marble is officially compatible with NVIDIA Robotics' Isaac Sim. Instead of showcasing their highlight reel, our team at Lightwheel created a full environment from a 360 pano using Marble and showed you can train directly in sim. The only non-generated assets are the robot arms, coffee, and bowl. Everything else was generated with Marble. Not sure how to do this yourself? NVIDIA wrote a technical blog with full instructions. You will learn how to: 1. Export scenes from World Labs' Marble as Gaussian splats 2. Convert to USD using NVIDIA Omniverse NuRec 3. Import into NVIDIA Isaac Sim 4. Add a robot and run the simulation Read the guide ➡️ #WorldModels #Robotics

Big news! World Labs' Marble is officially compatible with NVIDIA Robotics' Isaac Sim. Instead of showcasing their highlight reel, our team at Lightwheel created a full environment from a 360 pano using Marble and showed you can train directly in sim. The only non-generated assets are the robot arms, coffee, and bowl. Everything else was generated with Marble. Not sure how to do this yourself? NVIDIA wrote a technical blog with full instructions. You will learn how to: 1. Export scenes from World Labs' Marble as Gaussian splats 2. Convert to USD using NVIDIA Omniverse NuRec 3. Import into NVIDIA Isaac Sim 4. Add a robot and run the simulation Read the guide ➡️ #WorldModels #Robotics

Jonathan Stephens

13,783 просмотров • 5 месяцев назад

Is Voxels the future of 3D Gaussian Splats? 3DGS's days may be numbered... I dove into NVIDIA AI Developer's SVRaster over the past week and made a first impressions video. Link to full video: #computervision #3d #3DGS #VFX

Is Voxels the future of 3D Gaussian Splats? 3DGS's days may be numbered... I dove into NVIDIA AI Developer's SVRaster over the past week and made a first impressions video. Link to full video: #computervision #3d #3DGS #VFX

Jonathan Stephens

20,025 просмотров • 1 год назад

This is not 3D Gaussian Splatting, it's deformable beta splatting! Thanks to Rong Liu for letting me know code was live. Congrats to the whole team at USC! #3D #Radiancefield

This is not 3D Gaussian Splatting, it's deformable beta splatting! Thanks to Rong Liu for letting me know code was live. Congrats to the whole team at USC! #3D #Radiancefield

Jonathan Stephens

16,207 просмотров • 1 год назад

Now you can recreate your kid's messy playroom using NVIDIA AI Developer new ray tracing based gaussian splatting (3DGUT) on Windows! Follow my step by step tutorial and then enter their sweepstakes to win a free GPU!!! That's right, NVIDIA is running a sweepstakes for developers and creators using their new 3DGRUT project now available on gsplat. Learn more here: Watch my official tutorial on YouTube: #3D #AI #NVIDAResearch #AEC

Now you can recreate your kid's messy playroom using NVIDIA AI Developer new ray tracing based gaussian splatting (3DGUT) on Windows! Follow my step by step tutorial and then enter their sweepstakes to win a free GPU!!! That's right, NVIDIA is running a sweepstakes for developers and creators using their new 3DGRUT project now available on gsplat. Learn more here: Watch my official tutorial on YouTube: #3D #AI #NVIDAResearch #AEC

Jonathan Stephens

15,455 просмотров • 1 год назад

Fun fact: 11% of the 20M hours of data that trained NVIDIA AI Developer Cosmos was driving footage. I put it to the test. Here are some of my favorites. I have a guide for generating videos locally with Cosmos. Watch it here: #AI #GenerativeAI

Fun fact: 11% of the 20M hours of data that trained NVIDIA AI Developer Cosmos was driving footage. I put it to the test. Here are some of my favorites. I have a guide for generating videos locally with Cosmos. Watch it here: #AI #GenerativeAI

Jonathan Stephens

12,386 просмотров • 1 год назад

NVIDIA seems to be doing the impossible here! I break down how Difix3D+ is fixing bad novel views using diffusion techniques. Plus, if you watch till the end you learn how to enter my RTX 4080 GPU Giveaway! Difix3D+ Project Page: Giveaway details in 🧵

NVIDIA seems to be doing the impossible here! I break down how Difix3D+ is fixing bad novel views using diffusion techniques. Plus, if you watch till the end you learn how to enter my RTX 4080 GPU Giveaway! Difix3D+ Project Page: Giveaway details in 🧵

Jonathan Stephens

10,987 просмотров • 1 год назад

Больше нет контента для загрузки