Загрузка видео...

Не удалось загрузить видео

На главную

Fun weekend project: using AI at Meta Sam3 to segment objects inside a SpAItial AI scene. Pipeline: -> Generate a world on SpAItial AI app and export as .ply -> PlayCanvas viewer to render splat + a uniform spatial grid over splat centers -> Capture frames + camera pose,...

risingodegua's profile picture

RO2⚡️

8,846 subscribers

22,060 просмотров • 2 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

This is Segment Anything for #GaussianSplatting running on the web... but it comes with a caveat. This is SplatEdit, a powerful #3DGS editing software built on top of SuperSplat. Last week, it added a segmentation selection feature powered by Segment Anything, running locally in the browser. Just upload a splat, click on the "mug" icon and select entire objects in the scene. The problem is that everything after the object is also selected (as you can see in the video). To fix that, just use the brush tool to subtract the unwanted portion of the scene (hold CTRL while brushing). I feel we are getting closer and closer to having more editing control on these amazing fuzzy clouds 😉
0:37

This is Segment Anything for #GaussianSplatting running on the web... but it comes with a caveat. This is SplatEdit, a powerful #3DGS editing software built on top of SuperSplat. Last week, it added a segmentation selection feature powered by Segment Anything, running locally in the browser. Just upload a splat, click on the "mug" icon and select entire objects in the scene. The problem is that everything after the object is also selected (as you can see in the video). To fix that, just use the brush tool to subtract the unwanted portion of the scene (hold CTRL while brushing). I feel we are getting closer and closer to having more editing control on these amazing fuzzy clouds 😉

Gabriele Romagnoli

14,630 просмотров • 23 дней назад

In Uniform you can use multiple viewport panels to design the final render views for a project presentation. All viewports can have independent shading modes and allow scene and camera settings editing. This demo is running on a 2018 3rd gen iPad Pro.
0:54

In Uniform you can use multiple viewport panels to design the final render views for a project presentation. All viewports can have independent shading modes and allow scene and camera settings editing. This demo is running on a 2018 3rd gen iPad Pro.

Sparseal

64,594 просмотров • 2 лет назад

📢 OneCanvas: 3D Scene Understanding via Panoramic Reprojection We extract features from video frames and reproject them into one occlusion-free view of the whole scene that a 2D VLM reads just like a normal image. We can center this view on any viewpoint, including an agent's own pose for situated reasoning. The same projection lets us create spatial training tasks with no human annotation, solvable only by reasoning over the 3D positions of real object features placed on an otherwise empty canvas. The result is a stock 2D VLM that reasons in 3D, setting a new state of the art across spatial benchmarks at far less compute. 🌐 ▶️ Great work by Bartłomiej Baranowski & Dave Zhenyu Chen
0:44

📢 OneCanvas: 3D Scene Understanding via Panoramic Reprojection We extract features from video frames and reproject them into one occlusion-free view of the whole scene that a 2D VLM reads just like a normal image. We can center this view on any viewpoint, including an agent's own pose for situated reasoning. The same projection lets us create spatial training tasks with no human annotation, solvable only by reasoning over the 3D positions of real object features placed on an otherwise empty canvas. The result is a stock 2D VLM that reasons in 3D, setting a new state of the art across spatial benchmarks at far less compute. 🌐 ▶️ Great work by Bartłomiej Baranowski & Dave Zhenyu Chen

Matthias Niessner

25,169 просмотров • 1 месяц назад

Omg, this is such a hack for vibecoding mobile apps This is an image with a video on the top. The video is a character with no backround. I'm using: > Masko AI to generate the video > vibecode.dev to build the app In masko all you need to do is provide an image, and it will give you a video without a background. And then all you need to do is give it to vibecode.dev which is the best mobile app builder.
0:25

Omg, this is such a hack for vibecoding mobile apps This is an image with a video on the top. The video is a character with no backround. I'm using: > Masko AI to generate the video > vibecode.dev to build the app In masko all you need to do is provide an image, and it will give you a video without a background. And then all you need to do is give it to vibecode.dev which is the best mobile app builder.

Emily Lambert

29,840 просмотров • 6 месяцев назад

I still can't believe this works! Every filmmaker needs to try the hybrid AI filmmaking + Blender previs workflow for precise camera control. AI dialogue is still flat but I'm going to dive into solving that next. My workflow: 1. Generate a start frame in Midjourney 2. Block out the scene in Blender, animate the camera 3. Feed both to Seedance So much of cinematic storytelling is composition, timing, and giving the viewer a spatial understanding of the scene. You can solve all of that before you ever touch the video model.
0:36

I still can't believe this works! Every filmmaker needs to try the hybrid AI filmmaking + Blender previs workflow for precise camera control. AI dialogue is still flat but I'm going to dive into solving that next. My workflow: 1. Generate a start frame in Midjourney 2. Block out the scene in Blender, animate the camera 3. Feed both to Seedance So much of cinematic storytelling is composition, timing, and giving the viewer a spatial understanding of the scene. You can solve all of that before you ever touch the video model.

Reid Hannaford

17,266 просмотров • 1 месяц назад

This might be the craziest workflow you'll see this week. From #VR greyboxing to 3D worlds with the help of #AI. The 2 tools I used are ShapesXR and Marble from World Labs. Let me tell you why and how. - ShapesXR is the most powerful and fastest way to assemble 3D scenes at scale. - The library of assets + ability to sketch from procedural primitives gives the perfect balance between speed and freedom. - ShapesXR allows you to export your creation as a glTF that can be used as a "3D prompt" into Marble - Marble now allows you to create a panorama using a 3D scene as input increasing DRAMATICALLY the control you have during the creation process - Through the workflow you can adjust textures and objects in the scene using AI inpainting - There are tons of options on how to use the final output (create a #GaussianSplatting, export collision mesh and even a detailed mesh 🤯) This is the perfect examples of how VR creative tools improve and accelerate the design process... so make sure to follow me if that's your jam 😉.
1:01

This might be the craziest workflow you'll see this week. From #VR greyboxing to 3D worlds with the help of #AI. The 2 tools I used are ShapesXR and Marble from World Labs. Let me tell you why and how. - ShapesXR is the most powerful and fastest way to assemble 3D scenes at scale. - The library of assets + ability to sketch from procedural primitives gives the perfect balance between speed and freedom. - ShapesXR allows you to export your creation as a glTF that can be used as a "3D prompt" into Marble - Marble now allows you to create a panorama using a 3D scene as input increasing DRAMATICALLY the control you have during the creation process - Through the workflow you can adjust textures and objects in the scene using AI inpainting - There are tons of options on how to use the final output (create a #GaussianSplatting, export collision mesh and even a detailed mesh 🤯) This is the perfect examples of how VR creative tools improve and accelerate the design process... so make sure to follow me if that's your jam 😉.

Gabriele Romagnoli

20,126 просмотров • 8 месяцев назад

I'm excited to share the Geo Register Plugin for LichtFeld Studio from the LichtFeld community! This plugin helps bring Gaussian splat scenes into real-world geographic space. It registers a scene to WGS-84 and ECEF coordinates, so you can click any point on the model and get its latitude, longitude and altitude. It supports multiple georeferencing sources, including EXIF GPS data, image position CSVs, RealityScan camera parameters and saved similarity transforms. Once the scene is registered, you can export geo-referenced splat models as LAS, LAZ or 3D Tiles datasets for use in GIS and 3D mapping workflows. Built for anyone working with drone data, photogrammetry, Gaussian splatting, GIS, ArcGIS or CesiumJS. Link in the comment below!
0:22

I'm excited to share the Geo Register Plugin for LichtFeld Studio from the LichtFeld community! This plugin helps bring Gaussian splat scenes into real-world geographic space. It registers a scene to WGS-84 and ECEF coordinates, so you can click any point on the model and get its latitude, longitude and altitude. It supports multiple georeferencing sources, including EXIF GPS data, image position CSVs, RealityScan camera parameters and saved similarity transforms. Once the scene is registered, you can export geo-referenced splat models as LAS, LAZ or 3D Tiles datasets for use in GIS and 3D mapping workflows. Built for anyone working with drone data, photogrammetry, Gaussian splatting, GIS, ArcGIS or CesiumJS. Link in the comment below!

MrNeRF

51,761 просмотров • 2 месяцев назад

Another city. Another full 3D Gaussian Splat reconstruction. This entire neighborhood was captured while walking with an Insta360 X5, then processed with our GPU pipeline and transformed into a high-fidelity, machine-readable 3D environment with Over the Reality. What you’re seeing is not an isolated virtual world. It’s a 3D digital twin of a real-world location, built from structured spatial data where every building, street, and detail can become usable by AI, robotics, and XR. And it’s happening at scale, across cities in more than 180 countries every day. Have an Insta360 X5 and want to become a mapper? Apply here: Want to build a 3D Gaussian Splat from a video you already recorded? Do it for free at powered by Over the Reality.
2:42

Another city. Another full 3D Gaussian Splat reconstruction. This entire neighborhood was captured while walking with an Insta360 X5, then processed with our GPU pipeline and transformed into a high-fidelity, machine-readable 3D environment with Over the Reality. What you’re seeing is not an isolated virtual world. It’s a 3D digital twin of a real-world location, built from structured spatial data where every building, street, and detail can become usable by AI, robotics, and XR. And it’s happening at scale, across cities in more than 180 countries every day. Have an Insta360 X5 and want to become a mapper? Apply here: Want to build a 3D Gaussian Splat from a video you already recorded? Do it for free at powered by Over the Reality.

Over the Reality 🌐

400,097 просмотров • 3 месяцев назад

Fully local Code Assistant running on NVIDIA GPU! In this tutorial, I'll show you how to run Llama3 using TensorRT and Nvidia's Triton Inference Server to use it as a Code Assistant in VSCode In this thread 🧵, I'll walk you through the integration process, explaining each step simply and clearly 👇 NVIDIA AI NVIDIA AIDev AI at Meta
0:30

Fully local Code Assistant running on NVIDIA GPU! In this tutorial, I'll show you how to run Llama3 using TensorRT and Nvidia's Triton Inference Server to use it as a Code Assistant in VSCode In this thread 🧵, I'll walk you through the integration process, explaining each step simply and clearly 👇 NVIDIA AI NVIDIA AIDev AI at Meta

Daniel San

42,154 просмотров • 2 лет назад

NVIDIA AI Open-Sources ViPE (Video Pose Engine): A Powerful and Versatile 3D Video Annotation Tool for Spatial AI ViPE integrates bundle adjustment with dense optical flow, sparse keypoint tracking, and metric depth priors to estimate camera intrinsics, poses, and dense depth maps at 3–5 FPS on a single GPU. It significantly improves over prior uncalibrated pose estimation methods, achieving 18% and 50% error reduction on TUM and KITTI benchmarks, respectively, and shows robustness to dynamic scenes and diverse camera models. Beyond the method, the NVIDIA team also released a large-scale dataset comprising ~100K real-world internet videos, 1M AI-generated videos, and 2K panoramic videos (≈96M frames) annotated with metric depth and poses. This dataset and engine aim to accelerate training for spatial AI tasks such as 3D reconstruction, video generation, and robotics.... full analysis: paper: codes: NVIDIA NVIDIA AI NVIDIAnewsroom NVIDIA Robotics NVIDIA AIDev NVIDIAdeveloper
1:13

NVIDIA AI Open-Sources ViPE (Video Pose Engine): A Powerful and Versatile 3D Video Annotation Tool for Spatial AI ViPE integrates bundle adjustment with dense optical flow, sparse keypoint tracking, and metric depth priors to estimate camera intrinsics, poses, and dense depth maps at 3–5 FPS on a single GPU. It significantly improves over prior uncalibrated pose estimation methods, achieving 18% and 50% error reduction on TUM and KITTI benchmarks, respectively, and shows robustness to dynamic scenes and diverse camera models. Beyond the method, the NVIDIA team also released a large-scale dataset comprising ~100K real-world internet videos, 1M AI-generated videos, and 2K panoramic videos (≈96M frames) annotated with metric depth and poses. This dataset and engine aim to accelerate training for spatial AI tasks such as 3D reconstruction, video generation, and robotics.... full analysis: paper: codes: NVIDIA NVIDIA AI NVIDIAnewsroom NVIDIA Robotics NVIDIA AIDev NVIDIAdeveloper

Marktechpost AI Dev News ⚡

217,453 просмотров • 10 месяцев назад

Excited to share a new vibe coded prototype: an interactive lesson on ants and their habitat, designed for elementary school students. AI allowed me to finalize this demo quickly. Some educational details still need review, but the main chambers of an anthill are all represented. Everything was made in Omma AI , a Spline product. What's inside: - 3D models created with Omma, all I needed was one image of the anthill's exterior and another of its cross-section to generate both 3D models - Clouds done with sprites - A custom shader mask to reveal the inside of the anthill This was a really fun one to work on!
0:43

Excited to share a new vibe coded prototype: an interactive lesson on ants and their habitat, designed for elementary school students. AI allowed me to finalize this demo quickly. Some educational details still need review, but the main chambers of an anthill are all represented. Everything was made in Omma AI , a Spline product. What's inside: - 3D models created with Omma, all I needed was one image of the anthill's exterior and another of its cross-section to generate both 3D models - Clouds done with sprites - A custom shader mask to reveal the inside of the anthill This was a really fun one to work on!

Joseph Azar

19,853 просмотров • 2 месяцев назад

Finally solved all the big things needed to prove occlusion culling is viable in the Hyperfy engine. All culling happens directly at render time and instantly, we've taken over the Three.js render pipeline to use our spatial hierarchy and manually issue renders, postprocessing and shadow passes with both frustum and occlusion culling. Our pipeline now aligns much more closely to unity/unreal because we collect visible objects and issue custom instanced draws on the fly for only those objects every single frame. This is one of the biggest issues engines on the web like three, babylon and playcanvas have and imo the reason people coming from unity/unreal don't take them seriously. There's a lot to unpack here, maybe i'll make a thread soon.
0:50

Finally solved all the big things needed to prove occlusion culling is viable in the Hyperfy engine. All culling happens directly at render time and instantly, we've taken over the Three.js render pipeline to use our spatial hierarchy and manually issue renders, postprocessing and shadow passes with both frustum and occlusion culling. Our pipeline now aligns much more closely to unity/unreal because we collect visible objects and issue custom instanced draws on the fly for only those objects every single frame. This is one of the biggest issues engines on the web like three, babylon and playcanvas have and imo the reason people coming from unity/unreal don't take them seriously. There's a lot to unpack here, maybe i'll make a thread soon.

Ash

15,118 просмотров • 1 год назад

.Marc Andreessen 🇺🇸 says the winner of the US-China AI race could dictate what all 8 billion people on Earth use — but open source introduces a third possibility: "The stakes are basically: what is the world going to run on?" "In the long run, somebody's gonna win. And the world will either be running on American AI or be running on Chinese AI." "The open source thing is super fascinating because it throws a wrench into all of this. It raises a third possibility that neither the US nor China are going to be the platform — it may just be open source." Marc Andreessen with Jeetu Patel at Cisco AI Summit
3:41

.Marc Andreessen 🇺🇸 says the winner of the US-China AI race could dictate what all 8 billion people on Earth use — but open source introduces a third possibility: "The stakes are basically: what is the world going to run on?" "In the long run, somebody's gonna win. And the world will either be running on American AI or be running on Chinese AI." "The open source thing is super fascinating because it throws a wrench into all of this. It raises a third possibility that neither the US nor China are going to be the platform — it may just be open source." Marc Andreessen with Jeetu Patel at Cisco AI Summit

a16z

101,985 просмотров • 5 месяцев назад

The top question I got was does Halo support local LLM running on a Mac. Here's an example of the app using gemma-4-26b-a4b-qat via LM Studio running on a base M5/32GB. It's giving an average of 40 tok/s and in my brief testing is able to execute all tool calls correctly. Very usable with this app.
0:37

The top question I got was does Halo support local LLM running on a Mac. Here's an example of the app using gemma-4-26b-a4b-qat via LM Studio running on a base M5/32GB. It's giving an average of 40 tok/s and in my brief testing is able to execute all tool calls correctly. Very usable with this app.

Tanmay

11,682 просмотров • 1 месяц назад

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision
0:40

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision

Jonathan Stephens

30,663 просмотров • 1 год назад

You can run local AI models up to 120B without a $10,000 Mac Studio This phone-sized device is your own server for open-source models. 100% local and private. And you can use it to: - Power an agent like OpenClaw 24/7 - Completely replace a chatbot - Literally anything that requires an API It’s called Tiiny, and, once again, you can run the latest open-source models on it. Link below
16:10

You can run local AI models up to 120B without a $10,000 Mac Studio This phone-sized device is your own server for open-source models. 100% local and private. And you can use it to: - Power an agent like OpenClaw 24/7 - Completely replace a chatbot - Literally anything that requires an API It’s called Tiiny, and, once again, you can run the latest open-source models on it. Link below

Paul Couvert

41,245 просмотров • 4 месяцев назад

Look at how crispy this 3d gaussian splat came out! The reflections in the painting have me floored. Captured with a Sony a7iii camera and 18mm lens. I posed the imagery using epic's reality capture and trained it with lichtfeld studio. Lichtfeld is my new favorite way to train splats -- it integrates all the latest research and it’s completely free. Also what I love about 3d capture -- you can process old datasets with new algorithms and breathe new life into them.
0:27

Look at how crispy this 3d gaussian splat came out! The reflections in the painting have me floored. Captured with a Sony a7iii camera and 18mm lens. I posed the imagery using epic's reality capture and trained it with lichtfeld studio. Lichtfeld is my new favorite way to train splats -- it integrates all the latest research and it’s completely free. Also what I love about 3d capture -- you can process old datasets with new algorithms and breathe new life into them.

Bilawal Sidhu

78,613 просмотров • 5 месяцев назад

✨ Personal AI is the next computing platform. AI is shifting from something you access to something you build with, locally, at the edge, and across systems. We’re unlocking new possibilities for developers: • AMD Ryzen AI Halo, a local-first developer system, preorder starting in June, develop AI without limits on your desk • Gorgon Halo with up to 192GB unified memory, supporting 300B+ parameter models locally, run massive models locally We’re excited to partner with clem 🤗🤗, Co-founder and CEO of Hugging Face, to advance open-source AI for Ryzen AI. Our focus is seamless AI, from model to deployment. Cloud, edge, device. One continuum. Multi-agent systems, local inference at scale, open models as infrastructure. This is the next computing era 🚀
4:49

✨ Personal AI is the next computing platform. AI is shifting from something you access to something you build with, locally, at the edge, and across systems. We’re unlocking new possibilities for developers: • AMD Ryzen AI Halo, a local-first developer system, preorder starting in June, develop AI without limits on your desk • Gorgon Halo with up to 192GB unified memory, supporting 300B+ parameter models locally, run massive models locally We’re excited to partner with clem 🤗🤗, Co-founder and CEO of Hugging Face, to advance open-source AI for Ryzen AI. Our focus is seamless AI, from model to deployment. Cloud, edge, device. One continuum. Multi-agent systems, local inference at scale, open models as infrastructure. This is the next computing era 🚀

Jack Huynh

47,822 просмотров • 2 месяцев назад

From 2 second (faster on higher end GPUs) Gaussian splat generations, using Apple's "Sharp Monocular View Synthesis in Less Than a Second" code, running locally on my system. Imported into Octane Render 2026 which has fully path traced rendering of Gaussian splats. Stuff like this is why I've been so excited to see where Gaussian splatting research would go. Combined with a real-time Generative AI "renderer" this stuff could start to get quite insane. Combined with Dreams it would be even better ofc :D Code to the Apple code in the comments below.
0:33

From 2 second (faster on higher end GPUs) Gaussian splat generations, using Apple's "Sharp Monocular View Synthesis in Less Than a Second" code, running locally on my system. Imported into Octane Render 2026 which has fully path traced rendering of Gaussian splats. Stuff like this is why I've been so excited to see where Gaussian splatting research would go. Combined with a real-time Generative AI "renderer" this stuff could start to get quite insane. Combined with Dreams it would be even better ofc :D Code to the Apple code in the comments below.

Martin Nebelong

36,774 просмотров • 7 месяцев назад