Загрузка видео...

Не удалось загрузить видео

На главную

Fun weekend project: using AI at Meta Sam3 to segment objects inside a SpAItial AI scene. Pipeline: -> Generate a world on SpAItial AI app and export as .ply -> PlayCanvas viewer to render splat + a uniform spatial grid over splat centers -> Capture frames + camera pose,...

risingodegua's profile picture

RO2⚡️

8,885 subscribers

22,375 просмотров • 3 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

A fun way to try the new Claude Fable. I asked it to train a bot using RL to navigate 3D worlds created with SpAItial AI. This is all running in the browser including RL training! Link to demo and code in comments What you can do: -> Train a bot from scratch, watch it learn in minutes, then drop it into a world it has never seen and it keeps navigating obstacles. -> The worlds are real scenes generated from a text prompt or a photo via the SpAItial AI API (you get a splat + a collision mesh). -> Mesh is made into a 2D walkability + height grid that the RL environment runs on. -> The trick to generalization: the policy only ever sees egocentric inputs. A 16-ray lidar, the goal's body frame, and its speed. No global coordinates, no map. So it learns navigation, not a memorized world.
1:05

A fun way to try the new Claude Fable. I asked it to train a bot using RL to navigate 3D worlds created with SpAItial AI. This is all running in the browser including RL training! Link to demo and code in comments What you can do: -> Train a bot from scratch, watch it learn in minutes, then drop it into a world it has never seen and it keeps navigating obstacles. -> The worlds are real scenes generated from a text prompt or a photo via the SpAItial AI API (you get a splat + a collision mesh). -> Mesh is made into a 2D walkability + height grid that the RL environment runs on. -> The trick to generalization: the policy only ever sees egocentric inputs. A 16-ray lidar, the goal's body frame, and its speed. No global coordinates, no map. So it learns navigation, not a memorized world.

RO2⚡️

12,797 просмотров • 3 месяцев назад

This is Segment Anything for #GaussianSplatting running on the web... but it comes with a caveat. This is SplatEdit, a powerful #3DGS editing software built on top of SuperSplat. Last week, it added a segmentation selection feature powered by Segment Anything, running locally in the browser. Just upload a splat, click on the "mug" icon and select entire objects in the scene. The problem is that everything after the object is also selected (as you can see in the video). To fix that, just use the brush tool to subtract the unwanted portion of the scene (hold CTRL while brushing). I feel we are getting closer and closer to having more editing control on these amazing fuzzy clouds 😉
0:37

This is Segment Anything for #GaussianSplatting running on the web... but it comes with a caveat. This is SplatEdit, a powerful #3DGS editing software built on top of SuperSplat. Last week, it added a segmentation selection feature powered by Segment Anything, running locally in the browser. Just upload a splat, click on the "mug" icon and select entire objects in the scene. The problem is that everything after the object is also selected (as you can see in the video). To fix that, just use the brush tool to subtract the unwanted portion of the scene (hold CTRL while brushing). I feel we are getting closer and closer to having more editing control on these amazing fuzzy clouds 😉

Gabriele Romagnoli

14,767 просмотров • 2 месяцев назад

In Uniform you can use multiple viewport panels to design the final render views for a project presentation. All viewports can have independent shading modes and allow scene and camera settings editing. This demo is running on a 2018 3rd gen iPad Pro.
0:54

In Uniform you can use multiple viewport panels to design the final render views for a project presentation. All viewports can have independent shading modes and allow scene and camera settings editing. This demo is running on a 2018 3rd gen iPad Pro.

Sparseal

64,594 просмотров • 2 лет назад

📢 OneCanvas: 3D Scene Understanding via Panoramic Reprojection We extract features from video frames and reproject them into one occlusion-free view of the whole scene that a 2D VLM reads just like a normal image. We can center this view on any viewpoint, including an agent's own pose for situated reasoning. The same projection lets us create spatial training tasks with no human annotation, solvable only by reasoning over the 3D positions of real object features placed on an otherwise empty canvas. The result is a stock 2D VLM that reasons in 3D, setting a new state of the art across spatial benchmarks at far less compute. 🌐 ▶️ Great work by Bartłomiej Baranowski & Dave Zhenyu Chen
0:44

📢 OneCanvas: 3D Scene Understanding via Panoramic Reprojection We extract features from video frames and reproject them into one occlusion-free view of the whole scene that a 2D VLM reads just like a normal image. We can center this view on any viewpoint, including an agent's own pose for situated reasoning. The same projection lets us create spatial training tasks with no human annotation, solvable only by reasoning over the 3D positions of real object features placed on an otherwise empty canvas. The result is a stock 2D VLM that reasons in 3D, setting a new state of the art across spatial benchmarks at far less compute. 🌐 ▶️ Great work by Bartłomiej Baranowski & Dave Zhenyu Chen

Matthias Niessner

25,405 просмотров • 3 месяцев назад

Omg, this is such a hack for vibecoding mobile apps This is an image with a video on the top. The video is a character with no backround. I'm using: > Masko AI to generate the video > vibecode.dev to build the app In masko all you need to do is provide an image, and it will give you a video without a background. And then all you need to do is give it to vibecode.dev which is the best mobile app builder.
0:25

Omg, this is such a hack for vibecoding mobile apps This is an image with a video on the top. The video is a character with no backround. I'm using: > Masko AI to generate the video > vibecode.dev to build the app In masko all you need to do is provide an image, and it will give you a video without a background. And then all you need to do is give it to vibecode.dev which is the best mobile app builder.

Emily Lambert

29,840 просмотров • 7 месяцев назад

I still can't believe this works! Every filmmaker needs to try the hybrid AI filmmaking + Blender previs workflow for precise camera control. AI dialogue is still flat but I'm going to dive into solving that next. My workflow: 1. Generate a start frame in Midjourney 2. Block out the scene in Blender, animate the camera 3. Feed both to Seedance So much of cinematic storytelling is composition, timing, and giving the viewer a spatial understanding of the scene. You can solve all of that before you ever touch the video model.
0:36

I still can't believe this works! Every filmmaker needs to try the hybrid AI filmmaking + Blender previs workflow for precise camera control. AI dialogue is still flat but I'm going to dive into solving that next. My workflow: 1. Generate a start frame in Midjourney 2. Block out the scene in Blender, animate the camera 3. Feed both to Seedance So much of cinematic storytelling is composition, timing, and giving the viewer a spatial understanding of the scene. You can solve all of that before you ever touch the video model.

Reid Hannaford

17,468 просмотров • 2 месяцев назад

This might be the craziest workflow you'll see this week. From #VR greyboxing to 3D worlds with the help of #AI. The 2 tools I used are ShapesXR and Marble from World Labs. Let me tell you why and how. - ShapesXR is the most powerful and fastest way to assemble 3D scenes at scale. - The library of assets + ability to sketch from procedural primitives gives the perfect balance between speed and freedom. - ShapesXR allows you to export your creation as a glTF that can be used as a "3D prompt" into Marble - Marble now allows you to create a panorama using a 3D scene as input increasing DRAMATICALLY the control you have during the creation process - Through the workflow you can adjust textures and objects in the scene using AI inpainting - There are tons of options on how to use the final output (create a #GaussianSplatting, export collision mesh and even a detailed mesh 🤯) This is the perfect examples of how VR creative tools improve and accelerate the design process... so make sure to follow me if that's your jam 😉.
1:01

This might be the craziest workflow you'll see this week. From #VR greyboxing to 3D worlds with the help of #AI. The 2 tools I used are ShapesXR and Marble from World Labs. Let me tell you why and how. - ShapesXR is the most powerful and fastest way to assemble 3D scenes at scale. - The library of assets + ability to sketch from procedural primitives gives the perfect balance between speed and freedom. - ShapesXR allows you to export your creation as a glTF that can be used as a "3D prompt" into Marble - Marble now allows you to create a panorama using a 3D scene as input increasing DRAMATICALLY the control you have during the creation process - Through the workflow you can adjust textures and objects in the scene using AI inpainting - There are tons of options on how to use the final output (create a #GaussianSplatting, export collision mesh and even a detailed mesh 🤯) This is the perfect examples of how VR creative tools improve and accelerate the design process... so make sure to follow me if that's your jam 😉.

Gabriele Romagnoli

20,126 просмотров • 10 месяцев назад

I'm excited to share the Geo Register Plugin for LichtFeld Studio from the LichtFeld community! This plugin helps bring Gaussian splat scenes into real-world geographic space. It registers a scene to WGS-84 and ECEF coordinates, so you can click any point on the model and get its latitude, longitude and altitude. It supports multiple georeferencing sources, including EXIF GPS data, image position CSVs, RealityScan camera parameters and saved similarity transforms. Once the scene is registered, you can export geo-referenced splat models as LAS, LAZ or 3D Tiles datasets for use in GIS and 3D mapping workflows. Built for anyone working with drone data, photogrammetry, Gaussian splatting, GIS, ArcGIS or CesiumJS. Link in the comment below!
0:22

I'm excited to share the Geo Register Plugin for LichtFeld Studio from the LichtFeld community! This plugin helps bring Gaussian splat scenes into real-world geographic space. It registers a scene to WGS-84 and ECEF coordinates, so you can click any point on the model and get its latitude, longitude and altitude. It supports multiple georeferencing sources, including EXIF GPS data, image position CSVs, RealityScan camera parameters and saved similarity transforms. Once the scene is registered, you can export geo-referenced splat models as LAS, LAZ or 3D Tiles datasets for use in GIS and 3D mapping workflows. Built for anyone working with drone data, photogrammetry, Gaussian splatting, GIS, ArcGIS or CesiumJS. Link in the comment below!

MrNeRF

51,983 просмотров • 4 месяцев назад

Another city. Another full 3D Gaussian Splat reconstruction. This entire neighborhood was captured while walking with an Insta360 X5, then processed with our GPU pipeline and transformed into a high-fidelity, machine-readable 3D environment with Over the Reality. What you’re seeing is not an isolated virtual world. It’s a 3D digital twin of a real-world location, built from structured spatial data where every building, street, and detail can become usable by AI, robotics, and XR. And it’s happening at scale, across cities in more than 180 countries every day. Have an Insta360 X5 and want to become a mapper? Apply here: Want to build a 3D Gaussian Splat from a video you already recorded? Do it for free at powered by Over the Reality.
2:42

Another city. Another full 3D Gaussian Splat reconstruction. This entire neighborhood was captured while walking with an Insta360 X5, then processed with our GPU pipeline and transformed into a high-fidelity, machine-readable 3D environment with Over the Reality. What you’re seeing is not an isolated virtual world. It’s a 3D digital twin of a real-world location, built from structured spatial data where every building, street, and detail can become usable by AI, robotics, and XR. And it’s happening at scale, across cities in more than 180 countries every day. Have an Insta360 X5 and want to become a mapper? Apply here: Want to build a 3D Gaussian Splat from a video you already recorded? Do it for free at powered by Over the Reality.

Over the Reality 🌐

400,269 просмотров • 4 месяцев назад

Fully local Code Assistant running on NVIDIA GPU! In this tutorial, I'll show you how to run Llama3 using TensorRT and Nvidia's Triton Inference Server to use it as a Code Assistant in VSCode In this thread 🧵, I'll walk you through the integration process, explaining each step simply and clearly 👇 NVIDIA AI NVIDIA AIDev AI at Meta
0:30

Fully local Code Assistant running on NVIDIA GPU! In this tutorial, I'll show you how to run Llama3 using TensorRT and Nvidia's Triton Inference Server to use it as a Code Assistant in VSCode In this thread 🧵, I'll walk you through the integration process, explaining each step simply and clearly 👇 NVIDIA AI NVIDIA AIDev AI at Meta

Daniel San

42,154 просмотров • 2 лет назад

Excited to share a new vibe coded prototype: an interactive lesson on ants and their habitat, designed for elementary school students. AI allowed me to finalize this demo quickly. Some educational details still need review, but the main chambers of an anthill are all represented. Everything was made in Omma AI , a Spline product. What's inside: - 3D models created with Omma, all I needed was one image of the anthill's exterior and another of its cross-section to generate both 3D models - Clouds done with sprites - A custom shader mask to reveal the inside of the anthill This was a really fun one to work on!
0:43

Excited to share a new vibe coded prototype: an interactive lesson on ants and their habitat, designed for elementary school students. AI allowed me to finalize this demo quickly. Some educational details still need review, but the main chambers of an anthill are all represented. Everything was made in Omma AI , a Spline product. What's inside: - 3D models created with Omma, all I needed was one image of the anthill's exterior and another of its cross-section to generate both 3D models - Clouds done with sprites - A custom shader mask to reveal the inside of the anthill This was a really fun one to work on!

Joseph Azar

19,890 просмотров • 4 месяцев назад

Finally solved all the big things needed to prove occlusion culling is viable in the Hyperfy engine. All culling happens directly at render time and instantly, we've taken over the Three.js render pipeline to use our spatial hierarchy and manually issue renders, postprocessing and shadow passes with both frustum and occlusion culling. Our pipeline now aligns much more closely to unity/unreal because we collect visible objects and issue custom instanced draws on the fly for only those objects every single frame. This is one of the biggest issues engines on the web like three, babylon and playcanvas have and imo the reason people coming from unity/unreal don't take them seriously. There's a lot to unpack here, maybe i'll make a thread soon.
0:50

Finally solved all the big things needed to prove occlusion culling is viable in the Hyperfy engine. All culling happens directly at render time and instantly, we've taken over the Three.js render pipeline to use our spatial hierarchy and manually issue renders, postprocessing and shadow passes with both frustum and occlusion culling. Our pipeline now aligns much more closely to unity/unreal because we collect visible objects and issue custom instanced draws on the fly for only those objects every single frame. This is one of the biggest issues engines on the web like three, babylon and playcanvas have and imo the reason people coming from unity/unreal don't take them seriously. There's a lot to unpack here, maybe i'll make a thread soon.

Ash

15,118 просмотров • 1 год назад

.Marc Andreessen 🇺🇸 says the winner of the US-China AI race could dictate what all 8 billion people on Earth use — but open source introduces a third possibility: "The stakes are basically: what is the world going to run on?" "In the long run, somebody's gonna win. And the world will either be running on American AI or be running on Chinese AI." "The open source thing is super fascinating because it throws a wrench into all of this. It raises a third possibility that neither the US nor China are going to be the platform — it may just be open source." Marc Andreessen with Jeetu Patel at Cisco AI Summit
3:41

.Marc Andreessen 🇺🇸 says the winner of the US-China AI race could dictate what all 8 billion people on Earth use — but open source introduces a third possibility: "The stakes are basically: what is the world going to run on?" "In the long run, somebody's gonna win. And the world will either be running on American AI or be running on Chinese AI." "The open source thing is super fascinating because it throws a wrench into all of this. It raises a third possibility that neither the US nor China are going to be the platform — it may just be open source." Marc Andreessen with Jeetu Patel at Cisco AI Summit

a16z

101,985 просмотров • 7 месяцев назад

The top question I got was does Halo support local LLM running on a Mac. Here's an example of the app using gemma-4-26b-a4b-qat via LM Studio running on a base M5/32GB. It's giving an average of 40 tok/s and in my brief testing is able to execute all tool calls correctly. Very usable with this app.
0:37

The top question I got was does Halo support local LLM running on a Mac. Here's an example of the app using gemma-4-26b-a4b-qat via LM Studio running on a base M5/32GB. It's giving an average of 40 tok/s and in my brief testing is able to execute all tool calls correctly. Very usable with this app.

Tanmay

11,682 просмотров • 2 месяцев назад

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision
0:40

I created a 3D Gaussian Splat of my kitchen using 1/3rd the images it used to take me by using NVIDIA AI Developer's 3DGRUT! I can now use 180 degree fisheye images and ray tracing to make detailed splats. The only reason the scene isn't sharper is because my input images weren't super sharp - when I took the images back in October, I was still learning to use the lens. I plan to make a "first reactions/overview video". Tutorial after that. For reference, this took 206 images and the ultrawide on my iPhone took 608 images to capture. #3D #AEC #Computervision

Jonathan Stephens

30,714 просмотров • 1 год назад

Look at how crispy this 3d gaussian splat came out! The reflections in the painting have me floored. Captured with a Sony a7iii camera and 18mm lens. I posed the imagery using epic's reality capture and trained it with lichtfeld studio. Lichtfeld is my new favorite way to train splats -- it integrates all the latest research and it’s completely free. Also what I love about 3d capture -- you can process old datasets with new algorithms and breathe new life into them.
0:27

Look at how crispy this 3d gaussian splat came out! The reflections in the painting have me floored. Captured with a Sony a7iii camera and 18mm lens. I posed the imagery using epic's reality capture and trained it with lichtfeld studio. Lichtfeld is my new favorite way to train splats -- it integrates all the latest research and it’s completely free. Also what I love about 3d capture -- you can process old datasets with new algorithms and breathe new life into them.

Bilawal Sidhu

79,257 просмотров • 6 месяцев назад

You can run local AI models up to 120B without a $10,000 Mac Studio This phone-sized device is your own server for open-source models. 100% local and private. And you can use it to: - Power an agent like OpenClaw 24/7 - Completely replace a chatbot - Literally anything that requires an API It’s called Tiiny, and, once again, you can run the latest open-source models on it. Link below
16:10

You can run local AI models up to 120B without a $10,000 Mac Studio This phone-sized device is your own server for open-source models. 100% local and private. And you can use it to: - Power an agent like OpenClaw 24/7 - Completely replace a chatbot - Literally anything that requires an API It’s called Tiiny, and, once again, you can run the latest open-source models on it. Link below

Paul Couvert

41,245 просмотров • 6 месяцев назад

Fable 5 is nuts. I vibe coded terminator vision replays with no LiDAR or IMU, just 2D videos from Ray-Bans, GoPros, and iPhones -- all fused into a 4D reconstruction of my shooting range sessions. Full pipeline in the video ft. audio-waveform sync, SAM3 masked colmap & pi3 solves, 3D gaussian splats, and gemini as the hit/miss judge. Chapters: 0:00 Cold Open 0:39 Project IronSight 1:41 Origin Story 2:38 The Command Center 3:55 Inside the Machine — From 2D to 4D 5:54 Training the AI to See 8:18 The Payoff 9:28 Where This Goes 11:42 God's Eye View
12:23

Fable 5 is nuts. I vibe coded terminator vision replays with no LiDAR or IMU, just 2D videos from Ray-Bans, GoPros, and iPhones -- all fused into a 4D reconstruction of my shooting range sessions. Full pipeline in the video ft. audio-waveform sync, SAM3 masked colmap & pi3 solves, 3D gaussian splats, and gemini as the hit/miss judge. Chapters: 0:00 Cold Open 0:39 Project IronSight 1:41 Origin Story 2:38 The Command Center 3:55 Inside the Machine — From 2D to 4D 5:54 Training the AI to See 8:18 The Payoff 9:28 Where This Goes 11:42 God's Eye View

Bilawal Sidhu

23,091 просмотров • 2 месяцев назад