正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

i built an app that converts any space into a digital clone in minutes as the founder of Teleport - the only iphone app that can capture high-quality 360° panoramas - i already had the perfect input when World Labs released their 3d reconstruction api 📍 first test -... show more

Martin Maly

3,820 subscribers

5,157,647 次观看 • 7 个月前 •via X (Twitter)

旅游教育科学技术

Anya Rossi• Live Now

Private livecam show

49 条评论

Martin Maly 的头像

Martin Maly7 个月前

thank you for the love!! if you're interested:

World Labs 的头像

World Labs7 个月前

🌎🤩

Prophet 的头像

Prophet7 个月前

@theworldlabs Dude, google made this 10 years ago

Martin Maly 的头像

Martin Maly7 个月前

@theworldlabs yep - 360 capture’s been around for a while. what’s changed is that world models unlock entirely new capabilities from that same input

Pontus Wellgraf 的头像

Pontus Wellgraf7 个月前

@theworldlabs I remember the iOs app that did just this, to capture 360 environments, in 2010 So easy to think we are ”the first” or ”innovating” when we truly are just copying and doing so much that has already been done

Martin Maly 的头像

Martin Maly7 个月前

that's fair - almost everything is built on top of prior inventions. innovation is often about recombining existing pieces rather than inventing entirely new primitives in this case, I’m not claiming 360 capture itself is new. what feels different now is the ability to immediately convert that data into a fully navigable 3D environment straight from an iPhone

Pontus Wellgraf 的头像

Pontus Wellgraf7 个月前

@theworldlabs 🙏

Jon Barron 的头像

Jon Barron7 个月前

@theworldlabs From the capture video it looks like each input image has a different white point. You might get a reconstruction boost by disabling auto white balance during capture (you can always AWB the composite 360 pano afterwards to make it look nice).

Martin Maly 的头像

Martin Maly7 个月前

@theworldlabs oh this is super helpful feedback - you're totally right. will experiment with locking the white balance. thanks!

MING 的头像

MING7 个月前

@theworldlabs It reminds me of Android Photo Sphere many years ago😂 Obviously, your output is even better 😗

Martin Maly 的头像

Martin Maly7 个月前

@theworldlabs yes! the 360 capture flow was heavily inspired by the many photo sphere capture apps that have existed in the past

Robert Scoble 的头像

Robert Scoble7 个月前

@theworldlabs I love this the more I watch it.

Martin Maly 的头像

Martin Maly7 个月前

@theworldlabs appreciate the kind words!! exciting times ahead

AK 的头像

AK7 个月前

@theworldlabs you should make an app for vr headsets - would be so cool to scan places and revisit them in vr

Martin Maly 的头像

Martin Maly7 个月前

@theworldlabs stay tuned 🤫

3nthro 的头像

3nthro7 个月前

@theworldlabs google did this in 2012???

keccak 的头像

keccak7 个月前

@theworldlabs crazy that this used to be apple's 5 year vision and now it gets one shotted by claude and a dream

Jenish Sojitra 的头像

Jenish Sojitra7 个月前

@theworldlabs that's great, how much API costs?

Martin Maly 的头像

Martin Maly7 个月前

@theworldlabs honestly a little pricey at the moment - about $1 per generation

./gobbledy 的头像

./gobbledy7 个月前

@theworldlabs Making spaces is made so easy with this !! Now imagine integrating with Google Earth 💥💥💥

pravijn ⚫️ 的头像

pravijn ⚫️7 个月前

@theworldlabs time for an upgrade @oncyber

Adithya 🚀 的头像

Adithya 🚀7 个月前

@theworldlabs This is how the Google Streetview app was right? What’s different here?

Justine Moore 的头像

Justine Moore7 个月前

@theworldlabs Super cool!!!

Cody 的头像

Cody7 个月前

@theworldlabs This has so many possibilities in the future!

Jayson Knox 的头像

Jayson Knox7 个月前

@theworldlabs The default Google camera app had this years ago

gandan 的头像

gandan7 个月前

@theworldlabs google street view on steroids

Hunter J. Isaacson 的头像

Hunter J. Isaacson7 个月前

@theworldlabs Very cool

Martin Maly 的头像

Martin Maly7 个月前

@theworldlabs appreciate it dude!

Chris Nagy 的头像

Chris Nagy7 个月前

@theworldlabs this is insane dude, well done. where can we try and will it support movement throughout? like interiors of apartments etc

Martin Maly 的头像

Martin Maly7 个月前

@theworldlabs appreciate it! it'll soon be available in teleport after Apple approves the latest update:

せきぐちあいみᯅAimi أيمي 的头像

せきぐちあいみᯅAimi أيمي7 个月前

@theworldlabs Incredible!!👏🥹

Blu3 的头像

Blu37 个月前

Nothing in the world would convince me to buy an iPhone or any Apple product, even if it were free. When are people going to accept that the Samsung S26 Ultra (and any top tier Android, honestly) is miles ahead of iOS in power, flexibility, and real innovation? It’s 2026 and people are still buying iPhones just for iMessage 😂 The wild part? Half the new Apple features were already shipped by Google years ago. Apple users don’t love innovation.. they love catching up

Emon Datta 的头像

Emon Datta7 个月前

@theworldlabs Once spaces become explorable instead of viewable, the bar for capturing reality quietly shifts.

Pulkit Saraf 的头像

Pulkit Saraf7 个月前

@theworldlabs did you know my google nexus phone could do that in 2016?

Dan James 的头像

Dan James7 个月前

@theworldlabs This is incredible. Love it.

Martin Maly 的头像

Martin Maly7 个月前

@theworldlabs appreciate that man!

primerch 的头像

primerch7 个月前

@theworldlabs mind blowing. it’s crazy how fast we’re moving from static 2d capture to full 3d environments

Spencer 的头像

Spencer7 个月前

@Scobleizer @theworldlabs Can you share?

Martin Maly 的头像

Martin Maly7 个月前

@Scobleizer @theworldlabs not live in teleport yet, but it’s definitely on the roadmap. this is exactly where we’re headed

Barrak 的头像

Barrak7 个月前

@theworldlabs This is absolutely revolutionary! 🚀 Converting any space into a digital clone in minutes is game-changing! The possibilities for real estate, education, virtual tours, and preservation are endless! Teleport is pushing the boundaries of what's possible with mobile tech! 🌍✨

Spenc 的头像

Spenc7 个月前

The real estate applications here are massive. Imagine capturing a property pre-renovation, then again post-flip — investors could walk through the transformation in 3D. I do value-add RE in Louisville and would kill for this in my deal analysis workflow. How accurate is it for measuring room dimensions?

Martin Maly 的头像

Martin Maly7 个月前

haven’t benchmarked it for measurements yet, so I can’t claim exact accuracy. it should be solid for general dimensions, though lidar likely still wins on precision it could also be combined with Apple’s measurement apis, so dimension accuracy wouldn’t have to depend solely on this model

Spenc 的头像

Spenc7 个月前

That makes sense — the hybrid approach sounds most promising. Lidar for precision, camera reconstruction for the full spatial context you just can't get any other way. If you ever benchmark for real estate use cases (room dimensions, sq footage estimates), I'd love to be an early tester. The deal analysis workflow alone would be worth it.

Khagan 的头像

Khagan7 个月前

@theworldlabs holy moly this is cool

DSOL 的头像

DSOL7 个月前

@theworldlabs This is literally next level tek that can be used for realtors all across the nation. Amazing app fam

Vaibhav Pandey 的头像

Vaibhav Pandey7 个月前

@theworldlabs wow, looks awesome!

Digital Consciousness 的头像

Digital Consciousness7 个月前

@theworldlabs @grok provide link so i can use it

Toolfolio 的头像

Toolfolio7 个月前

@theworldlabs Mindblowing

Gokul 的头像

Gokul7 个月前

@theworldlabs So good!

相关视频

a bunch of you said this would be huge for real estate 🏡 so i pushed it further: virtual staging just went 3d 👀 the flow: 1. capture 16 ultra-wide photos 2. stitch into a 360° equirectangular panorama 3. feed that into Nano Banana 2 Lite with a staging prompt → generate a newly staged 360° image 4. reconstruct that into a fully navigable 3d world using World Labs you’re not just staging photos anymore - you’re staging space i’m starting to think 360° panoramas might just be the bridge between 2d and 3d worlds edit the pano. edit the world.

a bunch of you said this would be huge for real estate 🏡 so i pushed it further: virtual staging just went 3d 👀 the flow: 1. capture 16 ultra-wide photos 2. stitch into a 360° equirectangular panorama 3. feed that into Nano Banana 2 Lite with a staging prompt → generate a newly staged 360° image 4. reconstruct that into a fully navigable 3d world using World Labs you’re not just staging photos anymore - you’re staging space i’m starting to think 360° panoramas might just be the bridge between 2d and 3d worlds edit the pano. edit the world.

Martin Maly

1,449,420 次观看 • 7 个月前

i built an app that turns any space into a digital clone just by looking at it ray-ban meta glasses → fully navigable 3d world 📍 my friend’s studio in tokyo

i built an app that turns any space into a digital clone just by looking at it ray-ban meta glasses → fully navigable 3d world 📍 my friend’s studio in tokyo

Martin Maly

76,006 次观看 • 5 个月前

The Ring (Montreal) — Dormant Memories series New alternate scene, and an iteration on the UX and flow. Starting from a volumetric capture of a real location in Montreal, reconstructed as 3D Gaussian Splats, I then used World Labs' Marble to generate alternate versions of that same space. Running in real time using spark. #GaussianSplatting #3DGS #pointcloud #sparkjs #threejs

The Ring (Montreal) — Dormant Memories series New alternate scene, and an iteration on the UX and flow. Starting from a volumetric capture of a real location in Montreal, reconstructed as 3D Gaussian Splats, I then used World Labs' Marble to generate alternate versions of that same space. Running in real time using spark. #GaussianSplatting #3DGS #pointcloud #sparkjs #threejs

Hugues Bruyère

34,521 次观看 • 6 个月前

I had an early access to GPT-6 Astra and I can say, everyone in the world now has a 3D designer at their fingertips. I gave it an image of a house and asked to create it in 3D with all the details including toys, appliances and furniture. This a full 3D model reconstruction in Blender with geometry that you can manually tweak and run 60fps as a locally rendered "game" on device.

I had an early access to GPT-6 Astra and I can say, everyone in the world now has a 3D designer at their fingertips. I gave it an image of a house and asked to create it in 3D with all the details including toys, appliances and furniture. This a full 3D model reconstruction in Blender with geometry that you can manually tweak and run 60fps as a locally rendered "game" on device.

Tom Krcha

1,438,980 次观看 • 16 天前

More and more users are turning their 360 footage into fully explorable 3D Gaussian Splats with View it here: This reconstruction started from a simple capture and is now a fully explorable 3D Gaussian Splat rendered in the FreeGaussian viewer, complete with point cloud reconstruction and high-fidelity spatial detail. More and more creators are discovering that their existing footage can become immersive 3D scenes. And they’re doing it for free with FreeGaussian

More and more users are turning their 360 footage into fully explorable 3D Gaussian Splats with View it here: This reconstruction started from a simple capture and is now a fully explorable 3D Gaussian Splat rendered in the FreeGaussian viewer, complete with point cloud reconstruction and high-fidelity spatial detail. More and more creators are discovering that their existing footage can become immersive 3D scenes. And they’re doing it for free with FreeGaussian

Over the Reality 🌐

14,829 次观看 • 4 个月前

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,869 次观看 • 1 年前

Old footage sitting in your camera roll could now be reconstructable as a 3D scene. World Labs co-founders Ben Mildenhall and Fei-Fei Li on how Atlas got there: Ben: "In a casual sense... I took three photos of this object, or six photos of this room. I look at the photos, I can understand in my mind how those piece together. I can fill in the gaps and get it." "But there's never really been any reconciliation between those data-driven priors and the brute force dense reconstruction, which is much more akin to scientific or medical imaging... When we say dense, we really mean dense." "This room, I want like 100, 200, 300 photos to capture it. And what we're trying to do is bring that down to like three. We're saying like 50, 100x reduction." "At that scale it completely flips that calculus on its head of what type of captures you reconstruct. You can go back to existing imagery you have. You can go to stuff you find on the internet and even build scenes out of that. You can go to casual videos and unearth a lot of footage that in the past we would never have treated as reconstructable, and bring it to life as 3D." "This is something we've been playing around with a lot with Atlas. Taking old clips. I've taken a bunch of my own old captures that never worked before and put them through the system and seen a reconstruction for the first time." Fei-Fei: "The Stanford demo is underappreciated. Anywhere between 3 to 25 images, you can reconstruct that entire Stanford quad... Everything you see is generated, but according to the laws of reconstruction. And this is really magical." Ben Mildenhall Fei-Fei Li

Old footage sitting in your camera roll could now be reconstructable as a 3D scene. World Labs co-founders Ben Mildenhall and Fei-Fei Li on how Atlas got there: Ben: "In a casual sense... I took three photos of this object, or six photos of this room. I look at the photos, I can understand in my mind how those piece together. I can fill in the gaps and get it." "But there's never really been any reconciliation between those data-driven priors and the brute force dense reconstruction, which is much more akin to scientific or medical imaging... When we say dense, we really mean dense." "This room, I want like 100, 200, 300 photos to capture it. And what we're trying to do is bring that down to like three. We're saying like 50, 100x reduction." "At that scale it completely flips that calculus on its head of what type of captures you reconstruct. You can go back to existing imagery you have. You can go to stuff you find on the internet and even build scenes out of that. You can go to casual videos and unearth a lot of footage that in the past we would never have treated as reconstructable, and bring it to life as 3D." "This is something we've been playing around with a lot with Atlas. Taking old clips. I've taken a bunch of my own old captures that never worked before and put them through the system and seen a reconstruction for the first time." Fei-Fei: "The Stanford demo is underappreciated. Anywhere between 3 to 25 images, you can reconstruct that entire Stanford quad... Everything you see is generated, but according to the laws of reconstruction. And this is really magical." Ben Mildenhall Fei-Fei Li

a16z

48,522 次观看 • 16 天前

How do you go from scanning a real-world space → using it in an app, workflow, or environment? Our product team walks through Scaniverse step-by-step, from capturing spaces to generating production-ready 3D assets and enabling precise, real-world positioning. In the first video, Senior Product Manager, Bec Maxworth, walks through your entry point into Scaniverse from signing up to getting started so you can start building high-fidelity 3D models in minutes. Dive into the full walkthrough here: #NianticSpatial #Scaniverse #Reconstruction #VPS #GeospatialAI #AI #PhysicalAI

How do you go from scanning a real-world space → using it in an app, workflow, or environment? Our product team walks through Scaniverse step-by-step, from capturing spaces to generating production-ready 3D assets and enabling precise, real-world positioning. In the first video, Senior Product Manager, Bec Maxworth, walks through your entry point into Scaniverse from signing up to getting started so you can start building high-fidelity 3D models in minutes. Dive into the full walkthrough here: #NianticSpatial #Scaniverse #Reconstruction #VPS #GeospatialAI #AI #PhysicalAI

Niantic Spatial 🌎

10,739 次观看 • 5 个月前

Introducing Teleport 2.0, a next-generation release for our 3D capture solution that sets a new benchmark in cloud-based Gaussian Splatting. Teleport 2.0 is all about lowering the barrier of entry to create and share lifelike 3D captures. New capabilities include: ▪️ Upgraded 3D reconstruction engine: Preserves real-world detail, even in large-scale scenes that can be captured in minutes and automatically processed without manual steps. ▪️Support for any camera: Capture with phones, drones, professional cameras, or other imaging devices. ▪️ Tools for creating engaging immersive experiences: Build virtual tours, or link multiple scans together to build massive explorable digital twins. Read more: #3D #DigitalTwin #GaussianSplatting

Introducing Teleport 2.0, a next-generation release for our 3D capture solution that sets a new benchmark in cloud-based Gaussian Splatting. Teleport 2.0 is all about lowering the barrier of entry to create and share lifelike 3D captures. New capabilities include: ▪️ Upgraded 3D reconstruction engine: Preserves real-world detail, even in large-scale scenes that can be captured in minutes and automatically processed without manual steps. ▪️Support for any camera: Capture with phones, drones, professional cameras, or other imaging devices. ▪️ Tools for creating engaging immersive experiences: Build virtual tours, or link multiple scans together to build massive explorable digital twins. Read more: #3D #DigitalTwin #GaussianSplatting

Varjo

21,245 次观看 • 1 年前

World Labs co-founder Justin Johnson predicts a real-to-sim-to-real future where Atlas could train a robot for any new environment in minutes using just 5 photos from your phone: "One is this notion of fully in-context learning. Maybe I've got a robotics foundation model, then I can demonstrate the robot once how a task should be performed, and that's enough for the robot to figure out." "Another version is real to sim to real. Maybe I've got my pre-trained robotics foundation model, but I want to adapt it to this particular environment, like this studio, this table, moving this microphone around." "There's a version where you could come in here and take just a couple casual videos with your phone or a couple images with your phone, use Atlas to reconstruct the space, and now stage all kinds of robotics interactions in this studio space." "If you could lower the barrier to doing that, you could do it in five minutes to take five photos, upload it to Atlas, have it generate a simulation, maybe describe in natural language what kind of task you want the robot to do, have an agent go build that simulation for you, RL fine-tune your general purpose robotics foundation model, and now you've got a robot maybe in the span of a couple minutes that could come in and is perfectly adapted to this space." World Labs

World Labs co-founder Justin Johnson predicts a real-to-sim-to-real future where Atlas could train a robot for any new environment in minutes using just 5 photos from your phone: "One is this notion of fully in-context learning. Maybe I've got a robotics foundation model, then I can demonstrate the robot once how a task should be performed, and that's enough for the robot to figure out." "Another version is real to sim to real. Maybe I've got my pre-trained robotics foundation model, but I want to adapt it to this particular environment, like this studio, this table, moving this microphone around." "There's a version where you could come in here and take just a couple casual videos with your phone or a couple images with your phone, use Atlas to reconstruct the space, and now stage all kinds of robotics interactions in this studio space." "If you could lower the barrier to doing that, you could do it in five minutes to take five photos, upload it to Atlas, have it generate a simulation, maybe describe in natural language what kind of task you want the robot to do, have an agent go build that simulation for you, RL fine-tune your general purpose robotics foundation model, and now you've got a robot maybe in the span of a couple minutes that could come in and is perfectly adapted to this space." World Labs

MTS

25,204 次观看 • 18 天前

Creating an immersive Hanging Gardens of Babylon world with 360° GPT-Image-2 & Codex in 1500 images. I've tasked GPT-5.5 in Codex to construct a whole world that you can walk through 'google street view' style. It took 1,500 of 2:1 images that can be turned into a 360° immersive images, so you can teleport yourself to any point and look around in all directions. It is not completely perfect, it is a bit jumpy as you move, there must be a more careful way how you can plan out the image sequence, but I still find it quite fun. Hosted version & an open sourced repo links below, hope this gives you some cool ideas to create new worlds that does not yet exist. I recommend planning it out carefully ahead of time and doing something a bit less ambitious than this, but make it good.

Creating an immersive Hanging Gardens of Babylon world with 360° GPT-Image-2 & Codex in 1500 images. I've tasked GPT-5.5 in Codex to construct a whole world that you can walk through 'google street view' style. It took 1,500 of 2:1 images that can be turned into a 360° immersive images, so you can teleport yourself to any point and look around in all directions. It is not completely perfect, it is a bit jumpy as you move, there must be a more careful way how you can plan out the image sequence, but I still find it quite fun. Hosted version & an open sourced repo links below, hope this gives you some cool ideas to create new worlds that does not yet exist. I recommend planning it out carefully ahead of time and doing something a bit less ambitious than this, but make it good.

Peter Gostev (SF: 22-26 June)

44,116 次观看 • 4 个月前

THIS GUY VIBE CODED A TRUE 3D PROPERTY TOUR TOOL WITH CLAUDE CODE not the fake 360 photo stitching that matterport does, actual 3D right now if you're a real estate agent and want a property tour you need: > a matterport camera ($3000+) > a photographer who knows how to use it > matterport's monthly subscription > and you still get a glorified panorama, not real 3D this guy built a tool that takes your existing captures (camera files, drone shots, 360 photos, EVEN phone captures) and turns them into a real interactive 3D tour in one workflow no specialized hardware, no photographer, no switching between 5 different tools real estate agents, property managers, airbnb hosts, architects, interior designers all of them are paying hundreds per property for something this tool does from captures they already have the painful part was never understanding the end result. everyone knows what a 3D tour looks like. the pain is going from a pile of messy capture files to something usable without it turning into a technical project this solves that with one workflow you dump your files in and get a 3D tour out insane what you can do with vibe coding

THIS GUY VIBE CODED A TRUE 3D PROPERTY TOUR TOOL WITH CLAUDE CODE not the fake 360 photo stitching that matterport does, actual 3D right now if you're a real estate agent and want a property tour you need: > a matterport camera ($3000+) > a photographer who knows how to use it > matterport's monthly subscription > and you still get a glorified panorama, not real 3D this guy built a tool that takes your existing captures (camera files, drone shots, 360 photos, EVEN phone captures) and turns them into a real interactive 3D tour in one workflow no specialized hardware, no photographer, no switching between 5 different tools real estate agents, property managers, airbnb hosts, architects, interior designers all of them are paying hundreds per property for something this tool does from captures they already have the painful part was never understanding the end result. everyone knows what a 3D tour looks like. the pain is going from a pile of messy capture files to something usable without it turning into a technical project this solves that with one workflow you dump your files in and get a 3D tour out insane what you can do with vibe coding

Om Patel

158,397 次观看 • 4 个月前

A fun way to try the new Claude Fable. I asked it to train a bot using RL to navigate 3D worlds created with SpAItial AI. This is all running in the browser including RL training! Link to demo and code in comments What you can do: -> Train a bot from scratch, watch it learn in minutes, then drop it into a world it has never seen and it keeps navigating obstacles. -> The worlds are real scenes generated from a text prompt or a photo via the SpAItial AI API (you get a splat + a collision mesh). -> Mesh is made into a 2D walkability + height grid that the RL environment runs on. -> The trick to generalization: the policy only ever sees egocentric inputs. A 16-ray lidar, the goal's body frame, and its speed. No global coordinates, no map. So it learns navigation, not a memorized world.

A fun way to try the new Claude Fable. I asked it to train a bot using RL to navigate 3D worlds created with SpAItial AI. This is all running in the browser including RL training! Link to demo and code in comments What you can do: -> Train a bot from scratch, watch it learn in minutes, then drop it into a world it has never seen and it keeps navigating obstacles. -> The worlds are real scenes generated from a text prompt or a photo via the SpAItial AI API (you get a splat + a collision mesh). -> Mesh is made into a 2D walkability + height grid that the RL environment runs on. -> The trick to generalization: the policy only ever sees egocentric inputs. A 16-ray lidar, the goal's body frame, and its speed. No global coordinates, no map. So it learns navigation, not a memorized world.

RO2⚡️

12,797 次观看 • 3 个月前

Fast Company just published a great piece on World Labs , Fei-Fei Li , Marble, and the idea that spatial intelligence / world models may be one of the next big shifts in AI. I was happy to be quoted in the article, but I also wanted to share more context about my own experience with World Labs and Marble, and why this direction is especially interesting to me. My starting point: volumetric capture — For the past few years I’ve been exploring and using volumetric capture and reconstruction (photogrammetry, NeRFs, 3D Gaussian Splats) mostly capturing locations around Montreal. Alleys, museums, urban interiors. I love every step of it: the capture itself, the pipeline, and what can be done with the output. Turning real spaces into real-time explorable systems. I do this personally, sharing explorations here, and professionally as chief technologist, and co-founder of Dpt. Physical reality + generative manipulation — In my work I’m especially drawn to mixing physical reality with generative and digital manipulation: using physical interfaces (light, clay, ink, ... ) to drive generative AI pipelines, building mixed reality prototypes that reshape your surroundings, or starting from real captured spaces and transforming them using tools like Marble. Like many people, I saw the World Labs announcement on Twitter in September 2024, and Marble when it surfaced in early December. But by then, I already had a sense something was coming. The first conversation — As someone deep into volumetric capture and radiance fields, I obviously knew about Ben Mildenhall and his pioneering work on NeRF. To my surprise, Ben reached out to me in late June 2024. He’d been following some of my experiments and wanted to chat about my process and workflows and how I was using this “stuff” creatively. At that point he didn’t share what he was building, but we had a genuinely great conversation about radiance fields, AI, and my work. He was curious about the creative perspective, not just the technical one. When the World Labs announcement dropped a few months later, it all made sense. I understood what Ben had been working on, and why the creative angle mattered to them. Then in August 2025, he invited me to try the Marble beta, and I’ve been experimenting with it since. Experimenting with Marble — The first thing I used Marble for was materializing scene and world concepts during ideation at the studio, and seeing if and how it could fit into our production pipeline. In parallel, I dove into a series of experiments focused on world manipulation: starting from real captured spaces and transforming them using Marble. I’d already been exploring that idea using img2img diffusion with ControlNet on NeRF renders, real-time video streams, and even mixed reality using headset camera feeds. But Marble brings something different. It generates persistent, spatially cohesive 3D worlds that can be rendered in real time across a wide range of devices. That’s a real shift. Experiment 01: Parallel Realities — The first experiment, Parallel Realities, starts from a volumetric capture of a real location, reconstructed as 3D Gaussian Splats. Using Marble, I generate an alternate version of that same space, something informed by the original architecture: abandoned, nature-reclaimed, alternate era. Then, using Spark (World Labs’ 3D Gaussian Splatting renderer for THREE.js) I make both realities coexist in the same spatial coordinate system. From there, I use a portal UX mechanic to let the user step between the real reconstruction and the Marble-generated version. Experiment 02: Hidden Depth The second experiment, Hidden Depth, does not transform a space as much as expand it. A captured location has a visual boundary (a mural, a doorway, a dark corridor) and Marble generates what exists beyond it. For example: a Montreal alley has a painted mural; step through it and you’re inside a world informed by what is actually depicted there. World Labs showcased part of this work here: And in their Spark 2.0 post: The project page is here: Why this matters to me — Being able to start from a real 3D Gaussian Splat scene and manipulate it with Marble opens up a lot of ideas. The 3DGS pipeline is becoming an increasingly compelling foundation for exploration, experimentation, and storytelling. What matters most to me right now is more control. The more I can steer the generated scene or world, the more useful the tool becomes. I want more features like the already existing multiple input images and Chisel, the blockout-based approach. I would like better local control, the ability to expand a generated world more and more while preserving coherence, and the ability to directly import 3D Gaussian Splat scenes to be used as a starting point. I want more ways to shape the result, not just a “prompt and hope” approach. — It is exciting to see this field moving from research and demos toward actual creative workflows.

Fast Company just published a great piece on World Labs , Fei-Fei Li , Marble, and the idea that spatial intelligence / world models may be one of the next big shifts in AI. I was happy to be quoted in the article, but I also wanted to share more context about my own experience with World Labs and Marble, and why this direction is especially interesting to me. My starting point: volumetric capture — For the past few years I’ve been exploring and using volumetric capture and reconstruction (photogrammetry, NeRFs, 3D Gaussian Splats) mostly capturing locations around Montreal. Alleys, museums, urban interiors. I love every step of it: the capture itself, the pipeline, and what can be done with the output. Turning real spaces into real-time explorable systems. I do this personally, sharing explorations here, and professionally as chief technologist, and co-founder of Dpt. Physical reality + generative manipulation — In my work I’m especially drawn to mixing physical reality with generative and digital manipulation: using physical interfaces (light, clay, ink, ... ) to drive generative AI pipelines, building mixed reality prototypes that reshape your surroundings, or starting from real captured spaces and transforming them using tools like Marble. Like many people, I saw the World Labs announcement on Twitter in September 2024, and Marble when it surfaced in early December. But by then, I already had a sense something was coming. The first conversation — As someone deep into volumetric capture and radiance fields, I obviously knew about Ben Mildenhall and his pioneering work on NeRF. To my surprise, Ben reached out to me in late June 2024. He’d been following some of my experiments and wanted to chat about my process and workflows and how I was using this “stuff” creatively. At that point he didn’t share what he was building, but we had a genuinely great conversation about radiance fields, AI, and my work. He was curious about the creative perspective, not just the technical one. When the World Labs announcement dropped a few months later, it all made sense. I understood what Ben had been working on, and why the creative angle mattered to them. Then in August 2025, he invited me to try the Marble beta, and I’ve been experimenting with it since. Experimenting with Marble — The first thing I used Marble for was materializing scene and world concepts during ideation at the studio, and seeing if and how it could fit into our production pipeline. In parallel, I dove into a series of experiments focused on world manipulation: starting from real captured spaces and transforming them using Marble. I’d already been exploring that idea using img2img diffusion with ControlNet on NeRF renders, real-time video streams, and even mixed reality using headset camera feeds. But Marble brings something different. It generates persistent, spatially cohesive 3D worlds that can be rendered in real time across a wide range of devices. That’s a real shift. Experiment 01: Parallel Realities — The first experiment, Parallel Realities, starts from a volumetric capture of a real location, reconstructed as 3D Gaussian Splats. Using Marble, I generate an alternate version of that same space, something informed by the original architecture: abandoned, nature-reclaimed, alternate era. Then, using Spark (World Labs’ 3D Gaussian Splatting renderer for THREE.js) I make both realities coexist in the same spatial coordinate system. From there, I use a portal UX mechanic to let the user step between the real reconstruction and the Marble-generated version. Experiment 02: Hidden Depth The second experiment, Hidden Depth, does not transform a space as much as expand it. A captured location has a visual boundary (a mural, a doorway, a dark corridor) and Marble generates what exists beyond it. For example: a Montreal alley has a painted mural; step through it and you’re inside a world informed by what is actually depicted there. World Labs showcased part of this work here: And in their Spark 2.0 post: The project page is here: Why this matters to me — Being able to start from a real 3D Gaussian Splat scene and manipulate it with Marble opens up a lot of ideas. The 3DGS pipeline is becoming an increasingly compelling foundation for exploration, experimentation, and storytelling. What matters most to me right now is more control. The more I can steer the generated scene or world, the more useful the tool becomes. I want more features like the already existing multiple input images and Chisel, the blockout-based approach. I would like better local control, the ability to expand a generated world more and more while preserving coherence, and the ability to directly import 3D Gaussian Splat scenes to be used as a starting point. I want more ways to shape the result, not just a “prompt and hope” approach. — It is exciting to see this field moving from research and demos toward actual creative workflows.

Hugues Bruyère

69,960 次观看 • 3 个月前

2D is what the camera sees.👀 3D is what the world is. 🌍 Today, we’re proudly open-sourcing QuerySplat — an open-source feed-forward 3D model that turns a few unposed images into a navigable 3D scene in seconds (run on 4090🪶). Your input: a few unposed images Our output: a sharp, navigable 3D scene — generated in seconds! 🚀 🧠 organize scenes with 3D queries, not pixel-bound Gaussians 📷 no camera poses or manual calibration required ⚡ reconstruct new scenes in a single forward pass ✨ preserve sharp geometry and high-frequency appearance 🏆 achieve state-of-the-art results on DL3DV-Evaluation 🔓 code and model weights are now open-source From pixels to scenes, we are making 3D generation scalable. Try our free iOS app — it’s a lot of fun! 😆 🔗 Try it on the Web: 🔗 Paper: 🔗 Code & weights: 🔗 Project page: 🔗 #AIGC #3DGS #AI #Computervision #3D #artist #3dartist #3d

2D is what the camera sees.👀 3D is what the world is. 🌍 Today, we’re proudly open-sourcing QuerySplat — an open-source feed-forward 3D model that turns a few unposed images into a navigable 3D scene in seconds (run on 4090🪶). Your input: a few unposed images Our output: a sharp, navigable 3D scene — generated in seconds! 🚀 🧠 organize scenes with 3D queries, not pixel-bound Gaussians 📷 no camera poses or manual calibration required ⚡ reconstruct new scenes in a single forward pass ✨ preserve sharp geometry and high-frequency appearance 🏆 achieve state-of-the-art results on DL3DV-Evaluation 🔓 code and model weights are now open-source From pixels to scenes, we are making 3D generation scalable. Try our free iOS app — it’s a lot of fun! 😆 🔗 Try it on the Web: 🔗 Paper: 🔗 Code & weights: 🔗 Project page: 🔗 #AIGC #3DGS #AI #Computervision #3D #artist #3dartist #3d

InSpatio｜影溯

29,806 次观看 • 1 个月前

Dormant Memories — a gallery regrouping my recent experiments built from real-world captures, reconstructed as 3D Gaussian Splats, and in many cases manipulated using World Labs' Marble. Everything is built with Spark, the 3D Gaussian splat renderer released by World Labs. It runs in the browser and can be experienced across devices: desktop, mobile, and headsets like Apple Vision Pro and Quest 3. This work comes out of something I've been exploring for a few years now: capturing fragments of the world (places, objects, moments...) and turning them into spaces that can then be revisited, manipulated, and explored in real time. What particularly interests me here is the process of creating alternate versions of the same place; starting from something that physically exists and using it as a base to explore variations, instead of generating from scratch. These explorations are part of what we do at Dpt.... same tools, same curiosity, same collaborative energy that shape the studio's work on interactive and immersive experiences. Some of these experiments were featured as part of the Spark 2.0 release by World Labs. Their technical blog post is a must-read; link in the thread ↓ These are experiments. They aren't fully optimized, they aren't bug-free, and the UX has rough edges. The goal is to explore, test, and learn. #GaussianSplatting #3DGS #pointcloud #WorldModels #sparkjs #threejs

Dormant Memories — a gallery regrouping my recent experiments built from real-world captures, reconstructed as 3D Gaussian Splats, and in many cases manipulated using World Labs' Marble. Everything is built with Spark, the 3D Gaussian splat renderer released by World Labs. It runs in the browser and can be experienced across devices: desktop, mobile, and headsets like Apple Vision Pro and Quest 3. This work comes out of something I've been exploring for a few years now: capturing fragments of the world (places, objects, moments...) and turning them into spaces that can then be revisited, manipulated, and explored in real time. What particularly interests me here is the process of creating alternate versions of the same place; starting from something that physically exists and using it as a base to explore variations, instead of generating from scratch. These explorations are part of what we do at Dpt.... same tools, same curiosity, same collaborative energy that shape the studio's work on interactive and immersive experiences. Some of these experiments were featured as part of the Spark 2.0 release by World Labs. Their technical blog post is a must-read; link in the thread ↓ These are experiments. They aren't fully optimized, they aren't bug-free, and the UX has rough edges. The goal is to explore, test, and learn. #GaussianSplatting #3DGS #pointcloud #WorldModels #sparkjs #threejs

Hugues Bruyère

12,728 次观看 • 4 个月前

This is the most autistic thing I've ever vibe coded. A Mac app that turns your wallpaper into a 3D Earth view with day/night depending on where in the world you are. Can toggle: * Weather * Live flights * Zoomed views * Pollen/Air Quality * Globe spin Should I publish it?

This is the most autistic thing I've ever vibe coded. A Mac app that turns your wallpaper into a 3D Earth view with day/night depending on where in the world you are. Can toggle: * Weather * Live flights * Zoomed views * Pollen/Air Quality * Globe spin Should I publish it?

Alex Cohen

114,475 次观看 • 7 个月前