Gabriele Romagnoli
@GabRoXR • 8,456 subscribers
Showcasing the best of #XR & #AI for creatives and professionals | Tech Ambassador | Speaker Content Manager at @ShapesXR
Videos
"#AI will never create clean meshes" AI 👇 This is the new #AI model from NVIDIA called Meshtron (jeez what a name 😅). It generates meshes of complex 3D objects closely resembling those created by professional artists. It opens the door to a more realistic generation of detailed 3D assets for animation, gaming, and virtual environments. The current version accepts the following control inputs: - Point cloud: Determines the shape of the output mesh. - Face count: Determines the density of the output mesh. - Quad ratio: Switches between quad and triangle tessellation. - Creativity: Can be adjusted to generate extra details in the model. Meshtron can either serve as a stand-alone remesher to improve the quality of existing models, or be used in tandem with a text-to-3D or image-to-3D model to generate artist-grade meshes from scratch.
Gabriele Romagnoli2,176,484 Aufrufe • vor 1 Jahr
These are just 2 Keyframes. #AI did the rest. Cascadeur – The Future of Animation just released an impressive update that adds to the AI interpolation features that run locally on your PC 🤯. I am not an animator but creating a walk cycle took me literally 20 seconds... and the majority of that time was spent understanding how to add a new keyframe.
Gabriele Romagnoli728,455 Aufrufe • vor 6 Monaten
This web demo lets you create a portal to your own AI-generated world 👇 The Off-Axis spatial Kit built by Johannes Tscharn and Ian Curtis now supports the import function for your own splat. To get it, make sure to reach out to Ferhan Özkan and the SensAI Hackademy team, who have been building end-to-end workflow Kits for Unity, Unreal Engine and #WebXR.
Gabriele Romagnoli36,699 Aufrufe • vor 1 Monat
Animated #GaussianSplatting can now be streamed like a YouTube video. So far, you had to download large files BEFORE experiencing them anywhere, but Gracia has just released several demos that stream like a normal video. The first thing I tried was to make a side-by-side comparison, but I could hardly notice any difference. As long as you have a solid internet connection, you can play them on the web, headset and even on mobile in #augmentedreality. Interested in the latest in #3D, #XR and #AI? This is the type of news and updates I share every week on my newsletter (link on top of my profile).
Gabriele Romagnoli44,034 Aufrufe • vor 2 Monaten
This is the biggest announcement from Apple that nobody has talked about. Generating #GaussianSplatting with #AI using models like Trellis or Sam3D hasn’t made much sense...until now. Meshes are still far easier to edit, integrate cleanly into #3D workflows, and with recent retopology advances, you can get high-quality low-poly assets with detailed textures. That said, this is where the balance starts to shift toward 3DGS. Until recently, AI struggled to generate splats with proper view-dependent effects. That’s changed with LiTo, a new model from Apple that can generate 3DGS with realistic view-dependent rendering, and the results are impressive. They’re not ready for game development pipelines yet. But for VFX, e-commerce, configurators, architecture, and similar domains, the implications are significant. What would it take for you to seriously explore AI-generated 3DGS for non-gaming use cases?
Gabriele Romagnoli20,579 Aufrufe • vor 1 Monat
Playing #VR games "without" a headset... ... Absolute genius or total abomination? This is called PortalVR and it lets you play any SteamVR game or app on your Windows PC without wearing a headset. Just plug in your #MetaQuest via USB, put it on your desk facing you, and use your controllers while facing your screen. The setup and connection is quite seamless and the team came up with a clever locomotion system to simulate turning the user's head by just tapping the top of the thumbstick. Now the biggest question is: does this make any sense? In my opinion, to play "standard" VR #games, it doesn't. You lose the beauty of being immersed and reaching out to grab objects or looking around can be tricky. On the other hand, I think there might be experiences specifically designed with this interaction in mind like interactive installations or educational experiences that could greatly benefit from a more tactile interface than a mouse and a keyboard
Gabriele Romagnoli66,345 Aufrufe • vor 5 Monaten
This is the latest Google research on XR Interaction: World Mouse. Instead of using hand rays or gestures to point at things in the physical space, you move a cursor with a mouse and the system figures out where that cursor should sit in the 3D world. It works in two main ways: - On an object, the cursor follows the surface, so you can move precisely across walls, screens, furniture, or virtual objects. - Between objects, the system creates a smooth bridge through space, so the cursor can travel from one surface to another. That means you can go from a 2D panel to a 3D object, place content on a real wall, manipulate virtual objects, or interact with physical devices through digital proxies. This is the type of news, tools and product updates I share weekly in my newsletter. If you are into 3D XR and AI make sure to check it out (link on top of my profile). Based on “World Mouse: Exploring Interactions with a Cross-Reality Cursor” by Esen K. Tütüncü, Mar Gonzalez-Franco, Khushman Patel, and Eric J. Gonzalez
Gabriele Romagnoli28,846 Aufrufe • vor 2 Monaten
Here is what I learned from experiencing animated #GaussianSplatting in #mixedreality. File sizes are shrinking. Gracia manages to compress at around 1GB per minute, but once you are done with the download, the experience is mindblowing The crowded dance scene in the middle of the video (captured with a 180-degree rig) was a recent addition and a nice surprise to show how you can be immersed in other types of performances. Temporal stability is tricky. Fast-movements result in some artefacts (especially when you're getting closer). Transparencies are probably even more impressive than reflective surfaces The library keeps growing and I strongly advise you to check it out.
Gabriele Romagnoli50,448 Aufrufe • vor 5 Monaten
1:00
Sensitive content
This media may contain sensitive content.
Let me share a long-awaited behind-the-scenes video of how Simulon works. It all starts with setting up a scene. You set the location, lighting conditions and capture the light map with a few turns. After that, you can capture the environment mesh using the Lidar scanner. That's the moment the fun begins. You can bring in your #3D models with animations and capture a video of up to 50" using the #AR preview on screen. After that is done, you send the footage to the cloud and in a couple of minutes you get back an incredible #CGI shot. And this is just the beginning. Follow Gabriele Romagnoli to see how Simulon integrates with #Blender and in a full #VFX workflow. A special thanks to Divesh Naidoo for sharing early access and for building something 🤯
Gabriele Romagnoli142,504 Aufrufe • vor 2 Jahren
The new avatars on Apple #VisionPro are undoubtedly the biggest addition. Here is why 👇 The leap in quality when compared to the original persona 1.0 and 1.1 (shown in the video capture around 1 year ago) is just massive and reaches a level of fidelity that can compete with the Codec avatars showcased by Meta. The difference here is that it all happens on a standalone device, with no complicated setup (also shown in the video) and capitalises on a unique feature of the headset, with NO OTHER device coming even close. Co-presence is one of the aspects no one has nailed yet, and this is the biggest leap in that direction.
Gabriele Romagnoli65,263 Aufrufe • vor 11 Monaten
AI Mocap tools are evolving at breakneck speed... and I could find more than 10 all compiled and compared in a list you can get by subscribing to the XR AI Spotlight newsletter (link on top of my profile). This is Cartwheel and the stepping on that box was one of the most solid I saw. There is a bit of floatiness in the feet and they are not perfectly grounded, but since I tested the tool, the team has pushed massive improvement. What AI mocap tool do you use in your workflow? Let me know in the comments
Gabriele Romagnoli34,842 Aufrufe • vor 6 Monaten
Meta released SAM3 a model to reconstruct objects from segmented images. I tried it and here are 4 things I found out - The generation is incredibly fast when compared with other 3D GenAI models - The models generated are not meshes but Gaussian Splatting. I was surprised that in the paper it was not mentioned anywhere - When generating multiple objects the arrangement is a bit messed up. - Unfortunately, it is impossible to download any of the .ply generated 😔 I cannot stop thinking about the implications of this in XR. Objects in the real world could be almost instantly turned into photorealistic copies and even rearranged to the user's wish.
Gabriele Romagnoli32,424 Aufrufe • vor 6 Monaten
Typing was NOT the most surprising part of the surface keyboard pushed in #MetaQuest latest update. Let me start by telling you how to try it yourself (available only on Quest 3) 1- On your phone, open the Meta Horizon app. 2- Go to Devices and select your headset 3- Tap Headset settings, then tap Advanced settings. 4- Tap the toggle next to Public Test Channel to join Meta Quest PTC (Public Test Channel) 5- After go in the headset settings > experimental features > surface keyboard ⌨️ Typing felt very responsive but what caught me off guard was the touchpad. 🖱️Even when looking straight at the screen the cursor moves accurately as if you are using an actual trackpad. Tapping acts as a click and it just works. ✌️ It also responds to the scrolling motion using 2 Fingers!
Gabriele Romagnoli20,280 Aufrufe • vor 4 Monaten
Start designing for #VisionPro. Break the boundaries of 2D screens. I have done it... You can as well. Let me break down what happens in this video and explain how a tool like ShapesXR can help you #design for #spatialcomputing right now. 1️⃣ Assets and components you created in #Figma can be imported and synched in ShapesXR so you can start designing in #mixedreality without any #3d skills 2️⃣ You can prototype Gaze & Pinch interactions with head pose (on Quest 2 or 3) or actual eye tracking on Quest Pro. 3️⃣ You can switch seamlessly between VR and MR. Get tiny when you want to manipulate objects precisely or HUGE when you want to have a doll house view of your creation Be Brave, Be Bold, Think Spatial.
Gabriele Romagnoli67,836 Aufrufe • vor 2 Jahren
You can explore these AI-generated worlds on #VisionPro in 2 ways 👇 Option 1 is to get a link directly from Marble, the platform from World Labs that has recently opened beta access and allows users to use a prompt image to generate worlds as #GaussianSplatting. You just click on share and get a link you can input in your browser's headset. The viewer has several limitations since you cannot adjust scale, set the starting point or navigate the scene freely. Option 2 is to use Supersplat from Will Eastcott and his team. There you can do some editing and once in VR you can navigate around, looking and pinching to explore the generated portions of the model more freely. If you are into Gaussian Splatting and #VR remember to check out the list of updated tools that allow you to experience Gaussian Splatting in any headset. Just subscribe to my newsletter (link on top of my profile) and you will get it dircetly in your inbox.
Gabriele Romagnoli24,881 Aufrufe • vor 8 Monaten
If you are still in doubt about the power of 3DGS for immersive you should check this world in VRChat . It is called Spatialography and is one of the best showcases of #GaussianSplatting in #VR available in my list of apps, tools and tech demos to experience Gaussian Splatting on Quest or #VisionPro. 📢 To get the list just subscribe to my newsletter (link at the top of my profile). The list will be available for ONLY 5 more days, after that, it's GONE... why? Because I am launching something BIGGER, much more in-depth, that is going to be much "harder" to get
Gabriele Romagnoli15,554 Aufrufe • vor 4 Monaten
This might be the craziest workflow you'll see this week. From #VR greyboxing to 3D worlds with the help of #AI. The 2 tools I used are ShapesXR and Marble from World Labs. Let me tell you why and how. - ShapesXR is the most powerful and fastest way to assemble 3D scenes at scale. - The library of assets + ability to sketch from procedural primitives gives the perfect balance between speed and freedom. - ShapesXR allows you to export your creation as a glTF that can be used as a "3D prompt" into Marble - Marble now allows you to create a panorama using a 3D scene as input increasing DRAMATICALLY the control you have during the creation process - Through the workflow you can adjust textures and objects in the scene using AI inpainting - There are tons of options on how to use the final output (create a #GaussianSplatting, export collision mesh and even a detailed mesh 🤯) This is the perfect examples of how VR creative tools improve and accelerate the design process... so make sure to follow me if that's your jam 😉.
Gabriele Romagnoli20,126 Aufrufe • vor 6 Monaten