Gabriele Romagnoli
@GabRoXR • 8,522 subscribers
Showcasing the best of #XR & #AI for creatives and professionals | Tech Ambassador | Speaker Content Manager at @ShapesXR
Videos
"#AI will never create clean meshes" AI 👇 This is the new #AI model from NVIDIA called Meshtron (jeez what a name 😅). It generates meshes of complex 3D objects closely resembling those created by professional artists. It opens the door to a more realistic generation of detailed 3D assets for animation, gaming, and virtual environments. The current version accepts the following control inputs: - Point cloud: Determines the shape of the output mesh. - Face count: Determines the density of the output mesh. - Quad ratio: Switches between quad and triangle tessellation. - Creativity: Can be adjusted to generate extra details in the model. Meshtron can either serve as a stand-alone remesher to improve the quality of existing models, or be used in tandem with a text-to-3D or image-to-3D model to generate artist-grade meshes from scratch.
Gabriele Romagnoli2,176,749 次观看 • 1 年前
These are just 2 Keyframes. #AI did the rest. Cascadeur – The Future of Animation just released an impressive update that adds to the AI interpolation features that run locally on your PC 🤯. I am not an animator but creating a walk cycle took me literally 20 seconds... and the majority of that time was spent understanding how to add a new keyframe.
Gabriele Romagnoli729,894 次观看 • 7 个月前
I had to do this for "science" 🥼🔬😅. Jokes aside, there were a few things I learned by testing #mixedreality outdoor: 1- Sun was coming and going BUT the tracking on the #Quest3 was surprisingly stable.... including the controllers 2- After the initial hesitation to move around it was great to walk and build without the restrictions of walls or furniture. Right now the majority of the apps are designed for a standing experience without much movement or locomotion happening in the real world. That is a real shame because the moments when I move physically are when I feel the most immersed. Maybe #outdoor #MR can bring back some of that magic 3- Jumping in the whole DID feel believable and "kinda scary" to be honest. Are there other #MR apps that are great to test outdoors? Let me know below 👇
Gabriele Romagnoli598,220 次观看 • 2 年前
#GaussianSplatting are impossible to animate. Lazar Bogdan made this tutorial to prove otherwise: It is 1h 20 min and covers on how to rig, skin and animate a fly in Houdini. This is the type of finding I share with the Pro members of my newsletter XR AI Spotlight, every week. I spend countless hours finding and curating the latest updates and valuable resources for professionals and creatives at the intersection of hashtag#3D, hashtag#XR and hashtag#AI. Make sure to check it out.
Gabriele Romagnoli17,102 次观看 • 19 天前
Animated #GaussianSplatting can now be streamed like a YouTube video. So far, you had to download large files BEFORE experiencing them anywhere, but Gracia has just released several demos that stream like a normal video. The first thing I tried was to make a side-by-side comparison, but I could hardly notice any difference. As long as you have a solid internet connection, you can play them on the web, headset and even on mobile in #augmentedreality. Interested in the latest in #3D, #XR and #AI? This is the type of news and updates I share every week on my newsletter (link on top of my profile).
Gabriele Romagnoli44,321 次观看 • 4 个月前
Playing #VR games "without" a headset... ... Absolute genius or total abomination? This is called PortalVR and it lets you play any SteamVR game or app on your Windows PC without wearing a headset. Just plug in your #MetaQuest via USB, put it on your desk facing you, and use your controllers while facing your screen. The setup and connection is quite seamless and the team came up with a clever locomotion system to simulate turning the user's head by just tapping the top of the thumbstick. Now the biggest question is: does this make any sense? In my opinion, to play "standard" VR #games, it doesn't. You lose the beauty of being immersed and reaching out to grab objects or looking around can be tricky. On the other hand, I think there might be experiences specifically designed with this interaction in mind like interactive installations or educational experiences that could greatly benefit from a more tactile interface than a mouse and a keyboard
Gabriele Romagnoli66,345 次观看 • 7 个月前
This web demo lets you create a portal to your own AI-generated world 👇 The Off-Axis spatial Kit built by Johannes Tscharn and Ian Curtis now supports the import function for your own splat. To get it, make sure to reach out to Ferhan Özkan and the SensAI Hackademy team, who have been building end-to-end workflow Kits for Unity, Unreal Engine and #WebXR.
Gabriele Romagnoli36,773 次观看 • 3 个月前
Here is what I learned from experiencing animated #GaussianSplatting in #mixedreality. File sizes are shrinking. Gracia manages to compress at around 1GB per minute, but once you are done with the download, the experience is mindblowing The crowded dance scene in the middle of the video (captured with a 180-degree rig) was a recent addition and a nice surprise to show how you can be immersed in other types of performances. Temporal stability is tricky. Fast-movements result in some artefacts (especially when you're getting closer). Transparencies are probably even more impressive than reflective surfaces The library keeps growing and I strongly advise you to check it out.
Gabriele Romagnoli50,448 次观看 • 7 个月前
Let me share a long-awaited behind-the-scenes video of how Simulon works. It all starts with setting up a scene. You set the location, lighting conditions and capture the light map with a few turns. After that, you can capture the environment mesh using the Lidar scanner. That's the moment the fun begins. You can bring in your #3D models with animations and capture a video of up to 50" using the #AR preview on screen. After that is done, you send the footage to the cloud and in a couple of minutes you get back an incredible #CGI shot. And this is just the beginning. Follow Gabriele Romagnoli to see how Simulon integrates with #Blender and in a full #VFX workflow. A special thanks to Divesh Naidoo for sharing early access and for building something 🤯
Gabriele Romagnoli142,533 次观看 • 2 年前
This is the latest Google research on XR Interaction: World Mouse. Instead of using hand rays or gestures to point at things in the physical space, you move a cursor with a mouse and the system figures out where that cursor should sit in the 3D world. It works in two main ways: - On an object, the cursor follows the surface, so you can move precisely across walls, screens, furniture, or virtual objects. - Between objects, the system creates a smooth bridge through space, so the cursor can travel from one surface to another. That means you can go from a 2D panel to a 3D object, place content on a real wall, manipulate virtual objects, or interact with physical devices through digital proxies. This is the type of news, tools and product updates I share weekly in my newsletter. If you are into 3D XR and AI make sure to check it out (link on top of my profile). Based on “World Mouse: Exploring Interactions with a Cross-Reality Cursor” by Esen K. Tütüncü, Mar Gonzalez-Franco, Khushman Patel, and Eric J. Gonzalez
Gabriele Romagnoli28,945 次观看 • 4 个月前
This is the biggest announcement from Apple that nobody has talked about. Generating #GaussianSplatting with #AI using models like Trellis or Sam3D hasn’t made much sense...until now. Meshes are still far easier to edit, integrate cleanly into #3D workflows, and with recent retopology advances, you can get high-quality low-poly assets with detailed textures. That said, this is where the balance starts to shift toward 3DGS. Until recently, AI struggled to generate splats with proper view-dependent effects. That’s changed with LiTo, a new model from Apple that can generate 3DGS with realistic view-dependent rendering, and the results are impressive. They’re not ready for game development pipelines yet. But for VFX, e-commerce, configurators, architecture, and similar domains, the implications are significant. What would it take for you to seriously explore AI-generated 3DGS for non-gaming use cases?
Gabriele Romagnoli20,662 次观看 • 2 个月前
The new avatars on Apple #VisionPro are undoubtedly the biggest addition. Here is why 👇 The leap in quality when compared to the original persona 1.0 and 1.1 (shown in the video capture around 1 year ago) is just massive and reaches a level of fidelity that can compete with the Codec avatars showcased by Meta. The difference here is that it all happens on a standalone device, with no complicated setup (also shown in the video) and capitalises on a unique feature of the headset, with NO OTHER device coming even close. Co-presence is one of the aspects no one has nailed yet, and this is the biggest leap in that direction.
Gabriele Romagnoli65,263 次观看 • 1 年前
#VisionPro pushes for far interactions but is that the best way? With gaze and pinch, #Apple seems to discourage developers from implementing more direct hand interactions into their experiences. There are certainly some advantages but I thought the whole purpose of #VR and #spatialcomputing was to be immersed and feel like you are interacting with #digital objects as if they were real. I am even more convinced after testing this SDK from SpectreXR that takes advantage of the Nvidia physics engine to make hand interactions feel more natural. Would you use more direct interactions like in the video or stick with far gaze+pinch?
Gabriele Romagnoli104,043 次观看 • 2 年前
AI Mocap tools are evolving at breakneck speed... and I could find more than 10 all compiled and compared in a list you can get by subscribing to the XR AI Spotlight newsletter (link on top of my profile). This is Cartwheel and the stepping on that box was one of the most solid I saw. There is a bit of floatiness in the feet and they are not perfectly grounded, but since I tested the tool, the team has pushed massive improvement. What AI mocap tool do you use in your workflow? Let me know in the comments
Gabriele Romagnoli34,907 次观看 • 8 个月前
Meta released SAM3 a model to reconstruct objects from segmented images. I tried it and here are 4 things I found out - The generation is incredibly fast when compared with other 3D GenAI models - The models generated are not meshes but Gaussian Splatting. I was surprised that in the paper it was not mentioned anywhere - When generating multiple objects the arrangement is a bit messed up. - Unfortunately, it is impossible to download any of the .ply generated 😔 I cannot stop thinking about the implications of this in XR. Objects in the real world could be almost instantly turned into photorealistic copies and even rearranged to the user's wish.
Gabriele Romagnoli32,428 次观看 • 8 个月前
Typing was NOT the most surprising part of the surface keyboard pushed in #MetaQuest latest update. Let me start by telling you how to try it yourself (available only on Quest 3) 1- On your phone, open the Meta Horizon app. 2- Go to Devices and select your headset 3- Tap Headset settings, then tap Advanced settings. 4- Tap the toggle next to Public Test Channel to join Meta Quest PTC (Public Test Channel) 5- After go in the headset settings > experimental features > surface keyboard ⌨️ Typing felt very responsive but what caught me off guard was the touchpad. 🖱️Even when looking straight at the screen the cursor moves accurately as if you are using an actual trackpad. Tapping acts as a click and it just works. ✌️ It also responds to the scrolling motion using 2 Fingers!
Gabriele Romagnoli20,670 次观看 • 5 个月前
Start designing for #VisionPro. Break the boundaries of 2D screens. I have done it... You can as well. Let me break down what happens in this video and explain how a tool like ShapesXR can help you #design for #spatialcomputing right now. 1️⃣ Assets and components you created in #Figma can be imported and synched in ShapesXR so you can start designing in #mixedreality without any #3d skills 2️⃣ You can prototype Gaze & Pinch interactions with head pose (on Quest 2 or 3) or actual eye tracking on Quest Pro. 3️⃣ You can switch seamlessly between VR and MR. Get tiny when you want to manipulate objects precisely or HUGE when you want to have a doll house view of your creation Be Brave, Be Bold, Think Spatial.
Gabriele Romagnoli67,836 次观看 • 2 年前
Have you made some #3D models with #gaussiansplatting using Luma or polycam ? Here is how you can now clean them up directly on the web: People nowadays call them "Splats". These 3D models have the advantage of being extremely light compared to polygon-based 3D models. You can use various apps to generate these assets like the one I mentioned above. Often what you need is just a video or a set of images that you can shoot directly from your phone. To my knowledge, none of those apps right now offer the ability to edit and "clean" the model you get.... UNTIL NOW! You can go to from PlayCanvas import the splats you have downloaded and start cleaning it up directly on your #browser. After that the asset is ready to be exported as a PLY file Was this useful? Follow me for more content around #XR and #AI 😉
Gabriele Romagnoli70,966 次观看 • 2 年前