Loading video...

Video Failed to Load

Go Home

RPG Maker U2U has been officially revealed which will allow users to create HD-2D-style games. Called "P2D" (Perspective 2D), users can create 3D depth and rich effect expression to conventional 2D maps. 2D assets from previous RPG Maker titles can be used as map tiles for decoration, and more...

KaroshiMyriad's profile picture

Kars

99,997 subscribers

1,743,288 views • 2 months ago •via X (Twitter)

GamingEducationScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

fine, i'll build it myself, open source ofc Launching Three-Maps, a rapid blockout editor for building game maps for threejs games: Create a floor plan in 2D, automatically generate 3D levels from it, edit them, export to .glb and use in your games (see end of video) Features: - Floorplan 2D: A 2D canvas to map out the floor - Edit Mode for Objects: Edit Vertices/Edges/Faces, extrude, bevel, fillet and chamfer - Brush Mode: Rapidly build map elements with a RealtimeCSG like experience. Cubes, Stairs, Doors, Arch, Cylinder, Custom polygons. Yes, you can even curve the stairs - Modifiers: Yes, you read that right, slap that array modifier on your pillar and call it a day - Terrain Generator: Perlin, Voronoi, you name it. Oh yes, you can rasterize it and edit every single vertex in edit mode. Keyboard shortcuts are mostly same as blender to activate that muscle memory (G/R/S, axis lock XYZ) Soon: - AI integration. Boot that MCP and directly connect your fav AI agent to build the floorplan, generate 3D maps and more (you will love how this movie ends) It's a little rough and in early, very early alpha so come and join as a contributor instead of calling it slop It's all vibe coded btw, i have no idea what the code does I need this to build my game, as i'm a total blender noob and threejs is great, so why not? Outshipping every CAD and 3D startup just because i want to build a game, that's how you accelerate
1:54

fine, i'll build it myself, open source ofc Launching Three-Maps, a rapid blockout editor for building game maps for threejs games: Create a floor plan in 2D, automatically generate 3D levels from it, edit them, export to .glb and use in your games (see end of video) Features: - Floorplan 2D: A 2D canvas to map out the floor - Edit Mode for Objects: Edit Vertices/Edges/Faces, extrude, bevel, fillet and chamfer - Brush Mode: Rapidly build map elements with a RealtimeCSG like experience. Cubes, Stairs, Doors, Arch, Cylinder, Custom polygons. Yes, you can even curve the stairs - Modifiers: Yes, you read that right, slap that array modifier on your pillar and call it a day - Terrain Generator: Perlin, Voronoi, you name it. Oh yes, you can rasterize it and edit every single vertex in edit mode. Keyboard shortcuts are mostly same as blender to activate that muscle memory (G/R/S, axis lock XYZ) Soon: - AI integration. Boot that MCP and directly connect your fav AI agent to build the floorplan, generate 3D maps and more (you will love how this movie ends) It's a little rough and in early, very early alpha so come and join as a contributor instead of calling it slop It's all vibe coded btw, i have no idea what the code does I need this to build my game, as i'm a total blender noob and threejs is great, so why not? Outshipping every CAD and 3D startup just because i want to build a game, that's how you accelerate

robot 2.0

106,664 views • 5 months ago

In collaboration with Intel, our Depth Fusion showcases the power of our LDM3D diffusion model in generating 360° views from text prompts provided by the user. The LDM3D diffusion model generates a 2D RGB image and its corresponding relative depth map providing a complete RGBD representation corresponding to the text prompt. The LDM 3D model is a specialized version of the stable diffusion V 1.4 model that has been modified to fit both image and depth map data.The model was then fine tuned on a subset of the Laion400M data set - large scale image caption data set. The depth maps used to fine tune our model were generated by the DPTBeiT large 512 depth estimation model that provides highly accurate relative depth estimates for each pixel. We take the generated 2D RGB image and depth map and use them to compute a 360° projection using touchdesigner. Touchdesigner is a versatile platform that allows for the creation of immersive and interactive multimedia experiences. Our application harnesses the power of touchdesigner to bring the generated 360° views to life, providing users with a unique and engaging way to experience their text prompts, whether it’s a description of a tranquil forest, a noisy cityscape or a futuristic sci fi world. Our depth fusion can bring these concepts to life in a vivid and immersive detail. - Scottie Fox, VP Engineering Blockade Labs ScottieFox #AI #VR #3D #gamedev #stablediffusion
2:29

In collaboration with Intel, our Depth Fusion showcases the power of our LDM3D diffusion model in generating 360° views from text prompts provided by the user. The LDM3D diffusion model generates a 2D RGB image and its corresponding relative depth map providing a complete RGBD representation corresponding to the text prompt. The LDM 3D model is a specialized version of the stable diffusion V 1.4 model that has been modified to fit both image and depth map data.The model was then fine tuned on a subset of the Laion400M data set - large scale image caption data set. The depth maps used to fine tune our model were generated by the DPTBeiT large 512 depth estimation model that provides highly accurate relative depth estimates for each pixel. We take the generated 2D RGB image and depth map and use them to compute a 360° projection using touchdesigner. Touchdesigner is a versatile platform that allows for the creation of immersive and interactive multimedia experiences. Our application harnesses the power of touchdesigner to bring the generated 360° views to life, providing users with a unique and engaging way to experience their text prompts, whether it’s a description of a tranquil forest, a noisy cityscape or a futuristic sci fi world. Our depth fusion can bring these concepts to life in a vivid and immersive detail. - Scottie Fox, VP Engineering Blockade Labs ScottieFox #AI #VR #3D #gamedev #stablediffusion

Blockade Labs

11,439 views • 3 years ago

Exploring 3D Generation Capabilities in Alchemist AI🔮 Alchemist AI now enables users to create basic 3D assets, including models, objects, environments, and simulations, expanding its capabilities beyond 2D asset generation. What Does This Mean for You? • 3D Environment Creation: Generate virtual worlds, such as game environments or fantasy landscapes. Define parameters like terrain or structures to create dynamic and adaptable spaces tailored to your needs. • Customizable 3D Models and Objects: Design and modify 3D assets by adjusting dimensions, materials, and textures. Whether crafting prototypes or characters, users maintain full creative control. • Interactive Simulations: Build physics-based simulations or animated scenes. With upcoming support for sprite libraries and animation rigs, fine-tune object behaviors and interactions to suit your projects. With future API updates, Alchemist AI’s 3D generation capabilities will further expand, enhancing tools for creating models, environments, and simulations. AI-assisted text-to-3D will also be introduced—just describe your vision, such as 'space station' or 'orange sports car' and the system will generate customizable base assets.
0:20

Exploring 3D Generation Capabilities in Alchemist AI🔮 Alchemist AI now enables users to create basic 3D assets, including models, objects, environments, and simulations, expanding its capabilities beyond 2D asset generation. What Does This Mean for You? • 3D Environment Creation: Generate virtual worlds, such as game environments or fantasy landscapes. Define parameters like terrain or structures to create dynamic and adaptable spaces tailored to your needs. • Customizable 3D Models and Objects: Design and modify 3D assets by adjusting dimensions, materials, and textures. Whether crafting prototypes or characters, users maintain full creative control. • Interactive Simulations: Build physics-based simulations or animated scenes. With upcoming support for sprite libraries and animation rigs, fine-tune object behaviors and interactions to suit your projects. With future API updates, Alchemist AI’s 3D generation capabilities will further expand, enhancing tools for creating models, environments, and simulations. AI-assisted text-to-3D will also be introduced—just describe your vision, such as 'space station' or 'orange sports car' and the system will generate customizable base assets.

ALCHEMIST AI 🔮

35,673 views • 1 year ago

AI has entered the gaming space. We’ve gone from casually building websites to casually building games. This game was created using Verse8 , which is a cloud-native platform that turns text prompt into playable games. - Agent 8 is specialized for building 2D and 3D multiplayer games - It handles code, visuals, and sound - Everything runs in the browser, with instant publish and share Here are 6 examples and how you can try it 👇
1:08

AI has entered the gaming space. We’ve gone from casually building websites to casually building games. This game was created using Verse8 , which is a cloud-native platform that turns text prompt into playable games. - Agent 8 is specialized for building 2D and 3D multiplayer games - It handles code, visuals, and sound - Everything runs in the browser, with instant publish and share Here are 6 examples and how you can try it 👇

AshutoshShrivastava

15,729 views • 1 year ago

Midjourney announces the world’s first full-body ultrasound CT scanner • Goal is to bring affordable full-body imaging to everyone on Earth • Users are submerged in water during the scan • Creates detailed 3D body maps in under a minute • Can map more than 25 organs and anatomical structures in detail • No radiation is used • Working with the FDA for approvals on diagnostic use • Plans to bring the tech to market by the end of 2027 (via Midjourney)
2:02

Midjourney announces the world’s first full-body ultrasound CT scanner • Goal is to bring affordable full-body imaging to everyone on Earth • Users are submerged in water during the scan • Creates detailed 3D body maps in under a minute • Can map more than 25 organs and anatomical structures in detail • No radiation is used • Working with the FDA for approvals on diagnostic use • Plans to bring the tech to market by the end of 2027 (via Midjourney)

Culture Crave 🍿

1,115,692 views • 1 month ago

Available now: Stems 🥳 Pro & Premier users can now separate the vocals and instrumentals from songs, which will give you more control over how to use Suno. 1. Go to Library or Create, and click the vertical “...” on a song row 2. Click “Get Stems” 3. If you’re not there already, you’ll be navigated to the Create page where you can see two new clips that are generating. One will be the instrumentals, and the other will be the vocals. You can then download or extend each individual clip. We’re so excited to see what you create! ❤️ Team Suno
0:32

Available now: Stems 🥳 Pro & Premier users can now separate the vocals and instrumentals from songs, which will give you more control over how to use Suno. 1. Go to Library or Create, and click the vertical “...” on a song row 2. Click “Get Stems” 3. If you’re not there already, you’ll be navigated to the Create page where you can see two new clips that are generating. One will be the instrumentals, and the other will be the vocals. You can then download or extend each individual clip. We’re so excited to see what you create! ❤️ Team Suno

Suno

100,285 views • 2 years ago

You will soon be able to create full 3D games from a single prompt. Unity just announced Muse, an AI-powered assistant for video-game creation. The platform will be closely connected to the Unity Editor, enabling text prompts and sketches, rather than traditional manual coding which can be more time-consuming
0:41

You will soon be able to create full 3D games from a single prompt. Unity just announced Muse, an AI-powered assistant for video-game creation. The platform will be closely connected to the Unity Editor, enabling text prompts and sketches, rather than traditional manual coding which can be more time-consuming

Lior Alexander

63,178 views • 3 years ago

i was thinking of making some NTE renders so i imported some maps and wow they even used AI to create assets also you can see the ingame screenshot as well as much as i want this game to be good it keep becoming more trash now i wait for Ananta they didn't learn shit from tower of fanatasy but made it 100x worse #NTE #NevernesstoEverness
0:31

i was thinking of making some NTE renders so i imported some maps and wow they even used AI to create assets also you can see the ingame screenshot as well as much as i want this game to be good it keep becoming more trash now i wait for Ananta they didn't learn shit from tower of fanatasy but made it 100x worse #NTE #NevernesstoEverness

Urek_Mazino | comms open!

383,061 views • 2 months ago

Alibaba presents MIMO Controllable Character Video Synthesis with Spatial Decomposed Modeling Character video synthesis aims to produce realistic videos of animatable characters within lifelike scenes. As a fundamental problem in the computer vision and graphics community, 3D works typically require multi-view captures for per-case training, which severely limits their applicability of modeling arbitrary characters in a short time. Recent 2D methods break this limitation via pre-trained diffusion models, but they struggle for pose generality and scene interaction. To this end, we propose MIMO, a novel framework which can not only synthesize character videos with controllable attributes (i.e., character, motion and scene) provided by simple user inputs, but also simultaneously achieve advanced scalability to arbitrary characters, generality to novel 3D motions, and applicability to interactive real-world scenes in a unified framework. The core idea is to encode the 2D video to compact spatial codes, considering the inherent 3D nature of video occurrence. Concretely, we lift the 2D frame pixels into 3D using monocular depth estimators, and decompose the video clip to three spatial components (i.e., main human, underlying scene, and floating occlusion) in hierarchical layers based on the 3D depth. These components are further encoded to canonical identity code, structured motion code and full scene code, which are utilized as control signals of synthesis process. The design of spatial decomposed modeling enables flexible user control, complex motion expression, as well as 3D-aware synthesis for scene interactions. Experimental results demonstrate effectiveness and robustness of the proposed method.
1:29

Alibaba presents MIMO Controllable Character Video Synthesis with Spatial Decomposed Modeling Character video synthesis aims to produce realistic videos of animatable characters within lifelike scenes. As a fundamental problem in the computer vision and graphics community, 3D works typically require multi-view captures for per-case training, which severely limits their applicability of modeling arbitrary characters in a short time. Recent 2D methods break this limitation via pre-trained diffusion models, but they struggle for pose generality and scene interaction. To this end, we propose MIMO, a novel framework which can not only synthesize character videos with controllable attributes (i.e., character, motion and scene) provided by simple user inputs, but also simultaneously achieve advanced scalability to arbitrary characters, generality to novel 3D motions, and applicability to interactive real-world scenes in a unified framework. The core idea is to encode the 2D video to compact spatial codes, considering the inherent 3D nature of video occurrence. Concretely, we lift the 2D frame pixels into 3D using monocular depth estimators, and decompose the video clip to three spatial components (i.e., main human, underlying scene, and floating occlusion) in hierarchical layers based on the 3D depth. These components are further encoded to canonical identity code, structured motion code and full scene code, which are utilized as control signals of synthesis process. The design of spatial decomposed modeling enables flexible user control, complex motion expression, as well as 3D-aware synthesis for scene interactions. Experimental results demonstrate effectiveness and robustness of the proposed method.

AK

148,998 views • 1 year ago

We are excited to announce that “The Museum” has been nominated at this year’s The Game Awards. 10 maps have been chosen to represent the best of UEFN inside Fortnite. This will be a fan voting category so we hope you will take the time to cast a vote inside the game. UEFN has created a portal inside FortniteGame that connects all 10 nominated maps and an area to vote. It can be found at island code 0853-1358-8532. We have also added a portal inside “The Museum” which can be found at 5592-6865-3961. As many of you know we have been exploring the changing landscape of music and entertainment for a while now. Blockchain, virtual reality concerts and immersive worlds in UEFN provide us tools to create new and exciting experiences. Why just WATCH when you can participate? Why just LISTEN when you can play? With UEFN’s tools you can do it all. We are so incredibly proud to be nominated for this award and hope more artists create experiences for their communities. Please check out the a7x - fortnite channel on Discord for more information on our 3 maps and songs in Fortnite Festival. Now go vote!
0:22

We are excited to announce that “The Museum” has been nominated at this year’s The Game Awards. 10 maps have been chosen to represent the best of UEFN inside Fortnite. This will be a fan voting category so we hope you will take the time to cast a vote inside the game. UEFN has created a portal inside FortniteGame that connects all 10 nominated maps and an area to vote. It can be found at island code 0853-1358-8532. We have also added a portal inside “The Museum” which can be found at 5592-6865-3961. As many of you know we have been exploring the changing landscape of music and entertainment for a while now. Blockchain, virtual reality concerts and immersive worlds in UEFN provide us tools to create new and exciting experiences. Why just WATCH when you can participate? Why just LISTEN when you can play? With UEFN’s tools you can do it all. We are so incredibly proud to be nominated for this award and hope more artists create experiences for their communities. Please check out the a7x - fortnite channel on Discord for more information on our 3 maps and songs in Fortnite Festival. Now go vote!

Avenged Sevenfold

34,516 views • 1 year ago

Interesting times in the maps space, and its exciting there is so much buzz - maps are awesome :) My parents Rakesh and Rashmi Verma pioneered digital mapping in India in 1995, returning from the US after a successful career there with the passion and desire to do something unique for India. And it’s been 20 years for me personally in the mapping space, since I was a 19-year old Stanford engineering undergraduate student and got involved in starting India’s first interactive mapping portal, I realise MapmyIndia is a relatively lesser known company amidst more consumer facing global and local players, so it would be great if this post can be amplified, so that more people can be made aware 🙏 Warm regards, Rohan Verma CEO & ED, MapmyIndia *** A few thoughts on maps: 1) Accuracy and quality of maps is critical. I’d caution folks to check out quality and reliability of maps by browsing those maps in areas familiar to them, and if they notice errors in them, in terms of incorrect places marked wrongly on the map, it should serve as a reminder not to rely on such maps. I’ve personally looked at the maps of various global and local players, and find so many inaccuracies, which confirms my belief that the difficult art and science of map-making is not as easy and people may imagine or claim. 2) What’s exciting about Mappls MapmyIndia, as a home-grown indigenous deep tech digital products and platforms company, is that not just did we pioneer digital mapping in India since 1995, when there were no other digital maps available for the country, but back then, and even now, we’ve always built the most cutting-edge tech to build the most capable maps and empower our customers, users and developers with the most comprehensive and advanced solutions. Over the last 15 or so years, there have actually been many global and local players who have come into the mapping market, yet for some reason or the other, they haven’t sustained or maintained quality. On our side we have continuously innovated in our products and tech - already bringing and making the most advanced featured available into our 4D HD maps covering 360 RealViews and 3D drone and digital twin based maps, immersive views and RealVerses - and focused on solving the needs of Indian consumers and enterprises, and served customers and users with humility, passion and consistency, with a solid and sustainable business model to ensure and provide a long-term reliable mapping solution for customers and the country. 3) Here’s an explainer video of our maps, tech & APIs which focus on how developers, users and customers can leverage our solutions to get their needs solved in the best way. Do watch - you’ll be pleasantly surprised and happy at the offering. 4) To try out as a developer for yourself, visit We’re glad that tens of thousands of developers, and their hundreds of millions of users, benefit from Mappls MapmyIndia Maps & APIs everyday, using both our free plans and our commercial plans. Do try for your own needs as a developer. 5) In one sense, the quality and capability of Mappls MapmyIndia is proven to be better and more useful and valuable through our free consumer Mappls MapmyIndia app (learn more and download from which has gotten love from millions of consumers who are able to navigate safer and better. MapmyIndia Mappls Rakesh Verma @RashmiV1956
15:29

Interesting times in the maps space, and its exciting there is so much buzz - maps are awesome :) My parents Rakesh and Rashmi Verma pioneered digital mapping in India in 1995, returning from the US after a successful career there with the passion and desire to do something unique for India. And it’s been 20 years for me personally in the mapping space, since I was a 19-year old Stanford engineering undergraduate student and got involved in starting India’s first interactive mapping portal, I realise MapmyIndia is a relatively lesser known company amidst more consumer facing global and local players, so it would be great if this post can be amplified, so that more people can be made aware 🙏 Warm regards, Rohan Verma CEO & ED, MapmyIndia *** A few thoughts on maps: 1) Accuracy and quality of maps is critical. I’d caution folks to check out quality and reliability of maps by browsing those maps in areas familiar to them, and if they notice errors in them, in terms of incorrect places marked wrongly on the map, it should serve as a reminder not to rely on such maps. I’ve personally looked at the maps of various global and local players, and find so many inaccuracies, which confirms my belief that the difficult art and science of map-making is not as easy and people may imagine or claim. 2) What’s exciting about Mappls MapmyIndia, as a home-grown indigenous deep tech digital products and platforms company, is that not just did we pioneer digital mapping in India since 1995, when there were no other digital maps available for the country, but back then, and even now, we’ve always built the most cutting-edge tech to build the most capable maps and empower our customers, users and developers with the most comprehensive and advanced solutions. Over the last 15 or so years, there have actually been many global and local players who have come into the mapping market, yet for some reason or the other, they haven’t sustained or maintained quality. On our side we have continuously innovated in our products and tech - already bringing and making the most advanced featured available into our 4D HD maps covering 360 RealViews and 3D drone and digital twin based maps, immersive views and RealVerses - and focused on solving the needs of Indian consumers and enterprises, and served customers and users with humility, passion and consistency, with a solid and sustainable business model to ensure and provide a long-term reliable mapping solution for customers and the country. 3) Here’s an explainer video of our maps, tech & APIs which focus on how developers, users and customers can leverage our solutions to get their needs solved in the best way. Do watch - you’ll be pleasantly surprised and happy at the offering. 4) To try out as a developer for yourself, visit We’re glad that tens of thousands of developers, and their hundreds of millions of users, benefit from Mappls MapmyIndia Maps & APIs everyday, using both our free plans and our commercial plans. Do try for your own needs as a developer. 5) In one sense, the quality and capability of Mappls MapmyIndia is proven to be better and more useful and valuable through our free consumer Mappls MapmyIndia app (learn more and download from which has gotten love from millions of consumers who are able to navigate safer and better. MapmyIndia Mappls Rakesh Verma @RashmiV1956

Rohan Verma

16,210 views • 2 years ago

Let me explain what I'm doing here for those who are interested. I've been building a retro-style D3D12 game engine from scratch, and recently implemented a "Spatial Compression" effect. Essentially its a technique that makes distant geometry appear larger than standard perspective projection would, giving scenes a telephoto compression feel without changing the camera FOV globally (to keep things "normal" when close to the camera). The goal: foreground geometry completely unchanged, objects beyond a configurable start distance progressively scaled up, with a smooth power-curve falloff. The solution: Once the full view-projection transform has been applied, I'm holding 2D screen coordinates. Camera orientation is already resolved at that point, so it's done its job. Scaling those coordinates outward from screen center is a pure 2D operation with no dependency on where the camera is pointing. The scale factor itself is derived from Euclidean world-space distance between the camera and the vertex (not a view-space value, which would rotate with the camera). Combined with the 2D application, both ingredients are fully rotation-invariant. Critically, only the XY screen coordinates are modified. Depth values are left completely untouched, so the depth buffer and any shadow calculations remain correct. I then tried inverting the effect for the opposite in interiors, leading to "Spatial Expansion".
0:29

Let me explain what I'm doing here for those who are interested. I've been building a retro-style D3D12 game engine from scratch, and recently implemented a "Spatial Compression" effect. Essentially its a technique that makes distant geometry appear larger than standard perspective projection would, giving scenes a telephoto compression feel without changing the camera FOV globally (to keep things "normal" when close to the camera). The goal: foreground geometry completely unchanged, objects beyond a configurable start distance progressively scaled up, with a smooth power-curve falloff. The solution: Once the full view-projection transform has been applied, I'm holding 2D screen coordinates. Camera orientation is already resolved at that point, so it's done its job. Scaling those coordinates outward from screen center is a pure 2D operation with no dependency on where the camera is pointing. The scale factor itself is derived from Euclidean world-space distance between the camera and the vertex (not a view-space value, which would rotate with the camera). Combined with the 2D application, both ingredients are fully rotation-invariant. Critically, only the XY screen coordinates are modified. Depth values are left completely untouched, so the depth buffer and any shadow calculations remain correct. I then tried inverting the effect for the opposite in interiors, leading to "Spatial Expansion".

Analog Dream Dev

47,051 views • 1 month ago

General Intuition CEO Pim de Witte, who's building foundation models trained on video game controller input data ("action-labeled gameplay clips"), says general intelligence won't "taste like an LLM": "We have a scale of data that's going to allow us to jump to the frontier in one capability — which is any system that can be controlled with a game controller (which is most robots) — and then, you can use that to create a sufficiently general intelligence." "As humans, the decision to talk or type is a very, very small subset of the actions that we can actually take." "So in order to create a sufficiently general intelligence to play 10,000+ video games, the model has to be able to predict across the entire action space of human cognition when they're interacting with these environments. Which are 2D and 3D environments, interfaces, long-horizon tasks, short-horizon tasks, [etc.]." "It has to be a sufficiently general intelligence in order to predict actions. Therefore, the type of model you get out is not going to taste like an LLM. This model is going to be incredibly good at navigating unforeseen environments. It's going to be incredibly good at zero-shotting any task that can be done with a game controller."
1:53

General Intuition CEO Pim de Witte, who's building foundation models trained on video game controller input data ("action-labeled gameplay clips"), says general intelligence won't "taste like an LLM": "We have a scale of data that's going to allow us to jump to the frontier in one capability — which is any system that can be controlled with a game controller (which is most robots) — and then, you can use that to create a sufficiently general intelligence." "As humans, the decision to talk or type is a very, very small subset of the actions that we can actually take." "So in order to create a sufficiently general intelligence to play 10,000+ video games, the model has to be able to predict across the entire action space of human cognition when they're interacting with these environments. Which are 2D and 3D environments, interfaces, long-horizon tasks, short-horizon tasks, [etc.]." "It has to be a sufficiently general intelligence in order to predict actions. Therefore, the type of model you get out is not going to taste like an LLM. This model is going to be incredibly good at navigating unforeseen environments. It's going to be incredibly good at zero-shotting any task that can be done with a game controller."

TBPN

80,199 views • 1 month ago

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.
0:54

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.

Blaine Brown

1,468,576 views • 2 years ago

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360°, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360° head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data 🏠 🌍 🎥 Great work by Tobias Kirschstein and Simon Giebenhain!
0:30

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360°, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360° head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data 🏠 🌍 🎥 Great work by Tobias Kirschstein and Simon Giebenhain!

Matthias Niessner

95,991 views • 7 months ago

META automatically opted all users into its new deepfake AI system “Meta is facing backlash over a new AI feature just rolled out on Instagram” The app is called Muse Image. It allows users to make realistic deepfakes using other people's photos. Any adult with a public Instagram account was automatically included. There are ways to turn it off and opt out, if you don’t do this everything you post of yourself can be used for deepfakes - You go to your settings and search sharing and reuse - Then you scroll down to allow people to ‘create with and reuse your content’ and toggle both of those off
1:40

META automatically opted all users into its new deepfake AI system “Meta is facing backlash over a new AI feature just rolled out on Instagram” The app is called Muse Image. It allows users to make realistic deepfakes using other people's photos. Any adult with a public Instagram account was automatically included. There are ways to turn it off and opt out, if you don’t do this everything you post of yourself can be used for deepfakes - You go to your settings and search sharing and reuse - Then you scroll down to allow people to ‘create with and reuse your content’ and toggle both of those off

Wall Street Apes

49,848 views • 25 days ago

Phoenix GenAI’s new generative AI model Image-to-3D is now online. Create high fidelity 3D models importable to Unreal Engine 5 or Unity simply by sending an original image into PhoenixLLM. Better yet, generate the source image using Phoenix GenAI’s Flux and feed it into Image-to-3D. This release marks yet another upgrade of GenAI’s arsenal of capabilities, getting it ready for multi-workflow GenAI agents, in which users will be able to combine text-to-image, image-to-prompt, text-to-video, text-to-3D, and image-to-3D into complex multi-step workflows with simple commands via PhoenixLLM. Image-to-3D is yet another addition to Phoenix’s Vertical AI Solutions for gaming, content, and metaverse. Users are able to use it as a Phoenix-native alternative to SkyNet AI Marketplace’s Tripo Integration earlier this year. #Phoenix $PHB
1:37

Phoenix GenAI’s new generative AI model Image-to-3D is now online. Create high fidelity 3D models importable to Unreal Engine 5 or Unity simply by sending an original image into PhoenixLLM. Better yet, generate the source image using Phoenix GenAI’s Flux and feed it into Image-to-3D. This release marks yet another upgrade of GenAI’s arsenal of capabilities, getting it ready for multi-workflow GenAI agents, in which users will be able to combine text-to-image, image-to-prompt, text-to-video, text-to-3D, and image-to-3D into complex multi-step workflows with simple commands via PhoenixLLM. Image-to-3D is yet another addition to Phoenix’s Vertical AI Solutions for gaming, content, and metaverse. Users are able to use it as a Phoenix-native alternative to SkyNet AI Marketplace’s Tripo Integration earlier this year. #Phoenix $PHB

Phoenix AI

30,853 views • 1 year ago

Sora animates with this wonderful wonky, sketchy dreamlike quality that perfectly captures the nostalgic atmosphere of a hazy 90's suburban summer afternoon. While Seedance is just as sophisticated and in some aspects superior, I will miss the quality of Sora. Sora feels like 35mm film, with the nuanced way it captures lighting, color, and perspective, while Seedance feels digital - a bit too clean. I am confident that I can replicate this sketchy quality in Seedance by tweaking my prompts, but Sora has a unique way of animating that really should be preserved. There are many more bugs and mistakes with Sora, but when it gets it right, it REALLY hits a level of artful magic that sets it above all other models. I am shocked that I still appear to be the only person who is creating AI-Generated 2D animation with original characters while developing a unique Western style. Practically no one is doing 2D AI-animation at all, and when they do, it is usually an attempt to mimic an Anime style. 2D really should be attempted more by AI Filmmakers!! Especially with a Western model like Sora, before it is terminated in the fall. Really, everyone should be taking advantage of the way this model so skillfully animates, with the heart and soul of a seasoned professional. I personally feel AI-generated animation is superior to 3D/realistic AI filmmaking. While AI-generated realism is an attempt to mimic real life through a lens, 2D animation IS what it is - not a replica of anything, just a cartoon. I love all my AI filmmaker bros and all of the cutting edge work going on, but I urge all of you to give 2D animation a chance. : ) I'm still not giving up hope that we can save Sora somehow. Now that I see that Seedance is able to animate 2D brilliantly and beautifully, I know that it's not some hidden secret, and we can replicate the weights and maths of Sora. If we can't preserve Sora, we can make a comparable model. It's not just in the interest of AI-Animators like me to preserve this model, but in the interest of the entire American AI community if they want Western AI to be superior. If not, Chinese video models will dominate. And while I am perfectly happy to use a Chinese model and I am just eternally grateful that this technology exists at all, I would love to see American audiovisual models continue to be developed! #AIAnimation #AIFilmmaking #AIArt #SaveSora #SummerofSora #WillStancilShow Elon Musk Marc Andreessen 🇺🇸 Sora Bill Peebles NVIDIA Sam Altman
0:29

Sora animates with this wonderful wonky, sketchy dreamlike quality that perfectly captures the nostalgic atmosphere of a hazy 90's suburban summer afternoon. While Seedance is just as sophisticated and in some aspects superior, I will miss the quality of Sora. Sora feels like 35mm film, with the nuanced way it captures lighting, color, and perspective, while Seedance feels digital - a bit too clean. I am confident that I can replicate this sketchy quality in Seedance by tweaking my prompts, but Sora has a unique way of animating that really should be preserved. There are many more bugs and mistakes with Sora, but when it gets it right, it REALLY hits a level of artful magic that sets it above all other models. I am shocked that I still appear to be the only person who is creating AI-Generated 2D animation with original characters while developing a unique Western style. Practically no one is doing 2D AI-animation at all, and when they do, it is usually an attempt to mimic an Anime style. 2D really should be attempted more by AI Filmmakers!! Especially with a Western model like Sora, before it is terminated in the fall. Really, everyone should be taking advantage of the way this model so skillfully animates, with the heart and soul of a seasoned professional. I personally feel AI-generated animation is superior to 3D/realistic AI filmmaking. While AI-generated realism is an attempt to mimic real life through a lens, 2D animation IS what it is - not a replica of anything, just a cartoon. I love all my AI filmmaker bros and all of the cutting edge work going on, but I urge all of you to give 2D animation a chance. : ) I'm still not giving up hope that we can save Sora somehow. Now that I see that Seedance is able to animate 2D brilliantly and beautifully, I know that it's not some hidden secret, and we can replicate the weights and maths of Sora. If we can't preserve Sora, we can make a comparable model. It's not just in the interest of AI-Animators like me to preserve this model, but in the interest of the entire American AI community if they want Western AI to be superior. If not, Chinese video models will dominate. And while I am perfectly happy to use a Chinese model and I am just eternally grateful that this technology exists at all, I would love to see American audiovisual models continue to be developed! #AIAnimation #AIFilmmaking #AIArt #SaveSora #SummerofSora #WillStancilShow Elon Musk Marc Andreessen 🇺🇸 Sora Bill Peebles NVIDIA Sam Altman

Emily Youcis

18,940 views • 2 months ago

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.
0:50

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,798 views • 3 years ago