Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

New clean 3D AI workflow Control Rig physics in UEFN! Been exploring some more hybrid AI workflows lately, and for 3D assets splitting the model into parts is kinda essential. This video shows my first test using AssetHub, one of the cleanest ways i've used to keep the whole... generation process organized. It auto-detected the pose in my main image and broke it down into parts, then generated images for each. From there I can just iterate on each piece separately and generate the 3D models in one workflow. AssetHub is still in closed beta atm, so let me know if you want to try it out. Main reference image came from Grok Imagine. All the part images and high poly meshes were done in AssetHub, selected Tripo 3.1 as the main model. After that I ran it through Hunyuan 3D for retopo + unwrap. Final assembly in Blender. Baked normals from high to low poly. End result: under 60K tris, 3 materials, 2K textures. Also Control Rig physics in Unreal Engine are insane. It finally validates in UEFN too, so of course I had to test it out.show more

Jerome | InsaneUnreal

3,985 subscribers

81,416 views • 2 months ago •via X (Twitter)

Gaming Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

What about a 2 day version? After Andrew Price posted about the two extremes (fully manual vs fully AI car modeling), I wanted to try a hybrid approach. Not sure if his examples were more film or game-focused, but I’m aiming for a game-ready asset. Using AssetHub for the main workflow hub. Because of the canvas and part splitting tools , iterating is smooth here. Generating high poly meshes with Tripo inside AssetHub. Retopoly with Hunyuan 3D Studio. Texturing with Trellis2 in ComfyUI. Did some manual modeling for the rims since i still dont trust AI generated rims being 100% circular😄 Assembly in Blender. End result: car exterior + interior 36K tris. Wheels total still 32K hehe. I’d bake those tire detail normals onto a simple cylinder next. In UEFN / Unreal Engine i used a clear coat material to give the car some nice reflections. Is it perfect? No. retopo and texturing are still the weak spots, but that gap is closing. Would I use this in a game? 100%! (after improving the tires). If you want to try AssetHub, you can sign up and use JR1627 to skip the waitlist.

What about a 2 day version? After Andrew Price posted about the two extremes (fully manual vs fully AI car modeling), I wanted to try a hybrid approach. Not sure if his examples were more film or game-focused, but I’m aiming for a game-ready asset. Using AssetHub for the main workflow hub. Because of the canvas and part splitting tools , iterating is smooth here. Generating high poly meshes with Tripo inside AssetHub. Retopoly with Hunyuan 3D Studio. Texturing with Trellis2 in ComfyUI. Did some manual modeling for the rims since i still dont trust AI generated rims being 100% circular😄 Assembly in Blender. End result: car exterior + interior 36K tris. Wheels total still 32K hehe. I’d bake those tire detail normals onto a simple cylinder next. In UEFN / Unreal Engine i used a clear coat material to give the car some nice reflections. Is it perfect? No. retopo and texturing are still the weak spots, but that gap is closing. Would I use this in a game? 100%! (after improving the tires). If you want to try AssetHub, you can sign up and use JR1627 to skip the waitlist.

Jerome | InsaneUnreal

25,799 views • 1 month ago

From generated reference images to riding a robot raptor in Fortnite 🦖 For now mostly the same workflow, but I also used Nano Banana to get that toe claw in the image lol. Other steps: Images from Grok Imagine, Tripo for raw 3D generation, Hunyuan 3D Studio for lowpoly and UV unwrap, Trellis2 in ComfyUI for texturing and AI upscaling. Blender for cleanup, rigging and weight painting. Splitting up the body parts helped to get a more mid-poly result. Also with better control during cleanup. The final result has 28K triangles. Again with the awesome retargeting tools in Unreal Engine / UEFN. I’ll be testing different AI texturing approaches soon. Some parts can get a bit wobbly at times, most notable in hard-surface objects. Also curious if there are any upcoming weight painting tools 😄

From generated reference images to riding a robot raptor in Fortnite 🦖 For now mostly the same workflow, but I also used Nano Banana to get that toe claw in the image lol. Other steps: Images from Grok Imagine, Tripo for raw 3D generation, Hunyuan 3D Studio for lowpoly and UV unwrap, Trellis2 in ComfyUI for texturing and AI upscaling. Blender for cleanup, rigging and weight painting. Splitting up the body parts helped to get a more mid-poly result. Also with better control during cleanup. The final result has 28K triangles. Again with the awesome retargeting tools in Unreal Engine / UEFN. I’ll be testing different AI texturing approaches soon. Some parts can get a bit wobbly at times, most notable in hard-surface objects. Also curious if there are any upcoming weight painting tools 😄

Jerome | InsaneUnreal

40,169 views • 4 months ago

Hard surface model test: 3D model of an unfinished overland build. Got really nice images again from Grok. Also realised an ‘unfinished’ vehicle might not be the best idea lol. Anyways, I was well surprised with the results. I don’t trust rotating a generated wheel yet 😁, so I made the wheels in Blender. Very similar workflow as the mech character: Images from Grok Imagine, Tripo for raw 3D generation, Hunyuan 3D Studio for lowpoly and UV unwrap, Trellis2 in ComfyUI for texturing + upscaling. Blender final preparation. Unreal Engine / UEFN shaders. Next time i'd work a bit longer on the normals.

Hard surface model test: 3D model of an unfinished overland build. Got really nice images again from Grok. Also realised an ‘unfinished’ vehicle might not be the best idea lol. Anyways, I was well surprised with the results. I don’t trust rotating a generated wheel yet 😁, so I made the wheels in Blender. Very similar workflow as the mech character: Images from Grok Imagine, Tripo for raw 3D generation, Hunyuan 3D Studio for lowpoly and UV unwrap, Trellis2 in ComfyUI for texturing + upscaling. Blender final preparation. Unreal Engine / UEFN shaders. Next time i'd work a bit longer on the normals.

Jerome | InsaneUnreal

166,902 views • 4 months ago

Tested this concepting workflow. ▶️ Converting a level design image to a 3D asset gallery image ▶️ re-generating segmented subjects into separate high quality images. ▶️ using these images to generate the 3D meshes. First time using 'for loops' in ComfyUI, works great and the prompts are updated for every segment. I used the Nano Banana 2 API node in ComfyUI to generate the final high quality images. Those SAM3 nodes are crazy btw, just set some points and it creates a mask! For the 3D model generation I used the new Tripo Smart Mesh tool, which is fast! The output is not perfect yet. I think in a lot of cases not production ready right now but that depends on the input image and the goal as well. For pre-production / concepting phase it's perfect and for far distance meshes it’s fine. I’m sure these tools keep improving, curious what we can work with in 3-6 months. Minimal cleanup and scaling in Blender and tested the meshes in UEFN / Unreal Engine. After creating the ComfyUI workflow it took less than an hour to do everything.

Tested this concepting workflow. ▶️ Converting a level design image to a 3D asset gallery image ▶️ re-generating segmented subjects into separate high quality images. ▶️ using these images to generate the 3D meshes. First time using 'for loops' in ComfyUI, works great and the prompts are updated for every segment. I used the Nano Banana 2 API node in ComfyUI to generate the final high quality images. Those SAM3 nodes are crazy btw, just set some points and it creates a mask! For the 3D model generation I used the new Tripo Smart Mesh tool, which is fast! The output is not perfect yet. I think in a lot of cases not production ready right now but that depends on the input image and the goal as well. For pre-production / concepting phase it's perfect and for far distance meshes it’s fine. I’m sure these tools keep improving, curious what we can work with in 3-6 months. Minimal cleanup and scaling in Blender and tested the meshes in UEFN / Unreal Engine. After creating the ComfyUI workflow it took less than an hour to do everything.

Jerome | InsaneUnreal

17,267 views • 3 months ago

In this experiment I wanted to try something with facial expressions and have it work with native Unreal Engine tools. I did find that generating the head with an open mouth helped to get some extra details. Creating those blend shapes was still not easy and this result is far from perfect. Ideally i want 58 generated expression references and have AI figure that out haha. btw first time baking the high poly details onto the low poly generated character. That worked surprisingly well! Endresult has 34k triangles! Built this character in 4 separate parts in Blender and generated each part in Hunyuan 3D Studio. The original references came from Nano Banana 2 and Grok. Trellis 2 in ComfyUI for the texturing. The animation was done with Control Rig in UEFN / Unreal Engine. With all the experiments so far, file and version management start to become a challenge. Getting files from different platforms with each platform having their own filename logic. I’ll have to find something to keep it more organized. Any suggestions?

In this experiment I wanted to try something with facial expressions and have it work with native Unreal Engine tools. I did find that generating the head with an open mouth helped to get some extra details. Creating those blend shapes was still not easy and this result is far from perfect. Ideally i want 58 generated expression references and have AI figure that out haha. btw first time baking the high poly details onto the low poly generated character. That worked surprisingly well! Endresult has 34k triangles! Built this character in 4 separate parts in Blender and generated each part in Hunyuan 3D Studio. The original references came from Nano Banana 2 and Grok. Trellis 2 in ComfyUI for the texturing. The animation was done with Control Rig in UEFN / Unreal Engine. With all the experiments so far, file and version management start to become a challenge. Getting files from different platforms with each platform having their own filename logic. I’ll have to find something to keep it more organized. Any suggestions?

Jerome | InsaneUnreal

45,921 views • 3 months ago

✨ Made a new mini feature on Photo AI: [ Grab from 3d model ] So the problem is we're at that stage in time (typical for AI) where image-to-3d models are not good enough but are fun to play with, but we know they'll be good enough in 1-2 years With [ Make 3d model ] you already can turn any Photo AI pic into a 3d model but it still looks hyper clunky and deformed, but it works! One cool idea I had to make that more useful and made now: Let people make a 3d model then change the view of the it with the 3d viewer, then press [ o ] and it grabs a frame of the 3d That image you can then [ Remix ] (img2img), and it becomes a real photo again and that in turn you can then turn into a video again with [ Make video ] So that essentially gives you a fully freeform camera position control to take photos with One thing I need to fix is the background/skybox, I kinda need to take the original photo and remove the person and just get the background for the 3d model viewer, in this case it should be white, but it's a start!

✨ Made a new mini feature on Photo AI: [ Grab from 3d model ] So the problem is we're at that stage in time (typical for AI) where image-to-3d models are not good enough but are fun to play with, but we know they'll be good enough in 1-2 years With [ Make 3d model ] you already can turn any Photo AI pic into a 3d model but it still looks hyper clunky and deformed, but it works! One cool idea I had to make that more useful and made now: Let people make a 3d model then change the view of the it with the 3d viewer, then press [ o ] and it grabs a frame of the 3d That image you can then [ Remix ] (img2img), and it becomes a real photo again and that in turn you can then turn into a video again with [ Make video ] So that essentially gives you a fully freeform camera position control to take photos with One thing I need to fix is the background/skybox, I kinda need to take the original photo and remove the person and just get the background for the 3d model viewer, in this case it should be white, but it's a start!

@levelsio

119,210 views • 11 months ago

Here's a look behind the scenes of how I used a 3d model, generated from an image, as a "driver" for ai animation. In the near future we'll see more 3d tools that support these types of workflows, with much more emphasis on enhancing user input instead of just letting the ai do everything. I used the 3d model to generate keyframes that I then animated in Luma Dream. By using a 3d model with a diffusion "layer" done with Krea, I got quite high quality frames with a relatively high consistency as well since the AI didn't have to "hallucinate" that much. I upscaled and detailed the frames using Magnific. Using semi-detailed 3d models as drivers for gen ai is very powerful, and it allows us to use models that are sculpted in a more free-flowing type of workflow that doesn't rely so much on high surface detailing or very time-consuming finish, but instead relies more on gestural 3d sculpting. #art #ai

Here's a look behind the scenes of how I used a 3d model, generated from an image, as a "driver" for ai animation. In the near future we'll see more 3d tools that support these types of workflows, with much more emphasis on enhancing user input instead of just letting the ai do everything. I used the 3d model to generate keyframes that I then animated in Luma Dream. By using a 3d model with a diffusion "layer" done with Krea, I got quite high quality frames with a relatively high consistency as well since the AI didn't have to "hallucinate" that much. I upscaled and detailed the frames using Magnific. Using semi-detailed 3d models as drivers for gen ai is very powerful, and it allows us to use models that are sculpted in a more free-flowing type of workflow that doesn't rely so much on high surface detailing or very time-consuming finish, but instead relies more on gestural 3d sculpting. #art #ai

Martin Nebelong

48,200 views • 1 year ago

The battle of the Text/Image to 3D Model 👍- To make this fair, I tested the same image across all platforms and used the first generated result. No cherry picking allowed! I also tested what I think are the 4 top services to use right now. Let me know what you think is the best 3D mesh generation from these platforms however I think it's quite obvious that Hunyuan 2.5 is currently the king 👑 It wins in overall detail and sharpness, accuracy in relation to the input image, as well as texture generation and projection. Tripo 2.5 comes in second and Rodin 3rd with Trellis in 4th place. However, Rodin has the best topology since it can generate a mesh in quads BUT I left all of them in triangles to keep it fair. Hi3DGen can't be counted as it can't even generate textures and it has no multi-image support. Meshy was another option but the results were not great. Is there any other AI 3D Gen services that you know of that you want me to test as well?

The battle of the Text/Image to 3D Model 👍- To make this fair, I tested the same image across all platforms and used the first generated result. No cherry picking allowed! I also tested what I think are the 4 top services to use right now. Let me know what you think is the best 3D mesh generation from these platforms however I think it's quite obvious that Hunyuan 2.5 is currently the king 👑 It wins in overall detail and sharpness, accuracy in relation to the input image, as well as texture generation and projection. Tripo 2.5 comes in second and Rodin 3rd with Trellis in 4th place. However, Rodin has the best topology since it can generate a mesh in quads BUT I left all of them in triangles to keep it fair. Hi3DGen can't be counted as it can't even generate textures and it has no multi-image support. Meshy was another option but the results were not great. Is there any other AI 3D Gen services that you know of that you want me to test as well?

Travis Davids

60,813 views • 1 year ago

✨ I made a little demo of #vibejam sponsor Tripo and how you can use it to generate assets for your games In this case I'm generating an apartment flat that's half destroyed and open for my drone sim to fly around, take cover and shoot at enemies from First I write the prompt and select Nano Banana to generate an image first, I get a few different ones, and I select the one I like Then that one is converted into a 3d object called a Smart Mesh Smart Mesh is different than regular image-to-3d models because usually you get issues with uneven polygon distribution, too dense geometry and poor edge flow, so while you think you save time with generating it with AI you end up doing lots of editing after to fix the topology which kinda defeats the point Smart Mesh doesn't have any of this, it's fast and clean low-poly models you can drop into your game Anyway, after generating the model I can then see it in 3d, and if I don't like it, regenerate it, or if I like it start texturing it Once that's done, I export it as a GLB, FBX or OBJ and drop it in (usually I use GLB as ThreeJS seems to prefer that), in my case I just drop the URL into Cursor and let it figure out the placement of the asset into my game!

✨ I made a little demo of #vibejam sponsor Tripo and how you can use it to generate assets for your games In this case I'm generating an apartment flat that's half destroyed and open for my drone sim to fly around, take cover and shoot at enemies from First I write the prompt and select Nano Banana to generate an image first, I get a few different ones, and I select the one I like Then that one is converted into a 3d object called a Smart Mesh Smart Mesh is different than regular image-to-3d models because usually you get issues with uneven polygon distribution, too dense geometry and poor edge flow, so while you think you save time with generating it with AI you end up doing lots of editing after to fix the topology which kinda defeats the point Smart Mesh doesn't have any of this, it's fast and clean low-poly models you can drop into your game Anyway, after generating the model I can then see it in 3d, and if I don't like it, regenerate it, or if I like it start texturing it Once that's done, I export it as a GLB, FBX or OBJ and drop it in (usually I use GLB as ThreeJS seems to prefer that), in my case I just drop the URL into Cursor and let it figure out the placement of the asset into my game!

@levelsio

72,338 views • 1 month ago

3D AI is leveling up! Rodin 3D AI can create stunning, high-quality 3D models from just text or image inputs. And with its latest update, it can even generate 8K HDRI textures to bring your models to life. Check out the link in the comments!

3D AI is leveling up! Rodin 3D AI can create stunning, high-quality 3D models from just text or image inputs. And with its latest update, it can even generate 8K HDRI textures to bring your models to life. Check out the link in the comments!

el.cine

46,032 views • 1 year ago

✨ I can now generate 3d assets for my drone sim at directly from Cursor (sponsor of #vibejam) I need buildings that you'd see in a war torn city, like warehouses in ruins, broken down abandoned houses, bombed out bridges etc. Nano Banana Pro or 2 can generate them really well and then you can put them in an image-to-3d model and you get a GLB or FBX That one you can then import into your Three.js game, the models might be big though, in my case like 16MB, so I ask it to compress it and make it more low poly so it loads fast ThreeJS then loads the individual GLBs on page load and puts them in my drone sim somewhere randomly, I think I should remove some of the grass and match the sandy color of the ruins though to make it fit in more

✨ I can now generate 3d assets for my drone sim at directly from Cursor (sponsor of #vibejam) I need buildings that you'd see in a war torn city, like warehouses in ruins, broken down abandoned houses, bombed out bridges etc. Nano Banana Pro or 2 can generate them really well and then you can put them in an image-to-3d model and you get a GLB or FBX That one you can then import into your Three.js game, the models might be big though, in my case like 16MB, so I ask it to compress it and make it more low poly so it loads fast ThreeJS then loads the individual GLBs on page load and puts them in my drone sim somewhere randomly, I think I should remove some of the grass and match the sandy color of the ruins though to make it fit in more

@levelsio

134,777 views • 2 months ago

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!

New open-source 3D world-generation model. I'm rendering a couple of worlds in the video, so check it out. You'll find the GitHub and the Hugging Face links to the model below. This is a multi-modal world model that you can use for a bunch of things: • To generate new worlds • To reconstruct worlds • To simulate 3D interactive worlds from a prompt, images, or a video You can edit the 3D outputs in Unity and Unreal Engine (they export as meshes, 3DGS files, and point clouds). You can also generate 3D characters in the world and walk around. Pretty fun stuff!

Santiago

65,411 views • 2 months ago

This was way too easy... I created a cute 3D character in minutes instead of hours using this combo: Unbound + Tripo I’m not a 3D artist, but this made the whole process feel simple, playful, and actually accessible. Unbound helped me shape the character idea, then Tripo turned it into a 3D model without the usual complicated workflow If you are a 3D artist or you are just starting, try Unbound. It’s free, open for everyone to experiment with, and a fun way to create characters, creatures, and 3D models without feeling overwhelmed I had so much fun using it today, and I think many visual creators are going to love it. Link in the first comment 👇🏻

This was way too easy... I created a cute 3D character in minutes instead of hours using this combo: Unbound + Tripo I’m not a 3D artist, but this made the whole process feel simple, playful, and actually accessible. Unbound helped me shape the character idea, then Tripo turned it into a 3D model without the usual complicated workflow If you are a 3D artist or you are just starting, try Unbound. It’s free, open for everyone to experiment with, and a fun way to create characters, creatures, and 3D models without feeling overwhelmed I had so much fun using it today, and I think many visual creators are going to love it. Link in the first comment 👇🏻

Amira Zairi

186,621 views • 1 month ago

Phoenix GenAI’s new generative AI model Image-to-3D is now online. Create high fidelity 3D models importable to Unreal Engine 5 or Unity simply by sending an original image into PhoenixLLM. Better yet, generate the source image using Phoenix GenAI’s Flux and feed it into Image-to-3D. This release marks yet another upgrade of GenAI’s arsenal of capabilities, getting it ready for multi-workflow GenAI agents, in which users will be able to combine text-to-image, image-to-prompt, text-to-video, text-to-3D, and image-to-3D into complex multi-step workflows with simple commands via PhoenixLLM. Image-to-3D is yet another addition to Phoenix’s Vertical AI Solutions for gaming, content, and metaverse. Users are able to use it as a Phoenix-native alternative to SkyNet AI Marketplace’s Tripo Integration earlier this year. #Phoenix $PHB

Phoenix GenAI’s new generative AI model Image-to-3D is now online. Create high fidelity 3D models importable to Unreal Engine 5 or Unity simply by sending an original image into PhoenixLLM. Better yet, generate the source image using Phoenix GenAI’s Flux and feed it into Image-to-3D. This release marks yet another upgrade of GenAI’s arsenal of capabilities, getting it ready for multi-workflow GenAI agents, in which users will be able to combine text-to-image, image-to-prompt, text-to-video, text-to-3D, and image-to-3D into complex multi-step workflows with simple commands via PhoenixLLM. Image-to-3D is yet another addition to Phoenix’s Vertical AI Solutions for gaming, content, and metaverse. Users are able to use it as a Phoenix-native alternative to SkyNet AI Marketplace’s Tripo Integration earlier this year. #Phoenix $PHB

Phoenix AI

30,853 views • 1 year ago

Currently testing Hunyuan brand new 2.5 model for image to 3D. YUP! It's MAJOR improvement over 2.0, especially with making details look sharper, it can also reconstruct more detail. It also generates PBR materials now and you can automatically rig the character on the website. It's much better than Trellis and Tripo from my testing! Took it into Blender as well to compare Hunyuan 2.5 (on the left) and Hunyuan 2.0 (on the right). I wanted to see the mesh with a solid and material preview. If you're doing 3D printing, these would probably work very well. However...the biggest issue here is still topology which is a complete and utter mess, there's also still issues with texture projection...and the entire model is a single mesh, there needs to be a method to generate the model in parts BUT this tech is certainly improving. As someone that is also a 3D artist, I actually want to see what this tech is capable of generating to output 3D models.

Travis Davids

45,357 views • 1 year ago

You can now generate, edit, and convert a reference image into 3D inside one tool. Tripo's Image Generation runs Nano Banana and GPT to build the reference, then turns it into a model. Text or photo in, 3D out. One pipeline, zero handoffs.

You can now generate, edit, and convert a reference image into 3D inside one tool. Tripo's Image Generation runs Nano Banana and GPT to build the reference, then turns it into a model. Text or photo in, 3D out. One pipeline, zero handoffs.

Alvaro Cintas

55,824 views • 8 days ago

I built a fully playable 3D platformer game in exactly 3 days, using almost entirely AI-generated assets. To test whether AI can really replace the asset grind, I came up with a steampunk cat universe from scratch and brought it straight into Unreal Engine 5. Here is how it was done, step by step: 👇 🎨 World and prop concepting 🧩 Tripo 3.1 for low-poly smart meshes and PBR textures 🩹 Blender for geometry cleanup and AO baking 🦴 AccuRIG for character animation 🚀 UE5 for level design and blueprints The craziest part is that AI handled the routine, and I still had the creative energy to make the level actually fun. Watch the full workflow and gameplay reveal: #GameDev #TripoAI #UnrealEngine5 #3DModeling #IndieDev #Blender3D

I built a fully playable 3D platformer game in exactly 3 days, using almost entirely AI-generated assets. To test whether AI can really replace the asset grind, I came up with a steampunk cat universe from scratch and brought it straight into Unreal Engine 5. Here is how it was done, step by step: 👇 🎨 World and prop concepting 🧩 Tripo 3.1 for low-poly smart meshes and PBR textures 🩹 Blender for geometry cleanup and AO baking 🦴 AccuRIG for character animation 🚀 UE5 for level design and blueprints The craziest part is that AI handled the routine, and I still had the creative energy to make the level actually fun. Watch the full workflow and gameplay reveal: #GameDev #TripoAI #UnrealEngine5 #3DModeling #IndieDev #Blender3D

Stefan 3D AI

17,358 views • 1 month ago

The first AI design agent just dropped. It combines major models for image, video, and 3D asset generation all in one place. You can also edit the images, generate music, and voiceovers! Step-by-step tutorial 👇

The first AI design agent just dropped. It combines major models for image, video, and 3D asset generation all in one place. You can also edit the images, generate music, and voiceovers! Step-by-step tutorial 👇

Alvaro Cintas

120,917 views • 1 year ago

Not bad! 2K triangles and a bit of manual cleanup. This was a quick test creating a lowpoly 3D model. Tried Tripo for retopology and it did a good job. It does render some inverted normals though. For this lowpoly model it’s fine but inverted normals are a pain when working on highpoly models. I’m going to test some other options like the remesh tool in Blender sculpting. It’s crazy how this Trellis Texturing node works btw, It understands the geometry very well. Watch the end where I made the worst version of an Axe in Blender and it was able to wrap the texture on that model! I would probably spend a bit more time splitting the image up and generating the main parts separately. Also making any signs readable or just remove them.

Not bad! 2K triangles and a bit of manual cleanup. This was a quick test creating a lowpoly 3D model. Tried Tripo for retopology and it did a good job. It does render some inverted normals though. For this lowpoly model it’s fine but inverted normals are a pain when working on highpoly models. I’m going to test some other options like the remesh tool in Blender sculpting. It’s crazy how this Trellis Texturing node works btw, It understands the geometry very well. Watch the end where I made the worst version of an Axe in Blender and it was able to wrap the texture on that model! I would probably spend a bit more time splitting the image up and generating the main parts separately. Also making any signs readable or just remove them.

Jerome | InsaneUnreal

15,052 views • 5 months ago

This is probably the most complex workflow I’ve ever built, only with open-source tools. It took my 4 days. It takes four inputs: author, title, and style; and generates a full visual animated story in one click in ComfyUI . I worked on it for four days. There are still some bugs, but here’s the first preview. Here’s a quick breakdown: - The four inputs are sent to LLMs with precise instructions to generate: first, prompts for images and image modifications; second, prompts for animations; third, prompts for generating music. - All voices are generated from the text and timed precisely, as they determine the length of each animation segment. - The first image and video are generated to serve as the title, but also as the guide for all other images created for the video. - Titles and subtitles are also added automatically in Comfy. - I also developed a lot of custom nodes for minor frame calculations, mostly to match audio and video. - The full system is a large loop that, for each line of text, generates an image and then a video from that image. The loop was the hardest part to build in this workflow, so it can process either a 20-second video or a 2-minute video with the same input. - There are multiple combinations of LLMs that try to understand the text in the best way to provide the best prompts for images and video. - The final video is assembled entirely within ComfyUI. - The music is generated based on the LLM output and matches the exact timing of the full animation. - Done! For reference, this workflow uses a lot of models and only works on an RTX 6000 Pro with plenty of RAM. My goal is not to replace humans, as I’ll try to explain later, this workflow is highly controlled and can be adapted or reworked at any point by real artists! My aim was to create a tool that can animate text in one go, allowing the AI some freedom while keeping a strict flow. I don’t know yet how I’ll share this workflow with people, I still need to polish it properly, but maybe through Patreon. Anyway, I hope you enjoy my research, and let’s always keep pushing further! :)

This is probably the most complex workflow I’ve ever built, only with open-source tools. It took my 4 days. It takes four inputs: author, title, and style; and generates a full visual animated story in one click in ComfyUI . I worked on it for four days. There are still some bugs, but here’s the first preview. Here’s a quick breakdown: - The four inputs are sent to LLMs with precise instructions to generate: first, prompts for images and image modifications; second, prompts for animations; third, prompts for generating music. - All voices are generated from the text and timed precisely, as they determine the length of each animation segment. - The first image and video are generated to serve as the title, but also as the guide for all other images created for the video. - Titles and subtitles are also added automatically in Comfy. - I also developed a lot of custom nodes for minor frame calculations, mostly to match audio and video. - The full system is a large loop that, for each line of text, generates an image and then a video from that image. The loop was the hardest part to build in this workflow, so it can process either a 20-second video or a 2-minute video with the same input. - There are multiple combinations of LLMs that try to understand the text in the best way to provide the best prompts for images and video. - The final video is assembled entirely within ComfyUI. - The music is generated based on the LLM output and matches the exact timing of the full animation. - Done! For reference, this workflow uses a lot of models and only works on an RTX 6000 Pro with plenty of RAM. My goal is not to replace humans, as I’ll try to explain later, this workflow is highly controlled and can be adapted or reworked at any point by real artists! My aim was to create a tool that can animate text in one go, allowing the AI some freedom while keeping a strict flow. I don’t know yet how I’ll share this workflow with people, I still need to polish it properly, but maybe through Patreon. Anyway, I hope you enjoy my research, and let’s always keep pushing further! :)

Lovis Odin

58,571 views • 9 months ago