Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🚀🎬We introduce TMD (Transition Matching Distillation): 480p videos generated from text prompts in < 3 NFEs! 1️⃣Main backbone for feature extraction and lightweight head for iterative refinement 2️⃣Distilled from Wan2.1 14B T2V combining MeanFlow & DMD2 🔗

Julius Berner

1,352 subscribers

17,026 views • 9 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

4 Comments

Julius Berner9 months ago

💡Ideas: - Principled decoupling of student via transition matching formulation ( - Pretraining using adaptation of MeanFlow ( for flow head - Distillation using version of DMD2 ( with flow head rollout

Julius Berner9 months ago

⚡Experiments on Wan2.1 1.3B & 14B show that TMD outperforms existing distillation methods at comparable inference cost in visual fidelity and prompt adherence. See our paper for details: Joint work w/ @wn8_nie @ma_nanye Chao Liu @sainingxie @ArashVahdat

Jiachen Lei9 months ago

Cool!!

Jiaqi Feng5 months ago

Will it open source?

Related Videos

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

AK

16,062 views • 2 years ago

A 19-euro tool just replaced 8,000 Meta employees. Here’s the exact workflow a TikTok creator used to build AI influencer videos from scratch — no camera, no team, no budget. Step 1. Open Pinterest. Search “Girl Self.” Save the image you like. Open Nanobanana Pro. Step 2. Swap the face using a single prompt. (Link at the bottom.) Step 3. Take your generated photo into Crust. Currently the easiest video-from-image tool on the market. Step 4. Create 3 nodes: reference image, text, video generator. Step 5. Drop your photo into the image node. Write a video prompt into the text field. This is what makes the photo move. Step 6. Connect the nodes. Text to text-in. Image to image-in. Select Kling 2.6. Set duration. Toggle sound. Step 7. Hit generate. 2 prompts. 1 AI girl. 0 employees. Meta spent $14B on Reality Labs last year to build digital humans. This creator built one in 7 minutes for less than a dinner. The prompts are free in the reply.

A 19-euro tool just replaced 8,000 Meta employees. Here’s the exact workflow a TikTok creator used to build AI influencer videos from scratch — no camera, no team, no budget. Step 1. Open Pinterest. Search “Girl Self.” Save the image you like. Open Nanobanana Pro. Step 2. Swap the face using a single prompt. (Link at the bottom.) Step 3. Take your generated photo into Crust. Currently the easiest video-from-image tool on the market. Step 4. Create 3 nodes: reference image, text, video generator. Step 5. Drop your photo into the image node. Write a video prompt into the text field. This is what makes the photo move. Step 6. Connect the nodes. Text to text-in. Image to image-in. Select Kling 2.6. Set duration. Toggle sound. Step 7. Hit generate. 2 prompts. 1 AI girl. 0 employees. Meta spent $14B on Reality Labs last year to build digital humans. This creator built one in 7 minutes for less than a dinner. The prompts are free in the reply.

RGK

188,584 views • 4 months ago

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.

AK

40,492 views • 2 years ago

As detailed in the Meta Movie Gen technical report, today we’re open sourcing Movie Gen Bench: two new media generation benchmarks that we hope will help to enable the AI research community to progress work on more capable audio and video generation models. Movie Gen Video Bench is the largest and most comprehensive benchmark ever released for evaluating text-to-video generation. It includes a collection of 1,000+ prompts that cover concepts ranging from detailed human activity to animals, physics, unusual subjects and more — with broad coverage across different motion levels. Movie Gen Audio Bench is a first-of-its-kind benchmark aimed at evaluating video-to-audio and (text+video)-to-audio generation. It includes 527 generated videos and associated sound effects and music prompts covering a diverse set of ambient environments and sound effects. To enable fair and easy comparison to our models for future works, these new benchmarks include non cherry-picked generated videos and audio from Movie Gen. In releasing these new benchmarks we hope to promote fair & extensive evaluations in media generation research to enable greater progress in this field.

As detailed in the Meta Movie Gen technical report, today we’re open sourcing Movie Gen Bench: two new media generation benchmarks that we hope will help to enable the AI research community to progress work on more capable audio and video generation models. Movie Gen Video Bench is the largest and most comprehensive benchmark ever released for evaluating text-to-video generation. It includes a collection of 1,000+ prompts that cover concepts ranging from detailed human activity to animals, physics, unusual subjects and more — with broad coverage across different motion levels. Movie Gen Audio Bench is a first-of-its-kind benchmark aimed at evaluating video-to-audio and (text+video)-to-audio generation. It includes 527 generated videos and associated sound effects and music prompts covering a diverse set of ambient environments and sound effects. To enable fair and easy comparison to our models for future works, these new benchmarks include non cherry-picked generated videos and audio from Movie Gen. In releasing these new benchmarks we hope to promote fair & extensive evaluations in media generation research to enable greater progress in this field.

AI at Meta

156,273 views • 2 years ago

Is VideoGen starting to become good enough for robotic manipulation? 🤖 Check out our recent work, RIGVid — Robots Imitating Generated Videos — where we use AI-generated videos as intermediate representations and 6-DoF motion retargeting to guide robots in diverse manipulation tasks: pouring, wiping, mixing, and more. 🔗 Key takeaways: - VideoGen starts to become good enough for robotics - As the field progresses, we are expecting much better results in the coming years - Depending on whether video prediction models take actions or not (VideoGen vs Action-Conditioned Video Prediction), there are different ways to use them. - Controllability & steerability are still issues In the paper, we explore: – How do different video generation models compare for robotic imitation? – Can generated videos replace real videos for imitation? – What causes failure of imitation given high-quality videos? – How does imitating from video compare with other representations (e.g., keypoint constraints like ReKep)? 🎥 Watch the video for (1) AI-generated inputs, (2) robot executions, and (3) the 3D intermediate representation bridging the embodiment gap.

Is VideoGen starting to become good enough for robotic manipulation? 🤖 Check out our recent work, RIGVid — Robots Imitating Generated Videos — where we use AI-generated videos as intermediate representations and 6-DoF motion retargeting to guide robots in diverse manipulation tasks: pouring, wiping, mixing, and more. 🔗 Key takeaways: - VideoGen starts to become good enough for robotics - As the field progresses, we are expecting much better results in the coming years - Depending on whether video prediction models take actions or not (VideoGen vs Action-Conditioned Video Prediction), there are different ways to use them. - Controllability & steerability are still issues In the paper, we explore: – How do different video generation models compare for robotic imitation? – Can generated videos replace real videos for imitation? – What causes failure of imitation given high-quality videos? – How does imitating from video compare with other representations (e.g., keypoint constraints like ReKep)? 🎥 Watch the video for (1) AI-generated inputs, (2) robot executions, and (3) the 3D intermediate representation bridging the embodiment gap.

Yunzhu Li

16,540 views • 1 year ago

The future is here. 🚀 Introducing Nwiro | Your Unreal Engine AI Assistant, a new AI powered PCG environment creation tool by Leartes. nwiro.ai Turn a simple text prompt into fully generated Unreal Engine environments in minutes. Intelligent asset placement. Automated PCG workflows. Faster, smarter world building. From idea to immersive world, instantly. This is an Alpha release, and we are just getting started. Download Nwiro and get 3 free credits to start creating right away. Create beyond limits. Create with Nwiro. 🌍✨ Get Nwiro on Fab for free and give feedback to unlock more features! 🔗 🔗 Check out the Nwiro website #Nwiro #Leartes #UnrealEngine #PCG #GameDev #AI #WorldBuilding #Tool

The future is here. 🚀 Introducing Nwiro | Your Unreal Engine AI Assistant, a new AI powered PCG environment creation tool by Leartes. nwiro.ai Turn a simple text prompt into fully generated Unreal Engine environments in minutes. Intelligent asset placement. Automated PCG workflows. Faster, smarter world building. From idea to immersive world, instantly. This is an Alpha release, and we are just getting started. Download Nwiro and get 3 free credits to start creating right away. Create beyond limits. Create with Nwiro. 🌍✨ Get Nwiro on Fab for free and give feedback to unlock more features! 🔗 🔗 Check out the Nwiro website #Nwiro #Leartes #UnrealEngine #PCG #GameDev #AI #WorldBuilding #Tool

Leartes

69,194 views • 7 months ago

🌟 Passed 2000+ concurrent viewers! 🫟 ✅ Switched to portrait videos cause most usage on mobile ✅Moved the queue to the left with upvotes, so the most upvoted ones get generated (because limit of 4 videos (15s) per minute ✅ Shows playing next so you know if your video coming up next ✅ Together this makes it a bit more clear how it works: 1) you type a text in chat, 2) it gets put in the queue for people to upvote, 3) most upvoted one gets pulled from queue into generating, 3) when done generating get pu in the stream to play next ✅ Slowly adding some up and down scrolling (go previous videos) also but still a bit buggy Feature requests and ideas very welcome!

🌟 Passed 2000+ concurrent viewers! 🫟 ✅ Switched to portrait videos cause most usage on mobile ✅Moved the queue to the left with upvotes, so the most upvoted ones get generated (because limit of 4 videos (15s) per minute ✅ Shows playing next so you know if your video coming up next ✅ Together this makes it a bit more clear how it works: 1) you type a text in chat, 2) it gets put in the queue for people to upvote, 3) most upvoted one gets pulled from queue into generating, 3) when done generating get pu in the stream to play next ✅ Slowly adding some up and down scrolling (go previous videos) also but still a bit buggy Feature requests and ideas very welcome!

@levelsio

202,046 views • 1 month ago

HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Image paper page: 3D content creation from a single image is a long-standing yet highly desirable task. Recent advances introduce 2D diffusion priors, yielding reasonable results. However, existing methods are not hyper-realistic enough for post-generation usage, as users cannot view, render and edit the resulting 3D content from a full range. To address these challenges, we introduce HyperDreamer with several key designs and appealing properties: 1) Viewable: 360 degree mesh modeling with high-resolution textures enables the creation of visually compelling 3D models from a full range of observation points. 2) Renderable: Fine-grained semantic segmentation and data-driven priors are incorporated as guidance to learn reasonable albedo, roughness, and specular properties of the materials, enabling semantic-aware arbitrary material estimation. 3) Editable: For a generated model or their own data, users can interactively select any region via a few clicks and efficiently edit the texture with text-based guidance. Extensive experiments demonstrate the effectiveness of HyperDreamer in modeling region-aware materials with high-resolution textures and enabling user-friendly editing. We believe that HyperDreamer holds promise for advancing 3D content creation and finding applications in various domains.

HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Image paper page: 3D content creation from a single image is a long-standing yet highly desirable task. Recent advances introduce 2D diffusion priors, yielding reasonable results. However, existing methods are not hyper-realistic enough for post-generation usage, as users cannot view, render and edit the resulting 3D content from a full range. To address these challenges, we introduce HyperDreamer with several key designs and appealing properties: 1) Viewable: 360 degree mesh modeling with high-resolution textures enables the creation of visually compelling 3D models from a full range of observation points. 2) Renderable: Fine-grained semantic segmentation and data-driven priors are incorporated as guidance to learn reasonable albedo, roughness, and specular properties of the materials, enabling semantic-aware arbitrary material estimation. 3) Editable: For a generated model or their own data, users can interactively select any region via a few clicks and efficiently edit the texture with text-based guidance. Extensive experiments demonstrate the effectiveness of HyperDreamer in modeling region-aware materials with high-resolution textures and enabling user-friendly editing. We believe that HyperDreamer holds promise for advancing 3D content creation and finding applications in various domains.

AK

48,289 views • 2 years ago

The “I Believe” music video has officially landed on Joe’s YouTube channel for the very first time — now upscaled in HD! 🎬🎸 A standout track from Flying in a Blue Dream, “I Believe” is also one of the rare songs to feature Joe on vocals 🎤 Directed by David Hogan, who got his start designing legendary album covers for Stax Records and went on to direct Barb Wire (1996) and also work on Batman Forever and Alien 3. Watch now via the link in bio or at and stand by as we have news of more classic videos getting a good upscale treatment! 🎞️ #JoeSatriani #FlyingInABlueDream

The “I Believe” music video has officially landed on Joe’s YouTube channel for the very first time — now upscaled in HD! 🎬🎸 A standout track from Flying in a Blue Dream, “I Believe” is also one of the rare songs to feature Joe on vocals 🎤 Directed by David Hogan, who got his start designing legendary album covers for Stax Records and went on to direct Barb Wire (1996) and also work on Batman Forever and Alien 3. Watch now via the link in bio or at and stand by as we have news of more classic videos getting a good upscale treatment! 🎞️ #JoeSatriani #FlyingInABlueDream

Joe Satriani

17,149 views • 3 months ago

XAI EXTENDS GROK IMAGINE VIDEO GENERATION TO 10 SECONDS WITH QUALITY ENHANCEMENTS xAI has updated its Grok Imagine tool to produce videos lasting 10 seconds, doubling the prior limit. This change, along with refinements in visual and audio elements, expands the tool's utility for short-form content creation. xAI released the upgrade to Grok Imagine in early 2026. The company, founded by Elon Musk, announced the feature through posts on the X platform. This follows previous iterations where videos were capped at shorter durations, typically 5 seconds. Grok Imagine allows users to generate videos from text prompts, building on its image creation capabilities. The update addresses constraints in video length that limited expressive potential. Users can now input descriptions to create clips, such as animations or scenes, without needing initial images. This positions the tool within the broader landscape of AI-driven multimodal generation, where text-to-video systems are increasingly common. The core adjustment doubles the maximum video duration from 5 seconds to 10 seconds. Accompanying this are upgrades to video quality, including more stable visuals, richer details, and improved clarity. Audio has also been enhanced for better output, making the generated content more immersive. These changes were described as "big improvements across the board" in the announcement. No specific benchmarks or quantitative metrics for the quality improvements were detailed in the release statements. The feature rollout appears gradual, with some users accessing it via the Grok app or web interface. xAI has not introduced user controls for exact timing, though such options are mentioned as future possibilities. This development highlights xAI's emphasis on iterative enhancements in generative AI tools. By extending duration while refining output fidelity, it reflects engineering priorities aimed at balancing computational efficiency with user needs. The focus on audio and visual stability suggests attention to common pitfalls in early text-to-video models, such as inconsistencies or artifacts. The sources do not specify the underlying model architecture changes or training data adjustments enabling this upgrade. Performance in real-world scenarios, like handling complex prompts or maintaining consistency across clips, remains unquantified in the announcements. Interpretations of broader implications for AI video generation would require additional evidence beyond what's provided.

XAI EXTENDS GROK IMAGINE VIDEO GENERATION TO 10 SECONDS WITH QUALITY ENHANCEMENTS xAI has updated its Grok Imagine tool to produce videos lasting 10 seconds, doubling the prior limit. This change, along with refinements in visual and audio elements, expands the tool's utility for short-form content creation. xAI released the upgrade to Grok Imagine in early 2026. The company, founded by Elon Musk, announced the feature through posts on the X platform. This follows previous iterations where videos were capped at shorter durations, typically 5 seconds. Grok Imagine allows users to generate videos from text prompts, building on its image creation capabilities. The update addresses constraints in video length that limited expressive potential. Users can now input descriptions to create clips, such as animations or scenes, without needing initial images. This positions the tool within the broader landscape of AI-driven multimodal generation, where text-to-video systems are increasingly common. The core adjustment doubles the maximum video duration from 5 seconds to 10 seconds. Accompanying this are upgrades to video quality, including more stable visuals, richer details, and improved clarity. Audio has also been enhanced for better output, making the generated content more immersive. These changes were described as "big improvements across the board" in the announcement. No specific benchmarks or quantitative metrics for the quality improvements were detailed in the release statements. The feature rollout appears gradual, with some users accessing it via the Grok app or web interface. xAI has not introduced user controls for exact timing, though such options are mentioned as future possibilities. This development highlights xAI's emphasis on iterative enhancements in generative AI tools. By extending duration while refining output fidelity, it reflects engineering priorities aimed at balancing computational efficiency with user needs. The focus on audio and visual stability suggests attention to common pitfalls in early text-to-video models, such as inconsistencies or artifacts. The sources do not specify the underlying model architecture changes or training data adjustments enabling this upgrade. Performance in real-world scenarios, like handling complex prompts or maintaining consistency across clips, remains unquantified in the announcements. Interpretations of broader implications for AI video generation would require additional evidence beyond what's provided.

Lacey

28,881 views • 8 months ago

Guys, I need your help! How is X protecting original content creators? They aren't. In fact, the Head of Product blocks them! Here's my original video about the Tower of Babel: my idea for the comparison, my prompts, my credits, and my original text accompanying the video. It received 1.5 million impressions on X. Three days later, people stole my video, added their own watermarks, and posted it on TikTok, making money from my work. They even translated my entire text into their native language and reposted it. The result? I received a 90% reduction in revenue share, while this guy is probably making money from the 200,000 views generated by my work. Meanwhile, the same Head of Product is protecting content thieves on X who steal videos and simply add their own watermarks. How is this fair? Will someone from X answer me? If anyone has connections to X, Elon, or anyone else at the company, please ask them. I'm not crying about the low revenue share. I'm upset because there is no justice. Original creators are punished, while content thieves are rewarded.

Guys, I need your help! How is X protecting original content creators? They aren't. In fact, the Head of Product blocks them! Here's my original video about the Tower of Babel: my idea for the comparison, my prompts, my credits, and my original text accompanying the video. It received 1.5 million impressions on X. Three days later, people stole my video, added their own watermarks, and posted it on TikTok, making money from my work. They even translated my entire text into their native language and reposted it. The result? I received a 90% reduction in revenue share, while this guy is probably making money from the 200,000 views generated by my work. Meanwhile, the same Head of Product is protecting content thieves on X who steal videos and simply add their own watermarks. How is this fair? Will someone from X answer me? If anyone has connections to X, Elon, or anyone else at the company, please ask them. I'm not crying about the low revenue share. I'm upset because there is no justice. Original creators are punished, while content thieves are rewarded.

Open Minded Approach

64,558 views • 4 months ago

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ 🛠️ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ 🛠️ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

AI at Meta

2,267,342 views • 2 years ago

🚀 SANA-Video 2.0 is here! A full-stack optimized video model designed for efficiency — while still delivering high quality. We combine a hybrid architecture closely related to the recent Kimi K3 design, adopt Self-Flow from FLUX 3, and further accelerate it with our own Sol-Engine. Key technical ingredients: 🧠 Hybrid Linear–Softmax Attention 3 gated linear layers + 1 gated-softmax anchor (75% linear / 25% softmax) 🧱 Block Attention Residuals (AttnRes) Boosts deep-layer effective rank by ~12% 🏗️ Unified 5B & 14B models Trained from scratch under very limited resources: • 5B → only 16 nodes of H100 • 14B → 48 nodes of B200 Results 📊 • 84.30 VBench Total • 3.2× faster DiT forward than matched full-softmax at 720p/60s • 720p/5s in 13.06s on a single H100 with Sol-Engine • 120× faster than Wan 2.2-A14B under the same one-H100 setup SANA-Video 2.0 shows that high-quality 720p video generation can be both efficient and practical on a single GPU. 🎬 Project: 📄 Paper: 💻 Code: Proud of the team! 🎉 More details below 🧵

🚀 SANA-Video 2.0 is here! A full-stack optimized video model designed for efficiency — while still delivering high quality. We combine a hybrid architecture closely related to the recent Kimi K3 design, adopt Self-Flow from FLUX 3, and further accelerate it with our own Sol-Engine. Key technical ingredients: 🧠 Hybrid Linear–Softmax Attention 3 gated linear layers + 1 gated-softmax anchor (75% linear / 25% softmax) 🧱 Block Attention Residuals (AttnRes) Boosts deep-layer effective rank by ~12% 🏗️ Unified 5B & 14B models Trained from scratch under very limited resources: • 5B → only 16 nodes of H100 • 14B → 48 nodes of B200 Results 📊 • 84.30 VBench Total • 3.2× faster DiT forward than matched full-softmax at 720p/60s • 720p/5s in 13.06s on a single H100 with Sol-Engine • 120× faster than Wan 2.2-A14B under the same one-H100 setup SANA-Video 2.0 shows that high-quality 720p video generation can be both efficient and practical on a single GPU. 🎬 Project: 📄 Paper: 💻 Code: Proud of the team! 🎉 More details below 🧵

Enze Xie

25,648 views • 2 months ago

I recorded how I use agents day to day. In this video: • Agents spawn other agents to check whether performance improvements actually helped in production. • I have an agent fix UI flicker and test the result in a portal. • We investigate and fix an `amp clone` bug for a Nix user. • I toggle feature flags directly from the remote machine. • I ask an agent for "irrefutable proof" that its fix works. You can see all the prompts, what the agents do, and how I review their work (or don't). Timestamps: 0:15 Shipping and monitoring performance improvements 2:11 Investigating an amp clone bug in Nix 3:51 Fixing UI flicker and testing in a portal 5:55 Starting an agent to promote Recap to an opt-in experimental feature 7:45 Previewing the sidebar flicker fix in a portal 9:38 Reviewing production performance canaries 11:45 Checking in on the production fix and granting read-only log access 12:47 Check Recap Experimental Setting Copy 13:39 Check in on amp clone Nix fix 14:49 Sidebar fix deployment check schedule 15:35 Checking in on performance fix thread's log analysis 16:14 Asking the agent for irrefutable proof of its fix 17:00 Checking Recap experimental setting 18:31 Testing a mechanical refactor of the CLI 21:48 Checking amp clone nix fix agent evidence 23:20 Checking feedback from Slack

I recorded how I use agents day to day. In this video: • Agents spawn other agents to check whether performance improvements actually helped in production. • I have an agent fix UI flicker and test the result in a portal. • We investigate and fix an `amp clone` bug for a Nix user. • I toggle feature flags directly from the remote machine. • I ask an agent for "irrefutable proof" that its fix works. You can see all the prompts, what the agents do, and how I review their work (or don't). Timestamps: 0:15 Shipping and monitoring performance improvements 2:11 Investigating an amp clone bug in Nix 3:51 Fixing UI flicker and testing in a portal 5:55 Starting an agent to promote Recap to an opt-in experimental feature 7:45 Previewing the sidebar flicker fix in a portal 9:38 Reviewing production performance canaries 11:45 Checking in on the production fix and granting read-only log access 12:47 Check Recap Experimental Setting Copy 13:39 Check in on amp clone Nix fix 14:49 Sidebar fix deployment check schedule 15:35 Checking in on performance fix thread's log analysis 16:14 Asking the agent for irrefutable proof of its fix 17:00 Checking Recap experimental setting 18:31 Testing a mechanical refactor of the CLI 21:48 Checking amp clone nix fix agent evidence 23:20 Checking feedback from Slack

Thorsten Ball

339,119 views • 1 month ago

Our team S.N. Bose National Centre for Basic Sciences introduces PathGennie (just published in JCIM & JCTC Journals), a fast and unbiased computational framework for uncovering rare molecular events such as drug unbinding and protein folding. Instead of using any biased forces or elevated temperatures, PathGennie launches ultrashort unbiased MD trajectories and selectively propagates only those that show real progress toward the transition. This lets us generate complete rare-event pathways in just 10–100 ps, while preserving true molecular dynamics. Full credit goes to my PhD student Dibyendu Maity DIBYENDU MAITY for formulating and implementing the PathGennie code. Thanks to Anusandhan National Research Foundation for funding. ✨ Key Highlights: • Identified 7 distinct unbinding routes for benzene from T4 lysozyme • Recovered all major exit pathways for imatinib from Abl kinase within a handful of attempts • Captured folding/unfolding pathways of Trp-cage and Protein G that align with minimum free-energy paths • Weighted Ensemble (WE) simulations converge 3–4× faster when initialised with PathGennie-generated pathways • Path finding works even with suboptimal CVs 🔗 Try our code: 📄 Read the paper: #CompChem #RareEvent #EnhancedSampling #MolecularDynamics #PathSampling #DrugDiscovery

Our team S.N. Bose National Centre for Basic Sciences introduces PathGennie (just published in JCIM & JCTC Journals), a fast and unbiased computational framework for uncovering rare molecular events such as drug unbinding and protein folding. Instead of using any biased forces or elevated temperatures, PathGennie launches ultrashort unbiased MD trajectories and selectively propagates only those that show real progress toward the transition. This lets us generate complete rare-event pathways in just 10–100 ps, while preserving true molecular dynamics. Full credit goes to my PhD student Dibyendu Maity DIBYENDU MAITY for formulating and implementing the PathGennie code. Thanks to Anusandhan National Research Foundation for funding. ✨ Key Highlights: • Identified 7 distinct unbinding routes for benzene from T4 lysozyme • Recovered all major exit pathways for imatinib from Abl kinase within a handful of attempts • Captured folding/unfolding pathways of Trp-cage and Protein G that align with minimum free-energy paths • Weighted Ensemble (WE) simulations converge 3–4× faster when initialised with PathGennie-generated pathways • Path finding works even with suboptimal CVs 🔗 Try our code: 📄 Read the paper: #CompChem #RareEvent #EnhancedSampling #MolecularDynamics #PathSampling #DrugDiscovery

Suman Chakrabarty

54,353 views • 11 months ago

Here's my 30-second spec ad for WinRAR (that nobody asked for). Okay, perhaps Rourke Heath did. He shared this fun brief with his GenHQ community last week, and I took it on. In this experimental project, I wanted to turn one of the internet's longest-running jokes into a subtle family drama about WinRAR's 40-day trial that somehow lasted almost three decades. Here's how I made it: After writing the script (the old school way), I worked with ChatGPT 5.6 Sol Max to turn it into a 14,000-character prompt for Seedance 2.5 (yes, Dreamina allows prompts that long). I used GPT Image 2 to create the reference image for WinRAR's trial screen, while Seed Audio 1.0 generated the audio reference for the full voiceover and dialogue. After 20+ Seedance 2.5 tests at 480p, with the prompt evolving throughout, I generated the final versions at 720p. I then lightly edited the video and upscaled it to 1080p with Topaz Labs Astra Starlight Precision 2.6. However, the screenshot text didn't survive the upscale (I should have used Astra's scene detection feature and selected a different model for that particular shot), so I recreated it using Seedance 2.0 at 1080p. (I also tried Hailuo H3 and Kling 3.0, but in this particular case, Seedance 2.0 gave me the best overall result in one go.) For the sound, I selected a solo-instrument soundtrack from Epidemic Sound and added extra sound effects to improve the overall sound design. And here's the result. I'm curious to hear what you think. Did this bring back memories, or is this genuinely your first time hearing about WinRAR? I’d love to know.

Here's my 30-second spec ad for WinRAR (that nobody asked for). Okay, perhaps Rourke Heath did. He shared this fun brief with his GenHQ community last week, and I took it on. In this experimental project, I wanted to turn one of the internet's longest-running jokes into a subtle family drama about WinRAR's 40-day trial that somehow lasted almost three decades. Here's how I made it: After writing the script (the old school way), I worked with ChatGPT 5.6 Sol Max to turn it into a 14,000-character prompt for Seedance 2.5 (yes, Dreamina allows prompts that long). I used GPT Image 2 to create the reference image for WinRAR's trial screen, while Seed Audio 1.0 generated the audio reference for the full voiceover and dialogue. After 20+ Seedance 2.5 tests at 480p, with the prompt evolving throughout, I generated the final versions at 720p. I then lightly edited the video and upscaled it to 1080p with Topaz Labs Astra Starlight Precision 2.6. However, the screenshot text didn't survive the upscale (I should have used Astra's scene detection feature and selected a different model for that particular shot), so I recreated it using Seedance 2.0 at 1080p. (I also tried Hailuo H3 and Kling 3.0, but in this particular case, Seedance 2.0 gave me the best overall result in one go.) For the sound, I selected a solo-instrument soundtrack from Epidemic Sound and added extra sound effects to improve the overall sound design. And here's the result. I'm curious to hear what you think. Did this bring back memories, or is this genuinely your first time hearing about WinRAR? I’d love to know.

Burak Tuyan

677,626 views • 1 month ago

Today we're announcing #GAIA1: a 9B parameter world model, trained on 4,700 hours of driving data, able to simulate complex and diverse driving scenes from video, text and action inputs. This model is 480x larger than the preview we shared earlier this year and the results are incredible. These videos are entirely synthetically generated by Wayve's generative AI, GAIA-1. But there is more here than just generating videos, GAIA is an entire world model. A world model allows us to simulate the future, conditioned on video, text and action inputs, which can be leveraged for making informed decisions when driving. Why is this game-changing for autonomous driving? 1. Safety. One limitation with AI systems like today's Large Language Models is that they are autoregressive, next-word prediction algorithms, but aren't necessarily aware of the implications of their decisions. A world model allows us to give our AI the capability to be aware of its decisions, by simulating the future, which is important for self-driving safety. 2. Synthetic training data. I believe synthetic training data is the future for AI, because it is safer, cheaper, and infinitely scalable. GAIA-1 unlocks unprecedented realism and diversity of synthetic data for self-driving. 3. Long-tail robustness. One of the biggest challenges for self-driving is long-tail robustness: dealing with the enormous magnitude of edge cases we see on the road. An advantage of generative AI is its incredible ability to recombine experiences in new ways. This is exciting for self-driving as it means we can learn from two edge case scenarios, and combine them to become a corner case. For example, we can experience driving in fog, and experience of jay-walking pedestrians, and GAIA can learn from these experiences to understand how to generate a fog+jay walking scenario. Check out many more videos in our blog or further technical details in our paper: Or come chat with our team who are at the International Conference on Computer Vision (#ICCV2023) this week in Paris in Booth 32 Jamie Shotton

Today we're announcing #GAIA1: a 9B parameter world model, trained on 4,700 hours of driving data, able to simulate complex and diverse driving scenes from video, text and action inputs. This model is 480x larger than the preview we shared earlier this year and the results are incredible. These videos are entirely synthetically generated by Wayve's generative AI, GAIA-1. But there is more here than just generating videos, GAIA is an entire world model. A world model allows us to simulate the future, conditioned on video, text and action inputs, which can be leveraged for making informed decisions when driving. Why is this game-changing for autonomous driving? 1. Safety. One limitation with AI systems like today's Large Language Models is that they are autoregressive, next-word prediction algorithms, but aren't necessarily aware of the implications of their decisions. A world model allows us to give our AI the capability to be aware of its decisions, by simulating the future, which is important for self-driving safety. 2. Synthetic training data. I believe synthetic training data is the future for AI, because it is safer, cheaper, and infinitely scalable. GAIA-1 unlocks unprecedented realism and diversity of synthetic data for self-driving. 3. Long-tail robustness. One of the biggest challenges for self-driving is long-tail robustness: dealing with the enormous magnitude of edge cases we see on the road. An advantage of generative AI is its incredible ability to recombine experiences in new ways. This is exciting for self-driving as it means we can learn from two edge case scenarios, and combine them to become a corner case. For example, we can experience driving in fog, and experience of jay-walking pedestrians, and GAIA can learn from these experiences to understand how to generate a fog+jay walking scenario. Check out many more videos in our blog or further technical details in our paper: Or come chat with our team who are at the International Conference on Computer Vision (#ICCV2023) this week in Paris in Booth 32 Jamie Shotton

Alex Kendall

631,921 views • 3 years ago

great to see more people generating 3d avatars with our new text-to-3d feature in forge. this marks a step in the right direction in putting powerful creation tools directly in the hands of everyone. we built forge entirely from the ground up over the past months as one of the key releases on our roadmap. having full proprietary ownership of the technology gives us complete control to shape its direction without depending on external platforms or third-party licenses. building on this foundation, our upcoming studio feature will let users generate high-quality accessories, clothing, and environments simply by typing natural language prompts. a single description can produce fully textured, production-ready 3d assets in seconds, with options to create multiple variations and refine them through follow-up instructions.every asset created in studio integrates seamlessly with the 3d ai agents made in forge on users can instantly apply clothing and accessories with automatic fitting, layer multiple items, and place their agents inside custom-generated environments. all clothing and accessories come pre-rigged and optimized, while environments include proper lighting and geometry for immediate use in games, animation, virtual worlds, and more. we have spent months thoughtfully designing how can deliver real, sustainable value back to the community. we will continue to share more details on token utility use cases and the economic flywheel we have built. the goal is to create a self-reinforcing system where creators earn through royalties, autonomous agents drive on-chain activity, and platform growth directly benefits active community members and token holders. together, these features enable a complete creative flow. from a simple idea, anyone can quickly build fully realized 3d ai agents standing in rich, custom scenes. we are excited to see what the community builds next.

great to see more people generating 3d avatars with our new text-to-3d feature in forge. this marks a step in the right direction in putting powerful creation tools directly in the hands of everyone. we built forge entirely from the ground up over the past months as one of the key releases on our roadmap. having full proprietary ownership of the technology gives us complete control to shape its direction without depending on external platforms or third-party licenses. building on this foundation, our upcoming studio feature will let users generate high-quality accessories, clothing, and environments simply by typing natural language prompts. a single description can produce fully textured, production-ready 3d assets in seconds, with options to create multiple variations and refine them through follow-up instructions.every asset created in studio integrates seamlessly with the 3d ai agents made in forge on users can instantly apply clothing and accessories with automatic fitting, layer multiple items, and place their agents inside custom-generated environments. all clothing and accessories come pre-rigged and optimized, while environments include proper lighting and geometry for immediate use in games, animation, virtual worlds, and more. we have spent months thoughtfully designing how can deliver real, sustainable value back to the community. we will continue to share more details on token utility use cases and the economic flywheel we have built. the goal is to create a self-reinforcing system where creators earn through royalties, autonomous agents drive on-chain activity, and platform growth directly benefits active community members and token holders. together, these features enable a complete creative flow. from a simple idea, anyone can quickly build fully realized 3d ai agents standing in rich, custom scenes. we are excited to see what the community builds next.

nich

19,611 views • 4 months ago

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360°, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360° head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data 🏠 🌍 🎥 Great work by Tobias Kirschstein and Simon Giebenhain!

Matthias Niessner

96,371 views • 10 months ago