Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

🚀🎬We introduce TMD (Transition Matching Distillation): 480p videos generated from text prompts in < 3 NFEs! 1️⃣Main backbone for feature extraction and lightweight head for iterative refinement 2️⃣Distilled from Wan2.1 14B T2V combining MeanFlow & DMD2 🔗

16,985 görüntüleme • 7 ay önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.

AK

40,485 görüntüleme • 2 yıl önce

XAI EXTENDS GROK IMAGINE VIDEO GENERATION TO 10 SECONDS WITH QUALITY ENHANCEMENTS xAI has updated its Grok Imagine tool to produce videos lasting 10 seconds, doubling the prior limit. This change, along with refinements in visual and audio elements, expands the tool's utility for short-form content creation. xAI released the upgrade to Grok Imagine in early 2026. The company, founded by Elon Musk, announced the feature through posts on the X platform. This follows previous iterations where videos were capped at shorter durations, typically 5 seconds. Grok Imagine allows users to generate videos from text prompts, building on its image creation capabilities. The update addresses constraints in video length that limited expressive potential. Users can now input descriptions to create clips, such as animations or scenes, without needing initial images. This positions the tool within the broader landscape of AI-driven multimodal generation, where text-to-video systems are increasingly common. The core adjustment doubles the maximum video duration from 5 seconds to 10 seconds. Accompanying this are upgrades to video quality, including more stable visuals, richer details, and improved clarity. Audio has also been enhanced for better output, making the generated content more immersive. These changes were described as "big improvements across the board" in the announcement. No specific benchmarks or quantitative metrics for the quality improvements were detailed in the release statements. The feature rollout appears gradual, with some users accessing it via the Grok app or web interface. xAI has not introduced user controls for exact timing, though such options are mentioned as future possibilities. This development highlights xAI's emphasis on iterative enhancements in generative AI tools. By extending duration while refining output fidelity, it reflects engineering priorities aimed at balancing computational efficiency with user needs. The focus on audio and visual stability suggests attention to common pitfalls in early text-to-video models, such as inconsistencies or artifacts. The sources do not specify the underlying model architecture changes or training data adjustments enabling this upgrade. Performance in real-world scenarios, like handling complex prompts or maintaining consistency across clips, remains unquantified in the announcements. Interpretations of broader implications for AI video generation would require additional evidence beyond what's provided.

Lacey

28,881 görüntüleme • 7 ay önce

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date. Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in entirely new possibilities for casual creators and creative professionals alike. More details and examples of what Movie Gen can do ➡️ 🛠️ Movie Gen models and capabilities Movie Gen Video: 30B parameter transformer model that can generate high-quality and high-definition images and videos from a single text prompt. Movie Gen Audio: A 13B parameter transformer model that can take a video input along with optional text prompts for controllability to generate high-fidelity audio synced to the video. It can generate ambient sound, instrumental background music and foley sound — delivering state-of-the-art results in audio quality, video-to-audio alignment and text-to-audio alignment. Precise video editing: Using a generated or existing video and accompanying text instructions as an input it can perform localized edits such as adding, removing or replacing elements — or global changes like background or style changes. Personalized videos: Using an image of a person and a text prompt, the model can generate a video with state-of-the-art results on character preservation and natural movement in video. We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future.

AI at Meta

2,265,850 görüntüleme • 1 yıl önce

[SIGGRAPH 2025] Photoreal Scene Reconstruction from an Egocentric Device Contributions: 1. We address the importance of employing visual-inertial bundle adjustment (VIBA) that accounts for the rolling-shutter behavior of the RGB camera. This provides a continuous camera trajectory to model pixel movement in neural reconstruction. Our experiments demonstrate that using VIBA consistently improves the novel view quality in Gaussian Splatting by +1 dB in PSNR. 2. We introduce a rasterization-based image formulation pipeline that addresses common artifacts in physical image formation, including rolling shutter, lens shading, exposure, and gain compensation. Our approach is distinct in that we represent image poses as posed pixel arrays sampled from a continuous trajectory, rather than assigning a single camera pose per image, and preserve the merit of Gaussian rasterization. Unlike existing methods that require ray-tracing Gaussians, e.g., [Moenne-Loccoz et al. 2024], our formulation is applicable to general-purpose rasterization-based Gaussian splatting. When applied to 3D Gaussian Splatting (3DGS) [Kerbl et al. 2023], our approach can further enhance reconstruction quality by +1 dB. We outperform existing baselines and demonstrate a substantial quality improvement in handling complex scenes observed by egocentric devices. 3. To reduce the effect of blur from rapid head motion in darker indoor scenes, we propose a strategy of deliberately underexposing input videos during capture, inspired by HDR+ [Hasinoff et al. 2016]. We demonstrate that we can reconstruct high-quality, noise-free scene radiance from noisy, dim input videos, and further render sharp, blur-free videos at a higher dynamic range.

MrNeRF

15,244 görüntüleme • 1 yıl önce

Today we're announcing #GAIA1: a 9B parameter world model, trained on 4,700 hours of driving data, able to simulate complex and diverse driving scenes from video, text and action inputs. This model is 480x larger than the preview we shared earlier this year and the results are incredible. These videos are entirely synthetically generated by Wayve's generative AI, GAIA-1. But there is more here than just generating videos, GAIA is an entire world model. A world model allows us to simulate the future, conditioned on video, text and action inputs, which can be leveraged for making informed decisions when driving. Why is this game-changing for autonomous driving? 1. Safety. One limitation with AI systems like today's Large Language Models is that they are autoregressive, next-word prediction algorithms, but aren't necessarily aware of the implications of their decisions. A world model allows us to give our AI the capability to be aware of its decisions, by simulating the future, which is important for self-driving safety. 2. Synthetic training data. I believe synthetic training data is the future for AI, because it is safer, cheaper, and infinitely scalable. GAIA-1 unlocks unprecedented realism and diversity of synthetic data for self-driving. 3. Long-tail robustness. One of the biggest challenges for self-driving is long-tail robustness: dealing with the enormous magnitude of edge cases we see on the road. An advantage of generative AI is its incredible ability to recombine experiences in new ways. This is exciting for self-driving as it means we can learn from two edge case scenarios, and combine them to become a corner case. For example, we can experience driving in fog, and experience of jay-walking pedestrians, and GAIA can learn from these experiences to understand how to generate a fog+jay walking scenario. Check out many more videos in our blog or further technical details in our paper: Or come chat with our team who are at the International Conference on Computer Vision (#ICCV2023) this week in Paris in Booth 32 Jamie Shotton

Alex Kendall

631,869 görüntüleme • 2 yıl önce

great to see more people generating 3d avatars with our new text-to-3d feature in forge. this marks a step in the right direction in putting powerful creation tools directly in the hands of everyone. we built forge entirely from the ground up over the past months as one of the key releases on our roadmap. having full proprietary ownership of the technology gives us complete control to shape its direction without depending on external platforms or third-party licenses. building on this foundation, our upcoming studio feature will let users generate high-quality accessories, clothing, and environments simply by typing natural language prompts. a single description can produce fully textured, production-ready 3d assets in seconds, with options to create multiple variations and refine them through follow-up instructions.every asset created in studio integrates seamlessly with the 3d ai agents made in forge on users can instantly apply clothing and accessories with automatic fitting, layer multiple items, and place their agents inside custom-generated environments. all clothing and accessories come pre-rigged and optimized, while environments include proper lighting and geometry for immediate use in games, animation, virtual worlds, and more. we have spent months thoughtfully designing how can deliver real, sustainable value back to the community. we will continue to share more details on token utility use cases and the economic flywheel we have built. the goal is to create a self-reinforcing system where creators earn through royalties, autonomous agents drive on-chain activity, and platform growth directly benefits active community members and token holders. together, these features enable a complete creative flow. from a simple idea, anyone can quickly build fully realized 3d ai agents standing in rich, custom scenes. we are excited to see what the community builds next.

nich

19,611 görüntüleme • 2 ay önce

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360°, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360° head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data 🏠 🌍 🎥 Great work by Tobias Kirschstein and Simon Giebenhain!

Matthias Niessner

96,238 görüntüleme • 8 ay önce

New short course: Vibe Coding 101 with Replit! Learn to build and host applications with an AI agent in this course, built in partnership with Replit ⠕ and taught by its President Michele Catasta and Head of Developer Relations . Coding agents are changing how we write code. "Vibe coding" refers to a growing practice where you might barely look at the generated code, and instead focus on the architecture and features of your application. However, contrary to popular belief, effectively coding this way isn't done by just prompting, accepting all recommendations, and hoping for the best. It requires structuring your work, refining your prompts, and having a systematic process that lead to a more efficient and effective workflow. I code frequently using LLMs, and asking an LLM to do everything in one shot usually does not work. I'll typically take a problem, partition it into manageable modules, spend time creating prompts to specify each module, and use the model to produce the code one module at a time, and test/debug each module before moving on. A process like this is making me and many other developers faster and more efficient. In this video-only course, you’ll learn how to use Replit’s cloud environment--with an integrated code editor, package manager, and deployment tools--to build and deploy web applications. Along the way, you’ll learn strategies for working effectively with agents and improve your development skills. In detail, you’ll: - Understand principles of agentic code development such as being precise, giving agents one task at a time, making prompts specific, keeping projects tidy, starting with fresh sessions for each new feature, and how to approach debugging. - Learn how to get started with Replit, and key skills for vibe coding: Thinking, using frameworks, checkpoints, debugging, and providing context. - Create a product requirement document (PRD) and wireframe for your agent to build a prototype of a website performance analyzer. - See how to use an agent to make your prototype more visually appealing, and deploy it application others to access . - Learn to build a head-to-head national park ranking app, from a sample dataset, with voting capabilities and persistent data storage, and refine further ask the assistant to recap and explain what it built to find room for improvement and reinforce your learning. By the end of this course, you’ll have a solid foundation in building with coding agents, and a process you can use to keep vibe coding effectively. Please sign up here:

Andrew Ng

752,900 görüntüleme • 1 yıl önce

Sora 2 has the capabilities to make an entire anime episode with time and effort. This is a turning point for generative AI models and it is terrifying to think what is next to come. The difference between Sora and Sora 2 is absolutely staggering. Yes, this looks cool. I put the time into it to make it as close to art as possible. This was not simple to make and not just prompting and throwing it in some editor. Every frame is generated of another scene and even the reference image for characters I used to start this project were AI generated. This was all created with 10 second clips pulled apart and chunked out into the final product. Regardless of the imagery you see, I did not directly use any artist's art for this video. This 10-minute video was generated over the course of a week from more than 700 text prompts. It was built on technology trained by scraping the uncredited, uncompensated contributions of countless human artists and animators. The creation of this single video consumed an enormous amount of energy and water, equivalent to powering a home for days and requiring hundreds of liters of fresh water for cooling. But that is nothing. In the 10 minutes you spend watching this, the global network of AI video generators will create over 6,250 more short videos. The combined energy required for that 10 minutes of global creation is enough to power an average household for nearly 2 years. - It's a double-edged sword. While AI uses immense amount of energy has clear immoral issues with scalping the hard work of artists, this also provides those a medium who have potentially spent their life attempting to draw out the ideas in their head and failing to grasp it. My sister is an artist, I grew up always attempting to draw but never could get the image in my head on paper, I've spent the better part of over 20 years to teach myself through watching videos and practice, however I don't have steady hands and frankly have just been unable to make any vision come to life. Which brings me to the conflict here. This would have been a dream of mine to be able to get frames created and flesh out the story in my head. To make this video professionally, it would be an absurd amount of money. A small studio by itself would be over a million dollars just to start up. To hire a studio would be most likely over ~200k. I would love to make this story I have in my head through legit and traditional means, I would love to start a Kickstarter to get funding and hire artists, voice actors, and a production team. However I know that would most likely be an impossible reach that would further fuel the hate. Regardless, I understand the worry this causes. The fear this produces. However, we can't just ignore where AI is currently at and just tell people not to use it or even worse threaten people who use it. People will always use the shiny new toy in front of them, so the real question is how do we either make it work for us and work along side it, or how do we ACTUALLY implement a method to protect art rather than tell people to not use it. Respect your artists. Review their ToS and don't upload their hard work to a model without permission.

LUͦʷCͦk

18,417 görüntüleme • 10 ay önce