🔬 Built with an entirely new model architecture, our... diffusion-based approach uses 6B+ parameters and leverages the latest NVIDIA hardware. This is the most dynamic and wide-ranging video enhancing method we’ve ever created, setting a new standard for AI video restoration. Videos degrade due to compression artifacts, blurring, aliasing, noise, atmospheric distortion, missing pixels, etc. Each frame suffers from unique types of corruption, making AI video restoration a highly challenging task. Our technology solves this complexity by analyzing hundreds of frames to accurately restore details, delivering unmatched detail recovery combined with unparalleled temporal consistency.show more

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page:... Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.show more

Depth Any Video with Scalable Synthetic Data AI physicists... and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.show more

I am blown away 🤯. Check this out! CameraCtrl... II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models TL;DR: "To enable broader exploration of dynamic scenes, our model can generate new video clips of the same scene based on previously generated content and user-provided camera trajectories. This approach maintains dynamic capabilities, accurate camera control, and scene consistency throughout the extended exploration." "Our model enables precise camera control across diverse scenarios while preserving dynamic scene elements, e.g." "Our method can generate videos with strong 3D consistency, which enables high-quality 3D reconstruction using the camera-controlled videos." Contributions: 1) A systematic data curation pipeline for constructing a dynamic video dataset with camera trajectory annotations; 2) A lightweight camera control injection module and corresponding training strategy that preserves dynamic video generation capabilities while adding camera control effect; 3) A clip-wise autoregressive generation recipe that enables extended range exploration of generated scenes.show more

MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers paper... page: Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in practical applications. This study addresses this challenge by breaking down the text-based video editing process into two separate stages. In the first stage, we leverage an existing text-to-image diffusion model to simultaneously edit a few keyframes without additional fine-tuning. In the second stage, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the keyframes, benefiting from structural guidance provided by intermediate frames. Our comprehensive set of experiments illustrates the efficacy and efficiency of MaskINT when compared to other diffusion-based methodologies. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.show more

1/ Gemini 2.5 is here, and it’s our most... intelligent AI model ever. Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on Arena by a significant margin. With a model this intelligent, we wanted to get it to people as quickly as possible. Find it on Google AI Studio and in the Google Gemini for Gemini Advanced users now – and in Vertex in the coming weeks. This is the start of a new era of thinking models – and we can’t wait to see where things go from here.show more

Power Unrivalled 💪 Our infrastructure speaks for itself. From... ASIC miners to GPUs and Nodes, every piece of our hardware is real, operational, and we have full and complete ownership. With multiple facilities worldwide, we’re setting the standard others can only aspire to. Transparency for Hash AI is made simple as a result of genuine expertise and dedication to building profitable, scalable, and future-proof infrastructure. The video below showcases our GPUs actively mining in Ethiopia, one of our many operation centres. This is the sound of progress and power, built on a foundation of hard work and commitment to excellence. #BTC #DOGE #LTCshow more

This week we made it clear – X the... Everything App – is closer than everyone thinks! Nothing can slow us down. X is part of a constellation of companies working for the betterment of humanity. We're moving fast for our communities, creators, and businesses!! Big themes for X CES this year? Freedom of Speech, a new Video Ecosystem, and the Power of AI. It's all coming to life in plain sight and in real time. Here's what we discussed this week: – X is an app for everyone. We're building an information independence that's essential for society. From our live stream collaboration with CES to new content partnerships with Tulsi Gabbard 🌺, bluesky and Jim Rome, we’re expanding perspectives on X and unlocking new commercial opportunities. – We're building a new video ecosystem with our partners. There's never been more economic opportunity on X. New shopping experiences, financial partnerships for payments, AI collaborations, and a recruitment product are just the beginning. Our new partnerships with Shopify and Integral Ad Science hint at our ambitions for e-commerce and video. – AI is revolutionizing X. With 500 million searches daily, we're making information more relevant and useful. AI offers new opportunities for advertisers – better targeting to content creation. Imagine Grok as a concierge for all businesses! If 2023 was foundational, 2024 will be completely transformational for X!show more

doodles AI beta. next week. we're building the tools... for a new era of dynamic world-building. it starts with an image model that reimagines anything and everything through the doodles lens. this is the first iteration of many. as the product evolves, we'll introduce the ability to turn your generations into physical objects. video with sound and dialogue, realtime AR, and gaming are all on the roadmap. doodles AI aligns us with the speed and scale of the AI industry at large. our colourful world can now be plugged into new tech as it unfolds. create with us.show more

You can't 3D reconstruct glass from images... ...WRONG! Thanks... for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AIshow more

Wonderland: Navigating 3D Scenes from a Single Image Contributions:... • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.show more

Create a 3D model from a single image, set... of images or a text prompt in < 1 minute 😮‍💨 This new AI paper called CAT3D shows us that it’ll keep getting easier to produce 3D models from 2D images — whether it’s a sparser real world 3D scan (a few photos instead of hundreds) or your favorite 2D image generator like Midjourney (just an image). How does this magic work? “This architecture is similar to video diffusion models, but with camera pose embeddings for each image instead of time embeddings. The generated views are passed into a robust 3D reconstruction pipeline to create the 3D representation (Zip-NeRF or 3DGS)”show more

Google dropped a new AI paper called LUMIERE. It's... remarkably flexible, supporting video inpainting, image-to-video, AND stylized video generation tasks. Say hello to “space-time diffusion” for video generation! Now what the heck does that mean exactly?! 🌐⏳ → TL;DR it utilizes a “Space-Time UNet” architecture that generates the full duration of the video in one pass, rather than generating distant keyframes and interpolating between them like prior works. Because the computation is done in this “compressed space-time representation” to generate the full clip at once, it's far more temporally consistent. → Another benefit of generating the full video at once is that you can “direct” the video generation, making it easier to hand off to other models/tasks without having to stitch together partial solutions. You can condition generations on additional inputs, meaning you get the full stack of AI video capabilities – from video inpainting to image-to-video and beyond. → New SOTA for AI video generation? User study results in the paper suggest human evaluators preferred Lumiere over Runway Gen-2, Pika Labs, and Stable Video Diffusion in terms of quality, text alignment AND motion. But as always, we need to get hands-on with this tech when Google *actually* decides to ship it. → Could this end up inside YouTube? Y’all know i’m obsessed with blending reality and imagination – so it’s the video inpainting tech I'm most excited about. I really hope this model finds its way into YouTube's Generative AI efforts, and based on their prior announcements and the list of acknowledgments in the paper I think it might! 🤞🏽 Links: 🔗Paper: 🔗Project:show more

Emerging from Silence: A New Dawn After a two-year... period of silence, the team behind Anthrometa emerges to announce significant advancements. Our focus has been on refining our vision, and now, we're prepared to unveil our strategic direction. 🕹️We're giving away 0.5 $ETH and 100 $ICP RT, like, and tag a friend for your chance to win ! Community Governance with DAO Central to our transformation is the establishment of a Decentralized Autonomous Organization (#DAO). This structure ensures that each member of our community has a say in Anthrometa's governance, embodying our commitment to collective wisdom and shared governance. Multichain Integration with $ICP and $ETH Our project has expanded to become multichain, integrating with the Internet Computer Protocol ($ICP) and Ethereum ($ETH). This choice was made due to #ICP's exceptional infrastructure, allowing for smart contracts to operate at web speed with heightened security. Gameplay 3.0 with Unreal Engine 5.5 and AI At the heart of our gameplay lies the utilization of #UE5, the most advanced technology for creating immersive universes. This engine powers our environment with unparalleled visual fidelity and dynamic interaction capabilities. Anthrometa : The Advanced Agent Furthermore, we are pioneering the integration of artificial intelligence to form an advanced Agent named Anthrometa. This #AI-driven entity will adapt and learn from player interactions, providing personalized experiences and evolving the game world in real-time, thus blurring the lines between player and game narrative. We aim to tap into the gaming market, which is expected to reach a valuation of over $300 billion by 2027, positioning Anthrometa alongside giants like Axie Infinity and Decentraland. Introducing THE METATRIBES: REBORN REBORN's battle royale mode departs from the conventional, eschewing modern firearms like those found in #Fortnite or #CallofDuty. Instead, players will engage in combat using ancestral weapons, focusing on authentic, melee-based encounters. The gameplay emphasizes strategic positioning, tactical thinking, and mastery of ancient combat techniques, making every battle a test of wits and skill rather than just firepower. This approach invites players into a realm where strategy reigns supreme, and every fight is a dance of survival and cunning. If you read this, you are early Join us in shaping a future where technology and community converge. Your support has been invaluable during our silence, and now, with renewed purpose, #Anthrometa returns. Don't miss our update on December 4th; follow, turn on notifications, and engage to be part of it.show more

This AI UGC workflow is f*cking nuts 🤯 One... prompt -> five completely different characters + five full ad variations, all generated automatically in a single run. Perfect for DTC brands and creative agencies who need volume but don't have time to build each variation from scratch. Most people making AI UGC are doing it one video at a time. New character, new prompt, new script, new render. Repeat. It works, but it doesn't scale. By the time you've built 5 variations to test, you've burned half a day. This workflow solves it: → Enter one initial prompt with your product and angle → Auto-generates 5 unique AI characters → Builds hook scripts for each variation → Writes the bridge with your actual product image → Creates the CTA — all 5 versions in one shot No building each video manually. No copy-pasting prompts over and over. No bottleneck between idea and creative testing. What you get: > 5 unique character variations from a single prompt > Hook, bridge, and CTA scripts tailored to each character > Product image integration baked in > A repeatable system you can run every time you need fresh creative Built 100% with AI. Want a copy of the full workflow for free? > Like this post > Comment "UGC" And I'll send it over (must be following so I can DM)show more

🎬 $PALM Presents: the long-awaited Creator Studio. A token... holdings based usage system to use the latest generative AI for animations and graphics. Forget about downloadable tools or paying high subscription fees for no usage for mediocre results. Creator Studio allows you to access the tools our developers use for high-quality animated video making with the latest Generative AI tools. Creator Studio is part of the Parrot Framework that assigns tasks to AI agents when possible. You can navigate in a seamless Web UI, making the quality, three-dimensional, non-trippy AI videos and sequences you've ever wanted. We support up to 100 images per user stored in the cloud for you to make video sequences of up to 5 minutes with. Most importantly, we don't charge you a fee - your usage depends on your amount of $PALM tokens! The usage is reset monthly, so if you suddenly run out and need more points to complete your creation, you know what to do - buy some $PALM. This and more is being deployed live at where you can see a preview.show more

Phase Shift Initiated Since before GTC 2024, NVIDIA GDN... (Graphics Delivery Network) has been a strong catalyst for the enthusiasm we have seen for our innovation, not only among the community but also among the team. NVIDIA’s technology, platforms, and teams have consistently inspired us - with GDN being no exception. Recently, we’ve recalibrated our development efforts, doubling down on bringing our release to GDN’s cutting-edge infrastructure. Five of our developers are now fully focused on GDN integration, and in this week alone, we’ve achieved four major backend milestones, and are quickly closing in on three more. These advancements are propelling Web3 technology directly onto NVIDIA GeForce Servers. By harnessing NVIDIA GDN platform, we’re transforming high-fidelity 3D content into a seamless Web3 experience accessible anywhere—directly in your browser. No downloads. No accounts. Just Blockchain. This breakthrough eliminates the reliance on high-end hardware, redefining accessibility for industries like gaming, manufacturing, and media. With Kondux and GDN, even the most resource-intensive 3D applications can be effortlessly streamed to any device, delivering unmatched performance and interactivity. We’re not just overcoming barriers; we’re creating an entirely new playground for high-fidelity 3D assets.show more

Our @Grammarly AI agents are here! Today, we’re launching... eight new AI agents designed for students and professionals. We created many of these agents with students in mind because they’re the first generation entering a job market where employers expect both subject expertise AND AI fluency. These agents help with everything from finding credible sources to predicting reader reactions. One agent we’ve gotten great feedback on is AI Grader (I wish I had this in school), which you can see in the video below. It looks at your assignment rubric and gives you suggestions like your professor would, and a grade prediction before you submit your work. And these agents are available in docs, our new AI-native writing surface! I’m deeply proud of this launch—docs is powered by Coda (Superhuman Docs) technology and is a great integration moment between Grammarly and Coda. This is just the beginning of Grammarly’s journey to offering agents that work everywhere people work and collaborate. I’ve been loving using these agents, and I’m excited for our customers to get access. Try them for yourself here and let me know what you think:show more