Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

New Generation Model! 🚨 We're introducing the Mistral model to our expanding lineup of generation models. Mistral brings efficient performance and strong language understanding capabilities to our platform. Initial testing shows promising results in code comprehension and generation tasks, making it a valuable addition to development workflows. While we... show more

ALCHEMIST AI 🔮

42,823 subscribers

16,435 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

New Generation Model! 🚨 We're introducing the o3m model to our expanding lineup of generation models. While still in its beta phase, we're actively refining its output to push its capabilities even further. So far, we've observed a significant leap in physics calculations, AI-driven mechanics, and the overall fluidity of app and game interactions. As we continue to tweak and optimize, expect even greater advancements in precision, automation, and creative execution.

New Generation Model! 🚨 We're introducing the o3m model to our expanding lineup of generation models. While still in its beta phase, we're actively refining its output to push its capabilities even further. So far, we've observed a significant leap in physics calculations, AI-driven mechanics, and the overall fluidity of app and game interactions. As we continue to tweak and optimize, expect even greater advancements in precision, automation, and creative execution.

ALCHEMIST AI 🔮

17,321 Aufrufe • vor 1 Jahr

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

Show-o One Single Transformer to Unify Multimodal Understanding and Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.

AK

124,048 Aufrufe • vor 2 Jahren

gemini omni flash is here: our high-quality, cost-efficient model for video generation and conversational editing designed to support multimodal workflows, it enables you to refine videos using natural language and simple prompting start building with it today via ai studio and the gemini api

gemini omni flash is here: our high-quality, cost-efficient model for video generation and conversational editing designed to support multimodal workflows, it enables you to refine videos using natural language and simple prompting start building with it today via ai studio and the gemini api

Google AI Studio

970,611 Aufrufe • vor 2 Monaten

Video generation is powerful but too slow for real-world robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)

Video generation is powerful but too slow for real-world robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)

Shuang Li

67,506 Aufrufe • vor 1 Jahr

Robotics keeps hitting the same wall. Single task RL works, but... it does not scale to hundreds of tasks or new embodiments. This new paper looks like a real step toward fixing that. The team introduces MMBench, a benchmark with 200 tasks across many domains and robots, and Newt, a language conditioned world model trained online across all 200 tasks at once. The simple idea behind Newt: The model learns from demos to get the right priors It trains across many tasks through online interaction It uses language to ground the goal It adapts fast when a new task shows up What stood out to me: ✅ One model trained on 200 tasks at the same time ✅ Language conditioned control for both states and RGB ✅ Better data efficiency than strong baselines ✅ Strong open loop control ✅ Fast adaptation to new tasks and embodiments ✅ Full release of 200 checkpoints, 4000 demos, code, and benchmark This is a good push toward general control instead of one model per task. If you want the full paper: Project page: —- Weekly robotics and AI insights. Subscribe free:

Robotics keeps hitting the same wall. Single task RL works, but... it does not scale to hundreds of tasks or new embodiments. This new paper looks like a real step toward fixing that. The team introduces MMBench, a benchmark with 200 tasks across many domains and robots, and Newt, a language conditioned world model trained online across all 200 tasks at once. The simple idea behind Newt: The model learns from demos to get the right priors It trains across many tasks through online interaction It uses language to ground the goal It adapts fast when a new task shows up What stood out to me: ✅ One model trained on 200 tasks at the same time ✅ Language conditioned control for both states and RGB ✅ Better data efficiency than strong baselines ✅ Strong open loop control ✅ Fast adaptation to new tasks and embodiments ✅ Full release of 200 checkpoints, 4000 demos, code, and benchmark This is a good push toward general control instead of one model per task. If you want the full paper: Project page: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

70,090 Aufrufe • vor 9 Monaten

Each booth at Apsara 2025 is buzzing with the latest AI innovations! From the next-generation large language models of the Qwen3 family and the Wan 2.5 models to the new agent development framework feature in Model Studio, the updates are rolling in! Stay tuned as we zoom into the booths and bring the breakthroughs to you!🔥

Each booth at Apsara 2025 is buzzing with the latest AI innovations! From the next-generation large language models of the Qwen3 family and the Wan 2.5 models to the new agent development framework feature in Model Studio, the updates are rolling in! Stay tuned as we zoom into the booths and bring the breakthroughs to you!🔥

Alibaba Group

161,611 Aufrufe • vor 11 Monaten

LongWriter Unleashing 10,000+ Word Generation from Long Context LLMs discuss: Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.

LongWriter Unleashing 10,000+ Word Generation from Long Context LLMs discuss: Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.

AK

50,995 Aufrufe • vor 2 Jahren

We trained our Large Sensor Model (LSM) on over 40 million hours of de-identified multimodal sensor data from 165K users to demonstrate how it could improve performance in wearable tasks like exercise and activity recognition. Here’s what we found →

We trained our Large Sensor Model (LSM) on over 40 million hours of de-identified multimodal sensor data from 165K users to demonstrate how it could improve performance in wearable tasks like exercise and activity recognition. Here’s what we found →

Google AI

63,666 Aufrufe • vor 1 Jahr

Microsoft presents Windows Agent Arena Evaluating Multi-Modal OS Agents at Scale discuss: Large language models (LLMs) show remarkable potential to act as computer agents, enhancing human productivity and software accessibility in multi-modal tasks that require planning and reasoning. However, measuring agent performance in realistic environments remains a challenge since: (i) most benchmarks are limited to specific modalities or domains (e.g. text-only, web navigation, Q&A, coding) and (ii) full benchmark evaluations are slow (on order of magnitude of days) given the multi-step sequential nature of tasks. To address these challenges, we introduce the Windows Agent Arena: a reproducible, general environment focusing exclusively on the Windows operating system (OS) where agents can operate freely within a real Windows OS and use the same wide range of applications, tools, and web browsers available to human users when solving tasks. We adapt the OSWorld framework (Xie et al., 2024) to create 150+ diverse Windows tasks across representative domains that require agent abilities in planning, screen understanding, and tool usage. Our benchmark is scalable and can be seamlessly parallelized in Azure for a full benchmark evaluation in as little as 20 minutes. To demonstrate Windows Agent Arena's capabilities, we also introduce a new multi-modal agent, Navi. Our agent achieves a success rate of 19.5% in the Windows domain, compared to 74.5% performance of an unassisted human. Navi also demonstrates strong performance on another popular web-based benchmark, Mind2Web. We offer extensive quantitative and qualitative analysis of Navi's performance, and provide insights into the opportunities for future research in agent development and data generation using Windows Agent Arena.

Microsoft presents Windows Agent Arena Evaluating Multi-Modal OS Agents at Scale discuss: Large language models (LLMs) show remarkable potential to act as computer agents, enhancing human productivity and software accessibility in multi-modal tasks that require planning and reasoning. However, measuring agent performance in realistic environments remains a challenge since: (i) most benchmarks are limited to specific modalities or domains (e.g. text-only, web navigation, Q&A, coding) and (ii) full benchmark evaluations are slow (on order of magnitude of days) given the multi-step sequential nature of tasks. To address these challenges, we introduce the Windows Agent Arena: a reproducible, general environment focusing exclusively on the Windows operating system (OS) where agents can operate freely within a real Windows OS and use the same wide range of applications, tools, and web browsers available to human users when solving tasks. We adapt the OSWorld framework (Xie et al., 2024) to create 150+ diverse Windows tasks across representative domains that require agent abilities in planning, screen understanding, and tool usage. Our benchmark is scalable and can be seamlessly parallelized in Azure for a full benchmark evaluation in as little as 20 minutes. To demonstrate Windows Agent Arena's capabilities, we also introduce a new multi-modal agent, Navi. Our agent achieves a success rate of 19.5% in the Windows domain, compared to 74.5% performance of an unassisted human. Navi also demonstrates strong performance on another popular web-based benchmark, Mind2Web. We offer extensive quantitative and qualitative analysis of Navi's performance, and provide insights into the opportunities for future research in agent development and data generation using Windows Agent Arena.

AK

19,684 Aufrufe • vor 1 Jahr

LayerAI Works with DeepSeek to Supercharge the LayerAI Product Suite 🌐 LayerAI integrates DeepSeek by leveraging several of its features and capabilities, focusing on areas where DeepSeek's strengths align with LayerAI's objectives. 👉 Learn more about our work with DeepSeek: - Model Deployment: DeepSeek-V3 & Coder-V2 for superior code generation, debugging, and language tasks. - DeepSeek-Coder-V2: Coding in 338 languages with a 128K context for complex structures. - Efficient Inference: Enhanced performance with DeepSeek's MLA & MoE tech. - IDE Assist: Real-time coding help, error catch, and auto-complete. - GitHub Sync: Better collab with seamless version control and reviews. This integration elevates our AI tools, making coding, learning, and teamwork smarter. More:

LayerAI Works with DeepSeek to Supercharge the LayerAI Product Suite 🌐 LayerAI integrates DeepSeek by leveraging several of its features and capabilities, focusing on areas where DeepSeek's strengths align with LayerAI's objectives. 👉 Learn more about our work with DeepSeek: - Model Deployment: DeepSeek-V3 & Coder-V2 for superior code generation, debugging, and language tasks. - DeepSeek-Coder-V2: Coding in 338 languages with a 128K context for complex structures. - Efficient Inference: Enhanced performance with DeepSeek's MLA & MoE tech. - IDE Assist: Real-time coding help, error catch, and auto-complete. - GitHub Sync: Better collab with seamless version control and reviews. This integration elevates our AI tools, making coding, learning, and teamwork smarter. More:

LayerAI | AI2Earn

47,905 Aufrufe • vor 1 Jahr

In our constant commitment to bring value to holders, we’re excited to announce the addition of image generation to our dApp on April 3rd Next week, as part of our journey towards decentralization, we are starting to integrate our open-source-based AI model into the Solana blockchain. This marks the beginning of our transition from relying on third-party APIs to utilizing fine-tuned open source AI Models integrated with Solana, enhancing our dApp’s security and autonomy.

In our constant commitment to bring value to holders, we’re excited to announce the addition of image generation to our dApp on April 3rd Next week, as part of our journey towards decentralization, we are starting to integrate our open-source-based AI model into the Solana blockchain. This marks the beginning of our transition from relying on third-party APIs to utilizing fine-tuned open source AI Models integrated with Solana, enhancing our dApp’s security and autonomy.

PrimeCircle Studio

17,451 Aufrufe • vor 2 Jahren

Announcing our commercial partnership with Booster Robotics Booster builds humanoid robot hardware, OS, and developer tools to make humanoid robots more affordable, reliable, and practical. The partnership centers on using simulation to multiply the value of real-robot data—expanding teleoperation demonstrations into scalable training data across diverse tasks and scenes. This joint effort powers sim-real co-training and foundation-model development, accelerating progress from hardware iteration to deployable robot policies. Together, we're building simulation‑powered data infrastructure for Physical AI — making scalable training data accessible to model developers and the broader robotics ecosystem.

Announcing our commercial partnership with Booster Robotics Booster builds humanoid robot hardware, OS, and developer tools to make humanoid robots more affordable, reliable, and practical. The partnership centers on using simulation to multiply the value of real-robot data—expanding teleoperation demonstrations into scalable training data across diverse tasks and scenes. This joint effort powers sim-real co-training and foundation-model development, accelerating progress from hardware iteration to deployable robot policies. Together, we're building simulation‑powered data infrastructure for Physical AI — making scalable training data accessible to model developers and the broader robotics ecosystem.

Axis Robotics

68,340 Aufrufe • vor 24 Tagen

We are pleased to announce the availability of Stable Video 4D, our very first video-to-video generation model that allows users to upload a single video and receive dynamic novel-view videos of eight new angles, delivering a new level of versatility and creativity. In conjunction with this announcement, we are releasing a comprehensive technical report detailing the methodologies, challenges, and breakthroughs achieved during the development of this model. Learn more about this release and access the report here:

We are pleased to announce the availability of Stable Video 4D, our very first video-to-video generation model that allows users to upload a single video and receive dynamic novel-view videos of eight new angles, delivering a new level of versatility and creativity. In conjunction with this announcement, we are releasing a comprehensive technical report detailing the methodologies, challenges, and breakthroughs achieved during the development of this model. Learn more about this release and access the report here:

Stability AI

131,173 Aufrufe • vor 2 Jahren

The final coming of our Utility Hub is here in the form of $PALM Agentverse. A full pack of agents and enhanced models running on the V3 Framework in a dApp designed to be the one-stop solution for all $PALM holders. Enhanced by our partners, built on $PALM. The next-generation virtual agentic platform. Incorporates Agentic SDKs, GenAI plugins for Unreal Engine, trading algorithms, video workflows and much more into one. Never run out of utility while deploying your own meaningful agents that incorporate into your day-to-day tasks, whether they be in DeFi or in creative tasks or anything in-between. Releasing on the 1st of June.

The final coming of our Utility Hub is here in the form of $PALM Agentverse. A full pack of agents and enhanced models running on the V3 Framework in a dApp designed to be the one-stop solution for all $PALM holders. Enhanced by our partners, built on $PALM. The next-generation virtual agentic platform. Incorporates Agentic SDKs, GenAI plugins for Unreal Engine, trading algorithms, video workflows and much more into one. Never run out of utility while deploying your own meaningful agents that incorporate into your day-to-day tasks, whether they be in DeFi or in creative tasks or anything in-between. Releasing on the 1st of June.

PaLM AI - $PALM

11,511 Aufrufe • vor 1 Jahr

💻 Agent mode just went local! Today, Mistral AI released Devstral, an Apache-2.0, 24B-parameter model trained for tool calling in real-world software development environments It reached #1 on SWE-bench for open-source models, but more importantly in our real-world testing it has proven to be capable of navigating and autonomously editing codebases, while running entirely on a laptop To try it out, just add `ollama/devstral` in Continue. See below for a quickstart with ollama👇

💻 Agent mode just went local! Today, Mistral AI released Devstral, an Apache-2.0, 24B-parameter model trained for tool calling in real-world software development environments It reached #1 on SWE-bench for open-source models, but more importantly in our real-world testing it has proven to be capable of navigating and autonomously editing codebases, while running entirely on a laptop To try it out, just add `ollama/devstral` in Continue. See below for a quickstart with ollama👇

Continue

39,217 Aufrufe • vor 1 Jahr

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,849 Aufrufe • vor 1 Jahr

Introducing Muse Image and Muse Video, the first media generation models developed by Meta Superintelligence Labs. Muse Image is our most advanced image generation model yet. It follows instructions faithfully, edits with precision, composes from multiple references, and draws on Instagram for social context. It also brings agentic tool use capabilities to image generation and integrates with Muse Spark. You can try Muse Image in the Meta AI app and web, as well as in Instagram Stories and WhatsApp – starting in limited countries with more locations on the way. Today we’re also previewing Muse Video, which is built upon the same pretraining base as Muse Image to deliver exceptional visual fidelity with native audio support. Learn more about both models:

Introducing Muse Image and Muse Video, the first media generation models developed by Meta Superintelligence Labs. Muse Image is our most advanced image generation model yet. It follows instructions faithfully, edits with precision, composes from multiple references, and draws on Instagram for social context. It also brings agentic tool use capabilities to image generation and integrates with Muse Spark. You can try Muse Image in the Meta AI app and web, as well as in Instagram Stories and WhatsApp – starting in limited countries with more locations on the way. Today we’re also previewing Muse Video, which is built upon the same pretraining base as Muse Image to deliver exceptional visual fidelity with native audio support. Learn more about both models:

AI at Meta

848,426 Aufrufe • vor 1 Monat

INTRODUCING MON PROTOCOL PARTNER — XSY XSY (liltuesyvert) is a frontier-yield platform leading the creation of the next generation of digital synthetic assets and derivatives. In addition to be Launchpool partners, we have also deployed a portion of our treasury into $UTY, XSY’s principal-protected, ~11 % APY synthetic dollar. Learn more:

INTRODUCING MON PROTOCOL PARTNER — XSY XSY (liltuesyvert) is a frontier-yield platform leading the creation of the next generation of digital synthetic assets and derivatives. In addition to be Launchpool partners, we have also deployed a portion of our treasury into $UTY, XSY’s principal-protected, ~11 % APY synthetic dollar. Learn more:

MON Protocol 🐉 $MON

60,758 Aufrufe • vor 1 Jahr

Google dropped a new AI paper called LUMIERE. It's remarkably flexible, supporting video inpainting, image-to-video, AND stylized video generation tasks. Say hello to “space-time diffusion” for video generation! Now what the heck does that mean exactly?! 🌐⏳ → TL;DR it utilizes a “Space-Time UNet” architecture that generates the full duration of the video in one pass, rather than generating distant keyframes and interpolating between them like prior works. Because the computation is done in this “compressed space-time representation” to generate the full clip at once, it's far more temporally consistent. → Another benefit of generating the full video at once is that you can “direct” the video generation, making it easier to hand off to other models/tasks without having to stitch together partial solutions. You can condition generations on additional inputs, meaning you get the full stack of AI video capabilities – from video inpainting to image-to-video and beyond. → New SOTA for AI video generation? User study results in the paper suggest human evaluators preferred Lumiere over Runway Gen-2, Pika Labs, and Stable Video Diffusion in terms of quality, text alignment AND motion. But as always, we need to get hands-on with this tech when Google *actually* decides to ship it. → Could this end up inside YouTube? Y’all know i’m obsessed with blending reality and imagination – so it’s the video inpainting tech I'm most excited about. I really hope this model finds its way into YouTube's Generative AI efforts, and based on their prior announcements and the list of acknowledgments in the paper I think it might! 🤞🏽 Links: 🔗Paper: 🔗Project:

Google dropped a new AI paper called LUMIERE. It's remarkably flexible, supporting video inpainting, image-to-video, AND stylized video generation tasks. Say hello to “space-time diffusion” for video generation! Now what the heck does that mean exactly?! 🌐⏳ → TL;DR it utilizes a “Space-Time UNet” architecture that generates the full duration of the video in one pass, rather than generating distant keyframes and interpolating between them like prior works. Because the computation is done in this “compressed space-time representation” to generate the full clip at once, it's far more temporally consistent. → Another benefit of generating the full video at once is that you can “direct” the video generation, making it easier to hand off to other models/tasks without having to stitch together partial solutions. You can condition generations on additional inputs, meaning you get the full stack of AI video capabilities – from video inpainting to image-to-video and beyond. → New SOTA for AI video generation? User study results in the paper suggest human evaluators preferred Lumiere over Runway Gen-2, Pika Labs, and Stable Video Diffusion in terms of quality, text alignment AND motion. But as always, we need to get hands-on with this tech when Google actually decides to ship it. → Could this end up inside YouTube? Y’all know i’m obsessed with blending reality and imagination – so it’s the video inpainting tech I'm most excited about. I really hope this model finds its way into YouTube's Generative AI efforts, and based on their prior announcements and the list of acknowledgments in the paper I think it might! 🤞🏽 Links: 🔗Paper: 🔗Project:

Bilawal Sidhu

44,822 Aufrufe • vor 2 Jahren

We took a 30B model and split it in two to write tokens in parallel instead of one at a time. Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch. We found it kept 98.7% of the original model’s quality at 2.42× faster generation.

We took a 30B model and split it in two to write tokens in parallel instead of one at a time. Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch. We found it kept 98.7% of the original model’s quality at 2.42× faster generation.

NVIDIA AI

761,561 Aufrufe • vor 2 Monaten