Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

(1/n) Time to unify your favorite visual generative models, VLMs, and simulators for controllable visual generation—Introducing a Product of Experts (PoE) framework for inference-time knowledge composition from heterogeneous models.

zhang_yunzhi's profile picture

Yunzhi Zhang

2,736 subscribers

49,090 görüntüleme • 1 yıl önce •via X (Twitter)

Bilim & TeknolojiEğitimSanat
LIVE

Anya Rossi• Live Now

Private livecam show

6 Yorum

Yunzhi Zhang profil fotoğrafı
Yunzhi Zhang1 yıl önce

(2/n) The composition yields better controllability and provides flexible user interfaces for specifying visual synthesis goals, enabling applications such as composing physics simulation into generated videos…

Yunzhi Zhang profil fotoğrafı
Yunzhi Zhang1 yıl önce

(3/n) …inserting graphics engine rendering into images, and more.

Yunzhi Zhang profil fotoğrafı
Yunzhi Zhang1 yıl önce

(4/n) PoE sampling is non-trivial in high dimensions. We adopt Annealed Importance Sampling, where particles are initially drawn from a simple base distribution and steered towards the target, with transition kernels computed from expert models.. Two possible annealing paths:

Yunzhi Zhang profil fotoğrafı
Yunzhi Zhang1 yıl önce

(5/5) Page: More details in paper: Team work with the incredible Carson Murtuza-Lanier, @zizhang_li, @du_yilun, and @jiajunwu_cs!

Rainmaker profil fotoğrafı
Rainmaker2 yıl önce

Join me as I put several Machine Learning models head-to-head to see which one can beat the market and deliver strong returns. In this free Substack post I share several models that deliver better returns with much lower drawdown compared to Buy-and-Hold approach.

Aisha profil fotoğrafı
Aisha1 yıl önce

Whats the best visual video generative models in your experience ?

Benzer Videolar

Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing
1:11

Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing

AK

19,381 görüntüleme • 1 yıl önce

1/ 🚀 Introducing AIDO.StructureDiffusion: A generative model for structural protein design—enabling high-quality, controllable generation of monomers, complexes, and antibodies. 🧵
0:33

1/ 🚀 Introducing AIDO.StructureDiffusion: A generative model for structural protein design—enabling high-quality, controllable generation of monomers, complexes, and antibodies. 🧵

GenBio AI

918,241 görüntüleme • 1 yıl önce

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n
1:11

👀Humans compare images by looking back and forth. Many open-weight VLMs encode each image independently, and defer comparison to the LM. We introduce SVE: Stateful Visual Encoders for Vision-Language Models, where the visual encoder itself becomes change-aware. 🌐Project: 📰Paper: 💻Code: 1/n

Zirui "Colin" Wang

63,158 görüntüleme • 2 ay önce

Accurate and controllable scene generation has been difficult with natural language alone. You instead need a language for scenes. Introducing the Scene Language — a visual representation for high-quality 3D/4D generation by integrating programs, words, and embeddings — 🧵(1/6)
0:31

Accurate and controllable scene generation has been difficult with natural language alone. You instead need a language for scenes. Introducing the Scene Language — a visual representation for high-quality 3D/4D generation by integrating programs, words, and embeddings — 🧵(1/6)

Yunzhi Zhang

63,956 görüntüleme • 1 yıl önce

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇
0:48

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

Fangchen Liu

68,366 görüntüleme • 1 yıl önce

Introducing Ray2, a new frontier in video generative models. Scaled to 10x compute, #Ray2 creates realistic videos with natural and coherent motion, unlocking new freedoms of creative expression and visual storytelling. Available now. Learn more
1:02

Introducing Ray2, a new frontier in video generative models. Scaled to 10x compute, #Ray2 creates realistic videos with natural and coherent motion, unlocking new freedoms of creative expression and visual storytelling. Available now. Learn more

Luma

4,156,411 görüntüleme • 1 yıl önce

Can video generative models exhibit visuospatial intelligence? 🤔 Introducing Video4Spatial — a video-only framework that tackles spatial tasks. With just video context, our model can: 🔍 Ground objects by planning geometry-consistent paths 📸 Follow camera-pose instructions for scene navigation 🌐 Generalize to long contexts & unseen outdoor scenes A step toward video models as visual-spatial reasoners. Project: arXiv:
0:25

Can video generative models exhibit visuospatial intelligence? 🤔 Introducing Video4Spatial — a video-only framework that tackles spatial tasks. With just video context, our model can: 🔍 Ground objects by planning geometry-consistent paths 📸 Follow camera-pose instructions for scene navigation 🌐 Generalize to long contexts & unseen outdoor scenes A step toward video models as visual-spatial reasoners. Project: arXiv:

Xingang Pan

15,946 görüntüleme • 8 ay önce

Struggling with slow inference of diffusion and flow models? Check out the video below—I’ve been using our new FastGen library to achieve 7-28x acceleration for text-2-image and {text,image,video}-2-video generation without sacrificing visual fidelity!
2:20

Struggling with slow inference of diffusion and flow models? Check out the video below—I’ve been using our new FastGen library to achieve 7-28x acceleration for text-2-image and {text,image,video}-2-video generation without sacrificing visual fidelity!

Julius Berner

13,729 görüntüleme • 6 ay önce

World models are moving beyond offline generation towards interactive, real-time experiences. Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models: 🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds. Github: Docs: Research page: Join the #flashdreams Discord channel at FlashDreams is also the runtime backbone behind NVIDIA OmniDreams ( 1/n #AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA
0:46

World models are moving beyond offline generation towards interactive, real-time experiences. Introducing ⚡FlashDreams⚡: an open-source high-performance inference and serving library built for autoregressive world models: 🔥 Up to 3.10× faster LingBot-World inference 🔥 Up to 2.12× faster Self-Forcing inference 🔥 Up to 1.40× faster Wan2.1 inference 🔥 8 integrated models 🔥 Multi-GPU, streaming, low-latency serving 🔥 Agentic skills that teach you how to use it FlashDreams is designed for a new generation of AI systems that continuously evolve over time while responding to user interactions. It powers applications across robotics, autonomous vehicle simulation, gaming, and virtual worlds. Github: Docs: Research page: Join the #flashdreams Discord channel at FlashDreams is also the runtime backbone behind NVIDIA OmniDreams ( 1/n #AI #WorldModels #FastInference #PhysicalAI #OpenSource #NVIDIA

Ruilong Li

92,744 görüntüleme • 2 ay önce

The next generation of models won't just generate images - they'll understand worlds, motion, interaction, and action. We've been building toward this for a while. Visual intelligence is becoming real-time. Stephen spoke about where we're headed at AI Engineer:
2:12

The next generation of models won't just generate images - they'll understand worlds, motion, interaction, and action. We've been building toward this for a while. Visual intelligence is becoming real-time. Stephen spoke about where we're headed at AI Engineer:

Black Forest Labs

33,180 görüntüleme • 3 ay önce

Don't waste time choosing between models Introducing Multiple Model Generation: → Test up to 4 models at once → Same prompt. Same settings. Side by side → Find what works best for you, faster Time to pick the diamond of the season
0:21

Don't waste time choosing between models Introducing Multiple Model Generation: → Test up to 4 models at once → Same prompt. Same settings. Side by side → Find what works best for you, faster Time to pick the diamond of the season

Magnific

34,374 görüntüleme • 6 ay önce

Cerebras Inference powers the world's top models - like Qwen3 Coder at 2000 tokens/s 🤯 And you can now access these models directly in Visual Studio Code with the extension and an API key (that you can get for free from
0:22

Cerebras Inference powers the world's top models - like Qwen3 Coder at 2000 tokens/s 🤯 And you can now access these models directly in Visual Studio Code with the extension and an API key (that you can get for free from

Visual Studio Code

57,360 görüntüleme • 11 ay önce

Action-controllable world models are evolving fast. LingBot-World-Infinity can run an unbroken session of real-time world generation for up to an hour. A distilled model can stream 720p at 60 fps. World models are going to be a critical piece in training robotic agents for unseen worlds.
1:33

Action-controllable world models are evolving fast. LingBot-World-Infinity can run an unbroken session of real-time world generation for up to an hour. A distilled model can stream 720p at 60 fps. World models are going to be a critical piece in training robotic agents for unseen worlds.

The Humanoid Hub

14,007 görüntüleme • 1 ay önce

Nvidia presents Align Your Steps Optimizing Sampling Schedules in Diffusion Models Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed,
0:34

Nvidia presents Align Your Steps Optimizing Sampling Schedules in Diffusion Models Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed,

AK

32,888 görüntüleme • 2 yıl önce

Introducing Modality Forcing, a recipe for post-training T2I models for SOTA RGB-Depth generation! Text-to-image (T2I) models learn rich representations of the spatial world. How do we build on this prior for high-quality depth generation? 🧵 [1/6]
0:25

Introducing Modality Forcing, a recipe for post-training T2I models for SOTA RGB-Depth generation! Text-to-image (T2I) models learn rich representations of the spatial world. How do we build on this prior for high-quality depth generation? 🧵 [1/6]

Bardienus Duisterhof

65,326 görüntüleme • 2 ay önce

Our research team just released Flex-Forcing: a video generation method that lets a single model switch between generation methods at inference time. Right now there are two main approaches to video generation. Bidirectional diffusion models attend to every frame at once, holding structure well at the cost of speed. Autoregressive models generate frame by frame, so they stream fast and scale to long clips, but accumulate error and drift over time. Flex-Forcing trains a single model to do both, letting you choose from the range at inference based on your compute budget.
0:22

Our research team just released Flex-Forcing: a video generation method that lets a single model switch between generation methods at inference time. Right now there are two main approaches to video generation. Bidirectional diffusion models attend to every frame at once, holding structure well at the cost of speed. Autoregressive models generate frame by frame, so they stream fast and scale to long clips, but accumulate error and drift over time. Flex-Forcing trains a single model to do both, letting you choose from the range at inference based on your compute budget.

NVIDIA AI

31,412 görüntüleme • 1 ay önce

Elements for Image to Video generation gets a major upgrade — big improvements in subject consistency, dynamics, prompt adherence, and visual rendering quality! Time to build your scene with Elements!
0:45

Elements for Image to Video generation gets a major upgrade — big improvements in subject consistency, dynamics, prompt adherence, and visual rendering quality! Time to build your scene with Elements!

Kling AI

2,542,815 görüntüleme • 1 yıl önce

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵
2:39

New research from BlackForestLabsAI - Unofficial 🥳 Meet Self-Flow: our self-supervised framework for image, audio, video & world models 🤖 Do generative models really need DINO to learn strong representations? We propose teaching them directly via a joint framework instead 🧵

Hila Chefer

68,791 görüntüleme • 5 ay önce