Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Most video models 🤯forget the past 🐌slow down over time 🔁rely on bidirectional (not causal) attention Our state-space video world models (SSM) 🧠remember across hundreds of frames ⚡️generate at constant speed ⏩is fully causal, enabling real-time rollout 1/3

GordonWetzstein's profile picture

Gordon Wetzstein

4,920 subscribers

20,015 Aufrufe • vor 1 Jahr •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

The context size of video world models is only a few frames. Like a human with severe memory loss! We design a long-term memory for world models based on explicit 3D representations inspired by the human mind. This enables long-term consistency. 1/3
0:12

The context size of video world models is only a few frames. Like a human with severe memory loss! We design a long-term memory for world models based on explicit 3D representations inspired by the human mind. This enables long-term consistency. 1/3

Gordon Wetzstein

34,796 Aufrufe • vor 1 Jahr

(1/n) 🚀 With FastVideo, you can now generate a 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan series, a family of fast video generation models trained via a new recipe we term as “sparse distillation”, to speed up video denoising time by 70X! 🖥️ Live demo: (Thanks to @gmicloud for the support!) 🔗 Blog: 🔓 We fully open-source our models, code, and data with Apache-2.0 licenses
0:13

(1/n) 🚀 With FastVideo, you can now generate a 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan series, a family of fast video generation models trained via a new recipe we term as “sparse distillation”, to speed up video denoising time by 70X! 🖥️ Live demo: (Thanks to @gmicloud for the support!) 🔗 Blog: 🔓 We fully open-source our models, code, and data with Apache-2.0 licenses

Hao AI Lab

78,660 Aufrufe • vor 11 Monaten

NVIDIA's full-stack AI is enabling autonomous operations for the world's leading brands and platforms. Key capabilities driving this shift: 📈 Causal marketing analytics at enterprise scale 🤖 Secure, trustworthy agentic workflows ⚡Real-time, hyper-efficient auction bidding 🎬 Frame-level contextual video intelligence
0:15

NVIDIA's full-stack AI is enabling autonomous operations for the world's leading brands and platforms. Key capabilities driving this shift: 📈 Causal marketing analytics at enterprise scale 🤖 Secure, trustworthy agentic workflows ⚡Real-time, hyper-efficient auction bidding 🎬 Frame-level contextual video intelligence

NVIDIA

58,105 Aufrufe • vor 15 Tagen

Announcing our $320M Series A at a $2.3B valuation, led by Khosla Ventures, with General Catalyst, Eric Schmidt and Jeff Bezos. General Intuition is the frontier lab for acting in space and time. We build large action foundation models trained on billions of ground truth action-labeled gameplay clips from 17M monthly active users on Medal, and push the frontier of world models to generate infinite training environments.
0:18

Announcing our $320M Series A at a $2.3B valuation, led by Khosla Ventures, with General Catalyst, Eric Schmidt and Jeff Bezos. General Intuition is the frontier lab for acting in space and time. We build large action foundation models trained on billions of ground truth action-labeled gameplay clips from 17M monthly active users on Medal, and push the frontier of world models to generate infinite training environments.

General Intuition

515,516 Aufrufe • vor 14 Tagen

Video generation is powerful but too slow for real-world robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)
0:13

Video generation is powerful but too slow for real-world robotic tasks. How can we enable both video and action generation while ensuring real-time policy inference? Check out our work on the Unified Video Action Model (UVA) to find out! (1/7)

Shuang Li

67,404 Aufrufe • vor 1 Jahr

NVIDIA just released a very impressive text-to-video paper. Video Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:
0:14

NVIDIA just released a very impressive text-to-video paper. Video Latent Diffusion Models (Video LDMs) use a diffusion model in a compressed latent space to generate high-resolution videos. Here's a brief overview of how it works: 1. Pre-train image LDM on a dataset of images. 2. Turn the image LDM into a Video LDM by adding temporal layers to model video frames. 3. Fine-tune the Video LDM on encoded video sequences to create a video generator. 4. Temporally align diffusion model upsamplers to generate high-resolution videos. 5. Validate Video LDM on real driving videos of 512x1024 resolution, achieving state-of-the-art performance. 6. Apply the approach in creative content creation with text-to-video modeling. Paper: Project:

Lior Alexander

158,553 Aufrufe • vor 3 Jahren

Introducing Find the best local models based on real data 1. People run and submit benchmarks 2. Stats aggregated over models / devices 3. Find the best model for you `npx whatcanirun`, fully open-source
0:16

Introducing Find the best local models based on real data 1. People run and submit benchmarks 2. Stats aggregated over models / devices 3. Find the best model for you `npx whatcanirun`, fully open-source

⁵⁄₉

133,401 Aufrufe • vor 3 Monaten

🤩Video world models are cool, but it is cooler if they can simulate any 3D physical actions in real time! Introducing RealWonder⚡️: Now you can simulate 3D physical action (robot actions, 3D forces, force fields, etc.) consequences from a single image in real time! 🧵1/6
0:10

🤩Video world models are cool, but it is cooler if they can simulate any 3D physical actions in real time! Introducing RealWonder⚡️: Now you can simulate 3D physical action (robot actions, 3D forces, force fields, etc.) consequences from a single image in real time! 🧵1/6

Hong-Xing (Koven) Yu

29,756 Aufrufe • vor 4 Monaten

New AI video model MotionStream lets you control motion in real time. This is a glimpse at what all video models will allow you to do in the future.
0:19

New AI video model MotionStream lets you control motion in real time. This is a glimpse at what all video models will allow you to do in the future.

Dan Scalco

28,162 Aufrufe • vor 8 Monaten

Matrix-Game 2.0 — The FIRST open-source, real-time, long-sequence interactive world model Last week, DeepMind's Genie 3 shook the AI world with real-time interactive world models. But... it wasn't open-sourced. Today, Matrix-Game 2.0 changed the game. 🚀 25FPS. Minutes-long interaction. Fully open-source.
0:10

Matrix-Game 2.0 — The FIRST open-source, real-time, long-sequence interactive world model Last week, DeepMind's Genie 3 shook the AI world with real-time interactive world models. But... it wasn't open-sourced. Today, Matrix-Game 2.0 changed the game. 🚀 25FPS. Minutes-long interaction. Fully open-source.

Skywork

1,398,817 Aufrufe • vor 11 Monaten

The team at Runway is pushing the frontier of what's possible in world models. We're proud to work with them to power real-time inference for Runway Characters.
0:15

The team at Runway is pushing the frontier of what's possible in world models. We're proud to work with them to power real-time inference for Runway Characters.

Modal

68,896 Aufrufe • vor 3 Monaten

Every search, recommendation, simulation and AI-generated response depends on rapid access to #data.​​ From training large AI models to enabling real-time inference, #DRAM helps keep today's most demanding workloads moving at the speed innovation requires.​​ This National DRAM day, join us in celebrating the technology that helps bridge the gap between processors and data, enabling everything from smartphones and PCs to #AI infrastructure.
0:12

Every search, recommendation, simulation and AI-generated response depends on rapid access to #data.​​ From training large AI models to enabling real-time inference, #DRAM helps keep today's most demanding workloads moving at the speed innovation requires.​​ This National DRAM day, join us in celebrating the technology that helps bridge the gap between processors and data, enabling everything from smartphones and PCs to #AI infrastructure.

Micron Technology

15,826 Aufrufe • vor 1 Monat

Video understanding is the next frontier, but not all videos are alike. Models now reason over youtube clips and feature films, but what about the everyday spaces we—and our future AI assistants—navigate and experience? Introducing Thinking in Space, our latest study exploring how multimodal LLMs see, remember and recall spaces. 🧵[1/n]
0:11

Video understanding is the next frontier, but not all videos are alike. Models now reason over youtube clips and feature films, but what about the everyday spaces we—and our future AI assistants—navigate and experience? Introducing Thinking in Space, our latest study exploring how multimodal LLMs see, remember and recall spaces. 🧵[1/n]

Saining Xie

209,460 Aufrufe • vor 1 Jahr

This video is in real time 🤯
0:14

This video is in real time 🤯

Tevin

24,891 Aufrufe • vor 1 Jahr

🤔Applying Depth Estimation models directly to videos can result in inconsistency between frames. 💪Well, not anymore. 🔥ChronoDepth is a new approach to video depth estimation that focuses on achieving both accuracy within each frame and consistency across frames. 📜More👇
0:15

🤔Applying Depth Estimation models directly to videos can result in inconsistency between frames. 💪Well, not anymore. 🔥ChronoDepth is a new approach to video depth estimation that focuses on achieving both accuracy within each frame and consistency across frames. 📜More👇

Gradio

19,986 Aufrufe • vor 2 Jahren

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.
0:17

Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation paper page: Large text-to-image diffusion models have exhibited impressive proficiency in generating high-quality images. However, when applying these models to video domain, ensuring temporal consistency across video frames remains a formidable challenge. This paper proposes a novel zero-shot text-guided video-to-video translation framework to adapt image models to videos. The framework includes two parts: key frame translation and full video translation. The first part uses an adapted diffusion model to generate key frames, with hierarchical cross-frame constraints applied to enforce coherence in shapes, textures and colors. The second part propagates the key frames to other frames with temporal-aware patch matching and frame blending. Our framework achieves global style and local texture temporal consistency at a low cost (without re-training or optimization). The adaptation is compatible with existing image diffusion techniques, allowing our framework to take advantage of them, such as customizing a specific subject with LoRA, and introducing extra spatial guidance with ControlNet. Extensive experimental results demonstrate the effectiveness of our proposed framework over existing methods in rendering high-quality and temporally-coherent videos.

AK

375,090 Aufrufe • vor 3 Jahren

We are open-sourcing TTSKit! Run state-of-the-art text-to-speech models on your Mac and iPhone. The launch version supports Qwen Qwen3-TTS and generates audio faster than real-time playback with sub-200 ms time-to-first-byte. Voice cloning and advanced speed optimizations will be in the next version. Link to the GitHub repo and models on Hugging Face in comments.
0:16

We are open-sourcing TTSKit! Run state-of-the-art text-to-speech models on your Mac and iPhone. The launch version supports Qwen Qwen3-TTS and generates audio faster than real-time playback with sub-200 ms time-to-first-byte. Voice cloning and advanced speed optimizations will be in the next version. Link to the GitHub repo and models on Hugging Face in comments.

argmax

61,998 Aufrufe • vor 4 Monaten

Grok ranks #1 across many major AI coding platforms, with multi‑trillion token usage and the biggest lead over rival models in real‑world developer traffic
0:10

Grok ranks #1 across many major AI coding platforms, with multi‑trillion token usage and the biggest lead over rival models in real‑world developer traffic

X Freeze

427,925 Aufrufe • vor 7 Monaten

Our models run real time on a laptop 5090 at 100+ FPS. We're so excited for people to start playing with it.
0:10

Our models run real time on a laptop 5090 at 100+ FPS. We're so excited for people to start playing with it.

Louis Castricato @ lovecraftian horrors

137,685 Aufrufe • vor 10 Monaten