Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

VideoLLaMA3, latest MLLMs for image and video understanding. 🖐️ 7B models: DocVQA: 94.9, MathVision: 26.2, VideoMME: 66.2/70.3, MLVU: 73.0 🤏 2B models for edge devices: MMMU: 45.3, VideoMME: 59.6/63.4 👊 Frontier-class video model with ONLY 3M video-text pairs

Gradio

56,759 subscribers

37,154 views • 1 year ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

7 Comments

Gradio1 year ago

VideoLLaMA3 for Video understanding:

Gradio1 year ago

VideoLLaMA3 for Image understanding:

Gradio1 year ago

Learn more about VideoLLaMA3:

Rainmaker2 years ago

Join me as I put several Machine Learning models head-to-head to see which one can beat the market and deliver strong returns. In this free Substack post I share several models that deliver better returns with much lower drawdown compared to Buy-and-Hold approach.

Discord Guys1 year ago

Wow, VideoLLaMA3 sounds like a beast! Can't wait to try it out.

Sujith Reddy1 year ago

@_akhaliq @Gradio are these models production level models? Scalable?

ヌル1 year ago

3M pairs is a small dataset for training.

Related Videos

🤯DiffIR2VR-Zero: Zero-shot video restoration to high-resolution using pre-trained image restoration diffusion models. - Framework can handle video denoising and up to 8x super-resolution - outperforms trained models in generalizing across diverse datasets and extreme degradations - Compatible with every 2D restoration model

🤯DiffIR2VR-Zero: Zero-shot video restoration to high-resolution using pre-trained image restoration diffusion models. - Framework can handle video denoising and up to 8x super-resolution - outperforms trained models in generalizing across diverse datasets and extreme degradations - Compatible with every 2D restoration model

Gradio

20,342 views • 2 years ago

LTX 2.3 is out. Another big milestone for open-source video generation. Lightricks keeps pushing the frontier with faster, higher-quality video models the community can actually build with. Open video models are accelerating fast. Model:

LTX 2.3 is out. Another big milestone for open-source video generation. Lightricks keeps pushing the frontier with faster, higher-quality video models the community can actually build with. Open video models are accelerating fast. Model:

Hugging Models

35,935 views • 4 months ago

Struggling with slow inference of diffusion and flow models? Check out the video below—I’ve been using our new FastGen library to achieve 7-28x acceleration for text-2-image and {text,image,video}-2-video generation without sacrificing visual fidelity!

Struggling with slow inference of diffusion and flow models? Check out the video below—I’ve been using our new FastGen library to achieve 7-28x acceleration for text-2-image and {text,image,video}-2-video generation without sacrificing visual fidelity!

Julius Berner

13,643 views • 4 months ago

You’ve used language models, image models, video models, and voice models. Now it’s time for world models, thanks to World Labs.

You’ve used language models, image models, video models, and voice models. Now it’s time for world models, thanks to World Labs.

Amir Haghighat

838,889 views • 4 months ago

Meet Reka Edge – Our next-generation vision language model for physical AI. Uses 3x fewer input tokens and achieves 65% faster throughput compared to leading 8B models. Image understanding, video analysis, object detection, and tool use. Built for Action. Fast enough for production, deployable anywhere. Read more:

Meet Reka Edge – Our next-generation vision language model for physical AI. Uses 3x fewer input tokens and achieves 65% faster throughput compared to leading 8B models. Image understanding, video analysis, object detection, and tool use. Built for Action. Fast enough for production, deployable anywhere. Read more:

Reka

53,500 views • 4 months ago

Nano-Banana moment for Video models soon? In an interview two weeks ago, Demis Hassabis talked about the future of image and video models "When Video converges with Language models you will see some interesting combinaton of Capabilites"

Nano-Banana moment for Video models soon? In an interview two weeks ago, Demis Hassabis talked about the future of image and video models "When Video converges with Language models you will see some interesting combinaton of Capabilites"

AiBattle

47,820 views • 6 months ago

InfinityStar by Bytedance: A unified 8B spacetime autoregressive model for high-res image & video gen; - 5s 720p video ~10x faster than DiT; - scores 83.74 on VBench, topping other AR models and HunyuanVideo; - Flan-T5-XL as text encoder. - 480/720p, ~35Gb model

InfinityStar by Bytedance: A unified 8B spacetime autoregressive model for high-res image & video gen; - 5s 720p video ~10x faster than DiT; - scores 83.74 on VBench, topping other AR models and HunyuanVideo; - Flan-T5-XL as text encoder. - 480/720p, ~35Gb model

Wildminder

23,255 views • 8 months ago

Hello World, we’re We eval frontier models by putting them in simulations. So what happens when 6 frontier models compete in #Minecraft for GPUs? Video and 🧵

Hello World, we’re We eval frontier models by putting them in simulations. So what happens when 6 frontier models compete in #Minecraft for GPUs? Video and 🧵

Kradle

94,625 views • 10 months ago

Can a small academic team build a strong text-to-image model using only public datasets? Introducing i1: a simple, fully open recipe for strong text-to-image models

Can a small academic team build a strong text-to-image model using only public datasets? Introducing i1: a simple, fully open recipe for strong text-to-image models

Zhuang Liu

67,476 views • 17 days ago

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:

Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models turn the publicly available, state-of-the-art text-to-image LDM Stable Diffusion into an efficient and expressive text-to-video model with resolution up to 1280 x 2048 abs: project page:

AK

718,760 views • 3 years ago

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:

(1/2) LightIt: Illumination Modeling and Control for Diffusion Models! #CVPR2024 We facilitate lighting control for novel image generation from text prompts. We can also edit lighting for a given input image. Video: Project:

Matthias Niessner

19,849 views • 2 years ago

New Kling, Runway, Luma competitor? Open text and image video generation model Pyramidal Flow Matching for Efficient Video Generative Modeling

New Kling, Runway, Luma competitor? Open text and image video generation model Pyramidal Flow Matching for Efficient Video Generative Modeling

AK

90,965 views • 1 year ago

Demis Hassabis on the next 12 Months: - Full multimodal convergence: Models like Gemini will seamlessly take in and output text, images, audio, and video, with cross-pollination that boosts reasoning + creativity. - Breakthrough visual intelligence: Image models like Nano Banana Pro will produce highly accurate infographics and show near-human visual understanding. - Language + video fusion: Video models integrated with LLMs unlock richer analysis, storytelling, and step-by-step visual reasoning. - World models go mainstream like Genie 3 - Agents become reliable

Demis Hassabis on the next 12 Months: - Full multimodal convergence: Models like Gemini will seamlessly take in and output text, images, audio, and video, with cross-pollination that boosts reasoning + creativity. - Breakthrough visual intelligence: Image models like Nano Banana Pro will produce highly accurate infographics and show near-human visual understanding. - Language + video fusion: Video models integrated with LLMs unlock richer analysis, storytelling, and step-by-step visual reasoning. - World models go mainstream like Genie 3 - Agents become reliable

Chubby♨️

89,236 views • 7 months ago

Apple FastVLM-7B Efficient Vision Encoding for Vision Language Models larger variants using Qwen2-7B LLM outperform recent works like Cambrian-1-8B while using a single image encoder with a 7.9x faster TTFT vibe coding a video captioning app with it in anycoder

Apple FastVLM-7B Efficient Vision Encoding for Vision Language Models larger variants using Qwen2-7B LLM outperform recent works like Cambrian-1-8B while using a single image encoder with a 7.9x faster TTFT vibe coding a video captioning app with it in anycoder

AK

60,588 views • 10 months ago

𝗔𝗴𝗻𝗲𝘀 𝟮.𝟬: AI models ranked among the world's best. One suite for text, image, and video generation. Top 10 across leading AI benchmarks. Trained in Singapore and available now with no waitlist.

𝗔𝗴𝗻𝗲𝘀 𝟮.𝟬: AI models ranked among the world's best. One suite for text, image, and video generation. Top 10 across leading AI benchmarks. Trained in Singapore and available now with no waitlist.

Tech in Asia

4,765,968 views • 1 month ago

Ex-DeepMind Danijar Hafner: The debate about whether LLMs can reach AGI is outdated because we are already past that stage Frontier models are no longer just text engines; they now master image understanding, video processing, and generation "there's no debate even about that anymore"

Ex-DeepMind Danijar Hafner: The debate about whether LLMs can reach AGI is outdated because we are already past that stage Frontier models are no longer just text engines; they now master image understanding, video processing, and generation "there's no debate even about that anymore"

Haider.

22,431 views • 5 months ago

- if SAM3 is so good, do we still need other models? - yes. SAM3 is too large for low compute edge devices and too slow for real-time use. we still need light and fast models for edge workloads. distill SAM3 into smaller model and deploy it in seconds with Roboflow rapid

- if SAM3 is so good, do we still need other models? - yes. SAM3 is too large for low compute edge devices and too slow for real-time use. we still need light and fast models for edge workloads. distill SAM3 into smaller model and deploy it in seconds with Roboflow rapid

SkalskiP

93,025 views • 7 months ago