Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🚀Introducing LLaVA-NeXT Interleave: Now AI can understand and reason with multiple images at once - This opens up multi-image scenarios like multi-frame videos, multi-view 3D, and multiple inter-leaved images. - An all round LMM that can understand videos, images, and 3D More⬇️

Gradio

57,061 subscribers

27,665 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

8 Comments

Gradio2 years ago

LLaVA-NeXT-Interleave🔥 - Interleave data format unifies different tasks. - New datasets on 🤗Hub: 1️⃣M4-Instruct, high-quality dataset, 1.1M samples from domains: multi-image, video, 3D & single-image 2️⃣LLaVA-Interleave Bench - Set of tasks to evaluate multi-image capabilities

Gradio2 years ago

LLaVA-NeXT-Interleave💪 - Attached videos show how it can explain jokes and understand content spread in multiple images and videos 🤯 - SoTA Performance, both, in multi and single images - Matches in perf with LLaVA-NeXT - Improved performance in video tasks

Gradio2 years ago

Gradio Multimodal Demo for LLaVA-NeXT-Interleave😍 : Models and Datasets are on 🤗 Hub:

Stark2 years ago

how to finetune？

Omri Kaduri2 years ago

How can you refer to the order of the images in the prompt? Simply saying "first image" is enough? Like -"is the object in the first image shown in the second image"

Lily Zhang2 years ago

How does it understand 3D?

Gradio2 years ago

Different views as multiple image input

Gradio2 years ago

Love this! 💡

Related Videos

🔥 New Feature: Multi-Image to 3D! Now you can upload multiple images of the same object to generate a more complete and accurate 3D model. 😉 More improvements coming soon! Try it out and let us know what you think. #3dmodeling #ai

🔥 New Feature: Multi-Image to 3D! Now you can upload multiple images of the same object to generate a more complete and accurate 3D model. 😉 More improvements coming soon! Try it out and let us know what you think. #3dmodeling #ai

MeshyAI

59,314 views • 1 year ago

Introducing Stable Virtual Camera: This multi-view diffusion model transforms 2D images into immersive 3D videos with realistic depth and perspective—without complex reconstruction or scene-specific optimization.

Introducing Stable Virtual Camera: This multi-view diffusion model transforms 2D images into immersive 3D videos with realistic depth and perspective—without complex reconstruction or scene-specific optimization.

Stability AI

197,338 views • 1 year ago

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Michael Black

81,430 views • 2 years ago

Multi-view images to 3D with Tripo V2.0 and HD texture is updated on webapp and API platform👇 More user experience improvements:

Multi-view images to 3D with Tripo V2.0 and HD texture is updated on webapp and API platform👇 More user experience improvements:

Tripo

11,826 views • 1 year ago

You can now create long form AI Videos. Introducing Multi-Frame by Dreamina AI! Upload up to 10 images to be stitched together for a longer AI video output. You can control the duration of each transition and guide with a text prompt. Tutorial in the comments 🔽

You can now create long form AI Videos. Introducing Multi-Frame by Dreamina AI! Upload up to 10 images to be stitched together for a longer AI video output. You can control the duration of each transition and guide with a text prompt. Tutorial in the comments 🔽

Jerrod Lew

31,762 views • 8 months ago

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

AK

294,496 views • 2 years ago

We’ve just launched multi-reference in LTX Studio. You can now seamlessly integrate and composite objects, scenes, and characters from different images into a single frame with more control and precision. Now live.

We’ve just launched multi-reference in LTX Studio. You can now seamlessly integrate and composite objects, scenes, and characters from different images into a single frame with more control and precision. Now live.

LTX.io

1,521,321 views • 1 year ago

This is amazing! You can now create high-quality 3D Scenes from a single image using Multi-Instance Diffusion Models (MIDI) 🔥

This is amazing! You can now create high-quality 3D Scenes from a single image using Multi-Instance Diffusion Models (MIDI) 🔥

Gradio

41,770 views • 1 year ago

[1/N] Rotary Position Embeddings (RoPE) are ubiquitous across transformers that process tokens from 1D, 2D, or 3D grids e.g. language, images, or videos. Our RayRoPE formulation extends these to multi-view transformers. Paper and code:

[1/N] Rotary Position Embeddings (RoPE) are ubiquitous across transformers that process tokens from 1D, 2D, or 3D grids e.g. language, images, or videos. Our RayRoPE formulation extends these to multi-view transformers. Paper and code:

Shubham Tulsiani

73,064 views • 6 months ago

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors Aleksander Holynski, Ben Poole and an amazing team!

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors Aleksander Holynski, Ben Poole and an amazing team!

Ruiqi Gao

153,223 views • 2 years ago

No more waiting for one image to finish, you can now prompt edit multiple images at once in Figma! Keep working and stay in your flow as they generate and load in.

No more waiting for one image to finish, you can now prompt edit multiple images at once in Figma! Keep working and stay in your flow as they generate and load in.

miggi from figgi

14,374 views • 1 month ago

Introducing Claude for Creators. You can now create images, music, and 3D models from Claude Code or any agent you use. Connect once. Your agent gets them all.

Introducing Claude for Creators. You can now create images, music, and 3D models from Claude Code or any agent you use. Connect once. Your agent gets them all.

Shengkun

49,085 views • 1 month ago

Introducing Wan2.6 - A native multimodal model that turns your ideas into breathtaking videos and images! · Starring: Cast characters from reference videos into new scenes. Support human or human-like figures, enabling complex multi-person and human-object interactions with appearance and voice consistency. · Intelligent Multi-shot Narrative: Turn simple prompts into auto-storyboarded, multi-shot videos. Maintain visual consistency and upgrade storytelling from single shots to rich narratives. · Native A/V Sync: Generate multi-speaker dialogue with natural lip-sync and studio-quality audio. It doesn’t just look real - it sounds real. · Cinematic Quality: 15s 1080p HD generation with comprehensive upgrades to instruction adherence, motion physics, and aesthetic control. · Advanced Image Synthesis and Editing: Deliver cinematic photorealism with precise control over lens and lighting. Support multi-image referencing for commercial-grade consistency and faithful aesthetic transfer. · Storytelling with Structure: Generate interleaved texts and images powered by real-world knowledge and reasoning capabilities, enabling hierarchical and structured visual narratives.

Introducing Wan2.6 - A native multimodal model that turns your ideas into breathtaking videos and images! · Starring: Cast characters from reference videos into new scenes. Support human or human-like figures, enabling complex multi-person and human-object interactions with appearance and voice consistency. · Intelligent Multi-shot Narrative: Turn simple prompts into auto-storyboarded, multi-shot videos. Maintain visual consistency and upgrade storytelling from single shots to rich narratives. · Native A/V Sync: Generate multi-speaker dialogue with natural lip-sync and studio-quality audio. It doesn’t just look real - it sounds real. · Cinematic Quality: 15s 1080p HD generation with comprehensive upgrades to instruction adherence, motion physics, and aesthetic control. · Advanced Image Synthesis and Editing: Deliver cinematic photorealism with precise control over lens and lighting. Support multi-image referencing for commercial-grade consistency and faithful aesthetic transfer. · Storytelling with Structure: Generate interleaved texts and images powered by real-world knowledge and reasoning capabilities, enabling hierarchical and structured visual narratives.

Wan

3,847,733 views • 8 months ago

Text3D - a nice little gradio app that generates images that can be used to generate 3D artefacts downstream.

Text3D - a nice little gradio app that generates images that can be used to generate 3D artefacts downstream.

Gradio

42,475 views • 1 year ago

Introducing Wan 2.5 🚀 Create immersive AI videos with fluid motion, built-in audio and voice. Supporting text-to-video and start images Available now for all users

Introducing Wan 2.5 🚀 Create immersive AI videos with fluid motion, built-in audio and voice. Supporting text-to-video and start images Available now for all users

Magnific

170,229 views • 10 months ago

One prompt. Multiple gaming scenes 🎮 The Nano Banana agent can generate several images at once, using a reference image to keep the same person consistent across scenes. This example spans different gaming eras, but the same approach works for any theme or style. Images and videos, all in one place. Agent + prompt 👇

One prompt. Multiple gaming scenes 🎮 The Nano Banana agent can generate several images at once, using a reference image to keep the same person consistent across scenes. This example spans different gaming eras, but the same approach works for any theme or style. Images and videos, all in one place. Agent + prompt 👇

GLIF

10,882 views • 6 months ago