Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🚀Introducing LLaVA-NeXT Interleave: Now AI can understand and reason with multiple images at once - This opens up multi-image scenarios like multi-frame videos, multi-view 3D, and multiple inter-leaved images. - An all round LMM that can understand videos, images, and 3D More⬇️

Gradio's profile picture

Gradio

56,347 subscribers

27,655 Aufrufe • vor 2 Jahren •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

8 Kommentare

Profilbild von Gradio
Gradiovor 2 Jahren

LLaVA-NeXT-Interleave🔥 - Interleave data format unifies different tasks. - New datasets on 🤗Hub: 1️⃣M4-Instruct, high-quality dataset, 1.1M samples from domains: multi-image, video, 3D & single-image 2️⃣LLaVA-Interleave Bench - Set of tasks to evaluate multi-image capabilities

Profilbild von Gradio
Gradiovor 2 Jahren

LLaVA-NeXT-Interleave💪 - Attached videos show how it can explain jokes and understand content spread in multiple images and videos 🤯 - SoTA Performance, both, in multi and single images - Matches in perf with LLaVA-NeXT - Improved performance in video tasks

Profilbild von Gradio
Gradiovor 2 Jahren

Gradio Multimodal Demo for LLaVA-NeXT-Interleave😍 : Models and Datasets are on 🤗 Hub:

Profilbild von Stark
Starkvor 2 Jahren

how to finetune？

Profilbild von Omri Kaduri
Omri Kadurivor 2 Jahren

How can you refer to the order of the images in the prompt? Simply saying "first image" is enough? Like -"is the object in the first image shown in the second image"

Profilbild von Lily Zhang
Lily Zhangvor 2 Jahren

How does it understand 3D?

Profilbild von Gradio
Gradiovor 2 Jahren

Different views as multiple image input

Profilbild von Gradio
Gradiovor 2 Jahren

Love this! 💡

Ähnliche Videos

🔥 New Feature: Multi-Image to 3D! Now you can upload multiple images of the same object to generate a more complete and accurate 3D model. 😉 More improvements coming soon! Try it out and let us know what you think. #3dmodeling #ai
0:40

🔥 New Feature: Multi-Image to 3D! Now you can upload multiple images of the same object to generate a more complete and accurate 3D model. 😉 More improvements coming soon! Try it out and let us know what you think. #3dmodeling #ai

MeshyAI

59,314 Aufrufe • vor 1 Jahr

Introducing Stable Virtual Camera: This multi-view diffusion model transforms 2D images into immersive 3D videos with realistic depth and perspective—without complex reconstruction or scene-specific optimization.
0:39

Introducing Stable Virtual Camera: This multi-view diffusion model transforms 2D images into immersive 3D videos with realistic depth and perspective—without complex reconstruction or scene-specific optimization.

Stability AI

197,338 Aufrufe • vor 1 Jahr

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.
0:37

Multi-modal #LLMs understand a lot about humans. But do they understand our 3D pose? We train #PoseGPT to estimate, generate, and reason about 3D human pose (#SMPL) in images and text. This is the first true foundation model for understanding 3D humans.

Michael Black

81,406 Aufrufe • vor 2 Jahren

Multi-view images to 3D with Tripo V2.0 and HD texture is updated on webapp and API platform👇 More user experience improvements:
0:40

Multi-view images to 3D with Tripo V2.0 and HD texture is updated on webapp and API platform👇 More user experience improvements:

Tripo

11,826 Aufrufe • vor 1 Jahr

You can now create long form AI Videos. Introducing Multi-Frame by Dreamina AI! Upload up to 10 images to be stitched together for a longer AI video output. You can control the duration of each transition and guide with a text prompt. Tutorial in the comments 🔽
0:20

You can now create long form AI Videos. Introducing Multi-Frame by Dreamina AI! Upload up to 10 images to be stitched together for a longer AI video output. You can control the duration of each transition and guide with a text prompt. Tutorial in the comments 🔽

Jerrod Lew

31,762 Aufrufe • vor 8 Monaten

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.
0:44

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

AK

294,442 Aufrufe • vor 2 Jahren

We’ve just launched multi-reference in LTX Studio. You can now seamlessly integrate and composite objects, scenes, and characters from different images into a single frame with more control and precision. Now live.
0:21

We’ve just launched multi-reference in LTX Studio. You can now seamlessly integrate and composite objects, scenes, and characters from different images into a single frame with more control and precision. Now live.

LTX Studio

1,521,262 Aufrufe • vor 11 Monaten

This is amazing! You can now create high-quality 3D Scenes from a single image using Multi-Instance Diffusion Models (MIDI) 🔥
0:43

This is amazing! You can now create high-quality 3D Scenes from a single image using Multi-Instance Diffusion Models (MIDI) 🔥

Gradio

41,770 Aufrufe • vor 1 Jahr

[1/N] Rotary Position Embeddings (RoPE) are ubiquitous across transformers that process tokens from 1D, 2D, or 3D grids e.g. language, images, or videos. Our RayRoPE formulation extends these to multi-view transformers. Paper and code:
0:41

[1/N] Rotary Position Embeddings (RoPE) are ubiquitous across transformers that process tokens from 1D, 2D, or 3D grids e.g. language, images, or videos. Our RayRoPE formulation extends these to multi-view transformers. Paper and code:

Shubham Tulsiani

72,323 Aufrufe • vor 5 Monaten

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors Aleksander Holynski, Ben Poole and an amazing team!
1:21

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors Aleksander Holynski, Ben Poole and an amazing team!

Ruiqi Gao

152,867 Aufrufe • vor 2 Jahren

No more waiting for one image to finish, you can now prompt edit multiple images at once in Figma! Keep working and stay in your flow as they generate and load in.
0:27

No more waiting for one image to finish, you can now prompt edit multiple images at once in Figma! Keep working and stay in your flow as they generate and load in.

miggi from figgi

14,175 Aufrufe • vor 10 Tagen

Introducing Claude for Creators. You can now create images, music, and 3D models from Claude Code or any agent you use. Connect once. Your agent gets them all.
0:57

Introducing Claude for Creators. You can now create images, music, and 3D models from Claude Code or any agent you use. Connect once. Your agent gets them all.

Shengkun

49,085 Aufrufe • vor 21 Tagen

Introducing Wan2.6 - A native multimodal model that turns your ideas into breathtaking videos and images! · Starring: Cast characters from reference videos into new scenes. Support human or human-like figures, enabling complex multi-person and human-object interactions with appearance and voice consistency. · Intelligent Multi-shot Narrative: Turn simple prompts into auto-storyboarded, multi-shot videos. Maintain visual consistency and upgrade storytelling from single shots to rich narratives. · Native A/V Sync: Generate multi-speaker dialogue with natural lip-sync and studio-quality audio. It doesn’t just look real - it sounds real. · Cinematic Quality: 15s 1080p HD generation with comprehensive upgrades to instruction adherence, motion physics, and aesthetic control. · Advanced Image Synthesis and Editing: Deliver cinematic photorealism with precise control over lens and lighting. Support multi-image referencing for commercial-grade consistency and faithful aesthetic transfer. · Storytelling with Structure: Generate interleaved texts and images powered by real-world knowledge and reasoning capabilities, enabling hierarchical and structured visual narratives.
0:38

Introducing Wan2.6 - A native multimodal model that turns your ideas into breathtaking videos and images! · Starring: Cast characters from reference videos into new scenes. Support human or human-like figures, enabling complex multi-person and human-object interactions with appearance and voice consistency. · Intelligent Multi-shot Narrative: Turn simple prompts into auto-storyboarded, multi-shot videos. Maintain visual consistency and upgrade storytelling from single shots to rich narratives. · Native A/V Sync: Generate multi-speaker dialogue with natural lip-sync and studio-quality audio. It doesn’t just look real - it sounds real. · Cinematic Quality: 15s 1080p HD generation with comprehensive upgrades to instruction adherence, motion physics, and aesthetic control. · Advanced Image Synthesis and Editing: Deliver cinematic photorealism with precise control over lens and lighting. Support multi-image referencing for commercial-grade consistency and faithful aesthetic transfer. · Storytelling with Structure: Generate interleaved texts and images powered by real-world knowledge and reasoning capabilities, enabling hierarchical and structured visual narratives.

Wan

3,847,418 Aufrufe • vor 7 Monaten

Text3D - a nice little gradio app that generates images that can be used to generate 3D artefacts downstream.
0:21

Text3D - a nice little gradio app that generates images that can be used to generate 3D artefacts downstream.

Gradio

42,472 Aufrufe • vor 1 Jahr

Introducing Wan 2.5 🚀 Create immersive AI videos with fluid motion, built-in audio and voice. Supporting text-to-video and start images Available now for all users
0:23

Introducing Wan 2.5 🚀 Create immersive AI videos with fluid motion, built-in audio and voice. Supporting text-to-video and start images Available now for all users

Magnific

170,229 Aufrufe • vor 9 Monaten

In today's release of TanStack AI we bring you: 🚀 Tool calling improvements 🛠️ Multi-modality support 🗂️ Send things like images, videos, audio, documents and more from your client to your server! Check out the demo below 👇
0:29

In today's release of TanStack AI we bring you: 🚀 Tool calling improvements 🛠️ Multi-modality support 🗂️ Send things like images, videos, audio, documents and more from your client to your server! Check out the demo below 👇

TANSTACK

15,095 Aufrufe • vor 5 Monaten