正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

VITA-1.5 Towards GPT-4o Level Real-Time Vision and Speech Interaction

AK

438,985 subscribers

31,482 次观看 • 1 年前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

5 条评论

AK 的头像

AK1 年前

discuss:

AssemblyAI 的头像

AssemblyAI1 年前

Announcing: Our most advanced speech-to-text model goes beyond accuracy to capture the real-world complexity of human conversation and deliver reliable, source-of-truth audio data. Explore Universal-2 updates 👇

Cohorte 的头像

Cohorte1 年前

Curious how multimodal models like VITA-1.5 are reshaping AI? Discover how AI combines vision and language to power next-gen interactions:

Svebbi 的头像

Svebbi1 年前

@artificialguybr Amazing 🤩

Mingyu | Wapoo - Interactive Video Creation 的头像

Mingyu | Wapoo - Interactive Video Creation1 年前

your ideas are like a cat meme—always on point!

相关视频

BREAKING: ChatGPT GPT-4o was just announce by OpenAI. It improves on vision, audio and text. The ease of use is incredibly enhanced. It makes interaction with the GPT much more natural, especially with voice. GPT-4o reasons across voice, text and vision. GPT-4 wil be available to everyone.

BREAKING: ChatGPT GPT-4o was just announce by OpenAI. It improves on vision, audio and text. The ease of use is incredibly enhanced. It makes interaction with the GPT much more natural, especially with voice. GPT-4o reasons across voice, text and vision. GPT-4 wil be available to everyone.

Ed Krassenstein

21,605 次观看 • 2 年前

🚨 MiniCPM-o 2.6, an 8B multimodal LLM, outperforms GPT-4o, Gemini 1.5 Pro, and Sonnet in visual (70.2 avg on OpenCompass, 1.8M pixel - 1344 x 1344), speech (bilingual realtime, beats GPT-4o-realtime in ASR/STT), with 75% fewer vision tokens, supporting 30+ languages 🔥

🚨 MiniCPM-o 2.6, an 8B multimodal LLM, outperforms GPT-4o, Gemini 1.5 Pro, and Sonnet in visual (70.2 avg on OpenCompass, 1.8M pixel - 1344 x 1344), speech (bilingual realtime, beats GPT-4o-realtime in ASR/STT), with 75% fewer vision tokens, supporting 30+ languages 🔥

Vaibhav (VB) Srivastav

42,009 次观看 • 1 年前

Introducing GPT-4o, our new model which can reason across text, audio, and video in real time. It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction):

Introducing GPT-4o, our new model which can reason across text, audio, and video in real time. It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction):

Greg Brockman

4,360,353 次观看 • 2 年前

Is GPT-4o the only tool that can reason across audio, vision and text in real time?… No! So can projects powered $KARRAT

Is GPT-4o the only tool that can reason across audio, vision and text in real time?… No! So can projects powered $KARRAT

Karrat

58,910 次观看 • 2 年前

💥 Introducing MiniCPM-o 2.6: An 8B size, GPT-4o level Omni Model runs on device ✨ Highlights: ~Match GPT-4o-202405 in vision, audio and multimodal live streaming ~End-to-end real-time bilingual audio conversation ~Voice cloning & emotion control ~Advanced OCR & video understanding ~Offline iPad-compatible multimodal live streaming 🔗 Try it out: GitHub: HF: Demo:

💥 Introducing MiniCPM-o 2.6: An 8B size, GPT-4o level Omni Model runs on device ✨ Highlights: ~Match GPT-4o-202405 in vision, audio and multimodal live streaming ~End-to-end real-time bilingual audio conversation ~Voice cloning & emotion control ~Advanced OCR & video understanding ~Offline iPad-compatible multimodal live streaming 🔗 Try it out: GitHub: HF: Demo:

OpenBMB

97,663 次观看 • 1 年前

OpenAI’s new GPT-4o model can translate in real-time

OpenAI’s new GPT-4o model can translate in real-time

Dexerto

409,605 次观看 • 2 年前

Live audience request for GPT-4o vision capabilities

Live audience request for GPT-4o vision capabilities

OpenAI

628,932 次观看 • 2 年前

Live demo of GPT-4o realtime conversational speech

Live demo of GPT-4o realtime conversational speech

OpenAI

607,881 次观看 • 2 年前

OpenAI demos real-time language translation with its latest GPT-4o model.

OpenAI demos real-time language translation with its latest GPT-4o model.

TechCrunch

7,885,807 次观看 • 2 年前

GPT-4o level intelligence running on your phone! MiniCPM-V 4.5 delivers enterprise-grade AI performance in just 8B parameters, outperforming models like GPT-4o, Gemini-2.0 Pro on vision and language tasks. - 30+ language support - Runs smoothly on iPhone/iPad 100% open-source!

GPT-4o level intelligence running on your phone! MiniCPM-V 4.5 delivers enterprise-grade AI performance in just 8B parameters, outperforming models like GPT-4o, Gemini-2.0 Pro on vision and language tasks. - 30+ language support - Runs smoothly on iPhone/iPad 100% open-source!

Akshay 🚀

84,288 次观看 • 10 个月前

OpenAI has just demonstrated its new GPT-4o model doing real-time translations 🤯

OpenAI has just demonstrated its new GPT-4o model doing real-time translations 🤯

Tom Warren

8,427,790 次观看 • 2 年前

Say hello to GPT-4o, our new flagship model which can reason across audio, vision, and text in real time: Text and image input rolling out today in API and ChatGPT with voice and video in the coming weeks.

Say hello to GPT-4o, our new flagship model which can reason across audio, vision, and text in real time: Text and image input rolling out today in API and ChatGPT with voice and video in the coming weeks.

OpenAI

22,810,150 次观看 • 2 年前

🚀Introducing MiniCPM-V 2.6! 🔥 1、Surpassing GPT-4V in single image, multi-image and video understanding 📸🎥 2、Outperforms GPT-4o mini and Gemini 1.5 on OpenCompass 🏆 3、Real-time video analysis on iPad 📱💨 Try out the best on-device multimodal LLM here！ 👑 GitHub： Huggingface： #MLLM #MiniCPM

🚀Introducing MiniCPM-V 2.6! 🔥 1、Surpassing GPT-4V in single image, multi-image and video understanding 📸🎥 2、Outperforms GPT-4o mini and Gemini 1.5 on OpenCompass 🏆 3、Real-time video analysis on iPad 📱💨 Try out the best on-device multimodal LLM here！ 👑 GitHub： Huggingface： #MLLM #MiniCPM

OpenBMB

196,343 次观看 • 1 年前

Lullabies and whispers with GPT-4o

Lullabies and whispers with GPT-4o

OpenAI

529,238 次观看 • 2 年前

Point and learn Spanish with GPT-4o

Point and learn Spanish with GPT-4o

OpenAI

476,332 次观看 • 2 年前

Did you hear? OpenAI's newest model can reason across audio, vision, and text in real time. How does GPT-4o do with math tutoring?🤔 Salman Khan and his son test it on a Khan Academy math problem. You can get AI-powered math tutoring right now with Khanmigo:

Did you hear? OpenAI's newest model can reason across audio, vision, and text in real time. How does GPT-4o do with math tutoring?🤔 Salman Khan and his son test it on a Khan Academy math problem. You can get AI-powered math tutoring right now with Khanmigo:

Khan Academy

283,766 次观看 • 2 年前

Excited to introduce GPT-4o. Language, vision, and sound -- all together and all in real time. This thing has been so much fun to work on. It's been even more fun to play with -- with moments of magic where things feel totally fluid and I forget I'm video chatting with an AI.

Excited to introduce GPT-4o. Language, vision, and sound -- all together and all in real time. This thing has been so much fun to work on. It's been even more fun to play with -- with moments of magic where things feel totally fluid and I forget I'm video chatting with an AI.

Rowan Zellers

128,683 次观看 • 2 年前

OpenAI’s newly released GPT-4o can not only describe what it sees, such as clothes and facial expressions, but it also can communicate with another GPT-4o

OpenAI’s newly released GPT-4o can not only describe what it sees, such as clothes and facial expressions, but it also can communicate with another GPT-4o

TechCrunch

28,854 次观看 • 2 年前

Humans draw to facilitate reasoning and communication. Why not let LLMs do so? 🚀We introduce✏️Sketchpad, which gives multimodal LLMs a sketchpad to draw and facilitate reasoning! Sketchpad gives GPT-4o great boosts on many vision and math tasks 📈 The video shows how GPT-4o with Sketchpad reasons with interleaved visual and textual steps. For more, visit our project page: 📌 For math tasks, ✏️Sketchpad allows LLMs to draw auxiliary lines on geometry diagrams, plotting functions, graphs, and even games. GPT-4o does math better when it can sketch! (+12.7% acc on average) 📌 For computer vision tasks, ✏️Sketchpad allows LLMs to sketch with vision specialists (e.g., GroundingDINO draws bounding boxes, SegmentAnything draws masks). Sketchpad substantially improves GPT-4o's vision abilities. GPT-4o + Sketchpad compared with prior SOTAs: 1️⃣ V*Bench: 75.4% -> 80.3% 2️⃣ BLINK correspondence: 42.4% -> 80.8% 3️⃣ BLINK relative depth: 67.7% -> 83.9% 4️⃣ BLINK spatial relation: 76.2% -> 81.1% ... See more interesting examples in the thread!

Humans draw to facilitate reasoning and communication. Why not let LLMs do so? 🚀We introduce✏️Sketchpad, which gives multimodal LLMs a sketchpad to draw and facilitate reasoning! Sketchpad gives GPT-4o great boosts on many vision and math tasks 📈 The video shows how GPT-4o with Sketchpad reasons with interleaved visual and textual steps. For more, visit our project page: 📌 For math tasks, ✏️Sketchpad allows LLMs to draw auxiliary lines on geometry diagrams, plotting functions, graphs, and even games. GPT-4o does math better when it can sketch! (+12.7% acc on average) 📌 For computer vision tasks, ✏️Sketchpad allows LLMs to sketch with vision specialists (e.g., GroundingDINO draws bounding boxes, SegmentAnything draws masks). Sketchpad substantially improves GPT-4o's vision abilities. GPT-4o + Sketchpad compared with prior SOTAs: 1️⃣ V*Bench: 75.4% -> 80.3% 2️⃣ BLINK correspondence: 42.4% -> 80.8% 3️⃣ BLINK relative depth: 67.7% -> 83.9% 4️⃣ BLINK spatial relation: 76.2% -> 81.1% ... See more interesting examples in the thread!

Yushi Hu

145,048 次观看 • 2 年前