Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

MiniCPM-V 4.5 achieves an average score of 77.0 on OpenCompass, a comprehensive evaluation of 8 popular benchmarks. With only 8B parameters, it surpasses widely used proprietary models like GPT-4o-latest, Gemini-2.0 Pro, and strong open-source models like Qwen2.5-VL 72B powered by a new unified 3D-Resampler over images and videos, MiniCPM-V... show more

AK

505,718 subscribers

19,088 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

🚀 Introducing MiniCPM-V 4.5 8B: pushing the boundary of multimodal AI! ～ SOTA VL Capability: Surpasses GPT-4o, Gemini 2.0 Pro, Qwen2.5-VL 72B on OpenCompass! ～ "Eagle Eye" Video: 96x visual token compression for high refresh rate and long video understanding ～ Controllable Hybrid Fast/Deep Thinking ～ Strong OCR & Doc Parsing: Surpasses GPT-4o & Gemini 2.5 on OmniDocBench Get ready for the future of multimodal AI 👉 Huggingface｜ Github｜ Gradio｜ #AI #MiniCM #GPT #Gemini #OpenBMB #ArtificialIntelligence #MachineLearning

🚀 Introducing MiniCPM-V 4.5 8B: pushing the boundary of multimodal AI! ～ SOTA VL Capability: Surpasses GPT-4o, Gemini 2.0 Pro, Qwen2.5-VL 72B on OpenCompass! ～ "Eagle Eye" Video: 96x visual token compression for high refresh rate and long video understanding ～ Controllable Hybrid Fast/Deep Thinking ～ Strong OCR & Doc Parsing: Surpasses GPT-4o & Gemini 2.5 on OmniDocBench Get ready for the future of multimodal AI 👉 Huggingface｜ Github｜ Gradio｜ #AI #MiniCM #GPT #Gemini #OpenBMB #ArtificialIntelligence #MachineLearning

OpenBMB

25,223 views • 1 year ago

New open Omni model released! 👀OpenBMB MiniCPM-o 2.6 is a new 8B parameters, any-to-any multimodal model that can understand vision, speech, and language and runs on edge devices like phones and tablets. TL;DR: 🧠 8B total parameters (SigLip-400M + Whisper-300M + ChatTTS-200M + Qwen2.5-7B) 🔥 Outperforms GPT-4V on visual tasks with 70.2 average score on OpenCompass 🎙️ Best-in-class bilingual speech capabilities with real-time conversation and voice cloning 🎬 Supports multimodal streaming with support for continuous video/audio processing 📱 Runs on iPads and phones and supports 30+ languages 🖼️ Processes images up to 1.8M pixels (1344x1344) with OCR capabilities 🛠️ Easy integration with popular frameworks (llama.cpp, vLLM, Gradio) 🤗 commercial friendly (< 1 mio DAU) and available on Hugging Face

New open Omni model released! 👀OpenBMB MiniCPM-o 2.6 is a new 8B parameters, any-to-any multimodal model that can understand vision, speech, and language and runs on edge devices like phones and tablets. TL;DR: 🧠 8B total parameters (SigLip-400M + Whisper-300M + ChatTTS-200M + Qwen2.5-7B) 🔥 Outperforms GPT-4V on visual tasks with 70.2 average score on OpenCompass 🎙️ Best-in-class bilingual speech capabilities with real-time conversation and voice cloning 🎬 Supports multimodal streaming with support for continuous video/audio processing 📱 Runs on iPads and phones and supports 30+ languages 🖼️ Processes images up to 1.8M pixels (1344x1344) with OCR capabilities 🛠️ Easy integration with popular frameworks (llama.cpp, vLLM, Gradio) 🤗 commercial friendly (< 1 mio DAU) and available on Hugging Face

Philipp Schmid

19,700 views • 1 year ago

MiniCPM-o 4.5: Seeing, Listening, and Speaking — All at Once. 👁️👂🗣️ ✨Beyond traditional turn-taking, we’ve built a Native Full-Duplex engine that allows a 9B model to see, listen, and speak in one concurrent, non-blocking stream. Watch how it masters real-world complexity in real-time: 🔔 Proactive Auditory Interaction: Interrupts itself to alert you when it hears a "Ding!" while reading cards. 🎨 Temporal Flow Tracking: Follows your pen in real-time, narrating and "mind-reading" your drawing as you sketch. 🍎 Omni-Perception: Scans groceries & identifies prices on the fly. ✨Why it’s a category-leader: 📌Performance: Surpasses GPT-4o and Gemini 2.0 Pro on OpenCompass (Avg. 77.6). 📌Architecture: End-to-end fusion of SigLip2, Whisper, and CosyVoice2 on a Qwen3-8B base. 📌Efficiency: Full-duplex live streaming now runs locally on PCs via llama.cpp-omni. The era of "Wait-and-Response" AI is over. Proactive, real-time intelligence is now open-source. 🚀Experience it on Hugging Face: 🔗 #MiniCPM #Omnimodal #FullDuplex #EdgeAI #OpenSource #ComputerVision

MiniCPM-o 4.5: Seeing, Listening, and Speaking — All at Once. 👁️👂🗣️ ✨Beyond traditional turn-taking, we’ve built a Native Full-Duplex engine that allows a 9B model to see, listen, and speak in one concurrent, non-blocking stream. Watch how it masters real-world complexity in real-time: 🔔 Proactive Auditory Interaction: Interrupts itself to alert you when it hears a "Ding!" while reading cards. 🎨 Temporal Flow Tracking: Follows your pen in real-time, narrating and "mind-reading" your drawing as you sketch. 🍎 Omni-Perception: Scans groceries & identifies prices on the fly. ✨Why it’s a category-leader: 📌Performance: Surpasses GPT-4o and Gemini 2.0 Pro on OpenCompass (Avg. 77.6). 📌Architecture: End-to-end fusion of SigLip2, Whisper, and CosyVoice2 on a Qwen3-8B base. 📌Efficiency: Full-duplex live streaming now runs locally on PCs via llama.cpp-omni. The era of "Wait-and-Response" AI is over. Proactive, real-time intelligence is now open-source. 🚀Experience it on Hugging Face: 🔗 #MiniCPM #Omnimodal #FullDuplex #EdgeAI #OpenSource #ComputerVision

OpenBMB

115,537 views • 8 months ago

🚀 🚀Excited to announce the technical report of MiniCPM-o 4.5! MiniCPM-o 4.5 transitions #AI interaction from traditional turn-based processing to a real-time, native full-duplex stream-based paradigm. 🌊 The Omni-Flow Framework Instead of traditional VAD-based workarounds, we introduce the #Omni-#Flow framework. This unified stream paradigm aligns video, audio, and text on a synchronized millisecond timeline. • Native Full-Duplex: Simultaneous perception and response. • Proactive Interaction: Natively manages turn-taking without external VAD, supports proactive reminding. 📉 9B Scale, SOTA Performance MiniCPM-o 4.5 demonstrates SOTA multimodal intelligence at its scale: • Multimodal Benchmarks: Comparable to #Gemini 2.5 Flash on MMBench EN (87.6) and MathVista (80.1). • Streaming Evaluation: 54.4% win rate on LiveSports-3K-CC, surpassing specialized models. 💻 The Ultimate Edge AI — Fully Functional without Network Connection We are providing one-click installers for Windows (12G VRAM,RTX 5070) and macOS (M1-M5 Max/ M5 Pro). • Local API Support: Deploy your own inference server to integrate native full-duplex into custom apps. • Free Access: We are offering free community API services for exploration. • 100% Private: Your data never leaves your machine. Deploy in under 10 minutes. 🛠️👇 👐 Join the Open Future The weights are open. The protocol is public. 📄 Technical Report: 💻 GitHub: 🤗 HuggingFace: 🌐 Web Demo: #MiniCPMo #OpenSourceAI #EdgeAI #MachineLearning #ComputerVision #LLM

🚀 🚀Excited to announce the technical report of MiniCPM-o 4.5! MiniCPM-o 4.5 transitions #AI interaction from traditional turn-based processing to a real-time, native full-duplex stream-based paradigm. 🌊 The Omni-Flow Framework Instead of traditional VAD-based workarounds, we introduce the #Omni-#Flow framework. This unified stream paradigm aligns video, audio, and text on a synchronized millisecond timeline. • Native Full-Duplex: Simultaneous perception and response. • Proactive Interaction: Natively manages turn-taking without external VAD, supports proactive reminding. 📉 9B Scale, SOTA Performance MiniCPM-o 4.5 demonstrates SOTA multimodal intelligence at its scale: • Multimodal Benchmarks: Comparable to #Gemini 2.5 Flash on MMBench EN (87.6) and MathVista (80.1). • Streaming Evaluation: 54.4% win rate on LiveSports-3K-CC, surpassing specialized models. 💻 The Ultimate Edge AI — Fully Functional without Network Connection We are providing one-click installers for Windows (12G VRAM,RTX 5070) and macOS (M1-M5 Max/ M5 Pro). • Local API Support: Deploy your own inference server to integrate native full-duplex into custom apps. • Free Access: We are offering free community API services for exploration. • 100% Private: Your data never leaves your machine. Deploy in under 10 minutes. 🛠️👇 👐 Join the Open Future The weights are open. The protocol is public. 📄 Technical Report: 💻 GitHub: 🤗 HuggingFace: 🌐 Web Demo: #MiniCPMo #OpenSourceAI #EdgeAI #MachineLearning #ComputerVision #LLM

OpenBMB

148,094 views • 5 months ago

BREAKING NEWS: Anthropic just dropped Claude Ops 4.5!! It is by FAR the best coding model I've ever used. We've been testing it internally Every 📧 for the last few days, and it is an absolute paradigm shift for any kind of coding task. It extends the horizon of what you can vibe code The current generation of new models—Anthropic’s Sonnet 4.5, Google’s Gemini 3, or OpenAI’s Codex Max 5.1—can all competently build a minimum viable product in one shot, or fix a highly technical bug autonomously. But eventually, if you kept pushing them to vibe code more, they’d start to trip over their own feet: The code would be convoluted and contradictory, and you’d get stuck in endless bugs. We have not found that limit yet with Opus 4.5—it seems to be able to vibe code forever. Takes working in parallel to a whole new level because it's far better at planning and coding, it can work with more autonomy—meaning you can do more in parallel without breaking anything . Kieran Klaassen worked on 11 different projects in six hours—and had good results on all of them. Great at design iteration Opus 4.5 is incredibly skilled at iterating through a design autonomously using an MCP like Playwright. previous models would lose the thread after a few cycles, or say a design was done when it wasn't. Opus 4.5 is incredible at autonomously iterating until a design is pixel perfect. we have a full 4,000 word vibe check on Every 📧 right now with everything we tested:

BREAKING NEWS: Anthropic just dropped Claude Ops 4.5!! It is by FAR the best coding model I've ever used. We've been testing it internally Every 📧 for the last few days, and it is an absolute paradigm shift for any kind of coding task. It extends the horizon of what you can vibe code The current generation of new models—Anthropic’s Sonnet 4.5, Google’s Gemini 3, or OpenAI’s Codex Max 5.1—can all competently build a minimum viable product in one shot, or fix a highly technical bug autonomously. But eventually, if you kept pushing them to vibe code more, they’d start to trip over their own feet: The code would be convoluted and contradictory, and you’d get stuck in endless bugs. We have not found that limit yet with Opus 4.5—it seems to be able to vibe code forever. Takes working in parallel to a whole new level because it's far better at planning and coding, it can work with more autonomy—meaning you can do more in parallel without breaking anything . Kieran Klaassen worked on 11 different projects in six hours—and had good results on all of them. Great at design iteration Opus 4.5 is incredibly skilled at iterating through a design autonomously using an MCP like Playwright. previous models would lose the thread after a few cycles, or say a design was done when it wasn't. Opus 4.5 is incredible at autonomously iterating until a design is pixel perfect. we have a full 4,000 word vibe check on Every 📧 right now with everything we tested:

Dan Shipper 📧

272,699 views • 10 months ago

Codex (ChatGPT) + ChatCut is a new level of AI-assisted video editing. The plugin gives GPT 5.6 a full NLE it can drive natively: - Talking head editing from transcripts - Timeline editing on real tracks - Stunning, consistent, editable motion graphics - Video generation with Seedance 2.0, plus music, voiceover, and sound effects Editing costs nothing beyond your own GPT tokens. ChatCut credits only kick in for generative assets like Seedance. More details below👇

Codex (ChatGPT) + ChatCut is a new level of AI-assisted video editing. The plugin gives GPT 5.6 a full NLE it can drive natively: - Talking head editing from transcripts - Timeline editing on real tracks - Stunning, consistent, editable motion graphics - Video generation with Seedance 2.0, plus music, voiceover, and sound effects Editing costs nothing beyond your own GPT tokens. ChatCut credits only kick in for generative assets like Seedance. More details below👇

ChatCut

288,487 views • 3 months ago

AI app builders are officially obsolete. Stop building static sites. Start building AI-Native Apps. Enter Pro doesn’t just code a page; it builds apps with 'Brains', integrating Claude, GPT, and Gemini directly into your workflow via AI All. designed for build ai native apps or agent One sentence in. A real, emphasis on "ai native app", enter can build ai app or agent, with llm integrated. including the latest image and video models like nano banana, seedance 2.0(ready to launch), veo3.1 etc. Auth. Database. Payments. Handled.

AI app builders are officially obsolete. Stop building static sites. Start building AI-Native Apps. Enter Pro doesn’t just code a page; it builds apps with 'Brains', integrating Claude, GPT, and Gemini directly into your workflow via AI All. designed for build ai native apps or agent One sentence in. A real, emphasis on "ai native app", enter can build ai app or agent, with llm integrated. including the latest image and video models like nano banana, seedance 2.0(ready to launch), veo3.1 etc. Auth. Database. Payments. Handled.

Darshal Jaitwar

40,362 views • 8 months ago

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Harper Carroll

30,392 views • 2 months ago

Today we’re releasing V-JEPA, a method for teaching machines to understand and model the physical world by watching videos. This work is another important step towards Yann LeCun’s outlined vision of AI models that use a learned understanding of the world to plan, reason and accomplish complex tasks. Details ➡️ We're releasing a collection of V-JEPA vision models trained with a feature prediction objective using self-supervised learning. The models are able to understand and predict what is going on in a video, even with limited information. It learns by predicting missing or obscured parts of a video in its internal feature space. Unlike generative approaches that fill in missing pixels, this flexible approach enables up to 6x improvements in training and sample efficiency. The models were pre-trained on entirely unlabeled data, and a small amount of labeled data can be used to train a task-specific prediction head on top after pre-training. Our results show that, using a frozen backbone, our top V-JEPA models achieve 82.0% on Kinetics-400, 72.2% on Something-Something-v2 and 77.9% on ImageNet1K — competitive with or exceeding previous leading video models. We believe that this work is an important milestone on the path to advancing machine intelligence.

Today we’re releasing V-JEPA, a method for teaching machines to understand and model the physical world by watching videos. This work is another important step towards Yann LeCun’s outlined vision of AI models that use a learned understanding of the world to plan, reason and accomplish complex tasks. Details ➡️ We're releasing a collection of V-JEPA vision models trained with a feature prediction objective using self-supervised learning. The models are able to understand and predict what is going on in a video, even with limited information. It learns by predicting missing or obscured parts of a video in its internal feature space. Unlike generative approaches that fill in missing pixels, this flexible approach enables up to 6x improvements in training and sample efficiency. The models were pre-trained on entirely unlabeled data, and a small amount of labeled data can be used to train a task-specific prediction head on top after pre-training. Our results show that, using a frozen backbone, our top V-JEPA models achieve 82.0% on Kinetics-400, 72.2% on Something-Something-v2 and 77.9% on ImageNet1K — competitive with or exceeding previous leading video models. We believe that this work is an important milestone on the path to advancing machine intelligence.

AI at Meta

704,200 views • 2 years ago

The next big advantage in AI might not come from one breakthrough. It might come from how quickly a team can build, learn, and improve. Tencent Hunyuan has now released three major models in six months. Hy4 preview is the latest result of that pace. It is fully open source, with 770B total parameters, 49B active parameters, and support for more than 1M tokens of context. The model is designed around practical work, including coding, office productivity, science, planning, and longer workflows that require sustained execution. Access is priced at $0.834 per million input tokens, $2.501 per million output tokens, and $0.042 per million cache hits. That combination of rapid iteration, open access, and affordable usage is what makes this release worth paying attention to. Hy4 preview can be experienced free on WorkBuddy for the next two weeks. Tencent Hy Tencent AI WorkBuddy

The next big advantage in AI might not come from one breakthrough. It might come from how quickly a team can build, learn, and improve. Tencent Hunyuan has now released three major models in six months. Hy4 preview is the latest result of that pace. It is fully open source, with 770B total parameters, 49B active parameters, and support for more than 1M tokens of context. The model is designed around practical work, including coding, office productivity, science, planning, and longer workflows that require sustained execution. Access is priced at $0.834 per million input tokens, $2.501 per million output tokens, and $0.042 per million cache hits. That combination of rapid iteration, open access, and affordable usage is what makes this release worth paying attention to. Hy4 preview can be experienced free on WorkBuddy for the next two weeks. Tencent Hy Tencent AI WorkBuddy

Manish Kumar Shah

42,263 views • 1 month ago

PSA: General-purpose multimodal models, like Gemini 2.0, Grok-2, and GPT-4o are NOT good at interpreting medical images. This is expected, as they were not trained for this purpose. Last week, I shared a video of Gemini 2.0 Realtime looking at a CT that went viral. It was an extremely simple case (Case 1 in my call prep course on Navigating Radiology), and I asked very simple questions. Given the context - screensharing with a general model - I was genuinely impressed that it could identify some very basic anatomy and an abnormal pancreas in realtime! The video demonstrated the power of interacting with AI this way, and the potential for future medical-specific models to be used similarly. BUT many reposts of the video perpetuated misconceptions: AI replacing radiologists, Grok can read your scans, etc. Naturally, this video will likely not go viral, but here’s an example of what frequently happens with general-purpose models: many basic errors (calling the prostate the bladder, and the bladder the uterus), seeing masses that don't exist, etc - often probabilistic based on context. That said, interacting with LLMs real time with screensharing IS a very cool unlock, with a lot of potential. Very exciting!

PSA: General-purpose multimodal models, like Gemini 2.0, Grok-2, and GPT-4o are NOT good at interpreting medical images. This is expected, as they were not trained for this purpose. Last week, I shared a video of Gemini 2.0 Realtime looking at a CT that went viral. It was an extremely simple case (Case 1 in my call prep course on Navigating Radiology), and I asked very simple questions. Given the context - screensharing with a general model - I was genuinely impressed that it could identify some very basic anatomy and an abnormal pancreas in realtime! The video demonstrated the power of interacting with AI this way, and the potential for future medical-specific models to be used similarly. BUT many reposts of the video perpetuated misconceptions: AI replacing radiologists, Grok can read your scans, etc. Naturally, this video will likely not go viral, but here’s an example of what frequently happens with general-purpose models: many basic errors (calling the prostate the bladder, and the bladder the uterus), seeing masses that don't exist, etc - often probabilistic based on context. That said, interacting with LLMs real time with screensharing IS a very cool unlock, with a lot of potential. Very exciting!

Rajesh Bhayana

21,682 views • 1 year ago

Vercel AI Gateway now supports video generation. Grok Imagine Video & Image are 🆓 until tomorrow. We used v0 to create an open source Creative Studio powered by xAI Grok. Create images, videos, or make your own design tool! – it's quite fast. Some cool technical tidbits: ▪️ Vercel Workflows for reliable generation Video generation can take long. Users might restart their browsers or their wifi / LTE might drop. We smooth over that automatically. ▪️ Instant vector search Try searching the public generations. Now back to me. We used Mixedbread to index the content visually, so you can search 𝚋𝚕𝚊𝚌𝚔 𝚏𝚎𝚕𝚒𝚗𝚎 and get a cat (try it!) ▪️ Vercel AI Gateway All existing and future Grok models are accessible via our gateway without extra setup. When you deploy to Vercel, you get access to hundreds of models with a single account and unified billing It's obviously built with Next.js (RSC). Open it in v0 to make it your own! h/t Esteban Suárez for shipping this.

Vercel AI Gateway now supports video generation. Grok Imagine Video & Image are 🆓 until tomorrow. We used v0 to create an open source Creative Studio powered by xAI Grok. Create images, videos, or make your own design tool! – it's quite fast. Some cool technical tidbits: ▪️ Vercel Workflows for reliable generation Video generation can take long. Users might restart their browsers or their wifi / LTE might drop. We smooth over that automatically. ▪️ Instant vector search Try searching the public generations. Now back to me. We used Mixedbread to index the content visually, so you can search 𝚋𝚕𝚊𝚌𝚔 𝚏𝚎𝚕𝚒𝚗𝚎 and get a cat (try it!) ▪️ Vercel AI Gateway All existing and future Grok models are accessible via our gateway without extra setup. When you deploy to Vercel, you get access to hundreds of models with a single account and unified billing It's obviously built with Next.js (RSC). Open it in v0 to make it your own! h/t Esteban Suárez for shipping this.

Guillermo Rauch

15,876,082 views • 7 months ago

Black Forest Labs just announced FLUX 3: a unified multimodal model for image, video, audio and action prediction. Founded in Freiburg, Germany, one of the few globally significant European companies in the AI sector. The current release is gated early access, not open source. But if FLUX 3 Dev actually ships with usable open weights, this could become one of the most important open-model releases in generative video yet. Especially after the Chinese Minimax H3 release. BFL claims, based on preliminary internal comparisons, that FLUX 3 outperformed models such as Runway Gen-4.5, Luma Ray 3.2, Kling 3 Pro, and Seedance 2.0. However, these figures stem from the early-access phase and do not constitute independent validation. Be that as it may, it's great to see the quality that today's video models can produce at affordable prices. Something that would have been unthinkable a year ago. Really cool release!

Black Forest Labs just announced FLUX 3: a unified multimodal model for image, video, audio and action prediction. Founded in Freiburg, Germany, one of the few globally significant European companies in the AI sector. The current release is gated early access, not open source. But if FLUX 3 Dev actually ships with usable open weights, this could become one of the most important open-model releases in generative video yet. Especially after the Chinese Minimax H3 release. BFL claims, based on preliminary internal comparisons, that FLUX 3 outperformed models such as Runway Gen-4.5, Luma Ray 3.2, Kling 3 Pro, and Seedance 2.0. However, these figures stem from the early-access phase and do not constitute independent validation. Be that as it may, it's great to see the quality that today's video models can produce at affordable prices. Something that would have been unthinkable a year ago. Really cool release!

Chubby♨️

39,869 views • 2 months ago

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:

This is THE moment of Physical AI! We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀 - Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks. Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate. The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community. Welcome to the era of Physical AI. HuggingFace: Project Website: Code:

Max Zhaoshuo Li 李赵硕

1,080,311 views • 4 months ago

Dylan on why access to the most intelligent AI models will determine who wins in every industry: "What really matters is access to the most intelligent tokens and leveraging them at the highest-value tasks. A lot of folks will want tokens, but the shitty SaaS startup in SF who is using Claude is not necessarily creating a ton of value, and therefore they're going to get priced out of tokens. Anthropic only released Mythos to certain companies for cybersecurity. You also don't want people to distill your model so you release them to a fewer and fewer customers. Those customers are now wrestling over the tokens. Token usage and therefore the additional value generated on top of those tokens aggregates among fewer and fewer companies. I don't have Mythos but you know who has Mythos, top banks. There are people like Ken Griffin of Citadel who can just sign a deal with OpenAI or Antrhopic for access to your models. I'll buy the first $10 billion worth of tokens each year. So whenever you release the model, I'll spend the first 10 billion tokens and then everyone else can get the model after that. Now, he's going to crush everyone in the markets. I can envision because I have an enterprise Anthropic contract and Anthropic people like me, they're willing to give us slightly earlier access or higher rate limits, and then my competitor doesn't have that, and I'm able to f*cking crush them."

Dylan on why access to the most intelligent AI models will determine who wins in every industry: "What really matters is access to the most intelligent tokens and leveraging them at the highest-value tasks. A lot of folks will want tokens, but the shitty SaaS startup in SF who is using Claude is not necessarily creating a ton of value, and therefore they're going to get priced out of tokens. Anthropic only released Mythos to certain companies for cybersecurity. You also don't want people to distill your model so you release them to a fewer and fewer customers. Those customers are now wrestling over the tokens. Token usage and therefore the additional value generated on top of those tokens aggregates among fewer and fewer companies. I don't have Mythos but you know who has Mythos, top banks. There are people like Ken Griffin of Citadel who can just sign a deal with OpenAI or Antrhopic for access to your models. I'll buy the first $10 billion worth of tokens each year. So whenever you release the model, I'll spend the first 10 billion tokens and then everyone else can get the model after that. Now, he's going to crush everyone in the markets. I can envision because I have an enterprise Anthropic contract and Anthropic people like me, they're willing to give us slightly earlier access or higher rate limits, and then my competitor doesn't have that, and I'm able to f*cking crush them."

Patrick OShaughnessy

36,100 views • 5 months ago

Turn one product photo into a finished video ad. Storyboard to video with sound in minutes - inside a chat. Drop in a product photo and one sentence. Minutes later: a 9-frame storyboard, then a finished video ad with sound, ready to run. Powered by GPT Image 2 → Seedance 2.0 - one of the strongest storyboard-to-video pairings available right now, both in one chat at Why it's different: → No subscription. Most Seedance providers lock you into a monthly plan whether you make one clip or a hundred. Sogni charges per generation on a people-powered consumer GPU network. → Agent-native. Run the whole pipeline from Claude, Codex, or Hermes - "turn this into a 9-shot storyboard video" straight from your terminal. → Right model for the job. Frontier models when you need them, open-source models like LTX 2.3 when you don't. One photo. One sentence. One finished ad. Your move ↓

Turn one product photo into a finished video ad. Storyboard to video with sound in minutes - inside a chat. Drop in a product photo and one sentence. Minutes later: a 9-frame storyboard, then a finished video ad with sound, ready to run. Powered by GPT Image 2 → Seedance 2.0 - one of the strongest storyboard-to-video pairings available right now, both in one chat at Why it's different: → No subscription. Most Seedance providers lock you into a monthly plan whether you make one clip or a hundred. Sogni charges per generation on a people-powered consumer GPU network. → Agent-native. Run the whole pipeline from Claude, Codex, or Hermes - "turn this into a 9-shot storyboard video" straight from your terminal. → Right model for the job. Frontier models when you need them, open-source models like LTX 2.3 when you don't. One photo. One sentence. One finished ad. Your move ↓

Sogni.ai

1,089,620 views • 4 months ago

Really happy to see the interest around our “Hands-on with Gemini” video. In our developer blog yesterday, we broke down how Gemini was used to create it. We gave Gemini sequences of different modalities — image and text in this case — and had it respond by predicting what might come next. Devs can try similar things when access to Pro opens on 12/13 🚀. The knitting demo used Ultra⚡ All the user prompts and outputs in the video are real, shortened for brevity. The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers. When you’re building an app, you can get similar results (there’s always some variability with LLMs) by prompting Gemini with an instruction that allows the user to "configure" the behavior of the model, like inputting “you are an expert in science …” before a user can engage in the same kind of back and forth dialogue. Here’s a clip of what this looks like in AI Studio with Gemini Pro. We’ve come a long way since Flamingo 🦩 & PALI, looking forward to seeing what people build with it!

Really happy to see the interest around our “Hands-on with Gemini” video. In our developer blog yesterday, we broke down how Gemini was used to create it. We gave Gemini sequences of different modalities — image and text in this case — and had it respond by predicting what might come next. Devs can try similar things when access to Pro opens on 12/13 🚀. The knitting demo used Ultra⚡ All the user prompts and outputs in the video are real, shortened for brevity. The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers. When you’re building an app, you can get similar results (there’s always some variability with LLMs) by prompting Gemini with an instruction that allows the user to "configure" the behavior of the model, like inputting “you are an expert in science …” before a user can engage in the same kind of back and forth dialogue. Here’s a clip of what this looks like in AI Studio with Gemini Pro. We’ve come a long way since Flamingo 🦩 & PALI, looking forward to seeing what people build with it!

Oriol Vinyals

181,036 views • 2 years ago