Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

open sourcing Marlin-2B 🐟 a tiny VLM to extract structured information from videos Marlin is finetuned for two questions devs want to ask in their videos: what is happening, and when? Best open model in its weight class, competitive with Gemini-2.5-flash at only 2B params 🧵

Shubham Sharma

1,266 subscribers

341,382 просмотров • 1 месяц назад •via X (Twitter)

Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

The plot thickens. 🕵️ Use Gemini 2.5 Pro in the Google Gemini to turn random videos from your camera roll into a dramatic narrative. Try it yourself: 1️⃣ Go to Gemini and select “2.5 Pro” from the model dropdown. 2️⃣ Tap the plus icon and upload three videos from your camera roll. (Pro/Ultra subscribers, try using your Veo creations! 🎥) 3️⃣ Ask Gemini to write a short story based on the videos. Here’s our prompt to get you started: “Come up with a premise for an epic adventure based on these videos.”

The plot thickens. 🕵️ Use Gemini 2.5 Pro in the Google Gemini to turn random videos from your camera roll into a dramatic narrative. Try it yourself: 1️⃣ Go to Gemini and select “2.5 Pro” from the model dropdown. 2️⃣ Tap the plus icon and upload three videos from your camera roll. (Pro/Ultra subscribers, try using your Veo creations! 🎥) 3️⃣ Ask Gemini to write a short story based on the videos. Here’s our prompt to get you started: “Come up with a premise for an epic adventure based on these videos.”

Google

164,414 просмотров • 1 год назад

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Jim Fan

465,968 просмотров • 1 год назад

Inferact cofounder Simon Mo: “I fundamentally believe that open source, especially how vLLM itself is structured, is critical to the AI infrastructure in the world. And what we want to do with Inferact is to support, maintain, steward and push forward the open source ecosystem.” “Open source moves so fast that the only way to stay ahead is adopting... And in fact this is exactly why we're staying all in on open source.” Simon Mo

Inferact cofounder Simon Mo: “I fundamentally believe that open source, especially how vLLM itself is structured, is critical to the AI infrastructure in the world. And what we want to do with Inferact is to support, maintain, steward and push forward the open source ecosystem.” “Open source moves so fast that the only way to stay ahead is adopting... And in fact this is exactly why we're staying all in on open source.” Simon Mo

a16z

29,383 просмотров • 5 месяцев назад

"The Tower Heist" A small window is open to recover an ancient scroll. Created in early access with the fantastic new Kling AI 2.5 Turbo model bringing affordable generations with best in class prompt adherence and quality. The range of directed character motions now available to us is incredible! 🔊🔊🎧

"The Tower Heist" A small window is open to recover an ancient scroll. Created in early access with the fantastic new Kling AI 2.5 Turbo model bringing affordable generations with best in class prompt adherence and quality. The range of directed character motions now available to us is incredible! 🔊🔊🎧

Stevie Mac

114,555 просмотров • 9 месяцев назад

Ask Mel anything! Mel Reid is on Live From the U.S. Women's Open with us this week, and she’s ready to answer your best questions. Drop yours in the replies 👇 #AMA

Ask Mel anything! Mel Reid is on Live From the U.S. Women's Open with us this week, and she’s ready to answer your best questions. Drop yours in the replies 👇 #AMA

Golf Channel

20,808 просмотров • 1 месяц назад

Today at #GoogleIO, we’re demonstrating how our partnership with Google Cloud is redefining the landscape of Intelligent Content Management. Leveraging the advanced capabilities of Google’s Gemini 2.5 Pro, our new AI Agent with enhanced data extraction transforms complex, unstructured data from various sources—including scanned PDFs, images, and diverse forms—into structured, actionable insights. Check out the demo below to watch Gemini 2.5 Pro and Box AI agents in action. With the Box AI Agent powered by Gemini 2.5 Pro you can: ⭐ Automatically, and instantly, classify and organize documents ⭐ Extract accurate insights and information from documents and images ⭐ Enable workflow automation by leveraging structured and easily retrievable content Our partnership with Google is not just about enhancing current capabilities; it’s about unlocking the full potential of AI, and models like Gemini 2.5 Pro, for intricate enterprise use cases.

Today at #GoogleIO, we’re demonstrating how our partnership with Google Cloud is redefining the landscape of Intelligent Content Management. Leveraging the advanced capabilities of Google’s Gemini 2.5 Pro, our new AI Agent with enhanced data extraction transforms complex, unstructured data from various sources—including scanned PDFs, images, and diverse forms—into structured, actionable insights. Check out the demo below to watch Gemini 2.5 Pro and Box AI agents in action. With the Box AI Agent powered by Gemini 2.5 Pro you can: ⭐ Automatically, and instantly, classify and organize documents ⭐ Extract accurate insights and information from documents and images ⭐ Enable workflow automation by leveraging structured and easily retrievable content Our partnership with Google is not just about enhancing current capabilities; it’s about unlocking the full potential of AI, and models like Gemini 2.5 Pro, for intricate enterprise use cases.

Box

10,565 просмотров • 1 год назад

What happens when VLMs meet 3D foundation models? See VLM-3R (CVPR 2026). VLM-3R links a vision-language model (e.g., Qwen) with 3D geometric foundation models (e.g., CUT3R) at metric scale. Given an uncalibrated video, it moves beyond pixels to perceive and reason in 3D space. Code (open source):

What happens when VLMs meet 3D foundation models? See VLM-3R (CVPR 2026). VLM-3R links a vision-language model (e.g., Qwen) with 3D geometric foundation models (e.g., CUT3R) at metric scale. Given an uncalibrated video, it moves beyond pixels to perceive and reason in 3D space. Code (open source):

Zhiwen(Aaron) Fan

10,641 просмотров • 4 месяцев назад

Reasoning is central to purposeful action. Today we introduce MolmoAct — a fully open Action Reasoning Model (ARM) for robotics. Grounded in large-scale pre-training with action reasoning data, every predicted action is interpretable and user-steerable via visual trace. We are open-sourcing everything!

Reasoning is central to purposeful action. Today we introduce MolmoAct — a fully open Action Reasoning Model (ARM) for robotics. Grounded in large-scale pre-training with action reasoning data, every predicted action is interpretable and user-steerable via visual trace. We are open-sourcing everything!

Jiafei Duan

99,944 просмотров • 11 месяцев назад

Gemini 3 Flash is starting to roll out as the default model for AI Mode in Search globally. AI Mode can now tackle your most complicated questions with greater precision — without compromising speed. ⚡ With this upgrade, AI Mode becomes an all-around more powerful tool, better at understanding your needs. You can ask more nuanced questions and it will consider each aspect to provide a thoughtful response. And as always, you’ll have access to real-time information and useful links from across the web.

Gemini 3 Flash is starting to roll out as the default model for AI Mode in Search globally. AI Mode can now tackle your most complicated questions with greater precision — without compromising speed. ⚡ With this upgrade, AI Mode becomes an all-around more powerful tool, better at understanding your needs. You can ask more nuanced questions and it will consider each aspect to provide a thoughtful response. And as always, you’ll have access to real-time information and useful links from across the web.

Google

67,545 просмотров • 7 месяцев назад

What is next for Iran and the people around the world who make up the diaspora from the country? Yalda Hakim speaks to two women about what is happening in Iran and their fears for its future. 🎧

What is next for Iran and the people around the world who make up the diaspora from the country? Yalda Hakim speaks to two women about what is happening in Iran and their fears for its future. 🎧

Sky News

24,278 просмотров • 5 месяцев назад

GLM 4.7 is one of the top open-source models on LM Arena—and it's going toe-to-toe with Claude Opus 4.5 and Gemini Pro. We sat down with Anastasios Nikolas Angelopoulos, co-founder and CEO of Arena.ai, to break down 8,000+ developer votes: → Within 30 points of Gemini Pro in math & coding → Frontier-level multi-turn & instruction following → The open-source model devs are actually switching to The best part? You can run it at 1,500+ tokens/sec on Cerebras—for free.

GLM 4.7 is one of the top open-source models on LM Arena—and it's going toe-to-toe with Claude Opus 4.5 and Gemini Pro. We sat down with Anastasios Nikolas Angelopoulos, co-founder and CEO of Arena.ai, to break down 8,000+ developer votes: → Within 30 points of Gemini Pro in math & coding → Frontier-level multi-turn & instruction following → The open-source model devs are actually switching to The best part? You can run it at 1,500+ tokens/sec on Cerebras—for free.

Cerebras

28,515 просмотров • 5 месяцев назад

2026 internship applications are now open. If you want hands-on experience in the future of finance, this is your entry point. Here’s what to expect and a few highlights from 2025.

2026 internship applications are now open. If you want hands-on experience in the future of finance, this is your entry point. Here’s what to expect and a few highlights from 2025.

Gemini

103,554 просмотров • 7 месяцев назад

Gemini 3 Pro is the best model in the world for multimodal understanding. One of its most exciting capabilities is document understanding and reasoning. This means you can convert information in any format and into the medium that works best for you. Gemini 3 also has leading multilingual capabilities, enabling it to process, reason and even capture cultural relevance across a variety of languages. For example, here Gemini 3 is translating handwritten recipes in Korean and English to build a digital family cookbook in different languages.

Gemini 3 Pro is the best model in the world for multimodal understanding. One of its most exciting capabilities is document understanding and reasoning. This means you can convert information in any format and into the medium that works best for you. Gemini 3 also has leading multilingual capabilities, enabling it to process, reason and even capture cultural relevance across a variety of languages. For example, here Gemini 3 is translating handwritten recipes in Korean and English to build a digital family cookbook in different languages.

Google AI

36,696 просмотров • 7 месяцев назад

AI has its PhD and now it’s on the job market. Introducing the AI Productivity Index (APEX), a benchmark that measures how well we’ve automated the most valuable industries in the world. Most benchmarks study abstract capabilities. APEX evaluates model performance on real deliverables across law, finance, consulting, and medicine. The models most capable of doing work today, according to APEX: 🥇 GPT 5 🥈 Grok 4 🥉 Gemini 2.5 Flash Other findings: - GPT 5 demonstrates the strongest performance across all 4 domains - Some cheaper models outperform more expensive models from the same provider (e.g. Gemini 2.5 Flash vs. Gemini 2.5 Pro) - The best open source model, Qwen (7th), performs only 2% behind Grok 4 overall

AI has its PhD and now it’s on the job market. Introducing the AI Productivity Index (APEX), a benchmark that measures how well we’ve automated the most valuable industries in the world. Most benchmarks study abstract capabilities. APEX evaluates model performance on real deliverables across law, finance, consulting, and medicine. The models most capable of doing work today, according to APEX: 🥇 GPT 5 🥈 Grok 4 🥉 Gemini 2.5 Flash Other findings: - GPT 5 demonstrates the strongest performance across all 4 domains - Some cheaper models outperform more expensive models from the same provider (e.g. Gemini 2.5 Flash vs. Gemini 2.5 Pro) - The best open source model, Qwen (7th), performs only 2% behind Grok 4 overall

Brendan (can/do)

451,298 просмотров • 9 месяцев назад

The ZRP is appealing for information which can lead to the arrest of a suspect who is captured in a viral social media video stealing tobacco from an open truck loaded with tobacco bales. Anyone with information to report at any nearest Police Station.

The ZRP is appealing for information which can lead to the arrest of a suspect who is captured in a viral social media video stealing tobacco from an open truck loaded with tobacco bales. Anyone with information to report at any nearest Police Station.

Zimbabwe Republic Police

25,177 просмотров • 1 год назад

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

AI at Meta

97,755 просмотров • 1 год назад

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):

LiveKit

126,860 просмотров • 1 год назад

“We have a crisis of open source models in the Western world. Outside of China, there are no good open source models. We don't even have any in the US now. The talent, the capital and the focus to be best-in-class at pre-training, mid-training, post-training is an extremely scarce skillset. The answer for a lot of countries may just be to take an open weights model from China, post-train or fine-tune their own version, and have that be what they start from." Everett Randle Why does the West have such poor open source models and is it a national security threat to rely on open-source Chinese models Demis Hassabis Yann LeCun Aidan Gomez Lilian Weng Jan Leike

“We have a crisis of open source models in the Western world. Outside of China, there are no good open source models. We don't even have any in the US now. The talent, the capital and the focus to be best-in-class at pre-training, mid-training, post-training is an extremely scarce skillset. The answer for a lot of countries may just be to take an open weights model from China, post-train or fine-tune their own version, and have that be what they start from." Everett Randle Why does the West have such poor open source models and is it a national security threat to rely on open-source Chinese models Demis Hassabis Yann LeCun Aidan Gomez Lilian Weng Jan Leike

Harry Stebbings

63,397 просмотров • 26 дней назад

The upcoming OpenAI's open model is mixture of expert model. They take their proprietary training process and train a model with an model architecture that only contains publicly well known model arch layers & optimizations similar. This allows OpenAI to keep their internal model architecture an secret while they open weight an SOTA model. Thus, none of their long context optimizations needed for acutal real world production serving are in the open model architecture. OpenAI has created multiple different sizes of open weights distilled o3 including an 120B param that fits on a single node and 20B param that fits on a single chip.

The upcoming OpenAI's open model is mixture of expert model. They take their proprietary training process and train a model with an model architecture that only contains publicly well known model arch layers & optimizations similar. This allows OpenAI to keep their internal model architecture an secret while they open weight an SOTA model. Thus, none of their long context optimizations needed for acutal real world production serving are in the open model architecture. OpenAI has created multiple different sizes of open weights distilled o3 including an 120B param that fits on a single node and 20B param that fits on a single chip.

SemiAnalysis

80,891 просмотров • 11 месяцев назад

The biggest revelation from Deepseek is that Open Source has won. For a 1% difference in performance, it will be difficult for OpenAI to justify its price when the competition is free and formidable. -from my interview with Bloomberg

The biggest revelation from Deepseek is that Open Source has won. For a 1% difference in performance, it will be difficult for OpenAI to justify its price when the competition is free and formidable. -from my interview with Bloomberg

Kai-Fu Lee

759,461 просмотров • 1 год назад