Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Excited to release a preview of Moondream 3. A 9B param, 2B active MoE vision language model that makes no compromises; offering state-of-the-art visual reasoning while still retaining an efficient and deployment-friendly form factor.

vik

28,323 subscribers

228,372 views • 10 months ago •via X (Twitter)

Science & Technology News & Politics Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

The GeoAI Python package now supports interactive segmentation of remote sensing imagery with the Moondream AI Vision Language Models (VLMs), which provides powerful and efficient visual reasoning. A great alternative to Meta's SAM 3. GitHub: Notebook example: #GeoAI #geospatial #AI #opensource

The GeoAI Python package now supports interactive segmentation of remote sensing imagery with the Moondream AI Vision Language Models (VLMs), which provides powerful and efficient visual reasoning. A great alternative to Meta's SAM 3. GitHub: Notebook example: #GeoAI #geospatial #AI #opensource

Qiusheng Wu

11,655 views • 7 months ago

We present VLM-3R: a Vision-Language Model capable of 3D spatial reasoning from monocular video, grounding visual cues, geometry, and camera motion. ✅ No depth sensor ✅ No pre-built 3D maps ✅ End-to-end spatial + temporal reasoning 🔗 Code & benchmark: #VLM #3DVision #LLMs

We present VLM-3R: a Vision-Language Model capable of 3D spatial reasoning from monocular video, grounding visual cues, geometry, and camera motion. ✅ No depth sensor ✅ No pre-built 3D maps ✅ End-to-end spatial + temporal reasoning 🔗 Code & benchmark: #VLM #3DVision #LLMs

Zhiwen(Aaron) Fan

14,895 views • 1 year ago

mapping out the visual language of film using a multimodal llm: i fed frames of a short film to a vision-language model and mapped out its ratings of surrealism and presence of human figure in each moment along the timeline. the result is an interactive playback interface based on these 2 dimensions:

mapping out the visual language of film using a multimodal llm: i fed frames of a short film to a vision-language model and mapped out its ratings of surrealism and presence of human figure in each moment along the timeline. the result is an interactive playback interface based on these 2 dimensions:

Kat ⊷ the Poet Engineer

94,879 views • 1 year ago

NVIDIA Cosmos Reason 2 is here. 🥳 An open, highly accurate reasoning vision language model for physical AI, featuring: ✅ Improved spatio-temporal understanding and timestamp precision ✅ Flexible deployment with 2B and 8B model sizes ✅ Long-context reasoning with up to 256K tokens ✅ Expanded visual perception across complex environments We also have new Cosmos releases: Predict 2.5, Transfer 2.5, and the NVIDIA GR00T N1.6 robot foundation model. 📗Read our technical blog: 🤗 Download Cosmos Reason 2 on Hugging Face:

NVIDIA Cosmos Reason 2 is here. 🥳 An open, highly accurate reasoning vision language model for physical AI, featuring: ✅ Improved spatio-temporal understanding and timestamp precision ✅ Flexible deployment with 2B and 8B model sizes ✅ Long-context reasoning with up to 256K tokens ✅ Expanded visual perception across complex environments We also have new Cosmos releases: Predict 2.5, Transfer 2.5, and the NVIDIA GR00T N1.6 robot foundation model. 📗Read our technical blog: 🤗 Download Cosmos Reason 2 on Hugging Face:

NVIDIA AI Developer

45,677 views • 6 months ago

Introducing INTELLECT-3: Scaling RL to a 100B+ MoE model on our end-to-end stack Achieving state-of-the-art performance for its size across math, code and reasoning Built using the same tools we put in your hands, from environments & evals, RL frameworks, sandboxes & more

Introducing INTELLECT-3: Scaling RL to a 100B+ MoE model on our end-to-end stack Achieving state-of-the-art performance for its size across math, code and reasoning Built using the same tools we put in your hands, from environments & evals, RL frameworks, sandboxes & more

Prime Intellect

1,144,587 views • 8 months ago

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

Fangchen Liu

68,366 views • 1 year ago

🚀 Excited to announce the first release of a novel open source programming language and platform for language model interaction! Combining prompts, constraints & scripting, LMQL elevates the capabilities of large language models. 🧵1/6 A quick tour.

🚀 Excited to announce the first release of a novel open source programming language and platform for language model interaction! Combining prompts, constraints & scripting, LMQL elevates the capabilities of large language models. 🧵1/6 A quick tour.

LMQL (Language Model Query Language)

198,966 views • 3 years ago

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

AI at Meta

163,313 views • 1 year ago

🚀Introducing LLaVA Lightning: Train a lite, multimodal GPT-4 with just $40 in 3 hours! With our newly introduced datasets and the efficient design of LLaVA, you can now turbocharge your language model with image reasoning capabilities, in an incredibly affordable way.🧵

🚀Introducing LLaVA Lightning: Train a lite, multimodal GPT-4 with just $40 in 3 hours! With our newly introduced datasets and the efficient design of LLaVA, you can now turbocharge your language model with image reasoning capabilities, in an incredibly affordable way.🧵

Haotian Liu

302,319 views • 3 years ago

Emerging as the global frontier of model-reasoning has been a core vision of Affine since its inception. In pursuit of this vision, we've been working relentlessly to forge a repeatable training loop: a mechanism that uses benchmark driven post-training methodology to facilitate the improvement of reasoning itself. Through applying our training methodology onto Qwen3-32B as our base model, Affine’s miners have been able to push the Pareto frontier outward. We’ve now reached saturation; our models have become so good at reasoning that any further improvements are producing diminishing returns. As a result we now move our base model to Qwen3.6-35B-A3B. It provides a stronger basis, offering an extended context window, as well as an improved foundation for agentic work. The same training methodology will be applied going forward. Our objective is no longer simply to improve a single model, but to demonstrate that the process in itself is repeatable, scalable, and model agnostic. We view the commoditization of reasoning as vital to ensure our technology is not only transparent, but also co-owned by humanity.

Emerging as the global frontier of model-reasoning has been a core vision of Affine since its inception. In pursuit of this vision, we've been working relentlessly to forge a repeatable training loop: a mechanism that uses benchmark driven post-training methodology to facilitate the improvement of reasoning itself. Through applying our training methodology onto Qwen3-32B as our base model, Affine’s miners have been able to push the Pareto frontier outward. We’ve now reached saturation; our models have become so good at reasoning that any further improvements are producing diminishing returns. As a result we now move our base model to Qwen3.6-35B-A3B. It provides a stronger basis, offering an extended context window, as well as an improved foundation for agentic work. The same training methodology will be applied going forward. Our objective is no longer simply to improve a single model, but to demonstrate that the process in itself is repeatable, scalable, and model agnostic. We view the commoditization of reasoning as vital to ensure our technology is not only transparent, but also co-owned by humanity.

Affine

14,536 views • 1 month ago

♾Infinity TK♾ Looking for a tie designed for reducing pressure on the lower arms while still retaining the feel of a TK? This hybrid between a regular box-tie and the Butterfly Harness is pretty 𝒂𝒏𝒅 practical, offering more comfort for the model. ➡️

♾Infinity TK♾ Looking for a tie designed for reducing pressure on the lower arms while still retaining the feel of a TK? This hybrid between a regular box-tie and the Butterfly Harness is pretty 𝒂𝒏𝒅 practical, offering more comfort for the model. ➡️

Shibari Study

55,702 views • 2 years ago

Today we’re taking a big step on the path toward AGI and releasing Gemini 3— our most intelligent model yet. With Gemini 3, you can bring any idea to life. It is state-of-the-art in reasoning, the best model in the world for multimodal understanding, and our best agentic and vibe coding model.

Today we’re taking a big step on the path toward AGI and releasing Gemini 3— our most intelligent model yet. With Gemini 3, you can bring any idea to life. It is state-of-the-art in reasoning, the best model in the world for multimodal understanding, and our best agentic and vibe coding model.

Google AI

492,973 views • 8 months ago

🌍Today we release Mosaic, a probabilistic weather model that shifts the Pareto frontier of ML weather forecasting. It matches the skill of state-of-the-art models while generating a 24-member, 10-day global forecast in under 12 s on a single H100. Thread!

🌍Today we release Mosaic, a probabilistic weather model that shifts the Pareto frontier of ML weather forecasting. It matches the skill of state-of-the-art models while generating a 24-member, 10-day global forecast in under 12 s on a single H100. Thread!

Max Zhdanov

144,991 views • 2 months ago

The upcoming OpenAI's open model is mixture of expert model. They take their proprietary training process and train a model with an model architecture that only contains publicly well known model arch layers & optimizations similar. This allows OpenAI to keep their internal model architecture an secret while they open weight an SOTA model. Thus, none of their long context optimizations needed for acutal real world production serving are in the open model architecture. OpenAI has created multiple different sizes of open weights distilled o3 including an 120B param that fits on a single node and 20B param that fits on a single chip.

The upcoming OpenAI's open model is mixture of expert model. They take their proprietary training process and train a model with an model architecture that only contains publicly well known model arch layers & optimizations similar. This allows OpenAI to keep their internal model architecture an secret while they open weight an SOTA model. Thus, none of their long context optimizations needed for acutal real world production serving are in the open model architecture. OpenAI has created multiple different sizes of open weights distilled o3 including an 120B param that fits on a single node and 20B param that fits on a single chip.

SemiAnalysis

80,891 views • 1 year ago

“We’re excited to make PEPFAR more efficient. We think we can meet President Donald J. Trump’s goal of ending mother-to-child transmission by the time that he leaves office. There are a lot of different things that we’re excited about, and they’re a key part of our affirmative vision.”

“We’re excited to make PEPFAR more efficient. We think we can meet President Donald J. Trump’s goal of ending mother-to-child transmission by the time that he leaves office. There are a lot of different things that we’re excited about, and they’re a key part of our affirmative vision.”

Department of State

21,979 views • 1 year ago

New LLMs that control UIs! ByteDance Research releases UI-TARS, fine-tuned GUI agent that integrates reasoning, and action capabilities into a single vision-language model. Think of computer use but open. 👀 TL;DR; 3️⃣ Available in 3 sizes: 2B, 7B, and 72B parameters 🧠 Trained Qwen2-VL models with SFT & DPO 🥇 72B version achieves 82.8% on VisualWebBench (beating GPT-4 and Claude) 🏆 Achieves state-of-the-art results on 10+ GUI agent benchmarks 💡 Reasons before taking an action 🧑🏻‍💻 Can Click, Long Press, type, scroll, open app, navigate back/home, wait 🤗 Released under Apache 2.0 on Hugging Face

New LLMs that control UIs! ByteDance Research releases UI-TARS, fine-tuned GUI agent that integrates reasoning, and action capabilities into a single vision-language model. Think of computer use but open. 👀 TL;DR; 3️⃣ Available in 3 sizes: 2B, 7B, and 72B parameters 🧠 Trained Qwen2-VL models with SFT & DPO 🥇 72B version achieves 82.8% on VisualWebBench (beating GPT-4 and Claude) 🏆 Achieves state-of-the-art results on 10+ GUI agent benchmarks 💡 Reasons before taking an action 🧑🏻‍💻 Can Click, Long Press, type, scroll, open app, navigate back/home, wait 🤗 Released under Apache 2.0 on Hugging Face

Philipp Schmid

48,170 views • 1 year ago

EmbeddingGemma is our new best-in-class open embedding model designed for on-device AI. 📱 At just 308M parameters, it delivers state-of-the-art performance while being small and efficient enough to run anywhere - even without an internet connection.

EmbeddingGemma is our new best-in-class open embedding model designed for on-device AI. 📱 At just 308M parameters, it delivers state-of-the-art performance while being small and efficient enough to run anywhere - even without an internet connection.

Google DeepMind

584,655 views • 11 months ago

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 views • 3 years ago

Google just released Muse: a text to image model achieving state of the art performance. It is significantly more efficient than diffusion models. Muse also allows for inpainting, outpainting, and mask-free editing without finetuning or inverting the model.

Google just released Muse: a text to image model achieving state of the art performance. It is significantly more efficient than diffusion models. Muse also allows for inpainting, outpainting, and mask-free editing without finetuning or inverting the model.

bleedingedge.ai

183,750 views • 3 years ago

Excited to share an early preview of our gemini 2.0 flash thinking model with all it's raw thoughts visible. Here's the model trying to solve a Putnam 2024 with multiple approaches, and then self-verifies that it's answer was correct.

Excited to share an early preview of our gemini 2.0 flash thinking model with all it's raw thoughts visible. Here's the model trying to solve a Putnam 2024 with multiple approaches, and then self-verifies that it's answer was correct.

Ankesh Anand

80,391 views • 1 year ago