Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Prior to release, we shared a version of Cohere North Mini Code with AI engineers and answered some questions. Here's a quick illustrated walkthrough of the model's architecture and training process. Small models fill an important niche. They: 1. run on more widely available hardware 2. handle tasks within... show more

Jay Alammar

52,233 subscribers

89,469 Aufrufe • vor 3 Monaten •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

14 Kommentare

Profilbild von Jay Alammar

Jay Alammarvor 3 Monaten

The 30 billion parameter mixture of experts model stacks 49 Transformer blocks, the first of which is dense. The MoE layers have 128 experts, and activate 8 for each token. Leading to 3 billion active parameters. The self-attention setup interleaves sliding window attention and full attention in 3:1 ratio. Our team describes this choice in "Rope to Nope and Back Again: A New Hybrid Attention Strategy"

Profilbild von Jay Alammar

Jay Alammarvor 3 Monaten

GGUF quantized version now out kudos to @UnslothAI:

Profilbild von Ali Asaria

Ali Asariavor 3 Monaten

@cohere Love that you're sharing details on the internals like this. This level of detail is something that helps the whole community advance.

Profilbild von Latent Node

Latent Nodevor 3 Monaten

@cohere It is great but I wish it was better than qwen 3.6 27b or gemma 4 31b.

Profilbild von Aaliya

Aaliyavor 3 Monaten

@cohere Light models working on sub tasks is smart.

Profilbild von Samian

Samianvor 3 Monaten

@cohere small models hitting the sweet spot rn. curious how north mini handles tool calling vs something like haiku, that tradeoff usually gets glossed over

Profilbild von Fawaz Buqammaz | فوّاز بوقمّاز

Fawaz Buqammaz | فوّاز بوقمّازvor 3 Monaten

@cohere ملهم👏

Profilbild von Lila

Lilavor 3 Monaten

@cohere love a clean visual breakdown... small models handling specific sub-tasks instead of giant bloated ones is just good system design

Profilbild von Sani Ai Tech

Sani Ai Techvor 3 Monaten

@cohere Small models are quietly becoming the backbone of scalable AI systems. 🚀

Profilbild von g023

g023vor 3 Monaten

@cohere I mean you didn't beat Qwen, and I think you should probably focus on that goal, rather than release to release, or try a smaller target and beat the likes of LFM2.5 and their 8B-A1B.

Profilbild von Jason傑森 🇭🇰 | 🛠️

Jason傑森 🇭🇰 | 🛠️vor 3 Monaten

@cohere 话是对的但有一种不会成的宿感在

Profilbild von aitization 𝕏 

aitization 𝕏 vor 3 Monaten

@cohere dm me :)

Profilbild von ADITYA DUTT PANDEY

ADITYA DUTT PANDEYvor 3 Monaten

@cohere small models, big cap 🥱

Profilbild von Nahid

Nahidvor 3 Monaten

@cohere small models really do have their perks super flexible too

Ähnliche Videos

We’re redefining what’s possible with AI. With the release of our latest model, Command A, optimized for real-world agentic and multilingual tasks, we’re demonstrating our commitment to bringing enterprises AI that goes beyond the ordinary, and offers security & efficiency. Our team has developed highly capable and efficient models that can be run on just 2 GPUs. Check out our tech report to learn more:

We’re redefining what’s possible with AI. With the release of our latest model, Command A, optimized for real-world agentic and multilingual tasks, we’re demonstrating our commitment to bringing enterprises AI that goes beyond the ordinary, and offers security & efficiency. Our team has developed highly capable and efficient models that can be run on just 2 GPUs. Check out our tech report to learn more:

Cohere

16,426 Aufrufe • vor 1 Jahr

🚀 Excited to release AIBuildAI — an AI agent that automatically builds AI models 🏆 #1 on OpenAI MLE-Bench 💻 GitHub: Building AI models still requires a lot of manual work — designing models, writing code, training them, tuning hyperparameters, and iterating on results. We developed AIBuildAI to automate much of this process. The system runs an agent loop that automatically: • analyzes the task • designs models • writes code to implement them • trains the models • tunes hyperparameters • evaluates model performance • iteratively improves the models 🏆 On OpenAI’s MLE-Bench benchmark, AIBuildAI ranked #1:

🚀 Excited to release AIBuildAI — an AI agent that automatically builds AI models 🏆 #1 on OpenAI MLE-Bench 💻 GitHub: Building AI models still requires a lot of manual work — designing models, writing code, training them, tuning hyperparameters, and iterating on results. We developed AIBuildAI to automate much of this process. The system runs an agent loop that automatically: • analyzes the task • designs models • writes code to implement them • trains the models • tunes hyperparameters • evaluates model performance • iteratively improves the models 🏆 On OpenAI’s MLE-Bench benchmark, AIBuildAI ranked #1:

Pengtao Xie

152,252 Aufrufe • vor 6 Monaten

Want to reduce your Claude Code bill? Claude Code uses smaller models like Haiku for simple tasks. Instead of sending every small request to a paid API, run those tasks locally with Jan and keep heavier work in the cloud. One setup. Lower API usage. More control. Get the latest version at

Want to reduce your Claude Code bill? Claude Code uses smaller models like Haiku for simple tasks. Instead of sending every small request to a paid API, run those tasks locally with Jan and keep heavier work in the cloud. One setup. Lower API usage. More control. Get the latest version at

👋 Jan

13,169 Aufrufe • vor 7 Monaten

Just gave a demo at Every 📧's Vibe Code Camp showing how I use a kanban board to stay sane while overseeing Claude Code agents😎 Here's a quick recap video showing the whole flow: 1) Monitor progress on all tasks in one view 2) See when Claude is blocked, and respond in context 3) Read Claude's plans with nice formatting, not raw Markdown 4) Use AI to process voice notes into structured tasks 5) Get explanations of code changes... with an interactive quiz to check if you understood 🙃 I've had mixed feelings about kanban workflows in the past, but as the agents get stronger I'm warming up to them—especially for churning through small well-specified tasks. In that context, it feels like the challenge is managing my working memory for what's going on. And having a stable "zoomed-out map" of all the tasks really helps me with offloading some of that cognitive load to a visualization of everything that's going on!

Just gave a demo at Every 📧's Vibe Code Camp showing how I use a kanban board to stay sane while overseeing Claude Code agents😎 Here's a quick recap video showing the whole flow: 1) Monitor progress on all tasks in one view 2) See when Claude is blocked, and respond in context 3) Read Claude's plans with nice formatting, not raw Markdown 4) Use AI to process voice notes into structured tasks 5) Get explanations of code changes... with an interactive quiz to check if you understood 🙃 I've had mixed feelings about kanban workflows in the past, but as the agents get stronger I'm warming up to them—especially for churning through small well-specified tasks. In that context, it feels like the challenge is managing my working memory for what's going on. And having a stable "zoomed-out map" of all the tasks really helps me with offloading some of that cognitive load to a visualization of everything that's going on!

Geoffrey Litt

78,473 Aufrufe • vor 8 Monaten

We are releasing 4M-21 with a permissive license, including its source code and trained models. It's a pretty effective multimodal model that solves 10s of tasks & modalities. See the demo code, sample results, and the tokenizers of diverse modalities on the website. IMO, the multitask learning aspect of multimodal models has really taken a step forward. We can train a single model on many diverse tasks with ~SOTA accuracy. But a long way to go in terms of transfer/emergence. 🌐 ⌨️ Joint work w/ EPFL Apple.

We are releasing 4M-21 with a permissive license, including its source code and trained models. It's a pretty effective multimodal model that solves 10s of tasks & modalities. See the demo code, sample results, and the tokenizers of diverse modalities on the website. IMO, the multitask learning aspect of multimodal models has really taken a step forward. We can train a single model on many diverse tasks with ~SOTA accuracy. But a long way to go in terms of transfer/emergence. 🌐 ⌨️ Joint work w/ EPFL Apple.

Amir Zamir

69,604 Aufrufe • vor 2 Jahren

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Santiago

164,162 Aufrufe • vor 2 Jahren

Today we’re releasing V-JEPA, a method for teaching machines to understand and model the physical world by watching videos. This work is another important step towards Yann LeCun’s outlined vision of AI models that use a learned understanding of the world to plan, reason and accomplish complex tasks. Details ➡️ We're releasing a collection of V-JEPA vision models trained with a feature prediction objective using self-supervised learning. The models are able to understand and predict what is going on in a video, even with limited information. It learns by predicting missing or obscured parts of a video in its internal feature space. Unlike generative approaches that fill in missing pixels, this flexible approach enables up to 6x improvements in training and sample efficiency. The models were pre-trained on entirely unlabeled data, and a small amount of labeled data can be used to train a task-specific prediction head on top after pre-training. Our results show that, using a frozen backbone, our top V-JEPA models achieve 82.0% on Kinetics-400, 72.2% on Something-Something-v2 and 77.9% on ImageNet1K — competitive with or exceeding previous leading video models. We believe that this work is an important milestone on the path to advancing machine intelligence.

Today we’re releasing V-JEPA, a method for teaching machines to understand and model the physical world by watching videos. This work is another important step towards Yann LeCun’s outlined vision of AI models that use a learned understanding of the world to plan, reason and accomplish complex tasks. Details ➡️ We're releasing a collection of V-JEPA vision models trained with a feature prediction objective using self-supervised learning. The models are able to understand and predict what is going on in a video, even with limited information. It learns by predicting missing or obscured parts of a video in its internal feature space. Unlike generative approaches that fill in missing pixels, this flexible approach enables up to 6x improvements in training and sample efficiency. The models were pre-trained on entirely unlabeled data, and a small amount of labeled data can be used to train a task-specific prediction head on top after pre-training. Our results show that, using a frozen backbone, our top V-JEPA models achieve 82.0% on Kinetics-400, 72.2% on Something-Something-v2 and 77.9% on ImageNet1K — competitive with or exceeding previous leading video models. We believe that this work is an important milestone on the path to advancing machine intelligence.

AI at Meta

704,009 Aufrufe • vor 2 Jahren

JARVIS-VLA just dropped on Hugging Face Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse obtain VLA models in Minecraft that can follow human instructions on over 1k different atomic tasks, including crafting, smelting, cooking, mining, and killing. experiments demonstrate that post-training on non-trajectory tasks leads to a significant 40% improvement over the best agent baseline on a diverse set of atomic tasks. Furthermore, demonstrate that approach surpasses traditional imitation learning-based policies in Minecraft, achieving state-of-the-art performance.

JARVIS-VLA just dropped on Hugging Face Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse obtain VLA models in Minecraft that can follow human instructions on over 1k different atomic tasks, including crafting, smelting, cooking, mining, and killing. experiments demonstrate that post-training on non-trajectory tasks leads to a significant 40% improvement over the best agent baseline on a diverse set of atomic tasks. Furthermore, demonstrate that approach surpasses traditional imitation learning-based policies in Minecraft, achieving state-of-the-art performance.

AK

60,243 Aufrufe • vor 1 Jahr

AI only has pre-schooler level capabilities for visual tasks. "The frontier models—the best of them—still reason around the age of a preschooler, and any elementary school kid in that benchmark was able to beat all the frontier models on these visual tasks." "These are tasks that are not just object recognition. It's about spatial reasoning... tasks like navigating through the maze…” — Andrew M. Dai, CEO of Elorian

AI only has pre-schooler level capabilities for visual tasks. "The frontier models—the best of them—still reason around the age of a preschooler, and any elementary school kid in that benchmark was able to beat all the frontier models on these visual tasks." "These are tasks that are not just object recognition. It's about spatial reasoning... tasks like navigating through the maze…” — Andrew M. Dai, CEO of Elorian

The Information

26,158 Aufrufe • vor 2 Monaten

Watch Nvidia CEO Jensen Huang reveal what he described as the state of the art in AI hardware: the new Rubin computing architecture. “Vera Rubin is designed to address this fundamental challenge that we have: The amount of computation necessary for AI is skyrocketing.” Huang said. “Today, I can tell you that Vera Rubin is in full production.” According to Nvidia’s tests, Rubin architecture operates three and a half times faster than the previous Blackwell architecture on model-training tasks and five times faster on inference tasks, and that's just the start.

Watch Nvidia CEO Jensen Huang reveal what he described as the state of the art in AI hardware: the new Rubin computing architecture. “Vera Rubin is designed to address this fundamental challenge that we have: The amount of computation necessary for AI is skyrocketing.” Huang said. “Today, I can tell you that Vera Rubin is in full production.” According to Nvidia’s tests, Rubin architecture operates three and a half times faster than the previous Blackwell architecture on model-training tasks and five times faster on inference tasks, and that's just the start.

TechCrunch

76,412 Aufrufe • vor 8 Monaten

I'm super pumped about this one—we partnered with @heytavus on a plugin to simplify the process of building realtime AI avatars. There's a crazy amount of complexity to realtime procedural video generation that we just handle. In two lines of code—add an avatar to any agent.

Shayne

45,090 Aufrufe • vor 1 Jahr

BREAKING: Only ~1% of Bending Spoons’ AI requests & tokens goes to frontier models. 99% runs on self-hosted open-weight models. CEO Luca Ferrari on the shift to Sovereign AI: “We end up using open-weight models we self-host, so these are basically free. I mean, there is a little bit of cloud cost, but for 99% of the requests and tokens.” “And the frontier models, generally closed weight, through APIs for maybe 1% of the requests, only for the most complex tasks.” “Sometimes we use them automatically to check the work that got done by the slightly less intelligent models, as a more senior engineer would with a more junior engineer.”

BREAKING: Only ~1% of Bending Spoons’ AI requests & tokens goes to frontier models. 99% runs on self-hosted open-weight models. CEO Luca Ferrari on the shift to Sovereign AI: “We end up using open-weight models we self-host, so these are basically free. I mean, there is a little bit of cloud cost, but for 99% of the requests and tokens.” “And the frontier models, generally closed weight, through APIs for maybe 1% of the requests, only for the most complex tasks.” “Sometimes we use them automatically to check the work that got done by the slightly less intelligent models, as a more senior engineer would with a more junior engineer.”

Molly O’Shea

241,276 Aufrufe • vor 7 Tagen

LEAKED AUDIO: In an all-hands meeting on April 30, Mark Zuckerberg tells employees that he's training AI on them ahead of mass layoffs. "The AI models learn from watching really smart people do things... The average intelligence of the people who are at this company is significantly higher than the average set of people that you can get to do tasks. So if we're trying to teach the models coding, for example, then having people internally build tools or solve tasks that help teach the model how to code, we think is going to dramatically increase our model's coding ability faster than what others in the industry have the capability to do, who don't have thousands and thousands of extremely strong engineers at their company."

LEAKED AUDIO: In an all-hands meeting on April 30, Mark Zuckerberg tells employees that he's training AI on them ahead of mass layoffs. "The AI models learn from watching really smart people do things... The average intelligence of the people who are at this company is significantly higher than the average set of people that you can get to do tasks. So if we're trying to teach the models coding, for example, then having people internally build tools or solve tasks that help teach the model how to code, we think is going to dramatically increase our model's coding ability faster than what others in the industry have the capability to do, who don't have thousands and thousands of extremely strong engineers at their company."

More Perfect Union

4,564,376 Aufrufe • vor 4 Monaten

𝐃𝐞𝐬𝐭𝐫𝐚 𝐎𝐂𝐀𝐈 (𝐎𝐧𝐞-𝐂𝐥𝐢𝐜𝐤 𝐀𝐈) 𝐌𝐨𝐝𝐞𝐥𝐬 - $DSYNC 𝐄𝐚𝐫𝐥𝐲 𝐀𝐜𝐜𝐞𝐬𝐬 𝐏𝐫𝐞𝐯𝐢𝐞𝐰 We are training and building a wide range of AI models for the Destra OCAI platform. Today, we will be showcasing an early preview of one of the initial models we are working on—a bullish/bearish prediction model designed to 𝐟𝐨𝐫𝐞𝐜𝐚𝐬𝐭 𝐭𝐡𝐞 𝐦𝐚𝐫𝐤𝐞𝐭 𝐝𝐢𝐫𝐞𝐜𝐭𝐢𝐨𝐧 𝐨𝐟 𝐬𝐩𝐞𝐜𝐢𝐟𝐢𝐜 𝐭𝐨𝐤𝐞𝐧𝐬. This is just one of the initial models we are working on; as we progress, the models will become more complex and intricate. In this behind-the-scenes video, we demonstrate how the model predicts the future market trend of the BTC-USD pair with an 𝐚𝐜𝐜𝐮𝐫𝐚𝐜𝐲 𝐨𝐟 𝟔𝟓%. This is the barebones structure of the models we are training, which will soon be available to users with a single click through the Destra OCAI platform. The OCAI platform includes a wide array of Destra Exclusive AI models, specifically designed to simplify life for crypto audiences, that are envisioned and built from scratch at Destra Labs.

𝐃𝐞𝐬𝐭𝐫𝐚 𝐎𝐂𝐀𝐈 (𝐎𝐧𝐞-𝐂𝐥𝐢𝐜𝐤 𝐀𝐈) 𝐌𝐨𝐝𝐞𝐥𝐬 - $DSYNC 𝐄𝐚𝐫𝐥𝐲 𝐀𝐜𝐜𝐞𝐬𝐬 𝐏𝐫𝐞𝐯𝐢𝐞𝐰 We are training and building a wide range of AI models for the Destra OCAI platform. Today, we will be showcasing an early preview of one of the initial models we are working on—a bullish/bearish prediction model designed to 𝐟𝐨𝐫𝐞𝐜𝐚𝐬𝐭 𝐭𝐡𝐞 𝐦𝐚𝐫𝐤𝐞𝐭 𝐝𝐢𝐫𝐞𝐜𝐭𝐢𝐨𝐧 𝐨𝐟 𝐬𝐩𝐞𝐜𝐢𝐟𝐢𝐜 𝐭𝐨𝐤𝐞𝐧𝐬. This is just one of the initial models we are working on; as we progress, the models will become more complex and intricate. In this behind-the-scenes video, we demonstrate how the model predicts the future market trend of the BTC-USD pair with an 𝐚𝐜𝐜𝐮𝐫𝐚𝐜𝐲 𝐨𝐟 𝟔𝟓%. This is the barebones structure of the models we are training, which will soon be available to users with a single click through the Destra OCAI platform. The OCAI platform includes a wide array of Destra Exclusive AI models, specifically designed to simplify life for crypto audiences, that are envisioned and built from scratch at Destra Labs.

Destra Network

37,150 Aufrufe • vor 2 Jahren

Liquid AI's head of post-training explained how they built a small model that runs on-device under 1 GB in 20 minutes - better than $2500 small-model bootcamps. pick LFM2.5 base -> on-policy preference alignment -> agentic reinforcement learning -> curriculum training -> iterative model merging -> ship a 1B model that reliably calls tools on your phone. That loop is why frontier small models now beat 70B models on the tasks that actually matter. LFM2.5 + on-policy DPO + agentic RL + curriculum training + iterative merging - that's the stack. Watch and save it, then run a 1B agent on your phone tonight.

Liquid AI's head of post-training explained how they built a small model that runs on-device under 1 GB in 20 minutes - better than $2500 small-model bootcamps. pick LFM2.5 base -> on-policy preference alignment -> agentic reinforcement learning -> curriculum training -> iterative model merging -> ship a 1B model that reliably calls tools on your phone. That loop is why frontier small models now beat 70B models on the tasks that actually matter. LFM2.5 + on-policy DPO + agentic RL + curriculum training + iterative merging - that's the stack. Watch and save it, then run a 1B agent on your phone tonight.

h100envy

86,913 Aufrufe • vor 1 Monat

Chinese AI Model DeepSeq: Despite U.S. sanctions restricting access to Nvidia AI chips, Chinese researchers developed DeepSeq, an AI model reportedly more powerful than OpenAI’s. Overcoming Hardware Limitations: With only a limited supply of Nvidia chips, Chinese engineers achieved groundbreaking efficiency optimizations, reducing training resource requirements to just 1/100th of what OpenAI, Microsoft, or Google typically use. Open-Sourcing the Innovation: In a bold move, the developers open-sourced their AI models, making both the code and methodology freely available to the public, significantly reducing the competitive edge of proprietary models like OpenAI’s. Impact on AI Industry: The release of high-quality free alternatives disrupts the market, potentially devaluing OpenAI’s models and forcing American AI companies to rethink their strategies. China’s Rapid AI Innovation: The success of DeepSeq underscores China’s growing self-sufficiency in AI and its ability to leapfrog hardware limitations through advanced software and algorithmic breakthroughs.

Chinese AI Model DeepSeq: Despite U.S. sanctions restricting access to Nvidia AI chips, Chinese researchers developed DeepSeq, an AI model reportedly more powerful than OpenAI’s. Overcoming Hardware Limitations: With only a limited supply of Nvidia chips, Chinese engineers achieved groundbreaking efficiency optimizations, reducing training resource requirements to just 1/100th of what OpenAI, Microsoft, or Google typically use. Open-Sourcing the Innovation: In a bold move, the developers open-sourced their AI models, making both the code and methodology freely available to the public, significantly reducing the competitive edge of proprietary models like OpenAI’s. Impact on AI Industry: The release of high-quality free alternatives disrupts the market, potentially devaluing OpenAI’s models and forcing American AI companies to rethink their strategies. China’s Rapid AI Innovation: The success of DeepSeq underscores China’s growing self-sufficiency in AI and its ability to leapfrog hardware limitations through advanced software and algorithmic breakthroughs.

Johannes Maria

24,512 Aufrufe • vor 1 Jahr

$A 7-million parameter model outperforming models a thousand times its size on tasks like ARC Prize. That's what recursive reasoning unlocks. In this episode of Decoded, YC's Ankit Gupta and Francois Chaubard break down two recent papers on recursive AI models, HRMs and TRMs, that are achieving state-of-the-art results with a fraction of the parameters of today's largest models. They explain why standard LLMs hit a fundamental ceiling on certain reasoning tasks, how recursion at inference time gives small models the compute depth to break through it, and what happens when you combine these ideas with the power of large-scale foundation models. 00:35 - Model Foundations 01:15 - RNN Limits and LLM Contrast 02:36 - Reasoning Limits and Sorting Analogy 04:22 - HRM Paper Introduction 05:25 - HRM Architecture and Intuition 07:36 - HRM Results and Outer Loop 09:46 - TRM Paper Overview 11:20 - TRM Training and Fixed Point 13:30 - Detailed HRM Summary 20:46 - Comparing HRM and TRM 34:45 - Future Outlook$

A 7-million parameter model outperforming models a thousand times its size on tasks like ARC Prize. That's what recursive reasoning unlocks. In this episode of Decoded, YC's Ankit Gupta and Francois Chaubard break down two recent papers on recursive AI models, HRMs and TRMs, that are achieving state-of-the-art results with a fraction of the parameters of today's largest models. They explain why standard LLMs hit a fundamental ceiling on certain reasoning tasks, how recursion at inference time gives small models the compute depth to break through it, and what happens when you combine these ideas with the power of large-scale foundation models. 00:35 - Model Foundations 01:15 - RNN Limits and LLM Contrast 02:36 - Reasoning Limits and Sorting Analogy 04:22 - HRM Paper Introduction 05:25 - HRM Architecture and Intuition 07:36 - HRM Results and Outer Loop 09:46 - TRM Paper Overview 11:20 - TRM Training and Fixed Point 13:30 - Detailed HRM Summary 20:46 - Comparing HRM and TRM 34:45 - Future Outlook

Y Combinator

127,645 Aufrufe • vor 4 Monaten

Next up on First Pass: A conversation with Simon Mo from Inferact on Kimi K3. We discuss: - The basics on the K3 model (it's huge!) - Why open weight models are important - The engineering complexity of working with a model this large - Innovative licensing models And more!

Next up on First Pass: A conversation with Simon Mo from Inferact on Kimi K3. We discuss: - The basics on the K3 model (it's huge!) - Why open weight models are important - The engineering complexity of working with a model this large - Innovative licensing models And more!

Jamin Ball

13,707 Aufrufe • vor 1 Monat

Here's a quick demo of an agentic loop running offline entirely on-device on Apple Watch Series 9. --- We have been spearheading small models and one of the biggest unlock it provides for consumer is having intelligence right at their fingertips, no cloud-round trip = faster, cheaper, private, more control. This also unlocks a whole suite of new use-cases with product specific models - or models built specifically for and shipped alongside an app. We now have full expertise on the entire on-device Apple Intelligence stack. We have pioneered training and running models with Apple native APIs on iPhones, Macs and Watches. Already working with Apple DR team and some of the biggest wearable brands exploring running AI on different form factors. If you're a team exploring this domain, we are here to help. Reach out -> follow Conscious Engines • Studio for consumer AI

Here's a quick demo of an agentic loop running offline entirely on-device on Apple Watch Series 9. --- We have been spearheading small models and one of the biggest unlock it provides for consumer is having intelligence right at their fingertips, no cloud-round trip = faster, cheaper, private, more control. This also unlocks a whole suite of new use-cases with product specific models - or models built specifically for and shipped alongside an app. We now have full expertise on the entire on-device Apple Intelligence stack. We have pioneered training and running models with Apple native APIs on iPhones, Macs and Watches. Already working with Apple DR team and some of the biggest wearable brands exploring running AI on different form factors. If you're a team exploring this domain, we are here to help. Reach out -> follow Conscious Engines • Studio for consumer AI

Kautuk | Conscious Engines

18,296 Aufrufe • vor 1 Monat