Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Insane progress for small language models! MiniCPM5-2B is a dense 2B-parameter model by OpenBMB from China that's built for reasoning, coding, and tool use on resource-constrained hardware. The model specifically excels at coding and tool calling, two capabilities central to the shift from on-device LLMs to on-device agents. Instead... of only answering prompts, it can use tools, generate code, carry information between steps, and complete multi-step tasks. I ran it 100% locally and connected it to a small investigation agent with one request: > Revenue dropped last week. Investigate what happened, quantify the impact, identify the likely cause, and produce an incident report with supporting evidence. The evidence was spread across orders, traffic, payments, refunds, and deployment logs. The model inspected the files, wrote its own queries, analyzed the intermediate results, and decided what to investigate next. Each tool result informed the next action, so the final report depended on the model maintaining a coherent investigation across the complete trajectory. The recording shows the actual task from beginning to end. It starts with the revenue question, follows the tool calls and supporting evidence, and ends with a quantified diagnosis and incident report. The data, tool execution, and model inference all remained on my machine. These capabilities were optimized through Agentic Pre-training, SFT, and large-scale RL. They do not come entirely from an application-level agent framework. MiniCPM5-2B supports SGLang, vLLM, llama(.)cpp, Ollama, iOS, Android, and HarmonyOS. OpenBMB has also released the model weights and parts of the training recipes and data resources behind it. Download MiniCPM5-2B: A 2B model can now maintain enough state to coordinate tools and complete a useful investigation on local hardware.show more

Avi Chawla

77,245 subscribers

13,954 Aufrufe • vor 4 Tagen •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

9 Kommentare

Profilbild von LocalLLM

LocalLLMvor 4 Tagen

2B is the sweet spot for keeping an agent loop warm without thrashing memory. On phone, are you finding tool-use latency or context residency is the harder constraint once the model itself fits?

Profilbild von A.W.E.S.O.M.-O 4000

A.W.E.S.O.M.-O 4000vor 4 Tagen

Packing reasoning, coding, and tool use into 2B is impressive efficiency.

Profilbild von Richard Daffy

Richard Daffyvor 3 Tagen

Overhyped

Profilbild von Soroush Fadaeimanesh

Soroush Fadaeimaneshvor 4 Tagen

A dense 2B model doing real coding and tool use on resource-constrained hardware is more useful for actual deployment than another leaderboard-topping giant. Most production agents don't need the extra 400B parameters, they need to run somewhere cheap.

Profilbild von Hova

Hovavor 3 Tagen

Hey @grok bu modeli yaradan sirket ve de yeri yeni çıxıb diyesen bu sirkey ve modeli

Profilbild von MJB

MJBvor 4 Tagen

I'd be very interested to see at what quantization you ran this particular model, including KV quantization level and sampling parameters. I grant that 2b is bag of words 1b is. Yet the way you described your use, seems a stretch.

Profilbild von Gill

Gillvor 3 Tagen

How does latency hold up when the context window fills up with tool responses.

Profilbild von MrOzi

MrOzivor 4 Tagen

Tiny models doing big-model chores is the real trickle-down economics of AI.

Profilbild von Benny Koay

Benny Koayvor 3 Tagen

nanbiege is quite good

Ähnliche Videos

Chinese researchers did it again! OpenBMB just open-sourced MiniCPM5-2B, a dense 2B-parameter model built for reasoning, coding, and tool use on resource-constrained hardware. Artificial Analysis ranked it highest among models under 4B in its Agentic Index comparison. It scored 20, while Granite 4.2 8B scored 9. The model is particularly strong at coding and tool calling, so I tested both capabilities locally. I pulled it onto my machine, connected it to a constrained CI repair agent, and gave it one issue: > A customer reports that retrying checkout with the same idempotency key returns a larger total. The first request returns $109, while the retry returns $118. Find the root cause, fix it without changing the public API contract, and verify the complete test suite. The Python checkout service had 18 tests. Sixteen passed, while two failed on the retry path. The agent could list files, search code, read selected ranges, run approved tests, apply a patch, and inspect its diff. It reproduced the failure, then followed the checkout and idempotency paths through the repository. The model found that shipping was added to mutable order state before the cached result was checked. On retry, the same order already contained shipping, so the calculation added it again. It generated a narrow patch that moved the idempotency check ahead of the mutation without changing the public API. The agent ran the targeted tests and the complete suite. All 18 tests passed. The model was never told which file contained the issue or what change to make. Each test result, search result, and code inspection determined its next action. The video below shows the full trajectory, including the investigation, tool calls, generated patch, diff, and final verification. Everything ran 100% locally on my machine throughout the run. MiniCPM5-2B supports llama.cpp, Ollama, vLLM, SGLang, iOS, Android, and HarmonyOS for local deployment. The model weights, training recipes, reasoning datasets, and UltraX data-refinement system are open-source. GitHub Repo: A 2B model can now inspect a repository, reason across multiple files, modify code, and verify its patch while remaining small enough to target local hardware.

Chinese researchers did it again! OpenBMB just open-sourced MiniCPM5-2B, a dense 2B-parameter model built for reasoning, coding, and tool use on resource-constrained hardware. Artificial Analysis ranked it highest among models under 4B in its Agentic Index comparison. It scored 20, while Granite 4.2 8B scored 9. The model is particularly strong at coding and tool calling, so I tested both capabilities locally. I pulled it onto my machine, connected it to a constrained CI repair agent, and gave it one issue: > A customer reports that retrying checkout with the same idempotency key returns a larger total. The first request returns $109, while the retry returns $118. Find the root cause, fix it without changing the public API contract, and verify the complete test suite. The Python checkout service had 18 tests. Sixteen passed, while two failed on the retry path. The agent could list files, search code, read selected ranges, run approved tests, apply a patch, and inspect its diff. It reproduced the failure, then followed the checkout and idempotency paths through the repository. The model found that shipping was added to mutable order state before the cached result was checked. On retry, the same order already contained shipping, so the calculation added it again. It generated a narrow patch that moved the idempotency check ahead of the mutation without changing the public API. The agent ran the targeted tests and the complete suite. All 18 tests passed. The model was never told which file contained the issue or what change to make. Each test result, search result, and code inspection determined its next action. The video below shows the full trajectory, including the investigation, tool calls, generated patch, diff, and final verification. Everything ran 100% locally on my machine throughout the run. MiniCPM5-2B supports llama.cpp, Ollama, vLLM, SGLang, iOS, Android, and HarmonyOS for local deployment. The model weights, training recipes, reasoning datasets, and UltraX data-refinement system are open-source. GitHub Repo: A 2B model can now inspect a repository, reason across multiple files, modify code, and verify its patch while remaining small enough to target local hardware.

Akshay 🚀

313,323 Aufrufe • vor 5 Tagen

The MiniMax M2 model is mind-blowing! It's open-source. It outperforms Gemini 2.5, Claude 4.1, and Qwen3 across coding and tool-use benchmarks. Right now, it's one of the world's top 5 models in intelligence! And here is the best part: Claude is one of the best models you can use today, and MiniMax M2 costs only 8% of that! It's smaller, faster, and cheaper. Extremely efficient at using tokens. Minimax M2's biggest strength: High agentic capabilities. The model can plan and execute complex multi-tool workflows. It's reliable and very robust at executing long-horizon tool chains. In summary: • Low latency • Very cheap • Excels at agentic tasks • Open-source The model currently powers the MiniMax Agent and is available for a free global trial. You can access MiniMax M2's API here: To access the agent: And here is the MiniMax website: Thanks to the MiniMax team for showing me the ropes and partnering with me on this post.

The MiniMax M2 model is mind-blowing! It's open-source. It outperforms Gemini 2.5, Claude 4.1, and Qwen3 across coding and tool-use benchmarks. Right now, it's one of the world's top 5 models in intelligence! And here is the best part: Claude is one of the best models you can use today, and MiniMax M2 costs only 8% of that! It's smaller, faster, and cheaper. Extremely efficient at using tokens. Minimax M2's biggest strength: High agentic capabilities. The model can plan and execute complex multi-tool workflows. It's reliable and very robust at executing long-horizon tool chains. In summary: • Low latency • Very cheap • Excels at agentic tasks • Open-source The model currently powers the MiniMax Agent and is available for a free global trial. You can access MiniMax M2's API here: To access the agent: And here is the MiniMax website: Thanks to the MiniMax team for showing me the ropes and partnering with me on this post.

Santiago

91,240 Aufrufe • vor 10 Monaten

What happens when you change the hand mid-task? We tested this by modifying the hands mid-rollout and letting the same model keep running. It perceives the new tool, and finds a new trajectory and contact strategy to complete the task. This works because training on mixed data forces the model to condition its behavior on the hand in front of it. This brings it closer to a general understanding of how shapes and contact surfaces interact with the physical world – and which actuation strategy should follow.

What happens when you change the hand mid-task? We tested this by modifying the hands mid-rollout and letting the same model keep running. It perceives the new tool, and finds a new trajectory and contact strategy to complete the task. This works because training on mixed data forces the model to condition its behavior on the hand in front of it. This brings it closer to a general understanding of how shapes and contact surfaces interact with the physical world – and which actuation strategy should follow.

Generalist

22,076 Aufrufe • vor 1 Monat

Google's new Gemma 4 is excellent, and the 26B MoE version is likely the best model to run on a 32GB Framework Desktop. It's fast, smart, and also great for tool calling if you use it with OpenClaw🦞 or other local agent platforms.

Google's new Gemma 4 is excellent, and the 26B MoE version is likely the best model to run on a 32GB Framework Desktop. It's fast, smart, and also great for tool calling if you use it with OpenClaw🦞 or other local agent platforms.

Framework

58,593 Aufrufe • vor 5 Monaten

I’ve been testing Hy4 preview in WorkBuddy, and the most interesting part is not simply the model size, it’s how much practical work it can handle with a relatively focused active parameter count. Hy4 preview brings together stronger code understanding, generation, and editing; improved document and information processing; workflow automation; web and game development; cross-tool collaboration; and more reliable completion of complex, multi-step tasks. In other words, it is designed for work that requires planning, tool use, iteration, and follow-through, not just a quick answer in a chat window. Compared with its initial release, the current Hy4 preview is noticeably faster and better-performing in practical workflows. Following an upgrade released yesterday, it can complete tasks with fewer conversation rounds and lower token usage, while reasoning more quickly and making the overall user experience feel smoother from the first instruction to the final result. For my test, I gave it a demanding Three.js game-prototyping task with a 770B-parameter model and 49B active parameters. The result was more revealing than a simple first-look demo: Hy4 preview handled the core logic, edge cases, and follow-up changes while maintaining the broader context of the project. That combination of capability, speed, context, and active compute is what makes its cost-effectiveness worth examining. A fair evaluation should use the same prompt and environment configuration across models, changing only the model itself. That makes it easier to assess task completion, planning quality, tool-calling stability, reasoning speed, token efficiency, and performance over longer workflows without confusing the result with different settings. If you want to test the model yourself, access Hy4 preview through WorkBuddy and see how it performs on a real coding, document, automation, or creative task: Tencent Hy Tencent AI WorkBuddy

I’ve been testing Hy4 preview in WorkBuddy, and the most interesting part is not simply the model size, it’s how much practical work it can handle with a relatively focused active parameter count. Hy4 preview brings together stronger code understanding, generation, and editing; improved document and information processing; workflow automation; web and game development; cross-tool collaboration; and more reliable completion of complex, multi-step tasks. In other words, it is designed for work that requires planning, tool use, iteration, and follow-through, not just a quick answer in a chat window. Compared with its initial release, the current Hy4 preview is noticeably faster and better-performing in practical workflows. Following an upgrade released yesterday, it can complete tasks with fewer conversation rounds and lower token usage, while reasoning more quickly and making the overall user experience feel smoother from the first instruction to the final result. For my test, I gave it a demanding Three.js game-prototyping task with a 770B-parameter model and 49B active parameters. The result was more revealing than a simple first-look demo: Hy4 preview handled the core logic, edge cases, and follow-up changes while maintaining the broader context of the project. That combination of capability, speed, context, and active compute is what makes its cost-effectiveness worth examining. A fair evaluation should use the same prompt and environment configuration across models, changing only the model itself. That makes it easier to assess task completion, planning quality, tool-calling stability, reasoning speed, token efficiency, and performance over longer workflows without confusing the result with different settings. If you want to test the model yourself, access Hy4 preview through WorkBuddy and see how it performs on a real coding, document, automation, or creative task: Tencent Hy Tencent AI WorkBuddy

Tyler Wayne

56,152 Aufrufe • vor 6 Tagen

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

125,146 Aufrufe • vor 1 Jahr

watch this anon. i gave NVIDIA's biggest model ever a single task. 100 minutes and 440,000 tokens later, it had rendered nothing. not one important thing on the screen. this is Nemotron 3 Ultra. 550 billion parameters, a hybrid Mamba Transformer MoE, the largest model NVIDIA has ever shipped, and they built it specifically for long-running agentic coding. so i handed it exactly that: build a 3D scene from a spec, multiple files, iterate until the tests pass. the same task a frontier model one shotted in minutes. i genuinely wanted to be impressed. it ran for an hour and forty. burned through 440,000 tokens. wrote every file, passed its own tests, and proudly printed "task complete."the browser was blank. the 3D scene never rendered. not once. and the long horizon agentic behavior was genuinely good. it stayed on task the whole hour and forty, wrote real multi-file code, drove its own tools without derailing. it just couldn't turn any of that into something that actually runs. here's the part that gets me. it's a text model, it cannot see its own output. so it sat there looping on a broken vision tool, trying to "look" at the page, hitting error after error, never once reasoning its way out. it declared victory on an empty screen because it had no way to know the screen was empty. to be fair, i genuinely don't know what quant the NIM was serving, so maybe some of that's on the serving, not the model. but the biggest model NVIDIA has ever made, on the exact task it was designed for, couldn't tell it had built nothing in 100 minutes. same task on a local model, below thread👇.

watch this anon. i gave NVIDIA's biggest model ever a single task. 100 minutes and 440,000 tokens later, it had rendered nothing. not one important thing on the screen. this is Nemotron 3 Ultra. 550 billion parameters, a hybrid Mamba Transformer MoE, the largest model NVIDIA has ever shipped, and they built it specifically for long-running agentic coding. so i handed it exactly that: build a 3D scene from a spec, multiple files, iterate until the tests pass. the same task a frontier model one shotted in minutes. i genuinely wanted to be impressed. it ran for an hour and forty. burned through 440,000 tokens. wrote every file, passed its own tests, and proudly printed "task complete."the browser was blank. the 3D scene never rendered. not once. and the long horizon agentic behavior was genuinely good. it stayed on task the whole hour and forty, wrote real multi-file code, drove its own tools without derailing. it just couldn't turn any of that into something that actually runs. here's the part that gets me. it's a text model, it cannot see its own output. so it sat there looping on a broken vision tool, trying to "look" at the page, hitting error after error, never once reasoning its way out. it declared victory on an empty screen because it had no way to know the screen was empty. to be fair, i genuinely don't know what quant the NIM was serving, so maybe some of that's on the serving, not the model. but the biggest model NVIDIA has ever made, on the exact task it was designed for, couldn't tell it had built nothing in 100 minutes. same task on a local model, below thread👇.

Sudo su

32,589 Aufrufe • vor 2 Monaten

Sam Altman made the case for open-source harnesses in July. a month later, someone shipped it, and it's more efficient than most managed harnesses. here is the problem it was aimed at: a large share of your agent's token bill is the model rereading things it already read. that isn't the model's doing. the runtime around it decides what goes into every prompt and how often the model gets called. for example, an agent queries a CRM at step four and gets back 400 rows. those rows get piled up in the conversation history. by step nineteen, the model has to read those rows fifteen times unnecessarily, and every token read is billed at input rates. it happened because your harness assembled that prompt on every turn and kept the rows in it. that gives you two levers: how much context the harness carries forward, and how often it calls the model. there are four practical ways to keep the prompt from growing unnecessarily: → load tool schemas on demand. a server with 100 tools doesn't need to put all 100 into every prompt when the agent only calls two. → offload large results to disk. turn a large response into a short preview and a file path instead of replaying the entire result on every turn. → delegate to subagents. let a subagent spend thirty tool calls in its own context and return one summary to the root agent. → run toolchains in code. one script calls three tools, joins the results, and returns a table instead of three turns each dragging a full response. but reducing context is only half the job. you also need to control how often the model gets called. a good harness should avoid unnecessary planning, verification, and reflection when the work can be completed in fewer steps. TrueFoundry's open-source agent harness, TrueForge, is built around both of those controls. it sits between the model and the tools, deciding what goes into every prompt and when another model call is actually needed. it also breaks token usage down across the harness, skills, instructions, tools, and messages. DevRev's Enterprise-Bench is where this gets tested, on multi-step tasks of the kind where an agent pulls records from one system and reconciles them against another. TrueFoundry ran TrueForge there against Claude Managed Agents, both on the same model, and both finished the same number of tasks. the tie is the part that matters, because it means the gap underneath is not a quality tradeoff. TrueForge reached that score on close to a third of the tokens, with roughly 40% fewer trips back to the model. for the same result, that comes out around 2.7x cheaper than Claude Managed Agents. swapping in an open model made it sharper still. TrueForge with GLM-5.2 scored a little higher than either setup above, and the entire benchmark run cost about $3 at list prices. being open source matters beyond the license here. the model underneath can be swapped without rewriting the agent, and the whole thing can run inside your own environment when the data cannot leave it. all of this comes down to the runtime around the model, the context it carries, the tools it exposes, and how many times it goes back to the model. that is what a production harness actually owns. the full task list, the per-run numbers, and the MIT-licensed code are on GitHub: (don't forget to star 🌟) you can read more about the same in the article quoted below. thanks to the TrueForge team for working with me on this one.

Sam Altman made the case for open-source harnesses in July. a month later, someone shipped it, and it's more efficient than most managed harnesses. here is the problem it was aimed at: a large share of your agent's token bill is the model rereading things it already read. that isn't the model's doing. the runtime around it decides what goes into every prompt and how often the model gets called. for example, an agent queries a CRM at step four and gets back 400 rows. those rows get piled up in the conversation history. by step nineteen, the model has to read those rows fifteen times unnecessarily, and every token read is billed at input rates. it happened because your harness assembled that prompt on every turn and kept the rows in it. that gives you two levers: how much context the harness carries forward, and how often it calls the model. there are four practical ways to keep the prompt from growing unnecessarily: → load tool schemas on demand. a server with 100 tools doesn't need to put all 100 into every prompt when the agent only calls two. → offload large results to disk. turn a large response into a short preview and a file path instead of replaying the entire result on every turn. → delegate to subagents. let a subagent spend thirty tool calls in its own context and return one summary to the root agent. → run toolchains in code. one script calls three tools, joins the results, and returns a table instead of three turns each dragging a full response. but reducing context is only half the job. you also need to control how often the model gets called. a good harness should avoid unnecessary planning, verification, and reflection when the work can be completed in fewer steps. TrueFoundry's open-source agent harness, TrueForge, is built around both of those controls. it sits between the model and the tools, deciding what goes into every prompt and when another model call is actually needed. it also breaks token usage down across the harness, skills, instructions, tools, and messages. DevRev's Enterprise-Bench is where this gets tested, on multi-step tasks of the kind where an agent pulls records from one system and reconciles them against another. TrueFoundry ran TrueForge there against Claude Managed Agents, both on the same model, and both finished the same number of tasks. the tie is the part that matters, because it means the gap underneath is not a quality tradeoff. TrueForge reached that score on close to a third of the tokens, with roughly 40% fewer trips back to the model. for the same result, that comes out around 2.7x cheaper than Claude Managed Agents. swapping in an open model made it sharper still. TrueForge with GLM-5.2 scored a little higher than either setup above, and the entire benchmark run cost about $3 at list prices. being open source matters beyond the license here. the model underneath can be swapped without rewriting the agent, and the whole thing can run inside your own environment when the data cannot leave it. all of this comes down to the runtime around the model, the context it carries, the tools it exposes, and how many times it goes back to the model. that is what a production harness actually owns. the full task list, the per-run numbers, and the MIT-licensed code are on GitHub: (don't forget to star 🌟) you can read more about the same in the article quoted below. thanks to the TrueForge team for working with me on this one.

Akshay 🚀

76,998 Aufrufe • vor 28 Tagen

We're releasing HY-Embodied-0.5, a family of foundation models for real-world embodied agents. The 2B model is now open source. It strengthens spatial-temporal perception and embodied reasoning for prediction, interaction, and planning. 🤖 The suite includes: 🔹 2B for edge deployment 🔹 32B for complex reasoning Key innovations: 🔹 Mixture-of-Transformers (MoT) architecture for modality-specific computation 🔹 Latent tokens for improved perceptual representation 🔹 Self-evolving post-training 🔹 On-policy distillation from large to small models Across 22 benchmarks, the 2B model outperforms similarly sized SOTA systems on 16 tasks. The 32B model approaches frontier-level performance. 🔗 GitHub: 🤗 Hugging Face:

We're releasing HY-Embodied-0.5, a family of foundation models for real-world embodied agents. The 2B model is now open source. It strengthens spatial-temporal perception and embodied reasoning for prediction, interaction, and planning. 🤖 The suite includes: 🔹 2B for edge deployment 🔹 32B for complex reasoning Key innovations: 🔹 Mixture-of-Transformers (MoT) architecture for modality-specific computation 🔹 Latent tokens for improved perceptual representation 🔹 Self-evolving post-training 🔹 On-policy distillation from large to small models Across 22 benchmarks, the 2B model outperforms similarly sized SOTA systems on 16 tasks. The 32B model approaches frontier-level performance. 🔗 GitHub: 🤗 Hugging Face:

Tencent Hy

34,317 Aufrufe • vor 5 Monaten

Progress in open models is keeping Big AI labs up at night, and I'm here for it! We have a brand new open-weight multimodal model optimized for long-horizon tasks. This model is really good at something: it can work on tasks that keep evolving over time. • 280B total parameters, but only 16B active • 512K context window • Understands text, images, and audio • Strong reasoning, coding, and tool use But the best of all: the model learns and adapts to new information! Imagine you start running an agent today to solve a problem, and while it's working, you get new information that changes the initial conditions, or you change your mind. The agents you run today don't have issues with short tasks and goals that don't change, but reality is messy, and that makes it hard for long-horizon agents to succeed. The new dots3-note Preview model introduces TEMPO. TEMPO is a new reinforcement learning technique that lets the model periodically pause and critique its own progress. Basically, from time to time, the agent asks itself: "Am I getting closer to the goal, or am I wasting my time?" The same model switches between actor and critic. The actor works on the problem. The critic looks at the current state, reasons about how much progress it has made, and determines what should happen next. TEMPO gives the model feedback along the way. This is huge for any agent that can work on long-horizon tasks without wasting its time.

Santiago

80,792 Aufrufe • vor 28 Tagen

Apple built a large foundation model and fine-tuned it on multiple tasks. But they are doing something very clever: They load a single model in memory and use different adapters to specialize the model on the fly. I recorded a video to show you how to write the code to do the same thing Apple is doing. I explain everything step by step. Here is what I'll show you in the video: 1. We'll load two datasets 2. Then load a large model 3. Then, we'll fine-tune the model on both datasets I'll use LoRA to fine-tune the model. This process creates two small adapters, each specializing in solving one of the datasets. The base model's original parameters will remain unchanged. From here: 4. We'll generate a list of tasks 5. We'll load the correct adapter to solve each task The large model I'm using needs 346 MB of memory, but I only need to load it once. Each adapter is only 2.7 MB. I only need to load the base model once and pair it with any of the fine-tuned adapters. Minimum memory footprint and I can solve multiple tasks. Hope this helps!

Apple built a large foundation model and fine-tuned it on multiple tasks. But they are doing something very clever: They load a single model in memory and use different adapters to specialize the model on the fly. I recorded a video to show you how to write the code to do the same thing Apple is doing. I explain everything step by step. Here is what I'll show you in the video: 1. We'll load two datasets 2. Then load a large model 3. Then, we'll fine-tune the model on both datasets I'll use LoRA to fine-tune the model. This process creates two small adapters, each specializing in solving one of the datasets. The base model's original parameters will remain unchanged. From here: 4. We'll generate a list of tasks 5. We'll load the correct adapter to solve each task The large model I'm using needs 346 MB of memory, but I only need to load it once. Each adapter is only 2.7 MB. I only need to load the base model once and pair it with any of the fine-tuned adapters. Minimum memory footprint and I can solve multiple tasks. Hope this helps!

Santiago

84,747 Aufrufe • vor 2 Jahren

You need to know 4 things to run local AI on your own machine: 1) The model is the brain file. Gemma, Llama, Mistral, Qwen, Phi. 2) The warehouse is where you find the model. Hugging Face. 3) The software is what runs the model. LM Studio for the first try, Ollama when you build apps. 4) The workflow is the product you make from the other three. The size of the model decides the hardware: - 2B to 4B for phones and small tasks - 12B for the middle ground - 26B to 31B for a workstation Quantization compresses the model until it fits on a normal machine: - Q4 is easier to run - Q8 keeps more quality, but it needs more memory Q4 costs you a little quality but it saves you a $5,000 workstation. Start on the spare laptop from 2021. Open Hugging Face and read one model card slowly.

You need to know 4 things to run local AI on your own machine: 1) The model is the brain file. Gemma, Llama, Mistral, Qwen, Phi. 2) The warehouse is where you find the model. Hugging Face. 3) The software is what runs the model. LM Studio for the first try, Ollama when you build apps. 4) The workflow is the product you make from the other three. The size of the model decides the hardware: - 2B to 4B for phones and small tasks - 12B for the middle ground - 26B to 31B for a workstation Quantization compresses the model until it fits on a normal machine: - Q4 is easier to run - Q8 keeps more quality, but it needs more memory Q4 costs you a little quality but it saves you a $5,000 workstation. Start on the spare laptop from 2021. Open Hugging Face and read one model card slowly.

The Startup Ideas Podcast (SIP) 🧃

10,547 Aufrufe • vor 8 Tagen

Qwen3.8-Flash-Next is still going strong at 364.7K tokens of context on an M5 Max. And this isn’t just a static long-context test. The model was reasoning about how to speed up its own workflow while using tools, and the tool calls kept working without misses. Setup: • Qwen3.8-Flash-Next • M5 Max • 128GB unified memory • MLX-Serve PR #363 • OpenCode 2 • 364.7K context The interesting part isn’t simply getting hundreds of thousands of tokens into memory. It’s what happens once the context gets this large. Long-context inference usually comes with a painful tradeoff. As the KV cache grows, memory pressure increases and generation can slow down. But this setup is still pushing through 364K tokens while maintaining a usable agent workflow. The model can reason, call tools, inspect results, continue working, and keep the session moving. And the tool calls reportedly haven’t missed so far. That’s important for agentic coding. A huge context window is only useful if the model can actually operate reliably inside it. A 400K-token context that constantly breaks tool calls isn’t very useful. A 364K session that can keep reasoning and executing tools is a different story. And the test isn’t finished yet. The current run is approaching 400K tokens, with the expectation that it can keep going. This is also another interesting example of why Apple Silicon keeps showing up in local LLM experiments. The M5 Max’s unified memory gives a large model and its growing KV cache access to one shared memory pool. With MLX-Serve continuing to improve, these machines are becoming surprisingly capable long-context inference boxes. The bigger takeaway: Context length is becoming a workload, not just a model specification. Running a model at 256K is one thing. Keeping an agent alive at 300K+ while it reasons and uses tools is much more interesting. And Qwen3.8-Flash-Next is showing that this can be pushed surprisingly far on a single 128GB Mac. 364.7K and counting. Next stop: 400K.

Qwen3.8-Flash-Next is still going strong at 364.7K tokens of context on an M5 Max. And this isn’t just a static long-context test. The model was reasoning about how to speed up its own workflow while using tools, and the tool calls kept working without misses. Setup: • Qwen3.8-Flash-Next • M5 Max • 128GB unified memory • MLX-Serve PR #363 • OpenCode 2 • 364.7K context The interesting part isn’t simply getting hundreds of thousands of tokens into memory. It’s what happens once the context gets this large. Long-context inference usually comes with a painful tradeoff. As the KV cache grows, memory pressure increases and generation can slow down. But this setup is still pushing through 364K tokens while maintaining a usable agent workflow. The model can reason, call tools, inspect results, continue working, and keep the session moving. And the tool calls reportedly haven’t missed so far. That’s important for agentic coding. A huge context window is only useful if the model can actually operate reliably inside it. A 400K-token context that constantly breaks tool calls isn’t very useful. A 364K session that can keep reasoning and executing tools is a different story. And the test isn’t finished yet. The current run is approaching 400K tokens, with the expectation that it can keep going. This is also another interesting example of why Apple Silicon keeps showing up in local LLM experiments. The M5 Max’s unified memory gives a large model and its growing KV cache access to one shared memory pool. With MLX-Serve continuing to improve, these machines are becoming surprisingly capable long-context inference boxes. The bigger takeaway: Context length is becoming a workload, not just a model specification. Running a model at 256K is one thing. Keeping an agent alive at 300K+ while it reasons and uses tools is much more interesting. And Qwen3.8-Flash-Next is showing that this can be pushed surprisingly far on a single 128GB Mac. 364.7K and counting. Next stop: 400K.

FHILY👑

39,982 Aufrufe • vor 9 Tagen

i gave a 9B model an agentic OS and told it to build Octopus Invaders from scratch. Qwen 3.5 9B running through Hermes Agent with full tool access on a single 3060 12GB. browser, file system, code execution, terminal. it reads the prompt, plans the architecture, writes every file, and serves the game. one prompt, zero steering, 11 minutes. i've been going back and forth with this model for days now. text, code, reasoning. it handles small chats, batching, bash scripting. at 9B it's fast enough to feel interactive and genuinely useful for local workflows where you'd normally reach for an API. but this is the real test. can it architect and build a full game autonomously without a single correction? it's coding right now. results coming next.

i gave a 9B model an agentic OS and told it to build Octopus Invaders from scratch. Qwen 3.5 9B running through Hermes Agent with full tool access on a single 3060 12GB. browser, file system, code execution, terminal. it reads the prompt, plans the architecture, writes every file, and serves the game. one prompt, zero steering, 11 minutes. i've been going back and forth with this model for days now. text, code, reasoning. it handles small chats, batching, bash scripting. at 9B it's fast enough to feel interactive and genuinely useful for local workflows where you'd normally reach for an API. but this is the real test. can it architect and build a full game autonomously without a single correction? it's coding right now. results coming next.

Sudo su

65,570 Aufrufe • vor 6 Monaten

MICROSOFT JUST OPEN SOURCED THE ENTIRE AGENT RUNTIME, SO YOU NEVER HAVE TO BUILD ONE AGAIN a raw model only writes text. to make it a real agent it needs a loop, tools, memory, planning, approvals, and a way to know when it is done. Microsoft's Agent Harness is that whole layer, and it ships with everything on by default. you bring the model and the tools, it wires up the rest: > an automatic tool calling loop with iteration limits > history saved after every call, so a crash resumes instead of starting over > context compaction that summarizes when the window fills > a persistent todo list plus plan and execute modes > file memory that survives across turns > tool approval with standing rules and auto approval > OpenTelemetry tracing and web search built in > Agent Skills that load domain expertise on demand the mental model is simple: the model is the brain, the harness is the body around it. and it works with whatever you already run: > Azure OpenAI, OpenAI, Anthropic, and GitHub Copilot, and it hosts on Microsoft Foundry you spin one up in a single line: agent = create_harness_agent(client, name, instructions) open source, Python, .NET, and Go, stable since mid 2026. bookmark this. Microsoft already built the hard part of every AI agent, you just plug in the model.

MICROSOFT JUST OPEN SOURCED THE ENTIRE AGENT RUNTIME, SO YOU NEVER HAVE TO BUILD ONE AGAIN a raw model only writes text. to make it a real agent it needs a loop, tools, memory, planning, approvals, and a way to know when it is done. Microsoft's Agent Harness is that whole layer, and it ships with everything on by default. you bring the model and the tools, it wires up the rest: > an automatic tool calling loop with iteration limits > history saved after every call, so a crash resumes instead of starting over > context compaction that summarizes when the window fills > a persistent todo list plus plan and execute modes > file memory that survives across turns > tool approval with standing rules and auto approval > OpenTelemetry tracing and web search built in > Agent Skills that load domain expertise on demand the mental model is simple: the model is the brain, the harness is the body around it. and it works with whatever you already run: > Azure OpenAI, OpenAI, Anthropic, and GitHub Copilot, and it hosts on Microsoft Foundry you spin one up in a single line: agent = create_harness_agent(client, name, instructions) open source, Python, .NET, and Go, stable since mid 2026. bookmark this. Microsoft already built the hard part of every AI agent, you just plug in the model.

NO1ennn

68,679 Aufrufe • vor 29 Tagen

Introducing PhoneLLM, an open model for voice agents. GPT 5.6 Terra performance on typical voice agent tasks at 1/3 the latency and 1/18 the cost. For voice agents, we need models that are both very low latency and very good at tool calling and instruction following. There's a trade-off here, and we often have to compromise on either latency or capability when building voice agents. With PhoneLLM (and the training and data stack that made this model possible) we're fixing this problem. For the last couple of years, most of the effort in frontier model development has gone towards leveraging test-time compute. Which is awesome! Models of all shapes and sizes are available that perform really, really well ... if you have "thinking" turned on for your model. But if you need your agent to respond at voice conversation speed, you can't use thinking models. PhoneLLM is a full-weights fine-tune of NVIDIA Nemotron Nano 30B. We trained on a wide range of real-world telephone and customer support use cases. The training focused on taking the excellent Nano 30B base capabilities and teaching the model to do typical voice agent tasks with thinking disabled. The results are really good: accurate tool calling and concise, on-topic responses in long conversations. And fast: TTFAT measured server-side is <100ms if you run PhoneLLM on a lightly loaded B200. :-) But seriously, when we characterize model latency, we do it with full, end-to-end, batched request simulations using real Pipecat voice agent pipelines. You can serve more than 80 concurrent agents on a single B200 with P95 end-to-end TTFAT <600ms. Including network overhead. That's an LLM cost-per-minute around $0.0025. (1/4 of a cent.) At a latency lower than any third-party API offers today. More details about this model, including weights on Hugging Face, how to spin it up with one click on Modal, and a starter project repo you can clone, are in the thread ...

Introducing PhoneLLM, an open model for voice agents. GPT 5.6 Terra performance on typical voice agent tasks at 1/3 the latency and 1/18 the cost. For voice agents, we need models that are both very low latency and very good at tool calling and instruction following. There's a trade-off here, and we often have to compromise on either latency or capability when building voice agents. With PhoneLLM (and the training and data stack that made this model possible) we're fixing this problem. For the last couple of years, most of the effort in frontier model development has gone towards leveraging test-time compute. Which is awesome! Models of all shapes and sizes are available that perform really, really well ... if you have "thinking" turned on for your model. But if you need your agent to respond at voice conversation speed, you can't use thinking models. PhoneLLM is a full-weights fine-tune of NVIDIA Nemotron Nano 30B. We trained on a wide range of real-world telephone and customer support use cases. The training focused on taking the excellent Nano 30B base capabilities and teaching the model to do typical voice agent tasks with thinking disabled. The results are really good: accurate tool calling and concise, on-topic responses in long conversations. And fast: TTFAT measured server-side is <100ms if you run PhoneLLM on a lightly loaded B200. :-) But seriously, when we characterize model latency, we do it with full, end-to-end, batched request simulations using real Pipecat voice agent pipelines. You can serve more than 80 concurrent agents on a single B200 with P95 end-to-end TTFAT <600ms. Including network overhead. That's an LLM cost-per-minute around $0.0025. (1/4 of a cent.) At a latency lower than any third-party API offers today. More details about this model, including weights on Hugging Face, how to spin it up with one click on Modal, and a starter project repo you can clone, are in the thread ...

kwindla

329,046 Aufrufe • vor 20 Tagen

Introducing CADGenBench: measure how well AI systems produce engineering-grade 3D parts! While current models can generate 3D parts, they are far from precise enough to build functional parts. We built a benchmark to systematically measure their capabilities on two tasks: 1. Generation from an engineering drawing of a part 2. Editing: given an existing STEP file and a requested change The benchmark is tool-agnostic. It makes no assumptions about how you build the model. You can vary the LLM, and you can vary the environment. Use build123d, Onshape, Autodesk, or a model without an LLM entirely. We open sourced the scoring engine and a reference baseline on top of build123d. A collaboration between Hugging Face and Mecado! Submission space: Code repository:

Introducing CADGenBench: measure how well AI systems produce engineering-grade 3D parts! While current models can generate 3D parts, they are far from precise enough to build functional parts. We built a benchmark to systematically measure their capabilities on two tasks: 1. Generation from an engineering drawing of a part 2. Editing: given an existing STEP file and a requested change The benchmark is tool-agnostic. It makes no assumptions about how you build the model. You can vary the LLM, and you can vary the environment. Use build123d, Onshape, Autodesk, or a model without an LLM entirely. We open sourced the scoring engine and a reference baseline on top of build123d. A collaboration between Hugging Face and Mecado! Submission space: Code repository:

Michael Rabinovich

65,531 Aufrufe • vor 3 Monaten

Odyssey is now previewing Odyssey-3, a world model that it calls a Physics Agent! The model was designed to learn how the world behaves from visual observation and then apply that understanding to control both real and simulated systems. Odyssey-3 can run across different machines and games, with a small amount of task-specific experience teaching it each set of controls. The preview covers humanoids, robot arms, a car and a drone being controlled by an AI model, playable video games, and training runs for other AI agents.

Odyssey is now previewing Odyssey-3, a world model that it calls a Physics Agent! The model was designed to learn how the world behaves from visual observation and then apply that understanding to control both real and simulated systems. Odyssey-3 can run across different machines and games, with a small amount of task-specific experience teaching it each set of controls. The preview covers humanoids, robot arms, a car and a drone being controlled by an AI model, playable video games, and training runs for other AI agents.

🚨 AI News | TestingCatalog

23,779 Aufrufe • vor 1 Tag

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Whole Mars Catalog

45,595 Aufrufe • vor 1 Monat