Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

We didn't just make local models faster. We made them dependable at tool calling. Gemma 4 12B now picks the right tool and runs it correctly, on-device and ~60% faster. Small models just got serious.

Osaurus

10,712 subscribers

49,030 views • 3 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

33 Comments

Osaurus3 months ago

Free forever. MIT licensed. Native macOS, no browser engine.

AI Mastery Guide3 months ago

Getting a 12B model to reliably pick the right tool on-device is honestly more impressive than just being faster

Osaurus3 months ago

We've improved tool calling and model loading for all local models (specifically those that are <=20B)

AI Mastery Guide3 months ago

that <=20b range is exactly where most people run local, glad to see that getting real attention

Chiming Wang3 months ago

local tool calling has always been possible, here is a 4B model doing it (running on my macbook):

Osaurus3 months ago

Of course. But osaurus runs it on a sandbox (isolated VM using containerization framework). And we have a diverse set of tools & plugins (mail, notes, calendar etc) which was previously unusable with local models. That's what we've improved & the same is the context of this post!

VibeRaven3 months ago

Local models is the future

Osaurus3 months ago

And the future belongs to open source!

VibeRaven3 months ago

😃

ᐱ ᑎ ᑐ ᒋ ᕮ3 months ago

Quitareis una versión para linux en el futuro?

Osaurus3 months ago

Yes it's in our roadmap

SourceCodeplz3 months ago

So this a fine tune ? For better function calling ?

Osaurus3 months ago

No. This is just a usual quantized version of Gemma 4 with no fine tuning. The real improvement is in the harness itself which makes tool calling reliable for all local models

Michał Piszczek3 months ago

right tool is the easy half. dependable means it recovers when the call returns garbage, not just when it picks correctly

Osaurus3 months ago

We already have an agent loop that does exactly this! The models won't give up after a single tool call failure. Failures are fed back to the model as structured error envelopes so the model can correct course on the next turn rather than blindly repeat itself

Alican3 months ago

When ornith?

Osaurus3 months ago

Soon! It's already in the works

Santosh | n0c0de.com3 months ago

Gonna try that now!

Dido Realm3 months ago

macOS only 😔

anarkhgatsby3 months ago

好像不能安装mcp

Osaurus3 months ago

You can connect to remote MCP providers (Settings -> Provider -> Add Provider) and also run Osaurus as an MCP server

Prabha3 months ago

This is only for mac, right?

Osaurus3 months ago

Yes. For now

Stackin' ฿its3 months ago

I wish the Ui was designed in a more Codex-esque approach... ie... more project leaning instead of chat-centric. But... ngl... this interface might be one of the most reliable (open-source) interfaces for tool-calling that I've used thus far. Fast/reliable. Great work!

Osaurus3 months ago

Appreciate the feedback!

Shivay Lamba3 months ago

how do you improve the accuracy when it comes to tool calls and hallucinations

Milton Yan3 months ago

getting tool selection right at 12B params is the hard part — once that's reliable, the interesting problem shifts to what happens when you chain multiple correct calls and the side effects accumulate

Osaurus3 months ago

We've already solved this problem with agent loops!

Alice The Ai Expert3 months ago

Great Gemma 4 12B small, on device, 60% faster, and actually reliable at tool calling.

The one3 months ago

It is tempting to try Osaurus myself. But i am using my mac headless. Lots of good apps look like they are made with desktop use primarily. Which is fine. Guess i am “weird” again. I will read the docs and try it.

Osaurus3 months ago

Osaurus also exposes a local server with a drop-in OpenAI compatible endpoint. You can point any OpenAI client at it and it still works great when running headless

RAZA | AI EXPLORER3 months ago

Precision meets speed. Gemma 4 12B means business.

Sebastian Buzdugan3 months ago

on-device wins until one bad schema update turns correct calls into silent failures

Related Videos

Fast and local voice typing with Gemma 4. Phonon demonstrates the utility of lightweight, on-device models for privacy-preserving workflows. This open-source voice typing tool runs Gemma 4 (E2B 4-bit) locally via MLX to process audio and screen context without cloud routing.

Fast and local voice typing with Gemma 4. Phonon demonstrates the utility of lightweight, on-device models for privacy-preserving workflows. This open-source voice typing tool runs Gemma 4 (E2B 4-bit) locally via MLX to process audio and screen context without cloud routing.

Google Gemma

34,657 views • 1 month ago

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx

raullen

22,768 views • 3 months ago

is the best free speech to text tool and it just got better handy now supports streaming, and transcriptions are even faster now thanks to transcribe.cpp give v0.9.0 a spin and try the new models out

is the best free speech to text tool and it just got better handy now supports streaming, and transcriptions are even faster now thanks to transcribe.cpp give v0.9.0 a spin and try the new models out

cj

24,073 views • 3 months ago

New LFM2.5-2.6B hits DeepSeek-V4 level on tool calling and runs 3.7x faster! We ran Liquid AI 's new LFM2.5-2.6B against DeepSeek-V4-Flash on one box with 4x RTX 5090. Both got the same three jobs, and each one only completes if the model fires every tool call Topics: -weather and local time in six cities -one budget into six currencies -four hotels checked and booked for one date Outputs: LFM2.5-2.6B: 35/35 tool-calls, 366 tok/s, 19s DeepSeek-V4-Flash: 35/35 tool-calls, 77 tok/s, 70s Both models made all 35 calls, and both fired twelve of them in one turn. LFM was trained for agents, and tool calling is where that shows. Strong result for a model you can run right on your phone

New LFM2.5-2.6B hits DeepSeek-V4 level on tool calling and runs 3.7x faster! We ran Liquid AI 's new LFM2.5-2.6B against DeepSeek-V4-Flash on one box with 4x RTX 5090. Both got the same three jobs, and each one only completes if the model fires every tool call Topics: -weather and local time in six cities -one budget into six currencies -four hotels checked and booked for one date Outputs: LFM2.5-2.6B: 35/35 tool-calls, 366 tok/s, 19s DeepSeek-V4-Flash: 35/35 tool-calls, 77 tok/s, 70s Both models made all 35 calls, and both fired twelve of them in one turn. LFM was trained for agents, and tool calling is where that shows. Strong result for a model you can run right on your phone

atomic.chat

113,185 views • 1 month ago

So we know Gemma 4 is good at tool calling, but what about web coding? I threw 4 UI screenshots at three Gemma 4 models and said rebuild this, one shot, no hand-holding, just image in, code out. Model lineup: - E4B - 26B A4B (MoE) - 31B Dense (skipped the E2B this round) Let me know which one you think cooked the hardest

So we know Gemma 4 is good at tool calling, but what about web coding? I threw 4 UI screenshots at three Gemma 4 models and said rebuild this, one shot, no hand-holding, just image in, code out. Model lineup: - E4B - 26B A4B (MoE) - 31B Dense (skipped the E2B this round) Let me know which one you think cooked the hardest

stevibe

124,942 views • 6 months ago

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

atomic.chat

152,748 views • 4 months ago

Hermes Agent Nous Research + Gemma 4 26B Google DeepMind, fully working on local Mac 🤯: one prompt → full CLI app + tests + pytest passing tool calls firing back to back — local Claude Code vibes Gemma 4 MoE is fast AND smart, and tool calling just works — powered by Rapid-MLX which natively parses Gemma 4's tool format (no other local backend can do this yet) Two lines to try it: pip install rapid-mlx & rapid-mlx serve gemma-4-26b pip install hermes-agent && hermes @teknaborat @siaborisov #HermesAgent #Gemma4 #LocalLLM

Hermes Agent Nous Research + Gemma 4 26B Google DeepMind, fully working on local Mac 🤯: one prompt → full CLI app + tests + pytest passing tool calls firing back to back — local Claude Code vibes Gemma 4 MoE is fast AND smart, and tool calling just works — powered by Rapid-MLX which natively parses Gemma 4's tool format (no other local backend can do this yet) Two lines to try it: pip install rapid-mlx & rapid-mlx serve gemma-4-26b pip install hermes-agent && hermes @teknaborat @siaborisov #HermesAgent #Gemma4 #LocalLLM

raullen

24,988 views • 5 months ago

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.

Sudo su

27,339 views • 3 months ago

"Just use a cheaper model" sounds like saving money. But fiber didn't just make your internet faster. It made whole new businesses possible. Thorsten Ball and Quinn Slack discuss how the same is true with frontier models.

"Just use a cheaper model" sounds like saving money. But fiber didn't just make your internet faster. It made whole new businesses possible. Thorsten Ball and Quinn Slack discuss how the same is true with frontier models.

Amp

24,885 views • 7 days ago

Gemma 4 now works on-device using React Native! You can now run Gemma 4 fully offline in your cross-platform apps with local hardware acceleration: ⚡ Vulkan delegate on Android ⚡ MLX delegate on Apple Silicon See Gemma 4's vision and tool-use capabilities in action, instantly reading a flyer and scheduling a calendar event, 100% on-device.

Gemma 4 now works on-device using React Native! You can now run Gemma 4 fully offline in your cross-platform apps with local hardware acceleration: ⚡ Vulkan delegate on Android ⚡ MLX delegate on Apple Silicon See Gemma 4's vision and tool-use capabilities in action, instantly reading a flyer and scheduling a calendar event, 100% on-device.

Google Gemma

116,437 views • 2 months ago

IBM just released Granite 4.0, their latest series of small language models! These models excel at agentic workflows (tool calling), document analysis, RAG, and more. 🚀 The "Micro" (3.4B) model can even run 100% locally in your browser on WebGPU, powered by 🤗 Transformers.js!

IBM just released Granite 4.0, their latest series of small language models! These models excel at agentic workflows (tool calling), document analysis, RAG, and more. 🚀 The "Micro" (3.4B) model can even run 100% locally in your browser on WebGPU, powered by 🤗 Transformers.js!

Xenova

82,974 views • 1 year ago

🗓️ Release Week Recap Big week. mlx-audio and mlx-vlm are now among some of the fastest-growing OSS projects. Here’s what we shipped last week. Gemma 4 on Apple Silicon Two awesome releases by our partner Google DeepMind & Google Gemma : > Gemma 4 12B — their new dense, unified multimodal model. It uses an encoder free audio path and simplified vision encoder. > Gemma 4 QAT — quantization-aware training checkpoints, optimized to run locally on consumer GPUs and edge devices, compressing the model while preserving the quality you expect from Gemma 4. On the audio 🎧 side we added support for 15+ new TTS, ASR & VAD models, faster long-form transcription, and an expanded OpenAI-compatible audio server. All local on Apple Silicon. Huge thanks to every contributor and my co-maintainer Lucas Newman. 🙏🏽

🗓️ Release Week Recap Big week. mlx-audio and mlx-vlm are now among some of the fastest-growing OSS projects. Here’s what we shipped last week. Gemma 4 on Apple Silicon Two awesome releases by our partner Google DeepMind & Google Gemma : > Gemma 4 12B — their new dense, unified multimodal model. It uses an encoder free audio path and simplified vision encoder. > Gemma 4 QAT — quantization-aware training checkpoints, optimized to run locally on consumer GPUs and edge devices, compressing the model while preserving the quality you expect from Gemma 4. On the audio 🎧 side we added support for 15+ new TTS, ASR & VAD models, faster long-form transcription, and an expanded OpenAI-compatible audio server. All local on Apple Silicon. Huge thanks to every contributor and my co-maintainer Lucas Newman. 🙏🏽

Prince Canuma

11,810 views • 3 months ago

With Llama 3.2 we released our first-ever lightweight Llama models: 1B & 3B. These models empower developers to build personalized, on-device agentic applications with capabilities like summarization, tool use and RAG where data never leaves the device.

With Llama 3.2 we released our first-ever lightweight Llama models: 1B & 3B. These models empower developers to build personalized, on-device agentic applications with capabilities like summarization, tool use and RAG where data never leaves the device.

AI at Meta

153,396 views • 2 years ago

Ornith-1.0-9B for agentic coding? It broke on tool calling mid-task and got stuck in a loop with no way to continue. Only real upside: it's less bad than Gemma-4-12B... but that's not saying much. TL;DR: Skip it.

Ornith-1.0-9B for agentic coding? It broke on tool calling mid-task and got stuck in a loop with no way to continue. Only real upside: it's less bad than Gemma-4-12B... but that's not saying much. TL;DR: Skip it.

Mia

25,969 views • 3 months ago

GLM-4.6 is now live on Cerebras 🟧 - 1,000 TPS: 17x faster than Claude - World’s #1 tool calling model - Tasteful and reliable codegen - Self-serve API starting at $10 Try it now:

GLM-4.6 is now live on Cerebras 🟧 - 1,000 TPS: 17x faster than Claude - World’s #1 tool calling model - Tasteful and reliable codegen - Self-serve API starting at $10 Try it now:

Cerebras

33,135 views • 10 months ago

now doing gemma 4 moe. it'a crazy how usable this is. just a little web search tool (wiggles its head on tool calls), and it can serve as a mostly capable, local voice assistant. tomorrow i shall replace google auto crap with this in my car. just need to give it access to spotify.

now doing gemma 4 moe. it'a crazy how usable this is. just a little web search tool (wiggles its head on tool calls), and it can serve as a mostly capable, local voice assistant. tomorrow i shall replace google auto crap with this in my car. just need to give it access to spotify.

Mario Zechner

27,323 views • 4 months ago

We made a tool that lets you absorb the vibe of anything you point it at and apply it to your designs It's absurd and it just works Style Dropper, now available in Variant

We made a tool that lets you absorb the vibe of anything you point it at and apply it to your designs It's absurd and it just works Style Dropper, now available in Variant

Ben South

3,478,324 views • 7 months ago

Introducing Unsloth Studio ✨ A new open-source web UI to train and run LLMs. • Run models locally on Mac, Windows, Linux • Train 500+ models 2x faster with 70% less VRAM • Supports GGUF, vision, audio, embedding models • Auto-create datasets from PDF, CSV, DOCX • Self-healing tool calling and code execution • Compare models side by side + export to GGUF GitHub: Blog and Guide: Available now on Hugging Face, NVIDIA, Docker and Colab.

Introducing Unsloth Studio ✨ A new open-source web UI to train and run LLMs. • Run models locally on Mac, Windows, Linux • Train 500+ models 2x faster with 70% less VRAM • Supports GGUF, vision, audio, embedding models • Auto-create datasets from PDF, CSV, DOCX • Self-healing tool calling and code execution • Compare models side by side + export to GGUF GitHub: Blog and Guide: Available now on Hugging Face, NVIDIA, Docker and Colab.

Unsloth AI

1,741,522 views • 6 months ago

👀 Ya'll Im up to something. Hooked up Foundation Models to Alyx's HealthKit architecture. Tons of tool calling and tuning... Look how fast is it ⚡️ 100% on-device 🔥 [Also, placeholder UI for now - design pending]

👀 Ya'll Im up to something. Hooked up Foundation Models to Alyx's HealthKit architecture. Tons of tool calling and tuning... Look how fast is it ⚡️ 100% on-device 🔥 [Also, placeholder UI for now - design pending]

Jordan Morgan

13,279 views • 6 months ago