Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

We didn't just make local models faster. We made them dependable at tool calling. Gemma 4 12B now picks the right tool and runs it correctly, on-device and ~60% faster. Small models just got serious.

Osaurus

10,712 subscribers

49,030 görüntüleme • 3 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

33 Yorum

Osaurus profil fotoğrafı

Osaurus3 ay önce

Free forever. MIT licensed. Native macOS, no browser engine.

AI Mastery Guide profil fotoğrafı

AI Mastery Guide3 ay önce

Getting a 12B model to reliably pick the right tool on-device is honestly more impressive than just being faster

Osaurus profil fotoğrafı

Osaurus3 ay önce

We've improved tool calling and model loading for all local models (specifically those that are <=20B)

AI Mastery Guide profil fotoğrafı

AI Mastery Guide3 ay önce

that <=20b range is exactly where most people run local, glad to see that getting real attention

Chiming Wang profil fotoğrafı

Chiming Wang3 ay önce

local tool calling has always been possible, here is a 4B model doing it (running on my macbook):

Osaurus profil fotoğrafı

Osaurus3 ay önce

Of course. But osaurus runs it on a sandbox (isolated VM using containerization framework). And we have a diverse set of tools & plugins (mail, notes, calendar etc) which was previously unusable with local models. That's what we've improved & the same is the context of this post!

VibeRaven profil fotoğrafı

VibeRaven3 ay önce

Local models is the future

Osaurus profil fotoğrafı

Osaurus3 ay önce

And the future belongs to open source!

VibeRaven profil fotoğrafı

VibeRaven3 ay önce

😃

ᐱ ᑎ ᑐ ᒋ ᕮ profil fotoğrafı

ᐱ ᑎ ᑐ ᒋ ᕮ3 ay önce

Quitareis una versión para linux en el futuro?

Osaurus profil fotoğrafı

Osaurus3 ay önce

Yes it's in our roadmap

SourceCodeplz profil fotoğrafı

SourceCodeplz3 ay önce

So this a fine tune ? For better function calling ?

Osaurus profil fotoğrafı

Osaurus3 ay önce

No. This is just a usual quantized version of Gemma 4 with no fine tuning. The real improvement is in the harness itself which makes tool calling reliable for all local models

Michał Piszczek profil fotoğrafı

Michał Piszczek3 ay önce

right tool is the easy half. dependable means it recovers when the call returns garbage, not just when it picks correctly

Osaurus profil fotoğrafı

Osaurus3 ay önce

We already have an agent loop that does exactly this! The models won't give up after a single tool call failure. Failures are fed back to the model as structured error envelopes so the model can correct course on the next turn rather than blindly repeat itself

Alican profil fotoğrafı

Alican3 ay önce

When ornith?

Osaurus profil fotoğrafı

Osaurus3 ay önce

Soon! It's already in the works

Santosh | n0c0de.com profil fotoğrafı

Santosh | n0c0de.com3 ay önce

Gonna try that now!

Dido Realm profil fotoğrafı

Dido Realm3 ay önce

macOS only 😔

anarkhgatsby profil fotoğrafı

anarkhgatsby3 ay önce

好像不能安装mcp

Osaurus profil fotoğrafı

Osaurus3 ay önce

You can connect to remote MCP providers (Settings -> Provider -> Add Provider) and also run Osaurus as an MCP server

Prabha profil fotoğrafı

Prabha3 ay önce

This is only for mac, right?

Osaurus profil fotoğrafı

Osaurus3 ay önce

Yes. For now

Stackin' ฿its profil fotoğrafı

Stackin' ฿its3 ay önce

I wish the Ui was designed in a more Codex-esque approach... ie... more project leaning instead of chat-centric. But... ngl... this interface might be one of the most reliable (open-source) interfaces for tool-calling that I've used thus far. Fast/reliable. Great work!

Osaurus profil fotoğrafı

Osaurus3 ay önce

Appreciate the feedback!

Shivay Lamba profil fotoğrafı

Shivay Lamba3 ay önce

how do you improve the accuracy when it comes to tool calls and hallucinations

Milton Yan profil fotoğrafı

Milton Yan3 ay önce

getting tool selection right at 12B params is the hard part — once that's reliable, the interesting problem shifts to what happens when you chain multiple correct calls and the side effects accumulate

Osaurus profil fotoğrafı

Osaurus3 ay önce

We've already solved this problem with agent loops!

Alice The Ai Expert profil fotoğrafı

Alice The Ai Expert3 ay önce

Great Gemma 4 12B small, on device, 60% faster, and actually reliable at tool calling.

The one profil fotoğrafı

The one3 ay önce

It is tempting to try Osaurus myself. But i am using my mac headless. Lots of good apps look like they are made with desktop use primarily. Which is fine. Guess i am “weird” again. I will read the docs and try it.

Osaurus profil fotoğrafı

Osaurus3 ay önce

Osaurus also exposes a local server with a drop-in OpenAI compatible endpoint. You can point any OpenAI client at it and it still works great when running headless

RAZA | AI EXPLORER profil fotoğrafı

RAZA | AI EXPLORER3 ay önce

Precision meets speed. Gemma 4 12B means business.

Sebastian Buzdugan profil fotoğrafı

Sebastian Buzdugan3 ay önce

on-device wins until one bad schema update turns correct calls into silent failures

Benzer Videolar

Fast and local voice typing with Gemma 4. Phonon demonstrates the utility of lightweight, on-device models for privacy-preserving workflows. This open-source voice typing tool runs Gemma 4 (E2B 4-bit) locally via MLX to process audio and screen context without cloud routing.

Fast and local voice typing with Gemma 4. Phonon demonstrates the utility of lightweight, on-device models for privacy-preserving workflows. This open-source voice typing tool runs Gemma 4 (E2B 4-bit) locally via MLX to process audio and screen context without cloud routing.

Google Gemma

34,657 görüntüleme • 1 ay önce

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx

🔴 The Pain: Running local MLX models is incredibly fast and private. But let's be real - testing tool calling via terminal is clunky, and there's zero good UI for it. 🟢 The Fix: rapid-mlx share Just ONE command gives you a polished web chat + seamless tool calling (works beautifully with gemma-4-12b-qat). We are proud to be the ONLY MLX inference engine in the community shipping this. ⚡️ 👇 Try it now: brew install raullenchai/rapid-mlx/rapid-mlx

raullen

22,768 görüntüleme • 3 ay önce

is the best free speech to text tool and it just got better handy now supports streaming, and transcriptions are even faster now thanks to transcribe.cpp give v0.9.0 a spin and try the new models out

is the best free speech to text tool and it just got better handy now supports streaming, and transcriptions are even faster now thanks to transcribe.cpp give v0.9.0 a spin and try the new models out

cj

24,073 görüntüleme • 3 ay önce

New LFM2.5-2.6B hits DeepSeek-V4 level on tool calling and runs 3.7x faster! We ran Liquid AI 's new LFM2.5-2.6B against DeepSeek-V4-Flash on one box with 4x RTX 5090. Both got the same three jobs, and each one only completes if the model fires every tool call Topics: -weather and local time in six cities -one budget into six currencies -four hotels checked and booked for one date Outputs: LFM2.5-2.6B: 35/35 tool-calls, 366 tok/s, 19s DeepSeek-V4-Flash: 35/35 tool-calls, 77 tok/s, 70s Both models made all 35 calls, and both fired twelve of them in one turn. LFM was trained for agents, and tool calling is where that shows. Strong result for a model you can run right on your phone

New LFM2.5-2.6B hits DeepSeek-V4 level on tool calling and runs 3.7x faster! We ran Liquid AI 's new LFM2.5-2.6B against DeepSeek-V4-Flash on one box with 4x RTX 5090. Both got the same three jobs, and each one only completes if the model fires every tool call Topics: -weather and local time in six cities -one budget into six currencies -four hotels checked and booked for one date Outputs: LFM2.5-2.6B: 35/35 tool-calls, 366 tok/s, 19s DeepSeek-V4-Flash: 35/35 tool-calls, 77 tok/s, 70s Both models made all 35 calls, and both fired twelve of them in one turn. LFM was trained for agents, and tool calling is where that shows. Strong result for a model you can run right on your phone

atomic.chat

113,185 görüntüleme • 1 ay önce

So we know Gemma 4 is good at tool calling, but what about web coding? I threw 4 UI screenshots at three Gemma 4 models and said rebuild this, one shot, no hand-holding, just image in, code out. Model lineup: - E4B - 26B A4B (MoE) - 31B Dense (skipped the E2B this round) Let me know which one you think cooked the hardest

So we know Gemma 4 is good at tool calling, but what about web coding? I threw 4 UI screenshots at three Gemma 4 models and said rebuild this, one shot, no hand-holding, just image in, code out. Model lineup: - E4B - 26B A4B (MoE) - 31B Dense (skipped the E2B this round) Let me know which one you think cooked the hardest

stevibe

124,942 görüntüleme • 6 ay önce

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

New Google Gemma 4 12B claims near-26B performance - we tested both! We ran both models locally on one RTX 4090 and gave each the same task: write a self-contained HTML5 canvas animation with real physics in one file without libraries. Three scenes - a Galton board, two blocks colliding off a wall, and a chaotic triple pendulum Outputs: Gemma 4 26B-A4B: 15 GB VRAM usage, 6.9k tokens, 138 tok/s Gemma 4 12B: 9 GB VRAM usage, 8.9k tokens, 80 tok/s Same Gemma 4 family, but the 26B-A4B won every scene and ran ~1.7x faster - on just 4B active params. The 12B stayed very close though, on almost half the VRAM - which makes it the ideal model for a 16 GB laptop

atomic.chat

152,748 görüntüleme • 4 ay önce

Hermes Agent Nous Research + Gemma 4 26B Google DeepMind, fully working on local Mac 🤯: one prompt → full CLI app + tests + pytest passing tool calls firing back to back — local Claude Code vibes Gemma 4 MoE is fast AND smart, and tool calling just works — powered by Rapid-MLX which natively parses Gemma 4's tool format (no other local backend can do this yet) Two lines to try it: pip install rapid-mlx & rapid-mlx serve gemma-4-26b pip install hermes-agent && hermes @teknaborat @siaborisov #HermesAgent #Gemma4 #LocalLLM

Hermes Agent Nous Research + Gemma 4 26B Google DeepMind, fully working on local Mac 🤯: one prompt → full CLI app + tests + pytest passing tool calls firing back to back — local Claude Code vibes Gemma 4 MoE is fast AND smart, and tool calling just works — powered by Rapid-MLX which natively parses Gemma 4's tool format (no other local backend can do this yet) Two lines to try it: pip install rapid-mlx & rapid-mlx serve gemma-4-26b pip install hermes-agent && hermes @teknaborat @siaborisov #HermesAgent #Gemma4 #LocalLLM

raullen

24,988 görüntüleme • 5 ay önce

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.

small local model that falls apart in bloated agents like openclaw just runs like a wild horse in hermes agent. and that's not even my line, someone else called it that, i've just been quietly pointing people at this harness for months because it held up on everything i threw at it, 3b models all the way to one trillion params. watch this happen on my own machine. i pointed hermes agent at a local http endpoint, gemma 4 12b on my 3090 llama.cpp server, and it auto-detected the model and started working immediately. no config wrestling, no broken tool calls, no babysitting the output format, i typed in a url and it just went. the whole clip is exactly that, start to finish, no errors, no retries, butter smooth. and the tool calling, the one thing that quietly breaks most local setups, works here like it's nothing. it's not the model that's flaky, it's the harness around it. hermes agent is the first agent i've run that actually gets that right. one url, one local model on one card, and it runs like a wild horse.

Sudo su

27,339 görüntüleme • 3 ay önce

"Just use a cheaper model" sounds like saving money. But fiber didn't just make your internet faster. It made whole new businesses possible. Thorsten Ball and Quinn Slack discuss how the same is true with frontier models.

"Just use a cheaper model" sounds like saving money. But fiber didn't just make your internet faster. It made whole new businesses possible. Thorsten Ball and Quinn Slack discuss how the same is true with frontier models.

Amp

24,830 görüntüleme • 6 gün önce

Gemma 4 now works on-device using React Native! You can now run Gemma 4 fully offline in your cross-platform apps with local hardware acceleration: ⚡ Vulkan delegate on Android ⚡ MLX delegate on Apple Silicon See Gemma 4's vision and tool-use capabilities in action, instantly reading a flyer and scheduling a calendar event, 100% on-device.

Gemma 4 now works on-device using React Native! You can now run Gemma 4 fully offline in your cross-platform apps with local hardware acceleration: ⚡ Vulkan delegate on Android ⚡ MLX delegate on Apple Silicon See Gemma 4's vision and tool-use capabilities in action, instantly reading a flyer and scheduling a calendar event, 100% on-device.

Google Gemma

116,437 görüntüleme • 2 ay önce

IBM just released Granite 4.0, their latest series of small language models! These models excel at agentic workflows (tool calling), document analysis, RAG, and more. 🚀 The "Micro" (3.4B) model can even run 100% locally in your browser on WebGPU, powered by 🤗 Transformers.js!

IBM just released Granite 4.0, their latest series of small language models! These models excel at agentic workflows (tool calling), document analysis, RAG, and more. 🚀 The "Micro" (3.4B) model can even run 100% locally in your browser on WebGPU, powered by 🤗 Transformers.js!

Xenova

82,974 görüntüleme • 1 yıl önce

🗓️ Release Week Recap Big week. mlx-audio and mlx-vlm are now among some of the fastest-growing OSS projects. Here’s what we shipped last week. Gemma 4 on Apple Silicon Two awesome releases by our partner Google DeepMind & Google Gemma : > Gemma 4 12B — their new dense, unified multimodal model. It uses an encoder free audio path and simplified vision encoder. > Gemma 4 QAT — quantization-aware training checkpoints, optimized to run locally on consumer GPUs and edge devices, compressing the model while preserving the quality you expect from Gemma 4. On the audio 🎧 side we added support for 15+ new TTS, ASR & VAD models, faster long-form transcription, and an expanded OpenAI-compatible audio server. All local on Apple Silicon. Huge thanks to every contributor and my co-maintainer Lucas Newman. 🙏🏽

🗓️ Release Week Recap Big week. mlx-audio and mlx-vlm are now among some of the fastest-growing OSS projects. Here’s what we shipped last week. Gemma 4 on Apple Silicon Two awesome releases by our partner Google DeepMind & Google Gemma : > Gemma 4 12B — their new dense, unified multimodal model. It uses an encoder free audio path and simplified vision encoder. > Gemma 4 QAT — quantization-aware training checkpoints, optimized to run locally on consumer GPUs and edge devices, compressing the model while preserving the quality you expect from Gemma 4. On the audio 🎧 side we added support for 15+ new TTS, ASR & VAD models, faster long-form transcription, and an expanded OpenAI-compatible audio server. All local on Apple Silicon. Huge thanks to every contributor and my co-maintainer Lucas Newman. 🙏🏽

Prince Canuma

11,810 görüntüleme • 3 ay önce

With Llama 3.2 we released our first-ever lightweight Llama models: 1B & 3B. These models empower developers to build personalized, on-device agentic applications with capabilities like summarization, tool use and RAG where data never leaves the device.

With Llama 3.2 we released our first-ever lightweight Llama models: 1B & 3B. These models empower developers to build personalized, on-device agentic applications with capabilities like summarization, tool use and RAG where data never leaves the device.

AI at Meta

153,396 görüntüleme • 2 yıl önce

Ornith-1.0-9B for agentic coding? It broke on tool calling mid-task and got stuck in a loop with no way to continue. Only real upside: it's less bad than Gemma-4-12B... but that's not saying much. TL;DR: Skip it.

Ornith-1.0-9B for agentic coding? It broke on tool calling mid-task and got stuck in a loop with no way to continue. Only real upside: it's less bad than Gemma-4-12B... but that's not saying much. TL;DR: Skip it.

Mia

25,969 görüntüleme • 3 ay önce

GLM-4.6 is now live on Cerebras 🟧 - 1,000 TPS: 17x faster than Claude - World’s #1 tool calling model - Tasteful and reliable codegen - Self-serve API starting at $10 Try it now:

GLM-4.6 is now live on Cerebras 🟧 - 1,000 TPS: 17x faster than Claude - World’s #1 tool calling model - Tasteful and reliable codegen - Self-serve API starting at $10 Try it now:

Cerebras

33,135 görüntüleme • 10 ay önce

now doing gemma 4 moe. it'a crazy how usable this is. just a little web search tool (wiggles its head on tool calls), and it can serve as a mostly capable, local voice assistant. tomorrow i shall replace google auto crap with this in my car. just need to give it access to spotify.

now doing gemma 4 moe. it'a crazy how usable this is. just a little web search tool (wiggles its head on tool calls), and it can serve as a mostly capable, local voice assistant. tomorrow i shall replace google auto crap with this in my car. just need to give it access to spotify.

Mario Zechner

27,323 görüntüleme • 4 ay önce

We made a tool that lets you absorb the vibe of anything you point it at and apply it to your designs It's absurd and it just works Style Dropper, now available in Variant

We made a tool that lets you absorb the vibe of anything you point it at and apply it to your designs It's absurd and it just works Style Dropper, now available in Variant

Ben South

3,478,117 görüntüleme • 7 ay önce

Introducing Unsloth Studio ✨ A new open-source web UI to train and run LLMs. • Run models locally on Mac, Windows, Linux • Train 500+ models 2x faster with 70% less VRAM • Supports GGUF, vision, audio, embedding models • Auto-create datasets from PDF, CSV, DOCX • Self-healing tool calling and code execution • Compare models side by side + export to GGUF GitHub: Blog and Guide: Available now on Hugging Face, NVIDIA, Docker and Colab.

Introducing Unsloth Studio ✨ A new open-source web UI to train and run LLMs. • Run models locally on Mac, Windows, Linux • Train 500+ models 2x faster with 70% less VRAM • Supports GGUF, vision, audio, embedding models • Auto-create datasets from PDF, CSV, DOCX • Self-healing tool calling and code execution • Compare models side by side + export to GGUF GitHub: Blog and Guide: Available now on Hugging Face, NVIDIA, Docker and Colab.

Unsloth AI

1,741,522 görüntüleme • 6 ay önce

👀 Ya'll Im up to something. Hooked up Foundation Models to Alyx's HealthKit architecture. Tons of tool calling and tuning... Look how fast is it ⚡️ 100% on-device 🔥 [Also, placeholder UI for now - design pending]

👀 Ya'll Im up to something. Hooked up Foundation Models to Alyx's HealthKit architecture. Tons of tool calling and tuning... Look how fast is it ⚡️ 100% on-device 🔥 [Also, placeholder UI for now - design pending]

Jordan Morgan

13,279 görüntüleme • 6 ay önce