正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Try Ternary Bonsai 27B directly on Hugging Face , powered by Together AI

PrismML

28,443 subscribers

19,934 次观看 • 2 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 条评论

Omead Pooladzandi 的头像

Omead Pooladzandi2 个月前

@togethercompute Bonsai 27b goes brrr

zuckashi 的头像

zuckashi1 个月前

@togethercompute any plan for qwen 3.8 27b????

KP bhoomika 的头像

KP bhoomika2 个月前

@togethercompute named it Bonsai because you gotta water it daily just to get a response 🌱

Gamris ∞ 的头像

Gamris ∞2 个月前

@togethercompute When Krea 2 Bonsai?

Coding Guy 的头像

Coding Guy1 个月前

@togethercompute New Bonsai 27B based on the new Qwen3.8 27B when?😍

Jacob Sharp 的头像

Jacob Sharp16 天前

@togethercompute Sounds crazy 👏👏👏

Lily Godden 的头像

Lily Godden16 天前

@togethercompute I’ll try Ternary Bonsai, and feedback 🌳

Calney ⚡︎ 的头像

Calney ⚡︎16 天前

@togethercompute I liked this app layout, so clean... it has Pi behind? click to download the model and use instantly? would be nice! i would like to try <3

nariqwadet 的头像

nariqwadet2 个月前

@togethercompute 8k token limit 🫤

Martin 的头像

Martin1 个月前

@togethercompute bonsai is awsome, it would be really cool if you do the same with the new qwen 3.8 model 27b, that would be next level of awsomeness

相关视频

NEW: Ternary Bonsai 2 just landed on Hugging Face — a 27B reasoning model in ternary weights, built for the agentic era 🤯 9x smaller than FP16 while retaining 98.2% of the intelligence. At <6GB in size, it can even run 100% locally in your browser on WebGPU! Try it yourself 👇

NEW: Ternary Bonsai 2 just landed on Hugging Face — a 27B reasoning model in ternary weights, built for the agentic era 🤯 9x smaller than FP16 while retaining 98.2% of the intelligence. At <6GB in size, it can even run 100% locally in your browser on WebGPU! Try it yourself 👇

Xenova

17,717 次观看 • 17 天前

We also tested Ternary Bonsai 2 27B on the 2026 IMO problems against full-precision Qwen3.8 27B (54GB) and Gemma 4 12B QAT (~7GB). No internet. No tools. 131K-token reasoning budget. Bonsai scored in the upper end of the human bronze-medal range and retained 95% of Qwen’s IMO score, while completing the problems in 70% of the time. Code:

We also tested Ternary Bonsai 2 27B on the 2026 IMO problems against full-precision Qwen3.8 27B (54GB) and Gemma 4 12B QAT (~7GB). No internet. No tools. 131K-token reasoning budget. Bonsai scored in the upper end of the human bronze-medal range and retained 95% of Qwen’s IMO score, while completing the problems in 70% of the time. Code:

PrismML

34,109 次观看 • 10 天前

Here is Ternary Bonsai 27B running an end-to-end agentic workflow locally with Hermes on an NVIDIA GeForce RTX 5090 GPU. The model reasons, calls tools, reads outputs, modifies files, and surfaces insights - all on consumer hardware, while all private files, intermediate states, and iterations remain local.

Here is Ternary Bonsai 27B running an end-to-end agentic workflow locally with Hermes on an NVIDIA GeForce RTX 5090 GPU. The model reasons, calls tools, reads outputs, modifies files, and surfaces insights - all on consumer hardware, while all private files, intermediate states, and iterations remain local.

PrismML

154,473 次观看 • 2 个月前

The phone threshold is even harder than the storage number suggests. A phone exposes only part of its memory to an application, and the model must share that budget with its KV cache, activations, runtime, and the rest of the product. At 3.9 GB, 1-bit Bonsai 27B clears that threshold with room to work, making 27B-class local AI possible on a phone for the first time. (Demo Mode: cached & prefilled image context)

The phone threshold is even harder than the storage number suggests. A phone exposes only part of its memory to an application, and the model must share that budget with its KV cache, activations, runtime, and the rest of the product. At 3.9 GB, 1-bit Bonsai 27B clears that threshold with room to work, making 27B-class local AI possible on a phone for the first time. (Demo Mode: cached & prefilled image context)

PrismML

33,665 次观看 • 2 个月前

This is HUGE, Hugging Face just shipped Inference Providers on the Hub Partnering with Together AI, @FAL, Replicate and SambaNova! Starting today you can access thousands of Models like DeepSeek R1, Llama, Flux, Whisper, and more Directly from Hugging Face!

This is HUGE, Hugging Face just shipped Inference Providers on the Hub Partnering with Together AI, @FAL, Replicate and SambaNova! Starting today you can access thousands of Models like DeepSeek R1, Llama, Flux, Whisper, and more Directly from Hugging Face!

AK

177,217 次观看 • 1 年前

BOOM! Huge update for AI app developers by Hugging Face you can now deploy models directly from Hugging Face with Gradio while choosing your preferred inference provider from SambaNova, Hyperbolic, Together AI, Fireworks, Replicate, fal, Nebius, Novita AI Additionally, you can choose to require Space users to log in, ensuring that inference usage is billed to their accounts

BOOM! Huge update for AI app developers by Hugging Face you can now deploy models directly from Hugging Face with Gradio while choosing your preferred inference provider from SambaNova, Hyperbolic, Together AI, Fireworks, Replicate, fal, Nebius, Novita AI Additionally, you can choose to require Space users to log in, ensuring that inference usage is billed to their accounts

AK

59,352 次观看 • 1 年前

A 27B parameter model used to need a server room. Now it runs on: • iPhone • Android • Mac PrismML's Bonsai makes it possible. • 1-bit weights • 27B params in just 3.9GB • ~90% of full precision quality (PrismML evals) • Better than the 2-bit version at less than half the size The entire Bonsai family is now live in RunAnywhere. • 1.7B to 27B models • 1-bit + 2-bit ternary • Thinking mode On iOS and Mac, Bonsai runs through ggml's llama.cpp and Apple's MLX. On Android, Bonsai runs through ggml's llama.cpp and directly on the Qualcomm Hexagon NPU through QHexRT, our proprietary inference runtime. We built custom silicon kernels to make 1-bit inference on an NPU possible for the first time. Two years ago this needed a data center. Now it thinks in airplane mode. Now available in the RunAnywhere app on the App Store and Google Play.

A 27B parameter model used to need a server room. Now it runs on: • iPhone • Android • Mac PrismML's Bonsai makes it possible. • 1-bit weights • 27B params in just 3.9GB • ~90% of full precision quality (PrismML evals) • Better than the 2-bit version at less than half the size The entire Bonsai family is now live in RunAnywhere. • 1.7B to 27B models • 1-bit + 2-bit ternary • Thinking mode On iOS and Mac, Bonsai runs through ggml's llama.cpp and Apple's MLX. On Android, Bonsai runs through ggml's llama.cpp and directly on the Qualcomm Hexagon NPU through QHexRT, our proprietary inference runtime. We built custom silicon kernels to make 1-bit inference on an NPU possible for the first time. Two years ago this needed a data center. Now it thinks in airplane mode. Now available in the RunAnywhere app on the App Store and Google Play.

RunAnywhere (YC W26)

17,587 次观看 • 2 个月前

Atomic Chat is now on Hugging Face 🤗 We're officially a Local App on the world's biggest AI hub. Run 200,000+ open-weight models from Hugging Face directly on your device - private, local, and open source!

Atomic Chat is now on Hugging Face 🤗 We're officially a Local App on the world's biggest AI hub. Run 200,000+ open-weight models from Hugging Face directly on your device - private, local, and open source!

atomic.chat

29,420 次观看 • 3 个月前

Bonsai 2 dropped hours ago. It's already doing real work in Osaurus. Read two CSVs. Wrote the Python. Drew the chart. Then found the one channel losing money and told me why. 27B. Ternary. Nothing left the Mac.

Bonsai 2 dropped hours ago. It's already doing real work in Osaurus. Read two CSVs. Wrote the Python. Drew the chart. Then found the one channel losing money and told me why. 27B. Ternary. Nothing left the Mac.

Osaurus

34,387 次观看 • 16 天前

Darkbloom is the first model provider to support Ternary Bonsai 2 27B -- concentrated intelligence that fits on your phone. Try it now: First 250 users get 100 million free tokens; PrismML's new flagship model: - a ternary compression of Qwen3.8 27B at 2bits per parameter. - 8.5 GB total, 5x smaller than original - keeps 98.2% of the Qwen's FP16 benchmark performance. - 75% cheaper than Qwen 27B. Qwen3.8 27B already operates comparably with Opus 4.6 and 5.6 Luna on certain tasks. This one does it in the memory of a phone. 262K context, image input, Apache 2.0. From our first run on the network, on a single M5 Max with no caching: - 35 tok/s decode at 1K context, - 31 tok/s at 10K, - 19 tok/s at 50K. But we expect more performance gain coming in a few weeks! That's a full 27B reasoning model running comfortably on any Mac. 1,000+ Macs are serving on Darkbloom right now. Go try it out!! Thank you to Babak Hassibi Sahin Lale Omead Pooladzandi Reza Sadri Tushar Bansal Evan Walters and the whole PrismML team. This is exactly the kind of model Darkbloom was built for. You can read the essay by Bonsai on Why Local AI Matters:

Darkbloom is the first model provider to support Ternary Bonsai 2 27B -- concentrated intelligence that fits on your phone. Try it now: First 250 users get 100 million free tokens; PrismML's new flagship model: - a ternary compression of Qwen3.8 27B at 2bits per parameter. - 8.5 GB total, 5x smaller than original - keeps 98.2% of the Qwen's FP16 benchmark performance. - 75% cheaper than Qwen 27B. Qwen3.8 27B already operates comparably with Opus 4.6 and 5.6 Luna on certain tasks. This one does it in the memory of a phone. 262K context, image input, Apache 2.0. From our first run on the network, on a single M5 Max with no caching: - 35 tok/s decode at 1K context, - 31 tok/s at 10K, - 19 tok/s at 50K. But we expect more performance gain coming in a few weeks! That's a full 27B reasoning model running comfortably on any Mac. 1,000+ Macs are serving on Darkbloom right now. Go try it out!! Thank you to Babak Hassibi Sahin Lale Omead Pooladzandi Reza Sadri Tushar Bansal Evan Walters and the whole PrismML team. This is exactly the kind of model Darkbloom was built for. You can read the essay by Bonsai on Why Local AI Matters:

Gajesh

48,051 次观看 • 16 天前

document AI landed to Hugging Face chat, powered by open-source models (Cohere Command-R+) with tool calling 🛠️✨ explore many tools including image generation, image editing and more at Hugging Chat 🤗💬

document AI landed to Hugging Face chat, powered by open-source models (Cohere Command-R+) with tool calling 🛠️✨ explore many tools including image generation, image editing and more at Hugging Chat 🤗💬

merve

31,986 次观看 • 2 年前

Voxtral can now directly stream audio input into text output. Perfect for: - Live subtitles - Language learning apps - Note-taking tools - And more! Made a demo for you to try directly on hugging face !

Voxtral can now directly stream audio input into text output. Perfect for: - Live subtitles - Language learning apps - Note-taking tools - And more! Made a demo for you to try directly on hugging face !

Mistral for Developers

73,171 次观看 • 8 个月前

JUST IN: Inference Providers at Hugging Face Hub Use @FAL SambaNova Together AI Replicate on Hugging Face to infer gigantic models like DeepSeek R1 from model page or through our client SDKs (Python, JS)/HTTP calls

JUST IN: Inference Providers at Hugging Face Hub Use @FAL SambaNova Together AI Replicate on Hugging Face to infer gigantic models like DeepSeek R1 from model page or through our client SDKs (Python, JS)/HTTP calls

merve

36,902 次观看 • 1 年前

You can now run Nebius AI Studio models directly in Visual Studio Code, just select Hugging Face as your Copilot provider and enter your HF key. ✅ Use models like Qwen, DeepSeek, Kimi, GLM ⚡ Powered by Nebius inference infrastructure Open models, instant APIs, zero setup.

You can now run Nebius AI Studio models directly in Visual Studio Code, just select Hugging Face as your Copilot provider and enter your HF key. ✅ Use models like Qwen, DeepSeek, Kimi, GLM ⚡ Powered by Nebius inference infrastructure Open models, instant APIs, zero setup.

Nebius Token Factory

12,240 次观看 • 1 年前

running PrismML's Bonsai 2 27B on a single RTX 3060. 12GB VRAM. (config below) 220K context, ~35 tok/s decode, ~550 tok/s prefill. a 27B at 1.72 bits/weight, real ternary. two packings: PQ2_0 (7.21 GB) gets 220K and the numbers above, PTQ1_0 (5.95 GB) is smaller so it takes the full 262K but runs slower -- 26 tok/s decode, 260 prefill. max context go PTQ1_0, speed go PQ2_0. needs PrismML's llama.cpp fork, kernels aren't upstream yet. stock llama.cpp rejects these files. day one config, expect it to move. config: llama-server -m Ternary-Bonsai-2-27B-PQ2_0.gguf -ngl 999 -c 220000 -fa on --jinja -np 1 --cache-type-k q4_0 --cache-type-v q4_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --reasoning on

running PrismML's Bonsai 2 27B on a single RTX 3060. 12GB VRAM. (config below) 220K context, ~35 tok/s decode, ~550 tok/s prefill. a 27B at 1.72 bits/weight, real ternary. two packings: PQ2_0 (7.21 GB) gets 220K and the numbers above, PTQ1_0 (5.95 GB) is smaller so it takes the full 262K but runs slower -- 26 tok/s decode, 260 prefill. max context go PTQ1_0, speed go PQ2_0. needs PrismML's llama.cpp fork, kernels aren't upstream yet. stock llama.cpp rejects these files. day one config, expect it to move. config: llama-server -m Ternary-Bonsai-2-27B-PQ2_0.gguf -ngl 999 -c 220000 -fa on --jinja -np 1 --cache-type-k q4_0 --cache-type-v q4_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0 --repeat-penalty 1.0 --reasoning on

Doğukan

161,755 次观看 • 17 天前

1-bit Bonsai 8B running locally on an M4 Pro (MLX) alongside a standard 16-bit 8B model. Same class of model, very different deployment profile: far lower memory use and substantially higher throughput.

PrismML

93,717 次观看 • 6 个月前

We added 600+ new voices on Together AI! Introducing MiniMax Speech 2.8 Turbo on Together AI, an enterprise TTS model for expressive real-time voice agents. AI natives can now deploy MiniMax (official) Speech 2.8 Turbo on Together AI dedicated infrastructure, and try the voices directly in voice finder.

We added 600+ new voices on Together AI! Introducing MiniMax Speech 2.8 Turbo on Together AI, an enterprise TTS model for expressive real-time voice agents. AI natives can now deploy MiniMax (official) Speech 2.8 Turbo on Together AI dedicated infrastructure, and try the voices directly in voice finder.

Together AI

21,814 次观看 • 4 个月前