Загрузка видео...

Не удалось загрузить видео

На главную

Alibaba just open-sourced Qwen 3.6-35B A3B 35B MoE model (only ~3B active), multimodal, agentic coding, Apache 2.0. This isn’t just efficient—it’s built for real workflows. Open models are catching up fast.

13,063 просмотров • 5 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Someone is running Qwen3.8-35B-A3B on a single RTX 3060 with just 12GB of VRAM and 16GB of system RAM. And the reported numbers are surprisingly good. ~50 tok/s decode ~550 tok/s prefill 262K context window 12GB VRAM 16GB system RAM No multi-GPU setup. Just a consumer graphics card and a carefully configured local inference engine. The model is particularly interesting because it’s Qwen3.8 distilled into the Qwen3.6-35B-A3B architecture. That makes it worth investigating beyond raw inference speed. The developer currently uses Ornith-1.5 as the engine behind their Hermes and Pi agents. Now the question is whether this new model can deliver enough quality and reliability to replace it. The next step is testing it as an agent, not just a chatbot. The planned comparison includes four models: • Qwen3.8-35B-A3B • Ornith-1.5-35B • Qwen3.8-27B • Base Qwen3.6-35B-A3B An agentic coding benchmark should help reveal whether the distilled model can follow instructions, modify code, solve problems, and complete multi-step tasks effectively. That distinction matters because high token throughput doesn’t automatically translate into a better coding agent. A model can generate code quickly and still struggle with planning, debugging, tool calls, or completing a task without intervention. The hardware configuration is also worth examining. The developer is using llama-server with the Q4_K_M GGUF quantization, Flash Attention enabled, and 26 MoE layers assigned to CPU processing. The configuration also enables a 262,144-token context window, Q8 KV-cache quantization, and reasoning mode. This is how the setup is configured: llama-server \ -m Qwen3.8-35B-A3B-Q4_K_M.gguf \ -ngl 99 \ --n-cpu-moe 26 \ -c 262144 \ -fa on \ --jinja \ -np 1 \ --cache-type-k q8_0 \ --cache-type-v q8_0 \ --temp 0.6 \ --top-p 0.95 \ --top-k 20 \ --reasoning on One important detail: the model’s 35B parameter count doesn’t mean all 35 billion parameters are activated for every token. The A3B designation indicates roughly 3 billion active parameters per token. That sparse MoE architecture helps explain how a model of this size can deliver high generation speeds on modest hardware. CPU-based expert processing also helps work around the GPU’s limited memory, although the exact trade-offs depend on the workload. The 262K context claim is equally interesting. Supporting a context window that large doesn’t mean every request at that length will maintain the same speed. Prompt length, KV-cache memory, and CPU/GPU placement can all affect performance. The developer’s next test is particularly useful: graphing decode speed against context length. That should help show how performance changes as the prompt grows, rather than presenting one speed figure without the conditions behind it. If the coding benchmark also holds up, this could become a serious option for people who want capable local agents without investing in expensive GPUs. For now, the reported inference numbers are promising. The real test is whether Qwen3.8-35B-A3B can match or outperform Ornith-1.5-35B on actual agentic coding tasks. Fast local inference gets your attention. Reliable autonomous task completion is what earns a model a place in the stack.

FHILY👑

19,254 просмотров • 2 дней назад

Alibaba just dropped Qwen3.5-397B-A17B and there's a lot to unpack. 397B params, 17B active per forward pass. Sparse MoE done right. But the real story isn't the size—it's the architecture choices. The MoE Design Most MoE models feel like bolt-ons. Qwen 3.5's sparse activation is native—only 4.3% of parameters fire per token. That's how you get trillion-parameter-class performance without trillion-parameter inference costs. The 0.8 RMB/million tokens pricing isn't subsidized; it's structurally earned. Native Multimodal, Not Glued-On This is a vision-language model from the ground up. Heterogeneous architecture—separate processing pipelines for text, image, video that fuse early. Not a vision encoder slapped onto an LLM. The result: 90.8 on OmniDocBench, 79.0 on MMMU-Pro. Document understanding and visual reasoning without the usual brittleness. The Context Window Reality Qwen3.5-Plus (the hosted version) ships with 1M tokens by default. That's not a marketing number—they're actually positioning it for long-document workflows. With built-in adaptive tool use, it's clearly aimed at agentic automation, not just chat. What Actually Impressed Me • FP8 native pipeline: ~50% activation memory reduction • Async RL framework for continuous refinement—training and inference workloads separated • 201 languages (up from 119), 250k vocab for better low-resource encoding • Apache 2.0 license. Full weights on HuggingFace and ModelScope. The Benchmark Context 76.4 on SWE-bench Verified puts it in the range where it can handle real debugging workflows. 72.9 on BFCL v4 for agentic tool use. 88.4 on GPQA Diamond. These aren't SOTA in isolation, but the breadth is unusual—strong across reasoning, coding, multimodal, and agentic tasks. The Honest Caveat I haven't stress-tested the 1M context for needle-in-haystack retrieval yet. And "native multimodal" claims need real-world torture testing—PDFs with tables, charts, mixed layouts. Benchmarks are benchmarks. Bottom Line This isn't just another model release. It's a bet on efficient scale: big model capabilities, small active compute, open weights. At 1/18th the cost of Gemini 3 Pro, it's going to force pricing conversations across the board.

Bo Wang

13,221 просмотров • 7 месяцев назад

NVIDIA Releases Audex (Nemotron-Labs-Audex-30B-A3B): A Unified Audio-Text LLM That Preserves the Text Intelligence of Its Backbone Most unified audio models pay a text tax. Add audio output, and reasoning benchmarks drop — even when the only new output is speech. NVIDIA just released one that doesn't. Audex (Nemotron-Labs-Audex-30B-A3B) is a 30B MoE with 3B active parameters, built on the text-only Nemotron-Cascade-2-30B-A3B backbone. Audio inputs are projected into the text embedding space. Text tokens and quantized audio tokens are then generated the same way, inside one MoE decoder — no thinker–talker split, no stacked cascade. Here's what's actually interesting: → One model, audio in and out: understanding, ASR, translation, TTS, text-to-audio, speech-to-speech → Text holds vs its own backbone: IMO AnswerBench 81.1 vs 79.3, MMLU-Redux 86.4 vs 86.3 → Beats text-only Qwen3.5-35B-A3B on several tasks: LiveCodeBench v6 85.3 vs 74.6, IFBench 77.8 vs 70.2 → The usual tax, for contrast: Qwen3-Omni-30B-A3B-Thinking drops to 60.4 on HMMT vs 71.4 for its text backbone → 6.82 WER on OpenASR, ahead of Step-Audio-R1.1-33B (7.91) and Qwen3-Omni-Thinking (8.00) → Two codecs: X-Codec2 for speech (50 tok/s, FSQ, 65,536 codebook), X-Codec for general audio (200 tok/s, 4 flattened RVQ layers) — the only strong open model generating general audio beyond speech Full analysis: Paper: Model weights: NVIDIA AI NVIDIA Wei Ping

Marktechpost AI

28,301 просмотров • 3 месяцев назад

Alibaba just released a coding model that hits 82 percent on SWE-Bench Verified. That is the highest score ever published for an open-source model. The weights are free. The license is Apache 2.0. You can run it today. The model is Qwen 4 Coder 32B. Here is what 82 percent on SWE-Bench Verified actually means. SWE-Bench Verified tests whether an AI can autonomously resolve real bugs pulled from real production GitHub repositories. Not synthetic exercises. Real open-source projects that real teams depend on. A model gets a bug report, reads the code, writes a fix, and either passes the test suite or it does not. At 82 percent, Qwen 4 Coder 32B resolves 82 out of every 100 real production bugs it is given. Without a human guiding it. On code it has never seen before. For comparison: Qwen 4 Coder 32B: 82 percent SWE-Bench Verified. Open source. Apache 2.0. Claude Fable 5: 80.3 percent SWE-Bench Pro. $10 input / $50 output per million tokens. Currently suspended. GPT-5.6 Sol: Competitive on Terminal-Bench. $5 input / $30 output per million tokens. An open-weight model that you can download and run for free just beat both of them on the benchmark designed to measure real software engineering capability. Here is the architecture. Qwen 4 Coder 32B is a 32 billion parameter dense model. Not a Mixture-of-Experts. Every parameter is active on every request. This matters for inference: a dense 32B model runs on 22 gigabytes of VRAM, which fits on a single high-end consumer GPU or a MacBook Pro with 64GB of unified memory. The smaller variant, Qwen 4 Coder 4B, runs at approximately 135 tokens per second on an M5 Max and fits inside 8 gigabytes of RAM. For a model with usable coding capability, that is a new bar for what fits in a single laptop. The training methodology continued Alibaba's approach of reinforcement learning on verifiable coding tasks. The model gets rewarded when its code passes tests. It gets penalized when it fails. Over millions of training steps, the model learns to write code that actually runs rather than code that looks plausible. License: Apache 2.0. Full commercial use. No attribution requirement. No revenue threshold. No monthly active user ceiling. Weights: Hugging Face, available today. Runs on: vLLM, Ollama, SGLang, and any standard GGUF-compatible inference engine. Qwen 4 32B also runs at approximately 135 tokens per second on an M5 Max chip, setting a new bar for what a sub-8GB model can do on Apple Silicon. The open-source coding model just beat the best closed-source model in the world on the benchmark designed to test whether AI can actually do software engineering. The weights are free. The subscription is optional. Source: Autom8Labs AI Insight July 2026, State of Open Source LLMs June 2026, Kunal Ganglani blog June 2026.

Harman

41,278 просмотров • 3 месяцев назад