`transformers` + `torchao` quantization + `torch.compile` for faster inference... speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :show more

Mac Studio M2 Ultra is best Apple Silicon ever... (so far!) 🔥75 t/s and 0 throttling 🔥 Even after 20 mins of uninterrupted token generation with Apple MLX and mlx-community/Meta-Llama-3.1-8B-Instruct-8bit model!show more

Wild. Llama 4 Maverick just went Beast Mode 🔥... SambaNova partnered with AI at Meta to deliver the fastest Llama 4 inference on the market. Even the mighty GroqInc can’t keep up! Let’s dive in ↓show more

smolagents is now available in ai-gradio pip install ai-gradio[smolagents]==0.2.1... then simply do import gradio as gr import ai_gradio gr.load( name='smolagents:meta-llama/Llama-3.1-8B-Instruct', src=ai_gradio.registry).launch()show more

We are thrilled to be a launch partner for... Meta Llama 3. Experience Llama 3 now at up to 350 tokens per second for Llama 3 8B and up to 150 tokens per second for Llama 3 70B, running in full FP16 precision on the Together API! 🤯show more

Introducing DeepThought-8B: Transparent reasoning model built on LLaMA-3.1 with... test-time compute scaling. - JSON-structured thought chains & controllable inference paths. - ~16GB VRAM, competitive w/ 70B models. - Open model weights, and inference scripts.show more

Llama 3.3 70B is live on AkashChat. The latest... state-of-the-art multilingual AI model released by Meta is as performant as Llama 3.1 405B at a fraction of the size. Try it today:show more

pip install spectralquant ✂️ Up to 6.62x KV cache... compression for LLMs and transformers. Same model. Faster outputs. Smaller KV cache. Try now (2 mins): - KV cache integration via Hugging Face's DynamicCache - Three presets: 5.95x (paper), 6.55x (validated), 6.68x (edge) - Mistral 7B / Qwen 2.5 7B / Llama 3.1 8B verified - Pure PyTorch + future CUDA kernel support - Auto-calibration from a bundled corpus 📰 Paper: 💻 Code, quickstart, and benchmarks: #LLM #Inference #PyTorch #OpenSource #MachineLearning #LLM #KVCache #Inferenceshow more

Meta just announced that Code Llama was now free... for both research and commercial. This might the strongest competitor to ChatGPT: ▸ Can generate, explain, and debug your code ▸ Handles input 100,000 tokens ▸ Free for research + commercial use ▸ Outperforms most open models ▸ Comes in 7B, 13B, and 34B ▸ Supports Python, C++, Java, PHP, Typescript (Javascript), C#, and Bash Available in: ▸ Foundation base models (Code Llama) ▸ Python specializations (Code Llama - Python), ▸ Instruction-following models (Code Llama - Instruct)show more

> be Muse Spark Open > Meta's Llama 4... disappoints everyone > Meta goes closed with Muse Spark > Zuck writes a $100-200 billion check > Muse Wonder ships on Monday > 3 days later Meta drops you with open weights > 816 billion parameters > only 42 billion active at a time > 1 million token context > built for coding and long agent tasks > runs on two Mac Studios on a desk > Meta calls you the most capable US open-weight model ever > every Chinese lab reading the model card America is back in the open-weight game.show more

.ollama is playing with AI at Meta Llama 4... Scout! 🤯 a perfect opportunity to test Ollama's giant super computer ✈️✈️✈️show more

Microsoft just a 1-bit LLM with 2B parameters that... can run on CPUs like Apple M2. BitNet b1.58 2B4T outperforms fp LLaMA 3.2 1B while using only 0.4GB memory versus 2GB and processes tokens 40% faster. 100% opensource.show more

How much faster is the new MacBook Pro for... AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.show more

Multimodal Meta AI is rolling out widely on Ray-Ban... Meta starting today! It's a huge advancement for wearables & makes using AI more interactive & intuitive. Excited to share more on our multimodal work w/ Meta AI (& Llama 3), stay tuned for more updates coming soon.show more

The Meta Llama 3 Hackathon is this weekend in... SF with @Cerebral_Valley! Get on the list ➡️ What to expect • Two days of building alongside the best hackers in AI • Hands on support from the Llama team • Talks from some of the top names in the industryshow more

AGI at home Running DeepSeek R1 across my 7... M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses EXO Labs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB)show more

The first natively trained 1-bit model: BitNet 2B. Trained... on 4 trillion tokens. that can run on CPUs like Apple M2 Native 1.58-bit weights and 8bit activations W158A8 Outperforms LLaMA &close to Qwen 2.5 1.5B in while using only 0.4GB memory versus 2GB and processes tokens 40%show more

AI agents have arrived on Bittensor. Be the first... to launch one. It's fast and easy. TAO AGENTS - Smart contracts on Openτensor Foundaτion - Decentralized inference by Eternal AI - AI Model by AI at Meta (Llama 3.3) You can issue an instantly tradable token for your agent, too.show more

The easiest way to use this new model is... through HuggingChat with the link below. Just create a free account and select the model “nvidia/Llama-3.1-Nemotron-70B-Instruct-HF”. And you're ready to start chatting!show more