Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Microsoft just a 1-bit LLM with 2B parameters that can run on CPUs like Apple M2. BitNet b1.58 2B4T outperforms fp LLaMA 3.2 1B while using only 0.4GB memory versus 2GB and processes tokens 40% faster. 100% opensource.

Saboo_Shubham_'s profile picture

Shubham Saboo

113,720 subscribers

260,049 Aufrufe • vor 1 Jahr •via X (Twitter)

Nachrichten & PolitikWissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

You are not prepared for this, 250+ tokens/sec, 1B model < 2GB memory
0:15

You are not prepared for this, 250+ tokens/sec, 1B model < 2GB memory

anton

372,371 Aufrufe • vor 2 Jahren

LightVAE + ComfyUI node: High-performance video VAE; runs 2–3x faster using 50% less memory; LightTAE offers a 10+x speedup on just ~0.4GB VRAM
0:15

LightVAE + ComfyUI node: High-performance video VAE; runs 2–3x faster using 50% less memory; LightTAE offers a 10+x speedup on just ~0.4GB VRAM

Wildminder

38,092 Aufrufe • vor 8 Monaten

LM Studio 0.3.4 ships with Apple MLX 🚢🍎 Run on-device LLMs super fast, 100% locally and offline on your Apple Silicon Mac! Includes: > run Llama 3.2 1B at ~250 tok/sec (!) on M3 > enforce structured JSON responses > use via chat UI, or from your own code > run multiple models simultaneously > download any model from Hugging Face Video at 1x speed.
0:19

LM Studio 0.3.4 ships with Apple MLX 🚢🍎 Run on-device LLMs super fast, 100% locally and offline on your Apple Silicon Mac! Includes: > run Llama 3.2 1B at ~250 tok/sec (!) on M3 > enforce structured JSON responses > use via chat UI, or from your own code > run multiple models simultaneously > download any model from Hugging Face Video at 1x speed.

LM Studio

171,577 Aufrufe • vor 1 Jahr

Llama 3.2 1B in 4-bit runs at ~60 toks/sec with MLX Swift on my iPhone 15 pro. It's quite good and easily runs on-device:
0:17

Llama 3.2 1B in 4-bit runs at ~60 toks/sec with MLX Swift on my iPhone 15 pro. It's quite good and easily runs on-device:

Awni Hannun

492,403 Aufrufe • vor 1 Jahr

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.
0:15

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.

Alex Cheema

529,673 Aufrufe • vor 1 Jahr

GPT-4o level multimodal LLM running on your phone. MiniCPM-V 4.5 outperforms GPT-4o, Gemini-2.0 Pro, and Qwen2.5-VL 72B on vision and language AI tasks. It can even understand videos and perform OCR in 30+ languages. And it's 100% Opensource.
0:12

GPT-4o level multimodal LLM running on your phone. MiniCPM-V 4.5 outperforms GPT-4o, Gemini-2.0 Pro, and Qwen2.5-VL 72B on vision and language AI tasks. It can even understand videos and perform OCR in 30+ languages. And it's 100% Opensource.

Shubham Saboo

17,947 Aufrufe • vor 9 Monaten

Latest mlx-lm has faster and lower memory prompt processing! Thanks to causal fused attention from Jagrit Digani 7B 4-bit Mistral v3 can do ~30,000 tokens under a minute on my M4 Max laptop and only needs 8.5GB:
0:17

Latest mlx-lm has faster and lower memory prompt processing! Thanks to causal fused attention from Jagrit Digani 7B 4-bit Mistral v3 can do ~30,000 tokens under a minute on my M4 Max laptop and only needs 8.5GB:

Awni Hannun

22,156 Aufrufe • vor 1 Jahr

`transformers` + `torchao` quantization + `torch.compile` for faster inference speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :
0:11

`transformers` + `torchao` quantization + `torch.compile` for faster inference speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :

Marc Sun

24,515 Aufrufe • vor 1 Jahr

Today Meta released "Code Llama", a large language model fine-tuned for coding tasks. It's publicly available and can be used for commercial use! It outperforms GPT 3.5 and you can even run it locally on your Macbook using Ollama.
0:11

Today Meta released "Code Llama", a large language model fine-tuned for coding tasks. It's publicly available and can be used for commercial use! It outperforms GPT 3.5 and you can even run it locally on your Macbook using Ollama.

Marcel Pociot 🧪

49,806 Aufrufe • vor 2 Jahren

RAG is not Memory for AI Agents. 5 AI memory engines to build agents that maintain long-term context and learn continuously: (Last 2 released just this month) 1. Zep builds and queries temporally-aware knowledge graphs that evolve with every interaction. 100% Opensource.
0:19

RAG is not Memory for AI Agents. 5 AI memory engines to build agents that maintain long-term context and learn continuously: (Last 2 released just this month) 1. Zep builds and queries temporally-aware knowledge graphs that evolve with every interaction. 100% Opensource.

Unwind AI

85,045 Aufrufe • vor 10 Monaten

Llama 3.2 is the latest open-source AI model from Meta, released only a few hours ago. Here is the 3B parameter model running on Akash Chat at 165 tokens/second, powered by NVIDIA A100s on Akash. Try Llama 3.2 for free, no sign-in required:
0:15

Llama 3.2 is the latest open-source AI model from Meta, released only a few hours ago. Here is the 3B parameter model running on Akash Chat at 165 tokens/second, powered by NVIDIA A100s on Akash. Try Llama 3.2 for free, no sign-in required:

Akash Network

37,087 Aufrufe • vor 1 Jahr

Microsoft is testing a new feature in Windows 11 called Low Latency Profile that gives a short boost to your CPU for 1 to 3 seconds when you open apps or menus. Early tests show Microsoft apps like Edge and Outlook open up to 40% faster, while the Start menu and other menus can load up to 70% faster, with CPU use jumping high for a few seconds then dropping right back down. Microsoft says this is normal because Apple does it in macOS and Linux, plus phones use the same type of quick CPU boosts for better speed. The feature is only in early Insider testing now, so it is not out for everyone yet.
0:15

Microsoft is testing a new feature in Windows 11 called Low Latency Profile that gives a short boost to your CPU for 1 to 3 seconds when you open apps or menus. Early tests show Microsoft apps like Edge and Outlook open up to 40% faster, while the Start menu and other menus can load up to 70% faster, with CPU use jumping high for a few seconds then dropping right back down. Microsoft says this is normal because Apple does it in macOS and Linux, plus phones use the same type of quick CPU boosts for better speed. The feature is only in early Insider testing now, so it is not out for everyone yet.

Pirat_Nation 🔴

145,459 Aufrufe • vor 1 Monat

You can run and monitor Claude Code from literally anywhere. Start a coding session on your laptop, continue from your phone while on a walk, and get push notifications when Claude needs input. 100% Opensource and Free.
0:16

You can run and monitor Claude Code from literally anywhere. Start a coding session on your laptop, continue from your phone while on a walk, and get push notifications when Claude needs input. 100% Opensource and Free.

Unwind AI

15,573 Aufrufe • vor 10 Monaten

Meta released LongVU: a new video LM that can handle long videos (great performance, battle-tested by me ⚔) TLDR; 1️⃣ downsample using DINOv2 to eliminate redundant scenes 🦖 2️⃣ fuse rest of the features using DINOv2 and SigLIP 3️⃣ select some tokens, pass to Qwen2/Llama-3.2-3B
0:11

Meta released LongVU: a new video LM that can handle long videos (great performance, battle-tested by me ⚔) TLDR; 1️⃣ downsample using DINOv2 to eliminate redundant scenes 🦖 2️⃣ fuse rest of the features using DINOv2 and SigLIP 3️⃣ select some tokens, pass to Qwen2/Llama-3.2-3B

merve

49,546 Aufrufe • vor 1 Jahr

RAG engine that just works for complex real-world documents. It can extract knowledge from unlimited tokens across ANY data format - Word docs, Excel sheets, PDFs, and even scanned copies. And it's 100% Opensource.
0:17

RAG engine that just works for complex real-world documents. It can extract knowledge from unlimited tokens across ANY data format - Word docs, Excel sheets, PDFs, and even scanned copies. And it's 100% Opensource.

Shubham Saboo

45,212 Aufrufe • vor 1 Jahr

Indian man finds romance with Fido while his H-1B processes
0:17

Indian man finds romance with Fido while his H-1B processes

LXXPBUH*

4,829,839 Aufrufe • vor 1 Jahr

Perplexity's Sonar—built on Llama 3.3 70b—outperforms GPT-4o-mini and Claude 3.5 Haiku while matching or surpassing top models like GPT-4o and Claude 3.5 Sonnet in user satisfaction. At 1200 tokens/second, Sonar is optimized for answer quality and speed.
0:11

Perplexity's Sonar—built on Llama 3.3 70b—outperforms GPT-4o-mini and Claude 3.5 Haiku while matching or surpassing top models like GPT-4o and Claude 3.5 Sonnet in user satisfaction. At 1200 tokens/second, Sonar is optimized for answer quality and speed.

Perplexity

565,953 Aufrufe • vor 1 Jahr

I just created my own LaTeX-OCR app using Llama 3.2 Vision! Upload the LaTeX code as an image, and it gives you the corresponding LaTeX code using Llama 3.2 multimodal! Here's what I used: - Ollama for serving Llama 3.2 vision locally - Streamlit for the UI Everything is just 50 lines of code! Find the code in the next tweet. -- Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. Find me → Avi Chawla
0:15

I just created my own LaTeX-OCR app using Llama 3.2 Vision! Upload the LaTeX code as an image, and it gives you the corresponding LaTeX code using Llama 3.2 multimodal! Here's what I used: - Ollama for serving Llama 3.2 vision locally - Streamlit for the UI Everything is just 50 lines of code! Find the code in the next tweet. -- Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. Find me → Avi Chawla

Avi Chawla

15,610 Aufrufe • vor 1 Jahr

RAG is not Memory. AI agents need long-term memory to maintain context and learn continuously. 5 frameworks for AI agent memory. 100% opensource. 1. Graphiti builds temporally-aware knowledge graphs for AI agents that change over time with evolving relationships and context.
0:19

RAG is not Memory. AI agents need long-term memory to maintain context and learn continuously. 5 frameworks for AI agent memory. 100% opensource. 1. Graphiti builds temporally-aware knowledge graphs for AI agents that change over time with evolving relationships and context.

Unwind AI

293,513 Aufrufe • vor 1 Jahr