Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Microsoft just a 1-bit LLM with 2B parameters that can run on CPUs like Apple M2. BitNet b1.58 2B4T outperforms fp LLaMA 3.2 1B while using only 0.4GB memory versus 2GB and processes tokens 40% faster. 100% opensource.

Saboo_Shubham_'s profile picture

Shubham Saboo

121,404 subscribers

260,222 görüntüleme • 1 yıl önce •via X (Twitter)

Haberler & PolitikaBilim & TeknolojiEğitim
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

You are not prepared for this, 250+ tokens/sec, 1B model < 2GB memory
0:15

You are not prepared for this, 250+ tokens/sec, 1B model < 2GB memory

anton

372,392 görüntüleme • 3 yıl önce

LightVAE + ComfyUI node: High-performance video VAE; runs 2–3x faster using 50% less memory; LightTAE offers a 10+x speedup on just ~0.4GB VRAM
0:15

LightVAE + ComfyUI node: High-performance video VAE; runs 2–3x faster using 50% less memory; LightTAE offers a 10+x speedup on just ~0.4GB VRAM

Wildminder

38,092 görüntüleme • 1 yıl önce

LM Studio 0.3.4 ships with Apple MLX 🚢🍎 Run on-device LLMs super fast, 100% locally and offline on your Apple Silicon Mac! Includes: > run Llama 3.2 1B at ~250 tok/sec (!) on M3 > enforce structured JSON responses > use via chat UI, or from your own code > run multiple models simultaneously > download any model from Hugging Face Video at 1x speed.
0:19

LM Studio 0.3.4 ships with Apple MLX 🚢🍎 Run on-device LLMs super fast, 100% locally and offline on your Apple Silicon Mac! Includes: > run Llama 3.2 1B at ~250 tok/sec (!) on M3 > enforce structured JSON responses > use via chat UI, or from your own code > run multiple models simultaneously > download any model from Hugging Face Video at 1x speed.

LM Studio

171,873 görüntüleme • 2 yıl önce

Llama 3.2 1B in 4-bit runs at ~60 toks/sec with MLX Swift on my iPhone 15 pro. It's quite good and easily runs on-device:
0:17

Llama 3.2 1B in 4-bit runs at ~60 toks/sec with MLX Swift on my iPhone 15 pro. It's quite good and easily runs on-device:

Awni Hannun

492,545 görüntüleme • 2 yıl önce

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.
0:15

How much faster is the new MacBook Pro for AI inference? M4 Max is 27% faster with 72 tok/sec compared to 56 tok/sec of the M3 Max with MLX running Gemma 2 9B (4bit). The 27% speedup is the same with Llama-3.2-1b, Llama-3.2-3b and others. Next up: EXO Labs M4 cluster.

Alex Cheema

529,724 görüntüleme • 2 yıl önce

GPT-4o level multimodal LLM running on your phone. MiniCPM-V 4.5 outperforms GPT-4o, Gemini-2.0 Pro, and Qwen2.5-VL 72B on vision and language AI tasks. It can even understand videos and perform OCR in 30+ languages. And it's 100% Opensource.
0:12

GPT-4o level multimodal LLM running on your phone. MiniCPM-V 4.5 outperforms GPT-4o, Gemini-2.0 Pro, and Qwen2.5-VL 72B on vision and language AI tasks. It can even understand videos and perform OCR in 30+ languages. And it's 100% Opensource.

Shubham Saboo

18,112 görüntüleme • 1 yıl önce

`transformers` + `torchao` quantization + `torch.compile` for faster inference speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :
0:11

`transformers` + `torchao` quantization + `torch.compile` for faster inference speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :

Marc Sun

24,571 görüntüleme • 2 yıl önce

How does Edge0 run a 35B model with only 1–2.5 GB of peak memory? In simple terms: • The full model stays in storage. • Only the parameters needed for the current step are loaded into memory. • The next parameters are prepared while the current step is running. • Memory is reused instead of loading the entire model at once. Large model. Small working set. Fully on-device.
0:10

How does Edge0 run a 35B model with only 1–2.5 GB of peak memory? In simple terms: • The full model stays in storage. • Only the parameters needed for the current step are loaded into memory. • The next parameters are prepared while the current step is running. • Memory is reused instead of loading the entire model at once. Large model. Small working set. Fully on-device.

Samuel Zeng

34,026 görüntüleme • 2 ay önce

Today Meta released "Code Llama", a large language model fine-tuned for coding tasks. It's publicly available and can be used for commercial use! It outperforms GPT 3.5 and you can even run it locally on your Macbook using Ollama.
0:11

Today Meta released "Code Llama", a large language model fine-tuned for coding tasks. It's publicly available and can be used for commercial use! It outperforms GPT 3.5 and you can even run it locally on your Macbook using Ollama.

Marcel Pociot 🧪

50,094 görüntüleme • 3 yıl önce

RAG is not Memory for AI Agents. 5 AI memory engines to build agents that maintain long-term context and learn continuously: (Last 2 released just this month) 1. Zep builds and queries temporally-aware knowledge graphs that evolve with every interaction. 100% Opensource.
0:19

RAG is not Memory for AI Agents. 5 AI memory engines to build agents that maintain long-term context and learn continuously: (Last 2 released just this month) 1. Zep builds and queries temporally-aware knowledge graphs that evolve with every interaction. 100% Opensource.

Unwind AI

85,828 görüntüleme • 1 yıl önce

Microsoft is testing a new feature in Windows 11 called Low Latency Profile that gives a short boost to your CPU for 1 to 3 seconds when you open apps or menus. Early tests show Microsoft apps like Edge and Outlook open up to 40% faster, while the Start menu and other menus can load up to 70% faster, with CPU use jumping high for a few seconds then dropping right back down. Microsoft says this is normal because Apple does it in macOS and Linux, plus phones use the same type of quick CPU boosts for better speed. The feature is only in early Insider testing now, so it is not out for everyone yet.
0:15

Microsoft is testing a new feature in Windows 11 called Low Latency Profile that gives a short boost to your CPU for 1 to 3 seconds when you open apps or menus. Early tests show Microsoft apps like Edge and Outlook open up to 40% faster, while the Start menu and other menus can load up to 70% faster, with CPU use jumping high for a few seconds then dropping right back down. Microsoft says this is normal because Apple does it in macOS and Linux, plus phones use the same type of quick CPU boosts for better speed. The feature is only in early Insider testing now, so it is not out for everyone yet.

Pirat_Nation 🔴

146,285 görüntüleme • 6 ay önce

Meta released LongVU: a new video LM that can handle long videos (great performance, battle-tested by me ⚔) TLDR; 1️⃣ downsample using DINOv2 to eliminate redundant scenes 🦖 2️⃣ fuse rest of the features using DINOv2 and SigLIP 3️⃣ select some tokens, pass to Qwen2/Llama-3.2-3B
0:11

Meta released LongVU: a new video LM that can handle long videos (great performance, battle-tested by me ⚔) TLDR; 1️⃣ downsample using DINOv2 to eliminate redundant scenes 🦖 2️⃣ fuse rest of the features using DINOv2 and SigLIP 3️⃣ select some tokens, pass to Qwen2/Llama-3.2-3B

merve

49,547 görüntüleme • 2 yıl önce

Perplexity's Sonar—built on Llama 3.3 70b—outperforms GPT-4o-mini and Claude 3.5 Haiku while matching or surpassing top models like GPT-4o and Claude 3.5 Sonnet in user satisfaction. At 1200 tokens/second, Sonar is optimized for answer quality and speed.
0:11

Perplexity's Sonar—built on Llama 3.3 70b—outperforms GPT-4o-mini and Claude 3.5 Haiku while matching or surpassing top models like GPT-4o and Claude 3.5 Sonnet in user satisfaction. At 1200 tokens/second, Sonar is optimized for answer quality and speed.

Perplexity

566,570 görüntüleme • 1 yıl önce

Indian man finds romance with Fido while his H-1B processes
0:17

Indian man finds romance with Fido while his H-1B processes

LXXPBUH*

4,832,800 görüntüleme • 1 yıl önce

I just created my own LaTeX-OCR app using Llama 3.2 Vision! Upload the LaTeX code as an image, and it gives you the corresponding LaTeX code using Llama 3.2 multimodal! Here's what I used: - Ollama for serving Llama 3.2 vision locally - Streamlit for the UI Everything is just 50 lines of code! Find the code in the next tweet. -- Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. Find me → Avi Chawla
0:15

I just created my own LaTeX-OCR app using Llama 3.2 Vision! Upload the LaTeX code as an image, and it gives you the corresponding LaTeX code using Llama 3.2 multimodal! Here's what I used: - Ollama for serving Llama 3.2 vision locally - Streamlit for the UI Everything is just 50 lines of code! Find the code in the next tweet. -- Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. Find me → Avi Chawla

Avi Chawla

15,630 görüntüleme • 1 yıl önce

RAG is not Memory. AI agents need long-term memory to maintain context and learn continuously. 5 frameworks for AI agent memory. 100% opensource. 1. Graphiti builds temporally-aware knowledge graphs for AI agents that change over time with evolving relationships and context.
0:19

RAG is not Memory. AI agents need long-term memory to maintain context and learn continuously. 5 frameworks for AI agent memory. 100% opensource. 1. Graphiti builds temporally-aware knowledge graphs for AI agents that change over time with evolving relationships and context.

Unwind AI

294,052 görüntüleme • 1 yıl önce