Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

The first natively trained 1-bit model: BitNet 2B. Trained on 4 trillion tokens. that can run on CPUs like Apple M2 Native 1.58-bit weights and 8bit activations W158A8 Outperforms LLaMA &close to Qwen 2.5 1.5B in while using only 0.4GB memory versus 2GB and processes tokens 40%

0x0SojalSec's profile picture

Md Ismail Šojal 🕷️

44,919 subscribers

43,647 Aufrufe • vor 4 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec
0:16

DeepSeek R1 Qwen 7B 4bit M2 Ultra vs M4 Max on Apple MLX 🤫 Let them think... (video 4x in center part) M2 Ultra: 114.9 tokens per sec M4 Max (14"): 88.3 tokens per sec

Ivan Fioravanti ᯅ

60,200 Aufrufe • vor 1 Jahr

🔥Apple MLX first 6bit model is on Hugging Face!🔥 Qwen2.5-Coder-32B-Instruct-6bit! 3bit conversion and test in progress! Video 8x below on M4 Max 40GPU: - Prompt: 38 tokens, 61.731 tokens-per-sec - Generation: 1181 tokens, 16.939 tokens-per-sec - Peak memory: 25.122 GB
0:12

🔥Apple MLX first 6bit model is on Hugging Face!🔥 Qwen2.5-Coder-32B-Instruct-6bit! 3bit conversion and test in progress! Video 8x below on M4 Max 40GPU: - Prompt: 38 tokens, 61.731 tokens-per-sec - Generation: 1181 tokens, 16.939 tokens-per-sec - Peak memory: 25.122 GB

Ivan Fioravanti ᯅ

46,494 Aufrufe • vor 1 Jahr

DeepSeek R1 distilled to Qwen 1.5B easily runs on my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:
0:19

DeepSeek R1 distilled to Qwen 1.5B easily runs on my iPhone 16 with MLX swift. Here's the 4-bit model reasoning entirely on device at almost 60 toks/sec:

Awni Hannun

1,139,656 Aufrufe • vor 1 Jahr

You are not prepared for this, 250+ tokens/sec, 1B model < 2GB memory
0:15

You are not prepared for this, 250+ tokens/sec, 1B model < 2GB memory

anton

372,377 Aufrufe • vor 2 Jahren

Qwen 3.6 models are now 2.5x times faster on Atomic Chat with new MTP speedups. > MTP drafts several tokens ahead and verifies them in one pass. The speedup depends on the memory moved per pass. Users can run Qwen 3.6 models locally via the open-source Atomic Chat to test them!
0:14

Qwen 3.6 models are now 2.5x times faster on Atomic Chat with new MTP speedups. > MTP drafts several tokens ahead and verifies them in one pass. The speedup depends on the memory moved per pass. Users can run Qwen 3.6 models locally via the open-source Atomic Chat to test them!

🚨 AI News | TestingCatalog

46,013 Aufrufe • vor 1 Monat

GEMMA 4 FINISHED 14 MINUTES FASTER THAN QWEN 3.6 DESPITE LOWER TOKENS PER SECOND BY USING 5X FEWER TOKENS
0:16

GEMMA 4 FINISHED 14 MINUTES FASTER THAN QWEN 3.6 DESPITE LOWER TOKENS PER SECOND BY USING 5X FEWER TOKENS

0xMarioNawfal

71,957 Aufrufe • vor 2 Monaten

🧙🏼‍♂️! We are excited to announce the launch of $MIM and the first cauldrons on Berachain’s Artio Testnet!🐻⛓️ Users can now borrow testnet $MIM using HONEY/MIM LP tokens, natively on Artio!🔮🔥 Interact with Artio directly👇🏻
0:12

🧙🏼‍♂️! We are excited to announce the launch of $MIM and the first cauldrons on Berachain’s Artio Testnet!🐻⛓️ Users can now borrow testnet $MIM using HONEY/MIM LP tokens, natively on Artio!🔮🔥 Interact with Artio directly👇🏻

🧙🏼‍♂️

400,390 Aufrufe • vor 2 Jahren

Perplexity's Sonar—built on Llama 3.3 70b—outperforms GPT-4o-mini and Claude 3.5 Haiku while matching or surpassing top models like GPT-4o and Claude 3.5 Sonnet in user satisfaction. At 1200 tokens/second, Sonar is optimized for answer quality and speed.
0:11

Perplexity's Sonar—built on Llama 3.3 70b—outperforms GPT-4o-mini and Claude 3.5 Haiku while matching or surpassing top models like GPT-4o and Claude 3.5 Sonnet in user satisfaction. At 1200 tokens/second, Sonar is optimized for answer quality and speed.

Perplexity

565,953 Aufrufe • vor 1 Jahr

LLaVA Llama-3 and Phi-3 now on MLX 🎉🚀 You can now run inference locally on your Mac. pip install -U mlx-vlm I’m getting ~50 tokens on a M3 Max. Model cards 👇🏾
0:10

LLaVA Llama-3 and Phi-3 now on MLX 🎉🚀 You can now run inference locally on your Mac. pip install -U mlx-vlm I’m getting ~50 tokens on a M3 Max. Model cards 👇🏾

Prince Canuma

74,448 Aufrufe • vor 2 Jahren

MLX + M3 Ultra 512GB + Qwen3-235B-A22B-8bit = 🔥 "write a beautiful p5js particles animation that reacts to mouse clicks movements" Prompt: 22 tokens, 64.471 tokens-per-sec Generation: 6197 tokens, 18.916 tokens-per-sec Peak memory: 251.077 GB
0:14

MLX + M3 Ultra 512GB + Qwen3-235B-A22B-8bit = 🔥 "write a beautiful p5js particles animation that reacts to mouse clicks movements" Prompt: 22 tokens, 64.471 tokens-per-sec Generation: 6197 tokens, 18.916 tokens-per-sec Peak memory: 251.077 GB

Ivan Fioravanti ᯅ

24,160 Aufrufe • vor 1 Jahr

🚀 Igniter & Solaxy Wallet Are LIVE! 🛸 The wait is over. You can now launch tokens on Solaxy using Igniter, and interact with the rollup like never before using the official Solaxy Wallet! 🔹 Igniter Solaxy’s native launchpad. Launch your own token directly on the rollup. Tokens that hit a 1M $SOLX market cap are automatically rooted into Neptoon DEX.
0:16

🚀 Igniter & Solaxy Wallet Are LIVE! 🛸 The wait is over. You can now launch tokens on Solaxy using Igniter, and interact with the rollup like never before using the official Solaxy Wallet! 🔹 Igniter Solaxy’s native launchpad. Launch your own token directly on the rollup. Tokens that hit a 1M $SOLX market cap are automatically rooted into Neptoon DEX.

SOLAXY

303,047 Aufrufe • vor 11 Monaten

Today Meta released "Code Llama", a large language model fine-tuned for coding tasks. It's publicly available and can be used for commercial use! It outperforms GPT 3.5 and you can even run it locally on your Macbook using Ollama.
0:11

Today Meta released "Code Llama", a large language model fine-tuned for coding tasks. It's publicly available and can be used for commercial use! It outperforms GPT 3.5 and you can even run it locally on your Macbook using Ollama.

Marcel Pociot 🧪

50,094 Aufrufe • vor 2 Jahren

`transformers` + `torchao` quantization + `torch.compile` for faster inference speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :
0:11

`transformers` + `torchao` quantization + `torch.compile` for faster inference speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :

Marc Sun

24,515 Aufrufe • vor 1 Jahr

Qwen has just released a model on par with GPT-4o... And you can run it locally easily 🤯 Yep. GPT-4o level AI running offline on a laptop. - Fully open source - Only 3B active parameters - 262k context length natively Quick steps to run it on your machine and details below
0:15

Qwen has just released a model on par with GPT-4o... And you can run it locally easily 🤯 Yep. GPT-4o level AI running offline on a laptop. - Fully open source - Only 3B active parameters - 262k context length natively Quick steps to run it on your machine and details below

Paul Couvert

503,787 Aufrufe • vor 11 Monaten

This is amazing! Someone trained a 1.5B model (with GRPO) to solve mazes! model:
0:12

This is amazing! Someone trained a 1.5B model (with GRPO) to solve mazes! model:

Victor M

139,407 Aufrufe • vor 1 Jahr

/1 Gemma 4 31B just crushed Qwen 3.6 27B in a local LLM gamedev contest inside atomic.chat (prompt is below) Device: MacBook Pro M5 Max, 64GB RAM Results: Qwen 3.6 27B: 32 tokens/sec · 18m 04s · 33,946 tokens Gemma 4 31B: 27 tokens/sec · 3m 51s · 6,209 tokens So what is more important: tokens per second, or the quality of the final answer? Qwen made a very long response and showed more creativity and visual style. But Gemma gave a shorter, clearer, and more logical answer in much less time. In this one-shot Pac-Man gamedev contest, Gemma 4 31B was the clear winner. Its game logic was stronger: click reactions were smoother, and it handled interactions with elements like walls, ghosts, and particle effects better. But this was only one test. Maybe Qwen 3.6 27B can show better results with better settings. Open the comments, try our prompt, and share your result below.
0:16

/1 Gemma 4 31B just crushed Qwen 3.6 27B in a local LLM gamedev contest inside atomic.chat (prompt is below) Device: MacBook Pro M5 Max, 64GB RAM Results: Qwen 3.6 27B: 32 tokens/sec · 18m 04s · 33,946 tokens Gemma 4 31B: 27 tokens/sec · 3m 51s · 6,209 tokens So what is more important: tokens per second, or the quality of the final answer? Qwen made a very long response and showed more creativity and visual style. But Gemma gave a shorter, clearer, and more logical answer in much less time. In this one-shot Pac-Man gamedev contest, Gemma 4 31B was the clear winner. Its game logic was stronger: click reactions were smoother, and it handled interactions with elements like walls, ghosts, and particle effects better. But this was only one test. Maybe Qwen 3.6 27B can show better results with better settings. Open the comments, try our prompt, and share your result below.

Chubby♨️

72,566 Aufrufe • vor 2 Monaten

BREAKING: Grok set a new record on OpenRouter this year with over 16 trillion tokens of usage. 16,060,000,000,000 tokens. That is about 64% more usage than the next closest model.
0:11

BREAKING: Grok set a new record on OpenRouter this year with over 16 trillion tokens of usage. 16,060,000,000,000 tokens. That is about 64% more usage than the next closest model.

DogeDesigner

27,783 Aufrufe • vor 6 Monaten

Put your $ZEC, $DOGE, and $XRP to work on Unichain Thousands in daily rewards up for grabs to LPs on USDC pairs. Two ways: 1. Swap to ZEC/DOGE/XRP on Unichain and LP there 2. Bridge your native tokens using the Universal App on your Unichain wallet Zuniswap.
0:17

Put your $ZEC, $DOGE, and $XRP to work on Unichain Thousands in daily rewards up for grabs to LPs on USDC pairs. Two ways: 1. Swap to ZEC/DOGE/XRP on Unichain and LP there 2. Bridge your native tokens using the Universal App on your Unichain wallet Zuniswap.

Universal

27,512 Aufrufe • vor 8 Monaten

We are thrilled to be a launch partner for Meta Llama 3. Experience Llama 3 now at up to 350 tokens per second for Llama 3 8B and up to 150 tokens per second for Llama 3 70B, running in full FP16 precision on the Together API! 🤯
0:10

We are thrilled to be a launch partner for Meta Llama 3. Experience Llama 3 now at up to 350 tokens per second for Llama 3 8B and up to 150 tokens per second for Llama 3 70B, running in full FP16 precision on the Together API! 🤯

Together AI

88,229 Aufrufe • vor 2 Jahren