Loading video...

Video Failed to Load

Go Home

Microsoft killed the GPU mafia 🤯 They finally open-sourced their 1-bit LLM inference framework called bitnet.cpp. It lets you run 100B parameter models on your local CPU without GPUs. - 6.17x faster inference - 82.2% less energy on CPUs 100% Open Source.

oliviscusAI's profile picture

Oliver Prompts

18,995 subscribers

1,627,460 views • 4 months ago •via X (Twitter)

Health & WellnessEducationScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Laika AI x Inference Labs Excited to announce our partnership with Inference Labs We're providing our real-time RAG & AI model API to Inference Labs, powering their verification infrastructure with live blockchain data. Inference Labs delivers open-source, trustless verification for AI agent outputs, so you can trust what you see—without relying on centralized gatekeepers.
0:10

Laika AI x Inference Labs Excited to announce our partnership with Inference Labs We're providing our real-time RAG & AI model API to Inference Labs, powering their verification infrastructure with live blockchain data. Inference Labs delivers open-source, trustless verification for AI agent outputs, so you can trust what you see—without relying on centralized gatekeepers.

Laika AI

13,727 views • 1 year ago

Step 4 to achieve truly serverless GPUs for AI inference: skip over unserializable inference engine setup steps like CUDA graph capture and Torch compilation by stacking GPU snapshots and CPU snapshots.
0:11

Step 4 to achieve truly serverless GPUs for AI inference: skip over unserializable inference engine setup steps like CUDA graph capture and Torch compilation by stacking GPU snapshots and CPU snapshots.

Charles 🎉 Frye

17,384 views • 1 month ago

A catalogue is useless if you can’t run it. ⬟ The Playground is executable. Models are live, not listed. 100k+ models available for real inference. No local setup required. Browse less. Execute more.
0:16

A catalogue is useless if you can’t run it. ⬟ The Playground is executable. Models are live, not listed. 100k+ models available for real inference. No local setup required. Browse less. Execute more.

Nesa

15,378 views • 5 months ago

Claude Code got a massive open-source rival 🤯 This tool maps your entire codebase, automates terminal commands, and lets you use 500+ different AI models, including completely free ones. 100% Open-Source.
0:19

Claude Code got a massive open-source rival 🤯 This tool maps your entire codebase, automates terminal commands, and lets you use 500+ different AI models, including completely free ones. 100% Open-Source.

Alvaro Cintas

54,172 views • 3 months ago

Now that we have amazing open source TTS with fast inference, what are you building?
0:17

Now that we have amazing open source TTS with fast inference, what are you building?

Victor M

431,129 views • 1 year ago

just made it possible to evaluate generalist robotics models like Octo at 60-100x real world evaluation speeds via gpu simulation and rendering (~10x faster than original cpu sim code). All videos below are from our open source ManiSkill GPU sim!
0:12

just made it possible to evaluate generalist robotics models like Octo at 60-100x real world evaluation speeds via gpu simulation and rendering (~10x faster than original cpu sim code). All videos below are from our open source ManiSkill GPU sim!

Stone Tao

13,456 views • 1 year ago

Introducing Find the best local models based on real data 1. People run and submit benchmarks 2. Stats aggregated over models / devices 3. Find the best model for you `npx whatcanirun`, fully open-source
0:16

Introducing Find the best local models based on real data 1. People run and submit benchmarks 2. Stats aggregated over models / devices 3. Find the best model for you `npx whatcanirun`, fully open-source

⁵⁄₉

133,226 views • 2 months ago

Google just open-sourced LangExtract Python library! It uses LLMs to extract entities, attributes, and relations—with exact source grounding—from unstructured documents. Flexible LLM support (Gemini, OpenAI, Ollama) 100% open-source.
0:18

Google just open-sourced LangExtract Python library! It uses LLMs to extract entities, attributes, and relations—with exact source grounding—from unstructured documents. Flexible LLM support (Gemini, OpenAI, Ollama) 100% open-source.

Akshay 🚀

120,396 views • 10 months ago

If you love open models, you’ll love this: Crush now runs with Hugging Face Inference Providers 🤗✨
0:15

If you love open models, you’ll love this: Crush now runs with Hugging Face Inference Providers 🤗✨

Charm

38,931 views • 9 months ago

Someone just built a desktop app that that generates 3D models from images and runs 100% locally. It's called Modly. It runs entirely on your GPU, no cloud, no API bills. Just drop an image and get a 3D mesh. 100% Open Source.
0:11

Someone just built a desktop app that that generates 3D models from images and runs 100% locally. It's called Modly. It runs entirely on your GPU, no cloud, no API bills. Just drop an image and get a 3D mesh. 100% Open Source.

How To Prompt

222,673 views • 2 months ago

Introducing DeepThought-8B: Transparent reasoning model built on LLaMA-3.1 with test-time compute scaling. - JSON-structured thought chains & controllable inference paths. - ~16GB VRAM, competitive w/ 70B models. - Open model weights, and inference scripts.
0:16

Introducing DeepThought-8B: Transparent reasoning model built on LLaMA-3.1 with test-time compute scaling. - JSON-structured thought chains & controllable inference paths. - ~16GB VRAM, competitive w/ 70B models. - Open model weights, and inference scripts.

Ruliad

219,315 views • 1 year ago

We’re thrilled to launch GPT-OSS—the 120B-parameter open-source LLM—on Phala Cloud! 🚀 🔒 Run the fully OSS model inside a secure TEE for end-to-end confidential AI. Explore the build 👉
0:13

We’re thrilled to launch GPT-OSS—the 120B-parameter open-source LLM—on Phala Cloud! 🚀 🔒 Run the fully OSS model inside a secure TEE for end-to-end confidential AI. Explore the build 👉

Phala

12,176 views • 10 months ago

$AMD might be the most underrated AI stock on the market. Everyone focuses on GPUs. But CEO Lisa Su says the CPU market alone could hit $120B by 2030 because of AI agents. $AMD gives you BOTH: • AI GPUs • AI CPUs The AI boom won’t run on GPUs alone. Don’t miss out again…
0:11

$AMD might be the most underrated AI stock on the market. Everyone focuses on GPUs. But CEO Lisa Su says the CPU market alone could hit $120B by 2030 because of AI agents. $AMD gives you BOTH: • AI GPUs • AI CPUs The AI boom won’t run on GPUs alone. Don’t miss out again…

CMS Invests

13,616 views • 1 month ago

No more waitlist – Cerebras inference API is open to all! 1M free tokens/day 20x GPU speed Reasoning in ~1 second It's time to build!
0:18

No more waitlist – Cerebras inference API is open to all! 1M free tokens/day 20x GPU speed Reasoning in ~1 second It's time to build!

Cerebras

108,466 views • 1 year ago

this is the worst local AI will ever be. tomorrow it gets faster. next month the models get smarter. next year your GPU runs what a data center runs today. Qwen3.5-35B-A3B on a single 3090. told it to visualize its own expert routing. 256 experts, 8 active per token, rendered in 3D on the same GPU running inference. no API key. no subscription. no permission needed. closed AI isn't losing ground. it's losing the argument.
0:17

this is the worst local AI will ever be. tomorrow it gets faster. next month the models get smarter. next year your GPU runs what a data center runs today. Qwen3.5-35B-A3B on a single 3090. told it to visualize its own expert routing. 256 experts, 8 active per token, rendered in 3D on the same GPU running inference. no API key. no subscription. no permission needed. closed AI isn't losing ground. it's losing the argument.

Sudo su

106,710 views • 3 months ago

Fireworks blazing fast LLM inference is now available on Poe! Today, we’re excited to bring the power of the new Mistral 7B Instruct model to the Poe powered by our lightning-fast Fireworks inference platform. You can now have conversations with the Mistral 7B bot and even build your own bots on top of it. Try it here:
0:19

Fireworks blazing fast LLM inference is now available on Poe! Today, we’re excited to bring the power of the new Mistral 7B Instruct model to the Poe powered by our lightning-fast Fireworks inference platform. You can now have conversations with the Mistral 7B bot and even build your own bots on top of it. Try it here:

Fireworks AI

93,334 views • 2 years ago

🔥 BREAKING: Open source just leveled up AI agents Eigent gives you a fully local, customizable AI workforce....built to run on your laptop. → No vendor lock-in → No cloud dependency → 100% open source Just fast, private, parallel agents you control (Here's how):👇
0:15

🔥 BREAKING: Open source just leveled up AI agents Eigent gives you a fully local, customizable AI workforce....built to run on your laptop. → No vendor lock-in → No cloud dependency → 100% open source Just fast, private, parallel agents you control (Here's how):👇

Shruti

63,497 views • 10 months ago

You can now run open-source AI coding agents without paying for API keys 🤯 Cline CLI 2.0 just dropped with free access to Minimax M2.5. → Runs from your terminal → Parallel agents → Works with any editor Any model you want. 100% Open Source.
0:14

You can now run open-source AI coding agents without paying for API keys 🤯 Cline CLI 2.0 just dropped with free access to Minimax M2.5. → Runs from your terminal → Parallel agents → Works with any editor Any model you want. 100% Open Source.

Alvaro Cintas

161,893 views • 4 months ago

PRIMA.CPP Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday Home Clusters
0:17

PRIMA.CPP Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday Home Clusters

AK

48,241 views • 1 year ago

Tutorial Time: Run any open-source LLM locally. Now we will run an LLM on your M1/2 Mac. And its fast. All you need is moved to: @lmstudio let's get started. Good to be back. A thread
0:17

Tutorial Time: Run any open-source LLM locally. Now we will run an LLM on your M1/2 Mac. And its fast. All you need is moved to: @lmstudio let's get started. Good to be back. A thread

Linus ✦ Ekenstam

915,787 views • 2 years ago