Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Microsoft killed the GPU mafia 🤯 They finally open-sourced their 1-bit LLM inference framework called bitnet.cpp. It lets you run 100B parameter models on your local CPU without GPUs. - 6.17x faster inference - 82.2% less energy on CPUs 100% Open Source.

Oliver

23,446 subscribers

1,632,616 просмотров • 9 месяцев назад •via X (Twitter)

Здоровье и велнес Образование Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Biologists use specialized open-source models for tasks like modeling the structure of molecular systems, designing drug-like molecules, and predicting the effects of genetic mutations. But these models are often expensive to run, potentially limiting their impact. In our latest Science Blog, we share how Claude was able to optimize inference for more than 30 open-source models, making them 4x faster on average, partly by writing custom software for GPUs. We’re open sourcing all of the optimization code. Read more:

Biologists use specialized open-source models for tasks like modeling the structure of molecular systems, designing drug-like molecules, and predicting the effects of genetic mutations. But these models are often expensive to run, potentially limiting their impact. In our latest Science Blog, we share how Claude was able to optimize inference for more than 30 open-source models, making them 4x faster on average, partly by writing custom software for GPUs. We’re open sourcing all of the optimization code. Read more:

Anthropic

638,387 просмотров • 1 месяц назад

someone just open-sourced their own neuro-sama. and it might be better than the original. it's called airi. a fully autonomous ai companion that talks to you in real time, plays minecraft and factorio with you, chats on discord and telegram, and has a live2d/vrm avatar body. runs entirely on your machine. → real-time voice conversations, speech recognition → animated avatar with auto-blink, eye tracking, idle animations → persistent memory across sessions → local inference via webgpu, no api calls needed supports 30+ llm providers, openai, claude, gemini, deepseek, ollama, groq, mistral, xai, local models. swap the brain with a config change. runs on native cuda and apple metal for real gpu acceleration. 17.5k stars. 101 contributors. 46 releases. 100% free. open source.

Oliver

291,170 просмотров • 3 месяцев назад

Step 4 to achieve truly serverless GPUs for AI inference: skip over unserializable inference engine setup steps like CUDA graph capture and Torch compilation by stacking GPU snapshots and CPU snapshots.

Step 4 to achieve truly serverless GPUs for AI inference: skip over unserializable inference engine setup steps like CUDA graph capture and Torch compilation by stacking GPU snapshots and CPU snapshots.

Charles 🎉 Frye

17,452 просмотров • 6 месяцев назад

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers

Vikas gupta

14,138 просмотров • 2 месяцев назад

Cline Desktop just launched. An open-source workspace for open-weight models 🤯 > Import tasks from other coding agents > Run agents in parallel > Schedule tasks to run on their own Free models built in: DeepSeek V4 Flash, GLM 5.3 Flash, Laguna S 2.1. I had early access and I can assure you it feels incredible. Pick your model and provider, and run it. 100% open source.

Cline Desktop just launched. An open-source workspace for open-weight models 🤯 > Import tasks from other coding agents > Run agents in parallel > Schedule tasks to run on their own Free models built in: DeepSeek V4 Flash, GLM 5.3 Flash, Laguna S 2.1. I had early access and I can assure you it feels incredible. Pick your model and provider, and run it. 100% open source.

Alvaro Cintas

15,934 просмотров • 2 месяцев назад

NVIDIA JUST OPEN-SOURCED A FREE AI ROUTER THAT TURNS EVERY MACHINE ON YOUR HOME NETWORK INTO ONE LOCAL INFERENCE CLUSTER. ZERO CLOUD. ZERO API COSTS. ALL YOUR IDLE COMPUTE WORKS TOGETHER.

NVIDIA JUST OPEN-SOURCED A FREE AI ROUTER THAT TURNS EVERY MACHINE ON YOUR HOME NETWORK INTO ONE LOCAL INFERENCE CLUSTER. ZERO CLOUD. ZERO API COSTS. ALL YOUR IDLE COMPUTE WORKS TOGETHER.

0xMarioNawfal

50,952 просмотров • 2 месяцев назад

Excited to annouce that @askvenice is now integrated as our sole LLM inference provider for bunnyCompute. We love the product and recommend it for any LLM use case. Open-source models, no data retention, no censorship - that's why we chose them.

Excited to annouce that @askvenice is now integrated as our sole LLM inference provider for bunnyCompute. We love the product and recommend it for any LLM use case. Open-source models, no data retention, no censorship - that's why we chose them.

Bunny Open Society

69,386 просмотров • 4 месяцев назад

A catalogue is useless if you can’t run it. ⬟ The Playground is executable. Models are live, not listed. 100k+ models available for real inference. No local setup required. Browse less. Execute more.

A catalogue is useless if you can’t run it. ⬟ The Playground is executable. Models are live, not listed. 100k+ models available for real inference. No local setup required. Browse less. Execute more.

Nesa

15,494 просмотров • 10 месяцев назад

someone just open-sourced their own neuro-sama. and it might be better than the original. it's called airi. a fully autonomous ai companion that speaks with you in real time, joins you in minecraft and factorio, chats through discord and telegram, and comes with a live2d/vrm avatar. runs entirely on your machine. → real-time voice chat and speech recognition → animated avatar with automatic blinking, eye tracking, and idle movements → persistent memory between sessions → local inference through webgpu, no api calls required supports 30+ llm providers, including openai, claude, gemini, deepseek, ollama, groq, mistral, xai, and local models. switch the brain with a config change. uses native cuda and apple metal for real gpu acceleration. 17.5k stars. 101 contributors. 46 releases. completely free. open source.

Simplifying AI

197,185 просмотров • 1 месяц назад

Claude Code got a massive open-source rival 🤯 This tool maps your entire codebase, automates terminal commands, and lets you use 500+ different AI models, including completely free ones. 100% Open-Source.

Claude Code got a massive open-source rival 🤯 This tool maps your entire codebase, automates terminal commands, and lets you use 500+ different AI models, including completely free ones. 100% Open-Source.

Alvaro Cintas

54,172 просмотров • 8 месяцев назад

world models have their comfyui now. introducing alakazam studio, the first local, open source UI for world models. •⁠ ⁠sentence goes in => you edit the state graph => play •⁠ ⁠on a live model, yours, running in your browser •⁠ ⁠⁠apache-2.0, fully open source fork it and point it at your model 👇

world models have their comfyui now. introducing alakazam studio, the first local, open source UI for world models. •⁠ ⁠sentence goes in => you edit the state graph => play •⁠ ⁠on a live model, yours, running in your browser •⁠ ⁠⁠apache-2.0, fully open source fork it and point it at your model 👇

hugo

26,556 просмотров • 3 месяцев назад

Inference on UOMI is now free. Start building with frontier open-source models at no cost: • MiniMax M2.7 • Qwen 3.6 27B • Qwen 3.6 35B A3B • Google Gemma 4 26B A4B • Google Gemma 4 31B Powered by the UOMI Inference Network. Start inferencing:

Inference on UOMI is now free. Start building with frontier open-source models at no cost: • MiniMax M2.7 • Qwen 3.6 27B • Qwen 3.6 35B A3B • Google Gemma 4 26B A4B • Google Gemma 4 31B Powered by the UOMI Inference Network. Start inferencing:

Uomi

12,671 просмотров • 5 месяцев назад

RUN UNCENSORED AI OFFLINE ON YOUR ANDROID PHONE privatelm lets you download and run uncensored gguf models right on your phone. no internet needed after download how to set it up: 1. open the github repo 2. get the android build from the repo 3. open the app and let it check your phone's ram 4. download any gguf model, including uncensored ones 5. chat offline free and open source (mit)

RUN UNCENSORED AI OFFLINE ON YOUR ANDROID PHONE privatelm lets you download and run uncensored gguf models right on your phone. no internet needed after download how to set it up: 1. open the github repo 2. get the android build from the repo 3. open the app and let it check your phone's ram 4. download any gguf model, including uncensored ones 5. chat offline free and open source (mit)

Sakata

116,656 просмотров • 6 дней назад

NVIDIA open-sourced a 600M model that transcribes 40 languages in real-time at 80ms latency and it costs $0. that's faster than you can blink. across mandarin, arabic, hindi, portuguese, tagalog, whatever,from a SINGLE checkpoint. → 17x more concurrent streams than buffered ASR on the same H100. → punctuation + capitalization built-in. no post-processing. → runs on your own GPU. no API bill 100% Open Source.

NVIDIA open-sourced a 600M model that transcribes 40 languages in real-time at 80ms latency and it costs $0. that's faster than you can blink. across mandarin, arabic, hindi, portuguese, tagalog, whatever,from a SINGLE checkpoint. → 17x more concurrent streams than buffered ASR on the same H100. → punctuation + capitalization built-in. no post-processing. → runs on your own GPU. no API bill 100% Open Source.

Superman

105,212 просмотров • 4 месяцев назад

Run open models like Gemma 4 completely offline in the Antigravity SDK. Built on Google AI Edge’s LiteRT, you can now run Gemma 4 directly on your local GPU. Zero API costs, total data privacy, and no internet required.

Run open models like Gemma 4 completely offline in the Antigravity SDK. Built on Google AI Edge’s LiteRT, you can now run Gemma 4 directly on your local GPU. Zero API costs, total data privacy, and no internet required.

Google Antigravity

4,569,655 просмотров • 1 месяц назад

Google just open-sourced LangExtract Python library! It uses LLMs to extract entities, attributes, and relations—with exact source grounding—from unstructured documents. Flexible LLM support (Gemini, OpenAI, Ollama) 100% open-source.

Google just open-sourced LangExtract Python library! It uses LLMs to extract entities, attributes, and relations—with exact source grounding—from unstructured documents. Flexible LLM support (Gemini, OpenAI, Ollama) 100% open-source.

Akshay 🚀

120,532 просмотров • 1 год назад

just made it possible to evaluate generalist robotics models like Octo at 60-100x real world evaluation speeds via gpu simulation and rendering (~10x faster than original cpu sim code). All videos below are from our open source ManiSkill GPU sim!

just made it possible to evaluate generalist robotics models like Octo at 60-100x real world evaluation speeds via gpu simulation and rendering (~10x faster than original cpu sim code). All videos below are from our open source ManiSkill GPU sim!

Stone Tao

13,456 просмотров • 2 лет назад

Introducing Find the best local models based on real data 1. People run and submit benchmarks 2. Stats aggregated over models / devices 3. Find the best model for you `npx whatcanirun`, fully open-source

Introducing Find the best local models based on real data 1. People run and submit benchmarks 2. Stats aggregated over models / devices 3. Find the best model for you `npx whatcanirun`, fully open-source

⁵⁄₉

134,091 просмотров • 7 месяцев назад