Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Paving the way for unlimited context windows. Introducing ZML /attnd: sparse logarithmic attention, on CPU, over UDP, faster than GPU.

Steeve Morin

8,965 subscribers

121,949 Aufrufe • vor 9 Monaten •via X (Twitter)

Wissenschaft & Technologie Nachrichten & Politik Bildung

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Introducing Ziren (formerly zkMIPS) The most performant general-purpose zkVM on CPU, now scaled with GPU, distributed proving, and more. Build with confidence. Deploy without compromise 🟠 Full context: Technical updates:

Introducing Ziren (formerly zkMIPS) The most performant general-purpose zkVM on CPU, now scaled with GPU, distributed proving, and more. Build with confidence. Deploy without compromise 🟠 Full context: Technical updates:

ZKM

5,857,968 Aufrufe • vor 1 Jahr

$Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.$

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,193,147 Aufrufe • vor 2 Monaten

$INFQ GPU + CPU + QPU Frontier models are hitting saturation on tokens (data volume) and context windows (memory). $INFQ CTO walks through how they and $NVDA are working together to add a Quantum Processing Unit to the chain as the next inflection point.

$INFQ GPU + CPU + QPU Frontier models are hitting saturation on tokens (data volume) and context windows (memory). $INFQ CTO walks through how they and $NVDA are working together to add a Quantum Processing Unit to the chain as the next inflection point.

YeahDave

57,181 Aufrufe • vor 2 Monaten

Windows ML is now generally available! This milestone brings powerful, local AI capability to every Windows 11 device, harnessing the full power of the GPU, NPU, and CPU. Big thank you to the team and our silicon partners, AMD, Intel, NVIDIA, and Qualcomm, for their collaboration to unlock real-time, secure, and efficient AI experiences for developers on Windows. Let’s go 🚀!

Windows ML is now generally available! This milestone brings powerful, local AI capability to every Windows 11 device, harnessing the full power of the GPU, NPU, and CPU. Big thank you to the team and our silicon partners, AMD, Intel, NVIDIA, and Qualcomm, for their collaboration to unlock real-time, secure, and efficient AI experiences for developers on Windows. Let’s go 🚀!

Pavan Davuluri

11,663 Aufrufe • vor 9 Monaten

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,622 Aufrufe • vor 2 Jahren

Now introducing Replit Desktop, a seamless way to multitask and get the best of Replit right on your Desktop. Supported for both Windows and Mac.

Now introducing Replit Desktop, a seamless way to multitask and get the best of Replit right on your Desktop. Supported for both Windows and Mac.

Replit ⠕

82,858 Aufrufe • vor 20 Tagen

“Greater Manchester productivity rose over three times faster than London.” Great interview by Lord Jim O’Neil on #R4Today. GM is leading the way to growth for the next Government. 👍🏻

“Greater Manchester productivity rose over three times faster than London.” Great interview by Lord Jim O’Neil on #R4Today. GM is leading the way to growth for the next Government. 👍🏻

Andy Burnham

34,677 Aufrufe • vor 2 Jahren

Run Stable Audio 3 on ANY Computer with NO VRAM 1-click launcher for the official Stable Audio 3 gradio app. 1. Cross Platform: Works on ALL OS (Mac, Linux, Windows) out of the box 2. No GPU Needed: Even runs on CPU. 3. Ultra Fast: Generate a 2 minute clip in split seconds

Run Stable Audio 3 on ANY Computer with NO VRAM 1-click launcher for the official Stable Audio 3 gradio app. 1. Cross Platform: Works on ALL OS (Mac, Linux, Windows) out of the box 2. No GPU Needed: Even runs on CPU. 3. Ultra Fast: Generate a 2 minute clip in split seconds

cocktail peanut

71,399 Aufrufe • vor 2 Monaten

Introducing HydraDB. The graph native context infrastructure for agents. Purpose built to deliver precise context & observability into why agents act the way they do. We've always believed graphs are the best way to manage AI context, but they've been too expensive to scale or impractical for storing full context. Until now. HydraDB combines in memory, NVMe, and object storage into a single graph layer, making context delivery faster, cheaper, and more precise. We want context delivery to be extremely fast, 1000x cheap, and highly precise. Give your agents a brain.

Introducing HydraDB. The graph native context infrastructure for agents. Purpose built to deliver precise context & observability into why agents act the way they do. We've always believed graphs are the best way to manage AI context, but they've been too expensive to scale or impractical for storing full context. Until now. HydraDB combines in memory, NVMe, and object storage into a single graph layer, making context delivery faster, cheaper, and more precise. We want context delivery to be extremely fast, 1000x cheap, and highly precise. Give your agents a brain.

Nishkarsh

2,295,297 Aufrufe • vor 1 Monat

No forced towers. No violation of rights. Myth vs Reality. Telecommunications Amendment Bill protects private property, ensures consent, and clears myths, while paving the way for faster, cheaper internet across Pakistan! 🇵🇰 Facts over propaganda. Always! 🙌🏼

No forced towers. No violation of rights. Myth vs Reality. Telecommunications Amendment Bill protects private property, ensures consent, and clears myths, while paving the way for faster, cheaper internet across Pakistan! 🇵🇰 Facts over propaganda. Always! 🙌🏼

Saad Kaiser 🇵🇰

19,000 Aufrufe • vor 1 Monat

PHALA PHALA SAGA | uMkhonto weSizwe Party MP Mzwanele Manyi says he expected the process paving the way for impeachment proceedings to move faster, but is now frustrated by the legal advice being given to the Subcommittee on the Review of Assembly Rules.

PHALA PHALA SAGA | uMkhonto weSizwe Party MP Mzwanele Manyi says he expected the process paving the way for impeachment proceedings to move faster, but is now frustrated by the legal advice being given to the Subcommittee on the Review of Assembly Rules.

SABC News

20,525 Aufrufe • vor 1 Monat

Building on the proven success of Stretch for automated container unloading, DHL Group is paving the way for the global deployment of more than 1,000 additional robots. Learn how DHL is accelerating their automation strategy:

Building on the proven success of Stretch for automated container unloading, DHL Group is paving the way for the global deployment of more than 1,000 additional robots. Learn how DHL is accelerating their automation strategy:

Boston Dynamics

53,917 Aufrufe • vor 1 Jahr

Introducing the long awaited LANCOOL 217! 🚀 Built for effortless setups, the LANCOOL 217 features a tool-less design for panels and fan adjustments, making building and customizing easier than ever. Pre-installed with 5 fans 💨 • 2× 170mm front fans • 2×120 mm reverse fans at the bottom • 1× 140mm rear fan More features to love ❤️ • Supports two PSU mounting orientations • Dual modes: CPU and GPU modes • Included GPU anti-sag bracket for extra stability #lianli #case #lancool217 #pc #build

Introducing the long awaited LANCOOL 217! 🚀 Built for effortless setups, the LANCOOL 217 features a tool-less design for panels and fan adjustments, making building and customizing easier than ever. Pre-installed with 5 fans 💨 • 2× 170mm front fans • 2×120 mm reverse fans at the bottom • 1× 140mm rear fan More features to love ❤️ • Supports two PSU mounting orientations • Dual modes: CPU and GPU modes • Included GPU anti-sag bracket for extra stability #lianli #case #lancool217 #pc #build

LIAN LI Global

21,396 Aufrufe • vor 1 Jahr

QVAC SDK 0.15.0 is live. This release adds multiple prompts batching, brings a native AMD GPU backend to the stack, moves more vision encoders onto mobile GPUs, and adds a second local coding-agent integration. Main highlights: - Prompt batching for the LLM addon. Batch multiple prompts into one job and process them concurrently, with each answer returned the moment its generation finishes. - Native AMD GPU backend. A first-class HIP/ROCm backend in @qvac/vla-ggml, auto-selected over Vulkan with clean fallback when ROCm is absent. - A second local coding agent. OpenClaw joins OpenCode for local, cloud-free agent workflows. AGENTS - OpenCode plugin update (@qvac/opencode-plugin). Aligned with the current SDK, CLI, and AI SDK provider packages. A fresh install runs OpenCode against managed local QVAC models out of the box, from the default qvac/qwen3.5-9b, with no manual qvac serve setup. - OpenClaw plugin (@qvac/openclaw-plugin). A second coding-agent integration alongside OpenCode. A fresh setup installs the plugin, creates a local qvac provider through onboarding, and runs a QVAC model through OpenClaw🦞's local service path. LANGUAGE MODELS - Prompt batching (LLM addon). Batch multiple prompts in one job and run them concurrently, each answer returns the moment its generation finishes, no waiting on the others. - Reasoning-context trimming on hybrid + recurrent models (@qvac/llm-llamacpp). remove_thinking_from_context now works beyond pure-attention models. Same JS API, no throw. VOICE AND SPEECH - Transcription (transcription-parakeet 0.9.0). More robust CPU fallback on GPU failure and a faster Vulkan backend on Pixel 9. - Text-to-speech features (tts-ggml 0.4.0). Adds LavaSR for noise removal and adjustable output frequency up to 48 kHz, plus Japanese via Chatterbox. - Text-to-speech fixes (tts-ggml 0.4.1). CPU fallback on GPU failure, a q8_0 KV crash fix on Metal with Chatterbox. VISION - Qwen3.5 vision encoder on GPU (Android). Image encoder moves onto the phone GPU, with a smarter tile-grid preprocessor and default image-token caps, for flagship Android: Vulkan on Mali (Pixel 9 Pro) and OpenCL on Adreno 830 (Galaxy S25). - Gemma-4 vision encoder on GPU (Android). Vision encoder runs on the phone GPU instead of CPU, same flagship Android targets. PLATFORM AND PERFORMANCE - AMD GPU backend (@qvac/vla-ggml). Native HIP/ROCm backend, auto-selected over Vulkan with clean fallback when ROCm is absent (Linux x64 only). Comes with ~23% faster than Vulkan, ~14% faster than PyTorch-ROCm, parity preserved. Unified code style. A cleaner, more consistent, easier-to-contribute codebase. Let's build. npm install @qvac/sdk

QVAC SDK 0.15.0 is live. This release adds multiple prompts batching, brings a native AMD GPU backend to the stack, moves more vision encoders onto mobile GPUs, and adds a second local coding-agent integration. Main highlights: - Prompt batching for the LLM addon. Batch multiple prompts into one job and process them concurrently, with each answer returned the moment its generation finishes. - Native AMD GPU backend. A first-class HIP/ROCm backend in @qvac/vla-ggml, auto-selected over Vulkan with clean fallback when ROCm is absent. - A second local coding agent. OpenClaw joins OpenCode for local, cloud-free agent workflows. AGENTS - OpenCode plugin update (@qvac/opencode-plugin). Aligned with the current SDK, CLI, and AI SDK provider packages. A fresh install runs OpenCode against managed local QVAC models out of the box, from the default qvac/qwen3.5-9b, with no manual qvac serve setup. - OpenClaw plugin (@qvac/openclaw-plugin). A second coding-agent integration alongside OpenCode. A fresh setup installs the plugin, creates a local qvac provider through onboarding, and runs a QVAC model through OpenClaw🦞's local service path. LANGUAGE MODELS - Prompt batching (LLM addon). Batch multiple prompts in one job and run them concurrently, each answer returns the moment its generation finishes, no waiting on the others. - Reasoning-context trimming on hybrid + recurrent models (@qvac/llm-llamacpp). remove_thinking_from_context now works beyond pure-attention models. Same JS API, no throw. VOICE AND SPEECH - Transcription (transcription-parakeet 0.9.0). More robust CPU fallback on GPU failure and a faster Vulkan backend on Pixel 9. - Text-to-speech features (tts-ggml 0.4.0). Adds LavaSR for noise removal and adjustable output frequency up to 48 kHz, plus Japanese via Chatterbox. - Text-to-speech fixes (tts-ggml 0.4.1). CPU fallback on GPU failure, a q8_0 KV crash fix on Metal with Chatterbox. VISION - Qwen3.5 vision encoder on GPU (Android). Image encoder moves onto the phone GPU, with a smarter tile-grid preprocessor and default image-token caps, for flagship Android: Vulkan on Mali (Pixel 9 Pro) and OpenCL on Adreno 830 (Galaxy S25). - Gemma-4 vision encoder on GPU (Android). Vision encoder runs on the phone GPU instead of CPU, same flagship Android targets. PLATFORM AND PERFORMANCE - AMD GPU backend (@qvac/vla-ggml). Native HIP/ROCm backend, auto-selected over Vulkan with clean fallback when ROCm is absent (Linux x64 only). Comes with ~23% faster than Vulkan, ~14% faster than PyTorch-ROCm, parity preserved. Unified code style. A cleaner, more consistent, easier-to-contribute codebase. Let's build. npm install @qvac/sdk

QVAC

17,700,244 Aufrufe • vor 7 Tagen

locate-anything.cpp: native C++/ggml (ggml) inference for NVIDIA's LocateAnything-3B, open-vocabulary object detection / visual grounding, one of the neat detection VLMs out there, from the LocalAI team. Same detections as the official model, now running anywhere with no Python and faster, on CPU and GPU. Quantized GGUF on Hugging Face 🤗 Huge thanks to Georgi Gerganov for ggml and to NVIDIAAI for releasing LocateAnything! 🧵

locate-anything.cpp: native C++/ggml (ggml) inference for NVIDIA's LocateAnything-3B, open-vocabulary object detection / visual grounding, one of the neat detection VLMs out there, from the LocalAI team. Same detections as the official model, now running anywhere with no Python and faster, on CPU and GPU. Quantized GGUF on Hugging Face 🤗 Huge thanks to Georgi Gerganov for ggml and to NVIDIAAI for releasing LocateAnything! 🧵

Ettore Di Giacinto

13,404 Aufrufe • vor 1 Monat

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than before. tensor parallelism splits the model between machines in parallel instead of sequentially DeepSeek V3 full size without quantization. 183GB per machine. still 25 tokens per second. faster than most people can read Kimi K2 with trillion parameters and 256,000 context window. loaded, responded and adapted memory usage dynamically based on prompt size 4 machines consuming 66 watts total. cloud GPU for the same workload costs $1,900/month a cluster that used to cost millions now assembled from Mac Studios in a few hours

THIS DEVELOPER JUST RAN A TRILLION PARAMETER MODEL ON 4 MAC STUDIOS - 10X FASTER AND 5X CHEAPER THAN CLOUD CODE 19:00 he says it out loud. "we just ran a trillion parameter model. 30 something tokens per second. wow." RDMA over Thunderbolt made the cluster 10x faster than before. tensor parallelism splits the model between machines in parallel instead of sequentially DeepSeek V3 full size without quantization. 183GB per machine. still 25 tokens per second. faster than most people can read Kimi K2 with trillion parameters and 256,000 context window. loaded, responded and adapted memory usage dynamically based on prompt size 4 machines consuming 66 watts total. cloud GPU for the same workload costs $1,900/month a cluster that used to cost millions now assembled from Mac Studios in a few hours

Noisy

79,665 Aufrufe • vor 1 Monat

The infamous hyperspeed bug in Peter Jackson's King Kong is now fixed! The game tracks time with CPU cycles, but casts the frequency to a 32 bit value. On modern high-clock CPUs that overflows, so the engine thinks your CPU is way slower than it actually is. It then crams way too many physics & gameplay events into each tick - and since the real CPU is still blazing fast, everything goes into insane overdrive. Now properly fixed for smooth gameplay on today's hardware. Download link:

The infamous hyperspeed bug in Peter Jackson's King Kong is now fixed! The game tracks time with CPU cycles, but casts the frequency to a 32 bit value. On modern high-clock CPUs that overflows, so the engine thinks your CPU is way slower than it actually is. It then crams way too many physics & gameplay events into each tick - and since the real CPU is still blazing fast, everything goes into insane overdrive. Now properly fixed for smooth gameplay on today's hardware. Download link:

Fusion Fix

72,290 Aufrufe • vor 3 Monaten

Tested: Windows 11’s ‘faster’ File Explorer (preloaded) is still slower than Windows 10. On a 4GB RAM test machine, File Explorer used ~32.4MB of RAM without preloading. After turning on preloading, Windows quietly adds ~35MB more. So Explorer now sits at ~67.4MB in memory before you even open it. 35MB isn’t a big deal in 2025. Your browser eats that for breakfast. That's not the point of my testing. Why is the preloaded File Explorer still slower than Windows 10? I made a video to compare the two versions.

Tested: Windows 11’s ‘faster’ File Explorer (preloaded) is still slower than Windows 10. On a 4GB RAM test machine, File Explorer used ~32.4MB of RAM without preloading. After turning on preloading, Windows quietly adds ~35MB more. So Explorer now sits at ~67.4MB in memory before you even open it. 35MB isn’t a big deal in 2025. Your browser eats that for breakfast. That's not the point of my testing. Why is the preloaded File Explorer still slower than Windows 10? I made a video to compare the two versions.

Windows Latest

752,931 Aufrufe • vor 7 Monaten