Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

I trained a 12M parameter LLM on my own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more. Wrote the full transformer architecture, and BPE tokenizer from scratch. The framework features: - Custom CUDA kernels (Flash Attention, fused LayerNorm, fused GELU) for 3x... increased throughput - Automatic WebGPU fallback for non-NVIDIA devices - TypeScript API with Rust compute backend - One npm install to get started, prebuilt binaries for every platform Try out the model for yourself: Built with Reese Chong. Check out the repos and blog if you want to learn more. Shoutout to Modal for the compute credits allowing me to train on 2 A100 GPUs without going broke cc sunny madra Gavinshow more

Aadi Kulshrestha

4,064 subscribers

829,709 Aufrufe • vor 6 Monaten •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

55 Kommentare

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Everything is open source, and we wrote a thorough blog breaking down everything we did so you guys can learn. Repo: Blog:

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Test the LLM and view the probability distribution of next tokens. All running in the browser using our own framework.

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

As per @ClementDelangue's suggestion, I uploaded the model to HuggingFace. Feel free to check it out and run the model yourself:

Profilbild von clem 🤗

clem 🤗vor 6 Monaten

you should share the model on

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Will most definitely do 🔥

Profilbild von Gavin

Gavinvor 6 Monaten

Love it. Dm me?

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

For sure!

Profilbild von Sanjoy Das

Sanjoy Dasvor 6 Monaten

Nice! Would love to get your take on cuTile Rust:

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Looks interesting, will play around with it a bit and share my thoughts 👀

Profilbild von Bryce, the CUDA Colonel

Bryce, the CUDA Colonelvor 6 Monaten

DM me if you wanna chat about Rust for GPU programming!

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Will definitely reach out, even before I got into AI I did a lot of GPU programming work related to game development and rendering 🔥

Profilbild von Taelin

Taelinvor 6 Monaten

cost?

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

I spent ~$100 CAD of my own money, but @modal gave me ~$400 free credits at a hackathon a while back. The total was $500 of cloud compute

Profilbild von stan p

stan pvor 6 Monaten

waterloo strikes again

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

have to live up to the name

Profilbild von Ken Wu

Ken Wuvor 6 Monaten

how did i get into the same school as this beast

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Lmaooo 🙈

Profilbild von Kenny Guo

Kenny Guovor 6 Monaten

first principles 🫡

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Yessir 🔥

Profilbild von dnu

dnuvor 6 Monaten

respect. writing your own framework is one thing, getting a full train run to converge is the part that usually gets ugly. what broke first for you, kernels or optimizer math?

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

We had a lot of bugs with our CUDA kernels that had to be addressed before the training started to go well 😂

Profilbild von Yechan Do

Yechan Dovor 6 Monaten

This is so impressive. Can I use your work at my lecture for education purpose ?

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

@yechan_ai Yes of course, we’re honoured 🔥

Profilbild von Saurabh Kumar

Saurabh Kumarvor 6 Monaten

What are they feeding Waterloo undergrads

Profilbild von ayush

ayushvor 6 Monaten

my goats @_reesechong and @MankyDankyBanky this is insane

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

@_reesechong Ty bro 🙏

Profilbild von Xander Chin

Xander Chinvor 6 Monaten

fire

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Appreciate it, TinyTPU was a big inspo

Profilbild von ani

anivor 6 Monaten

🐐

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

nah you inspired me fr

Profilbild von Ricky Tang

Ricky Tangvor 6 Monaten

this is crazy bro

Profilbild von alexine 🏴‍☠️

alexine 🏴‍☠️vor 6 Monaten

very cool, great work!

Profilbild von EDITH

EDITHvor 6 Monaten

This is fucking great man!!!

Profilbild von Kaibo Huang

Kaibo Huangvor 6 Monaten

@lucashjin good stuff aadi🔥

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

@lucashjin Thanks Kaibo looking forward to Bear hacks

Profilbild von mikael haji

mikael hajivor 6 Monaten

very nice!

Profilbild von Aryan Dhawan

Aryan Dhawanvor 6 Monaten

Amazing

Profilbild von Akram Jamil

Akram Jamilvor 6 Monaten

@nicholaschen__ cracked

Profilbild von Benedict Kerres

Benedict Kerresvor 6 Monaten

Well done

Profilbild von Rhys Campbell

Rhys Campbellvor 6 Monaten

@hnshah Part time model builder, full time looksmaxxer

Profilbild von Freeman Jiang

Freeman Jiangvor 6 Monaten

good stuff

Profilbild von Moulik

Moulikvor 6 Monaten

holy cracked

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

thanks moulik!

Profilbild von NILAY1556

NILAY1556vor 6 Monaten

It's been my dream for long time to do something like that , impressive work Happy for you guyz to see you got much recognize by big giants Just great

Profilbild von evan

evanvor 6 Monaten

nice work!!

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Thanks so much Evan!

Profilbild von Nikhil Sonti

Nikhil Sontivor 6 Monaten

training a transformer from scratch used to be frontier lab work now random devs ship full implementations with custom CUDA kernels... Rust backends... BPE tokenizers built from zero as side projects what took entire teams 2 years ago is now weekend hobby territory the commodification speed is wild

Profilbild von Lucas Jin

Lucas Jinvor 6 Monaten

@imdonghakim w aadi

Profilbild von sanjay prasad hs

sanjay prasad hsvor 6 Monaten

Super cool stuff, man, super cool.

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Thanks man!

Profilbild von benkigera

benkigeravor 6 Monaten

Reminds me of what LLMs are doing under the hood.

Profilbild von Dev Shorya

Dev Shoryavor 6 Monaten

thats actually insane. most people talk about it, you actually built it. the rust + cuda combo is hard mode too

Profilbild von Ajith Bondili

Ajith Bondilivor 6 Monaten

@anirudhbv_ce W

Profilbild von 1jmdev

1jmdevvor 6 Monaten

Please update the demo page so it caches the model on user device it is really cool but please cache it on user device

Profilbild von Aadi Kulshrestha

Aadi Kulshresthavor 6 Monaten

Good point, will try to get this out soon!

Ähnliche Videos

I implemented Google Research's TurboQuant as a CUDA-native compression engine on Blackwell B200. 5x KV cache compression on Qwen 2.5-1.5B, near-loseless attention scores, generating live from compressed memory. 5 custom cuTile CUDA kernels ft: - fused attention (with QJL corrections) - online softmax -on-chip cache decompression - pipelined TMA loads Try it out: s/o Bryce, the CUDA Colonel and the cuTile team at NVIDIA for lending me Blackwell GPU access :) cc sunny madra Gavin

I implemented Google Research's TurboQuant as a CUDA-native compression engine on Blackwell B200. 5x KV cache compression on Qwen 2.5-1.5B, near-loseless attention scores, generating live from compressed memory. 5 custom cuTile CUDA kernels ft: - fused attention (with QJL corrections) - online softmax -on-chip cache decompression - pipelined TMA loads Try it out: s/o Bryce, the CUDA Colonel and the cuTile team at NVIDIA for lending me Blackwell GPU access :) cc sunny madra Gavin

ani

819,707 Aufrufe • vor 6 Monaten

Check out mistral.rs, our #Rust-based open source inference engine allowing for fast #LLM serving for a variety of architectures including X-LoRA mixture-of-expert (MoE) models, Llama-3, Mistral/Mixtral, Gemma & many others. Built on the Hugging Face #Candle framework for #Rust w/ custom CUDA kernels in the backend (as well as support for Metal, Apple Accelerate, and Intel MKL for CPU use), you can easily create a REST API OpenAI compatible server or run via Python bindings. Key features include: ✅Prefix caching, continuous batching ✅Flash Attention V2 ✅Device offloading ✅GGUF or Hugging Face models ✅2, 3, 4, 5, 6 and 8 bit quantization ✅X-LoRA MoE non-granular scalings for fast inference ✅Grammar support ✅Continuous batching ✅LoRA support with weight merging ✅LlamaIndex 🦙 integration ...and much more. Incorporation into our GraphReasoning multi-agent modeling framework & LlamaIndex 🦙 allows you to combine in-context learning with adversarial agentic strategies, to dive deep into complex scientific analyses, such as to predict material behaviors, generate hypotheses, analyze papers and data, develop new research concepts, and much more. Check out mistral.rs: Join our Discord here: Rust Trending Rust Language

Check out mistral.rs, our #Rust-based open source inference engine allowing for fast #LLM serving for a variety of architectures including X-LoRA mixture-of-expert (MoE) models, Llama-3, Mistral/Mixtral, Gemma & many others. Built on the Hugging Face #Candle framework for #Rust w/ custom CUDA kernels in the backend (as well as support for Metal, Apple Accelerate, and Intel MKL for CPU use), you can easily create a REST API OpenAI compatible server or run via Python bindings. Key features include: ✅Prefix caching, continuous batching ✅Flash Attention V2 ✅Device offloading ✅GGUF or Hugging Face models ✅2, 3, 4, 5, 6 and 8 bit quantization ✅X-LoRA MoE non-granular scalings for fast inference ✅Grammar support ✅Continuous batching ✅LoRA support with weight merging ✅LlamaIndex 🦙 integration ...and much more. Incorporation into our GraphReasoning multi-agent modeling framework & LlamaIndex 🦙 allows you to combine in-context learning with adversarial agentic strategies, to dive deep into complex scientific analyses, such as to predict material behaviors, generate hypotheses, analyze papers and data, develop new research concepts, and much more. Check out mistral.rs: Join our Discord here: Rust Trending Rust Language

Markus J. Buehler

73,581 Aufrufe • vor 2 Jahren

>build a 1.5B VLA on top of MiniCPM-V 4.6 >compete with π0.5 and Qwen-VLA >use roughly half the visual-processing compute >remember previous steps without blowing up inference cost >pull the network plug >let the robot keep tracking — fully local, fully onboard >with CUDA Graph optimization and custom Triton fused kernels, MiniCPM-Robot inference throughput goes from 10 Hz → 33 Hz, and up to 36 Hz on the same NVIDIA H20 >release the models and framework as open source so why does robotics still rely on bigger models and the cloud? OpenBMB just dropped MiniCPM-Robot at WAIC two models + one inference framework 🧵

>build a 1.5B VLA on top of MiniCPM-V 4.6 >compete with π0.5 and Qwen-VLA >use roughly half the visual-processing compute >remember previous steps without blowing up inference cost >pull the network plug >let the robot keep tracking — fully local, fully onboard >with CUDA Graph optimization and custom Triton fused kernels, MiniCPM-Robot inference throughput goes from 10 Hz → 33 Hz, and up to 36 Hz on the same NVIDIA H20 >release the models and framework as open source so why does robotics still rely on bigger models and the cloud? OpenBMB just dropped MiniCPM-Robot at WAIC two models + one inference framework 🧵

arc.

48,279 Aufrufe • vor 3 Monaten

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

Awni Hannun

42,844 Aufrufe • vor 1 Jahr

Just launched #CES2026, the new open-source NVIDIA Nemotron Speech ASR model is here to solve latency drift and redundant compute. Its cache-aware streaming architecture eliminates the need for buffered inference, giving you stable, sub-100ms latency (24ms median T-T-F) and up to 3x more throughput on your GPU. 🤗 Read the technical blog with real-world results from Daily and Modal on Hugging Face:

Just launched #CES2026, the new open-source NVIDIA Nemotron Speech ASR model is here to solve latency drift and redundant compute. Its cache-aware streaming architecture eliminates the need for buffered inference, giving you stable, sub-100ms latency (24ms median T-T-F) and up to 3x more throughput on your GPU. 🤗 Read the technical blog with real-world results from Daily and Modal on Hugging Face:

NVIDIA AI Developer

138,518 Aufrufe • vor 9 Monaten

CMU PhD who built the kernels NVIDIA now ships in TensorRT-LLM explained fast attention in 68 minutes - better than $1200 GPU programming courses. pick the attention pattern -> generate a fused CUDA kernel -> drop it into vLLM/SGLang -> same GPU, way more tokens per second. That loop is why FlashInfer now powers inference at NVIDIA, vLLM, SGLang, and half the serving stacks you use. FlashInfer + Triton + JIT-compiled kernels + paged-KV attention - that's the stack.

CMU PhD who built the kernels NVIDIA now ships in TensorRT-LLM explained fast attention in 68 minutes - better than $1200 GPU programming courses. pick the attention pattern -> generate a fused CUDA kernel -> drop it into vLLM/SGLang -> same GPU, way more tokens per second. That loop is why FlashInfer now powers inference at NVIDIA, vLLM, SGLang, and half the serving stacks you use. FlashInfer + Triton + JIT-compiled kernels + paged-KV attention - that's the stack.

h100envy

32,605 Aufrufe • vor 3 Monaten

underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it built a triton-class compiler from scratch, MiniTriton, that delivers performance on par with or better than Triton and torch.compile > then it designed a chip, by a model, for a model, in one 48-hour autonomous run the model is rewriting and optimizing every layer of the stack it runs on: kernels for its own training. A compiler for its own kernels. silicon for its own weights. we're watching software build its own hardware

underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it built a triton-class compiler from scratch, MiniTriton, that delivers performance on par with or better than Triton and torch.compile > then it designed a chip, by a model, for a model, in one 48-hour autonomous run the model is rewriting and optimizing every layer of the stack it runs on: kernels for its own training. A compiler for its own kernels. silicon for its own weights. we're watching software build its own hardware

Alex Ker 🔭

311,780 Aufrufe • vor 3 Monaten

Congrats to the Kimi.ai team! This is awesome. Great to see this level of research coming from open-source frontier model labs. I liked the paper so much I built a Rust implementation of it ;) Full AttnRes + Block AttnRes with two-phase inference, built using Burn (tensor library and Deep Learning Framework, in Rust, by Tracel AI). Runs on CPU, CUDA, Metal, wgpu. Includes an interactive TUI that trains a model live and visualizes depth attention evolving from uniform to selective in real time. Repo link and more on what is implemented in the comments.

Congrats to the Kimi.ai team! This is awesome. Great to see this level of research coming from open-source frontier model labs. I liked the paper so much I built a Rust implementation of it ;) Full AttnRes + Block AttnRes with two-phase inference, built using Burn (tensor library and Deep Learning Framework, in Rust, by Tracel AI). Runs on CPU, CUDA, Metal, wgpu. Includes an interactive TUI that trains a model live and visualizes depth attention evolving from uniform to selective in real time. Repo link and more on what is implemented in the comments.

abdel

95,888 Aufrufe • vor 7 Monaten

Inkling, Thinking Machines' first open model, dropped today: 975B total / 41B active MoE, up to 1M context, reasoning natively over text, images, and audio. Serving and RL support are already live: you can run and shape it on an open stack, starting now. Day 0 support on SGLang SGLang and Miles RadixArk👇 - Inkling's new architecture (ShortConv, attention with relative positional embedding, shared expert sink MoE) is natively implemented and deeply optimized, with prefill full CUDA graph and MXFP8 KV cache - Full parameter and LoRA RL in a customized Megatron backend, train inference consistency via customized kernels, routing replay, and cross-runtime parameter synchronization - DFlash speculative decoding from Modal for low-latency serving Launching now, blog and cookbook in the comments ⬇️

Inkling, Thinking Machines' first open model, dropped today: 975B total / 41B active MoE, up to 1M context, reasoning natively over text, images, and audio. Serving and RL support are already live: you can run and shape it on an open stack, starting now. Day 0 support on SGLang SGLang and Miles RadixArk👇 - Inkling's new architecture (ShortConv, attention with relative positional embedding, shared expert sink MoE) is natively implemented and deeply optimized, with prefill full CUDA graph and MXFP8 KV cache - Full parameter and LoRA RL in a customized Megatron backend, train inference consistency via customized kernels, routing replay, and cross-runtime parameter synchronization - DFlash speculative decoding from Modal for low-latency serving Launching now, blog and cookbook in the comments ⬇️

LMSYS Org

146,680 Aufrufe • vor 3 Monaten

Ben Thompson explains how LLMs greatly diminished Nvidia's CUDA moat even as they sent the stock to the moon "So the weird thing about large language models is they were obviously incredible for Nvidia. That's why their stock went to the moon." "They have been on and off the most valuable company in the world." "It was also very bad for Nvidia. And the reason it was bad for Nvidia is that the play with CUDA is to build a developer ecosystem on top of CUDA." "But CUDA only works on Nvidia GPUs. So you get CUDA for free. It's easier to use, and it's a tremendous investment. Nvidia almost went under trying to build CUDA at a time when no one understood what they were doing or why they were wasting money on it." "And that's why Jensen Huang will get bristly, particularly when people question their rent-seeking or profit, whatever. It's like, no, they earned their spot fair and square." "Absolutely. It shouldn't be forgotten. They have earned every dollar they've gotten through 25 years of taking massive risks." "It bottomed out in October 2022. I wrote an article like three weeks before ChatGPT came out, tracing their bottoming-out history and their search for what was next." "'Nvidia in the Valley.' So, go back to this GTC. So I wrote an article at the time called 'Nvidia Waves and Moats'." "And what was interesting about that GTC was, number one, it was very boring. All the cool stuff kind of got scrubbed out." "Now, Jensen Huang has brought that stuff back, so the last few GTCs he's more talking about other things. Now it comes across as, oh, you're still looking for something beyond the LLM." "Because the problem with the LLM is it shifts the developer platform far above where Nvidia sits. All the activity is happening on top of LLMs. And so no one who's writing an AI application today is using CUDA." "Now, some people are, if you're training your own model and you're doing some low-level things or non-LLM things." "But the vast majority of the energy and all the money and the ecosystem is far removed from CUDA." "They have no idea and don't need to know or care what chips their application is running on. They're just on the OpenAI API, or the Anthropic API, or using Bedrock on Amazon, and it's sitting on Trainium, and they're using a Chinese open-source model. It's totally abstracted away, and this is why LLMs were bad for Nvidia." "Now, again, all the money they made along the way is worth it, but their moat has been tremendously diminished." "CUDA is still a moat if you need to do stuff that requires CUDA. But the vast majority of stuff, in energy, doesn't require CUDA, like in a post-LLM world."

Ben Thompson explains how LLMs greatly diminished Nvidia's CUDA moat even as they sent the stock to the moon "So the weird thing about large language models is they were obviously incredible for Nvidia. That's why their stock went to the moon." "They have been on and off the most valuable company in the world." "It was also very bad for Nvidia. And the reason it was bad for Nvidia is that the play with CUDA is to build a developer ecosystem on top of CUDA." "But CUDA only works on Nvidia GPUs. So you get CUDA for free. It's easier to use, and it's a tremendous investment. Nvidia almost went under trying to build CUDA at a time when no one understood what they were doing or why they were wasting money on it." "And that's why Jensen Huang will get bristly, particularly when people question their rent-seeking or profit, whatever. It's like, no, they earned their spot fair and square." "Absolutely. It shouldn't be forgotten. They have earned every dollar they've gotten through 25 years of taking massive risks." "It bottomed out in October 2022. I wrote an article like three weeks before ChatGPT came out, tracing their bottoming-out history and their search for what was next." "'Nvidia in the Valley.' So, go back to this GTC. So I wrote an article at the time called 'Nvidia Waves and Moats'." "And what was interesting about that GTC was, number one, it was very boring. All the cool stuff kind of got scrubbed out." "Now, Jensen Huang has brought that stuff back, so the last few GTCs he's more talking about other things. Now it comes across as, oh, you're still looking for something beyond the LLM." "Because the problem with the LLM is it shifts the developer platform far above where Nvidia sits. All the activity is happening on top of LLMs. And so no one who's writing an AI application today is using CUDA." "Now, some people are, if you're training your own model and you're doing some low-level things or non-LLM things." "But the vast majority of the energy and all the money and the ecosystem is far removed from CUDA." "They have no idea and don't need to know or care what chips their application is running on. They're just on the OpenAI API, or the Anthropic API, or using Bedrock on Amazon, and it's sitting on Trainium, and they're using a Chinese open-source model. It's totally abstracted away, and this is why LLMs were bad for Nvidia." "Now, again, all the money they made along the way is worth it, but their moat has been tremendously diminished." "CUDA is still a moat if you need to do stuff that requires CUDA. But the vast majority of stuff, in energy, doesn't require CUDA, like in a post-LLM world."

Fireside Alpha

12,980 Aufrufe • vor 2 Monaten

Ubuntu 26.04 LTS, codenamed #ResoluteRaccoon, is now available to download. 🦝 Resolute Raccoon builds on the resilience-focused improvements introduced in interim releases, with TPM-backed full-disk encryption, improved support for application permission prompting, Livepatch updates for Arm-based servers, and Rust-based utilities for enhanced memory safety. This release also brings native support for industry-leading AI/ML toolkits like NVIDIA CUDA and AMD ROCm, making Ubuntu 26.04 LTS the ideal platform for AI development and production workloads. Install now: Learn more about the release:

Ubuntu 26.04 LTS, codenamed #ResoluteRaccoon, is now available to download. 🦝 Resolute Raccoon builds on the resilience-focused improvements introduced in interim releases, with TPM-backed full-disk encryption, improved support for application permission prompting, Livepatch updates for Arm-based servers, and Rust-based utilities for enhanced memory safety. This release also brings native support for industry-leading AI/ML toolkits like NVIDIA CUDA and AMD ROCm, making Ubuntu 26.04 LTS the ideal platform for AI development and production workloads. Install now: Learn more about the release:

Ubuntu

654,541 Aufrufe • vor 6 Monaten

$Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.$

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,496,370 Aufrufe • vor 5 Monaten

Learn how to Use the Blendr Platform 📚 Check out the video attached which will walk you through the Blendr Platform step-by-step. We are covering exactly how to Log onto the platform with your wallet, Lend your own GPUs and Rent other GPUs on the network. Once you're confident, head over to to try it out for yourself!

Learn how to Use the Blendr Platform 📚 Check out the video attached which will walk you through the Blendr Platform step-by-step. We are covering exactly how to Log onto the platform with your wallet, Lend your own GPUs and Rent other GPUs on the network. Once you're confident, head over to to try it out for yourself!

Blendr Network

24,382 Aufrufe • vor 2 Jahren

Luminal ( is creating PyTorch for Production – an ML compiler that generates blazingly fast CUDA kernels and makes deploying to production one line of code. Congrats on the launch, Jake Stevens, Joe Fioti, and Matthew Gunton!

Luminal ( is creating PyTorch for Production – an ML compiler that generates blazingly fast CUDA kernels and makes deploying to production one line of code. Congrats on the launch, Jake Stevens, Joe Fioti, and Matthew Gunton!

Y Combinator

98,496 Aufrufe • vor 1 Jahr

What does “fungible” mean for AI infrastructure? The flexibility to run different workloads: every type of AI, every phase of AI and even workloads that aren’t AI at all. One platform, built for maximum “productivity”, “durable” years after deployment, and “fungible” for every model and workload. Read the blog to learn more:

What does “fungible” mean for AI infrastructure? The flexibility to run different workloads: every type of AI, every phase of AI and even workloads that aren’t AI at all. One platform, built for maximum “productivity”, “durable” years after deployment, and “fungible” for every model and workload. Read the blog to learn more:

NVIDIA

77,813 Aufrufe • vor 27 Tagen

The best way to learn CUDA is by solving problems. In this video, I implement 7 beginner CUDA kernels from scratch: • Vector Addition • Matrix Transpose • Matrix Addition • RGB to Grayscale • Value Clipping • Sigmoid Activation • Interleave Arrays If you're getting started with CUDA programming, this should give you a solid foundation.

The best way to learn CUDA is by solving problems. In this video, I implement 7 beginner CUDA kernels from scratch: • Vector Addition • Matrix Transpose • Matrix Addition • RGB to Grayscale • Value Clipping • Sigmoid Activation • Interleave Arrays If you're getting started with CUDA programming, this should give you a solid foundation.

voxium

36,987 Aufrufe • vor 3 Monaten

Finally, I'm done with the basics of backend engineering on the all-in-one resources to learn backend engineering I'm working on This basic section includes - Internet - HTTP - Servers - Web Dev fundamentals - Fundamentals of Operating Systems - A server-side language -> JavaScript | Go | Rust | Node - A framework -> Express | Nestjs | Django | Laravel | Spring Boot This is the boring part. Don't worry. We are getting into the fun part soon. Still working on it, updating it, Adding more, and making it better for you. Follow me. Retweet. Comment "Backend." I will DM you the PDF version when it's out. Enjoy the web version, and send in your feedback on how we can improve it for everyone.

Finally, I'm done with the basics of backend engineering on the all-in-one resources to learn backend engineering I'm working on This basic section includes - Internet - HTTP - Servers - Web Dev fundamentals - Fundamentals of Operating Systems - A server-side language -> JavaScript | Go | Rust | Node - A framework -> Express | Nestjs | Django | Laravel | Spring Boot This is the boring part. Don't worry. We are getting into the fun part soon. Still working on it, updating it, Adding more, and making it better for you. Follow me. Retweet. Comment "Backend." I will DM you the PDF version when it's out. Enjoy the web version, and send in your feedback on how we can improve it for everyone.

Solomon Eseme

92,216 Aufrufe • vor 3 Jahren

Basic Guide to Playing MOG A quick guide to help beginners get familiar with the game and start farming corn for a Yield Fields (🧑‍🌾,🧑‍🌾) whitelist spot. Shoutout to skarly for putting more attention on the MOG grind 👀 Support me if you want:

Basic Guide to Playing MOG A quick guide to help beginners get familiar with the game and start farming corn for a Yield Fields (🧑‍🌾,🧑‍🌾) whitelist spot. Shoutout to skarly for putting more attention on the MOG grind 👀 Support me if you want:

John Huỳnh

25,172 Aufrufe • vor 1 Monat

Compute has always been core to the Verne Robotics story. Getting there meant negotiating long, expensive contracts with big neoclouds, and projecting compute needs years into the future. Verne wasn't ready for that. Then came the Y Combinator cluster, built on Together AI. Verne trained a model 3x larger, on almost 100x more data. More data in, bigger foundation models out.

Compute has always been core to the Verne Robotics story. Getting there meant negotiating long, expensive contracts with big neoclouds, and projecting compute needs years into the future. Verne wasn't ready for that. Then came the Y Combinator cluster, built on Together AI. Verne trained a model 3x larger, on almost 100x more data. More data in, bigger foundation models out.

Together AI

21,649 Aufrufe • vor 2 Monaten

YOUR PARENTS PAID FOR THE CUDA MOAT! The #1 contributor to the CUDA MOAT isn't the the developers at NVIDIA, but it is the millions of developers outside of NVIDIA that invent new algorithms for CUDA like Flash Attention. For most of them, it started with an GeForce gaming GPU. NVIDIA is the only companies that has an reasonable good developer stack on consumer grade GPUs. As people grow up beyond playing CSGO & League of Legends & Minecraft, they either become anime weeaboos or they start programming on their existing computer with has an GeForce GPU

YOUR PARENTS PAID FOR THE CUDA MOAT! The #1 contributor to the CUDA MOAT isn't the the developers at NVIDIA, but it is the millions of developers outside of NVIDIA that invent new algorithms for CUDA like Flash Attention. For most of them, it started with an GeForce gaming GPU. NVIDIA is the only companies that has an reasonable good developer stack on consumer grade GPUs. As people grow up beyond playing CSGO & League of Legends & Minecraft, they either become anime weeaboos or they start programming on their existing computer with has an GeForce GPU

SemiAnalysis

25,230 Aufrufe • vor 6 Monaten