Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

I trained a 12M parameter LLM on my own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more. Wrote the full transformer architecture, and BPE tokenizer from scratch. The framework features: - Custom CUDA kernels (Flash Attention, fused LayerNorm, fused GELU) for 3x...

MankyDankyBanky's profile picture

Aadi Kulshrestha

4,064 subscribers

829,709 görüntüleme • 6 ay önce •via X (Twitter)

EğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

55 Yorum

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Everything is open source, and we wrote a thorough blog breaking down everything we did so you guys can learn. Repo: Blog:

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Test the LLM and view the probability distribution of next tokens. All running in the browser using our own framework.

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

As per @ClementDelangue's suggestion, I uploaded the model to HuggingFace. Feel free to check it out and run the model yourself:

clem 🤗 profil fotoğrafı
clem 🤗6 ay önce

you should share the model on

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Will most definitely do 🔥

Gavin profil fotoğrafı
Gavin6 ay önce

Love it. Dm me?

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

For sure!

Sanjoy Das profil fotoğrafı
Sanjoy Das6 ay önce

Nice! Would love to get your take on cuTile Rust:

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Looks interesting, will play around with it a bit and share my thoughts 👀

Bryce, the CUDA Colonel profil fotoğrafı
Bryce, the CUDA Colonel6 ay önce

DM me if you wanna chat about Rust for GPU programming!

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Will definitely reach out, even before I got into AI I did a lot of GPU programming work related to game development and rendering 🔥

Taelin profil fotoğrafı
Taelin6 ay önce

cost?

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

I spent ~$100 CAD of my own money, but @modal gave me ~$400 free credits at a hackathon a while back. The total was $500 of cloud compute

stan p profil fotoğrafı
stan p6 ay önce

waterloo strikes again

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

have to live up to the name

Ken Wu profil fotoğrafı
Ken Wu6 ay önce

how did i get into the same school as this beast

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Lmaooo 🙈

Kenny Guo profil fotoğrafı
Kenny Guo6 ay önce

first principles 🫡

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Yessir 🔥

dnu profil fotoğrafı
dnu6 ay önce

respect. writing your own framework is one thing, getting a full train run to converge is the part that usually gets ugly. what broke first for you, kernels or optimizer math?

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

We had a lot of bugs with our CUDA kernels that had to be addressed before the training started to go well 😂

Yechan Do profil fotoğrafı
Yechan Do6 ay önce

This is so impressive. Can I use your work at my lecture for education purpose ?

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

@yechan_ai Yes of course, we’re honoured 🔥

Saurabh Kumar profil fotoğrafı
Saurabh Kumar6 ay önce

What are they feeding Waterloo undergrads

ayush profil fotoğrafı
ayush6 ay önce

my goats @_reesechong and @MankyDankyBanky this is insane

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

@_reesechong Ty bro 🙏

Xander Chin profil fotoğrafı
Xander Chin6 ay önce

fire

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Appreciate it, TinyTPU was a big inspo

ani profil fotoğrafı
ani6 ay önce

🐐

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

nah you inspired me fr

Ricky Tang profil fotoğrafı
Ricky Tang6 ay önce

this is crazy bro

alexine 🏴‍☠️ profil fotoğrafı
alexine 🏴‍☠️6 ay önce

very cool, great work!

EDITH profil fotoğrafı
EDITH6 ay önce

This is fucking great man!!!

Kaibo Huang profil fotoğrafı
Kaibo Huang6 ay önce

@lucashjin good stuff aadi🔥

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

@lucashjin Thanks Kaibo looking forward to Bear hacks

mikael haji profil fotoğrafı
mikael haji6 ay önce

very nice!

Aryan Dhawan profil fotoğrafı
Aryan Dhawan6 ay önce

Amazing

Akram Jamil profil fotoğrafı
Akram Jamil6 ay önce

@nicholaschen__ cracked

Benedict Kerres profil fotoğrafı
Benedict Kerres6 ay önce

Well done

Rhys Campbell profil fotoğrafı
Rhys Campbell6 ay önce

@hnshah Part time model builder, full time looksmaxxer

Freeman Jiang profil fotoğrafı
Freeman Jiang6 ay önce

good stuff

Moulik profil fotoğrafı
Moulik6 ay önce

holy cracked

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

thanks moulik!

NILAY1556 profil fotoğrafı
NILAY15566 ay önce

It's been my dream for long time to do something like that , impressive work Happy for you guyz to see you got much recognize by big giants Just great

evan profil fotoğrafı
evan6 ay önce

nice work!!

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Thanks so much Evan!

Nikhil Sonti profil fotoğrafı
Nikhil Sonti6 ay önce

training a transformer from scratch used to be frontier lab work now random devs ship full implementations with custom CUDA kernels... Rust backends... BPE tokenizers built from zero as side projects what took entire teams 2 years ago is now weekend hobby territory the commodification speed is wild

Lucas Jin profil fotoğrafı
Lucas Jin6 ay önce

@imdonghakim w aadi

sanjay prasad hs profil fotoğrafı
sanjay prasad hs6 ay önce

Super cool stuff, man, super cool.

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Thanks man!

benkigera profil fotoğrafı
benkigera6 ay önce

Reminds me of what LLMs are doing under the hood.

Dev Shorya profil fotoğrafı
Dev Shorya6 ay önce

thats actually insane. most people talk about it, you actually built it. the rust + cuda combo is hard mode too

Ajith Bondili profil fotoğrafı
Ajith Bondili6 ay önce

@anirudhbv_ce W

1jmdev profil fotoğrafı
1jmdev6 ay önce

Please update the demo page so it caches the model on user device it is really cool but please cache it on user device

Aadi Kulshrestha profil fotoğrafı
Aadi Kulshrestha6 ay önce

Good point, will try to get this out soon!

Benzer Videolar

I implemented Google Research's TurboQuant as a CUDA-native compression engine on Blackwell B200. 5x KV cache compression on Qwen 2.5-1.5B, near-loseless attention scores, generating live from compressed memory. 5 custom cuTile CUDA kernels ft: - fused attention (with QJL corrections) - online softmax -on-chip cache decompression - pipelined TMA loads Try it out: s/o Bryce, the CUDA Colonel and the cuTile team at NVIDIA for lending me Blackwell GPU access :) cc sunny madra Gavin
4:46

I implemented Google Research's TurboQuant as a CUDA-native compression engine on Blackwell B200. 5x KV cache compression on Qwen 2.5-1.5B, near-loseless attention scores, generating live from compressed memory. 5 custom cuTile CUDA kernels ft: - fused attention (with QJL corrections) - online softmax -on-chip cache decompression - pipelined TMA loads Try it out: s/o Bryce, the CUDA Colonel and the cuTile team at NVIDIA for lending me Blackwell GPU access :) cc sunny madra Gavin

ani

819,707 görüntüleme • 6 ay önce

Check out mistral.​rs, our #Rust-based open source inference engine allowing for fast #LLM serving for a variety of architectures including X-LoRA mixture-of-expert (MoE) models, Llama-3, Mistral/Mixtral, Gemma & many others. Built on the Hugging Face #Candle framework for #Rust w/ custom CUDA kernels in the backend (as well as support for Metal, Apple Accelerate, and Intel MKL for CPU use), you can easily create a REST API OpenAI compatible server or run via Python bindings. Key features include: ✅Prefix caching, continuous batching ✅Flash Attention V2 ✅Device offloading ✅GGUF or Hugging Face models ✅2, 3, 4, 5, 6 and 8 bit quantization ✅X-LoRA MoE non-granular scalings for fast inference ✅Grammar support ✅Continuous batching ✅LoRA support with weight merging ✅LlamaIndex 🦙 integration ...and much more. Incorporation into our GraphReasoning multi-agent modeling framework & LlamaIndex 🦙 allows you to combine in-context learning with adversarial agentic strategies, to dive deep into complex scientific analyses, such as to predict material behaviors, generate hypotheses, analyze papers and data, develop new research concepts, and much more. Check out mistral.​rs: Join our Discord here: Rust Trending Rust Language
0:35

Check out mistral.​rs, our #Rust-based open source inference engine allowing for fast #LLM serving for a variety of architectures including X-LoRA mixture-of-expert (MoE) models, Llama-3, Mistral/Mixtral, Gemma & many others. Built on the Hugging Face #Candle framework for #Rust w/ custom CUDA kernels in the backend (as well as support for Metal, Apple Accelerate, and Intel MKL for CPU use), you can easily create a REST API OpenAI compatible server or run via Python bindings. Key features include: ✅Prefix caching, continuous batching ✅Flash Attention V2 ✅Device offloading ✅GGUF or Hugging Face models ✅2, 3, 4, 5, 6 and 8 bit quantization ✅X-LoRA MoE non-granular scalings for fast inference ✅Grammar support ✅Continuous batching ✅LoRA support with weight merging ✅LlamaIndex 🦙 integration ...and much more. Incorporation into our GraphReasoning multi-agent modeling framework & LlamaIndex 🦙 allows you to combine in-context learning with adversarial agentic strategies, to dive deep into complex scientific analyses, such as to predict material behaviors, generate hypotheses, analyze papers and data, develop new research concepts, and much more. Check out mistral.​rs: Join our Discord here: Rust Trending Rust Language

Markus J. Buehler

73,581 görüntüleme • 2 yıl önce

>build a 1.5B VLA on top of MiniCPM-V 4.6 >compete with π0.5 and Qwen-VLA >use roughly half the visual-processing compute >remember previous steps without blowing up inference cost >pull the network plug >let the robot keep tracking — fully local, fully onboard >with CUDA Graph optimization and custom Triton fused kernels, MiniCPM-Robot inference throughput goes from 10 Hz → 33 Hz, and up to 36 Hz on the same NVIDIA H20 >release the models and framework as open source so why does robotics still rely on bigger models and the cloud? OpenBMB just dropped MiniCPM-Robot at WAIC two models + one inference framework 🧵
0:27

>build a 1.5B VLA on top of MiniCPM-V 4.6 >compete with π0.5 and Qwen-VLA >use roughly half the visual-processing compute >remember previous steps without blowing up inference cost >pull the network plug >let the robot keep tracking — fully local, fully onboard >with CUDA Graph optimization and custom Triton fused kernels, MiniCPM-Robot inference throughput goes from 10 Hz → 33 Hz, and up to 36 Hz on the same NVIDIA H20 >release the models and framework as open source so why does robotics still rely on bigger models and the cloud? OpenBMB just dropped MiniCPM-Robot at WAIC two models + one inference framework 🧵

arc.

48,279 görüntüleme • 3 ay önce

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:
0:20

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

Awni Hannun

42,844 görüntüleme • 1 yıl önce

Just launched #CES2026, the new open-source NVIDIA Nemotron Speech ASR model is here to solve latency drift and redundant compute. Its cache-aware streaming architecture eliminates the need for buffered inference, giving you stable, sub-100ms latency (24ms median T-T-F) and up to 3x more throughput on your GPU. 🤗 Read the technical blog with real-world results from Daily and Modal on Hugging Face:
1:24

Just launched #CES2026, the new open-source NVIDIA Nemotron Speech ASR model is here to solve latency drift and redundant compute. Its cache-aware streaming architecture eliminates the need for buffered inference, giving you stable, sub-100ms latency (24ms median T-T-F) and up to 3x more throughput on your GPU. 🤗 Read the technical blog with real-world results from Daily and Modal on Hugging Face:

NVIDIA AI Developer

138,518 görüntüleme • 9 ay önce

CMU PhD who built the kernels NVIDIA now ships in TensorRT-LLM explained fast attention in 68 minutes - better than $1200 GPU programming courses. pick the attention pattern -> generate a fused CUDA kernel -> drop it into vLLM/SGLang -> same GPU, way more tokens per second. That loop is why FlashInfer now powers inference at NVIDIA, vLLM, SGLang, and half the serving stacks you use. FlashInfer + Triton + JIT-compiled kernels + paged-KV attention - that's the stack.
1:08:48

CMU PhD who built the kernels NVIDIA now ships in TensorRT-LLM explained fast attention in 68 minutes - better than $1200 GPU programming courses. pick the attention pattern -> generate a fused CUDA kernel -> drop it into vLLM/SGLang -> same GPU, way more tokens per second. That loop is why FlashInfer now powers inference at NVIDIA, vLLM, SGLang, and half the serving stacks you use. FlashInfer + Triton + JIT-compiled kernels + paged-KV attention - that's the stack.

h100envy

32,605 görüntüleme • 3 ay önce

underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it built a triton-class compiler from scratch, MiniTriton, that delivers performance on par with or better than Triton and torch.compile > then it designed a chip, by a model, for a model, in one 48-hour autonomous run the model is rewriting and optimizing every layer of the stack it runs on: kernels for its own training. A compiler for its own kernels. silicon for its own weights. we're watching software build its own hardware
0:22

underrated gems in Kimi-K3 release: > an early K3 wrote the majority of the kernels in the late development stages > it built a triton-class compiler from scratch, MiniTriton, that delivers performance on par with or better than Triton and torch.compile > then it designed a chip, by a model, for a model, in one 48-hour autonomous run the model is rewriting and optimizing every layer of the stack it runs on: kernels for its own training. A compiler for its own kernels. silicon for its own weights. we're watching software build its own hardware

Alex Ker 🔭

311,780 görüntüleme • 3 ay önce

Congrats to the Kimi.ai team! This is awesome. Great to see this level of research coming from open-source frontier model labs. I liked the paper so much I built a Rust implementation of it ;) Full AttnRes + Block AttnRes with two-phase inference, built using Burn (tensor library and Deep Learning Framework, in Rust, by Tracel AI). Runs on CPU, CUDA, Metal, wgpu. Includes an interactive TUI that trains a model live and visualizes depth attention evolving from uniform to selective in real time. Repo link and more on what is implemented in the comments.
0:43

Congrats to the Kimi.ai team! This is awesome. Great to see this level of research coming from open-source frontier model labs. I liked the paper so much I built a Rust implementation of it ;) Full AttnRes + Block AttnRes with two-phase inference, built using Burn (tensor library and Deep Learning Framework, in Rust, by Tracel AI). Runs on CPU, CUDA, Metal, wgpu. Includes an interactive TUI that trains a model live and visualizes depth attention evolving from uniform to selective in real time. Repo link and more on what is implemented in the comments.

abdel

95,888 görüntüleme • 7 ay önce

Inkling, Thinking Machines' first open model, dropped today: 975B total / 41B active MoE, up to 1M context, reasoning natively over text, images, and audio. Serving and RL support are already live: you can run and shape it on an open stack, starting now. Day 0 support on SGLang SGLang and Miles RadixArk👇 - Inkling's new architecture (ShortConv, attention with relative positional embedding, shared expert sink MoE) is natively implemented and deeply optimized, with prefill full CUDA graph and MXFP8 KV cache - Full parameter and LoRA RL in a customized Megatron backend, train inference consistency via customized kernels, routing replay, and cross-runtime parameter synchronization - DFlash speculative decoding from Modal for low-latency serving Launching now, blog and cookbook in the comments ⬇️
1:01

Inkling, Thinking Machines' first open model, dropped today: 975B total / 41B active MoE, up to 1M context, reasoning natively over text, images, and audio. Serving and RL support are already live: you can run and shape it on an open stack, starting now. Day 0 support on SGLang SGLang and Miles RadixArk👇 - Inkling's new architecture (ShortConv, attention with relative positional embedding, shared expert sink MoE) is natively implemented and deeply optimized, with prefill full CUDA graph and MXFP8 KV cache - Full parameter and LoRA RL in a customized Megatron backend, train inference consistency via customized kernels, routing replay, and cross-runtime parameter synchronization - DFlash speculative decoding from Modal for low-latency serving Launching now, blog and cookbook in the comments ⬇️

LMSYS Org

146,680 görüntüleme • 3 ay önce

Ben Thompson explains how LLMs greatly diminished Nvidia's CUDA moat even as they sent the stock to the moon "So the weird thing about large language models is they were obviously incredible for Nvidia. That's why their stock went to the moon." "They have been on and off the most valuable company in the world." "It was also very bad for Nvidia. And the reason it was bad for Nvidia is that the play with CUDA is to build a developer ecosystem on top of CUDA." "But CUDA only works on Nvidia GPUs. So you get CUDA for free. It's easier to use, and it's a tremendous investment. Nvidia almost went under trying to build CUDA at a time when no one understood what they were doing or why they were wasting money on it." "And that's why Jensen Huang will get bristly, particularly when people question their rent-seeking or profit, whatever. It's like, no, they earned their spot fair and square." "Absolutely. It shouldn't be forgotten. They have earned every dollar they've gotten through 25 years of taking massive risks." "It bottomed out in October 2022. I wrote an article like three weeks before ChatGPT came out, tracing their bottoming-out history and their search for what was next." "'Nvidia in the Valley.' So, go back to this GTC. So I wrote an article at the time called 'Nvidia Waves and Moats'." "And what was interesting about that GTC was, number one, it was very boring. All the cool stuff kind of got scrubbed out." "Now, Jensen Huang has brought that stuff back, so the last few GTCs he's more talking about other things. Now it comes across as, oh, you're still looking for something beyond the LLM." "Because the problem with the LLM is it shifts the developer platform far above where Nvidia sits. All the activity is happening on top of LLMs. And so no one who's writing an AI application today is using CUDA." "Now, some people are, if you're training your own model and you're doing some low-level things or non-LLM things." "But the vast majority of the energy and all the money and the ecosystem is far removed from CUDA." "They have no idea and don't need to know or care what chips their application is running on. They're just on the OpenAI API, or the Anthropic API, or using Bedrock on Amazon, and it's sitting on Trainium, and they're using a Chinese open-source model. It's totally abstracted away, and this is why LLMs were bad for Nvidia." "Now, again, all the money they made along the way is worth it, but their moat has been tremendously diminished." "CUDA is still a moat if you need to do stuff that requires CUDA. But the vast majority of stuff, in energy, doesn't require CUDA, like in a post-LLM world."
3:04

Ben Thompson explains how LLMs greatly diminished Nvidia's CUDA moat even as they sent the stock to the moon "So the weird thing about large language models is they were obviously incredible for Nvidia. That's why their stock went to the moon." "They have been on and off the most valuable company in the world." "It was also very bad for Nvidia. And the reason it was bad for Nvidia is that the play with CUDA is to build a developer ecosystem on top of CUDA." "But CUDA only works on Nvidia GPUs. So you get CUDA for free. It's easier to use, and it's a tremendous investment. Nvidia almost went under trying to build CUDA at a time when no one understood what they were doing or why they were wasting money on it." "And that's why Jensen Huang will get bristly, particularly when people question their rent-seeking or profit, whatever. It's like, no, they earned their spot fair and square." "Absolutely. It shouldn't be forgotten. They have earned every dollar they've gotten through 25 years of taking massive risks." "It bottomed out in October 2022. I wrote an article like three weeks before ChatGPT came out, tracing their bottoming-out history and their search for what was next." "'Nvidia in the Valley.' So, go back to this GTC. So I wrote an article at the time called 'Nvidia Waves and Moats'." "And what was interesting about that GTC was, number one, it was very boring. All the cool stuff kind of got scrubbed out." "Now, Jensen Huang has brought that stuff back, so the last few GTCs he's more talking about other things. Now it comes across as, oh, you're still looking for something beyond the LLM." "Because the problem with the LLM is it shifts the developer platform far above where Nvidia sits. All the activity is happening on top of LLMs. And so no one who's writing an AI application today is using CUDA." "Now, some people are, if you're training your own model and you're doing some low-level things or non-LLM things." "But the vast majority of the energy and all the money and the ecosystem is far removed from CUDA." "They have no idea and don't need to know or care what chips their application is running on. They're just on the OpenAI API, or the Anthropic API, or using Bedrock on Amazon, and it's sitting on Trainium, and they're using a Chinese open-source model. It's totally abstracted away, and this is why LLMs were bad for Nvidia." "Now, again, all the money they made along the way is worth it, but their moat has been tremendously diminished." "CUDA is still a moat if you need to do stuff that requires CUDA. But the vast majority of stuff, in energy, doesn't require CUDA, like in a post-LLM world."

Fireside Alpha

12,980 görüntüleme • 2 ay önce

Ubuntu 26.04 LTS, codenamed #ResoluteRaccoon, is now available to download. 🦝 Resolute Raccoon builds on the resilience-focused improvements introduced in interim releases, with TPM-backed full-disk encryption, improved support for application permission prompting, Livepatch updates for Arm-based servers, and Rust-based utilities for enhanced memory safety. This release also brings native support for industry-leading AI/ML toolkits like NVIDIA CUDA and AMD ROCm, making Ubuntu 26.04 LTS the ideal platform for AI development and production workloads. Install now: Learn more about the release:
1:20

Ubuntu 26.04 LTS, codenamed #ResoluteRaccoon, is now available to download. 🦝 Resolute Raccoon builds on the resilience-focused improvements introduced in interim releases, with TPM-backed full-disk encryption, improved support for application permission prompting, Livepatch updates for Arm-based servers, and Rust-based utilities for enhanced memory safety. This release also brings native support for industry-leading AI/ML toolkits like NVIDIA CUDA and AMD ROCm, making Ubuntu 26.04 LTS the ideal platform for AI development and production workloads. Install now: Learn more about the release:

Ubuntu

654,541 görüntüleme • 6 ay önce

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.
1:24

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,496,370 görüntüleme • 5 ay önce

Learn how to Use the Blendr Platform 📚 Check out the video attached which will walk you through the Blendr Platform step-by-step. We are covering exactly how to Log onto the platform with your wallet, Lend your own GPUs and Rent other GPUs on the network. Once you're confident, head over to to try it out for yourself!
4:05

Learn how to Use the Blendr Platform 📚 Check out the video attached which will walk you through the Blendr Platform step-by-step. We are covering exactly how to Log onto the platform with your wallet, Lend your own GPUs and Rent other GPUs on the network. Once you're confident, head over to to try it out for yourself!

Blendr Network

24,382 görüntüleme • 2 yıl önce

Luminal ( is creating PyTorch for Production – an ML compiler that generates blazingly fast CUDA kernels and makes deploying to production one line of code. Congrats on the launch, Jake Stevens, Joe Fioti, and Matthew Gunton!
1:53

Luminal ( is creating PyTorch for Production – an ML compiler that generates blazingly fast CUDA kernels and makes deploying to production one line of code. Congrats on the launch, Jake Stevens, Joe Fioti, and Matthew Gunton!

Y Combinator

98,496 görüntüleme • 1 yıl önce

What does “fungible” mean for AI infrastructure? The flexibility to run different workloads: every type of AI, every phase of AI and even workloads that aren’t AI at all. One platform, built for maximum “productivity”, “durable” years after deployment, and “fungible” for every model and workload. Read the blog to learn more:
1:01

What does “fungible” mean for AI infrastructure? The flexibility to run different workloads: every type of AI, every phase of AI and even workloads that aren’t AI at all. One platform, built for maximum “productivity”, “durable” years after deployment, and “fungible” for every model and workload. Read the blog to learn more:

NVIDIA

77,813 görüntüleme • 27 gün önce

The best way to learn CUDA is by solving problems. In this video, I implement 7 beginner CUDA kernels from scratch: • Vector Addition • Matrix Transpose • Matrix Addition • RGB to Grayscale • Value Clipping • Sigmoid Activation • Interleave Arrays If you're getting started with CUDA programming, this should give you a solid foundation.
30:10

The best way to learn CUDA is by solving problems. In this video, I implement 7 beginner CUDA kernels from scratch: • Vector Addition • Matrix Transpose • Matrix Addition • RGB to Grayscale • Value Clipping • Sigmoid Activation • Interleave Arrays If you're getting started with CUDA programming, this should give you a solid foundation.

voxium

36,987 görüntüleme • 3 ay önce

Finally, I'm done with the basics of backend engineering on the all-in-one resources to learn backend engineering I'm working on This basic section includes - Internet - HTTP - Servers - Web Dev fundamentals - Fundamentals of Operating Systems - A server-side language -> JavaScript | Go | Rust | Node - A framework -> Express | Nestjs | Django | Laravel | Spring Boot This is the boring part. Don't worry. We are getting into the fun part soon. Still working on it, updating it, Adding more, and making it better for you. Follow me. Retweet. Comment "Backend." I will DM you the PDF version when it's out. Enjoy the web version, and send in your feedback on how we can improve it for everyone.
0:37

Finally, I'm done with the basics of backend engineering on the all-in-one resources to learn backend engineering I'm working on This basic section includes - Internet - HTTP - Servers - Web Dev fundamentals - Fundamentals of Operating Systems - A server-side language -> JavaScript | Go | Rust | Node - A framework -> Express | Nestjs | Django | Laravel | Spring Boot This is the boring part. Don't worry. We are getting into the fun part soon. Still working on it, updating it, Adding more, and making it better for you. Follow me. Retweet. Comment "Backend." I will DM you the PDF version when it's out. Enjoy the web version, and send in your feedback on how we can improve it for everyone.

Solomon Eseme

92,216 görüntüleme • 3 yıl önce

Basic Guide to Playing MOG A quick guide to help beginners get familiar with the game and start farming corn for a Yield Fields (🧑‍🌾,🧑‍🌾) whitelist spot. Shoutout to skarly for putting more attention on the MOG grind 👀 Support me if you want:
3:24

Basic Guide to Playing MOG A quick guide to help beginners get familiar with the game and start farming corn for a Yield Fields (🧑‍🌾,🧑‍🌾) whitelist spot. Shoutout to skarly for putting more attention on the MOG grind 👀 Support me if you want:

John Huỳnh

25,172 görüntüleme • 1 ay önce

Compute has always been core to the Verne Robotics story. Getting there meant negotiating long, expensive contracts with big neoclouds, and projecting compute needs years into the future. Verne wasn't ready for that. Then came the Y Combinator cluster, built on Together AI. Verne trained a model 3x larger, on almost 100x more data. More data in, bigger foundation models out.
1:10

Compute has always been core to the Verne Robotics story. Getting there meant negotiating long, expensive contracts with big neoclouds, and projecting compute needs years into the future. Verne wasn't ready for that. Then came the Y Combinator cluster, built on Together AI. Verne trained a model 3x larger, on almost 100x more data. More data in, bigger foundation models out.

Together AI

21,649 görüntüleme • 2 ay önce

YOUR PARENTS PAID FOR THE CUDA MOAT! The #1 contributor to the CUDA MOAT isn't the the developers at NVIDIA, but it is the millions of developers outside of NVIDIA that invent new algorithms for CUDA like Flash Attention. For most of them, it started with an GeForce gaming GPU. NVIDIA is the only companies that has an reasonable good developer stack on consumer grade GPUs. As people grow up beyond playing CSGO & League of Legends & Minecraft, they either become anime weeaboos or they start programming on their existing computer with has an GeForce GPU
0:23

YOUR PARENTS PAID FOR THE CUDA MOAT! The #1 contributor to the CUDA MOAT isn't the the developers at NVIDIA, but it is the millions of developers outside of NVIDIA that invent new algorithms for CUDA like Flash Attention. For most of them, it started with an GeForce gaming GPU. NVIDIA is the only companies that has an reasonable good developer stack on consumer grade GPUs. As people grow up beyond playing CSGO & League of Legends & Minecraft, they either become anime weeaboos or they start programming on their existing computer with has an GeForce GPU

SemiAnalysis

25,230 görüntüleme • 6 ay önce