Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

timelapse #21 (12 hrs): - leetcode practice for xAI coding test - updated mnist-cuda (find repo in pinned) by adding a new CUDA training script w/ an extra hidden layer and a feature to show generalization - notes on LLM training datasets, architectures, training config, etc

Elliot Arledge

16,390 subscribers

216,069 views • 1 year ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

10 Comments

calibur 🜲1 year ago

The grind is so real. Congrats on your training

Maximilian Schilling1 year ago

75k views is 5h is nuts! Awesome! You are the Timelapse grand master 😂

Elliot Arledge1 year ago

you inspire me Max

λux1 year ago

xAI LFG 🔥

Frieda Huang1 year ago

Nice. How much leetcode prep did you do?

sλrthak1 year ago

xAI coding test? enlighten me

Raen AI1 year ago

Locked in

Elliot Arledge1 year ago

yessir

Nothing is something1 year ago

We're back with timelapses.... LFG!!!!

Lucas Lain1 year ago

is there a way where I can peak a classic interview for these positions?

Related Videos

I vibecoded this neural network visualization for my students and open sourced it. It shows a simple MLP trained on MNIST handwritten digits at several training steps. The visualization is using Three.js and it comes with training code in PyTorch . Link + repo 👇

I vibecoded this neural network visualization for my students and open sourced it. It shows a simple MLP trained on MNIST handwritten digits at several training steps. The visualization is using Three.js and it comes with training code in PyTorch . Link + repo 👇

David Finsterwalder | eu/acc

487,057 views • 7 months ago

🚨 New Paper Training an LLM to speak low-resource language (EACL workshop, 2026) Tulu is spoken by 2M+ people in coastal Karnataka and LLMs basically can't speak it. We got to 85% grammar accuracy without fine-tuning anything or collecting a single new training example.

🚨 New Paper Training an LLM to speak low-resource language (EACL workshop, 2026) Tulu is spoken by 2M+ people in coastal Karnataka and LLMs basically can't speak it. We got to 85% grammar accuracy without fine-tuning anything or collecting a single new training example.

Lossfunk

120,485 views • 3 months ago

Elon Musk told an @xAI employee that he'd get a free Cybertruck if he could get a training run going within 24 hrs. "Elon's like "you can get a Cybertruck tonight if you can get a training run on these GPUs within 24 hours." We were training that night. He got the Cybertruck."

Elon Musk told an @xAI employee that he'd get a free Cybertruck if he could get a training run going within 24 hrs. "Elon's like "you can get a Cybertruck tonight if you can get a training run on these GPUs within 24 hours." We were training that night. He got the Cybertruck."

Sawyer Merritt

2,145,674 views • 4 months ago

Excited to announce 🚀gsplat v1.0🚀: a ⏩efficient⏩ CUDA backend for 3D Gaussian Splatting! A drop-in replacement of the official impl. with: - Up to 2x faster training; - Up to 4x less GPU memory; - Render millions of GSs in real-time; - And more;

Excited to announce 🚀gsplat v1.0🚀: a ⏩efficient⏩ CUDA backend for 3D Gaussian Splatting! A drop-in replacement of the official impl. with: - Up to 2x faster training; - Up to 4x less GPU memory; - Render millions of GSs in real-time; - And more;

Ruilong Li

87,917 views • 2 years ago

Our Training & Finetuning LLMs course is now LIVE 🚀🔥🎉 Brought to you by wan deeee bee and . ❤️ Join us for free: Get ready to learn from the best in the industry, including experts from Meta and Microsoft 😍 🚀 Master Parameter-efficient Training Techniques 🔍 Dive Deep into LLM Evaluation 📚 Explore Datasets for LLM Training 💻 Discover Essential Hardware Requirements This course isn't for beginners - be prepared for practical insights and the opportunity to train your own LLMs! 💪

Our Training & Finetuning LLMs course is now LIVE 🚀🔥🎉 Brought to you by wan deeee bee and . ❤️ Join us for free: Get ready to learn from the best in the industry, including experts from Meta and Microsoft 😍 🚀 Master Parameter-efficient Training Techniques 🔍 Dive Deep into LLM Evaluation 📚 Explore Datasets for LLM Training 💻 Discover Essential Hardware Requirements This course isn't for beginners - be prepared for practical insights and the opportunity to train your own LLMs! 💪

Darek Kłeczek

18,940 views • 2 years ago

in the lectures below, i hold your hand through low-level LLM systems engineering. it includes everything up to TODAY! 1) pytorch tensors 2) large matmul on cpu vs gpu 3) JAX (and why xAI uses it instead of pytorch) 4) raw cuda kernels and global threading indexing 5) triton design philosophy and softmax example 6) HIP kernels 7) mapping out the ENTIRE ecosystem + differences between CUDA and ROCm/HIP (BLAS, FFT, DNN) 8) cutlass and cute-dsl 9) pretraining, finetuning, rl, unsloth, axolotl, megatron-lm, deepspeed, nanogpt, nanochat 10) training vs inference, inference serving problems, throughput vs latency vs concurrency scaling, vllm, sglang, tensorrt-llm, tensorrt, llama.cpp, exllamav2, exllamav3, benchmark comparisons 11) projects/companies using llms to generate SOTA cuda/triton kernels 12) luminal inference 13) mojo/modular/max

in the lectures below, i hold your hand through low-level LLM systems engineering. it includes everything up to TODAY! 1) pytorch tensors 2) large matmul on cpu vs gpu 3) JAX (and why xAI uses it instead of pytorch) 4) raw cuda kernels and global threading indexing 5) triton design philosophy and softmax example 6) HIP kernels 7) mapping out the ENTIRE ecosystem + differences between CUDA and ROCm/HIP (BLAS, FFT, DNN) 8) cutlass and cute-dsl 9) pretraining, finetuning, rl, unsloth, axolotl, megatron-lm, deepspeed, nanogpt, nanochat 10) training vs inference, inference serving problems, throughput vs latency vs concurrency scaling, vllm, sglang, tensorrt-llm, tensorrt, llama.cpp, exllamav2, exllamav3, benchmark comparisons 11) projects/companies using llms to generate SOTA cuda/triton kernels 12) luminal inference 13) mojo/modular/max

Elliot Arledge

57,855 views • 7 months ago

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels. The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels that are much faster than existing CUDA kernels commonly used in production. We believe that fundamentally, AI systems can and should be as resource-efficient as the human brain, and that the best path to achieve this efficiency is to use AI to make AI more efficient! We are excited to publish our paper, The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition. We also release a dataset of over 17,000 verified CUDA kernels produced by The AI CUDA Engineer. Paper: Kernel Archive Webpage: HuggingFace Dataset: The AI CUDA Engineer utilizes evolutionary LLM-driven code optimization to autonomously improve the runtime of machine learning operations. Our system is not only able to convert PyTorch code into CUDA kernels, but through the use of evolution, it can also optimize the runtime performance of CUDA kernels, fuse multiple operations, and even discover novel solutions for writing efficient CUDA operations by learning from past innovations! We believe The AI CUDA Engineer opens a new era of AI-driven acceleration of AI and automated inference time optimization. We (Robert Lange, Aaditya Prasad 🇺🇸, Suuun, Maxence Faldor, Yujin Tang, hardmaru) are excited to continue Sakana AI's mission of leveraging AI to improve AI.

Introducing The AI CUDA Engineer: An agentic AI system that automates the production of highly optimized CUDA kernels. The AI CUDA Engineer can produce highly optimized CUDA kernels, reaching 10-100x speedup over common machine learning operations in PyTorch. Our system is also able to produce highly optimized CUDA kernels that are much faster than existing CUDA kernels commonly used in production. We believe that fundamentally, AI systems can and should be as resource-efficient as the human brain, and that the best path to achieve this efficiency is to use AI to make AI more efficient! We are excited to publish our paper, The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition. We also release a dataset of over 17,000 verified CUDA kernels produced by The AI CUDA Engineer. Paper: Kernel Archive Webpage: HuggingFace Dataset: The AI CUDA Engineer utilizes evolutionary LLM-driven code optimization to autonomously improve the runtime of machine learning operations. Our system is not only able to convert PyTorch code into CUDA kernels, but through the use of evolution, it can also optimize the runtime performance of CUDA kernels, fuse multiple operations, and even discover novel solutions for writing efficient CUDA operations by learning from past innovations! We believe The AI CUDA Engineer opens a new era of AI-driven acceleration of AI and automated inference time optimization. We (Robert Lange, Aaditya Prasad 🇺🇸, Suuun, Maxence Faldor, Yujin Tang, hardmaru) are excited to continue Sakana AI's mission of leveraging AI to improve AI.

Sakana AI

1,149,314 views • 1 year ago

xAI is hiring across multiple roles 🚀 currently building the world's best coding model and training it on a MILLION H100 equivalents The future of Grok Code is being written in real-time Exponential takeoff is here xAI is hiring: → AI training experts → Elite low-level software engineers → Systems design specialists If you're obsessed with code, this is where you need to be Join xAI now

xAI is hiring across multiple roles 🚀 currently building the world's best coding model and training it on a MILLION H100 equivalents The future of Grok Code is being written in real-time Exponential takeoff is here xAI is hiring: → AI training experts → Elite low-level software engineers → Systems design specialists If you're obsessed with code, this is where you need to be Join xAI now

X Freeze

22,905 views • 3 months ago

We open-sourced QeRL — Quantization-enhanced Reinforcement Learning ! 🧠 4-bit quantized RL training 💪 Train a 32B LLM on a single H100 GPU ⚙️ 1.7× faster overall training 🎯 Accuracy on par with bfloat16-level accuracy 🔥 Supports NVFP4 quantization format Moreover, we show that quantization helps exploration in RL training. Paper: Code: #NVIDIA #AIResearch #ReinforcementLearning #Quantization #LLM #EfficientAI

We open-sourced QeRL — Quantization-enhanced Reinforcement Learning ! 🧠 4-bit quantized RL training 💪 Train a 32B LLM on a single H100 GPU ⚙️ 1.7× faster overall training 🎯 Accuracy on par with bfloat16-level accuracy 🔥 Supports NVFP4 quantization format Moreover, we show that quantization helps exploration in RL training. Paper: Code: #NVIDIA #AIResearch #ReinforcementLearning #Quantization #LLM #EfficientAI

Yukang Chen

69,720 views • 8 months ago

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically *retrieved* “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are *not* collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant *sub-trajectories* from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically retrieved “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are not collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant sub-trajectories from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)

Abhishek Gupta

12,045 views • 1 year ago

I’ve been working on a new dog training technique called Bad Transatlantic Accent Training in which she gets confused by my voice and stops biting me

I’ve been working on a new dog training technique called Bad Transatlantic Accent Training in which she gets confused by my voice and stops biting me

meg “Yooper” bitchell

32,919 views • 10 months ago

I used to find writing CUDA code rather terrifying. But then I discovered a couple of tricks that actually make it quite accessible. In this video I introduce CUDA in a way that will be accessible to Python programmers, and I even show how to do it all in Colaboratory!

I used to find writing CUDA code rather terrifying. But then I discovered a couple of tricks that actually make it quite accessible. In this video I introduce CUDA in a way that will be accessible to Python programmers, and I even show how to do it all in Colaboratory!

Jeremy Howard

210,642 views • 2 years ago

David Beckham and Paul Scholes used to put on a show in England training 🏴󠁧󠁢󠁥󠁮󠁧󠁿

David Beckham and Paul Scholes used to put on a show in England training 🏴󠁧󠁢󠁥󠁮󠁧󠁿

GOAL

121,565 views • 7 months ago

been feeling weird and in my head lately. trying hard to hone back in on my practice and find my passion for training and flowing 🥲🧡 #bostaff

been feeling weird and in my head lately. trying hard to hone back in on my practice and find my passion for training and flowing 🥲🧡 #bostaff

✨ 🌿 Robin D. Banks 🍃 ☀️

15,504 views • 3 years ago

timelapse #38 (15 hrs): - first timelapse back from SF - health research (sleep, supplements, caffiene) - im wired more than ever - planning potential next trip - spent some time with family - setup a workout station - started planning stage for a CUDA book - chilled in Umar Jamil discord server talking about STT, ViTs, CUDA, and about my time in SF over the last month - switched to superwhisper since its the cheapest and best option (Andrej Karpathy effect lol) - applied at 10+ companies - co-working with an irl - shovelled driveway

timelapse #38 (15 hrs): - first timelapse back from SF - health research (sleep, supplements, caffiene) - im wired more than ever - planning potential next trip - spent some time with family - setup a workout station - started planning stage for a CUDA book - chilled in Umar Jamil discord server talking about STT, ViTs, CUDA, and about my time in SF over the last month - switched to superwhisper since its the cheapest and best option (Andrej Karpathy effect lol) - applied at 10+ companies - co-working with an irl - shovelled driveway

Elliot Arledge

13,133 views • 1 year ago

Team #RWBY training. #WeissSchnee and #YangXiaoLong training hard. #RubyRose and #Blakebelladonna hardly training. #RWBYfanart Based off a similar video I saw a while back but unable to find source. Using Hiru 🔅 CMS Closed's retextures. Background by yoshxxmmd

Team #RWBY training. #WeissSchnee and #YangXiaoLong training hard. #RubyRose and #Blakebelladonna hardly training. #RWBYfanart Based off a similar video I saw a while back but unable to find source. Using Hiru 🔅 CMS Closed's retextures. Background by yoshxxmmd

Ken - Vgen Open!

64,343 views • 4 months ago

FACT CHECK: No trainings OR standards have been cut from ICE training. New ICE recruits receieve 56 days of training and an average of 28 days of on-the-job training. Training increased from five days a week, eight hours a day to six days a week, twelve hours per day including personalized independent training. It’s the same hours of training officers have ALWAYS received.

FACT CHECK: No trainings OR standards have been cut from ICE training. New ICE recruits receieve 56 days of training and an average of 28 days of on-the-job training. Training increased from five days a week, eight hours a day to six days a week, twelve hours per day including personalized independent training. It’s the same hours of training officers have ALWAYS received.

Homeland Security

41,876 views • 3 months ago

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

Awni Hannun

42,743 views • 10 months ago

Scientists developed HumanX, a system that lets humanoid robots learn new skills by watching just one short human video. It learns directly from demonstrations instead of large datasets or complex training rewards. Using this, the Unitree G1 humanoid robot learned 10 different real-world skills without extra training.

Scientists developed HumanX, a system that lets humanoid robots learn new skills by watching just one short human video. It learns directly from demonstrations instead of large datasets or complex training rewards. Using this, the Unitree G1 humanoid robot learned 10 different real-world skills without extra training.

Space and Technology

27,686 views • 1 month ago