Sebastian Raschka
@rasbt • 510,760 subscribers
ML/AI research engineer. Ex stats professor. Author of "Build a Large Language Model From Scratch" (https://t.co/O8LAAMRzzW) & reasoning (https://t.co/5TueQKx2Fk)
Videos
Inference scaling part 1. Starting with a modded text generation function (temperature scaling, top-p filtering, multinomial sampling) to generate diverse outputs for self-consistency and best-of-N (improving answer accuracy by>2x) 00:00 Introduction and recap 00:31 Training-time and inference-time scaling 07:52 What we'll implement 11:47 Notebook setup and model loading 17:43 Building a flexible text generation function 24:40 Chain-of-thought prompting 28:26 Sampling and output diversity 33:43 Next-token logits and greedy decoding 38:20 Temperature scaling step by step 42:46 Softmax and token probabilities 47:42 Multinomial sampling 54:51 Adding temperature sampling to text generation 59:31 Top-p filtering step by step 1:10:23 Adding top-p filtering to text generation 1:13:43 Sampling and LLM watermarking 1:16:01 Self-consistency and majority voting 1:20:36 Implementing self-consistency 1:29:02 MATH-500 results 1:35:01 Accuracy and compute tradeoffs 1:36:50 Next steps and self-refinement
Sebastian Raschka42,748 просмотров • 4 дней назад
Reasoning from scratch round 3: This time, I cover generating a verifier for... a) ...evaluation (base model versus any future model improvement) b) ...the reinforcement learning with verifiable rewards (RLVR) training later on 00:00 Introduction 01:21 Four approaches to LLM evaluation 07:20 Verifiers and reinforcement learning with verifiable rewards 10:52 Notebook setup and dependencies 13:43 Section 3.1 Building a math verifier 18:57 Section 3.2 Loading a pre-trained model to generate text 24:34 Generating and displaying model answers 29:23 Section 3.3 Implementing a wrapper for easier text generation 34:00 Section 3.4 Extracting the final answer box 37:29 Handling answers without boxes 43:17 Section 3.5 Normalizing the extracted answer 46:56 Section 3.6 Verifying mathematical equivalence 53:32 Implementing the equality check 57:48 Section 3.7 Grading answers 59:20 Building and testing the answer grader 1:03:18 Section 3.8 Loading the evaluation dataset (MATH-500) 1:07:51 Section 3.9 Evaluating the model 1:08:34 Prompt templates for evaluation 1:10:47 Prompt sensitivity and memorization 1:13:55 A minimal evaluation example 1:15:32 Building the evaluation loop 1:20:27 Comparing CPU, MPS, and CUDA results 1:21:54 Reproducibility and floating-point math 1:23:37 Base model vs. reasoning model 1:25:30 Summary and next steps
Sebastian Raschka40,226 просмотров • 10 дней назад
Reasoning from scratch round 2: In this video, I cover the text generation process in LLMs and KV caching (to prepare the base model before adding reasoning techniques in the upcoming ones). 00:00 Introduction and reasoning model demo 01:55 How to work through the book 05:00 Chapter 2 overview 08:25 Checking PyTorch and hardware support 10:26 Apple silicon and MPS caveats 15:00 Cloud GPU options 16:08 Tokens and tokenization 18:20 Qwen3 and the Reasoning From Scratch package 23:05 Encoding and decoding text 26:24 Downloading weights and selecting a device 31:01 Loading the pretrained Qwen3 model 34:32 How LLMs generate text 36:47 Input tensors and batch dimensions 41:48 Running the model in inference mode 44:11 Logits and next-token predictions 49:21 Greedy decoding with argmax 52:28 Building a streaming text generator 01:01:28 Generating text and handling end-of-sequence tokens 01:06:00 Benchmarking text generation 01:14:34 How KV caching works 01:17:22 Adding KV caching and measuring the speedup 01:24:31 Model compilation with torch.compile 01:30:33 Combining compilation with KV caching 01:32:53 Comparing CPU and GPU performance 01:35:32 Recap and next steps
Sebastian Raschka65,595 просмотров • 17 дней назад
Some food for thought when designing benchmarks... So, here's a little computer-use (visual) comparison between GPT-5.6 Astra and Qwen3.8 Max. The task here was to recreate the image in the center using the Paint UI. Super interesting how the two different LLMs+Harnesses approached this totally differently by default. I.e., Astra tried to approach this by drawing and layering geometric shapes. Qwen approached this pixel by pixel. (Of course, the pixel-by-pixel result looks closer to the original, it's essentially a low-res version of that by nature.) So, the Qwen-generated image would surely score higher in the sense that it's closer to the original. But I wouldn’t conclude from this example that one LLM generalizes better than the other on other tasks. Also, I wouldn't say Qwen has better compute-use capabilities or better visual understanding than Astra. But it highlights an interesting point about how slippery benchmarks are when they only compare final results.
Sebastian Raschka23,056 просмотров • 8 дней назад
Thanks everyone for all the kind words and feedback. Super happy that you are enjoying Build a Reasoning Model (From Scratch)! Unfortunately, there's small typo in listing 6.5 on page 198 (see video below). The line "torch.manual_seed(0)" should be "torch.manual_seed(5)" This correction is needed to reproduce the generated response in listing 6.5 and the corresponding log-probability outputs later in Chapter 6. If you use 0, the generated response and the results that follow will be different. This will be fixed in the next printing. I am sorry about the oversight, and I hope this note saves you some debugging time.
Sebastian Raschka105,980 просмотров • 2 месяцев назад
Updated & turned my Big LLM Architecture Comparison article into a narrated video lecture. The 11 LLM architectures covered in this video: 1. DeepSeek V3/R1 2. OLMo 2 3. Gemma 3 4. Mistral Small 3.1 5. Llama 4 6. Qwen3 7. SmolLM3 8. Kimi 2 9. GPT-OSS 10. Grok 2.5 11. GLM-4.5
Sebastian Raschka199,792 просмотров • 1 год назад
Claude distillation has been a big topic this week while I am (coincidentally) writing Chapter 8 on model distillation. In that context, I shared some utilities to generate distillation data from all sorts of open-weight models via OpenRouter and Ollama:
Sebastian Raschka62,838 просмотров • 6 месяцев назад
Excited to be launching a series of short videos (5 min or less) to optimize the training performance of your PyTorch models! Covering - mixed-precision training - multi-GPU training strategies - Compiling PyTorch models - and finding good batch sizes 🔗
Sebastian Raschka120,808 просмотров • 3 лет назад
When doing machine learning and AI research (or writing books), making the code reproducible is usually desirable. Often, that's easier said than done! So, I recorded a video illustrating and dealing with 6 sources of randomness that occur when training deep neural networks and LLMs: 1. Model weight initialization 2. Dataset sampling and shuffling 3. Nondeterministic algorithms 4. Different runtime algorithms 5. Hardware and drivers 6. Randomness in generative AI models
Sebastian Raschka81,713 просмотров • 2 лет назад
I just added the new Llama 3.2 1B and 3B models to LitGPT, the open-source LLM library I help develop (focused on efficiency and code readability). LitGPT allows you to fine-tune and use these models on the cloud or a laptop. So, if you are looking for something to play with this weekend: # 1) Finetune the model litgpt finetune_lora meta-llama/Llama-3.2-1B \ --data JSON \ --data.json_path my_custom_dataset.json \ --train.epochs 1 \ --out_dir out/llama-3.2-finetuned \ --precision bf16-true # 2) Chat with the model litgpt chat out/llama-3.2-finetuned/final # 3) Serve the model via an API endpoint litgpt serve out/llama-3.2-finetuned/final
Sebastian Raschka65,529 просмотров • 2 лет назад
Больше нет контента для загрузки