Training LLMs @huggingface 🤗.
Author of https://t.co/4HAFxkzJMZ
Shorts
📖 The Ultimate MoE Training Handbook 📖 We spent 1/2M GPUh to optimize MoE training and wrote the most comprehensive blog about it. With 30k words and 50+ interactive figures it covers everything from profiling to large cluster training.
