Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Next level: QLoRA fine-tuning 4-bit Llama 3 8B on iPhone 15 pro. Incoming (Q)LoRA MLX Swift example by David Koski: works with lot's of models (Mistral, Gemma, Phi-2, etc)

Awni Hannun

44,845 subscribers

581,777 görüntüleme • 2 yıl önce •via X (Twitter)

Sağlık & İyilik Eğitim Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

10 Yorum

Awni Hannun profil fotoğrafı

Awni Hannun2 yıl önce

Stats: - 4 LoRA layers, batch size 1 - runs at about 45-50 toks/sec

Cavit Erginsoy profil fotoğrafı

Cavit Erginsoy2 yıl önce

Cool, but why 🤗

Ryan Ziolko 🚀 profil fotoğrafı

Ryan Ziolko 🚀2 yıl önce

Oh no, Wi-Fi stayed on in airplane mode (fear not, I believe you). Pretty wild to see fine-tuning on a phone. 💪

Daniel Merja profil fotoğrafı

Daniel Merja2 yıl önce

Cluster of IPhones next

SM profil fotoğrafı

SM2 yıl önce

My phones about to become a heater

David profil fotoğrafı

David2 yıl önce

Please forgive the ignorance on my part, but what is it being fine tuned *for* or *on*?

Awni Hannun profil fotoğrafı

Awni Hannun2 yıl önce

In this case wikiSQL. But it’s easy to change the data

Prem Kumar Aparanji 🤖👶🏼🐘 profil fotoğrafı

Prem Kumar Aparanji 🤖👶🏼🐘2 yıl önce

Mindblowing 🤯

Maziyar PANAHI profil fotoğrafı

Maziyar PANAHI2 yıl önce

you gotta be kidding! damn! 👏🏼

Will Townsend profil fotoğrafı

Will Townsend2 yıl önce

Lots of progress for local first LLMs being made, and I'm all here for it!

Benzer Videolar

MLX Swift example can also QLoRA fine-tune Llama 3.2. Here's the 1B fine-tuning on my iPhone 15 Pro at > 150 toks/sec. A this rate only takes a few minutes to learn some decent adapters fully on-device.

MLX Swift example can also QLoRA fine-tune Llama 3.2. Here's the 1B fine-tuning on my iPhone 15 Pro at > 150 toks/sec. A this rate only takes a few minutes to learn some decent adapters fully on-device.

Awni Hannun

91,961 görüntüleme • 2 yıl önce

Can an iPhone hold its own against a Mac? 💻📱 Ollama is running DeepSeek R1 Distill Llama 8B on a 16GB M2 MacBook Air, while the iPhone 15 Pro (8GB A17 Pro) runs the same model. Same model. Same sampling temperature. Same system prompt. While the iPhone is a bit slower, the 3-bit OmniQuant model reasons just as well as the 4-bit RTN model on the Mac. 🤯

Can an iPhone hold its own against a Mac? 💻📱 Ollama is running DeepSeek R1 Distill Llama 8B on a 16GB M2 MacBook Air, while the iPhone 15 Pro (8GB A17 Pro) runs the same model. Same model. Same sampling temperature. Same system prompt. While the iPhone is a bit slower, the 3-bit OmniQuant model reasons just as well as the 4-bit RTN model on the Mac. 🤯

Private LLM

26,544 görüntüleme • 1 yıl önce

Ex-NVIDIA engineer who built Unsloth explained RL, kernels, reasoning, quantization, and agents in 2 hours 42 minutes - better than $5000 fine-tuning bootcamps. pick the base model -> write triton kernels for 2x faster fine-tune -> quantize to 4-bit -> run GRPO/DPO -> ship a reasoning model on your single GPU. That loop is why Unsloth is the default way to fine-tune Llama, Qwen, Gemma, and Phi on hardware you already own. Unsloth + Triton kernels + 4-bit quantization + GRPO/DPO + single-GPU fine-tuning - that's the stack. Watch and save it, then fine-tune your first model tonight.

Ex-NVIDIA engineer who built Unsloth explained RL, kernels, reasoning, quantization, and agents in 2 hours 42 minutes - better than $5000 fine-tuning bootcamps. pick the base model -> write triton kernels for 2x faster fine-tune -> quantize to 4-bit -> run GRPO/DPO -> ship a reasoning model on your single GPU. That loop is why Unsloth is the default way to fine-tune Llama, Qwen, Gemma, and Phi on hardware you already own. Unsloth + Triton kernels + 4-bit quantization + GRPO/DPO + single-GPU fine-tuning - that's the stack. Watch and save it, then fine-tune your first model tonight.

h100envy

496,831 görüntüleme • 2 ay önce

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

Adrien Grondin

3,560,817 görüntüleme • 7 ay önce

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

OpenLedger

61,697 görüntüleme • 1 yıl önce

Meet iPhone 15 Pro & Pro Max! Powered by A17 Pro, which ushers in a new era of Apple Silicon, these products unlock new performance capabilities, amazing photography, next-level gaming, and more. And with an all-new titanium design, they're our lightest weight Pro models yet!

Meet iPhone 15 Pro & Pro Max! Powered by A17 Pro, which ushers in a new era of Apple Silicon, these products unlock new performance capabilities, amazing photography, next-level gaming, and more. And with an all-new titanium design, they're our lightest weight Pro models yet!

Tim Cook

5,795,630 görüntüleme • 3 yıl önce

Apple’s Live Translation feature requires AirPods 4 (ANC), AirPods Pro 2/3, iPhone 15 Pro or later, iOS 26+, and Apple Intelligence turned on Meanwhile, Google’s new Live Translate works with any headphones in 70+ languages This is crazy 💀 Really curious to see how it works in real life and how it compares with Apple

Apple’s Live Translation feature requires AirPods 4 (ANC), AirPods Pro 2/3, iPhone 15 Pro or later, iOS 26+, and Apple Intelligence turned on Meanwhile, Google’s new Live Translate works with any headphones in 70+ languages This is crazy 💀 Really curious to see how it works in real life and how it compares with Apple

Shishir

362,089 görüntüleme • 6 ay önce

Next mlx-vlm release will ship with continuous batching support on the server 🚀 What's coming: → Continuous batching — new requests join the active batch immediately, no waiting. Mixed image + text batches supported → OpenAI-compatible API — field-for-field match with mlx-lm, reasoning/content split for thinking models, tag-aware streaming → Multi-turn tool calling — full tool use support across streaming and non-streaming, works with Gemma4 and other templates → Vision feature caching — cache image embeddings across turns. Gemma4: 228x speedup, Qwen3.5: 23x on cache hit All running locally on Apple Silicon. Check our this demo running 4 concurrent requests (mixed image + text) to gemma-4-26B-A4B-IT by Google Gemma in bf16 using Pi + MLX-VLM server on my M3 Ultra. One of the requests ingests a 8K resolution image!

Next mlx-vlm release will ship with continuous batching support on the server 🚀 What's coming: → Continuous batching — new requests join the active batch immediately, no waiting. Mixed image + text batches supported → OpenAI-compatible API — field-for-field match with mlx-lm, reasoning/content split for thinking models, tag-aware streaming → Multi-turn tool calling — full tool use support across streaming and non-streaming, works with Gemma4 and other templates → Vision feature caching — cache image embeddings across turns. Gemma4: 228x speedup, Qwen3.5: 23x on cache hit All running locally on Apple Silicon. Check our this demo running 4 concurrent requests (mixed image + text) to gemma-4-26B-A4B-IT by Google Gemma in bf16 using Pi + MLX-VLM server on my M3 Ultra. One of the requests ingests a 8K resolution image!

Prince Canuma

82,349 görüntüleme • 5 ay önce

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO Sharon Zhou, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations: - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here:

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO Sharon Zhou, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations: - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here:

Andrew Ng

66,549 görüntüleme • 2 yıl önce

Only ~1.2B parameters active at a time. Edge0-8B-A1B-preview makes an 8B-class MoE practical for local inference.📜 Apache 2.0. 🤖 ⚡ Reaches 23.9–25.3 tokens/s with about 1.0 GiB peak active memory in the reported short-context benchmark. 🏆 Retains most of the FP16 base model’s quality, with an average gap of just 2.8 points across five benchmarks. MMLU-Pro rises from 65.8 to 70.1. 🧠 SSD expert offload keeps most weights outside active memory, while a prerouter predicts which experts each layer will need next. 🛠 Recover-LoRA offsets quality loss from 4-bit quantization and expert offloading. The current Preview runs locally through MLX on Apple Silicon.

Only ~1.2B parameters active at a time. Edge0-8B-A1B-preview makes an 8B-class MoE practical for local inference.📜 Apache 2.0. 🤖 ⚡ Reaches 23.9–25.3 tokens/s with about 1.0 GiB peak active memory in the reported short-context benchmark. 🏆 Retains most of the FP16 base model’s quality, with an average gap of just 2.8 points across five benchmarks. MMLU-Pro rises from 65.8 to 70.1. 🧠 SSD expert offload keeps most weights outside active memory, while a prerouter predicts which experts each layer will need next. 🛠 Recover-LoRA offsets quality loss from 4-bit quantization and expert offloading. The current Preview runs locally through MLX on Apple Silicon.

ModelScope

16,923 görüntüleme • 14 gün önce

غمرتوني بلطفكم جميعاً 🙏 التطبيق لسه في بداياته وقيد التطوير. يا ريت أقدر أضيفكم كلكم مرة وحدة، لكن لأسباب تقنية ولازم نمشي خطوة خطوة. الخطة حالياً كالتالي: حالياً التجربة لـ 20 مستخدم فقط (بصراحة ما كنت متوقع ألاقي حتى 5 😅) خلال أسبوع، بعد التعديل والتنقيح، نرفع العدد لـ 50 بعدها إطلاق على Apple للجميع بدون تتبع بدون اشتراكات وبدون حتى إنترنت الهدف أن الفائدة توصل للجميع. الأجهزة المدعومة حالياً: iPhone 13 / 14 الموديلات الصغيرة فقط. سريعة لكن إجاباتها بسيطة. iPhone 15 / Plus موديلات صغيرة ومتوسطة. توازن بين السرعة والذكاء. iPhone 15 Pro / 16 Pro كل الموديلات تشتغل. أفضل تجربة. موديلات المحادثة المتاحة حالياً: عربي + إنجليزي: Qwen 3 1.7B Qwen 3 4B Qwen 2.5 (0.5B, 1.5B, 3B, 7B) Gemma 3 1B Gemma 3 4B إنجليزي: Llama 3.2 (1B, 3B, 8B) DeepSeek R1 1.5B Phi 3.5 Phi 4 Mistral 7B Ministral 3B برمجة: Qwen 2.5 Coder 1.5B الصوت العربي حالياً تجريبي. مصري وسعودي وعراقي. يعمل بالكامل على الجهاز بدون إنترنت. الموديلات الصوتية مفتوحة المصدر هنا: بكل صراحة: الموديلات الصغيرة سريعة لكن بسيطة. لا تتوقعوا مستوى ChatGPT الحالي، الموديلات الأكبر أفضل بكثير لكنها تحتاج أجهزة أقوى. كل شيء يعمل محلياً على الجهاز. بدون إنترنت. بدون تتبع. بدون اشتراكات. التطبيق مجاني بالكامل. شكراً لدعمكم. ملاحظاتكم ودعمكم هي اللي راح تحسن التطبيق 🤝

غمرتوني بلطفكم جميعاً 🙏 التطبيق لسه في بداياته وقيد التطوير. يا ريت أقدر أضيفكم كلكم مرة وحدة، لكن لأسباب تقنية ولازم نمشي خطوة خطوة. الخطة حالياً كالتالي: حالياً التجربة لـ 20 مستخدم فقط (بصراحة ما كنت متوقع ألاقي حتى 5 😅) خلال أسبوع، بعد التعديل والتنقيح، نرفع العدد لـ 50 بعدها إطلاق على Apple للجميع بدون تتبع بدون اشتراكات وبدون حتى إنترنت الهدف أن الفائدة توصل للجميع. الأجهزة المدعومة حالياً: iPhone 13 / 14 الموديلات الصغيرة فقط. سريعة لكن إجاباتها بسيطة. iPhone 15 / Plus موديلات صغيرة ومتوسطة. توازن بين السرعة والذكاء. iPhone 15 Pro / 16 Pro كل الموديلات تشتغل. أفضل تجربة. موديلات المحادثة المتاحة حالياً: عربي + إنجليزي: Qwen 3 1.7B Qwen 3 4B Qwen 2.5 (0.5B, 1.5B, 3B, 7B) Gemma 3 1B Gemma 3 4B إنجليزي: Llama 3.2 (1B, 3B, 8B) DeepSeek R1 1.5B Phi 3.5 Phi 4 Mistral 7B Ministral 3B برمجة: Qwen 2.5 Coder 1.5B الصوت العربي حالياً تجريبي. مصري وسعودي وعراقي. يعمل بالكامل على الجهاز بدون إنترنت. الموديلات الصوتية مفتوحة المصدر هنا: بكل صراحة: الموديلات الصغيرة سريعة لكن بسيطة. لا تتوقعوا مستوى ChatGPT الحالي، الموديلات الأكبر أفضل بكثير لكنها تحتاج أجهزة أقوى. كل شيء يعمل محلياً على الجهاز. بدون إنترنت. بدون تتبع. بدون اشتراكات. التطبيق مجاني بالكامل. شكراً لدعمكم. ملاحظاتكم ودعمكم هي اللي راح تحسن التطبيق 🤝

Moshe

112,186 görüntüleme • 6 ay önce

🚀 OpenLLM 0.6 is out providing fast inference speed! Check out our video comparing #OpenLLM and #Ollama handling concurrent requests on the Llama 3 8B model. Ollama is ideal for local LLM deployment, but it is not designed for high concurrency scenarios essential for deployments in the cloud, powering applications with a large number of users. OpenLLM enables you to run any open-source model as OpenAI-compatible endpoints with just a single command. It supports a broad array of models like Llama 3, Mistral, and Qwen 2, including those fine-tuned with proprietary datasets and quantized versions. Ready to speed up your AI deployments? See the OpenLLM repo #BentoML #OpenSource

🚀 OpenLLM 0.6 is out providing fast inference speed! Check out our video comparing #OpenLLM and #Ollama handling concurrent requests on the Llama 3 8B model. Ollama is ideal for local LLM deployment, but it is not designed for high concurrency scenarios essential for deployments in the cloud, powering applications with a large number of users. OpenLLM enables you to run any open-source model as OpenAI-compatible endpoints with just a single command. It supports a broad array of models like Llama 3, Mistral, and Qwen 2, including those fine-tuned with proprietary datasets and quantized versions. Ready to speed up your AI deployments? See the OpenLLM repo #BentoML #OpenSource

BentoML

18,803 görüntüleme • 2 yıl önce

Check out mistral.rs, our #Rust-based open source inference engine allowing for fast #LLM serving for a variety of architectures including X-LoRA mixture-of-expert (MoE) models, Llama-3, Mistral/Mixtral, Gemma & many others. Built on the Hugging Face #Candle framework for #Rust w/ custom CUDA kernels in the backend (as well as support for Metal, Apple Accelerate, and Intel MKL for CPU use), you can easily create a REST API OpenAI compatible server or run via Python bindings. Key features include: ✅Prefix caching, continuous batching ✅Flash Attention V2 ✅Device offloading ✅GGUF or Hugging Face models ✅2, 3, 4, 5, 6 and 8 bit quantization ✅X-LoRA MoE non-granular scalings for fast inference ✅Grammar support ✅Continuous batching ✅LoRA support with weight merging ✅LlamaIndex 🦙 integration ...and much more. Incorporation into our GraphReasoning multi-agent modeling framework & LlamaIndex 🦙 allows you to combine in-context learning with adversarial agentic strategies, to dive deep into complex scientific analyses, such as to predict material behaviors, generate hypotheses, analyze papers and data, develop new research concepts, and much more. Check out mistral.rs: Join our Discord here: Rust Trending Rust Language

Check out mistral.rs, our #Rust-based open source inference engine allowing for fast #LLM serving for a variety of architectures including X-LoRA mixture-of-expert (MoE) models, Llama-3, Mistral/Mixtral, Gemma & many others. Built on the Hugging Face #Candle framework for #Rust w/ custom CUDA kernels in the backend (as well as support for Metal, Apple Accelerate, and Intel MKL for CPU use), you can easily create a REST API OpenAI compatible server or run via Python bindings. Key features include: ✅Prefix caching, continuous batching ✅Flash Attention V2 ✅Device offloading ✅GGUF or Hugging Face models ✅2, 3, 4, 5, 6 and 8 bit quantization ✅X-LoRA MoE non-granular scalings for fast inference ✅Grammar support ✅Continuous batching ✅LoRA support with weight merging ✅LlamaIndex 🦙 integration ...and much more. Incorporation into our GraphReasoning multi-agent modeling framework & LlamaIndex 🦙 allows you to combine in-context learning with adversarial agentic strategies, to dive deep into complex scientific analyses, such as to predict material behaviors, generate hypotheses, analyze papers and data, develop new research concepts, and much more. Check out mistral.rs: Join our Discord here: Rust Trending Rust Language

Markus J. Buehler

73,581 görüntüleme • 2 yıl önce

Many people are flat wrong about DeepSeek. You might think everyone freaked out about DeepSeek because the model is really good—which it is—or because it was from China, which is also true. But there's a more subtle reason: DeepSeek showed they could achieve those results using reinforcement learning instead of supervised fine-tuning. In other words, they didn't need an expensive phase where a bunch of people tuned the model by answering questions. They were able to do all of this faster and cheaper using reinforcement fine-tuning. This is a huge deal! I recorded the attached video with the following goals in mind: 1. Explain how reinforcement fine-tuning works 2. Show you how you can fine-tune your own models 3. Walk you through a complete code example

Many people are flat wrong about DeepSeek. You might think everyone freaked out about DeepSeek because the model is really good—which it is—or because it was from China, which is also true. But there's a more subtle reason: DeepSeek showed they could achieve those results using reinforcement learning instead of supervised fine-tuning. In other words, they didn't need an expensive phase where a bunch of people tuned the model by answering questions. They were able to do all of this faster and cheaper using reinforcement fine-tuning. This is a huge deal! I recorded the attached video with the following goals in mind: 1. Explain how reinforcement fine-tuning works 2. Show you how you can fine-tune your own models 3. Walk you through a complete code example

Santiago

105,589 görüntüleme • 1 yıl önce

You can now fine-tune Llama 3 without writing a single line of code! We are moving at breakneck speed. I recorded a video to show you how to fine-tune any open-source model in a few minutes. I'm using a GPT capable of taking a problem and turning it into a fine-tuned model that will solve it. You don't have to write any code. You only need to explain to a GPT what problem you want to solve and tell it you want to use Llama 3. For example, "fine-tune Llama 3" or "deploy zephyr." It feels magic. The system will recommend a dataset and fine-tune the model for you. I'm using Monster API, a platform that specializes in making fine-tuning and deploying open-source models easy and fast. Their stack is well-optimized to maximize fine-tuning efficiency using techniques like Q-Lora and vLLM. They are behind the GPT. Here is what you need to do: 1. Create an account at 2. Load the GPT with the link below This is as simple as it gets. When you are done, you can click a button to deploy the model and start using it. I have 10,000 free credits for anyone using the code "SANTIAGO" in the dashboard. You can use these credits to access, fine-tune, and deploy these open-source models. You can also keep up with their latest updates, and get free credits and special offers on their Discord server:

You can now fine-tune Llama 3 without writing a single line of code! We are moving at breakneck speed. I recorded a video to show you how to fine-tune any open-source model in a few minutes. I'm using a GPT capable of taking a problem and turning it into a fine-tuned model that will solve it. You don't have to write any code. You only need to explain to a GPT what problem you want to solve and tell it you want to use Llama 3. For example, "fine-tune Llama 3" or "deploy zephyr." It feels magic. The system will recommend a dataset and fine-tune the model for you. I'm using Monster API, a platform that specializes in making fine-tuning and deploying open-source models easy and fast. Their stack is well-optimized to maximize fine-tuning efficiency using techniques like Q-Lora and vLLM. They are behind the GPT. Here is what you need to do: 1. Create an account at 2. Load the GPT with the link below This is as simple as it gets. When you are done, you can click a button to deploy the model and start using it. I have 10,000 free credits for anyone using the code "SANTIAGO" in the dashboard. You can use these credits to access, fine-tune, and deploy these open-source models. You can also keep up with their latest updates, and get free credits and special offers on their Discord server:

Santiago

324,621 görüntüleme • 2 yıl önce

The #1 problem with local AI is now solved. There’s a free tool called that checks your hardware and tells you which models will actually run well before you download anything. So instead of guessing and hitting out-of-memory errors…it grades every model against your machine. What it does (right in your browser, no install): → detects your setup (RAM / CPU / GPU / VRAM) → scores each model for fit, speed, and context length → grades every quantization level (Q4_K_M, Q6_K, Q8_0, etc.) → labels what runs great vs okay vs too heavy It covers most of the open-weight stack, Llama, Qwen, Gemma, Mistral, DeepSeek, Phi and more, pulling requirements from llama.cpp, Ollama, and LM Studio. And it’s fully opensource

The #1 problem with local AI is now solved. There’s a free tool called that checks your hardware and tells you which models will actually run well before you download anything. So instead of guessing and hitting out-of-memory errors…it grades every model against your machine. What it does (right in your browser, no install): → detects your setup (RAM / CPU / GPU / VRAM) → scores each model for fit, speed, and context length → grades every quantization level (Q4_K_M, Q6_K, Q8_0, etc.) → labels what runs great vs okay vs too heavy It covers most of the open-weight stack, Llama, Qwen, Gemma, Mistral, DeepSeek, Phi and more, pulling requirements from llama.cpp, Ollama, and LM Studio. And it’s fully opensource

Alvaro Cintas

194,148 görüntüleme • 1 ay önce

New short course on Pretraining LLMs! Developed with Upstage and taught by their CEO Sung Kim and CSO Lucy Park. While prompting or fine-tuning existing models works well for many general language tasks, pretraining is valuable for specialized domains or languages with limited representation in current models. This course walks you through the LLM pretraining pipeline: 1. Data preparation: Learn to source, clean, and prepare training data using HuggingFace. 2. Model architecture: Configure transformer networks, including modifying existing models. 3. Training: Set up and run training using open-source libraries. 4. Evaluation: Benchmark performance using popular evaluation strategies. As an example use case, you'll also compare the output of a base model with its fine-tuned and further pretrained variants, to see the impact of pretraining on a model's ability to write Python. The course also explores an innovative technique called depth up-scaling, which Upstage used to train their Solar model family, reducing pretraining compute costs by up to 70%. This technique works by first duplicating layers of a smaller pretrained model to form a larger model, and then further pretraining the result. Sign up here!

New short course on Pretraining LLMs! Developed with Upstage and taught by their CEO Sung Kim and CSO Lucy Park. While prompting or fine-tuning existing models works well for many general language tasks, pretraining is valuable for specialized domains or languages with limited representation in current models. This course walks you through the LLM pretraining pipeline: 1. Data preparation: Learn to source, clean, and prepare training data using HuggingFace. 2. Model architecture: Configure transformer networks, including modifying existing models. 3. Training: Set up and run training using open-source libraries. 4. Evaluation: Benchmark performance using popular evaluation strategies. As an example use case, you'll also compare the output of a base model with its fine-tuned and further pretrained variants, to see the impact of pretraining on a model's ability to write Python. The course also explores an innovative technique called depth up-scaling, which Upstage used to train their Solar model family, reducing pretraining compute costs by up to 70%. This technique works by first duplicating layers of a smaller pretrained model to form a larger model, and then further pretraining the result. Sign up here!

Andrew Ng

85,714 görüntüleme • 2 yıl önce

An exciting new course: Fine-tuning and Reinforcement Learning for LLMs: Intro to Post-training, taught by Sharon Zhou, VP of AI at AMD. Available now at Post-training is the key technique used by frontier labs to turn a base LLM--a model trained on massive unlabeled text to predict the next word/token--into a helpful, reliable assistant that can follow instructions. I've also seen many applications where post-training is what turns a demo application that works only 80% of the time into a reliable system that consistently performs. This course will teach you the most important post-training techniques! In this 5 module course, Sharon walks you through the complete post-training pipeline: supervised fine-tuning, reward modeling, RLHF, and techniques like PPO and GRPO. You'll also learn to use LoRA for efficient training, and to design evals that catch problems before and after deployment. Skills you'll gain: - Apply supervised fine-tuning and reinforcement learning (RLHF, PPO, GRPO) to align models to desired behaviors - Use LoRA for efficient fine-tuning without retraining entire models - Prepare datasets and generate synthetic data for post-training - Understand how to operate LLM production pipelines, with go/no-go decision points and feedback loops These advanced methods aren’t limited to frontier AI labs anymore, and you can now use them in your own applications. Learn here:

An exciting new course: Fine-tuning and Reinforcement Learning for LLMs: Intro to Post-training, taught by Sharon Zhou, VP of AI at AMD. Available now at Post-training is the key technique used by frontier labs to turn a base LLM--a model trained on massive unlabeled text to predict the next word/token--into a helpful, reliable assistant that can follow instructions. I've also seen many applications where post-training is what turns a demo application that works only 80% of the time into a reliable system that consistently performs. This course will teach you the most important post-training techniques! In this 5 module course, Sharon walks you through the complete post-training pipeline: supervised fine-tuning, reward modeling, RLHF, and techniques like PPO and GRPO. You'll also learn to use LoRA for efficient training, and to design evals that catch problems before and after deployment. Skills you'll gain: - Apply supervised fine-tuning and reinforcement learning (RLHF, PPO, GRPO) to align models to desired behaviors - Use LoRA for efficient fine-tuning without retraining entire models - Prepare datasets and generate synthetic data for post-training - Understand how to operate LLM production pipelines, with go/no-go decision points and feedback loops These advanced methods aren’t limited to frontier AI labs anymore, and you can now use them in your own applications. Learn here:

Andrew Ng

132,304 görüntüleme • 11 ay önce

$PLTR cofounder Joe Lonsdale on investing in the 5 Layers of AI 🔮 “Over the next 10 years, every Fortune 500 company should benefit.” Layer 1: Chips Layer 2: Datacenters Level 3: Modeling (OpenAI, Anthropic, xAI) Level 4: Infrastructure & Deployment ( $META Llama, $PLTR 🔮(kinda) Level 5: Companies actually performing services in the economy “The big argument in venture is: is level 4 going to be eaten by level 3? Are the model companies themselves going to build out the infrastructure and own it? “The level 5 is about building things in the service economy. For example, healthcare billing is a $280B revenue industry in the U.S… We’re already making it 3x more productive. 3x higher margins… Via The Compound

$PLTR cofounder Joe Lonsdale on investing in the 5 Layers of AI 🔮 “Over the next 10 years, every Fortune 500 company should benefit.” Layer 1: Chips Layer 2: Datacenters Level 3: Modeling (OpenAI, Anthropic, xAI) Level 4: Infrastructure & Deployment ( $META Llama, $PLTR 🔮(kinda) Level 5: Companies actually performing services in the economy “The big argument in venture is: is level 4 going to be eaten by level 3? Are the model companies themselves going to build out the infrastructure and own it? “The level 5 is about building things in the service economy. For example, healthcare billing is a $280B revenue industry in the U.S… We’re already making it 3x more productive. 3x higher margins… Via The Compound

Jawwwn

151,179 görüntüleme • 1 yıl önce