Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

A 5-second 480P video, generated entirely on a Mac, in 30 seconds. No CUDA, no cloud, 3.9 GiB of memory. FastMetal brings the FastWan-QAD family to Apple Silicon. The DiT, the DMD sampler, and the decoder all run on Metal through MLX, INT8 by default. Three models: 1.3B at... show more

Hao AI Lab

6,680 subscribers

16,924 views • 5 days ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

(1/5) 5 seconds of video. 1.8s seconds of generation. One NVIDIA GeForce RTX 5090 on FastVideo. 🤯🚀 - FastWan-QAD, a new family of video generation models - Trained with FastVideo's Quantization-Aware Distillation (QAD) recipe. - Powered by FastVideo, we push a single NVIDIA GeForce RTX 5090 to its absolute limit: generating a 5-second 480P video in 1.8s end-to-end! 📜 Blog: 💻 Code: 💽 Model:

(1/5) 5 seconds of video. 1.8s seconds of generation. One NVIDIA GeForce RTX 5090 on FastVideo. 🤯🚀 - FastWan-QAD, a new family of video generation models - Trained with FastVideo's Quantization-Aware Distillation (QAD) recipe. - Powered by FastVideo, we push a single NVIDIA GeForce RTX 5090 to its absolute limit: generating a 5-second 480P video in 1.8s end-to-end! 📜 Blog: 💻 Code: 💽 Model:

Hao AI Lab

12,838 views • 2 months ago

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

Awni Hannun

42,761 views • 1 year ago

Tried creating a Pixar-style short. Generated in Seedance 2.5 at 480p then upscaled 4× with Topaz. Still working on prompting for 2.5. This video is text to video, no references used. You can check out the prompt in the replies. Generated and upscaled on Astorie AI

Tried creating a Pixar-style short. Generated in Seedance 2.5 at 480p then upscaled 4× with Topaz. Still working on prompting for 2.5. This video is text to video, no references used. You can check out the prompt in the replies. Generated and upscaled on Astorie AI

Kōda

13,832 views • 20 days ago

Latest Fireship video featured my run of DeepSeek R1 on M4 Mac Minis. Apple Silicon dominates in memory/bw unit economics, ideal for huge MoE models like R1 at batch_size=1 (the real-world use-case). SOTA AI may come from China, but it will run on American hardware.

Latest Fireship video featured my run of DeepSeek R1 on M4 Mac Minis. Apple Silicon dominates in memory/bw unit economics, ideal for huge MoE models like R1 at batch_size=1 (the real-world use-case). SOTA AI may come from China, but it will run on American hardware.

Alex Cheema - e/acc

79,775 views • 1 year ago

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.

Everyone wrote Apple off as the AI loser, but one hardware spec might flip that story upside down (Save this). @jason called Apple a screaming buy on the back of a single chip detail. The rumored M7 Ultra, expected around 2028, is designed to support up to 1.5TB of unified memory, enough to run frontier class trillion parameter AI models locally, with no cloud required. The Street's bear case on Apple is straightforward. Apple has no frontier model of its own, Siri has stumbled for years and the company effectively rents OpenAI's models for its hardest queries. That narrative treats Apple as the one Magnificent Seven name that missed the AI wave entirely but the bull case flips that framing on its head. If frontier AI models keep shrinking and getting cheaper to run, Apple doesn't need the smartest model in the world, it just needs to own the device that model runs on. And unified memory is the mechanism that makes this possible. Unlike traditional systems where the CPU and GPU each need separate memory, Apple's architecture lets the CPU, GPU and Neural Engine draw from one shared pool. A fully specced M7 Ultra could theoretically run something on the scale of a 1.2 trillion parameter model locally and that capability plugs directly into the one advantage Apple has spent over a decade building: privacy. Apple has already shipped Private Cloud Compute, a system designed so even Apple can't access user data processed off device. Apple doubled down on this at WWDC 2026, framing on device privacy as non-negotiable while rivals default to the cloud. If the best AI models get small enough to run on Apple silicon, the moat stops being the model and becomes the hardware it has to sit on. Milk Road Pro remains bullish on Apple and it remains as one of our core positions, if you want the full thesis + our full AI trades, come join us using the link below for just a $1.

Milk Road AI

37,459 views • 1 month ago

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵

I ran a 35-billion parameter AI agent on a $600 Mac mini. Specs: M4 Mac-Mini 16GB RAM The model doesn't fit in RAM. It pages from the SSD at 30 tokens/second. On NVIDIA, the same paging gives you 1.6 tok/s. Apple Silicon gives you 30. That's 18.6x faster. No cloud. No API keys. $0/month. Here's what it can do 🧵

thestreamingdev()

730,447 views • 5 months ago

NEW: Mistral AI releases Mistral 3, a family of multimodal models, including three start-of-the-art dense models (3B, 8B, and 14B) and Mistral Large 3 (675B, 41B active). All Apache 2.0! 🤗 Surprisingly, the 3B is small enough to run 100% locally in your browser on WebGPU! 🤯

NEW: Mistral AI releases Mistral 3, a family of multimodal models, including three start-of-the-art dense models (3B, 8B, and 14B) and Mistral Large 3 (675B, 41B active). All Apache 2.0! 🤗 Surprisingly, the 3B is small enough to run 100% locally in your browser on WebGPU! 🤯

Xenova

225,178 views • 8 months ago

$Introducing Antares: Cisco's family of small language models for locating known vulnerabilities in code. Antares-350M and Antares-1B are live on Hugging Face now. They can outperform many larger closed- and open-weight models at a fraction of the cost. Small enough to run locally. No shipping sensitive codebases to the cloud. Why it matters: vulnerability triage is expensive and slow. Antares helps democratize AI-assisted security for all. Explore the models + read the new Vulnerability Localization Benchmark:$

Introducing Antares: Cisco's family of small language models for locating known vulnerabilities in code. Antares-350M and Antares-1B are live on Hugging Face now. They can outperform many larger closed- and open-weight models at a fraction of the cost. Small enough to run locally. No shipping sensitive codebases to the cloud. Why it matters: vulnerability triage is expensive and slow. Antares helps democratize AI-assisted security for all. Explore the models + read the new Vulnerability Localization Benchmark:

Cisco AI

486,919 views • 1 month ago

Your cloud agent runs $10–20 a day. This box runs OpenAI's 120B model for $0. Same workload. It just sits in a closet on AMD silicon. A Minisforum mini PC, Strix Halo, 128GB unified memory. The model lives entirely on-device. 56 tokens a second. No API key. Nothing leaves the network. Cloud only handles the exceptions now. Claude or OpenAI by routing, not by default. Everyone calls this "free." Wrong word. It's not free. It's owned. The bill just stops existing. The cloud was never the only option. It was the default nobody bothered to question.

Your cloud agent runs $10–20 a day. This box runs OpenAI's 120B model for $0. Same workload. It just sits in a closet on AMD silicon. A Minisforum mini PC, Strix Halo, 128GB unified memory. The model lives entirely on-device. 56 tokens a second. No API key. Nothing leaves the network. Cloud only handles the exceptions now. Claude or OpenAI by routing, not by default. Everyone calls this "free." Wrong word. It's not free. It's owned. The bill just stops existing. The cloud was never the only option. It was the default nobody bothered to question.

RetroChainer

13,925 views • 2 months ago

$A used Mac Pro off Craigslist for $800, three weeks of upgrades and a $2,070 total build cost. 192GB RAM, an Afterburner card, 8TB NVMe and an eGPU enclosure later, video exports dropped from 18 minutes to 4, three local LLMs run simultaneously without breaking a sweat and models that normally need cloud compute run entirely locally. Apple wants $12,000 for similar specs. The used market sold him the same hardware for a fraction of that, and the $400 to $600 in monthly API savings paid the whole build back in under 4 months.$

A used Mac Pro off Craigslist for $800, three weeks of upgrades and a $2,070 total build cost. 192GB RAM, an Afterburner card, 8TB NVMe and an eGPU enclosure later, video exports dropped from 18 minutes to 4, three local LLMs run simultaneously without breaking a sweat and models that normally need cloud compute run entirely locally. Apple wants $12,000 for similar specs. The used market sold him the same hardware for a fraction of that, and the $400 to $600 in monthly API savings paid the whole build back in under 4 months.

0xMarioNawfal

138,169 views • 1 month ago

"Mac Minis for example are a very good fit" - Andrej Karpathy Andrej Karpathy shouted out my work on EXO Labs in his keynote at Y Combinator AI SUS! Here's the breakdown: Right now most AI workloads run in the cloud where requests from different users are continuously batched together. These workloads are FLOPS-bound and favors hardware with the best unit economics of $ per FLOP, i.e. enterprise GPUs. The personal computing revolution will shift these workloads to personal devices with lower batch sizes (mostly batch_size=1). batch_size=1 inference is memory-bound, because all of the model parameters need to be loaded into the GPU every time a token is generated. Apple Silicon with its Unified Memory architecture has a lot of memory and memory bandwidth per $ compared to other hardware: - M4 Pro Mac Mini, 24GB @ 273GB/s, $58.33/GB, $5.13/GB/s - H100, 80GB @ 3350GB/s, $625/GB, $14.93/GB/s The unit economics of Apple Silicon are becoming more compelling with every release of the Mac. The future of AI inference looks more like open weights models (OpenAI open weights model soonTM) run at low batch_size on personal devices.

"Mac Minis for example are a very good fit" - Andrej Karpathy Andrej Karpathy shouted out my work on EXO Labs in his keynote at Y Combinator AI SUS! Here's the breakdown: Right now most AI workloads run in the cloud where requests from different users are continuously batched together. These workloads are FLOPS-bound and favors hardware with the best unit economics of $ per FLOP, i.e. enterprise GPUs. The personal computing revolution will shift these workloads to personal devices with lower batch sizes (mostly batch_size=1). batch_size=1 inference is memory-bound, because all of the model parameters need to be loaded into the GPU every time a token is generated. Apple Silicon with its Unified Memory architecture has a lot of memory and memory bandwidth per $ compared to other hardware: - M4 Pro Mac Mini, 24GB @ 273GB/s, $58.33/GB, $5.13/GB/s - H100, 80GB @ 3350GB/s, $625/GB, $14.93/GB/s The unit economics of Apple Silicon are becoming more compelling with every release of the Mac. The future of AI inference looks more like open weights models (OpenAI open weights model soonTM) run at low batch_size on personal devices.

Alex Cheema

103,604 views • 1 year ago

I've been working on a new personal operating system that's fully local, private, and amnesic by default. Complete enough to be your only computer and the entire system is lean enough to boot from a USB stick, leaving no trace on the host machine. Models run locally on the user's own hardware. No account, no API key, no network calls after install. Ships with models preloaded, so the very first boot is productive with no network at all. One image, multiple modes, chosen at the boot menu to match your mission. Works offline, indefinitely.

I've been working on a new personal operating system that's fully local, private, and amnesic by default. Complete enough to be your only computer and the entire system is lean enough to boot from a USB stick, leaving no trace on the host machine. Models run locally on the user's own hardware. No account, no API key, no network calls after install. Ships with models preloaded, so the very first boot is productive with no network at all. One image, multiple modes, chosen at the boot menu to match your mission. Works offline, indefinitely.

nich

106,481 views • 4 months ago

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

Reese Chong

53,026 views • 4 months ago

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.

ollama

219,429 views • 2 years ago

100 MAC MINIS ON ONE METAL SHELF. IN THE CLOUD, EACH OF THOSE RENTS FOR $120 TO $500 A MONTH. that clip is a homemade rack: wire shelving packed with mac minis, each running macos, wired into one cluster. why macs and not a normal server? two reasons. you can only build, sign, and notarize ios and mac apps on apple hardware. cloud providers know it and charge for it. and apple silicon sips power and runs near-silent, so you can stack a lot of them on a shelf in a room, not a datacenter. the rent they're escaping: a single mac in the cloud runs ~$120 to $500 a month (macstadium up to aws) one engineer documented saving $4,000+ a month by self-hosting mac minis instead of cloud ci runners a used or new mini is a few hundred dollars once, then a few dollars of electricity what a shelf like this actually runs: ios and mac build farms, render jobs, app and device testing automation, and increasingly local ai inference. own the boxes, rent nothing. the uncomfortable part: the cloud was never the only option. it was the convenient one. the people who did the math bought the shelf. the honest caveat: it's real capex up front, real heat and power, and you become the sysadmin. no rented rack means no one to call at 3am. worth it at scale, overkill for one build a week. no rack rental, no per-hour metering, no fleet you don't own. save this before your cloud invoice renews again.

100 MAC MINIS ON ONE METAL SHELF. IN THE CLOUD, EACH OF THOSE RENTS FOR $120 TO $500 A MONTH. that clip is a homemade rack: wire shelving packed with mac minis, each running macos, wired into one cluster. why macs and not a normal server? two reasons. you can only build, sign, and notarize ios and mac apps on apple hardware. cloud providers know it and charge for it. and apple silicon sips power and runs near-silent, so you can stack a lot of them on a shelf in a room, not a datacenter. the rent they're escaping: a single mac in the cloud runs ~$120 to $500 a month (macstadium up to aws) one engineer documented saving $4,000+ a month by self-hosting mac minis instead of cloud ci runners a used or new mini is a few hundred dollars once, then a few dollars of electricity what a shelf like this actually runs: ios and mac build farms, render jobs, app and device testing automation, and increasingly local ai inference. own the boxes, rent nothing. the uncomfortable part: the cloud was never the only option. it was the convenient one. the people who did the math bought the shelf. the honest caveat: it's real capex up front, real heat and power, and you become the sysadmin. no rented rack means no one to call at 3am. worth it at scale, overkill for one build a week. no rack rental, no per-hour metering, no fleet you don't own. save this before your cloud invoice renews again.

RetroChainer

21,055 views • 1 month ago

Today, we’re thrilled to unveil the Arcee.ai Foundation Models, a new family of GenAI models designed from the ground up for enterprise reality. The first release—AFM-4.5B—is a 4.5-billion-parameter frontier model that delivers excellent accuracy, strict compliance, and very high cost efficiency. In short: enterprise-grade intelligence that can run anywhere—on a smartphone, at the edge, or in the cloud. For a quick taste, you can test AFM-4.5B in our playground and on For a deeper dive into the model’s training pipeline and benchmarks, details are available in our technical blog post. ➡️ Arcee AI playground: ➡️ Together AI playground: ➡️ Launch blog post: ➡️ Technical blog post: PS: As we’ll now focus on our AFM foundation models, and because we love open-source, we're opening up access to our previously closed-source language models. Details in the tech blog post!

Today, we’re thrilled to unveil the Arcee.ai Foundation Models, a new family of GenAI models designed from the ground up for enterprise reality. The first release—AFM-4.5B—is a 4.5-billion-parameter frontier model that delivers excellent accuracy, strict compliance, and very high cost efficiency. In short: enterprise-grade intelligence that can run anywhere—on a smartphone, at the edge, or in the cloud. For a quick taste, you can test AFM-4.5B in our playground and on For a deeper dive into the model’s training pipeline and benchmarks, details are available in our technical blog post. ➡️ Arcee AI playground: ➡️ Together AI playground: ➡️ Launch blog post: ➡️ Technical blog post: PS: As we’ll now focus on our AFM foundation models, and because we love open-source, we're opening up access to our previously closed-source language models. Details in the tech blog post!

Arcee.ai

21,934 views • 1 year ago

🚀 SANA-Video 2.0 is here! A full-stack optimized video model designed for efficiency — while still delivering high quality. We combine a hybrid architecture closely related to the recent Kimi K3 design, adopt Self-Flow from FLUX 3, and further accelerate it with our own Sol-Engine. Key technical ingredients: 🧠 Hybrid Linear–Softmax Attention 3 gated linear layers + 1 gated-softmax anchor (75% linear / 25% softmax) 🧱 Block Attention Residuals (AttnRes) Boosts deep-layer effective rank by ~12% 🏗️ Unified 5B & 14B models Trained from scratch under very limited resources: • 5B → only 16 nodes of H100 • 14B → 48 nodes of B200 Results 📊 • 84.30 VBench Total • 3.2× faster DiT forward than matched full-softmax at 720p/60s • 720p/5s in 13.06s on a single H100 with Sol-Engine • 120× faster than Wan 2.2-A14B under the same one-H100 setup SANA-Video 2.0 shows that high-quality 720p video generation can be both efficient and practical on a single GPU. 🎬 Project: 📄 Paper: 💻 Code: Proud of the team! 🎉 More details below 🧵

🚀 SANA-Video 2.0 is here! A full-stack optimized video model designed for efficiency — while still delivering high quality. We combine a hybrid architecture closely related to the recent Kimi K3 design, adopt Self-Flow from FLUX 3, and further accelerate it with our own Sol-Engine. Key technical ingredients: 🧠 Hybrid Linear–Softmax Attention 3 gated linear layers + 1 gated-softmax anchor (75% linear / 25% softmax) 🧱 Block Attention Residuals (AttnRes) Boosts deep-layer effective rank by ~12% 🏗️ Unified 5B & 14B models Trained from scratch under very limited resources: • 5B → only 16 nodes of H100 • 14B → 48 nodes of B200 Results 📊 • 84.30 VBench Total • 3.2× faster DiT forward than matched full-softmax at 720p/60s • 720p/5s in 13.06s on a single H100 with Sol-Engine • 120× faster than Wan 2.2-A14B under the same one-H100 setup SANA-Video 2.0 shows that high-quality 720p video generation can be both efficient and practical on a single GPU. 🎬 Project: 📄 Paper: 💻 Code: Proud of the team! 🎉 More details below 🧵

Enze Xie

25,338 views • 1 month ago

Mac user? You now have Suno at home. I ported ACE-Step 1.5 to native Swift - Offline. no Python, no cloud, no subscription. - Runs 100% on Apple Silicon (Metal GPU) - Full song generation in seconds - Personas (LoRA voice styles) built in - Audio editor + stem separation - Coming to the App Store FOR FREE Your music. Your hardware. No limits. Save this - dropping soon on App Store 🔥 #Suno #MusicAI #OpenSource #udio #aimusic #acestep #apple #swift #macOS #acestep

Mac user? You now have Suno at home. I ported ACE-Step 1.5 to native Swift - Offline. no Python, no cloud, no subscription. - Runs 100% on Apple Silicon (Metal GPU) - Full song generation in seconds - Personas (LoRA voice styles) built in - Audio editor + stem separation - Coming to the App Store FOR FREE Your music. Your hardware. No limits. Save this - dropping soon on App Store 🔥 #Suno #MusicAI #OpenSource #udio #aimusic #acestep #apple #swift #macOS #acestep

Vali Neagu

190,104 views • 6 months ago