Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

You can now run SQLCoder with a GUI on Apple Silicon or any NVIDIA GPU-enabled device! On Apple Silicon, just run CMAKE_ARGS="-DLLAMA_METAL=on" pip install "sqlcoder[llama-cpp]" sqlcoder launch The Apple Silicon version is not super accurate, but works great for simple questions – and everything happens (fairly fast) on just... show more

Rishabh Srivastava

12,304 subscribers

10,404 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

6 Comments

Cris Lenta2 years ago

This looks great! Have you tried also running it with .cpp models? Curious of the performance

Rishabh Srivastava2 years ago

Thank you! The demo above is for a llama.cpp model (with q5 km quantization). Haven't done a rigorous eval on those yet though – will do in a few hours!

Nerdynav2 years ago

hey great work, what's the license for this? Are code llama fine tunes allowed in their Meta's licensing?

Gautam Padiyar2 years ago

GPU with 20gb memory not required?

Rishabh Srivastava2 years ago

7B model, so you only need ~16GB or so of memory to make it run without beam search

Harsh Patel2 years ago

On Apple silicon how do I make it use the 7b-2 model? It is downloading the v1 I guess

Related Videos

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.

Locally AI - Local AI Chat

34,891 views • 5 months ago

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

Awni Hannun

42,743 views • 10 months ago

Weekend plans? How about diving into LLMs on Apple devices? 🕷️💻 Just dropped my first tutorial: Run LLMs on your Mac with MLX! Perfect for AI beginners with Apple Silicon. More tech adventures coming soon! Please watch & subscribe:

Weekend plans? How about diving into LLMs on Apple devices? 🕷️💻 Just dropped my first tutorial: Run LLMs on your Mac with MLX! Perfect for AI beginners with Apple Silicon. More tech adventures coming soon! Please watch & subscribe:

qnguyen3

27,144 views • 1 year ago

Alibaba’s Qwen 3.5 is now running fully on-device on the iPhone 17 Pro. It outperforms models 4x its size, delivers strong visual understanding, and lets you switch reasoning on or off. This demo uses the 2B 6-bit version, optimized with MLX for Apple Silicon.

Alibaba’s Qwen 3.5 is now running fully on-device on the iPhone 17 Pro. It outperforms models 4x its size, delivers strong visual understanding, and lets you switch reasoning on or off. This demo uses the 2B 6-bit version, optimized with MLX for Apple Silicon.

Hugging Models

202,648 views • 2 months ago

This is what native feels like. No Electron. No Python runtime. Just Swift on Apple Silicon.

This is what native feels like. No Electron. No Python runtime. Just Swift on Apple Silicon.

Osaurus

455,516 views • 5 months ago

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

Awni Hannun

88,726 views • 2 years ago

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Prince Canuma

123,480 views • 1 year ago

5 years ago today, Apple broke up with Intel for Apple Silicon And today, Everyone would agree that THERE IS NO BETTER LAPTOP THAN A MACBOOK

5 years ago today, Apple broke up with Intel for Apple Silicon And today, Everyone would agree that THERE IS NO BETTER LAPTOP THAN A MACBOOK

Apple Design

118,382 views • 11 months ago

Apple silicon. Simple, right? Full video:

Apple silicon. Simple, right? Full video:

Marques Brownlee

1,497,403 views • 7 months ago

Run the Official ACE-Step-v1.5 Gradio app on Macs (Apple Silicon only) with 1 Click! Previously the launcher was not working on macs because the default launch option used VLLM, which does not work on Macs, so I switched it out to use PyTorch instead, and it works!

Run the Official ACE-Step-v1.5 Gradio app on Macs (Apple Silicon only) with 1 Click! Previously the launcher was not working on macs because the default launch option used VLLM, which does not work on Macs, so I switched it out to use PyTorch instead, and it works!

cocktail peanut

17,017 views • 4 months ago

APPLE’S GLASSES GET CUSTOM SILICON—META’S RAY-BANS NOW HAVE A DEADLINE Apple isn’t rushing AR. But it’s definitely watching Meta. While Zuckerberg’s Ray-Bans lead the smart glasses race, Apple is quietly prepping its own contender—minus the full AR flash. Instead, think AI-powered frames with Siri support, on-device smarts, and custom chips tuned for low-power, multi-camera use. Built off Apple Watch silicon, fabricated by TSMC, and entirely Apple-designed, these glasses will bridge the gap until full AR is read Source: Bloomberg

APPLE’S GLASSES GET CUSTOM SILICON—META’S RAY-BANS NOW HAVE A DEADLINE Apple isn’t rushing AR. But it’s definitely watching Meta. While Zuckerberg’s Ray-Bans lead the smart glasses race, Apple is quietly prepping its own contender—minus the full AR flash. Instead, think AI-powered frames with Siri support, on-device smarts, and custom chips tuned for low-power, multi-camera use. Built off Apple Watch silicon, fabricated by TSMC, and entirely Apple-designed, these glasses will bridge the gap until full AR is read Source: Bloomberg

Mario Nawfal

51,607 views • 1 year ago

Run Qwen-TTS on Your Mac, natively. Blaine Brown has created Qwen3-TTS MLX WebUI Enhanced, a Web UI for running Qwen-TTS using the Mac-native MLX framework, powered by MLX-Audio from Prince Canuma, it runs very smooth! 1-click install on pinokio (Apple Silicon Macs)

Run Qwen-TTS on Your Mac, natively. Blaine Brown has created Qwen3-TTS MLX WebUI Enhanced, a Web UI for running Qwen-TTS using the Mac-native MLX framework, powered by MLX-Audio from Prince Canuma, it runs very smooth! 1-click install on pinokio (Apple Silicon Macs)

cocktail peanut

168,965 views • 3 months ago

My Siri replacement “Hey GPT” doesn’t just work on your iPhone… It works on all your Apple devices! See it in action on my Apple Watch - you can even scroll through the text. Apple better hurry up with the next Siri… Install:

My Siri replacement “Hey GPT” doesn’t just work on your iPhone… It works on all your Apple devices! See it in action on my Apple Watch - you can even scroll through the text. Apple better hurry up with the next Siri… Install:

Mckay Wrigley

698,871 views • 3 years ago

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

Adrien Grondin

3,552,070 views • 3 months ago

Private LLM for iOS v1.7.8 is now live on the App Store. 🎉 Experience the power of the latest Llama 3 8B Instruct model from AI at Meta, running privately, fully on-device with no internet connection or telemetry. Works on all Pro, Pro Max iPhones and Apple Silicon iPads. Also, full 8K context length on all devices with 8GB or more RAM: A feature you won’t see even on desktop LLM apps. Except ours, of course! 😅

Private LLM for iOS v1.7.8 is now live on the App Store. 🎉 Experience the power of the latest Llama 3 8B Instruct model from AI at Meta, running privately, fully on-device with no internet connection or telemetry. Works on all Pro, Pro Max iPhones and Apple Silicon iPads. Also, full 8K context length on all devices with 8GB or more RAM: A feature you won’t see even on desktop LLM apps. Except ours, of course! 😅

Private LLM

44,004 views • 2 years ago

All the wake-up sounds of Mac over 50 years: from the Apple I to the MacBook Pro on Apple Silicon

All the wake-up sounds of Mac over 50 years: from the Apple I to the MacBook Pro on Apple Silicon

Massimo

28,846 views • 2 months ago

Claude Code just got a lot easier to use 🙌 Just updated You can now install agents, commands, MCPs, and templates with a single command. Click on any card and it gives you the exact line to run. Everything is pulled from GitHub, the site runs on GitHub Pages, and downloads are logged with GitHub Actions.

Claude Code just got a lot easier to use 🙌 Just updated You can now install agents, commands, MCPs, and templates with a single command. Click on any card and it gives you the exact line to run. Everything is pulled from GitHub, the site runs on GitHub Pages, and downloads are logged with GitHub Actions.

Daniel San

135,186 views • 10 months ago

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.

argmax

30,154 views • 3 months ago

Apple Silicon never disappoints us... Full run CPUs + high rams used 🥵 4 MapKit Previews with the latest Xcode.

Apple Silicon never disappoints us... Full run CPUs + high rams used 🥵 4 MapKit Previews with the latest Xcode.

MING

55,837 views • 1 year ago