Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

You can now run SQLCoder with a GUI on Apple Silicon or any NVIDIA GPU-enabled device! On Apple Silicon, just run CMAKE_ARGS="-DLLAMA_METAL=on" pip install "sqlcoder[llama-cpp]" sqlcoder launch The Apple Silicon version is not super accurate, but works great for simple questions – and everything happens (fairly fast) on just...

rishdotblog's profile picture

Rishabh Srivastava

12,304 subscribers

10,404 Aufrufe • vor 2 Jahren •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

6 Kommentare

Profilbild von Cris Lenta
Cris Lentavor 2 Jahren

This looks great! Have you tried also running it with .cpp models? Curious of the performance

Profilbild von Rishabh Srivastava
Rishabh Srivastavavor 2 Jahren

Thank you! The demo above is for a llama.cpp model (with q5 km quantization). Haven't done a rigorous eval on those yet though – will do in a few hours!

Profilbild von Nerdynav
Nerdynavvor 2 Jahren

hey great work, what's the license for this? Are code llama fine tunes allowed in their Meta's licensing?

Profilbild von Gautam Padiyar
Gautam Padiyarvor 2 Jahren

GPU with 20gb memory not required?

Profilbild von Rishabh Srivastava
Rishabh Srivastavavor 2 Jahren

7B model, so you only need ~16GB or so of memory to make it run without beam search

Profilbild von Harsh Patel
Harsh Patelvor 2 Jahren

On Apple silicon how do I make it use the 7b-2 model? It is downloading the v1 I guess

Ähnliche Videos

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.
0:20

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.

Locally AI - Local AI Chat

34,891 Aufrufe • vor 5 Monaten

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:
0:20

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

Awni Hannun

42,743 Aufrufe • vor 10 Monaten

Weekend plans? How about diving into LLMs on Apple devices? 🕷️💻 Just dropped my first tutorial: Run LLMs on your Mac with MLX! Perfect for AI beginners with Apple Silicon. More tech adventures coming soon! Please watch & subscribe:
11:04

Weekend plans? How about diving into LLMs on Apple devices? 🕷️💻 Just dropped my first tutorial: Run LLMs on your Mac with MLX! Perfect for AI beginners with Apple Silicon. More tech adventures coming soon! Please watch & subscribe:

qnguyen3

27,144 Aufrufe • vor 1 Jahr

Alibaba’s Qwen 3.5 is now running fully on-device on the iPhone 17 Pro. It outperforms models 4x its size, delivers strong visual understanding, and lets you switch reasoning on or off. This demo uses the 2B 6-bit version, optimized with MLX for Apple Silicon.
0:24

Alibaba’s Qwen 3.5 is now running fully on-device on the iPhone 17 Pro. It outperforms models 4x its size, delivers strong visual understanding, and lets you switch reasoning on or off. This demo uses the 2B 6-bit version, optimized with MLX for Apple Silicon.

Hugging Models

202,648 Aufrufe • vor 2 Monaten

This is what native feels like. No Electron. No Python runtime. Just Swift on Apple Silicon.
0:24

This is what native feels like. No Electron. No Python runtime. Just Swift on Apple Silicon.

Osaurus

455,516 Aufrufe • vor 5 Monaten

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.
0:56

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

Awni Hannun

88,726 Aufrufe • vor 2 Jahren

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)
4:07

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Prince Canuma

123,480 Aufrufe • vor 1 Jahr

5 years ago today, Apple broke up with Intel for Apple Silicon And today, Everyone would agree that THERE IS NO BETTER LAPTOP THAN A MACBOOK
0:33

5 years ago today, Apple broke up with Intel for Apple Silicon And today, Everyone would agree that THERE IS NO BETTER LAPTOP THAN A MACBOOK

Apple Design

118,382 Aufrufe • vor 11 Monaten

Apple silicon. Simple, right? Full video:
1:12

Apple silicon. Simple, right? Full video:

Marques Brownlee

1,497,403 Aufrufe • vor 7 Monaten

Run the Official ACE-Step-v1.5 Gradio app on Macs (Apple Silicon only) with 1 Click! Previously the launcher was not working on macs because the default launch option used VLLM, which does not work on Macs, so I switched it out to use PyTorch instead, and it works!
0:51

Run the Official ACE-Step-v1.5 Gradio app on Macs (Apple Silicon only) with 1 Click! Previously the launcher was not working on macs because the default launch option used VLLM, which does not work on Macs, so I switched it out to use PyTorch instead, and it works!

cocktail peanut

17,017 Aufrufe • vor 4 Monaten

APPLE’S GLASSES GET CUSTOM SILICON—META’S RAY-BANS NOW HAVE A DEADLINE Apple isn’t rushing AR. But it’s definitely watching Meta. While Zuckerberg’s Ray-Bans lead the smart glasses race, Apple is quietly prepping its own contender—minus the full AR flash. Instead, think AI-powered frames with Siri support, on-device smarts, and custom chips tuned for low-power, multi-camera use. Built off Apple Watch silicon, fabricated by TSMC, and entirely Apple-designed, these glasses will bridge the gap until full AR is read Source: Bloomberg
1:57

APPLE’S GLASSES GET CUSTOM SILICON—META’S RAY-BANS NOW HAVE A DEADLINE Apple isn’t rushing AR. But it’s definitely watching Meta. While Zuckerberg’s Ray-Bans lead the smart glasses race, Apple is quietly prepping its own contender—minus the full AR flash. Instead, think AI-powered frames with Siri support, on-device smarts, and custom chips tuned for low-power, multi-camera use. Built off Apple Watch silicon, fabricated by TSMC, and entirely Apple-designed, these glasses will bridge the gap until full AR is read Source: Bloomberg

Mario Nawfal

51,607 Aufrufe • vor 1 Jahr

Run Qwen-TTS on Your Mac, natively. Blaine Brown has created Qwen3-TTS MLX WebUI Enhanced, a Web UI for running Qwen-TTS using the Mac-native MLX framework, powered by MLX-Audio from Prince Canuma, it runs very smooth! 1-click install on pinokio (Apple Silicon Macs)
0:40

Run Qwen-TTS on Your Mac, natively. Blaine Brown has created Qwen3-TTS MLX WebUI Enhanced, a Web UI for running Qwen-TTS using the Mac-native MLX framework, powered by MLX-Audio from Prince Canuma, it runs very smooth! 1-click install on pinokio (Apple Silicon Macs)

cocktail peanut

168,965 Aufrufe • vor 3 Monaten

My Siri replacement “Hey GPT” doesn’t just work on your iPhone… It works on all your Apple devices! See it in action on my Apple Watch - you can even scroll through the text. Apple better hurry up with the next Siri… Install:
1:02

My Siri replacement “Hey GPT” doesn’t just work on your iPhone… It works on all your Apple devices! See it in action on my Apple Watch - you can even scroll through the text. Apple better hurry up with the next Siri… Install:

Mckay Wrigley

698,871 Aufrufe • vor 3 Jahren

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.
0:25

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

Adrien Grondin

3,552,070 Aufrufe • vor 3 Monaten

Private LLM for iOS v1.7.8 is now live on the App Store. 🎉 Experience the power of the latest Llama 3 8B Instruct model from AI at Meta, running privately, fully on-device with no internet connection or telemetry. Works on all Pro, Pro Max iPhones and Apple Silicon iPads. Also, full 8K context length on all devices with 8GB or more RAM: A feature you won’t see even on desktop LLM apps. Except ours, of course! 😅
1:53

Private LLM for iOS v1.7.8 is now live on the App Store. 🎉 Experience the power of the latest Llama 3 8B Instruct model from AI at Meta, running privately, fully on-device with no internet connection or telemetry. Works on all Pro, Pro Max iPhones and Apple Silicon iPads. Also, full 8K context length on all devices with 8GB or more RAM: A feature you won’t see even on desktop LLM apps. Except ours, of course! 😅

Private LLM

44,004 Aufrufe • vor 2 Jahren

All the wake-up sounds of Mac over 50 years: from the Apple I to the MacBook Pro on Apple Silicon
1:47

All the wake-up sounds of Mac over 50 years: from the Apple I to the MacBook Pro on Apple Silicon

Massimo

28,846 Aufrufe • vor 2 Monaten

Claude Code just got a lot easier to use 🙌 Just updated You can now install agents, commands, MCPs, and templates with a single command. Click on any card and it gives you the exact line to run. Everything is pulled from GitHub, the site runs on GitHub Pages, and downloads are logged with GitHub Actions.
1:05

Claude Code just got a lot easier to use 🙌 Just updated You can now install agents, commands, MCPs, and templates with a single command. Click on any card and it gives you the exact line to run. Everything is pulled from GitHub, the site runs on GitHub Pages, and downloads are logged with GitHub Actions.

Daniel San

135,186 Aufrufe • vor 10 Monaten

Apple Silicon never disappoints us... Full run CPUs + high rams used 🥵 4 MapKit Previews with the latest Xcode.
0:26

Apple Silicon never disappoints us... Full run CPUs + high rams used 🥵 4 MapKit Previews with the latest Xcode.

MING

55,837 Aufrufe • vor 1 Jahr

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.
0:28

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.

argmax

30,154 Aufrufe • vor 3 Monaten