Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

You can now run SQLCoder with a GUI on Apple Silicon or any NVIDIA GPU-enabled device! On Apple Silicon, just run CMAKE_ARGS="-DLLAMA_METAL=on" pip install "sqlcoder[llama-cpp]" sqlcoder launch The Apple Silicon version is not super accurate, but works great for simple questions – and everything happens (fairly fast) on just...

rishdotblog's profile picture

Rishabh Srivastava

12,304 subscribers

10,404 görüntüleme • 2 yıl önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

6 Yorum

Cris Lenta profil fotoğrafı
Cris Lenta2 yıl önce

This looks great! Have you tried also running it with .cpp models? Curious of the performance

Rishabh Srivastava profil fotoğrafı
Rishabh Srivastava2 yıl önce

Thank you! The demo above is for a llama.cpp model (with q5 km quantization). Haven't done a rigorous eval on those yet though – will do in a few hours!

Nerdynav profil fotoğrafı
Nerdynav2 yıl önce

hey great work, what's the license for this? Are code llama fine tunes allowed in their Meta's licensing?

Gautam Padiyar profil fotoğrafı
Gautam Padiyar2 yıl önce

GPU with 20gb memory not required?

Rishabh Srivastava profil fotoğrafı
Rishabh Srivastava2 yıl önce

7B model, so you only need ~16GB or so of memory to make it run without beam search

Harsh Patel profil fotoğrafı
Harsh Patel2 yıl önce

On Apple silicon how do I make it use the 7b-2 model? It is downloading the v1 I guess

Benzer Videolar

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.
0:20

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.

Locally AI - Local AI Chat

34,891 görüntüleme • 5 ay önce

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:
0:20

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

Awni Hannun

42,743 görüntüleme • 10 ay önce

Weekend plans? How about diving into LLMs on Apple devices? 🕷️💻 Just dropped my first tutorial: Run LLMs on your Mac with MLX! Perfect for AI beginners with Apple Silicon. More tech adventures coming soon! Please watch & subscribe:
11:04

Weekend plans? How about diving into LLMs on Apple devices? 🕷️💻 Just dropped my first tutorial: Run LLMs on your Mac with MLX! Perfect for AI beginners with Apple Silicon. More tech adventures coming soon! Please watch & subscribe:

qnguyen3

27,144 görüntüleme • 1 yıl önce

Alibaba’s Qwen 3.5 is now running fully on-device on the iPhone 17 Pro. It outperforms models 4x its size, delivers strong visual understanding, and lets you switch reasoning on or off. This demo uses the 2B 6-bit version, optimized with MLX for Apple Silicon.
0:24

Alibaba’s Qwen 3.5 is now running fully on-device on the iPhone 17 Pro. It outperforms models 4x its size, delivers strong visual understanding, and lets you switch reasoning on or off. This demo uses the 2B 6-bit version, optimized with MLX for Apple Silicon.

Hugging Models

202,648 görüntüleme • 2 ay önce

This is what native feels like. No Electron. No Python runtime. Just Swift on Apple Silicon.
0:24

This is what native feels like. No Electron. No Python runtime. Just Swift on Apple Silicon.

Osaurus

455,516 görüntüleme • 5 ay önce

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.
0:56

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

Awni Hannun

88,726 görüntüleme • 2 yıl önce

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)
4:07

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Prince Canuma

123,480 görüntüleme • 1 yıl önce

5 years ago today, Apple broke up with Intel for Apple Silicon And today, Everyone would agree that THERE IS NO BETTER LAPTOP THAN A MACBOOK
0:33

5 years ago today, Apple broke up with Intel for Apple Silicon And today, Everyone would agree that THERE IS NO BETTER LAPTOP THAN A MACBOOK

Apple Design

118,382 görüntüleme • 11 ay önce

Apple silicon. Simple, right? Full video:
1:12

Apple silicon. Simple, right? Full video:

Marques Brownlee

1,497,403 görüntüleme • 7 ay önce

Run the Official ACE-Step-v1.5 Gradio app on Macs (Apple Silicon only) with 1 Click! Previously the launcher was not working on macs because the default launch option used VLLM, which does not work on Macs, so I switched it out to use PyTorch instead, and it works!
0:51

Run the Official ACE-Step-v1.5 Gradio app on Macs (Apple Silicon only) with 1 Click! Previously the launcher was not working on macs because the default launch option used VLLM, which does not work on Macs, so I switched it out to use PyTorch instead, and it works!

cocktail peanut

17,017 görüntüleme • 4 ay önce

APPLE’S GLASSES GET CUSTOM SILICON—META’S RAY-BANS NOW HAVE A DEADLINE Apple isn’t rushing AR. But it’s definitely watching Meta. While Zuckerberg’s Ray-Bans lead the smart glasses race, Apple is quietly prepping its own contender—minus the full AR flash. Instead, think AI-powered frames with Siri support, on-device smarts, and custom chips tuned for low-power, multi-camera use. Built off Apple Watch silicon, fabricated by TSMC, and entirely Apple-designed, these glasses will bridge the gap until full AR is read Source: Bloomberg
1:57

APPLE’S GLASSES GET CUSTOM SILICON—META’S RAY-BANS NOW HAVE A DEADLINE Apple isn’t rushing AR. But it’s definitely watching Meta. While Zuckerberg’s Ray-Bans lead the smart glasses race, Apple is quietly prepping its own contender—minus the full AR flash. Instead, think AI-powered frames with Siri support, on-device smarts, and custom chips tuned for low-power, multi-camera use. Built off Apple Watch silicon, fabricated by TSMC, and entirely Apple-designed, these glasses will bridge the gap until full AR is read Source: Bloomberg

Mario Nawfal

51,607 görüntüleme • 1 yıl önce

Run Qwen-TTS on Your Mac, natively. Blaine Brown has created Qwen3-TTS MLX WebUI Enhanced, a Web UI for running Qwen-TTS using the Mac-native MLX framework, powered by MLX-Audio from Prince Canuma, it runs very smooth! 1-click install on pinokio (Apple Silicon Macs)
0:40

Run Qwen-TTS on Your Mac, natively. Blaine Brown has created Qwen3-TTS MLX WebUI Enhanced, a Web UI for running Qwen-TTS using the Mac-native MLX framework, powered by MLX-Audio from Prince Canuma, it runs very smooth! 1-click install on pinokio (Apple Silicon Macs)

cocktail peanut

168,965 görüntüleme • 3 ay önce

My Siri replacement “Hey GPT” doesn’t just work on your iPhone… It works on all your Apple devices! See it in action on my Apple Watch - you can even scroll through the text. Apple better hurry up with the next Siri… Install:
1:02

My Siri replacement “Hey GPT” doesn’t just work on your iPhone… It works on all your Apple devices! See it in action on my Apple Watch - you can even scroll through the text. Apple better hurry up with the next Siri… Install:

Mckay Wrigley

698,871 görüntüleme • 3 yıl önce

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.
0:25

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

Adrien Grondin

3,552,070 görüntüleme • 3 ay önce

Private LLM for iOS v1.7.8 is now live on the App Store. 🎉 Experience the power of the latest Llama 3 8B Instruct model from AI at Meta, running privately, fully on-device with no internet connection or telemetry. Works on all Pro, Pro Max iPhones and Apple Silicon iPads. Also, full 8K context length on all devices with 8GB or more RAM: A feature you won’t see even on desktop LLM apps. Except ours, of course! 😅
1:53

Private LLM for iOS v1.7.8 is now live on the App Store. 🎉 Experience the power of the latest Llama 3 8B Instruct model from AI at Meta, running privately, fully on-device with no internet connection or telemetry. Works on all Pro, Pro Max iPhones and Apple Silicon iPads. Also, full 8K context length on all devices with 8GB or more RAM: A feature you won’t see even on desktop LLM apps. Except ours, of course! 😅

Private LLM

44,004 görüntüleme • 2 yıl önce

All the wake-up sounds of Mac over 50 years: from the Apple I to the MacBook Pro on Apple Silicon
1:47

All the wake-up sounds of Mac over 50 years: from the Apple I to the MacBook Pro on Apple Silicon

Massimo

28,846 görüntüleme • 2 ay önce

Claude Code just got a lot easier to use 🙌 Just updated You can now install agents, commands, MCPs, and templates with a single command. Click on any card and it gives you the exact line to run. Everything is pulled from GitHub, the site runs on GitHub Pages, and downloads are logged with GitHub Actions.
1:05

Claude Code just got a lot easier to use 🙌 Just updated You can now install agents, commands, MCPs, and templates with a single command. Click on any card and it gives you the exact line to run. Everything is pulled from GitHub, the site runs on GitHub Pages, and downloads are logged with GitHub Actions.

Daniel San

135,186 görüntüleme • 10 ay önce

Apple Silicon never disappoints us... Full run CPUs + high rams used 🥵 4 MapKit Previews with the latest Xcode.
0:26

Apple Silicon never disappoints us... Full run CPUs + high rams used 🥵 4 MapKit Previews with the latest Xcode.

MING

55,837 görüntüleme • 1 yıl önce

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.
0:28

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.

argmax

30,154 görüntüleme • 3 ay önce