Загрузка видео...

Не удалось загрузить видео

На главную

You can now run SQLCoder with a GUI on Apple Silicon or any NVIDIA GPU-enabled device! On Apple Silicon, just run CMAKE_ARGS="-DLLAMA_METAL=on" pip install "sqlcoder[llama-cpp]" sqlcoder launch The Apple Silicon version is not super accurate, but works great for simple questions – and everything happens (fairly fast) on just...

rishdotblog's profile picture

Rishabh Srivastava

12,304 subscribers

10,404 просмотров • 2 лет назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 6

Фото профиля Cris Lenta
Cris Lenta2 лет назад

This looks great! Have you tried also running it with .cpp models? Curious of the performance

Фото профиля Rishabh Srivastava
Rishabh Srivastava2 лет назад

Thank you! The demo above is for a llama.cpp model (with q5 km quantization). Haven't done a rigorous eval on those yet though – will do in a few hours!

Фото профиля Nerdynav
Nerdynav2 лет назад

hey great work, what's the license for this? Are code llama fine tunes allowed in their Meta's licensing?

Фото профиля Gautam Padiyar
Gautam Padiyar2 лет назад

GPU with 20gb memory not required?

Фото профиля Rishabh Srivastava
Rishabh Srivastava2 лет назад

7B model, so you only need ~16GB or so of memory to make it run without beam search

Фото профиля Harsh Patel
Harsh Patel2 лет назад

On Apple silicon how do I make it use the 7b-2 model? It is downloading the v1 I guess

Похожие видео

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.
0:20

Run the latest Ministral 3 models by Mistral AI on macOS. Running privately on-device and optimized for Apple Silicon with MLX.

Locally AI - Local AI Chat

34,891 просмотров • 5 месяцев назад

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:
0:20

The latest MLX has a CUDA back-end! To get started: pip install "mlx[cuda]" With the same codebase you can develop locally, run your model on Apple silicon, or in the cloud on Nvidia GPUs. MLX is designed around Apple silicon - which has a unified memory architecture. It uses the same design with CUDA. So there's no need to move arrays around from CPU memory to GPU memory. Note, this is early days - some operations are missing and performance is still being optimized. But it's already quite fast for Transformer training, text generation, and more! Here's a demo using mlx-lm to generate text with Llama 3 8B (bf16) on an A100:

Awni Hannun

42,743 просмотров • 10 месяцев назад

Weekend plans? How about diving into LLMs on Apple devices? 🕷️💻 Just dropped my first tutorial: Run LLMs on your Mac with MLX! Perfect for AI beginners with Apple Silicon. More tech adventures coming soon! Please watch & subscribe:
11:04

Weekend plans? How about diving into LLMs on Apple devices? 🕷️💻 Just dropped my first tutorial: Run LLMs on your Mac with MLX! Perfect for AI beginners with Apple Silicon. More tech adventures coming soon! Please watch & subscribe:

qnguyen3

27,144 просмотров • 1 год назад

Alibaba’s Qwen 3.5 is now running fully on-device on the iPhone 17 Pro. It outperforms models 4x its size, delivers strong visual understanding, and lets you switch reasoning on or off. This demo uses the 2B 6-bit version, optimized with MLX for Apple Silicon.
0:24

Alibaba’s Qwen 3.5 is now running fully on-device on the iPhone 17 Pro. It outperforms models 4x its size, delivers strong visual understanding, and lets you switch reasoning on or off. This demo uses the 2B 6-bit version, optimized with MLX for Apple Silicon.

Hugging Models

202,648 просмотров • 2 месяцев назад

This is what native feels like. No Electron. No Python runtime. Just Swift on Apple Silicon.
0:24

This is what native feels like. No Electron. No Python runtime. Just Swift on Apple Silicon.

Osaurus

455,516 просмотров • 5 месяцев назад

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.
0:56

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

Awni Hannun

88,726 просмотров • 2 лет назад

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)
4:07

Day 1 of 3 MLX Releases: Introducing MLX-Audio 🚀🔥 A text-to-speech (TTS) and Speech-to-Speech (STS) library built on Apple's MLX framework, providing efficient speech synthesis on Apple Silicon. Features ⚡️Fast inference on Apple Silicon (M series chips) 🤖Multiple language support 🗣️Voice customization options 🚀Quantization support for optimized performance Supported models: 🪶Kokoro - A multilingual TTS model with 82M params that supports various languages and voice styles. With more models coming soon. Get started: > pip install mlx-audio Please leave us a star and send a PR :)

Prince Canuma

123,480 просмотров • 1 год назад

5 years ago today, Apple broke up with Intel for Apple Silicon And today, Everyone would agree that THERE IS NO BETTER LAPTOP THAN A MACBOOK
0:33

5 years ago today, Apple broke up with Intel for Apple Silicon And today, Everyone would agree that THERE IS NO BETTER LAPTOP THAN A MACBOOK

Apple Design

118,382 просмотров • 11 месяцев назад

Apple silicon. Simple, right? Full video:
1:12

Apple silicon. Simple, right? Full video:

Marques Brownlee

1,497,403 просмотров • 7 месяцев назад

Run the Official ACE-Step-v1.5 Gradio app on Macs (Apple Silicon only) with 1 Click! Previously the launcher was not working on macs because the default launch option used VLLM, which does not work on Macs, so I switched it out to use PyTorch instead, and it works!
0:51

Run the Official ACE-Step-v1.5 Gradio app on Macs (Apple Silicon only) with 1 Click! Previously the launcher was not working on macs because the default launch option used VLLM, which does not work on Macs, so I switched it out to use PyTorch instead, and it works!

cocktail peanut

17,017 просмотров • 4 месяцев назад

APPLE’S GLASSES GET CUSTOM SILICON—META’S RAY-BANS NOW HAVE A DEADLINE Apple isn’t rushing AR. But it’s definitely watching Meta. While Zuckerberg’s Ray-Bans lead the smart glasses race, Apple is quietly prepping its own contender—minus the full AR flash. Instead, think AI-powered frames with Siri support, on-device smarts, and custom chips tuned for low-power, multi-camera use. Built off Apple Watch silicon, fabricated by TSMC, and entirely Apple-designed, these glasses will bridge the gap until full AR is read Source: Bloomberg
1:57

APPLE’S GLASSES GET CUSTOM SILICON—META’S RAY-BANS NOW HAVE A DEADLINE Apple isn’t rushing AR. But it’s definitely watching Meta. While Zuckerberg’s Ray-Bans lead the smart glasses race, Apple is quietly prepping its own contender—minus the full AR flash. Instead, think AI-powered frames with Siri support, on-device smarts, and custom chips tuned for low-power, multi-camera use. Built off Apple Watch silicon, fabricated by TSMC, and entirely Apple-designed, these glasses will bridge the gap until full AR is read Source: Bloomberg

Mario Nawfal

51,607 просмотров • 1 год назад

Run Qwen-TTS on Your Mac, natively. Blaine Brown has created Qwen3-TTS MLX WebUI Enhanced, a Web UI for running Qwen-TTS using the Mac-native MLX framework, powered by MLX-Audio from Prince Canuma, it runs very smooth! 1-click install on pinokio (Apple Silicon Macs)
0:40

Run Qwen-TTS on Your Mac, natively. Blaine Brown has created Qwen3-TTS MLX WebUI Enhanced, a Web UI for running Qwen-TTS using the Mac-native MLX framework, powered by MLX-Audio from Prince Canuma, it runs very smooth! 1-click install on pinokio (Apple Silicon Macs)

cocktail peanut

168,965 просмотров • 3 месяцев назад

My Siri replacement “Hey GPT” doesn’t just work on your iPhone… It works on all your Apple devices! See it in action on my Apple Watch - you can even scroll through the text. Apple better hurry up with the next Siri… Install:
1:02

My Siri replacement “Hey GPT” doesn’t just work on your iPhone… It works on all your Apple devices! See it in action on my Apple Watch - you can even scroll through the text. Apple better hurry up with the next Siri… Install:

Mckay Wrigley

698,871 просмотров • 3 лет назад

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.
0:25

The new Qwen 3.5 by Qwen running on-device on iPhone 17 Pro. Qwen 3.5 beats models 4 times its size, has strong visual understanding, and can toggle reasoning on or off. The 2B 6-bit model here is running with MLX optimized for Apple Silicon.

Adrien Grondin

3,552,070 просмотров • 3 месяцев назад

Private LLM for iOS v1.7.8 is now live on the App Store. 🎉 Experience the power of the latest Llama 3 8B Instruct model from AI at Meta, running privately, fully on-device with no internet connection or telemetry. Works on all Pro, Pro Max iPhones and Apple Silicon iPads. Also, full 8K context length on all devices with 8GB or more RAM: A feature you won’t see even on desktop LLM apps. Except ours, of course! 😅
1:53

Private LLM for iOS v1.7.8 is now live on the App Store. 🎉 Experience the power of the latest Llama 3 8B Instruct model from AI at Meta, running privately, fully on-device with no internet connection or telemetry. Works on all Pro, Pro Max iPhones and Apple Silicon iPads. Also, full 8K context length on all devices with 8GB or more RAM: A feature you won’t see even on desktop LLM apps. Except ours, of course! 😅

Private LLM

44,004 просмотров • 2 лет назад

All the wake-up sounds of Mac over 50 years: from the Apple I to the MacBook Pro on Apple Silicon
1:47

All the wake-up sounds of Mac over 50 years: from the Apple I to the MacBook Pro on Apple Silicon

Massimo

28,846 просмотров • 2 месяцев назад

Claude Code just got a lot easier to use 🙌 Just updated You can now install agents, commands, MCPs, and templates with a single command. Click on any card and it gives you the exact line to run. Everything is pulled from GitHub, the site runs on GitHub Pages, and downloads are logged with GitHub Actions.
1:05

Claude Code just got a lot easier to use 🙌 Just updated You can now install agents, commands, MCPs, and templates with a single command. Click on any card and it gives you the exact line to run. Everything is pulled from GitHub, the site runs on GitHub Pages, and downloads are logged with GitHub Actions.

Daniel San

135,186 просмотров • 10 месяцев назад

Apple Silicon never disappoints us... Full run CPUs + high rams used 🥵 4 MapKit Previews with the latest Xcode.
0:26

Apple Silicon never disappoints us... Full run CPUs + high rams used 🥵 4 MapKit Previews with the latest Xcode.

MING

55,837 просмотров • 1 год назад

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.
0:28

TTSKit now achieves sub-100ms time-to-first-byte for Qwen3-TTS 1.7b on Apple Silicon! Link to the code repo and details in comments.

argmax

30,154 просмотров • 3 месяцев назад