Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Running Llama 3.1 locally as a code assistant in VSCode with ollama ✨ Download Ollama ( ✨ Download CodeGPT ( ✨ Select the Llama 3.1 model (8b, 70b or 405b) ✨ Click download ✨ Enjoy! :)

dani_avila7's profile picture

Daniel San

33,089 subscribers

359,968 görüntüleme • 2 yıl önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

11 Yorum

Asfi profil fotoğrafı
Asfi2 yıl önce

@ollama What minimum ram do you recommend for 405 b on Ollama?

Daniel San profil fotoğrafı
Daniel San2 yıl önce

@ollama Probably like 256 GB 😨 Check this post:

Anubis profil fotoğrafı
Anubis2 yıl önce

@ollama I tested it today. ☹️ but an ant was faster. 🤔

OldMan Montoya profil fotoğrafı
OldMan Montoya2 yıl önce

@ollama Ram reqs are super high I was going to try codeqwen for autocomplete and it required a 20 gb minimum... my machine has 16gb :(

Pushkar Yadav profil fotoğrafı
Pushkar Yadav2 yıl önce

@ollama Would be quite device intensive, wouldn't be able to run vs code then 😂 Would be better to use some inference rather then local

Daniel San profil fotoğrafı
Daniel San2 yıl önce

@ollama You can use different model sizes, for example Llama 3.1 is also available in 8B and 70B. In the video we are running 8B and it runs pretty well 🙌

Douglas Lopes profil fotoğrafı
Douglas Lopes2 yıl önce

@ollama can be used as copilot too?

Matti Vilola profil fotoğrafı
Matti Vilola2 yıl önce

@ollama Sounds awesome. Any experience with using your extension with @cursor_ai? It is mentioned, but I wonder how is the interaction, does it confuse each other or basically replace each ones functionality?

Jogi profil fotoğrafı
Jogi2 yıl önce

@ollama What kind of hardware is needed for the 8B and 70 B model? I think 405 is out of reach for now.

Gilad Bar-Ilan profil fotoğrafı
Gilad Bar-Ilan2 yıl önce

@ollama do you have a step by step instructions how to install and run this?

Daniel San profil fotoğrafı
Daniel San2 yıl önce

@ollama Check this link:

Benzer Videolar

Deepseek running locally and privately for autocompletion in VSCode! 🙌 In less than a minute, I'll show you how to download Deepseek-coder and set it as the autocompletion model in VSCode. You’ll need to use ollama to download the model and CodeGPT to select it as the autocompletion model. Enjoy the best models running locally with :)
0:49

Deepseek running locally and privately for autocompletion in VSCode! 🙌 In less than a minute, I'll show you how to download Deepseek-coder and set it as the autocompletion model in VSCode. You’ll need to use ollama to download the model and CodeGPT to select it as the autocompletion model. Enjoy the best models running locally with :)

Daniel San

991,351 görüntüleme • 1 yıl önce

Introducing Ollama fully Integrated into VSCode with CodeGPT Now you can download open-source models from ollama with a single click and install them as code assistants in VSCode. In this case, I download the qwen2:0.5b (Qwen) model in under a minute, and it becomes fully functional with just one click. The era of locally running code assistants has begun! 🚀
1:14

Introducing Ollama fully Integrated into VSCode with CodeGPT Now you can download open-source models from ollama with a single click and install them as code assistants in VSCode. In this case, I download the qwen2:0.5b (Qwen) model in under a minute, and it becomes fully functional with just one click. The era of locally running code assistants has begun! 🚀

Daniel San

101,567 görüntüleme • 2 yıl önce

Running Qwen2.5-Coder (0.5B params) locally on a Mac Intel Core i5 with CodeGPT and Ollama 🤩 In seconds, it explains Python code... all private and secure! Download CodeGPT from the VSCode Marketplace! 👏 Qwen ollama CodeGPT
0:27

Running Qwen2.5-Coder (0.5B params) locally on a Mac Intel Core i5 with CodeGPT and Ollama 🤩 In seconds, it explains Python code... all private and secure! Download CodeGPT from the VSCode Marketplace! 👏 Qwen ollama CodeGPT

Daniel San

30,613 görüntüleme • 1 yıl önce

This is all the code you need to create a Streamlit app to chat with Llama 3.1 8B locally via ollama. Fast and private.
0:20

This is all the code you need to create a Streamlit app to chat with Llama 3.1 8B locally via ollama. Fast and private.

Pietro Schirano

310,018 görüntüleme • 2 yıl önce

With today’s launch of our Llama 3.1 collection of models we’re making history with the largest and most capable open source AI model ever released. 128K context length, multilingual support, and new safety tools. Download 405B and our improved 8B & 70B here.
2:57

With today’s launch of our Llama 3.1 collection of models we’re making history with the largest and most capable open source AI model ever released. 128K context length, multilingual support, and new safety tools. Download 405B and our improved 8B & 70B here.

Ahmad Al-Dahle

866,589 görüntüleme • 2 yıl önce

Llama 3.1 Now Available in VSCode as a Code Assistant via Groq Inc 🚨 You can now use this new AI at Meta model directly in VSCode using the CodeGPT extension My first impression: The model is incredible 🚀
0:32

Llama 3.1 Now Available in VSCode as a Code Assistant via Groq Inc 🚨 You can now use this new AI at Meta model directly in VSCode using the CodeGPT extension My first impression: The model is incredible 🚀

Daniel San

263,015 görüntüleme • 2 yıl önce

Running multi agents on open source models has always been a hit or miss until... Llama 3.1 on Groq Inc with 405b and 70b
0:41

Running multi agents on open source models has always been a hit or miss until... Llama 3.1 on Groq Inc with 405b and 70b

FlowiseAI

48,208 görüntüleme • 2 yıl önce

Introducing DeepSeek R1:1.5b Running Locally in Cursor! In less than 4 minutes, I set up the DeepSeek R1:1.5b model, download it, and run it locally to seamlessly work with code in Cursor The video is shown in real-time, with the model impressively running on an Intel Core i5 😱 using Ollama and the CodeGPT extension 👇
3:45

Introducing DeepSeek R1:1.5b Running Locally in Cursor! In less than 4 minutes, I set up the DeepSeek R1:1.5b model, download it, and run it locally to seamlessly work with code in Cursor The video is shown in real-time, with the model impressively running on an Intel Core i5 😱 using Ollama and the CodeGPT extension 👇

Daniel San

584,021 görüntüleme • 1 yıl önce

DeepSeek-R1. Now running *locally* on your Umbrel. Yes, you can now self-host open source AI models like R1, Llama, and more on your home cloud — thanks to ollama and Open WebUI. Download Open WebUI and Ollama from the Umbrel App Store to get started.
0:24

DeepSeek-R1. Now running *locally* on your Umbrel. Yes, you can now self-host open source AI models like R1, Llama, and more on your home cloud — thanks to ollama and Open WebUI. Download Open WebUI and Ollama from the Umbrel App Store to get started.

Umbrel ☂️

36,562 görüntüleme • 1 yıl önce

Introducing Docker 🐳 AI Models for local + private LLMs in VSCode 🤩 You can now use Docker as a local model provider in VSCode, JetBrains, Cursor, and soon Visual Studio Enterprise. With Docker Model Runner (v4.40+), run AI models locally on your machine... no data sharing, no cloud dependency 👏 Just update Docker, open CodeGPT, select the model, click "Download", and enjoy 👉 Here’s how to update Docker:
1:01

Introducing Docker 🐳 AI Models for local + private LLMs in VSCode 🤩 You can now use Docker as a local model provider in VSCode, JetBrains, Cursor, and soon Visual Studio Enterprise. With Docker Model Runner (v4.40+), run AI models locally on your machine... no data sharing, no cloud dependency 👏 Just update Docker, open CodeGPT, select the model, click "Download", and enjoy 👉 Here’s how to update Docker:

Daniel San

65,125 görüntüleme • 1 yıl önce

Qwen2.5-coder is a game changer for secure, private, and local code development in VSCode 🤩 We've been testing the latest Qwen models running locally in VSCode through ollama and CodeGPT, and the results are incredible I highly recommend downloading and trying out these models directly from #CodeGPT.. you'll be impressed!
0:24

Qwen2.5-coder is a game changer for secure, private, and local code development in VSCode 🤩 We've been testing the latest Qwen models running locally in VSCode through ollama and CodeGPT, and the results are incredible I highly recommend downloading and trying out these models directly from #CodeGPT.. you'll be impressed!

Daniel San

50,573 görüntüleme • 1 yıl önce

Ollama WebUI is amazing. Download with Docker, choose your open-source model, customize it without writing a single line of code.
0:32

Ollama WebUI is amazing. Download with Docker, choose your open-source model, customize it without writing a single line of code.

Muratcan Koylan

149,118 görüntüleme • 2 yıl önce

Copilot is now completely free and private thanks to moved to: @lmstudio and the OpenHermes-2.5-Mistral-7B model Finally, #LMStudio is seamlessly integrated with CodeGPT, allowing you to leverage all models of this incredible platform locally in VSCode. Download CodeGPT extension for FREE here: Download LMStudio here: Enjoy! ✌️ Miguel Ángel Durán Brais Moure Carlos Santana
0:38

Copilot is now completely free and private thanks to moved to: @lmstudio and the OpenHermes-2.5-Mistral-7B model Finally, #LMStudio is seamlessly integrated with CodeGPT, allowing you to leverage all models of this incredible platform locally in VSCode. Download CodeGPT extension for FREE here: Download LMStudio here: Enjoy! ✌️ Miguel Ángel Durán Brais Moure Carlos Santana

Daniel San

91,227 görüntüleme • 2 yıl önce

Exciting day today as Meta launches its new Llama 3.1 models and they’re available in Bedrock immediately. Llama 3.1 is a substantial step forward and the models (highlighted by the powerful 405B model) are impressive and powerful. Customers are gonna love these high performance models. Giddy up!
0:41

Exciting day today as Meta launches its new Llama 3.1 models and they’re available in Bedrock immediately. Llama 3.1 is a substantial step forward and the models (highlighted by the powerful 405B model) are impressive and powerful. Customers are gonna love these high performance models. Giddy up!

Andy Jassy

45,000 görüntüleme • 2 yıl önce

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:
7:24

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

Santiago

97,505 görüntüleme • 1 yıl önce

QwQ-32B model from Qwen in VSCode... Got 20GB of free space? 😅 This is the latest reasoning model developed by Qwen In this video, I walk you through how to download and install it in VSCode using ollama, completely free and private 👇
0:29

QwQ-32B model from Qwen in VSCode... Got 20GB of free space? 😅 This is the latest reasoning model developed by Qwen In this video, I walk you through how to download and install it in VSCode using ollama, completely free and private 👇

Daniel San

29,169 görüntüleme • 1 yıl önce

Starting today, open source is leading the way. Introducing Llama 3.1: Our most capable models yet. Today we’re releasing a collection of new Llama 3.1 models including our long awaited 405B. These models deliver improved reasoning capabilities, a larger 128K token context window and improved support for 8 languages among other improvements. Llama 3.1 405B rivals leading closed source models on state-of-the-art capabilities across a range of tasks in general knowledge, steerability, math, tool use and multilingual translation. The models are available to download now directly from Meta or Hugging Face. With today’s release the ecosystem is also ready to go with 25+ partners rolling out our latest models — including Amazon Web Services, NVIDIA, Databricks, Groq Inc, Dell Technologies, Microsoft Azure and Google Cloud ready on day one. More details in the full announcement ➡️ Download Llama 3.1 models ➡️ With these releases we’re setting the stage for unprecedented new opportunities and we can’t wait to see the innovation our newest models will unlock across all levels of the AI community.
2:57

Starting today, open source is leading the way. Introducing Llama 3.1: Our most capable models yet. Today we’re releasing a collection of new Llama 3.1 models including our long awaited 405B. These models deliver improved reasoning capabilities, a larger 128K token context window and improved support for 8 languages among other improvements. Llama 3.1 405B rivals leading closed source models on state-of-the-art capabilities across a range of tasks in general knowledge, steerability, math, tool use and multilingual translation. The models are available to download now directly from Meta or Hugging Face. With today’s release the ecosystem is also ready to go with 25+ partners rolling out our latest models — including Amazon Web Services, NVIDIA, Databricks, Groq Inc, Dell Technologies, Microsoft Azure and Google Cloud ready on day one. More details in the full announcement ➡️ Download Llama 3.1 models ➡️ With these releases we’re setting the stage for unprecedented new opportunities and we can’t wait to see the innovation our newest models will unlock across all levels of the AI community.

AI at Meta

1,268,774 görüntüleme • 2 yıl önce