Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Running Llama 3.1 locally as a code assistant in VSCode with ollama ✨ Download Ollama ( ✨ Download CodeGPT ( ✨ Select the Llama 3.1 model (8b, 70b or 405b) ✨ Click download ✨ Enjoy! :)

Daniel San

33,490 subscribers

359,980 Aufrufe • vor 2 Jahren •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Asfi

Asfivor 2 Jahren

@ollama What minimum ram do you recommend for 405 b on Ollama?

Profilbild von Daniel San

Daniel Sanvor 2 Jahren

@ollama Probably like 256 GB 😨 Check this post:

Profilbild von Anubis

Anubisvor 2 Jahren

@ollama I tested it today. ☹️ but an ant was faster. 🤔

Profilbild von OldMan Montoya

OldMan Montoyavor 2 Jahren

@ollama Ram reqs are super high I was going to try codeqwen for autocomplete and it required a 20 gb minimum... my machine has 16gb :(

Profilbild von Pushkar Yadav

Pushkar Yadavvor 2 Jahren

@ollama Would be quite device intensive, wouldn't be able to run vs code then 😂 Would be better to use some inference rather then local

Profilbild von Daniel San

Daniel Sanvor 2 Jahren

@ollama You can use different model sizes, for example Llama 3.1 is also available in 8B and 70B. In the video we are running 8B and it runs pretty well 🙌

Profilbild von Douglas Lopes

Douglas Lopesvor 2 Jahren

@ollama can be used as copilot too?

Profilbild von Matti Vilola

Matti Vilolavor 2 Jahren

@ollama Sounds awesome. Any experience with using your extension with @cursor_ai? It is mentioned, but I wonder how is the interaction, does it confuse each other or basically replace each ones functionality?

Profilbild von Jogi

Jogivor 2 Jahren

@ollama What kind of hardware is needed for the 8B and 70 B model? I think 405 is out of reach for now.

Profilbild von Gilad Bar-Ilan

Gilad Bar-Ilanvor 2 Jahren

@ollama do you have a step by step instructions how to install and run this?

Profilbild von Daniel San

Daniel Sanvor 2 Jahren

@ollama Check this link:

Ähnliche Videos

Deepseek running locally and privately for autocompletion in VSCode! 🙌 In less than a minute, I'll show you how to download Deepseek-coder and set it as the autocompletion model in VSCode. You’ll need to use ollama to download the model and CodeGPT to select it as the autocompletion model. Enjoy the best models running locally with :)

Deepseek running locally and privately for autocompletion in VSCode! 🙌 In less than a minute, I'll show you how to download Deepseek-coder and set it as the autocompletion model in VSCode. You’ll need to use ollama to download the model and CodeGPT to select it as the autocompletion model. Enjoy the best models running locally with :)

Daniel San

992,488 Aufrufe • vor 1 Jahr

Introducing Ollama fully Integrated into VSCode with CodeGPT Now you can download open-source models from ollama with a single click and install them as code assistants in VSCode. In this case, I download the qwen2:0.5b (Qwen) model in under a minute, and it becomes fully functional with just one click. The era of locally running code assistants has begun! 🚀

Introducing Ollama fully Integrated into VSCode with CodeGPT Now you can download open-source models from ollama with a single click and install them as code assistants in VSCode. In this case, I download the qwen2:0.5b (Qwen) model in under a minute, and it becomes fully functional with just one click. The era of locally running code assistants has begun! 🚀

Daniel San

101,567 Aufrufe • vor 2 Jahren

Running Qwen2.5-Coder (0.5B params) locally on a Mac Intel Core i5 with CodeGPT and Ollama 🤩 In seconds, it explains Python code... all private and secure! Download CodeGPT from the VSCode Marketplace! 👏 Qwen ollama CodeGPT

Running Qwen2.5-Coder (0.5B params) locally on a Mac Intel Core i5 with CodeGPT and Ollama 🤩 In seconds, it explains Python code... all private and secure! Download CodeGPT from the VSCode Marketplace! 👏 Qwen ollama CodeGPT

Daniel San

30,613 Aufrufe • vor 1 Jahr

This is all the code you need to create a Streamlit app to chat with Llama 3.1 8B locally via ollama. Fast and private.

This is all the code you need to create a Streamlit app to chat with Llama 3.1 8B locally via ollama. Fast and private.

Pietro Schirano

310,018 Aufrufe • vor 2 Jahren

Use Copilot for free and completely privately with the DeepSeek models in VSCode Follow these 5 steps: - Download the CodeGPT extension for VSCode: - Download @Ollama_ai: - Once installed, run the following command in the terminal: "ollama run deepseek-coder" - In CodeGPT settings in VSCode, select Ollama as the provider and the model as deepseek-coder. - All set! ✅ Start coding with these powerful expert models in mathematics and programming. In this new version of CodeGPT, the interface has been improved, and the following models have been added: ✨ codellama:13b ✨ zephyr ✨ deepseek-coder ✨ deepseek-coder:33b Open-source models are gradually taking their rightful place, and the best part is... they run on CPUs 🤩

Use Copilot for free and completely privately with the DeepSeek models in VSCode Follow these 5 steps: - Download the CodeGPT extension for VSCode: - Download @Ollama_ai: - Once installed, run the following command in the terminal: "ollama run deepseek-coder" - In CodeGPT settings in VSCode, select Ollama as the provider and the model as deepseek-coder. - All set! ✅ Start coding with these powerful expert models in mathematics and programming. In this new version of CodeGPT, the interface has been improved, and the following models have been added: ✨ codellama:13b ✨ zephyr ✨ deepseek-coder ✨ deepseek-coder:33b Open-source models are gradually taking their rightful place, and the best part is... they run on CPUs 🤩

Daniel San

190,584 Aufrufe • vor 2 Jahren

Llama 3.1 Now Available in VSCode as a Code Assistant via Groq Inc 🚨 You can now use this new AI at Meta model directly in VSCode using the CodeGPT extension My first impression: The model is incredible 🚀

Llama 3.1 Now Available in VSCode as a Code Assistant via Groq Inc 🚨 You can now use this new AI at Meta model directly in VSCode using the CodeGPT extension My first impression: The model is incredible 🚀

Daniel San

263,015 Aufrufe • vor 2 Jahren

With today’s launch of our Llama 3.1 collection of models we’re making history with the largest and most capable open source AI model ever released. 128K context length, multilingual support, and new safety tools. Download 405B and our improved 8B & 70B here.

With today’s launch of our Llama 3.1 collection of models we’re making history with the largest and most capable open source AI model ever released. 128K context length, multilingual support, and new safety tools. Download 405B and our improved 8B & 70B here.

Ahmad Al-Dahle

867,049 Aufrufe • vor 2 Jahren

Running multi agents on open source models has always been a hit or miss until... Llama 3.1 on Groq Inc with 405b and 70b

Running multi agents on open source models has always been a hit or miss until... Llama 3.1 on Groq Inc with 405b and 70b

FlowiseAI

48,208 Aufrufe • vor 2 Jahren

Introducing DeepSeek R1:1.5b Running Locally in Cursor! In less than 4 minutes, I set up the DeepSeek R1:1.5b model, download it, and run it locally to seamlessly work with code in Cursor The video is shown in real-time, with the model impressively running on an Intel Core i5 😱 using Ollama and the CodeGPT extension 👇

Introducing DeepSeek R1:1.5b Running Locally in Cursor! In less than 4 minutes, I set up the DeepSeek R1:1.5b model, download it, and run it locally to seamlessly work with code in Cursor The video is shown in real-time, with the model impressively running on an Intel Core i5 😱 using Ollama and the CodeGPT extension 👇

Daniel San

584,100 Aufrufe • vor 1 Jahr

Qwen2.5-coder is a game changer for secure, private, and local code development in VSCode 🤩 We've been testing the latest Qwen models running locally in VSCode through ollama and CodeGPT, and the results are incredible I highly recommend downloading and trying out these models directly from #CodeGPT.. you'll be impressed!

Qwen2.5-coder is a game changer for secure, private, and local code development in VSCode 🤩 We've been testing the latest Qwen models running locally in VSCode through ollama and CodeGPT, and the results are incredible I highly recommend downloading and trying out these models directly from #CodeGPT.. you'll be impressed!

Daniel San

50,818 Aufrufe • vor 1 Jahr

DeepSeek-R1. Now running *locally* on your Umbrel. Yes, you can now self-host open source AI models like R1, Llama, and more on your home cloud — thanks to ollama and Open WebUI. Download Open WebUI and Ollama from the Umbrel App Store to get started.

DeepSeek-R1. Now running locally on your Umbrel. Yes, you can now self-host open source AI models like R1, Llama, and more on your home cloud — thanks to ollama and Open WebUI. Download Open WebUI and Ollama from the Umbrel App Store to get started.

Umbrel ☂️

36,562 Aufrufe • vor 1 Jahr

Introducing Docker 🐳 AI Models for local + private LLMs in VSCode 🤩 You can now use Docker as a local model provider in VSCode, JetBrains, Cursor, and soon Visual Studio Enterprise. With Docker Model Runner (v4.40+), run AI models locally on your machine... no data sharing, no cloud dependency 👏 Just update Docker, open CodeGPT, select the model, click "Download", and enjoy 👉 Here’s how to update Docker:

Introducing Docker 🐳 AI Models for local + private LLMs in VSCode 🤩 You can now use Docker as a local model provider in VSCode, JetBrains, Cursor, and soon Visual Studio Enterprise. With Docker Model Runner (v4.40+), run AI models locally on your machine... no data sharing, no cloud dependency 👏 Just update Docker, open CodeGPT, select the model, click "Download", and enjoy 👉 Here’s how to update Docker:

Daniel San

65,228 Aufrufe • vor 1 Jahr

Ollama WebUI is amazing. Download with Docker, choose your open-source model, customize it without writing a single line of code.

Ollama WebUI is amazing. Download with Docker, choose your open-source model, customize it without writing a single line of code.

Muratcan Koylan

149,118 Aufrufe • vor 2 Jahren

Exciting day today as Meta launches its new Llama 3.1 models and they’re available in Bedrock immediately. Llama 3.1 is a substantial step forward and the models (highlighted by the powerful 405B model) are impressive and powerful. Customers are gonna love these high performance models. Giddy up!

Exciting day today as Meta launches its new Llama 3.1 models and they’re available in Bedrock immediately. Llama 3.1 is a substantial step forward and the models (highlighted by the powerful 405B model) are impressive and powerful. Customers are gonna love these high performance models. Giddy up!

Andy Jassy

45,000 Aufrufe • vor 2 Jahren

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

Santiago

97,543 Aufrufe • vor 1 Jahr

Copilot is now completely free and private thanks to moved to: @lmstudio and the OpenHermes-2.5-Mistral-7B model Finally, #LMStudio is seamlessly integrated with CodeGPT, allowing you to leverage all models of this incredible platform locally in VSCode. Download CodeGPT extension for FREE here: Download LMStudio here: Enjoy! ✌️ Miguel Ángel Durán Brais Moure Carlos Santana

Copilot is now completely free and private thanks to moved to: @lmstudio and the OpenHermes-2.5-Mistral-7B model Finally, #LMStudio is seamlessly integrated with CodeGPT, allowing you to leverage all models of this incredible platform locally in VSCode. Download CodeGPT extension for FREE here: Download LMStudio here: Enjoy! ✌️ Miguel Ángel Durán Brais Moure Carlos Santana

Daniel San

91,415 Aufrufe • vor 2 Jahren

You can now try Llama 3.1 405B for free (link below)! This is the largest open-source model out there, and for the first time, an open model is competitive with closed models. This time around, Meta did something new: Llama 3.1 has a license that allows developers to use it to enhance other models. For the first time, you can distill Llama 3.1 405B's capabilities into a smaller, more practical model for your use case. First, here is the link where you can play with Llama 3.1 for free: The model is hosted in Tune Studio, an end-to-end platform for developing applications using Large Language Models. They are sponsoring this post. Take a look at the attached video. It will show you how you can fine-tune a simple model using Llama 3.1 without leaving the platform: 1. You can create an empty dataset 2. Use the playground to generate and record interactions with Llama 3.1 3. Modify the dataset directly using the playground 4. Export the data and fine-tune a smaller model Fast and easy! As long as you have a web browser, you can start experimenting with fine-tuning and Llama 3.1. That's all it takes!

You can now try Llama 3.1 405B for free (link below)! This is the largest open-source model out there, and for the first time, an open model is competitive with closed models. This time around, Meta did something new: Llama 3.1 has a license that allows developers to use it to enhance other models. For the first time, you can distill Llama 3.1 405B's capabilities into a smaller, more practical model for your use case. First, here is the link where you can play with Llama 3.1 for free: The model is hosted in Tune Studio, an end-to-end platform for developing applications using Large Language Models. They are sponsoring this post. Take a look at the attached video. It will show you how you can fine-tune a simple model using Llama 3.1 without leaving the platform: 1. You can create an empty dataset 2. Use the playground to generate and record interactions with Llama 3.1 3. Modify the dataset directly using the playground 4. Export the data and fine-tune a smaller model Fast and easy! As long as you have a web browser, you can start experimenting with fine-tuning and Llama 3.1. That's all it takes!

Santiago

55,609 Aufrufe • vor 2 Jahren