Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

This is all the code you need to create a Streamlit app to chat with Llama 3.1 8B locally via ollama. Fast and private.

Pietro Schirano

104,447 subscribers

310,018 views • 1 year ago •via X (Twitter)

Science & Technology Education Arts

Anya Rossi• Live Now

Private livecam show

10 Comments

Pietro Schirano1 year ago

Here it is 👇

topmass1 year ago

@ollama there is a really cool chrome extension that runs all your ollama models locally too - no fluff just an extension that opens a chat where you select one of your already downloaded models and you can chat

Tom Wildflower1 year ago

@ollama I thought I was excited about the things I could build the last few weeks/months. I'm over the moon now, I can't wait to just be able to create. My brick and mortar businesses are legitimately on the chopping block. Lol

Sebastian BO1 year ago

@ollama llama-engineer...

Shadow Defense1 year ago

@ollama What is the easiest way to do this locally because I don't really know much about GitHub and Python and all this other stuff I just want to open it and it work like check GPT does what's the easiest method so that it can have unlimited chat

Lucky1 year ago

@ollama i trade u prompt for screenshot I took

Inka Hacker (sun/acc)1 year ago

@ollama Runs locally?

Pietro Schirano1 year ago

@ollama Yeah, the 8B version quantized is around 5GB.

someone1 year ago

@ollama can you please make changes in claude engineer so that it supports ollama?

S.D.Gwynn1 year ago

@ollama Nice. Is there a way to use Claude’s artifacts and projects setup locally?

Related Videos

Drag & Drop to build local chat app with memory using Llama 3.1 without writing a single line of code (100% free and without internet).

Drag & Drop to build local chat app with memory using Llama 3.1 without writing a single line of code (100% free and without internet).

Shubham Saboo

12,443 views • 1 year ago

Here's a mini-ChatGPT app that runs locally on your computer. You can chat with it just like you would chat with ChatGPT. We use: - DeepSeek R1 as the LLM - ollama to locally serve R1 - Chainlit for the UI Let's build it!

Here's a mini-ChatGPT app that runs locally on your computer. You can chat with it just like you would chat with ChatGPT. We use: - DeepSeek R1 as the LLM - ollama to locally serve R1 - Chainlit for the UI Let's build it!

Avi Chawla

40,265 views • 1 year ago

Instant Intelligence is wild with Llama 3.1 8B + Groq 🤯

Instant Intelligence is wild with Llama 3.1 8B + Groq 🤯

Min Choi

692,933 views • 1 year ago

$Fuck it! You can now run *any* GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡$

Fuck it! You can now run any GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡

Vaibhav (VB) Srivastav

317,790 views • 1 year ago

Running Qwen2.5-Coder (0.5B params) locally on a Mac Intel Core i5 with CodeGPT and Ollama 🤩 In seconds, it explains Python code... all private and secure! Download CodeGPT from the VSCode Marketplace! 👏 Qwen ollama CodeGPT

Running Qwen2.5-Coder (0.5B params) locally on a Mac Intel Core i5 with CodeGPT and Ollama 🤩 In seconds, it explains Python code... all private and secure! Download CodeGPT from the VSCode Marketplace! 👏 Qwen ollama CodeGPT

Daniel San

30,613 views • 1 year ago

I just created my own OCR app using Llama 3.2 vision! Upload an image, and it converts it into structured markdown using Llama 3.2 multimodal! Here's what I used: - Ollama for serving Llama 3.2 vision locally. - Streamlit for the UI. Everything in just 50 lines of code! Find the code in the next tweet. -- Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. Find me → Avi Chawla

I just created my own OCR app using Llama 3.2 vision! Upload an image, and it converts it into structured markdown using Llama 3.2 multimodal! Here's what I used: - Ollama for serving Llama 3.2 vision locally. - Streamlit for the UI. Everything in just 50 lines of code! Find the code in the next tweet. -- Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. Find me → Avi Chawla

Avi Chawla

131,139 views • 1 year ago

You can now try Llama 3.1 405B for free (link below)! This is the largest open-source model out there, and for the first time, an open model is competitive with closed models. This time around, Meta did something new: Llama 3.1 has a license that allows developers to use it to enhance other models. For the first time, you can distill Llama 3.1 405B's capabilities into a smaller, more practical model for your use case. First, here is the link where you can play with Llama 3.1 for free: The model is hosted in Tune Studio, an end-to-end platform for developing applications using Large Language Models. They are sponsoring this post. Take a look at the attached video. It will show you how you can fine-tune a simple model using Llama 3.1 without leaving the platform: 1. You can create an empty dataset 2. Use the playground to generate and record interactions with Llama 3.1 3. Modify the dataset directly using the playground 4. Export the data and fine-tune a smaller model Fast and easy! As long as you have a web browser, you can start experimenting with fine-tuning and Llama 3.1. That's all it takes!

You can now try Llama 3.1 405B for free (link below)! This is the largest open-source model out there, and for the first time, an open model is competitive with closed models. This time around, Meta did something new: Llama 3.1 has a license that allows developers to use it to enhance other models. For the first time, you can distill Llama 3.1 405B's capabilities into a smaller, more practical model for your use case. First, here is the link where you can play with Llama 3.1 for free: The model is hosted in Tune Studio, an end-to-end platform for developing applications using Large Language Models. They are sponsoring this post. Take a look at the attached video. It will show you how you can fine-tune a simple model using Llama 3.1 without leaving the platform: 1. You can create an empty dataset 2. Use the playground to generate and record interactions with Llama 3.1 3. Modify the dataset directly using the playground 4. Export the data and fine-tune a smaller model Fast and easy! As long as you have a web browser, you can start experimenting with fine-tuning and Llama 3.1. That's all it takes!

Santiago

55,609 views • 1 year ago

Build your own local ChatGPT-like interface! Powered by Llama 3.2 Vision, it runs 100% locally and supports: ↳Text chat ↳Chat with images Tech stack: ↳ Chainlit for the UI ↳ Ollama for serving Llama 3.2 vision Everything is just 50 lines of code! I have shared link to the code in next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights and tutorials on ML and AI Engineering!

Build your own local ChatGPT-like interface! Powered by Llama 3.2 Vision, it runs 100% locally and supports: ↳Text chat ↳Chat with images Tech stack: ↳ Chainlit for the UI ↳ Ollama for serving Llama 3.2 vision Everything is just 50 lines of code! I have shared link to the code in next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights and tutorials on ML and AI Engineering!

Akshay 🚀

49,952 views • 1 year ago

$Introducing `llama-405b-to-8b` ✍️ Get the quality of Llama 3.1 405B, at a fraction of the cost and latency. Give one example of your task, and 405B will teach 8B (~30x cheaper!!) how to do the task perfectly. And it's open-source:$

Introducing `llama-405b-to-8b` ✍️ Get the quality of Llama 3.1 405B, at a fraction of the cost and latency. Give one example of your task, and 405B will teach 8B (~30x cheaper!!) how to do the task perfectly. And it's open-source:

Matt Shumer

278,384 views • 1 year ago

Build a local LLM app with RAG to chat with Google Drive data using Mistral or Llama-3 running locally on your computer without wiring a single line of Python Code (100% free and without internet):

Build a local LLM app with RAG to chat with Google Drive data using Mistral or Llama-3 running locally on your computer without wiring a single line of Python Code (100% free and without internet):

Shubham Saboo

189,746 views • 1 year ago

DeepSeek-R1. Now running *locally* on your Umbrel. Yes, you can now self-host open source AI models like R1, Llama, and more on your home cloud — thanks to ollama and Open WebUI. Download Open WebUI and Ollama from the Umbrel App Store to get started.

DeepSeek-R1. Now running locally on your Umbrel. Yes, you can now self-host open source AI models like R1, Llama, and more on your home cloud — thanks to ollama and Open WebUI. Download Open WebUI and Ollama from the Umbrel App Store to get started.

Umbrel ☂️

36,562 views • 1 year ago

Today, we're building a Streamlit app to compare Alibaba's latest Qwen 3 against DeepSeek-R1 using RAG. Here's our tool stack: - @Llama_Index workflows for orchestration. - Comet Opik for evaluation. - ollama to serve both LLMs locally. Let's begin!

Today, we're building a Streamlit app to compare Alibaba's latest Qwen 3 against DeepSeek-R1 using RAG. Here's our tool stack: - @Llama_Index workflows for orchestration. - Comet Opik for evaluation. - ollama to serve both LLMs locally. Let's begin!

Avi Chawla

15,473 views • 1 year ago

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

Santiago

97,505 views • 1 year ago

DeepSeek bot | Step five. For fast trading, you need a sniper bot. It’s easier to use the existing ToxiSol - We also need to merge our code and create a guide for running it. ➤ Using my template 👇 "To trade selected tokens, use ToxiSol via TG. Also, add support to receive notifications for buy/sell actions and merge all the code into one."

DeepSeek bot | Step five. For fast trading, you need a sniper bot. It’s easier to use the existing ToxiSol - We also need to merge our code and create a guide for running it. ➤ Using my template 👇 "To trade selected tokens, use ToxiSol via TG. Also, add support to receive notifications for buy/sell actions and merge all the code into one."

DANNY

35,257 views • 1 year ago

people think running AI locally requires: → $3,000 MacBook Pro → RTX 4090 → $20/month cloud subscription nvidia just dropped a $249 computer. 67 TOPS. runs llama 3.1-8B locally. no internet. no API. no monthly fee. ever. smaller than your router. costs the same as AirPods. runs the same models you pay $240/year to access via ChatGPT. the local AI era just got a price tag. $249.

people think running AI locally requires: → $3,000 MacBook Pro → RTX 4090 → $20/month cloud subscription nvidia just dropped a $249 computer. 67 TOPS. runs llama 3.1-8B locally. no internet. no API. no monthly fee. ever. smaller than your router. costs the same as AirPods. runs the same models you pay $240/year to access via ChatGPT. the local AI era just got a price tag. $249.

self.dll

321,125 views • 1 month ago

Blend textures, characters, and objects with Veo 3.1 Ingredients to Video → Pro tip: use Nano Banana Pro in the Gemini app to create your ingredient images, which you can then use to create stunning videos with Veo 3.1 Ingredients to Video.

Blend textures, characters, and objects with Veo 3.1 Ingredients to Video → Pro tip: use Nano Banana Pro in the Gemini app to create your ingredient images, which you can then use to create stunning videos with Veo 3.1 Ingredients to Video.

Google Cloud Tech

22,578 views • 3 months ago

🤯 Wow! In one prompt Qwen3-Coder-Next generated a fully working flappy birds game in HTML. (0:05) Claude Code with Qwen3-Coder-Next (0:26) Shows the game running Run it fully locally: ollama pull qwen3-coder-next Ollama's cloud if you can't run it locally: ollama pull qwen3-coder-next:cloud Try launching it with Claude Code using ollama launch (link to play 🧵) So cool! Qwen Tongyi Lab Junyang Lin

🤯 Wow! In one prompt Qwen3-Coder-Next generated a fully working flappy birds game in HTML. (0:05) Claude Code with Qwen3-Coder-Next (0:26) Shows the game running Run it fully locally: ollama pull qwen3-coder-next Ollama's cloud if you can't run it locally: ollama pull qwen3-coder-next:cloud Try launching it with Claude Code using ollama launch (link to play 🧵) So cool! Qwen Tongyi Lab Junyang Lin

ollama

117,339 views • 4 months ago

My app AI Renamer now supports local AI models. You can easily use Ollama or LM Studio to privately rename all your files locally.

My app AI Renamer now supports local AI models. You can easily use Ollama or LM Studio to privately rename all your files locally.

Ozgur Ozer

17,059 views • 1 year ago