正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

This is all the code you need to create a Streamlit app to chat with Llama 3.1 8B locally via ollama. Fast and private.

Pietro Schirano

104,447 subscribers

310,018 次观看 • 2 年前 •via X (Twitter)

科学技术教育艺术

Anya Rossi• Live Now

Private livecam show

10 条评论

Pietro Schirano 的头像

Pietro Schirano2 年前

Here it is 👇

topmass 的头像

topmass2 年前

@ollama there is a really cool chrome extension that runs all your ollama models locally too - no fluff just an extension that opens a chat where you select one of your already downloaded models and you can chat

Tom Wildflower 的头像

Tom Wildflower2 年前

@ollama I thought I was excited about the things I could build the last few weeks/months. I'm over the moon now, I can't wait to just be able to create. My brick and mortar businesses are legitimately on the chopping block. Lol

Sebastian BO 的头像

Sebastian BO2 年前

@ollama llama-engineer...

Shadow Defense 的头像

Shadow Defense2 年前

@ollama What is the easiest way to do this locally because I don't really know much about GitHub and Python and all this other stuff I just want to open it and it work like check GPT does what's the easiest method so that it can have unlimited chat

Lucky 的头像

Lucky2 年前

@ollama i trade u prompt for screenshot I took

Inka Hacker (sun/acc) 的头像

Inka Hacker (sun/acc)2 年前

@ollama Runs locally?

Pietro Schirano 的头像

Pietro Schirano2 年前

@ollama Yeah, the 8B version quantized is around 5GB.

someone 的头像

someone2 年前

@ollama can you please make changes in claude engineer so that it supports ollama?

S.D.Gwynn 的头像

S.D.Gwynn2 年前

@ollama Nice. Is there a way to use Claude’s artifacts and projects setup locally?

相关视频

Running Llama 3.1 locally as a code assistant in VSCode with ollama ✨ Download Ollama ( ✨ Download CodeGPT ( ✨ Select the Llama 3.1 model (8b, 70b or 405b) ✨ Click download ✨ Enjoy! :)

Running Llama 3.1 locally as a code assistant in VSCode with ollama ✨ Download Ollama ( ✨ Download CodeGPT ( ✨ Select the Llama 3.1 model (8b, 70b or 405b) ✨ Click download ✨ Enjoy! :)

Daniel San

359,980 次观看 • 2 年前

Fully local RAG with Llama 3 on ollama & Streamlit Learn how to: 🦙Run Llama 3 using ollama 📄Add Website & PDFs to a knowledge base 💻Build an AI App using Streamlit 🏬Store chat history in a database PostgreSQL YT:

Fully local RAG with Llama 3 on ollama & Streamlit Learn how to: 🦙Run Llama 3 using ollama 📄Add Website & PDFs to a knowledge base 💻Build an AI App using Streamlit 🏬Store chat history in a database PostgreSQL YT:

Ashpreet Bedi

70,356 次观看 • 2 年前

GLM 4.6 in Claude Code to build a python app Took me around 3 mins to make this python app with a Streamlit web ui and llamacpp python bindings The app lets you run an inference via a simple chat interface plus configurable params 👇 (video 4x speed)

GLM 4.6 in Claude Code to build a python app Took me around 3 mins to make this python app with a Streamlit web ui and llamacpp python bindings The app lets you run an inference via a simple chat interface plus configurable params 👇 (video 4x speed)

Melvin Vivas

21,803 次观看 • 10 个月前

To illustrate the type of tech we now have access to, I built a Streamlit app that uses GroqInc instant LLama 3.1 8B and new 2.5 turbo model. We now have cheap, instant voice AI. ⚡️ Imagine the amazing things you can build. But I'll let Richard say more.

To illustrate the type of tech we now have access to, I built a Streamlit app that uses GroqInc instant LLama 3.1 8B and new 2.5 turbo model. We now have cheap, instant voice AI. ⚡️ Imagine the amazing things you can build. But I'll let Richard say more.

Pietro Schirano

74,626 次观看 • 2 年前

Drag & Drop to build local chat app with memory using Llama 3.1 without writing a single line of code (100% free and without internet).

Drag & Drop to build local chat app with memory using Llama 3.1 without writing a single line of code (100% free and without internet).

Shubham Saboo

12,446 次观看 • 2 年前

Here's a mini-ChatGPT app that runs locally on your computer. You can chat with it just like you would chat with ChatGPT. We use: - DeepSeek R1 as the LLM - ollama to locally serve R1 - Chainlit for the UI Let's build it!

Here's a mini-ChatGPT app that runs locally on your computer. You can chat with it just like you would chat with ChatGPT. We use: - DeepSeek R1 as the LLM - ollama to locally serve R1 - Chainlit for the UI Let's build it!

Avi Chawla

40,282 次观看 • 1 年前

$Fuck it! You can now run *any* GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡$

Fuck it! You can now run any GGUF on the Hugging Face Hub directly with ollama 🔥 This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub* *Without any changes whatsoever! ⚡ All you need to do is: ollama run hf. co/{username}/{reponame}:latest For example, to run the Llama 3.2 1B, you can run: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest If you want to run a specific quant, all you need to do is specify the Quant type: ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0 That's it! We'll work closely with Ollama to continue developing this further! ⚡

Vaibhav (VB) Srivastav

317,790 次观看 • 1 年前

Running Qwen2.5-Coder (0.5B params) locally on a Mac Intel Core i5 with CodeGPT and Ollama 🤩 In seconds, it explains Python code... all private and secure! Download CodeGPT from the VSCode Marketplace! 👏 Qwen ollama CodeGPT

Running Qwen2.5-Coder (0.5B params) locally on a Mac Intel Core i5 with CodeGPT and Ollama 🤩 In seconds, it explains Python code... all private and secure! Download CodeGPT from the VSCode Marketplace! 👏 Qwen ollama CodeGPT

Daniel San

30,613 次观看 • 1 年前

I just created my own OCR app using Llama 3.2 vision! Upload an image, and it converts it into structured markdown using Llama 3.2 multimodal! Here's what I used: - Ollama for serving Llama 3.2 vision locally. - Streamlit for the UI. Everything in just 50 lines of code! Find the code in the next tweet. -- Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. Find me → Avi Chawla

I just created my own OCR app using Llama 3.2 vision! Upload an image, and it converts it into structured markdown using Llama 3.2 multimodal! Here's what I used: - Ollama for serving Llama 3.2 vision locally. - Streamlit for the UI. Everything in just 50 lines of code! Find the code in the next tweet. -- Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs. Find me → Avi Chawla

Avi Chawla

131,139 次观看 • 1 年前

You can now try Llama 3.1 405B for free (link below)! This is the largest open-source model out there, and for the first time, an open model is competitive with closed models. This time around, Meta did something new: Llama 3.1 has a license that allows developers to use it to enhance other models. For the first time, you can distill Llama 3.1 405B's capabilities into a smaller, more practical model for your use case. First, here is the link where you can play with Llama 3.1 for free: The model is hosted in Tune Studio, an end-to-end platform for developing applications using Large Language Models. They are sponsoring this post. Take a look at the attached video. It will show you how you can fine-tune a simple model using Llama 3.1 without leaving the platform: 1. You can create an empty dataset 2. Use the playground to generate and record interactions with Llama 3.1 3. Modify the dataset directly using the playground 4. Export the data and fine-tune a smaller model Fast and easy! As long as you have a web browser, you can start experimenting with fine-tuning and Llama 3.1. That's all it takes!

You can now try Llama 3.1 405B for free (link below)! This is the largest open-source model out there, and for the first time, an open model is competitive with closed models. This time around, Meta did something new: Llama 3.1 has a license that allows developers to use it to enhance other models. For the first time, you can distill Llama 3.1 405B's capabilities into a smaller, more practical model for your use case. First, here is the link where you can play with Llama 3.1 for free: The model is hosted in Tune Studio, an end-to-end platform for developing applications using Large Language Models. They are sponsoring this post. Take a look at the attached video. It will show you how you can fine-tune a simple model using Llama 3.1 without leaving the platform: 1. You can create an empty dataset 2. Use the playground to generate and record interactions with Llama 3.1 3. Modify the dataset directly using the playground 4. Export the data and fine-tune a smaller model Fast and easy! As long as you have a web browser, you can start experimenting with fine-tuning and Llama 3.1. That's all it takes!

Santiago

55,609 次观看 • 2 年前

$Introducing `llama-405b-to-8b` ✍️ Get the quality of Llama 3.1 405B, at a fraction of the cost and latency. Give one example of your task, and 405B will teach 8B (~30x cheaper!!) how to do the task perfectly. And it's open-source:$

Introducing `llama-405b-to-8b` ✍️ Get the quality of Llama 3.1 405B, at a fraction of the cost and latency. Give one example of your task, and 405B will teach 8B (~30x cheaper!!) how to do the task perfectly. And it's open-source:

Matt Shumer

278,444 次观看 • 2 年前

Build a local LLM app with RAG to chat with Google Drive data using Mistral or Llama-3 running locally on your computer without wiring a single line of Python Code (100% free and without internet):

Build a local LLM app with RAG to chat with Google Drive data using Mistral or Llama-3 running locally on your computer without wiring a single line of Python Code (100% free and without internet):

Shubham Saboo

189,778 次观看 • 2 年前

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

Santiago

97,561 次观看 • 1 年前

people think running AI locally requires: → $3,000 MacBook Pro → RTX 4090 → $20/month cloud subscription nvidia just dropped a $249 computer. 67 TOPS. runs llama 3.1-8B locally. no internet. no API. no monthly fee. ever. smaller than your router. costs the same as AirPods. runs the same models you pay $240/year to access via ChatGPT. the local AI era just got a price tag. $249.

people think running AI locally requires: → $3,000 MacBook Pro → RTX 4090 → $20/month cloud subscription nvidia just dropped a $249 computer. 67 TOPS. runs llama 3.1-8B locally. no internet. no API. no monthly fee. ever. smaller than your router. costs the same as AirPods. runs the same models you pay $240/year to access via ChatGPT. the local AI era just got a price tag. $249.

self.dll

321,624 次观看 • 4 个月前

DeepSeek bot | Step five. For fast trading, you need a sniper bot. It’s easier to use the existing ToxiSol - We also need to merge our code and create a guide for running it. ➤ Using my template 👇 "To trade selected tokens, use ToxiSol via TG. Also, add support to receive notifications for buy/sell actions and merge all the code into one."

DeepSeek bot | Step five. For fast trading, you need a sniper bot. It’s easier to use the existing ToxiSol - We also need to merge our code and create a guide for running it. ➤ Using my template 👇 "To trade selected tokens, use ToxiSol via TG. Also, add support to receive notifications for buy/sell actions and merge all the code into one."

DANNY

35,258 次观看 • 1 年前