HOW TO RUN AN AI MODEL FROM A USB FLASH DRIVE

The real trick isn't the USB. It's having a model small enough to fit in memory and an inference runtime that can run locally.

True 😅

Keep in mind that while running LLMs off a USB drive works, it’s mostly meant for testing and light everyday tasks. Heavy-duty AI generation will demand a high-end system!

Niiiiice

Try it out

I legit just bought a portable SSD sandisk so I'll be setting this up hopefully this week

It's just for testing tho 😂

nawwwww I'm doing it... and then telling everyone you told me to 🤣 I needed portable storage anyway btw lol

Have fun 😂😂

Why do u need usb flash here?

@TranslateMom arabic

What's the procedure for a Linux usb or what's different in a Linux install

Can it work with vscode

Yes it can, but it completely depends on the model size. Small quantized models (1B–3B) only need ~2–4 GB of RAM and can run in the background while coding. However, I wouldn't advise relying on local setups unless you have a high-end PC. This video is mostly for testing and running offline experiments, competent coding models need significantly more resources to be practical.

not working!

Thanks!

With Arabic captions:

@SagirGurin

is the model free?

@xdownloadercom download this please.

Hmmm

So far it's not unsencored it's just a turn off for me

???

@lyonjacobsfe ...Flash Drive Modem

@GBallarani tiè...

That’s awesome! How can I do this on a Mac OS?

Any good for working with Visual studio or vs code. As direct plugins are very slow

•

You can install snd keep up to data ollama and python via the scoop package manager for Windows and the huggingface cli via pip...

Scoop is great for managing CLI tools on a primary PC. However, it requires an internet connection to install dependencies, which defeats the point of a portable, plug-and-play USB drive designed to run 100% offline on any PC without prior setup

$env:SCOOP_CACHE = "E:\ScoopCache" #drive letter of your usb drive scoop download python ... scoop install python ... mkdir E:\wheels pip download -d E:\wheels hungingface... And Bob's your uncle.

That pip part is a bit fussy maybe $env:PIP_CACHE_DIR = "E:\pip-cache" pip config set global.cache-dir "E:\pip-dir "E:\pip-cache Would be better.

How do I export the conversation to pdf? Can it?

It's running on your browser so Ctrl+ P and save as pdf