🚀 Watch how I connected Chrome's AI API to ollama, enabling ANY web app built for Chrome's local Gemini to seamlessly work with open-source LLMs! A🧵on making AI in Chrome more open and flexible...

Chrome 127+ introduced an experimental Prompt API for local AI experimentation using Gemini Nano. @googlechrome & @ChromiumDev are pushing boundaries with built-in AI capabilities.

Why is this exciting?🔥The Prompt API lets developers send natural language instructions to locally running models, accelerating AI use-case discovery and informing future task-specific APIs.

Shoutout to @huggingface for their amazing tutorial on running Gemini Nano locally:

But here's where it gets interesting...🎯What if we could use this same API with ANY model? That's where @ollama comes in!

I modified Chrome's AI implementation to talk to Ollama instead of Gemini Nano. The result? Any web app built for Chrome's AI API just works with Ollama's models! 🪄

Why is this important? 🤔 - Run any Ollama model locally - Keep your data private - Choose your own models - Zero changes needed for web apps

Big thanks to @ollama for their amazing work making local LLMs accessible. Their API design made this integration surprisingly straightforward! 🙏

Speaking of local AI champions... @exolabs have been a huge inspirations in the "AI at home" movement. Their work on making AI accessible on personal devices is groundbreaking.

- quick question: Should we expand this POC to work with an Exo cluster? The possibilities for distributed local AI in Chrome seem exciting!🤔

How it started😄

As many as 80% of AI projects fail, wasting time and money. Download your free guide to thoughtful ideation, development, and implementation of AI solutions today.