Meet hunch! Jev-like decisions in ~20ms without even needing a GPU, learning live from a teacher model. Built on @GoogleGemma's EmbeddingGemma 2, it can works with any teacher model like @CloudFlare's Clef. Multimodal, w/a systemone endpoint, slots into your existing stack 🧵

hunch was built as part of my work with undrgrnd, a stealth continual-learning startup. We needed to scan huge volumes of private data so we made it happen with hunch It turned out to be a powerful primitive so we open-sourced it Give us a ⭐️ on Github!

We'd lock to hear what sort of things you want to build with hunch. Give it a shot today and let us know below what you're going to try first. We also have a live demo that uses your own GPU!

@googlegemma @Cloudflare posting for posterity that I am the 17th star! I feel like this is gonna go viral. Gonna play around with this later to see if I can replace my usage of jev in my personal systems!

@googlegemma @Cloudflare Super-cool. We really hope it can take a good chunk of your existing decisions. It's really awesome for Local AI to keep the traffic to the more expensive models down.

@googlegemma @Cloudflare This is the coolest thing I've seen in a while, thank you for releasing it!

@googlegemma @Cloudflare Glad to hear you like it.. let us know what you'll do with it! Super curious to see what the community builds

@googlegemma @Cloudflare It's perfect timing as I had just vibe-built a small labelling tool over the weekend to help me explore how quickly I can go from data to pre-labels by LLM to manual verification to training a small LR on top of it. Your tool definitely pushes the boundary on "how quickly."

@googlegemma @Cloudflare oh man, that's in the sweet spot for hunch for sure. Embeddings are overlooked, but adding a systemone-style endpoint on top makes them so much easier to work with. Please share your project if you open-source it, sounds awesome (DM it to me, replies are overwhelming, hah)

@googlegemma @Cloudflare This project should win an internet award. I absolutely love everything about it. You know what? @grok make up an award and present it to these absolute legends.

@googlegemma @Cloudflare Very interesting. Inspired me to look into EmbeddingGemma2. I have this "hunch" that the most generalizable robotic controller is Codex/Claude Code as the planner, writing questions and context for a system1 in a loop that constantly reads sensor data and makes control choices.

@googlegemma @Cloudflare Take a look at eg2cpu in the repo! Happy to take patches to add SIMD support to whatever platform you'd like to run it on. It's probably the fastest way to run EG2 right now.

@googlegemma @Cloudflare I'd try it in my CRM first. DeepSeek reads the call transcripts there, and hunch could learn the routine questions from it and cut most of those API calls.

@googlegemma @Cloudflare I have a hunch this is gonna be great

@googlegemma @Cloudflare This is the way.

@googlegemma @Cloudflare Congrats on the launch. Excited to play with this.

@googlegemma @Cloudflare ty! Would love to hear your ideas on what you'd build

@googlegemma @Cloudflare Super pumped to try this!

@googlegemma @Cloudflare Fucking awesome

@googlegemma @Cloudflare Mind-blowing demo Quick question on scaling: what are the limits in terms of class count and granularity? Could this scale to hundreds/thousands of classes for retail/self-checkout use cases (e.g. cross-checking what a camera sees vs what barcode was scanned for loss prevention)?

@googlegemma @Cloudflare Super cool. Thank you. Will try this out in our compliance workflows

@googlegemma @Cloudflare Very interested to hear your results from this.

@googlegemma @Cloudflare It’s really cool to see more people building decision models. Am currently building one to answer questions on timeseries.

@googlegemma @Cloudflare Thanks! Already shared this with my son, believe it will drastically improve a project he has been working on! So many cool things this can be added to as well !

@googlegemma @Cloudflare 🔥 coming out great bro! Good work!

@googlegemma @Cloudflare ty... What a week! Happy to have this out here to start building fun stuff on.

@googlegemma @Cloudflare Very welcome, I can definitely see this expanding into classrooms 🫡

@googlegemma @Cloudflare Star #47 here! 😆🌰 I'm working on visual analysis for indie games, and hunch looks really promising for trailer-frame classification. Love the teacher/student approach. Excited to experiment with it! Thanks for open-sourcing this! 🚀

@googlegemma @Cloudflare I think that's a perfect usecase for it... let me know when you get results!

@googlegemma @Cloudflare 20ms on a cpu means it can ride inside the app itself.

@googlegemma @Cloudflare starred, excited to try this!

@googlegemma @Cloudflare I tried the same thing today for the industrial production system. Results were not good enough. It’s learning, the process works well, but in the 1000 examples it labels 813 correctly.

@googlegemma @Cloudflare I'd be interested to see if the teacher/student model in hunch works. It tries to fit on which questions it can't answer so the teacher stays in the mix. Would be happy to chat more via DM and see if there's improvements we can make (or file an issue!)

@googlegemma @Cloudflare Perfect, definetly will. Thx

@googlegemma @Cloudflare The checks after graduation are the detail that interests me. A classifier taking over routine calls still has a way back to its teacher when the data starts looking unfamiliar.

@googlegemma @Cloudflare The teacher/student setup is interesting for repetitive data decisions. How do you decide when Hunch should abstain and ask the teacher again, especially after the input distribution shifts?

@googlegemma @Cloudflare System 1 models are already super cheap and have good throughput. Whats the cost performance tradeoff here when using a lighter model here? I'm failing to understand use cases where this would be useful

@googlegemma @Cloudflare It's local friendly, optimized for cpu (no GPU required), hunch can outperform embeddings and decision models on their own. It's significantly faster on the questions it does answer. Cost of an AWS graviton cpu instance is super cheap vs a GPU one.

@googlegemma @Cloudflare Endpoint? Is there a hosted api?

@googlegemma @Cloudflare This is all self hosted for now - it runs on anything with a CPU (no GPU required). You can use any upstream systemone model for the teacher, or even a local one.