Multimodality and streaming is hard. I've been building something that allows you connect streaming devices like screen capture, microphone, camera, and text easily to craft generative streaming pipelines. It works well with Gemini 2.0 models. Happy to open source if anyone wants it.

It allows building pipelines with a little bit of configuration. It's super easy to quickly see what the models are capable of given a mixture of different modalities and context including custom components that can augment the context.

I’d like to use that

please do!

Really interested in something like this

👀

eggcellent! I'd like to try it.

Yes please!🙏

Very cool! I built a similar thing in python a couple months ago before all the realtime stuff came around. It's already open-source here:

+1 👆

Yes, please.