This is huge for video AI. Stream dropped Vision Agents, an open-source framework that lets AI watch and understand live video in real-time. - Sports coaching - Real-time threat detection - Live meeting assistants Combines YOLO, Gemini, OpenAI. Free. Open-source 🧵👇

Why this matters: Most AI apps = text or static images. Vision Agents = live video understanding in real-time. Join calls in 500ms. Audio/video latency under 30ms. Perfect for building AI that needs to see what's happening NOW.

You literally can get real-time video AI in ~5 lines. Example code:

To install it, just write the following command and that’s it: uv add vision-agents Here’s the repo (don’t forget to add a star): And the docs:

Follow @dr_cintas for AI news and gems like this. If you enjoyed this post, you can support with a like/repost :)

@_sawka_

Strong release. Real time video is a big unlock.

This is a game-changer. Real-time AI understanding of live video opens so many possibilities!

Real-time video understanding is the holy grail of embodied intelligence. Fusing YOLO’s spatial acuity with LLM’s reasoning unlocks serious edge applications. Most underestimate the latency and data pipeline complexities here.

wait isnt this what mediapipe has always been capable of ?

This is a big step because “video AI” only becomes truly useful when it’s real-time + stateful (track events over time, not just frame-by-frame labels). If Vision Agents is combining YOLO for detection with LLMs (Gemini/OpenAI) for reasoning, the key will be latency + reliability. Curious: does it support multi-object tracking + event triggers (e.g., “alert if X happens for >3 seconds”)?

👍

Its shit because it only works on 1 individual and very clear recording/light.

How does the integration of YOLO for detection with Gemini and OpenAI models handle edge cases in high-motion live videos, like chaotic sports plays?

Real-time video gets gnarly with jitter and backpressure. WebRTC transport plus processors feels like a clean split.

@dr_cintas, Vision Agents could really enhance real-time video analysis with its open-source framework.

How is it compared with Gemini 3 that can also take video file and youtube link?

this is really useful because its working realtime , deserve to star for sure

Great share

real win is who builds the best use cases

Real-time video understanding is the next frontier! The use cases mentioned (sports coaching, threat detection) are just the beginning. Imagine AI agents that can watch & learn from your screen in real-time 🤯

Good to see open source growing

This is a game-changer real-time video understanding opens up so many practical applications, and making it open-source will accelerate innovation across industries.