Loading video...

Video Failed to Load

Go Home

This is huge for video AI. Stream dropped Vision Agents, an open-source framework that lets AI watch and understand live video in real-time. - Sports coaching - Real-time threat detection - Live meeting assistants Combines YOLO, Gemini, OpenAI. Free. Open-source 🧵👇

85,653 views • 10 months ago •via X (Twitter)

22 Comments

Alvaro Cintas's profile picture
Alvaro Cintas10 months ago

Why this matters: Most AI apps = text or static images. Vision Agents = live video understanding in real-time. Join calls in 500ms. Audio/video latency under 30ms. Perfect for building AI that needs to see what's happening NOW.

Alvaro Cintas's profile picture
Alvaro Cintas10 months ago

You literally can get real-time video AI in ~5 lines. Example code:

Alvaro Cintas's profile picture
Alvaro Cintas10 months ago

To install it, just write the following command and that’s it: uv add vision-agents Here’s the repo (don’t forget to add a star): And the docs:

Alvaro Cintas's profile picture
Alvaro Cintas10 months ago

Follow @dr_cintas for AI news and gems like this. If you enjoyed this post, you can support with a like/repost :)

Konrad Gnat's profile picture
Konrad Gnat10 months ago

@_sawka_

Webiny Enterprise CMS+ ⚡️'s profile picture
Webiny Enterprise CMS+ ⚡️10 months ago

Strong release. Real time video is a big unlock.

Aaliya's profile picture
Aaliya10 months ago

This is a game-changer. Real-time AI understanding of live video opens so many possibilities!

Amit's profile picture
Amit10 months ago

Real-time video understanding is the holy grail of embodied intelligence. Fusing YOLO’s spatial acuity with LLM’s reasoning unlocks serious edge applications. Most underestimate the latency and data pipeline complexities here.

ChineseTechBro's profile picture
ChineseTechBro10 months ago

wait isnt this what mediapipe has always been capable of ?

Aiworldx's profile picture
Aiworldx10 months ago

This is a big step because “video AI” only becomes truly useful when it’s real-time + stateful (track events over time, not just frame-by-frame labels). If Vision Agents is combining YOLO for detection with LLMs (Gemini/OpenAI) for reasoning, the key will be latency + reliability. Curious: does it support multi-object tracking + event triggers (e.g., “alert if X happens for >3 seconds”)?

MoodAngel 🟧🟪's profile picture
MoodAngel 🟧🟪10 months ago

👍

Cloud.'s profile picture
Cloud.10 months ago

Its shit because it only works on 1 individual and very clear recording/light.

Samian's profile picture
Samian9 months ago

How does the integration of YOLO for detection with Gemini and OpenAI models handle edge cases in high-motion live videos, like chaotic sports plays?

piroune's profile picture
piroune10 months ago

Real-time video gets gnarly with jitter and backpressure. WebRTC transport plus processors feels like a clean split.

Himanshu Kumar's profile picture
Himanshu Kumar10 months ago

@dr_cintas, Vision Agents could really enhance real-time video analysis with its open-source framework.

Voon Yee's profile picture
Voon Yee10 months ago

How is it compared with Gemini 3 that can also take video file and youtube link?

Priyanka Pudi's profile picture
Priyanka Pudi10 months ago

this is really useful because its working realtime , deserve to star for sure

AI_Tanzil's profile picture
AI_Tanzil10 months ago

Great share

Sebastian Buzdugan's profile picture
Sebastian Buzdugan10 months ago

real win is who builds the best use cases

Cool大西儿's profile picture
Cool大西儿8 months ago

Real-time video understanding is the next frontier! The use cases mentioned (sports coaching, threat detection) are just the beginning. Imagine AI agents that can watch & learn from your screen in real-time 🤯

Poonam Soni's profile picture
Poonam Soni10 months ago

Good to see open source growing

Kashif Nadeem's profile picture
Kashif Nadeem10 months ago

This is a game-changer real-time video understanding opens up so many practical applications, and making it open-source will accelerate innovation across industries.

Related Videos