open sourcing Marlin-2B 🐟 a tiny VLM to extract structured information from videos Marlin is finetuned for two questions devs want to ask in their videos: what is happening, and when? Best open model in its weight class, competitive with Gemini-2.5-flash at only 2B params 🧵

Marlin was trained on two modes: 1. marlin.caption() returns a structured Scene + Events JSON with second-precise timestamps. You can use it to caption ig reels, index a video library or give your agent context of what happened and when in a video feed

2. marlin.find() returns (start, end) timestamps for any natural-language query over a video. fast enough to run inline in an agent loop, you can use it to locate sub-second moments in videos

strongest open model in its weight class on dense captioning (DREAM-1K, CaReBench), beating Tarsier-2 7B/34B and competitive with Gemini-2.5-flash. On temporal grounding (TimeLens-Bench) it sits a few points behind TimeLens-8B and close to Gemini-2.5-pro.

Back story: @rethinkNow348 and I were building video editing agents and using Gemini-2.5-Flash to parse ig reels for events. Kept hitting content filters on benign clips so we put a month and two H100s into this side quest and this is the result :)

🤗 Model: 🎮 Hosted Demo: Training recipe + a new dense-captioning/grounding benchmark dropping soon!!

Update for everyone tracking this: Marlin-2B has been relicensed to Apache 2.0. The BSL 1.1 LICENSE is replaced with standard Apache 2.0 text. NOTICE preserves attribution to Qwen3.5-2B-Base. HF metadata now reports license:apache-2.0. The license was a mistake — fair critique, now fixed. Truly open source 🙏

@_raghuvamsi Could be cool for your skill @LouiseDSadeleer

waiting for the fine tuning scripts to be dropped

this is amazing guys! will try to port it to mlx/coreml

crazy

pretty cool ill play with it and get back to you!

Have fun! Hit me up with any feedback you have 😊

Reminds me of the distilled LLM wave. Everyone chased massive context windows while tiny models ate their lunch. Strip the bloat, sharpen the task. Just hope those timestamps don't hallucinate.

We hope yes :) We also distilled gemini3-flash for our timestamp grounded data but used a couple of filter layers and CoT tricks to accurately label these videos.

the temporal grounding should let agents timestamp events in video feeds without extra parsing layers after inference.

brewed fresh outta @babayagalabs 🚀 insane!

2B params competing with Gemini Flash… small fish eating the big pond 🐟 open source really said hold my beer

Can the model understand and analyze basketball game videos?

We had some sports highlights into our data mixture but not sure how accurate would be, do try and hit me up if you face any difficulties!!

Tried the demo, it's not working.

We’ve only hosted inference for videos <2mins. Are your videos longer?

I mean, I successfully ran the demo, but it didn't work for the event of finding a goal in the basketball video.

nice one bro

Competitive with Gemini 2.5 Flash at 2B params is a serious claim. That weight class just got a lot more interesting

open sourcing is the way to go. letting others build on it can lead to unexpected innovations.

Looks good man, gg

Fine-tuning put to good use

crazyyy work! congratulations @HappyyPablo on this really amazing work!

cant wait to use this to tag all the 800 videos in my memes folder and be able to find any video on keywords such as "robot dies on stage"

crazy, huge unlock for robotic stuff. Even stuff as basic as vacuum robots being smarter with what to clean, just imagine

@acgfbr

It's so cool I would definitely use it for any of the projects I'm going to do with videos!!! @utk7arsh Seamless might be cheaper to run 😆

looks crazy bro 🚀

Very nice. I have my agent take screenshots and use Whisper to analyze video for editing, but a mini VLM could be handy to use instead.

this is really cool 😎 congrats and thank you for sharing 🙏

Impressive write the blogs ASAP!!

Marlin-2B looks like a huge win for local inference. Getting temporal grounding and dense captions at only 2B parameters is serious stuff. If it really competes with Gemini-2.5-Flash, I can finally run heavy video analysis on my own hardware without those massive API bills.

Wow this is incredibly useful!

I used it on my local device and it is more than amazing. Excited to see more updates on this.

Hey Shubham, would love to connect and distribute it across my audiences. The audience that I have is pretty much aligned on the same lines, and they would love to use the model, happy to connect and discuss! Dmed you

interesting - will try it out again tom

@llmgram

Looks great. I would love to run it in the background on all my family videos - new ones and ones I've collected over the years, then have an agent continuously create little story arcs, and auto-edit together mini-movies from our lives.

Pretty cool, when that event happened is so fucking difficult for the VLM models to answer directly rn. I'll play around and see if we can actually replace gemini in our workflows with this, would save us crazy money.

Do try and let me know brother! You guys have a solid product 🙌🏻🙌🏻

Cool one . Thanks for sharing Shubham Bravo to the team ! 👏