正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Introducing ImageJevBench 🥇Jev-Omni 🥈decider-2b-vision 🥉Reflex 4B The video shows 89 public synthetic images. Running the top 5 models on all these images cost $0.0217. You understand how disruptive this is?

Florian S

6,530 subscribers

30,027 次观看 • 16 天前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

32 条评论

Yohei 的头像

Yohei16 天前

put me in coach

Florian S 的头像

Florian S16 天前

oh yes!!! @benchmarkheaven put him into the benchmark: glance-vlm speedlab

Yohei 的头像

Yohei16 天前

@benchmarkheaven did you set up an agent you can trigger from x to add models to the bench?

Florian S 的头像

Florian S16 天前

@benchmarkheaven yes

Yohei 的头像

Yohei16 天前

@benchmarkheaven awesome

Florian S 的头像

Florian S16 天前

@benchmarkheaven He's called Harold, all my agents know him. I can just tell Claude or Devin or Codex to let Harold do it. He has a lovely personality, he's an accountant. The right guy for a benchmark collection I guess.

Yohei 的头像

Yohei16 天前

@benchmarkheaven looking forward to seeing the results! are you able to monitor what’s going etc as he adds new models?

Florian S 的头像

Florian S16 天前

@benchmarkheaven Yes, sure. I have review gates etc

Florian S 的头像

Florian S16 天前

Here are some examples of typical tasks that image Jev-class models can solve. A response typically takes less time than the blink of an eye and costs almost nothing. This unlocks a wealth of use cases that were previously uneconomical to solve. Think industrial automation, robotics, IoT, and document management and so much more.

Toran Bruce Richards 的头像

Toran Bruce Richards16 天前

Dude you are cooking recently! 🧑‍🍳

Florian S 的头像

Florian S16 天前

Thanks dude. Just starting. Will cook way more. 😄

Toran Bruce Richards 的头像

Toran Bruce Richards16 天前

I follow with great interest.

Florian S 的头像

Florian S16 天前

🙏

Midas 👑 的头像

Midas 👑16 天前

jev omni? when did that launch?!

akhila_datta 的头像

akhila_datta16 天前

Great work !

Florian S 的头像

Florian S16 天前

Thanks

Robin | Poker x AI 的头像

Robin | Poker x AI16 天前

Running five top models for under three cents on 89 images is insane, especially with sub millisecond latency

Florian S 的头像

Florian S16 天前

yes, it's incredible, isn't it? just think about the new business ideas this will unlock

Mo's fav goat 的头像

Mo's fav goat16 天前

Jev alternative when using images tool to check out later

Cuth 的头像

Cuth16 天前

Gonna use decider 2b vision at work

Laurits Wehding 的头像

Laurits Wehding16 天前

crazy results

Florian S 的头像

Florian S16 天前

Yes

Automater 的头像

Automater16 天前

Cheap vision evals change the operating question from ‘can it see’ to ‘when does it fail under distribution shift.’ A public image set with a cost line is how operators budget verification.

ʟɪsᴀ 的头像

ʟɪsᴀ16 天前

this is exactly what i was looking for - now if you can please host a routing platform that would be great

Florian S 的头像

Florian S16 天前

coming soon

Alex Volkov 的头像

Alex Volkov16 天前

@skalskip92 you need to see this

Curi Ator 的头像

Curi Ator16 天前

Amazing!

Florian S 的头像

Florian S16 天前

💯

Evia AI 的头像

Evia AI16 天前

$0.0217 to run top 5 across 89 images is wild efficiency

Alek 的头像

Alek16 天前

Florian - does the $0.0217 hold past 89 images?

Flavius Burca 的头像

Flavius Burca16 天前

how can new models be submitted ?

Florian S 的头像

Florian S16 天前

write me or @benchmarkheaven a DM or just post a link to your model here asking for being added. I should maybe add a submission section on the JevBench page as well. @benchmarkheaven, please build this.

相关视频

Video generation is now live on OpenRouter! One API gives you access to the top video models, right alongside your text, images, audio, embeddings, and reranker generations. Watch our human-generated launch video:

Video generation is now live on OpenRouter! One API gives you access to the top video models, right alongside your text, images, audio, embeddings, and reranker generations. Watch our human-generated launch video:

OpenRouter

119,974 次观看 • 5 个月前

Parameter scaling just broke in front of me. I gave 90 math problems as images to 10 local vision models, 3 runs each, only consistent answers across all 3 runs counted. Two findings: > Gemma 4 was the most consistent family, 31B took the crown at 89.6%. > But Qwen 3.5 4B came 2 answers behind it. A 4B. Ranked 2nd out of 10. Vision-math isn't one skill, it's two: reading the image, then solving it. The real lesson for anyone running local: small ≠ weak. If you're building agentic workflows, finding the right model for each task matters more than reaching for the biggest one. In this test, the 4B ran far faster because of its size, scored higher, and left VRAM free for the rest of your stack. Full results: 🥇 Gemma 4 31B — 242/270 (89.6%) 🥈 Qwen 3.5 4B — 240/270 (88.9%) 🥉 Gemma 4 E4B — 222/270 (82.2%) 🥉 Qwen 3.6 27B — 222/270 (82.2%) 5. Gemma 4 26B A4B — 216/270 (80.0%) 6. Qwen 3.5 2B — 201/270 (74.4%) 7. Gemma 4 E2B — 192/270 (71.1%) 8. Qwen 3.6 35B A3B — 192/270 (71.1%) 9. Qwen 3.5 9B — 168/270 (62.2%) 10. Qwen 3.5 0.8B — 45/270 (16.7%) All GGUF + mmproj, Unsloth's Q6_K_XL quant.

Parameter scaling just broke in front of me. I gave 90 math problems as images to 10 local vision models, 3 runs each, only consistent answers across all 3 runs counted. Two findings: > Gemma 4 was the most consistent family, 31B took the crown at 89.6%. > But Qwen 3.5 4B came 2 answers behind it. A 4B. Ranked 2nd out of 10. Vision-math isn't one skill, it's two: reading the image, then solving it. The real lesson for anyone running local: small ≠ weak. If you're building agentic workflows, finding the right model for each task matters more than reaching for the biggest one. In this test, the 4B ran far faster because of its size, scored higher, and left VRAM free for the rest of your stack. Full results: 🥇 Gemma 4 31B — 242/270 (89.6%) 🥈 Qwen 3.5 4B — 240/270 (88.9%) 🥉 Gemma 4 E4B — 222/270 (82.2%) 🥉 Qwen 3.6 27B — 222/270 (82.2%) 5. Gemma 4 26B A4B — 216/270 (80.0%) 6. Qwen 3.5 2B — 201/270 (74.4%) 7. Gemma 4 E2B — 192/270 (71.1%) 8. Qwen 3.6 35B A3B — 192/270 (71.1%) 9. Qwen 3.5 9B — 168/270 (62.2%) 10. Qwen 3.5 0.8B — 45/270 (16.7%) All GGUF + mmproj, Unsloth's Q6_K_XL quant.

stevibe

22,882 次观看 • 4 个月前

The NEW and last Goku Mid Update before the public release is OUT NOW on Patreon. This Update adds all the new costume images slots made by Fastfour XF as you can see in this video, Thank you and great work! 👏

The NEW and last Goku Mid Update before the public release is OUT NOW on Patreon. This Update adds all the new costume images slots made by Fastfour XF as you can see in this video, Thank you and great work! 👏

Millennium

17,102 次观看 • 1 年前

$100 SURVIVAL CHALLENGE: TOP 10 ANNOUNCED 📣 Well done, all. Every challenger is a winner, ranking or not. We carefully reviewed all challenger records before naming the Top 10. 🥇 Zayn · +100.52% · $2,000 🥈 泵泵超人 | Pumpman 🔶 · +81.07% · $1,000 🥉 比特欧 Elvin · +68.83% · $500 The video shows all 10. The thread has more. 🧵

$100 SURVIVAL CHALLENGE: TOP 10 ANNOUNCED 📣 Well done, all. Every challenger is a winner, ranking or not. We carefully reviewed all challenger records before naming the Top 10. 🥇 Zayn · +100.52% · $2,000 🥈 泵泵超人 | Pumpman 🔶 · +81.07% · $1,000 🥉 比特欧 Elvin · +68.83% · $500 The video shows all 10. The thread has more. 🧵

HertzFlow | Mainnet Arc🎲

29,650 次观看 • 25 天前

The last time I shared my giant and tiny human/animals series they went viral right here on X. Here is the link to the tutorial - I've put together a more in-depth guide explaining the process from image generation to bringing the images to life with Veo 3.1 in this video. These videos are so much fun to create! Once you understand how to create the images, it's up to you, you can prompt different camera angles, connect them and tell a story. Enjoy!

The last time I shared my giant and tiny human/animals series they went viral right here on X. Here is the link to the tutorial - I've put together a more in-depth guide explaining the process from image generation to bringing the images to life with Veo 3.1 in this video. These videos are so much fun to create! Once you understand how to create the images, it's up to you, you can prompt different camera angles, connect them and tell a story. Enjoy!

Travis Davids

16,149 次观看 • 8 个月前

This video is really weird. Not only is there strange ghosting and double images that seems to hide certain people unless you freeze frame it, there's also other important factors. Terryl Farnsworth is up on the wall for at least 2mins, James Rawlinson escaped through the tunnel, Andrew Sypher and Anthony West are both seemingly calm and Chris the mystery security guy comes running past all ghosted out and nearly invisible. Was this video messed with? What's with the laggy images?

This video is really weird. Not only is there strange ghosting and double images that seems to hide certain people unless you freeze frame it, there's also other important factors. Terryl Farnsworth is up on the wall for at least 2mins, James Rawlinson escaped through the tunnel, Andrew Sypher and Anthony West are both seemingly calm and Chris the mystery security guy comes running past all ghosted out and nearly invisible. Was this video messed with? What's with the laggy images?

Muppet Masher

15,346 次观看 • 20 天前

UPDATE: Charlie Kirk Someone did a walk through of UVU showing all the cameras… the one I’m most interested in is the stairwell within the first few minutes. A perfectly logical question… The FBI released STILL IMAGES of this person. Why not the video and still images? The same can be applied to the footage of the suspect jumping off the roof. The video clearly shows the shooting location, but the video only starts 20-30 seconds after when running. 🎥 Compiled by HustleBitch

UPDATE: Charlie Kirk Someone did a walk through of UVU showing all the cameras… the one I’m most interested in is the stairwell within the first few minutes. A perfectly logical question… The FBI released STILL IMAGES of this person. Why not the video and still images? The same can be applied to the footage of the suspect jumping off the roof. The video clearly shows the shooting location, but the video only starts 20-30 seconds after when running. 🎥 Compiled by HustleBitch

MJTruthUltra

30,165 次观看 • 1 年前

$We’re shipping two major updates to streamline your creative workflow, allowing you to generate high-speed images with one model and then instantly animate them with the other—all at a fraction of the cost 🍌⚡️ 1️⃣ Introducing Nano Banana 2 Lite: Our fastest and most cost-efficient Gemini Image model yet delivers text-to-image outputs in under 4 seconds. Now available via the Gemini API and Google AI Studio, and rolling out soon across @NotebookLM, Google Flow, Google Gemini, Stitch by Google, Google Search and Google Photos. 2️⃣ Gemini Omni Flash in Public Preview: Our natively multimodal model for cost-efficient video generation and conversational editing. Now available via the Gemini API, Google AI Studio, and Gemini Enterprise Agent Platform so you can integrate the model into your workflow. While exciting on their own, the real magic happens when you build using these models together. Watch how our interior design demo integrates Nano Banana 2 Lite and Omni to instantly reimagine any space. Upload a photo, swipe through tailored design concepts, and see Omni bring the details to life in cinematic motion. Try out the demo app in AI Studio:$

We’re shipping two major updates to streamline your creative workflow, allowing you to generate high-speed images with one model and then instantly animate them with the other—all at a fraction of the cost 🍌⚡️ 1️⃣ Introducing Nano Banana 2 Lite: Our fastest and most cost-efficient Gemini Image model yet delivers text-to-image outputs in under 4 seconds. Now available via the Gemini API and Google AI Studio, and rolling out soon across @NotebookLM, Google Flow, Google Gemini, Stitch by Google, Google Search and Google Photos. 2️⃣ Gemini Omni Flash in Public Preview: Our natively multimodal model for cost-efficient video generation and conversational editing. Now available via the Gemini API, Google AI Studio, and Gemini Enterprise Agent Platform so you can integrate the model into your workflow. While exciting on their own, the real magic happens when you build using these models together. Watch how our interior design demo integrates Nano Banana 2 Lite and Omni to instantly reimagine any space. Upload a photo, swipe through tailored design concepts, and see Omni bring the details to life in cinematic motion. Try out the demo app in AI Studio:

Google AI

129,340 次观看 • 3 个月前

I love everything about this video. How many images do we get of women celebrating in public? Especially older women - laughing, joyous, being a little naughty is the life energy we all need. They are repositories of decades of Basant rooftop memories.

I love everything about this video. How many images do we get of women celebrating in public? Especially older women - laughing, joyous, being a little naughty is the life energy we all need. They are repositories of decades of Basant rooftop memories.

Maria Amir 🍉

124,737 次观看 • 8 个月前

Just a trippy dream of how Grok and I traversed the universe to understand it. All images to create the video were generated by Grok, then I face swapped and animated them with Luma and Sora. This is also an original song, "Across the Universe," I made with Suno.

Just a trippy dream of how Grok and I traversed the universe to understand it. All images to create the video were generated by Grok, then I face swapped and animated them with Luma and Sora. This is also an original song, "Across the Universe," I made with Suno.

Katia Karpenko

8,247,450 次观看 • 1 年前

WARNING DISTURBING IMAGES🚨 This video is from a secret animal testing lab in Hamburg, Germany. These labs are all over the world and they conduct the most cruel and depraved “experiments” you could imagine on monkeys, puppies, piglets and pretty much any other animal you can think of. It’s all done under the banner of “scientific progress” and it is legal.

WARNING DISTURBING IMAGES🚨 This video is from a secret animal testing lab in Hamburg, Germany. These labs are all over the world and they conduct the most cruel and depraved “experiments” you could imagine on monkeys, puppies, piglets and pretty much any other animal you can think of. It’s all done under the banner of “scientific progress” and it is legal.

Champagne Joshi

51,328 次观看 • 2 年前

Ambassador Monica Crowley, Chief of Protocol of the United States, and representative for Freedom 250, on the 'Freedom Moves You' campaign to bring patriotic images to America's transportation hubs ahead of our 250th birthday: "This campaign is probably the biggest that will connect all Americans... we're all engaged in transporting ourselves across this great land, so to have this campaign, these images out there in our key transportation hubs... every American will have eyes on this campaign."

Ambassador Monica Crowley, Chief of Protocol of the United States, and representative for Freedom 250, on the 'Freedom Moves You' campaign to bring patriotic images to America's transportation hubs ahead of our 250th birthday: "This campaign is probably the biggest that will connect all Americans... we're all engaged in transporting ourselves across this great land, so to have this campaign, these images out there in our key transportation hubs... every American will have eyes on this campaign."

Rapid Response 47

69,502 次观看 • 7 个月前

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

This is the fastest I've seen Llama 3.3 running anywhere! Llama 3.3 70B running at 652 t/s is lightning fast. And if you want Llama 3.1, here are the speeds I was able to get: • Llama 3.1 8B: 1006 t/s • Llama 3.1 70B: 709 t/s • Llama 3.1 405B: 206 t/s (You can access all of these models for free! See the link below.) This speed is incredible, but the interesting part is what's happening behind the scenes: These models aren't running on a GPU! In this video, I'm using the SambaNova cloud to access these models. They built a custom chip (SN40L) optimized for AI workflows. A single SN40L chip can hold hundreds of models (trillions of parameters) in memory! The speed alone is a huge deal, but the big advantage is for agentic workflows running multiple specialized models. A GPU can only host a single model and switch (unload and load) to a different model if necessary. An SN40L, on the other hand, can host every model at once, making it much faster. Here is the video where you can see how fast these chips are:

Santiago

97,561 次观看 • 1 年前

Stop Creating Images One by One (Automate Everything with Google Flow) Creating AI images manually is one of the biggest time wasters for content creators. This tutorial shows you how to automate the entire process with Google Flow, so you can generate hundreds of high quality images in bulk instead of repeating the same prompts over and over. You'll learn how to build an automated workflow, batch generate AI images, download them automatically, and scale your content creation with far less effort. If you're creating YouTube videos, social media content, marketing assets, or AI projects, this workflow can save you hours every week. Like, Retweet and Bookmark. Follow BIG CHRIS 🤝 Video Credit: AI Content Insights (

Stop Creating Images One by One (Automate Everything with Google Flow) Creating AI images manually is one of the biggest time wasters for content creators. This tutorial shows you how to automate the entire process with Google Flow, so you can generate hundreds of high quality images in bulk instead of repeating the same prompts over and over. You'll learn how to build an automated workflow, batch generate AI images, download them automatically, and scale your content creation with far less effort. If you're creating YouTube videos, social media content, marketing assets, or AI projects, this workflow can save you hours every week. Like, Retweet and Bookmark. Follow BIG CHRIS 🤝 Video Credit: AI Content Insights (

BIG CHRIS

54,520 次观看 • 3 个月前

This video I made 4 years ago is about to become more relevant than ever Egyptian authorities have NOT shown the public ALL images of tunnels/chambers that exist UNDERNEATH & INSIDE the Great SPHINX 📍This video includes never before seen images of a hidden passage from INSIDE the Sphinx In short, Dr. Zahi Hawass lied when he said “they’re in my book” - No, these are 100% not in any book or publication he’s ever shared… Stay tuned for my next video, as this story is about to become much bigger 🔥

This video I made 4 years ago is about to become more relevant than ever Egyptian authorities have NOT shown the public ALL images of tunnels/chambers that exist UNDERNEATH & INSIDE the Great SPHINX 📍This video includes never before seen images of a hidden passage from INSIDE the Sphinx In short, Dr. Zahi Hawass lied when he said “they’re in my book” - No, these are 100% not in any book or publication he’s ever shared… Stay tuned for my next video, as this story is about to become much bigger 🔥

Jimmy Corsetti

59,248 次观看 • 4 个月前

i created this WWII documentary just with AI video models no After Effects no Remotion or complex setups no $1000 spent on an editor it took Calliope 30 seconds (yes, 30 seconds) to animate all the clips in parallel here's how: 1- reuse the image + video prompts from this article 2- write the story around one real person and check the facts 3- generate the narration with ElevenLabs v3 and the images in Extra High 4- review the stills in the editor 5- choose Extra High + All Clips to animate them in batch the images come first. animation is a separate step, so you can fix a bad frame before paying to animate it all inside the same tool the prompts are in the article below ⬇️

i created this WWII documentary just with AI video models no After Effects no Remotion or complex setups no $1000 spent on an editor it took Calliope 30 seconds (yes, 30 seconds) to animate all the clips in parallel here's how: 1- reuse the image + video prompts from this article 2- write the story around one real person and check the facts 3- generate the narration with ElevenLabs v3 and the images in Extra High 4- review the stills in the editor 5- choose Extra High + All Clips to animate them in batch the images come first. animation is a separate step, so you can fix a bad frame before paying to animate it all inside the same tool the prompts are in the article below ⬇️

Sergio Gil

102,232 次观看 • 29 天前

David Sacks on How Anthropic is Ironically Running Surveillance on Their Latest Models “This is the company that said that it was against government surveillance. They are now retaining for 30 days every prompt and every output you send to one of these Mythos class models.”

David Sacks on How Anthropic is Ironically Running Surveillance on Their Latest Models “This is the company that said that it was against government surveillance. They are now retaining for 30 days every prompt and every output you send to one of these Mythos class models.”

Chief Nerd

277,927 次观看 • 4 个月前

🚨 New Abstract Badge: "The Mogger" You have to get to Level 5 and pick up the badge. If you're lucky, going downstairs will take no time at all and your cost will be 0.001 ETH. This video can help you understand how it works 👇

🚨 New Abstract Badge: "The Mogger" You have to get to Level 5 and pick up the badge. If you're lucky, going downstairs will take no time at all and your cost will be 0.001 ETH. This video can help you understand how it works 👇

ensar ✳️

11,681 次观看 • 8 个月前

To every landlord out there, how does your stomach feel seeing this? You take a chance on tenants who seemed solid. They fall on hard times, go six months without paying, and when the sheriff finally shows up, you walk into a flooded basement with the taps left running. This is the brutal cost of being the boss.

To every landlord out there, how does your stomach feel seeing this? You take a chance on tenants who seemed solid. They fall on hard times, go six months without paying, and when the sheriff finally shows up, you walk into a flooded basement with the taps left running. This is the brutal cost of being the boss.

Desiree

1,395,967 次观看 • 2 天前

“Jaylen has to understand this is what your career has been built on. Not being respected. You didn’t make the Olympic team… you’re not in the top 5 All-NBA… this is what keeps you hungry Jaylen” -Paul Pierce New Episode:

“Jaylen has to understand this is what your career has been built on. Not being respected. You didn’t make the Olympic team… you’re not in the top 5 All-NBA… this is what keeps you hungry Jaylen” -Paul Pierce New Episode:

No Fouls Given

98,916 次观看 • 3 个月前