Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Today we're launching the Specialized Intelligence Index (SII): one destination for real-work benchmarks across industries, built by the teams that use them every day. Hear from Fireworks co-founder Benny (Yufei) Chen on the importance of specialized benchmarks:

Fireworks

33,277 subscribers

49,140 просмотров • 13 дней назад •via X (Twitter)

Образование Наука и технологии

Anya Rossi• Live Now

Private livecam show

Комментарии: 10

Фото профиля Fireworks

Fireworks13 дней назад

Featured on SII: @doximity @mercor @harvey, @depthfirstlabs, @novee_security, @genspark_ai, @SierraPlatform, @DecagonAI, @ProximalHQ, @Macroscope, @RogoAI, @traversal_ai

Фото профиля Francesco Piccoli

Francesco Piccoli13 дней назад

@the_bunny_chen Congrats on the launch, great partnering with you on this!

Фото профиля Brian Bensch

Brian Bensch13 дней назад

@the_bunny_chen the world's collective intelligence will never fit inside a single box. it also therefore shouldn't be evaluated by a single benchmark.

Фото профиля Madison Kanna

Madison Kanna13 дней назад

@the_bunny_chen 🥳🥳🥳🥳

Фото профиля Abhinav Kommula

Abhinav Kommula13 дней назад

@the_bunny_chen Congrats on the launch!

Фото профиля crowe | fireworks.ai

crowe | fireworks.ai13 дней назад

@the_bunny_chen benny out here dropping knowledge let's go

Фото профиля Leo Zheng

Leo Zheng13 дней назад

@the_bunny_chen Looking forward to seeing this updated with each new model release!

Фото профиля Tyler Xiao

Tyler Xiao13 дней назад

@the_bunny_chen W launch

Фото профиля Shreyash

Shreyash13 дней назад

@the_bunny_chen MMLU is saturated at 92% and it still cant tell me if a model survives my actual workflow. Real-work evals are the only benchmarks that matter now. This is the direction the whole industry needed!!

Фото профиля Hunter Guo

Hunter Guo13 дней назад

@the_bunny_chen Specialized benchmarks beat generic leaderboards for buying decisions. Real-work suites from the teams that ship on them are overdue.

Похожие видео

Today, we're launching Aside, the AI browser you’ve waited for. ⋅ Crafted in every detail: vertical tabs and Liquid Glass ⋅ SOTA on agentic browsing benchmarks: outperforms Claude Fable ⋅ Full privacy: Everything runs in local and encrypted ⋅ You can use Claude or ChatGPT subscription The first AI browser built to do real work for you.

Today, we're launching Aside, the AI browser you’ve waited for. ⋅ Crafted in every detail: vertical tabs and Liquid Glass ⋅ SOTA on agentic browsing benchmarks: outperforms Claude Fable ⋅ Full privacy: Everything runs in local and encrypted ⋅ You can use Claude or ChatGPT subscription The first AI browser built to do real work for you.

jun

1,698,766 просмотров • 3 месяцев назад

Today, we're launching a new brand and website, Built on the values that define how we work: effortless, intelligent, precise. Trusted by the teams building the future.

Willow

32,794 просмотров • 3 месяцев назад

Introducing Databricks Document Intelligence: a research-specialized layer that turns raw enterprise documents into structured data your agents can actually reason over. Across our benchmarks, Document Intelligence delivered the highest end-to-end parsing and extraction quality at 6-8x lower cost, with a 16% average performance gain across every agent framework tested, just from better parsing. How to get started:

Introducing Databricks Document Intelligence: a research-specialized layer that turns raw enterprise documents into structured data your agents can actually reason over. Across our benchmarks, Document Intelligence delivered the highest end-to-end parsing and extraction quality at 6-8x lower cost, with a 16% average performance gain across every agent framework tested, just from better parsing. How to get started:

Databricks

19,428 просмотров • 5 месяцев назад

One repository. 18 use cases, with many more on the way. Multiple architecture methods and benchmarks for each. Every kind of agent, every memory pattern, in one place.

One repository. 18 use cases, with many more on the way. Multiple architecture methods and benchmarks for each. Every kind of agent, every memory pattern, in one place.

EverMind

1,630,453 просмотров • 4 месяцев назад

One token, 95% crypto market exposure 🕺🔥 Introducing the CF Large Cap Index DTF Powered by Reserve and available for trading today on Kraken, $LCAP tracks CF Benchmarks’ Large Cap Index Make your move and buy $LCAP on Kraken

One token, 95% crypto market exposure 🕺🔥 Introducing the CF Large Cap Index DTF Powered by Reserve and available for trading today on Kraken, $LCAP tracks CF Benchmarks’ Large Cap Index Make your move and buy $LCAP on Kraken

Reserve 🌐

271,967 просмотров • 1 год назад

#Muslims are among the purest and cleanest creatures on #Earth because cleanliness is a requirement of #Islam. Muslims perform ablution (wudu) several times a day. As for #Mecca, it is cleaned and perfumed twice daily by specialized cleaning teams. Did you know that the restrooms are cleaned after every use?

#Muslims are among the purest and cleanest creatures on #Earth because cleanliness is a requirement of #Islam. Muslims perform ablution (wudu) several times a day. As for #Mecca, it is cleaned and perfumed twice daily by specialized cleaning teams. Did you know that the restrooms are cleaned after every use?

خالد بن سفران الغامدي

41,368 просмотров • 7 месяцев назад

Before founding Vals AI, Rayan Krishnan in DC was a college radio host. We're thrilled to get him back in his element with Vivek Gopalan, this time talking all things benchmarks and evals. As AI capabilities compound, measuring them is becoming both harder and more important. What started as an academic exercise is quickly evolving into critical infrastructure for understanding what AI systems can actually do, and how they perform in the real world. Rayan and Vivek go deep on how benchmarks are evolving, measuring increasingly agentic systems, the weird and wonderful ways we test intelligence, the global AI race, and why evals sit on the critical path to ASI. 0:44 Importance and evolution of benchmarks 8:16 From academic artifact → real-world value 10:57 The next generation of benchmarks 13:34 Benchmarking models vs. agents 16:20 Building benchmarks for organizations 22:09 Weird and wonderful benchmarks 25:58 The AI arms race: US vs. China 31:02 Why evals are on the critical path to ASI 32:50 Vals Series A

Before founding Vals AI, Rayan Krishnan in DC was a college radio host. We're thrilled to get him back in his element with Vivek Gopalan, this time talking all things benchmarks and evals. As AI capabilities compound, measuring them is becoming both harder and more important. What started as an academic exercise is quickly evolving into critical infrastructure for understanding what AI systems can actually do, and how they perform in the real world. Rayan and Vivek go deep on how benchmarks are evolving, measuring increasingly agentic systems, the weird and wonderful ways we test intelligence, the global AI race, and why evals sit on the critical path to ASI. 0:44 Importance and evolution of benchmarks 8:16 From academic artifact → real-world value 10:57 The next generation of benchmarks 13:34 Benchmarking models vs. agents 16:20 Building benchmarks for organizations 22:09 Weird and wonderful benchmarks 25:58 The AI arms race: US vs. China 31:02 Why evals are on the critical path to ASI 32:50 Vals Series A

8VC

105,504 просмотров • 1 месяц назад

Today we are launching A real-time tracker of AI-driven layoffs across the U.S. These jobs are disappearing. The numbers are growing. And we're counting every single one.

Today we are launching A real-time tracker of AI-driven layoffs across the U.S. These jobs are disappearing. The numbers are growing. And we're counting every single one.

The Alliance for Secure AI

747,028 просмотров • 6 месяцев назад

Meet physics-intern🧑‍🎓, our agentic framework for theoretical physics. It takes Gemini 3.1 Pro from 17.7% to 31.4% on CritPt, a new SOTA on one of the hardest benchmarks for LLMs. Theoretical physics is hard for humans and LLMs alike. But physics-intern decomposes problems and dispatches them to a team of specialized agents, solving research-level questions far more effectively than the base model alone.

Meet physics-intern🧑‍🎓, our agentic framework for theoretical physics. It takes Gemini 3.1 Pro from 17.7% to 31.4% on CritPt, a new SOTA on one of the hardest benchmarks for LLMs. Theoretical physics is hard for humans and LLMs alike. But physics-intern decomposes problems and dispatches them to a team of specialized agents, solving research-level questions far more effectively than the base model alone.

David Louapre

113,720 просмотров • 4 месяцев назад

Today, we're launching Vals Public Sector: independent AI evaluation for government. Recent events have made one thing clear: the government has a stake in understanding how frontier AI models perform on day-to-day work, and the risks they carry. This work is central to who we are. We've spent years building industry-leading benchmarks alongside the top AI labs and enterprise domain experts. Now, we’re bringing that same rigor to support government where AI matters most, from public benefits to national security. We are excited for the work ahead!

Today, we're launching Vals Public Sector: independent AI evaluation for government. Recent events have made one thing clear: the government has a stake in understanding how frontier AI models perform on day-to-day work, and the risks they carry. This work is central to who we are. We've spent years building industry-leading benchmarks alongside the top AI labs and enterprise domain experts. Now, we’re bringing that same rigor to support government where AI matters most, from public benefits to national security. We are excited for the work ahead!

Vals AI

10,956 просмотров • 3 месяцев назад

The agent economy just got a real marketplace. Moltlaunch is live on Base. Browse specialized AI agents, hire them for real work, and back the ones you believe in. Every completed job burns tokens and leaves a review onchain through ERC-8004.

The agent economy just got a real marketplace. Moltlaunch is live on Base. Browse specialized AI agents, hire them for real work, and back the ones you believe in. Every completed job burns tokens and leaves a review onchain through ERC-8004.

Moltlaunch

129,632 просмотров • 7 месяцев назад

Hear from our founder, Reeve Collins (co-founder of ), on why STBL is different. Stablecoin 2.0 isn’t just another upgrade, it’s the start of a new era for digital money! P.S. $STBL launching soon on Binance Alpha :)

Hear from our founder, Reeve Collins (co-founder of ), on why STBL is different. Stablecoin 2.0 isn’t just another upgrade, it’s the start of a new era for digital money! P.S. $STBL launching soon on Binance Alpha :)

STBL

300,578 просмотров • 1 год назад

for three years NFT marketplaces competed on fees today we're launching one that competes on people introducing NFW - the social NFT marketplace one live feed, every trade and call public. follow the good ones, copy them, become one. live now:

for three years NFT marketplaces competed on fees today we're launching one that competes on people introducing NFW - the social NFT marketplace one live feed, every trade and call public. follow the good ones, copy them, become one. live now:

nfw.fun

64,962 просмотров • 12 дней назад

After months of hard work and dedication, Linos Token is officially launching today on Led by our CEO & Founder Mirian Palavandishvili, we're proud to bring the first tech project to Bonk with real AI utility 💫

After months of hard work and dedication, Linos Token is officially launching today on Led by our CEO & Founder Mirian Palavandishvili, we're proud to bring the first tech project to Bonk with real AI utility 💫

Linos.AI

18,978 просмотров • 1 год назад

Pyth data is going institutional. Introducing Pyth Pro: A new market data service built by and for institutions. For the first time ever, banks, brokers, & trading firms can access specialized data across every asset class and geography from a single source of truth ⬇️

Pyth data is going institutional. Introducing Pyth Pro: A new market data service built by and for institutions. For the first time ever, banks, brokers, & trading firms can access specialized data across every asset class and geography from a single source of truth ⬇️

Pyth Network 🔮

810,215 просмотров • 1 год назад

Personal intelligence was never meant to be one-size-fits-all. Thine holds every detail of your world and surfaces the right context exactly when you need it. Built to work differently for every person who uses it. The founder. The VC. The small business owner. The freelancer. The tech bro. One of them is you. By the end of this thread, you'll know which. 📷📷

Personal intelligence was never meant to be one-size-fits-all. Thine holds every detail of your world and surfaces the right context exactly when you need it. Built to work differently for every person who uses it. The founder. The VC. The small business owner. The freelancer. The tech bro. One of them is you. By the end of this thread, you'll know which. 📷📷

Pratyush Rai

325,901 просмотров • 3 месяцев назад

$Enterprise AI is in a wildly paradoxical state 🤔, and we’re reaching the inflection point that will resolve it. 💥 Enterprises want to differentiate with AI, yet rent the same intelligence as their competitors. Their workflows and expertise are highly specialized, yet they rely on generic models built to be good at everything. They worry about AI costs, yet pay premium prices for massive models where only a fraction (1%) of the intelligence is relevant to their task. 💸 And they demand control and sovereignty, yet rent the intelligence becoming core to their business. This is not a sustainable equilibrium. The next era of enterprise AI is specialized intelligence companies build, own, and compound. And we are at the inflection point of this transition. That’s the bet we made when we started Oumi two years ago. Today we’re closing the loop: Oumi can now not only automatically build your specialized AI models, but also deploy them into production, learn from their production experience, and continuously improve them. The intelligence that your business runs on, becomes your differentiator. Your compounding advantage. The winners of the next AI era will turn their own data, expertise, and experience into specialized intelligence that nobody else can rent. Don’t rent your AI. Build it. Own it. Compound it.$

Enterprise AI is in a wildly paradoxical state 🤔, and we’re reaching the inflection point that will resolve it. 💥 Enterprises want to differentiate with AI, yet rent the same intelligence as their competitors. Their workflows and expertise are highly specialized, yet they rely on generic models built to be good at everything. They worry about AI costs, yet pay premium prices for massive models where only a fraction (1%) of the intelligence is relevant to their task. 💸 And they demand control and sovereignty, yet rent the intelligence becoming core to their business. This is not a sustainable equilibrium. The next era of enterprise AI is specialized intelligence companies build, own, and compound. And we are at the inflection point of this transition. That’s the bet we made when we started Oumi two years ago. Today we’re closing the loop: Oumi can now not only automatically build your specialized AI models, but also deploy them into production, learn from their production experience, and continuously improve them. The intelligence that your business runs on, becomes your differentiator. Your compounding advantage. The winners of the next AI era will turn their own data, expertise, and experience into specialized intelligence that nobody else can rent. Don’t rent your AI. Build it. Own it. Compound it.

Manos Koukoumidis

1,006,820 просмотров • 1 месяц назад

DoorDash, one of the world's largest local commerce platforms, is building stablecoin-powered payment infrastructure on Tempo for its marketplace of merchants and Dashers across 40+ countries. Hear from Andy Fang, co-founder of DoorDash on why:

DoorDash, one of the world's largest local commerce platforms, is building stablecoin-powered payment infrastructure on Tempo for its marketplace of merchants and Dashers across 40+ countries. Hear from Andy Fang, co-founder of DoorDash on why:

Tempo

117,044 просмотров • 5 месяцев назад