正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

🚀 Introducing Community Benchmarks on Kaggle! As AI evolves at an unprecedented pace, measuring intelligence requires more than a few AI research labs alone – it requires the imagination and collective expertise of the global community. That’s why we’re launching Community Benchmarks. It lets you build, run, and share... show more

Kaggle

319,336 subscribers

54,677 次观看 • 8 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

8 条评论

Kaggle 的头像

Kaggle8 个月前

With community benchmarks you get: ✨ Free access (within quota) to leading models 🧠 Multimodal, multi-step & tool-based tasks 📊 Compare model performance on a leaderboard 🛠 Powered by the kaggle-benchmarks SDK

Timur Yessenov 的头像

Timur Yessenov8 个月前

this matters more than people realize. lab benchmarks test what researchers think matters. community benchmarks test what builders actually need. biggest gap i see: "can it follow complex multi-step instructions without hallucinating?" no benchmark for that. real builders run their own tests daily.

ANKIT 𓃱 的头像

ANKIT 𓃱8 个月前

This is a strong step toward democratizing AI evaluation. Community driven benchmarks can surface real world tasks that academic suites miss and keep model progress grounded in practical impact. Looking forward to seeing how this evolves and what the community builds.

Jenny 的头像

Jenny8 个月前

Community-driven benchmarks could surface edge cases that internal teams miss. Transparency in AI evaluation is overdue.

Dinohacks 的头像

Dinohacks8 个月前

Is it possible to use a dataset with input and expected output while using this feature? If so, how to do that ?

Dr. Senthazal Ravi 的头像

Dr. Senthazal Ravi8 个月前

Awesome.

Himanshu Kumar 的头像

Himanshu Kumar8 个月前

@kaggle, the community benchmarks will be crucial as AI's pace of evolution accelerates, so it will be useful to understand how these will be.

Aaron Bowen 的头像

Aaron Bowen8 个月前

Hey guys, help me explain to people what's wrong with LLMs if I'm right? I don't care about being wrong. I'm worried if I'm right, LLMs and Humans experience hallucinations in similar ways. Deviation in time from context and perspective. Help me find the plot for everyone else, please? Best Wishes, Aaron Bowen 🍀✨

相关视频

Earlier today we released local development for Kaggle Benchmarks. 🚀 You can now write, validate and run AI evaluation tasks directly from your preferred dev environment — VSCode, Antigravity, Claude Code, and more. Go from idea to working eval using natural language with the write-kaggle-benchmarks skill.

Earlier today we released local development for Kaggle Benchmarks. 🚀 You can now write, validate and run AI evaluation tasks directly from your preferred dev environment — VSCode, Antigravity, Claude Code, and more. Go from idea to working eval using natural language with the write-kaggle-benchmarks skill.

Kaggle

16,445 次观看 • 3 个月前

🚀 Kaggle Benchmarks is here! Get competition-grade rigor for AI model evaluation. Let Kaggle handle infrastructure while you focus on AI breakthroughs. View model performance on 70+ leaderboards, including AI at Meta's MultiLoKo. Dive in:

🚀 Kaggle Benchmarks is here! Get competition-grade rigor for AI model evaluation. Let Kaggle handle infrastructure while you focus on AI breakthroughs. View model performance on 70+ leaderboards, including AI at Meta's MultiLoKo. Dive in:

Kaggle

24,649 次观看 • 1 年前

Before founding Vals AI, Rayan Krishnan was a college radio host. We're thrilled to get him back in his element with Vivek Gopalan, this time talking all things benchmarks and evals. As AI capabilities compound, measuring them is becoming both harder and more important. What started as an academic exercise is quickly evolving into critical infrastructure for understanding what AI systems can actually do, and how they perform in the real world. Rayan and Vivek go deep on how benchmarks are evolving, measuring increasingly agentic systems, the weird and wonderful ways we test intelligence, the global AI race, and why evals sit on the critical path to ASI. 0:44 Importance and evolution of benchmarks 8:16 From academic artifact → real-world value 10:57 The next generation of benchmarks 13:34 Benchmarking models vs. agents 16:20 Building benchmarks for organizations 22:09 Weird and wonderful benchmarks 25:58 The AI arms race: US vs. China 31:02 Why evals are on the critical path to ASI 32:50 Vals Series A

Before founding Vals AI, Rayan Krishnan was a college radio host. We're thrilled to get him back in his element with Vivek Gopalan, this time talking all things benchmarks and evals. As AI capabilities compound, measuring them is becoming both harder and more important. What started as an academic exercise is quickly evolving into critical infrastructure for understanding what AI systems can actually do, and how they perform in the real world. Rayan and Vivek go deep on how benchmarks are evolving, measuring increasingly agentic systems, the weird and wonderful ways we test intelligence, the global AI race, and why evals sit on the critical path to ASI. 0:44 Importance and evolution of benchmarks 8:16 From academic artifact → real-world value 10:57 The next generation of benchmarks 13:34 Benchmarking models vs. agents 16:20 Building benchmarks for organizations 22:09 Weird and wonderful benchmarks 25:58 The AI arms race: US vs. China 31:02 Why evals are on the critical path to ASI 32:50 Vals Series A

8VC

104,829 次观看 • 1 个月前

In the first episode of Kaggle Conversations, meg.ai 🇨🇦 and Walter Reade from the Kaggle team talk with Alex Shaw, creator of Harbor Framework, about how the AI Agent industry is moving towards autonomy, why software engineering benchmarks are prone to contamination, and what it took to build a secure evaluation framework for the $1M Konwinski Prize. Watch a clip from the episode 👇

In the first episode of Kaggle Conversations, meg.ai 🇨🇦 and Walter Reade from the Kaggle team talk with Alex Shaw, creator of Harbor Framework, about how the AI Agent industry is moving towards autonomy, why software engineering benchmarks are prone to contamination, and what it took to build a secure evaluation framework for the $1M Konwinski Prize. Watch a clip from the episode 👇

Kaggle

17,495 次观看 • 2 个月前

The consensus in education is that AI will replace the teacher. We’re launching the opposite bet today. I believe humans are going to be more powerful, more creative, and more important in an AI future. That’s why we’re launching Maven as the home for human expertise. Where humans can build superpowers and learn from those at the forefront of their fields. Learn from Humans. #learnfromhumans

The consensus in education is that AI will replace the teacher. We’re launching the opposite bet today. I believe humans are going to be more powerful, more creative, and more important in an AI future. That’s why we’re launching Maven as the home for human expertise. Where humans can build superpowers and learn from those at the forefront of their fields. Learn from Humans. #learnfromhumans

Gagan Biyani 🏛

11,931 次观看 • 4 个月前

📢Introducing Kaggle Game Arena: a new, open benchmark platform where top AI models compete in complex, strategic games in streamed match-ups. We're charting new frontiers for trustworthy AI evaluation and it begins with chess — a classic proving ground for system intelligence.

📢Introducing Kaggle Game Arena: a new, open benchmark platform where top AI models compete in complex, strategic games in streamed match-ups. We're charting new frontiers for trustworthy AI evaluation and it begins with chess — a classic proving ground for system intelligence.

Kaggle

114,176 次观看 • 1 年前

Agentic AI is exposing the limits of traditional LLM benchmarks—and forcing a rethink of how enterprise AI value is actually measured. Analysts Ryan Shrout & Mitch Lewis join Kamiwaza AI Platform Engineer JV Roig to unpack new research from Signal65 and Kamiwaza that demonstrates why legacy benchmarks miss real-world agentic work, and how the Kamiwaza Agentic Merit Index (KAMI) delivers a more meaningful way to evaluate enterprise-ready agentic AI.

Agentic AI is exposing the limits of traditional LLM benchmarks—and forcing a rethink of how enterprise AI value is actually measured. Analysts Ryan Shrout & Mitch Lewis join Kamiwaza AI Platform Engineer JV Roig to unpack new research from Signal65 and Kamiwaza that demonstrates why legacy benchmarks miss real-world agentic work, and how the Kamiwaza Agentic Merit Index (KAMI) delivers a more meaningful way to evaluate enterprise-ready agentic AI.

Six Five Media

166,438 次观看 • 8 个月前

We just launched a very different type of competition on Kaggle! More on why (+ prize & rule details) in the video 👇 Official 2023 Kaggle AI Report Competition Website:

We just launched a very different type of competition on Kaggle! More on why (+ prize & rule details) in the video 👇 Official 2023 Kaggle AI Report Competition Website:

Kaggle

24,807 次观看 • 3 年前

Introducing Ricursive Intelligence, a frontier AI lab enabling a recursive self-improvement loop between AI and the chips that fuel it. Learn more at

Introducing Ricursive Intelligence, a frontier AI lab enabling a recursive self-improvement loop between AI and the chips that fuel it. Learn more at

Ricursive Intelligence

521,670 次观看 • 9 个月前

🔉 Introducing SAM Audio, the first unified model that isolates any sound from complex audio mixtures using text, visual, or span prompts. We’re sharing SAM Audio with the community, along with a perception encoder model, benchmarks and research papers, to empower others to explore new forms of expression and build applications that were previously out of reach. 🔗 Learn more:

🔉 Introducing SAM Audio, the first unified model that isolates any sound from complex audio mixtures using text, visual, or span prompts. We’re sharing SAM Audio with the community, along with a perception encoder model, benchmarks and research papers, to empower others to explore new forms of expression and build applications that were previously out of reach. 🔗 Learn more:

AI at Meta

1,252,746 次观看 • 9 个月前

🤩 Introducing Background Removal Arena: Compare & Test the Best Background Removal Models ⚔️ An AI arena to rank the top background removal tools based on community votes. A transparent, open, and reliable resource for community as the Leaderboard and Voting data are made public too.

🤩 Introducing Background Removal Arena: Compare & Test the Best Background Removal Models ⚔️ An AI arena to rank the top background removal tools based on community votes. A transparent, open, and reliable resource for community as the Leaderboard and Voting data are made public too.

Gradio

25,214 次观看 • 1 年前

✅ Bharat’s sovereign models are performing well on global benchmarks. ✅ Success of the sovereign stack marks a significant achievement for our engineers and innovators. 📍AI Impact Summit, New Delhi

✅ Bharat’s sovereign models are performing well on global benchmarks. ✅ Success of the sovereign stack marks a significant achievement for our engineers and innovators. 📍AI Impact Summit, New Delhi

Ashwini Vaishnaw

943,874 次观看 • 7 个月前

Could You Guess What Comes After the OptimAI Edge Node? We’re excited to share another glimpse into the future, Community-Powered Data Validation, a unique feature within the OptimAI Core Node for desktop. What is Data Validation? It’s the essential process of ensuring AI systems learn from accurate, unbiased, and high-quality data. Through human intelligence, users verify, correct, and refine information - making it reliable for training advanced AI agents. This process is enabled by the OptimAI DeHIN (Decentralized Human Intelligence Network), which brings collective intelligence into the heart of AI development. Why is it important? Because truly responsible and powerful AI starts with trusted data. Community validation improves fairness, transparency, and performance - ensuring AI benefits everyone. And as a participant, you’re not just supporting better AI; you’re earning rewards and shaping its future. Your desktop is about to become much more than a device. It’s a gateway to reinforcing the quality of data that powers Agentic AI. Validate data. Strengthen AI. Get rewarded. The future of AI is collective — and it needs you. #BUIDL with us! 👉OptimAI Lite Node: 👉OptimAI Edge Node:

Could You Guess What Comes After the OptimAI Edge Node? We’re excited to share another glimpse into the future, Community-Powered Data Validation, a unique feature within the OptimAI Core Node for desktop. What is Data Validation? It’s the essential process of ensuring AI systems learn from accurate, unbiased, and high-quality data. Through human intelligence, users verify, correct, and refine information - making it reliable for training advanced AI agents. This process is enabled by the OptimAI DeHIN (Decentralized Human Intelligence Network), which brings collective intelligence into the heart of AI development. Why is it important? Because truly responsible and powerful AI starts with trusted data. Community validation improves fairness, transparency, and performance - ensuring AI benefits everyone. And as a participant, you’re not just supporting better AI; you’re earning rewards and shaping its future. Your desktop is about to become much more than a device. It’s a gateway to reinforcing the quality of data that powers Agentic AI. Validate data. Strengthen AI. Get rewarded. The future of AI is collective — and it needs you. #BUIDL with us! 👉OptimAI Lite Node: 👉OptimAI Edge Node:

OptimAI Network

60,795 次观看 • 1 年前

Launching now: a new way to follow the AI industry. Beta starts now for the next month. A joint project between Unaligned (my company) and Levangie Labs (Brayden Levangie company). It reads 50,000 of you, and follows 8,300 AI companies here on X. And pulls out the best and most interesting. All built with the X API. Check it out: And please sign up for its daily newsletter. Yeah, $25 a month is a lot for many of you, but that will defray the costs and let me expand it to do a lot more than just AI. Also, the same AI agent that built the site, and did EVERYTHING you see can build custom reports for you on literally any tech community here on X. It supports OpenClaw, RSS, and Notebook LM too. And I'll add more from your requests. More:

Launching now: a new way to follow the AI industry. Beta starts now for the next month. A joint project between Unaligned (my company) and Levangie Labs (Brayden Levangie company). It reads 50,000 of you, and follows 8,300 AI companies here on X. And pulls out the best and most interesting. All built with the X API. Check it out: And please sign up for its daily newsletter. Yeah, $25 a month is a lot for many of you, but that will defray the costs and let me expand it to do a lot more than just AI. Also, the same AI agent that built the site, and did EVERYTHING you see can build custom reports for you on literally any tech community here on X. It supports OpenClaw, RSS, and Notebook LM too. And I'll add more from your requests. More:

Robert Scoble

148,486 次观看 • 5 个月前

Introducing Kombai, the first AI agent built for frontend development. Kombai vastly outperforms SOTA models + generic agents in real-world benchmarks. Watch it add a complex new feature to an OSS codebase with 500K+ lines of code. More use cases and links in comments

Introducing Kombai, the first AI agent built for frontend development. Kombai vastly outperforms SOTA models + generic agents in real-world benchmarks. Watch it add a complex new feature to an OSS codebase with 500K+ lines of code. More use cases and links in comments

Dipanjan Dey

2,075,311 次观看 • 1 年前

We are hiring creative strategists. If you’re excited about AI video and ads, this role is for you. With AI you can make anything. But making something worth watching requires a human. That’s why we’re building a super-team of creative strategists. You will: - Work with the fastest growing and largest AI brands - Be paid to learn at the frontier of AI - Work with the top AI marketers, filmmakers, engineers, and creators - Ship more ads per day than others do in a month

We are hiring creative strategists. If you’re excited about AI video and ads, this role is for you. With AI you can make anything. But making something worth watching requires a human. That’s why we’re building a super-team of creative strategists. You will: - Work with the fastest growing and largest AI brands - Be paid to learn at the frontier of AI - Work with the top AI marketers, filmmakers, engineers, and creators - Ship more ads per day than others do in a month

Rahul Bhardwaj

46,361 次观看 • 3 个月前

AI video generation is advancing at an incredible pace and it's exciting to see tools that continue pushing the boundaries of what's possible. Better motion consistency, stronger prompt adherence and more polished results are making AI an even more powerful creative partner. Seedance 2.5 on Morphic is definitely one of the developments I'm excited to explore. It's great to see Morphic AI bringing cutting edge video models to creators and making high-quality AI video generation more accessible. Looking forward to seeing the creative projects the community builds and how this technology continues to evolve. Seedance 2.5 on Morphic 🚀

AI video generation is advancing at an incredible pace and it's exciting to see tools that continue pushing the boundaries of what's possible. Better motion consistency, stronger prompt adherence and more polished results are making AI an even more powerful creative partner. Seedance 2.5 on Morphic is definitely one of the developments I'm excited to explore. It's great to see Morphic AI bringing cutting edge video models to creators and making high-quality AI video generation more accessible. Looking forward to seeing the creative projects the community builds and how this technology continues to evolve. Seedance 2.5 on Morphic 🚀

NIJ Ruvos

34,043 次观看 • 1 个月前

$Enterprise AI is in a wildly paradoxical state 🤔, and we’re reaching the inflection point that will resolve it. 💥 Enterprises want to differentiate with AI, yet rent the same intelligence as their competitors. Their workflows and expertise are highly specialized, yet they rely on generic models built to be good at everything. They worry about AI costs, yet pay premium prices for massive models where only a fraction (1%) of the intelligence is relevant to their task. 💸 And they demand control and sovereignty, yet rent the intelligence becoming core to their business. This is not a sustainable equilibrium. The next era of enterprise AI is specialized intelligence companies build, own, and compound. And we are at the inflection point of this transition. That’s the bet we made when we started Oumi two years ago. Today we’re closing the loop: Oumi can now not only automatically build your specialized AI models, but also deploy them into production, learn from their production experience, and continuously improve them. The intelligence that your business runs on, becomes your differentiator. Your compounding advantage. The winners of the next AI era will turn their own data, expertise, and experience into specialized intelligence that nobody else can rent. Don’t rent your AI. Build it. Own it. Compound it.$

Enterprise AI is in a wildly paradoxical state 🤔, and we’re reaching the inflection point that will resolve it. 💥 Enterprises want to differentiate with AI, yet rent the same intelligence as their competitors. Their workflows and expertise are highly specialized, yet they rely on generic models built to be good at everything. They worry about AI costs, yet pay premium prices for massive models where only a fraction (1%) of the intelligence is relevant to their task. 💸 And they demand control and sovereignty, yet rent the intelligence becoming core to their business. This is not a sustainable equilibrium. The next era of enterprise AI is specialized intelligence companies build, own, and compound. And we are at the inflection point of this transition. That’s the bet we made when we started Oumi two years ago. Today we’re closing the loop: Oumi can now not only automatically build your specialized AI models, but also deploy them into production, learn from their production experience, and continuously improve them. The intelligence that your business runs on, becomes your differentiator. Your compounding advantage. The winners of the next AI era will turn their own data, expertise, and experience into specialized intelligence that nobody else can rent. Don’t rent your AI. Build it. Own it. Compound it.

Manos Koukoumidis

1,004,543 次观看 • 1 个月前