Загрузка видео...

Не удалось загрузить видео

На главную

Last week we launched Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency Watch our new video walkthrough to see how to build and run your own benchmark with Optima, and find the best model for your use case Start...

ArtificialAnlys's profile picture

Artificial Analysis

130,534 subscribers

18,569 просмотров • 5 дней назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at
2:33

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

Artificial Analysis

130,269 просмотров • 11 дней назад

Introducing Merge Gateway - Build Your Own Router. You're three sprints into your coding assistant. You pick the most hyped model, integrate, test, deploy. A month later, a new model drops. Now you re-test, re-integrate, re-deploy. Your product didn't change, but the benchmark did. That's how most AI teams operate. Chasing a "best" defined by people who've never seen their product. There is no best model. There's only the right one for your product, users, and use-cases. Build Your Own Router runs on your definition of good. Pick your benchmarks, weigh them, add your own evals. Merge routes every request to your winner. 👉$100 in credits to the first 200 people that comment
1:57

Introducing Merge Gateway - Build Your Own Router. You're three sprints into your coding assistant. You pick the most hyped model, integrate, test, deploy. A month later, a new model drops. Now you re-test, re-integrate, re-deploy. Your product didn't change, but the benchmark did. That's how most AI teams operate. Chasing a "best" defined by people who've never seen their product. There is no best model. There's only the right one for your product, users, and use-cases. Build Your Own Router runs on your definition of good. Pick your benchmarks, weigh them, add your own evals. Merge routes every request to your winner. 👉$100 in credits to the first 200 people that comment

Shensi Ding

3,667,066 просмотров • 3 месяцев назад

Meet #DBRX: a general-purpose LLM that sets a new standard for efficient open source models. Use the DBRX model in your RAG apps or use the DBRX design to build your own custom LLMs and improve the quality of your GenAI applications.
0:20

Meet #DBRX: a general-purpose LLM that sets a new standard for efficient open source models. Use the DBRX model in your RAG apps or use the DBRX design to build your own custom LLMs and improve the quality of your GenAI applications.

Databricks

327,800 просмотров • 2 лет назад

New short tutorial on building your own custom AI agent for business or personal use with ASI:ONE by Fetch.ai We walk through how we created the Stakecito Intern, our automated Customer Service Agent that answers all our validator business related questions. Bonus: Build your own custom AI and we will award the best submission with $100 worth of $FET.
6:31

New short tutorial on building your own custom AI agent for business or personal use with ASI:ONE by Fetch.ai We walk through how we created the Stakecito Intern, our automated Customer Service Agent that answers all our validator business related questions. Bonus: Build your own custom AI and we will award the best submission with $100 worth of $FET.

Stakecito | Validator ⚛️

14,334 просмотров • 8 месяцев назад

Wake up to a unique way to travel in 2025 with Caledonian Sleeper. 🛏️ Enjoy your own private room, 💤 and let your dreams take you to new adventures. 📍 Arrive in the heart of the city, refreshed and ready to explore. Watch our new video for inspiration and start planning your next journey today. You can book up to 12 months ahead at
1:00

Wake up to a unique way to travel in 2025 with Caledonian Sleeper. 🛏️ Enjoy your own private room, 💤 and let your dreams take you to new adventures. 📍 Arrive in the heart of the city, refreshed and ready to explore. Watch our new video for inspiration and start planning your next journey today. You can book up to 12 months ahead at

Caledonian Sleeper

33,933 просмотров • 1 год назад

Your perfect model is always ready. With Fotor, you can create your own AI model and use it to bring your marketing ideas to life. Generate consistent photos, videos, and campaign assets without the time and cost of traditional photoshoots. One AI model. Endless content. Create your model. Build your brand. Market smarter. Try Fotor AI Today！
0:31

Your perfect model is always ready. With Fotor, you can create your own AI model and use it to bring your marketing ideas to life. Generate consistent photos, videos, and campaign assets without the time and cost of traditional photoshoots. One AI model. Endless content. Create your model. Build your brand. Market smarter. Try Fotor AI Today！

Fotor

45,821 просмотров • 1 месяц назад

Our #developer documentation and #app management portal got a big upgrade today!🎉 Check out the new Spotify for Developers and learn about how to build your own app that integrates with Spotify.
0:36

Our #developer documentation and #app management portal got a big upgrade today!🎉 Check out the new Spotify for Developers and learn about how to build your own app that integrates with Spotify.

Spotify Engineering

10,574 просмотров • 3 лет назад

Introducing Arena Mode in Windsurf: One prompt. Two models. Your vote. Benchmarks don't reflect real-world coding quality. The best model for you depends on your codebase and stack. So we made real-world coding the benchmark. Free for the next week. May the best model win.
0:33

Introducing Arena Mode in Windsurf: One prompt. Two models. Your vote. Benchmarks don't reflect real-world coding quality. The best model for you depends on your codebase and stack. So we made real-world coding the benchmark. Free for the next week. May the best model win.

Devin Desktop

1,063,786 просмотров • 6 месяцев назад

We launched a webinar series on how to build your own programming language in C++. Come and join!
0:37

We launched a webinar series on how to build your own programming language in C++. Come and join!

PVS-Studio

1,834,366 просмотров • 17 дней назад

Introducing Lab: A full-stack platform for training your own agentic models Build, evaluate and train on your own environments at scale without managing the underlying infrastructure. Giving everyone their own frontier AI lab.
1:23

Introducing Lab: A full-stack platform for training your own agentic models Build, evaluate and train on your own environments at scale without managing the underlying infrastructure. Giving everyone their own frontier AI lab.

Prime Intellect

3,153,958 просмотров • 6 месяцев назад

Claude Managed Agents can operate in a sandbox you control, on your own infrastructure or with any provider you choose. Today we added new guides for Blaxel, E2B, Google Cloud, Namespace, and Superserve, so you can choose the best fit for your use case.
0:24

Claude Managed Agents can operate in a sandbox you control, on your own infrastructure or with any provider you choose. Today we added new guides for Blaxel, E2B, Google Cloud, Namespace, and Superserve, so you can choose the best fit for your use case.

ClaudeDevs

514,717 просмотров • 2 месяцев назад

🔥 New LangChain Academy Course: LangChain Essentials (Python & TypeScript) 🔥 Learn the basics of LangChain – our open source framework that makes it easy to start building agents with any model provider. Last week, we released LangChain 1.0. We’ve completely rewritten LangChain to be opinionated, focused, and powered by LangGraph’s runtime. It includes a new `create_agent` abstraction to build agents quickly, middleware for flexibility, and standard content blocks that work across any model provider. In this quickstart course, you'll learn how to: - Build an agent with the `create_agent` abstraction - Use LangChain’s core building blocks: Models, Messages, Memory, and Tools - Customize your agent with middleware - Debug and test your agent with LangSmith Observability & Evaluation Enroll for free ➡️
1:37

🔥 New LangChain Academy Course: LangChain Essentials (Python & TypeScript) 🔥 Learn the basics of LangChain – our open source framework that makes it easy to start building agents with any model provider. Last week, we released LangChain 1.0. We’ve completely rewritten LangChain to be opinionated, focused, and powered by LangGraph’s runtime. It includes a new `create_agent` abstraction to build agents quickly, middleware for flexibility, and standard content blocks that work across any model provider. In this quickstart course, you'll learn how to: - Build an agent with the `create_agent` abstraction - Use LangChain’s core building blocks: Models, Messages, Memory, and Tools - Customize your agent with middleware - Debug and test your agent with LangSmith Observability & Evaluation Enroll for free ➡️

LangChain

50,488 просмотров • 10 месяцев назад

Introducing MTPLX V1: The fastest and simplest way to run MTP compatible models on your Mac. - New Swift based app. 2x speed increase without bloat - Easy OpenCode, Hermes & Pi integration - Convert your own MTP models with forge And more try now at:
0:37

Introducing MTPLX V1: The fastest and simplest way to run MTP compatible models on your Mac. - New Swift based app. 2x speed increase without bloat - Easy OpenCode, Hermes & Pi integration - Convert your own MTP models with forge And more try now at:

Youssof Al Toukhi

31,741 просмотров • 2 месяцев назад

Introducing Vals-Smith: turn your code base into a customized benchmark. Public benchmarks tell you which model is strongest overall, not which model is the best on your code. Vals-Smith turns your merged pull requests into real coding tasks and measures the percentage a model can actually resolve. New models ship every week. Vals-Smith tells you which one to trust with your code.
0:37

Introducing Vals-Smith: turn your code base into a customized benchmark. Public benchmarks tell you which model is strongest overall, not which model is the best on your code. Vals-Smith turns your merged pull requests into real coding tasks and measures the percentage a model can actually resolve. New models ship every week. Vals-Smith tells you which one to trust with your code.

Vals AI

45,256 просмотров • 1 месяц назад

Create your own memories and own your own watch face with the itel 020 smart watch today! Get extra straps for extra drip and amazing new features!!! @itelsmartlifenaija #smartwatch#myfirstAIwatch#itelsmartwatch#itelFit020 #KassiaSule
1:45

Create your own memories and own your own watch face with the itel 020 smart watch today! Get extra straps for extra drip and amazing new features!!! @itelsmartlifenaija #smartwatch#myfirstAIwatch#itelsmartwatch#itelFit020 #KassiaSule

KASSIA

26,032 просмотров • 1 год назад

Welcome to Week 4 of the Ghost Gaming Recruitment Challenge powered by Own. Last week, you teamed up with fellow creators and collaborated. For the final week, show us how you build community and momentum through consistency. Use #GhostRC and tag Own. in your submissions.
0:39

Welcome to Week 4 of the Ghost Gaming Recruitment Challenge powered by Own. Last week, you teamed up with fellow creators and collaborated. For the final week, show us how you build community and momentum through consistency. Use #GhostRC and tag Own. in your submissions.

Ghost Gaming

10,913 просмотров • 3 месяцев назад

“Why don’t we do this? When you build your building, build your own electric plant. And nobody took it seriously. They didn’t believe me that it was happening. And I said no I’m serious. You’re going to build your own Con Edison in New York. You’re going to make your own.”
2:19

“Why don’t we do this? When you build your building, build your own electric plant. And nobody took it seriously. They didn’t believe me that it was happening. And I said no I’m serious. You’re going to build your own Con Edison in New York. You’re going to make your own.”

Duncan S. Campbell

39,392 просмотров • 1 год назад

We’re unveiling a new look for Artificial Analysis! We’ve come a long way since launching Artificial Analysis over 2 years ago. Today, we benchmark 400+ models, 50+ inference providers, and benchmark not only language models but also image, video, speech, music, hardware, and agents. Our mission to support the AI ecosystem with independent benchmarking remains the same, but our brand and website refresh is designed to better reflect how much we’ve grown and how much further we plan to go. A huge thank you to everyone who has been part of the Artificial Analysis community along the way: from developers choosing models and building agents, to labs, inference and hardware providers, and fellow independent researchers.
0:25

We’re unveiling a new look for Artificial Analysis! We’ve come a long way since launching Artificial Analysis over 2 years ago. Today, we benchmark 400+ models, 50+ inference providers, and benchmark not only language models but also image, video, speech, music, hardware, and agents. Our mission to support the AI ecosystem with independent benchmarking remains the same, but our brand and website refresh is designed to better reflect how much we’ve grown and how much further we plan to go. A huge thank you to everyone who has been part of the Artificial Analysis community along the way: from developers choosing models and building agents, to labs, inference and hardware providers, and fellow independent researchers.

Artificial Analysis

18,159 просмотров • 4 месяцев назад

Introducing Workflows, a new way to build your own tools inside of Runway. Now you can create your own custom node-based workflows chaining together multiple models, modalities and intermediary steps for even more control of your generations. Build the Workflows that work for you. Available now for our Creative Partners and Enterprise customers. Coming soon to all plans.
0:48

Introducing Workflows, a new way to build your own tools inside of Runway. Now you can create your own custom node-based workflows chaining together multiple models, modalities and intermediary steps for even more control of your generations. Build the Workflows that work for you. Available now for our Creative Partners and Enterprise customers. Coming soon to all plans.

Runway

227,095 просмотров • 10 месяцев назад

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.
2:47

We released Sonic-3.5 and Ink-2, the #1 streaming models for text to speech and speech to text you can use in your voice agents today. New architectures enable new frontiers for speed and quality. We're now the only provider to have #1 models for both speaking and listening.

Karan Goel

7,026,184 просмотров • 2 месяцев назад