Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Introducing Vals-Smith: turn your code base into a customized benchmark. Public benchmarks tell you which model is strongest overall, not which model is the best on your code. Vals-Smith turns your merged pull requests into real coding tasks and measures the percentage a model can actually resolve. New models... show more

Vals AI

28,754 subscribers

46,847 Aufrufe • vor 3 Monaten •via X (Twitter)

Wissenschaft & Technologie Bildung Gesundheit & Wellness

Anya Rossi• Live Now

Private livecam show

28 Kommentare

Profilbild von Vals AI

Vals AIvor 3 Monaten

Join the waitlist today!

Profilbild von Rayan Krishnan

Rayan Krishnanvor 3 Monaten

props to @OfirPress @jyangballin @KLieret for laying a lot of the ground work. by default we evaluate all models with mini swe agent, methodology is heavily swe bench inspired

Profilbild von Jerry Liu

Jerry Liuvor 3 Monaten

congrats on the launch!!

Profilbild von Gabriel Stengel

Gabriel Stengelvor 3 Monaten

Congrats @RayanKrishnan on the launch!

Profilbild von Rez Havaei

Rez Havaeivor 3 Monaten

Great work!

Profilbild von Aryo Patel

Aryo Patelvor 3 Monaten

Awesome release -- can't wait to try it out code speed about to double

Profilbild von Jihoon

Jihoonvor 3 Monaten

i had this idea for months and these days its great, you just have to wait because good ideas end up getting implemented thanks!

Profilbild von Braeden Norris

Braeden Norrisvor 3 Monaten

Super cool launch, congrats!

Profilbild von Inflectiv AI ⧉

Inflectiv AI ⧉vor 3 Monaten

🔥🔥

Profilbild von 安叫兽|Bird🕊️ 🔶 BNB

安叫兽|Bird🕊️ 🔶 BNBvor 3 Monaten

拿自己仓库测，比看榜单踏实多了

Profilbild von Ronak Malde

Ronak Maldevor 3 Monaten

This is very exciting!!

Profilbild von icosaedro

icosaedrovor 3 Monaten

PRs as the benchmark set is the only metric that matters. Generic evals lie.

Profilbild von Sumer Sao

Sumer Saovor 3 Monaten

congrats guys!

Profilbild von Jason Zhao

Jason Zhaovor 3 Monaten

Awesome stuff

Profilbild von Rona Wang

Rona Wangvor 3 Monaten

🚀

Profilbild von Vivek Gopalan

Vivek Gopalanvor 3 Monaten

Really important

Profilbild von Moritz Stephan

Moritz Stephanvor 3 Monaten

this is awesome

Profilbild von Daniel Rabinovich

Daniel Rabinovichvor 3 Monaten

This is really cool!

Profilbild von Faraz

Farazvor 3 Monaten

Really cool work @RayanKrishnan !!

Profilbild von Dhruv Mangtani

Dhruv Mangtanivor 3 Monaten

sounds expensive lol

Profilbild von Sergey Karayev

Sergey Karayevvor 3 Monaten

repo-specific evals are the way. we built this into @Superconductor too, and you can run your own on merged PRs today no waitlist:

Profilbild von Dhruvik Parikh 🌎

Dhruvik Parikh 🌎vor 3 Monaten

🔥🔥🔥

Profilbild von shyamal

shyamalvor 3 Monaten

congrats on the launch!

Profilbild von iTokens.io ⚡️ AI API

iTokens.io ⚡️ AI APIvor 3 Monaten

This feels like the direction coding evaluation is heading. The most useful benchmark isn't the one that ranks models globally—it's the one built from your own workflows and codebase.

Profilbild von Jihoon

Jihoonvor 3 Monaten

@grok competitor landscape in both closed and open source

Profilbild von RAZA | AI EXPLORER

RAZA | AI EXPLORERvor 3 Monaten

Measuring performance on your own code is a game-changer. Love this.

Profilbild von Lucas Campa

Lucas Campavor 3 Monaten

Looks awesome 🚀🚀

Profilbild von Panizzutti

Panizzuttivor 3 Monaten

I like this but vibe coded video

Ähnliche Videos

Introducing Arena Mode in Windsurf: One prompt. Two models. Your vote. Benchmarks don't reflect real-world coding quality. The best model for you depends on your codebase and stack. So we made real-world coding the benchmark. Free for the next week. May the best model win.

Introducing Arena Mode in Windsurf: One prompt. Two models. Your vote. Benchmarks don't reflect real-world coding quality. The best model for you depends on your codebase and stack. So we made real-world coding the benchmark. Free for the next week. May the best model win.

Devin Developers

1,073,143 Aufrufe • vor 8 Monaten

🧳 Bring Your Own AI Models to GitHub Copilot in VS Code Use your own Azure-hosted AI models with GitHub Copilot in VS Code. See how Bring Your Own Key (BYOK) lets you connect models from Microsoft Foundry, choose which models appear in Copilot, and use the right model for different coding tasks.

🧳 Bring Your Own AI Models to GitHub Copilot in VS Code Use your own Azure-hosted AI models with GitHub Copilot in VS Code. See how Bring Your Own Key (BYOK) lets you connect models from Microsoft Foundry, choose which models appear in Copilot, and use the right model for different coding tasks.

Visual Studio Code

38,536 Aufrufe • vor 1 Monat

They're still your ideas. [code]smith just writes the code. [code]smith is a cloud software engineer that works alongside you. Autofixing PRs, CI failures and reviews.

They're still your ideas. [code]smith just writes the code. [code]smith is a cloud software engineer that works alongside you. Autofixing PRs, CI failures and reviews.

Blacksmith

3,621,539 Aufrufe • vor 2 Monaten

SOMEONE MADE IT SO YOU CAN CODE WITH 174 AI MODELS FROM 23 PROVIDERS FOR FREE one npm package and one install: "npm i -g free-coding-models" switch between any model instantly. benchmark them against each other in real time to see which one actually codes best for your use case. no subscriptions. no API keys. no cost. 174 models. 23 providers, all free, all from your terminal. if you're tired of paying for 3 different AI subscriptions just to compare which model codes better, this is it

SOMEONE MADE IT SO YOU CAN CODE WITH 174 AI MODELS FROM 23 PROVIDERS FOR FREE one npm package and one install: "npm i -g free-coding-models" switch between any model instantly. benchmark them against each other in real time to see which one actually codes best for your use case. no subscriptions. no API keys. no cost. 174 models. 23 providers, all free, all from your terminal. if you're tired of paying for 3 different AI subscriptions just to compare which model codes better, this is it

Om Patel

93,660 Aufrufe • vor 6 Monaten

Introducing Shiro - the CUTEST ai coding agent which turns your prompts into pull requests. Link in the comments. @JoinShiro

Introducing Shiro - the CUTEST ai coding agent which turns your prompts into pull requests. Link in the comments. @JoinShiro

Dev

496,344 Aufrufe • vor 1 Jahr

This approach has made Sonnet the model of choice for developers worldwide. In addition to our new model, we're launching Claude Code, our first coding tool, in a limited research preview. With Claude Code, you can delegate substantial tasks to Claude—right from your terminal.

This approach has made Sonnet the model of choice for developers worldwide. In addition to our new model, we're launching Claude Code, our first coding tool, in a limited research preview. With Claude Code, you can delegate substantial tasks to Claude—right from your terminal.

Anthropic

1,141,101 Aufrufe • vor 1 Jahr

Introducing Claude Code on the web. You can now delegate coding tasks to Claude without opening your terminal.

Introducing Claude Code on the web. You can now delegate coding tasks to Claude without opening your terminal.

Claude

2,666,443 Aufrufe • vor 1 Jahr

Meet AutoRouter, the most intelligent way to pick models You’re wasting tokens selecting frontier models for every prompt, and sometimes for worse performance AutoRouter handles the decision layer: - Understands the prompt request - Picks the best model fit for the task - Saves you money at every turn - Plugs into your current setup Works with your plans & every leading model inside Codex/Claude Code Beta is live:

Meet AutoRouter, the most intelligent way to pick models You’re wasting tokens selecting frontier models for every prompt, and sometimes for worse performance AutoRouter handles the decision layer: - Understands the prompt request - Picks the best model fit for the task - Saves you money at every turn - Plugs into your current setup Works with your plans & every leading model inside Codex/Claude Code Beta is live:

ty

82,364 Aufrufe • vor 2 Tagen

🎙️ Dictate in Multiple Languages in #VSCode Dictation can detect your language and turn your speech into text using a local model. You can use it in Chat, the editor, and the integrated terminal.

🎙️ Dictate in Multiple Languages in #VSCode Dictation can detect your language and turn your speech into text using a local model. You can use it in Chat, the editor, and the integrated terminal.

Visual Studio Code

27,733 Aufrufe • vor 2 Monaten

Multimodal AI is here 🤯 GPT-4 can now turn your images into a text file in a snap with the new code interpreter model. Witness the OCR magic in action 🔥

Multimodal AI is here 🤯 GPT-4 can now turn your images into a text file in a snap with the new code interpreter model. Witness the OCR magic in action 🔥

Shubham Saboo

727,673 Aufrufe • vor 3 Jahren

Introducing OpenResearch: run parallel research agents with any model Bring your own compute and your code + data stays with you A virtual lab on your machine

Introducing OpenResearch: run parallel research agents with any model Bring your own compute and your code + data stays with you A virtual lab on your machine

alphaXiv

32,864 Aufrufe • vor 3 Monaten

Don't talk directly to a model. You can make your application 10x more flexible by simply adding one intermediate layer between your code and the model. This is the simplest trick you can use to improve the architecture of whatever you are building.

Don't talk directly to a model. You can make your application 10x more flexible by simply adding one intermediate layer between your code and the model. This is the simplest trick you can use to improve the architecture of whatever you are building.

Santiago

88,884 Aufrufe • vor 6 Monaten

Introducing Merge Gateway - Build Your Own Router. You're three sprints into your coding assistant. You pick the most hyped model, integrate, test, deploy. A month later, a new model drops. Now you re-test, re-integrate, re-deploy. Your product didn't change, but the benchmark did. That's how most AI teams operate. Chasing a "best" defined by people who've never seen their product. There is no best model. There's only the right one for your product, users, and use-cases. Build Your Own Router runs on your definition of good. Pick your benchmarks, weigh them, add your own evals. Merge routes every request to your winner. 👉$100 in credits to the first 200 people that comment

Introducing Merge Gateway - Build Your Own Router. You're three sprints into your coding assistant. You pick the most hyped model, integrate, test, deploy. A month later, a new model drops. Now you re-test, re-integrate, re-deploy. Your product didn't change, but the benchmark did. That's how most AI teams operate. Chasing a "best" defined by people who've never seen their product. There is no best model. There's only the right one for your product, users, and use-cases. Build Your Own Router runs on your definition of good. Pick your benchmarks, weigh them, add your own evals. Merge routes every request to your winner. 👉$100 in credits to the first 200 people that comment

Shensi Ding

3,668,527 Aufrufe • vor 5 Monaten

Introducing WrapdCode Claude Code with Bring Your Own Model 1. Use GPT, Gemini, Llama, Mistral - 300+ models 2. Same Claude Code UI, zero changes 3. Plug in your API key and go Stop being stuck on one model.

Introducing WrapdCode Claude Code with Bring Your Own Model 1. Use GPT, Gemini, Llama, Mistral - 300+ models 2. Same Claude Code UI, zero changes 3. Plug in your API key and go Stop being stuck on one model.

aditya

29,014 Aufrufe • vor 6 Monaten

Fusion Models by robinhood:0xa92768863a55d8a0591709f7f5e594a249d36ea3, explained properly. Why a panel beats one model: Ask three models the same hard question and they disagree. The disagreement is the information, where they contradict each other is where one of them is wrong, and a judge can see it. OpenRouter shipped exactly this in June and their panels beat any single frontier model. Credit to them and their $10B sale to stripe. Why the same panel for everyone is the wrong shape: You don't have one best model. You have a best model per job. The one you trust for code is not the one you trust for writing, and neither is the one you'd pick for a five-line reply. A panel chosen per request, identical for every user, can't know that. A panel arranged around you can. The number that matters isn't price: It's cost per answer that works. Model A costs $4 per million tokens and gets your job right one time in four. Model B costs $10 and gets it right three times in four. Per answer you actually keep, A costs $16 and B costs $13. The cheap model was the expensive one. Everyone comparing price lists is measuring the wrong thing. You never rate anything: Stopping an answer halfway is a rating. Regenerating is a rating. Switching model and asking again is the loudest rating there is. Every one of those already writes a row we can learn from. The router learns which models earn your credits, per job, and tries a cheaper one now and then to check. Fusion only when it pays: Cost mode tries the cheap model first and escalates only if a judge says it failed. Performance mode fans out to your panel and merges. Most simple prompts never fuse. That's why it costs less, not more. Put it together and you have a personal model: Not trained. Assembled. Your profile from a two-minute questionnaire and everything you've imported from Claude or ChatGPT. Your memory, shared across every model so switching never starts from zero. Your skills, workflows you build once and run on any model. Your scorecard, which models earn your credits per job. One per person. Nobody else has yours. Every model is someone else's; the arrangement is you, and it leaves with you. People aren't loyal to models. They're loyal to results. A moat can't be a commodity, but it can be our own creation.

askr

13,900 Aufrufe • vor 1 Monat

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

Artificial Analysis

133,793 Aufrufe • vor 2 Monaten

Introducing Cursor 2.0. Our first coding model and the best way to code with agents.

Introducing Cursor 2.0. Our first coding model and the best way to code with agents.

Cursor

3,572,368 Aufrufe • vor 1 Jahr

Stop hardcoding one model name in your code. Now you can give each request a policy: a small rule for what the call needs It picks the right model for the job, on your own keys This is unhardcoded, our new open source routing for AI models, live today

Stop hardcoding one model name in your code. Now you can give each request a policy: a small rule for what the call needs It picks the right model for the job, on your own keys This is unhardcoded, our new open source routing for AI models, live today

GenLayer

98,139 Aufrufe • vor 4 Monaten

Run your coding agent on a local model, with QVAC. OpenCode now plugs into a model on your own machine through our official plugin. That changes two things. Privacy by default. Your repo, your prompts, and every command the agent runs stay on your device. No third-party model ever sees your code. For an agent that reads your codebase and runs commands, that is the point, not a nice-to-have. New workflows. With no meter and no rate limit, the agent does not have to stop. Let it loop on a task, refactor a whole repo overnight, or run a watch-build-fix loop that keeps going. Setup takes a few minutes: one line in opencode.json and OpenCode runs on a local QVAC model. In the video demo, it handles three real developer tasks back to back, fully offline.

Run your coding agent on a local model, with QVAC. OpenCode now plugs into a model on your own machine through our official plugin. That changes two things. Privacy by default. Your repo, your prompts, and every command the agent runs stay on your device. No third-party model ever sees your code. For an agent that reads your codebase and runs commands, that is the point, not a nice-to-have. New workflows. With no meter and no rate limit, the agent does not have to stop. Let it loop on a task, refactor a whole repo overnight, or run a watch-build-fix loop that keeps going. Setup takes a few minutes: one line in opencode.json and OpenCode runs on a local QVAC model. In the video demo, it handles three real developer tasks back to back, fully offline.

QVAC

11,382,936 Aufrufe • vor 4 Monaten

STOP GUESSING WHICH LOCAL AI MODELS YOUR COMPUTER CAN RUN. LLMFIT SCANS YOUR HARDWARE AND TELLS YOU WHICH MODELS WILL ACTUALLY RUN WELL, INCLUDING THE BEST QUANTIZATION TO USE.

STOP GUESSING WHICH LOCAL AI MODELS YOUR COMPUTER CAN RUN. LLMFIT SCANS YOUR HARDWARE AND TELLS YOU WHICH MODELS WILL ACTUALLY RUN WELL, INCLUDING THE BEST QUANTIZATION TO USE.

0xMarioNawfal

48,706 Aufrufe • vor 1 Monat