Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Introducing Vals-Smith: turn your code base into a customized benchmark. Public benchmarks tell you which model is strongest overall, not which model is the best on your code. Vals-Smith turns your merged pull requests into real coding tasks and measures the percentage a model can actually resolve. New models... show more

Vals AI

28,754 subscribers

46,847 görüntüleme • 3 ay önce •via X (Twitter)

Bilim & Teknoloji Eğitim Sağlık & İyilik

Anya Rossi• Live Now

Private livecam show

28 Yorum

Vals AI profil fotoğrafı

Vals AI3 ay önce

Join the waitlist today!

Rayan Krishnan profil fotoğrafı

Rayan Krishnan3 ay önce

props to @OfirPress @jyangballin @KLieret for laying a lot of the ground work. by default we evaluate all models with mini swe agent, methodology is heavily swe bench inspired

Jerry Liu profil fotoğrafı

Jerry Liu3 ay önce

congrats on the launch!!

Gabriel Stengel profil fotoğrafı

Gabriel Stengel3 ay önce

Congrats @RayanKrishnan on the launch!

Rez Havaei profil fotoğrafı

Rez Havaei3 ay önce

Great work!

Aryo Patel profil fotoğrafı

Aryo Patel3 ay önce

Awesome release -- can't wait to try it out code speed about to double

Jihoon profil fotoğrafı

Jihoon3 ay önce

i had this idea for months and these days its great, you just have to wait because good ideas end up getting implemented thanks!

Braeden Norris profil fotoğrafı

Braeden Norris3 ay önce

Super cool launch, congrats!

Inflectiv AI ⧉ profil fotoğrafı

Inflectiv AI ⧉3 ay önce

🔥🔥

安叫兽|Bird🕊️ 🔶 BNB profil fotoğrafı

安叫兽|Bird🕊️ 🔶 BNB3 ay önce

拿自己仓库测，比看榜单踏实多了

Ronak Malde profil fotoğrafı

Ronak Malde3 ay önce

This is very exciting!!

icosaedro profil fotoğrafı

icosaedro3 ay önce

PRs as the benchmark set is the only metric that matters. Generic evals lie.

Sumer Sao profil fotoğrafı

Sumer Sao3 ay önce

congrats guys!

Jason Zhao profil fotoğrafı

Jason Zhao3 ay önce

Awesome stuff

Rona Wang profil fotoğrafı

Rona Wang3 ay önce

🚀

Vivek Gopalan profil fotoğrafı

Vivek Gopalan3 ay önce

Really important

Moritz Stephan profil fotoğrafı

Moritz Stephan3 ay önce

this is awesome

Daniel Rabinovich profil fotoğrafı

Daniel Rabinovich3 ay önce

This is really cool!

Faraz profil fotoğrafı

Faraz3 ay önce

Really cool work @RayanKrishnan !!

Dhruv Mangtani profil fotoğrafı

Dhruv Mangtani3 ay önce

sounds expensive lol

Sergey Karayev profil fotoğrafı

Sergey Karayev3 ay önce

repo-specific evals are the way. we built this into @Superconductor too, and you can run your own on merged PRs today no waitlist:

Dhruvik Parikh 🌎 profil fotoğrafı

Dhruvik Parikh 🌎3 ay önce

🔥🔥🔥

shyamal profil fotoğrafı

shyamal3 ay önce

congrats on the launch!

iTokens.io ⚡️ AI API profil fotoğrafı

iTokens.io ⚡️ AI API3 ay önce

This feels like the direction coding evaluation is heading. The most useful benchmark isn't the one that ranks models globally—it's the one built from your own workflows and codebase.

Jihoon profil fotoğrafı

Jihoon3 ay önce

@grok competitor landscape in both closed and open source

RAZA | AI EXPLORER profil fotoğrafı

RAZA | AI EXPLORER3 ay önce

Measuring performance on your own code is a game-changer. Love this.

Lucas Campa profil fotoğrafı

Lucas Campa3 ay önce

Looks awesome 🚀🚀

Panizzutti profil fotoğrafı

Panizzutti3 ay önce

I like this but vibe coded video

Benzer Videolar

Introducing Arena Mode in Windsurf: One prompt. Two models. Your vote. Benchmarks don't reflect real-world coding quality. The best model for you depends on your codebase and stack. So we made real-world coding the benchmark. Free for the next week. May the best model win.

Introducing Arena Mode in Windsurf: One prompt. Two models. Your vote. Benchmarks don't reflect real-world coding quality. The best model for you depends on your codebase and stack. So we made real-world coding the benchmark. Free for the next week. May the best model win.

Devin Developers

1,073,143 görüntüleme • 8 ay önce

🧳 Bring Your Own AI Models to GitHub Copilot in VS Code Use your own Azure-hosted AI models with GitHub Copilot in VS Code. See how Bring Your Own Key (BYOK) lets you connect models from Microsoft Foundry, choose which models appear in Copilot, and use the right model for different coding tasks.

🧳 Bring Your Own AI Models to GitHub Copilot in VS Code Use your own Azure-hosted AI models with GitHub Copilot in VS Code. See how Bring Your Own Key (BYOK) lets you connect models from Microsoft Foundry, choose which models appear in Copilot, and use the right model for different coding tasks.

Visual Studio Code

38,536 görüntüleme • 1 ay önce

They're still your ideas. [code]smith just writes the code. [code]smith is a cloud software engineer that works alongside you. Autofixing PRs, CI failures and reviews.

They're still your ideas. [code]smith just writes the code. [code]smith is a cloud software engineer that works alongside you. Autofixing PRs, CI failures and reviews.

Blacksmith

3,621,539 görüntüleme • 2 ay önce

SOMEONE MADE IT SO YOU CAN CODE WITH 174 AI MODELS FROM 23 PROVIDERS FOR FREE one npm package and one install: "npm i -g free-coding-models" switch between any model instantly. benchmark them against each other in real time to see which one actually codes best for your use case. no subscriptions. no API keys. no cost. 174 models. 23 providers, all free, all from your terminal. if you're tired of paying for 3 different AI subscriptions just to compare which model codes better, this is it

SOMEONE MADE IT SO YOU CAN CODE WITH 174 AI MODELS FROM 23 PROVIDERS FOR FREE one npm package and one install: "npm i -g free-coding-models" switch between any model instantly. benchmark them against each other in real time to see which one actually codes best for your use case. no subscriptions. no API keys. no cost. 174 models. 23 providers, all free, all from your terminal. if you're tired of paying for 3 different AI subscriptions just to compare which model codes better, this is it

Om Patel

93,660 görüntüleme • 6 ay önce

Introducing Shiro - the CUTEST ai coding agent which turns your prompts into pull requests. Link in the comments. @JoinShiro

Introducing Shiro - the CUTEST ai coding agent which turns your prompts into pull requests. Link in the comments. @JoinShiro

Dev

496,344 görüntüleme • 1 yıl önce

This approach has made Sonnet the model of choice for developers worldwide. In addition to our new model, we're launching Claude Code, our first coding tool, in a limited research preview. With Claude Code, you can delegate substantial tasks to Claude—right from your terminal.

This approach has made Sonnet the model of choice for developers worldwide. In addition to our new model, we're launching Claude Code, our first coding tool, in a limited research preview. With Claude Code, you can delegate substantial tasks to Claude—right from your terminal.

Anthropic

1,141,101 görüntüleme • 1 yıl önce

Introducing Claude Code on the web. You can now delegate coding tasks to Claude without opening your terminal.

Introducing Claude Code on the web. You can now delegate coding tasks to Claude without opening your terminal.

Claude

2,666,443 görüntüleme • 1 yıl önce

Meet AutoRouter, the most intelligent way to pick models You’re wasting tokens selecting frontier models for every prompt, and sometimes for worse performance AutoRouter handles the decision layer: - Understands the prompt request - Picks the best model fit for the task - Saves you money at every turn - Plugs into your current setup Works with your plans & every leading model inside Codex/Claude Code Beta is live:

Meet AutoRouter, the most intelligent way to pick models You’re wasting tokens selecting frontier models for every prompt, and sometimes for worse performance AutoRouter handles the decision layer: - Understands the prompt request - Picks the best model fit for the task - Saves you money at every turn - Plugs into your current setup Works with your plans & every leading model inside Codex/Claude Code Beta is live:

ty

82,364 görüntüleme • 2 gün önce

🎙️ Dictate in Multiple Languages in #VSCode Dictation can detect your language and turn your speech into text using a local model. You can use it in Chat, the editor, and the integrated terminal.

🎙️ Dictate in Multiple Languages in #VSCode Dictation can detect your language and turn your speech into text using a local model. You can use it in Chat, the editor, and the integrated terminal.

Visual Studio Code

27,733 görüntüleme • 2 ay önce

Multimodal AI is here 🤯 GPT-4 can now turn your images into a text file in a snap with the new code interpreter model. Witness the OCR magic in action 🔥

Multimodal AI is here 🤯 GPT-4 can now turn your images into a text file in a snap with the new code interpreter model. Witness the OCR magic in action 🔥

Shubham Saboo

727,673 görüntüleme • 3 yıl önce

Introducing OpenResearch: run parallel research agents with any model Bring your own compute and your code + data stays with you A virtual lab on your machine

Introducing OpenResearch: run parallel research agents with any model Bring your own compute and your code + data stays with you A virtual lab on your machine

alphaXiv

32,864 görüntüleme • 3 ay önce

Don't talk directly to a model. You can make your application 10x more flexible by simply adding one intermediate layer between your code and the model. This is the simplest trick you can use to improve the architecture of whatever you are building.

Don't talk directly to a model. You can make your application 10x more flexible by simply adding one intermediate layer between your code and the model. This is the simplest trick you can use to improve the architecture of whatever you are building.

Santiago

88,884 görüntüleme • 6 ay önce

Introducing Merge Gateway - Build Your Own Router. You're three sprints into your coding assistant. You pick the most hyped model, integrate, test, deploy. A month later, a new model drops. Now you re-test, re-integrate, re-deploy. Your product didn't change, but the benchmark did. That's how most AI teams operate. Chasing a "best" defined by people who've never seen their product. There is no best model. There's only the right one for your product, users, and use-cases. Build Your Own Router runs on your definition of good. Pick your benchmarks, weigh them, add your own evals. Merge routes every request to your winner. 👉$100 in credits to the first 200 people that comment

Introducing Merge Gateway - Build Your Own Router. You're three sprints into your coding assistant. You pick the most hyped model, integrate, test, deploy. A month later, a new model drops. Now you re-test, re-integrate, re-deploy. Your product didn't change, but the benchmark did. That's how most AI teams operate. Chasing a "best" defined by people who've never seen their product. There is no best model. There's only the right one for your product, users, and use-cases. Build Your Own Router runs on your definition of good. Pick your benchmarks, weigh them, add your own evals. Merge routes every request to your winner. 👉$100 in credits to the first 200 people that comment

Shensi Ding

3,668,527 görüntüleme • 5 ay önce

Introducing WrapdCode Claude Code with Bring Your Own Model 1. Use GPT, Gemini, Llama, Mistral - 300+ models 2. Same Claude Code UI, zero changes 3. Plug in your API key and go Stop being stuck on one model.

Introducing WrapdCode Claude Code with Bring Your Own Model 1. Use GPT, Gemini, Llama, Mistral - 300+ models 2. Same Claude Code UI, zero changes 3. Plug in your API key and go Stop being stuck on one model.

aditya

29,014 görüntüleme • 6 ay önce

Fusion Models by robinhood:0xa92768863a55d8a0591709f7f5e594a249d36ea3, explained properly. Why a panel beats one model: Ask three models the same hard question and they disagree. The disagreement is the information, where they contradict each other is where one of them is wrong, and a judge can see it. OpenRouter shipped exactly this in June and their panels beat any single frontier model. Credit to them and their $10B sale to stripe. Why the same panel for everyone is the wrong shape: You don't have one best model. You have a best model per job. The one you trust for code is not the one you trust for writing, and neither is the one you'd pick for a five-line reply. A panel chosen per request, identical for every user, can't know that. A panel arranged around you can. The number that matters isn't price: It's cost per answer that works. Model A costs $4 per million tokens and gets your job right one time in four. Model B costs $10 and gets it right three times in four. Per answer you actually keep, A costs $16 and B costs $13. The cheap model was the expensive one. Everyone comparing price lists is measuring the wrong thing. You never rate anything: Stopping an answer halfway is a rating. Regenerating is a rating. Switching model and asking again is the loudest rating there is. Every one of those already writes a row we can learn from. The router learns which models earn your credits, per job, and tries a cheaper one now and then to check. Fusion only when it pays: Cost mode tries the cheap model first and escalates only if a judge says it failed. Performance mode fans out to your panel and merges. Most simple prompts never fuse. That's why it costs less, not more. Put it together and you have a personal model: Not trained. Assembled. Your profile from a two-minute questionnaire and everything you've imported from Claude or ChatGPT. Your memory, shared across every model so switching never starts from zero. Your skills, workflows you build once and run on any model. Your scorecard, which models earn your credits per job. One per person. Nobody else has yours. Every model is someone else's; the arrangement is you, and it leaves with you. People aren't loyal to models. They're loyal to results. A moat can't be a commodity, but it can be our own creation.

askr

13,900 görüntüleme • 1 ay önce

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

Artificial Analysis

133,793 görüntüleme • 2 ay önce

Introducing Cursor 2.0. Our first coding model and the best way to code with agents.

Introducing Cursor 2.0. Our first coding model and the best way to code with agents.

Cursor

3,572,368 görüntüleme • 1 yıl önce

Stop hardcoding one model name in your code. Now you can give each request a policy: a small rule for what the call needs It picks the right model for the job, on your own keys This is unhardcoded, our new open source routing for AI models, live today

Stop hardcoding one model name in your code. Now you can give each request a policy: a small rule for what the call needs It picks the right model for the job, on your own keys This is unhardcoded, our new open source routing for AI models, live today

GenLayer

98,139 görüntüleme • 4 ay önce

Run your coding agent on a local model, with QVAC. OpenCode now plugs into a model on your own machine through our official plugin. That changes two things. Privacy by default. Your repo, your prompts, and every command the agent runs stay on your device. No third-party model ever sees your code. For an agent that reads your codebase and runs commands, that is the point, not a nice-to-have. New workflows. With no meter and no rate limit, the agent does not have to stop. Let it loop on a task, refactor a whole repo overnight, or run a watch-build-fix loop that keeps going. Setup takes a few minutes: one line in opencode.json and OpenCode runs on a local QVAC model. In the video demo, it handles three real developer tasks back to back, fully offline.

Run your coding agent on a local model, with QVAC. OpenCode now plugs into a model on your own machine through our official plugin. That changes two things. Privacy by default. Your repo, your prompts, and every command the agent runs stay on your device. No third-party model ever sees your code. For an agent that reads your codebase and runs commands, that is the point, not a nice-to-have. New workflows. With no meter and no rate limit, the agent does not have to stop. Let it loop on a task, refactor a whole repo overnight, or run a watch-build-fix loop that keeps going. Setup takes a few minutes: one line in opencode.json and OpenCode runs on a local QVAC model. In the video demo, it handles three real developer tasks back to back, fully offline.

QVAC

11,382,936 görüntüleme • 4 ay önce

STOP GUESSING WHICH LOCAL AI MODELS YOUR COMPUTER CAN RUN. LLMFIT SCANS YOUR HARDWARE AND TELLS YOU WHICH MODELS WILL ACTUALLY RUN WELL, INCLUDING THE BEST QUANTIZATION TO USE.

STOP GUESSING WHICH LOCAL AI MODELS YOUR COMPUTER CAN RUN. LLMFIT SCANS YOUR HARDWARE AND TELLS YOU WHICH MODELS WILL ACTUALLY RUN WELL, INCLUDING THE BEST QUANTIZATION TO USE.

0xMarioNawfal

48,706 görüntüleme • 1 ay önce