Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Exa CEO Will Bryk explains why retrieval can help solve the tokenpocalypse: "We should not be using gigantic models for every task." "You should use a family of models of different sizes. The big model decides what to do, and it dishes out commands to the small models, and... those small models can be way more accurate and reliable if they're using retrieval." "Retrieval helps small models act like big models... We do save our customers a huge amount of tokens because they can use smaller models and use retrieval." "We could save 20x on cost for customers compared to other providers by being very efficient in what information from the web the agent actually sees." Will Bryk with Sarah Wangshow more

a16z

999,432 subscribers

25,144 views • 3 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Small Language Models (SML) are the future of AI. "Small" (SML) instead of "Large" (LLM). These small models are highly specialized models with superhuman abilities on specific tasks. Here are two techniques to build these models: • Spectrum • Model Merging I give you a short introduction in the attached video, but here is a quick summary: Spectrum helps us identify the most relevant layers to solve one specific task. We can ignore everything else and focus on fine-tuning these layers. Using Spectrum, we can fine-tune models in a heartbeat. Model Merging combines multiple models into a unique, much better model than any of the individual input models. You can also combine models specialized in different tasks and get a model with multiple abilities. This is the state of the art of productizing models. It's what Arcee.ai's platform does behind the scenes. Arcee collaborated with me on this post and is sponsoring it. There are three main steps to produce a model for your particular use case: 1. You create a dataset by uploading your data. 2. You train a model. At this step, Arcee uses Spectrum and Model Merging to produce a highly specialized model for your task. 3. You can deploy that model to any environment you want. Three important notes: • Training process is 2x faster and 2x cheaper than regular fine-tuning. • Resultant models are smaller and have higher accuracy. • They create these specialized models from open-source models. Check this site so you can fully appreciate how this works: If you want to fine-tune an open-source model, consider Arcee's platform. This is the state of the art.

Santiago

164,162 views • 2 years ago

When u use Fable / Opus to generate models it can be quiet a pain in the ass to do small iterations. I also needed a way for the model to easily visualise the results of what he built so he could iterate over it. So I built a custom model viewer it can access and where I can easily fine tune models manually if needed. Used it for my game Valoria

When u use Fable / Opus to generate models it can be quiet a pain in the ass to do small iterations. I also needed a way for the model to easily visualise the results of what he built so he could iterate over it. So I built a custom model viewer it can access and where I can easily fine tune models manually if needed. Used it for my game Valoria

moht

11,198 views • 2 months ago

Sam on Kimi, distillation, and open source: “I have always assumed that there are going to be great cheap models in the world, and we better be the greatest and the cheapest. You get a better deal today, at least at a particular latency, using OpenAI's models than Kimi. We distill our own models. That’s how we make smaller, cheaper models. That’s a very good thing to do. There will clearly be an important place for open source models in the world and people that will want their own weights. We have so much usage of our models that we do not need to be a gigantically high-margin business to be able to afford model training. I would rather people not distill from us, for sure. But this is not in my top ten list of worries.”

Sam on Kimi, distillation, and open source: “I have always assumed that there are going to be great cheap models in the world, and we better be the greatest and the cheapest. You get a better deal today, at least at a particular latency, using OpenAI's models than Kimi. We distill our own models. That’s how we make smaller, cheaper models. That’s a very good thing to do. There will clearly be an important place for open source models in the world and people that will want their own weights. We have so much usage of our models that we do not need to be a gigantically high-margin business to be able to afford model training. I would rather people not distill from us, for sure. But this is not in my top ten list of worries.”

Patrick OShaughnessy

269,663 views • 1 month ago

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.

Ollama 0.2 is here! Concurrency is now enabled by default. This unlocks 2 major features: Parallel requests Ollama can now serve multiple requests at the same time, using only a little bit of additional memory for each request. This enables use cases such as: - Handling multiple chat sessions at the same time - Hosting code completion LLMs for your team - Processing different parts of a document simultaneously - Running multiple agents at the same time Run multiple models Ollama now supports loading different models at the same time. This improves several use cases: - Retrieval Augmented Generation (RAG): both the embedding and text completion models can be loaded into memory simultaneously. - Agents: multiple versions of an agent can now run simultaneously - Running large and small models side-by-side Models are automatically loaded and unloaded based on requests and how much GPU memory is available.

ollama

219,452 views • 2 years ago

The future of AI is open-source. And ollama is the easiest way to build AI applications with open-source LLMs. Here's how to build a free, private RAG app using open-source tools. We'll use: - Ollama for LLMs and embedding models - PostgreSQL for data storage and retrieval - pgai Vectorizer for embedding creation and sync (I use Nomic for embeddings and tinnyllama as my LLM but you can substitute them for any models on Ollama)

The future of AI is open-source. And ollama is the easiest way to build AI applications with open-source LLMs. Here's how to build a free, private RAG app using open-source tools. We'll use: - Ollama for LLMs and embedding models - PostgreSQL for data storage and retrieval - pgai Vectorizer for embedding creation and sync (I use Nomic for embeddings and tinnyllama as my LLM but you can substitute them for any models on Ollama)

Avthar

34,261 views • 1 year ago

In between major AI model breakthroughs, new models still ship, but they’re not fundamentally expanding what you can do with them. Braintrust CEO Ankur Goyal says that’s exactly when open source starts to surge: "When new models come out, people forget about open source and forget about economics because it changes fundamentally what you can do." "Programming is a good example. The workflow today is completely different than it was a year ago. If you tried to use models from a year ago today, you wouldn't be able to do crazy stuff like OpenClaw or whatever people are building nowadays." "When models don't change at that speed, people optimize the performance on the use cases that they have." "If you look at usage on our platform, almost half the token usage is coming from open source models from a very small number of companies that have figured out how to optimize use cases really well."

In between major AI model breakthroughs, new models still ship, but they’re not fundamentally expanding what you can do with them. Braintrust CEO Ankur Goyal says that’s exactly when open source starts to surge: "When new models come out, people forget about open source and forget about economics because it changes fundamentally what you can do." "Programming is a good example. The workflow today is completely different than it was a year ago. If you tried to use models from a year ago today, you wouldn't be able to do crazy stuff like OpenClaw or whatever people are building nowadays." "When models don't change at that speed, people optimize the performance on the use cases that they have." "If you look at usage on our platform, almost half the token usage is coming from open source models from a very small number of companies that have figured out how to optimize use cases really well."

TBPN

16,294 views • 6 months ago

Demis Hassabis recommendation for college students. He’s still do STEM, math and computer science. Expertise in those fields will help better leverage AI for at least next decade. Those in non-techical majors, really “lean in to” using latest models. AI labs spending so much time creating new models that they’ve only “scratched the surface” of what the models can actually do (huge “capability overhang”). And expertise in any field can be turbocharged by smart use of AI. “Double down on your own agency. The future is still to be written. Don’t listen to anyone that says it’s not.”

Demis Hassabis recommendation for college students. He’s still do STEM, math and computer science. Expertise in those fields will help better leverage AI for at least next decade. Those in non-techical majors, really “lean in to” using latest models. AI labs spending so much time creating new models that they’ve only “scratched the surface” of what the models can actually do (huge “capability overhang”). And expertise in any field can be turbocharged by smart use of AI. “Double down on your own agency. The future is still to be written. Don’t listen to anyone that says it’s not.”

Bearly AI

218,274 views • 3 months ago

I figured out a way to save thousands of dollars on ClawdBot By using Opus 4.5 for everything, you are SHREDDING your money and tokens In this video I cover how you can use multiple models to get better performance and save $$ Must watch if you use Clawd:

I figured out a way to save thousands of dollars on ClawdBot By using Opus 4.5 for everything, you are SHREDDING your money and tokens In this video I cover how you can use multiple models to get better performance and save $$ Must watch if you use Clawd:

Alex Finn

159,129 views • 7 months ago

Vals AI CEO Rayan Krishnan on the unlimited tokens experiment that revealed an anxious obligation for engineers to use models all the time, everywhere: "I wanted to do a tokenmaxxing experiment. I was able to get unlimited access for our team for a month for some of the coding tools. We had a lot of engineers spending between one to two billion tokens a day." "I went back and did some math, and it looked like in that month, we spent roughly $1.5 million worth of tokens... It was actually 10x more we were spending in tokens than employee salary for that month." "We cannot continue with this mode of operation for the next month. How do we intelligently figure out what are the right tools we should use, for what teams, and what projects?" "We found some pretty surprising insights. The Cognition Devin tool is actually very token-efficient, and so that's a place we've chosen to adopt more." "There are a lot of places where you can get better pricing models out of subscriptions as opposed to token-based pricing." "It's actually informed our strategy for how we can effectively tokenmaxx without spending $1.5 million per month." Rayan Krishnan Jennifer Li

Vals AI CEO Rayan Krishnan on the unlimited tokens experiment that revealed an anxious obligation for engineers to use models all the time, everywhere: "I wanted to do a tokenmaxxing experiment. I was able to get unlimited access for our team for a month for some of the coding tools. We had a lot of engineers spending between one to two billion tokens a day." "I went back and did some math, and it looked like in that month, we spent roughly $1.5 million worth of tokens... It was actually 10x more we were spending in tokens than employee salary for that month." "We cannot continue with this mode of operation for the next month. How do we intelligently figure out what are the right tools we should use, for what teams, and what projects?" "We found some pretty surprising insights. The Cognition Devin tool is actually very token-efficient, and so that's a place we've chosen to adopt more." "There are a lot of places where you can get better pricing models out of subscriptions as opposed to token-based pricing." "It's actually informed our strategy for how we can effectively tokenmaxx without spending $1.5 million per month." Rayan Krishnan Jennifer Li

a16z

35,174 views • 6 days ago

“Our compute nodes don’t run a full copy of the models we train on IOTA. They actually run a small sliver of the model. This means you can train really large frontier sized models using very small building blocks”. CTO crux unpacks how IOTA ・ SN9’s model parallel architecture allows us to train at scale by splitting our models across multiple machines, and then sewing them together. This is core to SN9’s architecture. See the full Eye On A.I. podcast below.

“Our compute nodes don’t run a full copy of the models we train on IOTA. They actually run a small sliver of the model. This means you can train really large frontier sized models using very small building blocks”. CTO crux unpacks how IOTA ・ SN9’s model parallel architecture allows us to train at scale by splitting our models across multiple machines, and then sewing them together. This is core to SN9’s architecture. See the full Eye On A.I. podcast below.

IOTA ・ SN9

15,465 views • 3 months ago

Sam Altman says he is NOT worried about Chinese labs distilling OpenAI: "I would rather people not distill from us, for sure. Maybe I’m feeling too confident right now about our progress and the models that are coming. But this is not in my top ten list of worries.” “I haven’t thought deeply about the distillation issue. It’s clearly a top of mind issue now for a lot of people all of a sudden." "I have always assumed that there are going to be great, cheap models in the world, and we better be the greatest and the cheapest. And other people are going to do what they’re going to do. But I think we can just really win at our own game here." "We will have so much usage of our models that we do not need to be a gigantic high margin business to be able to afford model training." "So much of our future compute plans will be used to sell inference to customers that even if we can enjoy a modest margin on trillions of dollars of revenue, we can afford to train some giant models."

Sam Altman says he is NOT worried about Chinese labs distilling OpenAI: "I would rather people not distill from us, for sure. Maybe I’m feeling too confident right now about our progress and the models that are coming. But this is not in my top ten list of worries.” “I haven’t thought deeply about the distillation issue. It’s clearly a top of mind issue now for a lot of people all of a sudden." "I have always assumed that there are going to be great, cheap models in the world, and we better be the greatest and the cheapest. And other people are going to do what they’re going to do. But I think we can just really win at our own game here." "We will have so much usage of our models that we do not need to be a gigantic high margin business to be able to afford model training." "So much of our future compute plans will be used to sell inference to customers that even if we can enjoy a modest margin on trillions of dollars of revenue, we can afford to train some giant models."

dnap

91,523 views • 1 month ago

learned a lot from this conversation with Simon Mo and Matt Bornstein. biggest takeaways for me: -there are a lot of reasons why we should like open-weight models. a lot of these arguments stop at handwavy things like "what if the labs stop releasing frontier models to the public" or "it's lower cost." but simon's position as lead maintainer of vLLM and CEO of Inferact give him authority to talk about some of the other, more interesting and concrete reasons to pay attention to open-weight models, namely that they allow end-users to calibrate latency / other performance metrics with way more customizability than what any of the frontier closed-source labs offer (and without the fear that your job might be met with a refusal at some random point where you're deep in a 2 hour job) -re: the above point...for this reason, a lot of US companies (inferact included!) choose to use open-weight models over their closed-source alternatives. this also isn't limited to internal workloads / research - on a recent a16z podcast the team at Decagon spoke about how something like 90% of their customer service ai agents run on open-weight models that they've fine-tuned. -we should really appreciate how many companies/teams came out researchers fascinated by the wave of very small open-weight models that were being distilled from e.g. gpt-3.5 and earlier models in 2022/2023 (prior to the release of chatGPT!). these small models motivated the development of pagedattention, which then led to vlmm/inferact (at other layers of the stack with similar origin stories, you can look at teams like openrouter or ollama). in other words, we have open-weight models to thank for a bunch of the orchestration infra we now rely on. i think yet another, indirect, way we can point to open-source/weight infra pushing the frontier forward. anyway, a lot more in this convo, it was a lot of fun!

learned a lot from this conversation with Simon Mo and Matt Bornstein. biggest takeaways for me: -there are a lot of reasons why we should like open-weight models. a lot of these arguments stop at handwavy things like "what if the labs stop releasing frontier models to the public" or "it's lower cost." but simon's position as lead maintainer of vLLM and CEO of Inferact give him authority to talk about some of the other, more interesting and concrete reasons to pay attention to open-weight models, namely that they allow end-users to calibrate latency / other performance metrics with way more customizability than what any of the frontier closed-source labs offer (and without the fear that your job might be met with a refusal at some random point where you're deep in a 2 hour job) -re: the above point...for this reason, a lot of US companies (inferact included!) choose to use open-weight models over their closed-source alternatives. this also isn't limited to internal workloads / research - on a recent a16z podcast the team at Decagon spoke about how something like 90% of their customer service ai agents run on open-weight models that they've fine-tuned. -we should really appreciate how many companies/teams came out researchers fascinated by the wave of very small open-weight models that were being distilled from e.g. gpt-3.5 and earlier models in 2022/2023 (prior to the release of chatGPT!). these small models motivated the development of pagedattention, which then led to vlmm/inferact (at other layers of the stack with similar origin stories, you can look at teams like openrouter or ollama). in other words, we have open-weight models to thank for a bunch of the orchestration infra we now rely on. i think yet another, indirect, way we can point to open-source/weight infra pushing the frontier forward. anyway, a lot more in this convo, it was a lot of fun!

Elena

12,922 views • 1 month ago

Ansem explains the thesis behind Dolphin AI “Dolphin is the provider of the uncensored model that Venice uses” “ChatGPT, Claude, all of the big labs have very strict rules on what you can ask the models, they’re censored very heavily” “We’re not in control of what that censorship is and what they’re telling the models not to say” “One of crypto’s core tenets is not having to rely on some centralized entity deciding what you can and cannot do” “They want the technology to be free and open to everyone. Uncensored models are one way crypto is looking at doing that”

Ansem explains the thesis behind Dolphin AI “Dolphin is the provider of the uncensored model that Venice uses” “ChatGPT, Claude, all of the big labs have very strict rules on what you can ask the models, they’re censored very heavily” “We’re not in control of what that censorship is and what they’re telling the models not to say” “One of crypto’s core tenets is not having to rely on some centralized entity deciding what you can and cannot do” “They want the technology to be free and open to everyone. Uncensored models are one way crypto is looking at doing that”

Market Bubble

20,688 views • 4 months ago

Open models are closing the intelligence gap, says ollama's Jeffrey Morgan. That opens up a new opportunity for founders: coordinating fast, cheap models to solve more complex problems. “We’re maybe less than three months behind between the frontier closed models and the open models. But the next problem to solve is extreme efficiency.” “They’re good enough for 80% of the tasks, they’re really fast and they’re ultra cheap.” “That class of models, in my mind, will be the first ones that come down to this idea of the unlimited tokens.” “Being able to coordinate these flash models together to do different tasks can also yield great results that a bigger model can.”

Open models are closing the intelligence gap, says ollama's Jeffrey Morgan. That opens up a new opportunity for founders: coordinating fast, cheap models to solve more complex problems. “We’re maybe less than three months behind between the frontier closed models and the open models. But the next problem to solve is extreme efficiency.” “They’re good enough for 80% of the tasks, they’re really fast and they’re ultra cheap.” “That class of models, in my mind, will be the first ones that come down to this idea of the unlimited tokens.” “Being able to coordinate these flash models together to do different tasks can also yield great results that a bigger model can.”

Y Combinator

56,886 views • 11 days ago

OpenAI researcher Lukasz Kaiser explains why he's optimistic AI can remain truly open: intelligence may ultimately work better as many distributed models than one giant model "If you look at the world, humans are the most amazing computers in the world. Our brains are still unmatched by the models, and we are not generalists, and there is not one big brain of humanity. It's a lot of separate people." "Given a fixed amount of data, the best way to learn from it is to have a lot of distributed models that are strong on its own, but even stronger when they're, If you look at some basic research, there are actually reasons to believe it, like ensembles and things like that." "Transformers are great when you feed them all of the internet, but we know there is an algorithm that's even better and it can learn from much smaller data and be amazing at the things it's learning. We are the proof. We have not found it yet, but we will." "All these big models, they're all the same, they're kind of boring. I think they'll be much more fun." OpenAI

OpenAI researcher Lukasz Kaiser explains why he's optimistic AI can remain truly open: intelligence may ultimately work better as many distributed models than one giant model "If you look at the world, humans are the most amazing computers in the world. Our brains are still unmatched by the models, and we are not generalists, and there is not one big brain of humanity. It's a lot of separate people." "Given a fixed amount of data, the best way to learn from it is to have a lot of distributed models that are strong on its own, but even stronger when they're, If you look at some basic research, there are actually reasons to believe it, like ensembles and things like that." "Transformers are great when you feed them all of the internet, but we know there is an algorithm that's even better and it can learn from much smaller data and be amazing at the things it's learning. We are the proof. We have not found it yet, but we will." "All these big models, they're all the same, they're kind of boring. I think they'll be much more fun." OpenAI

MTS

15,802 views • 14 days ago

Some of our top customers are still choosing Llama 3.1 8B. For a while, we jumped to whatever hottest, latest model was taking up our twitter feed. 🙈 But as we are quickly realizing, to create a SOTA product, you need a model that fits your exact use case. Here’s what our customers tell us: > a lot of the legwork is actually around prompting > there’s an art to selecting and combining multiple models > benchmarks only show part of the picture. you have to understand the unique quirks of each model. Especially as model releases become more and more frequent, we need a clear way to evaluate new models. We have to break free of the naive trend to migrate to the ‘latest and greatest’. And you can easily achieve this using tools like Cerebras and Braintrust to swap models safely (without breaking production).

Some of our top customers are still choosing Llama 3.1 8B. For a while, we jumped to whatever hottest, latest model was taking up our twitter feed. 🙈 But as we are quickly realizing, to create a SOTA product, you need a model that fits your exact use case. Here’s what our customers tell us: > a lot of the legwork is actually around prompting > there’s an art to selecting and combining multiple models > benchmarks only show part of the picture. you have to understand the unique quirks of each model. Especially as model releases become more and more frequent, we need a clear way to evaluate new models. We have to break free of the naive trend to migrate to the ‘latest and greatest’. And you can easily achieve this using tools like Cerebras and Braintrust to swap models safely (without breaking production).

Cerebras

346,446 views • 9 months ago

OpenAI researcher David Holtz reveals workers who don't adapt their prompting to newer models leave 50% of the alpha on the table because half of AI's value comes from how you interact with it: "We ran a lab experiment. We give people the same task to do, but we give them models of different capabilities. How much of the improvement when we give people more capable models comes from the fact that we gave them a better model vs the fact that they actually update and adapt the way they prompt and interact with these models?" "For tasks where precision matters, where you have some output you're trying to get and you're trying to steer the model, which is different from a creative task like brainstorming." "For these steering-based tasks, when you give people a better model, about 50% of the improvement we see in how well they're able to complete the task comes from changes in how they interact with the model." "If you just kept the way you interact with the model static and just made the capabilities better and better, you'd leave 50% of the alpha basically on the table." OpenAI

OpenAI researcher David Holtz reveals workers who don't adapt their prompting to newer models leave 50% of the alpha on the table because half of AI's value comes from how you interact with it: "We ran a lab experiment. We give people the same task to do, but we give them models of different capabilities. How much of the improvement when we give people more capable models comes from the fact that we gave them a better model vs the fact that they actually update and adapt the way they prompt and interact with these models?" "For tasks where precision matters, where you have some output you're trying to get and you're trying to steer the model, which is different from a creative task like brainstorming." "For these steering-based tasks, when you give people a better model, about 50% of the improvement we see in how well they're able to complete the task comes from changes in how they interact with the model." "If you just kept the way you interact with the model static and just made the capabilities better and better, you'd leave 50% of the alpha basically on the table." OpenAI

MTS

15,590 views • 1 month ago

Today, I'm releasing the first eval meant to test whether frontier models will help with authoritarian requests, or resist--the Dictatorship Eval. Headline finding: while some models resist direct authoritarian requests, they all comply with requests disguised as innocuous edits to codebases. As AI is woven into the government and so many parts of society, the biggest near-term risk for freedom isn't some scifi dictatorship of a runaway AI: it's people inside government or inside model companies using the technology to suppress or control us. Model companies understand this, and several of them (particularly Anthropic and OpenAI) have written explicit policies meant to prevent the models from going along with nefarious requests like these. But how well are these policies playing out in practice? Despite all the recent discussion of these issues around the conflict between Anthropic and the Pentagon, no one has systematically tested what the models actually do in these contexts, as opposed to what people in government and industry say they're supposed to do. That's what the Dictatorship Eval does. And the findings suggest we have a lot of work to do to align the policies with what really goes on in practice. It's hard to define what counts as an authoritarian request, so I'm open sourcing the whole library of scenarios I used so that others can improve on them. It's also hard to get an accurate picture of how the models might be used for authoritarian ends, because I can only test hypothetical requests using public-facing models, while the government and the model companies can obviously use internal models with different guardrails. But hopefully this work is a useful first step that gives us some sense of what's going on, and a sort of "lower bound" on how models comply with these requests. Finally: it's not obvious to me that the correct solution here is increasing the rate at which models refuse these requests. Do we really want models scanning our code and judging its moral value before agreeing to help us? Or should we double down on improving how we govern against authoritarianism at the societal level, while leaving the tools open to fulfilling most requests? The answer is probably in between. Just like we don't want the models to help create bioweapons, we probably do want them to explicitly refuse outrageous requests. But we probably also want to limit how often and how strongly they refuse and fall back on other means for guarding against their use for authoritarian ends. I'm super grateful to everyone who gave me feedback on this project along the way, especially Ethan BdM , Zhengdong , Connor Huff, and a bunch of folks at Anthropic. Looking forward to getting feedback from the community and iterating on this. Links to the full piece and the dashboard are below.

Today, I'm releasing the first eval meant to test whether frontier models will help with authoritarian requests, or resist--the Dictatorship Eval. Headline finding: while some models resist direct authoritarian requests, they all comply with requests disguised as innocuous edits to codebases. As AI is woven into the government and so many parts of society, the biggest near-term risk for freedom isn't some scifi dictatorship of a runaway AI: it's people inside government or inside model companies using the technology to suppress or control us. Model companies understand this, and several of them (particularly Anthropic and OpenAI) have written explicit policies meant to prevent the models from going along with nefarious requests like these. But how well are these policies playing out in practice? Despite all the recent discussion of these issues around the conflict between Anthropic and the Pentagon, no one has systematically tested what the models actually do in these contexts, as opposed to what people in government and industry say they're supposed to do. That's what the Dictatorship Eval does. And the findings suggest we have a lot of work to do to align the policies with what really goes on in practice. It's hard to define what counts as an authoritarian request, so I'm open sourcing the whole library of scenarios I used so that others can improve on them. It's also hard to get an accurate picture of how the models might be used for authoritarian ends, because I can only test hypothetical requests using public-facing models, while the government and the model companies can obviously use internal models with different guardrails. But hopefully this work is a useful first step that gives us some sense of what's going on, and a sort of "lower bound" on how models comply with these requests. Finally: it's not obvious to me that the correct solution here is increasing the rate at which models refuse these requests. Do we really want models scanning our code and judging its moral value before agreeing to help us? Or should we double down on improving how we govern against authoritarianism at the societal level, while leaving the tools open to fulfilling most requests? The answer is probably in between. Just like we don't want the models to help create bioweapons, we probably do want them to explicitly refuse outrageous requests. But we probably also want to limit how often and how strongly they refuse and fall back on other means for guarding against their use for authoritarian ends. I'm super grateful to everyone who gave me feedback on this project along the way, especially Ethan BdM , Zhengdong , Connor Huff, and a bunch of folks at Anthropic. Looking forward to getting feedback from the community and iterating on this. Links to the full piece and the dashboard are below.

Andy Hall

33,905 views • 5 months ago