Safety-oriented interpretability researchers should be focused on AI systems, not individual model…
@ChrisGPotts17K followers
16,512 views
6 Comments
Comments from the original post on X (Twitter)

@iamabdelhadi02
Hello Prof. Potts, you often emphasize compound systems—how challenging are they to interpret mechanistically?
1 like

@ChrisGPotts
@AlgerianAb26687 I think they will be challenging to interpret because they have all the complexity of traditional software and all the complexity of foundation models.
1 like

@AJakkli
I'm confused why would these systems be fundamentally that different from individual model artifacts? And also aren't systems being studied via creating tool calling envs?
1 like

@ChrisGPotts
We have an overview of the differences between models and compound AI systems here: My concern is specifically about what interp researchers are doing; I know security folks and others are focused on the nature and integrity of compound AI systems.
2 likes

@AJakkli
read the article - do you have examples of interp work that could be done on a compound system?

@codewithimanshu
Christopher is right, focusing on the AI system is crucial for safety, not just individual components.