Skip navigation
Safety-oriented interpretability researchers should be focused on AI systems, not individual model…
0:00 / 0:37

Safety-oriented interpretability researchers should be focused on AI systems, not individual model…

Christopher Potts

@ChrisGPotts17K followers

Visit on X
166 likesDownloadOriginal post

16,512 views

Safety-oriented interpretability researchers should be focused on AI systems, not individual model artifacts. A snippet from the NeurIPS CogInterp workshop panel on Sunday:

6 Comments

Comments from the original post on X (Twitter)

  • ‏عبد الهادي‏'s profile picture

    @iamabdelhadi02

    Hello Prof. Potts, you often emphasize compound systems—how challenging are they to interpret mechanistically?

    1 like

  • Christopher Potts's profile picture

    @ChrisGPotts

    @AlgerianAb26687 I think they will be challenging to interpret because they have all the complexity of traditional software and all the complexity of foundation models.

    1 like

  • arya's profile picture

    @AJakkli

    I'm confused why would these systems be fundamentally that different from individual model artifacts? And also aren't systems being studied via creating tool calling envs?

    1 like

  • Christopher Potts's profile picture

    @ChrisGPotts

    We have an overview of the differences between models and compound AI systems here: My concern is specifically about what interp researchers are doing; I know security folks and others are focused on the nature and integrity of compound AI systems.

    2 likes

  • arya's profile picture

    @AJakkli

    read the article - do you have examples of interp work that could be done on a compound system?

  • Himanshu Kumar's profile picture

    @codewithimanshu

    Christopher is right, focusing on the AI system is crucial for safety, not just individual components.