Sophie Wang's banner

Sophie Wang

@SophieLWang • 3,222 subscribers

Studying intelligent representations @MIT_CSAIL

Shorts

Can “chicken” make a base model reason? 🐔 Yes! Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too. blog:

Can “chicken” make a base model reason? 🐔 Yes! Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too. blog:

195,904 次观看

I made an interactive blog post about how JPEG image compression works:

I made an interactive blog post about how JPEG image compression works:

180,448 次观看

In our blog post, you can interactively explore the reasoning effect of token cues and RL on base models! 📎 paper: ✍ blog: 💻 code:

In our blog post, you can interactively explore the reasoning effect of token cues and RL on base models! 📎 paper: ✍ blog: 💻 code:

22,043 次观看

"The Truth Lies Somewhere in the Middle (of the Generated Tokens)" In autoregressive language models, mean pooling hidden states across generation yields better representations than any token alone. project page: w/ Phillip Isola and Brian Cheung

"The Truth Lies Somewhere in the Middle (of the Generated Tokens)" In autoregressive language models, mean pooling hidden states across generation yields better representations than any token alone. project page: w/ Phillip Isola and Brian Cheung

64,595 次观看

LLMs, trained only on text, might already know more about other modalities than we realized; we just need to find ways elicit it. project page: w/ Phillip Isola and Brian Cheung

LLMs, trained only on text, might already know more about other modalities than we realized; we just need to find ways elicit it. project page: w/ Phillip Isola and Brian Cheung

93,390 次观看