Understanding OpenAI o1: Noam Brown on integrating reasoning into the model. Takeaways: - Avoid MCTS and current paradigm of using processes outside of the model during inference - Think about how to directly integrate reasoning into the model architecture

This is why entropix or Quiet-STaR represents as an interesting way to directly build in better reasoning.

My take is that online planning is integrated at the model level. On top of that, they have a unique RL technique that truly enables the model to reason. I think @natolambert has some good posts on the RL in o1, also has a high-level blog:

He didnt say to avoid it he said to go broader then just mcts

Avoid MCTS was kind of inferred from the context. In general, I think what Noam is trying to convey is to think outside of the easy to-go-to methods like MCTS. Instead, think about it differently - how can we directly integrate search into the model?

Let me repeat:

Nowhere did he imply to avoid MCTS…

Did he mention to avoid MCTS?

Yes, in the 1 minute clip above, he mentions it at the end

I don’t think he mean avoid MCTS

I provided some clarification here on what I meant. Emphasis on model-level search.