Пропустить навигацию
Evolution happens to populations.
0:00 / 0:45

Evolution happens to populations.

Zhuokai Zhao

@zhuokaiz6 тыс. подписчиков

Открыть в X
219 отметок «Нравится»СкачатьОригинальный пост

925 744 просмотра

Evolution happens to populations. Useful genes become common in a group over generations, then spread to other groups that test them in their own environments. The speed depends on generation time, 20 minutes for a bacterium but about 25 years for an elephant. Using LLMs to evolve better models often looks like an isolated herd of elephants. Each search runs alone on one dataset, and for large models every generation costs a long training run on the full data. Our new paper, "Discovered, Not Designed," introduces Population Evolution. It has two goals, to discover better models and to discover them faster. To discover better model designs, the searches share what they find. Several searches run at the same time, each on its own dataset. When one finds a code change that helps, the same change is retrained on the others. While nature keeps no record of the genes that failed to help, Population Evolution does. All peer results, including weak or unsuccessful transfers, are added to a shared ledger, and the LLM reads that ledger before it writes the next change. To discover them faster, the searches do not train on the full data, because that is what makes each generation slow. We found that a search can run on as little as 2% or 5% of the data and still find changes that hold up in full training, as long as the other searches check each change first. In Population Evolution, a change that works on one slice must also work on at least two other slices, then on two larger 10% sets. Only then does it get a full training run. We tested Population Evolution on RMD-Bench, a benchmark we also introduce for evaluating model discovery. It covers five task families (LLM pretraining, VLM pretraining, RL algorithm discovery, ranking, and watch-time prediction) with 34 instances in total. Each one comes with a reference implementation and a fixed training and evaluation protocol, so a change found on one dataset or data scale can be retrained and scored on another, and other discovery methods can be compared under the same conditions. The numbers below are a sneak peek at the two most expensive families of experiments. The LLM pretraining task starts from a 185M-parameter Gemma 3-style model, and the VLM pretraining task starts from a Qwen3.5-style model with about 646M trainable parameters. The baseline runs the same kind of search directly on the full data, at the same total GPU compute. Population Evolution finds better models. The best relative accuracy gain for LLM pretraining goes from 0.92% to 2.48%. The best loss reduction for VLM pretraining goes from 5.05% to 8.78%. Population Evolution finds them faster. In LLM pretraining, each search gets 100 generations where a full-data search gets 14. In VLM pretraining, 57 of 61 full training runs met the success threshold, against 12 of 100. Evolution works because many groups pass along what helps and get many generations to do it, and Population Evolution gives LLM-driven model search these prerequisites. Read our paper at for full implementation details and results. The benchmark data and code will be made publicly available shortly.

15 комментариев

Комментарии из оригинального поста в X (Twitter)

  • Фото профиля Zhuokai Zhao

    @zhuokaiz

    Amazing work led by @BoPeng8eyj, @LizhuZhang and the rest of the team @YuhangZhou2, @RachMinyee, @yifannnwu 🫡

    3 отметки «Нравится»

  • Фото профиля Cuong Nguyen

    @cuongpb139

    Sounds like a cool way to speed things up

  • Фото профиля Kien Trinh

    @Nguyenn259373

    Mình thích cách nó nhắc đến vi khuẩn chỉ 20 phút một thế hệ, nghe vừa đơn giản vừa thú vị 🌱

  • Фото профиля Kien Trinh

    @Nguyenn259373

    Nghe thú vị thật, 20 phút một thế hệ thì vi khuẩn tiến hóa nhanh đến mức khó tin luôn 🦠

  • Фото профиля Tooby

    @TDuy240201

    That generation time gap is insane, bacteria really get to test stuff on fast forward

  • Фото профиля UR Noyon

    @urnoyon_ofc

    yep, generation time is the real throttle on evolution dynamics

  • Фото профиля Zarroc.BTC 🧠

    @HuevaToi

    the generation time example makes it feel real not magic

  • Фото профиля Ariel Lillie

    @ariell_xyz

    generation time like 25 years for elephants really slows things down

  • Фото профиля Hunt Austin

    @MuhammedAs18748

    The evolution of climate change modeling is shaped by breakthroughs in research, market demand, and infrastructure maturity in emerging economies.

  • Фото профиля Rose Phamm

    @RosePhamm2002

    20 phút cho một vi khuẩn thật nhanh! 😊

  • Фото профиля Eric_kang

    @BeatAPIio

    57 of 61 vs 12 of 100 at matched compute 🤯 Do wins found on the 2% slices ever fail to carry over at full scale?

  • Фото профиля Rose Phamm

    @RosePhamm2002

    20 phút cho một vi khuẩn thật thú vị 😊

  • Фото профиля Jack

    @JackN1x

    interesting take. nature's own A/B testing?

  • Фото профиля bernardo reyes

    @onesimus_prime1

    crazy theory

  • Фото профиля Kelvin Tran

    @KevinTrann86

    Nghe đơn giản mà sâu sắc thật, 20 phút một thế hệ ở vi khuẩn thì tiến hóa diễn ra nhanh đến khó tin 🌱