正在加载视频...
视频加载失败
Introducing LoopCD our latest research at 🍎 Apple MLR. Heard the rumors that frontier models like GPT-6 Astra and Gemini 4 use Looped Transformers? We present a new inference-time method that makes pre-trained looped models better for (almost) free! AIME 2024 (Ouro-2.6B): 61.9 → 73.3 HumanEval (Huginn): 22.6 →... show more
153,564 次观看 • 1 个月前 •via X (Twitter)
10 条评论

joint work with our amazing intern @WeihaoUIC and the incredibly talented team @UnderGroundJeg @richard_baihe @iamct_r @YuyangW95 @rohit_dilip_ @Yizhu_Jiao

looping the transformer is free until the extra passes show up in your inference bill, which is also kind of the point.

looped transformers go back to the 2018 universal transformer. tied weights, iterated in depth instead of stacked. loopcd's actual trick is keeping the early loop states around as a guidance signal instead of discarding them.

incredible work!

Great timing, right after the Astra looped-transformer reports. Does it stack with speculative decoding?

i can't believe apple allows claudish in public comms. there's like ten whole sentences in the video and yet they are still ai generated

Fascinating result — training-free gains from earlier loop states, 61.88% to 73.33% on AIME24 plus ~48% fewer FLOPs at half depth, is exactly the kind of efficiency trick that moves from paper to production fast. We cover reasoning-model efficiency advances like this in the BayAIU curriculum.

@aisearchio video ?

in my experiments it doesn't worth the additional inference costs

How early does the signal show up, first loop or only the later ones?


