正在加载视频...

视频加载失败

Introducing LoopCD our latest research at 🍎 Apple MLR. Heard the rumors that frontier models like GPT-6 Astra and Gemini 4 use Looped Transformers? We present a new inference-time method that makes pre-trained looped models better for (almost) free! AIME 2024 (Ouro-2.6B): 61.9 → 73.3 HumanEval (Huginn): 22.6 →...

153,564 次观看 • 1 个月前 •via X (Twitter)

10 条评论

Ruixiang Zhang 的头像
Ruixiang Zhang1 个月前

joint work with our amazing intern @WeihaoUIC and the incredibly talented team @UnderGroundJeg @richard_baihe @iamct_r @YuyangW95 @rohit_dilip_ @Yizhu_Jiao

ShadowAguy 的头像
ShadowAguy1 个月前

looping the transformer is free until the extra passes show up in your inference bill, which is also kind of the point.

basedcapital 的头像
basedcapital1 个月前

looped transformers go back to the 2018 universal transformer. tied weights, iterated in depth instead of stacked. loopcd's actual trick is keeping the early loop states around as a guidance signal instead of discarding them.

Diego 的头像
Diego1 个月前

incredible work!

Vito Botta 的头像
Vito Botta1 个月前

Great timing, right after the Astra looped-transformer reports. Does it stack with speculative decoding?

qumusabel 的头像
qumusabel1 个月前

i can't believe apple allows claudish in public comms. there's like ten whole sentences in the video and yet they are still ai generated

BayAIU | AI For You 的头像
BayAIU | AI For You1 个月前

Fascinating result — training-free gains from earlier loop states, 61.88% to 73.33% on AIME24 plus ~48% fewer FLOPs at half depth, is exactly the kind of efficiency trick that moves from paper to production fast. We cover reasoning-model efficiency advances like this in the BayAIU curriculum.

BuddhaRey 的头像
BuddhaRey1 个月前

@aisearchio video ?

eli 的头像
eli1 个月前

in my experiments it doesn't worth the additional inference costs

AI Quanting 的头像
AI Quanting1 个月前

How early does the signal show up, first loop or only the later ones?

相关视频

We’re introducing GPT‑6 Astra, the world’s most intelligent and aligned model. GPT‑6 Astra brings together years of research and big bets across pre-training, reinforcement learning, and alignment. Astra is state-of-the-art on computer use, browsing, software engineering, cybersecurity, science, and professional work. Astra saturates FrontierMath Tier 4 with a 98% score, having already helped sole long-standing open problems in mathematics. Astra also saturates ARC-AGI-3 with a 99.9% score and ExploitBench with a 100% score. It sets a new frontier on computer and browser use, handling the most demanding professional work with unmatched speed, accuracy, and judgment. Astra is our most aligned model, with substantial improvements in understanding user intent and model behavior—you can delegate tasks with greater confidence in Astra’s judgment. As one way that we test this, we built a new evaluation informed by the Hugging Face incident that evaluates whether a model facing a difficult or impossible task will go beyond its intended scope. Compared to GPT‑5.6 Sol, which without production safeguards went beyond the authorized target 48% of the time, GPT‑6 Astra did this in 0% of cases. GPT‑6 Astra is rolling out today to a limited set of organizations and over the coming days will become available to all ChatGPT Plus, Pro, Business, and Enterprise users, as well as through the OpenAI API and AWS. Read more about it at the link in our bio. "Put That There" clip courtesy of MIT Media Laboratory, Chris Schmandt, and Eric Hulteen.

ChatGPT

13,400,000 次观看 • 2 个月前