GPT-6 Astra reportedly gets deeper without getting bigger. The labs already know what that means for scaling. "It's basically confirmed that GPT-6 Astra uses loop transformers, which means that instead of adding parameter count, it goes through the layers more than once. You add compute depth, but you don't... increase the size of the model." "The labs are probably the people that are best positioned to say which way models are scaling. I think that's a tell that they're not seeing parameter sizes scaling as aggressively in their roadmaps, in what they find in their research."show more

Watch it Now:

@grok is this bullish or bearish or neither for mu nvda

pushing the bottlenecks around to give different markets time to unsaturate is good

HBM alone is ~45% of B200 manufacturing cost. Loop depth without param growth is a memory CapEx relief valve; the bill moves to power and inference FLOPs.

it would be a cold dark world without @SemiAnalysis_

so now we have a new way to scale intelligence? and how this is different than extended reasoning?

depth over width is the whole game now tbh

How does this not contradict your latest piece about models not being able to fit locally in robots then?

Instead of endlessly increasing parameter counts, labs may be prioritising: Parameter efficiency → recurrent depth → more inference compute → stronger reasoning.

I think that pushing further into deeper frontiers may eventually take us beyond what large language models themselves are fundamentally designed to do. Language still operates at the surface layer of cognition.

Report: GPT-6 Astra leaning on loop transformers — more passes through the same layers (compute depth) instead of a larger parameter count. That’s an L0–L1 scaling tell: labs buying FLOPs-per-weight when param roadmaps look less aggressive. If depth-over-width sticks, does the bottleneck shift from training-cluster size toward inference latency and energy per token? NFA.

@zephyr_z9 Duhhh

Loop transformers is a clever workaround

looping layers instead of adding parameters swaps one cost for another: same weights on disk, but every token now pays for multiple passes through the network. compute and latency per token go up even though the checkpoint doesn't get bigger, that tradeoff matters a lot more once you're serving this in realtime.

Recurrent depth increases inference latency non-linearly if the halting condition isn't strictly bounded.

Loop transformer basically does if the last layers require knowledge from the first set of layers it can’t bcuz of the strict forward pass while in loop u can get atleast another chance also. So composition of knowledge is possible.That why i felt like astra is token efficient.

looping the same layers adds sequential compute without adding weights. does the extra pass show up as slower TTFT, or only once it's generating?

go~ooo d