Загрузка видео...
Не удалось загрузить видео
Introducing HRM-Text. An ultra-lean 1B-parameter reasoning language model designed to deliver strong general performance with a fraction of the data, compute, and infrastructure. Trained on just 40B structured tokens, HRM-Text achieves competitive performance while using ~1/1000 of the training data of comparable models. The kicker? The full model trains... show more
519,365 просмотров • 5 месяцев назад •via X (Twitter)
Комментарии: 34
Download HRM-Text 🔗 Github: Hugging Face:
Am I missing something ? HRM-Text is trained on a corpus where every benchmark's training set is present and often upsampled 10×. Of course you hit the same benchmark score with 150× less data when 150× more of your data is on-distribution for the eval.
Watched it twice. The $1k pretraining number is going to do more for independent research than any open-weights release this year
Those are impressive results. Any reason you didn’t scale it up? It’s a bit strange to have a flagship that trains for a thousand dollars.
The training mix (data_io) includes GSM8k-train ×10, MATH-train ×10, and FLAN (which contains HellaSwag/ARC/BoolQ/Winogrande train splits). Most of the 150× saving is the data mix, not the architecture. But I do have to applaude the ingenious H/L split :)
those are the numbers for Qwen3 Qwen3.5s dataset size has not been disclosed and if you really believe in this AND its so cheap you should scale it up to at least 32B range and get llama.cpp compatability otherwise nobody can even use it or care about it
Very interesting! What about for bigger models larger than 1b?
$1000 for a 1B parameter is excessive. I full pretrained my 1.5B RLM model with an RTX Pro 6000 for 13 hours. It cost less than $4(with Google Colab).
looks promising, can't wait to try it out. I really need a small parameter model that I can run on my old PC
training a 1B model in a day for $1k is insane the barrier to AI research just dropped through the floor
Amazing result, this is where I don't understand why you didn't start a company instead of letting the LLM companies just use it to train much smarter models.
This release is incompatible with me going to sleep in time
Brilliant. Hierarchical dual-timescale reasoning in a 1B model that beats far larger ones on 40B tokens is exactly the efficiency breakthrough we need. This is how we get to real intelligence, not just scale.
awesome work, congrats!
wait it's only 1B and trained in a day?? That's wild
"chain-of-thought" slander for 5 whole mins. great approach, moving from grid-based puzzling in HRM to implementing latent recursive reasoning in language modelling through HRM-text.
Paper:
one day and a thousand bucks is wild, but comparable models is doing a lot of work in that claim what evals are you running and what's the accuracy gap?
That's super, so maybe we can use our new approach of cloning the LLM behavioural shadow here for only 10 USD and then inference it with CPU. Have to investigate.
I’ve been using my personal agent for a while now So this could be a game-changer!
This is amazing
40B structured tokens, 1B params. Tokenization doing the heavy lifting. What counts as 'structured'?
Tried in Colab T4, it's not even forming a coherent sentence 😭
$1k pretraining cost is the part that broke my brain
Where is the paper?
Def interesting stuff. Would love to see if this scales. What kind of context length can you use?
1k to train a 1b model, big labs about to cry into their gpu invoices
looks awesomeee i watchedd
wait… a 1B reasoning model trained for around $1k? that’s actually insane
Finally something that doesn't need a million GPUs lol.
The most interesting part is the compute barrier collapsing this fast .. feels like we’re entering the era of garage-built AI again
The real AGI is already here
the $1k pretrain cost is what hit me. small teams can actually do this now
The future of AGI
![General Intuition CEO Pim de Witte, who's building foundation models trained on video game controller input data ("action-labeled gameplay clips"), says general intelligence won't "taste like an LLM": "We have a scale of data that's going to allow us to jump to the frontier in one capability — which is any system that can be controlled with a game controller (which is most robots) — and then, you can use that to create a sufficiently general intelligence." "As humans, the decision to talk or type is a very, very small subset of the actions that we can actually take." "So in order to create a sufficiently general intelligence to play 10,000+ video games, the model has to be able to predict across the entire action space of human cognition when they're interacting with these environments. Which are 2D and 3D environments, interfaces, long-horizon tasks, short-horizon tasks, [etc.]." "It has to be a sufficiently general intelligence in order to predict actions. Therefore, the type of model you get out is not going to taste like an LLM. This model is going to be incredibly good at navigating unforeseen environments. It's going to be incredibly good at zero-shotting any task that can be done with a game controller."](https://image.24vids.com/tw-2071731641366806768/amplify_video_thumb/2071731439247523840/img/Ar0HzGY4TZXz_4fN.jpg)
