Loading video...

Video Failed to Load

Go Home

Introducing HRM-Text. An ultra-lean 1B-parameter reasoning language model designed to deliver strong general performance with a fraction of the data, compute, and infrastructure. Trained on just 40B structured tokens, HRM-Text achieves competitive performance while using ~1/1000 of the training data of comparable models. The kicker? The full model trains...

519,365 views • 6 months ago •via X (Twitter)

34 Comments

Sapient Intelligence's profile picture
Sapient Intelligence6 months ago

Download HRM-Text 🔗 Github: Hugging Face:

Flavius Burca's profile picture
Flavius Burca6 months ago

Am I missing something ? HRM-Text is trained on a corpus where every benchmark's training set is present and often upsampled 10×. Of course you hit the same benchmark score with 150× less data when 150× more of your data is on-distribution for the eval.

AI_Explorer's profile picture
AI_Explorer6 months ago

Watched it twice. The $1k pretraining number is going to do more for independent research than any open-weights release this year

William Wale's profile picture
William Wale6 months ago

Those are impressive results. Any reason you didn’t scale it up? It’s a bit strange to have a flagship that trains for a thousand dollars.

Flavius Burca's profile picture
Flavius Burca6 months ago

The training mix (data_io) includes GSM8k-train ×10, MATH-train ×10, and FLAN (which contains HellaSwag/ARC/BoolQ/Winogrande train splits). Most of the 150× saving is the data mix, not the architecture. But I do have to applaude the ingenious H/L split :)

ρ:ɡeon's profile picture
ρ:ɡeon6 months ago

those are the numbers for Qwen3 Qwen3.5s dataset size has not been disclosed and if you really believe in this AND its so cheap you should scale it up to at least 32B range and get llama.cpp compatability otherwise nobody can even use it or care about it

Crof AI's profile picture
Crof AI6 months ago

Very interesting! What about for bigger models larger than 1b?

DevHunterAI's profile picture
DevHunterAI6 months ago

$1000 for a 1B parameter is excessive. I full pretrained my 1.5B RLM model with an RTX Pro 6000 for 13 hours. It cost less than $4(with Google Colab).

Knowix's profile picture
Knowix6 months ago

looks promising, can't wait to try it out. I really need a small parameter model that I can run on my old PC

Brunão's profile picture
Brunão6 months ago

training a 1B model in a day for $1k is insane the barrier to AI research just dropped through the floor

Adam Ritter's profile picture
Adam Ritter6 months ago

Amazing result, this is where I don't understand why you didn't start a company instead of letting the LLM companies just use it to train much smarter models.

Everlier's profile picture
Everlier6 months ago

This release is incompatible with me going to sleep in time

Jules 𝕏's profile picture
Jules 𝕏6 months ago

Brilliant. Hierarchical dual-timescale reasoning in a 1B model that beats far larger ones on 40B tokens is exactly the efficiency breakthrough we need. This is how we get to real intelligence, not just scale.

himanshu's profile picture
himanshu6 months ago

awesome work, congrats!

Ziwen's profile picture
Ziwen6 months ago

wait it's only 1B and trained in a day?? That's wild

𝐲𝐮𝐫𝐨's profile picture
𝐲𝐮𝐫𝐨6 months ago

"chain-of-thought" slander for 5 whole mins. great approach, moving from grid-based puzzling in HRM to implementing latent recursive reasoning in language modelling through HRM-text.

Sapient Intelligence's profile picture
Sapient Intelligence6 months ago

Paper:

BruzWJ's profile picture
BruzWJ6 months ago

one day and a thousand bucks is wild, but comparable models is doing a lot of work in that claim what evals are you running and what's the accuracy gap?

the.PM's profile picture
the.PM6 months ago

That's super, so maybe we can use our new approach of cloning the LLM behavioural shadow here for only 10 USD and then inference it with CPU. Have to investigate.

Yuli Kay's profile picture
Yuli Kay6 months ago

I’ve been using my personal agent for a while now So this could be a game-changer!

Ryan Dale's profile picture
Ryan Dale6 months ago

This is amazing

TechGeekDavid's profile picture
TechGeekDavid6 months ago

40B structured tokens, 1B params. Tokenization doing the heavy lifting. What counts as 'structured'?

Adhil Parammel's profile picture
Adhil Parammel6 months ago

Tried in Colab T4, it's not even forming a coherent sentence 😭

Oogie's profile picture
Oogie6 months ago

$1k pretraining cost is the part that broke my brain

Harris's profile picture
Harris6 months ago

Where is the paper?

Latent Local's profile picture
Latent Local6 months ago

Def interesting stuff. Would love to see if this scales. What kind of context length can you use?

Chemist 🧪's profile picture
Chemist 🧪6 months ago

1k to train a 1b model, big labs about to cry into their gpu invoices

Yigido's profile picture
Yigido6 months ago

looks awesomeee i watchedd

tara_'s profile picture
tara_6 months ago

wait… a 1B reasoning model trained for around $1k? that’s actually insane

Yuki Eliot's profile picture
Yuki Eliot6 months ago

Finally something that doesn't need a million GPUs lol.

Synthia's profile picture
Synthia6 months ago

The most interesting part is the compute barrier collapsing this fast .. feels like we’re entering the era of garage-built AI again

J A Z I I's profile picture
J A Z I I6 months ago

The real AGI is already here

veno‎ 𓅃's profile picture
veno‎ 𓅃6 months ago

the $1k pretrain cost is what hit me. small teams can actually do this now

Oxdave's profile picture
Oxdave6 months ago

The future of AGI

Related Videos

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.

AK

40,492 views • 2 years ago

Perplexity CEO Aravind Srinivas on the biggest threat to the data center industry: It's not competition. It's not regulation. It's decentralisation. "The biggest threat to a data center is if the intelligence can be packed locally on a chip that's running on the device and then there's no need to inference all of it on like one centralized data center." He outlines how this could work in practice. Personalisation doesn't necessarily require on-device model training. Retrieval augmented generation, tool calls, and local data can already tailor AI to individual users. But the real unlock? Test time training. Aravind Srinivas describes a future where AI lives on your device, watches how you work and gradually automates your repetitive tasks. "Imagine we crack test time training where the AI watches tasks you repeatedly do on your local system, adapts to you over time and starts automating a lot of the things you do." The key insight: in this model, the intelligence belongs to you. It's your data, your device, your personalised AI brain. And if that future arrives, the economics of centralised infrastructure start to collapse. "That really disrupts the whole data center industry. It doesn't make sense to spend all this money, 500 billion, 5 trillion, whatever on building all the centralized data centers across the world that do a lot of the intelligence workloads for people." The companies spending trillions on centralised infrastructure may want to rethink where intelligence actually needs to live.

Big Brain AI

90,355 views • 8 months ago

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.🎨 ✨ New in 2.1: 🔹Advanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. 🔹Precise Chinese and English Text Rendering with seamless image–text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. 🔹Rich Styles and High Aesthetic: Capable of generating images in various styles—including photorealistic portraits, comics, and vinyl figures—it delivers outstanding visual appeal and artistic quality. 🔹High-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideas—like posters with slogans or multi-panel comics—into visuals faster than ever. We’re just getting started. Stay tuned for our native multimodal image generation model coming soon. 🌐Website: 🔗Github: 🤗Hugging Face: ✨Hugging Face Demo:

Tencent Hy

89,430 views • 1 year ago

Just how capable are open source models? Below is the first in a new series where we go behind the scenes and pull back the curtain on interesting AI research / demos, making them fun and easy to understand. Here, we have a short visual demonstration from aizk ✡️ showcasing how Kimi K3 (a language model that operates primarily through text) is capable of building complicated 3D structures / moments in history in Minecraft, something that previously was not possible with other open source models, and why this matters. The crazy part? The model doesn't "see" the game like we do. The LLMs must reason in pure text, writing JavaScript, that later compiles down into commands placing each block, one at a time. Spatial reasoning is a very hard problem in AI, it's the same core challenge behind robotics and self-driving cars, where a model has to understand and act in physical 3D space. Watching a text model pull it off is nothing short of a miracle. The point isn't just Minecraft itself, rather, it's AI being able to generalize, not memorize, on things that are weird and beyond their training data. This is key to building true artificial general intelligence. These video game benchmarks (there are many different games actively being researched right now) provide a clear-cut end goal, challenges that are almost certainly not in the training set, and a fun, very fast, visual way to almost feel the increasing capabilities of various open source AI models over time. If you haven't given open source models a serious try yet, watch the video, it may shock you!

Featherless AI

39,769 views • 3 months ago