Video wird geladen...
Video konnte nicht geladen werden
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data. Language modeling on raw bytes is... show more
1,023,296 Aufrufe • vor 7 Tagen •via X (Twitter)
16 Kommentare

Blog: Part 1 Part 2 (Voiceover created entirely by Cartesia's Sonic text-to-speech model!)

This is the kind of work that makes me excited to be at @cartesia. Huge props to the team behind H-Nets~

So exciting to see h-net results! The In-context tokenization part is so cool - show's the real power of hnet imo. Feels like it's directionarlly a step on the path to a true continuous learner / infinite context.

so cool to see it work on images and patchify background as one token

amazing

bitter lesson never misses, we just keep forgetting it every hype cycle

Raw bytes still force the hierarchy

raw data, huh. language already comes pre-chewed by people. the real test is what happens where nobody bothered to name things.

what happened here?

the efficiency jump is huge! the bitter lesson is always true

Other people have spoken of a cognitive core with search and tools. A lot of the initial wow came from the fact that some x million documents have been consumed and can be recalled verbatim. That's a different task from getting work done.

What does staying at the byte level cost you at inference time?

does any of the insights translate to image/video-gen?

10x less compute is absurd

Incredible work!!

beautiful!
