Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data. Language modeling on raw bytes is...

1,023,296 Aufrufe • vor 7 Tagen •via X (Twitter)

16 Kommentare

Profilbild von Albert Gu
Albert Guvor 7 Tagen

Blog: Part 1 Part 2 (Voiceover created entirely by Cartesia's Sonic text-to-speech model!)

Profilbild von Nathan (Quan) Zhou
Nathan (Quan) Zhouvor 7 Tagen

This is the kind of work that makes me excited to be at @cartesia. Huge props to the team behind H-Nets~

Profilbild von Justin Waugh
Justin Waughvor 7 Tagen

So exciting to see h-net results! The In-context tokenization part is so cool - show's the real power of hnet imo. Feels like it's directionarlly a step on the path to a true continuous learner / infinite context.

Profilbild von Karan Bania
Karan Baniavor 7 Tagen

so cool to see it work on images and patchify background as one token

Profilbild von Tim Shi
Tim Shivor 7 Tagen

amazing

Profilbild von Saumya Verma
Saumya Vermavor 7 Tagen

bitter lesson never misses, we just keep forgetting it every hype cycle

Profilbild von Tommy
Tommyvor 7 Tagen

Raw bytes still force the hierarchy

Profilbild von hailports
hailportsvor 7 Tagen

raw data, huh. language already comes pre-chewed by people. the real test is what happens where nobody bothered to name things.

Profilbild von Jonathan Chang
Jonathan Changvor 7 Tagen

what happened here?

Profilbild von Charis Mitsakis
Charis Mitsakisvor 7 Tagen

the efficiency jump is huge! the bitter lesson is always true

Profilbild von Chirag
Chiragvor 7 Tagen

Other people have spoken of a cognitive core with search and tools. A lot of the initial wow came from the fact that some x million documents have been consumed and can be recalled verbatim. That's a different task from getting work done.

Profilbild von AI Quanting
AI Quantingvor 7 Tagen

What does staying at the byte level cost you at inference time?

Profilbild von Ir rev
Ir revvor 7 Tagen

does any of the insights translate to image/video-gen?

Profilbild von Gravity Research
Gravity Researchvor 7 Tagen

10x less compute is absurd

Profilbild von Nancy Xu
Nancy Xuvor 7 Tagen

Incredible work!!

Profilbild von Sophia Yang
Sophia Yangvor 7 Tagen

beautiful!

Ähnliche Videos

#WATCH | India AI Impact Summit 2026 | Delhi: Founder Chairman and CEO of Sampark Foundation & former CEO of HCL Technologies, Vineet Nayar says, "...From an employment point of view I think it is very important for us to understand that Indian companies, including Indian IT companies, are going to be profit-driven and therefore if you believe that they are going to create employment you must be dreaming. Therefore, the question is how do we create employment in this environment, and that employment comes from mass scale startups, which is what this government has already doing. So, how do we create new sets of people who are trying to solve new sets of problems not new sets of technology and if we do that we will get it right. I think we as Indians have to be very careful on who does data belong to and that is the debate we have a problem with. The LLM models which exist worldwide are far superior than the Indian models. Unfortunately, in India, we never develop products, so therefore we do not have SLMs and LLMs which are world-class. On one side, we have global LLM products which are coming to India and trading on our Indian data. Should we allowed that or should we not allowed that? But on the other side if we don't allow that then we have the data but we don't have the LLM models. So, how do we encourage technology completely to develop the LLM models. This needs radicals strategic thinking and a very important aspect otherwise we will either give up a data. So, I think it's a very critical aspect for us to think about - who does this data belong, what is the kind of incentives we are going to give to develop LLM technologies or SLM technologies fast so that we train on our data otherwise an LLM will come in with our data and we'll immediately see return and we'll celebrate and we will do all these kind of press releases but the India will lose a competitive advantage on something which is very critical for the next decade."

ANI

18,753 Aufrufe • vor 8 Monaten