Loading video...

Video Failed to Load

Go Home

Variable-length compressive tokenization is a promising direction. Not just for efficiency, but for actually learning powerful representations. FlexTok scratched the surface of this direction with images. But real-world data has additional structure that can be tapped, such as the temporal structure of videos. VideoFlexTok develops this concept for video,...

zamir_ar's profile picture

Amir Zamir

5,902 subscribers

17,787 views • 6 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

1 Comments

Sivan Doveh's profile picture
Sivan Doveh6 months ago

Great paper Amir! Super interesting

Related Videos

Test-time scaling, reasoning, and generally search-like processes clearly drive significant gains in LLMs. Largely owed to the structure of language. One would think the same could apply to non-linguistic domains, like image generation, but that obviously depends on whether the structure of the domain's representation lends itself to search. 1D ordered tokens (e.g., image FlexTok, video FlexTok) seem like a natural fit since they enable a step-by-step coarse-to-fine generation. We investigated that and found they indeed enable search and scale far better with test-time compute than 2D grids. See the visuals on the webpage. Appearing in ICML Conference 2026. 🔗 📄
0:23

Test-time scaling, reasoning, and generally search-like processes clearly drive significant gains in LLMs. Largely owed to the structure of language. One would think the same could apply to non-linguistic domains, like image generation, but that obviously depends on whether the structure of the domain's representation lends itself to search. 1D ordered tokens (e.g., image FlexTok, video FlexTok) seem like a natural fit since they enable a step-by-step coarse-to-fine generation. We investigated that and found they indeed enable search and scale far better with test-time compute than 2D grids. See the visuals on the webpage. Appearing in ICML Conference 2026. 🔗 📄

Amir Zamir

15,925 views • 5 months ago

Introducing ESM Cambrian. Unsupervised learning can invert biology at scale to reveal the hidden structure of the natural world. We’ve scaled up compute and data to train a new generation of protein language models. ESM C defines a new state of the art for protein representation learning.
1:29

Introducing ESM Cambrian. Unsupervised learning can invert biology at scale to reveal the hidden structure of the natural world. We’ve scaled up compute and data to train a new generation of protein language models. ESM C defines a new state of the art for protein representation learning.

Alex Rives

207,631 views • 1 year ago

A lot of robot learning pipelines treat embodiment mismatch as a software problem. Collect data with one device, retarget the motion to another robot, and try to deal with the differences later. TwinDEX flips that idea around. This system is from a Chinese robotics company called X Square. Instead of starting with a generic data-collection device, the team co-designs a wearable three-finger end-effector and a robotic hand as two sides of the same learning system. They align the things that actually matter during manipulation: joint structure, link proportions, contact geometry, surface properties, visual appearance, sensing, and timing. That matters because dexterous manipulation is much more than matching hand poses. A few millimeters of contact error or a different force relationship can completely change how a task is executed. In the collection evaluation, TwinDEX delivered up to 5.3 times the effective throughput of on-robot teleoperation. It’s a useful reminder that scaling robot learning may not only be about bigger models and more data. Sometimes the hardware collecting the data is part of the data problem itself.
0:20

A lot of robot learning pipelines treat embodiment mismatch as a software problem. Collect data with one device, retarget the motion to another robot, and try to deal with the differences later. TwinDEX flips that idea around. This system is from a Chinese robotics company called X Square. Instead of starting with a generic data-collection device, the team co-designs a wearable three-finger end-effector and a robotic hand as two sides of the same learning system. They align the things that actually matter during manipulation: joint structure, link proportions, contact geometry, surface properties, visual appearance, sensing, and timing. That matters because dexterous manipulation is much more than matching hand poses. A few millimeters of contact error or a different force relationship can completely change how a task is executed. In the collection evaluation, TwinDEX delivered up to 5.3 times the effective throughput of on-robot teleoperation. It’s a useful reminder that scaling robot learning may not only be about bigger models and more data. Sometimes the hardware collecting the data is part of the data problem itself.

Alamin

48,112 views • 2 months ago

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.
1:47

Before AI can understand the physical world, it needs data about the physical world. Not just images. Spatial environments. Real streets, buildings, parks, stores, museums, and public spaces captured in 3D. OVER has already reconstructed 290,000+ real-world 3D scenes, creating a growing dataset for the next generation of spatial and embodied AI. And this is exactly the kind of real-world spatial data AI companies are looking for. We're already in conversations with multiple AI and Robotics companies exploring how OVER's dataset can support the development and training of models that need to understand the physical world. The next AI race will be about real-world data.

Over the Reality 🌐

24,343 views • 1 month ago

The ice forming here during the landing burn is also interesting to me. This structure is effectively like a raceway running around the base of the booster, for context. Links up each of the 40 junction boxes on the exterior that route commodities to and from each engine and such. There shouldn't be anything venting in the cavity there. We've never seen condensation, especially to the point of ice, form on these surfaces. Yes, it's affected by exterior moisture as you can see it fluctuate as it passes through a cloud here, but the difference is negligible and it only really starts forming as soon as the landing burn begins. If something broke in there...not saying it did but it's an interesting prospect
0:27

The ice forming here during the landing burn is also interesting to me. This structure is effectively like a raceway running around the base of the booster, for context. Links up each of the 40 junction boxes on the exterior that route commodities to and from each engine and such. There shouldn't be anything venting in the cavity there. We've never seen condensation, especially to the point of ice, form on these surfaces. Yes, it's affected by exterior moisture as you can see it fluctuate as it passes through a cloud here, but the difference is negligible and it only really starts forming as soon as the landing burn begins. If something broke in there...not saying it did but it's an interesting prospect

TheSpaceEngineer

32,300 views • 1 month ago

one of my favorite aspects of this new work is that it is generative, not based on an arbitrary random number, but based on the chaos of real life. the physical sculpture is constantly pulling in news and real-time data from all over the world and the NFT is updated daily as well with this hectic collage of information overload... i believe in the future more work will be dynamic like this allowing us to reimagine real-time data in new ways based on different events happening in the world...
0:34

one of my favorite aspects of this new work is that it is generative, not based on an arbitrary random number, but based on the chaos of real life. the physical sculpture is constantly pulling in news and real-time data from all over the world and the NFT is updated daily as well with this hectic collage of information overload... i believe in the future more work will be dynamic like this allowing us to reimagine real-time data in new ways based on different events happening in the world...

beeple

155,819 views • 2 years ago

World Labs CEO Dr. Fei-Fei Li & SceniX Co-Founder Yunzhu Li on the data bottleneck in robotics and how world models help: "The lack of data in training, the lack of data in evaluation, this is very, very different from language models, where data is abundant on the internet." "And we know that in order for robotics to work, we have to somehow unlock the power of scaling law. But where does that come from?" "It's a profound problem that everybody's battling with in robotics." "We see a lot of unlock in being able to do this whole process through the modeling of the environments." "Being able to create these digital worlds... that's just going to unlock so much more potential for being able to replace all the costly and the unsafe data in the real environments with the data generated from the worlds for robots to be able to do scalable learning and evaluations." Fei-Fei Li Yunzhu Li martin_casado
1:49

World Labs CEO Dr. Fei-Fei Li & SceniX Co-Founder Yunzhu Li on the data bottleneck in robotics and how world models help: "The lack of data in training, the lack of data in evaluation, this is very, very different from language models, where data is abundant on the internet." "And we know that in order for robotics to work, we have to somehow unlock the power of scaling law. But where does that come from?" "It's a profound problem that everybody's battling with in robotics." "We see a lot of unlock in being able to do this whole process through the modeling of the environments." "Being able to create these digital worlds... that's just going to unlock so much more potential for being able to replace all the costly and the unsafe data in the real environments with the data generated from the worlds for robots to be able to do scalable learning and evaluations." Fei-Fei Li Yunzhu Li martin_casado

a16z

80,437 views • 3 months ago

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.
0:33

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

AK

122,567 views • 3 years ago

I saw a trending video of a group of Nigerians in Edo State scooping rice from a watery gutter, and I almost shed tears. The state of the nation has left so many people poor and hungry that they no longer care about their dignity. This was not the Nigeria I envisioned when I was a young man. It hurts to see that things have gotten this bad. It truly hurts. At times like this, I wonder what the rest of the world must think of us when they see videos like this. When the President and the APC brags about having structure, it becomes more obvious to us that their structure is a structure that supports the foundation of poverty and hunger in Nigeria.
3:01

I saw a trending video of a group of Nigerians in Edo State scooping rice from a watery gutter, and I almost shed tears. The state of the nation has left so many people poor and hungry that they no longer care about their dignity. This was not the Nigeria I envisioned when I was a young man. It hurts to see that things have gotten this bad. It truly hurts. At times like this, I wonder what the rest of the world must think of us when they see videos like this. When the President and the APC brags about having structure, it becomes more obvious to us that their structure is a structure that supports the foundation of poverty and hunger in Nigeria.

Pat Utomi

22,026 views • 17 days ago

"Commander, you've been wondering what your Spartans died for? They died for 𝘵𝘩𝘪𝘴." I love this Halsey line. It heightens the mystery of just what could be important enough to justify the deaths of two Spartans, AND ties in the rest of the campaign up until this point. It has two meanings, too. In universe, the looming Forerunner structure and the data it contains (the "latchkey" discovery) are worth the sacrifice... but to the player, the one who carries the data is equally as important: Cortana.
0:26

"Commander, you've been wondering what your Spartans died for? They died for 𝘵𝘩𝘪𝘴." I love this Halsey line. It heightens the mystery of just what could be important enough to justify the deaths of two Spartans, AND ties in the rest of the campaign up until this point. It has two meanings, too. In universe, the looming Forerunner structure and the data it contains (the "latchkey" discovery) are worth the sacrifice... but to the player, the one who carries the data is equally as important: Cortana.

GoldAthena 💜 (VTuber Arc)

22,315 views • 5 months ago

We have data in abundance now. The real value isn't within the RAW Data, but is this data validated and verified? More value in data refinement. We have a solution for this which is validated by 6 years of research and development with the Department of Defense Now securing and validating commercial data from partners like Common Crawl and Panasonic
1:03

We have data in abundance now. The real value isn't within the RAW Data, but is this data validated and verified? More value in data refinement. We have a solution for this which is validated by 6 years of research and development with the Department of Defense Now securing and validating commercial data from partners like Common Crawl and Panasonic

Constellation²

10,567 views • 1 year ago

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,867 views • 4 months ago

This is a very big lie. Ali Larijani’s brother, Sadegh Larijani, had no standing with the IRGC or the core power structure because his financial corruption had been publicly exposed. The selection of Mojtaba Khamenei was laid down as early as 2009. Two years ago, Mojtaba also publicly announced the suspension of his fiqh classes and stepped away from clerical positions in order to prepare for power (this is the only short video that exists of him speaking). No one has the ability to compete with the security structure that has been built over the past 37 years. As we have said before, the IRGC oligarchy and the clerical circles in Mashhad and Qom have embedded themselves in the power structure in such a way that it does not fall into the hands of outsiders. This is how it works!
1:35

This is a very big lie. Ali Larijani’s brother, Sadegh Larijani, had no standing with the IRGC or the core power structure because his financial corruption had been publicly exposed. The selection of Mojtaba Khamenei was laid down as early as 2009. Two years ago, Mojtaba also publicly announced the suspension of his fiqh classes and stepped away from clerical positions in order to prepare for power (this is the only short video that exists of him speaking). No one has the ability to compete with the security structure that has been built over the past 37 years. As we have said before, the IRGC oligarchy and the clerical circles in Mashhad and Qom have embedded themselves in the power structure in such a way that it does not fall into the hands of outsiders. This is how it works!

Ehsan AriaN

16,524 views • 7 months ago

Russian Genius Found Structure in The Mess of Turbulence In 1941, Andrey Kolmogorov proposed a simple but powerful picture of turbulence: Energy does not disappear where it is injected but moves. Large eddies pass energy to smaller ones, those smaller ones feed even finer motion, and viscosity finally removes it at the tiniest scales. That cascade picture became one of the foundations of modern turbulence theory. The point is not just that turbulence looks messy. The point is that the mess has structure, and Kolmogorov’s great insight was to treat that structure statistically.
0:30

Russian Genius Found Structure in The Mess of Turbulence In 1941, Andrey Kolmogorov proposed a simple but powerful picture of turbulence: Energy does not disappear where it is injected but moves. Large eddies pass energy to smaller ones, those smaller ones feed even finer motion, and viscosity finally removes it at the tiniest scales. That cascade picture became one of the foundations of modern turbulence theory. The point is not just that turbulence looks messy. The point is that the mess has structure, and Kolmogorov’s great insight was to treat that structure statistically.

Mathelirium

59,138 views • 6 months ago

Why quantum theory could have a fractal structure: explained in 83 minutes. The notion of true quantum nonlocality is absurd. In the latest episode of the Quantum Foundations Podcast, Prof Tim Palmer from the University of Oxford suggests that there is a hidden assumption in standard quantum mechanics, and dropping it will save us from this absurdity. Namely, the reality of counterfactuals: the physics of what could have happened but did not. Inspired by chaos theory and the fractal structure widespread in atmospheric physics, Palmer has developed a new underlying structure for quantum theory, with radical implications for our fundamental principles of quantum physics; the limits of quantum computation; and perhaps even the search for quantum gravity. Watch on my YouTube channel, or listen on Spotify and Apple Podcasts!
1:34

Why quantum theory could have a fractal structure: explained in 83 minutes. The notion of true quantum nonlocality is absurd. In the latest episode of the Quantum Foundations Podcast, Prof Tim Palmer from the University of Oxford suggests that there is a hidden assumption in standard quantum mechanics, and dropping it will save us from this absurdity. Namely, the reality of counterfactuals: the physics of what could have happened but did not. Inspired by chaos theory and the fractal structure widespread in atmospheric physics, Palmer has developed a new underlying structure for quantum theory, with radical implications for our fundamental principles of quantum physics; the limits of quantum computation; and perhaps even the search for quantum gravity. Watch on my YouTube channel, or listen on Spotify and Apple Podcasts!

Dr Maria Violaris

18,602 views • 9 months ago

Licensed pharmacist says there is a real way you can actually reduce the amount of days your sick if you begin to get a cold He says some of the items sold in America are nothing more than placebos but there is something that’s works and has data to show it works to shorten the length of time you’re sick It’s recommended by the American Academy of Family Physicians for the reduction in length of the common cold
3:00

Licensed pharmacist says there is a real way you can actually reduce the amount of days your sick if you begin to get a cold He says some of the items sold in America are nothing more than placebos but there is something that’s works and has data to show it works to shorten the length of time you’re sick It’s recommended by the American Academy of Family Physicians for the reduction in length of the common cold

Wall Street Apes

330,779 views • 1 month ago

"The data that we've been working with as people now becomes extremely useful in a world of agents. You're having this rebirth of the file system." Aaron Levie on Navigators with Paul Klein IV on why unstructured data, the 90% of enterprise data that has always been the hardest to automate, is now the biggest unlock for AI agents. And why the real AI conversation is a context conversation.
2:40

"The data that we've been working with as people now becomes extremely useful in a world of agents. You're having this rebirth of the file system." Aaron Levie on Navigators with Paul Klein IV on why unstructured data, the 90% of enterprise data that has always been the hardest to automate, is now the biggest unlock for AI agents. And why the real AI conversation is a context conversation.

Box

66,225 views • 3 months ago

Terence Tao, UCLA professor and the most decorated mathematician alive: "Funds pay $750K to combine weak signals into one real edge. I proved the thing that makes it work and makes it dangerous: in any long enough sequence, hidden structure is unavoidable. it always accumulates. the whole job is telling the real structure from the noise that only looks like it." this free lecture is the most decorated mathematician alive on the exact problem sitting underneath every factor model, and it costs nothing. at the board it's simple. Tao's lifelong theme is the line between structure and randomness. The Erdős discrepancy problem asks a deceptively simple thing: can you write an endless string of plus-ones and minus-ones that stays perfectly balanced forever? Tao proved you cannot. No matter how cleverly you try, imbalance, hidden structure, is forced to accumulate as the sequence grows. There is no such thing as a long stream of pure, structureless noise. That's the whole idea, minus the jargon. Which is exactly why a multi-factor model can work, and exactly why it can kill you. Stack enough weak signals and real structure will appear, because at scale structure is unavoidable. But so will fake structure, patterns that exist only because the data is long enough to force them. Same point as the post above: finding structure is guaranteed. Knowing which structure is an edge is the rare and expensive part. the mathematics is free and public. what nobody can sell you is the judgment to tell the structure the market will pay you for from the structure that exists only because you looked hard enough. That judgment is the alpha, and it takes years to build.
51:34

Terence Tao, UCLA professor and the most decorated mathematician alive: "Funds pay $750K to combine weak signals into one real edge. I proved the thing that makes it work and makes it dangerous: in any long enough sequence, hidden structure is unavoidable. it always accumulates. the whole job is telling the real structure from the noise that only looks like it." this free lecture is the most decorated mathematician alive on the exact problem sitting underneath every factor model, and it costs nothing. at the board it's simple. Tao's lifelong theme is the line between structure and randomness. The Erdős discrepancy problem asks a deceptively simple thing: can you write an endless string of plus-ones and minus-ones that stays perfectly balanced forever? Tao proved you cannot. No matter how cleverly you try, imbalance, hidden structure, is forced to accumulate as the sequence grows. There is no such thing as a long stream of pure, structureless noise. That's the whole idea, minus the jargon. Which is exactly why a multi-factor model can work, and exactly why it can kill you. Stack enough weak signals and real structure will appear, because at scale structure is unavoidable. But so will fake structure, patterns that exist only because the data is long enough to force them. Same point as the post above: finding structure is guaranteed. Knowing which structure is an edge is the rare and expensive part. the mathematics is free and public. what nobody can sell you is the judgment to tell the structure the market will pay you for from the structure that exists only because you looked hard enough. That judgment is the alpha, and it takes years to build.

Rossst.03

214,575 views • 3 months ago

This is the real world. Overwhelming support, all the time! The concept of a transgender child is a lie, and we’re in the only generation in human history where this lie has been pushed. Our institutions and politicians need to stop catering to an angry, deceived, and deceptive minority. There is no such thing as a transgender child!
2:36

This is the real world. Overwhelming support, all the time! The concept of a transgender child is a lie, and we’re in the only generation in human history where this lie has been pushed. Our institutions and politicians need to stop catering to an angry, deceived, and deceptive minority. There is no such thing as a transgender child!

Billboard Chris 🌎

26,105 views • 8 months ago

Matryoshka Representation Learning (MRL) is a super exciting approach to improving the quality and efficiency of embedding models and strategies ✨ MRL allows models to store more information in the earlier dimensions of their data vectors. This method not only boosts performance in tasks like classification and retrieval, but is also a super cool compression technique! Paper: For compression: It’s been so much fun learning and making this video with Daniel, thanks for all your help!
2:39

Matryoshka Representation Learning (MRL) is a super exciting approach to improving the quality and efficiency of embedding models and strategies ✨ MRL allows models to store more information in the earlier dimensions of their data vectors. This method not only boosts performance in tasks like classification and retrieval, but is also a super cool compression technique! Paper: For compression: It’s been so much fun learning and making this video with Daniel, thanks for all your help!

Victoria Slocum

120,470 views • 2 years ago