Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Variable-length compressive tokenization is a promising direction. Not just for efficiency, but for actually learning powerful representations. FlexTok scratched the surface of this direction with images. But real-world data has additional structure that can be tapped, such as the temporal structure of videos. VideoFlexTok develops this concept for video,...

zamir_ar's profile picture

Amir Zamir

5,896 subscribers

17,775 Aufrufe • vor 4 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

1 Kommentare

Profilbild von Sivan Doveh
Sivan Dovehvor 4 Monaten

Great paper Amir! Super interesting

Ähnliche Videos

Test-time scaling, reasoning, and generally search-like processes clearly drive significant gains in LLMs. Largely owed to the structure of language. One would think the same could apply to non-linguistic domains, like image generation, but that obviously depends on whether the structure of the domain's representation lends itself to search. 1D ordered tokens (e.g., image FlexTok, video FlexTok) seem like a natural fit since they enable a step-by-step coarse-to-fine generation. We investigated that and found they indeed enable search and scale far better with test-time compute than 2D grids. See the visuals on the webpage. Appearing in ICML Conference 2026. 🔗 📄
0:23

Test-time scaling, reasoning, and generally search-like processes clearly drive significant gains in LLMs. Largely owed to the structure of language. One would think the same could apply to non-linguistic domains, like image generation, but that obviously depends on whether the structure of the domain's representation lends itself to search. 1D ordered tokens (e.g., image FlexTok, video FlexTok) seem like a natural fit since they enable a step-by-step coarse-to-fine generation. We investigated that and found they indeed enable search and scale far better with test-time compute than 2D grids. See the visuals on the webpage. Appearing in ICML Conference 2026. 🔗 📄

Amir Zamir

15,686 Aufrufe • vor 4 Monaten

A lot of robot learning pipelines treat embodiment mismatch as a software problem. Collect data with one device, retarget the motion to another robot, and try to deal with the differences later. TwinDEX flips that idea around. This system is from a Chinese robotics company called X Square. Instead of starting with a generic data-collection device, the team co-designs a wearable three-finger end-effector and a robotic hand as two sides of the same learning system. They align the things that actually matter during manipulation: joint structure, link proportions, contact geometry, surface properties, visual appearance, sensing, and timing. That matters because dexterous manipulation is much more than matching hand poses. A few millimeters of contact error or a different force relationship can completely change how a task is executed. In the collection evaluation, TwinDEX delivered up to 5.3 times the effective throughput of on-robot teleoperation. It’s a useful reminder that scaling robot learning may not only be about bigger models and more data. Sometimes the hardware collecting the data is part of the data problem itself.
0:20

A lot of robot learning pipelines treat embodiment mismatch as a software problem. Collect data with one device, retarget the motion to another robot, and try to deal with the differences later. TwinDEX flips that idea around. This system is from a Chinese robotics company called X Square. Instead of starting with a generic data-collection device, the team co-designs a wearable three-finger end-effector and a robotic hand as two sides of the same learning system. They align the things that actually matter during manipulation: joint structure, link proportions, contact geometry, surface properties, visual appearance, sensing, and timing. That matters because dexterous manipulation is much more than matching hand poses. A few millimeters of contact error or a different force relationship can completely change how a task is executed. In the collection evaluation, TwinDEX delivered up to 5.3 times the effective throughput of on-robot teleoperation. It’s a useful reminder that scaling robot learning may not only be about bigger models and more data. Sometimes the hardware collecting the data is part of the data problem itself.

Alamin

48,112 Aufrufe • vor 14 Tagen

one of my favorite aspects of this new work is that it is generative, not based on an arbitrary random number, but based on the chaos of real life. the physical sculpture is constantly pulling in news and real-time data from all over the world and the NFT is updated daily as well with this hectic collage of information overload... i believe in the future more work will be dynamic like this allowing us to reimagine real-time data in new ways based on different events happening in the world...
0:34

one of my favorite aspects of this new work is that it is generative, not based on an arbitrary random number, but based on the chaos of real life. the physical sculpture is constantly pulling in news and real-time data from all over the world and the NFT is updated daily as well with this hectic collage of information overload... i believe in the future more work will be dynamic like this allowing us to reimagine real-time data in new ways based on different events happening in the world...

beeple

155,806 Aufrufe • vor 1 Jahr

World Labs CEO Dr. Fei-Fei Li & SceniX Co-Founder Yunzhu Li on the data bottleneck in robotics and how world models help: "The lack of data in training, the lack of data in evaluation, this is very, very different from language models, where data is abundant on the internet." "And we know that in order for robotics to work, we have to somehow unlock the power of scaling law. But where does that come from?" "It's a profound problem that everybody's battling with in robotics." "We see a lot of unlock in being able to do this whole process through the modeling of the environments." "Being able to create these digital worlds... that's just going to unlock so much more potential for being able to replace all the costly and the unsafe data in the real environments with the data generated from the worlds for robots to be able to do scalable learning and evaluations." Fei-Fei Li Yunzhu Li martin_casado
1:49

World Labs CEO Dr. Fei-Fei Li & SceniX Co-Founder Yunzhu Li on the data bottleneck in robotics and how world models help: "The lack of data in training, the lack of data in evaluation, this is very, very different from language models, where data is abundant on the internet." "And we know that in order for robotics to work, we have to somehow unlock the power of scaling law. But where does that come from?" "It's a profound problem that everybody's battling with in robotics." "We see a lot of unlock in being able to do this whole process through the modeling of the environments." "Being able to create these digital worlds... that's just going to unlock so much more potential for being able to replace all the costly and the unsafe data in the real environments with the data generated from the worlds for robots to be able to do scalable learning and evaluations." Fei-Fei Li Yunzhu Li martin_casado

a16z

77,065 Aufrufe • vor 1 Monat

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.
0:33

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

AK

122,567 Aufrufe • vor 3 Jahren

We have data in abundance now. The real value isn't within the RAW Data, but is this data validated and verified? More value in data refinement. We have a solution for this which is validated by 6 years of research and development with the Department of Defense Now securing and validating commercial data from partners like Common Crawl and Panasonic
1:03

We have data in abundance now. The real value isn't within the RAW Data, but is this data validated and verified? More value in data refinement. We have a solution for this which is validated by 6 years of research and development with the Department of Defense Now securing and validating commercial data from partners like Common Crawl and Panasonic

Constellation²

10,567 Aufrufe • vor 1 Jahr

"Commander, you've been wondering what your Spartans died for? They died for 𝘵𝘩𝘪𝘴." I love this Halsey line. It heightens the mystery of just what could be important enough to justify the deaths of two Spartans, AND ties in the rest of the campaign up until this point. It has two meanings, too. In universe, the looming Forerunner structure and the data it contains (the "latchkey" discovery) are worth the sacrifice... but to the player, the one who carries the data is equally as important: Cortana.
0:26

"Commander, you've been wondering what your Spartans died for? They died for 𝘵𝘩𝘪𝘴." I love this Halsey line. It heightens the mystery of just what could be important enough to justify the deaths of two Spartans, AND ties in the rest of the campaign up until this point. It has two meanings, too. In universe, the looming Forerunner structure and the data it contains (the "latchkey" discovery) are worth the sacrifice... but to the player, the one who carries the data is equally as important: Cortana.

GoldAthena 💜 (VTuber Arc)

22,315 Aufrufe • vor 3 Monaten

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,772 Aufrufe • vor 3 Monaten

This is a very big lie. Ali Larijani’s brother, Sadegh Larijani, had no standing with the IRGC or the core power structure because his financial corruption had been publicly exposed. The selection of Mojtaba Khamenei was laid down as early as 2009. Two years ago, Mojtaba also publicly announced the suspension of his fiqh classes and stepped away from clerical positions in order to prepare for power (this is the only short video that exists of him speaking). No one has the ability to compete with the security structure that has been built over the past 37 years. As we have said before, the IRGC oligarchy and the clerical circles in Mashhad and Qom have embedded themselves in the power structure in such a way that it does not fall into the hands of outsiders. This is how it works!
1:35

This is a very big lie. Ali Larijani’s brother, Sadegh Larijani, had no standing with the IRGC or the core power structure because his financial corruption had been publicly exposed. The selection of Mojtaba Khamenei was laid down as early as 2009. Two years ago, Mojtaba also publicly announced the suspension of his fiqh classes and stepped away from clerical positions in order to prepare for power (this is the only short video that exists of him speaking). No one has the ability to compete with the security structure that has been built over the past 37 years. As we have said before, the IRGC oligarchy and the clerical circles in Mashhad and Qom have embedded themselves in the power structure in such a way that it does not fall into the hands of outsiders. This is how it works!

Ehsan AriaN

16,524 Aufrufe • vor 6 Monaten

Russian Genius Found Structure in The Mess of Turbulence In 1941, Andrey Kolmogorov proposed a simple but powerful picture of turbulence: Energy does not disappear where it is injected but moves. Large eddies pass energy to smaller ones, those smaller ones feed even finer motion, and viscosity finally removes it at the tiniest scales. That cascade picture became one of the foundations of modern turbulence theory. The point is not just that turbulence looks messy. The point is that the mess has structure, and Kolmogorov’s great insight was to treat that structure statistically.
0:30

Russian Genius Found Structure in The Mess of Turbulence In 1941, Andrey Kolmogorov proposed a simple but powerful picture of turbulence: Energy does not disappear where it is injected but moves. Large eddies pass energy to smaller ones, those smaller ones feed even finer motion, and viscosity finally removes it at the tiniest scales. That cascade picture became one of the foundations of modern turbulence theory. The point is not just that turbulence looks messy. The point is that the mess has structure, and Kolmogorov’s great insight was to treat that structure statistically.

Mathelirium

59,102 Aufrufe • vor 5 Monaten

Why quantum theory could have a fractal structure: explained in 83 minutes. The notion of true quantum nonlocality is absurd. In the latest episode of the Quantum Foundations Podcast, Prof Tim Palmer from the University of Oxford suggests that there is a hidden assumption in standard quantum mechanics, and dropping it will save us from this absurdity. Namely, the reality of counterfactuals: the physics of what could have happened but did not. Inspired by chaos theory and the fractal structure widespread in atmospheric physics, Palmer has developed a new underlying structure for quantum theory, with radical implications for our fundamental principles of quantum physics; the limits of quantum computation; and perhaps even the search for quantum gravity. Watch on my YouTube channel, or listen on Spotify and Apple Podcasts!
1:34

Why quantum theory could have a fractal structure: explained in 83 minutes. The notion of true quantum nonlocality is absurd. In the latest episode of the Quantum Foundations Podcast, Prof Tim Palmer from the University of Oxford suggests that there is a hidden assumption in standard quantum mechanics, and dropping it will save us from this absurdity. Namely, the reality of counterfactuals: the physics of what could have happened but did not. Inspired by chaos theory and the fractal structure widespread in atmospheric physics, Palmer has developed a new underlying structure for quantum theory, with radical implications for our fundamental principles of quantum physics; the limits of quantum computation; and perhaps even the search for quantum gravity. Watch on my YouTube channel, or listen on Spotify and Apple Podcasts!

Dr Maria Violaris

18,594 Aufrufe • vor 7 Monaten

"The data that we've been working with as people now becomes extremely useful in a world of agents. You're having this rebirth of the file system." Aaron Levie on Navigators with Paul Klein IV on why unstructured data, the 90% of enterprise data that has always been the hardest to automate, is now the biggest unlock for AI agents. And why the real AI conversation is a context conversation.
2:40

"The data that we've been working with as people now becomes extremely useful in a world of agents. You're having this rebirth of the file system." Aaron Levie on Navigators with Paul Klein IV on why unstructured data, the 90% of enterprise data that has always been the hardest to automate, is now the biggest unlock for AI agents. And why the real AI conversation is a context conversation.

Box

66,225 Aufrufe • vor 2 Monaten

Terence Tao, UCLA professor and the most decorated mathematician alive: "Funds pay $750K to combine weak signals into one real edge. I proved the thing that makes it work and makes it dangerous: in any long enough sequence, hidden structure is unavoidable. it always accumulates. the whole job is telling the real structure from the noise that only looks like it." this free lecture is the most decorated mathematician alive on the exact problem sitting underneath every factor model, and it costs nothing. at the board it's simple. Tao's lifelong theme is the line between structure and randomness. The Erdős discrepancy problem asks a deceptively simple thing: can you write an endless string of plus-ones and minus-ones that stays perfectly balanced forever? Tao proved you cannot. No matter how cleverly you try, imbalance, hidden structure, is forced to accumulate as the sequence grows. There is no such thing as a long stream of pure, structureless noise. That's the whole idea, minus the jargon. Which is exactly why a multi-factor model can work, and exactly why it can kill you. Stack enough weak signals and real structure will appear, because at scale structure is unavoidable. But so will fake structure, patterns that exist only because the data is long enough to force them. Same point as the post above: finding structure is guaranteed. Knowing which structure is an edge is the rare and expensive part. the mathematics is free and public. what nobody can sell you is the judgment to tell the structure the market will pay you for from the structure that exists only because you looked hard enough. That judgment is the alpha, and it takes years to build.
51:34

Terence Tao, UCLA professor and the most decorated mathematician alive: "Funds pay $750K to combine weak signals into one real edge. I proved the thing that makes it work and makes it dangerous: in any long enough sequence, hidden structure is unavoidable. it always accumulates. the whole job is telling the real structure from the noise that only looks like it." this free lecture is the most decorated mathematician alive on the exact problem sitting underneath every factor model, and it costs nothing. at the board it's simple. Tao's lifelong theme is the line between structure and randomness. The Erdős discrepancy problem asks a deceptively simple thing: can you write an endless string of plus-ones and minus-ones that stays perfectly balanced forever? Tao proved you cannot. No matter how cleverly you try, imbalance, hidden structure, is forced to accumulate as the sequence grows. There is no such thing as a long stream of pure, structureless noise. That's the whole idea, minus the jargon. Which is exactly why a multi-factor model can work, and exactly why it can kill you. Stack enough weak signals and real structure will appear, because at scale structure is unavoidable. But so will fake structure, patterns that exist only because the data is long enough to force them. Same point as the post above: finding structure is guaranteed. Knowing which structure is an edge is the rare and expensive part. the mathematics is free and public. what nobody can sell you is the judgment to tell the structure the market will pay you for from the structure that exists only because you looked hard enough. That judgment is the alpha, and it takes years to build.

Rossst.03

213,931 Aufrufe • vor 1 Monat

This is the real world. Overwhelming support, all the time! The concept of a transgender child is a lie, and we’re in the only generation in human history where this lie has been pushed. Our institutions and politicians need to stop catering to an angry, deceived, and deceptive minority. There is no such thing as a transgender child!
2:36

This is the real world. Overwhelming support, all the time! The concept of a transgender child is a lie, and we’re in the only generation in human history where this lie has been pushed. Our institutions and politicians need to stop catering to an angry, deceived, and deceptive minority. There is no such thing as a transgender child!

Billboard Chris 🌎

26,105 Aufrufe • vor 7 Monaten

Matryoshka Representation Learning (MRL) is a super exciting approach to improving the quality and efficiency of embedding models and strategies ✨ MRL allows models to store more information in the earlier dimensions of their data vectors. This method not only boosts performance in tasks like classification and retrieval, but is also a super cool compression technique! Paper: For compression: It’s been so much fun learning and making this video with Daniel, thanks for all your help!
2:39

Matryoshka Representation Learning (MRL) is a super exciting approach to improving the quality and efficiency of embedding models and strategies ✨ MRL allows models to store more information in the earlier dimensions of their data vectors. This method not only boosts performance in tasks like classification and retrieval, but is also a super cool compression technique! Paper: For compression: It’s been so much fun learning and making this video with Daniel, thanks for all your help!

Victoria Slocum

120,470 Aufrufe • vor 2 Jahren

This thief went in through the burglary proof structure to steal but on his way out, fate played a dirty one on him. He couldn't come out and got stuck in the structure. This is a classic case of "Everyday is for the thief but one day is for the homeowner". 😀
1:30

This thief went in through the burglary proof structure to steal but on his way out, fate played a dirty one on him. He couldn't come out and got stuck in the structure. This is a classic case of "Everyday is for the thief but one day is for the homeowner". 😀

Olaudah Equiano®

24,216 Aufrufe • vor 5 Monaten

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:
0:36

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:

Unreal Engine

61,187 Aufrufe • vor 5 Monaten

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.
1:43

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.

AK

40,489 Aufrufe • vor 2 Jahren

#WATCH | Delhi: On National Statistics Day, Saurabh Garg, Secretary, Ministry of Statistics and Programme Implementation, says, "The theme for National Statistics Day celebration is ' Unlocking the potential of administrative data'. Over the past decade, we have a lot of administrative data, especially because of the growth of Digital India. How we can leverage this administrative data for decision-making is the theme for today's discussions. The PM has been focusing on data-driven decision-making. This is in the context that if we are talking of a Viksit Bharat by 2047, the policies that we take, the decisions that we make, have to be so much more focused and more data-driven. Ministry of Statistics is helping to ensure that the data is more harmonised to make it interoperable."
2:39

#WATCH | Delhi: On National Statistics Day, Saurabh Garg, Secretary, Ministry of Statistics and Programme Implementation, says, "The theme for National Statistics Day celebration is ' Unlocking the potential of administrative data'. Over the past decade, we have a lot of administrative data, especially because of the growth of Digital India. How we can leverage this administrative data for decision-making is the theme for today's discussions. The PM has been focusing on data-driven decision-making. This is in the context that if we are talking of a Viksit Bharat by 2047, the policies that we take, the decisions that we make, have to be so much more focused and more data-driven. Ministry of Statistics is helping to ensure that the data is more harmonised to make it interoperable."

ANI

30,758 Aufrufe • vor 2 Monaten