Test-Time Training (TTT) is now on Video! And not just a 5-second video. We can generate a full 1-min video! TTT module is an RNN module that provides an explicit and efficient memory mechanism. It models the hidden state of an RNN with a machine learning model, which is... updated via gradient descent. Combined with a Diffusion Transformer, we are able to generate a 1-min Tom and Jerry cartoon. Enjoy our video with input script (not seen before): Jerry happily eats cheese in a tidy kitchen until Tom playfully takes it away, teasing him. Annoyed, Jerry packs his belongings and leaves home, dragging a small suitcase behind him. Later, Tom notices Jerry's absence, feels sad, and follows Jerry's tiny footprints all the way to San Francisco. Jerry sits disheartened in an alleyway, where Tom finds him, gently offering cheese as an apology. Jerry forgives Tom, accepts the cheese, and the two return home together, their friendship restored.show more

Input: Tom and Jerry visit a lively carnival, where Tom eagerly tries a ball-tossing game but repeatedly misses, causing Jerry to laugh at him. Frustrated, Tom becomes determined, but despite his best efforts, he can't knock down the can. Jerry confidently steps in and easily succeeds, knocking over the can and winning a shiny gold trophy. Jerry happily celebrates his victory, leaving Tom astonished, jealous, and annoyed as they leave the carnival together—Jerry proudly holding his prize while Tom trails behind grumpily. Output:

Input: Tom is happily eating an apple pie at the kitchen table. Jerry looks longingly wishing he had some. Jerry goes outside the front door of the house and rings the doorbell. While Tom comes to open the door, Jerry runs around the back to the kitchen. Jerry steals Tom's apple pie. Jerry runs to his mousehole carrying the pie, while Tom is chasing him. Just as Tom is about to catch Jerry, he makes it through the mouse hole and Tom slams into the wall. Output:

Input: In an underwater adventure, Jerry locates a treasure map and searches for the treasure while evading Tom in a chase through coral reefs and kelp forests. Jerry triumphantly discovers treasure inside a shipwreck, blissfully celebrating while Tom's pursuit leads him into trouble with a hungry shark. Output:

Input: On a sunny morning in New York, Tom, a blue-gray cat carrying a briefcase, arrives at his office in the World Trade Center. As he settles in, his computer suddenly shuts down – Jerry, a mischievous brown mouse, has chewed the cable. A chase ensues, ending with Tom crashing into the wall as Jerry escapes into his mousehole. Determined, Tom bursts through an office door, accidentally interrupting a meeting led by Spike, an irritated bulldog, who angrily sends him away. Safe in his cozy mousehole, Jerry laughs at the chaos. Output:

Website: Paper: Previous work on TTT for language modeling: Great collaboration with: @karansdalal @danielkoceja @GashonHussein @Jerry_XU_Jiarui @__yuezhao__ Youjin Song, Shihao Han, Ka Chun Cheung, @jankautz @guestrin @tatsu_hashimoto @sanmikoyejo @YejinChoinka , Yu Sun

Which Machine Learning model delivers stronger trading results? Check out this free Substack post where I compare several powerful models that beat the market and show yearly returns of over 20%.

Seems like some meta-learning is happening

Amazing results! It would be interesting to apply it to policy learning for long horizon tasks.

Wow.. it actually took me a while to notice the animation was AI generated. I didn’t expect such tech to be so impressive this quickly. Im surprised.

Super cool indeed!

Woah this is so cool… never would have guessed it was AI