Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

MetaAI's SAM 2 struggles when things move fast or when there are crowded, fast-moving objects! Introducing SAMURAI: An adaptation of the Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory. 100% Open Source

Sumanth

76,668 subscribers

324,599 views • 1 year ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

11 Comments

Sumanth1 year ago

Github Repo:

Sumanth1 year ago

If you find this useful, RT to share it with your friends. Don't forget to follow me @Sumanth_077 for more such content and tutorials on Python, AI & ML!

Page to Pixel Publishing2 years ago

The Art of Flight is a homage to 80s/90s arcade action shmups with a fresh twist on the genre. Pilot multiple ships at the same time to take on oncoming waves of enemies in this fast paced space shooter. Wishlist on Steam today!

Rethynk AI1 year ago

That’s a brilliant upgrade! SAMURAI sounds like a game-changer for dynamic environments where traditional SAM models fall short. Motion-aware memory could make zero-shot visual tracking far more robust, especially in real-world applications like sports analysis or autonomous vehicles.

Sumanth1 year ago

Absolutely!

anarki🌟1 year ago

instant @arXivBangers haha let’s gooooo!!!!!

Pérry Odé 🇪🇺🇩🇪🇺🇦🇮🇱1 year ago

If I were a producer of self-shooting and AI-driven drones, SAMURAI would be preferable in this case. 👍

Machine Learning Community ⭐️1 year ago

Impressive!

Sumanth1 year ago

Indeed!

Appy Pie1 year ago

MetaAI's SAM 2 meets its match with fast motion and crowded scenes, but SAMURAI steps in! Motion-aware memory and zero-shot visual tracking make it a game-changer. Plus, it's 100% open source!

tarama1 year ago

@BenjaminDEKR fyi

Related Videos

SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware check out this SAM2 vs SAMURAI comparison! - paper: - code: - license: Apache-2.0

SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware check out this SAM2 vs SAMURAI comparison! - paper: - code: - license: Apache-2.0

SkalskiP

124,419 views • 1 year ago

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.

Bilawal Sidhu

47,246 views • 1 year ago

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

AI at Meta

97,755 views • 2 years ago

Introducing VT Tensiontrack a cylindrical tracking scenario with a mix of fast, short & long strafes. Uniquely made on Aimlabs with an indicator that informs the player when to increase tension or loosen up. Great for practicing tension control! ✏️ VT LG56 🚀 | 🎯 VT Matty

Introducing VT Tensiontrack a cylindrical tracking scenario with a mix of fast, short & long strafes. Uniquely made on Aimlabs with an indicator that informs the player when to increase tension or loosen up. Great for practicing tension control! ✏️ VT LG56 🚀 | 🎯 VT Matty

Voltaic

27,036 views • 2 years ago

🚨Fast Moving Objects Captured With Infrared Camera Nov. 24, 2025 around 10pm PST - Los Angeles, CA Just found this interesting capture. The OP states: "3 objects captured with an infrared camera in the night sky above Los Angeles. They were moving fast across the sky, much faster than typical airliners I see. These objects did not appear on my flight tracker app. There are two parts of this clip, both played back in real time; one that shows the original camera perspective, the second part of clip is zoomed in and stabilized." #ufotwitter #uapX Source:

🚨Fast Moving Objects Captured With Infrared Camera Nov. 24, 2025 around 10pm PST - Los Angeles, CA Just found this interesting capture. The OP states: "3 objects captured with an infrared camera in the night sky above Los Angeles. They were moving fast across the sky, much faster than typical airliners I see. These objects did not appear on my flight tracker app. There are two parts of this clip, both played back in real time; one that shows the original camera perspective, the second part of clip is zoomed in and stabilized." #ufotwitter #uapX Source:

Skywatch Signal

53,070 views • 7 months ago

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!

Akshay 🚀

46,421 views • 8 months ago

THAT MOMENT WHEN NVIDIA CEO JENSEN HUANG WAS ASKED ABOUT HIS COMMENTS ON CHINA BEATING THE U.S. IN THE AI RACE: “THAT’S NOT WHAT I SAID. WHAT I SAID WAS THAT CHINA HAS VERY GOOD AI TECHNOLOGY. THEY HAVE MANY AI RESEARCHERS. IN FACT, 50% OF THE WORLD’S AI RESEARCHERS ARE IN CHINA. THEY DEVELOP VERY GOOD AI TECHNOLOGY. THE MOST POPULAR OPEN-SOURCE AI MODELS IN THE WORLD TODAY ARE FROM CHINA. SO THEY ARE MOVING VERY, VERY FAST. THE UNITED STATES HAS TO CONTINUE TO MOVE INCREDIBLY FAST. OTHERWISE, THE WORLD IS VERY COMPETITIVE, SO WE HAVE TO RUN FAST.”

THAT MOMENT WHEN NVIDIA CEO JENSEN HUANG WAS ASKED ABOUT HIS COMMENTS ON CHINA BEATING THE U.S. IN THE AI RACE: “THAT’S NOT WHAT I SAID. WHAT I SAID WAS THAT CHINA HAS VERY GOOD AI TECHNOLOGY. THEY HAVE MANY AI RESEARCHERS. IN FACT, 50% OF THE WORLD’S AI RESEARCHERS ARE IN CHINA. THEY DEVELOP VERY GOOD AI TECHNOLOGY. THE MOST POPULAR OPEN-SOURCE AI MODELS IN THE WORLD TODAY ARE FROM CHINA. SO THEY ARE MOVING VERY, VERY FAST. THE UNITED STATES HAS TO CONTINUE TO MOVE INCREDIBLY FAST. OTHERWISE, THE WORLD IS VERY COMPETITIVE, SO WE HAVE TO RUN FAST.”

Vivek Sen

155,599 views • 2 months ago

🇺🇸VIVEK: TRUMP WORKS FAST—HOW IS THAT A BAD THING? "He definitely is moving at a faster pace than probably any president has in modern history or in our lifetimes. The fact that the best allegation [Democrats] are coming up with is that 'the pace is just too crazy' when you had open borders and a flailing economy. I do think that Fast and Furious is exactly what we need." Source: Vivek Ramaswamy, Shawn Ryan Show

🇺🇸VIVEK: TRUMP WORKS FAST—HOW IS THAT A BAD THING? "He definitely is moving at a faster pace than probably any president has in modern history or in our lifetimes. The fact that the best allegation [Democrats] are coming up with is that 'the pace is just too crazy' when you had open borders and a flailing economy. I do think that Fast and Furious is exactly what we need." Source: Vivek Ramaswamy, Shawn Ryan Show

Mario Nawfal

604,774 views • 1 year ago

💥 A 450M model just beat bigger VLAs on real robot tasks, and it’s 100% open source [📍 bookmark for later] Came across SmolVLA, a new vision-language-action model for robotics that’s compact, fast, and trained entirely on open community datasets from LeRobot via Hugging Face. What stood out to me is how it matches or outperforms much larger models like ACT using noisy, real-world community data instead of giant private datasets. Why it’s worth a look ✅ 26% performance boost from pretraining on open-source data ✅ Runs on consumer hardware, even a MacBook ✅ 30% faster responses with async inference and smart architecture tweaks ✅ Strong results across Meta-World, LIBERO, SO100, and SO101 ✅ Fully open source: weights, code, training pipeline, eval stack They also introduced smart efficiency tricks like using fewer visual tokens, pulling outputs from mid-layer, and separating perception from action to make it all run fast. SmolVLA is a strong case for what can happen when the robotics community shares data and builds in the open. Definitely worth keeping an eye on.

💥 A 450M model just beat bigger VLAs on real robot tasks, and it’s 100% open source [📍 bookmark for later] Came across SmolVLA, a new vision-language-action model for robotics that’s compact, fast, and trained entirely on open community datasets from LeRobot via Hugging Face. What stood out to me is how it matches or outperforms much larger models like ACT using noisy, real-world community data instead of giant private datasets. Why it’s worth a look ✅ 26% performance boost from pretraining on open-source data ✅ Runs on consumer hardware, even a MacBook ✅ 30% faster responses with async inference and smart architecture tweaks ✅ Strong results across Meta-World, LIBERO, SO100, and SO101 ✅ Fully open source: weights, code, training pipeline, eval stack They also introduced smart efficiency tricks like using fewer visual tokens, pulling outputs from mid-layer, and separating perception from action to make it all run fast. SmolVLA is a strong case for what can happen when the robotics community shares data and builds in the open. Definitely worth keeping an eye on.

Ilir Aliu - eu/acc

17,353 views • 11 months ago

Meta released SAM3 a model to reconstruct objects from segmented images. I tried it and here are 4 things I found out - The generation is incredibly fast when compared with other 3D GenAI models - The models generated are not meshes but Gaussian Splatting. I was surprised that in the paper it was not mentioned anywhere - When generating multiple objects the arrangement is a bit messed up. - Unfortunately, it is impossible to download any of the .ply generated 😔 I cannot stop thinking about the implications of this in XR. Objects in the real world could be almost instantly turned into photorealistic copies and even rearranged to the user's wish.

Meta released SAM3 a model to reconstruct objects from segmented images. I tried it and here are 4 things I found out - The generation is incredibly fast when compared with other 3D GenAI models - The models generated are not meshes but Gaussian Splatting. I was surprised that in the paper it was not mentioned anywhere - When generating multiple objects the arrangement is a bit messed up. - Unfortunately, it is impossible to download any of the .ply generated 😔 I cannot stop thinking about the implications of this in XR. Objects in the real world could be almost instantly turned into photorealistic copies and even rearranged to the user's wish.

Gabriele Romagnoli

32,439 views • 8 months ago

CAPTCHA IS OFFICIALLY OUTDATED A new open-source library called Cap is changing how websites stop bots. No puzzles. No traffic lights. No “select all bikes” anymore. Instead, it uses a SHA-256 proof-of-work system — simple, silent, and fast. Why devs are switching: • Only ~20KB in size • Zero tracking, zero data collection • No images, no user friction • Works with any JS runtime • Fully customizable (visible, invisible, floating modes) • Zero dependencies • Can be deployed instantly via Docker This is a full replacement for traditional CAPTCHA systems. Cleaner UX. Faster websites. Better privacy. 100% open-source on GitHub Link in comments.

CAPTCHA IS OFFICIALLY OUTDATED A new open-source library called Cap is changing how websites stop bots. No puzzles. No traffic lights. No “select all bikes” anymore. Instead, it uses a SHA-256 proof-of-work system — simple, silent, and fast. Why devs are switching: • Only ~20KB in size • Zero tracking, zero data collection • No images, no user friction • Works with any JS runtime • Fully customizable (visible, invisible, floating modes) • Zero dependencies • Can be deployed instantly via Docker This is a full replacement for traditional CAPTCHA systems. Cleaner UX. Faster websites. Better privacy. 100% open-source on GitHub Link in comments.

divyansh tiwari

113,183 views • 3 months ago

Introducing FoundationMotion. A large-scale, video-derived motion annotation dataset & auto-labeling pipeline + advanced models for motion understanding. Fully open-source: code, datasets, and models, free to use and build on. Understanding motion is core to physical reasoning, yet today’s leading models still struggle with simple spatial actions like “turn right” or “move up” or “flip the toast” - mainly due to the lack of large, fine-grained motion datasets. We present FoundationMotion, a fully automated pipeline that: • detects & tracks objects in videos • extracts trajectories • uses LLMs + frames to generate rich motion captions & QA pairs → creating large-scale, high-quality motion datasets at scale. After fine-tuning the open-source models Qwen and NVILA on our annotations, these models now outperform the closed-source Gemini-3-Flash and GPT-5.1 on spatial understanding tasks across autonomous driving, robotics, and everyday scenarios. 📜Paper: 🌐Webpage: 💻 Code: 🕸️Model: 📊 Dataset: 👉 Interactive Demo: Let’s move research forward together. FoundationMotion is also referred to as Wolf V2 🐺, the second chapter in the Wolf series:

Introducing FoundationMotion. A large-scale, video-derived motion annotation dataset & auto-labeling pipeline + advanced models for motion understanding. Fully open-source: code, datasets, and models, free to use and build on. Understanding motion is core to physical reasoning, yet today’s leading models still struggle with simple spatial actions like “turn right” or “move up” or “flip the toast” - mainly due to the lack of large, fine-grained motion datasets. We present FoundationMotion, a fully automated pipeline that: • detects & tracks objects in videos • extracts trajectories • uses LLMs + frames to generate rich motion captions & QA pairs → creating large-scale, high-quality motion datasets at scale. After fine-tuning the open-source models Qwen and NVILA on our annotations, these models now outperform the closed-source Gemini-3-Flash and GPT-5.1 on spatial understanding tasks across autonomous driving, robotics, and everyday scenarios. 📜Paper: 🌐Webpage: 💻 Code: 🕸️Model: 📊 Dataset: 👉 Interactive Demo: Let’s move research forward together. FoundationMotion is also referred to as Wolf V2 🐺, the second chapter in the Wolf series:

Boyi Li

66,999 views • 8 months ago

Is There a 70-Year Plan Behind the Global Crisis? There are warnings of a global famine. Not someday. But something being set in motion right now. Key trade routes are under pressure. The Strait of Hormuz — one of the most critical pathways for global energy and supply. If that gets disrupted… it doesn’t just affect oil. It affects food. Supply chains. Entire nations. And when systems like this start breaking down… the consequences move fast. So the real question is: Is this a crisis… or something being engineered? Alex Newman

Is There a 70-Year Plan Behind the Global Crisis? There are warnings of a global famine. Not someday. But something being set in motion right now. Key trade routes are under pressure. The Strait of Hormuz — one of the most critical pathways for global energy and supply. If that gets disrupted… it doesn’t just affect oil. It affects food. Supply chains. Entire nations. And when systems like this start breaking down… the consequences move fast. So the real question is: Is this a crisis… or something being engineered? Alex Newman

Sarah Westall

471,584 views • 4 months ago

1. Smear frames replicate real life motion blur that happens in cameras and our eyes. So be careful to only use them when a movement is too fast to follow. That usually means an object is moving a large distance in 1 to 2 frames. A basic rule is that the audience should have trouble telling if you used a smear or not. (But don’t worry they will feel it in the smoothness of motion) 2. Old looney tunes cartoons are great to reference when practicing smears as those animators really pushed the limits of what could work :) 3. Anticipation and follow through are the secret to making smears effective. They allow audiences to make sense of what happened during a quick motion.

1. Smear frames replicate real life motion blur that happens in cameras and our eyes. So be careful to only use them when a movement is too fast to follow. That usually means an object is moving a large distance in 1 to 2 frames. A basic rule is that the audience should have trouble telling if you used a smear or not. (But don’t worry they will feel it in the smoothness of motion) 2. Old looney tunes cartoons are great to reference when practicing smears as those animators really pushed the limits of what could work :) 3. Anticipation and follow through are the secret to making smears effective. They allow audiences to make sense of what happened during a quick motion.

Hater

67,404 views • 4 months ago

remind-reid-tracker REMIND — RE-Identification with Memory for INDoor Navigation REMIND addresses a core challenge in visual tracking: re-identifying objects that disappear and reappear, look similar to one another, or are observed from changing viewpoints. Rather than relying on position or motion cues, REMIND builds appearance-based identity models per object using DINOv3 patch features, decomposed into: Global and part-level descriptors — K-means and attention-guided semantic parts extracted per detection Relational context (neighbor sets) — structural scene layout encoded as co-occurrence graphs of neighboring objects Known-set distance disambiguation — geometry-aware resolution of visually ambiguous groups Adaptive memory — per-object appearance, part, and background models that update over time The association pipeline runs a per-frame sequence of visual evidence building, context activation, global Hungarian assignment, and post-assignment guards, producing explicit uncertainty signals (ambiguous, provisional) alongside confident identity decisions. Evaluation outputs span case, object, frame, class, scene, and batch levels, with full internal telemetry for diagnostic analysis.

remind-reid-tracker REMIND — RE-Identification with Memory for INDoor Navigation REMIND addresses a core challenge in visual tracking: re-identifying objects that disappear and reappear, look similar to one another, or are observed from changing viewpoints. Rather than relying on position or motion cues, REMIND builds appearance-based identity models per object using DINOv3 patch features, decomposed into: Global and part-level descriptors — K-means and attention-guided semantic parts extracted per detection Relational context (neighbor sets) — structural scene layout encoded as co-occurrence graphs of neighboring objects Known-set distance disambiguation — geometry-aware resolution of visually ambiguous groups Adaptive memory — per-object appearance, part, and background models that update over time The association pipeline runs a per-frame sequence of visual evidence building, context activation, global Hungarian assignment, and post-assignment guards, producing explicit uncertainty signals (ambiguous, provisional) alongside confident identity decisions. Evaluation outputs span case, object, frame, class, scene, and batch levels, with full internal telemetry for diagnostic analysis.

Ryohei Sasaki@engineer

18,478 views • 18 days ago

🚨🇺🇸 SEC. RUBIO ON 'AGE OF DISCLOSURE': "THERE ARE OBJECTS FLYING OVER RESTRICTED AIRSPACE... AND THEY’RE NOT OURS" Secretary of State Marco Rubio addressed his appearance in the "Age of Disclosure" documentary, standing by his previous statements about unidentified objects flying over military installations and classified programs hidden even from presidents: "First of all, I'm not disavowing it. It was an interview done maybe three or four years ago when I was in the Senate. I was describing what people had said to me, navy pilots, admirals, generals, who came forward saying there were programs in the U.S. government that not even presidents were made aware of. The fundamental comments are true: there have been things that fly over restricted airspace, where we conduct military exercises, and everyone in the government says they're not ours. What I worry most about is that some adversary, another country, has developed an asymmetric capability for surveillance or something we just don't fully understand yet." When the Secretary of State confirms objects are flying over military bases and the government claims they're not ours, that's either aliens or a massive national security failure. Neither option is great. Source: Sean Hannity 🇺🇸

🚨🇺🇸 SEC. RUBIO ON 'AGE OF DISCLOSURE': "THERE ARE OBJECTS FLYING OVER RESTRICTED AIRSPACE... AND THEY’RE NOT OURS" Secretary of State Marco Rubio addressed his appearance in the "Age of Disclosure" documentary, standing by his previous statements about unidentified objects flying over military installations and classified programs hidden even from presidents: "First of all, I'm not disavowing it. It was an interview done maybe three or four years ago when I was in the Senate. I was describing what people had said to me, navy pilots, admirals, generals, who came forward saying there were programs in the U.S. government that not even presidents were made aware of. The fundamental comments are true: there have been things that fly over restricted airspace, where we conduct military exercises, and everyone in the government says they're not ours. What I worry most about is that some adversary, another country, has developed an asymmetric capability for surveillance or something we just don't fully understand yet." When the Secretary of State confirms objects are flying over military bases and the government claims they're not ours, that's either aliens or a massive national security failure. Neither option is great. Source: Sean Hannity 🇺🇸

Mario Nawfal

16,925 views • 8 months ago

Why do time slow down as you approach the speed of light? Because space and time are not separate things. They are woven together into spacetime, and the speed of light is built into its structure. Every object moves through spacetime at the same fundamental rate. When you are at rest, almost all of that motion is through time. As you move faster through space, some of that motion is redirected away from time. The total stays constant, so your clock must tick more slowly. This is not philosophy or metaphor. It has been measured with atomic clocks on airplanes and satellites. Move fast enough, and you do not just cross distances differently, you experience less time itself.

Why do time slow down as you approach the speed of light? Because space and time are not separate things. They are woven together into spacetime, and the speed of light is built into its structure. Every object moves through spacetime at the same fundamental rate. When you are at rest, almost all of that motion is through time. As you move faster through space, some of that motion is redirected away from time. The total stays constant, so your clock must tick more slowly. This is not philosophy or metaphor. It has been measured with atomic clocks on airplanes and satellites. Move fast enough, and you do not just cross distances differently, you experience less time itself.

The Scientific Lens

12,149 views • 6 months ago

How can we build a general-purpose “foundation model” for robot motion? Zhengyi “Zen” Luo joins us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a model with a generally-useful embedding space that can be controlled by a VLA, or from human video, to perform a wide variety of humanoid whole-body-control tasks, including with zero-shot transfer to previously unseen motions. Watch episode 72 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, now!

How can we build a general-purpose “foundation model” for robot motion? Zhengyi “Zen” Luo joins us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a model with a generally-useful embedding space that can be controlled by a VLA, or from human video, to perform a wide variety of humanoid whole-body-control tasks, including with zero-shot transfer to previously unseen motions. Watch episode 72 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, now!

RoboPapers

24,802 views • 4 months ago

SITUATION EXPLAINED: What happens to open source when the US restricts access to frontier AI models? We asked Adrian Dittmann: "China is releasing these models [open source] to gain international dominance, people will work on these things and say, 'Hey, maybe I can contribute to this if it's open source.' This is the thing that actually gets them talent." "If you're not allowed into the next GPT release or you're not allowed to use Fable, then what are you gonna do? You're stuck with the existing stuff, or you will have to use an open source model that might have subtly better capabilities." "The United States will catch up to open source in some form as well. They seem to just be maximizing product creation." "The problem with open source models is you have to be kind of a nerd in order to use them properly. Consumer facing, the effects will be minimal because the average person doesn't care about open source models. They only care whether or not they can do a quick search with Gemini or ChatGPT."

SITUATION EXPLAINED: What happens to open source when the US restricts access to frontier AI models? We asked Adrian Dittmann: "China is releasing these models [open source] to gain international dominance, people will work on these things and say, 'Hey, maybe I can contribute to this if it's open source.' This is the thing that actually gets them talent." "If you're not allowed into the next GPT release or you're not allowed to use Fable, then what are you gonna do? You're stuck with the existing stuff, or you will have to use an open source model that might have subtly better capabilities." "The United States will catch up to open source in some form as well. They seem to just be maximizing product creation." "The problem with open source models is you have to be kind of a nerd in order to use them properly. Consumer facing, the effects will be minimal because the average person doesn't care about open source models. They only care whether or not they can do a quick search with Gemini or ChatGPT."

MTS

14,406 views • 1 month ago