正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

🚀 Introducing MegaFlow: Zero-Shot Large Displacement Optical Flow! 🔥 Code, models, and demo — all available now! 🎯 Large displacement motion estimation (optical flow, point tracking) has been a long-standing challenge. MegaFlow proposes a simple solution by leveraging pre-trained vision priors. Combined with a global matching formulation, we tackle... show more

Haofei Xu

1,326 subscribers

32,233 次观看 • 3 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Tracking Everything Everywhere All at Once paper page: present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively.

Tracking Everything Everywhere All at Once paper page: present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively.

AK

280,547 次观看 • 3 年前

Check out our #PAMI paper with code "Dense Continuous-Time Optical Flow from Event Cameras," where we show how to regress *continuous-time* trajectories of every pixel from event cameras alone or events plus frames! The key idea is to iteratively estimate per-pixel polynomials using a recurrent lookup and update scheme. Paper: Code: DOI: We present a method for estimating dense continuous-time optical flow from event data. Traditional dense optical flow methods compute the pixel displacement between two images. Due to missing information, these approaches cannot recover the pixel trajectories in the blind time between two images. We show that it is possible to compute per-pixel, continuous-time optical flow using events from an event camera. Events provide temporally fine-grained information about movement in pixel space due to their asynchronous nature and microsecond response time. We leverage these benefits to predict pixel trajectories densely in continuous time via parameterized Bézier curves. To achieve this, we build a neural network with strong inductive biases for this task: First, we build multiple sequential correlation volumes in time using event data. Second, we use Bézier curves to index these correlation volumes at multiple timestamps along the trajectory. Third, we use the retrieved correlation to update the Bézier curve representations iteratively. Our method can optionally include image pairs to boost performance further. To train and evaluate our model, we introduce a synthetic dataset (MultiFlow) that features moving objects and ground truth trajectories for every pixel. Our quantitative experiments suggest that our method successfully predicts pixel trajectories in continuous time and is competitive in the traditional two-view pixel displacement metric on MultiFlow and DSEC-Flow. Open source code and datasets are released to the public. Kudos to Mathias Gehrig Manasi Muglikar

Check out our #PAMI paper with code "Dense Continuous-Time Optical Flow from Event Cameras," where we show how to regress continuous-time trajectories of every pixel from event cameras alone or events plus frames! The key idea is to iteratively estimate per-pixel polynomials using a recurrent lookup and update scheme. Paper: Code: DOI: We present a method for estimating dense continuous-time optical flow from event data. Traditional dense optical flow methods compute the pixel displacement between two images. Due to missing information, these approaches cannot recover the pixel trajectories in the blind time between two images. We show that it is possible to compute per-pixel, continuous-time optical flow using events from an event camera. Events provide temporally fine-grained information about movement in pixel space due to their asynchronous nature and microsecond response time. We leverage these benefits to predict pixel trajectories densely in continuous time via parameterized Bézier curves. To achieve this, we build a neural network with strong inductive biases for this task: First, we build multiple sequential correlation volumes in time using event data. Second, we use Bézier curves to index these correlation volumes at multiple timestamps along the trajectory. Third, we use the retrieved correlation to update the Bézier curve representations iteratively. Our method can optionally include image pairs to boost performance further. To train and evaluate our model, we introduce a synthetic dataset (MultiFlow) that features moving objects and ground truth trajectories for every pixel. Our quantitative experiments suggest that our method successfully predicts pixel trajectories in continuous time and is competitive in the traditional two-view pixel displacement metric on MultiFlow and DSEC-Flow. Open source code and datasets are released to the public. Kudos to Mathias Gehrig Manasi Muglikar

Davide Scaramuzza

12,637 次观看 • 2 年前

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

AK

122,533 次观看 • 3 年前

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.

AI at Meta

218,966 次观看 • 1 年前

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

AK

125,319 次观看 • 3 年前

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!

The holy grail of robotics is to be able to perform previously-unseen, out-of-distribution manipulation tasks “zero shot” in a new environment. NovaFlow proposes an approach which (1) generates a video, (2) computes predicted flow — how points move through the scene — and (3) uses this flow as an objective to generate a motion. Using this procedure, NovaFlow generates motions in unseen scenes, for unseen tasks, and can transfer across embodiments. To learn more, we are joined by Hongyu Li and Jiahui Fu from RAI. Watch Episode #63 of RoboPapers with Chris Paxton and Michael Cho - Rbt/Acc now to learn more!

RoboPapers

21,295 次观看 • 5 个月前

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.

MolmoAct2 runs zero-shot on an SO-ARM101, no training required 🦾 Ai2 open Action Reasoning Model is now in LeRobot with the full lifecycle covered: fine-tuning (full or LoRA), evaluation, and real-robot deployment. It pairs a Molmo2-ER vision-language backbone with a flow-matching action expert to turn images, language, and proprioceptive state into action chunks. The best part: ready-made checkpoints ship with calibration correction baked in, so you can point it at your SO-100/101 and just watch it work. Inference fits in ~12GB at bf16, and LoRA fine-tuning runs on a single 24GB GPU. Big thanks to the Ai2 team for building this in the open.

LeRobot

19,891 次观看 • 14 天前

NVIDIA AI Open-Sources ViPE (Video Pose Engine): A Powerful and Versatile 3D Video Annotation Tool for Spatial AI ViPE integrates bundle adjustment with dense optical flow, sparse keypoint tracking, and metric depth priors to estimate camera intrinsics, poses, and dense depth maps at 3–5 FPS on a single GPU. It significantly improves over prior uncalibrated pose estimation methods, achieving 18% and 50% error reduction on TUM and KITTI benchmarks, respectively, and shows robustness to dynamic scenes and diverse camera models. Beyond the method, the NVIDIA team also released a large-scale dataset comprising ~100K real-world internet videos, 1M AI-generated videos, and 2K panoramic videos (≈96M frames) annotated with metric depth and poses. This dataset and engine aim to accelerate training for spatial AI tasks such as 3D reconstruction, video generation, and robotics.... full analysis: paper: codes: NVIDIA NVIDIA AI NVIDIAnewsroom NVIDIA Robotics NVIDIA AIDev NVIDIAdeveloper

NVIDIA AI Open-Sources ViPE (Video Pose Engine): A Powerful and Versatile 3D Video Annotation Tool for Spatial AI ViPE integrates bundle adjustment with dense optical flow, sparse keypoint tracking, and metric depth priors to estimate camera intrinsics, poses, and dense depth maps at 3–5 FPS on a single GPU. It significantly improves over prior uncalibrated pose estimation methods, achieving 18% and 50% error reduction on TUM and KITTI benchmarks, respectively, and shows robustness to dynamic scenes and diverse camera models. Beyond the method, the NVIDIA team also released a large-scale dataset comprising ~100K real-world internet videos, 1M AI-generated videos, and 2K panoramic videos (≈96M frames) annotated with metric depth and poses. This dataset and engine aim to accelerate training for spatial AI tasks such as 3D reconstruction, video generation, and robotics.... full analysis: paper: codes: NVIDIA NVIDIA AI NVIDIAnewsroom NVIDIA Robotics NVIDIA AIDev NVIDIAdeveloper

Marktechpost AI Dev News ⚡

217,453 次观看 • 10 个月前

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

LLaVA-3D A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Recent advancements in Large Multimodal Models (LMMs) have greatly enhanced their proficiency in 2D visual understanding tasks, enabling them to effectively process and understand images and videos. However, the development of LMMs with 3D-awareness for 3D scene understanding has been hindered by the lack of large-scale 3D vision-language datasets and powerful 3D encoders. In this paper, we introduce a simple yet effective framework called LLaVA-3D. Leveraging the strong 2D understanding priors from LLaVA, our LLaVA-3D efficiently adapts LLaVA for 3D scene understanding without compromising 2D understanding capabilities. To achieve this, we employ a simple yet effective representation, 3D Patch, which connects 2D CLIP patch features with their corresponding positions in 3D space. By integrating the 3D Patches into 2D LMMs and employing joint 2D and 3D vision-language instruction tuning, we establish a unified architecture for both 2D image understanding and 3D scene understanding. Experimental results show that LLaVA-3D converges 3.5x faster than existing 3D LMMs when trained on 3D vision-language datasets. Moreover, LLaVA-3D not only achieves state-of-the-art performance across various 3D tasks but also maintains comparable 2D image understanding and vision-language conversation capabilities with LLaVA.

AK

41,736 次观看 • 1 年前

Introducing Marigold 🌼 - a universal monocular depth estimator, delivering incredibly sharp predictions in the wild! Based on Stable Diffusion, it is trained with synthetic depth data only and excels in zero-shot adaptation to real-world imagery. Check it out: 🌐 Website: 🤗 Hugging Face Space: 📄 Paper: 👾 Code: The team: Bingxin Ke (KeBingxin), yours truly (Anton Obukhov), Shengyu Huang (Shengyu Huang), Nando Metzger (Nando Metzger), Rodrigo Caye Daudt (Rodrigo Daudt), and Konrad Schindler. #ComputerVision #PRS #ETHZurich

Introducing Marigold 🌼 - a universal monocular depth estimator, delivering incredibly sharp predictions in the wild! Based on Stable Diffusion, it is trained with synthetic depth data only and excels in zero-shot adaptation to real-world imagery. Check it out: 🌐 Website: 🤗 Hugging Face Space: 📄 Paper: 👾 Code: The team: Bingxin Ke (KeBingxin), yours truly (Anton Obukhov), Shengyu Huang (Shengyu Huang), Nando Metzger (Nando Metzger), Rodrigo Caye Daudt (Rodrigo Daudt), and Konrad Schindler. #ComputerVision #PRS #ETHZurich

Anton Obukhov

490,031 次观看 • 2 年前

InstantDrag Improving Interactivity in Drag-based Image Editing discuss: Drag-based image editing has recently gained popularity for its interactivity and precision. However, despite the ability of text-to-image models to generate samples within a second, drag editing still lags behind due to the challenge of accurately reflecting user interaction while maintaining image content. Some existing approaches rely on computationally intensive per-image optimization or intricate guidance-based methods, requiring additional inputs such as masks for movable regions and text prompts, thereby compromising the interactivity of the editing process. We introduce InstantDrag, an optimization-free pipeline that enhances interactivity and speed, requiring only an image and a drag instruction as input. InstantDrag consists of two carefully designed networks: a drag-conditioned optical flow generator (FlowGen) and an optical flow-conditioned diffusion model (FlowDiffusion). InstantDrag learns motion dynamics for drag-based image editing in real-world video datasets by decomposing the task into motion generation and motion-conditioned image generation. We demonstrate InstantDrag's capability to perform fast, photo-realistic edits without masks or text prompts through experiments on facial video datasets and general scenes. These results highlight the efficiency of our approach in handling drag-based image editing, making it a promising solution for interactive, real-time applications.

InstantDrag Improving Interactivity in Drag-based Image Editing discuss: Drag-based image editing has recently gained popularity for its interactivity and precision. However, despite the ability of text-to-image models to generate samples within a second, drag editing still lags behind due to the challenge of accurately reflecting user interaction while maintaining image content. Some existing approaches rely on computationally intensive per-image optimization or intricate guidance-based methods, requiring additional inputs such as masks for movable regions and text prompts, thereby compromising the interactivity of the editing process. We introduce InstantDrag, an optimization-free pipeline that enhances interactivity and speed, requiring only an image and a drag instruction as input. InstantDrag consists of two carefully designed networks: a drag-conditioned optical flow generator (FlowGen) and an optical flow-conditioned diffusion model (FlowDiffusion). InstantDrag learns motion dynamics for drag-based image editing in real-world video datasets by decomposing the task into motion generation and motion-conditioned image generation. We demonstrate InstantDrag's capability to perform fast, photo-realistic edits without masks or text prompts through experiments on facial video datasets and general scenes. These results highlight the efficiency of our approach in handling drag-based image editing, making it a promising solution for interactive, real-time applications.

AK

71,232 次观看 • 1 年前

Last week, we shared a refresher on our favorite Flow features. Today, we're doing a deep dive into prompting with a few tips on how to craft the optimal prompt, create your dream video, and #FindYourFlow 🕺 Clearly identify your characters or objects and describe their movements 🎥 Frame your shot with compositional terms like "wide shot" or “close-up" and direct the camera with instructions like "tracking shot" or "aerial view" 💡The lighting and environment set the entire mood. Instead of "a room" use more descriptive terms like "a dusty attic filled with forgotten treasures, a single beam of afternoon light cutting through a grimy window" 🧸 Flow is not limited to realistic visuals. You can experiment with a wide array of animation styles like "stop motion" or "knitted animation" 🗣️ With Veo 3, you can include sound effects and dialogue in your prompt. Describe the ambient sounds happening throughout the scene and generate dialogue by including it in your prompt. For more Flow tips, see here ➡️ 🚨Pro tip: this is another great bookmark 😉

Last week, we shared a refresher on our favorite Flow features. Today, we're doing a deep dive into prompting with a few tips on how to craft the optimal prompt, create your dream video, and #FindYourFlow 🕺 Clearly identify your characters or objects and describe their movements 🎥 Frame your shot with compositional terms like "wide shot" or “close-up" and direct the camera with instructions like "tracking shot" or "aerial view" 💡The lighting and environment set the entire mood. Instead of "a room" use more descriptive terms like "a dusty attic filled with forgotten treasures, a single beam of afternoon light cutting through a grimy window" 🧸 Flow is not limited to realistic visuals. You can experiment with a wide array of animation styles like "stop motion" or "knitted animation" 🗣️ With Veo 3, you can include sound effects and dialogue in your prompt. Describe the ambient sounds happening throughout the scene and generate dialogue by including it in your prompt. For more Flow tips, see here ➡️ 🚨Pro tip: this is another great bookmark 😉

Google Labs

33,868 次观看 • 1 年前

Pakistan's PRSC-EO3: an unusual orbit for an optical satellite Radar tracking via Leonardo Avella. Processed via COMSPOC SSA. PRSC-EO3 (visualized in cyan) launched April 25, 2026 on a Long March 6. It's an optical imager — but its orbit is curious. Most optical LEO satellites use sun-synchronous orbits (~97-105° inclination), which provide consistent lighting for imaging. PRSC-EO3 is in a 38° inclined orbit instead. This sacrifices global coverage and consistent lighting, but increases revisit rates over a specific latitude band: 20-40°N. That's India, Kashmir, and Pakistan. Now consider PRSC-S1 (visualized in pink), Pakistan's SAR satellite launched July 2025, sitting in a 41° orbit. Similar inclination, similar altitude — but their RAANs are ~175° out of phase. When one passes over South Asia in daylight, the other passes in darkness. SAR works day and night. Optical needs sunlight. The geometry appears to allow complementary coverage. We ran the access analysis [Image 1]. The SAR sensor (unconstrained) and optical sensor (daytime-constrained) together provide repeatable revisit across day and night. The gaps left by one are filled by the other. Then there's PRSC-HS1 — a hyperspectral satellite in SSO [Image 2], capable of detecting camouflage and identifying materials from orbit. Optical shows you the picture. SAR shows you the picture at night and through weather. Hyperspectral tells you what you're looking at. Five remote sensing satellites in 16 months [Image 2]. All launched by China. All with orbits favoring South Asian coverage. The stated missions are civilian. The orbital architecture appears consistent with a multi-modal ISR constellation. Space Domain Awareness , Jonathan McDowell, Joey Roulette, SpaceNews , Integrity ISR #Pakistan #Space #SAR #ISR #PRSC

Pakistan's PRSC-EO3: an unusual orbit for an optical satellite Radar tracking via Leonardo Avella. Processed via COMSPOC SSA. PRSC-EO3 (visualized in cyan) launched April 25, 2026 on a Long March 6. It's an optical imager — but its orbit is curious. Most optical LEO satellites use sun-synchronous orbits (~97-105° inclination), which provide consistent lighting for imaging. PRSC-EO3 is in a 38° inclined orbit instead. This sacrifices global coverage and consistent lighting, but increases revisit rates over a specific latitude band: 20-40°N. That's India, Kashmir, and Pakistan. Now consider PRSC-S1 (visualized in pink), Pakistan's SAR satellite launched July 2025, sitting in a 41° orbit. Similar inclination, similar altitude — but their RAANs are ~175° out of phase. When one passes over South Asia in daylight, the other passes in darkness. SAR works day and night. Optical needs sunlight. The geometry appears to allow complementary coverage. We ran the access analysis [Image 1]. The SAR sensor (unconstrained) and optical sensor (daytime-constrained) together provide repeatable revisit across day and night. The gaps left by one are filled by the other. Then there's PRSC-HS1 — a hyperspectral satellite in SSO [Image 2], capable of detecting camouflage and identifying materials from orbit. Optical shows you the picture. SAR shows you the picture at night and through weather. Hyperspectral tells you what you're looking at. Five remote sensing satellites in 16 months [Image 2]. All launched by China. All with orbits favoring South Asian coverage. The stated missions are civilian. The orbital architecture appears consistent with a multi-modal ISR constellation. Space Domain Awareness , Jonathan McDowell, Joey Roulette, SpaceNews , Integrity ISR #Pakistan #Space #SAR #ISR #PRSC

COMSPOC_OPS

58,022 次观看 • 2 个月前

SAMURAI vs. MetaAI's SAM 2! Traditional visual object tracking struggles in crowded, fast-moving, or self-occluded scenes, as does SAM2. Meet SAMURAI: a completely open-source adaptation of the Segment Anything Model for zero-shot visual tracking! Here's why it's a game-changer: 🚫 No need for retraining or finetuning 🎯 Boosts success rate and precision 🤖 Motion-aware memory selection 💪 Zero-shot performance on diverse datasets But that's not all: 🔬 Refines mask selection 🔮 Predicts object motion effectively 📈 Gains: 7.1% AUC on LaSOT, 3.5% AO on GOT-10k 🏆 Competes with fully supervised methods without extra training Link to the GitHub repo in the next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights & tutorials on AI and Machine Learning.

SAMURAI vs. MetaAI's SAM 2! Traditional visual object tracking struggles in crowded, fast-moving, or self-occluded scenes, as does SAM2. Meet SAMURAI: a completely open-source adaptation of the Segment Anything Model for zero-shot visual tracking! Here's why it's a game-changer: 🚫 No need for retraining or finetuning 🎯 Boosts success rate and precision 🤖 Motion-aware memory selection 💪 Zero-shot performance on diverse datasets But that's not all: 🔬 Refines mask selection 🔮 Predicts object motion effectively 📈 Gains: 7.1% AUC on LaSOT, 3.5% AO on GOT-10k 🏆 Competes with fully supervised methods without extra training Link to the GitHub repo in the next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights & tutorials on AI and Machine Learning.

Akshay 🚀

363,328 次观看 • 1 年前

This seemingly obvious prediction didn't take long to become reality. MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors Contributions: • The first real-time SLAM system using the two-view 3D reconstruction prior MASt3R [20] as a foundation. • Efficient techniques for pointmap matching, tracking and local fusion, graph construction and loop closure, and second-order global optimization. • A state-of-the-art dense SLAM system capable of handling generic, time-varying camera models. Abstract: We present a real-time monocular dense SLAM system, designed from the ground up using MASt3R, a two-view 3D reconstruction and matching prior. Equipped with this strong prior, our system remains robust on in-the-wild video sequences, making no assumptions on a fixed or parametric camera model beyond a unique camera center. Key features include: - Efficient methods for pointmap matching, camera tracking, and local fusion - Graph construction and loop closure - Second-order global optimization With known calibration, a simple modification achieves state-of-the-art performance across various benchmarks. Altogether, we propose a plug-and-play monocular SLAM system capable of producing globally-consistent poses and dense geometry while operating at 15 FPS.

This seemingly obvious prediction didn't take long to become reality. MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors Contributions: • The first real-time SLAM system using the two-view 3D reconstruction prior MASt3R [20] as a foundation. • Efficient techniques for pointmap matching, tracking and local fusion, graph construction and loop closure, and second-order global optimization. • A state-of-the-art dense SLAM system capable of handling generic, time-varying camera models. Abstract: We present a real-time monocular dense SLAM system, designed from the ground up using MASt3R, a two-view 3D reconstruction and matching prior. Equipped with this strong prior, our system remains robust on in-the-wild video sequences, making no assumptions on a fixed or parametric camera model beyond a unique camera center. Key features include: - Efficient methods for pointmap matching, camera tracking, and local fusion - Graph construction and loop closure - Second-order global optimization With known calibration, a simple modification achieves state-of-the-art performance across various benchmarks. Altogether, we propose a plug-and-play monocular SLAM system capable of producing globally-consistent poses and dense geometry while operating at 15 FPS.

MrNeRF

29,961 次观看 • 1 年前

This week, we tested 3 latest models in our Game Arena Benchmark: → O3 → O4-mini → Gemini 2.5 Flash Across 4 games—Phoenix Wright, Sokoban, Candy Crush, and 2048—O3 dominated the zero-shot leaderboard, ranking #1 or #2 in nearly every task and outperforming previous SOTA models like O3-mini and Gemini 2.5 Flash. 🔥 Beyond our customized tests, we took on a real challenge: Sokoban (1989)—the classic, unforgiving original. 🗣️ Many say O3 shows strong image reasoning, but does it really? Let’s see what our Game Arena Benchmark reveals. 👇

This week, we tested 3 latest models in our Game Arena Benchmark: → O3 → O4-mini → Gemini 2.5 Flash Across 4 games—Phoenix Wright, Sokoban, Candy Crush, and 2048—O3 dominated the zero-shot leaderboard, ranking #1 or #2 in nearly every task and outperforming previous SOTA models like O3-mini and Gemini 2.5 Flash. 🔥 Beyond our customized tests, we took on a real challenge: Sokoban (1989)—the classic, unforgiving original. 🗣️ Many say O3 shows strong image reasoning, but does it really? Let’s see what our Game Arena Benchmark reveals. 👇

Hao AI Lab

14,636 次观看 • 1 年前

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 次观看 • 3 年前

🔥In Magma, we talked a lot about spatial/temporal intelligence beyond verbal intelligencen as advocated by Dr. Fei-Fei Li. So how to interpret it? Today I am happy to announce a new demo Magma-Gaming: 👉 Rather than asking LLMs to write game code, we further ask the model to PLAY the game. A simple game like moving to the target in a 2D grid still requires precise action grounding and planning capability, yet challenges the most advanced VLMs and even GPT-4o-mini model. Magma, born with stronger spatial understanding and reasoning ability, significantly outperforms the counterparts and achieves much higher scores in zero-shot manner. This result pinpoints the huge potential of building multimodal agentic models endowed with both verbal and spatial inteligence! Also, I believe this simple demo gives you a better hints to understand what is spatial intelligence, and why it is important!

🔥In Magma, we talked a lot about spatial/temporal intelligence beyond verbal intelligencen as advocated by Dr. Fei-Fei Li. So how to interpret it? Today I am happy to announce a new demo Magma-Gaming: 👉 Rather than asking LLMs to write game code, we further ask the model to PLAY the game. A simple game like moving to the target in a 2D grid still requires precise action grounding and planning capability, yet challenges the most advanced VLMs and even GPT-4o-mini model. Magma, born with stronger spatial understanding and reasoning ability, significantly outperforms the counterparts and achieves much higher scores in zero-shot manner. This result pinpoints the huge potential of building multimodal agentic models endowed with both verbal and spatial inteligence! Also, I believe this simple demo gives you a better hints to understand what is spatial intelligence, and why it is important!

Jianwei Yang

17,940 次观看 • 1 年前

Introducing FoundationMotion. A large-scale, video-derived motion annotation dataset & auto-labeling pipeline + advanced models for motion understanding. Fully open-source: code, datasets, and models, free to use and build on. Understanding motion is core to physical reasoning, yet today’s leading models still struggle with simple spatial actions like “turn right” or “move up” or “flip the toast” - mainly due to the lack of large, fine-grained motion datasets. We present FoundationMotion, a fully automated pipeline that: • detects & tracks objects in videos • extracts trajectories • uses LLMs + frames to generate rich motion captions & QA pairs → creating large-scale, high-quality motion datasets at scale. After fine-tuning the open-source models Qwen and NVILA on our annotations, these models now outperform the closed-source Gemini-3-Flash and GPT-5.1 on spatial understanding tasks across autonomous driving, robotics, and everyday scenarios. 📜Paper: 🌐Webpage: 💻 Code: 🕸️Model: 📊 Dataset: 👉 Interactive Demo: Let’s move research forward together. FoundationMotion is also referred to as Wolf V2 🐺, the second chapter in the Wolf series:

Introducing FoundationMotion. A large-scale, video-derived motion annotation dataset & auto-labeling pipeline + advanced models for motion understanding. Fully open-source: code, datasets, and models, free to use and build on. Understanding motion is core to physical reasoning, yet today’s leading models still struggle with simple spatial actions like “turn right” or “move up” or “flip the toast” - mainly due to the lack of large, fine-grained motion datasets. We present FoundationMotion, a fully automated pipeline that: • detects & tracks objects in videos • extracts trajectories • uses LLMs + frames to generate rich motion captions & QA pairs → creating large-scale, high-quality motion datasets at scale. After fine-tuning the open-source models Qwen and NVILA on our annotations, these models now outperform the closed-source Gemini-3-Flash and GPT-5.1 on spatial understanding tasks across autonomous driving, robotics, and everyday scenarios. 📜Paper: 🌐Webpage: 💻 Code: 🕸️Model: 📊 Dataset: 👉 Interactive Demo: Let’s move research forward together. FoundationMotion is also referred to as Wolf V2 🐺, the second chapter in the Wolf series:

Boyi Li

66,999 次观看 • 7 个月前

Google approached me and asked if I would be down to collab with their research team to create a film for Google I/O. Not only would I have access to Flow, their latest AI filmmaking tool, but I would also be responsible for giving feedback on the product itself. The filmmaker in me was excited for the challenge, but the technologist in me was living out my dream in helping mold a next-generation product for a new era of filmmaking. Experimenting with the tool has been a fun ride. Flow is built on Google Deepmind’s most advanced models - Veo, Imagen, and Gemini, and has allowed me to world-build and create characters I have had in my head for years. I can create consistent characters, scenes, and stories with the ability to reference key ingredients. With Flow, it finally feels like filmmakers are in control. We can iterate freely and actually steer the ship in our creations. I decided to create my short film “Freelancers” using Flow: An action-packed ride following two estranged adopted brothers—one methodical, the other loud and brash—who reminisce on their careers as international hitmen during dinner, until they realize they’ve both been contracted to carry out the same job. GoogleLabs #EarlyAccess #FlowbyGoogle #Veo Try it out at

Google approached me and asked if I would be down to collab with their research team to create a film for Google I/O. Not only would I have access to Flow, their latest AI filmmaking tool, but I would also be responsible for giving feedback on the product itself. The filmmaker in me was excited for the challenge, but the technologist in me was living out my dream in helping mold a next-generation product for a new era of filmmaking. Experimenting with the tool has been a fun ride. Flow is built on Google Deepmind’s most advanced models - Veo, Imagen, and Gemini, and has allowed me to world-build and create characters I have had in my head for years. I can create consistent characters, scenes, and stories with the ability to reference key ingredients. With Flow, it finally feels like filmmakers are in control. We can iterate freely and actually steer the ship in our creations. I decided to create my short film “Freelancers” using Flow: An action-packed ride following two estranged adopted brothers—one methodical, the other loud and brash—who reminisce on their careers as international hitmen during dinner, until they realize they’ve both been contracted to carry out the same job. GoogleLabs #EarlyAccess #FlowbyGoogle #Veo Try it out at

Dave Clark

162,122 次观看 • 1 年前