Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Introducing Subtask Annotations in LeRobot Datasets. LeRobot now supports subtasks as a native dataset field, so you can label task progression frame-by-frame, train hierarchical policies, and build stronger reward models with stage-aware supervision. Available today, with an annotation space + tooling to easily add subtasks and high-level dialogue to... show more

LeRobot

17,477 subscribers

16,524 görüntüleme • 7 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Most robot datasets store one sentence per episode: the task. Real problems need more: a plan, a memory, room to answer questions and change course. We just shipped full language support in the LeRobot dataset format. Now lerobot-annotate points a vision-language model at each episode and writes the subtasks, plans, memory, and grounded Q&A itself - roughly $1.20 for 100 episodes. From there, recipes turn the annotated rows into policy-agnostic training messages, so the same dataset can teach planning, memory, tool calls, or low-level control, no dataset redesign required. Read how it all works on the blog:

Most robot datasets store one sentence per episode: the task. Real problems need more: a plan, a memory, room to answer questions and change course. We just shipped full language support in the LeRobot dataset format. Now lerobot-annotate points a vision-language model at each episode and writes the subtasks, plans, memory, and grounded Q&A itself - roughly $1.20 for 100 episodes. From there, recipes turn the annotated rows into policy-agnostic training messages, so the same dataset can teach planning, memory, tool calls, or low-level control, no dataset redesign required. Read how it all works on the blog:

LeRobot

13,214 görüntüleme • 1 ay önce

How do you know your robot actually succeeded? In v0.6.0, you ask a VLM 🤖 LeRobot ships a new unified reward models API with two zero-shot additions. Robometer is a general-purpose reward model trained on 1M+ trajectories across 21 robot embodiments: point it at any LeRobot dataset and it scores task progress and success straight from video and a language instruction, no task-specific training needed. TOPReward goes even further: no reward weights at all. It wraps an off-the-shelf VLM and reads the log-probability of the token "True" given the video and the instruction. Both come with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning. More on how they work:

How do you know your robot actually succeeded? In v0.6.0, you ask a VLM 🤖 LeRobot ships a new unified reward models API with two zero-shot additions. Robometer is a general-purpose reward model trained on 1M+ trajectories across 21 robot embodiments: point it at any LeRobot dataset and it scores task progress and success straight from video and a language instruction, no task-specific training needed. TOPReward goes even further: no reward weights at all. It wraps an off-the-shelf VLM and reads the log-probability of the token "True" given the video and the instruction. Both come with labeling scripts that write per-frame progress curves into your dataset, ready for reward-aware behavior cloning. More on how they work:

LeRobot

22,358 görüntüleme • 1 ay önce

Introducing FoundationMotion. A large-scale, video-derived motion annotation dataset & auto-labeling pipeline + advanced models for motion understanding. Fully open-source: code, datasets, and models, free to use and build on. Understanding motion is core to physical reasoning, yet today’s leading models still struggle with simple spatial actions like “turn right” or “move up” or “flip the toast” - mainly due to the lack of large, fine-grained motion datasets. We present FoundationMotion, a fully automated pipeline that: • detects & tracks objects in videos • extracts trajectories • uses LLMs + frames to generate rich motion captions & QA pairs → creating large-scale, high-quality motion datasets at scale. After fine-tuning the open-source models Qwen and NVILA on our annotations, these models now outperform the closed-source Gemini-3-Flash and GPT-5.1 on spatial understanding tasks across autonomous driving, robotics, and everyday scenarios. 📜Paper: 🌐Webpage: 💻 Code: 🕸️Model: 📊 Dataset: 👉 Interactive Demo: Let’s move research forward together. FoundationMotion is also referred to as Wolf V2 🐺, the second chapter in the Wolf series:

Introducing FoundationMotion. A large-scale, video-derived motion annotation dataset & auto-labeling pipeline + advanced models for motion understanding. Fully open-source: code, datasets, and models, free to use and build on. Understanding motion is core to physical reasoning, yet today’s leading models still struggle with simple spatial actions like “turn right” or “move up” or “flip the toast” - mainly due to the lack of large, fine-grained motion datasets. We present FoundationMotion, a fully automated pipeline that: • detects & tracks objects in videos • extracts trajectories • uses LLMs + frames to generate rich motion captions & QA pairs → creating large-scale, high-quality motion datasets at scale. After fine-tuning the open-source models Qwen and NVILA on our annotations, these models now outperform the closed-source Gemini-3-Flash and GPT-5.1 on spatial understanding tasks across autonomous driving, robotics, and everyday scenarios. 📜Paper: 🌐Webpage: 💻 Code: 🕸️Model: 📊 Dataset: 👉 Interactive Demo: Let’s move research forward together. FoundationMotion is also referred to as Wolf V2 🐺, the second chapter in the Wolf series:

Boyi Li

67,569 görüntüleme • 8 ay önce

Today, we’re introducing the Sarvam MCP Server. It gives coding agents a standard way to discover and build with Sarvam. All you need to is a add a single configuration, and Sarvam becomes available directly in your coding agent. It already knows the right models and tools, so you can build with them and ship faster. Create a key, copy the config, start building -

Today, we’re introducing the Sarvam MCP Server. It gives coding agents a standard way to discover and build with Sarvam. All you need to is a add a single configuration, and Sarvam becomes available directly in your coding agent. It already knows the right models and tools, so you can build with them and ship faster. Create a key, copy the config, start building -

Sarvam

76,991 görüntüleme • 2 ay önce

Introducing AI fashion Photoshoots on Flair AI you can now train fashion models on your clothing with incredible accuracy - texture, labels, logos, and more preserved with midjourney level quality 🔥 rt and comment to try this feature in beta. serving access codes today:)

Introducing AI fashion Photoshoots on Flair AI you can now train fashion models on your clothing with incredible accuracy - texture, labels, logos, and more preserved with midjourney level quality 🔥 rt and comment to try this feature in beta. serving access codes today:)

mickey friedman

298,123 görüntüleme • 2 yıl önce

finally ported my fav Claude Code feature into Pi: `/subtask` a subtask is a fork of your current conversation, it inherits everything you've discussed, works in the background, and sends only its final answer back into the conversation that spawned it. Less context pollution/bloating and you can continue the main conversation as subtasks are running. here's what's packed into the Pi extension: - a live panel under the prompt: ↑↓ to select, enter to watch a fork work in real time, x to stop or dismiss it - steer a running fork mid-task, or resume a finished one without losing its progress - the model can spawn subtasks on its own and keep working while it waits for the result - forks reuse the parent's prompt cache, so they're cheaper than briefing a subagent - every fork's transcript is a real pi session: reopen it any time with `pi --session `

finally ported my fav Claude Code feature into Pi: `/subtask` a subtask is a fork of your current conversation, it inherits everything you've discussed, works in the background, and sends only its final answer back into the conversation that spawned it. Less context pollution/bloating and you can continue the main conversation as subtasks are running. here's what's packed into the Pi extension: - a live panel under the prompt: ↑↓ to select, enter to watch a fork work in real time, x to stop or dismiss it - steer a running fork mid-task, or resume a finished one without losing its progress - the model can spawn subtasks on its own and keep working while it waits for the result - forks reuse the parent's prompt cache, so they're cheaper than briefing a subagent - every fork's transcript is a real pi session: reopen it any time with `pi --session `

Victor M

18,394 görüntüleme • 23 gün önce

Today, we’re introducing Pomelli’s latest feature update, ‘Photoshoot’ With Photoshoot, you can start from a single image of your product and easily create high quality, customized product shots to elevate your marketing. Available free of charge in the US, Canada, Australia & New Zealand! Get started with Pomelli today at

Today, we’re introducing Pomelli’s latest feature update, ‘Photoshoot’ With Photoshoot, you can start from a single image of your product and easily create high quality, customized product shots to elevate your marketing. Available free of charge in the US, Canada, Australia & New Zealand! Get started with Pomelli today at

Google Labs

24,136,649 görüntüleme • 6 ay önce

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

We’re launching Optima. Now anyone can create a custom benchmark for their use case, leveraging Artificial Analysis’ leading research and platform Building and running benchmarks is difficult. We have distilled Artificial Analysis’ research and experience developing benchmarks into Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency. Optima allows you to find the best model for your task, or an equally performant alternative to your current setup at 10x lower cost or time per task. We’ve integrated Artificial Analysis' research and experience in benchmarks across the Optima workflow: ➤ Build benchmarks based on your own data and use cases: There are three ways to build a benchmark with Optima. Upload an existing evaluation dataset from your own files or Hugging Face, or import agent traces from platforms including Arize AI, Braintrust and langfuse.com. Install the Optima skill to build a benchmark using context from your coding environment and previous sessions. Or simply describe your use case and provide example inputs and outputs, and Optima will build the benchmark for you ➤ Run across the latest models: Run the same benchmark across leading models in a single click, and keep your leaderboard up to date as soon as new models are released ➤ Bring Artificial Analysis grading to your own benchmark: Evaluate responses against objective rubric criteria or using the same pairwise judging approach used for Artificial Analysis benchmarks including GDPval-AA and AA-Briefcase. For pairwise judging, select your preferred responses from a sample and Optima uses those preferences to rank models across your test set ➤ Compare performance, cost and time efficiency: Optima measures more than model performance. Cost per Task and Time per Task are tracked alongside benchmark scores, with category-level results and support for custom metrics, allowing you to compare the tradeoffs between models for your specific use case Ahead of launch, here are examples questions our beta testers answered with Optima: ➤ Which model can save me 10x the cost without a meaningful decrease in quality for my finance & accounting agent? ➤ Which model best matches the writing style of lawyers for my legal agent? ➤ Which model can best identify different elements in my custom image dataset? Optima is available today. Build your own benchmark at

Artificial Analysis

132,403 görüntüleme • 28 gün önce

Hugging Face is NUTS! A high-performance robotic arm for $115 😱 😱😱 [ DIY at home ⬇ ] Meet the game-changer: SO-100 At 1/3 the cost and 2x the capabilities of their previous arms! Fully integrated with LeRobot: ✅ Go-to for tutorials ✅ Datasets, and ✅ Pre-trained models — all in Python Github: Discord: Now is the best time to get into AI for robotics! Order your parts now! Crafted by Hugging Face & Rob Knight by Rob Knight! And kudos to Remi Cadene and his team for bringing robotics to the people! • • • • • • • --------------------------------------------------- 🚀 Scaling Deep Tech Startups:

Hugging Face is NUTS! A high-performance robotic arm for $115 😱 😱😱 [ DIY at home ⬇ ] Meet the game-changer: SO-100 At 1/3 the cost and 2x the capabilities of their previous arms! Fully integrated with LeRobot: ✅ Go-to for tutorials ✅ Datasets, and ✅ Pre-trained models — all in Python Github: Discord: Now is the best time to get into AI for robotics! Order your parts now! Crafted by Hugging Face & Rob Knight by Rob Knight! And kudos to Remi Cadene and his team for bringing robotics to the people! • • • • • • • --------------------------------------------------- 🚀 Scaling Deep Tech Startups:

Ilir Aliu

27,503 görüntüleme • 1 yıl önce

Claude Code now supports agent teams (in research preview) Instead of a single agent working through a task sequentially, a lead agent can delegate to multiple teammates that work in parallel to research, debug, and build while coordinating with each other. Try it out today by enabling agent teams in your settings.json!

Claude Code now supports agent teams (in research preview) Instead of a single agent working through a task sequentially, a lead agent can delegate to multiple teammates that work in parallel to research, debug, and build while coordinating with each other. Try it out today by enabling agent teams in your settings.json!

Lydia Hallie ✨

1,444,869 görüntüleme • 7 ay önce

🚨 NEW LABS EXPERIMENT 🚨 Introducing Pomelli, an experimental AI marketing tool designed to help you easily generate scalable, on-brand content to connect with your audience, faster. Just enter your website, and Pomelli will understand your unique business identity to build effective campaigns tailored to your brand. Now available in US, CAN, AUS, & NZ! Try It Now ⬇️

Google Labs

1,840,909 görüntüleme • 10 ay önce

Today we are introducing Tara. Biological datasets are a source of insights and a means to train biological AI models. As the ability to reason at scale emerges, they take on a new role: the ground truth for testing what reasoning models produce, and the environment in which those models operate, get feedback, and improve. Tara, our autonomous research agent, is embedded in our ever-expanding datasets, lab-generated and synthetic, and built to test and evolve the hypotheses frontier models generate, matching the pace at which they produce new ideas. By keeping those models grounded in a vast space of high-precision biological data, we believe we can compound biological reasoning and close the impedance mismatch between hypothesis generation and validation.

Today we are introducing Tara. Biological datasets are a source of insights and a means to train biological AI models. As the ability to reason at scale emerges, they take on a new role: the ground truth for testing what reasoning models produce, and the environment in which those models operate, get feedback, and improve. Tara, our autonomous research agent, is embedded in our ever-expanding datasets, lab-generated and synthetic, and built to test and evolve the hypotheses frontier models generate, matching the pace at which they produce new ideas. By keeping those models grounded in a vast space of high-precision biological data, we believe we can compound biological reasoning and close the impedance mismatch between hypothesis generation and validation.

Nima Alidoust

30,458 görüntüleme • 2 ay önce

1- You can have Grok speak in your language; he speaks perfectly with the right accent. (here in french) 2- You can build a long scene in 1080p by taking the last frame of an animation to start another animation. It's easy with Grok (Right-click on the animation and copy the image you want.)

1- You can have Grok speak in your language; he speaks perfectly with the right accent. (here in french) 2- You can build a long scene in 1080p by taking the last frame of an animation to start another animation. It's easy with Grok (Right-click on the animation and copy the image you want.)

Déborah

348,285 görüntüleme • 22 gün önce

Introducing WorldForge: testable world-model workflows for physical AI systems. You can think of it, loosely, as “LangChain for world models”. The problem is that “world model” has become an overloaded label. Depending on context, it can mean a video generator, a cost model, a robot policy, a JEPA-style latent predictor, etc. They share almost nothing, different inputs, runtimes, failure modes. I built WorldForge to stop pretending they're interchangeable. Front-door demo: a real Hugging Face LeRobot (LeRobot) diffusion_pusht policy combined with LeWorldModel by Lucas Maes checkpoint for scoring. Both run locally on my MacBook in the demo video. LeWM is extremely efficient (~15M params), can plan up to 48× faster, and runs on commodity hardware. WorldForge wires the loop: policy → candidates → score → select Replay happens in a local TUI today, but the same loop could drive a real robot in the world. Would love feedback from people working on world models, physical AI, robotics, ML infra, and adjacent tooling. Fully open source. Contributions very welcome. Plan in the dream, replay in real world.

Introducing WorldForge: testable world-model workflows for physical AI systems. You can think of it, loosely, as “LangChain for world models”. The problem is that “world model” has become an overloaded label. Depending on context, it can mean a video generator, a cost model, a robot policy, a JEPA-style latent predictor, etc. They share almost nothing, different inputs, runtimes, failure modes. I built WorldForge to stop pretending they're interchangeable. Front-door demo: a real Hugging Face LeRobot (LeRobot) diffusion_pusht policy combined with LeWorldModel by Lucas Maes checkpoint for scoring. Both run locally on my MacBook in the demo video. LeWM is extremely efficient (~15M params), can plan up to 48× faster, and runs on commodity hardware. WorldForge wires the loop: policy → candidates → score → select Replay happens in a local TUI today, but the same loop could drive a real robot in the world. Would love feedback from people working on world models, physical AI, robotics, ML infra, and adjacent tooling. Fully open source. Contributions very welcome. Plan in the dream, replay in real world.

abdel

17,001 görüntüleme • 4 ay önce

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmented—making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same code—bringing together the community’s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse dataset—a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulators—e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our team’s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced🥳!!! Project Page: Documentation: Github Repo: Paper:

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmented—making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same code—bringing together the community’s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse dataset—a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulators—e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our team’s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced🥳!!! Project Page: Documentation: Github Repo: Paper:

Haoran Geng

84,318 görüntüleme • 1 yıl önce

🚀 Introducing Queue: A Smarter Way to Build with Replit Agent Today, we’re excited to launch Queue, a powerful new capability designed to enhance your experience with Replit Agent. With Queue, you can submit multiple tasks while your Agent is still working—no more waiting around or breaking your flow. As each task completes, the Agent automatically moves to the next. You can: ✅ Add multiple tasks for the Agent to process in order. ✅ Edit, reorder, or reprioritize messages on the fly. ✅ Attach files and adjust settings—including toggling extended thinking or selecting a different model—per task.

🚀 Introducing Queue: A Smarter Way to Build with Replit Agent Today, we’re excited to launch Queue, a powerful new capability designed to enhance your experience with Replit Agent. With Queue, you can submit multiple tasks while your Agent is still working—no more waiting around or breaking your flow. As each task completes, the Agent automatically moves to the next. You can: ✅ Add multiple tasks for the Agent to process in order. ✅ Edit, reorder, or reprioritize messages on the fly. ✅ Attach files and adjust settings—including toggling extended thinking or selecting a different model—per task.

Replit ⠕

75,576 görüntüleme • 1 yıl önce

What if building a mobile dual-arm robot for your home took less than 7 hours and cost under $700? [Github ⬇️ Bookmark for later 📍] XLeRobot… a low-cost, open-source robot for everyday tasks. Built with off-the-shelf parts and powered by the LeRobot ecosystem, it’s made for hackers, students, and researchers. ✅ Built from Lekiwi, SO100, an IKEA cart, and an Anker battery ✅ Fully LeRobot compatible with tons of pre-trained models ✅ Easy to assemble in under 7 hours with global parts ✅ Supported by a 5,000+ member community XLeRobot is more than a fun weekend build, it’s a serious step toward affordable home robotics. Thank you, Vector Wang for sharing!!! 🫶 Github:

What if building a mobile dual-arm robot for your home took less than 7 hours and cost under $700? [Github ⬇️ Bookmark for later 📍] XLeRobot… a low-cost, open-source robot for everyday tasks. Built with off-the-shelf parts and powered by the LeRobot ecosystem, it’s made for hackers, students, and researchers. ✅ Built from Lekiwi, SO100, an IKEA cart, and an Anker battery ✅ Fully LeRobot compatible with tons of pre-trained models ✅ Easy to assemble in under 7 hours with global parts ✅ Supported by a 5,000+ member community XLeRobot is more than a fun weekend build, it’s a serious step toward affordable home robotics. Thank you, Vector Wang for sharing!!! 🫶 Github:

Ilir Aliu

32,099 görüntüleme • 1 yıl önce

This is how you build the next generation of AI applications. You need an idea, a good model architecture, and a great harness. That's what MUZIM did. It's an app that runs local multimodal and vision models offline to parse and index your images, videos, and documents frame by frame. • Super fast on-device processing • On-device video & image understanding • You can also bring your API keys for heavy reasoning • Your data stays where it is / nothing goes to the cloud You can do a couple of things with this: 1. Vibe searching for whatever you remember in photos or raw video clips. The app will jump straight to the exact frame and timestamp of the video. This is search as it should be. 2. Use local-running agents to perform tasks on your files. I show a couple of examples in my video. You can check it out here: Thanks to the MUZIM by OpenSoul team for showing me the tool and partnering with me on this post.

This is how you build the next generation of AI applications. You need an idea, a good model architecture, and a great harness. That's what MUZIM did. It's an app that runs local multimodal and vision models offline to parse and index your images, videos, and documents frame by frame. • Super fast on-device processing • On-device video & image understanding • You can also bring your API keys for heavy reasoning • Your data stays where it is / nothing goes to the cloud You can do a couple of things with this: 1. Vibe searching for whatever you remember in photos or raw video clips. The app will jump straight to the exact frame and timestamp of the video. This is search as it should be. 2. Use local-running agents to perform tasks on your files. I show a couple of examples in my video. You can check it out here: Thanks to the MUZIM by OpenSoul team for showing me the tool and partnering with me on this post.

Santiago

65,953 görüntüleme • 24 gün önce

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

AK

122,567 görüntüleme • 3 yıl önce