Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Pi0 from Physical Intelligence is one of the best generalist Vision-Language-Action (VLA) models for robotics (and predecessor of Pi0.5). 🧵 Here's how it works + how you can fine-tune it on your own robot for a simple task:

Ilia

1,170 subscribers

32,956 görüntüleme • 10 ay önce •via X (Twitter)

Bilim & Teknoloji Eğitim Sanat

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️

Ted Xiao

152,445 görüntüleme • 1 yıl önce

VLAExplain — Interpreting Vision-Language-Action (VLA) Models VLAExplain is an interpretability toolkit designed to help users visually understand the inner workings of Vision-Language-Action (VLA) models. Currently, attention analysis is supported for both the pi05 and unifolm-vla models. For details, please check pi05 and UnifoLM-VLA readme files respectively. Demo of pi05 in action:

VLAExplain — Interpreting Vision-Language-Action (VLA) Models VLAExplain is an interpretability toolkit designed to help users visually understand the inner workings of Vision-Language-Action (VLA) models. Currently, attention analysis is supported for both the pi05 and unifolm-vla models. For details, please check pi05 and UnifoLM-VLA readme files respectively. Demo of pi05 in action:

Ryohei Sasaki@engineer

12,774 görüntüleme • 2 ay önce

3D-VLA A 3D Vision-Language-Action Generative World Model Recent vision-language-action (VLA) models rely on 2D inputs, lacking integration with the broader realm of the 3D physical world. Furthermore, they perform action prediction by learning a direct mapping from

3D-VLA A 3D Vision-Language-Action Generative World Model Recent vision-language-action (VLA) models rely on 2D inputs, lacking integration with the broader realm of the 3D physical world. Furthermore, they perform action prediction by learning a direct mapping from

AK

85,570 görüntüleme • 2 yıl önce

⭐ The first foundational model available on LeRobot ⭐ Pi0 is the most advanced Vision Language Action model. It takes natural language commands as input and directly output autonomous behavior. It was trained by Physical Intelligence and ported to pytorch by Pablo Montalvo 👇🧵

⭐ The first foundational model available on LeRobot ⭐ Pi0 is the most advanced Vision Language Action model. It takes natural language commands as input and directly output autonomous behavior. It was trained by Physical Intelligence and ported to pytorch by Pablo Montalvo 👇🧵

Remi Cadene

130,941 görüntüleme • 1 yıl önce

Qwen-VLA feels like one of the first real robotics foundation models. A single system trained across robot manipulation, navigation, egocentric human video, simulation, and vision-language reasoning instead of isolated robot policies.

Qwen-VLA feels like one of the first real robotics foundation models. A single system trained across robot manipulation, navigation, egocentric human video, simulation, and vision-language reasoning instead of isolated robot policies.

Robots Digest 🤖

14,663 görüntüleme • 1 ay önce

UK-based startup 'Humanoid' announced KinetIQ, an AI framework with a Vision-Language-Action (VLA) model at its core. It uses a four-layer architecture: fleet orchestration, task decomposition, VLA, and RL for whole-body control. It works on both bipedal and wheeled robots.

UK-based startup 'Humanoid' announced KinetIQ, an AI framework with a Vision-Language-Action (VLA) model at its core. It uses a four-layer architecture: fleet orchestration, task decomposition, VLA, and RL for whole-body control. It works on both bipedal and wheeled robots.

The Humanoid Hub

21,954 görüntüleme • 5 ay önce

Chain-of-thought reasoning is a powerful tool to enable language models to work through complex problems. Can we use this with robots? With embodied chain-of-thought, vision-language-action (VLA) models can think through perception and planning! A 🧵👇

Chain-of-thought reasoning is a powerful tool to enable language models to work through complex problems. Can we use this with robots? With embodied chain-of-thought, vision-language-action (VLA) models can think through perception and planning! A 🧵👇

Sergey Levine

30,388 görüntüleme • 2 yıl önce

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇

Fangchen Liu

68,353 görüntüleme • 1 yıl önce

Microsoft just dropped VITRA-VLA, a new Vision-Language-Action model for robotics on Hugging Face. It learns dexterous manipulation from over 1 million real-life human hand activity videos.

Microsoft just dropped VITRA-VLA, a new Vision-Language-Action model for robotics on Hugging Face. It learns dexterous manipulation from over 1 million real-life human hand activity videos.

DailyPapers

19,177 görüntüleme • 7 ay önce

This is Microsoft's new agentic Copilot feature for Windows 11 in action. It will take your task and complete it in a separate desktop environment, and you can watch it while it works or minimize and get on with your own task.

This is Microsoft's new agentic Copilot feature for Windows 11 in action. It will take your task and complete it in a separate desktop environment, and you can watch it while it works or minimize and get on with your own task.

Zac Bowden

110,244 görüntüleme • 9 ay önce

Exciting progress on Vision-Language-Action models from a collaboration between San Francisco-based Physical Intelligence (π) and China’s AGIBOT: (π)’s single model can autonomously perform diverse tasks on the AGIBOT G1 robot, using both humanoid hands and two-finger grippers.

Exciting progress on Vision-Language-Action models from a collaboration between San Francisco-based Physical Intelligence (π) and China’s AGIBOT: (π)’s single model can autonomously perform diverse tasks on the AGIBOT G1 robot, using both humanoid hands and two-finger grippers.

The Humanoid Hub

32,575 görüntüleme • 1 yıl önce

Robot AI brains, aka Vision-Language-Action models, cannot adapt to new tasks as easily as LLMs like Gemini, ChatGPT, or Grok. LLMs can adapt quickly with their in-context learning (ICL) capabilities. But can we inject ICL abilities into a pre-trained VLA like pi0? Yes! Introducing RICL (Retraining for In-Context Learning), our Conference on Robot Learning (CoRL) 2025 paper. Our RICL-pi0 model can adapt to unseen objects, novel motions, and new scenes with just ICL and RAG (retrieval-augmented generation). RICL-pi0 also boosts performance on the long-tail of tasks. A quick 1 minute video summary:

Robot AI brains, aka Vision-Language-Action models, cannot adapt to new tasks as easily as LLMs like Gemini, ChatGPT, or Grok. LLMs can adapt quickly with their in-context learning (ICL) capabilities. But can we inject ICL abilities into a pre-trained VLA like pi0? Yes! Introducing RICL (Retraining for In-Context Learning), our Conference on Robot Learning (CoRL) 2025 paper. Our RICL-pi0 model can adapt to unseen objects, novel motions, and new scenes with just ICL and RAG (retrieval-augmented generation). RICL-pi0 also boosts performance on the long-tail of tasks. A quick 1 minute video summary:

Kaustubh Sridhar

52,158 görüntüleme • 11 ay önce

You can't control the uncontrollables. But that doesn't mean you can't control the future of robotics. 𝐓𝐞𝐥𝐞𝐨𝐩 𝐢𝐬 𝐥𝐢𝐯𝐞 𝐨𝐧 𝐏𝐫𝐢𝐬𝐦𝐚𝐗. You can now train vision-language action models and earn PrismaX points for your impact.

You can't control the uncontrollables. But that doesn't mean you can't control the future of robotics. 𝐓𝐞𝐥𝐞𝐨𝐩 𝐢𝐬 𝐥𝐢𝐯𝐞 𝐨𝐧 𝐏𝐫𝐢𝐬𝐦𝐚𝐗. You can now train vision-language action models and earn PrismaX points for your impact.

PrismaX

23,438 görüntüleme • 11 ay önce

This is crazy... Have you ever thought about the combination of AI and podcast? Now you can build your own AI podcast agent. Here's how it works:

This is crazy... Have you ever thought about the combination of AI and podcast? Now you can build your own AI podcast agent. Here's how it works:

Angry Tom

25,328 görüntüleme • 1 yıl önce

The next evolution: VLA+ models Just yesterday Microsoft Research released Rho-alpha (ρα) – their first robotics model, built on the Phi family. While most Vision-Language-Action (VLA) models stop at vision and language, Rho-alpha adds: ▪️ Tactile sensing to feel objects during manipulation ▪️ Online learning that lets it improve from human corrections (via teleoperation, 3D mouse or other tools) in real-time even after deployment. Both these sides make adaptability central rather than incidental. Microsoft calls it a VLA+ model, positioning it as an extension beyond what current VLA systems support. ➡️ Today Rho-alpha can control dual-arm robot setups to perform tasks such as: • Manipulating the BusyBox following natural-language instructions • Plug insertion • Toolbox packing and object arrangement with bimanual coordination But to understand why this "plus" matters, we need to understand what came before. Here, we'll take you through the entire landscape of VLA models – Gemini Robotics, π0, SmolVLA, Helix, ACoT-VLA and others:

The next evolution: VLA+ models Just yesterday Microsoft Research released Rho-alpha (ρα) – their first robotics model, built on the Phi family. While most Vision-Language-Action (VLA) models stop at vision and language, Rho-alpha adds: ▪️ Tactile sensing to feel objects during manipulation ▪️ Online learning that lets it improve from human corrections (via teleoperation, 3D mouse or other tools) in real-time even after deployment. Both these sides make adaptability central rather than incidental. Microsoft calls it a VLA+ model, positioning it as an extension beyond what current VLA systems support. ➡️ Today Rho-alpha can control dual-arm robot setups to perform tasks such as: • Manipulating the BusyBox following natural-language instructions • Plug insertion • Toolbox packing and object arrangement with bimanual coordination But to understand why this "plus" matters, we need to understand what came before. Here, we'll take you through the entire landscape of VLA models – Gemini Robotics, π0, SmolVLA, Helix, ACoT-VLA and others:

Turing Post

62,362 görüntüleme • 5 ay önce

GPT 5.6 SOL IS HERE! How to run your personal + business life with GPT 5.6 Sol + Codex (full 49 min masterclass) We tested it for 30 days and the video it's the CLEAREST look at the FUTURE of work: Here's what's possible once you set it up: 1. Your inbox becomes cards every morning, each with a summary and a reply drafted in your own voice. Y 2. Your Slack, meeting notes, and company updates can turn into one daily feed with a clear next action. It learns what you care about over time and rewrites its own prompts to get sharper. 3. You can give your agent its own email address, so your other tools and even your team's Slack bot email it directly and it just handles things. 4. You can have it watch you do a task once and turn it into a skill it repeats forever. 5. You can set a long goal and walk away. You can have it run for 20 hours straight, and fine-tune your own models, something that was out of reach for non-engineers 12 months ago. How to start: Open Codex, give it access to your computer, and ask it to suggest things it could do for you based on how you already work. Full episode on The Startup Ideas Podcast (SIP) 🧃 (thanks Dan Shipper 📧 for sharing your entire workflow and review of GPT 5.6) Start with one boring task, get it working, and build from there. You'll learn exactly how to make something similar. GPT 5.6 Sol is impressive. Sol (according to openAI benchmarks) is the best coding model out right now. It set a new state of the art on Terminal-Bench 2.1 at 88.8%, and its "ultra mode" hits 91.9%, beating Claude Opus 4.8, Fable 5, and even Mythos 5 this masterclass is 100% free, like always. For more The Startup Ideas Podcast (SIP) 🧃 Watch

GPT 5.6 SOL IS HERE! How to run your personal + business life with GPT 5.6 Sol + Codex (full 49 min masterclass) We tested it for 30 days and the video it's the CLEAREST look at the FUTURE of work: Here's what's possible once you set it up: 1. Your inbox becomes cards every morning, each with a summary and a reply drafted in your own voice. Y 2. Your Slack, meeting notes, and company updates can turn into one daily feed with a clear next action. It learns what you care about over time and rewrites its own prompts to get sharper. 3. You can give your agent its own email address, so your other tools and even your team's Slack bot email it directly and it just handles things. 4. You can have it watch you do a task once and turn it into a skill it repeats forever. 5. You can set a long goal and walk away. You can have it run for 20 hours straight, and fine-tune your own models, something that was out of reach for non-engineers 12 months ago. How to start: Open Codex, give it access to your computer, and ask it to suggest things it could do for you based on how you already work. Full episode on The Startup Ideas Podcast (SIP) 🧃 (thanks Dan Shipper 📧 for sharing your entire workflow and review of GPT 5.6) Start with one boring task, get it working, and build from there. You'll learn exactly how to make something similar. GPT 5.6 Sol is impressive. Sol (according to openAI benchmarks) is the best coding model out right now. It set a new state of the art on Terminal-Bench 2.1 at 88.8%, and its "ultra mode" hits 91.9%, beating Claude Opus 4.8, Fable 5, and even Mythos 5 this masterclass is 100% free, like always. For more The Startup Ideas Podcast (SIP) 🧃 Watch

GREG ISENBERG

152,821 görüntüleme • 10 gün önce

LLM post-training used to mean fine-tuning to a downstream task Robotics has been stuck in this setting, needing task-specific fine-tuning for best performance π07 changes this: It works out of the box & outperforms fine-tuned specialists Details:

LLM post-training used to mean fine-tuning to a downstream task Robotics has been stuck in this setting, needing task-specific fine-tuning for best performance π07 changes this: It works out of the box & outperforms fine-tuned specialists Details:

Chelsea Finn

63,780 görüntüleme • 3 ay önce

Pretraining is essential for good performance on a wide variety of robotics tasks, and so most vision-language-action models build off of a vision language model (VLM) trained on a wide variety of image-language data. But how does the choice of VLM translate to downstream robotics performance? Jianke Zhang and @GYanjiang join us to talk about this key part of the robot policy, looking at a wide variety of different VLMs and how they perform. Interestingly, they see that performance on auxiliary tasks like quesiton answering did not lead to downstream improvements in control. To learn more, watch episode 65 of RoboPapers now, with Chris Paxton and Jiafei Duan!

Pretraining is essential for good performance on a wide variety of robotics tasks, and so most vision-language-action models build off of a vision language model (VLM) trained on a wide variety of image-language data. But how does the choice of VLM translate to downstream robotics performance? Jianke Zhang and @GYanjiang join us to talk about this key part of the robot policy, looking at a wide variety of different VLMs and how they perform. Interestingly, they see that performance on auxiliary tasks like quesiton answering did not lead to downstream improvements in control. To learn more, watch episode 65 of RoboPapers now, with Chris Paxton and Jiafei Duan!

RoboPapers

23,905 görüntüleme • 4 ay önce

Why LLMs are a dead end for human-level intelligence, and especially for Physical AI / Robotics. The next leap isn’t bigger language models. It’s World Models. I just dropped a full 1-hour presentation from Shanghai: “World Models: the ChatGPT moment for robotics?” → Why LLMs hit a wall → Why action-conditioned world models planning in latent space are the real path → Live World Forge demo with LeWorldModel + Hugging Face LeRobot Watch here. The future of intelligence is embodied, not just chatty.

Why LLMs are a dead end for human-level intelligence, and especially for Physical AI / Robotics. The next leap isn’t bigger language models. It’s World Models. I just dropped a full 1-hour presentation from Shanghai: “World Models: the ChatGPT moment for robotics?” → Why LLMs hit a wall → Why action-conditioned world models planning in latent space are the real path → Live World Forge demo with LeWorldModel + Hugging Face LeRobot Watch here. The future of intelligence is embodied, not just chatty.

abdel

37,349 görüntüleme • 1 ay önce