Загрузка видео...

Не удалось загрузить видео

На главную

Our newest model, π0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!

physical_int's profile picture

Physical Intelligence

45,516 subscribers

456,897 просмотров • 2 месяцев назад •via X (Twitter)

ОбразованиеНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

We finished evaluating π0.7, our new model at Physical Intelligence. What I'm most excited about with π0.7 is that it's starting to show some surprising emergent compositional generalization, being able to both perform complex tasks and learn new tasks just from instructions.
1:07

We finished evaluating π0.7, our new model at Physical Intelligence. What I'm most excited about with π0.7 is that it's starting to show some surprising emergent compositional generalization, being able to both perform complex tasks and learn new tasks just from instructions.

Sergey Levine

60,336 просмотров • 2 месяцев назад

Sim2Real RL for Vision-Based Dexterous Manipulation on Humanoids TLDR - we train a humanoid robot with two multifingered hands to perform a range of dexterous manipulation tasks robust generalization and high performance without human demonstration :D
0:20

Sim2Real RL for Vision-Based Dexterous Manipulation on Humanoids TLDR - we train a humanoid robot with two multifingered hands to perform a range of dexterous manipulation tasks robust generalization and high performance without human demonstration :D

Toru

49,561 просмотров • 1 год назад

Physical Intelligence's π₀, a general-purpose robot foundation model that combines Internet-scale vision-language pretraining with robot interaction data to execute tasks. They aim "to develop foundation models that can control any robot to perform any task" Autonomous demos:
3:36

Physical Intelligence's π₀, a general-purpose robot foundation model that combines Internet-scale vision-language pretraining with robot interaction data to execute tasks. They aim "to develop foundation models that can control any robot to perform any task" Autonomous demos:

The Humanoid Hub

196,055 просмотров • 1 год назад

How can we build a general-purpose “foundation model” for robot motion? Zhengyi “Zen” Luo joins us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a model with a generally-useful embedding space that can be controlled by a VLA, or from human video, to perform a wide variety of humanoid whole-body-control tasks, including with zero-shot transfer to previously unseen motions. Watch episode 72 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, now!
59:44

How can we build a general-purpose “foundation model” for robot motion? Zhengyi “Zen” Luo joins us to talk about SONIC, which uses motion tracking as a foundational task for humanoid robot control, and scales humanoid control training to 9k GPU hours and 100 million frames worth of data. The result: a model with a generally-useful embedding space that can be controlled by a VLA, or from human video, to perform a wide variety of humanoid whole-body-control tasks, including with zero-shot transfer to previously unseen motions. Watch episode 72 of RoboPapers, with Michael Cho - Rbt/Acc and Jiafei Duan, now!

RoboPapers

24,802 просмотров • 2 месяцев назад

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!
54:52

Every home is different. That means that to build a useful home robot, we must be able to perform zero-shot generalization on a wide range of tasks. Humanoid company 1X has a solution: world models. 1X Director of Evaluations Daniel Ho joins us on RoboPapers to talk about: - why world models are the future for scaling robot learning - how to use world models for robot control - what world models unlock for evaluating robot model performance - how we can hill-climb from here to general purpose robots Watch Episode #61 of RoboPapers, with Michael Cho - Rbt/Acc and Chris Paxton, now!

RoboPapers

27,567 просмотров • 4 месяцев назад

For the first time in human history, we are teaching a Foundation Model to master the diverse tasks of medicinal chemists, biologists, and computational scientists all in one place. In our latest collaboration with Liquid AI, we are moving away from fragmented, specialized tools toward a single, super-intelligent model. What surprised me most? This model isn't just performing at reasonable levels—it has started outperforming specialist models across physics-based tasks, imaging, and longitudinal data. Why this changes everything: -Synergy over Specialization: Fine-tuning on specific tasks has unlocked unexpected capabilities in synergetic areas, opening a new frontier in multimodal AI research. -Zero-Shot Potential: We are building a model that can perform out-of-scope tasks, moving us closer to an "AI deity" for drug discovery. -Quality First: The goal isn't just to bypass regulations to save time; it’s about using these synergies to develop better, more effective drugs. We are no longer just looking at linear regression or simple text; we are looking at the future of how humanity fights disease. #LiquidAI #InsilicoMedicine #GenerativeAI #DrugDiscovery #DeepTech #BiotechInnovation
1:53

For the first time in human history, we are teaching a Foundation Model to master the diverse tasks of medicinal chemists, biologists, and computational scientists all in one place. In our latest collaboration with Liquid AI, we are moving away from fragmented, specialized tools toward a single, super-intelligent model. What surprised me most? This model isn't just performing at reasonable levels—it has started outperforming specialist models across physics-based tasks, imaging, and longitudinal data. Why this changes everything: -Synergy over Specialization: Fine-tuning on specific tasks has unlocked unexpected capabilities in synergetic areas, opening a new frontier in multimodal AI research. -Zero-Shot Potential: We are building a model that can perform out-of-scope tasks, moving us closer to an "AI deity" for drug discovery. -Quality First: The goal isn't just to bypass regulations to save time; it’s about using these synergies to develop better, more effective drugs. We are no longer just looking at linear regression or simple text; we are looking at the future of how humanity fights disease. #LiquidAI #InsilicoMedicine #GenerativeAI #DrugDiscovery #DeepTech #BiotechInnovation

Alex Zhavoronkov, PhD (aka Aleksandrs Zavoronkovs)

10,544 просмотров • 3 месяцев назад

A few examples of π0.7 in action Turn the shirt right-side out and fold it:
1:22

A few examples of π0.7 in action Turn the shirt right-side out and fold it:

Karol Hausman

22,841 просмотров • 2 месяцев назад

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️
0:31

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️

Ted Xiao

152,423 просмотров • 1 год назад

We are excited to share new experiments with AgiBot @AgiBot_zhiyuan on multi-task, multi-embodiment VLAs! With one model that can perform many tasks with both two-finger grippers and multi-fingered hands, we take another step toward one model for all robots and tasks.
3:15

We are excited to share new experiments with AgiBot @AgiBot_zhiyuan on multi-task, multi-embodiment VLAs! With one model that can perform many tasks with both two-finger grippers and multi-fingered hands, we take another step toward one model for all robots and tasks.

Physical Intelligence

75,796 просмотров • 1 год назад

Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.
3:50

Our model can now learn from its own experience with RL! Our new π*0.6 model can more than double throughput over a base model trained without RL, and can perform real-world tasks: making espresso drinks, folding diverse laundry, and assembling boxes. More in the thread below.

Physical Intelligence

704,626 просмотров • 7 месяцев назад

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.
2:02

We discovered an emergent property of VLAs like π0/π0.5/π0.6: as we scale up pre-training, the model learns to align human videos and robot data! This gives us a simple way to leverage human videos. Once π0.5 knows how to control robots, it can naturally learn from human video.

Physical Intelligence

1,182,229 просмотров • 6 месяцев назад

Pretraining is essential for good performance on a wide variety of robotics tasks, and so most vision-language-action models build off of a vision language model (VLM) trained on a wide variety of image-language data. But how does the choice of VLM translate to downstream robotics performance? Jianke Zhang and @GYanjiang join us to talk about this key part of the robot policy, looking at a wide variety of different VLMs and how they perform. Interestingly, they see that performance on auxiliary tasks like quesiton answering did not lead to downstream improvements in control. To learn more, watch episode 65 of RoboPapers now, with Chris Paxton and Jiafei Duan!
1:03:46

Pretraining is essential for good performance on a wide variety of robotics tasks, and so most vision-language-action models build off of a vision language model (VLM) trained on a wide variety of image-language data. But how does the choice of VLM translate to downstream robotics performance? Jianke Zhang and @GYanjiang join us to talk about this key part of the robot policy, looking at a wide variety of different VLMs and how they perform. Interestingly, they see that performance on auxiliary tasks like quesiton answering did not lead to downstream improvements in control. To learn more, watch episode 65 of RoboPapers now, with Chris Paxton and Jiafei Duan!

RoboPapers

23,905 просмотров • 3 месяцев назад

Another home-use humanoid robot The cute and friendly Onero H1 comes from Switchbot. It looks like a wheeled humanoid robot that grew out of a robotic vacuum cleaner base. It can perform tasks such as making coffee, doing laundry, folding clothes, tidying up, and acting as a life assistant. Its embedded OmniSense VLA model allows it to adapt to complex household tasks. Notably, smart home brands such as LG, Samsung, Haier, Midea, and Xiaomi are all launching home-use humanoid robots, marking a new stage in smart home living.
1:34

Another home-use humanoid robot The cute and friendly Onero H1 comes from Switchbot. It looks like a wheeled humanoid robot that grew out of a robotic vacuum cleaner base. It can perform tasks such as making coffee, doing laundry, folding clothes, tidying up, and acting as a life assistant. Its embedded OmniSense VLA model allows it to adapt to complex household tasks. Notably, smart home brands such as LG, Samsung, Haier, Midea, and Xiaomi are all launching home-use humanoid robots, marking a new stage in smart home living.

CyberRobo

20,970 просмотров • 5 месяцев назад

We've partnered with OpenAI as one of their early research preview partners for the new computer use model. With this model, AI can control a browser better than ever before 👀 Of course, we had to launch a free hosted version + open source implementation. Check it out!
0:53

We've partnered with OpenAI as one of their early research preview partners for the new computer use model. With this model, AI can control a browser better than ever before 👀 Of course, we had to launch a free hosted version + open source implementation. Check it out!

Browserbase

40,232 просмотров • 1 год назад

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation
0:27

🤖What if a robot could perform a new task just from a natural language command, with zero demonstrations? Our new work, NovaFlow, makes it possible! We use pre-trained video generative model to create a video of the task, then translate it into a plan for real-world robot execution. 1/6 #Robotics #AI #ZeroShot #Manipulation

Hongyu Li

105,471 просмотров • 8 месяцев назад

Chinese company Robot Era has unveiled their next-gen robot L7, a 5′7″ tall humanoid. The company has also showcased the ERA-42 Vision-Language-Action model running on L7 robot to autonomously execute dexterous tasks.
2:23

Chinese company Robot Era has unveiled their next-gen robot L7, a 5′7″ tall humanoid. The company has also showcased the ERA-42 Vision-Language-Action model running on L7 robot to autonomously execute dexterous tasks.

The Humanoid Hub

38,547 просмотров • 11 месяцев назад

Excited to share our latest progress on DYNA-1 pre-training! 🤖 The base model now can perform diverse, dexterous tasks (laundry folding, package sorting, …) without any post-training, even in unseen environments. This powerful base also allows extremely efficient fine-tuning to ~100% success on challenging new tasks with as little as 1 hour of data! 🤯 Watch it master two of them: cup stacking & celery chopping on repeat, no failures. 👇
1:53

Excited to share our latest progress on DYNA-1 pre-training! 🤖 The base model now can perform diverse, dexterous tasks (laundry folding, package sorting, …) without any post-training, even in unseen environments. This powerful base also allows extremely efficient fine-tuning to ~100% success on challenging new tasks with as little as 1 hour of data! 🤯 Watch it master two of them: cup stacking & celery chopping on repeat, no failures. 👇

Dyna Robotics

66,619 просмотров • 7 месяцев назад

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.
4:10

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 просмотров • 3 лет назад

Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability. This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.
2:23

Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability. This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.

Zu Wang

18,402 просмотров • 1 месяц назад

Collecting dexterous humanoid robot data is difficult to scale. That's why Mengda Xu and Han Zhang built DexUMI: a tool for demonstrating how to control a dexterous robot hand, which allows you to quickly collect task data. Co-hosted by Michael Cho - Rbt/Acc and Chris Paxton
58:42

Collecting dexterous humanoid robot data is difficult to scale. That's why Mengda Xu and Han Zhang built DexUMI: a tool for demonstrating how to control a dexterous robot hand, which allows you to quickly collect task data. Co-hosted by Michael Cho - Rbt/Acc and Chris Paxton

RoboPapers

26,344 просмотров • 10 месяцев назад