Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Language following is a tough problem for VLAs: while these models can follow complex language, in practice getting datasets that enable language following is hard. We developed a method to counterfactually and automatically label data to improve language following! 🧵👇

svlevine's profile picture

Sergey Levine

132,500 subscribers

44,236 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Chain-of-thought reasoning is a powerful tool to enable language models to work through complex problems. Can we use this with robots? With embodied chain-of-thought, vision-language-action (VLA) models can think through perception and planning! A 🧵👇
1:39

Chain-of-thought reasoning is a powerful tool to enable language models to work through complex problems. Can we use this with robots? With embodied chain-of-thought, vision-language-action (VLA) models can think through perception and planning! A 🧵👇

Sergey Levine

30,479 Aufrufe • vor 2 Jahren

Compared to conventional VLAs, S1 is a step-change improvement. For known tasks, S1 can match the performance of language-prompted VLAs. For novel tasks, S1 exponentially outperforms any existing language-prompted VLAs as we scale the pre-training. We believe this is a promising direction towards establishing scaling laws for robotics.
0:27

Compared to conventional VLAs, S1 is a step-change improvement. For known tasks, S1 can match the performance of language-prompted VLAs. For novel tasks, S1 exponentially outperforms any existing language-prompted VLAs as we scale the pre-training. We believe this is a promising direction towards establishing scaling laws for robotics.

Skild AI

301,156 Aufrufe • vor 1 Tag

Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).
3:10

Vision-language models can control robots, but what if the prompt is too complex for the robot to follow directly? We developed a way to get robots to “think through” complex instructions, feedback, and interjections. We call it the Hierarchical Interactive Robot (Hi Robot).

Physical Intelligence

117,221 Aufrufe • vor 1 Jahr

Goal-conditioned RL (GCRL) is great - unsupervised, can use data (in offline mode), flexibility to define tasks at test time. But can we run GCRL on *language data*?? In our new work we show that language GCRL enables sophisticated test-time reasoning for interactive tasks! 🧵👇
0:22

Goal-conditioned RL (GCRL) is great - unsupervised, can use data (in offline mode), flexibility to define tasks at test time. But can we run GCRL on *language data*?? In our new work we show that language GCRL enables sophisticated test-time reasoning for interactive tasks! 🧵👇

Sergey Levine

18,782 Aufrufe • vor 1 Jahr

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.
2:53

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.

The Humanoid Hub

69,442 Aufrufe • vor 1 Jahr

Practice what you preach - Adviced to CM Vijay by our Annamalai ji. CM Vijay runs a school known as Vijay Vidyashram Three language system is practiced as part of CBSE curriculam So - 3 language for your children but two language for TVK Karyakartas !! Why Bro? So why is he denying right to study three language to children of TN!? DMK has injected a poison in minds of TN people saying 3 language means - imposition of Hindi In reality - No its only asking you to study any three language of your preference Just following #DMK foot steps - I guess !
1:22

Practice what you preach - Adviced to CM Vijay by our Annamalai ji. CM Vijay runs a school known as Vijay Vidyashram Three language system is practiced as part of CBSE curriculam So - 3 language for your children but two language for TVK Karyakartas !! Why Bro? So why is he denying right to study three language to children of TN!? DMK has injected a poison in minds of TN people saying 3 language means - imposition of Hindi In reality - No its only asking you to study any three language of your preference Just following #DMK foot steps - I guess !

RajeIyer

221,204 Aufrufe • vor 3 Monaten

Hard - Learning a 2nd language Harder - Giving a talk in a 2nd language Hardcore - Live coding while speaking a 2nd language This is for all of you who have to focus on English just to speak at conferences.
0:30

Hard - Learning a 2nd language Harder - Giving a talk in a 2nd language Hardcore - Live coding while speaking a 2nd language This is for all of you who have to focus on English just to speak at conferences.

David East

28,580 Aufrufe • vor 2 Jahren

Serious concern is mounting following remarks reportedly made by a Syrian army commander that use hostile and dehumanizing language toward the Kurdish population.
0:41

Serious concern is mounting following remarks reportedly made by a Syrian army commander that use hostile and dehumanizing language toward the Kurdish population.

Rania Hamzeh 🇸🇨🧉

10,232 Aufrufe • vor 7 Monaten

6. Learn the language of money The reason most stay poor is that they think money is complicated. But wealth speaks a simple language: - Make offers - Close deals - Buy assets - Repeat To learn a language, you have to practice.
1:02

6. Learn the language of money The reason most stay poor is that they think money is complicated. But wealth speaks a simple language: - Make offers - Close deals - Buy assets - Repeat To learn a language, you have to practice.

Codie Sanchez

20,440 Aufrufe • vor 1 Jahr

Prostate Enlargement in Igbo language via YouTube Prostate Enlargement in Igbo Language. Prostate Enlargement is a condition that can affect your ability to urinate. In this video in Igbo language; I’ve define what the prostate is.
2:20

Prostate Enlargement in Igbo language via YouTube Prostate Enlargement in Igbo Language. Prostate Enlargement is a condition that can affect your ability to urinate. In this video in Igbo language; I’ve define what the prostate is.

Sylvia

55,868 Aufrufe • vor 3 Jahren

🚀 Excited to announce the first release of a novel open source programming language and platform for language model interaction! Combining prompts, constraints & scripting, LMQL elevates the capabilities of large language models. 🧵1/6 A quick tour.
0:20

🚀 Excited to announce the first release of a novel open source programming language and platform for language model interaction! Combining prompts, constraints & scripting, LMQL elevates the capabilities of large language models. 🧵1/6 A quick tour.

LMQL (Language Model Query Language)

198,966 Aufrufe • vor 3 Jahren

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.
0:48

MotionGPT: Human Motion as a Foreign Language paper page: Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

AK

125,353 Aufrufe • vor 3 Jahren

We figured out how to train VLAs with diffusion outputs much faster (7.5x faster), inheriting better language following from the VLM, and leading to better results. The key: protect the VLM backbone during training with knowledge insulation. Let’s talk about what we learned👇
0:31

We figured out how to train VLAs with diffusion outputs much faster (7.5x faster), inheriting better language following from the VLM, and leading to better results. The key: protect the VLM backbone during training with knowledge insulation. Let’s talk about what we learned👇

Physical Intelligence

100,649 Aufrufe • vor 1 Jahr

As kindness is a language that can be understood by anybody, it makes sense that it is the language to which this cat responds🤍
0:28

As kindness is a language that can be understood by anybody, it makes sense that it is the language to which this cat responds🤍

Tansu Yegen

647,197 Aufrufe • vor 3 Jahren

I was an English-as-Second-Language learner when I moved to Canada with my family many years ago. I remember doing endless fill-in-the-blank exercises to practice English. Deep Learning Math is also a language. So I thought: why not use the same method to practice this math language? See more 👉
0:34

I was an English-as-Second-Language learner when I moved to Canada with my family many years ago. I remember doing endless fill-in-the-blank exercises to practice English. Deep Learning Math is also a language. So I thought: why not use the same method to practice this math language? See more 👉

Tom Yeh

87,893 Aufrufe • vor 8 Monaten

A Nigerian lady from a minority ethnic group in Bayelsa state who couldn’t speak her language decided to learn Yoruba so that she can at least speak one Nigerian language and pass it down. Yoruba language is the standard!!!
1:50

A Nigerian lady from a minority ethnic group in Bayelsa state who couldn’t speak her language decided to learn Yoruba so that she can at least speak one Nigerian language and pass it down. Yoruba language is the standard!!!

🇨🇦 Ademola🇨🇭

25,841 Aufrufe • vor 3 Monaten

woman is experiencing a priceless joy upon realizing that her dog, whom she has been trying to teach sign language for a while, has finally started to understand sign language.
0:37

woman is experiencing a priceless joy upon realizing that her dog, whom she has been trying to teach sign language for a while, has finally started to understand sign language.

Puppies & Positivity

1,051,894 Aufrufe • vor 7 Monaten

Humans learn and improve from failures. Similarly, foundation models adapt based on human feedback. Can we leverage this failure understanding to enhance robotics systems that use foundation models? Introducing AHA—a vision-language model for detecting and reasoning over failures in robotic manipulation. Project page: 🧵Thread👇 Aha!
1:29

Humans learn and improve from failures. Similarly, foundation models adapt based on human feedback. Can we leverage this failure understanding to enhance robotics systems that use foundation models? Introducing AHA—a vision-language model for detecting and reasoning over failures in robotic manipulation. Project page: 🧵Thread👇 Aha!

Jiafei Duan

48,793 Aufrufe • vor 1 Jahr

New research from Meta FAIR: Large Concept Models (LCM) is a fundamentally different paradigm for language modeling that decouples reasoning from language representation, inspired by how humans can plan high-level thoughts to communicate.
0:36

New research from Meta FAIR: Large Concept Models (LCM) is a fundamentally different paradigm for language modeling that decouples reasoning from language representation, inspired by how humans can plan high-level thoughts to communicate.

AI at Meta

531,765 Aufrufe • vor 1 Jahr