Загрузка видео...

Не удалось загрузить видео

На главную

How can agents understand the world from diverse language? 🌎 Excited to introduce Dynalang, an agent that learns to understand language by 𝙢𝙖𝙠𝙞𝙣𝙜 𝙥𝙧𝙚𝙙𝙞𝙘𝙩𝙞𝙤𝙣𝙨 𝙖𝙗𝙤𝙪𝙩 𝙩𝙝𝙚 𝙛𝙪𝙩𝙪𝙧𝙚 with a multimodal world model!

realJessyLin's profile picture

Jessy Lin

5,278 subscribers

107,491 просмотров • 3 лет назад •via X (Twitter)

Наука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

π0.7 handles diverse prompts that don't just say what to do, but also how to do it, including rich language and multimodal information, such as visual subgoal images. At test time, these images can be produced by a lightweight world model.
0:29

π0.7 handles diverse prompts that don't just say what to do, but also how to do it, including rich language and multimodal information, such as visual subgoal images. At test time, these images can be produced by a lightweight world model.

Physical Intelligence

33,748 просмотров • 3 месяцев назад

LARP: Language-Agent Role Play for Open-World Games paper page: Language agents have shown impressive problem-solving skills within defined settings and brief timelines. Yet, with the ever-evolving complexities of open-world simulations, there's a pressing need for agents that can flexibly adapt to complex environments and consistently maintain a long-term memory to ensure coherent actions. To bridge the gap between language agents and open-world games, we introduce Language Agent for Role-Playing (LARP), which includes a cognitive architecture that encompasses memory processing and a decision-making assistant, an environment interaction module with a feedback-driven learnable action space, and a postprocessing method that promotes the alignment of various personalities. The LARP framework refines interactions between users and agents, predefined with unique backgrounds and personalities, ultimately enhancing the gaming experience in open-world contexts. Furthermore, it highlights the diverse uses of language models in a range of areas such as entertainment, education, and various simulation scenarios.
1:03

LARP: Language-Agent Role Play for Open-World Games paper page: Language agents have shown impressive problem-solving skills within defined settings and brief timelines. Yet, with the ever-evolving complexities of open-world simulations, there's a pressing need for agents that can flexibly adapt to complex environments and consistently maintain a long-term memory to ensure coherent actions. To bridge the gap between language agents and open-world games, we introduce Language Agent for Role-Playing (LARP), which includes a cognitive architecture that encompasses memory processing and a decision-making assistant, an environment interaction module with a feedback-driven learnable action space, and a postprocessing method that promotes the alignment of various personalities. The LARP framework refines interactions between users and agents, predefined with unique backgrounds and personalities, ultimately enhancing the gaming experience in open-world contexts. Furthermore, it highlights the diverse uses of language models in a range of areas such as entertainment, education, and various simulation scenarios.

AK

143,974 просмотров • 2 лет назад

The world keeps trying to reach a deal with Iran, while the Iranian regime keeps adding more and more demands. When will the world understand that with a regime like Iran’s, the only language they understand is strength?
0:57

The world keeps trying to reach a deal with Iran, while the Iranian regime keeps adding more and more demands. When will the world understand that with a regime like Iran’s, the only language they understand is strength?

Hananya Naftali

16,794 просмотров • 4 дней назад

Excited to introduce SketchAgent!👩‍🎨 We leverage the prior of pretrained multimodal LLMs for language-driven, sequential sketch generation and human-agent collaborative sketching! ✨ Try our fun interface here:
1:21

Excited to introduce SketchAgent!👩‍🎨 We leverage the prior of pretrained multimodal LLMs for language-driven, sequential sketch generation and human-agent collaborative sketching! ✨ Try our fun interface here:

Yael Vinker

31,956 просмотров • 1 год назад

We are excited to announce the 1st version of our multimodal assistant, Yasa-1, a language assistant with visual and auditory sensors that can take actions via code execution 🪄. Yasa-1 can understand text, images, videos, sounds & more! 🚀 Check out more details below👇
0:45

We are excited to announce the 1st version of our multimodal assistant, Yasa-1, a language assistant with visual and auditory sensors that can take actions via code execution 🪄. Yasa-1 can understand text, images, videos, sounds & more! 🚀 Check out more details below👇

Reka

814,303 просмотров • 2 лет назад

NEWS: Nvidia has just introduced "Cosmos," a world foundation model created to understand the physical world. The model can generate synthetic data to train robotics.
2:34

NEWS: Nvidia has just introduced "Cosmos," a world foundation model created to understand the physical world. The model can generate synthetic data to train robotics.

Sawyer Merritt

254,296 просмотров • 1 год назад

Eric Schmidt: the point at which AI agents can talk to each other in a language we can't understand, we should unplug the computers
0:59

Eric Schmidt: the point at which AI agents can talk to each other in a language we can't understand, we should unplug the computers

Tsarathustra

154,517 просмотров • 2 лет назад

🚀 We introduce Neural Theorizer (NEO) — a new type of world model that learns to theorize the world from observation, without language or LLM supervision. Selected as an ICML 2026 oral presentation — 0.7% of submitted papers. The paper asks: "What does it mean to understand the world and build a world model?" Today’s world models are often trained to predict the future: the next frame, next latent state, or next observation. But is prediction enough? We argue that a world model should be a theory-building system: one that discovers reusable primitives, composes them into executable explanations, and transfers those explanations to novel phenomena. NEO is our first step toward this vision — a World Theory Model that learns explicit, compositional theories from raw observation. This work was led by my wonderful students: Doojin Baek*(Doojin Baek), Gyubin Lee* (GyuBin Lee), Junyeob Baek (Junyeob Baek), and Hosung Lee (Hosung Lee). For more details, take a look at the paper — and if you’re attending ICML, let’s talk there! 📄 arXiv: 🌐 Project page:
0:00

🚀 We introduce Neural Theorizer (NEO) — a new type of world model that learns to theorize the world from observation, without language or LLM supervision. Selected as an ICML 2026 oral presentation — 0.7% of submitted papers. The paper asks: "What does it mean to understand the world and build a world model?" Today’s world models are often trained to predict the future: the next frame, next latent state, or next observation. But is prediction enough? We argue that a world model should be a theory-building system: one that discovers reusable primitives, composes them into executable explanations, and transfers those explanations to novel phenomena. NEO is our first step toward this vision — a World Theory Model that learns explicit, compositional theories from raw observation. This work was led by my wonderful students: Doojin Baek*(Doojin Baek), Gyubin Lee* (GyuBin Lee), Junyeob Baek (Junyeob Baek), and Hosung Lee (Hosung Lee). For more details, take a look at the paper — and if you’re attending ICML, let’s talk there! 📄 arXiv: 🌐 Project page:

Sungjin Ahn

99,136 просмотров • 1 месяц назад

Openclaw / Agents can now command ANY robot to grasp objects via language Integrate dimensional with your personal agent to unlock the physical world Fully open source. Join the hundreds of developers already vibecoding atoms
1:09

Openclaw / Agents can now command ANY robot to grasp objects via language Integrate dimensional with your personal agent to unlock the physical world Fully open source. Join the hundreds of developers already vibecoding atoms

stash

19,682 просмотров • 4 месяцев назад

Embeddings (and how to create them) are, perhaps, the most interesting idea behind Large Language Models. I built a simple model to help you understand embeddings from scratch. Here is a step-by-step video explanation:
51:43

Embeddings (and how to create them) are, perhaps, the most interesting idea behind Large Language Models. I built a simple model to help you understand embeddings from scratch. Here is a step-by-step video explanation:

Santiago

44,107 просмотров • 1 год назад

The only way for Americans to understand the true impact of Trump tariffs is to speak to them in a language they can understand 👇
0:51

The only way for Americans to understand the true impact of Trump tariffs is to speak to them in a language they can understand 👇

Cyrus Janssen

117,507 просмотров • 1 год назад

Perceptron’s platform is here — built for Physical AI Developers can now use Isaac-0.1 or Qwen3VL 235B via: Perceptron API — fast, reliable multimodal intelligence Python SDK — simple, grounded prompting for vision + language Build apps that see and understand the world.
0:27

Perceptron’s platform is here — built for Physical AI Developers can now use Isaac-0.1 or Qwen3VL 235B via: Perceptron API — fast, reliable multimodal intelligence Python SDK — simple, grounded prompting for vision + language Build apps that see and understand the world.

Perceptron AI

36,095 просмотров • 9 месяцев назад

If you understand that the common language of the world is humanity, you will understand the importance of this message. The zio regime is still butchering Palestinians. Please go to part 2 #humanity #truth
3:40

If you understand that the common language of the world is humanity, you will understand the importance of this message. The zio regime is still butchering Palestinians. Please go to part 2 #humanity #truth

Nicole Jenes

33,134 просмотров • 1 год назад

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models paper page: Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.
0:22

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models paper page: Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.

AK

141,440 просмотров • 2 лет назад

1- I want to introduce you to Palestinian heritage 🧵 This song is from the resistance heritage. If you don't understand the song, it's not because you don't know Arabic. Arabs don't understand it either because it's a coded language that Palestinians invented.
1:47

1- I want to introduce you to Palestinian heritage 🧵 This song is from the resistance heritage. If you don't understand the song, it's not because you don't know Arabic. Arabs don't understand it either because it's a coded language that Palestinians invented.

Muhammad ( Taqi al-Din )

121,681 просмотров • 2 лет назад

I built a Multimodal AI Agent using Gemini 2.0 that can understand videos and search the internet at the same time. 100% Opensource Code with step-by-step tutorial.
0:48

Sensitive content

I built a Multimodal AI Agent using Gemini 2.0 that can understand videos and search the internet at the same time. 100% Opensource Code with step-by-step tutorial.

Shubham Saboo

97,816 просмотров • 1 год назад

The only way world can survive this community .. It's to follow China 🇨🇳 way .. Sooner the world 🌎 can understand this .. Better it would be ..
1:27

The only way world can survive this community .. It's to follow China 🇨🇳 way .. Sooner the world 🌎 can understand this .. Better it would be ..

AstroCounselKK 🇮🇳

20,974 просмотров • 7 месяцев назад

3D-VLA A 3D Vision-Language-Action Generative World Model Recent vision-language-action (VLA) models rely on 2D inputs, lacking integration with the broader realm of the 3D physical world. Furthermore, they perform action prediction by learning a direct mapping from
1:43

3D-VLA A 3D Vision-Language-Action Generative World Model Recent vision-language-action (VLA) models rely on 2D inputs, lacking integration with the broader realm of the 3D physical world. Furthermore, they perform action prediction by learning a direct mapping from

AK

85,570 просмотров • 2 лет назад

Warren Buffett: "You have to understand accounting. It's the language [of business]. It's like being in a foreign country without knowing the language if you're in business and you don't understand accounting." "You want to get as comfortable with [accounting] as you are with the English language."
1:56

Warren Buffett: "You have to understand accounting. It's the language [of business]. It's like being in a foreign country without knowing the language if you're in business and you don't understand accounting." "You want to get as comfortable with [accounting] as you are with the English language."

Kevin Carpenter

36,031 просмотров • 1 месяц назад