正在加载视频...

视频加载失败

返回首页

Build AR applications that recognize physical objects and provide real-time spatial guidance. Stijn Spanhove and Pavlo Tkachenko used Gemini 2.5 Pro’s multimodal vision and sound effect prompting capabilities to create an immersive experience with LEGO Smart Bricks and Snap Spectacles.

googleaidevs's profile picture

Google AI Developers

116,031 subscribers

48,222 次观看 • 5 个月前 •via X (Twitter)

科学技术
LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

🚀 Just launched… a Starship! (Well, in AR 😉) Built an AR experience for Snapchat Spectacles where you launch a SpaceX Starship, and guide the booster back to a 3D-printed launch tower using a pinch gesture. Super interesting to blend physical objects with spatial interaction!
0:46

🚀 Just launched… a Starship! (Well, in AR 😉) Built an AR experience for Snapchat Spectacles where you launch a SpaceX Starship, and guide the booster back to a 3D-printed launch tower using a pinch gesture. Super interesting to blend physical objects with spatial interaction!

Stijn Spanhove

10,799 次观看 • 1 年前

We unveiled the new Multimodal Live API for Gemini 2.0 Flash Experimental. ⚡ Developers can harness the power of real-time audio and video streaming to create anything from interactive assistants that respond to your voice to live translation applications. ↓
2:30

We unveiled the new Multimodal Live API for Gemini 2.0 Flash Experimental. ⚡ Developers can harness the power of real-time audio and video streaming to create anything from interactive assistants that respond to your voice to live translation applications. ↓

Google DeepMind

117,701 次观看 • 1 年前

Engineer builds AR ad blocker for real life using Snap Spectacles and Gemini AI | Aamir Khollam, Interesting Engineering A prototype AR app detects ads in your surroundings and blocks them live through Snap Spectacles using Google’s Gemini AI. You can pay for YouTube Premium to avoid watching ads during your sacred food-with-YouTube ritual, or pay for Spotify to not pester you with ads while you try to get a morning workout in, but what can you do about real-life advertisements? A Belgian software developer may have found a workaround. Stijn Spanhove has created an experimental augmented reality app that detects and digitally covers physical ads like billboards and soda cans in the real world. Built for Snap’s fifth-generation AR glasses (Snap Spectacles), the prototype uses Google’s Gemini AI to identify branded content and instantly mask it. Instead of the original imagery, the app places a bright red square over the detected ad. These red blocks also name the hidden brand, like “Bol. billboard,” turning ad removal into a kind of real-time brand callout. “It’s exciting to imagine a future where you control the physical content you see,” Spanhove posted on X (formerly Twitter). In follow-up replies, he hinted at additional features, including options to replace the red square with personal photos or text from a notes app. Check it out below – Combines Snap and Google tools The app relies on Snap’s Depth Cache API to register objects in 3D space and maintain spatial consistency as the user moves. Gemini, Google’s generative AI model, identifies the ads themselves, whether on large posters, newspaper pages, or food packaging. This allows the blocker to function beyond obvious signage. In demo clips shared by Spanhove, it successfully covers ads on cereal boxes, magazines, and public signage, though not without a delay of a second or two. The red overlays appear to float stably, following head movements and perspective shifts with accuracy. Still, it’s early days. Spanhove describes the software as “experimental,” and the user experience reflects that. Because Snap Spectacles use transparent displays, the overlays can’t fully block light, so the original ad sometimes faintly shows through. Also, the Spectacles’ narrow 46-degree field of view limits coverage to only what’s directly ahead. Design stirs debate The app has sparked strong reactions online. Many praised the concept of user-controlled visual space, while others criticized the bold red boxes as more jarring than the ads themselves. One suggestion, from user Hexographer ⬣Backup account⬣, proposed replacing the red blocks with more visually pleasing alternatives, such as “images of local foliage or animal life.” They also suggested that users could set a custom folder of replacement visuals, including personal or family photos. Others asked for cross-platform support, but the app currently only works with Snap Spectacles. Devices like the Apple Vision Pro or Meta Quest will need separate development efforts. Despite its limitations, the project arrives at a moment when major tech firms like Meta and Microsoft have scaled back their AR ambitions. Snap, meanwhile, continues renting its Spectacles to developers at $99 per month, making experiments like Spanhove’s possible. Read more:
0:29

Engineer builds AR ad blocker for real life using Snap Spectacles and Gemini AI | Aamir Khollam, Interesting Engineering A prototype AR app detects ads in your surroundings and blocks them live through Snap Spectacles using Google’s Gemini AI. You can pay for YouTube Premium to avoid watching ads during your sacred food-with-YouTube ritual, or pay for Spotify to not pester you with ads while you try to get a morning workout in, but what can you do about real-life advertisements? A Belgian software developer may have found a workaround. Stijn Spanhove has created an experimental augmented reality app that detects and digitally covers physical ads like billboards and soda cans in the real world. Built for Snap’s fifth-generation AR glasses (Snap Spectacles), the prototype uses Google’s Gemini AI to identify branded content and instantly mask it. Instead of the original imagery, the app places a bright red square over the detected ad. These red blocks also name the hidden brand, like “Bol. billboard,” turning ad removal into a kind of real-time brand callout. “It’s exciting to imagine a future where you control the physical content you see,” Spanhove posted on X (formerly Twitter). In follow-up replies, he hinted at additional features, including options to replace the red square with personal photos or text from a notes app. Check it out below – Combines Snap and Google tools The app relies on Snap’s Depth Cache API to register objects in 3D space and maintain spatial consistency as the user moves. Gemini, Google’s generative AI model, identifies the ads themselves, whether on large posters, newspaper pages, or food packaging. This allows the blocker to function beyond obvious signage. In demo clips shared by Spanhove, it successfully covers ads on cereal boxes, magazines, and public signage, though not without a delay of a second or two. The red overlays appear to float stably, following head movements and perspective shifts with accuracy. Still, it’s early days. Spanhove describes the software as “experimental,” and the user experience reflects that. Because Snap Spectacles use transparent displays, the overlays can’t fully block light, so the original ad sometimes faintly shows through. Also, the Spectacles’ narrow 46-degree field of view limits coverage to only what’s directly ahead. Design stirs debate The app has sparked strong reactions online. Many praised the concept of user-controlled visual space, while others criticized the bold red boxes as more jarring than the ads themselves. One suggestion, from user Hexographer ⬣Backup account⬣, proposed replacing the red blocks with more visually pleasing alternatives, such as “images of local foliage or animal life.” They also suggested that users could set a custom folder of replacement visuals, including personal or family photos. Others asked for cross-platform support, but the app currently only works with Snap Spectacles. Devices like the Apple Vision Pro or Meta Quest will need separate development efforts. Despite its limitations, the project arrives at a moment when major tech firms like Meta and Microsoft have scaled back their AR ambitions. Snap, meanwhile, continues renting its Spectacles to developers at $99 per month, making experiments like Spanhove’s possible. Read more:

Owen Gregorian

45,774 次观看 • 1 年前

How does an AI model actually learn to see? 🤖 Learn about the tech behind native multimodality, how models reason over visual data like documents and video, and the future of proactive AI assistants with Logan Kilpatrick and Gemini Model Behavior Product Lead, Ani Baddepudi. ↓ Timestamps: 01:12 Why Gemini is natively multimodal 02:23 The technology behind multimodal models 05:15 Video understanding with Gemini 2.5 09:25 Deciding what to build next 13:23 Building new product experiences with multimodal AI 17:15 The vision for proactive assistants 24:13 Improving video usability with variable FPS and frame tokenization 27:35 What’s next for Gemini’s multimodal development 31:47 Deep dive on Gemini’s document understanding capabilities 37:56 The teamwork and collaboration behind Gemini 40:56 What’s next with model behavior
44:17

How does an AI model actually learn to see? 🤖 Learn about the tech behind native multimodality, how models reason over visual data like documents and video, and the future of proactive AI assistants with Logan Kilpatrick and Gemini Model Behavior Product Lead, Ani Baddepudi. ↓ Timestamps: 01:12 Why Gemini is natively multimodal 02:23 The technology behind multimodal models 05:15 Video understanding with Gemini 2.5 09:25 Deciding what to build next 13:23 Building new product experiences with multimodal AI 17:15 The vision for proactive assistants 24:13 Improving video usability with variable FPS and frame tokenization 27:35 What’s next for Gemini’s multimodal development 31:47 Deep dive on Gemini’s document understanding capabilities 37:56 The teamwork and collaboration behind Gemini 40:56 What’s next with model behavior

Google AI

58,703 次观看 • 1 年前

imagine a world where geospatial AI, VPS, and AR glasses meet. we’re actively exploring the magic of what that looks like with Dots as spatially intelligent agents for navigation. cc: Spectacles Peridot Niantic Spatial 🌎 ICYMI, we also announced our multi-year strategic partnership, including an investment from Snap Inc., to build the next-gen AI-powered map of the real world.
1:14

imagine a world where geospatial AI, VPS, and AR glasses meet. we’re actively exploring the magic of what that looks like with Dots as spatially intelligent agents for navigation. cc: Spectacles Peridot Niantic Spatial 🌎 ICYMI, we also announced our multi-year strategic partnership, including an investment from Snap Inc., to build the next-gen AI-powered map of the real world.

asim ᯅ

51,901 次观看 • 1 年前

Listen up 🔊 We’ve made some updates to our Gemini Audio models and capabilities: — Gemini’s live speech-to-speech translation capability is rolling out in a beta experience to the Google Translate app, bringing you real-time audio translation that captures the nuance of human speech — Gemini 2.5 Flash and 2.5 Pro Text-to-Speech preview models bring improved adherence to style prompts, precision pacing with context-aware speed adjustments, and character voice consistency for multi-speaker scenarios — Gemini 2.5 Flash Native Audio is now updated, with improvements to handle complex workflows, navigate user instructions, and hold natural conversations
2:50

Listen up 🔊 We’ve made some updates to our Gemini Audio models and capabilities: — Gemini’s live speech-to-speech translation capability is rolling out in a beta experience to the Google Translate app, bringing you real-time audio translation that captures the nuance of human speech — Gemini 2.5 Flash and 2.5 Pro Text-to-Speech preview models bring improved adherence to style prompts, precision pacing with context-aware speed adjustments, and character voice consistency for multi-speaker scenarios — Gemini 2.5 Flash Native Audio is now updated, with improvements to handle complex workflows, navigate user instructions, and hold natural conversations

Google AI

1,092,883 次观看 • 7 个月前

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:
2:49

Explore state-of-the-art multimodal prompting in our new short course Large Multimodal Model Prompting with Gemini, taught by Erwin Huizenga in collaboration with Google Cloud. One interesting insight from this course: with multimodal models, prompt structure matters significantly. Placing text inputs, such as a patient's medical history, before image inputs, like an X-ray, can enhance the model's ability to contextualize and interpret visual data effectively. In other contexts, such as image captioning, you may get better results by putting the image first. Multimodal models behave differently than text-only LLMs, and effective prompting for models varies depending on the model you’re using. In this course you’ll learn how to effectively prompt Gemini models. Gemini's multimodal capabilities also enable new approaches in AI application development, for example: - The Gemini library handles various video formats (MP4, MOV, MPEG), streamlining applications using these formats. - Large context window (up to 1 million tokens) enables processing of extensive content, like analyzing multiple 50-minute videos simultaneously. - Function calling feature integrates real-time data (e.g., current exchange rates) into model responses. The course demonstrates building multimodal applications with real-world examples including document analyzers that reason across text and graphs simultaneously, video content extractors that find and timestamp specific information from multiple hours of footage, and automated expense report systems processing receipt images while cross-referencing company policies. Sign up here:

Andrew Ng

74,060 次观看 • 1 年前

VITA Towards Open-Source Interactive Omni Multimodal LLM discuss: The remarkable multimodal capabilities and interactive experience of GPT-4o underscore their necessity in practical applications, yet open-source models rarely excel in both areas. In this paper, we introduce VITA, the first-ever open-source Multimodal Large Language Model (MLLM) adept at simultaneous processing and analysis of Video, Image, Text, and Audio modalities, and meanwhile has an advanced multimodal interactive experience. Starting from Mixtral 8x7B as a language foundation, we expand its Chinese vocabulary followed by bilingual instruction tuning. We further endow the language model with visual and audio capabilities through two-stage multi-task learning of multimodal alignment and instruction tuning. VITA demonstrates robust foundational capabilities of multilingual, vision, and audio understanding, as evidenced by its strong performance across a range of both unimodal and multimodal benchmarks. Beyond foundational capabilities, we have made considerable progress in enhancing the natural multimodal human-computer interaction experience. To the best of our knowledge, we are the first to exploit non-awakening interaction and audio interrupt in MLLM. VITA is the first step for the open-source community to explore the seamless integration of multimodal understanding and interaction. While there is still lots of work to be done on VITA to get close to close-source counterparts, we hope that its role as a pioneer can serve as a cornerstone for subsequent research.
2:21

VITA Towards Open-Source Interactive Omni Multimodal LLM discuss: The remarkable multimodal capabilities and interactive experience of GPT-4o underscore their necessity in practical applications, yet open-source models rarely excel in both areas. In this paper, we introduce VITA, the first-ever open-source Multimodal Large Language Model (MLLM) adept at simultaneous processing and analysis of Video, Image, Text, and Audio modalities, and meanwhile has an advanced multimodal interactive experience. Starting from Mixtral 8x7B as a language foundation, we expand its Chinese vocabulary followed by bilingual instruction tuning. We further endow the language model with visual and audio capabilities through two-stage multi-task learning of multimodal alignment and instruction tuning. VITA demonstrates robust foundational capabilities of multilingual, vision, and audio understanding, as evidenced by its strong performance across a range of both unimodal and multimodal benchmarks. Beyond foundational capabilities, we have made considerable progress in enhancing the natural multimodal human-computer interaction experience. To the best of our knowledge, we are the first to exploit non-awakening interaction and audio interrupt in MLLM. VITA is the first step for the open-source community to explore the seamless integration of multimodal understanding and interaction. While there is still lots of work to be done on VITA to get close to close-source counterparts, we hope that its role as a pioneer can serve as a cornerstone for subsequent research.

AK

23,958 次观看 • 1 年前

AI Guidance for Physical Work David Lieb Much of the AI conversation focuses on which desk jobs will get replaced. But for physical work, AI can't yet act in the world. With multimodal models and ubiquitous hardware, there’s an opportunity to give workers live AI coaching that dramatically reduces training time and unlocks skilled labor at scale.
1:40

AI Guidance for Physical Work David Lieb Much of the AI conversation focuses on which desk jobs will get replaced. But for physical work, AI can't yet act in the world. With multimodal models and ubiquitous hardware, there’s an opportunity to give workers live AI coaching that dramatically reduces training time and unlocks skilled labor at scale.

Y Combinator

26,533 次观看 • 5 个月前

InternLM-XComposer-2.5 A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output We present InternLM-XComposer-2.5 (IXC-2.5), a versatile large-vision language model that supports long-contextual input and output. IXC-2.5 excels in various text-image comprehension and composition applications, achieving GPT-4V level capabilities with merely 7B LLM backend. Trained with 24K interleaved image-text contexts, it can seamlessly extend to 96K long contexts via RoPE extrapolation. This long-context capability allows IXC-2.5 to excel in tasks requiring extensive input and output contexts. Compared to its previous 2.0 version, InternLM-XComposer-2.5 features three major upgrades in vision-language comprehension: (1) Ultra-High Resolution Understanding, (2) Fine-Grained Video Understanding, and (3) Multi-Turn Multi-Image Dialogue. In addition to comprehension, IXC-2.5 extends to two compelling applications using extra LoRA parameters for text-image composition: (1) Crafting Webpages and (2) Composing High-Quality Text-Image Articles. IXC-2.5 has been evaluated on 28 benchmarks, outperforming existing open-source state-of-the-art models on 16 benchmarks. It also surpasses or competes closely with GPT-4V and Gemini Pro on 16 key tasks.
2:45

InternLM-XComposer-2.5 A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output We present InternLM-XComposer-2.5 (IXC-2.5), a versatile large-vision language model that supports long-contextual input and output. IXC-2.5 excels in various text-image comprehension and composition applications, achieving GPT-4V level capabilities with merely 7B LLM backend. Trained with 24K interleaved image-text contexts, it can seamlessly extend to 96K long contexts via RoPE extrapolation. This long-context capability allows IXC-2.5 to excel in tasks requiring extensive input and output contexts. Compared to its previous 2.0 version, InternLM-XComposer-2.5 features three major upgrades in vision-language comprehension: (1) Ultra-High Resolution Understanding, (2) Fine-Grained Video Understanding, and (3) Multi-Turn Multi-Image Dialogue. In addition to comprehension, IXC-2.5 extends to two compelling applications using extra LoRA parameters for text-image composition: (1) Crafting Webpages and (2) Composing High-Quality Text-Image Articles. IXC-2.5 has been evaluated on 28 benchmarks, outperforming existing open-source state-of-the-art models on 16 benchmarks. It also surpasses or competes closely with GPT-4V and Gemini Pro on 16 key tasks.

AK

66,472 次观看 • 2 年前

Wait, did Edy just say you can build full software applications on PWR? 🎥 Check out this video breaking down how developers can create powerful software applications, not just limited to what smart contracts can handle 👇 Use Familiar Coding Languages – Develop on blockchain using the programming languages you already know and love, like Python, Java or JavaScript. Automated & Gas-Free – Say hello to advanced features like automation and zero gas fees, making your applications smarter and more efficient. Versatile & Powerful – Build apps that can process complex logic, interact with a range of algorithms and offer more capabilities than ever before.
0:26

Wait, did Edy just say you can build full software applications on PWR? 🎥 Check out this video breaking down how developers can create powerful software applications, not just limited to what smart contracts can handle 👇 Use Familiar Coding Languages – Develop on blockchain using the programming languages you already know and love, like Python, Java or JavaScript. Automated & Gas-Free – Say hello to advanced features like automation and zero gas fees, making your applications smarter and more efficient. Versatile & Powerful – Build apps that can process complex logic, interact with a range of algorithms and offer more capabilities than ever before.

PWR Chain

20,821 次观看 • 1 年前

Boom! Open source LegoGPT is a building AI, and sure it can be used for Legos but it can also be used for Lego-like building of homes. LegoGPT converts meshes to Lego in one step using 1×1, 1×2, 1×4, 1×6, 1×8, 2×2, 2×4, and 2×6 bricks. Then they evaluate the stability of the design. Finally, they render an image and ask GPT-4o to produce captions to go with the image. This sent to robots and they complete the real-time build. This can absolutely scale and as with Legos, you can use smaller pieces for higher resolution. Testing it detail now with robot assembly. Link:
0:45

Boom! Open source LegoGPT is a building AI, and sure it can be used for Legos but it can also be used for Lego-like building of homes. LegoGPT converts meshes to Lego in one step using 1×1, 1×2, 1×4, 1×6, 1×8, 2×2, 2×4, and 2×6 bricks. Then they evaluate the stability of the design. Finally, they render an image and ask GPT-4o to produce captions to go with the image. This sent to robots and they complete the real-time build. This can absolutely scale and as with Legos, you can use smaller pieces for higher resolution. Testing it detail now with robot assembly. Link:

Brian Roemmele

22,363 次观看 • 1 年前

Really happy to see the interest around our “Hands-on with Gemini” video. In our developer blog yesterday, we broke down how Gemini was used to create it. We gave Gemini sequences of different modalities — image and text in this case — and had it respond by predicting what might come next. Devs can try similar things when access to Pro opens on 12/13 🚀. The knitting demo used Ultra⚡ All the user prompts and outputs in the video are real, shortened for brevity. The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers. When you’re building an app, you can get similar results (there’s always some variability with LLMs) by prompting Gemini with an instruction that allows the user to "configure" the behavior of the model, like inputting “you are an expert in science …” before a user can engage in the same kind of back and forth dialogue. Here’s a clip of what this looks like in AI Studio with Gemini Pro. We’ve come a long way since Flamingo 🦩 & PALI, looking forward to seeing what people build with it!
2:24

Really happy to see the interest around our “Hands-on with Gemini” video. In our developer blog yesterday, we broke down how Gemini was used to create it. We gave Gemini sequences of different modalities — image and text in this case — and had it respond by predicting what might come next. Devs can try similar things when access to Pro opens on 12/13 🚀. The knitting demo used Ultra⚡ All the user prompts and outputs in the video are real, shortened for brevity. The video illustrates what the multimodal user experiences built with Gemini could look like. We made it to inspire developers. When you’re building an app, you can get similar results (there’s always some variability with LLMs) by prompting Gemini with an instruction that allows the user to "configure" the behavior of the model, like inputting “you are an expert in science …” before a user can engage in the same kind of back and forth dialogue. Here’s a clip of what this looks like in AI Studio with Gemini Pro. We’ve come a long way since Flamingo 🦩 & PALI, looking forward to seeing what people build with it!

Oriol Vinyals

180,966 次观看 • 2 年前

Unreal Engine 5.7 is here, with the power to help you: 🌎Build and render rich, dense open worlds. 🌿Render beautiful, detailed vegetation in real-time with Nanite Foliage. ✨Blend with true physical accuracy using Substrate Materials. 🎬Experience smoother animation workflows. 💪Get expert guidance from the new in-editor AI Assistant. And there’s even more to explore, see what else is new:
2:50

Unreal Engine 5.7 is here, with the power to help you: 🌎Build and render rich, dense open worlds. 🌿Render beautiful, detailed vegetation in real-time with Nanite Foliage. ✨Blend with true physical accuracy using Substrate Materials. 🎬Experience smoother animation workflows. 💪Get expert guidance from the new in-editor AI Assistant. And there’s even more to explore, see what else is new:

Unreal Engine

234,670 次观看 • 8 个月前

🚨XI TEASES FAST, SMART CODING AI WITH VIDEO NEXT A specialized coding model is already in the works, built to be fast and smart, not just powerful. Next up: fixing Grok’s blurry multimodal vision. The new model will see, hear, and understand the world like a human. After that comes video. Think: infinite scroll, generative content, and real-time story-building. Training begins in weeks using 100,000+ GB200s. XI’s goal isn’t just smarter AI, it’s real-time intelligence with a throttle. Source: xAI
2:39

🚨XI TEASES FAST, SMART CODING AI WITH VIDEO NEXT A specialized coding model is already in the works, built to be fast and smart, not just powerful. Next up: fixing Grok’s blurry multimodal vision. The new model will see, hear, and understand the world like a human. After that comes video. Think: infinite scroll, generative content, and real-time story-building. Training begins in weeks using 100,000+ GB200s. XI’s goal isn’t just smarter AI, it’s real-time intelligence with a throttle. Source: xAI

Mario Nawfal

80,589 次观看 • 1 年前

In their latest video, Boston Dynamics’s AI team explains how they make the Atlas humanoid perceive and interact with the world. Atlas uses an agile perception system to understand both the shape and context of objects in complex environments. Atlas combines 2D and 3D awareness, keypoint-based localization, and an object pose tracking system that fuses vision, kinematics, and object knowledge to handle occlusion and uncertainty. Accurate calibration ensures precise hand–eye coordination for reliable manipulation. The team is now working toward a unified model that merges perception and control – pushing beyond spatial AI toward physical intelligence.
6:34

In their latest video, Boston Dynamics’s AI team explains how they make the Atlas humanoid perceive and interact with the world. Atlas uses an agile perception system to understand both the shape and context of objects in complex environments. Atlas combines 2D and 3D awareness, keypoint-based localization, and an object pose tracking system that fuses vision, kinematics, and object knowledge to handle occlusion and uncertainty. Accurate calibration ensures precise hand–eye coordination for reliable manipulation. The team is now working toward a unified model that merges perception and control – pushing beyond spatial AI toward physical intelligence.

The Humanoid Hub

50,051 次观看 • 1 年前