正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

It’s finally here! 🎊 We open sourced our #vtuber motion capture solution at #GoogleIO! Our new MediaPipe model predicts 478 face landmarks + 52 blendshapes from your webcam and is compatible with any ARKit rigged avatar! 😺🧵

Rich 🍈

4,479 subscribers

508,202 次观看 • 3 年前 •via X (Twitter)

教育艺术科学技术 #vtuber #GoogleIO

Anya Rossi• Live Now

Private livecam show

11 条评论

Rich 🍈 的头像

Rich 🍈3 年前

Try out our web demo at: Ready for Android, JavaScript, C++ or Python developers. We’re gonna see a lot more new Vtuber apps 👀

Rich 🍈 的头像

Rich 🍈3 年前

So glad to finally be able to share this. It’s been amazing to work with such talented teammates at Google AI to bring this into reality. Super cathartic moment. 😊

Rich 🍈 的头像

Rich 🍈3 年前

Oh hey, you can find my beginner example on the IO website but here's a direct link to the Codepen. It's a super minimal example showcasing Mediapipe's 52 blendshapes + transformation matrix for AR pinning. A great starting point to understand the API!

🥉 Pipkin Pippa 🔌🐰 Phase-Connect 的头像

🥉 Pipkin Pippa 🔌🐰 Phase-Connect3 年前

This looks amazing!!

Rich 🍈 的头像

Rich 🍈3 年前

Whoa it’s the real Pippa 👀! Thanks!

Fireproof 🐱 VTuber 的头像

Fireproof 🐱 VTuber3 年前

This is insanely cool! Is there a way to get it to work with .vrm models from VRoid or would I have to add ARKit support to that avatar?

Rich 🍈 的头像

Rich 🍈3 年前

One recommended way is to follow some tutorials on Vroid to PerfectSync blendshape conversion. If the avatar was created in Vroid Studio, Perfect Sync can autorig it to ARKit spec.

Butz Yung 的头像

Butz Yung3 年前

Checking the demo right now. Blink is a bit off when I have my glasses on. Things are fine when glasses are off. Now I need to think about how to map blendshapes to MMD morphs lol

TigerHix 的头像

TigerHix3 年前

Amazing work!! Congrats on the release. Surely integrating this into @hakuyalabs soon!

bell ᶘ •̀ ᴥ•́ ᶅ✧ 的头像

bell ᶘ •̀ ᴥ•́ ᶅ✧3 年前

this feels like an overly paranoid question but I just want to be sure, is any of the image/cam data being sent back to a server on google's end, or is it kept entirely client-side? with the state of AI datasets being non-con by default I'm wary of anything with the AI label

Rich 🍈 的头像

Rich 🍈3 年前

Yup, it’s all done locally. These types of prediction models are all local on your device. There’s no need to run a server for this type of AI. Generative stuff is what typically uses a server because it requires much more processing power.

相关视频

VideoJAM is our new framework for improved motion generation from AI at Meta We show that video generators struggle with motion because the training objective favors appearance over dynamics. VideoJAM directly adresses this **without any extra data or scaling** 👇🧵

VideoJAM is our new framework for improved motion generation from AI at Meta We show that video generators struggle with motion because the training objective favors appearance over dynamics. VideoJAM directly adresses this without any extra data or scaling 👇🧵

Hila Chefer

168,802 次观看 • 1 年前

I vibecoded a new game this morning: the floor is lava ♨️ Save our cactus friend from pits of lava by jumping around your living room 🌵 Created with Three.js, Rosebud AI, and MediaPipe computer vision (body pose model). The game is free and live on the web now! Just need a regular webcam to play. Code is also open source ↴

I vibecoded a new game this morning: the floor is lava ♨️ Save our cactus friend from pits of lava by jumping around your living room 🌵 Created with Three.js, Rosebud AI, and MediaPipe computer vision (body pose model). The game is free and live on the web now! Just need a regular webcam to play. Code is also open source ↴

AA

47,670 次观看 • 1 年前

Here is a look at the new curtain bangs for our customizable 3D vtuber model! Being the first of it's kind, we want to go all out with the quality and details! 💜✨

Here is a look at the new curtain bangs for our customizable 3D vtuber model! Being the first of it's kind, we want to go all out with the quality and details! 💜✨

Pastell & Palette🤍 3D Customizable Model!

15,611 次观看 • 2 个月前

VIGGLE'S NEW MODEL IS HERE! Controllable video generation is our mission, and character is where we start. Animate any character just as you want, simply with a prompt, or by uploading a reference video featuring clear motion. Try Viggle for FREE:

VIGGLE'S NEW MODEL IS HERE! Controllable video generation is our mission, and character is where we start. Animate any character just as you want, simply with a prompt, or by uploading a reference video featuring clear motion. Try Viggle for FREE:

ViggleAI

37,520 次观看 • 2 年前

Avatar IV is here and it changes everything. The most advanced avatar model we’ve ever built. Upload one photo and a script. That’s it. Our new audio to expression engine captures your tone, rhythm, and emotion, then generates facial motion so real it feels alive. And it’s not just for humans. Pets. Aliens. Anything you can imagine brought to life in seconds. This isn’t animation. It’s expression. Drop your creations below. We can't wait to see what you dream up👇

HeyGen

4,400,674 次观看 • 1 年前

Your Enterprise Notebooks just got an upgrade. You can now use Google Cloud Workbench Instances directly from your favorite IDE with the new VS Code extension. With this new extension you can connect to and run notebooks on managed cloud environments directly within your local IDE. As part of our commitment to the developer ecosystem, the extension is fully open-sourced to support community-driven innovation. 🧵

Your Enterprise Notebooks just got an upgrade. You can now use Google Cloud Workbench Instances directly from your favorite IDE with the new VS Code extension. With this new extension you can connect to and run notebooks on managed cloud environments directly within your local IDE. As part of our commitment to the developer ecosystem, the extension is fully open-sourced to support community-driven innovation. 🧵

Google for Developers

33,113 次观看 • 1 个月前

Can we scale 4D pretraining to learn general space-time representations that reconstruct an object from a few views at any time to any view at any other time? Introducing 4D-LRM: a Large Space-Time Reconstruction Model that ... 🔹 Predicts 4D Gaussian primitives directly from multi-view tokens (no motion vectors, no HexPlane); 🔹 Uses a clean, minimal Transformer backbone; 🔹 Generalizes with fast, high-quality feedforward rendering at any view and infinite frame rate. Check out more interactive demos and scaling behaviors on our homepage/paper. 👉Website: 👉Paper:

Can we scale 4D pretraining to learn general space-time representations that reconstruct an object from a few views at any time to any view at any other time? Introducing 4D-LRM: a Large Space-Time Reconstruction Model that ... 🔹 Predicts 4D Gaussian primitives directly from multi-view tokens (no motion vectors, no HexPlane); 🔹 Uses a clean, minimal Transformer backbone; 🔹 Generalizes with fast, high-quality feedforward rendering at any view and infinite frame rate. Check out more interactive demos and scaling behaviors on our homepage/paper. 👉Website: 👉Paper:

Martin Ziqiao Ma

21,811 次观看 • 1 年前

EDIT MOTION IN VIDEOS!!! Quit prompting and start directing I've been shouting for YEARS about 3D as the control layer. Here it is, signs of life of our Universal Video Editor!!! The workflow is: take your video, capture with comic 4, edit with the motion editor, re-render with your favorite video to video model (e.g., Runway and Google Gemini have good ones) Here we have a fashion clip where we wanted our actress to high step and show a bit more pizzaz - but shoot day is over, and traditionally it would cost many thousands to reshoot. Watch the video to see how it works instead in cartwheel. cc Logan Kilpatrick Cristóbal Valenzuela 👀

EDIT MOTION IN VIDEOS!!! Quit prompting and start directing I've been shouting for YEARS about 3D as the control layer. Here it is, signs of life of our Universal Video Editor!!! The workflow is: take your video, capture with comic 4, edit with the motion editor, re-render with your favorite video to video model (e.g., Runway and Google Gemini have good ones) Here we have a fashion clip where we wanted our actress to high step and show a bit more pizzaz - but shoot day is over, and traditionally it would cost many thousands to reshoot. Watch the video to see how it works instead in cartwheel. cc Logan Kilpatrick Cristóbal Valenzuela 👀

Andrew Carr 🤸

11,622 次观看 • 2 个月前

Man from Iran has a warning for America Islam, “This religion is not compatible with Western values” “A lot of people call me an Islamophobe. My answer is, you're goddamn right. I'm an Islamophobe. I'm from Iran. 9 million of us escaped that brutal, brutal regime because of what that religion did to our country” “This religion is not compatible with Western values, and the irony is they know it themselves. They escaped the shithole. They were from hoping they could make a better version of it in the new land they came to, but they can't because those, those ideologies are just not compatible with what we stand for and what we cherish. Open your eyes around your new reality”

Man from Iran has a warning for America Islam, “This religion is not compatible with Western values” “A lot of people call me an Islamophobe. My answer is, you're goddamn right. I'm an Islamophobe. I'm from Iran. 9 million of us escaped that brutal, brutal regime because of what that religion did to our country” “This religion is not compatible with Western values, and the irony is they know it themselves. They escaped the shithole. They were from hoping they could make a better version of it in the new land they came to, but they can't because those, those ideologies are just not compatible with what we stand for and what we cherish. Open your eyes around your new reality”

Wall Street Apes

283,731 次观看 • 4 个月前

We previously shared our research on Layer Skip, an end-to-end solution for accelerating LLMs from researchers at Meta FAIR. It achieves this by executing a subset of an LLM’s layers and utilizing subsequent layers for verification and correction. We’re now releasing inference code and fine-tuned checkpoints for this work. Model weights on Hugging Face ➡️ More details ➡️ We hope that releasing this work will open up new areas of experimentation and innovative new research in optimization and interpretability.

We previously shared our research on Layer Skip, an end-to-end solution for accelerating LLMs from researchers at Meta FAIR. It achieves this by executing a subset of an LLM’s layers and utilizing subsequent layers for verification and correction. We’re now releasing inference code and fine-tuned checkpoints for this work. Model weights on Hugging Face ➡️ More details ➡️ We hope that releasing this work will open up new areas of experimentation and innovative new research in optimization and interpretability.

AI at Meta

156,618 次观看 • 1 年前

"Beautiful, isn't it?” Did you catch the message from Rodin? To our #Bayonetta fans around the world: it’s been far too long. 🔻Special website🔻 I’m Suzuki, Lead Designer at #IIJAN. A new contract has been forged— the collaboration with "The Gates of Hell" is back in motion. We are preparing a fresh arsenal of new items, alongside the return of our most requested pieces. Release details will be unveiled shortly. And to our devoted fans in Europe and South America: the wait is over. We can finally deliver these treasures to your doorstep. Thank you for your patience and for believing in IIJAN. We couldn’t have done this without your support. Stay tuned, and have a hell of a weekend. Suzuki IIJAN #Bayonetta #Bayonetta2 #TheGatesOfHell

"Beautiful, isn't it?” Did you catch the message from Rodin? To our #Bayonetta fans around the world: it’s been far too long. 🔻Special website🔻 I’m Suzuki, Lead Designer at #IIJAN. A new contract has been forged— the collaboration with "The Gates of Hell" is back in motion. We are preparing a fresh arsenal of new items, alongside the return of our most requested pieces. Release details will be unveiled shortly. And to our devoted fans in Europe and South America: the wait is over. We can finally deliver these treasures to your doorstep. Thank you for your patience and for believing in IIJAN. We couldn’t have done this without your support. Stay tuned, and have a hell of a weekend. Suzuki IIJAN #Bayonetta #Bayonetta2 #TheGatesOfHell

IIJAN_EN

37,287 次观看 • 4 个月前

Your data is NOT yours—and it’s a global crisis! 🚨 Every 39 seconds, a cyberattack occurs. That’s not just a stat, it’s a ticking time bomb targeting OUR sensitive information. From social media profiles to our bank details, corporations are hoarding, storing, and SELLING our data. These centralized giants have made us a product, leaving us exposed to breaches, hacks, and exploitation. Our digital identity isn’t just at risk, it’s being hijacked. This is the cost of centralized systems: zero transparency, zero ownership, and zero security. As a solution, we introduced Pangea’s self-sovereign identity. 🔒 Powered by next-gen cryptography, it’s your ticket to taking back control. Own. Protect. Liberate. With decentralized technology, your data is YOURS, secure, untouchable, and free from exploitation. This isn’t just about privacy, it’s about POWER. It’s about FREEDOM. The future belongs to those who dare to demand it. Don’t wait to be the next statistic. Reclaim what’s yours. The revolution is decentralized, and it’s already here. 🌐

Your data is NOT yours—and it’s a global crisis! 🚨 Every 39 seconds, a cyberattack occurs. That’s not just a stat, it’s a ticking time bomb targeting OUR sensitive information. From social media profiles to our bank details, corporations are hoarding, storing, and SELLING our data. These centralized giants have made us a product, leaving us exposed to breaches, hacks, and exploitation. Our digital identity isn’t just at risk, it’s being hijacked. This is the cost of centralized systems: zero transparency, zero ownership, and zero security. As a solution, we introduced Pangea’s self-sovereign identity. 🔒 Powered by next-gen cryptography, it’s your ticket to taking back control. Own. Protect. Liberate. With decentralized technology, your data is YOURS, secure, untouchable, and free from exploitation. This isn’t just about privacy, it’s about POWER. It’s about FREEDOM. The future belongs to those who dare to demand it. Don’t wait to be the next statistic. Reclaim what’s yours. The revolution is decentralized, and it’s already here. 🌐

Tonomy

18,135 次观看 • 1 年前

Parse text from any PDF in seconds and give it to Claude Code 📑🤖 LiteParse is our open-source, model-free document parser that lets you digitalize text from any document in seconds. This is especially useful for coding agents, which are great at reading plaintext files but terrible at reading traditional document formats (PDF, Office docs). We have a one-line installable skill that lets you plug LiteParse into Claude Code and 40+ other agents. Repo is here:

Parse text from any PDF in seconds and give it to Claude Code 📑🤖 LiteParse is our open-source, model-free document parser that lets you digitalize text from any document in seconds. This is especially useful for coding agents, which are great at reading plaintext files but terrible at reading traditional document formats (PDF, Office docs). We have a one-line installable skill that lets you plug LiteParse into Claude Code and 40+ other agents. Repo is here:

Jerry Liu

30,668 次观看 • 4 个月前

Dear friends, Today, I am so excited to share our Studio’s groundbreaking project, the Large Nature Model - the world’s first open-source, generative AI model focused on Nature. This model, built from an extensive, ethically sourced natural world dataset, is central to our ambitious DATALAND project. DATALAND is a unique museum and Web3 platform dedicated to data visualization and AI art. It’s a collaborative effort, bringing together pioneers in diverse fields including the arts, scientific researchers, institutional archives, and advanced technology under the artistic leadership of Refik Anadol Studio. Here at the World Economic Forum, we’re presenting “Living Archive: Nature,” an installation featuring the Large Nature Model’s early artistic experimentations based on the Large Nature Model, blending visuals, sound, and scent. These initial explorations will continue to evolve into large-scale artworks and exhibitions, starting with our first solo show next month at Serpentine in London! Please join our community at

Dear friends, Today, I am so excited to share our Studio’s groundbreaking project, the Large Nature Model - the world’s first open-source, generative AI model focused on Nature. This model, built from an extensive, ethically sourced natural world dataset, is central to our ambitious DATALAND project. DATALAND is a unique museum and Web3 platform dedicated to data visualization and AI art. It’s a collaborative effort, bringing together pioneers in diverse fields including the arts, scientific researchers, institutional archives, and advanced technology under the artistic leadership of Refik Anadol Studio. Here at the World Economic Forum, we’re presenting “Living Archive: Nature,” an installation featuring the Large Nature Model’s early artistic experimentations based on the Large Nature Model, blending visuals, sound, and scent. These initial explorations will continue to evolve into large-scale artworks and exhibitions, starting with our first solo show next month at Serpentine in London! Please join our community at

Refik Anadol

444,632 次观看 • 2 年前

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito

Matthias Niessner

74,763 次观看 • 1 年前

💬: what does it mean to be performing at times square on new year’s eve for you? 🐶: oh my god it is absolutely surreal to be here welcoming the new year with everybody - with my best friends. here in new york. thank you guys so much for tuning it and special shoutout to our fans, our fearnots. thank you so much we love you. it’s an honor to be here. yunjin will take any chance to call le sserafim her ‘best friends’ 🥹

💬: what does it mean to be performing at times square on new year’s eve for you? 🐶: oh my god it is absolutely surreal to be here welcoming the new year with everybody - with my best friends. here in new york. thank you guys so much for tuning it and special shoutout to our fans, our fearnots. thank you so much we love you. it’s an honor to be here. yunjin will take any chance to call le sserafim her ‘best friends’ 🥹

nat ☆彡

74,967 次观看 • 7 个月前

Our team has been BUSY this June. Here’s a quick recap of what’s new: • The Video Agent waitlist is open. Our Agent can turn a prompt, PDF, or raw footage into a fully edited video. • Product Placement is live. Create UGC-style videos from a product photo and a person in minutes. • Avatar IV got a major upgrade with better expressions, motion, and gesture control. Now on iOS. • AI Studio is here. Edit videos faster and easier right inside HeyGen. • We built a Twitter bot. Tag HeyGen Labs under an image and we’ll turn it into a video. Stay tuned. We’re just getting started 😏

HeyGen

20,147 次观看 • 1 年前

Something powerful has emerged from this NDP leadership race: hope. Across the country, our campaign is sparking energy, solidarity, and new possibilities. Hope is a rare and fragile thing — but it’s here. The NDP is roaring back to life with the organizing and ideas to meet this historic moment. Voting is now open. Cast your ballot today and join us in going all in on hope.

Something powerful has emerged from this NDP leadership race: hope. Across the country, our campaign is sparking energy, solidarity, and new possibilities. Hope is a rare and fragile thing — but it’s here. The NDP is roaring back to life with the organizing and ideas to meet this historic moment. Voting is now open. Cast your ballot today and join us in going all in on hope.

Avi Lewis

17,031 次观看 • 5 个月前

🐰: and our member sehunnie’s discharge is now just one month away… ah… right, i think (we) first talked about this back at smtown in january, and now it’s finally almost here… his comeback, sehunnie’s comeback… so, as soon as sehunnie is discharged, exo will start preparing a new album and come to japan again with new music and performances, so please look forward to it lots~

🐰: and our member sehunnie’s discharge is now just one month away… ah… right, i think (we) first talked about this back at smtown in january, and now it’s finally almost here… his comeback, sehunnie’s comeback… so, as soon as sehunnie is discharged, exo will start preparing a new album and come to japan again with new music and performances, so please look forward to it lots~

🌸 사랑둥이 귀염둥이 세훈이 🌸

111,199 次观看 • 1 年前