Loading video...

Video Failed to Load

Go Home

🧵GEMINI + MEDIAPIPE = HANDPUPPET🖐✨ I built an experiment turning my hand into a puppet w/ Gemini 2.0 Flash & Mediapipe. By detecting when my hand is open + capturing webcam images in real-time, Gemini analyzes my environment and generates context-aware thoughts. 📸

wwwtinazone's profile picture

Tina Tarighian

1,661 subscribers

61,067 views • 1 year ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

i started making a new game the mission: control a robot with hand movements, smash some buildings, dodge some tanks created with threejs, rapier physics, mediapipe computer vision, gemini 3, my macbook webcam continuing my journey to resurrect wii-kinect-eyetoy AR games :)
0:21

i started making a new game the mission: control a robot with hand movements, smash some buildings, dodge some tanks created with threejs, rapier physics, mediapipe computer vision, gemini 3, my macbook webcam continuing my journey to resurrect wii-kinect-eyetoy AR games :)

AA

72,773 views • 8 months ago

I used Gemini 2.0 Flash real-time screen-sharing feature, and it is incredible check this out. It could analyze my code, terminal, and issues, then suggest bug fixes. Truly outstanding work from Google gemini team.
1:04

I used Gemini 2.0 Flash real-time screen-sharing feature, and it is incredible check this out. It could analyze my code, terminal, and issues, then suggest bug fixes. Truly outstanding work from Google gemini team.

AshutoshShrivastava

72,466 views • 1 year ago

Gemini 2.0 real-time AI is absolutely wild! Watch how I use it as an AI research assistant by sharing my screen and asking it about an AI paper. 10x your paper reading skills or just let Gemini summarize key points. What an incredible time to be alive!
3:31

Gemini 2.0 real-time AI is absolutely wild! Watch how I use it as an AI research assistant by sharing my screen and asking it about an AI paper. 10x your paper reading skills or just let Gemini summarize key points. What an incredible time to be alive!

elvis

260,229 views • 1 year ago

it's day 2 of making minecraft* in the browser *except it's way less convenient because the only way to control the game is by waving your hands around i added a GTA radial menu to choose between characters (all 3D models by quaternius) tech stack: mediapipe computer vision, threejs, gemini 3, my macbook webcam
0:23

it's day 2 of making minecraft* in the browser *except it's way less convenient because the only way to control the game is by waving your hands around i added a GTA radial menu to choose between characters (all 3D models by quaternius) tech stack: mediapipe computer vision, threejs, gemini 3, my macbook webcam

AA

160,016 views • 8 months ago

I love Hermes Agent and Gemini Live is also one of my for voice conversation, so over the weekend I built a small vibe-coding project that brings the two together called IRIS. The idea is simple: Gemini Live becomes the conversational interface, while Hermes Agent handles execution, automation, and long-running tasks behind the scenes. I can talk to Gemini naturally, and whenever something needs real work, Gemini formulates the request and hands it off to Hermes. The best part is that the conversation never stops. Gemini keeps talking with me naturally while Hermes works in the background. When Hermes finishes, Gemini proactively tells me the results are ready, switches context to the Hermes output, helps me review it, and then we return to our normal conversation. That manager/worker flow feels really good. Because I wanted the app to be as hands-free as possible, I also added a camera-based gesture layer using MediaPipe Gesture Recognizer. I can point at Hermes task cards, open details, scroll through results with my hand, and close them using gestures. Small weekend project, but I had a lot of fun building it. Also, the anti-flicker macOS app I vibe-coded yesterday ended up being incredibly useful for this project. It fixed my webcam flickering issue, so I can finally record videos properly. 🙂
1:29

I love Hermes Agent and Gemini Live is also one of my for voice conversation, so over the weekend I built a small vibe-coding project that brings the two together called IRIS. The idea is simple: Gemini Live becomes the conversational interface, while Hermes Agent handles execution, automation, and long-running tasks behind the scenes. I can talk to Gemini naturally, and whenever something needs real work, Gemini formulates the request and hands it off to Hermes. The best part is that the conversation never stops. Gemini keeps talking with me naturally while Hermes works in the background. When Hermes finishes, Gemini proactively tells me the results are ready, switches context to the Hermes output, helps me review it, and then we return to our normal conversation. That manager/worker flow feels really good. Because I wanted the app to be as hands-free as possible, I also added a camera-based gesture layer using MediaPipe Gesture Recognizer. I can point at Hermes task cards, open details, scroll through results with my hand, and close them using gestures. Small weekend project, but I had a lot of fun building it. Also, the anti-flicker macOS app I vibe-coded yesterday ended up being incredibly useful for this project. It fixed my webcam flickering issue, so I can finally record videos properly. 🙂

AshutoshShrivastava

149,739 views • 2 months ago

Update on my hand-tracking pottery app 🏺 New features just dropped: - Colorize your clay in real time - Change the shape of the clay base Got a webcam? Use your hands. Don't have one? Your mouse works just as well. Everything built within Omma + threeJs + mediaPipe Demo link in the comments 👇
0:35

Update on my hand-tracking pottery app 🏺 New features just dropped: - Colorize your clay in real time - Change the shape of the clay base Got a webcam? Use your hands. Don't have one? Your mouse works just as well. Everything built within Omma + threeJs + mediaPipe Demo link in the comments 👇

Joseph Azar

13,973 views • 2 months ago

Google Gemini 2.0 realtime AI is insane. Watch me turn it into a live code tutor just by sharing my screen and talking to it. We’re living in future. I’m speechless.
5:37

Google Gemini 2.0 realtime AI is insane. Watch me turn it into a live code tutor just by sharing my screen and talking to it. We’re living in future. I’m speechless.

Mckay Wrigley

624,926 views • 1 year ago

Today I created Tetris: but it has an unnecessary learning curve and makes my arm really tired 🟦🥵 Built with Three.js and MediaPipe hand-tracking. This runs on the web and uses a regular laptop / webcam. Let me know if you want to play, and I'll fight some bugs and ship! ⛴️
0:44

Today I created Tetris: but it has an unnecessary learning curve and makes my arm really tired 🟦🥵 Built with Three.js and MediaPipe hand-tracking. This runs on the web and uses a regular laptop / webcam. Let me know if you want to play, and I'll fight some bugs and ship! ⛴️

AA

20,114 views • 1 year ago

i made a drum machine that turns hand gestures into drum loops and helps me create backing tracks while i'm playing piano it runs in the browser using my webcam; no special software or hardware created with Three.js, mediapipe computer vision, Kimi.ai k2.5 code, and some free drum samples
0:44

i made a drum machine that turns hand gestures into drum loops and helps me create backing tracks while i'm playing piano it runs in the browser using my webcam; no special software or hardware created with Three.js, mediapipe computer vision, Kimi.ai k2.5 code, and some free drum samples

AA

18,605 views • 6 months ago

In India, March is synonymous with exam season, and so I built a Study Assistant for my kid—thanks to Gradio and Gemini 2.0 Flash. Why Gradio? • No complex UI coding! With just a few lines of Python, I could create an interactive chatbot that offers features like chat history, image uploads (for questions about pictures, charts, and diagrams), and a notepad for quick study notes. Powered by Gemini 2.0 Flash • Multilingual support - for Hindi • Real-time Google Search Integration for current affairs and fact-checking. • Kid-friendly—it filters out distractions like gaming and social media topics. Will share the code along with a blogpost soon. AK 🍉 Abubakar Abid Logan Kilpatrick
0:47

In India, March is synonymous with exam season, and so I built a Study Assistant for my kid—thanks to Gradio and Gemini 2.0 Flash. Why Gradio? • No complex UI coding! With just a few lines of Python, I could create an interactive chatbot that offers features like chat history, image uploads (for questions about pictures, charts, and diagrams), and a notepad for quick study notes. Powered by Gemini 2.0 Flash • Multilingual support - for Hindi • Real-time Google Search Integration for current affairs and fact-checking. • Kid-friendly—it filters out distractions like gaming and social media topics. Will share the code along with a blogpost soon. AK 🍉 Abubakar Abid Logan Kilpatrick

Parul Pandey

10,795 views • 1 year ago

Learning from Human Demonstrations: Show the Robot How to Act! The pipeline is very similar to older experiments using Gemini & pi0 with LeRobot. Pi-zero runs locally, while Gemini Flash generates the affordances and the high-level task. (More details are in the thread.) The new component is learning from demonstrations via Gemini 2.5 Pro. I capture a video while demoing & take one of the last frames. Gemini 2.5 Pro then extracts the instructions & passes them to Gemini Flash to process the scene. The fun part is that there's no fancy insight that came from me; other than the days spent figuring out the right prompts. It's the bitter lesson hitting you in the face -> Enhanced Gemini capabilities make this possible. For example, Gemini Flash cannot do Russian doll stacking, but Gemini 2.5 Pro can do it consistently. The current limitation is low-level manipulation: - As you can see, I'm aligning the objects so they are easy to grasp using the same technique from the training data. I couldn't get Gemini Flash to consistently output an accurate grasping angle, and Gemini 1.5 Pro was too expensive and slow for real-time deployment. - Getting a symmetrical gripper should also help a lot. Adding rubber to the tips would probably also help prevent objects from slipping. Collecting & curating the data was the most time consuming & labor intensive part. Next, to improve low-level manipulation and make the system more real-time, I'm shifting to focus more on sims & synthetic data. This aligns better with my core competence. I'm open to tips and suggestions.
3:35

Learning from Human Demonstrations: Show the Robot How to Act! The pipeline is very similar to older experiments using Gemini & pi0 with LeRobot. Pi-zero runs locally, while Gemini Flash generates the affordances and the high-level task. (More details are in the thread.) The new component is learning from demonstrations via Gemini 2.5 Pro. I capture a video while demoing & take one of the last frames. Gemini 2.5 Pro then extracts the instructions & passes them to Gemini Flash to process the scene. The fun part is that there's no fancy insight that came from me; other than the days spent figuring out the right prompts. It's the bitter lesson hitting you in the face -> Enhanced Gemini capabilities make this possible. For example, Gemini Flash cannot do Russian doll stacking, but Gemini 2.5 Pro can do it consistently. The current limitation is low-level manipulation: - As you can see, I'm aligning the objects so they are easy to grasp using the same technique from the training data. I couldn't get Gemini Flash to consistently output an accurate grasping angle, and Gemini 1.5 Pro was too expensive and slow for real-time deployment. - Getting a symmetrical gripper should also help a lot. Adding rubber to the tips would probably also help prevent objects from slipping. Collecting & curating the data was the most time consuming & labor intensive part. Next, to improve low-level manipulation and make the system more real-time, I'm shifting to focus more on sims & synthetic data. This aligns better with my core competence. I'm open to tips and suggestions.

Shreyas Gite

22,555 views • 1 year ago

I think people are still sleeping on realtime voice driven consumer experiences.. Releasing my latest side project geared towards kids: Upon A Voice ✨ You can watch my daughter interact with it in the video below but the highlights are as follows: - generates a "live" storybook based on who your child is and how they look. - the story is completely driven by voice and what the kid wants to see in the book. - images are generated in real time keeping the character consistent. - at the end, you get a full PDF that you can print and hand to your child to read as a bedtime story Live now at uponavoice dot com with a BYO Gemini key!
2:00

I think people are still sleeping on realtime voice driven consumer experiences.. Releasing my latest side project geared towards kids: Upon A Voice ✨ You can watch my daughter interact with it in the video below but the highlights are as follows: - generates a "live" storybook based on who your child is and how they look. - the story is completely driven by voice and what the kid wants to see in the book. - images are generated in real time keeping the character consistent. - at the end, you get a full PDF that you can print and hand to your child to read as a bedtime story Live now at uponavoice dot com with a BYO Gemini key!

Nikunj Kothari

27,953 views • 1 month ago

“Because that’s when i stabbed your Brother in the chest.” Can’t get this scene out of my head. Dexter being so real & intrigued in Gemini. The duality they share with the way Brian is worked into the moment Resurrection is just so fucking good man.
3:02

“Because that’s when i stabbed your Brother in the chest.” Can’t get this scene out of my head. Dexter being so real & intrigued in Gemini. The duality they share with the way Brian is worked into the moment Resurrection is just so fucking good man.

Craig

20,604 views • 2 months ago

(1/4) Gemini 3.5 Flash is in a league of it's own! ⚡️ It's the perfect combo of intelligence, speed, & cost. It's now my daily driver in both Spark & Antigravity! Watch 3.5 spawn subagents organize a set of marketing assets, rename them, and put them into folders
0:51

(1/4) Gemini 3.5 Flash is in a league of it's own! ⚡️ It's the perfect combo of intelligence, speed, & cost. It's now my daily driver in both Spark & Antigravity! Watch 3.5 spawn subagents organize a set of marketing assets, rename them, and put them into folders

Tulsee Doshi

43,846 views • 3 months ago