Matthieu LC's banner

Matthieu LC

@matthieulc • 1,119 subscribers

robotics @gen_intuition / built typeless, acq by @Doctolib

Shorts

worked on having gpt4o drive open-loop motor primitives like waving or grabbing from stereo vision. special tokens like get streamed straight into the transcript stream, which gpt4o reacts too. next up is integrating the closed-loop hand tracking. main challenge there is making policy orchestration smooth.

worked on having gpt4o drive open-loop motor primitives like waving or grabbing from stereo vision. special tokens like get streamed straight into the transcript stream, which gpt4o reacts too. next up is integrating the closed-loop hand tracking. main challenge there is making policy orchestration smooth.

2,481,765 views

hi all, excited to join! i'm building an expressive mini "shoggoth" robot which will eventually be hooked up to gpt4o realtime voice. i'm currently working on the low-level policies, which are trained in a mujoco simulation with RL. to delay working on raw-pixels for now, i trained a pose-estimation model using deeplabcut and triangulate the position in 3d space using the stereo cameras. eventually, i'll use gpt4o's tool calling capabilities to activate several of these policies (closed and open loop) based on the dialog flow! captions: manual actuation of the tentacle / 3d pose estimation / target design

hi all, excited to join! i'm building an expressive mini "shoggoth" robot which will eventually be hooked up to gpt4o realtime voice. i'm currently working on the low-level policies, which are trained in a mujoco simulation with RL. to delay working on raw-pixels for now, i trained a pose-estimation model using deeplabcut and triangulate the position in 3d space using the stereo cameras. eventually, i'll use gpt4o's tool calling capabilities to activate several of these policies (closed and open loop) based on the dialog flow! captions: manual actuation of the tentacle / 3d pose estimation / target design

45,506 views

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Releasing Shoggoth Mini!🐙 Soft tentacle robot meets GPT-4o & RL. I built it to explore the boundaries of weird: expressiveness, aliveness, and AI embodiment. Blogpost:

Releasing Shoggoth Mini!🐙 Soft tentacle robot meets GPT-4o & RL. I built it to explore the boundaries of weird: expressiveness, aliveness, and AI embodiment. Blogpost:

43,302 views • 10 months ago

hi all, excited to join! i'm building an expressive mini "shoggoth" robot which will eventually be hooked up to gpt4o realtime voice. i'm currently working on the low-level policies, which are trained in a mujoco simulation with RL. to delay working on raw-pixels for now, i trained a pose-estimation model using deeplabcut and triangulate the position in 3d space using the stereo cameras. eventually, i'll use gpt4o's tool calling capabilities to activate several of these policies (closed and open loop) based on the dialog flow! captions: manual actuation of the tentacle / 3d pose estimation / target design

hi all, excited to join! i'm building an expressive mini "shoggoth" robot which will eventually be hooked up to gpt4o realtime voice. i'm currently working on the low-level policies, which are trained in a mujoco simulation with RL. to delay working on raw-pixels for now, i trained a pose-estimation model using deeplabcut and triangulate the position in 3d space using the stereo cameras. eventually, i'll use gpt4o's tool calling capabilities to activate several of these policies (closed and open loop) based on the dialog flow! captions: manual actuation of the tentacle / 3d pose estimation / target design

45,506 views • 1 year ago

in dexterity tests, shoggoth mini reveals strong passive grasping generalization

in dexterity tests, shoggoth mini reveals strong passive grasping generalization

10,888 views • 11 months ago

No more content to load