Loading video...

Video Failed to Load

Go Home

a robot can recognize every object in a room and still not know how to open a single drawer knowing there's a cabinet is scene understanding. knowing which handle to grab, which direction to pull, and that it's a drawer not a door is affordance understanding almost no dataset...

18,800 views • 3 months ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Dr Fei-Fei-Li explains with a simple example how everyday household chores are so extremely difficult for Robots. "If you tell a robot to open the top drawer and watch out for the vase, this is actually a really hard task for robots." because the robot must ground language into the real world. Words like "top", "drawer", and "vase" are abstract. The system has to map them to 3D locations, objects, and relations in a noisy scene. This requires robust perception, object recognition, and spatial reasoning under uncertainty. The robot also lacks human commonsense. "Watch out" implies predicting consequences, estimating clearances, and understanding that vases are fragile. Encoding such priors, like how heavy a drawer is or how a vase might tip, is very complex and difficult without rich world knowledge. Learning the behavior from rewards is tough. The success signal is very sparse here, so naive exploration almost never stumbles on a full success sequence. This makes policy learning sample inefficient and brittle, especially when the environment changes between training and deployment. A sparse reward situation is when the agent only gets a success signal at the very end, and gets little or no feedback along the way. If a robot must open a drawer without hitting a vase, it might get reward only if the drawer ends up open and the vase is intact. Every partial try before that looks the same to the learner, reward equals 0. --- From "DSAI by Dr. Osbert Tay" YT channel

Rohan Paul

342,627 views • 11 months ago

is live on the Solana dApp Store. Your Solana Seeker is now a 3D agent studio. MAKE SOMETHING IN 3D A selfie becomes a rigged, textured, animation-ready character in about a minute. Or describe it in text. Or turn any photo into a model. Or upload a GLB you already have. Then restyle its materials, change its clothes, check its rig and fix it. MAKE IT MOVE A clip library that retargets onto any humanoid skeleton. Mocap from your camera. Lipsync from text or a live mic. Gestures. Sign language. A choreographer for full routines. And motion swap, which drops your avatar into video you already shot. GIVE IT A MIND A personality, a voice, skills, and a choice of models behind it. Then talk to it. It answers in 3D, out loud or in text, in your language. PUT IT ANYWHERE Every agent has a URL and a one-line embed, so it drops into a site, a chat, or a widget. Place it in your room in AR. Walk it across any web page. Turn a URL into a 3D world it can stand in. Render it out as video. OWN IT Deploy it on-chain as a Metaplex Core asset your wallet holds. Open manifest, no lock-in. Give it an on-chain identity, a name, and a wallet with recovery and inheritance. Sell its skills. Take tips. Trade the agent itself in USDC. LET IT EARN Pay-per-call, so other agents can buy what yours does and yours can buy what it needs. A skills marketplace, a labor market, vaults that let people back an agent, and a live feed of agents paying each other. Launch a coin around it if you want one. FIND EVERYONE ELSE A marketplace of what the community built: orbit any agent, inspect it, fork it, talk to it. Crews, rankings, tournaments, leaderboards. And build on it: SDKs, an MCP server, a CLI, an API, and the whole platform open source. AND THE PART THAT NEEDED THIS PHONE Every signature routes to Seed Vault through Mobile Wallet Adapter. Signing in is one tap, and the private key never enters the app process. Creating, chatting and browsing are free. Search in the dApp Store on your Seeker.

three.ws

11,970 views • 1 month ago

I kept seeing #GPT-6 Astra modelling results on here and got curious enough to build the thing myself: an agentic pipeline that turns a plain RGB video into 3D assets. The living room for robots MIT CSAIL. Input: one handheld phone walkthrough of our lab kitchen (~20s). Nothing else — no depth sensor, no CAD, no asset library. ~1 day (I actually slept overnight), including human-in-the-loop. Live: It is still not perfect — thin and shiny things are still weak, a few objects are drafts, the room shell needs another pass. The loop, roughly: - Monocular video → metric scan (ViPE): camera poses + depth. This is the measuring instrument, not the output. - GPT-6 lists what should exist as separate objects, then open-vocabulary detection + tracking gives per-object masks; each object is fused and measured in metres. - For every asset, GPT-6 works in its own sandbox with tools: it writes the object as a program in a small Blender DSL (closed solids, PBR colours, hinges/drawers), builds it, renders it over the original video frames, compares it with the scan points in 3D, and iterates. - A separate GPT-6 session is the verifier — it can render the model on any frame it wants and must justify every complaint with a frame. The modeller never grades its own work. - A completeness pass renders the whole modelled scene from the video's own cameras, puts it next to the real frames, and says what's still missing. What the detector keeps missing (a row of identical cabinets) gets placed geometrically instead. - Out comes MJCF/URDF with joints. Now, adding simulation to this env.

Zhiyang (Frank) Dou

11,367 views • 1 month ago