This also shows up in the representations learned by... the model. We plot the model’s representations of human and robot images. As pre-training is scaled up, the representation of humans and robots become more aligned: to a scaled-up model, human videos "look" like robot demos.show more

How can we leverage diverse human videos to improve... robot manipulation? Excited to introduce EgoVLA — a Vision-Language-Action model trained on egocentric human videos by explicitly modeling wrist & hand motion. We build a shared action space between humans and robots, enabling seamless transfer. With some robot demos, EgoVLA becomes a powerful, generalizable robot policy.show more

Super excited about Hydra-0 from Hongyu Li and team!... The key idea is to use flow as a shared visual interface across embodiments/objects for controllable video generation, allowing a single generalist world model to learn from human, handheld-gripper, and robot interaction data. My favorite result is the video below: start from a real video of a human doing the task (left), extract the desired object flow, and condition the model on that flow (right). The model then hallucinates a plausible robot motion that could produce the same object motion. Very cool glimpse of how a generalist world model can bridge human demonstrations and robot control.show more

Text-to-image diffusion transformer models learn to align text and... image representations as a byproduct of their conditional denoising task. By taking the dot product between the text and image representations of a DiT model (like Flux 2), you can create rich saliency maps.show more

🤯 This is what humanoid robots may look like... in the very near future. A human face. An exposed mechanical body. And AI controlling the machine. Android X looks less like a robot… and more like the first generation of something entirely new. 🤖 The uncanny valley is getting real.show more

how Smooth Is Human-Robot Table Tennis Match? Intelligent robots... play table tennis accurately, bringing fun human-machine competitions. The combination of sports and technology shows the infinite vitality of scientific innovation.#ICIF #Shenzhen #tennisshow more

Robots have developed the skill to learn by watching... videos: The Visual-Robotics Bridge (VRB) model can teach a robot a new task in roughly 25 minutes using only reference video - without needing human interaction or a simulated environment.show more

Came to the office and ended up training a... robot 🤖show more

TESLA’S NEXT-GEN OPTIMUS V3 HAND IS GETTING CLOSE TO... HUMAN-LEVEL Tesla engineers just shared new details on the Optimus V3 hand — and the progress is striking. They’re now saying that as they move into Gen-3 and mass production, the hand is getting very close to human functionality and form factor. One engineer described it clearly: “It won’t even look like a robot. It will look like a human in a superhero suit. It will be something revolutionary.” This level of dexterity and human-like design is a critical milestone. The hand is one of the hardest parts of building a truly useful humanoid robot — and Tesla is iterating extremely fast.show more

(1/2) MonoNPHM will be presented as a #CVPR2024 Highlight!... Our Neural Parametric Head Model parametrizes both geometry and appearance. With the learned model, we can then 3D reconstruct and track human heads from images or videos.show more

In the future, it might just be humanoid robots... doing the refueling!🤖⛽️ Sinopec plans to introduce humanoid robot workers across its gas stations. They will handle tasks like fueling, picking up items, restocking, and inspections,working alongside human staff, especially during peak holiday periods in China to ease service pressure. With more than 30,000 gas stations across the country (most of them equipped with convenience stores), this implies a deployment of at least 30,000 humanoid robots. The robot is a wheeled humanoid developed by Beijing-based company FIVEAGES.show more

So… the 1X NEO home robot is not actually... autonomous. Behind the scenes, it’ll often be teleoperated by humans, meaning someone, somewhere, could literally remote-control a robot inside your living room. I wanted the dawn of embodied AI. Instead, I’m apparently paying $499/month for a robot avatar with a human pilot. It’s impressive tech. But also… kind of dystopian? A robot that looks alive, yet secretly puppeteered, the uncanny valley just got a new basement level. Feels less like the “post-labor future,” and more like we just outsourced physical presence itself.show more

This isn’t a robot it’s a Dominican Papibot $tsla... Boston Dynamics just raised the bar again. The new Atlas moves with fluid, human-like coordination—then goes past human limits. Balance, recovery, and whole-body control are happening in real time, not pre-scripted demos. $nvdashow more

The context size of video world models is only... a few frames. Like a human with severe memory loss! We design a long-term memory for world models based on explicit 3D representations inspired by the human mind. This enables long-term consistency. 1/3show more

JUST IN: Reimagine Robotics has just emerged from stealth!... 🥷🏻 Its approach to robot training is one of the most human-centric I've seen. The founder is Jonathan Scholz, the person who built and led Google DeepMind's Applied Robotics team in London for seven years. This is not a first-time founder taking a swing at robotics. He has spent a decade at the frontier of the field. The philosophy is powerful. He calls it "monkey-see, monkey-do." 🐒 A worker shows the robot what to do. Watches it attempt the task. Corrects it on the spot. The robot learns. No specialist programmers. No months of integration. And it's already working in the real world: → A made-to-order plastics business trained robots to tend 3D printers overnight, removing print beds, operating latches, pressing controls → A hard drive disassembly facility built a three-robot cell combining robots and people to recover critical materials → Time to prototype and test a new robot behaviour reduced from one day to 10 MINUTES That last number is the one that changes everything. When testing a new behaviour takes 10 minutes instead of a day, the entire pace of deployment transforms. Scholz's framing of the human-robot relationship is worth reading carefully: "A robot that learns on the job depends on people. The worker identifies the bottleneck, shows the robot how to help, and corrects it until it is useful." It's August, and we keep getting robotics bangers week in week. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →show more

Turns out you can train humanoid hands without any... robot data. The idea in HUG is quite simple: (a) collect human data with smart glasses, (b) train a human manipulation model, (c) retarget to multi-fingered robot hands.show more

New startup out of stealth in Cambridge MA: Eka... Robotics. The company is building intelligence for the physical world in its native language: FORCE. Their core solution to mastering fast, reliable adaptive robots: Vision-Force-Action (VFA) model: - sim-only reinforcement learning - no human teleop data - robot practices thousands of hours in a physics-rich simulator (mass, inertia) and comes up with its own solutions, AlphaZero-style. - custom grippers add touch sensing; the model maps pixels + felt force to actions.show more

The Hidden Language of Diffusion Models paper page: tackle... the challenge of understanding concept representations in text-to-image models by decomposing an input text prompt into a small set of interpretable elements. This is achieved by learning a pseudo-token that is a sparse weighted combination of tokens from the model's vocabulary, with the objective of reconstructing the images generated for the given concept. Applied over the state-of-the-art Stable Diffusion model, this decomposition reveals non-trivial and surprising structures in the representations of concepts. For example, we find that some concepts such as "a president" or "a composer" are dominated by specific instances (e.g., "Obama", "Biden") and their interpolations. Other concepts, such as "happiness" combine associated terms that can be concrete ("family", "laughter") or abstract ("friendship", "emotion"). In addition to peering into the inner workings of Stable Diffusion, our method also enables applications such as single-image decomposition to tokens, bias detection and mitigation, and semantic image manipulationshow more

Afficianados of robot learning: join us in Hall B2... at #NeurIPS2023 for some cutting-edge talks, posters, a spicy debate, and live robot demos! The robots are here, are you? We also have some GPUs for a "Spicy Question of the Day Prize" 🌶️, don't miss outshow more

Your model can't zoom up as much as your... friends? Send this to your rigger❗️ - In the video, left side is adjusted exported param, right side is the default. - The left model has a more close up default state and can zoom up more than the right The export parameters that you should pay attention to - Center of model Y vertically affects the center of exported model - Canvas scale unit: How big the default state of your model is Personally I would recommend having - Center of model Y at 0.25 - 0.15 ( to focus on the face) - Canvas scale unit: 4.0 #live2d #live2dtip #vtubestudioshow more