D4RL is a great benchmark, but is saturated. Introducing... OGBench, a new benchmark for offline goal-conditioned RL and offline RL! Tasks include HumanoidMaze, Puzzle, Drawing, and more 🙂 Project page: GitHub: 🧵↓show more

Robotics keeps hitting the same wall. Single task RL... works, but... it does not scale to hundreds of tasks or new embodiments. This new paper looks like a real step toward fixing that. The team introduces MMBench, a benchmark with 200 tasks across many domains and robots, and Newt, a language conditioned world model trained online across all 200 tasks at once. The simple idea behind Newt: The model learns from demos to get the right priors It trains across many tasks through online interaction It uses language to ground the goal It adapts fast when a new task shows up What stood out to me: ✅ One model trained on 200 tasks at the same time ✅ Language conditioned control for both states and RGB ✅ Better data efficiency than strong baselines ✅ Strong open loop control ✅ Fast adaptation to new tasks and embodiments ✅ Full release of 200 checkpoints, 4000 demos, code, and benchmark This is a good push toward general control instead of one model per task. If you want the full paper: Project page: —- Weekly robotics and AI insights. Subscribe free:show more

RL is back! But is it always the best... choice? In a new paper, we investigate under what circumstances neuroevolution outperforms RL in transfer learning tasks. See more details in the thread below 🧵 While NE performs best in simpler domains, it will be interesting to see if the lessons learned here can also be applied to more complex systems/tasks (LLMs?).show more

Can MLLMs actually track what's happening in a video?... Introducing VSTAT 🎯, our new benchmark for visual state tracking. The tasks are simple: count cups, read typed words, count page flips. Humans solve them easily. MLLMs don't. 🧵 [1/11]show more

🧵🚀 New WebAgent Benchmark Alert! 🚀 There is hope... for human workers! We released WorkArena++, a new challenging benchmark for WebAgents. Our best agent achieves 0% accuracy on this benchmark, while human evaluators still obtain 94%! 🔗show more

🚨Current scalable RL algos train a policy w/o value... func, which is limiting with learning in open-ended, non-stationary, dynamic environments. But, how to scale value-based RL with more data/compute is unclear... Not anymore: presenting scaling laws for value-based RL 🧵⬇️show more

ManiFeel is a benchmark for contact-rich manipulation tasks. Force-... and tactile-based tasks with occlusions and partially-observable state are going to be crucial for true general purpose robotics. Website:show more

🤖 Another zero-shot reward model is now in LeRobot:... ROBOMETER. A general-purpose, zero-shot video-language reward model from University of South Carolina, UT Dallas, Massachusetts Institute of Technology (MIT), University of Washington, Ai2, and NVIDIA that predicts frame-level task progress. Trained on 1M+ trajectories from 21 robot embodiments, generalizes zero-shot to unseen tasks, scenes, and robots. 2.4–4.5x better downstream success rates across online RL, offline RL, data filtering, failure detection, and data retrieval for IL. Project: Paper:show more

Pantograph robot building with jenga blocks. Focusing on RL... is a great and fairly unique strategyshow more

We are finally releasing a new benchmark!! #RSS2023 Main... highlights are: * For real-world * Reproducible (3D printing) * 9 furniture assembly long-horizon tasks * 200+ hrs of demonstrations This apparently is my 3rd furniture-related benchmark.🥹show more

📢Announcing our 3D head avatar benchmark📢 Two tasks with... hidden test sets: - Dynamic Novel View Synthesis on Heads - Monocular FLAME-driven Head Avatar Reconstruction Our goal is to make research on 3D head avatars more comparable and ultimately increase the realism of digital humans. The benchmark studies distinct phenomena of 3D head avatar creation, such as extreme facial expressions, slow motion captures of shaking long hair, or complicated light reflection and refraction patterns of glasses. The two benchmark tasks assess two core desiderata of 3D avatars: While the novel view synthesis challenge focuses on best possible rendering quality of complex moving scenes, the avatar animation challenge is concerned with how well a driving signal is translated into an avatar. Evaluations are light-weight and consist of diverse video recordings from the popular NeRSemble dataset with a hidden test set. Participation in the benchmark is therefore straight-forward and requires only 5 reconstructions per task. Leaderboard and benchmark submission: Benchmark data access and toolkit: Great work by Tobias Kirschstein Simon Giebenhainshow more

new collab from Paradigm and OpenAI: evmbench is a... benchmark and agent harness for exploiting smart contract bugs a few months ago, the best models found <20% of critical, fund-draining @Code4rena bugs in our benchmark. today they find > 70%show more

New project! Flow Policy Gradients for Robot Control tldr;... a simple online RL recipe for training and fine-tuning flow policies for robots co-led w/ Hongsuk Benjamin Choi:show more

Is there a female equivalent to what "benching your... bodyweight" is as a baisc benchmark for men's fitness? Feel like a split is close but probably tougher to do. More akin to "benching 225"show more

they could break the internet by posting a selfie... together in this but it is offline man number one and twoshow more

An actress gives a great performance and somehow the... conversation circles back to Alia 😂✨ That's not coincidence that's impact She is the benchmark She is the standard. ❤️👑show more

Introducing CQN: Coarse-to-fine Q-Network, a value-based RL algorithm for... continuous control🦾Initialized with 20~50 demonstrations, it learns to solve real-world robotic tasks within 10 mins of training, without any pre-training and shaped rewards! (1/4)show more

Robots struggle with strict action rules…memory and symbols help... them learn fast. [Project + Full video link ⬇️] Robots struggle when tasks require specific steps in a fixed order. What if memory helped them think symbolically and learn faster? Solving tasks like unlocking a door then opening it is hard for deep RL. But by learning constraint relationships and storing them in memory, robots can solve these tasks much faster; with fewer trials and less training. Why it works ✅ Learns symbolic rules about action constraints ✅ Uses memory to transfer what it learned across tasks ✅ Handles real-world exploration with just 30 minutes of data ✅ Needs 10x fewer episodes than deep RL approaches This memory-based method shows a promising path forward for robots learning structured, real-world tasks. Full video: Paper: Thank you, Mrinal Verghese for sharing this amazing work! 🙏show more

Frontier research just crossed a new threshold. Mind Lab... is now public. And it begins with trillion-scale RL trained at only 10% of the usual compute.show more

We are investing in the frontiers of agentic capabilities... with a few early prototypes. Project Mariner is built with Gemini 2.0 and is able to understand and reason across information - pixels, text, code, images + forms - on your browser screen, and then uses that info to complete tasks for you. When evaluated against the WebVoyager benchmark, it achieved a state-of-the-art result of 83.5% working as a single agent setup.show more

ok Grok 4.5 is wild It just built me... this rocket tracking app with live data and a 3D globe 🚀 I might need a new benchmark after this 😭show more