Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than...

52,308 Aufrufe • vor 2 Jahren •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

The term "continual learning" has become overloaded if you see it as an ML problem. One classic thread is about memorization: regularization-based continual learning methods, such as EWC, MAS, and SI, estimate which parameters mattered for previous tasks and resist changing them too much. One modern thread is about adaptation: test-time training and inference-time learning methods, such as TTT, adapt part of the model on the incoming test stream before making predictions. These are sometimes discussed as separate threads. But in modern scalable architectures, I think they are better seen as complementary constraints: a model that learns quickly at test time also benefits from a mechanism for deciding what not to forget. In our #ECCV2026 paper, we study this in large-scale 4D reconstruction: how to build fast spatial memory that can adapt over long observation streams while reducing collapse and forgetting. Instead of using fully plastic test-time updates, we stabilize fast-weight adaptation with an elastic prior that balances adaptation and memory. Key ideas: - Elastic Test-Time Training: Fisher-weighted consolidation for fast-weight updates - EMA anchor weights that provide a moving reference for stability - Chunk-by-chunk inference for long 3D/4D observation streams We show that this scales across large 3D/4D pretraining settings, including both LRM-style and LVSM-style models, and improves reconstruction across benchmarks including Stereo4D, NVIDIA, and DL3DV-140. We release model checkpoints across different design choices: resolution, post-training curriculum, and whether the model uses an explicit 4DGS intermediate representation. - Homepage: - Paper: - Code: - Models: This work is co-led with Xueyang Yu, contributed by Haoyu Zhen Yuncong Yang, and advised by Michigan SLED Lab Chuang Gan.

Martin Ziqiao Ma

33,940 Aufrufe • vor 2 Monaten

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.

Robora

23,522 Aufrufe • vor 11 Monaten

A viral paper "Language Model Represents Space and Time" recently claims that LLMs learn "world models". As much as I like Max Tegmark's works, I disagree with their definition of world model. World model is a core concept in AI agent and decision making. It is our mental simulation of how the world works given interventions (or lack thereof). A world model captures causality and intuitive physics, telling the agent what is likely and what is impossible. It can and should be used for counterfactual reasoning, i.e. "what ifs": what would happen if I knock over a cup of water? Where would I have been if I had not taken that bus? Yann LeCun Yann LeCun says it well in his position paper ( I quote: "Using such world models, animals can learn new skills with very few trials. They can predict the consequences of their actions, they can reason, plan, explore, and imagine new solutions to problems. Importantly, they can also avoid making dangerous mistakes when facing an unknown situation." The first use of the term World Model in deep policy learning is attributed to hardmaru & Jürgen Schmidhuber: In their seminal paper, an agent masters shooting skills in the popular game Doom (demo below) by learning in imagination, using an internal world model as a "physics simulator". To put in a simple Python math formula, world model learns a function F(s[0:t-1], a) -> s[t:], which takes as input the observed past and current action, and outputs plausible future states. Now the definition of World Model in Tegmark's paper seems to be about predicting GPS coordinates and time eras. I see this as just a classification task with no causal learning and simulation going on. You cannot make meaningful interventions against that model, nor can you optimize any decision making in a closed feedback loop. As for the "space & time neurons", I think they are most similar to the "sentiment neuron" that OpenAI published in 2017: Predicting GPS is conceptually no different from predicting sentiment in my opinion. I don't think their experimental results are wrong - just that their conclusion is on shaky grounds. I welcome any debate! Paper link:

Jim Fan

594,014 Aufrufe • vor 2 Jahren

We are at NeurIPS Conference for our 4th #MyoChallenge and 7th #MyoSymposium! What started as a discussion with Vittorio Caggiano is a global community now MyoSuite 💪 In 2022, we started with two key hypotheses - 1⃣𝑺𝒄𝒂𝒍𝒊𝒏𝒈 𝑯𝒚𝒑𝒐𝒕𝒉𝒆𝒔𝒊𝒔: can we scale data driven learnings to achieve human level motor control? 2⃣𝑬𝒎𝒃𝒐𝒅𝒊𝒎𝒆𝒏𝒕 𝑯𝒚𝒑𝒐𝒕𝒉𝒆𝒔𝒊𝒔: Akin to Neurons's inspiration behind NN, are there embodied priors that will form the critical substrate to get to human performance? After 3 years, both these hypotheses are running strong. But in different ways than we anticipated -scaling hypothesis predicted vanilla RL algorithms (developed over OpenAI Gym and robotics tasks) will scale & realize human level motor control. RL did scale with better simulation & compute infrastructure but the curse of dimensionality became the limiter for high dimensional MSK systems. This is where our 2nd Embodiment Hypothesis kicked in. Spatial as well as morphological embodied priors facilitated development of next generation of algorithms at the intersection of representation and reinforcement learning - (DepRL from Pierre Schumacher et al, MyoDex & SAR from Cameron Vittorio Caggiano et al, Kinesis from Alberto Chiappa, muscleVAE from Yusen et al, etc) Current #MyoChallenge result evaluations phase was humbling realization - what started with a team of two evolved as a global community, the entry barriers has been lowered enough for even high school students, and underrepresented groups with limited resources to participate. It's incredible to realize the progress we have seen in 3 years. All the same time idiosyncrasies of the behaviors leave us quite unsatisfied and wanting more. Ahead of us there are exciting challenges on all frontiers -- embodiment, proprioceptive+exteroceptive sensing and control, validation -- presenting large real world potentials in health, wellness, sports, robotics. While there is a lot for us to be proud of, open challenges in understanding, as well as emulating human level motor intelligence remains. Join MyoSuite team for the awaited #MyoSymposium in discussing these frontiers on Saturday, Dec. 6th from 8-11 am: Ballroom 6D.

Vikash Kumar

11,145 Aufrufe • vor 9 Monaten

A Letter to Our Community: The Road Ahead for Robotics To our Community and Partners, As we step into 2026, our mission at Axis is clearer than ever: Constructing the definitive End-to-End Scaling Layer for Robotics. Our goal is to accelerate the transfer of diverse human intelligence into Robotics General Intelligence (RGI). By owning the critical path of intelligence creation, we are turning the physical limitations of robotics into a scalable, software-driven future. Here is our strategic outlook and roadmap for the year ahead. The Core Thesis: Simulation is the Only Way Out The path to RGI is currently blocked by Data Scarcity, Generalization Fragility, and Hardware Fragmentation. At Axis, we believe Simulation is the only way out. Our Simulation Data Platform and Data Augmentation Engine transform raw data into "Synthetic Gold". Backed by academic milestones like Roboverse, Skill Blending, and GraspVLA, we have proven that pure simulation can achieve the generalization required for the real world. We don’t just collect data; we architect it. The Engine: Why Crypto? We believe RGI should come from all, not a few. Crypto is not just a feature; it is the primitive that powers our entire ecosystem flywheel: - Incentive Mechanism: Democratizing contribution and rewarding the trainers and developers. - Assetization: Turning proprietary data and refined models into liquid, ownable assets. - Verifiable Workflow: We are opening the "Black Box" of AI. By bringing total transparency to the Task Generation → Data Collection → Model Training pipeline, we ensure every byte of intelligence is verifiable, traceable, and secure. 2026 Strategic Deliverables This year, we are committed to delivering three foundational pillars: - The World's Largest Training Dataset for Robots: A robot training set—diverse, high-quality interaction data at an unprecedented scale. - A Robotics Foundation Model: A universal robotic brain trained on our pure simulation and synthetic data, capable of robust cross-embodiment transfer and open-world adaptability. - Evolvable Robot Hardware: Robots deployed with Axis models that autonomously evolve through continuous interaction, turning every deployment into a self-improving node within our RGI network. The Ultimate Vision We are building more than models; we are architecting the Distributed Machine Economy. A future where every dataset, model, and robotic embodiment is a verifiable asset in a global, autonomous network. Thank you for building the future of intelligence with us✌️📷

Axis Robotics

28,096 Aufrufe • vor 8 Monaten

AI has had exactly two scaling axes that worked so far, and the second one is starting to look finite too the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before you’re just polishing RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and that’s much closer to actual RL and it’s what bought us planning and reasoning (arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing) so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- it’s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet... and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inference

Sasha Malysheva

14,445 Aufrufe • vor 1 Monat

Depth Any Video with Scalable Synthetic Data AI physicists and chemists continue to make strides in depth estimation from video. Check out this new paper featuring some impressive examples. See the thread for more details (unfortunately no code yet). Abstract: Video depth estimation has long been hindered by the scarcity of consistent and scalable ground truth data, leading to inconsistent and unreliable results. In this paper, we introduce Depth Any Video, a model that tackles the challenge through two key innovations. First, we develop a scalable synthetic data pipeline, capturing real-time video depth data from diverse game environments, yielding 40,000 video clips of 5-second duration, each with precise depth annotations. Second, we leverage the powerful priors of generative video diffusion models to handle real-world videos effectively, integrating advanced techniques such as rotary position encoding and flow matching to further enhance flexibility and efficiency. Unlike previous models, which are limited to fixed-length video sequences, our approach introduces a novel mixed-duration training strategy that handles videos of varying lengths and performs robustly across different frame rates 0 - even on single frames. At inference, we propose a depth interpolation method that enables our model to infer high-resolution video depth across sequences of up to 150 frames. Our model outperforms all previous generative depth models in terms of spatial accuracy and temporal consistency.

MrNeRF

27,428 Aufrufe • vor 1 Jahr

OFFICIAL: It has been 134 days since the release of Totenreich, which is longer than the search for both Samantha's Drawing, and Gong COMBINED. With the release of s6 next week, it felt fitting to use our platform to get as many eyes on it as possible. As a quick disclaimer, just because things have been 'tested' does not mean it is not the solve. This post is also not an invitation to share your personal thoughts on datamining/leaks. This relic, by everything shown in the game files, can NOT be datamined: "Even in the current state of the game, relics are a lot harder to solve via datamining and ingame hack tools... For relics, only the sequential item ones (like Dancing Arnie and Agarthan device) are incredibly easy. Agarthan device with the helmet, radio, and mr peeks is actually the easiest. Dancing Arnie becomes hard due to it becoming a kill challenge for the lobster and you cant datamine that". Because of that, the leading theory is that the missing piece has to be something kill condition related, or player action related. Below is literally all of the information I could find on the belt that I thought was worth mentioning, to try and help the community solve this together. CONFIRMED: There is a magazine with a sticker of Mr Peeks holding a Geiger Counter that has a pickup interaction, and is located at the top of the lighthouse while you are picking up the Jotunn Star. This only appears in cursed mode, and appears without any relics active. Considering the only other grim relic (power switch) is solved, we know this mag fits in with the wrestlers belt in some way. The sticker comes directly from intel located on Paradox Junction. THEORIES: Treyarch Dev Tweet - One Treyarch dev tweeted out: "It feels very fitting this is the final unsolved relic given the reward... good luck" Coda Wallbuy Sounds - In a video, a user hears a distinct ding sound when buying ammo off of the coda wall buy in spawn. This has been difficult to replicate, but has been replicated by the original user. Disappearing Boxes - There are 7 boxes around the map that have a hit box, and disappear when hit/shot (they disappear fast with melee macchiato). Currently it is believed by the majority of the community that this is nothing Clocks - There are five clocks near Richtofen's office. Dataminers have said "The hour, minute and second hands are all separate individual models stacked on top of the base model. lol I mean it could mean something or it could just be nothing. Their design choices on some stuff is weird But it is a model without a script and isn’t setup to move in any way" Swordfish - The mag located at the top of the lighthouse is the same model of a mag from the Swordfish A1. Some people believe that is a hint for the swordfish being needed from the box in order to complete the relic. Here are links to a spreadsheet managed by one community member that has things that community members have tested directly. Feel free to look through it, and see if you have an idea that has not already been tested: If you are interested in hunting this with other community members, please feel free to join the Mystery and ZoneX discords, as many of the community is hunting throughout there regularly. You can also feel free to comment under this post anything you might have found interesting.

COD: Zombies News

45,414 Aufrufe • vor 5 Tagen

Dear Tarun Chitra 1. We are the original creators of DeSci back in 2016. What DeSci has become today is largely unrelated with its original model of producing rigorous peer-reviewed scientific studies published in reputable medical journals. The model we introduced. We are tirelessly fighting against pseudoscience, and we are showing the world that people can understand the difference between legit science and pseudoscience with the success of $INNBCV. Yes, meritocracy is possible in crypto. Even against all odds. 2. We are the only project in the entire crypto space that ever funded, performed, and published highly innovative HIV cure research ( We are the project that produced the first peer-reviewed study on blockchain-based biomedical data storage in the world’s most reputable scientific network, Springer Nature ( $INNBCV is not for the privileged few; it is for the many. We resisted all the pressure from those who wanted us to provide big allocations to VIPs of other DAOs “because it is good for the marketing” and put our users first, ensuring a fair launch, a launch for the people, and they turned $70k into $2,000,000. $INNBCV shows that you can have a sustainable model, provided you are backed by actual science. And thanks to the amazing guys at daos.fun baoskee and Solana community. Behind our project there is the sweat and blood of years of work to produce publications in the most reputable medical journals. Just to put things into perspective, it took us 3 years to publish our latest work in Springer Nature. 3. Unlike many other projects, we had no ICO/VCs, meaning we had to prove ourselves every single day because we are only supported by our community. If we deliver products, we survive; it is either publish or perish for us, and that’s why we have such a close connection to our community. $INNBCV is a struggler, $INNBCV is a survivor, $INNBCV is not for the privilege of the few but for the people. Our community makes it possible by supporting us. You guys are the real heroes.

InnovativeBioresearch🇮🇹

10,867 Aufrufe • vor 1 Jahr

$IREN "we haven't disclosed the specific amount of GPUs" 1. 🤮 reminds me of $NBIS 2. Setting a terrible precedent here for future deals 3. Making it purposely difficult, to not let analysts properly value your 2027 revenue 4. Increasing the polarized view on IREN by the market However: "approximately 60MW of air-cooled Blackwells" 1. You typically don't talk about gross capacity in a deployment like this 2. If it would be gross capacity, the GPU hour rate at IT level would be crazy high (at PUE 1.2, $680m / 50 = 13.6m/MW) 3. At 60MW IT load, and ~14kW draw at DGX server level, we can get to ~4,286 DGX systems with 8 GPUs per. 4. Based on this we can conclude that 60MW of IT load can run approximately 34k DGX B300. 5. 34k DGX B300 at $680m/yr, would represent a GPU hour price of $2.28 Now this is the problem with not disclosing your GPU quantity. You purposely make your business model look bad, because by approach, you get to a GPU hour price that would imply a payback period of 4 years, where only the last year of the contract is 100% margin. But of course, we can also take "the glass is half full" approach. IREN has ordered 50K B300s from Dell. They have 2 purchase orders for this, 1 between Dell Canada and IE CA Leasing Ltd for 4 phases, and 1 between Dell USA and IE US Hardware 1 Inc (amended from IE US Hardware 4 Inc on April 27, 2026). The order for Canada is divided in 4 phases, and are going to Mackenzie for 80MW of gross capacity, which happens to be 4 buildings of 20MW. The order for Childress is divided in 2 phases, and are going to DC35 and DC36, (as depicted in the earnings presentation) and those are 50MW gross. The purchase price of the order for Childress was $1.2B, and for Canada it was $2.3B If we go with 50,000 B300s for a total of $3.5B then $1.2 would represent 34.285% of the 50,000 GPUs, or 17,140 B300s rounded down. For this calculation I will consider that $IREN will deploy 17,140 GPUs in 50MW gross capacity in DC35 and DC36 of block 3 in Childress.. That would imply at 1.2 PUE, IREN can run 17,140 B300s in 41.67MW IT load. Now by that ratio, they can run 24,680 GPUs in 60MW IT load — a massive difference with 34k units through the Nvidia DGX reference calculation. If common sense is applied, you can still get to 2 completely different outcomes, that show a difference of more than 9k GPUs. The GPU hour rate at 24.68k GPUs would be $3.145 per B300, as MASSIVE difference from the earlier calculated $2.28. Sure, the DGX system may be a factor here. And I'm sure that the reality is somewhere in the middle. But I personally hate this as an investor, to be unable to calculate profitability on unit economic basis. After all, contracts are signed on a $/GPU hour basis. Why hide this from your investors? Not being able to calculate payback periods, unable to calculate ROIC. And most importantly, we cannot properly assess the $NVDA deal on a contract basis. I really hope the payback period of this contract is not 4 years. I want the glass to be half full, but by starting to censor the purchases, IREN is taking a step in the wrong direction. Not a fan of this.

Frans Bakker

148,167 Aufrufe • vor 4 Monaten

The discourse around the Eze penalty last night is fascinating. If nothing else it provides Arteta with an excuse as to how his team has been hard done by and robbed. He loves excuses. It’s a fascinating situation because I actually think the right outcome was reached albeit through the wrong process. Once given on the field under the current rules it shouldn't have been overturned. For that Arsenal can feel aggrieved. Don't forget Arsenal were the beneficiaries of an incredibly soft penalty in Leverkusen. It wasn’t overturned. I find the attached video interesting because it clearly shows how minimal the contact is. The defender doesn’t pin Ezes foot to the ground. Doesn’t smash into it. It brushes down the side of it. It’s where the still pictures of it were wildly misleading. Are we really saying that contact such as that is worthy of a penalty? What’s clear in the video is that Eze has absolutely made the most of the slight contact. I love the straight left leg. A very natural position. It’s a brilliant dive and normally it would have been rewarded. By saying that is a stone wall penalty all we are doing is encouraging such theatrics and cheating. Whenever you hear the words “that was clever” by a commentator they are intimating the player has cheated/made the most of it. We complain about refs all the time but what chance do they have when players are doing stuff like this. Until players are punished properly there is no disincentive for players to keep doing this stuff. I also find it fascinating when you have a foot brushed like this. When you compare it to all the holding/pushing/grappling at other times. Sometimes a contact sport and others not. Is this really what football has become?

Luke Paton

104,573 Aufrufe • vor 4 Monaten

We’re excited to introduce ShinkaEvolve: An open-source framework that evolves programs for scientific discovery with unprecedented sample-efficiency. Blog: Code: Like AlphaEvolve and its variants, our framework leverages LLMs to find state-of-the-art solutions to complex problems, but using orders of magnitude fewer resources! Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature. ‘Shinka’ (進化) is Japanese for evolution, and we designed our system to be just as resourceful. On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a big leap in efficiency compared to previous methods that required thousands of evaluations. We applied ShinkaEvolve to a diverse set of hard problems with real-world applications: 1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering an entire Pareto frontier of solutions trading performance for efficiency. 2/ Competitive Programming: On ALE-Bench (a benchmark for NP-Hard optimization problems), ShinkaEvolve took the best existing agent's solutions and improved them, turning a 5th place solution on one task into a 2nd place leaderboard rank in a competitive programming competition. 3/ LLM Training: We even turned ShinkaEvolve inward to improve LLMs themselves. It tackled the open challenge of designing load balancing losses for Mixture-of-Experts (MoE) models. It discovered a novel loss function that leads to better expert specialization and consistently improves model performance and perplexity. ShinkaEvolve achieves its remarkable sample-efficiency through three key innovations that work together: (1) an adaptive parent sampling strategy to balance exploration and exploitation, (2) novelty-based rejection filtering to avoid redundant work, and (3) a bandit-based LLM ensemble that dynamically picks the best model for the job. By making ShinkaEvolve open-source and highly sample-efficient, our goal is to democratize access to advanced, open-ended discovery tools. Our vision for ShinkaEvolve is to be an easy-to-use companion tool to help scientists and engineers with their daily work. We believe that building more efficient, nature-inspired systems is key to unlocking the future of AI-driven scientific research. We are excited to see what the community builds with it! Learn more in our technical report:

Sakana AI

360,407 Aufrufe • vor 11 Monaten

Dear Pioneers 🌏 We are living in a decisive moment! Pi Network has the potential to become a true revolution in the global financial landscape, but for that to happen, we must act wisely and united. We are millions of Pioneers around the world, driven by the same purpose: to create a decentralized, accessible, and fair financial system for all. The real value of Pi is not determined by the traditional market but by our confidence and engagement. The GCV of $314,159 represents much more than just a number – it symbolizes our conviction about Pi's potential. Our Strength Lies in the Community For Pi to reach its true value, we need to act strategically and with commitment. No great project can sustain itself without the support of its participants. If we want to see Pi shine, we must: ✅ Avoid having our Pi traded on exchanges prematurely, creating scarcity and strengthening its value ✅ Not accept selling it for prices lower than it truly deserves ✅ Acquire and withdraw Pi from exchanges, consolidating its strength in the market ✅ Use it in real transactions, promoting its growth as a global digital currency ✅ Spread awareness and encourage its daily use, expanding its acceptance Each Pioneer plays a crucial role in this process. The more we use and value Pi within the ecosystem, the more it will establish itself as an asset of great global impact. The Future Is in Our Hands We can allow speculators to dictate the value of Pi, or we can take control of our own destiny and ensure that it reaches its true potential. The decision is ours! If we act together, with strategy and determination, nothing can stop us from making Pi Network a milestone in cryptocurrency history. The time is now, and each of us is a fundamental part of this revolution. Let's strengthen Pi together and make the GCV of $314,159 an undeniable reality! Sérgio Cruz, Portugal GCV Ambassador 🇵🇹

Doris Yin 东方紫莲🪷

25,065 Aufrufe • vor 1 Jahr