🤖 Does VLA models really listen to language instructions?... Maybe not 👀 🚀 Introducing our RSS paper: CodeDiffuser -- using VLM-generated code to bridge the gap between **high-level language** and **low-level visuomotor policy** 🎮 Try the live demo: (1/9)show more

🎙️MiniCPM-o 4.5: Full-duplex interaction in motion. Watch the 9B... model track and identify fruit price tags in a dynamic live stream. Unlike traditional reactive systems, MiniCPM-o 4.5 processes continuous video and audio inputs to see, listen, and respond simultaneously—without mutual blocking. 🚀This end-to-end architecture enables low-latency, proactive feedback even while the device is moving, bridging the gap between static vision-language tasks and real-world live interaction. Try the demo and share your feedback with us! Hugging Face 👉 #MiniCPMo45 #MLLM #EdgeAI #OpenSourceshow more

1/ Happy to share VADER: Video Diffusion Alignment via... Reward Gradients. We adapt foundational video diffusion models using pre-trained reward models to generate high-quality, aligned videos for various end-applications. Below we generated a short movie using VADER 😀, we used ChatGPT to write a script and an off-the-shelf AI music generator to generate the sound. Our code & weights are open-sourced:show more

(1/n) 🚀 With FastVideo, you can now generate a... 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan series, a family of fast video generation models trained via a new recipe we term as “sparse distillation”, to speed up video denoising time by 70X! 🖥️ Live demo: (Thanks to @gmicloud for the support!) 🔗 Blog: 🔓 We fully open-source our models, code, and data with Apache-2.0 licensesshow more

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical... Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.show more

🛠️ What if a robot could invent its own... tools. And teach itself how to use them? That’s exactly what VLMgineer does: a new framework that lets Vision Language Models (VLMs) design physical tools and the actions to use them, entirely on their own. No templates. No human demonstrations. Just raw, AI-driven creativity. Why it matters ✅ Co-designs tools and actions together using VLMs, ensuring tight coupling between form and function ✅ Uses VLM-guided evolution (not random search) to refine designs intelligently ✅ Outperforms human-designed tools by +64.7% in task success across 12 RoboToolBench challenges ✅ Produces better-than-everyday tools for real manipulation tasks—measured in success rate and elegance It builds on the emerging trend of large-model-guided evolutionary design (like Eureka and AlphaEvolve) and brings it into physical robotics. It opens the door to general-purpose, automated hardware design, no strong priors needed. Code & paper: —- Weekly robotics and AI insights. Subscribe free:show more

Haven't been to a conference in a while, really... excited to be at #NeurIPS2024! I'll be helping present 4 of our group's recent papers: 1. Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL 2. Distributional Successor Features Enable Zero-Shot Policy Optimization 3. Learning to Cooperate with Humans using Generative Agents 4. Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning Find more details on each paper and where to find us in this thread (1/6)show more

Disappointed with your ICLR paper being rejected? Ten years... ago today, Sergey and I finished training some of the first end-to-end neutral nets for robot control 🤖 We submitted the paper to RSS on January 23, 2015. It was rejected for being "incremental" and "unlikely to have much impact" Our resubmission to NeurIPS was also rejected It now has >4,000 citations (and more importantly, end-to-end training is widely accepted!) It's also cool to think about what's changed and what's the same -- - The network was 92k parameters and trained on ~15 minutes of data - The code was a combination of matlab, caffe, ROS, a custom CUDA kernel for speed, and a low-level 20 Hz controller in C++, all talking to each other. ROS+matlab was as bad as it sounds. - We pre-trained the encoder and did inference off-board on a workstation with a larger GPU. - We were paranoid about varying lighting messing up the network, so we did all the experiments after sunset (so long nights running experiments on the robot past 3 am) Now, we have manipulation policies that are far more dextrous, far more generalizable, and maybe on the cusp of breaking into the real world. :) (the paper:show more

I’m excited to announce the launch of @Span_Platform’s AI... Code Detector! 🚀 We all know how transformative AI coding assistants have been, but it's still hard to know what's AI vs. not & what impact it's ultimately having on quality, velocity, and security. Now you can with Span—powered by our machine learning model, span-detect-1, which detects AI-generated code with an industry-best 95% accuracy. We can’t wait to see how this helps engineering leaders who want to lead with hard data, not hype. Try it out today—completely free.show more

Most video models do one thing well. MiniMax H3... does everything, together. Our next-gen open-weight multimodal model reads text, image, video, and audio as a single creative language: motion, sound, emotion, cinematography, all connected. The result: precision-level generation with real cinematic quality. → Commercial-grade: film, ads, MVs, UI, game CG → 2K/24FPS, native stereo audio → Voice cloning + multi-asset reference for characters, motion & camera Concept to production. One seamless flow. | #MiniMaxH3 Hailuo AI (MiniMax)show more

✨What are Lost Animon? ✨ Lost Animon are incredibly... rare creatures, believed to be extinct by the people of Talea. They’re not simple recolors, they feature new shapes and redesigned forms, transforming the original creature into something truly unique. 🔎How do you find a Lost Animon? While exploring the map, listen carefully. If you hear a shimmering sparkle sound through the air…it means a Lost Animon is somewhere nearby. 👀 For anyone wondering: Lost Animon are already available in our free demo. The real question is… will you be able to find one? 📷 🚀Try out the new free Steam Next Fest demo and wishlist now!show more

Contrail lesson! 1. “Chemtrails” don’t exist. Just to get... that out of the way. 2. Observe the satellite loop and Skew-T chart. In the IR satellite loop you can see yesterday, the West Coast had a decent short wave ridge suppressing moisture over California and Nevada. Today, you can see moisture from a low pressure over the Pacific spilling over the ridge that is now moving east of California. This is upper level moisture ADVECTING into the area. This upper level moisture is mainly above the 500mb level, or 20,000ft. 3. Now observe the Skew-T chart. Particularly clue into the 300mb level. This is a perfect example of what I talk about all the time, and why it’s important to pay attention to the 300mb level. This moisture layer is advecting particularly at the 300mb level, and synoptic scale cirrus development, and advection, typically occurs at 300mb. This is key because aircraft are flying at and above the 300mb level. 4. So, lastly, observe the pictures that I took of the sky over northern Nevada at the time of this post. You can see the layer of cirrus as well as contrails persisting in that moisture layer, exactly as depicted in the satellite shot AND confirmed by the Skew-T chart. Keep in mind that temperatures at this level of the atmosphere are typically -20 to -50°C. In this case, you can see that the temperature at 300mb is -40°C and relative humidities at this level are far different than what you experience at the surface. Any decrease in the gap between temperature and dewpoint at this level can significantly increase the relative humidity. This is why it’s referred to as “relative”because it’s far different than temperatures and dew points at the surface. So, to bring it all together, aircraft flying at these altitudes, which most commercial and military aircraft do, injecting warm, moist air from the engines rapidly into the super cooled environment, not only instantly form contrails, but when relative humidities are as depicted in this example, will enable contrails to persist for hours at a time supported by the moisture existing in that layer. This is what causes persistent contrails. These ARE NOT “chemtrails” and because they persist, does not, and will not ever, make them “chemtrails.” Now that you all needed your government to tell you that climate change was a hoax and I’ve been telling you for years that the “Geoengineering” and “chemtrail” nonsense are propaganda directly related to the climate change hoax, hopefully you can take some time to learn the basics of the atmosphere and understand what I’m showing you here, and how it works, so you’re not fooled by climate propaganda going forward. Thank you for your attention to this matter. 💪🏼🇺🇸show more

Introducing MiniMax H3 on Argil Some days, the future... decides to accelerate. Two days ago, we launched the new version of Argil. Today, I’m extremely happy to announce our partnership with MiniMax (official): H3 is live in Argil. We tested it internally. The quality is really solid. An intelligence that reads text, images, video and audio as one language. That captures creative intent at a glance and already delivers production-ready content. MiniMax H3 will soon be open source. And that’s a real game changer. For the occasion, Golden Ticket: 40% off Pro & Business yearly during MiniMax Hive until tonight. Comment "argil" I’ll send you the code in DM. To those who create: the field just got wider. Link 👇show more

Remember when we as football fans had to rely... solely on paper draft guides, sports radio rumors, and gut feelings to predict draft day decisions? Excited that fans now have access to the NFL's Draft IQ powered by Amazon Web Services ( – the most sophisticated tool yet for following the NFL draft and your favorite team's strategy. Draft IQ is built on Amazon QuickSight, our cloud business intelligence service that makes it easy to analyze and visualize massive amounts of data. QuickSight processes real-time data to give fans unprecedented insight into team decision-making, updating the entire draft landscape every five minutes. You can explore team needs, draft capital, and front office tendencies through personalized team dashboards, plus get AWS-powered machine learning predictions about potential trades and picks. During draft week, fans can track picks, prospects, and Next Gen Stats in real-time. We're also introducing Amazon Q Business integration, our generative AI-powered assistant. Q Business leverages large language models to understand and respond to natural language queries, allowing fans to ask detailed questions about draft prospects, team strategies, and historical draft data. It can provide AI-generated insights based on the same historical Next Gen Stats research data that powers Draft IQ, giving fans a new way to engage with the draft experience (check out the example below). Can't wait to see what stories the data tells us as teams make their selections and excited to dig into the Giants' data myself :)show more

After the Striker spends 100 MPH videos blew up... not too long ago, you couldn’t find a video without seeing comments of his mechanics needing improvement and just kind of absurd reasons as to why his mechanics need to improve. Keep in mind this is a 16 year old kid throwing 100 MPH. Can he really throw 100 MPH with as many of the mechanical flaws as the Instagram comments make him out to have? Or are mechanics subjective and majority of what people think creates elite throwers just not really matter like they think it does? With that being said here are couple subjective thoughts that I have on this throw: Passive lower half, doesn’t push/jump to try to create velocity. Setting up torso in a great spot at foot plant, not forcing it True rotation, pelvis rotates torso follows, minimal counter rotation You can agree or disagree with what my thoughts are, which makes the point of speaking the same language between coaches and athletes constantly so important. What I said can be interpreted hundreds of ways if you don’t communicate with the same language every single time you communicate with an athlete.show more

The ATG School One-Pager I’m not trying to reinvent... schooling. There are just a handful of things I believe in which I haven’t seen in any school I’ve been around as a student or parent. Policy #1: Each student gets to be responsible for growing some of their own food, no matter how small, and THROUGHOUT schooling (not just a quickie project here or there). Policy #2: Minimum 1:1 ratio of time NOT SITTING IN THE CLASSROOM. What you do with this is up to you. There are so many real world skills, sports, gardening, music, etc. The strict ratio in the school day is the key for me. Common sense and personal interests can take it from there. Policy #3: Daily time to read whatever you want to read about. The biggest barrier for my reading was INTEREST. Be there to ensure the book is at their level, and to help them if they don’t understand something. Other than that, LET THEM ENJOY READING, ALL THE WAY THROUGH SCHOOL, not just in early years. Policy #4: (This is the most unusual yet the biggest reason I’m in education.) High school is a 50/50 bridge to winning in real life. Mornings are for actual work, making and SAVING UP MONEY. Afternoons are for learning finances and professional skills of YOUR INTEREST. With average work, you’ll finish school with $50,000-$100,000 in the bank, more skills than the norm, and a greater chance of creating your life and work from there on out, rather than conforming to make a paycheck. Policy #5: As part of the high school 50/50 system, ensure each student learns the adult financial red tape in your state/country before you’ve got bills, kids, etc.show more

STEVE-1: A Generative Model for Text-to-Behavior in Minecraft paper... page: Constructing AI models that respond to text instructions is challenging, especially for sequential decision-making tasks. This work introduces an instruction-tuned Video Pretraining (VPT) model for Minecraft called STEVE-1, demonstrating that the unCLIP approach, utilized in DALL-E 2, is also effective for creating instruction-following sequential decision-making agents. STEVE-1 is trained in two steps: adapting the pretrained VPT model to follow commands in MineCLIP's latent space, then training a prior to predict latent codes from text. This allows us to finetune VPT through self-supervised behavioral cloning and hindsight relabeling, bypassing the need for costly human text annotations. By leveraging pretrained models like VPT and MineCLIP and employing best practices from text-conditioned image generation, STEVE-1 costs just $60 to train and can follow a wide range of short-horizon open-ended text and visual instructions in Minecraft. STEVE-1 sets a new bar for open-ended instruction following in Minecraft with low-level controls (mouse and keyboard) and raw pixel inputs, far outperforming previous baselines. We provide experimental evidence highlighting key factors for downstream performance, including pretraining, classifier-free guidance, and data scaling. All resources, including our model weights, training scripts, and evaluation tools are made available for further research.show more

We’re excited to introduce Text-to-LoRA: a Hypernetwork that generates... task-specific LLM adapters (LoRAs) based on a text description of the task. Catch our presentation at #ICML2025! Paper: Code: Biological systems are capable of rapid adaptation, given limited sensory cues. For example, our human visual system can quickly adapt and tune its light sensitivity to our surroundings. While modern LLMs exhibit a wide variety of capabilities and knowledge, they remain rigid when adding task-specific capabilities. Traditionally, customizing these models requires gathering large datasets and performing often expensive, time-consuming fine-tuning for specific applications. To bypass these limitations, Text-to-LoRA (T2L) meta-learns a “hypernetwork” that takes in a text description of a desired task, as a prompt, and generates a task-specific LoRA that performs well on the task. In our experiments, we show that T2L can encode hundreds of existing LoRA adapters. While the compression is lossy, T2L maintains the performance of task-specifically tuned LoRA adapters. We also show that T2L can even generalize to unseen tasks given a natural language description of the tasks. Importantly, Text-to-LoRA is parameter-efficient. It generates LoRAs in a single, inexpensive step, based solely on a simple text description of the task. This approach is a step towards dramatically lowering the technical and computational barriers, allowing non-technical users to specialize foundation models using plain language, rather than needing deep technical expertise or large compute resources.show more