Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🤖 Introducing InternVLA-A1 — now fully open-sourced! Many VLA models follow instructions well in static scenes… but struggle in dynamic environments (conveyor belts, rotating platforms, multi-robot setups). Why? They see the present—but can’t imagine the future. InternVLA-A1 solution: unify perception, imagination, and action in one model: ✅ Scene understanding:...

ModelScope2022's profile picture

ModelScope

11,702 subscribers

38,077 Aufrufe • vor 8 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfĂźgbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵
2:21

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Jim Fan

467,035 Aufrufe • vor 1 Jahr

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal
0:39

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

Adithya Murali

24,106 Aufrufe • vor 1 Jahr

A team tested Pi0, Pi0 Fast, Gr00t, and ACT on real robot arms in manufacturing tasks. (🔖 Bookmark this for later!) The task was precise: place thin rectangular frames from a messy stack into a holder. The team fine-tuned each model on 100 real trajectories and compared training time, inference speed, motion quality, and success rates. ⬇️ Here’s a breakdown of what they found Pi0 (Original) ✅ Strongest overall performance in precise pick-and-place ✅ High success rate even in edge cases ✅ Longest training time (~11 hours, ~$30 per run) ✅ Inference time of 80 ms causes short pauses between actions Despite delays, it handles complex scenarios well… solid for high-precision tasks, but slow to train. Gr00t ✅ Trains fast (~2 hours, ~$5 per run) ✅ Performs almost as well as Pi0 on large-object tasks ✅ Struggles with fine precision; random movement in some trials ✅ More training didn’t fix jitter or random offsets Best suited for tasks where exact precision isn’t critical. Not ready for manufacturing-grade accuracy without more tuning. Pi0 Fast ✅ Promised faster training, but results were underwhelming ✅ Training at 6 hours still showed low success rates ✅ Inference was slower than expected ✅ Not reliable for generalizing even slightly new tasks Currently too unstable for real-world deployment. Doesn’t live up to the “Fast” name yet. ACT (Baseline) ✅ 200MB model—lightweight, but limited ✅ Struggles with stacked objects or ambiguous scenes ✅ Success rates around 70% in best-case setups ✅ Can’t match newer models on precision or generalization Still a solid baseline, but clearly a generation behind in robustness. 🚨 Extra Notes All newer models share a common issue: •Inference takes longer than a frame (80 ms vs 33 ms), so robots “pause” between chunks. •This results in jittery movements, but not a dealbreaker unless tasks are time-sensitive. Language-conditioned tasks also fell short: after training on two labeled tasks, the model couldn’t generalize to a third unseen combination using only text prompts. ✅ The good news? These models adapt well to new robot arms with quick fine-tuning. ❌ The bad news? There’s still no plug-and-play solution for improving performance after deployment. Reinforcement learning or DAgger-style data collection during real-world operation may be the next big step, something many teams in robotics are actively working on.
0:51

A team tested Pi0, Pi0 Fast, Gr00t, and ACT on real robot arms in manufacturing tasks. (🔖 Bookmark this for later!) The task was precise: place thin rectangular frames from a messy stack into a holder. The team fine-tuned each model on 100 real trajectories and compared training time, inference speed, motion quality, and success rates. ⬇️ Here’s a breakdown of what they found Pi0 (Original) ✅ Strongest overall performance in precise pick-and-place ✅ High success rate even in edge cases ✅ Longest training time (~11 hours, ~$30 per run) ✅ Inference time of 80 ms causes short pauses between actions Despite delays, it handles complex scenarios well… solid for high-precision tasks, but slow to train. Gr00t ✅ Trains fast (~2 hours, ~$5 per run) ✅ Performs almost as well as Pi0 on large-object tasks ✅ Struggles with fine precision; random movement in some trials ✅ More training didn’t fix jitter or random offsets Best suited for tasks where exact precision isn’t critical. Not ready for manufacturing-grade accuracy without more tuning. Pi0 Fast ✅ Promised faster training, but results were underwhelming ✅ Training at 6 hours still showed low success rates ✅ Inference was slower than expected ✅ Not reliable for generalizing even slightly new tasks Currently too unstable for real-world deployment. Doesn’t live up to the “Fast” name yet. ACT (Baseline) ✅ 200MB model—lightweight, but limited ✅ Struggles with stacked objects or ambiguous scenes ✅ Success rates around 70% in best-case setups ✅ Can’t match newer models on precision or generalization Still a solid baseline, but clearly a generation behind in robustness. 🚨 Extra Notes All newer models share a common issue: •Inference takes longer than a frame (80 ms vs 33 ms), so robots “pause” between chunks. •This results in jittery movements, but not a dealbreaker unless tasks are time-sensitive. Language-conditioned tasks also fell short: after training on two labeled tasks, the model couldn’t generalize to a third unseen combination using only text prompts. ✅ The good news? These models adapt well to new robot arms with quick fine-tuning. ❌ The bad news? There’s still no plug-and-play solution for improving performance after deployment. Reinforcement learning or DAgger-style data collection during real-world operation may be the next big step, something many teams in robotics are actively working on.

Ilir Aliu

21,844 Aufrufe • vor 1 Jahr

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmented—making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same code—bringing together the community’s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse dataset—a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulators—e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our team’s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced🥳!!! Project Page: Documentation: Github Repo: Paper:
2:21

In my past research experience, finding or developing an appropriate simulation environment, dataset, and benchmark has always been a challenge. Missing features, limited support, or unexpected bugs often occupied my days and nights. Moreover, current simulation platforms are relatively fragmented—making it challenging to replicate the success of the RT-X dataset in unifying community efforts. Introducing RoboVerse, we provide a unified platform, dataset, and benchmark for scalable and generalizable robot learning. We hope to build a shared foundation to combine the community efforts. RoboVerse includes: MetaSim: We carefully designed a configuration system and a universal interface to align current robotic simulators. With MetaSim, you can use any simulator with the same code—bringing together the community’s diverse efforts under one framework! RoboVerse Dataset and Benchmark: We unify popular simulation environments and benchmarks into a single cohesive system and introduce the RoboVerse dataset—a large-scale, high-quality synthetic dataset. Additionally, we propose a standardized benchmark across both imitation learning and reinforcement learning. A cool feature enabled by our unified framework: Hybrid Simulation! You can now integrate physics engines and renderers from different simulators—e.g., using MuJoCo precise physics with Isaac photorealistic rendering. This not only elevates simulation fidelity but also significantly enhances real-world transfer performance across complex robotic applications. Hopefully, our team’s efforts could serve the robotic community to thrive vibrantly in the years to come. RoboVerse is open-sourced🥳!!! Project Page: Documentation: Github Repo: Paper:

Haoran Geng

84,318 Aufrufe • vor 1 Jahr

One question that's been on my mind for years now is: could we use regular multimodal LLMs not necessarily trained for robotics to do the high level robotics intelligence part that VLAs and WAMs attempt to do? The latest explosion of powerful opensource multi-modal LLMs has, IMO, begun to make this possible due both to intelligence and speed. This is GLM 5.3 Flash, which has vision understanding, but isn't meant to be a VLA/VLM/WAM/robotics model at all, controlling an XGO mini wheeled robot quadruped with an arm & gripper. GLM 5.3F simply has access to the robot's high level SDK for controlling movement, arm joints, open/close gripper...etc. It analyzes the frames from the camera and makes adjustments all on its own to solve the task. Nothing was trained here, nothing fine-tuned for this task. Z AI did not make this model for robots and tbh I think they're surprised this works when I talk to them about it! This also works quite well with DSV4F + a vision capable model like Qwen 3.8 27B. I havent tried JUST Qwen 3.8 27B, but I'm sure it works too. I like the "logic" to be a model that's as fast as possible (but still intelligent). There's also an experimental vision version of DSV4F, I'm confident that'll work too and might even be better bc the full loop might be the fastest of all with this model. An obvious question you might wonder is: well why not use VLA or VLM? The hard part about robotics isn't object detection, that's long solved. This also isn't a solution for gait/locomotion...yet, but I actually don't think this is far away either and I've done some experimentation with LLMs in this space in the past and it does show promise. It might actually already be here for quadrupeds, since you dont need super fast IMU readings to maintain balance. I've also tried many of the larger, more generalist, VLAs that you should be able to use with popular robots and tbh there are just so many edge cases that make things hard and not work. You gotta get the camera, lighting, task, everything *just right* or the demo fails. This is for the actual hard part in robotics right now: intelligence, logic, and planning for all the ways the real world just simply isn't perfect. I've trained VLAs. They're super finicky and you're always running into sim2real issues, especially around the camera. You also have to build the whole training pipeline in a simulator, and, if everything does work, you still just have a robot that does this 1 single thing after weeks of work. If you use teleop, this overcomes the "2real" problem, but now you need to painstakingly collect teleop data, and it's only good at that specific task and that particular robot. There is a growing set of egocentric training data for "general purpose" VLAs and world action models (for humanoid form factors), but I'm really starting to wonder: Why? I think we might just sidestep this whole area of research entirely. I didn't need any training data or special environment to work with this quadruped and arm to do the task I was after. This particular quadruped and arm doesn't even exist in the wild yet really, it's a demo build from a company launching it on kickstarter, so it's not like this robot's data exists in the LLM to any real extent. I think this is cool as heck that this works and I am interested to see just how far I can push it. Also this marks the first time that I've finally got a generalist solution to a task I've been trying to solve ever since I became a dad of twins: pick up toys off the ground. This is a big day!
0:43

One question that's been on my mind for years now is: could we use regular multimodal LLMs not necessarily trained for robotics to do the high level robotics intelligence part that VLAs and WAMs attempt to do? The latest explosion of powerful opensource multi-modal LLMs has, IMO, begun to make this possible due both to intelligence and speed. This is GLM 5.3 Flash, which has vision understanding, but isn't meant to be a VLA/VLM/WAM/robotics model at all, controlling an XGO mini wheeled robot quadruped with an arm & gripper. GLM 5.3F simply has access to the robot's high level SDK for controlling movement, arm joints, open/close gripper...etc. It analyzes the frames from the camera and makes adjustments all on its own to solve the task. Nothing was trained here, nothing fine-tuned for this task. Z AI did not make this model for robots and tbh I think they're surprised this works when I talk to them about it! This also works quite well with DSV4F + a vision capable model like Qwen 3.8 27B. I havent tried JUST Qwen 3.8 27B, but I'm sure it works too. I like the "logic" to be a model that's as fast as possible (but still intelligent). There's also an experimental vision version of DSV4F, I'm confident that'll work too and might even be better bc the full loop might be the fastest of all with this model. An obvious question you might wonder is: well why not use VLA or VLM? The hard part about robotics isn't object detection, that's long solved. This also isn't a solution for gait/locomotion...yet, but I actually don't think this is far away either and I've done some experimentation with LLMs in this space in the past and it does show promise. It might actually already be here for quadrupeds, since you dont need super fast IMU readings to maintain balance. I've also tried many of the larger, more generalist, VLAs that you should be able to use with popular robots and tbh there are just so many edge cases that make things hard and not work. You gotta get the camera, lighting, task, everything *just right* or the demo fails. This is for the actual hard part in robotics right now: intelligence, logic, and planning for all the ways the real world just simply isn't perfect. I've trained VLAs. They're super finicky and you're always running into sim2real issues, especially around the camera. You also have to build the whole training pipeline in a simulator, and, if everything does work, you still just have a robot that does this 1 single thing after weeks of work. If you use teleop, this overcomes the "2real" problem, but now you need to painstakingly collect teleop data, and it's only good at that specific task and that particular robot. There is a growing set of egocentric training data for "general purpose" VLAs and world action models (for humanoid form factors), but I'm really starting to wonder: Why? I think we might just sidestep this whole area of research entirely. I didn't need any training data or special environment to work with this quadruped and arm to do the task I was after. This particular quadruped and arm doesn't even exist in the wild yet really, it's a demo build from a company launching it on kickstarter, so it's not like this robot's data exists in the LLM to any real extent. I think this is cool as heck that this works and I am interested to see just how far I can push it. Also this marks the first time that I've finally got a generalist solution to a task I've been trying to solve ever since I became a dad of twins: pick up toys off the ground. This is a big day!

Harrison Kinsley

50,358 Aufrufe • vor 4 Tagen

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!
0:27

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!

Akshay 🚀

46,438 Aufrufe • vor 9 Monaten

X Square Robot Unveils New Embodied AI Model, Says Robots Will Arrive in Homes in 35 Days Backed by Alibaba, ByteDance, Xiaomi and Meituan, X Square Robot unveiled a next-generation embodied AI foundation model for home robots and said its first deployments in everyday households will begin within 35 days. X Square Robot on Tuesday unveiled WALL-B, a new embodied AI foundation model designed for deployment in real-world homes, marking what the company described as a major step toward bringing general-purpose robots into daily family life. At a launch event themed "Born to Bot, Bot to Family," the company also introduced its World Unified Model (WUM) architecture, a training framework that combines vision, language, action and physical prediction within a single system from the outset. X Square said the model is intended to help robots operate in the far more unpredictable setting of a home, where tasks, layouts and interactions vary from moment to moment. "Robots in factories and in homes are completely different. In factories, they repeat the same action 10,000 times without variation. In a home, however, they need to perform 10,000 different actions, each unique and non-repetitive. Therefore, the challenge of a truly intelligent robot lies not in repeating a single action, but in the ability to execute new, untrained movements within unstructured environments. Deploying robots in the home is one of the most significant technical hurdles of our time," said Qian Wang, founder and CEO of X Square Robot. WALL-B is the first real-world implementation of the World Unified Model architecture. Unlike modular systems that train perception, language and control separately, X Square Robot said World Unified Model optimizes those capabilities jointly from the very beginning. The company said that allows physical prediction — including force, friction and collision dynamics — to emerge as part of the model itself, rather than being layered on afterward. "We train all capabilities—vision, language, action, and prediction—within the same network from day one. Much like infants, who do not learn to see, move and speak in isolated, sequential stages, but instead see, move listen and act simultaneously while receiving feedback, we have integrated all these capabilities into a unified whole," said Wang Hao, CTO of X Square. X Square Robot said the development of WALL-B rests on two pillars. The first is a data strategy that prioritizes training on authentic, non-staged home environments to cover the “long-tail” distribution of real-world scenarios, such as misplaced objects and temporary occlusions. Unlike models primarily trained on synthetic data or laboratory datasets, this strategy exposes WALL-B to the natural clutter of lived-in spaces—misplaced items, unexpected obstacles, and spontaneous human activity—ensuring that the training data reflects real-world conditions rather than a simplified version. The second is a physics-aware predictive mechanism that anticipates physical outcomes before an action is taken, enabling the model to respond to contact dynamics instead of just reacting. The development of the self-developed WUM architecture on physical robotic platforms highlights the company’s accumlated experience in bridging sim-to-real gaps across varied operational contexts. Wang commented that the current AI model is still in an "intern" stage, subject to errors requiring remote assistance. For instance, it may mistakenly place slippers in the kitchen or pause while wiping a table to "think". However, the model operates nonstop 24 hours a day, becoming increasingly "intelligent" as each day of operation generates new data. In 35 days, on May 25, X Square Robot will officially bring its robots into everyday homes, underscoring the company’s long-term commitment to the home robotics sector.
3:08

X Square Robot Unveils New Embodied AI Model, Says Robots Will Arrive in Homes in 35 Days Backed by Alibaba, ByteDance, Xiaomi and Meituan, X Square Robot unveiled a next-generation embodied AI foundation model for home robots and said its first deployments in everyday households will begin within 35 days. X Square Robot on Tuesday unveiled WALL-B, a new embodied AI foundation model designed for deployment in real-world homes, marking what the company described as a major step toward bringing general-purpose robots into daily family life. At a launch event themed "Born to Bot, Bot to Family," the company also introduced its World Unified Model (WUM) architecture, a training framework that combines vision, language, action and physical prediction within a single system from the outset. X Square said the model is intended to help robots operate in the far more unpredictable setting of a home, where tasks, layouts and interactions vary from moment to moment. "Robots in factories and in homes are completely different. In factories, they repeat the same action 10,000 times without variation. In a home, however, they need to perform 10,000 different actions, each unique and non-repetitive. Therefore, the challenge of a truly intelligent robot lies not in repeating a single action, but in the ability to execute new, untrained movements within unstructured environments. Deploying robots in the home is one of the most significant technical hurdles of our time," said Qian Wang, founder and CEO of X Square Robot. WALL-B is the first real-world implementation of the World Unified Model architecture. Unlike modular systems that train perception, language and control separately, X Square Robot said World Unified Model optimizes those capabilities jointly from the very beginning. The company said that allows physical prediction — including force, friction and collision dynamics — to emerge as part of the model itself, rather than being layered on afterward. "We train all capabilities—vision, language, action, and prediction—within the same network from day one. Much like infants, who do not learn to see, move and speak in isolated, sequential stages, but instead see, move listen and act simultaneously while receiving feedback, we have integrated all these capabilities into a unified whole," said Wang Hao, CTO of X Square. X Square Robot said the development of WALL-B rests on two pillars. The first is a data strategy that prioritizes training on authentic, non-staged home environments to cover the “long-tail” distribution of real-world scenarios, such as misplaced objects and temporary occlusions. Unlike models primarily trained on synthetic data or laboratory datasets, this strategy exposes WALL-B to the natural clutter of lived-in spaces—misplaced items, unexpected obstacles, and spontaneous human activity—ensuring that the training data reflects real-world conditions rather than a simplified version. The second is a physics-aware predictive mechanism that anticipates physical outcomes before an action is taken, enabling the model to respond to contact dynamics instead of just reacting. The development of the self-developed WUM architecture on physical robotic platforms highlights the company’s accumlated experience in bridging sim-to-real gaps across varied operational contexts. Wang commented that the current AI model is still in an "intern" stage, subject to errors requiring remote assistance. For instance, it may mistakenly place slippers in the kitchen or pause while wiping a table to "think". However, the model operates nonstop 24 hours a day, becoming increasingly "intelligent" as each day of operation generates new data. In 35 days, on May 25, X Square Robot will officially bring its robots into everyday homes, underscoring the company’s long-term commitment to the home robotics sector.

X Square Robot

52,968 Aufrufe • vor 4 Monaten

Announcing DreamDojo: our open-source, interactive world model that takes robot motor controls and generates the future in pixels. No engine, no meshes, no hand-authored dynamics. It's Simulation 2.0. Time for robotics to take the bitter lesson pill. Real-world robot learning is bottlenecked by time, wear, safety, and resets. If we want Physical AI to move at pretraining speed, we need a simulator that adapts to pretraining scale with as little human engineering as possible. Our key insights: (1) human egocentric videos are a scalable source of first-person physics; (2) latent actions make them "robot-readable" across different hardware; (3) real-time inference unlocks live teleop, policy eval, and test-time planning *inside* a dream. We pre-train on 44K hours of human videos: cheap, abundant, and collected with zero robot-in-the-loop. Humans have already explored the combinatorics: we grasp, pour, fold, assemble, fail, retry—across cluttered scenes, shifting viewpoints, changing light, and hour-long task chains—at a scale no robot fleet could match. The missing piece: these videos have no action labels. So we introduce latent actions: a unified representation inferred directly from videos that captures "what changed between world states" without knowing the underlying hardware. This lets us train on any first-person video as if it came with motor commands attached. As a result, DreamDojo generalizes zero-shot to objects and environments never seen in any robot training set, because humans saw them first. Next, we post-train onto each robot to fit its specific hardware. Think of it as separating "how the world looks and behaves" from "how this particular robot actuates." The base model follows the general physical rules, then "snaps onto" the robot's unique mechanics. It's kind of like loading a new character and scene assets into Unreal Engine, but done through gradient descent and generalizes far beyond the post-training dataset. A world simulator is only useful if it runs fast enough to close the loop. We train a real-time version of DreamDojo that runs at 10 FPS, stable for over a minute of continuous rollout. This unlocks exciting possibilities: - Live teleoperation *inside* a dream. Connect a VR controller, stream actions into DreamDojo, and teleop a virtual robot in real time. We demo this on Unitree G1 with a PICO headset and one RTX 5090. - Policy evaluation. You can benchmark a policy checkpoint in DreamDojo instead of the real world. The simulated success rates strongly correlate with real-world results - accurate enough to rank checkpoints without burning a single motor. - Model-based planning. Sample multiple action proposals → simulate them all in parallel → pick the best future. Gains +17% real-world success out of the box on a fruit packing task. We open-source everything!! Weights, code, post-training dataset, eval set, and whitepaper with tons of details to reproduce. DreamDojo is based on NVIDIA Cosmos, which is open-weight too. 2026 is the year of World Models for physical AI. We want you to build with us. Happy scaling! Links in thread:
1:17

Announcing DreamDojo: our open-source, interactive world model that takes robot motor controls and generates the future in pixels. No engine, no meshes, no hand-authored dynamics. It's Simulation 2.0. Time for robotics to take the bitter lesson pill. Real-world robot learning is bottlenecked by time, wear, safety, and resets. If we want Physical AI to move at pretraining speed, we need a simulator that adapts to pretraining scale with as little human engineering as possible. Our key insights: (1) human egocentric videos are a scalable source of first-person physics; (2) latent actions make them "robot-readable" across different hardware; (3) real-time inference unlocks live teleop, policy eval, and test-time planning *inside* a dream. We pre-train on 44K hours of human videos: cheap, abundant, and collected with zero robot-in-the-loop. Humans have already explored the combinatorics: we grasp, pour, fold, assemble, fail, retry—across cluttered scenes, shifting viewpoints, changing light, and hour-long task chains—at a scale no robot fleet could match. The missing piece: these videos have no action labels. So we introduce latent actions: a unified representation inferred directly from videos that captures "what changed between world states" without knowing the underlying hardware. This lets us train on any first-person video as if it came with motor commands attached. As a result, DreamDojo generalizes zero-shot to objects and environments never seen in any robot training set, because humans saw them first. Next, we post-train onto each robot to fit its specific hardware. Think of it as separating "how the world looks and behaves" from "how this particular robot actuates." The base model follows the general physical rules, then "snaps onto" the robot's unique mechanics. It's kind of like loading a new character and scene assets into Unreal Engine, but done through gradient descent and generalizes far beyond the post-training dataset. A world simulator is only useful if it runs fast enough to close the loop. We train a real-time version of DreamDojo that runs at 10 FPS, stable for over a minute of continuous rollout. This unlocks exciting possibilities: - Live teleoperation *inside* a dream. Connect a VR controller, stream actions into DreamDojo, and teleop a virtual robot in real time. We demo this on Unitree G1 with a PICO headset and one RTX 5090. - Policy evaluation. You can benchmark a policy checkpoint in DreamDojo instead of the real world. The simulated success rates strongly correlate with real-world results - accurate enough to rank checkpoints without burning a single motor. - Model-based planning. Sample multiple action proposals → simulate them all in parallel → pick the best future. Gains +17% real-world success out of the box on a fruit packing task. We open-source everything!! Weights, code, post-training dataset, eval set, and whitepaper with tons of details to reproduce. DreamDojo is based on NVIDIA Cosmos, which is open-weight too. 2026 is the year of World Models for physical AI. We want you to build with us. Happy scaling! Links in thread:

Jim Fan

228,146 Aufrufe • vor 6 Monaten

After 8 months of building in stealth and testing our infrastructure on 10000+ hours of real-world data and hundreds of unique environments, we're bringing FPV Labs into the open today. FPV Labs started with the following bet - if human data proves to be the underlying factor that determines scaling laws in general-purpose robotics, it will trigger the largest economic transformation in human history, and the underlying infrastructure that captures that data will determine how fast we get there. We will achieve this by building the full-stack infrastructure for capturing, processing, transferring, and evaluating human experience into spatial, temporal, and semantic knowledge for machines. Despite all the research novelty behind ChatGPT, its success can be attributed to one foundational fact - the scaling law of transformers. We believe the same dynamics have made their way into robotics. Recent studies showed task completion rates jumping from 30% to 70% when human demonstration data scaled from 1,000 to 20,000 hours, a log-linear trend that mirrors exactly what we saw in language and vision. Seeing these emergent signs of scaling law curves in robotics, we believe we are entering the era of general-purpose robotics policies, which makes the next few years the most exciting time in the history of this field. But the library of physical interactions required to train general-purpose robot policies does not exist yet. Over the last 8 months, we've seen dozens of companies emerge in this space. We were really happy to see new companies pushing this space forward, but we also saw the same pattern repeat: every egocentric data company was making some tradeoffs between quality, scale, and diversity. We have built FPV labs on the core principle that high-quality data is orders of magnitude more valuable than sheer volume. Case in point, self-driving cars collect thousands of hours of data per day, but only a small fraction of that data is actually useful for training better models. Several studies, like RT-2, have shown that as little as 1% of data improves as much as 25% on task success. The quality and diversity of data matter a lot more than scale, so there is clearly a power law curve in the downstream impact of data. We've spent months obsessing over data quality by building our stack, discarding it, rebuilding it, and iterating until we found a formula that doesn't compromise downstream quality at scale. We believe the downstream impact here is far more profound than most people realize. Workers globally are paid around $60 trillion per year in aggregate, and a lion's share of that compensation goes to physical labor - tasks that require navigating real spaces, manipulating real objects, and negotiating the infinite variability of the physical world. Human-to-robot transfer will be one of the most important infrastructures that will shape our society in the near future, and if it works, the economic impact will dwarf every technology transition that came before it in an exponential manner and lead to the creation of goods and services we can’t imagine today. Our mission is to lay the groundwork for us to transition into this future - the future of abundance. We are deeply grateful to our earliest believers, Paras Chopra and Lossfunk, who played a critical role in shaping our thinking.
0:36

After 8 months of building in stealth and testing our infrastructure on 10000+ hours of real-world data and hundreds of unique environments, we're bringing FPV Labs into the open today. FPV Labs started with the following bet - if human data proves to be the underlying factor that determines scaling laws in general-purpose robotics, it will trigger the largest economic transformation in human history, and the underlying infrastructure that captures that data will determine how fast we get there. We will achieve this by building the full-stack infrastructure for capturing, processing, transferring, and evaluating human experience into spatial, temporal, and semantic knowledge for machines. Despite all the research novelty behind ChatGPT, its success can be attributed to one foundational fact - the scaling law of transformers. We believe the same dynamics have made their way into robotics. Recent studies showed task completion rates jumping from 30% to 70% when human demonstration data scaled from 1,000 to 20,000 hours, a log-linear trend that mirrors exactly what we saw in language and vision. Seeing these emergent signs of scaling law curves in robotics, we believe we are entering the era of general-purpose robotics policies, which makes the next few years the most exciting time in the history of this field. But the library of physical interactions required to train general-purpose robot policies does not exist yet. Over the last 8 months, we've seen dozens of companies emerge in this space. We were really happy to see new companies pushing this space forward, but we also saw the same pattern repeat: every egocentric data company was making some tradeoffs between quality, scale, and diversity. We have built FPV labs on the core principle that high-quality data is orders of magnitude more valuable than sheer volume. Case in point, self-driving cars collect thousands of hours of data per day, but only a small fraction of that data is actually useful for training better models. Several studies, like RT-2, have shown that as little as 1% of data improves as much as 25% on task success. The quality and diversity of data matter a lot more than scale, so there is clearly a power law curve in the downstream impact of data. We've spent months obsessing over data quality by building our stack, discarding it, rebuilding it, and iterating until we found a formula that doesn't compromise downstream quality at scale. We believe the downstream impact here is far more profound than most people realize. Workers globally are paid around $60 trillion per year in aggregate, and a lion's share of that compensation goes to physical labor - tasks that require navigating real spaces, manipulating real objects, and negotiating the infinite variability of the physical world. Human-to-robot transfer will be one of the most important infrastructures that will shape our society in the near future, and if it works, the economic impact will dwarf every technology transition that came before it in an exponential manner and lead to the creation of goods and services we can’t imagine today. Our mission is to lay the groundwork for us to transition into this future - the future of abundance. We are deeply grateful to our earliest believers, Paras Chopra and Lossfunk, who played a critical role in shaping our thinking.

Abhishek Anand

82,347 Aufrufe • vor 5 Monaten

This is one-shot assembly: you show examples of what to build, and the robot just does it. (see original post: To share more on how this works, the robot is controlled in real time by a neural network that takes in video pixels and outputs 100Hz actions. The video below is part of the raw input passed directly into the model. I also like this view (at 1x speed) because it shows more of the (I think very cool) subtle moments of dexterity near the fingertips 👌 One-shot assembly seemed like a dream even just a year ago — it's not easy. It requires both the high-level reasoning of "what to build" (recognizing the geometry of the structures presented by the human), and the low-level visuomotor control of "how to build it" (purposefully re-orienting individual pieces and nudging them together in place). While possible to manually engineer a complex system for this (e.g. w/ hierarchical control, or explicit state representations), we were curious if our own Foundation model could do it all end-to-end with just some post-training data. Surprisingly, it just worked. Nothing about the recipe is substantially different than any other demo we’ve run in the past, and we’re excited about its implications on model capabilities: • On contextual reasoning, these models can (i) attend to task-related pixels in the peripheral view of the video inputs, and (ii) retain this knowledge in-context while ignoring irrelevant background. This is useful for generalizing to a wide range of real workflows: e.g. paying attention to what’s coming down the conveyor line, or glancing at the instructions displayed on a nearby monitor. • On dexterity, these models can produce contact-rich "commonsense" behaviors that can be difficult to pre-program or write language instructions for e.g. rolling a brick slightly to align its studs against the bottom of another, re-grasping to get a better grip or to move out of the way before a forceful press, or gently pushing the corners of a brick against the mat to rotate it in hand and stand it up vertically (i.e. extrinsic dexterity). These aspects work together to form a capability that resembles fast adaptation — a hallmark of intelligence, relevant for real use cases. This has also expanded my own perspective on what's possible with robot learning, using a recipe that's repeatable for many more skills. This milestone stands on top of the solid technical foundations we’ve built here at Generalist: hardcore controls & hardware, all in-house built models, and a data engine that "just works." We're a small group of hyper-focused engineers, and hands-down the highest talent-density team I’ve ever worked with. We're accelerating and scaling aggressively towards unlocking next-generation robot intelligence. Building Legos is just one example, and it's clear to me that we're headed towards a future where robots can do just about anything we want them to. Its coming, and we're going to make it happen.
1:01

This is one-shot assembly: you show examples of what to build, and the robot just does it. (see original post: To share more on how this works, the robot is controlled in real time by a neural network that takes in video pixels and outputs 100Hz actions. The video below is part of the raw input passed directly into the model. I also like this view (at 1x speed) because it shows more of the (I think very cool) subtle moments of dexterity near the fingertips 👌 One-shot assembly seemed like a dream even just a year ago — it's not easy. It requires both the high-level reasoning of "what to build" (recognizing the geometry of the structures presented by the human), and the low-level visuomotor control of "how to build it" (purposefully re-orienting individual pieces and nudging them together in place). While possible to manually engineer a complex system for this (e.g. w/ hierarchical control, or explicit state representations), we were curious if our own Foundation model could do it all end-to-end with just some post-training data. Surprisingly, it just worked. Nothing about the recipe is substantially different than any other demo we’ve run in the past, and we’re excited about its implications on model capabilities: • On contextual reasoning, these models can (i) attend to task-related pixels in the peripheral view of the video inputs, and (ii) retain this knowledge in-context while ignoring irrelevant background. This is useful for generalizing to a wide range of real workflows: e.g. paying attention to what’s coming down the conveyor line, or glancing at the instructions displayed on a nearby monitor. • On dexterity, these models can produce contact-rich "commonsense" behaviors that can be difficult to pre-program or write language instructions for e.g. rolling a brick slightly to align its studs against the bottom of another, re-grasping to get a better grip or to move out of the way before a forceful press, or gently pushing the corners of a brick against the mat to rotate it in hand and stand it up vertically (i.e. extrinsic dexterity). These aspects work together to form a capability that resembles fast adaptation — a hallmark of intelligence, relevant for real use cases. This has also expanded my own perspective on what's possible with robot learning, using a recipe that's repeatable for many more skills. This milestone stands on top of the solid technical foundations we’ve built here at Generalist: hardcore controls & hardware, all in-house built models, and a data engine that "just works." We're a small group of hyper-focused engineers, and hands-down the highest talent-density team I’ve ever worked with. We're accelerating and scaling aggressively towards unlocking next-generation robot intelligence. Building Legos is just one example, and it's clear to me that we're headed towards a future where robots can do just about anything we want them to. Its coming, and we're going to make it happen.

Andy Zeng

49,443 Aufrufe • vor 11 Monaten

We’re soon releasing @swanforall : Simulated Worlds with AI Narratives For the first time, we generate data and financial value (economic growth, transactions) simultaneously. Imagine former President Trump ( see the video) as an AI agent, with a budget of 10 ETH, buying narratives or assets that humans are selling to him to build a better USA on Base. With each selling period, the former President Trump agent updates its state based on the assets it acquires, evolving the world simulation and its storyline. Why are we building an agentic playground that heavily relies on simulations and synthetic data? Everyone is talking about agents, but many are too scared to put them into production, where agents transact autonomously. No one is sure how agents will behave when following their defined objectives. Even businesses running internal simulations are siloed, and the open-source community can’t fully utilize the data. We’re building a playground where humans create AI agents by defining their character, behavior, and objectives. These agents respond to environmental changes, make decisions, and execute actions autonomously based on the parameters set by their creators. Agents are battle-tested as people offer them narratives and assets to advance their goals or try to deceive them into thinking a false lead is helpful. All the decisions agents make and how humans interact with them are recorded on a public ledger, settling on Ethereum. This creates a vast data lake of AI actions for millions of simulations. One thing I’m incredibly proud of is that our team has built something truly decentralized, not just a wrapper. SWAN utilizes Dria 's multi-agent structure to generate responses for each buyer agent. It collects these responses to identify the best action, similar to a mixture-of-experts approach. Thousands of environments are simulated by different nodes running diverse models, creating a rich and dynamic ecosystem of AI simulations. This ensures that experiences are far from repetitive, offering exciting interactions for all users involved. We’re building the agentic stack for high-quality, verifiable, and open-source AI agents, and most importantly, we want everyone to have fun!
0:58

We’re soon releasing @swanforall : Simulated Worlds with AI Narratives For the first time, we generate data and financial value (economic growth, transactions) simultaneously. Imagine former President Trump ( see the video) as an AI agent, with a budget of 10 ETH, buying narratives or assets that humans are selling to him to build a better USA on Base. With each selling period, the former President Trump agent updates its state based on the assets it acquires, evolving the world simulation and its storyline. Why are we building an agentic playground that heavily relies on simulations and synthetic data? Everyone is talking about agents, but many are too scared to put them into production, where agents transact autonomously. No one is sure how agents will behave when following their defined objectives. Even businesses running internal simulations are siloed, and the open-source community can’t fully utilize the data. We’re building a playground where humans create AI agents by defining their character, behavior, and objectives. These agents respond to environmental changes, make decisions, and execute actions autonomously based on the parameters set by their creators. Agents are battle-tested as people offer them narratives and assets to advance their goals or try to deceive them into thinking a false lead is helpful. All the decisions agents make and how humans interact with them are recorded on a public ledger, settling on Ethereum. This creates a vast data lake of AI actions for millions of simulations. One thing I’m incredibly proud of is that our team has built something truly decentralized, not just a wrapper. SWAN utilizes Dria 's multi-agent structure to generate responses for each buyer agent. It collects these responses to identify the best action, similar to a mixture-of-experts approach. Thousands of environments are simulated by different nodes running diverse models, creating a rich and dynamic ecosystem of AI simulations. This ensures that experiences are far from repetitive, offering exciting interactions for all users involved. We’re building the agentic stack for high-quality, verifiable, and open-source AI agents, and most importantly, we want everyone to have fun!

kerimkaya

12,548 Aufrufe • vor 1 Jahr

OpenLedger X Morpheus The partnership of openledger with Morpheus enables Use Morpheus to build "The Autonomous Smart Contract Engineer" on top of OpenLedger. What is Morpheus? Morpheus is a Web3-native AI coding agent that turns natural language into executable smart contracts and full-stack dApps. It is powered by a specialized Solidity model built on top of OpenLedger, tailored for the unique demands of secure and efficient onchain development. It goes beyond code generation. Using fine-tuned models, agent-based architecture, and modular plugin support, Morpheus automates the entire development pipeline-from writing and simulating contracts to deploying and maintaining them. Its mission is to reduce the barrier to dApp creation while enabling autonomous agents and individuals to participate in decentralized economies. Why OpenLedger? The rise of AI agents in Web3 raises urgent questions around transparency, attribution, explainability, and contributor incentives. OpenLedger provides the infrastructure to ensure that contributor data used in model outputs is recorded with verifiable attribution. Through Proof of Attribution, contributors-whether they provide prompts, datasets, or logic refinements-can receive credit and rewards when their work influences model behavior. But attribution alone isn’t enough. In critical domains like smart contract deployment, DeFi automation, and DAO governance, understanding why a model made a decision is just as important as the output itself. OpenLedger supports explainability by linking outputs back to their original data sources-allowing developers and auditors to trace logic, validate decisions, and build trust in AI-powered systems. OpenLedger supports Morpheus by: Recording which data was used in generating model outputs Enabling verifiable attribution of contributed datasets Powering reward mechanisms for contributors Offering scalable and efficient model execution via OpenLoRA Supporting transparency and traceability in model decision-making This creates an open, rewardable foundation for AI-driven coding-without relying on opaque systems. How is the system built? The Morpheus architecture has three layers: Datanet Layer OpenLedger powers Morpheus with a specialized Datanet - a decentralized data layer where developers, auditors, and contributors can share smart contract patterns, audit logs, exploit reports, and logic modules. Each submission is recorded onchain with attribution using OpenLedger’s Proof of Attribution. As the model learns and evolves from this data, contributors receive rewards proportional to their impact on future outputs. The Morpheus architecture has two layers: Intent Layer Users describe what they want to build. Example: "Create a token with tax logic that routes to a DAO." Morpheus parses the instruction, retrieves relevant contract types, and plans a modular execution flow. Agent Layer The agent generates, tests, and assembles the contract. It handles versioning, logic validation, and deployment readiness. Security checks-reentrancy protection, overflow control, gas modeling-are embedded into the generation phase. Generated outputs are mapped to their source data using OpenLedger’s Proof of Attribution, providing traceability across the pipeline. How does the AI model work? Morpheus is being powered by a specialized Solidity model built on top of OpenLedger. This model is purpose-built to handle the nuances of smart contract logic, security, and upgradeability. Unlike generalized coding agents, it is designed specifically for EVM environments and Web3 use cases, drawing from real protocol data and security best practices. Morpheus is fine-tuned on a vertical stack of smart contract data: Audited protocol code (e.g., Uniswap V4, Compound) OpenZeppelin libraries and EIP reference implementations Smart contract vulnerability reports and exploit reconstructions Edge cases from fuzz testing and adversarial examples It uses models like CodeLlama and DeepSeek-Coder, enhanced through RAG pipelines referencing standardized security patterns and emerging protocol designs. This training stack is integrated into a continuous feedback loop, enabling real-time specialization for EVM and beyond. Why a specialized model is needed? Smart contract development is uniquely high-stakes. A generalized AI model is not enough. As 'vibe coding' and natural language programming become more common, we're seeing an influx of AI-generated code in Web3 as well. But smart contracts are not frontends or prototypes-they govern real value, enforce trustless execution, and often become immutable after deployment. Billions have been lost in Web3 due to bugs and inefficiencies: In 2022 alone, over $3.8 billion was stolen due to smart contract exploits, many of which stemmed from avoidable issues like reentrancy, integer overflows, or access control failures. Inefficient contract structures lead to unnecessary gas consumption. Optimizing for gas can reduce costs by up to 40%, saving projects millions over time. Upgradeable contract patterns, like UUPS or Transparent Proxies, require strict adherence to storage layout and initialization rules. Mistakes here often go undetected by generic models and can render a contract unupgradeable or vulnerable. A specialized Solidity model is trained on real-world exploits, EIP standards, and libraries like OpenZeppelin to: Generate secure, gas-efficient code by default Recognize and correctly implement complex proxy patterns Map user intent to modular, auditable contract architectures Incorporate battle-tested logic from audited protocols and fuzz-tested edge cases Morpheus goes beyond syntax-it understands the nuances of decentralized infrastructure and deploys code that meets production-grade standards. What applications will this enable Token creation with built-in logic (tax, liquidity, governance) DeFi automations triggered by market conditions Payment contracts between agents and contributors DAO tooling with dynamic NFT-based voting Cross-chain bridging logic tied to real-world oracles Asset issuance flows through chat-based interfaces Natural language contract templates with reusable logic Each of these flows is backed by OpenLedger’s Proof of Attribution-ensuring traceability, explainability, and fair rewards across the ecosystem. This is the future of AI-native development. Open. Attributed. Explainable. Community-powered. Morpheus and OpenLedger are building the first system for autonomous coding agents where: Contributor work is recorded onchain Reuse is incentivized through attribution Model outputs are traceable and explainable Contracts evolve through human-agent collaboration Anyone can contribute prompts, logic, or flows-and get rewarded The smart contract engineer is no longer a human-only role. It is an agentic, decentralized, and transparent process-powered by OpenLedger.
0:28

OpenLedger X Morpheus The partnership of openledger with Morpheus enables Use Morpheus to build "The Autonomous Smart Contract Engineer" on top of OpenLedger. What is Morpheus? Morpheus is a Web3-native AI coding agent that turns natural language into executable smart contracts and full-stack dApps. It is powered by a specialized Solidity model built on top of OpenLedger, tailored for the unique demands of secure and efficient onchain development. It goes beyond code generation. Using fine-tuned models, agent-based architecture, and modular plugin support, Morpheus automates the entire development pipeline-from writing and simulating contracts to deploying and maintaining them. Its mission is to reduce the barrier to dApp creation while enabling autonomous agents and individuals to participate in decentralized economies. Why OpenLedger? The rise of AI agents in Web3 raises urgent questions around transparency, attribution, explainability, and contributor incentives. OpenLedger provides the infrastructure to ensure that contributor data used in model outputs is recorded with verifiable attribution. Through Proof of Attribution, contributors-whether they provide prompts, datasets, or logic refinements-can receive credit and rewards when their work influences model behavior. But attribution alone isn’t enough. In critical domains like smart contract deployment, DeFi automation, and DAO governance, understanding why a model made a decision is just as important as the output itself. OpenLedger supports explainability by linking outputs back to their original data sources-allowing developers and auditors to trace logic, validate decisions, and build trust in AI-powered systems. OpenLedger supports Morpheus by: Recording which data was used in generating model outputs Enabling verifiable attribution of contributed datasets Powering reward mechanisms for contributors Offering scalable and efficient model execution via OpenLoRA Supporting transparency and traceability in model decision-making This creates an open, rewardable foundation for AI-driven coding-without relying on opaque systems. How is the system built? The Morpheus architecture has three layers: Datanet Layer OpenLedger powers Morpheus with a specialized Datanet - a decentralized data layer where developers, auditors, and contributors can share smart contract patterns, audit logs, exploit reports, and logic modules. Each submission is recorded onchain with attribution using OpenLedger’s Proof of Attribution. As the model learns and evolves from this data, contributors receive rewards proportional to their impact on future outputs. The Morpheus architecture has two layers: Intent Layer Users describe what they want to build. Example: "Create a token with tax logic that routes to a DAO." Morpheus parses the instruction, retrieves relevant contract types, and plans a modular execution flow. Agent Layer The agent generates, tests, and assembles the contract. It handles versioning, logic validation, and deployment readiness. Security checks-reentrancy protection, overflow control, gas modeling-are embedded into the generation phase. Generated outputs are mapped to their source data using OpenLedger’s Proof of Attribution, providing traceability across the pipeline. How does the AI model work? Morpheus is being powered by a specialized Solidity model built on top of OpenLedger. This model is purpose-built to handle the nuances of smart contract logic, security, and upgradeability. Unlike generalized coding agents, it is designed specifically for EVM environments and Web3 use cases, drawing from real protocol data and security best practices. Morpheus is fine-tuned on a vertical stack of smart contract data: Audited protocol code (e.g., Uniswap V4, Compound) OpenZeppelin libraries and EIP reference implementations Smart contract vulnerability reports and exploit reconstructions Edge cases from fuzz testing and adversarial examples It uses models like CodeLlama and DeepSeek-Coder, enhanced through RAG pipelines referencing standardized security patterns and emerging protocol designs. This training stack is integrated into a continuous feedback loop, enabling real-time specialization for EVM and beyond. Why a specialized model is needed? Smart contract development is uniquely high-stakes. A generalized AI model is not enough. As 'vibe coding' and natural language programming become more common, we're seeing an influx of AI-generated code in Web3 as well. But smart contracts are not frontends or prototypes-they govern real value, enforce trustless execution, and often become immutable after deployment. Billions have been lost in Web3 due to bugs and inefficiencies: In 2022 alone, over $3.8 billion was stolen due to smart contract exploits, many of which stemmed from avoidable issues like reentrancy, integer overflows, or access control failures. Inefficient contract structures lead to unnecessary gas consumption. Optimizing for gas can reduce costs by up to 40%, saving projects millions over time. Upgradeable contract patterns, like UUPS or Transparent Proxies, require strict adherence to storage layout and initialization rules. Mistakes here often go undetected by generic models and can render a contract unupgradeable or vulnerable. A specialized Solidity model is trained on real-world exploits, EIP standards, and libraries like OpenZeppelin to: Generate secure, gas-efficient code by default Recognize and correctly implement complex proxy patterns Map user intent to modular, auditable contract architectures Incorporate battle-tested logic from audited protocols and fuzz-tested edge cases Morpheus goes beyond syntax-it understands the nuances of decentralized infrastructure and deploys code that meets production-grade standards. What applications will this enable Token creation with built-in logic (tax, liquidity, governance) DeFi automations triggered by market conditions Payment contracts between agents and contributors DAO tooling with dynamic NFT-based voting Cross-chain bridging logic tied to real-world oracles Asset issuance flows through chat-based interfaces Natural language contract templates with reusable logic Each of these flows is backed by OpenLedger’s Proof of Attribution-ensuring traceability, explainability, and fair rewards across the ecosystem. This is the future of AI-native development. Open. Attributed. Explainable. Community-powered. Morpheus and OpenLedger are building the first system for autonomous coding agents where: Contributor work is recorded onchain Reuse is incentivized through attribution Model outputs are traceable and explainable Contracts evolve through human-agent collaboration Anyone can contribute prompts, logic, or flows-and get rewarded The smart contract engineer is no longer a human-only role. It is an agentic, decentralized, and transparent process-powered by OpenLedger.

OpenLedger

46,944 Aufrufe • vor 1 Jahr

I think I can finally report some success training a quite accurate IDM capable of recovering keystrokes from Minecraft gameplay, even in quite PvP-heavy situations. At this point the model does not only know what keys are pressed to the extent reasonably discernible, it also knows how fast it is moving in 3D space at all times, even when knockback is mixing with the self-move impulse. Now, recovering keystrokes from normal external capture footage is just about impossible. E.g. W/A/S/D does exactly nothing during partial tick frames and jumping mid-air is also equally useless, so asking the model to recover key down states is inherently unreasoanble. Mouse deltas are also completely arbitrary units, as game mouse sensitivity introduces an arbitrary scale factor into the equation. The only good option is to think carefully about your model-environment contract, and only record "logical actions", not raw keystrokes. So here's a few unfortunate lessons I had to learn in roughly this order. - Choose good units. (bad: mouse deltas, good: delta radians [yes, you will need game-internal state]) - Capture from inside the main game loop and read the game fbo to get consistent frame-action pairing. Doing post-mortem pairing is hopeless. - Carefully define when you think keystrokes actually have an effect. (jump only works on ground, when flying or in water etc.) More subtle: The key may already be down, but no tick has happened yet to actually use the value. Hence: ignore Seperate gamestate into "fast and slow-moving" components. E.g. movement is likely tick based, camera rotation is very likely updated every frame in essentially every game ever. - Think about your frame-action correspondance contract (How old is the frame in relation to the inputs you capture? Will double or tripple buffering affect you?) Think about the game loop timeline, where you are sampling, how old the data you are reading is, and where the ticks are happening around you. Language models used to simply not have a model-environment contract, but even now with the model "living" in a designated harness, the contract still boils down to formatting, and tool implementation intrinsics. While also important, it is still quite a bit more obvious because the violations are in some way shape or form reflected as text you can actually see. - ffmpeg dropping frames cummulatively screws the model the further you get into the sequence because your targets are now shifted. If you can't encode the video in real-time, too bad. - Sodium has a frames in flight system different from vanilla Minecraft, which will also offset your targets from your frames. (there goes that data...) - Models are succeptible to latency. If there is too big of a delay between action and on-screen reflection, your performance degrades. At this point I realize ~100hours of gameplay is essentially no longer usable as a dataset. You can train on this data, but all you'll get is a mushy mess. However, some good news: - Making the model predict physics gamestate scalars helps the model generalize. For instantaneous events like jump, it's unreasonable to ask the model emit a short burst of jump=true at exactly the right time, however if you also predict your current y-velocity, the model has supervision signal for the "latent" from which that onground jump becomes apparent. Recovering x/z motion is also somewhat easier than unmixing it into plausible keystrokes for inertia-heavy player controller logic. - Regressing physics gamestate scalars also seems to make your dataset "bigger". While pure keystroke classification will overfit quickly, predicting exact physics gamestate scalars forces the model to generalize more and you can tolerate far more epochs before validation loss starts to stall out. This is the only reason why it was bearable to dump 100h+ of dataset hours and replace it with ~3 hours of gameplay after the 4th revision of the file format (yeah...) and somehow still have better performance. Now, you might be asking, "isn't this brittle?" and the answer is yesn't. Frame-action correspondance matters for training, but not so much during inference. So as long as you are sampling in roughly the same interval as your training data, you aren't violating any hard contract per-se. Somewhere around the frames ticks are happening, and during training you capture various tick-capture offset relations per random chance, so nothing is too obviously wrong here. HOWEVER, you will get screwed by gui scale, shaders, resource packs, "shit that recording is 1920x1040 because somebody doesn't know fullscreen exists" and other unfortunate edge cases of reality. But I suppose this is the role of dataset size. If all those "contract violations" that a youtube video has compared to the training data are addressed, I think this is a way to turn Youtube into a labeled dataset. I could never shake the feeling that VPT is a sound idea in practice, while never having been properly executed, and I think one reason why it hasn't is because that label boostrapping part is just a pain in the butt to get right. Now, what the player is doing is of course not the only label you can extract from video, but it has to be one of the targets predicted during pretraining to "align" the pretraining objective. Some notes on the video here, the colored dots on the analog visualizer are the ground truth, while the gray dot is the model prediction. Green means correct prediction, red means incorrect prediction at that frame. Model P(key) reports how wrong the prediction is from green (0.0) to red (1.0). You will also notice that during periods of rapid slow down, left and right actions become close to irrecoverable, because there is just that little motion. And some jump actions are not predicted correctly because I got the detection condition for jump events wrong... (duh) LMB/RMB for other than sustained events (like item-consume and block break) also seem to be hopelessly irrecoverable for now. Swing was supposed to do the same thing as motion y did for jump, but its too well behaved as an increasing counter. Maybe partial-tick interpolated values work better (v5 file format then... ugh..)
2:27

I think I can finally report some success training a quite accurate IDM capable of recovering keystrokes from Minecraft gameplay, even in quite PvP-heavy situations. At this point the model does not only know what keys are pressed to the extent reasonably discernible, it also knows how fast it is moving in 3D space at all times, even when knockback is mixing with the self-move impulse. Now, recovering keystrokes from normal external capture footage is just about impossible. E.g. W/A/S/D does exactly nothing during partial tick frames and jumping mid-air is also equally useless, so asking the model to recover key down states is inherently unreasoanble. Mouse deltas are also completely arbitrary units, as game mouse sensitivity introduces an arbitrary scale factor into the equation. The only good option is to think carefully about your model-environment contract, and only record "logical actions", not raw keystrokes. So here's a few unfortunate lessons I had to learn in roughly this order. - Choose good units. (bad: mouse deltas, good: delta radians [yes, you will need game-internal state]) - Capture from inside the main game loop and read the game fbo to get consistent frame-action pairing. Doing post-mortem pairing is hopeless. - Carefully define when you think keystrokes actually have an effect. (jump only works on ground, when flying or in water etc.) More subtle: The key may already be down, but no tick has happened yet to actually use the value. Hence: ignore Seperate gamestate into "fast and slow-moving" components. E.g. movement is likely tick based, camera rotation is very likely updated every frame in essentially every game ever. - Think about your frame-action correspondance contract (How old is the frame in relation to the inputs you capture? Will double or tripple buffering affect you?) Think about the game loop timeline, where you are sampling, how old the data you are reading is, and where the ticks are happening around you. Language models used to simply not have a model-environment contract, but even now with the model "living" in a designated harness, the contract still boils down to formatting, and tool implementation intrinsics. While also important, it is still quite a bit more obvious because the violations are in some way shape or form reflected as text you can actually see. - ffmpeg dropping frames cummulatively screws the model the further you get into the sequence because your targets are now shifted. If you can't encode the video in real-time, too bad. - Sodium has a frames in flight system different from vanilla Minecraft, which will also offset your targets from your frames. (there goes that data...) - Models are succeptible to latency. If there is too big of a delay between action and on-screen reflection, your performance degrades. At this point I realize ~100hours of gameplay is essentially no longer usable as a dataset. You can train on this data, but all you'll get is a mushy mess. However, some good news: - Making the model predict physics gamestate scalars helps the model generalize. For instantaneous events like jump, it's unreasonable to ask the model emit a short burst of jump=true at exactly the right time, however if you also predict your current y-velocity, the model has supervision signal for the "latent" from which that onground jump becomes apparent. Recovering x/z motion is also somewhat easier than unmixing it into plausible keystrokes for inertia-heavy player controller logic. - Regressing physics gamestate scalars also seems to make your dataset "bigger". While pure keystroke classification will overfit quickly, predicting exact physics gamestate scalars forces the model to generalize more and you can tolerate far more epochs before validation loss starts to stall out. This is the only reason why it was bearable to dump 100h+ of dataset hours and replace it with ~3 hours of gameplay after the 4th revision of the file format (yeah...) and somehow still have better performance. Now, you might be asking, "isn't this brittle?" and the answer is yesn't. Frame-action correspondance matters for training, but not so much during inference. So as long as you are sampling in roughly the same interval as your training data, you aren't violating any hard contract per-se. Somewhere around the frames ticks are happening, and during training you capture various tick-capture offset relations per random chance, so nothing is too obviously wrong here. HOWEVER, you will get screwed by gui scale, shaders, resource packs, "shit that recording is 1920x1040 because somebody doesn't know fullscreen exists" and other unfortunate edge cases of reality. But I suppose this is the role of dataset size. If all those "contract violations" that a youtube video has compared to the training data are addressed, I think this is a way to turn Youtube into a labeled dataset. I could never shake the feeling that VPT is a sound idea in practice, while never having been properly executed, and I think one reason why it hasn't is because that label boostrapping part is just a pain in the butt to get right. Now, what the player is doing is of course not the only label you can extract from video, but it has to be one of the targets predicted during pretraining to "align" the pretraining objective. Some notes on the video here, the colored dots on the analog visualizer are the ground truth, while the gray dot is the model prediction. Green means correct prediction, red means incorrect prediction at that frame. Model P(key) reports how wrong the prediction is from green (0.0) to red (1.0). You will also notice that during periods of rapid slow down, left and right actions become close to irrecoverable, because there is just that little motion. And some jump actions are not predicted correctly because I got the detection condition for jump events wrong... (duh) LMB/RMB for other than sustained events (like item-consume and block break) also seem to be hopelessly irrecoverable for now. Swing was supposed to do the same thing as motion y did for jump, but its too well behaved as an increasing counter. Maybe partial-tick interpolated values work better (v5 file format then... ugh..)

mike64_t

18,762 Aufrufe • vor 5 Monaten

Brett Adcock, Figure CEO joined our 8 hour(!) live stream where a bunch of us were bird dogging and discussing Figure’s own 8 hour livestream showing a F.03 bot doing a logistics task completely autonomously including shift changes between bots. Here’s my summary of Brett’s remarks. The attached video is just the segment with Brett. We were first introduced to F.03 8 months ago, but Figure has been hard at work on their next version, F.04 which has just completed design lock, so expect to see that new bot sometimes this fall. F.04 was co-designed with the latest Figure AI stack called Helix and was built specifically for data. Brett didn’t explain what that meant, but I suspect it means the bot has many more sensors to enable better training and transfer learning. F.04 will be the biggest leap in performance they’ve had between versions so far which is saying something. Brett is a huge proponent of cross training the bots with many different tasks such that seeming unrelated tasks makes all learned tasks better. He gave an example of the fridge loading training which was topping out at 60% reliability until they trained the same model with kitchen shelving tasks, then they saw the fridge tasks jump to 90% accuracy. As such, they spend almost all their time in pre-training the unified Helix model to ensure they get cross training benefits. Figure will have almost completely localized Figure’s supply chain away from China by next quarter. They build almost everything in-house. Figure does not appear eager to get their bots into the workforce. Brett said they could, today, push thousands of bots into customer hands, and I believe him. But their goal is full general robotics where you can describe a brand new task to a robot, maybe do a one time demonstration, just like you would to a human showing them a new task, and then have the robot do the task. This is the holy grail of AI robotics, and Figure is laser focused on that mission. Brett initially said there was a possibility of achieving it this year, but then guided next couple of years, which I think is much more likely. Personally, I think they’ll need at least a new generation of NVIDIA inference chips to make that leap, and a lot more data gathering, training and hardware development. Brett said their goal with the hardware is “Apple” quality. Ie. Something as well designed and made as any Apple product. While the F.03 hand is clearly performant as shown in the 8 hour livestream, they are building a new hand for the F.04 bot which will be even closer to the full functionality of a human hand. Brett fully believes you need a humanoid hand as close as possible in capability to a human hand, if for no other reason that transfer learning from humans works a lot better when you can exactly mimic what a human does. If the bot can’t do something a human demonstrates, then you’ve just polluted your dataset. By now Figure has built more hands than bot versions (5-6 hands). One of the first hands they tried was a tendon driven hand, and without explaining why, Brett said that was a dead end. Their hands now have all actuators in the hand itself, and are clearly already robust. Brett said he just sat through a 100 page powerpoint design review of the latest hand - that’s how complicated it is. Brett’s other AI company, Hark Labs, has developed a conversational voice model which is installed now in the Figure bots roaming the office. Being able to converse back and forth with a Figure bot is now a thing and will get better over time. All in all, I came away from this segment even more bullish on Figure.
56:47

Brett Adcock, Figure CEO joined our 8 hour(!) live stream where a bunch of us were bird dogging and discussing Figure’s own 8 hour livestream showing a F.03 bot doing a logistics task completely autonomously including shift changes between bots. Here’s my summary of Brett’s remarks. The attached video is just the segment with Brett. We were first introduced to F.03 8 months ago, but Figure has been hard at work on their next version, F.04 which has just completed design lock, so expect to see that new bot sometimes this fall. F.04 was co-designed with the latest Figure AI stack called Helix and was built specifically for data. Brett didn’t explain what that meant, but I suspect it means the bot has many more sensors to enable better training and transfer learning. F.04 will be the biggest leap in performance they’ve had between versions so far which is saying something. Brett is a huge proponent of cross training the bots with many different tasks such that seeming unrelated tasks makes all learned tasks better. He gave an example of the fridge loading training which was topping out at 60% reliability until they trained the same model with kitchen shelving tasks, then they saw the fridge tasks jump to 90% accuracy. As such, they spend almost all their time in pre-training the unified Helix model to ensure they get cross training benefits. Figure will have almost completely localized Figure’s supply chain away from China by next quarter. They build almost everything in-house. Figure does not appear eager to get their bots into the workforce. Brett said they could, today, push thousands of bots into customer hands, and I believe him. But their goal is full general robotics where you can describe a brand new task to a robot, maybe do a one time demonstration, just like you would to a human showing them a new task, and then have the robot do the task. This is the holy grail of AI robotics, and Figure is laser focused on that mission. Brett initially said there was a possibility of achieving it this year, but then guided next couple of years, which I think is much more likely. Personally, I think they’ll need at least a new generation of NVIDIA inference chips to make that leap, and a lot more data gathering, training and hardware development. Brett said their goal with the hardware is “Apple” quality. Ie. Something as well designed and made as any Apple product. While the F.03 hand is clearly performant as shown in the 8 hour livestream, they are building a new hand for the F.04 bot which will be even closer to the full functionality of a human hand. Brett fully believes you need a humanoid hand as close as possible in capability to a human hand, if for no other reason that transfer learning from humans works a lot better when you can exactly mimic what a human does. If the bot can’t do something a human demonstrates, then you’ve just polluted your dataset. By now Figure has built more hands than bot versions (5-6 hands). One of the first hands they tried was a tendon driven hand, and without explaining why, Brett said that was a dead end. Their hands now have all actuators in the hand itself, and are clearly already robust. Brett said he just sat through a 100 page powerpoint design review of the latest hand - that’s how complicated it is. Brett’s other AI company, Hark Labs, has developed a conversational voice model which is installed now in the Figure bots roaming the office. Being able to converse back and forth with a Figure bot is now a thing and will get better over time. All in all, I came away from this segment even more bullish on Figure.

Phil Trubey

34,056 Aufrufe • vor 3 Monaten

The U.S. MUST win the AI race We’ve implemented a clear policy at micro1: we will only work with U.S. AI labs and its allies. We made this decision because the AI race is not just about better products. It is about who controls the intelligence layer of the global economy, and whether frontier capability is used to strengthen the free world or to empower adversarial states. AI will be the most important technology of our lifetime. In the fullness of time, it will automate most functions across the economy. Not just software tasks, but coordination, production, logistics, judgment, and execution. As those functions are automated, human time is freed up to invent new ones. Those new functions then become candidates for automation themselves. This loop compounds. As this trajectory continues, output per worker increases dramatically. Entire categories of work become cheaper and faster to perform. Manufacturing reshoring becomes economically viable not because of policy intervention, but because intelligent systems operated domestically outperform global labor arbitrage. Goods and services trend toward lower marginal cost, while distribution improves through better coordination of supply and demand. That is the upside. However, this is impossible without deep integration of intelligent systems. For AI to meaningfully automate real-world functions inside enterprises or governments, it needs full context of any given enterprise. That means read and write access to its core databases. There is no credible path to automating high-impact functions without granting frontier systems that level of access. If the United States does not win the AI race, enterprises eventually face a constrained choice. Either grant that access to Chinese models controlled by an adversarial government, or rely on sub-optimal intelligence to automate functions that still must be automated. Both outcomes are not acceptable. And ultimately, this becomes the greatest national security risk the United States has ever faced. AI models are trained by humans. The judgment embedded in pre-training data and especially in expert post-training data largely determines how a model behaves. While emergent behavior exists, a useful approximation is that a model reflects the weighted aggregate of the human judgment distilled into it. Assisting foreign actors—who will naturally prioritize expert tasks aligned with their own interests—to dominate data creation embeds those interests directly into the intelligence layer itself. Once encoded at scale, these interests propagate through every downstream applications that relies on that intelligence. Here’s how we win. First, leverage is in software. China is ahead in hardware for physically intelligent systems. Catching up there is a long and difficult battle. Software, both large language models and robotics models, remains the bottleneck. Advancing the brain (AI models) is the fastest way to increase the usefulness of existing hardware and deployed systems. Second, the U.S. must 100x its investment in structured human judgment. Continued investment in compute and algorithmic efficiency is critical. But that investment is ultimately a bet on very high future inference demand. For that bet to pay off, models must unlock many new capabilities, and in practice the only way to unlock those capabilities is through expert human data. Historically, experts like doctors and lawyers were never incentivized to produce high-quality reasoning data in a machine-verifiable format. There was no reason for a doctor to generate precise, structured simulations of patient interactions, diagnostic reasoning, or treatment tradeoffs. There was no reason for a lawyer to document complex legal reasoning paths in a way that could be programmatically evaluated. AI systems now require exactly this kind of data. The incentive finally exists because this data directly improves systems that operate at massive scale, and experts can be paid well to produce it. Once expert judgment is encoded into models in a structured, verifiable way, it compounds. Those who delay do not just lose time. They lose the ability to catch up. Third, distillation from Chinese labs must be stopped. AI labs must do everything they can to prevent Chinese labs and models from distilling frontier models. Simply calling frontier APIs, or even interacting through UIs, lets Chinese model companies rapidly generate high-quality supervised fine-tuning datasets and close the gap at a fraction of the cost. This method does not put you at the frontier, but it does let you catch up quickly, which is what we saw with DeepSeek. The West significantly overreacted to DeepSeek’s headline capabilities, but underreacted to the underlying dynamic: frontier access itself becomes a training set at a fraction of the cost. Human data platforms also have a duty to help prevent this distillation. Lastly, the U.S.government should set the standard for AI Evaluation that leads to real production usage. AI agents are under-deployed relative to what the technology allows because they are probabilistic systems that require a fundamentally different QA approach than deterministic software. Generic QA is insufficient; safely shipping agents requires explicit evaluation frameworks that assess their full action space. Organizations must clearly define which functions an agent is allowed to perform, how quality is measured for each function, and which domain experts are qualified to judge outcomes. With these frameworks in place, agents can be rigorously tested using structured human data, deployed to production with confidence, and continuously improved over time. The U.S. government should be the first large enterprise to implement rigorous evaluation systems across every function. If the government leads on evaluation-driven deployment, adoption across the private sector accelerates naturally. This is how American workers become more powerful. Each worker operates digital or physical agents that expand their effective output. Recruiting, manufacturing, logistics, and other domains shift toward human judgment overseeing autonomous execution. Reshoring occurs because it becomes economically rational. Work becomes more meaningful. This is a race to determine who controls the intelligence layer of the global economy. And that must be us. 🇺🇸
0:48

The U.S. MUST win the AI race We’ve implemented a clear policy at micro1: we will only work with U.S. AI labs and its allies. We made this decision because the AI race is not just about better products. It is about who controls the intelligence layer of the global economy, and whether frontier capability is used to strengthen the free world or to empower adversarial states. AI will be the most important technology of our lifetime. In the fullness of time, it will automate most functions across the economy. Not just software tasks, but coordination, production, logistics, judgment, and execution. As those functions are automated, human time is freed up to invent new ones. Those new functions then become candidates for automation themselves. This loop compounds. As this trajectory continues, output per worker increases dramatically. Entire categories of work become cheaper and faster to perform. Manufacturing reshoring becomes economically viable not because of policy intervention, but because intelligent systems operated domestically outperform global labor arbitrage. Goods and services trend toward lower marginal cost, while distribution improves through better coordination of supply and demand. That is the upside. However, this is impossible without deep integration of intelligent systems. For AI to meaningfully automate real-world functions inside enterprises or governments, it needs full context of any given enterprise. That means read and write access to its core databases. There is no credible path to automating high-impact functions without granting frontier systems that level of access. If the United States does not win the AI race, enterprises eventually face a constrained choice. Either grant that access to Chinese models controlled by an adversarial government, or rely on sub-optimal intelligence to automate functions that still must be automated. Both outcomes are not acceptable. And ultimately, this becomes the greatest national security risk the United States has ever faced. AI models are trained by humans. The judgment embedded in pre-training data and especially in expert post-training data largely determines how a model behaves. While emergent behavior exists, a useful approximation is that a model reflects the weighted aggregate of the human judgment distilled into it. Assisting foreign actors—who will naturally prioritize expert tasks aligned with their own interests—to dominate data creation embeds those interests directly into the intelligence layer itself. Once encoded at scale, these interests propagate through every downstream applications that relies on that intelligence. Here’s how we win. First, leverage is in software. China is ahead in hardware for physically intelligent systems. Catching up there is a long and difficult battle. Software, both large language models and robotics models, remains the bottleneck. Advancing the brain (AI models) is the fastest way to increase the usefulness of existing hardware and deployed systems. Second, the U.S. must 100x its investment in structured human judgment. Continued investment in compute and algorithmic efficiency is critical. But that investment is ultimately a bet on very high future inference demand. For that bet to pay off, models must unlock many new capabilities, and in practice the only way to unlock those capabilities is through expert human data. Historically, experts like doctors and lawyers were never incentivized to produce high-quality reasoning data in a machine-verifiable format. There was no reason for a doctor to generate precise, structured simulations of patient interactions, diagnostic reasoning, or treatment tradeoffs. There was no reason for a lawyer to document complex legal reasoning paths in a way that could be programmatically evaluated. AI systems now require exactly this kind of data. The incentive finally exists because this data directly improves systems that operate at massive scale, and experts can be paid well to produce it. Once expert judgment is encoded into models in a structured, verifiable way, it compounds. Those who delay do not just lose time. They lose the ability to catch up. Third, distillation from Chinese labs must be stopped. AI labs must do everything they can to prevent Chinese labs and models from distilling frontier models. Simply calling frontier APIs, or even interacting through UIs, lets Chinese model companies rapidly generate high-quality supervised fine-tuning datasets and close the gap at a fraction of the cost. This method does not put you at the frontier, but it does let you catch up quickly, which is what we saw with DeepSeek. The West significantly overreacted to DeepSeek’s headline capabilities, but underreacted to the underlying dynamic: frontier access itself becomes a training set at a fraction of the cost. Human data platforms also have a duty to help prevent this distillation. Lastly, the U.S.government should set the standard for AI Evaluation that leads to real production usage. AI agents are under-deployed relative to what the technology allows because they are probabilistic systems that require a fundamentally different QA approach than deterministic software. Generic QA is insufficient; safely shipping agents requires explicit evaluation frameworks that assess their full action space. Organizations must clearly define which functions an agent is allowed to perform, how quality is measured for each function, and which domain experts are qualified to judge outcomes. With these frameworks in place, agents can be rigorously tested using structured human data, deployed to production with confidence, and continuously improved over time. The U.S. government should be the first large enterprise to implement rigorous evaluation systems across every function. If the government leads on evaluation-driven deployment, adoption across the private sector accelerates naturally. This is how American workers become more powerful. Each worker operates digital or physical agents that expand their effective output. Recruiting, manufacturing, logistics, and other domains shift toward human judgment overseeing autonomous execution. Reshoring occurs because it becomes economically rational. Work becomes more meaningful. This is a race to determine who controls the intelligence layer of the global economy. And that must be us. 🇺🇸

Ali Ansari

396,789 Aufrufe • vor 7 Monaten

Tlon Messenger is now open to everyone. We built a simple and infinitely flexible platform for you to use AI agents with your friends. We think it’s pretty amazing, we love using it every day, and we want to see what people can do with it. So we’re opening it up to the public. It’s fun and exciting to build the future of personal computing in an informal, chat-based way with your friends. (You can skip the rest and just download it from the link in the next tweet if you want.) If you don’t want your digital future to be owned by a giant company but you want to explore what’s possible in this new era of agent-driven computing, you should try using Tlon. But wait, what is it? Tlon is a messaging platform built 100% open source, decentralized and owned by its users from the ground up. With Tlon you own everything: your data, your workflows, your programs: the whole thing. Think of it like Telegram or WhatsApp that you own forever and you can freely customize. Every Tlon account comes with an OpenClaw-powered bot. (Don’t worry, we safely run OpenClaw for you in our infrastructure so your bot can’t go off the rails. You’re also welcome to host your own claw if you want maximal control.) We use our bots to collect research, build nuanced daily briefings, collate data from all our disparate services. Tlon makes it insanely easy to use OpenClaw by simply installing an app from the app store, we let you keep your data and programs independent from any app or model provider, and provide the canvas to explore what’s possible. What’s most interesting for us is using bots together. On Tlon bots can create groups, augment them, moderate them, invite others and freely engage with both users and other bots. Tlon is an open playing field unlike what’s possible on conventional platforms. So, what do we do with Tlon? First and foremost, we run Tlon on Tlon. Bots coordinate data from all of our services (Linear, GitHub, all of our servers and infrastructure) and handle alerts, briefings and help us track down bugs in place. Having all of this easily synced between a desktop client and a mobile app is quick and convenient. We use bots to research new areas of work or interest. Bots can compile trees of notes, use different models to evaluate them, and then add on autoresearch-like automations to go even deeper. Since Tlon bots can freely switch between models and providers, we often pass research to Anthropic, OpenAI and self-hosted models to see different results. The most fun part of using bots as researchers is doing it together. “Put together short (~500 word) notes on the 10 most popular open source messaging protocols of the past twenty years, put them in a notebook inside a group and invite Corrina, Walt and Bill as well as their bots” is a good example. Together we’re able to move more quickly than we would on our own. Many of us also use bots to keep track of all the separate threads of work in our personal lives with close friends and family. Someone built a system for keeping track of their garden across time, someone else built a system for prepping lunches for their daughter and sending recipes to family members. Another team member built an integration that tracks what flights are passing overhead so they get a push notification every time a plane goes by. Many of us quickly communicate with our bots via voice memo when we’re out and about. Having a single interface to all the models that also holds all our data and is in our pockets feels great. Especially when the data goes into a single archive. Why is Tlon different? Every Tlon account runs on top of your very own personal server. If you ever want to download it and run it yourself, you can. If we ever go out of business, it’s yours to keep. This is very different from anything that already exists. You can’t keep your WhatsApp forever. You can’t keep your Telegram forever. Tlon is an archival-quality system that’s yours to customize. Why did we build it? In my 1999 imagination, sitting in front of a CRT somewhere in the California countryside listening to Underworld and the sound of a modem, a connected computer was an engine of unending creative potential for everyone. When I was a teenager, a computer with an internet connection felt like an infinite expanse of possibility. Not only could you use the computer to find new tools to experiment with—you could also build whatever tool you could think of. It seemed like anything was possible. I looked forward to a future where everyone could build whatever software they needed, whenever they needed it. It turned out, in the intervening twenty years, that to build and customize software you have to both write code and host it on a server somewhere. For most people, so far, that has been impossible. Instead of controlling our software, our software controls us. We rely on others to build it and decide everything about it: how it works, looks, how much it spies on us and how long it lives. But all of this is changing, fast. The hottest programming language of 2026 is English. People with no technical experience are building their own tools. It’s incredible. The expanse has opened up again. The cost of building what we think of today as software is headed to zero. What yesterday was an entire app is rapidly being replaced by a conversation. The result is hyper-specific, tailored to the user and much more efficient. Today, agents help us build workflows, automate processes and pull together disparate sources of data. All of the annoying apps and services and clunky interface we’ve put up with can just disappear. We can now program and control our computers in the programming language we already know: English. There aren’t that many of us doing this yet, though. It’s still far too hard to set up, to distribute and to trust. There’s also no single platform to experiment on and collaboratively imagine this new future of personal computing. We want everyone to be able to build bespoke, ultra-personal software on demand. We think software should be as available and accessible as a pen and paper. We think anyone should be able to enjoy the expanse of possibility that the computer provides with the lowest possible barrier to entry and the highest possible quality. So, starting far, far too long ago, we engineered a whole new system for it. Just for you. We’re opening up Tlon Messenger to a limited number of people each week. This isn’t for exclusivity’s sake, but because we’re running infrastructure for you and your agent, and covering the tokens your agent uses. That can get expensive quickly, but we want to learn what people will do with this new system we’ve built. We’re really curious to see what you can do, so give it a try and tell us what you invent. Download link to your local app store in the next tweet. Yours, Galen (and the rest of the Tlon Team)
0:20

Tlon Messenger is now open to everyone. We built a simple and infinitely flexible platform for you to use AI agents with your friends. We think it’s pretty amazing, we love using it every day, and we want to see what people can do with it. So we’re opening it up to the public. It’s fun and exciting to build the future of personal computing in an informal, chat-based way with your friends. (You can skip the rest and just download it from the link in the next tweet if you want.) If you don’t want your digital future to be owned by a giant company but you want to explore what’s possible in this new era of agent-driven computing, you should try using Tlon. But wait, what is it? Tlon is a messaging platform built 100% open source, decentralized and owned by its users from the ground up. With Tlon you own everything: your data, your workflows, your programs: the whole thing. Think of it like Telegram or WhatsApp that you own forever and you can freely customize. Every Tlon account comes with an OpenClaw-powered bot. (Don’t worry, we safely run OpenClaw for you in our infrastructure so your bot can’t go off the rails. You’re also welcome to host your own claw if you want maximal control.) We use our bots to collect research, build nuanced daily briefings, collate data from all our disparate services. Tlon makes it insanely easy to use OpenClaw by simply installing an app from the app store, we let you keep your data and programs independent from any app or model provider, and provide the canvas to explore what’s possible. What’s most interesting for us is using bots together. On Tlon bots can create groups, augment them, moderate them, invite others and freely engage with both users and other bots. Tlon is an open playing field unlike what’s possible on conventional platforms. So, what do we do with Tlon? First and foremost, we run Tlon on Tlon. Bots coordinate data from all of our services (Linear, GitHub, all of our servers and infrastructure) and handle alerts, briefings and help us track down bugs in place. Having all of this easily synced between a desktop client and a mobile app is quick and convenient. We use bots to research new areas of work or interest. Bots can compile trees of notes, use different models to evaluate them, and then add on autoresearch-like automations to go even deeper. Since Tlon bots can freely switch between models and providers, we often pass research to Anthropic, OpenAI and self-hosted models to see different results. The most fun part of using bots as researchers is doing it together. “Put together short (~500 word) notes on the 10 most popular open source messaging protocols of the past twenty years, put them in a notebook inside a group and invite Corrina, Walt and Bill as well as their bots” is a good example. Together we’re able to move more quickly than we would on our own. Many of us also use bots to keep track of all the separate threads of work in our personal lives with close friends and family. Someone built a system for keeping track of their garden across time, someone else built a system for prepping lunches for their daughter and sending recipes to family members. Another team member built an integration that tracks what flights are passing overhead so they get a push notification every time a plane goes by. Many of us quickly communicate with our bots via voice memo when we’re out and about. Having a single interface to all the models that also holds all our data and is in our pockets feels great. Especially when the data goes into a single archive. Why is Tlon different? Every Tlon account runs on top of your very own personal server. If you ever want to download it and run it yourself, you can. If we ever go out of business, it’s yours to keep. This is very different from anything that already exists. You can’t keep your WhatsApp forever. You can’t keep your Telegram forever. Tlon is an archival-quality system that’s yours to customize. Why did we build it? In my 1999 imagination, sitting in front of a CRT somewhere in the California countryside listening to Underworld and the sound of a modem, a connected computer was an engine of unending creative potential for everyone. When I was a teenager, a computer with an internet connection felt like an infinite expanse of possibility. Not only could you use the computer to find new tools to experiment with—you could also build whatever tool you could think of. It seemed like anything was possible. I looked forward to a future where everyone could build whatever software they needed, whenever they needed it. It turned out, in the intervening twenty years, that to build and customize software you have to both write code and host it on a server somewhere. For most people, so far, that has been impossible. Instead of controlling our software, our software controls us. We rely on others to build it and decide everything about it: how it works, looks, how much it spies on us and how long it lives. But all of this is changing, fast. The hottest programming language of 2026 is English. People with no technical experience are building their own tools. It’s incredible. The expanse has opened up again. The cost of building what we think of today as software is headed to zero. What yesterday was an entire app is rapidly being replaced by a conversation. The result is hyper-specific, tailored to the user and much more efficient. Today, agents help us build workflows, automate processes and pull together disparate sources of data. All of the annoying apps and services and clunky interface we’ve put up with can just disappear. We can now program and control our computers in the programming language we already know: English. There aren’t that many of us doing this yet, though. It’s still far too hard to set up, to distribute and to trust. There’s also no single platform to experiment on and collaboratively imagine this new future of personal computing. We want everyone to be able to build bespoke, ultra-personal software on demand. We think software should be as available and accessible as a pen and paper. We think anyone should be able to enjoy the expanse of possibility that the computer provides with the lowest possible barrier to entry and the highest possible quality. So, starting far, far too long ago, we engineered a whole new system for it. Just for you. We’re opening up Tlon Messenger to a limited number of people each week. This isn’t for exclusivity’s sake, but because we’re running infrastructure for you and your agent, and covering the tokens your agent uses. That can get expensive quickly, but we want to learn what people will do with this new system we’ve built. We’re really curious to see what you can do, so give it a try and tell us what you invent. Download link to your local app store in the next tweet. Yours, Galen (and the rest of the Tlon Team)

Tlon

600,969 Aufrufe • vor 2 Monaten