Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Today, we enable AutoResearch in the physical world for the first time! Introducing ENPIRE: we give 8 Codex agents a fleet of robots, an allocation of GPUs, and generous token budget. We set them free with a simple goal: solve the task as quickly as possible, keep the robots...

675,804 görüntüleme • 2 ay önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

I made Physical AutoResearch sound simple (conceptually), but it took a village to pull off and lots of design thinking into the robot /loopcraft. The hardest part is everything we need to setup *before* pressing Enter. Here's a behind-the-scene tour: 1. Safety harness Letting 8 robots run unattended overnight means safety has to be more than a hint in the system prompt. ENPIRE hardwires it in 2 layers: (1) hard kinematic limit that trips an immediate task failure and auto-resets as soon as a robot leaves its safety envelope, and (2) a torque-limited compliant gripper so a bad contact or misaligned insertion ends in a safe stall, instead of crushing the robot or the object at hand. We make safety more conservative than usual so humans can sleep tight. In reality, we still need a few human operators to watch over the "robots of loving grace". 2. Definition of /done An agent that can edit its own reward will game it for sure. ENPIRE fixes the goalposts before the fleet can move them. Here's the recipe: Collect a few minutes of success & failure demos -> Ask agent to write code using computer vision tools to classify success and measure against groundtruth -> Agent hill-climbs on classifier until reliably good -> This classifier becomes the real-time reward function that directly computes on sensor streams -> *Freeze* the reward function before AutoResearch. It's sacred, enshrined in a Gym env that no one can touch. 3. System telemetry design Robot-seconds is by far the scarcest resource, followed by GPU-seconds, and finally tokens. We instrument all three and surface them to ENPIRE for live resource awareness rather than letting it hill-climb in a vacuum. We define: - Mean Robot Utilization ("MRU"): the fraction of wall-clock time when the robot is actively executing an experiment. Otherwise the hardware is sitting idle and waiting for the next code commit. - Mean Token Utilization ("MTU"): tokens consumed per minute, our proxy for how hard the agent is actually thinking. A low MTU means the agent is stalled, waiting on a robot rollout to finish instead of doing research. - GPU utilization: fraction of wall-clock time when GPU is active. ... and evaluate on two budget-to-outcome metrics: 1. Tokens-to-Success: token budget the fleet burns to complete /goal. 2. Time-to-Success: wall-clock time to /goal

Jim Fan

107,951 görüntüleme • 2 ay önce

Today, we give robots a /skills library that self-evolves and compounds indefinitely! Introducing ASPIRE: a robot solving its 100th task is no longer as clueless as solving its first. Coding agents observe multimodal sensory traces from simulation and real robots, launch an evolutionary search over control programs, and distill the best know-how into an ever-expanding library. ASPIRE is a new type of continual learning: "training" is skill refinement instead of gradient descent. "Trained model" is a repo of sensorimotor skills instead of floating weights. “Distributed training” is a panel of agents each practicing a different skill instead of sharded minibatches. Here's the beauty: ASPIRE gives the tired terms "sim2real transfer" and "cross-embodiment transfer" a whole new meaning. Bridging the sim-to-real gap is notoriously brutal. An end-to-end policy has to swallow both the visual shift (sim looks toyish next to a real camera) and the subtle contact physics it never quite gets right. ASPIRE sidesteps the mess, because it doesn't ship pixels or weights across the gap, but ships the know-how. The robot still has to practice in the real world, not zero-shot, but it gets there way faster because it isn't rediscovering the strategy from scratch. Same for going single-arm to bimanual hardware, which usually requires new data and retraining from zero. ASPIRE achieves up to ~10x cut in "transfer learning” tokens (yes, tokens are the new unit of *training* compute ;) Check out our gallery of 150+ tasks and 90+ skills the robots taught themselves, all on the website! Kind of wild that we can ship the "learned weights" as an HTML page rather than a GGUF. We'll open-source the full stack so your own robot library starts compounding from ours! Deep dive in thread:

Jim Fan

209,534 görüntüleme • 2 ay önce

Jensen Huang: Physical AI is a $50T industry, in 3-5 years robots will be everywhere Nvidia's CEO on the coming explosion of robots in society Jensen: “ Physical AI, as a large category, is the technology industry's first opportunity to address a $50 trillion industry that has largely been void of technology until now.” “And so we need to invent all of the technology necessary to do that.” @jason: “We had this very weird, I don't know if I want to call it a lost decade or 20 years, of Boston Dynamics, Google bought a bunch of companies, they then wound up selling them and spinning them out, where people just thought, ‘Eh, robotics is just not ready for primetime.’” Jensen: “We invented the industry, largely. America invented it. You could argue we got into it too soon, and we got exhausted, we got tired about five years before the enabling technology appeared.” “We got tired of it just a little too soon. Okay, that's number one.” But it's here now.” “Now the question is how much longer? In three to five years, we're going to have robots all over the place.” Jason: “Elon seems to think we're going to have one robot for every human. 8 billion for 8 billion.” Jensen: “I'm hoping more. First of all, there's a whole bunch of robots that are going to be in factories working around the clock. There's going be a whole bunch of robots that don't move.” “Almost everything will be robotic.”

The All-In Podcast

27,735 görüntüleme • 5 ay önce

Milestone! We (robotic arms for gadgets assembly) finished the first commercial order, which brought the first revenue. Here are some learnings from this: The customer was a smart toy manufacturer. The task was to add a heatsink to Raspberry Pi. We received parts from them and returned the assembled modules back. Currently, it's done by teleoperation. Later it will be done by a remote employee via the Internet. Then it will be automated action by action, reducing the operator's time on this and making the task profitable. ps. If you have an assembly task that we can do for you asynchronically - leave a comment below. Learning 1. It's possible! This task which is usually done by the human arm with 5 fingers can be done with a two-finger gripper with the addition of a couple of simple tooling. The task was not simplified. We peeled off thin films from stickers, unpacked paper boxes, moved PCB boards full of components, etc. And no unsolvable problems have been encountered yet. Challenges: 1) The paper box shifted during the opening Solved with the plastic walls that you can lean against 2) Heat pad, stuck to the gripper instead of heat sync. Can be solved by gripper with a pump, but this time solved with the patience of the operator 3) The film on the pad is very thin. Turned out that sub-millimeter arm precision is enough to peel it off with just a regular gripper. 4) The working area has not enough space. You'll only know this by doing real tasks in bulk. This could be solved by an extra pair of long arms, but in this case, solved with the patience of the operator. I think that in the end, we will have 5-10 types of universal tooling and 5-10 types of grippers to solve almost all the problems in such assembly tasks. Learning 2. It's slow. It took 5 times more time, than doing it with human hands. But the good news is there's a lot of room for improvement. We now have specific “time for task” metrics, which we will decrease with iterations. The main reasons for slowness: 1) To rotate the gripper to a steep angle you are forced to control one robot arm with two hands instead of using both arms. We can fix this by just making more room for rotations. 2) Grabbing PCB board with two arms is hard. A slight difference in rotation can break the board, and it's hard to control these angles visually. To solve this, the best way is to use force feedback so you can feel the pressure applied to the item. 3) Accuracy and steadiness is still can be improved We will try a metal version and double the motors to do this. 4) It is physically difficult for the human hands to move with such precision To solve this, we will add a pad for the hands like in surgical robots Learning 3. It's a good business model The "Factory in the cloud" is a good business model for this stage. You send us parts and we send back assembled modules. Currently, it's more convenient than sending a robot to your place, as we can iterate/fix the robot quickly and utilize it 100% of the time. When we polish the set-up over time - we can send robots to your place. So if we can assemble something for you in the USA with Chinese prices by using modern automation - leave a comment below.

Igor Kulakov

37,266 görüntüleme • 1 yıl önce

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data. 2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro -> RoboCasa produces N (varying visuals) -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are building tools to enable everyone in the ecosystem to scale up with us. Links in thread:

Jim Fan

364,565 görüntüleme • 2 yıl önce

US to Iranian Regime: “Give up your nuclear program, give up the enriched uranium, give up your ballistic missile program, stop funding proxies and terrorism, stop killing your citizens, and just be a normal decent country” Iranian Regime to US: “Let’s compromise. Stop the war. Give us a lot of money. Release all our frozen assets. Commit to never attacking us again. And we have full control of the Strait of Hormuz. In return, we keep our nuclear program, we keep the uranium, we keep our missile program, we keep funding our proxies, we keep killing our people… oh, and death to Israel and death to America” US: “We think you misunderstood. You cannot have nukes. No uranium. You must stop killing civilians. You must stop funding proxies, and you must open the Strait of Hormuz” Iranian Regime: “Ah, now we understand. Ok then, let’s negotiate. Stop the war. Give us a lot of money. Release all our frozen assets. Commit to never attacking us again. And we have full control of the Strait of Hormuz. In return, we keep our nuclear program, we keep the uranium, we keep our missile program, we keep funding our proxies, we keep killing our people… oh, and death to Israel and death to America” US: “Listen, you either agree to our terms or we will have no choice but to strike again and all hell will break loose. You have 3 days.” Iranian Regime: “Fine. You’re twisting our arm. We know you won’t attack and if you do we will burn the whole region and humiliate you. But… we are willing to negotiate in good faith. So… Stop the war. Give us a lot of money. Release all our frozen assets. Commit to never attacking us again. And we have full control of the Strait of Hormuz. In return, we keep our nuclear program, we keep the uranium, we keep our missile program, we keep funding our proxies, we keep killing our people… oh, and death to Israel and death to America” US: “Enough. We are giving you one more last chance. Agree to our terms or else. No more chances. You have one week.” Iranian Regime: “We are prepared to negotiate and meet you half way. It has to be fair. So… Stop the war. Give us a lot of money. Release all our frozen assets. Commit to never attacking us again. And we have full control of the Strait of Hormuz. In return, we keep our nuclear program, we keep the uranium, we keep our missile program, we keep funding our proxies, we keep killing our people… oh, and death to Israel and death to America” US: “Stop playing games. This is serious. We will not give you any more chances. You have two weeks or all hell will break loose” Iranian Regime: “We accept to agree to negotiate where we can discuss your terms over the next 30 days. But… Stop the war. Give us a lot of money. Release all our frozen assets. Commit to never attacking us again. And we have full control of the Strait of Hormuz. In return, we keep our nuclear program, we keep the uranium, we keep our missile program, we keep funding our proxies, we keep killing our people… oh, and death to Israel and death to America” And round and round and round we go. Dear President Trump. There is only one way to negotiate with the Islamic regime or any Islamic terrorists. Just one way👇🏼 President Donald J. Trump Secretary of War Pete Hegseth

Mor Edge Insight

29,686 görüntüleme • 3 ay önce

Loneliness On The Path & Transcending The Matrix Many of us can relate to the sense and experience of feeling alone as we shed the matrix-cultural social programming and sincerely start to seek truth. As we embark on the path, we start to learn, often remember, realize, and begin to see a deeper reality within and without us. This new sight is contrary to what the masses believe in and how they live their lives. It is also opposed to whatever we found to be true at first. But there was a splinter in the mind - a seed deep within that inspired us to question what we've been told and taught most often after a period of intense suffering responding to the yearning of the soul's call to awaken. It pushed us to answer the call and step into the unknown, and down the rabbit hole we tumble. As we progress on our path, going through our trials initiations and process of disillusionment, truth starts to become stranger than fiction. We can no longer relate to pleasures, social interactions, and enjoyments we may have engaged in enthusiastically in the past. Goals, ambitions, and desires we once had and were very attached to start to feel completely foreign to us. We realize that most of our so-called needs and desires, wants, including group and tribal identifications, were based on lies, wounds, and matrix programs, which we mistook stemming from our true self. They do not lead us to nor reflect true fulfillment joy, and love - the pure state of our soul hidden deep within us beyond the ego personality - our unique soul embodied individuality centered in the Divine. We start to see through appearances and all the pretense - the mechanical unconscious social interactions with the unconscious feeding mechanisms between people - emotionally, energetically sexually - where almost everyone is a copy of a copy sleepwalking and dreaming of being awake. We realize how little of real individuality and authenticity there is in the world, and how people mistake their conditioned and programmed ego personality for authenticity and also mistake all kinds of feelings, desires, and cravings within them for the soul being. All of which we have done in the past as well to varying degrees and still may do. We realize how little of true love there is. If we are honest with ourselves, we also question what love truly is and if we have really ever experienced it. The more we uncover within us and work through our shadow aspects, mechanical behavior, wounds, and traumas, the more we see all that in others. As the masks fall away, we start to see the unseen and a whole new world - the real world opens up. At first this view is not pleasant at all, and we may question our sanity, feelings of despair, depression, shame, guilt, anger, sadness, and loneliness come up everything that we have been hiding and have suppressed for lifetimes starts to come to the surface the darkness within and without is being made conscious as we descend into the underworld. The higher we want to rise, the deeper we need to go, The more light we possess, the more darkness we uncover. It is the esoteric law of descent and ascent at the same time. If we are sincere in our process, we start to realize thatwe cannot blame others or anything externally anymore. We are asked to take full self responsibility with radical self-honesty. The more we awaken, the more we are also confronted with the illusion of control, the ego likes to cling to. There is something else, a higher divine force that is pulling us out of inertia and sleep, asking us to surrender to it while, at the same time, the lower forces of nature want to keep us chained to our past, our wounds, programs, and identifications trapped in the victim-blame cycle. The occult adverse forces tag into our lower nature using temptations, egohooks and thought injections targeting our blind spots and wounds to keep us plugged into the matrix. Ultimately, we realize that the battle is and always has been through us. We then also start to see and sense the mighty occult hyperdimensional forces beyond our five sensory cognition that manipulate not only us but all people like puppets with strings attached. We realize that there is something else hidden from the mind of ordinary consciousness that controls and manipulates human beings from occult realms - something that feeds on humanity and keeps us in a frequency prison of fear and survival-based egoic service to self-consciousness. We also start to see the wounds and traumas people carry masked and armored under a thick social mask disconnected from their bodies. In fact, we begin to see how most people suffer deep inside but don't know that they are suffering for all the attention and energy is used to escape it by dissociation and then externalizing or suppressing their pain to avoid the discomfort of feeling it within. Like unconscious drug addicts, most people are constantly looking for a fix, trying to fill a bottomless hole in, always looking for love, fulfillment, and happiness externally- just like we have done in our unconscious sleeping state, dreaming of being awake. We then start to encounter humans who don't seem to have a soul but are empty shells occupied by something else. The eyes always give it away - all of which becomes more obvious as we cleanse our perception by internal work. We don't get fooled by appearances anymore. We begin to see via inner vision and our guidance comes more and more from within, not influenced by the group hive mind or the conditioned mind with all its fragmented thought loops. We let go of comparison, ambition and competition and surrender to our soul's calling to unite with the Divine. However, not only do we experience isolation and loneliness pulling back more and more from exterior life we also become subject to the projections and attacks of people still wandering in the wilderness of the collective matrix program. We realize that longtime friends and even family members or our spouse and partner are not at all who we thought they were - that these connections were often based on the illusion of love, trauma bonds, and social programming. We realize how we mistook love for all kinds of behaviors, feelings, and things that are not love but were based on wounds and unconscious feeding mechanisms. When we tell others about our discoveries and realizations we are most often met with ridicule made fun of or even attacked and shamed. We are being told that we are negative, crazy and live in fear or engage in fear-mongering-when, in fact, this reaction is their unconscious fear of the truth projected onto us. It is their false mechanical ego personality trying to protect itself from disillusionment with occult forces, working through them, using them as unconscious matrix agents and attempting to keep us from awakening, speaking truth, and anchoring the divine force. Hence, we also realize that we cannot possibly take anything personally anymore because it is not them doing anything. Moreover, we also recognize that these attacks and interferences serve a teaching function in the bigger picture of the evolution of consciousness and in the process of our unique soul evolution for all there is are lessons. These realizations and confronting the lies we've been telling ourselves do indeed lead to disillusionment - a moral bankruptcy. It's the necessary death everyone has to go through to be reborn in the true self. It is the way out of the matrix. Yet this loneliness we experience and the inability to fit in anywhere has its purpose, including the suffering we experience. If we keep on the track with faith and trust, learning our lessons with sincerity, patience, humility, and courage, it will lead us to true love, real companionship, and a conscious connection with Spirit as we find our way back home, reuniting with the Divine. We start to experience magical positive synchronicities and there is help and support of the divine force that responds to our call and supports and guides us in unexpected ways. But we can only perceive this guidance if we get out of the tunnel vision of ego identification and out of our heads. Doors open where there have been no doors before. Out of the dark, we emerge into the light, and we realize and experience on an embodied level—not merely philosophically or intellectually—that we were never alone but under a spell of separation from all that is, as we experience and perceive the Divine in all. Godspeed. Bernhard Guenther “We must become so alone, so utterly alone, that we withdraw into our innermost self. It is a way of bitter suffering. But then our solitude is overcome, we are no longer alone, for we find that our innermost self is the spirit, that it is Creator, the indivisible. And suddenly, we find ourselves in the midst of the world, yet undisturbed by its multiplicity, for our innermost soul we know ourselves to be one with all being.” - Hermann Hesse --------------------- Written and narrated by Bernhard Guenther Video and audio editing by Victor Fontane

Bernhard Guenther

136,740 görüntüleme • 2 yıl önce