Loading video...

Video Failed to Load

Go Home

WebGPU + WebXR Three.js physics stress tests w/ eye-tracking input on Apple Vision Pro M5 🏟️ 12k bricks, 100 ragdolls, rope & cloth Based on @[email protected]'s work on AVBD sim, rec'd by Cody Bennett. The scale is amazing + little-to-no clipping! Fun to play w/ on next gen h/w

14,678 views • 13 days ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

This is big. NVIDIA and Apple just unlocked the next level for Vision Pro with CloudXR. Here’s what you need to know: It streams from a PC or the cloud directly to your Vision Pro. No cables. Up to 4K at 120fps. The technology is called dynamic foveated streaming. Your eyes only see in full resolution at the center of your gaze. CloudXR tracks exactly where you’re looking and delivers maximum resolution there. Everything in your periphery gets optimized. The stream stays efficient without you ever noticing a difference. Why does this matter? Vision Pro is already the most advanced spatial computer ever made. But standalone processing has a ceiling. There are workflows that need more compute than any headset can carry. CloudXR removes that ceiling by connecting Vision Pro to the full power of NVIDIA RTX in real time. This is not a workaround. It is a native visionOS integration. Also worth knowing. Gaze data never leaves your device. Not to the app. Not to the server. Developers get the full performance benefit of foveation without ever touching your raw eye tracking data. Privacy built into the architecture. Three industries are already on this: Kia, Rivian, and Volvo are running 1:1 scale design reviews with photorealistic accuracy. Full size vehicle models evaluated in spatial computing before a physical prototype exists. That is a fundamental shift in how design works. Foxconn is walking factory floors digitally, optimizing facilities before construction begins. Switch is managing data center infrastructure through a full digital twin. Companies using this approach are reporting up to 30% improvement in development processes. And sim enthusiasts finally get to cut the cord. iRacing and X-Plane 12 are the first titles. Full GeForce RTX power, streamed wirelessly to Vision Pro, inside your physical space. For developers: one Xcode template, one codebase, deployed across Vision Pro, iPhone, and iPad. And multiple headsets can share the same streamed environment at once. Some fully immersed, others on a tablet. That collaborative layer is what enterprise has been asking for. Coming this spring with visionOS 26.4. It’s wild to think about what this unlocks. The future of spatial computing is incredibly bright. Live from GTC. More coming soon.

Justin Ryan

43,685 views • 5 months ago

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data. 2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro -> RoboCasa produces N (varying visuals) -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are building tools to enable everyone in the ecosystem to scale up with us. Links in thread:

Jim Fan

364,565 views • 2 years ago

Alibaba just released a coding model that hits 82 percent on SWE-Bench Verified. That is the highest score ever published for an open-source model. The weights are free. The license is Apache 2.0. You can run it today. The model is Qwen 4 Coder 32B. Here is what 82 percent on SWE-Bench Verified actually means. SWE-Bench Verified tests whether an AI can autonomously resolve real bugs pulled from real production GitHub repositories. Not synthetic exercises. Real open-source projects that real teams depend on. A model gets a bug report, reads the code, writes a fix, and either passes the test suite or it does not. At 82 percent, Qwen 4 Coder 32B resolves 82 out of every 100 real production bugs it is given. Without a human guiding it. On code it has never seen before. For comparison: Qwen 4 Coder 32B: 82 percent SWE-Bench Verified. Open source. Apache 2.0. Claude Fable 5: 80.3 percent SWE-Bench Pro. $10 input / $50 output per million tokens. Currently suspended. GPT-5.6 Sol: Competitive on Terminal-Bench. $5 input / $30 output per million tokens. An open-weight model that you can download and run for free just beat both of them on the benchmark designed to measure real software engineering capability. Here is the architecture. Qwen 4 Coder 32B is a 32 billion parameter dense model. Not a Mixture-of-Experts. Every parameter is active on every request. This matters for inference: a dense 32B model runs on 22 gigabytes of VRAM, which fits on a single high-end consumer GPU or a MacBook Pro with 64GB of unified memory. The smaller variant, Qwen 4 Coder 4B, runs at approximately 135 tokens per second on an M5 Max and fits inside 8 gigabytes of RAM. For a model with usable coding capability, that is a new bar for what fits in a single laptop. The training methodology continued Alibaba's approach of reinforcement learning on verifiable coding tasks. The model gets rewarded when its code passes tests. It gets penalized when it fails. Over millions of training steps, the model learns to write code that actually runs rather than code that looks plausible. License: Apache 2.0. Full commercial use. No attribution requirement. No revenue threshold. No monthly active user ceiling. Weights: Hugging Face, available today. Runs on: vLLM, Ollama, SGLang, and any standard GGUF-compatible inference engine. Qwen 4 32B also runs at approximately 135 tokens per second on an M5 Max chip, setting a new bar for what a sub-8GB model can do on Apple Silicon. The open-source coding model just beat the best closed-source model in the world on the benchmark designed to test whether AI can actually do software engineering. The weights are free. The subscription is optional. Source: Autom8Labs AI Insight July 2026, State of Open Source LLMs June 2026, Kunal Ganglani blog June 2026.

Harman

41,278 views • 1 month ago

Apple Vision Pro: My Initial Impressions I agree with some others. This is one of the most impressive pieces of tech I’ve ever used, but it's not perfect. The Good: • The eye tracking and hand gestures are simply incredible. The accuracy is like 99%. It’s quite intuitive and feels very natural. You can put your hands almost anywhere and it'll sense them. • I’m not exaggerating when I say Apple’s immersive videos made for the Vision Pro literally make you feel like you’re there in person. I felt like I was literally in the room with Alicia Keys while she sang, or traversing 3,000 feet above Norway’s breathtaking fjords while walking on a rope. My only complaint is that while Apple Immersive Videos are 180-degree 3D 8K recordings, they are still a little blurry. • You can use the Vision Pro while lying down in bed in complete darkness and the hand gestures will still work just fine (just like FaceID does in the dark). • In 2013 Apple introduced TouchID. In 2017 FaceID. With Vision Pro, Apple has introduced OpticID. It scans your iris and unlocks your Vision Pro when you put it on. It’s incredibly quick and accurate. And like FaceID, it's secure. It's what you will use to pay for things on the device as well. • I didn’t experience any vertigo, dizziness, etc while using the device. • I'm a bit of a display nerd. For the first time outside of a theatre, Avatar The Way of Water feels true to the directors creative intent. High frame rate (HFR) wasn’t possible in any other device. The 3D is exactly like the theatre. I went to a dolby cinema to watch it originally, and the Vision Pro gets it about as close to a theatre video quality experience as you can get. • The sound quality is really good. The spacial audio is very convincing. • The Vision Pro is heavy, but I found myself not caring all that much if i used the dual loop headband. • I was skeptical of the productivity side of the Vision pro, but wow does the mac virtual display work insanely well. You can easily edit videos or browse the web. • You can complain about plenty of things, but built quality is not one of them. It's really good and solid. The device feels very dense. • Neck strain was not an issue • I watched John Wick in one of the immersive environments, and the movie was reflecting onto the water perfectly. It’s was incredible. I'm not joking when i saw it literally felt like I was watching a movie on a 4K HDR screen in then middle of the wilderness. • Battery life is fine most of the time. The Not So Good: • The app ecosystem just isn’t there yet, which is understandable. That will happen over time, but currently there are only 600 apps built for the Vision. For developer, It doesn't make sense to make a custom Vision Pro app when there are only ~200k users in the ecosystem. • The field of view is kinda of small, which is disappointing, and sometimes a little distracting. It's not terrible, but I wish the FOV was bigger. • The passthrough video is dissapointinly low quality, though it’s still better than anything else on the market. You can’t read your small text on your phone. Colors are muted, and it's a little jittery. Future versions will be better no doubt. • It is sort of an isolating device. I almost guarantee you future versions of the Vision Pro will be able to connect with one another so two people can be looking at the same virtual safari window, movie, etc. • Price: It's expensive. • As I said above, while heavy, the weight of the VP was not an issue for me, but that doesn't mean I don't want it to be lighter. It does press against your face pretty hard, which can cause minor cheek area fatigue. Future versions will no doubt be lighter. Final Thoughts: Going back to my Mac felt weird. It suddenly felt like a technology of the past. If you have the money and want to be an early adopter, you’ll be happy with the Vision Pro. But I have to say, the second or third generation of this device, I think, will be better and cheaper, so it’s probably worth waiting for that. While it’s one of the greatest pieces of tech I’ve ever tried, I can’t see myself using this in my daily life yet, and for $3,500, I should be able to envision that. I'm going to give it a few more days, but I'll likely be returning it. The app ecosystem needs to improve and expand a lot. That's ultimately what will make or break this thing's success. Apple knocked it out of the park engineering this thing. It's mind blowing, but if i'm already mind blown, Imagine how good future versions will be.

Sawyer Merritt

1,602,952 views • 2 years ago

Back in the day, if you were a client of a bespoke tailor and wanted a tweed jacket, there's a good chance you would have phoned up W. Bill in London. They started as a shop selling things like Scottish sweaters, tweedy ties, and pillow covers. Eventually, they exclusively became a cloth merchant. For some reason, during the time I interacted with the shop, it was owned by a large online hardware company, such as the kind that sells buckets and hammers. The owner of that business assigned Ray Hammett to run W. Bill. Ray apparently was a bit of a quiet menace — a soft-spoken man who *loved* British cloth, especially tweeds. The owner of the parent company was wealthy and didn't really supervise Ray, so Ray basically had an endless budget. And so, Ray bought whatever caught his fancy. W. Bill's shop eventually became so stacked with tweeds that the light from the buzzing fluorescent tubes overhead never penetrated to all corners. The inventory was stacked up to the ceiling, and you had to squeeze yourself between the aisles. Apparently, Ray became known across Britain as the "man who would buy any cloth," so other merchants flocked there to sell even more tweeds. If you were looking for a specific tweed — a Shetland or Cheviot, herringbone or houndstooth, in a particular shade of olive or brown — he would most certainly have it. The great thing about W. Bill is that he had stock from independent, little-known weavers, so if you wanted something made by someone in a small shack in some remote corner of Scotland, he'd have that too. You'd phone him up and hear this soft voice on the other end from a man who was enthusiastic to talk to anyone about tweed. Ray passed away some years ago. W. Bill was eventually acquired by Harrison's of Edinburgh, who cleared out the stock that was never going to sell and kept the good stuff that eventually sold. IMO, Harrison's today is the best merchant if you, like me, favor British cloths. They continue to make things under the W. Bill name — linen, cotton, overcoatings, whipcords, and, of course, tweeds. Some of the designs are a bit playful and eccentric, much like what caught Ray's eye.

derek guy

66,320 views • 11 months ago