Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

You can actually interact with the world simulator directly in the browser. 🤖 Here is a quick screen recording (8x speed) of me playing with it: real-time action-conditioned video prediction across rigid objects, deformable objects, rope, and object piles. Try it yourself (no install required): Huge kudos to my...

41,513 Aufrufe • vor 5 Monaten •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

If you think this is just another silly demo made with AI, read this post. You might change your mind, because this demo is about MATH. What you see on the screen is not a render from Blender (obviously, it’s not that good). It’s a three.js app built with Toolcraft. Available on the web and rendered in real time(link in the comments). But Blender still has a lot to do with it. Blender has Geometry Nodes - a powerful node-based system for creating and manipulating procedural geometry. In other words, it’s math. And math is a universal language. And who do you think is pretty good at math? >>> AI. Now you can download or buy Blender files from marketplaces, and when they contain Geometry Nodes for procedural animations, objects, surfaces, or effects, you can transfer that logic to the web. Make it real-time, make it interactive. Materials are a separate story, of course. They can still suck unless you use the right tricks: PBR, HDRIs, material blending, displacement, and faked surface relief. So why is Blender important here? Blender is open source, and many tools around it are open source too. An AI trained on their code. That means it can translate the math from one environment to another quite accurately. If you’ve been struggling to reproduce some idea with AI that you had in your head or seen in some references, and it has something to do with Geometry Nodes, and you can find that idea or a close one in the Blender ecosystem - it means you can transfer it to the web. Thank me later.

Alex Barashkov

28,300 Aufrufe • vor 1 Monat

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

Jonathan Stephens

17,712 Aufrufe • vor 7 Monaten