Loading video...

Video Failed to Load

Go Home

Introducing 🫂HUGS: Human Gaussian Splats - capable of creating animatable (3DGS) avatars from a casual video (50-100 frames) in ~30 mins. Our avatars can easily be embedded into other (NeRF) scenes. (1/4) Project: arXiv:

274,194 views • 2 years ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

home page hero ✨ Design notes: - "Forever" hero text dot pixel FX done in Unicorn Studio. (I will do a whole tutorial on this later. Unicorn's WebGL engine is absolutely wild and very powerful / robust) - built in Framer - I wanted to recreate the iOS unlock effect where your home screen icons cascade into place in a beautifully timed choreography. This took a lot of careful timing using Framer's "Appear" effect on the hero text and surrounding avatars because it was super important that we didn't lose the legibility of our main message ("Build Your Forever Audience") with all the animations. - If you look closely, the choreography is setup to lead your eye through the hero text first starting with "Build Your" then "Forever" and finally "Audience." - With those text layers in place + the surrounding avatars, there is a slight 1 sec pause before the remaining elements slide in below and above (How it works, CTA buttons, announcement badge, and lastly the main nav). - All told the entire loading sequence is 6 seconds - Custom particle system powers the interactive star field (the stars slowly gravitate to your pointer position, and the star field perspective changes ever so subtly as you move your mouse around on the page) - I have 3 shooting stars made of small white line layers that start out off canvas rotated at different angles that shoot across to another point off canvas at random times on a loop effect. - Given this hero scene is in space, I wanted the surrounding avatar elements to "float" in low gravity mode. For this I used Framer's loop effect that slowly oscillates the layer's y position. I then offset the delay of each element randomly to stagger the floating loop so each avatar floats independently/randomly - The final major treatment for this hero scene was the scroll animations on the avatars. I wanted to create a bit of a warp speed effect when you scroll down, as if the avatars were being pulled or sucked into a worm hole as you scroll down below this hero fold. - To accomplish this, I applied Framer's scroll transform affect set to "section in view" on each of the floating avatars, and set the "scroll to" position of the upper avatars to be much, much further away on the y-axis than the "scroll to" position of the lower avatars. (eg. -1700px on upper most avatars vs. -600px on lowest positioned avatars). This effectively causes the upper avatars to slide up off the hero canvas with much greater velocity than the lower positioned avatars. - And when you scroll back up to the hero section, the inverse happens where the lower positioned avatars "arrive back in place" from up above the hero canvas before the upper avatars come back into the scene and settle in place. - Overall I wanted this hero section to feel alive. The floating avatars, particle system with very subtle star movements, and the Caustics effect on the "Forever" text all sort of move at the pace of slow breathing - which is a great pace to create a sense of life and comfort in your scene. Conversion Results (so far) - When this new Framer site launched along with Calaxy v1.9 release on Base a couple weeks ago, we saw a surge in traffic, around 20k page views in the first few days. - Of those 20k hits, 11k visited the app install page ( - which is our main CTA - We saw around 10K new users in the first week after v1.9 launch Overall I'm very happy with the new site and early performance metrics. Lots of tweaking to do but its a good start. If you are a designer building in Framer - hit me with any questions on the above hero notes. Happy to share more specifics! 👾

Chadd Weston

16,281 views • 9 months ago

📢 Our lab has been exploring 3D world models for years — and we’re thrilled to share **PhysTwin**: a milestone that reconstructs object appearance, geometry, and dynamics from just a few seconds of interaction! Led by the amazing Hanxiao Jiang 👉 PhysTwin combines **Gaussian splatting** with **inverse dynamics optimization** based on simple **spring-mass** systems. ⚙️ The result? Real-time, action-conditioned 3D video prediction under novel interactions (i.e., 3D world models). 🔑 A few key takeaways: 1. Having the right structure (e.g., particles/masses) helps navigate the trade-off between sample efficiency, generalization, and broad applicability. 2. Visual foundation models (VFMs) have matured to the point where they can provide rich supervision for world modeling (e.g., tracking, shape completion). 3. Beyond VFMs, many crucial components have come together in recent years: Gaussian splats for rendering, NVIDIA Warp for high-performance simulation, and scene/asset generation from a wide range of labs and companies. The future of 3D world models is looking bright! ✨ 4. The resulting digital twin supports a wide range of downstream applications—especially in data generation and policy evaluation, thanks to its realistic rendering and simulation capabilities. 🎥 All code and data to reproduce the results, along with interactive demos, are available on the website. Check the following visualizations of: (1) observations, (2) reconstructed state/actions, (3) interactive digital twins, and (4) the overlays between real-world robot teleoperation and our model’s open-loop predictions.

Yunzhu Li

25,279 views • 1 year ago

The ATG School One-Pager I’m not trying to reinvent schooling. There are just a handful of things I believe in which I haven’t seen in any school I’ve been around as a student or parent. Policy #1: Each student gets to be responsible for growing some of their own food, no matter how small, and THROUGHOUT schooling (not just a quickie project here or there). Policy #2: Minimum 1:1 ratio of time NOT SITTING IN THE CLASSROOM. What you do with this is up to you. There are so many real world skills, sports, gardening, music, etc. The strict ratio in the school day is the key for me. Common sense and personal interests can take it from there. Policy #3: Daily time to read whatever you want to read about. The biggest barrier for my reading was INTEREST. Be there to ensure the book is at their level, and to help them if they don’t understand something. Other than that, LET THEM ENJOY READING, ALL THE WAY THROUGH SCHOOL, not just in early years. Policy #4: (This is the most unusual yet the biggest reason I’m in education.) High school is a 50/50 bridge to winning in real life. Mornings are for actual work, making and SAVING UP MONEY. Afternoons are for learning finances and professional skills of YOUR INTEREST. With average work, you’ll finish school with $50,000-$100,000 in the bank, more skills than the norm, and a greater chance of creating your life and work from there on out, rather than conforming to make a paycheck. Policy #5: As part of the high school 50/50 system, ensure each student learns the adult financial red tape in your state/country before you’ve got bills, kids, etc.

KneeOverToesGuy

31,525 views • 4 months ago

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

Jonathan Stephens

17,712 views • 6 months ago