Andrew Carr 🤸's banner

Andrew Carr 🤸

@andrew_n_carr • 28,114 subscribers

co-founder leading science @getcartwheel co-founder advisor @arcade_ai Past: Codex @OpenAI, Brain @GoogleAI, world ranked Tetris player

Shorts

Most “physics from video” papers replay the clip or do some re-decode LaGSplat learns a Lagrangian from a few seconds of monocular video, then lets you push on the object with a force that was never measured. It's so neat that the same latent q drives the Gaussians and the equations of motion. That’s why a click on the image becomes J^f

Most “physics from video” papers replay the clip or do some re-decode LaGSplat learns a Lagrangian from a few seconds of monocular video, then lets you push on the object with a force that was never measured. It's so neat that the same latent q drives the Gaussians and the equations of motion. That’s why a click on the image becomes J^f

28,187 views

I believe, fundamentally, that the human body should be the platform for creative and physical AI. move yourself -> move everything so here's a fun little demo of our universal retargeter coming to life on faces :)

I believe, fundamentally, that the human body should be the platform for creative and physical AI. move yourself -> move everything so here's a fun little demo of our universal retargeter coming to life on faces :)

14,986 views

Disney's new research on neural rendering really upped the game. They start by path tracing and then running light gathering on those paths. Then they train a little network, kinda like a NERF, to produce those lighting results directly. It essentially keeps the lighting team from requiring a full (complex) re-render every-time they move a light. But it keeps the lighting team fully in the driver's seat creatively. Definitely a great use of AI for advanced storytelling and animation.

Disney's new research on neural rendering really upped the game. They start by path tracing and then running light gathering on those paths. Then they train a little network, kinda like a NERF, to produce those lighting results directly. It essentially keeps the lighting team from requiring a full (complex) re-render every-time they move a light. But it keeps the lighting team fully in the driver's seat creatively. Definitely a great use of AI for advanced storytelling and animation.

76,653 views

One of the biggest challenges in video to motion is scale! Most of the time people think of it as distance to the camera, but when you add multiple people the scale between them almost matters more! At cartwheel - with our newest model - we're really proud to capture distance and character scale accurately

One of the biggest challenges in video to motion is scale! Most of the time people think of it as distance to the camera, but when you add multiple people the scale between them almost matters more! At cartwheel - with our newest model - we're really proud to capture distance and character scale accurately

24,455 views

Huggingface has entered their node editor era with Daggr (DAG-based Gradio workflows). Really really cool and well done

Huggingface has entered their node editor era with Daggr (DAG-based Gradio workflows). Really really cool and well done

29,325 views

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

The best spritesheet animation I've ever seen!! A reddit user found a dead simple prompt for H3 to generate 2d game ready animations Prompt: ```integrated_multimodal_description: [Shot 1] 16bit retro 2D Game Sprite Animation, A cute round spherical dragon, dragon's head and body is one round piece, small little 4 legs, thick short tail, angry expression, he is standing on all his 4 legs, big cute eyes, big cute mouth, dragon is pink, cute little black wings, dragon have no arms, only 4 legs, dragon have cute little black horns. [0s] - Sitting idle animation: He sits on his butt and looks around, then a small blue butterfly fly from the right, dragon look at butterfly and track it with his eyes his eyes pupils become big and cute while he look at the butterfly with cute face expression until it fly away. his tail swing, eyes blinking, eyes looking in different directions. overall_soundscape: Dragon movements sounds. non_diegetic_music: N/A``` The workflow is also crazy simple 1. generate sprite 2. generate video with animation 3. get key frames from this video 4. run auto-background removal More discussion here:

The best spritesheet animation I've ever seen!! A reddit user found a dead simple prompt for H3 to generate 2d game ready animations Prompt: ```integrated_multimodal_description: [Shot 1] 16bit retro 2D Game Sprite Animation, A cute round spherical dragon, dragon's head and body is one round piece, small little 4 legs, thick short tail, angry expression, he is standing on all his 4 legs, big cute eyes, big cute mouth, dragon is pink, cute little black wings, dragon have no arms, only 4 legs, dragon have cute little black horns. [0s] - Sitting idle animation: He sits on his butt and looks around, then a small blue butterfly fly from the right, dragon look at butterfly and track it with his eyes his eyes pupils become big and cute while he look at the butterfly with cute face expression until it fly away. his tail swing, eyes blinking, eyes looking in different directions. overall_soundscape: Dragon movements sounds. non_diegetic_music: N/A``` The workflow is also crazy simple 1. generate sprite 2. generate video with animation 3. get key frames from this video 4. run auto-background removal More discussion here:

Andrew Carr 🤸

72,888 views • 11 days ago

Most “physics from video” papers replay the clip or do some re-decode LaGSplat learns a Lagrangian from a few seconds of monocular video, then lets you push on the object with a force that was never measured. It's so neat that the same latent q drives the Gaussians and the equations of motion. That’s why a click on the image becomes J^f

Most “physics from video” papers replay the clip or do some re-decode LaGSplat learns a Lagrangian from a few seconds of monocular video, then lets you push on the object with a force that was never measured. It's so neat that the same latent q drives the Gaussians and the equations of motion. That’s why a click on the image becomes J^f

Andrew Carr 🤸

28,187 views • 4 days ago

Just a few months after releasing Scoot, which we feel redefined motion generation quality, our team has made substantial improvements in speed, quality, subtlety, and instruction following. This comes in concert with our newly scaled data collection and labeling efforts, compute increase, substantial architectural improvements, and more stable / scalable training dynamics. Swing is our new flagship movement model. It's live today in Cartwheel studio with infinite batch generation coming soon to our API. Treating Movement as a fundamentally new data type has really unlocked our ability to rapidly make improvements. As it turns out there are far more use cases than you might imagine from entertainment to robotics.

Just a few months after releasing Scoot, which we feel redefined motion generation quality, our team has made substantial improvements in speed, quality, subtlety, and instruction following. This comes in concert with our newly scaled data collection and labeling efforts, compute increase, substantial architectural improvements, and more stable / scalable training dynamics. Swing is our new flagship movement model. It's live today in Cartwheel studio with infinite batch generation coming soon to our API. Treating Movement as a fundamentally new data type has really unlocked our ability to rapidly make improvements. As it turns out there are far more use cases than you might imagine from entertainment to robotics.

Andrew Carr 🤸

150,785 views • 7 months ago

I recorded my screen, it took me less than 10 minutes to go from text -> 3d printed object. I brainstormed with Gemini, used Imagen 3.0 to create the image, used Trellis to turn it into a GLB, translated that into an STL and printed it with my wife's bambulabs printer. I've been dreaming about this for years, being able to create things from pure thought! We're getting closer.

I recorded my screen, it took me less than 10 minutes to go from text -> 3d printed object. I brainstormed with Gemini, used Imagen 3.0 to create the image, used Trellis to turn it into a GLB, translated that into an STL and printed it with my wife's bambulabs printer. I've been dreaming about this for years, being able to create things from pure thought! We're getting closer.

Andrew Carr 🤸

360,374 views • 1 year ago

The term "markerless" gets thrown around a lot in motion capture. What does it mean? Well...surprise! There are still markers (we put them on you instead of you getting in a suit) When we capture motion from your videos, we still need to estimate important landmarks, we've just built a way to do it on a web camera instead of a $100k mocap stage. Multi person + scale aware with extremely good camera pose estimation. Try it out and let me know what you think

The term "markerless" gets thrown around a lot in motion capture. What does it mean? Well...surprise! There are still markers (we put them on you instead of you getting in a suit) When we capture motion from your videos, we still need to estimate important landmarks, we've just built a way to do it on a web camera instead of a $100k mocap stage. Multi person + scale aware with extremely good camera pose estimation. Try it out and let me know what you think

Andrew Carr 🤸

72,246 views • 3 months ago

animation is the past, present, and future of storytelling - but it can be painful and tedious. I'm pretty excited about this workflow cartwheel in blender! capture motion, edit, generate, repath, repose, retime - all in your DCC. we're testing this out, so let me know if you're interested (DMs open), we're hoping to launch a cleaned up version early next month

animation is the past, present, and future of storytelling - but it can be painful and tedious. I'm pretty excited about this workflow cartwheel in blender! capture motion, edit, generate, repath, repose, retime - all in your DCC. we're testing this out, so let me know if you're interested (DMs open), we're hoping to launch a cleaned up version early next month

Andrew Carr 🤸

31,375 views • 3 months ago

EDIT MOTION IN VIDEOS!!! Quit prompting and start directing I've been shouting for YEARS about 3D as the control layer. Here it is, signs of life of our Universal Video Editor!!! The workflow is: take your video, capture with comic 4, edit with the motion editor, re-render with your favorite video to video model (e.g., Runway and Google Gemini have good ones) Here we have a fashion clip where we wanted our actress to high step and show a bit more pizzaz - but shoot day is over, and traditionally it would cost many thousands to reshoot. Watch the video to see how it works instead in cartwheel. cc Logan Kilpatrick Cristóbal Valenzuela 👀

EDIT MOTION IN VIDEOS!!! Quit prompting and start directing I've been shouting for YEARS about 3D as the control layer. Here it is, signs of life of our Universal Video Editor!!! The workflow is: take your video, capture with comic 4, edit with the motion editor, re-render with your favorite video to video model (e.g., Runway and Google Gemini have good ones) Here we have a fashion clip where we wanted our actress to high step and show a bit more pizzaz - but shoot day is over, and traditionally it would cost many thousands to reshoot. Watch the video to see how it works instead in cartwheel. cc Logan Kilpatrick Cristóbal Valenzuela 👀

Andrew Carr 🤸

11,622 views • 2 months ago

I'm pretty sure the "AI Mode" / "Generative Search" model is something unreleased. Time to figure out it's mmlu-pro score

I'm pretty sure the "AI Mode" / "Generative Search" model is something unreleased. Time to figure out it's mmlu-pro score

Andrew Carr 🤸

18,866 views • 8 months ago

motion is transferable!! capture, create, and craft on human - then retarget to ANY character. cartographer is our new labor of love, true retargeting for the masses - live today on cartwheel :)

motion is transferable!! capture, create, and craft on human - then retarget to ANY character. cartographer is our new labor of love, true retargeting for the masses - live today on cartwheel :)

Andrew Carr 🤸

10,689 views • 4 months ago

Introducing Cartwheel 🤸 a text-to-animation tool!

Introducing Cartwheel 🤸 a text-to-animation tool!

Andrew Carr 🤸

40,037 views • 2 years ago

No more content to load