Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

diffusion policy trained on 26 minutes of data (80 demos, 20 seconds each) using DDIM noise scheduler for inference with 10 denoising steps

ahad

1,731 subscribers

318,744 views • 7 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models "Block diffusion sequentially generates blocks of tokens by performing diffusion within each block and conditioning on previous blocks. By combining strength from autoregressive and diffusion models, block diffusion overcomes the limitations of both approaches by supporting variable-length, higher-quality generation and improving inference efficiency with KV caching and parallel sampling."

Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models "Block diffusion sequentially generates blocks of tokens by performing diffusion within each block and conditioning on previous blocks. By combining strength from autoregressive and diffusion models, block diffusion overcomes the limitations of both approaches by supporting variable-length, higher-quality generation and improving inference efficiency with KV caching and parallel sampling."

Tanishq Mathew Abraham, Ph.D.

21,813 views • 1 year ago

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.

MrNeRF

17,052 views • 1 year ago

MuJoCo 3.1.6 has just been released! MJX now comes with out-of-the-box support for height fields. Check out this joystick policy that Baruch Tabanpour trained (~2M steps per second on an A100). Colab to reproduce:

MuJoCo 3.1.6 has just been released! MJX now comes with out-of-the-box support for height fields. Check out this joystick policy that Baruch Tabanpour trained (~2M steps per second on an A100). Colab to reproduce:

Kevin Zakka

30,916 views • 2 years ago

NVIDIA DROPPED A MOTION DIFFUSION MODEL FOR HUMANOID ROBOTS trained on 700 hours of mocap data kimodo generates high-quality 3D human and robot motions from text prompts you control it with: → full-body pose keyframes → end-effector positions/rotations → 2D paths and waypoints works on human skeletons and unitree G1 robot plug the outputs directly into mujoco or retarget to other robots using GMR has a web-based interactive demo with a timeline editor. runs locally needs ~17GB VRAM to run inference open source under apache 2.0

NVIDIA DROPPED A MOTION DIFFUSION MODEL FOR HUMANOID ROBOTS trained on 700 hours of mocap data kimodo generates high-quality 3D human and robot motions from text prompts you control it with: → full-body pose keyframes → end-effector positions/rotations → 2D paths and waypoints works on human skeletons and unitree G1 robot plug the outputs directly into mujoco or retarget to other robots using GMR has a web-based interactive demo with a timeline editor. runs locally needs ~17GB VRAM to run inference open source under apache 2.0

Vaishnavi

17,572 views • 3 months ago

Let your wearable data "speak" for itself! Introducing SensorLM, a family of sensor-language foundation models trained on ~60 million hours of data, enabling robust wearable data understanding with natural language. →

Let your wearable data "speak" for itself! Introducing SensorLM, a family of sensor-language foundation models trained on ~60 million hours of data, enabling robust wearable data understanding with natural language. →

Google Research

74,487 views • 1 year ago

(1/n) 🚀 With FastVideo, you can now generate a 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan series, a family of fast video generation models trained via a new recipe we term as “sparse distillation”, to speed up video denoising time by 70X! 🖥️ Live demo: (Thanks to @gmicloud for the support!) 🔗 Blog: 🔓 We fully open-source our models, code, and data with Apache-2.0 licenses

(1/n) 🚀 With FastVideo, you can now generate a 5-second video in 5 seconds on a single H200 GPU! Introducing FastWan series, a family of fast video generation models trained via a new recipe we term as “sparse distillation”, to speed up video denoising time by 70X! 🖥️ Live demo: (Thanks to @gmicloud for the support!) 🔗 Blog: 🔓 We fully open-source our models, code, and data with Apache-2.0 licenses

Hao AI Lab

78,660 views • 1 year ago

Today, Nesa is partnering with Nillion to power the future of AI. Nesa and Nillion are forging a strong strategic partnership to integrate each other's unique technologies. Nillion will also be using Nesa for its AI inference. Stay tuned for more updates on this ground-breaking partnership.

Today, Nesa is partnering with Nillion to power the future of AI. Nesa and Nillion are forging a strong strategic partnership to integrate each other's unique technologies. Nillion will also be using Nesa for its AI inference. Stay tuned for more updates on this ground-breaking partnership.

Nesa

157,352 views • 2 years ago

This is the coolest Diffusion work I've seen in a while! It generates Visual Anagrams, a type of optical illusion where an image looks like one thing, but changes appearance when transformed. It works with any orthogonal transformation matrices, which luckily include rotation, permutation (jigsaw puzzles), and color negation. Intuitively, the method first inverts the noise from multiple image transforms (with different text prompts), and then average them. After taking a diffusion step in the averaged noise, the resulting image becomes an anagram that aligns with the texts in different views. It does very little computation, using pre-trained Stable Diffusion. Simple, elegant, and inexpensive technique for non-professionals to create some interesting art! Paper: Website: It's open-source! Authors: Daniel Geng, Inbum Park, Andrew Owens.

This is the coolest Diffusion work I've seen in a while! It generates Visual Anagrams, a type of optical illusion where an image looks like one thing, but changes appearance when transformed. It works with any orthogonal transformation matrices, which luckily include rotation, permutation (jigsaw puzzles), and color negation. Intuitively, the method first inverts the noise from multiple image transforms (with different text prompts), and then average them. After taking a diffusion step in the averaged noise, the resulting image becomes an anagram that aligns with the texts in different views. It does very little computation, using pre-trained Stable Diffusion. Simple, elegant, and inexpensive technique for non-professionals to create some interesting art! Paper: Website: It's open-source! Authors: Daniel Geng, Inbum Park, Andrew Owens.

Jim Fan

188,968 views • 2 years ago

LTX-2 is amazing! WIth the distilled model on a RTX 5060 took less than 3 minutes for generating this 10 seconds 720p clip. I used Wan2GP on Pinokio

LTX-2 is amazing! WIth the distilled model on a RTX 5060 took less than 3 minutes for generating this 10 seconds 720p clip. I used Wan2GP on Pinokio

franzipol

21,437 views • 6 months ago

i manage 160+ accounts for 20 brands with less than 2 days of work per month. most agencies need 10 employees for this. i use 3 VA's and a scheduler. here's the entire workflow: > va's warmup 10 accounts per niche manually for 7 days ($100/account) > load into social scheduler > queue 120 DansUGC vids (reaction + demo) at $8/vid done. while you're hiring your 4th social media manager i'm scaling to 100 accounts.

i manage 160+ accounts for 20 brands with less than 2 days of work per month. most agencies need 10 employees for this. i use 3 VA's and a scheduler. here's the entire workflow: > va's warmup 10 accounts per niche manually for 7 days ($100/account) > load into social scheduler > queue 120 DansUGC vids (reaction + demo) at $8/vid done. while you're hiring your 4th social media manager i'm scaling to 100 accounts.

daniel

16,183 views • 7 months ago

We’re proud to share that Tamarind Bio has been selected to build, host, and operate the inference infrastructure layer for TuneLab2.0, the next evolution of the platform. Eli Lilly and Company TuneLab is a first-of-its-kind, collaborative AI/ML drug discovery platform, bringing models trained on over $1B worth of Lilly proprietary data to the biotech ecosystem. Tamarind will power TuneLab’s scalable drug discovery workflows and model inference.

We’re proud to share that Tamarind Bio has been selected to build, host, and operate the inference infrastructure layer for TuneLab2.0, the next evolution of the platform. Eli Lilly and Company TuneLab is a first-of-its-kind, collaborative AI/ML drug discovery platform, bringing models trained on over $1B worth of Lilly proprietary data to the biotech ecosystem. Tamarind will power TuneLab’s scalable drug discovery workflows and model inference.

Deniz Kavi

43,309 views • 1 month ago

Using Grok Build to create the "brain" for hundreds of thousands of files over 20 years. Allows Grok to instantly retrieve, synthesize, and analyze specific documents on demand with all permissions respected. Enterprise private data layer enablement. Intelligence.

Using Grok Build to create the "brain" for hundreds of thousands of files over 20 years. Allows Grok to instantly retrieve, synthesize, and analyze specific documents on demand with all permissions respected. Enterprise private data layer enablement. Intelligence.

Jon Shulkin

16,555 views • 1 month ago

Tesla is extensively mapping Austin in preparation for Robotaxi. Bullish on TSLA! Saw 2 Model Y’s with Lidar mounted on the roof within 10 minutes from each other.

Tesla is extensively mapping Austin in preparation for Robotaxi. Bullish on TSLA! Saw 2 Model Y’s with Lidar mounted on the roof within 10 minutes from each other.

Nikola Brussels

201,441 views • 1 year ago

🏃‍♂️ Remote workers, save this: relieve lower back pain in minutes An osteopath trainer shared a simple routine for those with sedentary jobs. Remote workers, save this: — Support yourself on your hands, alternate stepping your legs with a torso twist — 3×20 each; — Knees bent, rotate your hips side to side — 3×20; — Lying on your back: lift your hips and alternate raising your legs — 3×20. 10 minutes — less tension, better blood flow.

🏃‍♂️ Remote workers, save this: relieve lower back pain in minutes An osteopath trainer shared a simple routine for those with sedentary jobs. Remote workers, save this: — Support yourself on your hands, alternate stepping your legs with a torso twist — 3×20 each; — Knees bent, rotate your hips side to side — 3×20; — Lying on your back: lift your hips and alternate raising your legs — 3×20. 10 minutes — less tension, better blood flow.

NEXTA

54,451 views • 3 months ago

80% BIG SALE PROMO! 10/20 STILL AVAILABLE SLOT AVAIL NOW!🥵 📍LIFETIME CHANNEL 📍PRIVATE CHANNEL AVAIL MY 2 CHANNEL FOR ONLY ₱500.00 PESOS WITH FREE VCS (5 MINUTES) WITH ALSO FACEREVEAL!🤤 TG: #alterpinay #alterph #alterworld #AlterManila

Sensitive content

80% BIG SALE PROMO! 10/20 STILL AVAILABLE SLOT AVAIL NOW!🥵 📍LIFETIME CHANNEL 📍PRIVATE CHANNEL AVAIL MY 2 CHANNEL FOR ONLY ₱500.00 PESOS WITH FREE VCS (5 MINUTES) WITH ALSO FACEREVEAL!🤤 TG: #alterpinay #alterph #alterworld #AlterManila

𝚂𝚎𝚡𝚢 𝙰𝚢𝚎𝚜𝚑𝚊🍑

14,587 views • 3 years ago

Diffusion models are an amazing tool for cofolding, they allow us to predict a protein and the molecule bound to it at once. But they are not exactly fast and require a lot of denoising steps to get accurate predictions. So we distilled ours. Meet DeCAF-Pearl: the first flow map model for all-atom cofolding. Instead of inching along the denoising trajectory, a flow map learns to jump across it. DeCAF-Pearl runs structure generation ~5x faster than Pearl, our SOTA model, while still maintaining the performance of the teacher model. That speed up allows us to run larger experiments and generate more synthetic data to improve our models. Getting there meant reparameterizing into noise-level space to stabilize gradients, committing to clean-structure prediction to keep the rigid-alignment loss biomolecules needed, and building DeCAF-Search, one steering algorithm for every compute budget. For more technical details, read out blog post: And the paper:

Diffusion models are an amazing tool for cofolding, they allow us to predict a protein and the molecule bound to it at once. But they are not exactly fast and require a lot of denoising steps to get accurate predictions. So we distilled ours. Meet DeCAF-Pearl: the first flow map model for all-atom cofolding. Instead of inching along the denoising trajectory, a flow map learns to jump across it. DeCAF-Pearl runs structure generation ~5x faster than Pearl, our SOTA model, while still maintaining the performance of the teacher model. That speed up allows us to run larger experiments and generate more synthetic data to improve our models. Getting there meant reparameterizing into noise-level space to stabilize gradients, committing to clean-structure prediction to keep the rigid-alignment loss biomolecules needed, and building DeCAF-Search, one steering algorithm for every compute budget. For more technical details, read out blog post: And the paper:

Sergey Edunov

36,985 views • 1 month ago

1 sol giveaway, 2 winners ($80 each) In 15 minutes This will be the last round, 🔔on Winners of the last giveaway DM me please👇 (used X picker for this one)

1 sol giveaway, 2 winners ($80 each) In 15 minutes This will be the last round, 🔔on Winners of the last giveaway DM me please👇 (used X picker for this one)

Mr B

20,971 views • 8 months ago

New research from Databricks: LLMs Can Learn to Reason via Off-Policy RL Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning. It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations. For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.

New research from Databricks: LLMs Can Learn to Reason via Off-Policy RL Optimal Advantage-based Policy Optimization with Lagged Inference policy (OAPL) shows you don’t need strict on-policy training to improve reasoning. It matches or beats Group Relative Policy Optimization (GRPO), stays stable with large policy lag, and uses ~3× fewer training generations. For Databricks customers, it’s a simpler, practical, and equally powerful approach to RL that Databricks is pioneering internally — and bringing directly to Databricks customers, so enterprises can improve agents using the same methods we use for our in-house agents, without complex infrastructure changes.

Databricks AI Research

12,633 views • 5 months ago

🐣: after living with each other for 5 years, we practically know everything about each other, which I’m proud of 🐣: I hope we stick together for 10–20 years BRING MY FAMILY BACK !!!

🐣: after living with each other for 5 years, we practically know everything about each other, which I’m proud of 🐣: I hope we stick together for 10–20 years BRING MY FAMILY BACK !!!

•・PG・• 🐆 fan account

73,251 views • 4 months ago

Data for Policy 2024 call for contributions has launched! Be part of the global conversation on trustworthy governance *with* AI - Full papers benefit from integrated review in dataandpolicy - Deadline 27 Nov All info #dataforpolicy2024

Data for Policy 2024 call for contributions has launched! Be part of the global conversation on trustworthy governance with AI - Full papers benefit from integrated review in dataandpolicy - Deadline 27 Nov All info #dataforpolicy2024

Data for Policy CIC #dataforpolicy2026

14,717 views • 2 years ago