Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

If you have a policy that uses diffusion/flow (e.g. diffusion VLA), you can run RL where the actor chooses the noise, which is then denoised by the policy to produce an action. This method, which we call diffusion steering (DSRL), leads to a remarkably efficient RL method! 🧵👇

152,824 Aufrufe • vor 1 Jahr •via X (Twitter)

9 Kommentare

Profilbild von Sergey Levine
Sergey Levinevor 1 Jahr

DSRL trains an actor and Q-function, treating the diffusion noise as the action space. Because samples from the noise prior map to reasonable actions for the policy, DSRL essentially explores "inside" the set of reasonable pre-trained behaviors, making it extremely efficient.

Profilbild von Sergey Levine
Sergey Levinevor 1 Jahr

DSRL learns essentially in real time, with good results in as little as 50 trials (it's so efficient that a person can literally sit in front of the robot and push a button to assign sparse rewards).

Profilbild von Sergey Levine
Sergey Levinevor 1 Jahr

This was a really fun collaboration led by @ajwagenmaker Project website with paper: To find out more, check out his thread here:

Profilbild von ahad
ahadvor 1 Jahr

would a supervised learning version of this work? where the noise distribution is a parmeter that is also optimized along with policy weights

Profilbild von ahad
ahadvor 1 Jahr

how long would it take to get that first sparse reward with this method?

Profilbild von Himanshu Kumar
Himanshu Kumarvor 1 Jahr

Controlling the noise instead of the action itself is a surprisingly effective approach.

Profilbild von Andres Franco
Andres Francovor 1 Jahr

This is pretty amazing, and the visualization made everything so easy to understand😅

Profilbild von Ran Cheng
Ran Chengvor 1 Jahr

Will making the initial noise distribution a learnable parameter reduce randomness and thus make the model more prone to overfitting?

Profilbild von Joanne Mercado
Joanne Mercadovor 1 Jahr

😅🥹

Ähnliche Videos

DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior paper page: present DreamCraft3D, a hierarchical 3D content generation method that produces high-fidelity and coherent 3D objects. We tackle the problem by leveraging a 2D reference image to guide the stages of geometry sculpting and texture boosting. A central focus of this work is to address the consistency issue that existing works encounter. To sculpt geometries that render coherently, we perform score distillation sampling via a view-dependent diffusion model. This 3D prior, alongside several training strategies, prioritizes the geometry consistency but compromises the texture fidelity. We further propose Bootstrapped Score Distillation to specifically boost the texture. We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized. The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene. Notably, through an alternating optimization of the diffusion prior and 3D scene representation, we achieve mutually reinforcing improvements: the optimized 3D scene aids in training the scene-specific diffusion model, which offers increasingly view-consistent guidance for 3D optimization. The optimization is thus bootstrapped and leads to substantial texture boosting. With tailored 3D priors throughout the hierarchical generation, DreamCraft3D generates coherent 3D objects with photorealistic renderings, advancing the state-of-the-art in 3D content generation.

AK

161,530 Aufrufe • vor 2 Jahren