Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Diffusion Policies and VLAs with Formal Safety Guarantees🛡️ We just released our new paper: “From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies” 📑Paper: 🌐Website:

Learning Systems and Robotics Lab (is hiring!)

3,003 subscribers

20,356 görüntüleme • 9 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Today, we are releasing Stable Video Diffusion, our first foundation model for generative AI video based on the image model, Stable Diffusion. As part of this research preview, the code, weights, and research paper are now available. Additionally, today you can sign up for our waitlist to access a new upcoming web experience featuring a Text-To-Video interface. To access the model & sign up for our waitlist, visit our website here:

Stability AI

1,024,682 görüntüleme • 2 yıl önce

What we are seeing from federal enforcement right now unacceptable. Oregon will continue to push back against policies that harm our neighbors and fight for justice, compassion, and safety.

What we are seeing from federal enforcement right now unacceptable. Oregon will continue to push back against policies that harm our neighbors and fight for justice, compassion, and safety.

Governor Tina Kotek

41,016 görüntüleme • 7 ay önce

Excited to release RT-Affordance! We propose conditioning policies on visual affordance plans as an intermediate representation that allows us to learn new tasks without collecting any new robot trajectories. Website and paper: Here’s a short 🧵

Excited to release RT-Affordance! We propose conditioning policies on visual affordance plans as an intermediate representation that allows us to learn new tasks without collecting any new robot trajectories. Website and paper: Here’s a short 🧵

Soroush Nasiriany

27,495 görüntüleme • 1 yıl önce

Sanctuary city policies don’t just undermine public safety, they also enable illegal aliens to defraud American taxpayers. We need cooperation at the federal, state, and local level to keep both our law enforcement and American citizens safe. Sanctuary cities should be defunded.

Sanctuary city policies don’t just undermine public safety, they also enable illegal aliens to defraud American taxpayers. We need cooperation at the federal, state, and local level to keep both our law enforcement and American citizens safe. Sanctuary cities should be defunded.

Senator Katie Boyd Britt

11,682 görüntüleme • 6 ay önce

Federal policies related to marriage and family have disincentivized both for generations. Our new family policy paper outlines a robust plan to eliminate those disincentives. Read it here: Roger Severino

Federal policies related to marriage and family have disincentivized both for generations. Our new family policy paper outlines a robust plan to eliminate those disincentives. Read it here: Roger Severino

Heritage Foundation

10,674 görüntüleme • 7 ay önce

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.

Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models paper page: Traditional 3D content creation tools empower users to bring their imagination to life by giving them direct control over a scene's geometry, appearance, motion, and camera path. Creating computer-generated videos, however, is a tedious manual process, which can be automated by emerging text-to-video diffusion models. Despite great promise, video diffusion models are difficult to control, hindering a user to apply their own creativity rather than amplifying it. To address this challenge, we present a novel approach that combines the controllability of dynamic 3D meshes with the expressivity and editability of emerging diffusion models. For this purpose, our approach takes an animated, low-fidelity rendered mesh as input and injects the ground truth correspondence information obtained from the dynamic mesh into various stages of a pre-trained text-to-image generation model to output high-quality and temporally consistent frames. We demonstrate our approach on various examples where motion can be obtained by animating rigged assets or changing the camera path.

AK

117,066 görüntüleme • 2 yıl önce

Restore Britain has just released our energy policy paper outlining how we can make energy cheap, reliable and abundant. This is a key step on the path to restoring Britain.

Restore Britain has just released our energy policy paper outlining how we can make energy cheap, reliable and abundant. This is a key step on the path to restoring Britain.

Rupert Lowe MP

118,737 görüntüleme • 3 ay önce

I am running for re-election as Attorney General. As Attorney General, I’ve deployed every tool available to keep Virginians safe and violent criminals off our streets. We are not going back to the failed policies of the past; our families' safety and Virginia's future depends on it.

I am running for re-election as Attorney General. As Attorney General, I’ve deployed every tool available to keep Virginians safe and violent criminals off our streets. We are not going back to the failed policies of the past; our families' safety and Virginia's future depends on it.

Jason Miyares

333,498 görüntüleme • 1 yıl önce

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:

Heng Yang

19,166 görüntüleme • 5 ay önce

🌍Are LLMs aware of cultural and legal safety in today’s geo-diverse world? 🚀Introducing SafeWorld, our #NeurIPS2024 paper and benchmark assessing LLMs’ understanding of geo-diverse safety, based on cultural norms and policies across 50 countries and 493 regions/races. ⚖️We also propose a multi-dimensional framework for evaluating contextual appropriateness, accuracy, and comprehensiveness, revealing major gaps in current LLMs. 🧨To address this, we train SafeWorldLM using DPO, achieving SOTA performance and a 20% higher global human evaluator rating in helpfulness and harmfulness over competing models, including GPT-4o. 🔗Paper: 💻 GitHub: 🫶🏻This is a joint leading effort with Da Yin. Also many thanks to the amazing team Kung-Hsiang Steeve Huang Kai-Wei Chang, and Violet Peng for their hard work. Check out more details and results we conclude from our paper in the thread below. 🧵

🌍Are LLMs aware of cultural and legal safety in today’s geo-diverse world? 🚀Introducing SafeWorld, our #NeurIPS2024 paper and benchmark assessing LLMs’ understanding of geo-diverse safety, based on cultural norms and policies across 50 countries and 493 regions/races. ⚖️We also propose a multi-dimensional framework for evaluating contextual appropriateness, accuracy, and comprehensiveness, revealing major gaps in current LLMs. 🧨To address this, we train SafeWorldLM using DPO, achieving SOTA performance and a 20% higher global human evaluator rating in helpfulness and harmfulness over competing models, including GPT-4o. 🔗Paper: 💻 GitHub: 🫶🏻This is a joint leading effort with Da Yin. Also many thanks to the amazing team Kung-Hsiang Steeve Huang Kai-Wei Chang, and Violet Peng for their hard work. Check out more details and results we conclude from our paper in the thread below. 🧵

Haoyi Qiu

16,344 görüntüleme • 1 yıl önce

📣Big news! We just released a new capture mode that lets you take 360 photos with just your iPhone for free. The main stitching happens on-device in ~15 s, and then we complete the sphere with stable diffusion🌎

📣Big news! We just released a new capture mode that lets you take 360 photos with just your iPhone for free. The main stitching happens on-device in ~15 s, and then we complete the sphere with stable diffusion🌎

polycam

261,441 görüntüleme • 3 yıl önce

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 görüntüleme • 3 yıl önce

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

AK

294,519 görüntüleme • 3 yıl önce

🚨Today marks three years since my life was forever altered by the murder of my innocent teenage brother, Christian, in Chicago. We launched the Caldwell Institute for Public Safety to challenge broken policies and restore common sense to public safety. Woke, soft-on-crime policies put everyday people at risk. When law enforcement is defunded or tied up in bureaucracy, crime wins. This year, we pushed back—with research, legislation, and support for leaders who prioritize safety, training, and accountability. This video captures what we've accomplished—and where we’re headed. Because safety is not a luxury. It’s the foundation of a thriving community. Thank you for supporting our pursuit of safer communities. Watch our one-year recap and explore to see our progress. @caldwellinstitute

🚨Today marks three years since my life was forever altered by the murder of my innocent teenage brother, Christian, in Chicago. We launched the Caldwell Institute for Public Safety to challenge broken policies and restore common sense to public safety. Woke, soft-on-crime policies put everyday people at risk. When law enforcement is defunded or tied up in bureaucracy, crime wins. This year, we pushed back—with research, legislation, and support for leaders who prioritize safety, training, and accountability. This video captures what we've accomplished—and where we’re headed. Because safety is not a luxury. It’s the foundation of a thriving community. Thank you for supporting our pursuit of safer communities. Watch our one-year recap and explore to see our progress. @caldwellinstitute

Gianno Caldwell

101,175 görüntüleme • 1 yıl önce

Our nation’s safety is crumbling faster than ever before. This makes it clear that we have zero control over our borders and our streets! Britain must be taken back with bold, intelligent, just security policies. Only BRUV has the right plan to deliver it! Check our charter: 🇬🇧

Our nation’s safety is crumbling faster than ever before. This makes it clear that we have zero control over our borders and our streets! Britain must be taken back with bold, intelligent, just security policies. Only BRUV has the right plan to deliver it! Check our charter: 🇬🇧

BRUV

35,181 görüntüleme • 7 ay önce

We have released Draft Version 1 of our ICPX paper. As we mature and reach a milestone of launching the full web application, it is time to start releasing the first draft of our paper and roadmap. Read below 👇 Our website will have a small overhaul too, with updated information and interactive roadmap. A Universal Compute Orchestration Protocol

We have released Draft Version 1 of our ICPX paper. As we mature and reach a milestone of launching the full web application, it is time to start releasing the first draft of our paper and roadmap. Read below 👇 Our website will have a small overhaul too, with updated information and interactive roadmap. A Universal Compute Orchestration Protocol

ICPX Protocol

11,521 görüntüleme • 2 ay önce

🔥 Introducing MVLift: Generate realistic 3D motion without any 3D training data - just using 2D poses from monocular videos! Applicable to human motion, human-object interaction & animal motion. Joint work w/ Jiajun Wu & Karen 💡 How? We reformulate 3D motion estimation as generating consistent multi-view 2D pose sequences. Our framework uses 2D motion diffusion to progressively establish multi-view consistency, requiring only single-view 2D pose sequences for training. Project: Video with demonstration: Paper:

🔥 Introducing MVLift: Generate realistic 3D motion without any 3D training data - just using 2D poses from monocular videos! Applicable to human motion, human-object interaction & animal motion. Joint work w/ Jiajun Wu & Karen 💡 How? We reformulate 3D motion estimation as generating consistent multi-view 2D pose sequences. Our framework uses 2D motion diffusion to progressively establish multi-view consistency, requiring only single-view 2D pose sequences for training. Project: Video with demonstration: Paper:

Jiaman Li

15,886 görüntüleme • 1 yıl önce