Mihir Prabhudesai's banner

Mihir Prabhudesai

@mihirp98 • 2,761 subscribers

PhD student at Carnegie Mellon

Shorts

1/ Happy to share UniDisc - Unified Multimodal Discrete Diffusion – We train a 1.5 billion parameter transformer model from scratch on 250 million image/caption pairs using a **discrete diffusion objective**. Our model has all the benefits of diffusion models but now in multimodal space! - flexible compute-quality tradeoff, zero-shot inpainting and editing, better control via classifier-free guidance and lower latency! We open source everything - our code, weights and the training dataset.

1/ Happy to share UniDisc - Unified Multimodal Discrete Diffusion – We train a 1.5 billion parameter transformer model from scratch on 250 million image/caption pairs using a discrete diffusion objective. Our model has all the benefits of diffusion models but now in multimodal space! - flexible compute-quality tradeoff, zero-shot inpainting and editing, better control via classifier-free guidance and lower latency! We open source everything - our code, weights and the training dataset.

104,934 views

1/ Happy to share VADER: Video Diffusion Alignment via Reward Gradients. We adapt foundational video diffusion models using pre-trained reward models to generate high-quality, aligned videos for various end-applications. Below we generated a short movie using VADER 😀, we used ChatGPT to write a script and an off-the-shelf AI music generator to generate the sound. Our code & weights are open-sourced:

1/ Happy to share VADER: Video Diffusion Alignment via Reward Gradients. We adapt foundational video diffusion models using pre-trained reward models to generate high-quality, aligned videos for various end-applications. Below we generated a short movie using VADER 😀, we used ChatGPT to write a script and an off-the-shelf AI music generator to generate the sound. Our code & weights are open-sourced:

13,403 views

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

What if AI learned physics the way Newton did – by experiencing it? We built Sim2Reason: train LLMs inside virtual worlds governed by real physics laws, zero human annotation. Result: +5–10% improvement on International Physics Olympiad, zero-shot. 🧵

What if AI learned physics the way Newton did – by experiencing it? We built Sim2Reason: train LLMs inside virtual worlds governed by real physics laws, zero human annotation. Result: +5–10% improvement on International Physics Olympiad, zero-shot. 🧵

Mihir Prabhudesai

248,746 views • 4 months ago

Images, video, audio, actions — generative modeling has converged on one recipe: compress into continuous latents, generate in latent space, decode. Language is the lone exception: still generated token by token, as a long discrete stream. Latent Thought Flows: We compress 256 text tokens into 8 continuous latents, generate them with a one-step flow model, and read them out as text using an autoregressive decoder. Result: a better inference-compute vs. generation-quality Pareto frontier than a tuned autoregressive baseline — thanks to compression and one-step generation. Co-led w/ Zhengyang Geng 🧵

Images, video, audio, actions — generative modeling has converged on one recipe: compress into continuous latents, generate in latent space, decode. Language is the lone exception: still generated token by token, as a long discrete stream. Latent Thought Flows: We compress 256 text tokens into 8 continuous latents, generate them with a one-step flow model, and read them out as text using an autoregressive decoder. Result: a better inference-compute vs. generation-quality Pareto frontier than a tuned autoregressive baseline — thanks to compression and one-step generation. Co-led w/ Zhengyang Geng 🧵

Mihir Prabhudesai

81,598 views • 1 month ago

No more content to load