Humphrey Shi
@humphrey_shi • 4,591 subscribers
Vice President @NVIDIA · Professor @GeorgiaTech | Engineer–Researcher building next-generation high-performance, multimodal, and creative AI systems
Videos
While frontier closed models like Google’s Nano Banana can autonomously produce rich interleaved content (e.g., illustrated tutorials), open-source models still lag in both task coverage and generation quality. We introduce DuoGen, a dual transformer–diffusion framework that narrows this gap via an efficient decoupled design: a pretrained Multimodal LLM performs semantic reasoning and decides when to generate images, while a Video Diffusion Transformer ensures high-fidelity, consistent visuals—without costly mixed-modality pretraining. Enabled by a new large-scale interleaved instruction-tuning dataset. Code & data will be open-sourced. Paper: Project: Work led by Min Shi 🐝 originating from his summer internship at NVIDIA Research and continuing beyond, in collaboration with Ming-Yu Liu, Xiaohui Zeng, Jiannan Huang, Yin Cui, Jialuo Li, Tsung-Yi Lin, Max Li 李赵硕, Francesco Ferroni, Xiao Fu, Yogesh Balaji, Chieh-Yun Chen, and other colleagues 🚀
Humphrey Shi16,692 просмотров • 4 месяцев назад
Over 4 years into our journey bridging Convolutions and Transformers, we introduce Generalized Neighborhood Attention—Multi-dimensional Sparse Attention at the Speed of Light: A collaboration with the best minds in AI and HPC. 🐝🟩🟧 Georgia Tech Computing NVIDIA
Humphrey Shi29,110 просмотров • 1 год назад
Больше нет контента для загрузки