Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

๐Ÿ“Š VBench: Comprehensive Benchmark Suite for Video Generative Models ๐ŸŽž๏ธ ๐Ÿ›๏ธ Hierarchical and Disentangled Dimensions ๐Ÿ‘๏ธ Human-Alignment in Each Dimension ๐ŸŒŸ Valuable and Multi-Perspective Insights Website: Code:

19,008 Aufrufe โ€ข vor 2 Jahren โ€ขvia X (Twitter)

1 Kommentare

Profilbild von Tica Lin
Tica Linvor 2 Jahren

Hi Ziqi, great work! Since I couldn't find details in your paper, I am curious how did you recruit annotators, and how many annotators and tasks each complete? Thanks!

ร„hnliche Videos

DisCo: Disentangled Control for Referring Human Dance Generation in Real World paper page: Generative AI has made significant strides in computer vision, particularly in image/video synthesis conditioned on text descriptions. Despite the advancements, it remains challenging especially in the generation of human-centric content such as dance synthesis. Existing dance synthesis methods struggle with the gap between synthesized content and real-world dance scenarios. In this paper, we define a new problem setting: Referring Human Dance Generation, which focuses on real-world dance scenarios with three important properties: (i) Faithfulness: the synthesis should retain the appearance of both human subject foreground and background from the reference image, and precisely follow the target pose; (ii) Generalizability: the model should generalize to unseen human subjects, backgrounds, and poses; (iii) Compositionality: it should allow for composition of seen/unseen subjects, backgrounds, and poses from different sources. To address these challenges, we introduce a novel approach, DISCO, which includes a novel model architecture with disentangled control to improve the faithfulness and compositionality of dance synthesis, and an effective human attribute pre-training for better generalizability to unseen humans. Extensive qualitative and quantitative results demonstrate that DISCO can generate high-quality human dance images and videos with diverse appearances and flexible motions.

AK

161,478 Aufrufe โ€ข vor 3 Jahren

Today is a good day for open science. As part of our continued commitment to the growth and development of an open ecosystem, today at Meta FAIR weโ€™re announcing four new publicly available AI models and additional research artifacts to inspire innovation in the community and help advance AI in a responsible way. More in the video from Joelle Pineau. What weโ€™re releasing: ๐ŸฆŽ Meta Chameleon 7B & 34B language models that support mixed-modal input and text-only outputs. ๐Ÿช™ Meta Multi-Token Prediction Pretrained Language Models for code completion using Multi-Token Prediction. ๐ŸŽผ Meta JASCO Generative text-to-music models capable of accepting various conditioning inputs for greater controllability. Paper available today with a pretrained model coming soon. ๐Ÿ—ฃ๏ธ Meta AudioSeal An audio watermarking model that we believe is the first designed specifically for the localized detection of AI-generated speech, available under a commercial license. ๐Ÿ“ Additional RAI artifacts Including research, data and code to measure and improve the representation of geographical and cultural preferences and diversity in AI systems. We believe that access to state-of-the-art AI creates opportunities for everyone โ€“ not just a small handful of Big Tech companies. Weโ€™re excited to share this work and to see how the community learns, iterates and builds using this technology. Details and access to everything released by FAIR today โžก๏ธ

AI at Meta

380,822 Aufrufe โ€ข vor 2 Jahren

๐Ÿ”ฌ Exciting News! Our manuscript, "scGPT: toward building a foundation model for single-cell multi-omics using generative AI" is now finally published in Nature Methods (Nature Methods) ๐ŸŽ‰ !!! (Re-)Introducing scGPT: A transformative foundation model engineered for single-cell omics analysis. Developed through the analysis of over 33 million human cells, scGPT sets a new benchmark for application versatility, offering both fine-tuning and zero-shot capabilities. Since its preprint in May 2023, scGPT has significantly impacted the field, evidenced by 13K+ installations, 600+ GitHub stars ๐ŸŒŸ, and 40+ citations before its official publication! scGPT has been validated by numerous benchmark studies as a leading foundation model in single-cell analysis. Its pre-trained embeddings extend its utility beyond single-cell studies, enhancing a variety of downstream tasks including protein enrichment and genetic perturbation predictions. Some key updates lately: ---Expanded zero-shot applications for efficient reference mapping and integration, now with CellXGene census integration. ---Advanced perturbation analysis capabilities, including genome-scale perturb-seq data analysis and bulk sequencing data generalization. ---Upgraded scGPT package, offering versatile model loading compatible with PyTorch and flash-attn, for both GPU and CPU. ---Cloud-based scGPT applications for reference mapping, cell annotation, and gene regulatory network inference are available on ---Integration with Hugging Face for easier model training. Limitations: scGPT is an early foray into foundation models for single-cell omics, facing challenges like limited zero-shot learning in some tasks, pretraining constraints, data quality issues, and evaluation limitations. See our Supplementary Notes for details. ๐Ÿš€ Future Work? Short-Term Goals: 1. Releasing a Mouse Model for broader analysis. 2. Developing a comprehensive evaluation suite for foundation models in single-cell analysis. 3. Creating a foundation model for single-cell spatial omics. 4. Enhancing zero-shot capacity by integrating scGPT with RAG (e.g., knowledge graphs). Long-Term Goals: 1. Expanding scGPT for comprehensive single-cell multi-omics analysis. 2. Developing an in-silico perturbation model for predicting genetic perturbation effects. 3. Merging scGPT with multi-modal genomic sequence models for a deeper understanding of cell biology. ๐Ÿ“š Access the paper on Nature Methods: ๐Ÿ”ฌPreprint in Bioarixv: ๐Ÿ’ป All our codes/data/weights are open source: Wholehearted congratulations to all the authors, especially the two co-first authors, Haotian (Haotian Cui ) and Chloe (ChloeXWang), who are really the emerging superstars in AI and biology! Vector Institute Peter Munk Cardiac Centre AI U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology University Health Network University of Toronto #scGPT #GenerativeAI #AI4Science #Combio #opensource

Bo Wang

199,718 Aufrufe โ€ข vor 2 Jahren