Video yükleniyor...
Video Yüklenemedi
Excited to release METAGENE-1, a 7B parameter metagenomic foundation model, built to aid in pathogen detection & pandemic monitoring. Pretrained on 1.5 trillion base pairs of DNA/RNA sequenced from wastewater. A collab w/ USC, Prime Intellect, & the Nucleic Acid Observatory. 🧵
13,390 görüntüleme • 1 yıl önce •via X (Twitter)
12 Yorum

Metagenomic sequencing of wastewater produces vast amounts of data that can capture public health trends at a societal scale. Our goal is to train a model on this data to help in large-scale wastewater monitoring & detection of novel bio threats. 🌐

Our data pipeline is: human microbiome > wastewater > metagenomic sequences > tokens > training data. Wastewater provides a rich source of data from tens of thousands of species across the human-adjacent microbiome. In total we pretrain on over 1.5T base pairs of DNA/RNA.

METAGENE-1 shows state-of-the-art results on pathogen detection, metagenomic embedding, and other genomic tasks. We also release new benchmarks for genomic detection and embedding (eg, Gene-MTEB, based on MTEB for LLMs). See our paper for details:

We leverage the ecosystem of modern LLM tooling—in tokenization, model architecture, training, infra, etc—for performance and extensibility. METAGENE-1 is standardized & easy to use. Hugging Face: Github:

Our paper also contains an in-depth discussion on safety when releasing metagenomic models. s/o to great coauthors @olliezliu @samsja19 @johannes_hage @UpupWang @_jason136, & jefftk Looking for collaborators to build on this with us — please reach out!

RT @InstaMAT_io: 👀 Are you looking for an alternative to Adobe Substance tools? Do not miss out on InstaMAT from killer node-based material…

@PrimeIntellect Exciting work! For next steps, are you thinking about how this would compare to classic bioinformatics workflows (e.g. for pathogen detection? It could also be relevant to compare to the @tatta_bio gLM model and DGEB benchmark.

@PrimeIntellect @tatta_bio Thanks for the links! Yes, these are great ideas for next steps.

@USC @PrimeIntellect Wow, awesome work @willieneis and team! Congrats on the release 🎉

@USC @PrimeIntellect Thanks Alon!

@USC @PrimeIntellect Congrats Willie! Looks very cool.

@USC @PrimeIntellect Thanks Ben!
