Yu Su

@ysu_nlp • 14,285 subscribers

co-founder @NeoCognition | prof. @osunlp | sloan fellow | building towards abundance of specialized intelligence

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Introducing NeoCognition, the agent lab for specialized intelligence. Everyone needs experts, but human expertise does not scale. Backed by $40M seed funding, we build self-learning agents that specialize across domains to make expertise abundant.

Introducing NeoCognition, the agent lab for specialized intelligence. Everyone needs experts, but human expertise does not scale. Backed by $40M seed funding, we build self-learning agents that specialize across domains to make expertise abundant.

187,461 次观看 • 1 个月前

Introducing BioCLIP: A Vision Foundation Model for the Tree of Life A foundation model that strongly generalizes on the tree of life (2M+ species), outperforming OpenAI CLIP by 18% in zero-shot classification, and supports open-ended classification over almost the entire tree of life What's the secrete ingredients? > Data: we curate and release TreeOfLife-10M, the largest and most diverse ML-ready dataset of organism images to date. It contains 10.4M images for over 450K taxa, sourced from iNaturalist, BIOSCAN, and Encyclopedia of Life. > Modeling: we creatively repurposes CLIP's multimodal contrastive learning objective for hierarchical image classification. The autoregressive language model naturally encodes the hierarchy of the tree of life taxonomy, which in turn bakes the hierarchical representation into the vision transformer encoder. Key results > Strong zero/few-shot classification for animals/plants/fungi, including rare species, outperforming CLIP by avg 16-18% absolute. > T-sne visualization shows that BioCLIP's vision encoder has captued the fine-grained hierarchical structure of the tree of life > BioCLIP is a kind of universal classifier for the tree of life. Just give it an organism image and it will likely find the correct species (among top 5)! But use it with caution; it's not perfect yet.. Final remarks > AI for Science is really hard but extremely rewarding! It took us a ton of time (1+ year) and frustration trying to find a plausible way to integrate the tree of life taxonomy into foundation model training. But when the "Eureka!" moment came and the idea hit us (by the great Wei-Lun Chao) that CLIP's multimodal contrastive learning objective can be repurposed for that, everything just follows naturally. It was truly a moment of joy and excitement! > BioCLIP is our first attempt at foundation models for biology, but it certainly won't be the last! There's so much more to do at the intersection of one of the oldest scientific disciplines and the young but thriving field of AI. Biological intelligence is the foundation for artificial intelligence, and artificial intelligence will in turn become the most important tool for us to unraval the mysteries of biological intelligence. We are hiring postdocs and PhDs in the NSF Imageomics Institute institute to explore this exciting field! Drop us an email. also happy to chat about it at #NeurIPS2023 with any of Tanya, Wei-Lun Chao, or me. - paper: - project: - demo: - model: - data (TreeOfLife-10M): to be released on Hugging Face soon joint work with the amazing Imageomics Institute team: @samstevens6860 Lisa Wu, Matt Thompson, Elizabeth Campolongo Chan Hee (Luke) Song @CVPR2026 David Carlyn Li Dong Wasila Dahdul Chuck Stewart, Tanya Berger-Wolf Wei-Lun Chao Yu Su

Introducing BioCLIP: A Vision Foundation Model for the Tree of Life A foundation model that strongly generalizes on the tree of life (2M+ species), outperforming OpenAI CLIP by 18% in zero-shot classification, and supports open-ended classification over almost the entire tree of life What's the secrete ingredients? > Data: we curate and release TreeOfLife-10M, the largest and most diverse ML-ready dataset of organism images to date. It contains 10.4M images for over 450K taxa, sourced from iNaturalist, BIOSCAN, and Encyclopedia of Life. > Modeling: we creatively repurposes CLIP's multimodal contrastive learning objective for hierarchical image classification. The autoregressive language model naturally encodes the hierarchy of the tree of life taxonomy, which in turn bakes the hierarchical representation into the vision transformer encoder. Key results > Strong zero/few-shot classification for animals/plants/fungi, including rare species, outperforming CLIP by avg 16-18% absolute. > T-sne visualization shows that BioCLIP's vision encoder has captued the fine-grained hierarchical structure of the tree of life > BioCLIP is a kind of universal classifier for the tree of life. Just give it an organism image and it will likely find the correct species (among top 5)! But use it with caution; it's not perfect yet.. Final remarks > AI for Science is really hard but extremely rewarding! It took us a ton of time (1+ year) and frustration trying to find a plausible way to integrate the tree of life taxonomy into foundation model training. But when the "Eureka!" moment came and the idea hit us (by the great Wei-Lun Chao) that CLIP's multimodal contrastive learning objective can be repurposed for that, everything just follows naturally. It was truly a moment of joy and excitement! > BioCLIP is our first attempt at foundation models for biology, but it certainly won't be the last! There's so much more to do at the intersection of one of the oldest scientific disciplines and the young but thriving field of AI. Biological intelligence is the foundation for artificial intelligence, and artificial intelligence will in turn become the most important tool for us to unraval the mysteries of biological intelligence. We are hiring postdocs and PhDs in the NSF Imageomics Institute institute to explore this exciting field! Drop us an email. also happy to chat about it at #NeurIPS2023 with any of Tanya, Wei-Lun Chao, or me. - paper: - project: - demo: - model: - data (TreeOfLife-10M): to be released on Hugging Face soon joint work with the amazing Imageomics Institute team: @samstevens6860 Lisa Wu, Matt Thompson, Elizabeth Campolongo Chan Hee (Luke) Song @CVPR2026 David Carlyn Li Dong Wasila Dahdul Chuck Stewart, Tanya Berger-Wolf Wei-Lun Chao Yu Su

80,637 次观看 • 2 年前

没有更多内容可加载