Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Evo 2, our fully open-source biological foundation model trained on trillions of DNA tokens spanning the entire tree of life, is out in nature today We & the scientific community have done a lot with this Arc Institute NVIDIA model in the last year! 🧵👇

Patrick Hsu

86,211 subscribers

250,801 Aufrufe • vor 6 Monaten •via X (Twitter)

Gesundheit & Wellness Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Evo 2, the largest fully open biological AI model to date, is now published in nature.

Evo 2, the largest fully open biological AI model to date, is now published in nature.

Arc Institute

68,978 Aufrufe • vor 6 Monaten

1/n: We are excited to share that our paper on Chroma, a general purpose diffusion model for proteins, is out today in nature! A couple of my favorite highlights in the 🧵below 👇

1/n: We are excited to share that our paper on Chroma, a general purpose diffusion model for proteins, is out today in nature! A couple of my favorite highlights in the 🧵below 👇

Andrew Beam

362,107 Aufrufe • vor 2 Jahren

MolmoAct 2 artifacts have been downloaded 400K+ times in under 1 month. Today we're opening up the full code & training data: everything you need to fine-tune or build on our fully open robotics foundation model. 🧵

MolmoAct 2 artifacts have been downloaded 400K+ times in under 1 month. Today we're opening up the full code & training data: everything you need to fine-tune or build on our fully open robotics foundation model. 🧵

Ai2

17,870 Aufrufe • vor 3 Monaten

We are releasing Carbon: a crazy fast DNA model Carbon is 275x faster than the next best model. So fast you can process the whole human genome on a single GPU in <2 days. Here are the tricks we used: When modelling DNA sequences a lot of the performance comes down to tokenizing the sequences in a smart way. BPE tokenizer struggle because there are no whitespaces and character (called base in DNA) level tokenizers waste a lot of compute on too many tokens. Carbon is built with a unique tokenizer: we split sequences in chunks of 6 bases, but during both training and inference we can work with single base resolution. That's similar to having word tokens but resolving them at the character level. All possible thanks to the DNA tokens unique structure. The architecture combined with the tokenizer makes the model 275x faster than the previous SoTA (Evo2) at this size. We built an interactive demo so you can explore how the model can generate DNA sequences, investigate the structure of genes, predict the effect of mutations, generate and fold proteins and even reconstruct parts of the tree of life.

We are releasing Carbon: a crazy fast DNA model Carbon is 275x faster than the next best model. So fast you can process the whole human genome on a single GPU in <2 days. Here are the tricks we used: When modelling DNA sequences a lot of the performance comes down to tokenizing the sequences in a smart way. BPE tokenizer struggle because there are no whitespaces and character (called base in DNA) level tokenizers waste a lot of compute on too many tokens. Carbon is built with a unique tokenizer: we split sequences in chunks of 6 bases, but during both training and inference we can work with single base resolution. That's similar to having word tokens but resolving them at the character level. All possible thanks to the DNA tokens unique structure. The architecture combined with the tokenizer makes the model 275x faster than the previous SoTA (Evo2) at this size. We built an interactive demo so you can explore how the model can generate DNA sequences, investigate the structure of genes, predict the effect of mutations, generate and fold proteins and even reconstruct parts of the tree of life.

Leandro von Werra

405,615 Aufrufe • vor 3 Monaten

Earlier this week at GTC, we announced our partnership with Nvidia. We will work with Nvidia to build strong, American open-source models that are at the frontier of scientific reasoning. These models will be essential for the US to compete with China on science in the coming decades. Jensen is committing to spend tens of billions of dollars developing open-source models, and we are excited to be a partner with them in figuring out how to benchmark, train and use those agents to accelerate scientific research. We have already open-sourced some of the work we have done with them, and are looking forward to open-sourcing more. There are few things today that are more important. See our blog post below, and watch the video to learn more, narrated by the man himself.

Earlier this week at GTC, we announced our partnership with Nvidia. We will work with Nvidia to build strong, American open-source models that are at the frontier of scientific reasoning. These models will be essential for the US to compete with China on science in the coming decades. Jensen is committing to spend tens of billions of dollars developing open-source models, and we are excited to be a partner with them in figuring out how to benchmark, train and use those agents to accelerate scientific research. We have already open-sourced some of the work we have done with them, and are looking forward to open-sourcing more. There are few things today that are more important. See our blog post below, and watch the video to learn more, narrated by the man himself.

Sam Rodriques

23,763 Aufrufe • vor 5 Monaten

Today in nature, we report MouseMapper: foundation-model AI to map disease perturbations across the entire mouse body cell-by-cell. In obesity, it revealed body-wide inflammation & unexpected facial nerve damage. 🧵👇🔉 led by Doris Kaltenecker & Ying Chen

Today in nature, we report MouseMapper: foundation-model AI to map disease perturbations across the entire mouse body cell-by-cell. In obesity, it revealed body-wide inflammation & unexpected facial nerve damage. 🧵👇🔉 led by Doris Kaltenecker & Ying Chen

Ali Max Erturk

262,769 Aufrufe • vor 3 Monaten

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

The All-In Podcast

79,718 Aufrufe • vor 1 Jahr

Jensen on Nemotron 3 Ultra and it's role in the open ecosystem. It's great to have Nvidia leading on this, with so much turnover in open model providers in recent years. Super excited to see how the model turns out.

Jensen on Nemotron 3 Ultra and it's role in the open ecosystem. It's great to have Nvidia leading on this, with so much turnover in open model providers in recent years. Super excited to see how the model turns out.

Nathan Lambert

26,650 Aufrufe • vor 5 Monaten

After operating in stealth for the last 18 months , we’re excited today to finally show the world what we’ve been working on. We believe we’re on a path to physical AGI with the launch of our brand new foundation model, the Direct Video Action (DVA) model.

Jagdeep Singh

270,776 Aufrufe • vor 5 Monaten

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥

Chaz Wargnier ♨️

565,905 Aufrufe • vor 7 Monaten

We are now Cult.Repo. The original creators behind the iconic Open Source documentaries (Vue, React, GraphQL) are back together! Honeypot is now called Cult.Repo and is independently-owned. We are on a mission to document the complete history of every major open source language ever created. With 250K subscribers and over 15M views, we're building on our foundation as creators of the acclaimed Open Source Origin Story documentary format. Coming in 2025: Python Software Foundation, Vite ⚡ documentaries and much more. Our independence is core to our mission. We believe the human stories behind open source deserve authentic, in-depth storytelling. Our new model is powered directly by: - Community support from viewers - Sponsorships from organizations that champion open source This transition represents our commitment to independent, meaningful tech storytelling that honours the collaborative spirit of open source itself.

We are now Cult.Repo. The original creators behind the iconic Open Source documentaries (Vue, React, GraphQL) are back together! Honeypot is now called Cult.Repo and is independently-owned. We are on a mission to document the complete history of every major open source language ever created. With 250K subscribers and over 15M views, we're building on our foundation as creators of the acclaimed Open Source Origin Story documentary format. Coming in 2025: Python Software Foundation, Vite ⚡ documentaries and much more. Our independence is core to our mission. We believe the human stories behind open source deserve authentic, in-depth storytelling. Our new model is powered directly by: - Community support from viewers - Sponsorships from organizations that champion open source This transition represents our commitment to independent, meaningful tech storytelling that honours the collaborative spirit of open source itself.

Cult.Repo

71,390 Aufrufe • vor 1 Jahr

1/ We are releasing Playground v2.5, our latest foundation model to create images. We tested our model across 20K+ users in a rigorous benchmark that went beyond anything we've seen to date. This model is open weights. More information in the tweets below. 👇

1/ We are releasing Playground v2.5, our latest foundation model to create images. We tested our model across 20K+ users in a rigorous benchmark that went beyond anything we've seen to date. This model is open weights. More information in the tweets below. 👇

Suhail

288,552 Aufrufe • vor 2 Jahren

The first truly open-source audio-video model. LTX-2 is a DiT-based foundation model with all core video generation capabilities in one unified model. Designed to run locally on consumer GPUs. - text-to-video - image-to-video - and video-to-video modes 100% open-source.

The first truly open-source audio-video model. LTX-2 is a DiT-based foundation model with all core video generation capabilities in one unified model. Designed to run locally on consumer GPUs. - text-to-video - image-to-video - and video-to-video modes 100% open-source.

Akshay 🚀

66,197 Aufrufe • vor 8 Monaten

Red Hat AI just shipped DFlash speculator checkpoints for two of NVIDIA AI's most powerful open models: → Nemotron Ultra 550B → Nemotron Super 120B On math and reasoning: ~5 out of 7 draft tokens accepted on average. On code (HumanEval): ~3.4 out of 7. Both checkpoints trained with the open source Speculators library from vLLM. Apache 2.0. Validated on NVIDIA B200. One flag to enable in vLLM: --spec-model RedHatAI/NVIDIA-Nemotron-3-Ultra-550B-A55B-speculator.dflash --spec-tokens 7 --spec-method dflash 🔗 Ultra 550B: 🔗 Super 120B:

Red Hat AI just shipped DFlash speculator checkpoints for two of NVIDIA AI's most powerful open models: → Nemotron Ultra 550B → Nemotron Super 120B On math and reasoning: ~5 out of 7 draft tokens accepted on average. On code (HumanEval): ~3.4 out of 7. Both checkpoints trained with the open source Speculators library from vLLM. Apache 2.0. Validated on NVIDIA B200. One flag to enable in vLLM: --spec-model RedHatAI/NVIDIA-Nemotron-3-Ultra-550B-A55B-speculator.dflash --spec-tokens 7 --spec-method dflash 🔗 Ultra 550B: 🔗 Super 120B:

Red Hat AI

14,923 Aufrufe • vor 1 Monat

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

Segment Anything Model 2 (SAM 2) is a foundation model from Meta FAIR for promptable visual segmentation in images & videos. Available now for anyone to build on for free, open source under an Apache license. Try the demo ➡️

AI at Meta

97,755 Aufrufe • vor 2 Jahren

One of the last Model S Signature for Europe was delivered today to a very good friend in northern Germany 🇩🇪 It’s the number 89 and we are celebrating it with our Tesla Europe, Middle East & Africa Community. Welcome on the road Model S Signature #89!

One of the last Model S Signature for Europe was delivered today to a very good friend in northern Germany 🇩🇪 It’s the number 89 and we are celebrating it with our Tesla Europe, Middle East & Africa Community. Welcome on the road Model S Signature #89!

RalF

10,067 Aufrufe • vor 1 Monat

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Whole Mars Catalog

45,595 Aufrufe • vor 1 Monat

Introducing BioCLIP: A Vision Foundation Model for the Tree of Life A foundation model that strongly generalizes on the tree of life (2M+ species), outperforming OpenAI CLIP by 18% in zero-shot classification, and supports open-ended classification over almost the entire tree of life What's the secrete ingredients? > Data: we curate and release TreeOfLife-10M, the largest and most diverse ML-ready dataset of organism images to date. It contains 10.4M images for over 450K taxa, sourced from iNaturalist, BIOSCAN, and Encyclopedia of Life. > Modeling: we creatively repurposes CLIP's multimodal contrastive learning objective for hierarchical image classification. The autoregressive language model naturally encodes the hierarchy of the tree of life taxonomy, which in turn bakes the hierarchical representation into the vision transformer encoder. Key results > Strong zero/few-shot classification for animals/plants/fungi, including rare species, outperforming CLIP by avg 16-18% absolute. > T-sne visualization shows that BioCLIP's vision encoder has captued the fine-grained hierarchical structure of the tree of life > BioCLIP is a kind of universal classifier for the tree of life. Just give it an organism image and it will likely find the correct species (among top 5)! But use it with caution; it's not perfect yet.. Final remarks > AI for Science is really hard but extremely rewarding! It took us a ton of time (1+ year) and frustration trying to find a plausible way to integrate the tree of life taxonomy into foundation model training. But when the "Eureka!" moment came and the idea hit us (by the great Wei-Lun Chao) that CLIP's multimodal contrastive learning objective can be repurposed for that, everything just follows naturally. It was truly a moment of joy and excitement! > BioCLIP is our first attempt at foundation models for biology, but it certainly won't be the last! There's so much more to do at the intersection of one of the oldest scientific disciplines and the young but thriving field of AI. Biological intelligence is the foundation for artificial intelligence, and artificial intelligence will in turn become the most important tool for us to unraval the mysteries of biological intelligence. We are hiring postdocs and PhDs in the NSF Imageomics Institute institute to explore this exciting field! Drop us an email. also happy to chat about it at #NeurIPS2023 with any of Tanya, Wei-Lun Chao, or me. - paper: - project: - demo: - model: - data (TreeOfLife-10M): to be released on Hugging Face soon joint work with the amazing Imageomics Institute team: @samstevens6860 Lisa Wu, Matt Thompson, Elizabeth Campolongo Chan Hee (Luke) Song David Carlyn Li Dong Wasila Dahdul Chuck Stewart, Tanya Berger-Wolf Wei-Lun Chao Yu Su

Introducing BioCLIP: A Vision Foundation Model for the Tree of Life A foundation model that strongly generalizes on the tree of life (2M+ species), outperforming OpenAI CLIP by 18% in zero-shot classification, and supports open-ended classification over almost the entire tree of life What's the secrete ingredients? > Data: we curate and release TreeOfLife-10M, the largest and most diverse ML-ready dataset of organism images to date. It contains 10.4M images for over 450K taxa, sourced from iNaturalist, BIOSCAN, and Encyclopedia of Life. > Modeling: we creatively repurposes CLIP's multimodal contrastive learning objective for hierarchical image classification. The autoregressive language model naturally encodes the hierarchy of the tree of life taxonomy, which in turn bakes the hierarchical representation into the vision transformer encoder. Key results > Strong zero/few-shot classification for animals/plants/fungi, including rare species, outperforming CLIP by avg 16-18% absolute. > T-sne visualization shows that BioCLIP's vision encoder has captued the fine-grained hierarchical structure of the tree of life > BioCLIP is a kind of universal classifier for the tree of life. Just give it an organism image and it will likely find the correct species (among top 5)! But use it with caution; it's not perfect yet.. Final remarks > AI for Science is really hard but extremely rewarding! It took us a ton of time (1+ year) and frustration trying to find a plausible way to integrate the tree of life taxonomy into foundation model training. But when the "Eureka!" moment came and the idea hit us (by the great Wei-Lun Chao) that CLIP's multimodal contrastive learning objective can be repurposed for that, everything just follows naturally. It was truly a moment of joy and excitement! > BioCLIP is our first attempt at foundation models for biology, but it certainly won't be the last! There's so much more to do at the intersection of one of the oldest scientific disciplines and the young but thriving field of AI. Biological intelligence is the foundation for artificial intelligence, and artificial intelligence will in turn become the most important tool for us to unraval the mysteries of biological intelligence. We are hiring postdocs and PhDs in the NSF Imageomics Institute institute to explore this exciting field! Drop us an email. also happy to chat about it at #NeurIPS2023 with any of Tanya, Wei-Lun Chao, or me. - paper: - project: - demo: - model: - data (TreeOfLife-10M): to be released on Hugging Face soon joint work with the amazing Imageomics Institute team: @samstevens6860 Lisa Wu, Matt Thompson, Elizabeth Campolongo Chan Hee (Luke) Song David Carlyn Li Dong Wasila Dahdul Chuck Stewart, Tanya Berger-Wolf Wei-Lun Chao Yu Su

Yu Su

80,660 Aufrufe • vor 2 Jahren

FLUX.2 is LIVE on Higgsfield. Our 65% Black Friday lineup expands with the best open-source image model. Fully unrestricted, uncensored & FREE. ALL image models UNLIMITED for 1 year! Two days left to lock this in 🔥 For 9 hours: comment&retweet&like for 240 FREE credits

FLUX.2 is LIVE on Higgsfield. Our 65% Black Friday lineup expands with the best open-source image model. Fully unrestricted, uncensored & FREE. ALL image models UNLIMITED for 1 year! Two days left to lock this in 🔥 For 9 hours: comment&retweet&like for 240 FREE credits

Higgsfield AI 🧩

1,097,050 Aufrufe • vor 9 Monaten