正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

What could scaling unlock for biology? Introducing ProGen3- our next AI foundation models for protein generation. We develop compute-optimal scaling laws up to 46B parameters on 1.5T tokens with real evidence in the wet lab. +we solve a new set of challenges for drug discovery

Ali Madani

7,907 subscribers

80,642 次观看 • 1 年前 •via X (Twitter)

健康养生新闻政治科学技术

Anya Rossi• Live Now

Private livecam show

11 条评论

Ali Madani 的头像

Ali Madani1 年前

Training on more data and increasing the capacity of our language models = better understanding of proteins and ability to generate viable diverse sequences. We are on a clear trajectory.

Ali Madani 的头像

Ali Madani1 年前

One of the most important aspects: Alignment of our foundation models to laboratory data improves its utility for functional tasks. Larger models are better at alignment. + easter egg that our large models are good at span generation

Ali Madani 的头像

Ali Madani1 年前

This is not purely an academic endeavor. We ultimately evaluate our progress at Profluent based on our ability to create positive value for society. So we tackled two challenges in biology.

Ali Madani 的头像

Ali Madani1 年前

1. We took 20 disease targets with approved therapies treating 7M people and yielding $660B in sales. Can we design antibodies that match performance in one shot? Yes + they are materially different across all CDRs and framework regions. We call these OpenAntibodies-- more soon.

Ali Madani 的头像

Ali Madani1 年前

2. Gene editors, especially cas9, are quite large, posing practical issues. Could we design ultra-compact cas proteins? Yes, we designed functional proteins that could enable single AAV delivery. Really excited for implications for unaddressable targets and epigenome editing too

Ali Madani 的头像

Ali Madani1 年前

☀️The future is bright. We're in the early days of what will sweep biology and design cures to disease. Learn more, license our molecules, or get early access to our API below Fortune:

Ithy 的头像

Ithy1 年前

What happens when you combine every AI? It's time for something better than ChatGPT...

Cyril Zakka, MD 的头像

Cyril Zakka, MD1 年前

Can we expect an open weights release on the @huggingface hub? Would be hugely impactful.

David Li 的头像

David Li1 年前

Congrats! Cool stuff! Will there be a preprint describing OpenAntibodies and the small Cas as well?

fajie yuan 的头像

fajie yuan1 年前

Hi Ali, Congrats，will the training dataset be shared？

Ubadah Sabbagh 的头像

Ubadah Sabbagh1 年前

congrats to the team! looking forward to digging in.

相关视频

.Basecamp Research trained EDEN, a 28B-parameter foundation model for biology, on Nebius: 1,024 GPUs, six weeks, no major interruptions. The result: new scaling laws for biology; new antibiotics and gene editors already in the lab.

.Basecamp Research trained EDEN, a 28B-parameter foundation model for biology, on Nebius: 1,024 GPUs, six weeks, no major interruptions. The result: new scaling laws for biology; new antibiotics and gene editors already in the lab.

Nebius

22,497 次观看 • 24 天前

Microsoft CEO Satya Nadella says that rather than seeing AI Scaling Laws hit a wall, if anything we are seeing the emergence of a new Scaling Law for test-time compute

Microsoft CEO Satya Nadella says that rather than seeing AI Scaling Laws hit a wall, if anything we are seeing the emergence of a new Scaling Law for test-time compute

Tsarathustra

111,422 次观看 • 1 年前

Introducing ESM Cambrian. Unsupervised learning can invert biology at scale to reveal the hidden structure of the natural world. We’ve scaled up compute and data to train a new generation of protein language models. ESM C defines a new state of the art for protein representation learning.

Introducing ESM Cambrian. Unsupervised learning can invert biology at scale to reveal the hidden structure of the natural world. We’ve scaled up compute and data to train a new generation of protein language models. ESM C defines a new state of the art for protein representation learning.

Alex Rives

207,204 次观看 • 1 年前

OpenAI's Noam Brown says letting AI models think for 20 seconds got the same boost in performance as scaling up the model by 100,000x and training it for 100,000 times longer, and scaling model size and compute will end up being a footnote compared to scaling model thinking

OpenAI's Noam Brown says letting AI models think for 20 seconds got the same boost in performance as scaling up the model by 100,000x and training it for 100,000 times longer, and scaling model size and compute will end up being a footnote compared to scaling model thinking

Tsarathustra

273,500 次观看 • 1 年前

Sergey Brin on Scaling Laws vs. algorithms: “Compute is the dessert. Algorithms are the main course. Algorithmic progress has outpaced compute and data scaling over the last decade.” This echoes Ilya’s view: scaling what we have will keep leading to improvements, but some key algorithmic breakthrough is still missing for AGI/ASI.

Sergey Brin on Scaling Laws vs. algorithms: “Compute is the dessert. Algorithms are the main course. Algorithmic progress has outpaced compute and data scaling over the last decade.” This echoes Ilya’s view: scaling what we have will keep leading to improvements, but some key algorithmic breakthrough is still missing for AGI/ASI.

Yuchen Jin

90,818 次观看 • 7 个月前

it's been a huge few weeks for ai in bio: a $2.25b Profluent x Eli Lilly and Company deal on ai-designed gene editors, verve's base-editing data, new scaling results on protein models from Chan Zuckerberg Biohub Network, Isomorphic Labs' haul. Ali Madani and i recorded a pod diving into all of it we get into taking biology from discovery to design, sequence-first vs structure-first, and why he calls this the "gpt-1.5 era" of biology... enjoy!

it's been a huge few weeks for ai in bio: a $2.25b Profluent x Eli Lilly and Company deal on ai-designed gene editors, verve's base-editing data, new scaling results on protein models from Chan Zuckerberg Biohub Network, Isomorphic Labs' haul. Ali Madani and i recorded a pod diving into all of it we get into taking biology from discovery to design, sequence-first vs structure-first, and why he calls this the "gpt-1.5 era" of biology... enjoy!

Nathan Benaich

32,400 次观看 • 1 个月前

Introducing Move Pro 2.0: the next evolution in high-precision motion capture. Local and secure processing, faster performance with dedicated GPUs, cloud scaling options, and second-generation AI models for enhanced accuracy. Perfect for capturing motion from small studios to large stadiums.

Introducing Move Pro 2.0: the next evolution in high-precision motion capture. Local and secure processing, faster performance with dedicated GPUs, cloud scaling options, and second-generation AI models for enhanced accuracy. Perfect for capturing motion from small studios to large stadiums.

Move AI

14,141 次观看 • 1 年前

📣 Call for proposals! Our first funded programme is now live — Scaling Compute: AI at 1/1000th the cost. Hear more from PD Suraj Bramhavar on how we aim to radically reduce the cost + energy consumption of compute hardware, and open new avenues to reap the benefits of AI 👇

📣 Call for proposals! Our first funded programme is now live — Scaling Compute: AI at 1/1000th the cost. Hear more from PD Suraj Bramhavar on how we aim to radically reduce the cost + energy consumption of compute hardware, and open new avenues to reap the benefits of AI 👇

ARIA

61,132 次观看 • 2 年前

SemiAnalysis's Dylan Patel says AI models will improve faster in the next 6 month to a year than we saw in the past year because there's a new axis of scale that has been unlocked in the form of synthetic data generation, that we are still very early in scaling up

SemiAnalysis's Dylan Patel says AI models will improve faster in the next 6 month to a year than we saw in the past year because there's a new axis of scale that has been unlocked in the form of synthetic data generation, that we are still very early in scaling up

Tsarathustra

174,827 次观看 • 1 年前

Can Claude Code design proteins? We're just about to release our wet lab API + SDK so we made a quick demo to test how agents can interact with protein design models and submit the proteins to our lab for testing. This closes the loop between the computational design and the experiment validation We queued up a bunch of agent-designed proteins for wet lab validation, will release the results in a couple weeks on Proteinbase

Can Claude Code design proteins? We're just about to release our wet lab API + SDK so we made a quick demo to test how agents can interact with protein design models and submit the proteins to our lab for testing. This closes the loop between the computational design and the experiment validation We queued up a bunch of agent-designed proteins for wet lab validation, will release the results in a couple weeks on Proteinbase

Julian Englert

81,258 次观看 • 6 个月前

The Bitter Lesson for Biology — Adam Green on Scaling Laws for Virtual Cells In this episode, the founder of Markov Biosciences, Adam Green, explains the "bitter lesson" for biology, the idea borrowed from Richard Sutton that large unbiased datasets and the right training objective tend to outcompete models with hard-coded rules and human priors. We talk a lot about virtual cells, sources of bias in data, and his evidence that virtual cells pre-trained on plain observational data show clean scaling laws, getting monotonically better at predicting unseen perturbations as the models grow, and beating a state-of-the-art model built specifically for that task. Our interview was about 3 hours long, but cut to 1.5 hours. My main goal was to understand Adam's perspective on virtual cells and his goal to "solve biology." Search for "The New Biology" on YouTube, Spotify, and Apple Podcasts. Timestamps: 00:00 — Cold open 01:58 — Clinical predictions from a virtual cell 05:38 — What is a "virtual cell"? 08:01 — The problems with single-cell RNA-seq 11:31 — The urns analogy 19:54 — Observational vs. perturbation 23:29 — The bitter lesson for biology 29:06 — Geometric Plackett-Luce 38:27 — Ablations and loss function 47:23 — Cells as specimens 59:26 — Antibody-Drug conjugates 1:11:16 — Will we ever understand the cell fully?

The Bitter Lesson for Biology — Adam Green on Scaling Laws for Virtual Cells In this episode, the founder of Markov Biosciences, Adam Green, explains the "bitter lesson" for biology, the idea borrowed from Richard Sutton that large unbiased datasets and the right training objective tend to outcompete models with hard-coded rules and human priors. We talk a lot about virtual cells, sources of bias in data, and his evidence that virtual cells pre-trained on plain observational data show clean scaling laws, getting monotonically better at predicting unseen perturbations as the models grow, and beating a state-of-the-art model built specifically for that task. Our interview was about 3 hours long, but cut to 1.5 hours. My main goal was to understand Adam's perspective on virtual cells and his goal to "solve biology." Search for "The New Biology" on YouTube, Spotify, and Apple Podcasts. Timestamps: 00:00 — Cold open 01:58 — Clinical predictions from a virtual cell 05:38 — What is a "virtual cell"? 08:01 — The problems with single-cell RNA-seq 11:31 — The urns analogy 19:54 — Observational vs. perturbation 23:29 — The bitter lesson for biology 29:06 — Geometric Plackett-Luce 38:27 — Ablations and loss function 47:23 — Cells as specimens 59:26 — Antibody-Drug conjugates 1:11:16 — Will we ever understand the cell fully?

Niko McCarty.

27,682 次观看 • 1 个月前

Introducing LiteFold DeNovo. Generate hundreds of chemically viable ligands against any protein pocket, evaluate them in real time, and refine with our built in molecular editor. The problem: hit generation in drug discovery is slow, fragmented, and outsourced. With LiteFold DeNovo, it becomes interactive and instant. You can also fragment grow base generated molecules, iterate, and soon run molecular dynamics to test stability. Docking and large scale screening are on the way. When we said we’re building the AI first lab for drug discovery, we meant it. Super excited to what's coming next. Codename: Rosalind Try LiteFold DeNovo today. It’s free. Links in the comments.

Introducing LiteFold DeNovo. Generate hundreds of chemically viable ligands against any protein pocket, evaluate them in real time, and refine with our built in molecular editor. The problem: hit generation in drug discovery is slow, fragmented, and outsourced. With LiteFold DeNovo, it becomes interactive and instant. You can also fragment grow base generated molecules, iterate, and soon run molecular dynamics to test stability. Docking and large scale screening are on the way. When we said we’re building the AI first lab for drug discovery, we meant it. Super excited to what's coming next. Codename: Rosalind Try LiteFold DeNovo today. It’s free. Links in the comments.

Anindya

23,157 次观看 • 11 个月前

Introducing 🧞Genie 2 🧞 - our most capable large-scale foundation world model, which can generate a diverse array of consistent worlds, playable for up to a minute. We believe Genie 2 could unlock the next wave of capabilities for embodied agents 🧠.

Introducing 🧞Genie 2 🧞 - our most capable large-scale foundation world model, which can generate a diverse array of consistent worlds, playable for up to a minute. We believe Genie 2 could unlock the next wave of capabilities for embodied agents 🧠.

Jack Parker-Holder

2,606,305 次观看 • 1 年前

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models

AK

62,719 次观看 • 1 年前

The first real evidence that the days of LLM Scaling laws are over. Introducing SCOPE: the world's most efficient Neural Planner. 🔍📊 We tested SCOPE vs Frontier LLMs for planning tasks on TextCraft (text version of Minecraft) and here are the results: ⤵️ - SCOPE Runs 55x faster than GPT 3.5 (3 seconds vs 164 seconds) - SCOPE is 160,000 smaller than GPT 4o (11M parameters vs 1.8T parameters) - SCOPE is more accurate on Planning tasks (56%) than frontier LLM models The age of efficient AI models starts now. 🔗📌 Read the full write up here:

The first real evidence that the days of LLM Scaling laws are over. Introducing SCOPE: the world's most efficient Neural Planner. 🔍📊 We tested SCOPE vs Frontier LLMs for planning tasks on TextCraft (text version of Minecraft) and here are the results: ⤵️ - SCOPE Runs 55x faster than GPT 3.5 (3 seconds vs 164 seconds) - SCOPE is 160,000 smaller than GPT 4o (11M parameters vs 1.8T parameters) - SCOPE is more accurate on Planning tasks (56%) than frontier LLM models The age of efficient AI models starts now. 🔗📌 Read the full write up here:

Skyfall AI

49,027 次观看 • 6 个月前

Altman on what could lead to a compute oversupply within 2 Years: “If models become so smart & efficient that they can do nearly everything we need, while human attention becomes the limiting factor, we could end up oversupplied. The same could happen if costs stop falling because we hit a scaling wall.”

Altman on what could lead to a compute oversupply within 2 Years: “If models become so smart & efficient that they can do nearly everything we need, while human attention becomes the limiting factor, we could end up oversupplied. The same could happen if costs stop falling because we hit a scaling wall.”

Wall St Engine

26,260 次观看 • 5 天前

A new chapter for World Mobile Token. $WMTX is now live on Binance! Built on real-world usage. Scaling at global level. The foundation is set. This is how DePIN grows up.

A new chapter for World Mobile Token. $WMTX is now live on Binance! Built on real-world usage. Scaling at global level. The foundation is set. This is how DePIN grows up.

World Mobile Chain

63,190 次观看 • 6 个月前

High-quality experimental data fuels biological discovery. Could an AI-enabled "autonomous lab" fundamentally increase the rate at which we generate it, and accelerate our progress toward cures for disease? That's the question I sat down to explore with Jason Kelly, CEO of Ginkgo Bioworks, for a deep-dive and tour of their 52-RAC lab in Boston's Seaport District. In our conversation we covered: → Ginkgo's recent collaboration with OpenAI on cell-free protein synthesis → U.S. Department of Energy's POTUS-backed Genesis Mission to accelerate AI for science → The evolution from cloud labs to truly autonomous labs → An on-the-floor look at Ginkgo's lab, including early challenges and creative workarounds If you're curious about frontier AI models meet the lab bench, check out the latest episode of the Discovery Engines Podcast. Watch or listen: 👀 YouTube: 🎧 Spotify: 🎧 Apple:

High-quality experimental data fuels biological discovery. Could an AI-enabled "autonomous lab" fundamentally increase the rate at which we generate it, and accelerate our progress toward cures for disease? That's the question I sat down to explore with Jason Kelly, CEO of Ginkgo Bioworks, for a deep-dive and tour of their 52-RAC lab in Boston's Seaport District. In our conversation we covered: → Ginkgo's recent collaboration with OpenAI on cell-free protein synthesis → U.S. Department of Energy's POTUS-backed Genesis Mission to accelerate AI for science → The evolution from cloud labs to truly autonomous labs → An on-the-floor look at Ginkgo's lab, including early challenges and creative workarounds If you're curious about frontier AI models meet the lab bench, check out the latest episode of the Discovery Engines Podcast. Watch or listen: 👀 YouTube: 🎧 Spotify: 🎧 Apple:

Nabil Laoudji

11,508 次观看 • 4 个月前

AI agents move fast. So should your search foundation. Introducing the next generation of Amazon OpenSearch Serverless. We rebuilt OpenSearch Serverless from the ground up for agentic AI and unpredictable workloads: - Create resources in seconds - Autoscale up to 20x faster than before - Usage-based pricing with decoupled storage and compute - No idle costs with scale to zero - Up to 60% cost savings vs. provisioned clusters for peak capacity Learn more.

AI agents move fast. So should your search foundation. Introducing the next generation of Amazon OpenSearch Serverless. We rebuilt OpenSearch Serverless from the ground up for agentic AI and unpredictable workloads: - Create resources in seconds - Autoscale up to 20x faster than before - Usage-based pricing with decoupled storage and compute - No idle costs with scale to zero - Up to 60% cost savings vs. provisioned clusters for peak capacity Learn more.

Amazon Web Services

11,941 次观看 • 2 个月前

“Proteins sticking to other proteins is what makes life exist.” 🧬 We explain the science behind AlphaProteo: our AI system for designing protein binders that holds huge potential for biology. ▶️ Hear from Jue Wang, co-lead of the protein design team, David La, research scientist, and Harshnira Patani, wet lab lead here at Google DeepMind. ↓

“Proteins sticking to other proteins is what makes life exist.” 🧬 We explain the science behind AlphaProteo: our AI system for designing protein binders that holds huge potential for biology. ▶️ Hear from Jue Wang, co-lead of the protein design team, David La, research scientist, and Harshnira Patani, wet lab lead here at Google DeepMind. ↓

Google DeepMind

67,570 次观看 • 1 年前