Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

In two new papers we have found that the ESM2 language model generalizes beyond natural proteins, and enables programmable generation of complex and modular protein structures.

Alex Rives

14,600 subscribers

202,826 views • 3 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Comments

Alex Rives3 years ago

ESM2 learns the design principles of proteins. With @uwproteindesign we experimentally validated 152 ESM2 designs, including de novo generations outside the space of natural proteins (<20% sequence identity to known proteins). 📄Read the paper here:

Alex Rives3 years ago

We implemented a high level programming language for generative protein design with ESM2. This made it possible to program the generation of large proteins and complexes with intricate and modular structures. 📄Read the paper here:

Alex Rives3 years ago

Thread by @TomSercu on how language models generalize to de novo proteins ⬇️

Alex Rives3 years ago

🧵@BrianHie on a high level programming language for generative protein design

The Guy3 years ago

Well, that probably won't have any significant long term impacts. Holy Mackerel! Are you kidding me! Top of the line, cutting edge work guys. Wow! Congratulations. I'm gonna be over here freakin' out for a while.

shb3 years ago

in the appendix it looks like you used the 650M param version for this... why not one of the bigger models? didn't they train up to 15B?

decipher3 years ago

congratulations, I was expecting novel approaches to solve inverse folding-docking problem after esm & aplhafold2

LUIS VIZCAYA3 years ago

Sometimes I have no words for the amazing progress we are having in AI

Ifigeneia Apostolopoulou3 years ago

it's the class of problems to which generative models should be applied ;)

Peter Morgan3 years ago

Phenomenal

Related Videos

What if we could universally recombine, insert, delete, or invert any two pieces of DNA? In back-to-back nature papers, we report the discovery of bridge RNAs and 3 atomic structures of the first natural RNA-guided recombinase - a new mechanism for programmable genome design

What if we could universally recombine, insert, delete, or invert any two pieces of DNA? In back-to-back nature papers, we report the discovery of bridge RNAs and 3 atomic structures of the first natural RNA-guided recombinase - a new mechanism for programmable genome design

Patrick Hsu

1,862,442 views • 2 years ago

We're thrilled to present ESM3 in Science Magazine. ESM3 is a generative language model that reasons over the three fundamental properties of proteins: sequence, structure, and function. Today we're making ESM3 available free to researchers worldwide via the public beta of an API for biological intelligence. Trained with over a trillion teraflops of compute, this is the first time a model of this scale has been trained for biology, pushing the frontier of AI for biological discovery and engineering. ESM3 learns to represent the immense complexity of protein biology, learning from billions of natural proteins. From this training it developed the capability to design proteins, responding to complex prompts combining atomic level details and high level instructions to generate new proteins. ESM3 can explore protein space far beyond natural evolution. We prompted ESM3 to generate a fluorescent protein at a far distance from any known fluorescent proteins, searching an unknown region of protein space, to discover a new fluorescent protein. We estimate this is equivalent to simulating five hundred million years of evolution.

We're thrilled to present ESM3 in Science Magazine. ESM3 is a generative language model that reasons over the three fundamental properties of proteins: sequence, structure, and function. Today we're making ESM3 available free to researchers worldwide via the public beta of an API for biological intelligence. Trained with over a trillion teraflops of compute, this is the first time a model of this scale has been trained for biology, pushing the frontier of AI for biological discovery and engineering. ESM3 learns to represent the immense complexity of protein biology, learning from billions of natural proteins. From this training it developed the capability to design proteins, responding to complex prompts combining atomic level details and high level instructions to generate new proteins. ESM3 can explore protein space far beyond natural evolution. We prompted ESM3 to generate a fluorescent protein at a far distance from any known fluorescent proteins, searching an unknown region of protein space, to discover a new fluorescent protein. We estimate this is equivalent to simulating five hundred million years of evolution.

Alex Rives

227,480 views • 1 year ago

What seemed like an intractable problem is now possible: To design proteins with a specified nonlinear mechanical response, capturing complex folding and unfolding mechanisms in singe and few-shot computations. We present ForceGen, an end-to-end algorithm for de novo protein generation based on nonlinear mechanical unfolding responses. Rooted in the physics of protein mechanics, this generative strategy provides a powerful way to design new proteins rapidly, including exquisite and rapid predictions about their dynamical behavior. Proteins, like any other mechanical object, respond to forces in peculiar ways. Think of the different response you'd get from pulling on a steel cable versus pulling on a rubber band, or the difference between honey and glass. Now, we can design proteins with a set of desirable mechanical characteristics, with applications from health to sustainable plastics. The key to solving this problem was to integrate a protein language model with denoising diffusion methods, and using accurate atomistic-level physical simulation data to endow the model a first-principles understanding. ForceGen can solve both forward and inverse tasks: In the forward task, we can predict how stable a protein is, how it will unfold and what the forces involved are, all given just the sequence of amino acids. In the inverse task, we can design new proteins that meet complex nonlinear mechanical signature targets. Read the paper, led by LAMM@MIT postdoc Bo Ni, published in Science Advances: Why do we care about the mechanics of proteins? The mechanics of proteins are critical elements of many living systems - as evidenced in many studies of mechanobiology. Through evolution, nature has presented a set of remarkable protein materials with unique mechanical functions like elastins, silks, keratins or collagens that play crucial roles in biology. However, going beyond natural designs to discover proteins that meet specified mechanical properties remains challenging. So far, the only way to do this was to use existing evolutionary concepts or to manually alter proteins. With our new generative model we can directly design proteins to meet complex nonlinear mechanical property-design objectives. ForceGen leverages deep knowledge on protein sequences from a pretrained protein language model and maps mechanical unfolding responses to create proteins. Via full-atom molecular simulations for direct validation from physical and chemical principles, we demonstrate that the designed proteins are de novo, and fulfill the targeted mechanical properties, including unfolding energy and mechanical strength, and a detailed unfolding force-separation curves. ForceGen offers rapid pathways to explore the enormous mechanobiological protein sequence space unconstrained by biological synthesis, to enable the discovery of new protein materials with superior mechanical properties. B. Ni, D.L. Kaplan, M.J. Buehler, ForceGen: End-to-end de novo protein generation based on nonlinear mechanical unfolding responses using a language diffusion model. Sci. Adv. 10, eadl4000 (2024). DOI: 10.1126/sciadv.adl4000 Codes and model weights available Hugging Face: David Kaplan

What seemed like an intractable problem is now possible: To design proteins with a specified nonlinear mechanical response, capturing complex folding and unfolding mechanisms in singe and few-shot computations. We present ForceGen, an end-to-end algorithm for de novo protein generation based on nonlinear mechanical unfolding responses. Rooted in the physics of protein mechanics, this generative strategy provides a powerful way to design new proteins rapidly, including exquisite and rapid predictions about their dynamical behavior. Proteins, like any other mechanical object, respond to forces in peculiar ways. Think of the different response you'd get from pulling on a steel cable versus pulling on a rubber band, or the difference between honey and glass. Now, we can design proteins with a set of desirable mechanical characteristics, with applications from health to sustainable plastics. The key to solving this problem was to integrate a protein language model with denoising diffusion methods, and using accurate atomistic-level physical simulation data to endow the model a first-principles understanding. ForceGen can solve both forward and inverse tasks: In the forward task, we can predict how stable a protein is, how it will unfold and what the forces involved are, all given just the sequence of amino acids. In the inverse task, we can design new proteins that meet complex nonlinear mechanical signature targets. Read the paper, led by LAMM@MIT postdoc Bo Ni, published in Science Advances: Why do we care about the mechanics of proteins? The mechanics of proteins are critical elements of many living systems - as evidenced in many studies of mechanobiology. Through evolution, nature has presented a set of remarkable protein materials with unique mechanical functions like elastins, silks, keratins or collagens that play crucial roles in biology. However, going beyond natural designs to discover proteins that meet specified mechanical properties remains challenging. So far, the only way to do this was to use existing evolutionary concepts or to manually alter proteins. With our new generative model we can directly design proteins to meet complex nonlinear mechanical property-design objectives. ForceGen leverages deep knowledge on protein sequences from a pretrained protein language model and maps mechanical unfolding responses to create proteins. Via full-atom molecular simulations for direct validation from physical and chemical principles, we demonstrate that the designed proteins are de novo, and fulfill the targeted mechanical properties, including unfolding energy and mechanical strength, and a detailed unfolding force-separation curves. ForceGen offers rapid pathways to explore the enormous mechanobiological protein sequence space unconstrained by biological synthesis, to enable the discovery of new protein materials with superior mechanical properties. B. Ni, D.L. Kaplan, M.J. Buehler, ForceGen: End-to-end de novo protein generation based on nonlinear mechanical unfolding responses using a language diffusion model. Sci. Adv. 10, eadl4000 (2024). DOI: 10.1126/sciadv.adl4000 Codes and model weights available Hugging Face: David Kaplan

Markus J. Buehler

47,269 views • 2 years ago

We have trained ESM3 and we're excited to introduce EvolutionaryScale. ESM3 is a generative language model for programming biology. In experiments, we found ESM3 can simulate 500M years of evolution to generate new fluorescent proteins. Read more:

We have trained ESM3 and we're excited to introduce EvolutionaryScale. ESM3 is a generative language model for programming biology. In experiments, we found ESM3 can simulate 500M years of evolution to generate new fluorescent proteins. Read more:

Alex Rives

1,537,330 views • 2 years ago

Internet Computer's Caffeine is Now Available in Claude Internet Computer (DFINITY Foundation)'s Caffeine launches a direct integration with Anthropic’s Claude, enabling the instantaneous generation of production-ready applications through natural language interfaces. This deployment enables "vibe coders" and enterprise teams to use caffeine’s modular frameworks to build, iterate on, and deploy complex software without leaving the LLM environment.

Internet Computer's Caffeine is Now Available in Claude Internet Computer (DFINITY Foundation)'s Caffeine launches a direct integration with Anthropic’s Claude, enabling the instantaneous generation of production-ready applications through natural language interfaces. This deployment enables "vibe coders" and enterprise teams to use caffeine’s modular frameworks to build, iterate on, and deploy complex software without leaving the LLM environment.

BSCN

31,877 views • 2 months ago

”David Baker, Demis Hassabis and John Jumper, your ground-breaking work in computational protein design and protein structure prediction has revolutionised these fields. It has opened up completely new possibilities to design proteins that have never been seen before, and we now have access to predicted structures of all 200 million known proteins. These are truly great achievements.” Watch the very moment the chemistry laureates received their Nobel Prize diplomas and medals during the 2024 Nobel Prize award ceremony. #NobelPrize

”David Baker, Demis Hassabis and John Jumper, your ground-breaking work in computational protein design and protein structure prediction has revolutionised these fields. It has opened up completely new possibilities to design proteins that have never been seen before, and we now have access to predicted structures of all 200 million known proteins. These are truly great achievements.” Watch the very moment the chemistry laureates received their Nobel Prize diplomas and medals during the 2024 Nobel Prize award ceremony. #NobelPrize

The Nobel Prize

295,309 views • 1 year ago

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.

New paper introduces NaVILA, a vision-language-action (VLA) model that integrates high-level visual-language understanding and low-level locomotion control. It enables humanoid or quadruped robots to navigate unseen environments with natural language instructions.

The Humanoid Hub

20,466 views • 1 year ago

Excited to show a glimpse of the next generation of AlphaFold from our teams at Isomorphic Labs and Google DeepMind. This model expands beyond proteins to include other molecules like small molecules and nucleic acids, and improves accuracy on proteins 1/n

Excited to show a glimpse of the next generation of AlphaFold from our teams at Isomorphic Labs and Google DeepMind. This model expands beyond proteins to include other molecules like small molecules and nucleic acids, and improves accuracy on proteins 1/n

Max Jaderberg

117,467 views • 2 years ago

Introducing UI-TARS-1.5, a vision-language model that beats OpenAI Operator and Claude 3.7 on GUI Agent and Game Agent tasks. We've open-sourced a small-size version model for research purposes, more details can be found in our blog. TARS learns solely from a screen, but generalizes beyond a screen! Blog: Model: App:

Introducing UI-TARS-1.5, a vision-language model that beats OpenAI Operator and Claude 3.7 on GUI Agent and Game Agent tasks. We've open-sourced a small-size version model for research purposes, more details can be found in our blog. TARS learns solely from a screen, but generalizes beyond a screen! Blog: Model: App:

Yujia Qin

85,212 views • 1 year ago

Introducing ESM Cambrian. Unsupervised learning can invert biology at scale to reveal the hidden structure of the natural world. We’ve scaled up compute and data to train a new generation of protein language models. ESM C defines a new state of the art for protein representation learning.

Introducing ESM Cambrian. Unsupervised learning can invert biology at scale to reveal the hidden structure of the natural world. We’ve scaled up compute and data to train a new generation of protein language models. ESM C defines a new state of the art for protein representation learning.

Alex Rives

207,283 views • 1 year ago

Explosives of ‘devastating power’ were found near the TurkStream pipeline in Serbia that carries Russian natural gas to Hungary and beyond, leaders from the two countries said

Explosives of ‘devastating power’ were found near the TurkStream pipeline in Serbia that carries Russian natural gas to Hungary and beyond, leaders from the two countries said

Reuters

137,269 views • 4 months ago

🎉We present DreamCraft3D++, an extension of DreamCraft3D that enables efficient high-quality generation of complex 3D assets in 10 minutes. Project: Paper:

🎉We present DreamCraft3D++, an extension of DreamCraft3D that enables efficient high-quality generation of complex 3D assets in 10 minutes. Project: Paper:

Jingxiang Sun

28,318 views • 1 year ago

Cooking in kitchens is fun. BUT doing it collaboratively with two robots is even more satisfying! We introduce MOSAIC, a modular framework that coordinates multiple robots to closely collaborate and cook with humans via natural language interaction and a repository of skills.

Cooking in kitchens is fun. BUT doing it collaboratively with two robots is even more satisfying! We introduce MOSAIC, a modular framework that coordinates multiple robots to closely collaborate and cook with humans via natural language interaction and a repository of skills.

Sanjiban Choudhury

26,423 views • 2 years ago

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.

Two weeks ago, Figure's CEO announced the end of their partnership with OpenAI and promised to show the in-house AI development – here we are: Helix is a model that unifies perception, language and learned control to execute humanoid tasks by following natural language prompts.

The Humanoid Hub

69,442 views • 1 year ago

NVIDIA CEO Jensen Huang just gave us a glimpse into the future of AI, and it's far bigger than text and images. He asked: "If you can understand worlds, is it possible that you understand proteins and chemicals that have structure?” This is the quiet revolution happening in AI right now. We're moving from an AI that understands human language to an AI that understands the language of nature itself. The "letters" are atoms. The "words" are molecules. The "sentences" are the complex structures of life, networks, and reality. The same foundational models that generate video are now being used to: *Decode proteins to fight disease *Design new materials at the atomic level *Simulate complex systems in quantum computing These were once the domain of supercomputers and Nobel laureates. Now, they're becoming solvable problems for a new generation of startups. AI is transitioning from a tool for communication to a tool for creation and discovery in the physical world. This is where the next wave of generational companies will be built. The "hard-to-solve" problems are now within reach.

NVIDIA CEO Jensen Huang just gave us a glimpse into the future of AI, and it's far bigger than text and images. He asked: "If you can understand worlds, is it possible that you understand proteins and chemicals that have structure?” This is the quiet revolution happening in AI right now. We're moving from an AI that understands human language to an AI that understands the language of nature itself. The "letters" are atoms. The "words" are molecules. The "sentences" are the complex structures of life, networks, and reality. The same foundational models that generate video are now being used to: Decode proteins to fight disease Design new materials at the atomic level *Simulate complex systems in quantum computing These were once the domain of supercomputers and Nobel laureates. Now, they're becoming solvable problems for a new generation of startups. AI is transitioning from a tool for communication to a tool for creation and discovery in the physical world. This is where the next wave of generational companies will be built. The "hard-to-solve" problems are now within reach.

Konstantine Buhler

13,014 views • 10 months ago

Introducing FlexOlmo, a new paradigm for language model training that enables the co-development of AI through data collaboration. 🧵

Introducing FlexOlmo, a new paradigm for language model training that enables the co-development of AI through data collaboration. 🧵

Ai2

377,039 views • 1 year ago

ODesign: A World Model for Biomolecular Interaction Design | Odin Zhang et al. - "ODesign allows scientists to specify epitopes on arbitrary targets and generate diverse classes of binding partners with fine-grained control. Across entity-, token-, and atom-level benchmarks in the protein modality, ODesign demonstrates superior controllability and performance to modality-specific baselines. Extending beyond proteins, it generalizes to nucleic acid and small-molecule design, enabling interaction types such as protein-binding RNA/DNA and RNA/DNA-binding ligands that were previously inaccessible." Technical Report: Project Page:

ODesign: A World Model for Biomolecular Interaction Design | Odin Zhang et al. - "ODesign allows scientists to specify epitopes on arbitrary targets and generate diverse classes of binding partners with fine-grained control. Across entity-, token-, and atom-level benchmarks in the protein modality, ODesign demonstrates superior controllability and performance to modality-specific baselines. Extending beyond proteins, it generalizes to nucleic acid and small-molecule design, enabling interaction types such as protein-binding RNA/DNA and RNA/DNA-binding ligands that were previously inaccessible." Technical Report: Project Page:

Leo Zang

17,064 views • 10 months ago

New Anthropic research: Emotion concepts and their function in a large language model. All LLMs sometimes act like they have emotions. But why? We found internal representations of emotion concepts that can drive Claude’s behavior, sometimes in surprising ways.

New Anthropic research: Emotion concepts and their function in a large language model. All LLMs sometimes act like they have emotions. But why? We found internal representations of emotion concepts that can drive Claude’s behavior, sometimes in surprising ways.

Anthropic

3,944,870 views • 5 months ago

Proteins are the machinery of life. Scientists have cataloged billions of protein sequences—but their biology is still mostly unknown. Today we're releasing a world model of protein biology: a scientific engine for prediction, design, and discovery that consists of ESMFold2, ESMC, and ESM Atlas. Together, they're helping to open up a new way for researchers to design proteins and speed up scientific discovery. Our mission is to cure or prevent disease. To do that, we need to accelerate science. That's why we're releasing all three openly.

Proteins are the machinery of life. Scientists have cataloged billions of protein sequences—but their biology is still mostly unknown. Today we're releasing a world model of protein biology: a scientific engine for prediction, design, and discovery that consists of ESMFold2, ESMC, and ESM Atlas. Together, they're helping to open up a new way for researchers to design proteins and speed up scientific discovery. Our mission is to cure or prevent disease. To do that, we need to accelerate science. That's why we're releasing all three openly.

biohub

79,510 views • 3 months ago