Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🦓 Self-supervised multimodal ML is promising the next AI breakthrough - in our new work published in nature, we debut @CEBRAai: for self-supervised hypothesis- and discovery-driven science. 📝 💻 🦓 🧵⬇️

Mackenzie Weygandt Mathis, PhD

22,788 subscribers

249,881 views • 3 years ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

10 Comments

Mackenzie Mathis, PhD3 years ago

First, this is a story about people 🥳 @stes_io & @jinnnnnlee are co-first authors and it was absolute pleasure to work with them & see the 🦓 CEBRA magic come to life. A few happy/fun bits in the @Nature research briefing! 🔗

Mackenzie Mathis, PhD3 years ago

The work: we focused in on 4 open-access datasets: synthetic, hippocampus, sensorimotor, and vision across species & recording methods to show the general performance and many features of #CEBRA cc @AllenInstitute @sejdevries @PresNCM & more (🙏)

Mackenzie Mathis, PhD3 years ago

The strength of CEBRA is the flexibility & performance: no generative model = no restrictions on data, & it can be used unsupervised (Time) or w/labels (Behavior). Here, we demo on hippocampus with CEBRA-Time and then do hypo. testing for a quantitative readout of model fit!

Mackenzie Mathis, PhD3 years ago

One big #CEBRA moment for us was seeing the impressive video decoding results: 🐭🧠+🎥->🦓= - high perf. >95% accuracy at frame prediction 🔥 - highly similar latents across Neuropixels & 2-photon data ✅ - differences in performance across the visual system of 🐭

Mackenzie Mathis, PhD3 years ago

To be continued... we won't tweet all the science today, so stay tuned for more highlights later 🖤🦓🍾

Mackenzie Mathis, PhD3 years ago

✨🦓🖤 Our Research Briefing can now be found at - the problem, our solution, and future directions … 🚨including how CEBRA is not limited to neural data🚨— if you use t-SNE or UMAP, consider using CEBRA for more consistent and higher accuracy results

NeurodataWithoutBorders3 years ago

@Nature @cebraAI Exciting news! 🎉 The @cebraAI paper just got published in @Nature, using secondary analysis of an NWB dataset on the @DANDIArchive 🧠. This marks a new milestone in scientific research and collaboration. Congrats to the team! #OpenScience #research #collaboration #datasharing

Mackenzie Mathis, PhD3 years ago

@Nature @cebraAI @DANDIarchive Thanks very much! This is one thing we are proud of in this paper - all data was open source and useable! 🥰 #dandi #figshare #crcns

Fabian Theis3 years ago

@Nature @cebraAI Congrats Mackenzie and @stes_io - exciting new avenue, let’s try to learn eg cellular or other dynamics.

Ana Marija Jakšić, PhD🦄🧬🪰3 years ago

@sainsbury_tom @Nature @cebraAI Stunning work, love seeing it out!! Congratulations!!! 🥳 🍾 🥂

Related Videos

🧪 Breakthrough new method for producing fluorochemicals is published in Science Magazine today 🎥 Take a look at our “Behind the Paper” video…

🧪 Breakthrough new method for producing fluorochemicals is published in Science Magazine today 🎥 Take a look at our “Behind the Paper” video…

Oxford Chemistry

28,582 views • 3 years ago

Elon Musk says Self-Driving Cars is a solved problem. “Tesla has rolled out Robotaxi Service in a few cities, I think we will be very wide-spread by the end of this year within the US. We hope to get Supervised Self-Driving Approval in Europe hopefully next month—maybe a similar timing for China.”

Elon Musk says Self-Driving Cars is a solved problem. “Tesla has rolled out Robotaxi Service in a few cities, I think we will be very wide-spread by the end of this year within the US. We hope to get Supervised Self-Driving Approval in Europe hopefully next month—maybe a similar timing for China.”

Nic Cruz Patane

62,158 views • 6 months ago

Self-supervised representation learning looks a bit like RL. What if we literally use RL as a SSL method for visual representations? Turns out that it works quite well. In new work by Dibya Ghosh, we show how this can be done:

Self-supervised representation learning looks a bit like RL. What if we literally use RL as a SSL method for visual representations? Turns out that it works quite well. In new work by Dibya Ghosh, we show how this can be done:

Sergey Levine

48,751 views • 1 year ago

BREAKING: Tesla's FSD (Supervised) is officially launching today to the general public across Australia and New Zealand. All HW4 Model 3s and Model Ys now have access. Launch details: • Eligible customers will be able to upgrade their vehicle to enable FSD (Supervised) by way of an ‘over-the-air’ update from anywhere with a WiFi connection. • FSD (Supervised) is available as an optional package for every new Tesla vehicle purchased in Australia and New Zealand. • Customers who have already taken delivery of an eligible vehicle without purchasing Full Self-Driving Capability can upgrade by purchasing the FSD (Supervised) package. Tesla is also looking to make FSD (Supervised) available soon by way of a subscription model for eligible vehicles for $149/month (AUD). No timeline for HW3 owners yet. Source:

BREAKING: Tesla's FSD (Supervised) is officially launching today to the general public across Australia and New Zealand. All HW4 Model 3s and Model Ys now have access. Launch details: • Eligible customers will be able to upgrade their vehicle to enable FSD (Supervised) by way of an ‘over-the-air’ update from anywhere with a WiFi connection. • FSD (Supervised) is available as an optional package for every new Tesla vehicle purchased in Australia and New Zealand. • Customers who have already taken delivery of an eligible vehicle without purchasing Full Self-Driving Capability can upgrade by purchasing the FSD (Supervised) package. Tesla is also looking to make FSD (Supervised) available soon by way of a subscription model for eligible vehicles for $149/month (AUD). No timeline for HW3 owners yet. Source:

Sawyer Merritt

342,041 views • 10 months ago

As an AI researcher, are you interested in tracking trends from CV/NLP/ML to robotics—even Nature/Science. Our paper “Real Deep Research for AI, Robotics & Beyond” automates survey generation and trend/topic discovery across fields 🔥Explore RDR at

As an AI researcher, are you interested in tracking trends from CV/NLP/ML to robotics—even Nature/Science. Our paper “Real Deep Research for AI, Robotics & Beyond” automates survey generation and trend/topic discovery across fields 🔥Explore RDR at

Xueyan Zou

62,815 views • 9 months ago

ELON: SELF-DRIVING IS SOLVED “Some insurance companies have said Tesla Full Self-Driving is so safe they’re offering customers half price insurance if they use it. I think self-driving cars are essentially a solved problem at this point. Tesla has rolled out robotaxi service in a few cities and it will be very widespread by the end of this year in the U.S. We also hope to get supervised Full Self-Driving approval in Europe, hopefully next month.” Source: ELON CLIPS

ELON: SELF-DRIVING IS SOLVED “Some insurance companies have said Tesla Full Self-Driving is so safe they’re offering customers half price insurance if they use it. I think self-driving cars are essentially a solved problem at this point. Tesla has rolled out robotaxi service in a few cities and it will be very widespread by the end of this year in the U.S. We also hope to get supervised Full Self-Driving approval in Europe, hopefully next month.” Source: ELON CLIPS

Mario Nawfal

46,665 views • 6 months ago

📢 SHeaP: Self-Supervised Head Predictor Learned via 2D Gaussians 📢 Given a single input image, we predict accurate 3D head geometry, pose, and expression. Previous works (e.g. DECA, EMOCA) use differentiable mesh rasterization to learn a self-supervised head geometry predictor via a photometric reconstruction loss. We borrow these ideas, but our key insight is to replace the mesh rendering with 2D Gaussian Splatting. This leads to much higher accuracy of the underlying predicted geometry and thus more gradient signal during training. 🌍 🎥 Great work by Liam Schoneveld Davide Davoli Jiapeng Tang

📢 SHeaP: Self-Supervised Head Predictor Learned via 2D Gaussians 📢 Given a single input image, we predict accurate 3D head geometry, pose, and expression. Previous works (e.g. DECA, EMOCA) use differentiable mesh rasterization to learn a self-supervised head geometry predictor via a photometric reconstruction loss. We borrow these ideas, but our key insight is to replace the mesh rendering with 2D Gaussian Splatting. This leads to much higher accuracy of the underlying predicted geometry and thus more gradient signal during training. 🌍 🎥 Great work by Liam Schoneveld Davide Davoli Jiapeng Tang

Matthias Niessner

28,559 views • 1 year ago

We aim to strengthen the “tech” in BioNTech with our agreement to acquire InstaDeep to support our strategy to build world leading capabilities in #AI- and #ML-driven #drugdiscovery and development of next-generation immunotherapies and vaccines.

We aim to strengthen the “tech” in BioNTech with our agreement to acquire InstaDeep to support our strategy to build world leading capabilities in #AI- and #ML-driven #drugdiscovery and development of next-generation immunotherapies and vaccines.

BioNTech SE

33,314 views • 3 years ago

Good morning FSD users across the country. Tesla Full Self-Driving (Supervised) 12.5.6.3 is rolling out now to AI4 cars with the new end to end highway stack. Let us know if you get it and how it's working for you. Watch in 4K: 1x:

Good morning FSD users across the country. Tesla Full Self-Driving (Supervised) 12.5.6.3 is rolling out now to AI4 cars with the new end to end highway stack. Let us know if you get it and how it's working for you. Watch in 4K: 1x:

Whole Mars Catalog

69,615 views • 1 year ago

Pfizer CEO Albert Bourla believes we are closer than ever to the next major breakthrough in fighting cancer. While progress is being made in neuroscience and Alzheimer’s, cancer research is delivering game-changing advancements every year—with new discoveries monthly and significant milestones quarterly. The science has matured, and the future looks promising.

Pfizer CEO Albert Bourla believes we are closer than ever to the next major breakthrough in fighting cancer. While progress is being made in neuroscience and Alzheimer’s, cancer research is delivering game-changing advancements every year—with new discoveries monthly and significant milestones quarterly. The science has matured, and the future looks promising.

Camus

18,113 views • 1 year ago

Elon Musk: “In order to solve full self-driving properly, you have to solve real-world AI. The road networks are designed to work with a biological neural net — our brains — and with vision — our eyes. So, in order to make it work with computers, you need to solve real-world AI and vision. We need cameras and silicon neural nets for self-driving to work in a system designed for eyes and biological neural nets. When you put it that way, it’s quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision.”

Elon Musk: “In order to solve full self-driving properly, you have to solve real-world AI. The road networks are designed to work with a biological neural net — our brains — and with vision — our eyes. So, in order to make it work with computers, you need to solve real-world AI and vision. We need cameras and silicon neural nets for self-driving to work in a system designed for eyes and biological neural nets. When you put it that way, it’s quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision.”

Elonogy

14,331 views • 1 month ago

We’re excited to share our latest work published today in Nature Biotechnology: Protein2PAM, an AI model that enables the rapid design of CRISPR editors with new PAM recognition And we’re making the model freely available for research and commercial use:

We’re excited to share our latest work published today in Nature Biotechnology: Protein2PAM, an AI model that enables the rapid design of CRISPR editors with new PAM recognition And we’re making the model freely available for research and commercial use:

Profluent

16,511 views • 5 months ago

Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes Contributions: • We propose STORM, the first feed-forward, self-supervised method for fast and accurate reconstruction of dynamic 3D scenes from sparse, multi-timestep, posed camera images. • Our bottom-up framework aggregates and transforms per-frame 3D Gaussian Splats into a cohesive scene representation, enabling self-supervised motion estimation. Furthermore, we introduce motion tokens that capture common motion primitives and regularize motion predictions, facilitating dynamic motion group segmentation without explicit motion or correspondence supervision. • We present several enhancements for in-the-wild scenarios, including sky modeling, camera exposure inconsistency handling, large novel-view extrapolation, and fine-grained human motions reconstruction, making STORM well-suited for real-world applications.

Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenes Contributions: • We propose STORM, the first feed-forward, self-supervised method for fast and accurate reconstruction of dynamic 3D scenes from sparse, multi-timestep, posed camera images. • Our bottom-up framework aggregates and transforms per-frame 3D Gaussian Splats into a cohesive scene representation, enabling self-supervised motion estimation. Furthermore, we introduce motion tokens that capture common motion primitives and regularize motion predictions, facilitating dynamic motion group segmentation without explicit motion or correspondence supervision. • We present several enhancements for in-the-wild scenarios, including sky modeling, camera exposure inconsistency handling, large novel-view extrapolation, and fine-grained human motions reconstruction, making STORM well-suited for real-world applications.

MrNeRF

53,292 views • 1 year ago

AI agents won’t just assist us, they’ll become our primary partners in how we work. Aidan Gomez, Cofounder and CEO of Cohere, shares his perspective on the future of AI-driven productivity and why open collaboration is essential for building what’s next. #AdvancingAI

AI agents won’t just assist us, they’ll become our primary partners in how we work. Aidan Gomez, Cofounder and CEO of Cohere, shares his perspective on the future of AI-driven productivity and why open collaboration is essential for building what’s next. #AdvancingAI

AMD

53,591 views • 1 year ago

Computer science is for everyone, and the LGBTQ+ community thrives in this field. It's a space for self-discovery and connection. Code is the new language, and using it to build inclusive tech is crucial. Diversity fuels innovation! When a variety of voices come together, groundbreaking solutions emerge. Let's make tech fair and equitable for all. #CodeForAll

Computer science is for everyone, and the LGBTQ+ community thrives in this field. It's a space for self-discovery and connection. Code is the new language, and using it to build inclusive tech is crucial. Diversity fuels innovation! When a variety of voices come together, groundbreaking solutions emerge. Let's make tech fair and equitable for all. #CodeForAll

CodeAI

15,847 views • 2 years ago

🚀 New $VOX Utility is LIVE! 🚀 The NeuraVox team keeps delivering AI innovation, and today we’re introducing our latest breakthrough: 🎥 AI Video Agent Podcast – bringing a face to AI for the most immersive experience yet! 🔹 Realistic AI-driven interactions 🔹 Next-gen engagement 🔹 A new era of AI communication 🔥 Available to try in few hours ! #VOX #AI #Crypto

🚀 New $VOX Utility is LIVE! 🚀 The NeuraVox team keeps delivering AI innovation, and today we’re introducing our latest breakthrough: 🎥 AI Video Agent Podcast – bringing a face to AI for the most immersive experience yet! 🔹 Realistic AI-driven interactions 🔹 Next-gen engagement 🔹 A new era of AI communication 🔥 Available to try in few hours ! #VOX #AI #Crypto

NeuraVox

23,924 views • 1 year ago

Elon Musk: The only way to solve full self-driving properly is to solve real-world AI. ”What are the road networks designed to work with? They're designed to work with a biological neural net, our brains, and with vision, our eyes. And so, in order to make it work with computers, you basically need to solve real-world AI and vision. We need cameras and silicon neural nets in order to have self-driving work for a system that was designed for eyes and biological neural nets. I guess when you put it that way, it's quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision.” TED Interview, April 14, 2022

Elon Musk: The only way to solve full self-driving properly is to solve real-world AI. ”What are the road networks designed to work with? They're designed to work with a biological neural net, our brains, and with vision, our eyes. And so, in order to make it work with computers, you basically need to solve real-world AI and vision. We need cameras and silicon neural nets in order to have self-driving work for a system that was designed for eyes and biological neural nets. I guess when you put it that way, it's quite obvious that the only way to solve full self-driving is to solve real-world AI and sophisticated vision.” TED Interview, April 14, 2022

ELON CLIPS

20,411 views • 1 year ago

Modern AI is confined to the digital world. At Skild AI, we are building towards AGI for the real world, unconstrained by robot type or task — a single, omni-bodied brain. Today, we are sharing our journey, starting with early milestones, with more to come in the weeks ahead. Our Mission: Artificial General Intelligence grounded in the physical world. We believe AGI that can truly understand and reason in the real world can only be built through grounding in the physical world. Our Vision: Any robot, Any task, One brain. We tackle robotics in its full generality – building a continually improving, omni-bodied brain that can control any hardware for any task. Who are we? A passionate group of scientists & engineers driven by our shared vision. We have been researching AI and robotics for more than a decade. Our team includes pioneers of self-supervised learning, curiosity-driven exploration, end-to-end sim2real for visual locomotion, dexterous manipulation, learning from human videos, robot parkour, and many more. Many of these works have won awards at top-tier AI and Robotics conferences. Our team has also built production-ready systems at Anduril, Tesla, Nvidia, Meta, Kitty Hawk, Google, Everyday Robotics, and Amazon. Join us in our mission to build the robot brains of tomorrow.

Modern AI is confined to the digital world. At Skild AI, we are building towards AGI for the real world, unconstrained by robot type or task — a single, omni-bodied brain. Today, we are sharing our journey, starting with early milestones, with more to come in the weeks ahead. Our Mission: Artificial General Intelligence grounded in the physical world. We believe AGI that can truly understand and reason in the real world can only be built through grounding in the physical world. Our Vision: Any robot, Any task, One brain. We tackle robotics in its full generality – building a continually improving, omni-bodied brain that can control any hardware for any task. Who are we? A passionate group of scientists & engineers driven by our shared vision. We have been researching AI and robotics for more than a decade. Our team includes pioneers of self-supervised learning, curiosity-driven exploration, end-to-end sim2real for visual locomotion, dexterous manipulation, learning from human videos, robot parkour, and many more. Many of these works have won awards at top-tier AI and Robotics conferences. Our team has also built production-ready systems at Anduril, Tesla, Nvidia, Meta, Kitty Hawk, Google, Everyday Robotics, and Amazon. Join us in our mission to build the robot brains of tomorrow.

Skild AI

382,738 views • 1 year ago

Since Tesla FSD Supervised got approved to use in Belgium so much footage has been published here on X. We cannot be anything but impressed. I will add one more to that list. This was our first testdrive with FSD Supervised in Belgium.🇧🇪 FSD avoided an crash with a Range Rover. It stopped instantly and saw the car before we saw it. The Range Rover was driving too fast and stopped last minute. The footage doesn’t show us exactly how it felt but believe me when I say Tesla_Calle and myself were deeply impressed. This week we will get a Tesla with FSD Supervised to use for a few days. Can’t wait to also start posting about it here! Thanks for the amazing work to make Europe safer Tesla Europe, Middle East & Africa

Since Tesla FSD Supervised got approved to use in Belgium so much footage has been published here on X. We cannot be anything but impressed. I will add one more to that list. This was our first testdrive with FSD Supervised in Belgium.🇧🇪 FSD avoided an crash with a Range Rover. It stopped instantly and saw the car before we saw it. The Range Rover was driving too fast and stopped last minute. The footage doesn’t show us exactly how it felt but believe me when I say Tesla_Calle and myself were deeply impressed. This week we will get a Tesla with FSD Supervised to use for a few days. Can’t wait to also start posting about it here! Thanks for the amazing work to make Europe safer Tesla Europe, Middle East & Africa

Esther Rebers

11,599 views • 27 days ago

NEWS: For the first time, Tesla has launched a whole page on its website dedicated just to FSD, and they revealed a couple new pieces of info on it! • 3.8 billion miles have bene driven on FSD (Supervised) • FSD (Supervised) is 54% safer than a human driver when it's engaged • FSD (Supervised) is trained on what amounts to over 100 years of anonymous real-world driving scenarios from Tesla's fleet of over six million vehicles. Tesla's fleet collectively experiences a lifetime of driving scenarios in 10 minutes. Link to the FSD page in the thread below.

NEWS: For the first time, Tesla has launched a whole page on its website dedicated just to FSD, and they revealed a couple new pieces of info on it! • 3.8 billion miles have bene driven on FSD (Supervised) • FSD (Supervised) is 54% safer than a human driver when it's engaged • FSD (Supervised) is trained on what amounts to over 100 years of anonymous real-world driving scenarios from Tesla's fleet of over six million vehicles. Tesla's fleet collectively experiences a lifetime of driving scenarios in 10 minutes. Link to the FSD page in the thread below.

Sawyer Merritt

367,796 views • 1 year ago