Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Scaling laws describe how loss changes with scale. Do neurons inside models change predictably too? We study vision and language models up to 30B params and find systematic scaling in neuron universality, specialization, and selectivity. Paper+code: 1/n

Amil Dravid

1,292 subscribers

205,643 views • 1 month ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

E11 Bio is excited to unveil PRISM technology for mapping brain wiring with simple light microscopes. Today, brain mapping in humans and other mammals is bottlenecked by accurate neuron tracing. PRISM uses molecular ID codes and AI to help neurons trace themselves. We discovered a new cell barcoding approach exceeding comparable methods by more than 750x. This is the heart of PRISM. We integrated this capability with microscopy and AI image analysis to automatically trace neurons at high resolution and annotate them with molecular features. This is a key advance towards economically viable brain mapping - 95% of costs stem from neuron tracing. It is also an important step towards democratizing neuron tracing for everyday neuroscience. Solving these problems is critical for curing brain disorders, building safer and human-like AI, and even simulating brain function. In our first pilot study, we acquired a unique dataset in mouse hippocampus. Barcodes improved the accuracy of tracing genetically labelled neurons by 8x – with a clear path to 100x or more. They also permit tracing across spatial gaps – essential for mitigating tissue section loss in whole-brain scaling. Using molecular annotation, we uncover an intriguing feature of synaptic organization, demonstrating how PRISM can be used for systematic discovery 🧵

E11 Bio is excited to unveil PRISM technology for mapping brain wiring with simple light microscopes. Today, brain mapping in humans and other mammals is bottlenecked by accurate neuron tracing. PRISM uses molecular ID codes and AI to help neurons trace themselves. We discovered a new cell barcoding approach exceeding comparable methods by more than 750x. This is the heart of PRISM. We integrated this capability with microscopy and AI image analysis to automatically trace neurons at high resolution and annotate them with molecular features. This is a key advance towards economically viable brain mapping - 95% of costs stem from neuron tracing. It is also an important step towards democratizing neuron tracing for everyday neuroscience. Solving these problems is critical for curing brain disorders, building safer and human-like AI, and even simulating brain function. In our first pilot study, we acquired a unique dataset in mouse hippocampus. Barcodes improved the accuracy of tracing genetically labelled neurons by 8x – with a clear path to 100x or more. They also permit tracing across spatial gaps – essential for mitigating tissue section loss in whole-brain scaling. Using molecular annotation, we uncover an intriguing feature of synaptic organization, demonstrating how PRISM can be used for systematic discovery 🧵

Andrew Payne

112,064 views • 9 months ago

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots. We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:

(1/6) X-Humanoid 🤖: Scaling up data for Humanoid Robots. We convert human daily activity videos (from Ego-Exo4D) into humanoid videos (i.e., Tesla Optimus) performing tasks like cooking or fixing a bike. This data can be potentially used to train robot policies and world models. 🔥 Project page: Paper link:

Mike Shou

87,622 views • 7 months ago

My first PhD paper!🎉We learn *diffusion* models for code generation that learn to directly *edit* syntax trees of programs. The result is a system that can incrementally write code, see the execution output, and debug it. 🧵1/n

My first PhD paper!🎉We learn diffusion models for code generation that learn to directly edit syntax trees of programs. The result is a system that can incrementally write code, see the execution output, and debug it. 🧵1/n

Shreyas Kapur

742,493 views • 2 years ago

Imagine you want to learn about the technology behind LLMs. You instantly get an 100 page book with chapters, content, and structure. What if you find the language too technical? You can change the prompt and the book – all 100 pages – adapts to your needs. What if you want to learn how to code models in python? You can ask for that too, and in seconds, every chapter has code examples alongside the instructional content. That’s the power of an Infinite Bookshelf. Open source, Powered by Groq Inc

Imagine you want to learn about the technology behind LLMs. You instantly get an 100 page book with chapters, content, and structure. What if you find the language too technical? You can change the prompt and the book – all 100 pages – adapts to your needs. What if you want to learn how to code models in python? You can ask for that too, and in seconds, every chapter has code examples alongside the instructional content. That’s the power of an Infinite Bookshelf. Open source, Powered by Groq Inc

Ben Klieger

62,600 views • 1 year ago

TOMORROW, 9:30AM PT We go live with an Odyssey announcement (or two) I've been looking forward to for some time. I think these releases will make where world models are headed, and how powerful they can become, much more obvious. So hyped. GPUs scaling as we speak.

TOMORROW, 9:30AM PT We go live with an Odyssey announcement (or two) I've been looking forward to for some time. I think these releases will make where world models are headed, and how powerful they can become, much more obvious. So hyped. GPUs scaling as we speak.

Oliver Cameron

79,586 views • 2 months ago

AI’s next frontier is Spatial Intelligence, a technology that will turn seeing into reasoning, perception into action, and imagination into creation. But what is it? Why does it matter? How do we build it? And how can we use it? Today, I want to share with you my thoughts on building and using world models to unlock spatial intelligence in this essay below. 1/n

AI’s next frontier is Spatial Intelligence, a technology that will turn seeing into reasoning, perception into action, and imagination into creation. But what is it? Why does it matter? How do we build it? And how can we use it? Today, I want to share with you my thoughts on building and using world models to unlock spatial intelligence in this essay below. 1/n

Fei-Fei Li

926,730 views • 8 months ago

Video understanding is the next frontier, but not all videos are alike. Models now reason over youtube clips and feature films, but what about the everyday spaces we—and our future AI assistants—navigate and experience? Introducing Thinking in Space, our latest study exploring how multimodal LLMs see, remember and recall spaces. 🧵[1/n]

Video understanding is the next frontier, but not all videos are alike. Models now reason over youtube clips and feature films, but what about the everyday spaces we—and our future AI assistants—navigate and experience? Introducing Thinking in Space, our latest study exploring how multimodal LLMs see, remember and recall spaces. 🧵[1/n]

Saining Xie

209,460 views • 1 year ago

First ever color Kindle Scribe is here and think customers are going to really like it. We completely redesigned it from the ground up, and it’s now ultra-thin, writing and sketching feels like it’s happening on paper, and moving between pages is almost instantaneous. The AI we’ve baked in (featuring our Nova models) is especially useful, helping people find their notes, create note summaries, and ask questions of their notes. Think this is gonna change how folks work.

First ever color Kindle Scribe is here and think customers are going to really like it. We completely redesigned it from the ground up, and it’s now ultra-thin, writing and sketching feels like it’s happening on paper, and moving between pages is almost instantaneous. The AI we’ve baked in (featuring our Nova models) is especially useful, helping people find their notes, create note summaries, and ask questions of their notes. Think this is gonna change how folks work.

Andy Jassy

65,109 views • 7 months ago

* SEGA GENESIS/MEGADRIVE - SCALING PART 1 * I've been exploring software scaling on the 68k cpu using a background layer rather than scaling sprites. There are a few advantages, no sprite boundaries to worry about so moving pixels is faster also cheaper vertical scaling by adjusting vertical scroll midscreen. The net result is you can manage larger scales more efficiently as the CPU can spend more time working on the horizontal scaling whilst the VDP helps with the vertical scaling (there is still a non trivial cpu cost there though). Here I'm scaling an early image of the Lufthoheit logo - but the goal is to use this in the game also for very large scales / either background effects or bosses. The scaler and Interupt handler are written in Assembly for speed and I think could be faster yet with some optimisations, word alignment moves etc. The scaler has the ability to expand > full screen in height (from a more limited base height) and up to 3x the image width at present. I need to add double buffering and possibly go a bit larger yet. I'm excited to get these effects into the game as large scaling effects we didn't often see on the Genesis ! CYBERDEOUS - Crouzet Laurent Carsten666

Shannon Birt

12,775 views • 1 year ago

What are black holes? How do we find and study them? NASA scientists have answers to these questions (and more!) in our new video series! Check out a clip, and dive deeper into black holes with “Astro-Investigates” on YouTube: #BlackHoleWeek

What are black holes? How do we find and study them? NASA scientists have answers to these questions (and more!) in our new video series! Check out a clip, and dive deeper into black holes with “Astro-Investigates” on YouTube: #BlackHoleWeek

NASA Universe

14,456 views • 2 years ago

With two teenagers I never realized how used to dirty floors I had gotten. Now my robot cleans up every night. We love our Matic Robots. The most advanced computer vision in a vacuum. And, yes, I have a list of everyone in computer vision (look at my profile and find “view lists” to find that).

Robert Scoble

25,973 views • 1 year ago

Introducing ASAL: Automating the Search for Artificial Life with Foundation Models Artificial Life (ALife) research holds key insights that can transform and accelerate progress in AI. By speeding up ALife discovery with AI, we accelerate our understanding of emergence, evolution, and intelligence–core principles that can inspire the next generation of AI systems! We proudly collaborated with MIT, OpenAI, Swiss AI Lab IDSIA, and Ken Stanley on this exciting project. Full Paper (Website): Full Paper (arxiv): Code: In this work, we propose a new algorithm called Automated Search for Artificial Life (“ASAL”) to automate the discovery of artificial life using vision-language foundation models. Instead of tediously hand-designing every tiny rule of an Alife simulation, simply describe the space of simulations to search over, and ASAL will automatically discover the most interesting and open-ended artificial lifeforms! Because of the generality of foundation models, ASAL can discover new lifeforms across a diverse range of seminal ALife simulations, including Boids, Particle Life, Game of Life, Lenia, and Neural Cellular Automata. ASAL even discovered novel cellular automata rules that are more open-ended and expressive than the original Conway’s Game of Life. We believe this new paradigm may reignite ALife research by overcoming the bottleneck of manually designed simulations, thus advancing beyond the limits of human ingenuity.

Introducing ASAL: Automating the Search for Artificial Life with Foundation Models Artificial Life (ALife) research holds key insights that can transform and accelerate progress in AI. By speeding up ALife discovery with AI, we accelerate our understanding of emergence, evolution, and intelligence–core principles that can inspire the next generation of AI systems! We proudly collaborated with MIT, OpenAI, Swiss AI Lab IDSIA, and Ken Stanley on this exciting project. Full Paper (Website): Full Paper (arxiv): Code: In this work, we propose a new algorithm called Automated Search for Artificial Life (“ASAL”) to automate the discovery of artificial life using vision-language foundation models. Instead of tediously hand-designing every tiny rule of an Alife simulation, simply describe the space of simulations to search over, and ASAL will automatically discover the most interesting and open-ended artificial lifeforms! Because of the generality of foundation models, ASAL can discover new lifeforms across a diverse range of seminal ALife simulations, including Boids, Particle Life, Game of Life, Lenia, and Neural Cellular Automata. ASAL even discovered novel cellular automata rules that are more open-ended and expressive than the original Conway’s Game of Life. We believe this new paradigm may reignite ALife research by overcoming the bottleneck of manually designed simulations, thus advancing beyond the limits of human ingenuity.

Sakana AI

750,610 views • 1 year ago

We have 6 different models of CNC machines already designed and validated for production but we would need crores to scale up production and offer it to other manufacturers. Hence, we only build machines for our own production facility. Capital is a bottleneck for us. Currently we just have a tiny space within our own factory which is our R&D lab. We are a team of mere three people. Scaling hardware isn’t easy but it’s something that must be done for us to truly be self-reliant. Really appreciate your efforts Sridhar Vembu to build these hardware tools here. We must build more factories and hardware products in the country. Long way to go before I sleep.

We have 6 different models of CNC machines already designed and validated for production but we would need crores to scale up production and offer it to other manufacturers. Hence, we only build machines for our own production facility. Capital is a bottleneck for us. Currently we just have a tiny space within our own factory which is our R&D lab. We are a team of mere three people. Scaling hardware isn’t easy but it’s something that must be done for us to truly be self-reliant. Really appreciate your efforts Sridhar Vembu to build these hardware tools here. We must build more factories and hardware products in the country. Long way to go before I sleep.

Vishakh Ranotra

12,538 views • 1 month ago

Box Hubs turns content chaos into a searchable, AI-powered knowledge base - built in minutes, no code required. The problem isn't lack of information. It's content sprawl making the right information impossible to find. Box Hubs changes that: 🔷Single source of truth that auto-updates when files change 🔷Ask natural language questions, get instant answers with citations via Box AI 🔷Drag-and-drop builder with multi-page portals, change notifications, and link previews 🔷Enterprise-grade security and governance from the start Stop duplicating files. Stop answering the same questions. Start putting your content to work. Find out how:

Box Hubs turns content chaos into a searchable, AI-powered knowledge base - built in minutes, no code required. The problem isn't lack of information. It's content sprawl making the right information impossible to find. Box Hubs changes that: 🔷Single source of truth that auto-updates when files change 🔷Ask natural language questions, get instant answers with citations via Box AI 🔷Drag-and-drop builder with multi-page portals, change notifications, and link previews 🔷Enterprise-grade security and governance from the start Stop duplicating files. Stop answering the same questions. Start putting your content to work. Find out how:

Box

251,013 views • 5 months ago

🚨 BREAKING: Mathematicians invented a kind of “QR code” for knots. That might sound simple. It isn’t. Knot theory has struggled for decades with a brutal tradeoff: the best invariants are hard to compute, and the easy ones are often too weak to separate complicated knots. This new one appears to do both: powerful enough to distinguish highly complex knots, and efficient enough to handle hundreds of crossings. That could be a serious leap for topology. Could a computable knot “QR code” change how we study complexity in mathematics itself? What do you think? Follow me for more frontier physics and maths.

🚨 BREAKING: Mathematicians invented a kind of “QR code” for knots. That might sound simple. It isn’t. Knot theory has struggled for decades with a brutal tradeoff: the best invariants are hard to compute, and the easy ones are often too weak to separate complicated knots. This new one appears to do both: powerful enough to distinguish highly complex knots, and efficient enough to handle hundreds of crossings. That could be a serious leap for topology. Could a computable knot “QR code” change how we study complexity in mathematics itself? What do you think? Follow me for more frontier physics and maths.

TheNewPhysics

11,370 views • 2 months ago

Brilliant open-source tool from Anthropic Visualize internal LLM pathways through open attribution graphs. → Attribution graphs reveal which neurons and features drive each output token, turning the LLM’s black box into a clear map. They open-sourced a library that generates attribution graphs to trace how LLMs arrive at outputs. These graphs reveal internal activation flows and feature influences in popular open-weights models. 🧠 Attribution Graphs Graphs link model activations to output tokens. Nodes represent neurons or features. Edges show influence strength. Researchers inspect paths to see which components drive each decision. 🛠️ Toolchain and Frontend Circuit-tracer library works with models like Gemma-2-2b and Llama-3.2-1b. Demo notebooks illustrate multi-step reasoning examples. Neuronpedia hosts an interactive UI for graph exploration, annotation, and sharing. 🔍 Hypothesis Testing Users tweak feature values in graphs, regenerate outputs, and observe changes. This lets researchers validate whether specific circuits control behaviors like language translation or reasoning chains.

Brilliant open-source tool from Anthropic Visualize internal LLM pathways through open attribution graphs. → Attribution graphs reveal which neurons and features drive each output token, turning the LLM’s black box into a clear map. They open-sourced a library that generates attribution graphs to trace how LLMs arrive at outputs. These graphs reveal internal activation flows and feature influences in popular open-weights models. 🧠 Attribution Graphs Graphs link model activations to output tokens. Nodes represent neurons or features. Edges show influence strength. Researchers inspect paths to see which components drive each decision. 🛠️ Toolchain and Frontend Circuit-tracer library works with models like Gemma-2-2b and Llama-3.2-1b. Demo notebooks illustrate multi-step reasoning examples. Neuronpedia hosts an interactive UI for graph exploration, annotation, and sharing. 🔍 Hypothesis Testing Users tweak feature values in graphs, regenerate outputs, and observe changes. This lets researchers validate whether specific circuits control behaviors like language translation or reasoning chains.

Rohan Paul

27,197 views • 1 year ago

Weekend read: 𝗔𝗴𝗲𝗻𝘁𝘀 𝘀𝗵𝗼𝘂𝗹𝗱 𝗯𝗲 𝗦𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 (𝗮𝗻𝗱 𝗗𝘂𝗿𝗮𝗯𝗹𝗲) Serverless platforms are amazing for many AI agents, especially if they do a lot of bursty work (chat sessions with long inactivity) or react to events (approvals). Two things are needed to make this work well: (1) Durable execution / orchestration, to let agents recover and suspend (scale to zero) when they wait for input. (2) A durable execution orchestrator that has a push model that works with serverless platforms. Super fun to see agents with restate go brrrrrrrrrr: starting as durable functions on Modal's compute platform, scaling up, running, scaling down. Just out of the box, no extra code, no workers deployed. 👇

Weekend read: 𝗔𝗴𝗲𝗻𝘁𝘀 𝘀𝗵𝗼𝘂𝗹𝗱 𝗯𝗲 𝗦𝗲𝗿𝘃𝗲𝗿𝗹𝗲𝘀𝘀 (𝗮𝗻𝗱 𝗗𝘂𝗿𝗮𝗯𝗹𝗲) Serverless platforms are amazing for many AI agents, especially if they do a lot of bursty work (chat sessions with long inactivity) or react to events (approvals). Two things are needed to make this work well: (1) Durable execution / orchestration, to let agents recover and suspend (scale to zero) when they wait for input. (2) A durable execution orchestrator that has a push model that works with serverless platforms. Super fun to see agents with restate go brrrrrrrrrr: starting as durable functions on Modal's compute platform, scaling up, running, scaling down. Just out of the box, no extra code, no workers deployed. 👇

Stephan Ewen

18,135 views • 9 months ago

We release Diamond Maps💎 unlocking accurate and efficient guidance for diffusion models. Our experiments show that our methods scale incredibly well. Excited to see what people will build with this! Accurate guidance has been a notoriously hard problem, but in this work, we’re bringing TWO (!) solutions to the table. The recipe for success: 1️⃣ Speed: Use distilled models (flow maps, mean flows, consistency models). 2️⃣ Exploration: Inject stochasticity to properly explore your search space. Because this fundamentally improves anything using flow matching and diffusion, we see a lot of potential for applications across audio, robotics, molecules, and beyond. Paper: Code: Huge thanks to an amazing team: Douglas Chen, Luca Eyring @ ICML26, Ishin Shah, Giri Anantharaman, Yutong (Kelly) He, Zeynep Akata, Tommi Jaakkola, Nicholas Boffi, and Max Simchowitz. It was awesome bringing this to life together!

We release Diamond Maps💎 unlocking accurate and efficient guidance for diffusion models. Our experiments show that our methods scale incredibly well. Excited to see what people will build with this! Accurate guidance has been a notoriously hard problem, but in this work, we’re bringing TWO (!) solutions to the table. The recipe for success: 1️⃣ Speed: Use distilled models (flow maps, mean flows, consistency models). 2️⃣ Exploration: Inject stochasticity to properly explore your search space. Because this fundamentally improves anything using flow matching and diffusion, we see a lot of potential for applications across audio, robotics, molecules, and beyond. Paper: Code: Huge thanks to an amazing team: Douglas Chen, Luca Eyring @ ICML26, Ishin Shah, Giri Anantharaman, Yutong (Kelly) He, Zeynep Akata, Tommi Jaakkola, Nicholas Boffi, and Max Simchowitz. It was awesome bringing this to life together!

Peter Holderrieth

60,103 views • 2 months ago

What if you could watch chromatin states change inside living neurons.....IN REAL TIME? Repost or share if you or someone you know dreams of this! The Solecki Lab is building an imaging-based biosensor platform to make chromatin biology visible, measurable, and experimentally testable as neurons mature and wire into brain circuits (see enhancer nucleosomes fluttering below). We’re looking for an Associate Scientist who wants to do more than run experiments. Someone who wants to help build the framework for seeing how the neuronal genome changes as the brain is built.

What if you could watch chromatin states change inside living neurons.....IN REAL TIME? Repost or share if you or someone you know dreams of this! The Solecki Lab is building an imaging-based biosensor platform to make chromatin biology visible, measurable, and experimentally testable as neurons mature and wire into brain circuits (see enhancer nucleosomes fluttering below). We’re looking for an Associate Scientist who wants to do more than run experiments. Someone who wants to help build the framework for seeing how the neuronal genome changes as the brain is built.

David Solecki (pronounced so_lets_ki 😉)

32,032 views • 25 days ago