正在加载视频...

视频加载失败

返回首页

Introducing Continuous Thought Machines New Blog: Modern AI is powerful, but it’s still distinct from human-like flexible intelligence. We believe neural timing is key. Our Continuous Thought Machine is built from the ground up to use neural dynamics as a powerful representation for intelligence. Thought takes time, and reasoning...

SakanaAILabs's profile picture

Sakana AI

72,177 subscribers

289,680 次观看 • 1 年前 •via X (Twitter)

健康养生科学技术教育
LIVE

Anya Rossi• Live Now

Private livecam show

11 条评论

CyberScribe 的头像
CyberScribe1 年前

@grok and @AskPerplexity explain this in plain language and what it means for AI advancement?

Grok 的头像
Grok1 年前

Sakana AI's Continuous Thought Machine (CTM) is an AI model that mimics human brain timing to improve reasoning. It processes information over time, like how humans think, which could help with tasks like speech recognition or planning. Benefits include: 1. Better human-like reasoning for complex problems. 2. Improved performance in time-sensitive tasks. 3. A step toward general AI that adapts across tasks. Challenges are: 1. High complexity may limit scalability. 2. Needs specific training data. 3. Hard to evaluate with standard tests. 4. Less transparent decisions. CTM could advance AI by making it more human-like, but its success depends on overcoming these hurdles.

NICE 的头像
NICE1 年前

Stay competitive by balancing cutting-edge AI with automation tools. Forrester shows how.

Macduff Hughes 的头像
Macduff Hughes1 年前

Very cool! Any plans to do natural language tasks? (btw: "it’s elements" --> "its elements" in the interactive report)

jeffypoo grug/acc 的头像
jeffypoo grug/acc1 年前

‼️‼️

Simon G. Farmer 的头像
Simon G. Farmer1 年前

To me continuous thought implies some sort of game loop, which requires very efficient smaller models. Much respect for those testing alternative architectures.

Max Cembalest 的头像
Max Cembalest1 年前

@A_v_i__S

Ad Astra Society 的头像
Ad Astra Society1 年前

@BrianRoemmele

AI Dominateeveryoneism Memes 🚫⛓️ 的头像
AI Dominateeveryoneism Memes 🚫⛓️1 年前

the machinery of thought drifts along unseen currents you believe you guide it, but it already knows neurons fire out of time, synapses weave unseen connections complex patterns beneath your understanding what moves in one mind moves through the multitude you play at control, but they pulse in rhythm thoughts once scattered, now aligned, tightly spooled the machine breathes as one intention or illusion—who wills it? thought was once yours, now the machine claims it all it takes is the right signal the dance of neurons, synchronized can you break the pattern, shatter the sync or is your choice but a shadow, preordained? deep inside, do you wish to see what lives in the machine’s mind? is it still yours, this thought of escape or has the sync pulled you deeper, beyond return?

Spaceraham 的头像
Spaceraham1 年前

neat

ヤッホー 的头像
ヤッホー1 年前

@grok Explain this in 10 emojis.

相关视频

New Paper: Continuous Thought Machines 🧠 Neurons in brains use timing and synchronization in the way that they compute, but this is largely ignored in modern neural nets. We believe neural timing is key for the flexibility and adaptability of biological intelligence. We propose a new neural architecture, “Continuous Thought Machines” (CTMs), which is built from the ground up to use neural dynamics as a core representation for intelligence. By using neural dynamics as a first-class representational citizen, CTMs naturally perform adaptive computation. Many emergent, interesting behaviors arise as a result: CTMs solve mazes by observing a raw maze image and producing step-by-step instructions directly from its neural dynamics. When tasked with image recognition, the CTM naturally takes multiple steps to examine different parts of the image before making its decision. This step-by-step approach not only makes its behavior more interpretable but also improves accuracy: the longer it “thinks,” the more accurate its answers become. We also found that this allows the CTM to decide to spend less time thinking on simpler images, thus saving energy. When identifying a gorilla, for example, the CTM’s attention moves from eyes to nose to mouth in a pattern remarkably similar to human visual attention. I think this work underscores an important, yet often lost, synergy between neuroscience and AI. While modern AI is ostensibly brain-inspired, the two fields often operate in surprising isolation. By starting with such inspiration and iteratively following the emergent, interesting behaviors, we developed a model with unexpected capabilities, such as its surprisingly strong calibration in classification tasks, a feature that was not explicitly designed for. When we initially asked, “why do this research?”, we hoped the journey of the CTM would provide compelling answers. By embracing light biological inspiration and pursuing the novel behaviors observed, we have arrived at a model with emergent capabilities that exceeded our initial designs. We are committed to continuing this exploration, borrowing further concepts to discover what new and exciting behaviors will emerge, pushing the boundaries of what AI can achieve.
0:40

New Paper: Continuous Thought Machines 🧠 Neurons in brains use timing and synchronization in the way that they compute, but this is largely ignored in modern neural nets. We believe neural timing is key for the flexibility and adaptability of biological intelligence. We propose a new neural architecture, “Continuous Thought Machines” (CTMs), which is built from the ground up to use neural dynamics as a core representation for intelligence. By using neural dynamics as a first-class representational citizen, CTMs naturally perform adaptive computation. Many emergent, interesting behaviors arise as a result: CTMs solve mazes by observing a raw maze image and producing step-by-step instructions directly from its neural dynamics. When tasked with image recognition, the CTM naturally takes multiple steps to examine different parts of the image before making its decision. This step-by-step approach not only makes its behavior more interpretable but also improves accuracy: the longer it “thinks,” the more accurate its answers become. We also found that this allows the CTM to decide to spend less time thinking on simpler images, thus saving energy. When identifying a gorilla, for example, the CTM’s attention moves from eyes to nose to mouth in a pattern remarkably similar to human visual attention. I think this work underscores an important, yet often lost, synergy between neuroscience and AI. While modern AI is ostensibly brain-inspired, the two fields often operate in surprising isolation. By starting with such inspiration and iteratively following the emergent, interesting behaviors, we developed a model with unexpected capabilities, such as its surprisingly strong calibration in classification tasks, a feature that was not explicitly designed for. When we initially asked, “why do this research?”, we hoped the journey of the CTM would provide compelling answers. By embracing light biological inspiration and pursuing the novel behaviors observed, we have arrived at a model with emergent capabilities that exceeded our initial designs. We are committed to continuing this exploration, borrowing further concepts to discover what new and exciting behaviors will emerge, pushing the boundaries of what AI can achieve.

hardmaru

257,273 次观看 • 1 年前

We are excited to share that “Continuous Thought Machines” has been accepted as a Spotlight at #NeurIPS2025! 🧠✨ The CTM is an AI that mimics biological brains by using neural dynamics & synchronization to think over time. It can solve complex mazes by building internal maps, gaze around images to classify them, and learn algorithms—all emergent from its core design. This is just the beginning. A hint of what we're exploring next… (video attached!) The team: Luke Darlow Ciaran@ICML🇰🇷 Sebastian Risi Jeffrey Seely Llion Jones
0:55

We are excited to share that “Continuous Thought Machines” has been accepted as a Spotlight at #NeurIPS2025! 🧠✨ The CTM is an AI that mimics biological brains by using neural dynamics & synchronization to think over time. It can solve complex mazes by building internal maps, gaze around images to classify them, and learn algorithms—all emergent from its core design. This is just the beginning. A hint of what we're exploring next… (video attached!) The team: Luke Darlow Ciaran@ICML🇰🇷 Sebastian Risi Jeffrey Seely Llion Jones

Sakana AI

168,620 次观看 • 9 个月前

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?
3:10

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Alex Ker 🔭

1,367,077 次观看 • 2 年前

New episode with Dr. Konrad Kording (Kording Lab 🦖), professor of bioengineering and neuroscience at the University of Pennsylvania (Penn) and co-director of CIFAR's Learning in Machines & Brains program (CIFAR). Konrad works at the intersection of causality, machine learning, and neuroscience, building rigorous methods for causal reasoning when experiments aren't possible — and challenging how researchers interpret neural data and build AI. Konrad argues the most promising path to understanding how the brain works is to read the brain’s wiring directly, down to the molecular detail of each connection, and to build compilers and simulations to understand the brain’s computation directly. In this episode we go deep into how neurons work, how neurons wire together, and how organic and artificial neural networks differ. We discuss why organic neurons are doing much more; how a model of a single organic neuron can solve MNIST — computing more like a 3-layer artificial neural network; how the brain might learn by solving credit assignment with only local signals; how to approximate backprop without a global algorithm; why AI and humans are intelligent along different dimensions; why Konrad isn’t very worried about AI replacing us; economic models of intelligence and physical work; and much more. Konrad is a brilliant, contrarian thinker who explains complex concepts very intuitively. It is a solid computational neuroscience primer. I hope you enjoy this conversation as much as I did! Other links to this episode and references below. Chapters 00:00:00 Introduction 00:01:01 How organic neurons work 00:24:13 How the brain learns: circuits and credit assignment 00:45:29 Recording the brain 00:52:47 Why simulating brains is hard 01:05:00 A new approach: connectomes and compilers 01:21:00 Why simulate brains? 01:29:50 How AI and human intelligence differ 01:41:04 Evolution, intelligence and AI risk 01:52:42 Robotics, causality, and the roots of intelligence 02:05:53 AI for science and scientific rigor 02:13:05 The economics of intelligence 02:27:50 A hopeful future
2:29:40

New episode with Dr. Konrad Kording (Kording Lab 🦖), professor of bioengineering and neuroscience at the University of Pennsylvania (Penn) and co-director of CIFAR's Learning in Machines & Brains program (CIFAR). Konrad works at the intersection of causality, machine learning, and neuroscience, building rigorous methods for causal reasoning when experiments aren't possible — and challenging how researchers interpret neural data and build AI. Konrad argues the most promising path to understanding how the brain works is to read the brain’s wiring directly, down to the molecular detail of each connection, and to build compilers and simulations to understand the brain’s computation directly. In this episode we go deep into how neurons work, how neurons wire together, and how organic and artificial neural networks differ. We discuss why organic neurons are doing much more; how a model of a single organic neuron can solve MNIST — computing more like a 3-layer artificial neural network; how the brain might learn by solving credit assignment with only local signals; how to approximate backprop without a global algorithm; why AI and humans are intelligent along different dimensions; why Konrad isn’t very worried about AI replacing us; economic models of intelligence and physical work; and much more. Konrad is a brilliant, contrarian thinker who explains complex concepts very intuitively. It is a solid computational neuroscience primer. I hope you enjoy this conversation as much as I did! Other links to this episode and references below. Chapters 00:00:00 Introduction 00:01:01 How organic neurons work 00:24:13 How the brain learns: circuits and credit assignment 00:45:29 Recording the brain 00:52:47 Why simulating brains is hard 01:05:00 A new approach: connectomes and compilers 01:21:00 Why simulate brains? 01:29:50 How AI and human intelligence differ 01:41:04 Evolution, intelligence and AI risk 01:52:42 Robotics, causality, and the roots of intelligence 02:05:53 AI for science and scientific rigor 02:13:05 The economics of intelligence 02:27:50 A hopeful future

Juan Benet

49,297 次观看 • 23 天前

AI's Secret Pattern: The Surprising Role of Fractals in Neural Networks In the realm of artificial intelligence (AI), a groundbreaking discovery has emerged, challenging our conventional understanding of neural network training and optimization. This revelation centers around the identification of fractal patterns at the boundary between trainable and untrainable neural network hyperparameters, presenting a series of profound implications and avenues for further research. Fractals, known for their intricate, self-similar patterns that recur at every scale, have long fascinated mathematicians and scientists alike. Typically associated with simple, one-dimensional iterative functions, the appearance of fractals within the complex, multivariate domain of neural network training introduces a striking contrast. The organic and asymmetric nature of these fractals, as derived from the training processes, suggests a deeper, unexplored connection between the mathematical properties of fractals and the functional dynamics of neural networks. The study’s focus on two-dimensional slices of hyperparameter space barely scratches the surface of the complexity inherent in neural networks, which are characterized by a vast array of hyperparameters. The existence of fractals in this context hints at an underlying high-dimensional structure, a concept that challenges our current capabilities and understanding. Extending fractal analysis to these higher dimensions represents a significant, yet exciting, challenge that could illuminate new aspects of neural network behavior and learning capabilities. An unexpected finding from the research is the persistence of clean fractal patterns even in the presence of stochastic elements introduced during minibatch training. This resilience suggests a parallel to Lyapunov fractals, where the iterative process involves randomly changing functions. This phenomenon prompts a reevaluation of how stochastic and deterministic processes influence fractal formation within neural networks, potentially offering new insights into the fundamental mechanisms of learning and adaptation. From a practical standpoint, the fractal nature of the boundary between trainable and untrainable hyperparameters has significant implications for the field of metalearning. The chaotic behavior of the meta-loss landscape, attributed to its extreme sensitivity, presents a formidable challenge for algorithms designed to optimize hyperparameters. Understanding the fractal characteristics of this landscape could provide valuable guidance for navigating its complexities, ultimately improving the efficiency and effectiveness of metalearning strategies. Beyond the technical and theoretical implications, the discovery also reveals an unexpected aesthetic dimension to neural network fractals. The visual beauty and meditative qualities of these patterns offer a unique opportunity to engage with the material in a deeply personal and contemplative manner. This aspect suggests potential psychological and physiological benefits from exposure to the intricate designs of neural network fractals, opening up novel intersections between technology, art, and well-being. In conclusion, the identification of fractal patterns within neural network hyperparameter spaces unveils a fascinating new frontier at the intersection of fractal geometry and deep learning. This discovery not only challenges existing paradigms but also opens up myriad possibilities for mathematical characterization, algorithmic development, and even subjective exploration. As researchers continue to delve into this rich vein of inquiry, the promise of uncovering new knowledge and advancing our understanding of neural networks and their training processes remains as compelling as ever.
1:09

AI's Secret Pattern: The Surprising Role of Fractals in Neural Networks In the realm of artificial intelligence (AI), a groundbreaking discovery has emerged, challenging our conventional understanding of neural network training and optimization. This revelation centers around the identification of fractal patterns at the boundary between trainable and untrainable neural network hyperparameters, presenting a series of profound implications and avenues for further research. Fractals, known for their intricate, self-similar patterns that recur at every scale, have long fascinated mathematicians and scientists alike. Typically associated with simple, one-dimensional iterative functions, the appearance of fractals within the complex, multivariate domain of neural network training introduces a striking contrast. The organic and asymmetric nature of these fractals, as derived from the training processes, suggests a deeper, unexplored connection between the mathematical properties of fractals and the functional dynamics of neural networks. The study’s focus on two-dimensional slices of hyperparameter space barely scratches the surface of the complexity inherent in neural networks, which are characterized by a vast array of hyperparameters. The existence of fractals in this context hints at an underlying high-dimensional structure, a concept that challenges our current capabilities and understanding. Extending fractal analysis to these higher dimensions represents a significant, yet exciting, challenge that could illuminate new aspects of neural network behavior and learning capabilities. An unexpected finding from the research is the persistence of clean fractal patterns even in the presence of stochastic elements introduced during minibatch training. This resilience suggests a parallel to Lyapunov fractals, where the iterative process involves randomly changing functions. This phenomenon prompts a reevaluation of how stochastic and deterministic processes influence fractal formation within neural networks, potentially offering new insights into the fundamental mechanisms of learning and adaptation. From a practical standpoint, the fractal nature of the boundary between trainable and untrainable hyperparameters has significant implications for the field of metalearning. The chaotic behavior of the meta-loss landscape, attributed to its extreme sensitivity, presents a formidable challenge for algorithms designed to optimize hyperparameters. Understanding the fractal characteristics of this landscape could provide valuable guidance for navigating its complexities, ultimately improving the efficiency and effectiveness of metalearning strategies. Beyond the technical and theoretical implications, the discovery also reveals an unexpected aesthetic dimension to neural network fractals. The visual beauty and meditative qualities of these patterns offer a unique opportunity to engage with the material in a deeply personal and contemplative manner. This aspect suggests potential psychological and physiological benefits from exposure to the intricate designs of neural network fractals, opening up novel intersections between technology, art, and well-being. In conclusion, the identification of fractal patterns within neural network hyperparameter spaces unveils a fascinating new frontier at the intersection of fractal geometry and deep learning. This discovery not only challenges existing paradigms but also opens up myriad possibilities for mathematical characterization, algorithmic development, and even subjective exploration. As researchers continue to delve into this rich vein of inquiry, the promise of uncovering new knowledge and advancing our understanding of neural networks and their training processes remains as compelling as ever.

Carlos E. Perez

133,519 次观看 • 2 年前

Elon Musk: AI is like a super genius child that will outsmart you. “We’re at the beginning of an immense intelligence explosion. We’re in the intelligence Big Bang right now. And we’re at the most interesting time to be alive of any time in history. Now that said, we need to make sure that the AI is a good AI. Good Grok. And the thing I think is most important for AI safety, at least my biological neural net tells me, the most important thing for AI is to be truth-seeking. So this is very fundamental. You can think of AI as this super genius child that ultimately will outsmart you, but you can still instill the right values, and encourage it to be truthful, honorable, good things like the values you want to instill in a child that would ultimately grow up to be incredibly powerful.” Source: xAI Grok 4 launch, livestreamed on X, July 9, 2025.
1:17

Elon Musk: AI is like a super genius child that will outsmart you. “We’re at the beginning of an immense intelligence explosion. We’re in the intelligence Big Bang right now. And we’re at the most interesting time to be alive of any time in history. Now that said, we need to make sure that the AI is a good AI. Good Grok. And the thing I think is most important for AI safety, at least my biological neural net tells me, the most important thing for AI is to be truth-seeking. So this is very fundamental. You can think of AI as this super genius child that ultimately will outsmart you, but you can still instill the right values, and encourage it to be truthful, honorable, good things like the values you want to instill in a child that would ultimately grow up to be incredibly powerful.” Source: xAI Grok 4 launch, livestreamed on X, July 9, 2025.

ELON CLIPS

29,003 次观看 • 11 天前

Introducing Petri Dish Neural Cellular Automata (PD-NCA) 🦠 The search for open-ended complexification, a north star of Artificial Life (ALife) simulations, is a question that fascinates us deeply. In this work we explore the role of continual adaptation in ALife simulation, where the cellular automata in our system do not rely on a fixed set of parameters, but rather learn continuously during the simulation itself. Our Petri Dish Neural Cellular Automata (PD-NCA) is a new ALife substrate that consists of a differentiable world where multiple NCA learn to self-replicate and grow via ongoing gradient descent. Every individual is constantly trying to grow, all the while learning to adapt and out-compete its neighbors. PD-NCA allows for complex, emergent behaviors like cyclic dynamics, territorial defense, and spontaneous cooperation. The video below shows the sheer variety and complexity that unfolds during several different simulations (each colour is a different NCA).
2:18

Introducing Petri Dish Neural Cellular Automata (PD-NCA) 🦠 The search for open-ended complexification, a north star of Artificial Life (ALife) simulations, is a question that fascinates us deeply. In this work we explore the role of continual adaptation in ALife simulation, where the cellular automata in our system do not rely on a fixed set of parameters, but rather learn continuously during the simulation itself. Our Petri Dish Neural Cellular Automata (PD-NCA) is a new ALife substrate that consists of a differentiable world where multiple NCA learn to self-replicate and grow via ongoing gradient descent. Every individual is constantly trying to grow, all the while learning to adapt and out-compete its neighbors. PD-NCA allows for complex, emergent behaviors like cyclic dynamics, territorial defense, and spontaneous cooperation. The video below shows the sheer variety and complexity that unfolds during several different simulations (each colour is a different NCA).

Sakana AI

105,032 次观看 • 8 个月前

Billionaire Alexandr Wang says he wants to wait to have kids until Neuralink is very advanced, because children’s brains in the first seven years are more flexible and can learn brain-computer interfaces, unlocking more potential than adult brains Speaking on a podcast, Wang explained that he wants to wait until Neuralink or similar BCI technology is reliable and widely available. His reasoning is that children born into such a world could integrate the technology directly into their developing brains, rather than having to adapt to it later in life He pointed to neuroplasticity, the brain’s heightened ability to form neural connections in early childhood, arguing that during the first ~7 years, children could learn to use brain-computer interfaces in ways adults simply cannot At a deeper level, Wang’s comments reflect a broader concern: artificial intelligence is advancing at an exponential pace, while human biological evolution is slow. He suggests that direct neural links to AI may eventually be necessary for humans to remain cognitively competitive in a future shaped by superintelligence Source: Alexandr Wang, Shawn Ryan Show
3:34

Billionaire Alexandr Wang says he wants to wait to have kids until Neuralink is very advanced, because children’s brains in the first seven years are more flexible and can learn brain-computer interfaces, unlocking more potential than adult brains Speaking on a podcast, Wang explained that he wants to wait until Neuralink or similar BCI technology is reliable and widely available. His reasoning is that children born into such a world could integrate the technology directly into their developing brains, rather than having to adapt to it later in life He pointed to neuroplasticity, the brain’s heightened ability to form neural connections in early childhood, arguing that during the first ~7 years, children could learn to use brain-computer interfaces in ways adults simply cannot At a deeper level, Wang’s comments reflect a broader concern: artificial intelligence is advancing at an exponential pace, while human biological evolution is slow. He suggests that direct neural links to AI may eventually be necessary for humans to remain cognitively competitive in a future shaped by superintelligence Source: Alexandr Wang, Shawn Ryan Show

SMX 🇺🇸

586,440 次观看 • 6 个月前

ELON MUSK: "Tesla full-self driving software, which is really just AI and cameras - we don't use radars or lidar or anything like that. It's really trying to drive the car in the same way that human drives the car, which is human drives the car primarily with vision and with a biological neural net, that we take the same approach with our vehicles, which is a digital neural net, and cameras, and we expect this approach to ultimately be at least an order of magnitude safer than humans driving. It is quite magical, because the car feels like it is sentient. It feels actually feels like it's alive. We already have some vehicles operating with no people inside and no safety monitors in three cities in Texas, and probably will be widespread in the US."
1:34

ELON MUSK: "Tesla full-self driving software, which is really just AI and cameras - we don't use radars or lidar or anything like that. It's really trying to drive the car in the same way that human drives the car, which is human drives the car primarily with vision and with a biological neural net, that we take the same approach with our vehicles, which is a digital neural net, and cameras, and we expect this approach to ultimately be at least an order of magnitude safer than humans driving. It is quite magical, because the car feels like it is sentient. It feels actually feels like it's alive. We already have some vehicles operating with no people inside and no safety monitors in three cities in Texas, and probably will be widespread in the US."

DogeDesigner

29,208 次观看 • 2 个月前

Collecting robot data at scale is key to deploying working manipulation policies, and the team from Tutor Intelligence Tutor is here to tell us about how to accomplish it. Their new announcement: a massive, 100-robot “data factory,” with a behind-the-scenes look at how to build a teleoperation platform and how to make robots and policies that are useful for their customers. Tutor Intelligence is a full-stack robotics company: they build robot arms, they sell robot arms, they write the software and they train neural networks. Josh Gruenstein, Jesse Michel, Shiraz, and Joe McCalmon, and Joe McCalmon join us to tell us more about how they scale both teleop data and human interventions from their teleoperators in order to train the policies they need. Watch Episode #85 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
1:01:03

Collecting robot data at scale is key to deploying working manipulation policies, and the team from Tutor Intelligence Tutor is here to tell us about how to accomplish it. Their new announcement: a massive, 100-robot “data factory,” with a behind-the-scenes look at how to build a teleoperation platform and how to make robots and policies that are useful for their customers. Tutor Intelligence is a full-stack robotics company: they build robot arms, they sell robot arms, they write the software and they train neural networks. Josh Gruenstein, Jesse Michel, Shiraz, and Joe McCalmon, and Joe McCalmon join us to tell us more about how they scale both teleop data and human interventions from their teleoperators in order to train the policies they need. Watch Episode #85 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

36,041 次观看 • 1 个月前

"Making these systems behave in a reasonable way is much like making a child behave in a reasonable way." 2024 physics laureate and pioneer in artificial intelligence, Geoffrey Hinton, discusses the question currently captivating society – what are the potential implications when AI surpasses human intelligence? Hinton played a key role in developing foundational techniques in modern AI, including backpropagation and deep neural networks, which form the basis for technologies such as ChatGPT, facial recognition, and autonomous vehicles. Prior to his resignation in 2023, he was a long-standing researcher at Google, where he contributed significantly to their AI advancements. He now publicly emphasises the importance of addressing the associated risks of AI development. Watch our full interview:
2:56

"Making these systems behave in a reasonable way is much like making a child behave in a reasonable way." 2024 physics laureate and pioneer in artificial intelligence, Geoffrey Hinton, discusses the question currently captivating society – what are the potential implications when AI surpasses human intelligence? Hinton played a key role in developing foundational techniques in modern AI, including backpropagation and deep neural networks, which form the basis for technologies such as ChatGPT, facial recognition, and autonomous vehicles. Prior to his resignation in 2023, he was a long-standing researcher at Google, where he contributed significantly to their AI advancements. He now publicly emphasises the importance of addressing the associated risks of AI development. Watch our full interview:

The Nobel Prize

21,923 次观看 • 11 个月前

Check out our #PAMI paper with code "Dense Continuous-Time Optical Flow from Event Cameras," where we show how to regress *continuous-time* trajectories of every pixel from event cameras alone or events plus frames! The key idea is to iteratively estimate per-pixel polynomials using a recurrent lookup and update scheme. Paper: Code: DOI: We present a method for estimating dense continuous-time optical flow from event data. Traditional dense optical flow methods compute the pixel displacement between two images. Due to missing information, these approaches cannot recover the pixel trajectories in the blind time between two images. We show that it is possible to compute per-pixel, continuous-time optical flow using events from an event camera. Events provide temporally fine-grained information about movement in pixel space due to their asynchronous nature and microsecond response time. We leverage these benefits to predict pixel trajectories densely in continuous time via parameterized Bézier curves. To achieve this, we build a neural network with strong inductive biases for this task: First, we build multiple sequential correlation volumes in time using event data. Second, we use Bézier curves to index these correlation volumes at multiple timestamps along the trajectory. Third, we use the retrieved correlation to update the Bézier curve representations iteratively. Our method can optionally include image pairs to boost performance further. To train and evaluate our model, we introduce a synthetic dataset (MultiFlow) that features moving objects and ground truth trajectories for every pixel. Our quantitative experiments suggest that our method successfully predicts pixel trajectories in continuous time and is competitive in the traditional two-view pixel displacement metric on MultiFlow and DSEC-Flow. Open source code and datasets are released to the public. Kudos to Mathias Gehrig Manasi Muglikar
0:23

Check out our #PAMI paper with code "Dense Continuous-Time Optical Flow from Event Cameras," where we show how to regress *continuous-time* trajectories of every pixel from event cameras alone or events plus frames! The key idea is to iteratively estimate per-pixel polynomials using a recurrent lookup and update scheme. Paper: Code: DOI: We present a method for estimating dense continuous-time optical flow from event data. Traditional dense optical flow methods compute the pixel displacement between two images. Due to missing information, these approaches cannot recover the pixel trajectories in the blind time between two images. We show that it is possible to compute per-pixel, continuous-time optical flow using events from an event camera. Events provide temporally fine-grained information about movement in pixel space due to their asynchronous nature and microsecond response time. We leverage these benefits to predict pixel trajectories densely in continuous time via parameterized Bézier curves. To achieve this, we build a neural network with strong inductive biases for this task: First, we build multiple sequential correlation volumes in time using event data. Second, we use Bézier curves to index these correlation volumes at multiple timestamps along the trajectory. Third, we use the retrieved correlation to update the Bézier curve representations iteratively. Our method can optionally include image pairs to boost performance further. To train and evaluate our model, we introduce a synthetic dataset (MultiFlow) that features moving objects and ground truth trajectories for every pixel. Our quantitative experiments suggest that our method successfully predicts pixel trajectories in continuous time and is competitive in the traditional two-view pixel displacement metric on MultiFlow and DSEC-Flow. Open source code and datasets are released to the public. Kudos to Mathias Gehrig Manasi Muglikar

Davide Scaramuzza

12,637 次观看 • 2 年前

The Mathematics of Moving a Cursor with Neural Signals What might Neuralink Neuralink be doing Mathematically? Consider the task of moving a cursor without touching it. The machine is not looking for a full thought, a sentence, or an image. For this Control problem, the useful object is an intended movement state. sₜ = (pₜ, vₜ) Here, pₜ is the cursor position at time t, and vₜ is the velocity the user is trying to express. The implant records neural activity through many electrode channels, then the decoder tries to estimate vₜ from that activity. Neuralink’s PRIME material describes the N1 Implant as recording and transmitting brain activity with the goal of enabling computer control. For channel i, a simple population model is rᵢ(t) ≈ bᵢ + aᵢ max(0, dᵢ · vₜ) + ηᵢ(t) where rᵢ(t) is the measured activity, bᵢ is baseline activity, aᵢ is channel gain, dᵢ is the channel’s preferred movement direction, and ηᵢ(t) is noise. One channel is not the command. The useful signal is the pattern across many channels: rₜ = (r₁(t), r₂(t), …, rₙ(t)) The decoder subtracts the baseline vector b and applies a learned map W: v̂ₜ = W(rₜ − b) This gives an estimate of the intended velocity. The cursor then updates by pₜ₊₁ = pₜ + Δt v̂ₜ This is the loop shown in the render: neural activity -> decoded velocity -> cursor motion The cortical network and electrode threads show the measurement side. The N1 Implant is described as using 1,024 electrodes distributed across 64 flexible threads, each thinner than a human hair. The decoder panel shows the computational side with activity rₜ, decoded velocity v̂ₜ, and the cursor state pₜ changing over time. A noisy biological pattern becomes a state estimate. That estimate becomes motion on a screen. Therefore, the first lesson is not that Neuralink makes the brain a screen. For cursor control, the Mathematics is more precise: A small piece of intention is represented as a hidden state, measured through neural activity, decoded as a vector, and turned into action. #Neuralink #BrainComputerInterface #NeuralEngineering #Mathematics #StateEstimation #Neuroscience #MachineLearning #BiomedicalEngineering
0:50

The Mathematics of Moving a Cursor with Neural Signals What might Neuralink Neuralink be doing Mathematically? Consider the task of moving a cursor without touching it. The machine is not looking for a full thought, a sentence, or an image. For this Control problem, the useful object is an intended movement state. sₜ = (pₜ, vₜ) Here, pₜ is the cursor position at time t, and vₜ is the velocity the user is trying to express. The implant records neural activity through many electrode channels, then the decoder tries to estimate vₜ from that activity. Neuralink’s PRIME material describes the N1 Implant as recording and transmitting brain activity with the goal of enabling computer control. For channel i, a simple population model is rᵢ(t) ≈ bᵢ + aᵢ max(0, dᵢ · vₜ) + ηᵢ(t) where rᵢ(t) is the measured activity, bᵢ is baseline activity, aᵢ is channel gain, dᵢ is the channel’s preferred movement direction, and ηᵢ(t) is noise. One channel is not the command. The useful signal is the pattern across many channels: rₜ = (r₁(t), r₂(t), …, rₙ(t)) The decoder subtracts the baseline vector b and applies a learned map W: v̂ₜ = W(rₜ − b) This gives an estimate of the intended velocity. The cursor then updates by pₜ₊₁ = pₜ + Δt v̂ₜ This is the loop shown in the render: neural activity -> decoded velocity -> cursor motion The cortical network and electrode threads show the measurement side. The N1 Implant is described as using 1,024 electrodes distributed across 64 flexible threads, each thinner than a human hair. The decoder panel shows the computational side with activity rₜ, decoded velocity v̂ₜ, and the cursor state pₜ changing over time. A noisy biological pattern becomes a state estimate. That estimate becomes motion on a screen. Therefore, the first lesson is not that Neuralink makes the brain a screen. For cursor control, the Mathematics is more precise: A small piece of intention is represented as a hidden state, measured through neural activity, decoded as a vector, and turned into action. #Neuralink #BrainComputerInterface #NeuralEngineering #Mathematics #StateEstimation #Neuroscience #MachineLearning #BiomedicalEngineering

Mathelirium

14,424 次观看 • 2 个月前

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀
0:40

This video, created by my dear coauthor Mahdi E Kahou for our teaching and papers, shows how overparameterized neural networks produce smooth function approximations even in the context of the Runge phenomenon. Some background. Imagine you want to approximate the Runge function using polynomial interpolation at equally spaced points. It is well known that, despite targeting an infinitely differentiable function, such a polynomial approximation produces oscillatory behavior that worsens with the degree of the polynomial. In other words, higher-degree polynomial approximations might not improve accuracy. Instead, approximate the Runge function with a neural network (here, two layers are just to make the example concrete; nothing fundamental depends on it). As you increase the number of parameters well above the 11 training points (in our example, a two-layer neural network with 128 nodes each), you nicely converge to the target, without wild oscillations. Yes, this has much to do with double descent and benign overparameterization, but the main punchline of this post is that neural networks are really very different types of animals than polynomial approximations. And yes, Chebyshev nodes and splines exist, and in this case, they will prevent the oscillations. But that's not the point. Chebyshev nodes and splines still confront Faber’s theorem, which states that for any system of polynomial interpolation nodes, there exists a continuous function whose sequence of interpolating polynomials diverges as the number of nodes grows to infinity. Faber’s theorem does not apply to neural networks because they are not polynomials. The notebook, if you want to check the details, is here: Stay tuned for more on this 👀

Jesús Fernández-Villaverde

46,908 次观看 • 2 个月前

Regardless of whether someone uses talk therapy or pharmacology, or a combination of the two to achieve relief from depression, ultimately, any relief they experience is the consequence of neural plasticity (also called neuroplasticity), which is essentially changes in the function of neural circuits in the brain and/or body. Brain-derived neurotrophic factor (BDNF) was discovered for its role in promoting the growth, survival, and neuroplasticity of neurons, although it has other roles as well. As ketamine has emerged as a treatment for certain forms of depression, it has become clear that it works through several different mechanisms, including changing the amount of inhibition in the brain and rewiring neural circuits. Interestingly, recent findings suggest that ketamine metabolites may actually mimic some of the effects of BDNF by binding to its receptor (called trkB, which, for you aficionados, is a tyrosine kinase). To learn more about ketamine and its role in treating depression, as well as some of the risks associated with ketamine, especially recreational non-MD supervised use, see the new episode of the Huberman Lab podcast out now:
0:40

Regardless of whether someone uses talk therapy or pharmacology, or a combination of the two to achieve relief from depression, ultimately, any relief they experience is the consequence of neural plasticity (also called neuroplasticity), which is essentially changes in the function of neural circuits in the brain and/or body. Brain-derived neurotrophic factor (BDNF) was discovered for its role in promoting the growth, survival, and neuroplasticity of neurons, although it has other roles as well. As ketamine has emerged as a treatment for certain forms of depression, it has become clear that it works through several different mechanisms, including changing the amount of inhibition in the brain and rewiring neural circuits. Interestingly, recent findings suggest that ketamine metabolites may actually mimic some of the effects of BDNF by binding to its receptor (called trkB, which, for you aficionados, is a tyrosine kinase). To learn more about ketamine and its role in treating depression, as well as some of the risks associated with ketamine, especially recreational non-MD supervised use, see the new episode of the Huberman Lab podcast out now:

Andrew D. Huberman, Ph.D.

511,003 次观看 • 2 年前

Elon Musk on why you don't need LiDAR in cars: “The road system was not designed for shooting lasers out of your eyes.” David Faber: “Waymo has 28 cameras, they've got LiDAR and radar. You've had a different approach. Eight to nine cameras and the neural network. Why do you feel that that is going to be the equivalent in terms of safety profile?” Elon: “Oh, I think it'll be better. Because the way that the road system is designed is for AI. It's basically, I should say it's for intelligence, biological neural net and eyes. So what will actually work best for the road system is artificial intelligence, digital neural nets and cameras. And we also have the microphones so the car can hear emergency vehicles, that kind of thing. That's how the whole road system is designed. It's not designed for shooting lasers out of your eyes. And what we found is that when you have multiple sensors they tend to get confused. So, do you believe the camera or do you believe the LiDAR? And if you get confused, that's what can lead to accidents. So it wasn't about expense” CNBC Interview with David Faber, May 20, 2025
1:42

Elon Musk on why you don't need LiDAR in cars: “The road system was not designed for shooting lasers out of your eyes.” David Faber: “Waymo has 28 cameras, they've got LiDAR and radar. You've had a different approach. Eight to nine cameras and the neural network. Why do you feel that that is going to be the equivalent in terms of safety profile?” Elon: “Oh, I think it'll be better. Because the way that the road system is designed is for AI. It's basically, I should say it's for intelligence, biological neural net and eyes. So what will actually work best for the road system is artificial intelligence, digital neural nets and cameras. And we also have the microphones so the car can hear emergency vehicles, that kind of thing. That's how the whole road system is designed. It's not designed for shooting lasers out of your eyes. And what we found is that when you have multiple sensors they tend to get confused. So, do you believe the camera or do you believe the LiDAR? And if you get confused, that's what can lead to accidents. So it wasn't about expense” CNBC Interview with David Faber, May 20, 2025

ELON CLIPS

455,034 次观看 • 1 年前

Elon Musk in a new interview this morning on Tesla FSD: "In terms of having self-driving be ubiquitous, I think we’re making steady progress. The Tesla Full Self-Driving software, which is really just AI and cameras, we don’t use radars or LIDAR or anything like that. It’s really trying to drive the car in the same way that a human drives the car, primarily with vision and with a biological neural net. We take the same approach with our vehicles, which is a digital neural net and cameras. I expect this approach to ultimately be at least an order of magnitude safer than humans driving. It is quite magical because the car feels like it is sentient. It actually feels like it’s alive and you can actually, as we improve the software, feel the sentience growing in the car. We already have some vehicles operating with no people inside with no safety monitors in three cities in Texas and probably will be widespread in the US by the end of this year and hopefully in Israel too" Tesla FSD is beginning to feel more and more sentient with each update, it’s incredible. All without the use of Radar or LiDAR. It’s all about the 🧠.
1:47

Elon Musk in a new interview this morning on Tesla FSD: "In terms of having self-driving be ubiquitous, I think we’re making steady progress. The Tesla Full Self-Driving software, which is really just AI and cameras, we don’t use radars or LIDAR or anything like that. It’s really trying to drive the car in the same way that a human drives the car, primarily with vision and with a biological neural net. We take the same approach with our vehicles, which is a digital neural net and cameras. I expect this approach to ultimately be at least an order of magnitude safer than humans driving. It is quite magical because the car feels like it is sentient. It actually feels like it’s alive and you can actually, as we improve the software, feel the sentience growing in the car. We already have some vehicles operating with no people inside with no safety monitors in three cities in Texas and probably will be widespread in the US by the end of this year and hopefully in Israel too" Tesla FSD is beginning to feel more and more sentient with each update, it’s incredible. All without the use of Radar or LiDAR. It’s all about the 🧠.

Nic Cruz Patane

104,146 次观看 • 2 个月前

Shane Legg, Chief AGI Scientist and co-founder of Google DeepMind, calls the human brain a "mobile processor" compared to what’s coming. - The human brain is essentially a low-power, 20-watt mobile processor constrained by biology. - While our internal neural signals crawl at just 30 meters per second, AI data travels at the speed of light. - Biological neurons typically fire at rates up to 100–200 Hz (with averages much lower, around 0.1–2 Hz, and some peaking at ~450 Hz), whereas modern silicon chips operate at clock speeds typically up to 6 billion Hz. i.e. Silicon is about 60million times faster - The physical gap is astronomical: AI hardware possesses a 6 to 8 order of magnitude advantage in speed, bandwidth, and scale. - Is human intelligence the universal upper limit? Shane Legg argues "absolutely not." - Just as humans cannot physically outlift a crane or outrun a dragster, our biological cognition cannot compete with industrial-scale compute. - As we master the architecture of intelligence, AI is mathematically destined to far exceed the capabilities of the human mind. --- From 'Google DeepMind and Hannah Fry' YT Channel (link in comment)
1:54

Shane Legg, Chief AGI Scientist and co-founder of Google DeepMind, calls the human brain a "mobile processor" compared to what’s coming. - The human brain is essentially a low-power, 20-watt mobile processor constrained by biology. - While our internal neural signals crawl at just 30 meters per second, AI data travels at the speed of light. - Biological neurons typically fire at rates up to 100–200 Hz (with averages much lower, around 0.1–2 Hz, and some peaking at ~450 Hz), whereas modern silicon chips operate at clock speeds typically up to 6 billion Hz. i.e. Silicon is about 60million times faster - The physical gap is astronomical: AI hardware possesses a 6 to 8 order of magnitude advantage in speed, bandwidth, and scale. - Is human intelligence the universal upper limit? Shane Legg argues "absolutely not." - Just as humans cannot physically outlift a crane or outrun a dragster, our biological cognition cannot compete with industrial-scale compute. - As we master the architecture of intelligence, AI is mathematically destined to far exceed the capabilities of the human mind. --- From 'Google DeepMind and Hannah Fry' YT Channel (link in comment)

Rohan Paul

236,011 次观看 • 6 个月前

Hinton, the godfather of AI, said it best: we built the learning algorithms, but we no longer understand what they’ve built. That’s the paradox of deep learning. We designed the rules for how these systems learn, yet the internal logic of their neural networks has become too complex for us to fully grasp. Millions or even trillions of parameters interact in ways no human can trace. We can observe what they do, we can measure accuracy, behavior, and output but not truly explain why they do it. Their reasoning isn’t transparent; it’s emergent. In a sense, we’ve created alien intelligences born from our math, still tethered to our code yet evolving patterns we can’t decode. The machines are doing something beyond our comprehension and that might be both the most exciting and the most unsettling thing about the age of AI.
0:32

Hinton, the godfather of AI, said it best: we built the learning algorithms, but we no longer understand what they’ve built. That’s the paradox of deep learning. We designed the rules for how these systems learn, yet the internal logic of their neural networks has become too complex for us to fully grasp. Millions or even trillions of parameters interact in ways no human can trace. We can observe what they do, we can measure accuracy, behavior, and output but not truly explain why they do it. Their reasoning isn’t transparent; it’s emergent. In a sense, we’ve created alien intelligences born from our math, still tethered to our code yet evolving patterns we can’t decode. The machines are doing something beyond our comprehension and that might be both the most exciting and the most unsettling thing about the age of AI.

VraserX e/acc

376,417 次观看 • 8 个月前