Loading video...
Video Failed to Load
We have developed DELIVR, a user-friendly deep-learning pipeline for automatically detecting and analyzing cells from whole-brain images to map neuronal activity via cFOS. DELIVR uses virtual reality to substantially accelerate the generation of training data and is accessible to biologists without advanced coding skills through a simple Fiji plugin.... show more
15,773 views • 1 year ago •via X (Twitter)
0 Comments
No comments available
Comments from the original post will appear here
![This is probably the most entertaining way to understand one of AI’s hardest AI debates. Transformer vs Post-Transformer, argued by leading researchers, inside a real physical boxing ring. Both technically deep and genuinely entertaining. I was glued for the entire 1 hour 20 minutes. So many super cool points to learn. 🥊 Transformers - Transformers still own the present because they work at scale. They are simple, trainable, hardware-friendly, and already power the strongest AI systems we use today. - The Transformer is basically a memory machine. It stores information as keys and values, then uses attention to pull back the most useful parts when answering. - The real Transformer advantage is not just “attention.” The bigger advantage is that it fits modern hardware extremely well, so it can process huge batches of tokens fast. - Scaling is still the brutal rule. If you give Transformers more compute, more data, and more parameters, they usually keep getting better. Any Post-Transformer architecture has to scale just as well, or better. - It is not enough to look clever on small tests, because the real question is whether it improves faster than Transformers when scaled up. - A replacement cannot be slightly better. Because the whole AI stack is already built around Transformers, the next architecture may need to be around 10x better to force everyone to switch. - Transformers are powerful, but they may be brute force. A human does not need to read the entire internet many times to become smart, but current LLMs need enormous data and compute. 🥊 Post-Transformer - Post-Transformer people are not saying Transformers are bad. They are saying Transformers may be the best current tool, not the final form of machine intelligence. - The biggest Post-Transformer target is native reasoning and continual learning. Today’s LLM reasoning often feels like text-based step-by-step work added on top, instead of thinking happening naturally inside the model. - Latent reasoning is one possible next step. That means the model reasons inside its own hidden internal space, instead of writing every thought out as words. - Continual learning is still a major weakness. Humans keep learning from experience, but most Transformer-based models are trained, frozen, and then only adapt inside the prompt. - Long context is not the same as real memory. A model can read a huge prompt, but that is different from building a life history, learning from mistakes, and updating beliefs over time. - The future may be hybrid, not a clean replacement. Transformers may stay as 1 building block while newer systems add better memory, better reasoning, and better learning loops. - The most interesting possibility is that Transformers may help discover their own successor. AI agents are already getting better at research and coding, so the next architecture may come from AI-assisted architecture search. ------- - Benchmarks are a problem. Many public benchmarks are easy to game, so they may show leaderboard strength without proving deeper intelligence. - Perplexity is still probably a great metric to evaluate frontier models,, because it tests prediction quality. --- Overall, Transformers continue to dominate, but the frontier is clearly widening. Pathway’s BDH (Dragon Hatchling — brain-inspired reasoning architecture), Sakana AI’s CTMs (Continuous Thought Machines — models that think over time), and Liquid AI’s LFMs (Liquid Foundation Models — efficient multimodal foundation models) - all of these show how the frontier is expanding. --- From “Pathway (pathway[.]com)” Youtube channel (link in comment) Zuzanna Stamirowska](https://image.24vids.com/tw-2060361249477795942/amplify_video_thumb/2060360206559965185/img/bgZldwDwZ0I6C1r0.jpg)
![Legendary technologist Jann Tallin: “Extinction from godlike AI is not just possible, but imminent.” “We are close.” “AI will not leave any survivors“ “On the current trajectory, you are not going to live very long” “A recent poll found that 88% of AI engineers think that AI could destroy the world.” PARTIAL TRANSCRIPT: “Humanity is akin to a teenager with rapidly developing physical abilities, lagging wisdom and self control, little thought for its long term future, and an unhealthy appetite for risk. There is an increasing consensus: Alan Turing, in 1951, predicted that we should expect to lose control to machines, and the inventor of deep learning itself, Geoff Hinton, starting to have doubts about his life work. There are now hundreds of AI experts sounding their alarm bells. A recent poll found that 76% of American voters believe AI is a threat to our existence. Just yesterday, there was news that one of the leading superforecaster groups published their prediction that their estimate for AI catastrophic risk is 30%. 30%! The battle for establishing that AI is an existential risk, a battle that I spent roughly 15 years of my career on, has now all but won. I'm going to show that there are fundamental reasons why underlying godlike AI will not leave any survivors. That we are now close to such AI but have no idea how to align it. And how skeptics’ counterarguments are, sadly, extremely weak. [AI will be like a new apex species. And humans - an apex species - have driven other countless other species to extinction.] Godlike AI will not care about humans because of a dirty secret of the AI industry: AIs are not built, they are grown. The ‘p’ in Chat GPT stands for pretrained. Pretraining - "summoning" - is a process where simple two page program is soaked in terabytes of data and megawatts of electricity and left like that for months. And then, after that, attempts are made to tame the emergent alien mind. Importantly, those methods of taming rely on the AI being less competent than the humans who are taming it now. The reason why we expect that we are close to godlike AI is that the trend of AI is getting more powerful and is now visible to everyone. It's obvious. Just look at capability differences between GPT2, GPT3, and GPT4. GPT-2 was released in 2019. A simple extrapolation would take us to GPT-7 before this decade is over. So, in summary, we are blindly growing increasingly competent minds while hoping that they are not so competent that they spin out of control and destroy our living environment. Unfortunately, that hope is not justified, which explains increasing anxiety among the AI developers themselves. Of course, at this point, just like a patient that has received a terminal diagnosis, you are encouraged to seek a second opinion. Unfortunately, having been part of this debate for more than a decade, I already know what you're going to hear. First, labeling. These are arguments like “Oh, this is science fiction” or “This is alarmism” “These are doomsayers” “Don't listen to people with that non-virtuous property, X.” Second, frame control. “AI is like X, and X is very nice, right?” This has now reached grotesque levels. One prominent VC claimed recently that “AI is basically just math, so why should we worry?” Imagine the captain of Titanic announcing, “don't worry, passengers, this is just water.” Third class of arguments, human supremacy. “AI can never do X” Or “we are very far from AI doing X.” Unfortunately, reality has been very harsh judge here recently. The set of things that only humans can do is collapsing really rapidly. There's now growing global consensus that the unregulated, blind AI scaling is reckless and dangerous. So we need to constrain AI or ban AI altogether. Just like we banned human cloning. You have received a terminal diagnosis. Please don't simply ignore it.” --- Jann Tallin is founder of Skype and Kazaa](https://image.24vids.com/tw-1749384479519457298/ext_tw_video_thumb/1749383664159387648/pu/img/ks6RSq7HRZHpjBHu.jpg)
