Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

New lecture for the book! Nominally about synthetic data, but mostly is a walk through of the distillation literature from the Hinton 2015 paper to multi-teach on-policy distillation of today! At 7.4 hours of video in my post-training brain dump and counting :) It was fun to stare at...

natolambert's profile picture

Nathan Lambert

96,546 subscribers

92,497 Aufrufe • vor 1 Monat •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast
1:29

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

AI Evolution

15,565 Aufrufe • vor 2 Monaten

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹
1:31:19

I had an awesome time interviewing idan shenfeld and Jonas Hübotter from MIT and ETH Zurich about self-distillation. this very promising post-training paradigm where the model acts as its own teacher by conditioning on environment feedback or demonstrations. we cover the SDPO algo for reinforcement learning with rich feedback and SDFT for continual learning without forgetting along with many applications. we dig into how it works, why it's simpler and faster than GRPO, and where this is already showing up in production systems. table of content: 0:00 - what is self distillation 2:50 - idan (MIT) and jonas (ETH Zurich) introduction and motivation 18:40 - different perspective of on-policy self-distillation (presentation) 36:00 - metacognition and specificity in self-distillation 37:24 - very long hard task and self-distillation 42:00 - continual learning with self-distillation (presentation) 1:16:50 - what is next in this research direction? 1:20:00 - is there any experience with subjective feedbacks? 1:22:50 - quality vs number of feedbacks? 1:26:40 - what setting would self-distillation struggle vs GRPO? the slides were super crisp really cool of them to share! enjoy my guys 🌹

Yacine Mahdid

13,023 Aufrufe • vor 3 Monaten

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄
0:32

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

NVIDIA Robotics

22,413 Aufrufe • vor 1 Monat

Sequoia's Dean Meyer: "If distillation is real, the question is, how do we address the asymmetry?" "Western companies aren't going to set up 35,000 proxy accounts to steal from Anthropic." "There *is* a structural asymmetry, and it's something we have to think about." "The gap between closed and open source has closed a lot." "And the key conversation online, over the past week, was distillation the reason for this? Or is there some other, mysterious reason?" "The truth is, from the conversations we've had, is that it's somewhere in the middle." "Whether it's 20% or 40%, it doesn't really matter as much. But it's meaningful that we know it's happening." "We know from the labs that there's 10s of millions of queries through these large-scale proxy distillation accounts." "We also know, if you speak to anyone who's leading in synthetic data gen on pre- or post-training at the labs, they will tell you that even if you do hardcore KYC, and then just get access to the output (let's assume you just get the output from the prompt), it is still extremely valuable." "That is 100% true. We know distillation is real."
3:21

Sequoia's Dean Meyer: "If distillation is real, the question is, how do we address the asymmetry?" "Western companies aren't going to set up 35,000 proxy accounts to steal from Anthropic." "There *is* a structural asymmetry, and it's something we have to think about." "The gap between closed and open source has closed a lot." "And the key conversation online, over the past week, was distillation the reason for this? Or is there some other, mysterious reason?" "The truth is, from the conversations we've had, is that it's somewhere in the middle." "Whether it's 20% or 40%, it doesn't really matter as much. But it's meaningful that we know it's happening." "We know from the labs that there's 10s of millions of queries through these large-scale proxy distillation accounts." "We also know, if you speak to anyone who's leading in synthetic data gen on pre- or post-training at the labs, they will tell you that even if you do hardcore KYC, and then just get access to the output (let's assume you just get the output from the prompt), it is still extremely valuable." "That is 100% true. We know distillation is real."

TBPN

43,027 Aufrufe • vor 11 Tagen

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.
2:05

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 Aufrufe • vor 1 Jahr

#WATCH | Chicago, US: Chairman of Indian Overseas Congress, Sam Pitroda says, "...This is a policy issue. Congress party would frame a policy through which the wealth distribution would be better...We don't have a minimum wage (in India)...If we come up with a minimum wage in the country saying you must pay so much money to the poor, that's the distribution of wealth. Today, rich people don't pay their peons, servants, and home help enough but they spend that money on vacation in Dubai and London...When you talk about the distribution of wealth, it is not that you sit on a chair and say I have this much money and I'll be distributing it to everybody. It's naive to think like that. The PM of a country thinks like that, that means I got some concerns about his brain...You are really dealing with the policy issues to redistribute wealth and when you ask for the data, you are really trying to understand what is the distribution today. We don't have good data on all this. I think we need data to decide on policy issues. We don't need data to distribute wealth. We need data to decide on policy issues going forward..."
3:30

#WATCH | Chicago, US: Chairman of Indian Overseas Congress, Sam Pitroda says, "...This is a policy issue. Congress party would frame a policy through which the wealth distribution would be better...We don't have a minimum wage (in India)...If we come up with a minimum wage in the country saying you must pay so much money to the poor, that's the distribution of wealth. Today, rich people don't pay their peons, servants, and home help enough but they spend that money on vacation in Dubai and London...When you talk about the distribution of wealth, it is not that you sit on a chair and say I have this much money and I'll be distributing it to everybody. It's naive to think like that. The PM of a country thinks like that, that means I got some concerns about his brain...You are really dealing with the policy issues to redistribute wealth and when you ask for the data, you are really trying to understand what is the distribution today. We don't have good data on all this. I think we need data to decide on policy issues. We don't need data to distribute wealth. We need data to decide on policy issues going forward..."

ANI

244,459 Aufrufe • vor 2 Jahren

How to extract data from papers for literature review in seconds? 1. Go to 2. Upload 2-3 papers you already know are relevant 3. Start with clicking on one paper 4. ResearchCollab do the following for the paper ✦ Provide a short overview of the paper ✦ Identify weaknesses in the paper ✦ Share papers that contrast the existing paper ✦ Extract meta data about the paper ✦ Evaluate each part (e.g., methodology) of the paper ✦ Provide AI chat to extract any other data 5. Now click on the Related Papers button at top right 6. You will get papers related to the seed paper 7. You can click on each paper and read its abstract 8. If you find it relevant, just add it to your list of papers 9. Do the same for the other 2 seed papers 10. This way you will collect all relevant papers 11. And extract data from those papers After this, analyse the data and report the findings. Try ResearchCollab today:
0:47

How to extract data from papers for literature review in seconds? 1. Go to 2. Upload 2-3 papers you already know are relevant 3. Start with clicking on one paper 4. ResearchCollab do the following for the paper ✦ Provide a short overview of the paper ✦ Identify weaknesses in the paper ✦ Share papers that contrast the existing paper ✦ Extract meta data about the paper ✦ Evaluate each part (e.g., methodology) of the paper ✦ Provide AI chat to extract any other data 5. Now click on the Related Papers button at top right 6. You will get papers related to the seed paper 7. You can click on each paper and read its abstract 8. If you find it relevant, just add it to your list of papers 9. Do the same for the other 2 seed papers 10. This way you will collect all relevant papers 11. And extract data from those papers After this, analyse the data and report the findings. Try ResearchCollab today:

Faheem Ullah

26,494 Aufrufe • vor 6 Monaten

Today, we're joined by Aakanksha Chowdhery, member of technical staff at Reflection, to explore the fundamental shifts required to build true agentic AI. While the industry has largely focused on post-training techniques to improve reasoning, Aakanksha draws on her experience leading pre-training efforts for Google’s PaLM and early Gemini models to argue that pre-training itself must be rethought to move beyond static benchmarks. We explore the limitations of next-token prediction for multi-step workflows and examine how attention mechanisms, loss objectives, and training data must evolve to support long-form reasoning and planning. Aakanksha shares insights on the difference between context retrieval and actual reasoning, the importance of "trajectory" training data, and why scaling remains essential for discovering emergent agentic capabilities like error recovery and dynamic tool learning. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 02:26 - Reflection 04:54 - Limitations of post-training for building agents 07:31 - Rethinking pre-training in agents 10:51 - Scaling 11:27 - Evolving attention mechanisms for agentic capabilities 12:39 - Memory as a tool 14:13 - Loss objectives and training data 15:50 - Fine-tuning loss in agent performance 19:37 - Training data 21:29 - Augmenting dominant training data source 24:11 - Overcoming challenges in training on synthetic data 25:47 - Benchmarks 30:44 - Scaling laws in large models versus small models 33:20 - Long-form versus short-form reasoning 37:57 - Agent’s ability to recover from failure 40:15 - Hallucinations and failure recovery 43:53 - Tool use in agents 46:38 - Coding agents 48:37 - How researchers can contribute to agentic AI
51:54

Today, we're joined by Aakanksha Chowdhery, member of technical staff at Reflection, to explore the fundamental shifts required to build true agentic AI. While the industry has largely focused on post-training techniques to improve reasoning, Aakanksha draws on her experience leading pre-training efforts for Google’s PaLM and early Gemini models to argue that pre-training itself must be rethought to move beyond static benchmarks. We explore the limitations of next-token prediction for multi-step workflows and examine how attention mechanisms, loss objectives, and training data must evolve to support long-form reasoning and planning. Aakanksha shares insights on the difference between context retrieval and actual reasoning, the importance of "trajectory" training data, and why scaling remains essential for discovering emergent agentic capabilities like error recovery and dynamic tool learning. 🗒️ For the full list of resources for this episode, visit the show notes page: 📖 CHAPTERS =============================== 00:00 - Introduction 02:26 - Reflection 04:54 - Limitations of post-training for building agents 07:31 - Rethinking pre-training in agents 10:51 - Scaling 11:27 - Evolving attention mechanisms for agentic capabilities 12:39 - Memory as a tool 14:13 - Loss objectives and training data 15:50 - Fine-tuning loss in agent performance 19:37 - Training data 21:29 - Augmenting dominant training data source 24:11 - Overcoming challenges in training on synthetic data 25:47 - Benchmarks 30:44 - Scaling laws in large models versus small models 33:20 - Long-form versus short-form reasoning 37:57 - Agent’s ability to recover from failure 40:15 - Hallucinations and failure recovery 43:53 - Tool use in agents 46:38 - Coding agents 48:37 - How researchers can contribute to agentic AI

The TWIML AI Podcast

44,888 Aufrufe • vor 7 Monaten

Elon Musk On What It Takes To Build A Competitive AI Model Elon Musk breaks down the three factors that decide whether a foundation model can compete, and why the next frontier isn't human data at all. Speaking with Garry Tan, President and CEO of Y Combinator, Elon lays out what's actually required to build a large foundation model that's competitive: "You've got to get a lot of GPUs and have them train coherently and stably. Then it's like, what unique access to data do you have? I guess distribution matters to some degree as well like, how do people get exposed to your AI? Those are critical factors." But there's a problem with the data part of that equation. Echoing what a friend in the field has said, Elon explains that the industry has essentially run out of human-generated pre-training data: "You run out of tokens pretty fast, certainly of high-quality tokens. And then you need to essentially create synthetic data, and be able to accurately judge the synthetic data that you're creating, to verify: is this real synthetic data, or is it a hallucination that doesn't actually match reality?" That verification step is the hard part: "Achieving grounding in reality is tricky. But we are at the stage where there's more effort put into synthetic data. Right now we're training Grok 3.5, which is a heavy focus on reasoning." On reasoning, Garry Tan adds an interesting detail from researchers he's spoken to: hard science, particularly physics textbooks is very useful for training reasoning, whereas social science is "totally useless" for it. Elon's response: "Yes, that's probably true." He then points to where all of this is heading: "Something that's going to be very important in the future is combining deep AI in the data center or supercluster with robotics. So, things like the Optimus humanoid robot."
2:02

Elon Musk On What It Takes To Build A Competitive AI Model Elon Musk breaks down the three factors that decide whether a foundation model can compete, and why the next frontier isn't human data at all. Speaking with Garry Tan, President and CEO of Y Combinator, Elon lays out what's actually required to build a large foundation model that's competitive: "You've got to get a lot of GPUs and have them train coherently and stably. Then it's like, what unique access to data do you have? I guess distribution matters to some degree as well like, how do people get exposed to your AI? Those are critical factors." But there's a problem with the data part of that equation. Echoing what a friend in the field has said, Elon explains that the industry has essentially run out of human-generated pre-training data: "You run out of tokens pretty fast, certainly of high-quality tokens. And then you need to essentially create synthetic data, and be able to accurately judge the synthetic data that you're creating, to verify: is this real synthetic data, or is it a hallucination that doesn't actually match reality?" That verification step is the hard part: "Achieving grounding in reality is tricky. But we are at the stage where there's more effort put into synthetic data. Right now we're training Grok 3.5, which is a heavy focus on reasoning." On reasoning, Garry Tan adds an interesting detail from researchers he's spoken to: hard science, particularly physics textbooks is very useful for training reasoning, whereas social science is "totally useless" for it. Elon's response: "Yes, that's probably true." He then points to where all of this is heading: "Something that's going to be very important in the future is combining deep AI in the data center or supercluster with robotics. So, things like the Optimus humanoid robot."

High Signal AI

21,377 Aufrufe • vor 18 Tagen

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)
1:07

Jensen Huang on "distillation" On his new interview with axios, he was asked this question "Should open source model companies be allowed to distill closed models" "Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something. The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI. As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI." ---- From "Axios" YouTube channel, (full video link in comment)

Rohan Paul

440,435 Aufrufe • vor 14 Tagen

New podcast with finbarr! We survey the latest post-training recipes, from GLM 5.1, Kimi K2.6, DeepSeek V4, Xiaomi MiMo V2.5, Nemotron Ultra, etc. and discuss: - Why the industry slowly shifted to multi-teacher on-policy distillation (MOPD). - What an Olmo-style recipe would need improvements in - How post-training works / suits larger organizational efforts - Career advice in the foothills of the singularity - and other topics I heard y'all wanted me to start doing this, so making some time when I'm in funemployment! Chapters: 00:00 Introduction & Olmo reflections 06:28 Post-train recipes review (history) 23:00 2026’s model recipes (MiMo Flash, DeepSeek V4, GLM 5, Kimi K2.6, etc.) 39:05 Open-ended post-training discussions 48:22 Career advice in the LLM race Links below, please follow Interconnects and like and subscribe and buy my book?
56:35

New podcast with finbarr! We survey the latest post-training recipes, from GLM 5.1, Kimi K2.6, DeepSeek V4, Xiaomi MiMo V2.5, Nemotron Ultra, etc. and discuss: - Why the industry slowly shifted to multi-teacher on-policy distillation (MOPD). - What an Olmo-style recipe would need improvements in - How post-training works / suits larger organizational efforts - Career advice in the foothills of the singularity - and other topics I heard y'all wanted me to start doing this, so making some time when I'm in funemployment! Chapters: 00:00 Introduction & Olmo reflections 06:28 Post-train recipes review (history) 23:00 2026’s model recipes (MiMo Flash, DeepSeek V4, GLM 5, Kimi K2.6, etc.) 39:05 Open-ended post-training discussions 48:22 Career advice in the LLM race Links below, please follow Interconnects and like and subscribe and buy my book?

Nathan Lambert

42,732 Aufrufe • vor 1 Monat

Here's my conversation with Lucas Atkins and the team at Arcee.ai on their path to training and releasing Trinity Large today. From going all in on open models built end to end in the US 6 months ago to having the model in hand is no easy feet. I loved this conversation on how to design a startup around open models and take a bold step to scale it up. I'm openly an Arcee fan, watching them take risk and pull it off. We discuss: - The state (and future) of open vs. closed models, - The business of selling open models for on-prem deployments, - The story of Arcee AI & going “all-in” on this training run, - The ATOM project, - Building frontier model training teams in 6 months, - and other great topics. I really loved this one, and think you well too. Chapters: 00:00:00 Intro: Arcee AI, Trinity Models & Trinity Large 00:08:26 Transitioning a Company to Pre-training 00:13:00 Technical Decisions: Muon and MoE 00:18:41 Scaling and MoE Training Pain 00:23:14 Post-training and RL Strategies 00:28:09 Team Structure and Data Scaling 00:31:31 The Trinity Manifesto: US Open Weights 00:42:31 Specialized Models and Distillation 00:47:12 Infrastructure and Hosting 400B 00:50:53 Open Source as a Business Moat 00:56:31 Predictions: Best Model in 2026 01:02:29 Lightning Round & Conclusions More great open model builder podcasts coming soon!
1:12:15

Here's my conversation with Lucas Atkins and the team at Arcee.ai on their path to training and releasing Trinity Large today. From going all in on open models built end to end in the US 6 months ago to having the model in hand is no easy feet. I loved this conversation on how to design a startup around open models and take a bold step to scale it up. I'm openly an Arcee fan, watching them take risk and pull it off. We discuss: - The state (and future) of open vs. closed models, - The business of selling open models for on-prem deployments, - The story of Arcee AI & going “all-in” on this training run, - The ATOM project, - Building frontier model training teams in 6 months, - and other great topics. I really loved this one, and think you well too. Chapters: 00:00:00 Intro: Arcee AI, Trinity Models & Trinity Large 00:08:26 Transitioning a Company to Pre-training 00:13:00 Technical Decisions: Muon and MoE 00:18:41 Scaling and MoE Training Pain 00:23:14 Post-training and RL Strategies 00:28:09 Team Structure and Data Scaling 00:31:31 The Trinity Manifesto: US Open Weights 00:42:31 Specialized Models and Distillation 00:47:12 Infrastructure and Hosting 400B 00:50:53 Open Source as a Business Moat 00:56:31 Predictions: Best Model in 2026 01:02:29 Lightning Round & Conclusions More great open model builder podcasts coming soon!

Nathan Lambert

26,872 Aufrufe • vor 6 Monaten

Your AI is only as smart as the context it runs on. Have you built your shared enterprise brain yet? At Databricks Data+AI Summit, I spoke with Prukalpa ✨, co-founder and co-CEO of Atlan, to go deeper on the AI Context Layer in practice. The concept of a shared enterprise brain resonated with me. It is a foundation that every AI agent draws from, learns from, and contributes back to. "The future of organizations are going to be the fastest learning organizations." - Prukalpa ✨ Sankar, co-founder and co-CEO of Atlan Watch the full interview on YouTube: Check out the live demo by Atlan on Building Context for Your AI Agents - #EnterpriseAI #ContextLayer #Data #AI #AtlanPartner Matthias Heidrich
2:03

Your AI is only as smart as the context it runs on. Have you built your shared enterprise brain yet? At Databricks Data+AI Summit, I spoke with Prukalpa ✨, co-founder and co-CEO of Atlan, to go deeper on the AI Context Layer in practice. The concept of a shared enterprise brain resonated with me. It is a foundation that every AI agent draws from, learns from, and contributes back to. "The future of organizations are going to be the fastest learning organizations." - Prukalpa ✨ Sankar, co-founder and co-CEO of Atlan Watch the full interview on YouTube: Check out the live demo by Atlan on Building Context for Your AI Agents - #EnterpriseAI #ContextLayer #Data #AI #AtlanPartner Matthias Heidrich

Helen Yu

10,953 Aufrufe • vor 1 Monat

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:
0:36

Unreal Engine 5 is becoming the go-to platform for robotics teams, not just as a simulator but as a synthetic data factory powering the next generation of robotics systems🦾 From synthetic data to sim-to-real transfer, read our piece on how UE5 is quietly reshaping how robots learn to see, move, and act in the real world:

Unreal Engine

61,187 Aufrufe • vor 4 Monaten

Human generated data has fueled incredible AI progress, but what comes next? 📈 On the latest episode of our podcast, Hannah Fry and David Silver, VP of Reinforcement Learning, talk about how we could move from the era of relying on human data to one where AI could learn for itself. Watch now → 00:00 Introduction 01:50 Era of experience 03:45 AlphaZero 10:19 Move 37 15:20 Reinforcement learning and human feedback 24:30 AlphaProof 29:50 Math Olympiads 35:00 Experience based methods 42:56 Hannah's reflections 44:00 Fan Hui joins
49:30

Human generated data has fueled incredible AI progress, but what comes next? 📈 On the latest episode of our podcast, Hannah Fry and David Silver, VP of Reinforcement Learning, talk about how we could move from the era of relying on human data to one where AI could learn for itself. Watch now → 00:00 Introduction 01:50 Era of experience 03:45 AlphaZero 10:19 Move 37 15:20 Reinforcement learning and human feedback 24:30 AlphaProof 29:50 Math Olympiads 35:00 Experience based methods 42:56 Hannah's reflections 44:00 Fan Hui joins

Google DeepMind

227,202 Aufrufe • vor 1 Jahr

Hot take from my panel on decentralized AI at UC Berkeley: Your data is worthless. The idea that you'll get rich selling your data to train AI is a fantasy. The frontier is high-quality synthetic data, not your dumb tweets. Second hot take: the government censorship of AI has not started yet. It will come soon. And when it does, decentralized AI will matter in the same way that the rise of financial surveillance made Bitcoin matter. Spicy back and forth with Avichal - Electric ϟ Capital and Franklin Bi 👇
33:40

Hot take from my panel on decentralized AI at UC Berkeley: Your data is worthless. The idea that you'll get rich selling your data to train AI is a fantasy. The frontier is high-quality synthetic data, not your dumb tweets. Second hot take: the government censorship of AI has not started yet. It will come soon. And when it does, decentralized AI will matter in the same way that the rise of financial surveillance made Bitcoin matter. Spicy back and forth with Avichal - Electric ϟ Capital and Franklin Bi 👇

Haseeb ＞|＜

81,012 Aufrufe • vor 1 Jahr

அன்றே கணித்தார் Sriram Krishnan A year ago, on the Good time show, Sriram Krishnan and Aarthi Ramamurthy, along with Alexandr Wang, discussed how crypto can solve the problem of AI data attribution. They also talked about how, when a synthetic AI generation is created from a base data IP and a derivative IP is generated, the base data's IP owner should still receive rightful royalties. At the time, many weren’t aware of the significance, but now this has become a key focus for every DEAI crypto company. At Openledger, our mission is to build an AI DATA infrastructure from start to finish. We gather data from people, create personas from that data, attribute the right contributors—whether for data contribution or synthetic personas (derivatives)—and provide incentives. If you’re a tech founder, make sure to check out Aarthi and Sriram’s podcast at to get some real tech insights.
5:10

அன்றே கணித்தார் Sriram Krishnan A year ago, on the Good time show, Sriram Krishnan and Aarthi Ramamurthy, along with Alexandr Wang, discussed how crypto can solve the problem of AI data attribution. They also talked about how, when a synthetic AI generation is created from a base data IP and a derivative IP is generated, the base data's IP owner should still receive rightful royalties. At the time, many weren’t aware of the significance, but now this has become a key focus for every DEAI crypto company. At Openledger, our mission is to build an AI DATA infrastructure from start to finish. We gather data from people, create personas from that data, attribute the right contributors—whether for data contribution or synthetic personas (derivatives)—and provide incentives. If you’re a tech founder, make sure to check out Aarthi and Sriram’s podcast at to get some real tech insights.

Kamesh 𝐎𝐩𝐞𝐧𝐋𝐞𝐝𝐠𝐞𝐫 🇺🇸

24,736 Aufrufe • vor 1 Jahr

Perplexity CEO Aravind Srinivas on the biggest threat to the data center industry: It's not competition. It's not regulation. It's decentralisation. "The biggest threat to a data center is if the intelligence can be packed locally on a chip that's running on the device and then there's no need to inference all of it on like one centralized data center." He outlines how this could work in practice. Personalisation doesn't necessarily require on-device model training. Retrieval augmented generation, tool calls, and local data can already tailor AI to individual users. But the real unlock? Test time training. Aravind Srinivas describes a future where AI lives on your device, watches how you work and gradually automates your repetitive tasks. "Imagine we crack test time training where the AI watches tasks you repeatedly do on your local system, adapts to you over time and starts automating a lot of the things you do." The key insight: in this model, the intelligence belongs to you. It's your data, your device, your personalised AI brain. And if that future arrives, the economics of centralised infrastructure start to collapse. "That really disrupts the whole data center industry. It doesn't make sense to spend all this money, 500 billion, 5 trillion, whatever on building all the centralized data centers across the world that do a lot of the intelligence workloads for people." The companies spending trillions on centralised infrastructure may want to rethink where intelligence actually needs to live.
1:15

Perplexity CEO Aravind Srinivas on the biggest threat to the data center industry: It's not competition. It's not regulation. It's decentralisation. "The biggest threat to a data center is if the intelligence can be packed locally on a chip that's running on the device and then there's no need to inference all of it on like one centralized data center." He outlines how this could work in practice. Personalisation doesn't necessarily require on-device model training. Retrieval augmented generation, tool calls, and local data can already tailor AI to individual users. But the real unlock? Test time training. Aravind Srinivas describes a future where AI lives on your device, watches how you work and gradually automates your repetitive tasks. "Imagine we crack test time training where the AI watches tasks you repeatedly do on your local system, adapts to you over time and starts automating a lot of the things you do." The key insight: in this model, the intelligence belongs to you. It's your data, your device, your personalised AI brain. And if that future arrives, the economics of centralised infrastructure start to collapse. "That really disrupts the whole data center industry. It doesn't make sense to spend all this money, 500 billion, 5 trillion, whatever on building all the centralized data centers across the world that do a lot of the intelligence workloads for people." The companies spending trillions on centralised infrastructure may want to rethink where intelligence actually needs to live.

Big Brain AI

90,241 Aufrufe • vor 5 Monaten

PhD Students – How to extract data from papers for your literature review in seconds? Extracting data from papers takes a lot of time. You can automate this process with Bohrium 𝐇𝐨𝐰 𝐭𝐨 𝐚𝐮𝐭𝐨𝐦𝐚𝐭𝐢𝐜𝐚𝐥𝐥𝐲 𝐞𝐱𝐭𝐫𝐚𝐜𝐭 𝐝𝐚𝐭𝐚 𝐟𝐫𝐨𝐦 𝐩𝐚𝐩𝐞𝐫𝐬? 1. Go to and log in 2. Click on 𝐾𝑛𝑜𝑤𝑙𝑒𝑑𝑔𝑒 𝐵𝑎𝑠𝑒 from the left menu 3. Upload the papers you selected for literature review 4. You will see the following option against each paper - Read PDF - Key Takeaway - AI Poster 5. Click on 𝑅𝑒𝑎𝑑 𝑃𝐷𝐹 for the first paper in your list 6. Write a prompt for the data you want to extract 7. For example, you can enter datasets, methodology etc. 8. It will extract the required data from the paper 9. If you want to extract Key Takeaways from the paper 10. Go back and click on 𝐾𝑒𝑦 𝑇𝑎𝑘𝑒𝑎𝑤𝑎𝑦𝑠 11. Bohrium will extract Key Takeaways from the paper 12. In addition to this, you also have 2 more options - AI Poster - Podcast 13. Click on 𝐴𝐼 𝑃𝑜𝑠𝑡𝑒𝑟 and it will create a poster for you 14. This is the poster based on the given research paper 15. If you click on 𝑃𝑜𝑑𝑐𝑎𝑠𝑡, it will convert the paper to audio 16. You can listen to the paper instead of reading it Repeat this cycle for all the papers in your pool. You will end up with the required data. You can use this data to write your literature review Try Bohrium today for FREE: Anything you’d like to add?
0:59

PhD Students – How to extract data from papers for your literature review in seconds? Extracting data from papers takes a lot of time. You can automate this process with Bohrium 𝐇𝐨𝐰 𝐭𝐨 𝐚𝐮𝐭𝐨𝐦𝐚𝐭𝐢𝐜𝐚𝐥𝐥𝐲 𝐞𝐱𝐭𝐫𝐚𝐜𝐭 𝐝𝐚𝐭𝐚 𝐟𝐫𝐨𝐦 𝐩𝐚𝐩𝐞𝐫𝐬? 1. Go to and log in 2. Click on 𝐾𝑛𝑜𝑤𝑙𝑒𝑑𝑔𝑒 𝐵𝑎𝑠𝑒 from the left menu 3. Upload the papers you selected for literature review 4. You will see the following option against each paper - Read PDF - Key Takeaway - AI Poster 5. Click on 𝑅𝑒𝑎𝑑 𝑃𝐷𝐹 for the first paper in your list 6. Write a prompt for the data you want to extract 7. For example, you can enter datasets, methodology etc. 8. It will extract the required data from the paper 9. If you want to extract Key Takeaways from the paper 10. Go back and click on 𝐾𝑒𝑦 𝑇𝑎𝑘𝑒𝑎𝑤𝑎𝑦𝑠 11. Bohrium will extract Key Takeaways from the paper 12. In addition to this, you also have 2 more options - AI Poster - Podcast 13. Click on 𝐴𝐼 𝑃𝑜𝑠𝑡𝑒𝑟 and it will create a poster for you 14. This is the poster based on the given research paper 15. If you click on 𝑃𝑜𝑑𝑐𝑎𝑠𝑡, it will convert the paper to audio 16. You can listen to the paper instead of reading it Repeat this cycle for all the papers in your pool. You will end up with the required data. You can use this data to write your literature review Try Bohrium today for FREE: Anything you’d like to add?

Faheem Ullah

13,433 Aufrufe • vor 11 Monaten

0.32 has shipped, and it's a massive release from Rerun. There's a ton of cool new features, and I wanted to highlight 2 in particular 1. OSS Server streaming from disk 2. Dataset review I walk you through them in the video, so take a look. I'll have a much longer blog post next week about the entire pipeline. With 0.32, much of the foundation is set for a unified data layer for physical data, and I'll be getting into the details of it with all that I've built over the past year. This will cover 1. Raw Data Collection 2. Data Ingestion 3. Catalog Registration 4. Query and Review 5. Post Process 6. Training so lots to share
1:54

0.32 has shipped, and it's a massive release from Rerun. There's a ton of cool new features, and I wanted to highlight 2 in particular 1. OSS Server streaming from disk 2. Dataset review I walk you through them in the video, so take a look. I'll have a much longer blog post next week about the entire pipeline. With 0.32, much of the foundation is set for a unified data layer for physical data, and I'll be getting into the details of it with all that I've built over the past year. This will cover 1. Raw Data Collection 2. Data Ingestion 3. Catalog Registration 4. Query and Review 5. Post Process 6. Training so lots to share

Pablo Vela

11,264 Aufrufe • vor 2 Monaten