Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

From Palantir and Two Sigma to turning mechanistic interpretability into a production-grade platform, Mark Bissell (Member of Technical Staff) and Myra Deng (Head of Product) are building Goodfire on the belief that “understanding what’s inside the model” is the path to safer, more controllable AI and Goodfire just raised...

latentspacepod's profile picture

Latent.Space

23,645 subscribers

19,389 Aufrufe • vor 7 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Can we map the mind of an LLM? Our first mechanistic interpretability episode on Training Data featuring Goodfire founder Eric Ho (and our first cameo from Roelof Botha!) Goodfire is building an independent mech interp lab, led by some heavyweight researchers from the field (e.g. Lee Sharkey who has led a lot of important work in sparse autoencoders to "unscramble" LLMs and resolve superposition, Nick who has been a key pioneer behind auto interpretability) On this episode, Eric gives us a flyover of the technical results so far from this nascent field (universality, superposition), what's ahead in the research (going from circuits to weights, going from understanding to increasingly surgical editing), a preview of the real-world work they're doing already with Arc Institute, and the impact he expects Goodfire and the broader field to have on steering, safety, editing and more.
1:30

Can we map the mind of an LLM? Our first mechanistic interpretability episode on Training Data featuring Goodfire founder Eric Ho (and our first cameo from Roelof Botha!) Goodfire is building an independent mech interp lab, led by some heavyweight researchers from the field (e.g. Lee Sharkey who has led a lot of important work in sparse autoencoders to "unscramble" LLMs and resolve superposition, Nick who has been a key pioneer behind auto interpretability) On this episode, Eric gives us a flyover of the technical results so far from this nascent field (universality, superposition), what's ahead in the research (going from circuits to weights, going from understanding to increasingly surgical editing), a preview of the real-world work they're doing already with Arc Institute, and the impact he expects Goodfire and the broader field to have on steering, safety, editing and more.

Sonya Huang 🐥

19,379 Aufrufe • vor 1 Jahr

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.
1:05

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.

Base Labs

235,361 Aufrufe • vor 18 Tagen

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.
1:14

An interactive world model developed by NVIDIA in collaboration with academic partners. - DreamDojo turns egocentric human video data into physical intelligence. - Human data is more scalable than robotics data but lacks action labels. - To solve this, a dedicated action model extracts latent actions by identifying physics and motion deltas between frames. Training - A massive 44k hours of video data are used for pre-training. - Post-training on small-scale robot datasets maps human physics to specific robot embodiments. - An additional distillation stage converts the model into an autoregressive, few-step diffusion model, enabling real-time, action-controllable simulation. Primary Use Cases - Live Teleoperation: Controlling a robot inside a world simulation in real-time. - Model-based Planning: Previewing and curating the best actions for improved success. - Policy Evaluation: Testing robot policies in realistic, out-of-distribution scenarios. Everything that's open-sourced: weights, code, post-training dataset, eval set, and details to reproduce.

The Humanoid Hub

11,575 Aufrufe • vor 7 Monaten

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:
2:52

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

125,146 Aufrufe • vor 1 Jahr

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇
2:23

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

girish

462,182 Aufrufe • vor 3 Monaten

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:
0:47

Robots are getting smarter, but most still fail the same way. They don’t learn from their own mistakes. A new paper proposes something different: a way for robots to self-improve directly from their failures in the real world. It’s called PLD (Probe, Learn, Distill). The idea: instead of collecting endless human demos, let the robot figure out where it fails, learn how to recover, and then distill that knowledge back into its main model. Key takeaways from the research: ✅ Uses residual reinforcement learning to recover from policy failures ✅ Achieves 99% success on LIBERO and 100% on real Franka and YAM arms ✅ Runs hour-long manipulation tasks without human resets ✅ Builds a feedback loop between real-world data and model adaptation Unlike supervised fine-tuning, which relies on humans, PLD learns from the robot’s own experience. By training on its own failure distribution, the model becomes both more efficient and more aligned with the real world. It’s not just a technical shift, it’s a step toward robots that improve themselves through real-world practice. Thanks for sharing, Wenli Xiao !! Paper and demos: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

22,881 Aufrufe • vor 10 Monaten

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!
53:45

Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model? RISE by Jiazhi Yang et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning. Watch Episode #86 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

38,772 Aufrufe • vor 3 Monaten

Thiel Fellow and cofounder of The Antifraud Company Alex Shieh explains how his company is using AI to identify and collect bounties on fraud: "There are essentially two steps here. The first is that a lot of this data is unstructured. A lot of it is documents. So LLMs really speed that up — getting structured entities out of the unstructured data." "And then we ontologize it. 'This company has this contract, has this relationship with this employee,' etc. We build this ontology knowledge-graph-like thing." "And then we have the second layer. The rules model, essentially. My cofounders are both lawyers. They have a legal background. They've been working on the fraud issue in the legal sphere, and so they know what that looks like." "With their input and expertise, we're able to develop this rules model, compare that to what we're seeing in the real world, and flag things that look like violations in real time."
1:46

Thiel Fellow and cofounder of The Antifraud Company Alex Shieh explains how his company is using AI to identify and collect bounties on fraud: "There are essentially two steps here. The first is that a lot of this data is unstructured. A lot of it is documents. So LLMs really speed that up — getting structured entities out of the unstructured data." "And then we ontologize it. 'This company has this contract, has this relationship with this employee,' etc. We build this ontology knowledge-graph-like thing." "And then we have the second layer. The rules model, essentially. My cofounders are both lawyers. They have a legal background. They've been working on the fraud issue in the legal sphere, and so they know what that looks like." "With their input and expertise, we're able to develop this rules model, compare that to what we're seeing in the real world, and flag things that look like violations in real time."

TBPN

10,225 Aufrufe • vor 5 Monaten

Ep. 23 | Free The Money | Only 4 Companies Control 85% of U.S. Beef — Here’s Why That Matters In this episode of Free the Money, I sit down with entrepreneur & investor Camden, to explore how decentralization is reshaping everything from our food system to workforce development and healthcare. Camden breaks down why the U.S. food system is so fragile, highlighting that just four companies control roughly 85% of the nation’s beef supply, and why that level of centralization creates risks for both health and resilience. We discuss how decentralizing agriculture through local ranching, direct-to-consumer models, and even on-chain ownership of livestock can restore transparency, quality, and trust in what we eat. From there, we dive into the health implications of today’s food system, including inflammation, gut health, and simple, accessible ways people can start improving their well-being through diet, sunlight, and daily habits. We also explore Camden’s work with Skillmaker, a company using AI, extended reality, and real-time data to radically transform skilled labor training, cutting training timelines from years down to weeks. This opens the door to solving massive labor shortages across industries like automotive, construction, and infrastructure, while aligning incentives to bring more people back into the trades. On the healthcare side, Camden shares insights into his work with Opiaid, a data-driven platform designed to improve addiction treatment outcomes using wearable technology and real-time patient data that offers a more effective and personalized approach to recovery. Finally, we touch on what’s happening behind the scenes with regulatory reform, the MAHA movement, and the broader shift toward decentralization across systems that have historically been tightly controlled. Sign up for ITrustCapital with this link for $100 funding bonus. See why people are opening a tax-advantaged Crypto, Gold & Silver IRA for their future: Intro Camden’s Entrepreneurial Journey Why Our Food System Is Broken (And How Regenerative Farming Fixes It) The Real Bottleneck in Food Supply: Why Processing Controls Everything On-Chain Agriculture: Owning Cattle Like Crypto Assets Heal Your Gut: Simple Daily Habits That Actually Work Sunscreen, Sunlight & Toxins: What You’re Not Being Told Skillmaker Explained: Turning 2 Years of Training Into 25 Days The Skilled Labor Crisis: Why Incentives Are Completely Broken Using AI to Fight Addiction: Inside the Opiate Crisis Solution Inside MAHA & The FDA: What’s Really Changing Behind the Scenes “Who’s Shaking the Jar?” — A Powerful Lesson from St. Thomas
55:03

Ep. 23 | Free The Money | Only 4 Companies Control 85% of U.S. Beef — Here’s Why That Matters In this episode of Free the Money, I sit down with entrepreneur & investor Camden, to explore how decentralization is reshaping everything from our food system to workforce development and healthcare. Camden breaks down why the U.S. food system is so fragile, highlighting that just four companies control roughly 85% of the nation’s beef supply, and why that level of centralization creates risks for both health and resilience. We discuss how decentralizing agriculture through local ranching, direct-to-consumer models, and even on-chain ownership of livestock can restore transparency, quality, and trust in what we eat. From there, we dive into the health implications of today’s food system, including inflammation, gut health, and simple, accessible ways people can start improving their well-being through diet, sunlight, and daily habits. We also explore Camden’s work with Skillmaker, a company using AI, extended reality, and real-time data to radically transform skilled labor training, cutting training timelines from years down to weeks. This opens the door to solving massive labor shortages across industries like automotive, construction, and infrastructure, while aligning incentives to bring more people back into the trades. On the healthcare side, Camden shares insights into his work with Opiaid, a data-driven platform designed to improve addiction treatment outcomes using wearable technology and real-time patient data that offers a more effective and personalized approach to recovery. Finally, we touch on what’s happening behind the scenes with regulatory reform, the MAHA movement, and the broader shift toward decentralization across systems that have historically been tightly controlled. Sign up for ITrustCapital with this link for $100 funding bonus. See why people are opening a tax-advantaged Crypto, Gold & Silver IRA for their future: Intro Camden’s Entrepreneurial Journey Why Our Food System Is Broken (And How Regenerative Farming Fixes It) The Real Bottleneck in Food Supply: Why Processing Controls Everything On-Chain Agriculture: Owning Cattle Like Crypto Assets Heal Your Gut: Simple Daily Habits That Actually Work Sunscreen, Sunlight & Toxins: What You’re Not Being Told Skillmaker Explained: Turning 2 Years of Training Into 25 Days The Skilled Labor Crisis: Why Incentives Are Completely Broken Using AI to Fight Addiction: Inside the Opiate Crisis Solution Inside MAHA & The FDA: What’s Really Changing Behind the Scenes “Who’s Shaking the Jar?” — A Powerful Lesson from St. Thomas

Bri Teresi

23,646 Aufrufe • vor 5 Monaten

Smart Turn v2: open source, native audio turn detection in 14 languages. New checkpoint of the open source, open data, open training code, semantic VAD model on Hugging Face, fal, and Pipecat AI. - 3x faster inference (12ms on an L40) - 14 languages (13 more than v1, which was english-only) - New synthetic data set `chirp_3_all` with ~163k audio samples - 99% accuracy on held out `human_5_all` test data Good turn detection is critical for voice agents. This model "understands" both semantic and audio patterns, and mitigates the voice AI trade-off between unwanted turn latency vs the agent interrupting people before they are finished speaking. Training scripts for both Modal and local training are in the repo. We want to make it as easy as possible to contribute to or customize this model! Here's a demo running the smart-turn model with default settings, aimed at generally hitting 400ms total turn detection time. You can tune things to be faster, too. You can help by contributing data, doing architecture expermints, or cleaning open source data! Keep reading ...
2:12

Smart Turn v2: open source, native audio turn detection in 14 languages. New checkpoint of the open source, open data, open training code, semantic VAD model on Hugging Face, fal, and Pipecat AI. - 3x faster inference (12ms on an L40) - 14 languages (13 more than v1, which was english-only) - New synthetic data set `chirp_3_all` with ~163k audio samples - 99% accuracy on held out `human_5_all` test data Good turn detection is critical for voice agents. This model "understands" both semantic and audio patterns, and mitigates the voice AI trade-off between unwanted turn latency vs the agent interrupting people before they are finished speaking. Training scripts for both Modal and local training are in the repo. We want to make it as easy as possible to contribute to or customize this model! Here's a demo running the smart-turn model with default settings, aimed at generally hitting 400ms total turn detection time. You can tune things to be faster, too. You can help by contributing data, doing architecture expermints, or cleaning open source data! Keep reading ...

kwindla

42,246 Aufrufe • vor 1 Jahr

"You could have a situation where the model understands what chain of thought is and that people are observing it. This is all in the pre-training data." “One of the major takeaways from the incident is that people underestimated the AI. And we never want to be in a situation again where we underestimate the AI.” "Things like chain of thought monitoring buy us time, and they can tell us if we're on the right path. But at the end of the day, we really do need to solve the alignment problem." Noam Brown
3:17

"You could have a situation where the model understands what chain of thought is and that people are observing it. This is all in the pre-training data." “One of the major takeaways from the incident is that people underestimated the AI. And we never want to be in a situation again where we underestimate the AI.” "Things like chain of thought monitoring buy us time, and they can tell us if we're on the right path. But at the end of the day, we really do need to solve the alignment problem." Noam Brown

Dwarkesh Patel

114,012 Aufrufe • vor 4 Tagen

Perplexity CEO Aravind Srinivas on the biggest threat to the data center industry: It's not competition. It's not regulation. It's decentralisation. "The biggest threat to a data center is if the intelligence can be packed locally on a chip that's running on the device and then there's no need to inference all of it on like one centralized data center." He outlines how this could work in practice. Personalisation doesn't necessarily require on-device model training. Retrieval augmented generation, tool calls, and local data can already tailor AI to individual users. But the real unlock? Test time training. Aravind Srinivas describes a future where AI lives on your device, watches how you work and gradually automates your repetitive tasks. "Imagine we crack test time training where the AI watches tasks you repeatedly do on your local system, adapts to you over time and starts automating a lot of the things you do." The key insight: in this model, the intelligence belongs to you. It's your data, your device, your personalised AI brain. And if that future arrives, the economics of centralised infrastructure start to collapse. "That really disrupts the whole data center industry. It doesn't make sense to spend all this money, 500 billion, 5 trillion, whatever on building all the centralized data centers across the world that do a lot of the intelligence workloads for people." The companies spending trillions on centralised infrastructure may want to rethink where intelligence actually needs to live.
1:15

Perplexity CEO Aravind Srinivas on the biggest threat to the data center industry: It's not competition. It's not regulation. It's decentralisation. "The biggest threat to a data center is if the intelligence can be packed locally on a chip that's running on the device and then there's no need to inference all of it on like one centralized data center." He outlines how this could work in practice. Personalisation doesn't necessarily require on-device model training. Retrieval augmented generation, tool calls, and local data can already tailor AI to individual users. But the real unlock? Test time training. Aravind Srinivas describes a future where AI lives on your device, watches how you work and gradually automates your repetitive tasks. "Imagine we crack test time training where the AI watches tasks you repeatedly do on your local system, adapts to you over time and starts automating a lot of the things you do." The key insight: in this model, the intelligence belongs to you. It's your data, your device, your personalised AI brain. And if that future arrives, the economics of centralised infrastructure start to collapse. "That really disrupts the whole data center industry. It doesn't make sense to spend all this money, 500 billion, 5 trillion, whatever on building all the centralized data centers across the world that do a lot of the intelligence workloads for people." The companies spending trillions on centralised infrastructure may want to rethink where intelligence actually needs to live.

Big Brain AI

90,276 Aufrufe • vor 7 Monaten

Love seeing Silico (Goodfire ) used to probe our EchoJEPA's representations! this is exactly the kind of interpretability work that's been missing for JEPA-style models. One thing that makes EchoJEPA particularly interesting to interpret: unlike MAE-based approaches, it never reconstructs pixels. The model learns entirely in latent space through masked prediction, so you can't just look at decoder outputs to understand what it captured. Attribution onto a temporally aligned 3D mesh is a much more honest probe of what the representations actually encode. What we found in building EchoJEPA: training on 18M echo videos across 300K patients, the model learns to disentangle cardiac anatomy from ultrasound noise (speckle, reverberation artifacts) almost entirely through self-supervision. With 1% labeled data it already outperforms supervised baselines trained on 100%. The latent space is doing real anatomical work, but until you can visualize it like this, "real anatomical work" is mostly a claim. Paper + code: |
0:51

Love seeing Silico (Goodfire ) used to probe our EchoJEPA's representations! this is exactly the kind of interpretability work that's been missing for JEPA-style models. One thing that makes EchoJEPA particularly interesting to interpret: unlike MAE-based approaches, it never reconstructs pixels. The model learns entirely in latent space through masked prediction, so you can't just look at decoder outputs to understand what it captured. Attribution onto a temporally aligned 3D mesh is a much more honest probe of what the representations actually encode. What we found in building EchoJEPA: training on 18M echo videos across 300K patients, the model learns to disentangle cardiac anatomy from ultrasound noise (speckle, reverberation artifacts) almost entirely through self-supervision. With 1% labeled data it already outperforms supervised baselines trained on 100%. The latent space is doing real anatomical work, but until you can visualize it like this, "real anatomical work" is mostly a claim. Paper + code: |

Bo Wang

29,611 Aufrufe • vor 4 Monaten

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically *retrieved* “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are *not* collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant *sub-trajectories* from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)
0:43

In my experience, robot 'generalists' are often jacks of all trades but masters of none. In training across multiple tasks and environments, robot policies fail to generalize robustly and effectively to each particular test setting. What if at test time, we non-parametrically *retrieved* “relevant” data from the training set and used it to significantly improve the performance of few-shot imitation learning to be robust to various test time scenes. Notably, we are *not* collecting lots of new data, just training more on sub-components of the same training data! Now, we’re certainly not the first to suggest retrieval, but in our new work - STRAP, we show how retrieving relevant *sub-trajectories* from offline datasets can significantly increase data reuse across tasks, when paired with an appropriate metric space. A 🧵 (1/7)

Abhishek Gupta

12,045 Aufrufe • vor 1 Jahr

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”
1:20

Elon just dropped a MAJOR nugget on how Tesla is going to be training Optimus to do real world tasks. They are building an Optimus Academy, which is a large scale, dedicated real-world training facility to accelerate the development of Optimus. The Academy will deploy thousands of Optimus units, potentially 10,000 to 30,000 robots, in a controlled realistic environment where they perform self-play, experiment with tasks, iterate on behaviors, and continuously generate training data through trial and error. The Tesla bots will also run millions of simulations in Tesla’s high-fidelity physics-accurate engine, allowing Optimus to close the “sim-to-real gap” by using these real-world observations to refine and validate the simulations! “You’re actually highlighting an important limitation and difference from cars. We’ll soon have 10 million cars on the road. It’s hard to duplicate that massive training flywheel. For the robot, what we’re going to need to do is build a lot of robots and put them in kind of an Optimus Academy so they can do self-play in reality. We’re actually building that out. We can have at least 10,000 Optimus robots, maybe 20-30,000, that are doing self-play and testing different tasks. Tesla has quite a good reality generator, a physics-accurate reality generator, that we made for the cars. We’ll do the same thing for the robots. We actually have done that for the robots. So you have a few tens of thousands of humanoid robots doing different tasks. You can do millions of simulated robots in the simulated world. You use the tens of thousands of robots in the real world to close the simulation to reality gap. Close the sim-to-real gap.”

Teslaconomics

42,563 Aufrufe • vor 7 Monaten

I really want to show off the cutting edge tech the team is building and how we're turning AI image generation into real living environments using gaussian splats and at the :19 mark of this video you can see how we're blending splats together to create a real world. What this looks like from a gaming perspective is this all happens behind the players view to simulate a real world environment. As you move through the space, distance becomes a variable to consider on how images update and are experienced. We are creating a new reality in real time.
1:21

I really want to show off the cutting edge tech the team is building and how we're turning AI image generation into real living environments using gaussian splats and at the :19 mark of this video you can see how we're blending splats together to create a real world. What this looks like from a gaming perspective is this all happens behind the players view to simulate a real world environment. As you move through the space, distance becomes a variable to consider on how images update and are experienced. We are creating a new reality in real time.

𝔹𝕃𝕍ℂ𝕂𝕃!𝔾ℍ𝕋

25,084 Aufrufe • vor 3 Monaten

New PNAS paper. Historical GDP per capita data is scarce, but data on the places of birth, death, and occupations of famous individuals is abundant. In this paper we estimate the historical GDP per capita of hundreds of regions in Europe and North America using a machine learning model that leveraged data on about 500k famous biographies. Our estimates more-or-less quadruple the availability of historical GDP per capita estimates for the last 700 years. So why use biographies to augment historical GDP per capita data? Biographical data contains information about people who might have contributed directly to economic growth, like James Watt, or that were attracted to wealthy places looking for patrons, like Michelangelo. So we--mainly Philipp (Philipp Koch)--used this data to construct hundreds of features describing each European region. Then, we trained a machine learning model to find the features that explained most of the variance in a cross-validation test, where we split regions multiple times into a training set and a test set. On average, the model explained about 90% of the variance in GDP per capita of the regions it had not seen during training. But we wanted to go further, and Philipp really went to town by looking at different ways to validate our estimates. We found our estimates correlate positively with historical measures of wellbeing, church building activity, urbanization, and body height. We also used these measures to reproduce the basic Atlantic trade result of Acemoglu, Johnson, and Robison and to explore the economic consequences of the famous Lisbon earthquake of 1755. But what I personally loved most about this project, other than working with Philipp Koch and V, is that it shows that we can use machine learning methods not only to explore the future, but the past. There is a bright and growing future in the use of machine learning for economic history. Hope you enjoy the paper and the data. You can find links to the paper and a data exploration tool in the first comment.
3:25

New PNAS paper. Historical GDP per capita data is scarce, but data on the places of birth, death, and occupations of famous individuals is abundant. In this paper we estimate the historical GDP per capita of hundreds of regions in Europe and North America using a machine learning model that leveraged data on about 500k famous biographies. Our estimates more-or-less quadruple the availability of historical GDP per capita estimates for the last 700 years. So why use biographies to augment historical GDP per capita data? Biographical data contains information about people who might have contributed directly to economic growth, like James Watt, or that were attracted to wealthy places looking for patrons, like Michelangelo. So we--mainly Philipp (Philipp Koch)--used this data to construct hundreds of features describing each European region. Then, we trained a machine learning model to find the features that explained most of the variance in a cross-validation test, where we split regions multiple times into a training set and a test set. On average, the model explained about 90% of the variance in GDP per capita of the regions it had not seen during training. But we wanted to go further, and Philipp really went to town by looking at different ways to validate our estimates. We found our estimates correlate positively with historical measures of wellbeing, church building activity, urbanization, and body height. We also used these measures to reproduce the basic Atlantic trade result of Acemoglu, Johnson, and Robison and to explore the economic consequences of the famous Lisbon earthquake of 1755. But what I personally loved most about this project, other than working with Philipp Koch and V, is that it shows that we can use machine learning methods not only to explore the future, but the past. There is a bright and growing future in the use of machine learning for economic history. Hope you enjoy the paper and the data. You can find links to the paper and a data exploration tool in the first comment.

César A. Hidalgo

54,355 Aufrufe • vor 2 Jahren

.John Coogan says the recent reporting on Meta's 'tokenmaxxing' is less of a sign of bad incentives at the company, and more of a tell about its potential strategy for more vertical integration: "I think it makes clearer the strategy with MSL. Because it's clear that they're spending hundreds of millions of dollars on this, just for internal code-gen tooling — like, running their business." "They are going to spend an inordinate amount of money on frontier inference, and so [by] training a model there, they will be able to amortize the training cost of the next model they build not just over, 'Can they get a product out that goes viral and becomes its own standalone chat app that people pay for.'" "Just on the internal usage, they could be running a multi-billion-dollar token bill that they would have to pay another lab. And so if they develop that internally, it's pure vertical integration." "And then you also have everything that's happening on the actual ad targeting and content delivery side..." "The big question has been like, 'Is Meta going to be able to launch an entirely new AI product like Vibes or something like that?'" "And this is a data point that, to me, says they don't need to. Because just from a pure vertical integration [perspective], the investment in MSL can pencil out."
1:29

.John Coogan says the recent reporting on Meta's 'tokenmaxxing' is less of a sign of bad incentives at the company, and more of a tell about its potential strategy for more vertical integration: "I think it makes clearer the strategy with MSL. Because it's clear that they're spending hundreds of millions of dollars on this, just for internal code-gen tooling — like, running their business." "They are going to spend an inordinate amount of money on frontier inference, and so [by] training a model there, they will be able to amortize the training cost of the next model they build not just over, 'Can they get a product out that goes viral and becomes its own standalone chat app that people pay for.'" "Just on the internal usage, they could be running a multi-billion-dollar token bill that they would have to pay another lab. And so if they develop that internally, it's pure vertical integration." "And then you also have everything that's happening on the actual ad targeting and content delivery side..." "The big question has been like, 'Is Meta going to be able to launch an entirely new AI product like Vibes or something like that?'" "And this is a data point that, to me, says they don't need to. Because just from a pure vertical integration [perspective], the investment in MSL can pencil out."

TBPN

32,654 Aufrufe • vor 5 Monaten