Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Today, AWS CEO Matt Garman announced Nova Forge, a model builder which lets companies inject their own data during the pre-training phase. "You [tell Forge]: 'Here's my corpus of corporate data, here's everything I need to know about my industry.' We then mix that in and finish pre-training the... show more

TBPN

687,770 subscribers

96,978 views • 7 months ago •via X (Twitter)

Education News & Politics Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Meet Amazon Nova Forge, the easiest and most cost-effective path to your own frontier models. * Early Nova checkpoints across pre-training, mid-training, and post-training phases * Blend proprietary data with Amazon Nova-curated training data * Reinforcement Fine Tuning (RFT) with reward functions in your environment * Custom content moderation settings

Meet Amazon Nova Forge, the easiest and most cost-effective path to your own frontier models. * Early Nova checkpoints across pre-training, mid-training, and post-training phases * Blend proprietary data with Amazon Nova-curated training data * Reinforcement Fine Tuning (RFT) with reward functions in your environment * Custom content moderation settings

Amazon Web Services

2,344,052 views • 7 months ago

Still following your human intuition to mix corpora from different sources for language model pre-training 🧠? Everyone says that data mixture has a big impact on model performance, but how - and why🕵️? Did you know that web corpora are actually highly impactful for downstream tasks 🏆? Let's check out our preprint "RegMix: Data Mixture as Regression for Language Model Pre-training" 📄 🔬In this paper, we've proposed an automatic data mixture method RegMix that achieves a 6.3% improvement over human selection on the widely used HellaSwag benchmark - and it only needs a 2% extra training FLOPs! 📈 Details in the thread 🧵

Still following your human intuition to mix corpora from different sources for language model pre-training 🧠? Everyone says that data mixture has a big impact on model performance, but how - and why🕵️? Did you know that web corpora are actually highly impactful for downstream tasks 🏆? Let's check out our preprint "RegMix: Data Mixture as Regression for Language Model Pre-training" 📄 🔬In this paper, we've proposed an automatic data mixture method RegMix that achieves a 6.3% improvement over human selection on the widely used HellaSwag benchmark - and it only needs a 2% extra training FLOPs! 📈 Details in the thread 🧵

Qian Liu

54,961 views • 2 years ago

New short course on Fine-tuning LLMs! Many developers are moving beyond only prompting, to also fine-tuning LLMs - that is, taking a pre-trained model and training it further on your own data, which can deliver superior results inexpensively. In this course, Sharon Zhou, CEO of Lamini (disclosure: I’m a minor shareholder) shows you how to recognize when fine-tuning can be help, and how to train an open-source LLM on your own data. I hope you enjoy the course!

New short course on Fine-tuning LLMs! Many developers are moving beyond only prompting, to also fine-tuning LLMs - that is, taking a pre-trained model and training it further on your own data, which can deliver superior results inexpensively. In this course, Sharon Zhou, CEO of Lamini (disclosure: I’m a minor shareholder) shows you how to recognize when fine-tuning can be help, and how to train an open-source LLM on your own data. I hope you enjoy the course!

Andrew Ng

502,781 views • 2 years ago

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

TBPN

76,634 views • 1 year ago

Tether Data, AI model training platform preview. This PaaS will be available to any company interested in (pre-)training own models. Bonus, at the core of this platform we're leveraging Holepunch's tech for all data-structures to make training and models highly-resilient and unstoppable. Soon available via Northern Data Group , leveraging 24k+ H100 GPUs.

Tether Data, AI model training platform preview. This PaaS will be available to any company interested in (pre-)training own models. Bonus, at the core of this platform we're leveraging Holepunch's tech for all data-structures to make training and models highly-resilient and unstoppable. Soon available via Northern Data Group , leveraging 24k+ H100 GPUs.

Paolo Ardoino 🤖

28,092 views • 1 year ago

Building a truly private AI model isn’t as simple as the tech world wants you to believe. despite all the marketing around one click deployments, RAG systems, fine tuning services, and memory features, we still don’t have models that are genuinely trained only on your personal data with complete privacy from third parties. the reality is that most AI solutions today require you to trust someone else with your books, notes, journals, and training data. even when companies promise privacy, there’s usually a way for the provider or other parties to access your information during training or inference. Phala is taking a different approach by building the infrastructure needed for real data ownership. They’re developing confidential runtime environments where your AI training happens inside secure enclaves with attestation guarantees. The key innovation is their in-enclave keying system through dstack, which means that once your model is training, even Phala themselves cannot see your data or model weights.

Building a truly private AI model isn’t as simple as the tech world wants you to believe. despite all the marketing around one click deployments, RAG systems, fine tuning services, and memory features, we still don’t have models that are genuinely trained only on your personal data with complete privacy from third parties. the reality is that most AI solutions today require you to trust someone else with your books, notes, journals, and training data. even when companies promise privacy, there’s usually a way for the provider or other parties to access your information during training or inference. Phala is taking a different approach by building the infrastructure needed for real data ownership. They’re developing confidential runtime environments where your AI training happens inside secure enclaves with attestation guarantees. The key innovation is their in-enclave keying system through dstack, which means that once your model is training, even Phala themselves cannot see your data or model weights.

soulman 🎮

11,702 views • 9 months ago

Today we’re releasing K2 Think V2, our most capable open-source reasoning model to date. This is a fully sovereign model: trained end-to-end on IFM-curated and synthesized data, with complete transparency from pre-training through final reasoning alignment.

Today we’re releasing K2 Think V2, our most capable open-source reasoning model to date. This is a fully sovereign model: trained end-to-end on IFM-curated and synthesized data, with complete transparency from pre-training through final reasoning alignment.

MBZUAI

287,725 views • 5 months ago

The Model is the Product!⚡️ Bringing a great pod with Alexander Doria - we'd talked about pre-training recipes, common corpus, mid-training, agentic systems, good post-training and everything ai. [podcast link in replies]

The Model is the Product!⚡️ Bringing a great pod with Alexander Doria - we'd talked about pre-training recipes, common corpus, mid-training, agentic systems, good post-training and everything ai. [podcast link in replies]

himanshu

21,827 views • 9 months ago

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

girish

451,913 views • 28 days ago

We just launched @ai_browser's pre-beta – it's a browser that contains your very own team of AI interns that you can teach to your grunt work. Here's me spinning up hundreds of parallel research agents to augment sheet data, right in the browser.

We just launched @ai_browser's pre-beta – it's a browser that contains your very own team of AI interns that you can teach to your grunt work. Here's me spinning up hundreds of parallel research agents to augment sheet data, right in the browser.

Charles Maddock

52,477 views • 1 year ago

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗

NVIDIA Robotics

13,429 views • 5 months ago

Introducing Ψ₀ ( — an open foundation model for universal humanoid loco-manipulation. 🏆 Outperforms GR00T N1.6 by 40%+ overall success rate 📉 Uses only ~10% of the pre-training data 📦 Fully open-source: model, data, code, and deployment pipeline 1/10

Introducing Ψ₀ ( — an open foundation model for universal humanoid loco-manipulation. 🏆 Outperforms GR00T N1.6 by 40%+ overall success rate 📉 Uses only ~10% of the pre-training data 📦 Fully open-source: model, data, code, and deployment pipeline 1/10

Yue Wang

19,503 views • 3 months ago

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”

TBPN

18,811 views • 1 year ago

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

DeepLearning.AI

16,771 views • 11 months ago

Google's Jeff Dean says current pre-training is passive: initialize a model, stream the internet past it, let it observe But models need to learn not just from data, but by acting, predicting, and choosing what to learn from next "we have this artificial distinction now between pre and post-training, and it shouldn't exist long term"

Google's Jeff Dean says current pre-training is passive: initialize a model, stream the internet past it, let it observe But models need to learn not just from data, but by acting, predicting, and choosing what to learn from next "we have this artificial distinction now between pre and post-training, and it shouldn't exist long term"

Haider.

55,694 views • 3 months ago

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

AI Evolution

15,565 views • 1 month ago

Lightspeed's Bucky Moore says the real opportunity in the AI app layer is in large industries far enough afield from where the model providers are today — and where the context engineering to get customer data into the model is extremely nuanced and messy. "I think this is kind of the elephant in the room right now — whether post-training open-source models combined with the unique user feedback you get from being an application provider is defensible enough." "That is going to be an inevitable challenge for any of these industries that hit a maturation point of AI adoption, like legal and software engineering have." "But on the other hand, there are some industries where they're very large, they're far enough afield from where the model providers are today — and probably will continue to be — and the context engineering to actually get the customer data into the model is just so messy. It requires going across different business functions, it requires a lot of hands-on forward-deployed engineering." "Those are the kind of companies that we get really excited about. Because I think being really good at that is not only defensible, but it also allows you to generate a feedback loop with your customers, where you hear a lot of their secrets. And those secrets allow you to feed that back into how you make your product better at the expense of anyone else playing in the space. Because if you're serving the customer, they're only serving you those secrets." "I think Palantir is a good example of this in the pre-AI era, and I think we're going to see many companies ascend in that same way."

Lightspeed's Bucky Moore says the real opportunity in the AI app layer is in large industries far enough afield from where the model providers are today — and where the context engineering to get customer data into the model is extremely nuanced and messy. "I think this is kind of the elephant in the room right now — whether post-training open-source models combined with the unique user feedback you get from being an application provider is defensible enough." "That is going to be an inevitable challenge for any of these industries that hit a maturation point of AI adoption, like legal and software engineering have." "But on the other hand, there are some industries where they're very large, they're far enough afield from where the model providers are today — and probably will continue to be — and the context engineering to actually get the customer data into the model is just so messy. It requires going across different business functions, it requires a lot of hands-on forward-deployed engineering." "Those are the kind of companies that we get really excited about. Because I think being really good at that is not only defensible, but it also allows you to generate a feedback loop with your customers, where you hear a lot of their secrets. And those secrets allow you to feed that back into how you make your product better at the expense of anyone else playing in the space. Because if you're serving the customer, they're only serving you those secrets." "I think Palantir is a good example of this in the pre-AI era, and I think we're going to see many companies ascend in that same way."

TBPN

46,746 views • 3 months ago

Engram cofounder Jack Morris just raised $98M to build a new type of AI. He says models don't need to get smarter over time. Instead, they just need to know you better and better over time. Jack describes what he's building: "Our product is a new type of AI. We have a pretty different vision from a lot of the frontier labs, which are working on one model per lab, and trying to make that model smarter every month." "There's another way to think about it, which is that the model doesn't need to get smarter every month. It needs to know you better." "So we're working on a whole different stack, which is a way to train models that train themselves to know your world better and adjust to the things that you say." "So: new ways of training, new ways of running the models."

Engram cofounder Jack Morris just raised $98M to build a new type of AI. He says models don't need to get smarter over time. Instead, they just need to know you better and better over time. Jack describes what he's building: "Our product is a new type of AI. We have a pretty different vision from a lot of the frontier labs, which are working on one model per lab, and trying to make that model smarter every month." "There's another way to think about it, which is that the model doesn't need to get smarter every month. It needs to know you better." "So we're working on a whole different stack, which is a way to train models that train themselves to know your world better and adjust to the things that you say." "So: new ways of training, new ways of running the models."

TBPN

67,455 views • 10 days ago

David Friedberg says Anthropic asked big pharma for their data and nearly everyone said no "There's been an effort by Anthropic to sign up life sciences companies to contribute to a new life sciences focused model. They're approaching these large companies with large proprietary data sets and saying, if you share your data, we will give you early access, some sort of proprietary value. Sign this NDA and you can participate with us." "I think nearly everyone I've spoken with has woken up to the fact that they are trying to commoditize everyone's business. If all of the tens of billions of dollars you have invested in experiments and product development, and you've generated all of this proprietary data along the way, that data is a true asset of your organization. It's an asset that you've spent billions of dollars developing." "And by handing it over to a model company to then combine with other people's data, you are commoditizing the one core differentiation that you have. And so everyone is largely saying no." "I think what everyone's realizing is they're better off developing their own weights and their own models using either an open source basis or there might be some intermediary business model that evolves."

David Friedberg says Anthropic asked big pharma for their data and nearly everyone said no "There's been an effort by Anthropic to sign up life sciences companies to contribute to a new life sciences focused model. They're approaching these large companies with large proprietary data sets and saying, if you share your data, we will give you early access, some sort of proprietary value. Sign this NDA and you can participate with us." "I think nearly everyone I've spoken with has woken up to the fact that they are trying to commoditize everyone's business. If all of the tens of billions of dollars you have invested in experiments and product development, and you've generated all of this proprietary data along the way, that data is a true asset of your organization. It's an asset that you've spent billions of dollars developing." "And by handing it over to a model company to then combine with other people's data, you are commoditizing the one core differentiation that you have. And so everyone is largely saying no." "I think what everyone's realizing is they're better off developing their own weights and their own models using either an open source basis or there might be some intermediary business model that evolves."

dnap

261,648 views • 3 days ago