Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Today, AWS CEO Matt Garman announced Nova Forge, a model builder which lets companies inject their own data during the pre-training phase. "You [tell Forge]: 'Here's my corpus of corporate data, here's everything I need to know about my industry.' We then mix that in and finish pre-training the... show more

TBPN

1,606,999 subscribers

97,039 görüntüleme • 11 ay önce •via X (Twitter)

Eğitim Haberler & Politika Bilim & Teknoloji

Anya Rossi• Live Now

Private livecam show

12 Yorum

arti profil fotoğrafı

arti11 ay önce

Aka you get a Low rank finetune and they tell you they trained a model for you

Yacine Mahdid profil fotoğrafı

Yacine Mahdid11 ay önce

soooo all of the companies are mixing their proprietary data with all the others in a generative model? that’s fun

Youssef El Manssouri profil fotoğrafı

Youssef El Manssouri11 ay önce

RAG is a patch. Pre-training is the cure. For deep technical domains, the model needs to actually know the material, not just look it up.

Abhishek Goyal profil fotoğrafı

Abhishek Goyal11 ay önce

Incredible how they can confidently repackage and announce something available since 2023

Himanshu Kumar profil fotoğrafı

Himanshu Kumar11 ay önce

Right, TBPN, so it's like a customized AI, but what about data privacy, na?

Mohamed Anis profil fotoğrafı

Mohamed Anis11 ay önce

The shift from fine tuning to pre-training integration fundamentally changes how enterprises think about AI moats.

dleer profil fotoğrafı

dleer11 ay önce

Imagin the pre-processing

PelicanInvasion profil fotoğrafı

PelicanInvasion11 ay önce

Is this going to be available for SOTA models? If it's or training doesn't that mean they have to do their entire training run?

ZJZ profil fotoğrafı

ZJZ11 ay önce

$AMZN shareholders announce new bankruptcy

Bret Greenstein profil fotoğrafı

Bret Greenstein11 ay önce

Company data is not unique enough to be worth training a model on it, and it is rarely organized enough to be good training data. It is far better to use a leading edge pretrained model with your corporate data as context (over MCP, etc) than to create an entire model.

Manas Ranjan Kar profil fotoğrafı

Manas Ranjan Kar11 ay önce

Does this have ability to mix data sources in desired ratios and I assume do Continued Pre Training from some standard checkpoint? Also CPT will not be cheap. IMO we are looking at hundreds of thousands if not million $ surprise bills here.

Andrés- e/acc profil fotoğrafı

Andrés- e/acc11 ay önce

This is huge. I am impressed that people are not exited about this.

Benzer Videolar

Grass is betting that better AI still starts with better pre-training data “Whatever you put into your model ultimately determines what your model's capabilities are” “Several of the top frontier AI labs in the world are working directly with Grass to acquire large amounts of pre-training data” “If that information is not present in your model, your model doesn't know how to do that thing in the first place” Andrej puts Grass close to one of the most fundamental inputs in AI: the training corpus itself. As frontier models scale, they need enormous amounts of diverse, high-quality web data. They are building the infrastructure to collect and supply that data at internet scale. Model capability starts with what the model gets to learn from.

Grass is betting that better AI still starts with better pre-training data “Whatever you put into your model ultimately determines what your model's capabilities are” “Several of the top frontier AI labs in the world are working directly with Grass to acquire large amounts of pre-training data” “If that information is not present in your model, your model doesn't know how to do that thing in the first place” Andrej puts Grass close to one of the most fundamental inputs in AI: the training corpus itself. As frontier models scale, they need enormous amounts of diverse, high-quality web data. They are building the infrastructure to collect and supply that data at internet scale. Model capability starts with what the model gets to learn from.

Kaiz 🐂🀄️

13,989 görüntüleme • 28 gün önce

Meet Amazon Nova Forge, the easiest and most cost-effective path to your own frontier models. * Early Nova checkpoints across pre-training, mid-training, and post-training phases * Blend proprietary data with Amazon Nova-curated training data * Reinforcement Fine Tuning (RFT) with reward functions in your environment * Custom content moderation settings

Meet Amazon Nova Forge, the easiest and most cost-effective path to your own frontier models. * Early Nova checkpoints across pre-training, mid-training, and post-training phases * Blend proprietary data with Amazon Nova-curated training data * Reinforcement Fine Tuning (RFT) with reward functions in your environment * Custom content moderation settings

Amazon Web Services

2,344,230 görüntüleme • 11 ay önce

Still following your human intuition to mix corpora from different sources for language model pre-training 🧠? Everyone says that data mixture has a big impact on model performance, but how - and why🕵️? Did you know that web corpora are actually highly impactful for downstream tasks 🏆? Let's check out our preprint "RegMix: Data Mixture as Regression for Language Model Pre-training" 📄 🔬In this paper, we've proposed an automatic data mixture method RegMix that achieves a 6.3% improvement over human selection on the widely used HellaSwag benchmark - and it only needs a 2% extra training FLOPs! 📈 Details in the thread 🧵

Still following your human intuition to mix corpora from different sources for language model pre-training 🧠? Everyone says that data mixture has a big impact on model performance, but how - and why🕵️? Did you know that web corpora are actually highly impactful for downstream tasks 🏆? Let's check out our preprint "RegMix: Data Mixture as Regression for Language Model Pre-training" 📄 🔬In this paper, we've proposed an automatic data mixture method RegMix that achieves a 6.3% improvement over human selection on the widely used HellaSwag benchmark - and it only needs a 2% extra training FLOPs! 📈 Details in the thread 🧵

Qian Liu

54,961 görüntüleme • 2 yıl önce

New short course on Fine-tuning LLMs! Many developers are moving beyond only prompting, to also fine-tuning LLMs - that is, taking a pre-trained model and training it further on your own data, which can deliver superior results inexpensively. In this course, Sharon Zhou, CEO of Lamini (disclosure: I’m a minor shareholder) shows you how to recognize when fine-tuning can be help, and how to train an open-source LLM on your own data. I hope you enjoy the course!

New short course on Fine-tuning LLMs! Many developers are moving beyond only prompting, to also fine-tuning LLMs - that is, taking a pre-trained model and training it further on your own data, which can deliver superior results inexpensively. In this course, Sharon Zhou, CEO of Lamini (disclosure: I’m a minor shareholder) shows you how to recognize when fine-tuning can be help, and how to train an open-source LLM on your own data. I hope you enjoy the course!

Andrew Ng

503,002 görüntüleme • 3 yıl önce

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

We asked Sholto Douglas from Anthropic about the costs of RL (Reinforcement Learning) runs. "In Dario Amodei's essay, he said that RL runs cost only $1M back in December." "RL is a more naively parallelizable and scalable than pre-training." "With pre-training, you need everything in one big data center ideally. For RL, in theory, you could scale all over the world."

TBPN

76,696 görüntüleme • 1 yıl önce

Tether Data, AI model training platform preview. This PaaS will be available to any company interested in (pre-)training own models. Bonus, at the core of this platform we're leveraging Holepunch's tech for all data-structures to make training and models highly-resilient and unstoppable. Soon available via Northern Data Group , leveraging 24k+ H100 GPUs.

Tether Data, AI model training platform preview. This PaaS will be available to any company interested in (pre-)training own models. Bonus, at the core of this platform we're leveraging Holepunch's tech for all data-structures to make training and models highly-resilient and unstoppable. Soon available via Northern Data Group , leveraging 24k+ H100 GPUs.

Paolo Ardoino 🤖

28,092 görüntüleme • 1 yıl önce

Today we’re releasing K2 Think V2, our most capable open-source reasoning model to date. This is a fully sovereign model: trained end-to-end on IFM-curated and synthesized data, with complete transparency from pre-training through final reasoning alignment.

Today we’re releasing K2 Think V2, our most capable open-source reasoning model to date. This is a fully sovereign model: trained end-to-end on IFM-curated and synthesized data, with complete transparency from pre-training through final reasoning alignment.

MBZUAI

287,728 görüntüleme • 9 ay önce

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

“don’t train your own model” is common ai advice. it's wrong. your token bill's the proof. today, we’re excited to launch castform into open preview. castform is the easiest way for you to train your own model, on your own data. open-weights models are performant and much cheaper. when trained on your task & proprietary data, they beat closed models. the thing standing between you and that was weeks of plumbing & years of ml expertise. with castform, model training is as simple as prompt engineering. castform bring your agent traces or raw corpora. castform turns it into training data, picks the right algorithmic recipes, manages gpus, and gives you an ide to watch and chat with your model as it learns. see what you can build with castform👇

girish

464,371 görüntüleme • 4 ay önce

We just launched 's pre-beta – it's a browser that contains your very own team of AI interns that you can teach to your grunt work. Here's me spinning up hundreds of parallel research agents to augment sheet data, right in the browser.

We just launched 's pre-beta – it's a browser that contains your very own team of AI interns that you can teach to your grunt work. Here's me spinning up hundreds of parallel research agents to augment sheet data, right in the browser.

Charles Maddock

52,762 görüntüleme • 1 yıl önce

Andrej reveals that Grass has crossed $35M in trailing 12-month revenue from AI pre-training data sales, with H1 revenue growing nearly 7x YoY “Every single book that any human has ever published represents one percent of the amount of data that current pre-training runs actually need to properly scale” “When you think of things from that perspective, it becomes pretty obvious why there's a massive data shortage and why we've been focusing so much on pre-training data” “This has also translated to certain business metrics. Grass has 7x revenue purely on pre-training data sales since it began monetizing data” “It went from $2.7 million in the first half of last year, and then in the last 12 months crossed $35 million” “Thinking of things from this scale is incredibly interesting, but it also illustrates why our focus has been where it's been” w/ Andrej

Andrej reveals that Grass has crossed $35M in trailing 12-month revenue from AI pre-training data sales, with H1 revenue growing nearly 7x YoY “Every single book that any human has ever published represents one percent of the amount of data that current pre-training runs actually need to properly scale” “When you think of things from that perspective, it becomes pretty obvious why there's a massive data shortage and why we've been focusing so much on pre-training data” “This has also translated to certain business metrics. Grass has 7x revenue purely on pre-training data sales since it began monetizing data” “It went from $2.7 million in the first half of last year, and then in the last 12 months crossed $35 million” “Thinking of things from this scale is incredibly interesting, but it also illustrates why our focus has been where it's been” w/ Andrej

Kaiz 🐂🀄️

14,984 görüntüleme • 20 gün önce

The first time S1 flipped a pancake, we assumed pancake flipping must have been in its pre-training data. We searched our whole pre-training data and found no examples of flipping. S1 inferred the out-of-distribution task from one video prompt.

The first time S1 flipped a pancake, we assumed pancake flipping must have been in its pre-training data. We searched our whole pre-training data and found no examples of flipping. S1 inferred the out-of-distribution task from one video prompt.

Skild AI

316,005 görüntüleme • 2 ay önce

Brett Adcock says Figure's newest model is "baby AGI" and the most significant moment he's ever had in robotics. He also said it was trained last month on the most pre-training data and GPUs in Figure's history, with an even larger model ready in a few weeks.

Brett Adcock says Figure's newest model is "baby AGI" and the most significant moment he's ever had in robotics. He also said it was trained last month on the most pre-training data and GPUs in Figure's history, with an even larger model ready in a few weeks.

The Humanoid Labs

65,284 görüntüleme • 9 gün önce

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗

Real-world robot data is expensive and slow to collect, creating a major challenge for humanoid development. 🤖 The NVIDIA GR00T N1.6 open vision language action model is pre-trained on a diverse mix of data, including thousands of hours of Stanford Vision and Learning Lab’s BEHAVIOR simulation data, which covers long-horizon everyday manipulation tasks. This diverse training is the key to robust cross-embodiment performance and real-world adaptability. 🌍 Read the blog 🔗

NVIDIA Robotics

13,562 görüntüleme • 9 ay önce

Why the companies of the future will have specialized models trained on their own data “You are creating value for the model company versus capturing that value for yourself. We need to have more of that surplus that gets created by the use of intelligence remain at the firm. Every firm will have a private eval benchmark, which is their core IP, and every model comes in and tries to hill climb on that eval.” Satya Nadella Love to hear your thoughts Harrison Chase Shreya Shankar Jerry Liu Anjney Midha

Why the companies of the future will have specialized models trained on their own data “You are creating value for the model company versus capturing that value for yourself. We need to have more of that surplus that gets created by the use of intelligence remain at the firm. Every firm will have a private eval benchmark, which is their core IP, and every model comes in and tries to hill climb on that eval.” Satya Nadella Love to hear your thoughts Harrison Chase Shreya Shankar Jerry Liu Anjney Midha

Harry Stebbings

12,629 görüntüleme • 4 gün önce

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”

We asked Angus about the future of robotics and AI training. “I built an industrial-grade kinematic solver for 6 degrees of freedom motion, with full path planning and joint control." "You can stream AI model outputs into it, or run it like a traditional industrial robot." "The benefit of that is all of that information you can stream and record from those joints… you can use that as the training data." "Frankly, moving a robot with a lever is really crap data for training robots. But this gives you smooth, high-quality motion, without needing two robots.”

TBPN

18,811 görüntüleme • 1 yıl önce

Introducing Ψ₀ ( — an open foundation model for universal humanoid loco-manipulation. 🏆 Outperforms GR00T N1.6 by 40%+ overall success rate 📉 Uses only ~10% of the pre-training data 📦 Fully open-source: model, data, code, and deployment pipeline 1/10

Introducing Ψ₀ ( — an open foundation model for universal humanoid loco-manipulation. 🏆 Outperforms GR00T N1.6 by 40%+ overall success rate 📉 Uses only ~10% of the pre-training data 📦 Fully open-source: model, data, code, and deployment pipeline 1/10

Yue Wang

30,472 görüntüleme • 7 ay önce

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

DeepLearning.AI

16,779 görüntüleme • 1 yıl önce