Загрузка видео...

Не удалось загрузить видео

На главную

No data, no problem introducing agentic synthetic data generation with Cosmos 3 share a few examples, generate more data, automate model training, automatically deploy the latest version with no downtime in a benchmark run with Corning Incorporated's optical fiber manufacturing engineering team, a model trained on 8 real defect...

roboflow's profile picture

Roboflow

13,748 subscribers

39,115 просмотров • 2 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.
2:05

A humanoid robot policy trained solely on synthetic data generated by a world model. Research Scientist Joel Jang presents NVIDIA's DreamGen pipeline: ⦿ Post-train the world model Cosmos-Predict2 with a small set of real teleoperation demos. ⦿ Prompt the world model to generate synthetic video data with verbs and scenarios not used in the world model’s post-training. ⦿ Auto-label synthetic video data with action sequences. ⦿ Train robot policies using only synthetic data. That's it. Deploy zero-shot to a real humanoid robot.

The Humanoid Hub

20,968 просмотров • 1 год назад

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move
3:58

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

OpenLedger

61,662 просмотров • 1 год назад

We’re excited to share that 🥇Llama Nemotron Super 49B v1.5 -- our latest open reasoning model -- is now #1 on the Artificial Analysis Intelligence Index - a leaderboard that spans advanced math, science, and agentic tasks, in the 70B open model category. Llama Nemotron Super 49B v1.5 is trained with high-quality reasoning synthetic data generated from models like Qwen3-235B and DeepSeek R1. It delivers state-of-the-art accuracy and throughput, running on a single H100. Key features: 🎯 Leading accuracy on multi-step reasoning, math, coding, and function-calling 🏗️ Post-trained using RPO, DPO, and RLVR across 26M+ synthetic examples 📊 Fully transparent training data and techniques If you're building AI agents and want a high accuracy, fully-open, and transparent reasoning model that you can deploy anywhere, try Super v1.5 on or download from Hugging Face 🤗 ➡️ Leaderboard:
0:54

We’re excited to share that 🥇Llama Nemotron Super 49B v1.5 -- our latest open reasoning model -- is now #1 on the Artificial Analysis Intelligence Index - a leaderboard that spans advanced math, science, and agentic tasks, in the 70B open model category. Llama Nemotron Super 49B v1.5 is trained with high-quality reasoning synthetic data generated from models like Qwen3-235B and DeepSeek R1. It delivers state-of-the-art accuracy and throughput, running on a single H100. Key features: 🎯 Leading accuracy on multi-step reasoning, math, coding, and function-calling 🏗️ Post-trained using RPO, DPO, and RLVR across 26M+ synthetic examples 📊 Fully transparent training data and techniques If you're building AI agents and want a high accuracy, fully-open, and transparent reasoning model that you can deploy anywhere, try Super v1.5 on or download from Hugging Face 🤗 ➡️ Leaderboard:

NVIDIA AI Developer

100,506 просмотров • 1 год назад

Announcing Roboflow Rapid: the first prompt-based model creation engine. Data labeling is dead. Go from idea to deployed model in minutes without labeling data. Upload a video, type a text prompt, and get an API. No data labeling teams. No manual annotation. No infra / dependency hell. If you are spending 95% of your time labeling data and only 5% building your app, you are doing it wrong.
2:07

Announcing Roboflow Rapid: the first prompt-based model creation engine. Data labeling is dead. Go from idea to deployed model in minutes without labeling data. Upload a video, type a text prompt, and get an API. No data labeling teams. No manual annotation. No infra / dependency hell. If you are spending 95% of your time labeling data and only 5% building your app, you are doing it wrong.

Roboflow

22,959 просмотров • 8 месяцев назад

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.
1:43

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.

AK

40,485 просмотров • 2 лет назад

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast
1:29

🚨 Jensen Huang says everyone panicked about the AI data when MOST training data was never REAL to begin with. Ilya Sutskever told the industry pre-training was over. "Ilya said, 'We're out of data,' or something like that. 'Pre-training is over,' or something like that," Huang says. "The industry panicked, you know, that this is the end of AI." "And of course, of course that's obviously not true. We're gonna keep on scaling the amount of data that we have to train with." "A lot of that data is probably gonna be synthetic." That's where the panic came from — synthetic data sounds like cheating. "Most of the data that we are training, that we teach each other with, inform each other with, is synthetic." "It's synthetic because it didn't come out of nature." "You created it. I'm consuming it. I modify it, augment it, I regenerate it, somebody else consumes it." The textbook in your hand is synthetic. The post you're reading is synthetic. The lecture you took is synthetic. Nature didn't make any of it. Humans did. AI just learned to do the same thing — faster. "Training is now limited by compute," Huang says. "Data is now limited by compute." The data wall wasn't a wall. It was a mirror. If you're new here, follow @AiEvolutio for the latest on ChatGPT, Claude, and the AI tools shaping how we work and create. — Jensen Huang ( NVIDIA ), NVIDIA CEO, on Lex Fridman's ( Lex Fridman ) podcast

AI Evolution

15,565 просмотров • 2 месяцев назад

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.
0:51

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

AK

367,110 просмотров • 1 год назад

Google engineer explained how to fine-tune a tiny LLM from 46% to 90% accuracy on your phone in 21 minutes - better than $1500 on-device AI bootcamps. pick Gemma 270M -> generate synthetic task data -> fine-tune with LoRA -> quantize to int4 -> deploy to Pixel and hit 2000 tokens per second. That loop is how a 270M model beats a 70B one on your task, running fully offline in your pocket. Gemma 270M + synthetic data + LoRA + int4 quantization + on-device runtime - that's the stack. Watch and save it, then fine-tune your own tiny agent tonight.
20:45

Google engineer explained how to fine-tune a tiny LLM from 46% to 90% accuracy on your phone in 21 minutes - better than $1500 on-device AI bootcamps. pick Gemma 270M -> generate synthetic task data -> fine-tune with LoRA -> quantize to int4 -> deploy to Pixel and hit 2000 tokens per second. That loop is how a 270M model beats a 70B one on your task, running fully offline in your pocket. Gemma 270M + synthetic data + LoRA + int4 quantization + on-device runtime - that's the stack. Watch and save it, then fine-tune your own tiny agent tonight.

h100envy

700,758 просмотров • 22 дней назад

Today we announced the new Einstein Copilot Search and Salesforce Vector Database in Data Cloud to power semantic search and retrieval augmented generation. This will enhance Einstein Copilot's ability to understand, generate outputs, and automate actions across a wide variety of use cases, contexts, and data/content types. LLMs can't be relied on to recall specific data they were trained on, so the way to make them work in the enterprise, where accuracy is paramount, is to feed them with the right data using hybrid keyword and vector search-powered RAG. By bringing all this seamlessly into our apps and Einstein Copilot UX as well as Hyperforce cloud infrastructure, we can offer high performance, low latency, addressing data privacy, security, and residency requirements🌟
3:06

Today we announced the new Einstein Copilot Search and Salesforce Vector Database in Data Cloud to power semantic search and retrieval augmented generation. This will enhance Einstein Copilot's ability to understand, generate outputs, and automate actions across a wide variety of use cases, contexts, and data/content types. LLMs can't be relied on to recall specific data they were trained on, so the way to make them work in the enterprise, where accuracy is paramount, is to feed them with the right data using hybrid keyword and vector search-powered RAG. By bringing all this seamlessly into our apps and Einstein Copilot UX as well as Hyperforce cloud infrastructure, we can offer high performance, low latency, addressing data privacy, security, and residency requirements🌟

Clara Shih

49,620 просмотров • 2 лет назад

Learn to train an LLM with distributed data while ensuring privacy using federated learning in a new two-part short course, Intro to Federated Learning and Federated Fine-tuning of LLMs with Private Data, created with Flower and taught by Daniel J. Beutel and nic lane. Federated learning allows a single model to be trained across multiple devices, such as phones, or multiple organizations, such as hospitals, without the need to share data to a central server. This two-part course gives you an introduction to federated learning, and then teaches you how to fine-tune your large language model with distributed data using Flower Lab’s open source federated learning framework. You’ll learn: - How to use federated learning to train a variety of models, ranging from speech and vision models to LLMs, across distributed data while offering data privacy options to users and organizations. - Privacy Enhancing Technologies like differential privacy (DP), which obscures individual data by adding calibrated noise to query results. - Two variants of differential privacy - Central and Local - and how to choose depending on your use case. - How to measure and decrease bandwidth usage to make federated learning more practical and efficient with techniques like using pre-trained models and Parameter-Efficient Fine-Tuning - How federated LLM fine-tuning reduces the risk of leaking training data. Sign up here!
4:34

Learn to train an LLM with distributed data while ensuring privacy using federated learning in a new two-part short course, Intro to Federated Learning and Federated Fine-tuning of LLMs with Private Data, created with Flower and taught by Daniel J. Beutel and nic lane. Federated learning allows a single model to be trained across multiple devices, such as phones, or multiple organizations, such as hospitals, without the need to share data to a central server. This two-part course gives you an introduction to federated learning, and then teaches you how to fine-tune your large language model with distributed data using Flower Lab’s open source federated learning framework. You’ll learn: - How to use federated learning to train a variety of models, ranging from speech and vision models to LLMs, across distributed data while offering data privacy options to users and organizations. - Privacy Enhancing Technologies like differential privacy (DP), which obscures individual data by adding calibrated noise to query results. - Two variants of differential privacy - Central and Local - and how to choose depending on your use case. - How to measure and decrease bandwidth usage to make federated learning more practical and efficient with techniques like using pre-trained models and Parameter-Efficient Fine-Tuning - How federated LLM fine-tuning reduces the risk of leaking training data. Sign up here!

Andrew Ng

64,558 просмотров • 2 лет назад

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models
0:30

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models

AK

62,719 просмотров • 1 год назад

We're unlocking next-level AI capabilities with the launch of our Retrieval Augmentation Generation (RAG) architecture! 🌟 By integrating a sophisticated information retrieval system, RAG empowers our Large Language Model (LLM), Qwen, with an extra layer of context, allowing it to access industry-specific insights or a company's proprietary data on demand. At the heart of RAG's knowledgebase lies the Vector Engine – a powerhouse of semantic recall ensuring all data is meticulously preserved and readily accessible. Plus, with Alibaba Cloud's suite of vector database solutions, we're at the forefront of augmented data retrieval, offering unprecedented precision and speed. Get ready to transform your data strategy with RAG! #AlibabaCloud #AI #Innovation #DataStrategy #VectorEngine #RAG #AIandBigDataSummit2024 #AIForward
0:22

We're unlocking next-level AI capabilities with the launch of our Retrieval Augmentation Generation (RAG) architecture! 🌟 By integrating a sophisticated information retrieval system, RAG empowers our Large Language Model (LLM), Qwen, with an extra layer of context, allowing it to access industry-specific insights or a company's proprietary data on demand. At the heart of RAG's knowledgebase lies the Vector Engine – a powerhouse of semantic recall ensuring all data is meticulously preserved and readily accessible. Plus, with Alibaba Cloud's suite of vector database solutions, we're at the forefront of augmented data retrieval, offering unprecedented precision and speed. Get ready to transform your data strategy with RAG! #AlibabaCloud #AI #Innovation #DataStrategy #VectorEngine #RAG #AIandBigDataSummit2024 #AIForward

Alibaba Cloud

9,636,416 просмотров • 2 лет назад

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)
0:41

So I heard we need more data for robot learning :) Purely real world teleop is expensive and slow, making large scale data collection challenging. I’ve been excited about getting more data into robot learning, going beyond just real-world teleop data. To this end, we’ve been scaling up data generation with RL in realistic simulations generated on the fly from crowdsourced videos. Enables realistic data collection, much more cheaply than purely real world teleop. Importantly, data collection becomes even*cheaper* with more environments, allowing training with over 100x more data. Transfers to real robots for generalizable manipulation. A 🧵 (1/N)

Abhishek Gupta

13,350 просмотров • 1 год назад

NVIDIA CEO Jensen Huang on Tesla FSD at CES: “Tesla’s FSD stack is completely world-class. They’ve been working on it for quite some time. It’s world-class. Not only in the number of miles it has, but it’s world-class in the way that it’s designed, the way they do training to data collection and curation, synthetic data generation, all of their simulation technologies. Of course, the latest generation is end-to-end full self-driving, meaning it’s just one large model that is end-to-end trained. Elon’s AV system is, in every way, 100% state-of-the-art. And so I’m really quite impressed by the technology. I have it and I drive it in our house and it works incredibly well.”
1:11

NVIDIA CEO Jensen Huang on Tesla FSD at CES: “Tesla’s FSD stack is completely world-class. They’ve been working on it for quite some time. It’s world-class. Not only in the number of miles it has, but it’s world-class in the way that it’s designed, the way they do training to data collection and curation, synthetic data generation, all of their simulation technologies. Of course, the latest generation is end-to-end full self-driving, meaning it’s just one large model that is end-to-end trained. Elon’s AV system is, in every way, 100% state-of-the-art. And so I’m really quite impressed by the technology. I have it and I drive it in our house and it works incredibly well.”

Nic Cruz Patane

52,826 просмотров • 7 месяцев назад

Introducing HRM-Text. An ultra-lean 1B-parameter reasoning language model designed to deliver strong general performance with a fraction of the data, compute, and infrastructure. Trained on just 40B structured tokens, HRM-Text achieves competitive performance while using ~1/1000 of the training data of comparable models. The kicker? The full model trains in roughly one day on a $1,000 budget. This opens the door to a new generation of AI that is powerful, accessible, and radically easier to adapt. Theories and research concepts once deemed too expensive to test are officially back in the game. Sapient Intelligence invites you to help us shape a new paradigm for general intelligence.
5:31

Introducing HRM-Text. An ultra-lean 1B-parameter reasoning language model designed to deliver strong general performance with a fraction of the data, compute, and infrastructure. Trained on just 40B structured tokens, HRM-Text achieves competitive performance while using ~1/1000 of the training data of comparable models. The kicker? The full model trains in roughly one day on a $1,000 budget. This opens the door to a new generation of AI that is powerful, accessible, and radically easier to adapt. Theories and research concepts once deemed too expensive to test are officially back in the game. Sapient Intelligence invites you to help us shape a new paradigm for general intelligence.

Sapient Intelligence

515,089 просмотров • 2 месяцев назад

FRIEDBERG: VIDEO DATA WILL POWER THE NEXT GENERATION OF AI Friedberg broke down the scale shift coming to artificial intelligence, arguing that text based models like GPT are just the beginning, and that the real revolution will come from video-trained systems: “The internet and all these LLMs are language models trained on text from the internet, around 50 billion words total, maybe one to five terabytes of data in their training sets. But if you look at the video data out there, there are hundreds of billions of hours, much of it on YouTube. By some estimates, there’s a thousand exabytes of video data on the internet, about a billion times more than text data. I think we just saw that play out with the new video model that launched yesterday. Google has all this YouTube data, whether or not they’re using it to train, I don’t know. I’ve heard from insiders they’re not allowed to yet and would have to redo the terms of service.” Source: AIFinInsights david friedberg
2:12

FRIEDBERG: VIDEO DATA WILL POWER THE NEXT GENERATION OF AI Friedberg broke down the scale shift coming to artificial intelligence, arguing that text based models like GPT are just the beginning, and that the real revolution will come from video-trained systems: “The internet and all these LLMs are language models trained on text from the internet, around 50 billion words total, maybe one to five terabytes of data in their training sets. But if you look at the video data out there, there are hundreds of billions of hours, much of it on YouTube. By some estimates, there’s a thousand exabytes of video data on the internet, about a billion times more than text data. I think we just saw that play out with the new video model that launched yesterday. Google has all this YouTube data, whether or not they’re using it to train, I don’t know. I’ve heard from insiders they’re not allowed to yet and would have to redo the terms of service.” Source: AIFinInsights david friedberg

Mario Nawfal

15,855 просмотров • 8 месяцев назад

Building a truly private AI model isn’t as simple as the tech world wants you to believe. despite all the marketing around one click deployments, RAG systems, fine tuning services, and memory features, we still don’t have models that are genuinely trained only on your personal data with complete privacy from third parties. the reality is that most AI solutions today require you to trust someone else with your books, notes, journals, and training data. even when companies promise privacy, there’s usually a way for the provider or other parties to access your information during training or inference. Phala is taking a different approach by building the infrastructure needed for real data ownership. They’re developing confidential runtime environments where your AI training happens inside secure enclaves with attestation guarantees. The key innovation is their in-enclave keying system through dstack, which means that once your model is training, even Phala themselves cannot see your data or model weights.
0:34

Building a truly private AI model isn’t as simple as the tech world wants you to believe. despite all the marketing around one click deployments, RAG systems, fine tuning services, and memory features, we still don’t have models that are genuinely trained only on your personal data with complete privacy from third parties. the reality is that most AI solutions today require you to trust someone else with your books, notes, journals, and training data. even when companies promise privacy, there’s usually a way for the provider or other parties to access your information during training or inference. Phala is taking a different approach by building the infrastructure needed for real data ownership. They’re developing confidential runtime environments where your AI training happens inside secure enclaves with attestation guarantees. The key innovation is their in-enclave keying system through dstack, which means that once your model is training, even Phala themselves cannot see your data or model weights.

soulman 🎮

11,702 просмотров • 10 месяцев назад

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:
0:46

Synthetic data will provide the next trillion tokens to fuel our hungry models. I'm excited to announce MimicGen: massively scaling up data pipeline for robot learning! We multiply high-quality human data in simulation with digital twins. Using 50,000 training episodes across 18 tasks, multiple simulators, and even in the real-world! The idea is simple: 1. Humans tele-operate the robot to complete a task. It is extremely high-quality but also very slow and expensive. 2. We create a digital twin of the robot and the scene in high-fidelity, GPU-accelerated simulation. 3. We can now move objects around, replace with new assets, and even change the robot hand - basically augment the training data with procedural generation. 4. Export the successful episodes, and feed that to a neural network! You now have an near-infinite stream of data. One of the key reasons that robotics lags far behind other AI fields is the lack of data: you cannot scrape control signals from the internet. They simply don't exist in-the-wild. MimicGen shows the power of synthetic data and simulation to keep our scaling laws alive. I believe this principle apply beyond robotics. We are quickly exhausting the high-quality, real tokens from the web. Artificial intelligence from artificial data will be the way forward. We are big fans of the OSS community. As usual, we open-source everything, including the generated dataset! - Website: - Paper: - Dataset is hosted on HuggingFace (thanks AK!!): - Code: MimicGen is led by Ajay Mandlekar, deep dive in the thread:

Jim Fan

332,238 просмотров • 2 лет назад

To everyone wondering if Tesla's FSD moat has eroded thanks to Nvidia's keynote - here's the answer: Think of Nvidia as a seller of a toolkit. You can buy pieces built specifically for a job but once you have the tools you still have to build the entire project. In this case...a model Many seem to be in awe over Nvidia's Cosmos which is a platform that includes World Foundation Models that can generate synthetic data for training AI systems like autonomous vehicles I'll go into much more depth in today's episode but here's a clip of Ashok Elluswamy at CVPR '23 explaining how Tesla is already using a similar approach to augment its real world data set This also doesn't even consider the fact that much of the auto industry using Nvidia "tools" will be forced to pay 50%+ margins just to buy the toolkit and will be locked in to Nvidia's system Nor does this touch on legacy auto needing to hire top ML engineering talent to actually put these tools to work The path to autonomy will be real world data as the foundation and simulations/synthetic data as a supplement. There is no path to autonomy with synthetic data alone. More to come later $TSLA As Elon said earlier this year, "it's remarkable how quickly we run out of human-created data. Reality itself and synthetic data ftw" "What you are seeing here is purely generated video sequences - given the past videos the network predicts some sample from the future, hopefully the most likely sample. It is being predicted not just for one camera, but it predicts for all 8 cameras around the car jointly" - Ashok
4:41

To everyone wondering if Tesla's FSD moat has eroded thanks to Nvidia's keynote - here's the answer: Think of Nvidia as a seller of a toolkit. You can buy pieces built specifically for a job but once you have the tools you still have to build the entire project. In this case...a model Many seem to be in awe over Nvidia's Cosmos which is a platform that includes World Foundation Models that can generate synthetic data for training AI systems like autonomous vehicles I'll go into much more depth in today's episode but here's a clip of Ashok Elluswamy at CVPR '23 explaining how Tesla is already using a similar approach to augment its real world data set This also doesn't even consider the fact that much of the auto industry using Nvidia "tools" will be forced to pay 50%+ margins just to buy the toolkit and will be locked in to Nvidia's system Nor does this touch on legacy auto needing to hire top ML engineering talent to actually put these tools to work The path to autonomy will be real world data as the foundation and simulations/synthetic data as a supplement. There is no path to autonomy with synthetic data alone. More to come later $TSLA As Elon said earlier this year, "it's remarkable how quickly we run out of human-created data. Reality itself and synthetic data ftw" "What you are seeing here is purely generated video sequences - given the past videos the network predicts some sample from the future, hopefully the most likely sample. It is being predicted not just for one camera, but it predicts for all 8 cameras around the car jointly" - Ashok

Dillon Loomis

60,753 просмотров • 1 год назад