Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Krishna Srinivasan, Founder and CEO of Data Bootstrap and former Google DeepMind researcher, discusses the challenges of data scraping and scalability. With experience at prior Apple, Yahoo, and IBM watsonx, Krishna demonstrates the difference between scraping with a single machine versus leveraging OpenLedger's community nodes. The results speak for... show more

OpenLedger

396,332 subscribers

12,079 views • 1 year ago •via X (Twitter)

Science & Technology News & Politics Education

Anya Rossi• Live Now

Private livecam show

10 Comments

Chioma Chukwurah ⚛️🥷⚔️1 year ago

@GoogleDeepMind @Apple @Yahoo @IBMwatsonx Indeed Great innovation! 🔥🔥

Kjael // Skaling Ventures1 year ago

When I talk to SaaS Founders, they often struggle with subjectivity and bias in decision-making. The solution? An objective point of view informed via benchmarks and base rates. Here are 12x KPIs to assess any SaaS firm clearly and effectively:

Utkarsh0x 🐙1 year ago

@GoogleDeepMind @Apple @Yahoo @IBMwatsonx Great innovation #opnup

Namphox1 year ago

@GoogleDeepMind @Apple @Yahoo @IBMwatsonx #Opnup moon

SON63 🐳1 year ago

@GoogleDeepMind @Apple @Yahoo @IBMwatsonx Thanks boss 🙏🍀🚀🍀🍀

Namiii 🟧 Allo (✸,✸) 🦙🔥1 year ago

@GoogleDeepMind @Apple @Yahoo @IBMwatsonx Perfect 👍 #Opnup

Charisan ♦️꧁IP꧂1 year ago

@GoogleDeepMind @Apple @Yahoo @IBMwatsonx Data scraping sounds like trying to catch a greased pig at a county fair! 🐖 With #PublicAI, we’re all about making data sharing a team sport instead! 🎉

fahadg.base.eth⛺🍸 Tabi 🟧 . Ordzaar ꧁IP꧂ (Ø,G)1 year ago

@GoogleDeepMind @Apple @Yahoo @IBMwatsonx LFG

Wizjust42 ꧁IP꧂1 year ago

@GoogleDeepMind @Apple @Yahoo @IBMwatsonx Looking for a $BOOST Join OpenLedger. Best node yet and it is just beginning. Anyone say DROP ilevzcob4a

Vicky Sharma (✸,✸) CLONE ⛺ Kaisar1 year ago

@GoogleDeepMind @Apple @Yahoo @IBMwatsonx cooking ❤️‍🔥 #Openledger #Opnup

Related Videos

Meet watsonx, our AI and data platform built for business. Leveraging the power of foundation models, generative AI and ML, #watsonx can make the most of your data and offers trusted, scalable, adaptable #AI to embed deep into your business. Learn more:

Meet watsonx, our AI and data platform built for business. Leveraging the power of foundation models, generative AI and ML, #watsonx can make the most of your data and offers trusted, scalable, adaptable #AI to embed deep into your business. Learn more:

IBM

13,319 views • 3 years ago

Scale and accelerate AI across your business with #watsonx. 💡 Here, IBM SVP & Director of Research Dario Gil discusses our enterprise-ready #AI and data platform aimed at providing businesses with a competitive advantage on @SixFivePodcast. Learn more:

Scale and accelerate AI across your business with #watsonx. 💡 Here, IBM SVP & Director of Research Dario Gil discusses our enterprise-ready #AI and data platform aimed at providing businesses with a competitive advantage on @SixFivePodcast. Learn more:

IBM

20,099 views • 3 years ago

“Without data, there is no AI.” As Micron Chairman, President and CEO Sanjay Mehrotra shares, AI’s future depends on the ability to move, store, and access data at incredible scale — and that’s exactly where Micron delivers. From cloud to edge, our memory and storage innovations help transform raw data into real intelligence. Learn more about how we’re powering the #AI era: #IntelligenceAccelerated Sanjay Mehrotra

“Without data, there is no AI.” As Micron Chairman, President and CEO Sanjay Mehrotra shares, AI’s future depends on the ability to move, store, and access data at incredible scale — and that’s exactly where Micron delivers. From cloud to edge, our memory and storage innovations help transform raw data into real intelligence. Learn more about how we’re powering the #AI era: #IntelligenceAccelerated Sanjay Mehrotra

Micron Technology

20,156 views • 5 months ago

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Here's how I would learn data engineering basics in 2025: - Find a data source you care about (examples: gaming APIs, stock market, web scraping, etc) - Use Python to interact and ingest your source. Initially just write the data to a CSV. - Setup an account with Snowflake or Google BigQuery. - update your Python script to load a table in Snowflake/BigQuery - schedule your script with CRON in the cloud with a service like Heroku. - build aggregations and visualizations on top of your ingested data Only thing this misses is data quality and complex job orchestration which you can learn later! How would you learn data engineering nowadays?

Zach Wilson

20,368 views • 1 year ago

Enrollment is now open for the Data Engineering Professional Certificate! Data engineers are the architects of modern organizations, ensuring data is reliable, accessible, and ready for analytics and machine learning. This professional certificate is tailored to equip you with the critical skills, through frameworks and hands-on practice, to excel in this role. Taught by industry expert Joe Reis, co-author of the best-selling book "Fundamentals of Data Engineering," along with 17 guest instructors from the data field, you will gain expertise to start and further your career in the high-demand field of data engineering. Key focus areas: 🗂️ Data Engineering Lifecycle: Learn the important stages of building an efficient data pipeline that creates business value. 📥 Data Ingestion: Learn how to efficiently gather data from various sources. 💾 Data Storage: Master the techniques for storing data securely and cost-effectively. 🔄 Data Transformation: Understand how to clean, organize, and prepare data for analysis and machine learning. 🏗️ Data Architecture Design: Build robust architectures that support scalable, efficient data workflows. 📊 Serving Data: Ensure that data is available to stakeholders when and where they need it to drive business decisions. Enroll now!

Enrollment is now open for the Data Engineering Professional Certificate! Data engineers are the architects of modern organizations, ensuring data is reliable, accessible, and ready for analytics and machine learning. This professional certificate is tailored to equip you with the critical skills, through frameworks and hands-on practice, to excel in this role. Taught by industry expert Joe Reis, co-author of the best-selling book "Fundamentals of Data Engineering," along with 17 guest instructors from the data field, you will gain expertise to start and further your career in the high-demand field of data engineering. Key focus areas: 🗂️ Data Engineering Lifecycle: Learn the important stages of building an efficient data pipeline that creates business value. 📥 Data Ingestion: Learn how to efficiently gather data from various sources. 💾 Data Storage: Master the techniques for storing data securely and cost-effectively. 🔄 Data Transformation: Understand how to clean, organize, and prepare data for analysis and machine learning. 🏗️ Data Architecture Design: Build robust architectures that support scalable, efficient data workflows. 📊 Serving Data: Ensure that data is available to stakeholders when and where they need it to drive business decisions. Enroll now!

DeepLearning.AI

20,833 views • 1 year ago

It's easy for robots to nail a demo. It's much harder to get them to work reliably in the real world. The bottleneck is data. At Axis Robotics we're building a compounding data engine that combines simulation, ego-centric data and DAgger correction data at scale. Watch our CEO ChrisF(✱,✱) break down the data bottleneck, why messy data can still train good models, and what it takes to get robots out of the lab with blocmates. ⬇️

It's easy for robots to nail a demo. It's much harder to get them to work reliably in the real world. The bottleneck is data. At Axis Robotics we're building a compounding data engine that combines simulation, ego-centric data and DAgger correction data at scale. Watch our CEO ChrisF(✱,✱) break down the data bottleneck, why messy data can still train good models, and what it takes to get robots out of the lab with blocmates. ⬇️

Axis Robotics

22,634 views • 13 days ago

Introducing the Future of AI-Powered Sports Data with $FXI.⚡️ We’re not just talking about innovation, we’re building it. 🚀 From #AI-driven data capture to on-chain validation and data monetization, the FX1 ecosystem is rewriting the rules of the sports and data economy. Our platform empowers leagues, teams, and creators to unlock the full potential of their data with cutting-edge AI solutions. 🧠 Why FX1? ➡️ AI-Powered Precision: Transform raw sports data into actionable insights. ➡️ Data Validation on Chain: Building trust in the data economy through blockchain. ➡️ AI-Powered Revenue Streams: Use sports data to drive predictive analytics, odds modeling, and other high-value solutions that unlock new ways to monetize. FX1 is leading the charge in AI innovation, creating a seamless bridge between sports, blockchain, and the data economy. This isn’t just evolution - it’s a revolution. 💡 The future of sports is here. And it’s powered by AI. 🤖 Watch the full video and see how we’re turning vision into reality. 👇

Introducing the Future of AI-Powered Sports Data with $FXI.⚡️ We’re not just talking about innovation, we’re building it. 🚀 From #AI-driven data capture to on-chain validation and data monetization, the FX1 ecosystem is rewriting the rules of the sports and data economy. Our platform empowers leagues, teams, and creators to unlock the full potential of their data with cutting-edge AI solutions. 🧠 Why FX1? ➡️ AI-Powered Precision: Transform raw sports data into actionable insights. ➡️ Data Validation on Chain: Building trust in the data economy through blockchain. ➡️ AI-Powered Revenue Streams: Use sports data to drive predictive analytics, odds modeling, and other high-value solutions that unlock new ways to monetize. FX1 is leading the charge in AI innovation, creating a seamless bridge between sports, blockchain, and the data economy. This isn’t just evolution - it’s a revolution. 💡 The future of sports is here. And it’s powered by AI. 🤖 Watch the full video and see how we’re turning vision into reality. 👇

FX1

23,913 views • 1 year ago

How they built surveillance capitalism: 1) Big Tech stole our data 2) Sold it to other companies 3) These companies now use our data to charge us higher prices for goods & services This is exactly why they’re building all these data centers: to surveil, track, and weaponize our own data against us. The age of 𝐀𝐈 𝐬𝐮𝐫𝐯𝐞𝐢𝐥𝐥𝐚𝐧𝐜𝐞 𝐜𝐚𝐩𝐢𝐭𝐚𝐥𝐢𝐬𝐦 is here and it’s even more dystopian than you thought.

How they built surveillance capitalism: 1) Big Tech stole our data 2) Sold it to other companies 3) These companies now use our data to charge us higher prices for goods & services This is exactly why they’re building all these data centers: to surveil, track, and weaponize our own data against us. The age of 𝐀𝐈 𝐬𝐮𝐫𝐯𝐞𝐢𝐥𝐥𝐚𝐧𝐜𝐞 𝐜𝐚𝐩𝐢𝐭𝐚𝐥𝐢𝐬𝐦 is here and it’s even more dystopian than you thought.

Power to the People ☭🕊

19,703 views • 13 days ago

Data powers digital transformation! ST Engineering MRAS is leveraging Spot to get a full view of critical equipment and facility. Working autonomously, Spot collects information about machine health and, with the integration of the Leica BLK ARC for reality capture, detailed and accurate point cloud data for their digital twin. Watch the full video:

Data powers digital transformation! ST Engineering MRAS is leveraging Spot to get a full view of critical equipment and facility. Working autonomously, Spot collects information about machine health and, with the integration of the Leica BLK ARC for reality capture, detailed and accurate point cloud data for their digital twin. Watch the full video:

Boston Dynamics

19,483 views • 6 months ago

1/ Cherry AI Robotics is building the world’s first data layer for robotics. Every autonomous robot needs a neural network to operate, which requires massive domain-specific data. Cherry AI Robotics has an entire ecosystem of products with 200k+ communities and 800k+ users. These products generate large amounts of data, which is parsed to create valuable data sets for robotics. Learn more:

1/ Cherry AI Robotics is building the world’s first data layer for robotics. Every autonomous robot needs a neural network to operate, which requires massive domain-specific data. Cherry AI Robotics has an entire ecosystem of products with 200k+ communities and 800k+ users. These products generate large amounts of data, which is parsed to create valuable data sets for robotics. Learn more:

CherryAI Robotics

105,357 views • 11 months ago

Scale alone is not enough for AI data. Quality and complexity are equally critical. Excited to support all of these for LLM developers with Snorkel AI Data-as-a-Service, and to share our new leaderboard! — Our decade-plus of research and work in AI data has a simple point: scale alone is not enough. AI success is all about the quality, complexity, and distribution of data—in addition to volume. We’re excited to be powering leading LLM developers with Snorkel AI Expert Data-as-a-Service, our white glove service for custom, expert-level AI datasets—and to now preview some of what we’re building via our new Expert Data Leaderboard (🔗 in 🧵) + upcoming OSS dataset releases! Snorkel Expert Data-as-a-Service is built to meet the rapidly evolving data needs of the agentic AI world—where success is built on the quality, complexity, and distribution of datasets, in addition to size and scale. This kind of high-quality, frontier AI data can only come from a union of technology and human expertise. With Snorkel Expert Data-as-a-Service, we’re powering frontier LLM developers across agentic, expert knowledge, reasoning, coding, multi-modal, and other task types via the combination of these two key components: - (1) The Snorkel Expert Network: A global team of subject matter experts focused wholly on specialized knowledge–spanning thousands of topics in STEM/academic, vertical/professional, and consumer/lifestyle domains. - (2) Snorkel AI Data Development Platform: Our unique programmatic data curation and quality control platform, accelerating and improving expert authoring and review through principled techniques developed over the last decade of R&D. Now: we’re incredibly excited to showcase some of the power of Snorkel Expert Data-as-a-Service via the new Snorkel Leaderboard—putting frontier models to the test in complex, agentic, and reasoning settings inspired by real industry scenarios (not esoteric puzzles)! We’ll be releasing new leaderboards and accompanying expert-verified open source datasets (coming soon!) regularly. To start, we’re sharing three initial ones in preview: - SnorkelFinance: Q&A over financial documents requiring agentic tool-calling and reasoning - SnorkelUnderwrite: Agentic insurance tasks requiring industry-specific reasoning and tool use - SnorkelSequences: Mathematical tasks requiring compositional multi-step reasoning

Scale alone is not enough for AI data. Quality and complexity are equally critical. Excited to support all of these for LLM developers with Snorkel AI Data-as-a-Service, and to share our new leaderboard! — Our decade-plus of research and work in AI data has a simple point: scale alone is not enough. AI success is all about the quality, complexity, and distribution of data—in addition to volume. We’re excited to be powering leading LLM developers with Snorkel AI Expert Data-as-a-Service, our white glove service for custom, expert-level AI datasets—and to now preview some of what we’re building via our new Expert Data Leaderboard (🔗 in 🧵) + upcoming OSS dataset releases! Snorkel Expert Data-as-a-Service is built to meet the rapidly evolving data needs of the agentic AI world—where success is built on the quality, complexity, and distribution of datasets, in addition to size and scale. This kind of high-quality, frontier AI data can only come from a union of technology and human expertise. With Snorkel Expert Data-as-a-Service, we’re powering frontier LLM developers across agentic, expert knowledge, reasoning, coding, multi-modal, and other task types via the combination of these two key components: - (1) The Snorkel Expert Network: A global team of subject matter experts focused wholly on specialized knowledge–spanning thousands of topics in STEM/academic, vertical/professional, and consumer/lifestyle domains. - (2) Snorkel AI Data Development Platform: Our unique programmatic data curation and quality control platform, accelerating and improving expert authoring and review through principled techniques developed over the last decade of R&D. Now: we’re incredibly excited to showcase some of the power of Snorkel Expert Data-as-a-Service via the new Snorkel Leaderboard—putting frontier models to the test in complex, agentic, and reasoning settings inspired by real industry scenarios (not esoteric puzzles)! We’ll be releasing new leaderboards and accompanying expert-verified open source datasets (coming soon!) regularly. To start, we’re sharing three initial ones in preview: - SnorkelFinance: Q&A over financial documents requiring agentic tool-calling and reasoning - SnorkelUnderwrite: Agentic insurance tasks requiring industry-specific reasoning and tool use - SnorkelSequences: Mathematical tasks requiring compositional multi-step reasoning

Alex Ratner

495,851 views • 1 year ago

My team is hiring in multimodal representation learning. We are working on tactile representations to equip robots with a physical understanding of the world. Bored of scraping disembodied internet data and looking for a new challenge? Apply here:

My team is hiring in multimodal representation learning. We are working on tactile representations to equip robots with a physical understanding of the world. Bored of scraping disembodied internet data and looking for a new challenge? Apply here:

Mustafa Mukadam

56,704 views • 2 years ago

Highlights of the OpenLedger journey: OpenLedger is the AI Blockchain unlocking liquidity for data, models, and agents, backed by Polychain, Balaji Srinivasan (ex-Coinbase CTO), and more. $14.7M+ in revenue from global enterprise projects and $91M+ in investment commitments secured. Worked with global enterprises like Walmart, Sony, Meta LLaMA, Viacom, and others. Built on Stanford’s data attribution research paper, enabling verifiable, fair rewards for contributors. 40+ projects, including ex-Google DeepMind teams and top DeFi builders, are building specialized AI models. 1M+ AI nodes, 25M+ transactions, and 20K+ models, real-world usage at scale.

Highlights of the OpenLedger journey: OpenLedger is the AI Blockchain unlocking liquidity for data, models, and agents, backed by Polychain, Balaji Srinivasan (ex-Coinbase CTO), and more. $14.7M+ in revenue from global enterprise projects and $91M+ in investment commitments secured. Worked with global enterprises like Walmart, Sony, Meta LLaMA, Viacom, and others. Built on Stanford’s data attribution research paper, enabling verifiable, fair rewards for contributors. 40+ projects, including ex-Google DeepMind teams and top DeFi builders, are building specialized AI models. 1M+ AI nodes, 25M+ transactions, and 20K+ models, real-world usage at scale.

OpenLedger

739,225 views • 1 year ago

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.

Web scraping is a critical skill, and yet nobody talks about it. How do you think companies are training their Large Language Models? Where do you think the data comes from? But web scraping goes beyond all of that. Imagine giving an AI agent access to any public online data in real time! I like to call this "web-scrapping on demand", and I'm pretty sure it's going to unlock unlimited power for AI applications. I recorded a quick video to show you how you can do this using Apify. I've talked about them before, and they are collaborating with me on this post. They have one of the best open-source web scraping and browser automation libraries out there: But it gets much better than this! You can use MCP to connect your AI Agents and applications to the Apify platform and use any specialized actor on demand to scrape and process online data. In the video, I used Cursor to scrape LinkedIn posts with the words "Machine Learning" in real time. Worked like a charm with no code needed! Here is a link to the platform: Think about this: You can now feed your AI applications with any public data on demand! We aren't ready for what's coming.

Santiago

101,473 views • 1 year ago

Today we're launching Hebbian Robotics (YC S26) Hebbian Robotics (YC S26) . We build APIs for evaluating data quality, without training a robotics model. Data vendors use our API to search and analyze Physical AI data at scale, getting on-demand quality signals and metrics without managing infrastructure. High-quality data is the bottleneck to the next frontier of intelligence in robotics. Buyers want data they can trust. Data providers are focused on operational excellence. They don't have the bandwidth to understand the quality of their data at scale. We're fixing this. Vendors use our APIs to understand the impact of their decisions on their data quality: - Buyers demanding per-collector hour limits? Prove compliance while we stay ahead of the workarounds. - Changed your collection SOPs? Monitor for quality drift and regression. - Suspect duplicate samples? Verify with a single API call. Before Hebbian, I trained AI models for industrial cleaning robots, and kingston kuan spent years building large-scale infrastructure at Jane Street.

Today we're launching Hebbian Robotics (YC S26) Hebbian Robotics (YC S26) . We build APIs for evaluating data quality, without training a robotics model. Data vendors use our API to search and analyze Physical AI data at scale, getting on-demand quality signals and metrics without managing infrastructure. High-quality data is the bottleneck to the next frontier of intelligence in robotics. Buyers want data they can trust. Data providers are focused on operational excellence. They don't have the bandwidth to understand the quality of their data at scale. We're fixing this. Vendors use our APIs to understand the impact of their decisions on their data quality: - Buyers demanding per-collector hour limits? Prove compliance while we stay ahead of the workarounds. - Changed your collection SOPs? Monitor for quality drift and regression. - Suspect duplicate samples? Verify with a single API call. Before Hebbian, I trained AI models for industrial cleaning robots, and kingston kuan spent years building large-scale infrastructure at Jane Street.

Brandon Ong

165,365 views • 15 days ago