正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

What if you could instantly access & analyze all the dYdX data in a reliable way? KYVE makes this happen ⚡ Recently, KYVE hosted a developer workshop providing an overview of how dYdX users can leverage KYVE’s tooling for an enhanced data & overall building experience. KYVE’s Data Pipeline... show more

KYVE 💫

69,644 subscribers

16,289 次观看 • 1 年前 •via X (Twitter)

科学技术教育财经

Anya Rossi• Live Now

Private livecam show

2 条评论

KYVE 💫 的头像

KYVE 💫1 年前

ICYMI, here’s the full replay of KYVE’s latest Dev Workshop for @dYdX nodes, devs, data analysts, & other data users ⤵️

denbecha 的头像

denbecha1 年前

@dYdX Get ready for $ISLM to smash through all-time highs!

相关视频

⚡️ What if you could sync your node to any dYdX block height in seconds? Look no further than KYVE’s KSYNC! 🤝 No more unstable peer-to-peer connections or missing archive nodes, KSYNC enables seamless access to historical blocks, state snapshots, or full blockchain history of the dYdX chain through KYVE. Here’s how dYdX users can use KSYNC ⤵️

⚡️ What if you could sync your node to any dYdX block height in seconds? Look no further than KYVE’s KSYNC! 🤝 No more unstable peer-to-peer connections or missing archive nodes, KSYNC enables seamless access to historical blocks, state snapshots, or full blockchain history of the dYdX chain through KYVE. Here’s how dYdX users can use KSYNC ⤵️

KYVE 💫

39,035 次观看 • 1 年前

"If you're an analytical SaaS company, it's over. It's over." - Palo Alto Networks CEO Nikesh Arora "What is an analytical SaaS company? It is a company that says, “I'm going to collect a lot of data for you and analyze it for you.” I don't need you to analyze it for me. I can run models against the data and analyze it myself. Think about it: almost every SaaS company has a marketplace. You can buy applications from the Salesforce Marketplace. What do those applications say? “You have Salesforce data. I'm a marketplace app. Use me, and I'll help you analyze that data.” But I don't need to do that. You don't need those applications anymore. I can simply run an LLM against the data. Therefore, all the additional functionality that has traditionally been sold to us as incremental software modules may no longer need to be sold separately. I would much rather have an LLM run against the data." ---- From "All-In Podcast" YouTube channel, (full video link in comment)

"If you're an analytical SaaS company, it's over. It's over." - Palo Alto Networks CEO Nikesh Arora "What is an analytical SaaS company? It is a company that says, “I'm going to collect a lot of data for you and analyze it for you.” I don't need you to analyze it for me. I can run models against the data and analyze it myself. Think about it: almost every SaaS company has a marketplace. You can buy applications from the Salesforce Marketplace. What do those applications say? “You have Salesforce data. I'm a marketplace app. Use me, and I'll help you analyze that data.” But I don't need to do that. You don't need those applications anymore. I can simply run an LLM against the data. Therefore, all the additional functionality that has traditionally been sold to us as incremental software modules may no longer need to be sold separately. I would much rather have an LLM run against the data." ---- From "All-In Podcast" YouTube channel, (full video link in comment)

Rohan Paul

40,959 次观看 • 27 天前

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Santiago

40,627 次观看 • 1 年前

#WATCH | Delhi: On the #CasteCensus, author and Advaita teacher Acharya Prashant says, "Any action depends on the intention. Caste census means collecting more data, authentic data and data in itself are neither forward nor backwards, good nor bad. It depends on what you want to what you intend to do with the data. If you want to identify the marginalised groups in a better way, so that more accurate and more efficient policy making can happen, then it's a good step, a forward step... On the other hand, if you use this data to cement cost identification further, then obviously it's a step backwards. If you use this data for electoral gains by targeting particular caste groups concentrated in particular electorates or geographies, then obviously it's a backwards thing..."

#WATCH | Delhi: On the #CasteCensus, author and Advaita teacher Acharya Prashant says, "Any action depends on the intention. Caste census means collecting more data, authentic data and data in itself are neither forward nor backwards, good nor bad. It depends on what you want to what you intend to do with the data. If you want to identify the marginalised groups in a better way, so that more accurate and more efficient policy making can happen, then it's a good step, a forward step... On the other hand, if you use this data to cement cost identification further, then obviously it's a step backwards. If you use this data for electoral gains by targeting particular caste groups concentrated in particular electorates or geographies, then obviously it's a backwards thing..."

ANI

82,457 次观看 • 1 年前

Major program launch: Data Analytics Professional Certificate! This large, five-course sequence takes you all the way to being job-ready as a data analyst, and shows how to use Generative AI as a thought partner to enhance your work in this role. Offered by on Coursera, this is taught by Sean Barnes, Ph.D., a Data Science & Engineering Leader at Netflix. Analyzing data remains one of the most important skills in where the world is going with AI. This comprehensive certificate takes you all the way to being job-ready. Each course comes with practical projects demonstrated in real-world contexts, such as analyzing sales data for a Korean bakery, video game sales trends across different regions, or identifying factors impacting customer retention for a communications company. You'll also work on estimating fire distribution for forest fire prevention, analyzing how a diamond's properties affect its market value, and developing predictive models for retail sales analysis, carbon emissions, and coral reef conservation. Here's some of what you'll learn: - How to define data and categorize it into its many types such as discrete & continuous numerical, structured & unstructured, time series, categorical, and know what insights can be derived from the different types of data categories. - How to differentiate between data-related job roles and their responsibilities, and how data flows through an organization from the moment of capture to decision-making. - How to perform data processing functions and apply conditional formatting in spreadsheets to extract business value from your data using statistical calculations and best practices for visualizing and interpreting data. - How to use LLMs for stakeholder analysis, data exploration, and data visualization. - Best practices for using LLMs for as a thought partner to data analysis work By the end of this professional certificate program, you will have learned core statistical concepts, analysis techniques, and visualization methodologies that will serve as the foundation for working as a data analyst. The world needs more data analysts, especially ones who know how to use modern generative AI. With data science roles projected to grow 36% by 2033, the skills taught in this program create new professional opportunities in data. Sign up here!

Major program launch: Data Analytics Professional Certificate! This large, five-course sequence takes you all the way to being job-ready as a data analyst, and shows how to use Generative AI as a thought partner to enhance your work in this role. Offered by on Coursera, this is taught by Sean Barnes, Ph.D., a Data Science & Engineering Leader at Netflix. Analyzing data remains one of the most important skills in where the world is going with AI. This comprehensive certificate takes you all the way to being job-ready. Each course comes with practical projects demonstrated in real-world contexts, such as analyzing sales data for a Korean bakery, video game sales trends across different regions, or identifying factors impacting customer retention for a communications company. You'll also work on estimating fire distribution for forest fire prevention, analyzing how a diamond's properties affect its market value, and developing predictive models for retail sales analysis, carbon emissions, and coral reef conservation. Here's some of what you'll learn: - How to define data and categorize it into its many types such as discrete & continuous numerical, structured & unstructured, time series, categorical, and know what insights can be derived from the different types of data categories. - How to differentiate between data-related job roles and their responsibilities, and how data flows through an organization from the moment of capture to decision-making. - How to perform data processing functions and apply conditional formatting in spreadsheets to extract business value from your data using statistical calculations and best practices for visualizing and interpreting data. - How to use LLMs for stakeholder analysis, data exploration, and data visualization. - Best practices for using LLMs for as a thought partner to data analysis work By the end of this professional certificate program, you will have learned core statistical concepts, analysis techniques, and visualization methodologies that will serve as the foundation for working as a data analyst. The world needs more data analysts, especially ones who know how to use modern generative AI. With data science roles projected to grow 36% by 2033, the skills taught in this program create new professional opportunities in data. Sign up here!

Andrew Ng

85,107 次观看 • 1 年前

Millions of users scroll through endless transaction feeds trying to answer simple questions about their money. That’s why we built Conversational Transaction Insights directly into PhonePe. When building Conversational Transaction Insights for PhonePe, we wanted to eliminate the friction of digging through statements or tweaking filters. You should be able to ask a spending question in plain language via text or voice and get an accurate answer instantly. The core engineering challenge wasn't just wrapping an LLM around data - it was combining natural language flexibility with deterministic analytical precision and uncompromising user privacy. Whether tracking food expenses over the last 6 months or analyzing utility trends, financial clarity is now just a quick conversation / text away. Here’s a look at how it works!

Millions of users scroll through endless transaction feeds trying to answer simple questions about their money. That’s why we built Conversational Transaction Insights directly into PhonePe. When building Conversational Transaction Insights for PhonePe, we wanted to eliminate the friction of digging through statements or tweaking filters. You should be able to ask a spending question in plain language via text or voice and get an accurate answer instantly. The core engineering challenge wasn't just wrapping an LLM around data - it was combining natural language flexibility with deterministic analytical precision and uncompromising user privacy. Whether tracking food expenses over the last 6 months or analyzing utility trends, financial clarity is now just a quick conversation / text away. Here’s a look at how it works!

Rahul Chari

15,238 次观看 • 18 天前

This is, by far, one of the best uses of modern AI. If you don't use embeddings when querying your database, you are definitely leaving a lot on the table. In this video, I'll show you how to run semantic searches using OpenAI and PostgreSQL. It's all thanks to Pgai, an open-source PostgreSQL extension: Here's what will happen: 1. We'll create a simple table with news articles 2. We'll generate embeddings for those articles 3. We'll run queries on top of those embeddings For this video, I generated the embeddings using a simple query, but pgai Vectorizer would do the same automatically as new information makes it into the database. This is awesome! If you have a PostgreSQL database with data you are searching over, you should start experimenting with semantic searches immediately. For most use cases, a combination of full-text search + semantic search is the best approach. If you don't have a PostgreSQL database around, you can try free for 30 days using Timescale: Thanks to the TimescaleDB (by Tiger Data) team for partnering with me on this post!

This is, by far, one of the best uses of modern AI. If you don't use embeddings when querying your database, you are definitely leaving a lot on the table. In this video, I'll show you how to run semantic searches using OpenAI and PostgreSQL. It's all thanks to Pgai, an open-source PostgreSQL extension: Here's what will happen: 1. We'll create a simple table with news articles 2. We'll generate embeddings for those articles 3. We'll run queries on top of those embeddings For this video, I generated the embeddings using a simple query, but pgai Vectorizer would do the same automatically as new information makes it into the database. This is awesome! If you have a PostgreSQL database with data you are searching over, you should start experimenting with semantic searches immediately. For most use cases, a combination of full-text search + semantic search is the best approach. If you don't have a PostgreSQL database around, you can try free for 30 days using Timescale: Thanks to the TimescaleDB (by Tiger Data) team for partnering with me on this post!

Santiago

110,133 次观看 • 1 年前

New JavaScript short course: Build a full-stack web application that uses RAG in JavaScript RAG Web Apps with LlamaIndex, taught by Laurie Voss, VP of Developer Relations at LlamaIndex 🦙 and npm co-founder. - Build a RAG application for querying your own data - Develop tools to interact with multiple data sources using an agent that intelligently selects the right tool for your queries - Create a full-stack web app that can chat with your data - Dig further into production-ready techniques, like how to persist your data so you aren’t constantly reindexing, and try the create-llama command line tool from LlamaIndex You can sign up here:

New JavaScript short course: Build a full-stack web application that uses RAG in JavaScript RAG Web Apps with LlamaIndex, taught by Laurie Voss, VP of Developer Relations at LlamaIndex 🦙 and npm co-founder. - Build a RAG application for querying your own data - Develop tools to interact with multiple data sources using an agent that intelligently selects the right tool for your queries - Create a full-stack web app that can chat with your data - Dig further into production-ready techniques, like how to persist your data so you aren’t constantly reindexing, and try the create-llama command line tool from LlamaIndex You can sign up here:

Andrew Ng

218,703 次观看 • 2 年前

MCP is mind-blowing. Here is a genius setup that will earn you a promotion with your manager. I have multiple data sources: data in S3, MySQL, and local files. Using Spice (an open-source project), I can co-locate all of that data behind a single SQL endpoint and augment it using MCP. Before, accessing data was slow, and I had to make multiple requests to get anything. After using Spice, I can query the data much faster (since it's mirrored locally), and I don't have to worry about where the data is coming from. Spice acts as an MCP server, so I can connect to it from my IDE or the Claude Desktop app and have access to all the data. But even more cool than that: Spice also acts as an MCP client, so it can connect to other MCP servers. This is where things get really interesting! Watch the video. I'm querying multiple data sources from my IDE and using the Google Maps MCP server for driving directions.

MCP is mind-blowing. Here is a genius setup that will earn you a promotion with your manager. I have multiple data sources: data in S3, MySQL, and local files. Using Spice (an open-source project), I can co-locate all of that data behind a single SQL endpoint and augment it using MCP. Before, accessing data was slow, and I had to make multiple requests to get anything. After using Spice, I can query the data much faster (since it's mirrored locally), and I don't have to worry about where the data is coming from. Spice acts as an MCP server, so I can connect to it from my IDE or the Claude Desktop app and have access to all the data. But even more cool than that: Spice also acts as an MCP client, so it can connect to other MCP servers. This is where things get really interesting! Watch the video. I'm querying multiple data sources from my IDE and using the Google Maps MCP server for driving directions.

Santiago

142,696 次观看 • 1 年前

The Financial Revolution You Didn't Know You Needed: Account Aggregator You may have caught Amitabh Bachchan discussing Account Aggregator (AA) on Kaun Banega Crorepati recently. It wasn't just talk; it is a financial revolution in the making. The Problem: Meet Priya, an entrepreneur with a vision but no credit history. Traditional banks shut their doors on her, labeling her a 'risk.' The Solution: Enter Account Aggregator. With AA, Priya could consolidate her financial data, from bank accounts to tax returns, into one secure platform. She could then share this data with lenders on her terms. The Outcome: Priya secured a loan within days, not weeks. She took control of her financial destiny, all thanks to AA. Why Choose Agya for Your AA Journey: Security: Your data is encrypted and secure. Efficiency: One platform for all your financial data. Inclusivity: Whether you're a student, a homemaker, or an entrepreneur, Agya is for you. Amitabh Bachchan's Take: "Account Aggregator is more than a service; it's a paradigm shift in how we handle our finances." Your Next Step: Priya's story is not an outlier; it's a testament to what Agya can do for you. Take control of your financial data. Shape your financial future. Credit for the video: Kaun Banega Crorepati on Sony Pictures Entertainment Narendra Modi Nirmala Sitharaman Sahamati Nandan Nilekani ReserveBankOfIndia Ministry of Finance BG Mahesh Siddharth Shetty Pramod Varma Dilip Asbe 🇮🇳 Amrish Rau Sahil Kini Vinay Kesari irfn Vikram Bhat

The Financial Revolution You Didn't Know You Needed: Account Aggregator You may have caught Amitabh Bachchan discussing Account Aggregator (AA) on Kaun Banega Crorepati recently. It wasn't just talk; it is a financial revolution in the making. The Problem: Meet Priya, an entrepreneur with a vision but no credit history. Traditional banks shut their doors on her, labeling her a 'risk.' The Solution: Enter Account Aggregator. With AA, Priya could consolidate her financial data, from bank accounts to tax returns, into one secure platform. She could then share this data with lenders on her terms. The Outcome: Priya secured a loan within days, not weeks. She took control of her financial destiny, all thanks to AA. Why Choose Agya for Your AA Journey: Security: Your data is encrypted and secure. Efficiency: One platform for all your financial data. Inclusivity: Whether you're a student, a homemaker, or an entrepreneur, Agya is for you. Amitabh Bachchan's Take: "Account Aggregator is more than a service; it's a paradigm shift in how we handle our finances." Your Next Step: Priya's story is not an outlier; it's a testament to what Agya can do for you. Take control of your financial data. Shape your financial future. Credit for the video: Kaun Banega Crorepati on Sony Pictures Entertainment Narendra Modi Nirmala Sitharaman Sahamati Nandan Nilekani ReserveBankOfIndia Ministry of Finance BG Mahesh Siddharth Shetty Pramod Varma Dilip Asbe 🇮🇳 Amrish Rau Sahil Kini Vinay Kesari irfn Vikram Bhat

Nikhil Kumar

35,314 次观看 • 2 年前

10,000 valid trajectories collected — and we did it ahead of schedule, in just 5 days. We’ve reached our data milestone for "Little Prince's Rose." This community-driven dataset is now entering the pipeline for cleaning, augmentation, and model training, advancing the intelligence of Franka at this very moment. While the Discord role claiming is closing, the experiment will remain open as a welcoming portal for everyone to experience the Axis and how we tackle the data bottleneck for robotics. Robot intelligence is not built by a few; it's built by all.🌹

10,000 valid trajectories collected — and we did it ahead of schedule, in just 5 days. We’ve reached our data milestone for "Little Prince's Rose." This community-driven dataset is now entering the pipeline for cleaning, augmentation, and model training, advancing the intelligence of Franka at this very moment. While the Discord role claiming is closing, the experiment will remain open as a welcoming portal for everyone to experience the Axis and how we tackle the data bottleneck for robotics. Robot intelligence is not built by a few; it's built by all.🌹

Axis Robotics

23,424 次观看 • 7 个月前

🔎🤖LangSmith Insights Agent Really excited to launch our first in-product agent This agent lives inside LangSmith and combs through traces, giving you insights into: 🧑‍🤝‍🧑how users are using your agent ⁉️how your agent may be messing up 🛃{your custom insight here} The problem we saw was that people were launching agents... and didn't know how their users were actually using them! You put a chat box in front of people, and they may ask it anything - the surface area for agents is often super wide In addition - agents would fail silently. They could give a bad response - this wouldn't show up in error logs, but its good to know. If you know what look for, you can set up LLM as a judge evaluators. But what if you don't? (most people don't initially) The best way to figure this out - as Hamel Husain says - "look at your data". But LLMs are really good at looking at your data! So can they do it for you? This is exactly what insights agent attempts to do. It's live in LangSmith today. You can read more about it here:

🔎🤖LangSmith Insights Agent Really excited to launch our first in-product agent This agent lives inside LangSmith and combs through traces, giving you insights into: 🧑‍🤝‍🧑how users are using your agent ⁉️how your agent may be messing up 🛃{your custom insight here} The problem we saw was that people were launching agents... and didn't know how their users were actually using them! You put a chat box in front of people, and they may ask it anything - the surface area for agents is often super wide In addition - agents would fail silently. They could give a bad response - this wouldn't show up in error logs, but its good to know. If you know what look for, you can set up LLM as a judge evaluators. But what if you don't? (most people don't initially) The best way to figure this out - as Hamel Husain says - "look at your data". But LLMs are really good at looking at your data! So can they do it for you? This is exactly what insights agent attempts to do. It's live in LangSmith today. You can read more about it here:

Harrison Chase

98,520 次观看 • 10 个月前

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?

Building Data Pipelines has levels to it: - level 0 Understand the basic flow: Extract → Transform → Load (ETL) or ELT This is the foundation. - Extract: Pull data from sources (APIs, DBs, files) - Transform: Clean, filter, join, or enrich the data - Load: Store into a warehouse or lake for analysis You’re not a data engineer until you’ve scheduled a job to pull CSVs off an SFTP server at 3AM! level 1 Master the tools: - Airflow for orchestration - dbt for transformations - Spark or PySpark for big data - Snowflake, BigQuery, Redshift for warehouses - Kafka or Kinesis for streaming Understand when to batch vs stream. Most companies think they need real-time data. They usually don’t. level 2 Handle complexity with modular design: - DAGs should be atomic, idempotent, and parameterized - Use task dependencies and sensors wisely - Break transformations into layers (staging → clean → marts) - Design for failure recovery. If a step fails, how do you re-run it? From scratch or just that part? Learn how to backfill without breaking the world. level 3 Data quality and observability: - Add tests for nulls, duplicates, and business logic - Use tools like Great Expectations, Monte Carlo, or built-in dbt tests - Track lineage so you know what downstream will break if upstream changes Know the difference between: - a late-arriving dimension - a broken SCD2 - and a pipeline silently dropping rows At this level, you understand that reliability > cleverness. level 4 Build for scale and maintainability: - Version control your pipeline configs - Use feature flags to toggle behavior in prod - Push vs pull architecture - Decouple compute and storage (e.g. Iceberg and Delta Lake) - Data mesh, data contracts, streaming joins, and CDC are words you throw around because you know how and when to use them. What else belongs in the journey to mastering data pipelines?

Zach Wilson

16,688 次观看 • 1 年前

Your AI Agents can only analyze 1GB of data Enterprises have 1,000,000x as much data We fixed that... It took 3 years and building an entirely new execution environment to pull this off. TextQL lets AI agents analyze your ENTIRE enterprise data - not just the 0.0001% they can handle today. Here's what makes it so different: 1. ALL your data, not samples. AI today: 1GB demos. Your reality: 350TB across 47 systems. TextQL: Everything. At once. 2. Real enterprise questions.ChatGPT writes poems. We answer why renewal rates dropped 3% after ERP upgrades. 3. No more sandbox limitations.Everyone else: Jupyter notebook with 10GB RAM. Us: A data center. ------------------------------ In 2021, I watched Fortune 500s spend millions on AI that could only analyze data that fits on a USB stick. Today: - We process 10+ petabytes daily - Our biggest customer analyzes 100TB in real-time - Average query spans 50+ data sources And we just proved what everyone said was impossible - AI that can actually build a digital twin of your enterprise ------------------------------ To celebrate, we're giving away free data architecture assessments. We'll show you exactly how much of your data is invisible to AI right now (spoiler: it's probably 99.9%). Comment "TextQL" and we'll send you the assessment link.

Your AI Agents can only analyze 1GB of data Enterprises have 1,000,000x as much data We fixed that... It took 3 years and building an entirely new execution environment to pull this off. TextQL lets AI agents analyze your ENTIRE enterprise data - not just the 0.0001% they can handle today. Here's what makes it so different: 1. ALL your data, not samples. AI today: 1GB demos. Your reality: 350TB across 47 systems. TextQL: Everything. At once. 2. Real enterprise questions.ChatGPT writes poems. We answer why renewal rates dropped 3% after ERP upgrades. 3. No more sandbox limitations.Everyone else: Jupyter notebook with 10GB RAM. Us: A data center. ------------------------------ In 2021, I watched Fortune 500s spend millions on AI that could only analyze data that fits on a USB stick. Today: - We process 10+ petabytes daily - Our biggest customer analyzes 100TB in real-time - Average query spans 50+ data sources And we just proved what everyone said was impossible - AI that can actually build a digital twin of your enterprise ------------------------------ To celebrate, we're giving away free data architecture assessments. We'll show you exactly how much of your data is invisible to AI right now (spoiler: it's probably 99.9%). Comment "TextQL" and we'll send you the assessment link.

ethan ding 📊

162,101 次观看 • 1 年前

The $KLED Enterprise Portal Full Demo – Search and License Any Data in Seconds: For the last 6 weeks, we've had thousands of users sign up for the first data marketplace. We've paid out over $200,000 to them for uploading their camera rolls, homework, dashcam footage, etc. We've now amassed petabytes and petabytes of data from these users and large enterprises, and tagged it all with rich metadata. Now OpenAI, xAI, Google AI, or any other large lab can easily search and license that data using our newly completed enterprise portal. If you're a large lab, send me a DM and get access to the largest, never-before-seen consumer dataset on the planet. This is the biggest update for Kled since launch.

The $KLED Enterprise Portal Full Demo – Search and License Any Data in Seconds: For the last 6 weeks, we've had thousands of users sign up for the first data marketplace. We've paid out over $200,000 to them for uploading their camera rolls, homework, dashcam footage, etc. We've now amassed petabytes and petabytes of data from these users and large enterprises, and tagged it all with rich metadata. Now OpenAI, xAI, Google AI, or any other large lab can easily search and license that data using our newly completed enterprise portal. If you're a large lab, send me a DM and get access to the largest, never-before-seen consumer dataset on the planet. This is the biggest update for Kled since launch.

Avi Patel

91,697 次观看 • 1 年前