正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Most people think Rerun is a visualization tool. In reality, it's a database masquerading as a visualizer. I wanted to showcase this functionality by building a full data pipeline consisting of: ingestion → baseline method → eval → finetuning for SLAM on egocentric data. I'll eventually extend this to... the rest of my ego/exo datasets, but I wanted to start with a smaller bunch of datasets first. Rerun allows you to expose your saved .rrd files to a catalog where you store datasets. You can query, filter, and join them like any database using DataFusion under the hood. These are the same .rrd files that are automatically generated whenever you visualize anything in Rerun and decide to save it to disk. I brought in 109 VSLAM-LAB sequences across 14 datasets into the Rerun catalog as an example. These include 7Scenes, Euroc, eth3d, and others. Now I can query them with segment_table, filter_segments, and filter_contents instead of parsing CSVs and YAML files. With a strong set of ground-truth datasets for SLAM, baseline additions become nearly automatic with agents like Opus/Codex. This unification of data and visualization is imo the largest missing part for Physical AI. Visualization becomes a natural byproduct of having your data properly structured and queryable. The catalog API is what makes it a database, not just a viewer. I initially focused on VSLAM-LAB data, but I'll migrate all the egoexo data to this format in the coming days to really show just how useful this is.show more

Pablo Vela

3,226 subscribers

35,256 次观看 • 6 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

3 条评论

Adel Bucetta 的头像

Adel Bucetta6 个月前

@rerundotio most people miss the underlying structure, not just the visualization itself

Pablo Vela 的头像

Pablo Vela6 个月前

@rerundotio Agreed!

Pablo Vela 的头像

Pablo Vela6 个月前

Code here - <

相关视频

More progress! I now have two Dockerized Gradio | Rerun apps. The first one takes as input a "raw" rrd file that consists of the synchronized egocentric and exocentric MP4 files. This runs the pipeline and produces an "annotated" rrd file. This has the camera parameters, 3D joints, and projected 2D joints (with 6DOF mano soon). The second app takes this "annotated" rrd file and allows for manual labeling. This is a crucial step in addressing any major failures in the pipeline. Right now, it is only the ego view that can be modified. But I'll eventually extend to all. This results in a final "gt" rrd file. From here, the plan is to improve quality and start building a data loop. Excited to start really scaling this. I'm basically going all in on keeping my data stored as Rerun rrd files. As always, I want to emphasize how crucial it is to LOOK AT YOUR data! The rrd format makes it incredibly easy to do so. Getting the data out to use is a bit of a hassle right now, but for me, it's well worth the tradeoff.

More progress! I now have two Dockerized Gradio | Rerun apps. The first one takes as input a "raw" rrd file that consists of the synchronized egocentric and exocentric MP4 files. This runs the pipeline and produces an "annotated" rrd file. This has the camera parameters, 3D joints, and projected 2D joints (with 6DOF mano soon). The second app takes this "annotated" rrd file and allows for manual labeling. This is a crucial step in addressing any major failures in the pipeline. Right now, it is only the ego view that can be modified. But I'll eventually extend to all. This results in a final "gt" rrd file. From here, the plan is to improve quality and start building a data loop. Excited to start really scaling this. I'm basically going all in on keeping my data stored as Rerun rrd files. As always, I want to emphasize how crucial it is to LOOK AT YOUR data! The rrd format makes it incredibly easy to do so. Getting the data out to use is a bit of a hassle right now, but for me, it's well worth the tradeoff.

Pablo Vela

19,527 次观看 • 1 年前

I've been on a SLAM/SFM kick. It's one of the more underexplored and lacking areas when it comes to human teleop/data collections, so I've brought over Deep Patch Visual Odometry/SLAM to Rerun and Gradio. With this example, we now have 1. pycuvslam 2. pycolmap/glomap 3. mast3r-slam 4. dpvo/slam all integrated into rerun. The question becomes, which method should be used in what situations? They all make different trade-offs with different camera requirements and throughput/accuracy. What about when a new method comes out? Now that I have several different methods, I plan to use VSLAM-LAB for evaluation. It uses prefix.dev to isolate all the dependencies of each of these methods and easily compare them against each other. In particular, I'll be converting the data preprocessing, algorithm outputs, and evaluation into rerun recordings (rrd files). This will allow both programmatic querying of anything stored in the files (which method had the highest ATE-to-FPS ratio? Which dataset/sequence caused the most difficulty? etc. etc.), all with easy visual inspection using the rerun server to link them all together. Another really important side effect of this is how it impacts agents. As Karpathy said ``` LLMs are exceptionally good at looping until they meet specific goals, and this is where most of the "feel the AGI" magic is to be found. Don't tell it what to do, give it success criteria, and watch it go. ``` by having accuracy and throughput metrics deeply tied with human inspectable artifacts. One can really accelerate agentic development with an actual understanding of how the method/data performs. I think this is another killer use case that I'll be really leaning into to make ingestion of new datasets/methods trivial with an agent. I'm making it my mission for folks to understand that rerun as a visualization tool only scratches the surface of what its true benefit is. Deep integration between data and visuals, with powerful query capabilities. I'll be focusing on the SLAM use case first and then bringing this into the full egocentric/exocentric data collection domain!

I've been on a SLAM/SFM kick. It's one of the more underexplored and lacking areas when it comes to human teleop/data collections, so I've brought over Deep Patch Visual Odometry/SLAM to Rerun and Gradio. With this example, we now have 1. pycuvslam 2. pycolmap/glomap 3. mast3r-slam 4. dpvo/slam all integrated into rerun. The question becomes, which method should be used in what situations? They all make different trade-offs with different camera requirements and throughput/accuracy. What about when a new method comes out? Now that I have several different methods, I plan to use VSLAM-LAB for evaluation. It uses prefix.dev to isolate all the dependencies of each of these methods and easily compare them against each other. In particular, I'll be converting the data preprocessing, algorithm outputs, and evaluation into rerun recordings (rrd files). This will allow both programmatic querying of anything stored in the files (which method had the highest ATE-to-FPS ratio? Which dataset/sequence caused the most difficulty? etc. etc.), all with easy visual inspection using the rerun server to link them all together. Another really important side effect of this is how it impacts agents. As Karpathy said ``` LLMs are exceptionally good at looping until they meet specific goals, and this is where most of the "feel the AGI" magic is to be found. Don't tell it what to do, give it success criteria, and watch it go. ``` by having accuracy and throughput metrics deeply tied with human inspectable artifacts. One can really accelerate agentic development with an actual understanding of how the method/data performs. I think this is another killer use case that I'll be really leaning into to make ingestion of new datasets/methods trivial with an agent. I'm making it my mission for folks to understand that rerun as a visualization tool only scratches the surface of what its true benefit is. Deep integration between data and visuals, with powerful query capabilities. I'll be focusing on the SLAM use case first and then bringing this into the full egocentric/exocentric data collection domain!

Pablo Vela

41,361 次观看 • 6 个月前

0.32 has shipped, and it's a massive release from Rerun. There's a ton of cool new features, and I wanted to highlight 2 in particular 1. OSS Server streaming from disk 2. Dataset review I walk you through them in the video, so take a look. I'll have a much longer blog post next week about the entire pipeline. With 0.32, much of the foundation is set for a unified data layer for physical data, and I'll be getting into the details of it with all that I've built over the past year. This will cover 1. Raw Data Collection 2. Data Ingestion 3. Catalog Registration 4. Query and Review 5. Post Process 6. Training so lots to share

0.32 has shipped, and it's a massive release from Rerun. There's a ton of cool new features, and I wanted to highlight 2 in particular 1. OSS Server streaming from disk 2. Dataset review I walk you through them in the video, so take a look. I'll have a much longer blog post next week about the entire pipeline. With 0.32, much of the foundation is set for a unified data layer for physical data, and I'll be getting into the details of it with all that I've built over the past year. This will cover 1. Raw Data Collection 2. Data Ingestion 3. Catalog Registration 4. Query and Review 5. Post Process 6. Training so lots to share

Pablo Vela

11,264 次观看 • 5 个月前

We have HOT3D! I've started using Claude to port more datasets into Rerun and exoego-forge. I'd been meaning to bring in the HOT3D dataset from Meta for a while, but with Claude, it's way easier. My goal is to take any egocentric, exocentric, or both datasets and ingest them into a standardized schema. Getting everything into Rerun means we can easily query and transform data via the in-memory OSS server. This lets us generate SQL-like queries such as: "Find me all frames that only contain left hands in the leftmost camera view." Most people think of Rerun as a viewer, but this is the actual superpower. So far we have: 1. HOT3D 2. Hocap 3. UmeTrack 4. Assembly101 5. EgoDex Planning to add more, and with every addition, it gets easier as we build up agent skills and better code examples. Hoping to make it almost fully automatic for adding new datasets. The next few I'm looking at are Harmony4D and Aria Pilot Gen2 After we have enough samples, I'll work on bringing in all the different algorithms I've worked on to transform the data 🙂

We have HOT3D! I've started using Claude to port more datasets into Rerun and exoego-forge. I'd been meaning to bring in the HOT3D dataset from Meta for a while, but with Claude, it's way easier. My goal is to take any egocentric, exocentric, or both datasets and ingest them into a standardized schema. Getting everything into Rerun means we can easily query and transform data via the in-memory OSS server. This lets us generate SQL-like queries such as: "Find me all frames that only contain left hands in the leftmost camera view." Most people think of Rerun as a viewer, but this is the actual superpower. So far we have: 1. HOT3D 2. Hocap 3. UmeTrack 4. Assembly101 5. EgoDex Planning to add more, and with every addition, it gets easier as we build up agent skills and better code examples. Hoping to make it almost fully automatic for adding new datasets. The next few I'm looking at are Harmony4D and Aria Pilot Gen2 After we have enough samples, I'll work on bringing in all the different algorithms I've worked on to transform the data 🙂

Pablo Vela

36,545 次观看 • 7 个月前

As I'm working on finding more datasets to test my pipeline, I found a really cool one from Meta. This is Show3D and it does ego + exo data capture on a mobile capture rig, its focused on hand/object interaction in the willd. I've so far registered it to the Rerun OSS catalog, but I think I found a few issues. The tracking confidence is more unstable then I would expect and the object often has low tracking confidence on large swaths of of the segment. I'll spend more time on it and see if its a me error, or something with the data labels. Also sadly, they only provide the stereo images, no IMU =/

As I'm working on finding more datasets to test my pipeline, I found a really cool one from Meta. This is Show3D and it does ego + exo data capture on a mobile capture rig, its focused on hand/object interaction in the willd. I've so far registered it to the Rerun OSS catalog, but I think I found a few issues. The tracking confidence is more unstable then I would expect and the object often has low tracking confidence on large swaths of of the segment. I'll spend more time on it and see if its a me error, or something with the data labels. Also sadly, they only provide the stereo images, no IMU =/

Pablo Vela

15,541 次观看 • 1 个月前

This is, by far, one of the best uses of modern AI. If you don't use embeddings when querying your database, you are definitely leaving a lot on the table. In this video, I'll show you how to run semantic searches using OpenAI and PostgreSQL. It's all thanks to Pgai, an open-source PostgreSQL extension: Here's what will happen: 1. We'll create a simple table with news articles 2. We'll generate embeddings for those articles 3. We'll run queries on top of those embeddings For this video, I generated the embeddings using a simple query, but pgai Vectorizer would do the same automatically as new information makes it into the database. This is awesome! If you have a PostgreSQL database with data you are searching over, you should start experimenting with semantic searches immediately. For most use cases, a combination of full-text search + semantic search is the best approach. If you don't have a PostgreSQL database around, you can try free for 30 days using Timescale: Thanks to the TimescaleDB (by Tiger Data) team for partnering with me on this post!

This is, by far, one of the best uses of modern AI. If you don't use embeddings when querying your database, you are definitely leaving a lot on the table. In this video, I'll show you how to run semantic searches using OpenAI and PostgreSQL. It's all thanks to Pgai, an open-source PostgreSQL extension: Here's what will happen: 1. We'll create a simple table with news articles 2. We'll generate embeddings for those articles 3. We'll run queries on top of those embeddings For this video, I generated the embeddings using a simple query, but pgai Vectorizer would do the same automatically as new information makes it into the database. This is awesome! If you have a PostgreSQL database with data you are searching over, you should start experimenting with semantic searches immediately. For most use cases, a combination of full-text search + semantic search is the best approach. If you don't have a PostgreSQL database around, you can try free for 30 days using Timescale: Thanks to the TimescaleDB (by Tiger Data) team for partnering with me on this post!

Santiago

110,133 次观看 • 1 年前

Today we're launching Amplitude MCP MCP was made for data analytics. Ask a vague question and have an AI model iteratively query Amplitude to find you insights. Amplitude MCP exposes all of Amplitude's functionality so an agent can interact with it directly. It allows you to use Amplitude without having to know anything about Amplitude's user interface or your data taxonomy. It uses the intelligence of off the shelf AI agents, including Claude, Gemini, and Cursor, to query Amplitude data you already have to find answers. It leverages all of your existing Amplitude data and how it's used to navigate through large complex behavioral datasets. You can have conversations about your product data directly inside any AI model with MCP. It can create Amplitude dashboards and notebooks with what it finds and even make recommendations. This is the best MCP use case I've ever seen. Watching an AI agent think, reason, query Amplitude, and then repeat that process iteratively is magical. See it in action:

Today we're launching Amplitude MCP MCP was made for data analytics. Ask a vague question and have an AI model iteratively query Amplitude to find you insights. Amplitude MCP exposes all of Amplitude's functionality so an agent can interact with it directly. It allows you to use Amplitude without having to know anything about Amplitude's user interface or your data taxonomy. It uses the intelligence of off the shelf AI agents, including Claude, Gemini, and Cursor, to query Amplitude data you already have to find answers. It leverages all of your existing Amplitude data and how it's used to navigate through large complex behavioral datasets. You can have conversations about your product data directly inside any AI model with MCP. It can create Amplitude dashboards and notebooks with what it finds and even make recommendations. This is the best MCP use case I've ever seen. Watching an AI agent think, reason, query Amplitude, and then repeat that process iteratively is magical. See it in action:

Spenser Skates

32,477 次观看 • 1 年前

A couple of weeks ago the NTSB issued an update on the UPS MD11 crash and I wanted to use the graphs of flight data in a visualization combined with the ADS-B data and surveilance videos. This is what I ended up with, one hard part about this is the flight is so close to the ground the viewpoint needs to account for the pitch of the aircraft raising and lowering the cockpit.

A couple of weeks ago the NTSB issued an update on the UPS MD11 crash and I wanted to use the graphs of flight data in a visualization combined with the ADS-B data and surveilance videos. This is what I ended up with, one hard part about this is the flight is so close to the ground the viewpoint needs to account for the pitch of the aircraft raising and lowering the cockpit.

Scott Manley

207,872 次观看 • 9 个月前

Your agents can't keep up with real-time data. Especially when it's scattered across dozens of sources. Most teams waste weeks building custom connectors for every database, API, and data warehouse. Then they build ETL pipelines to sync everything. By the time your agent retrieves the data, it's already outdated. Picture this: Your Postgres database updated 5 minutes ago. Your MongoDB collection changed 2 minutes ago. Your agent is still pulling from yesterday's snapshot. This is why most production RAG systems fail. There's a better approach: MindsDB is an open-source AI platform with a federated data engine that lets you query multiple data sources in real-time using SQL - without moving any data. Here's what makes it different: ↳ Your data stays in place. No ETL pipelines or data duplication ↳ Query Postgres, MongoDB, REST APIs, and more using consistent SQL ↳ JOIN across different sources in real-time with a unified interface ↳ Works with both structured and un-structured data And here's the best part: You don't even need to write SQL. Just describe what you want in plain English, and MindsDB converts it to SQL automatically. The system does all the heavy lifting. The breakthrough for AI agents is simple: When data updates at the source, your agent gets fresh results immediately. No sync delays. No stale embeddings. No custom code for each integration. You can literally write a SQL query that joins a Postgres table with a MongoDB collection and gets live results. This is what production AI applications need but rarely get. In this video, I give you a complete walkthrough of what we just discussed and how to actually do it. Make sure you watch this till the end. I've shared the link to MindsDB's GitHub repo in the next tweet!

Your agents can't keep up with real-time data. Especially when it's scattered across dozens of sources. Most teams waste weeks building custom connectors for every database, API, and data warehouse. Then they build ETL pipelines to sync everything. By the time your agent retrieves the data, it's already outdated. Picture this: Your Postgres database updated 5 minutes ago. Your MongoDB collection changed 2 minutes ago. Your agent is still pulling from yesterday's snapshot. This is why most production RAG systems fail. There's a better approach: MindsDB is an open-source AI platform with a federated data engine that lets you query multiple data sources in real-time using SQL - without moving any data. Here's what makes it different: ↳ Your data stays in place. No ETL pipelines or data duplication ↳ Query Postgres, MongoDB, REST APIs, and more using consistent SQL ↳ JOIN across different sources in real-time with a unified interface ↳ Works with both structured and un-structured data And here's the best part: You don't even need to write SQL. Just describe what you want in plain English, and MindsDB converts it to SQL automatically. The system does all the heavy lifting. The breakthrough for AI agents is simple: When data updates at the source, your agent gets fresh results immediately. No sync delays. No stale embeddings. No custom code for each integration. You can literally write a SQL query that joins a Postgres table with a MongoDB collection and gets live results. This is what production AI applications need but rarely get. In this video, I give you a complete walkthrough of what we just discussed and how to actually do it. Make sure you watch this till the end. I've shared the link to MindsDB's GitHub repo in the next tweet!

Akshay 🚀

65,672 次观看 • 1 年前

$BTC Statistical Study using Claude - A Beginner's Workflow Here's an example of a z-score study on $BTC - still tinkering so don't take this as overtly useful information but the creation of a dashboard for the visualization of statistical data is phenomenal. my current workflow: > import $BTC time data - .csv file (can get this information from multiple venues - Binance is where I got mine) > creating bins - if you have a larger data set - you can use recent data to prevent overt bias in the long direction or filter consolidation and trending regime data into separate bins for statistical analysis - however, you will have to define thresholds and determine what that entails. > defining metrics in Claude that you want to use for statistical analysis e.g. for z-score what is it based on and what type of calculation? make sure you understand the calculations being performed for any metrics that you are doing a study for and modify them accordingly. > prompting Claude to do a statistical analysis with specific instructions and then tell it to create visualization for this. I've been messing around with this and I'm seriously impressed by the output.

$BTC Statistical Study using Claude - A Beginner's Workflow Here's an example of a z-score study on $BTC - still tinkering so don't take this as overtly useful information but the creation of a dashboard for the visualization of statistical data is phenomenal. my current workflow: > import $BTC time data - .csv file (can get this information from multiple venues - Binance is where I got mine) > creating bins - if you have a larger data set - you can use recent data to prevent overt bias in the long direction or filter consolidation and trending regime data into separate bins for statistical analysis - however, you will have to define thresholds and determine what that entails. > defining metrics in Claude that you want to use for statistical analysis e.g. for z-score what is it based on and what type of calculation? make sure you understand the calculations being performed for any metrics that you are doing a study for and modify them accordingly. > prompting Claude to do a statistical analysis with specific instructions and then tell it to create visualization for this. I've been messing around with this and I'm seriously impressed by the output.

Stoic

19,816 次观看 • 1 年前

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Traditional data pipelines don't work for RAG applications. There are 3 issues with them: 1. Traditional data engineering solutions are optimized to handle structured data. RAG applications rely primarily on unstructured data. 2. The connector ecosystem to load data from unstructured data sources is very immature. 3. Traditional solutions do not offer any way to transform unstructured data into an optimized vector search index. The goal of a RAG Pipeline is to solve these problems. The number one objective is to create a reliable vector search index using factual knowledge and relevant context. This sounds easy, but it's one of the biggest challenges we face when building RAG applications. At a high level, there are four different stages in the architecture of a RAG pipeline: 1. Ingestion: Here is where the pipeline loads the information from the data source. 2. Extraction: Where the pipeline processes the input data and decides how to retrieve the text contained inside them. 3. Transform: Where the pipeline chunks the data and generates document embeddings. 4. Load: Where the pipeline creates a search index in a vector database and loads the document embeddings. There are different rabbit holes at each one of these stages. Here are three of them: 1. Ingesting data once is simple. The hard part is refreshing the vector database whenever the original data source changes. 2. Extracting the content of a plain text document is simple. The hard part is to extract content from complex documents containing tables, images, or cross-references. 3. A simple continual chunking strategy with an overlap is simple. The hard part is to find the optimal strategy for your specific knowledge base and the way you are planning to query it. In the attached video, I'll show you how you can build an enterprise-grade RAG Pipeline that solves every one of the above problems. I'll use Vectorize. They partnered with me on this post. You can use them to build RAG pipelines optimized for accurate context retrieval. If you have a few documents lying around, set up a free account and give it a try.

Santiago

40,646 次观看 • 2 年前

As part of my exo ego pipeline, I haven't yet found a VIO/Slam solution that hits all of the features I want. Now that agents make it super easy to write low level systems code, I've almost finished building a portable rust/python slam pipeline that works on both a 5090 AND a Rockchip-class computer. To really make this work, I needed a good ground truth, and the wonderful work from Collabora and Mateo de Mayo on the monado slam dataset has been indispensable. I registered it as a Rerun dataset in the OSS catalog to make it very easy to verify that the GT actually looks right and use it as an eval, and it has worked REALLY well. I'll have more on the VIO pipeline here in the coming days

As part of my exo ego pipeline, I haven't yet found a VIO/Slam solution that hits all of the features I want. Now that agents make it super easy to write low level systems code, I've almost finished building a portable rust/python slam pipeline that works on both a 5090 AND a Rockchip-class computer. To really make this work, I needed a good ground truth, and the wonderful work from Collabora and Mateo de Mayo on the monado slam dataset has been indispensable. I registered it as a Rerun dataset in the OSS catalog to make it very easy to verify that the GT actually looks right and use it as an eval, and it has worked REALLY well. I'll have more on the VIO pipeline here in the coming days

Pablo Vela

20,588 次观看 • 1 个月前

🎁 A New Year’s gift to the geospatial community: a brand-new QGIS plugin that unlocks access to 80+ petabytes of satellite imagery and geospatial datasets with zero coding required. In my first video of 2026, I’m excited to share what might be one of the most powerful QGIS plugins I’ve built so far. This plugin makes it incredibly easy to explore the official Earth Engine Data Catalog and the Awesome GEE Community Catalog directly inside QGIS. With just a few clicks, you can browse massive Earth Engine datasets, visualize them on the map, generate time series from ImageCollections, export data for further analysis, and even convert Earth Engine JavaScript workflows to Python, all without writing a single line of code. If you’ve ever wanted the power of Earth Engine inside QGIS without the learning curve, this plugin is for you. 👉 Download the plugin and follow the full step-by-step video tutorial to get started today: QGIS Plugin Page: GitHub Repository: Video Tutorial: Happy New Year, and I hope this tool helps you do more with geospatial data in 2026 🚀 #QGIS #geospatial #EarthEngine #Python #datascience

Qiusheng Wu

85,313 次观看 • 10 个月前

Colmap 4.0 was very recently released, so it inspired me to do some work to better understand it and its new capabilities with Rerun. I want to really understand how Colmap, and in particular, pycolmap, works outside of just calling it via the CLI. So my goal is to use the low-level pycolmap API to log every part of the pipeline. The explicit goal is to have an alternative to the SQLite database that I can utilize. Instead of SQLite, I want to try logging everything directly to rerun and use RRD. This means I can have deep inspectability and still save the features/matches/2D view geometry, but be able to view it directly in rerun. I think this is one of the superpowers that rerun provides; data and visualizations are deeply integrated. As I'm often working with sequential data (videos), I'm going to specifically focus on four things: 1. Monocular Video Simple: Calls high-level APIs such as pycolmap.extract_features, pycolmap.match_sequential, pycolmap.incremental_mapping. These are basically identical to the CLI options and provide a good baseline. 2. Monocular Video Streamed: Take the above high-level APIs and break them down to their iterator version, logging each component in a streamed manner. This way, I can stream the intermediate features to rerun while the extraction/matching/mapping is happening. 3. Rig with unknown calibration: <- WHAT THE VIDEO SHOWS This is probably the most interesting version and the first one I've been working on. It allows one to set a rig between known sensors, such as in VR/AR devices, leading to much better reconstructions with multiple cameras. This is the case where we don't know the calibration a priori, so we have to run a reconstruction twice: once as a normal Colmap reconstruction with no rig constraints, use this to generate the constraints, and then do it again with the newly found rig. 4. Rig with known calibration: This is the RoboCap example, where we have a pre-calibrated set of sensors, so we don't need to run the two reconstructions and also gain better matching between cameras, both spatially and temporally. Again, this leads to a much better reconstruction! Along with all this, GLOMAP has become a first-class global mapper, making it super easy to use directly within pycolmap! I'm excited to do more with this and compare it to things like pycuvslam, vipe, and other alternatives.

Colmap 4.0 was very recently released, so it inspired me to do some work to better understand it and its new capabilities with Rerun. I want to really understand how Colmap, and in particular, pycolmap, works outside of just calling it via the CLI. So my goal is to use the low-level pycolmap API to log every part of the pipeline. The explicit goal is to have an alternative to the SQLite database that I can utilize. Instead of SQLite, I want to try logging everything directly to rerun and use RRD. This means I can have deep inspectability and still save the features/matches/2D view geometry, but be able to view it directly in rerun. I think this is one of the superpowers that rerun provides; data and visualizations are deeply integrated. As I'm often working with sequential data (videos), I'm going to specifically focus on four things: 1. Monocular Video Simple: Calls high-level APIs such as pycolmap.extract_features, pycolmap.match_sequential, pycolmap.incremental_mapping. These are basically identical to the CLI options and provide a good baseline. 2. Monocular Video Streamed: Take the above high-level APIs and break them down to their iterator version, logging each component in a streamed manner. This way, I can stream the intermediate features to rerun while the extraction/matching/mapping is happening. 3. Rig with unknown calibration: <- WHAT THE VIDEO SHOWS This is probably the most interesting version and the first one I've been working on. It allows one to set a rig between known sensors, such as in VR/AR devices, leading to much better reconstructions with multiple cameras. This is the case where we don't know the calibration a priori, so we have to run a reconstruction twice: once as a normal Colmap reconstruction with no rig constraints, use this to generate the constraints, and then do it again with the newly found rig. 4. Rig with known calibration: This is the RoboCap example, where we have a pre-calibrated set of sensors, so we don't need to run the two reconstructions and also gain better matching between cameras, both spatially and temporally. Again, this leads to a much better reconstruction! Along with all this, GLOMAP has become a first-class global mapper, making it super easy to use directly within pycolmap! I'm excited to do more with this and compare it to things like pycuvslam, vipe, and other alternatives.

Pablo Vela

30,070 次观看 • 7 个月前

$Google open-sourced MCP Toolbox for Databases. I gave it access to everything else. For context, Google's MCP Toolbox for Databases is an open-source server that lets AI agents securely query structured databases like PostgreSQL and MySQL through the MCP protocol However, most enterprise knowledge doesn't actually live in databases. It's scattered across emails, Slack threads, GitHub repos, Salesforce records, customer reviews, and internal docs. So Agents can't see any of it, which means they're working with a fraction of the context they need. I fixed that using MindsDB. It acts as a universal SQL layer that sits on top of all your data sources: structured, semi-structured, and unstructured. This means you can query Salesforce, Gmail, GitHub, S3 files, Jira, and 200+ more sources using SQL syntax. The clever part is how it connects to the MCP Toolbox. MindsDB exposes everything through MySQL, so from the Agent's perspective, it's just running SQL and getting context back. It doesn't know or care that the data came from five different sources behind the scenes. This setup unlocks some powerful capabilities: → One SQL interface for dozens of enterprise sources → Cross-datasource joins (combine GitHub and CRM data in a single query) → Built-in ML capabilities for working with unstructured data → Simple MCP tools that now have massively expanded reach In the video below, the Agent queries GitHub data and a customer review database in one SQL query. So what used to require ETL pipelines and weeks of engineering effort now happens instantly. At the end of the day, AI agents are only as useful as the data they can access. This gives them a lot more to work with. I have shared the GitHub repo in the replies, where you can find more details about this.$

Google open-sourced MCP Toolbox for Databases. I gave it access to everything else. For context, Google's MCP Toolbox for Databases is an open-source server that lets AI agents securely query structured databases like PostgreSQL and MySQL through the MCP protocol However, most enterprise knowledge doesn't actually live in databases. It's scattered across emails, Slack threads, GitHub repos, Salesforce records, customer reviews, and internal docs. So Agents can't see any of it, which means they're working with a fraction of the context they need. I fixed that using MindsDB. It acts as a universal SQL layer that sits on top of all your data sources: structured, semi-structured, and unstructured. This means you can query Salesforce, Gmail, GitHub, S3 files, Jira, and 200+ more sources using SQL syntax. The clever part is how it connects to the MCP Toolbox. MindsDB exposes everything through MySQL, so from the Agent's perspective, it's just running SQL and getting context back. It doesn't know or care that the data came from five different sources behind the scenes. This setup unlocks some powerful capabilities: → One SQL interface for dozens of enterprise sources → Cross-datasource joins (combine GitHub and CRM data in a single query) → Built-in ML capabilities for working with unstructured data → Simple MCP tools that now have massively expanded reach In the video below, the Agent queries GitHub data and a customer review database in one SQL query. So what used to require ETL pipelines and weeks of engineering effort now happens instantly. At the end of the day, AI agents are only as useful as the data they can access. This gives them a lot more to work with. I have shared the GitHub repo in the replies, where you can find more details about this.

Akshay 🚀

40,562 次观看 • 8 个月前

The era of manually analyzing data will come to an end. AI can now do a lot of this automatically. It's a huge time saver. I do data analysis for a living, and I'm a huge fan of writing Jupyter notebooks to do it all, but it's now hard to justify manually writing code that you can generate in a few seconds. I still check everything manually, but breaking down datasets into tables and charts is now 10x easier than it's ever been. Here is a video where I'm using Retool. I load a dataset and generate a few charts as quickly as I can think. The speed at which we can go from one idea to a working solution is astonishing.

The era of manually analyzing data will come to an end. AI can now do a lot of this automatically. It's a huge time saver. I do data analysis for a living, and I'm a huge fan of writing Jupyter notebooks to do it all, but it's now hard to justify manually writing code that you can generate in a few seconds. I still check everything manually, but breaking down datasets into tables and charts is now 10x easier than it's ever been. Here is a video where I'm using Retool. I load a dataset and generate a few charts as quickly as I can think. The speed at which we can go from one idea to a working solution is astonishing.

Santiago

70,812 次观看 • 1 年前

I got a smart meter recently and saw you can download a HDF file with the data so I had the idea of writing a script that could parse that and show the data in a useful manner. However I discovered that someone has already done this and done a really good job on it. The video explains how it works. In simple terms it uses the ESB smart meter data and shows a breakdown of how much data you are using and when and also recommends plans and estimates what each one would cost based on the data. The tool is available at easier to read on Laptop or Tablet or then your phone to Landscape.

I got a smart meter recently and saw you can download a HDF file with the data so I had the idea of writing a script that could parse that and show the data in a useful manner. However I discovered that someone has already done this and done a really good job on it. The video explains how it works. In simple terms it uses the ESB smart meter data and shows a breakdown of how much data you are using and when and also recommends plans and estimates what each one would cost based on the data. The tool is available at easier to read on Laptop or Tablet or then your phone to Landscape.

Carlow Weather

298,565 次观看 • 3 年前

Working on adding a new dataset to the lineup. Ported ego-dex over to Rerun With rerun now stabilizing RRD format between versions (0.23 -> 0.24), this is the perfect time to start encoding all of the datasets I've been using to RRD 1. I'm starting with ego-dex and then adding others, such as HOCAP/Assembly 101 2. Looking to see if it also makes sense to port to webdatasets RRD 3. I've started including visualizing confidence — green (high), yellow (medium), red (low). More info on Friday

Working on adding a new dataset to the lineup. Ported ego-dex over to Rerun With rerun now stabilizing RRD format between versions (0.23 -> 0.24), this is the perfect time to start encoding all of the datasets I've been using to RRD 1. I'm starting with ego-dex and then adding others, such as HOCAP/Assembly 101 2. Looking to see if it also makes sense to port to webdatasets RRD 3. I've started including visualizing confidence — green (high), yellow (medium), red (low). More info on Friday

Pablo Vela

34,317 次观看 • 1 年前