正在加载视频...

视频加载失败

More progress! I now have two Dockerized Gradio | Rerun apps. The first one takes as input a "raw" rrd file that consists of the synchronized egocentric and exocentric MP4 files. This runs the pipeline and produces an "annotated" rrd file. This has the camera parameters, 3D joints, and...

19,527 次观看 • 8 个月前 •via X (Twitter)

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Most people think Rerun is a visualization tool. In reality, it's a database masquerading as a visualizer. I wanted to showcase this functionality by building a full data pipeline consisting of: ingestion → baseline method → eval → finetuning for SLAM on egocentric data. I'll eventually extend this to the rest of my ego/exo datasets, but I wanted to start with a smaller bunch of datasets first. Rerun allows you to expose your saved .rrd files to a catalog where you store datasets. You can query, filter, and join them like any database using DataFusion under the hood. These are the same .rrd files that are automatically generated whenever you visualize anything in Rerun and decide to save it to disk. I brought in 109 VSLAM-LAB sequences across 14 datasets into the Rerun catalog as an example. These include 7Scenes, Euroc, eth3d, and others. Now I can query them with segment_table, filter_segments, and filter_contents instead of parsing CSVs and YAML files. With a strong set of ground-truth datasets for SLAM, baseline additions become nearly automatic with agents like Opus/Codex. This unification of data and visualization is imo the largest missing part for Physical AI. Visualization becomes a natural byproduct of having your data properly structured and queryable. The catalog API is what makes it a database, not just a viewer. I initially focused on VSLAM-LAB data, but I'll migrate all the egoexo data to this format in the coming days to really show just how useful this is.

Pablo Vela

34,840 次观看 • 1 个月前

I've been on a SLAM/SFM kick. It's one of the more underexplored and lacking areas when it comes to human teleop/data collections, so I've brought over Deep Patch Visual Odometry/SLAM to Rerun and Gradio. With this example, we now have 1. pycuvslam 2. pycolmap/glomap 3. mast3r-slam 4. dpvo/slam all integrated into rerun. The question becomes, which method should be used in what situations? They all make different trade-offs with different camera requirements and throughput/accuracy. What about when a new method comes out? Now that I have several different methods, I plan to use VSLAM-LAB for evaluation. It uses prefix.dev to isolate all the dependencies of each of these methods and easily compare them against each other. In particular, I'll be converting the data preprocessing, algorithm outputs, and evaluation into rerun recordings (rrd files). This will allow both programmatic querying of anything stored in the files (which method had the highest ATE-to-FPS ratio? Which dataset/sequence caused the most difficulty? etc. etc.), all with easy visual inspection using the rerun server to link them all together. Another really important side effect of this is how it impacts agents. As Karpathy said ``` LLMs are exceptionally good at looping until they meet specific goals, and this is where most of the "feel the AGI" magic is to be found. Don't tell it what to do, give it success criteria, and watch it go. ``` by having accuracy and throughput metrics deeply tied with human inspectable artifacts. One can really accelerate agentic development with an actual understanding of how the method/data performs. I think this is another killer use case that I'll be really leaning into to make ingestion of new datasets/methods trivial with an agent. I'm making it my mission for folks to understand that rerun as a visualization tool only scratches the surface of what its true benefit is. Deep integration between data and visuals, with powerful query capabilities. I'll be focusing on the SLAM use case first and then bringing this into the full egocentric/exocentric data collection domain!

Pablo Vela

40,744 次观看 • 2 个月前

Colmap 4.0 was very recently released, so it inspired me to do some work to better understand it and its new capabilities with Rerun. I want to really understand how Colmap, and in particular, pycolmap, works outside of just calling it via the CLI. So my goal is to use the low-level pycolmap API to log every part of the pipeline. The explicit goal is to have an alternative to the SQLite database that I can utilize. Instead of SQLite, I want to try logging everything directly to rerun and use RRD. This means I can have deep inspectability and still save the features/matches/2D view geometry, but be able to view it directly in rerun. I think this is one of the superpowers that rerun provides; data and visualizations are deeply integrated. As I'm often working with sequential data (videos), I'm going to specifically focus on four things: 1. Monocular Video Simple: Calls high-level APIs such as pycolmap.extract_features, pycolmap.match_sequential, pycolmap.incremental_mapping. These are basically identical to the CLI options and provide a good baseline. 2. Monocular Video Streamed: Take the above high-level APIs and break them down to their iterator version, logging each component in a streamed manner. This way, I can stream the intermediate features to rerun while the extraction/matching/mapping is happening. 3. Rig with unknown calibration: <- WHAT THE VIDEO SHOWS This is probably the most interesting version and the first one I've been working on. It allows one to set a rig between known sensors, such as in VR/AR devices, leading to much better reconstructions with multiple cameras. This is the case where we don't know the calibration a priori, so we have to run a reconstruction twice: once as a normal Colmap reconstruction with no rig constraints, use this to generate the constraints, and then do it again with the newly found rig. 4. Rig with known calibration: This is the RoboCap example, where we have a pre-calibrated set of sensors, so we don't need to run the two reconstructions and also gain better matching between cameras, both spatially and temporally. Again, this leads to a much better reconstruction! Along with all this, GLOMAP has become a first-class global mapper, making it super easy to use directly within pycolmap! I'm excited to do more with this and compare it to things like pycuvslam, vipe, and other alternatives.

Pablo Vela

30,070 次观看 • 2 个月前

This Is Absolutely Mind Blowing ⚠️ Barack Obama’s New Movie, Leave The World Behind, When Downloaded & Put Into An Editing Program Has 4 Audio Files Instead Of The Standard 1 One Of The Audio Files Is Just A Frequency 🚨 “Low Frequency Infrasound” Other Movies Do Not Have This. The Guy Has Done Thousands Of Movies & Never Seen This “When you open this this movie right here is Jurassic Park, for instance. Okay? I'm just showing this to people who have never seen this. For the last 15 years that I have been using Sony Vegas and other editing software and have gone through downloading movies And using them in our own video production, 100% of the time, it's either an MKV or a MP For occasionally, you'll download a torrent that's in AVR, but not very often. And this is what it will always look like. One video file right here. One audio file right here always. That's been my experience. Now let me show you what happened when we downloaded and put, Sony put the, uh, movie Leave the World Behind onto Sony Vegas. Here you go. Stunning. Stunning. One video file, 4 audio files opened up. And this one, This 4th audio file seems to be the weapon. This is a low ex uh, ELF, Uh, Low Frequency Infrasound. Infrasonic wave right here. And, um, so we We are gonna let me put the camera back on again. So this is big news just just so people realize. In in 15 years of video editing, I have never downloaded now speaking specifically a movie file. Okay. A Hollywood movie from a torrent downloading it in either MKV, MP4, occasionally AVI. 100% of the time, over a thousands movies I've done this with, you have video file, audio file, period. This is the very first time I've ever seen anything like this with the 4 files. Bonnie's gonna go into now talking about the The infrasonic sound waves and what we found out about that and the fact that there's one here. We're in the point is we're going to go back after Bonnie gives you some information about what it is we're dealing with, how this is a weapon. Then we're gonna go back, uh, towards the end of the live stream here. And I'm gonna hit specific places where this weapon was used against anybody who watches it. Okay.”

Wall Street Apes

3,769,004 次观看 • 2 年前

The Digital ID Will Change the Lives of Every Australian - For the Worse ... As much as the Government attempts to downplay the importance of introducing a single central digital identifier for all Australians, the truth is that this legislation is the most significant I've encountered during my time in the Senate. It's the glue that holds together the digital control agenda by which every Australian will be controlled, corralled, exploited and then gagged when they speak or act in opposition. The government knows Digital ID will be compulsory by the device of preventing access to government services, banking services, air travel and major purchases for any Australian who does not have a Digital ID. The Digital ID will, in effect, create a live data file of your movements, purchases, accounts and associates containing reference to every piece of data being held in the private and government sector as a first step in a wider agenda. Tech giants have been building huge data files on every Australian for years. Those huge data files that contain every website you visited, every post you made on their social media, everything you have ever bought online. Keywords scanned from conversations overheard by Siri and Alexa in your home are now unmasked. Until now, that data was anonymised using a unique identifier rather than name and address, which has always been there as well. However, tech companies were not allowed to use it or share data with others that included the person's name and address. Until Now. Look for the tech giants to ask for your Digital ID as a requirement of using their service. The point of that exercise is to ensure they put the right name on the right data treasure trove. This is why the Liberal Party have moved amendments to the Digital ID Bill to bring private corporations into this roll out earlier. All those treasure troves of data worth billions, trillions, that have been accumulated for years illegally, by retailers, tech and data companies - all that unrealised profit just sitting there has been too much of a temptation for the Liberal Nationals to resist and is now joined with Labor in pushing Digital ID. There will be no escape from the digital ID. Australians now have a digital version of "papers please" and Australians will never be the same.

Malcolm Roberts 🇦🇺

217,182 次观看 • 2 年前