Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

A preview of what's next, visualized with Rerun and PlayCanvas supersplat ✨ (Also, feel free to send me a DM 📩; I’ll be in San Francisco from July 21–29, and I'm looking to meet like-minded folks!) I'm convinced that Gaussian Splats will be an integral part of any data...

26,036 görüntüleme • 1 yıl önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Colmap 4.0 was very recently released, so it inspired me to do some work to better understand it and its new capabilities with Rerun. I want to really understand how Colmap, and in particular, pycolmap, works outside of just calling it via the CLI. So my goal is to use the low-level pycolmap API to log every part of the pipeline. The explicit goal is to have an alternative to the SQLite database that I can utilize. Instead of SQLite, I want to try logging everything directly to rerun and use RRD. This means I can have deep inspectability and still save the features/matches/2D view geometry, but be able to view it directly in rerun. I think this is one of the superpowers that rerun provides; data and visualizations are deeply integrated. As I'm often working with sequential data (videos), I'm going to specifically focus on four things: 1. Monocular Video Simple: Calls high-level APIs such as pycolmap.extract_features, pycolmap.match_sequential, pycolmap.incremental_mapping. These are basically identical to the CLI options and provide a good baseline. 2. Monocular Video Streamed: Take the above high-level APIs and break them down to their iterator version, logging each component in a streamed manner. This way, I can stream the intermediate features to rerun while the extraction/matching/mapping is happening. 3. Rig with unknown calibration: <- WHAT THE VIDEO SHOWS This is probably the most interesting version and the first one I've been working on. It allows one to set a rig between known sensors, such as in VR/AR devices, leading to much better reconstructions with multiple cameras. This is the case where we don't know the calibration a priori, so we have to run a reconstruction twice: once as a normal Colmap reconstruction with no rig constraints, use this to generate the constraints, and then do it again with the newly found rig. 4. Rig with known calibration: This is the RoboCap example, where we have a pre-calibrated set of sensors, so we don't need to run the two reconstructions and also gain better matching between cameras, both spatially and temporally. Again, this leads to a much better reconstruction! Along with all this, GLOMAP has become a first-class global mapper, making it super easy to use directly within pycolmap! I'm excited to do more with this and compare it to things like pycuvslam, vipe, and other alternatives.

Pablo Vela

30,070 görüntüleme • 4 ay önce

Everyone's sleeping on image-to-3D AI models. They can make your app look incredibly unique, with just a little effort. Here's how. This is my calorie tracker, built in a week with nothing but prompting. Just Claude Code + a couple APIs. The visuals are all AI-generated. I'll be sharing the full workflow + all the crazy technical stuff Claude and I did to make this work, so nobody has to struggle through it like me. Deep dive coming soon! Till then, this is the high-level idea: 1. Get a clean image of the food (or whatever your asset is) - In my app, the user describes foods via text, or attaches images (or both) - If text, an LLM extracts the food description and formats it into a specific prompt I tuned for this design, and we generate an image using Z-Image Turbo through fal - If image, we do the same thing but with FLUX.2 [dev] to edit the user image into our reference design - Originally, both used Google Nano Banana, but switching to open models cut costs and latency a ton 2. Gaussian splatting (2D image → 3D model) - I tried various 2D-to-3D options on fal and ended up with TripoSplat as my preferred balance of speed, cost, latency; this turns an image into a 3D model that looks super high quality (link below) - The app displays the 2D image while our backend generates the 3D splat - We "groom" the splat to reduce size and load time by culling low-opacity/scale points 3. Render efficiently on device Originally, it looked great but ran at 10 FPS. Getting to 120 FPS was a crazy journey. TL;DR: - SwiftUI had to go; it forced us to render each asset in independent MTKViews, which wasn't workable - Instead, we composite every dish into one full-bleed CAMetalLayer using MetalSplatter (link below) - We had to make some optimizations within MetalSplatter's code too, to reduce the overhead of sorting points per render Then I added some finishing touches like the subtle rotation and parallax as they move around. I think it turned out pretty cool :) Overall, this took some effort, but we still got it done in less than a day. Hopefully your agent can follow in the footsteps of mine and do it much faster. Keep an eye out for the bigger writeup, which'll give your agent everything it needs. If you have any questions, drop em below!

Anshu

19,931 görüntüleme • 1 ay önce

OK, I have a definitive word on the CJ Abrams play from today's Pittsburgh Pirates at Washington Nationals game after talking with Elias Sports Bureau on this. This play will stay as a sacrifice fly. The originial ruling of NOT a sacrifice fly was for the exact same reason that I thought, which is that the infielder is not running into the outfield, which is year's past would have been correct As it was explained to me, in past years, an infielder had to be running almost in a straight line towards the outfield wall to be considered "running in the outfield". Here, since he is running, and he ends up further away from home (157 feet) than when he started (145 feet), this is going to count as a sacrifice fly. That definition is changing, in part from this play to help bring greater consistency, and to take some of the guesswork out of it (the argument that he is running into the outfield as opposed to more parallel). Now, folks all the time ask "why doesn't MLB publish the OS Manual" and I always say because it is a living document that can have the wording change, and the wording for this play will be modified to something like "more towards the outfield wall than towards home plate" to eliminate any confusion. The big key to this play is that he was running on a full sprint. Also, and this is helpful for me, but for all fly ball outs that score a run, Elias Saba reviews to ensure consistency. So, yes, it's a sacrifice fly, and now that I have that info from Elias themselves, that sort of settles this one. Sounds like the guidelines for this definition will be changing, either this season, or certainly for next season.

MLB Scoring Changes

49,141 görüntüleme • 1 ay önce

Dagknight technical progress As would be mentioned in a still unshared post by Michael Sutton, the dagknight effort is split into v0 devnet, v1 testnet and v2 mainnet candidate. I’ve been testing the current v0-based implementation in a small devnet with the help of some testers who run nodes and miners with me. The DK work can be thought of as split into two parts: (1) implementing the actual protocol and (2) wiring it up and using it. The testing and development over the last month has been focused on (2). Obviously, DK is a consensus change for selecting parents. What’s not so obvious is that such a change affects DAA, coinbase, IBD, pruning and a lot more. Each of these areas is very sensitive and requires proper understanding to wire correctly. An important consideration and difference from GD is that DK does not focus on maximizing a property like blue work. So to maintain topological properties of blue work, an independent (free) GD implementation is kept running specifically for maintaining blue work. This allows us to keep using the property for topology. Coloring and blue score use the megachain induced by DK. The wiring around DK as of this posting is in a working state, but still needs to be reviewed. Next efforts will be focused on protocol specific components, particularly Tie-Breaking and incremental UMC. Attached are some captures from the internal devnet. The dense DAG image is what happens when things related to DAA or other similar consensus parameter causes a node to insist on their POV. The video is a recent snippet of the KGI running on the devnet showing (perhaps not obviously) DK at work. The current “dagknight” branch is now posted on the main repo. A topic in the Public R&D has been opened for Dagknight development.

coderofstuff

52,857 görüntüleme • 5 ay önce