Video wird geladen...
Video konnte nicht geladen werden
What if any dataset, any size, any number of dimensions, opened in a browser tab from a link? 🔬🧪💻 Luxar is out today: write it in Python, share it as a link, explore it in any browser. Open source. 🧵 biohub
17,834 Aufrufe • vor 8 Tagen •via X (Twitter)
48 Kommentare

🪰 500 timepoints of Drosophila gastrulation playing in a browser tab from static files, orbited live. SiMView recording with Philipp Keller at @HHMIJanelia. #lightsheet Preprint: Code: Demo:

How it works. Describe the scene in Python: points, lines, meshes and Gaussian splats, any number of dimensions. Compile once into a chunked, indexed .luxar.zarr archive, level-of-detail ladder optional. Put it on any static host. The browser does the rest. #dataviz

🐟 The microscopy piece: Gaussian splatting, volumes as Gaussian mixtures, ~100x smaller. 17 volumes, 12 datasets, 4 modalities: median 99x (6 to 340x) at 26 to 67 dB PSNR, minutes per volume on one GPU. This frame: 2.2 M splats. #lightsheet Demo:

The human nuclear pore complex, 4,937,064 atoms in 808 chains, approached head-on and flown through its central channel. Demo:

Luxar is made for 3D UMAPs. 500K Tabula Sapiens cells from 24 tissues as points coloured by organ, with a legend in the scene; hovering a cell names its cell type and tissue. Millions of points stay interactive. #SingleCell Demo:

Quick start: pip install luxar luxar demo run cloud A cumulus cloud billows, matures and sinks back, a 3D+time scene you can play. luxar demo lists 89 more. Any public .luxar.zarr archive opens at

🌌 Beyond biology. All 9.75 million galaxies and quasars of DESI DR1 (DESI Collaboration), placed in 3D by redshift, streamed through a level-of-detail ladder. #dataviz Demo:

575,503 Swiss-Prot proteins, ESM C embeddings (@EvoscaleAI, now part of @biohub) in a 3D UMAP coloured by taxon, flown through in fly mode. Demo:

Four geometry types, one scene graph, any number of dimensions. C. elegans: nuclei as Gaussian splats, lineage as polylines, current positions as points, played through time with the tracks clipped to the slice. Demo:

Annotation is native. A map of arXiv, bioRxiv and medRxiv papers: hover a point and GPU picking names the paper; right-click copies the title or opens the link. #dataviz Demo:

Hover can show images too. The CytoSelf protein-localisation map from OpenCell: hover a protein and its source micrograph appears. Demo:

🌍 The ocean currents of Earth. Demo:

A cluster fly, macro-photogrammetry Gaussian splats by Dany Bittel (CC BY 4.0), imported from a PLY file. Luxar reads INRIA PLY, .splat, .spz, SuperSplat and PlayCanvas SOG (@playcanvas) and writes PLY back. #GaussianSplatting Demo:

Scale is not the constraint. A hundred embryos on a line, each an adaptive level-of-detail group; colour is the level drawn. Detail never exceeds what the screen can show, so the line could be far longer at almost no extra cost. Demo:

Level of detail is a grammar, not a preset. Six topologies from one Tribolium fit, one flag each: flat, stream, levels, tiles, overview, adaptive. Here the levels column swaps from coarse (red) to finest (teal) as it fills the screen. Demo:

The archive is a Zarr hierarchy indexed in space and along every other dimension; the viewer fetches only the chunks, levels and timepoints the view needs. Cold load at 30 Mbit/s: coarse rung first, then detail, then time steps. Demo:

Points. The atoms of ATP synthase (PDB 5DN6), sharpness swept from soft Gaussian sprites to hard spheres along a hidden dimension of the scene. 60 FPS through a million elements. Demo:

Lines. A single cell's genome (Dip-C) as haplotype-resolved polylines drawn as capsules: joints that neither brighten nor open at corners, at any zoom. Demo:

Meshes. Cells3D isosurfaces as shaded surfaces under a view-anchored headlight, in the same scene graph as the points, lines and splats. Demo:

Gaussian splats. Additive or volumetric blending, switched live on a Drosophila embryo. Volumetric gives absorption and depth; additive gives the glow. Demo:

🧠 The HCP-1065 white-matter atlas (Yeh 2022), 87 tracts as polylines, each tract its own colour, orbited. Demo:

A clinical CT coloured by 117 TotalSegmentator labels. Hovering names the structure; skeleton, organs, vessels, nerves and muscles toggle as layers. Demo:

Scenes can carry guided tours: camera flights, overlays and narration, authored in Python. Sound on. This is the hemoglobin story on the protein landscape. Demo:

The same viewer on an iPad: one finger rotates, two pan, a pinch zooms, a twist rolls, a double tap recentres. Demo:

Ship it your way: a link, an offline folder, a native app, or the viewer embedded in any web page from npm. luxar export does all of it, and the recording panel writes a movie.

It plays with the tools you have: luxar gsplat napari opens a fit in @napari_imaging, any NumPy array is one call from a scene, and every scene is a Zarr store you can read back.

How a volume becomes a scene: luxar gsplat fit turns the light-sheet stack into Gaussians, fifteen lines of Python compile the scene, a link opens it. The GPU rasterises the splats directly. No voxel grid, ever. Demo:

How many Gaussians? Training PSNR keeps rising, so it cannot say. Score the fit on voxels it never saw instead: held-out PSNR rises, peaks, then falls as the splats start memorising noise. The peak is the budget. Kidney nuclei: 64K splats, and a 14.6 dB gap by 512K.

The how, from self-supervised denoising (Noise2Self, Batson & Royer 2019; Noise2Void): mask 5% of voxels, fill each from its neighbours, fit, score only at the masked voxels. Noise is independent between voxels, so only structure predicts them. One command: luxar gsplat cal.

Fits stop at the noise ceiling. A volume's noise sets a PSNR ceiling; at the cross-validated budget every fit sits below it (median 4.4 dB). Push on and training PSNR crosses it on the noisy confocal volumes: memorised noise. Only the signal is stored; the compression follows.

Not every curve turns down. Fusion and deconvolution correlate the noise, so on light-sheet volumes held-out PSNR plateaus or rises to 2M splats; Tribolium is still 5.5 dB under its 61 dB ceiling. Where a ceiling is estimable the curve stays below it: the ceiling is the stop.

🐟 Channels and timepoints are just axes. A zebrafish neuromast (iSIM, 100 timepoints) recorded by Adrian Jacobo's lab: membranes and nuclei as two layers, each with its own window, colormap and blending. #lightsheet Demo:

🪰 MultiColor FlpOut neurons of a Drosophila brain (FlyLight, @HHMIJanelia) as Gaussian splats, flown through. Demo:

The same brain, framed: a raw 63x FlyLight tile, every labelled neurite a splat, in its bounding box. Demo:

A cryo-EM density map, the PBCV-1 giant-virus capsid (EMD-5384), as Gaussian splats. Splats are for any scalar field, fluorescence or otherwise. Demo:

🐟 Zebrahub: a zebrafish single-cell UMAP with RNA-velocity streamlines, cells as points and flow as lines in one scene. #SingleCell Demo:

🌌 Cosmicflows-4: the Laniakea supercluster with its velocity streamlines. #dataviz Demo:

A cumulus cloud evolving: a 4D Gaussian-splat volume played in time. Demo:

A Mandelbulb, because a fractal is a volume too. Demo:

A particle collision, animated through time. Demo:

Every line of Luxar was designed, generated, debugged, tested with the help of Claude Code: 33,000+ tests across Python, CUDA, TypeScript, Rust and Go, 91% coverage. Pre-AI this was thirty person-years. Analyses, figures and this thread too, under my direction and verification.

Thanks to Merlin Lange for the zebrafish recording, to Adrian Jacobo and his lab for the neuromast, and to Philipp Keller and @HHMIJanelia for permission to share the Drosophila time-lapse as splats.

Thanks to the teams behind the public data: OpenCell, FlyLight at Janelia and FISBe, the Cell Tracking Challenge, the Image Data Resource, the Allen Institute for Cell Science, scikit-image, Tabula Sapiens, DESI, Cosmicflows, PDB, HYCOM, Kaggle and more, credited in every scene.

Thanks to Kyle Harrington for discussions, to the communities behind Zarr, @threejs, @PyTorch, @numpy_team and @napari_imaging, and to @biohub for funding this work, with the generous support of Priscilla Chan and Mark Zuckerberg.

Try it, break it, tell me what you would build with it. pip install luxar Code: Demos: Preprint: #microscopy #opensource #bioimaging #dataviz

@biohub Just want to say this is more than awesome work to bring gaussian splatting to bioimage.

@biohub This is awesome can i DM you?

@biohub 🤯
