Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

What if any dataset, any size, any number of dimensions, opened in a browser tab from a link? 🔬🧪💻 Luxar is out today: write it in Python, share it as a link, explore it in any browser. Open source. 🧵 biohub

17,834 Aufrufe • vor 8 Tagen •via X (Twitter)

48 Kommentare

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

🪰 500 timepoints of Drosophila gastrulation playing in a browser tab from static files, orbited live. SiMView recording with Philipp Keller at @HHMIJanelia. #lightsheet Preprint: Code: Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

How it works. Describe the scene in Python: points, lines, meshes and Gaussian splats, any number of dimensions. Compile once into a chunked, indexed .luxar.zarr archive, level-of-detail ladder optional. Put it on any static host. The browser does the rest. #dataviz

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

🐟 The microscopy piece: Gaussian splatting, volumes as Gaussian mixtures, ~100x smaller. 17 volumes, 12 datasets, 4 modalities: median 99x (6 to 340x) at 26 to 67 dB PSNR, minutes per volume on one GPU. This frame: 2.2 M splats. #lightsheet Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

The human nuclear pore complex, 4,937,064 atoms in 808 chains, approached head-on and flown through its central channel. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Luxar is made for 3D UMAPs. 500K Tabula Sapiens cells from 24 tissues as points coloured by organ, with a legend in the scene; hovering a cell names its cell type and tissue. Millions of points stay interactive. #SingleCell Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Quick start: pip install luxar luxar demo run cloud A cumulus cloud billows, matures and sinks back, a 3D+time scene you can play. luxar demo lists 89 more. Any public .luxar.zarr archive opens at

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

🌌 Beyond biology. All 9.75 million galaxies and quasars of DESI DR1 (DESI Collaboration), placed in 3D by redshift, streamed through a level-of-detail ladder. #dataviz Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

575,503 Swiss-Prot proteins, ESM C embeddings (@EvoscaleAI, now part of @biohub) in a 3D UMAP coloured by taxon, flown through in fly mode. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Four geometry types, one scene graph, any number of dimensions. C. elegans: nuclei as Gaussian splats, lineage as polylines, current positions as points, played through time with the tracks clipped to the slice. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Annotation is native. A map of arXiv, bioRxiv and medRxiv papers: hover a point and GPU picking names the paper; right-click copies the title or opens the link. #dataviz Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Hover can show images too. The CytoSelf protein-localisation map from OpenCell: hover a protein and its source micrograph appears. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

🌍 The ocean currents of Earth. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

A cluster fly, macro-photogrammetry Gaussian splats by Dany Bittel (CC BY 4.0), imported from a PLY file. Luxar reads INRIA PLY, .splat, .spz, SuperSplat and PlayCanvas SOG (@playcanvas) and writes PLY back. #GaussianSplatting Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Scale is not the constraint. A hundred embryos on a line, each an adaptive level-of-detail group; colour is the level drawn. Detail never exceeds what the screen can show, so the line could be far longer at almost no extra cost. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Level of detail is a grammar, not a preset. Six topologies from one Tribolium fit, one flag each: flat, stream, levels, tiles, overview, adaptive. Here the levels column swaps from coarse (red) to finest (teal) as it fills the screen. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

The archive is a Zarr hierarchy indexed in space and along every other dimension; the viewer fetches only the chunks, levels and timepoints the view needs. Cold load at 30 Mbit/s: coarse rung first, then detail, then time steps. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Points. The atoms of ATP synthase (PDB 5DN6), sharpness swept from soft Gaussian sprites to hard spheres along a hidden dimension of the scene. 60 FPS through a million elements. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Lines. A single cell's genome (Dip-C) as haplotype-resolved polylines drawn as capsules: joints that neither brighten nor open at corners, at any zoom. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Meshes. Cells3D isosurfaces as shaded surfaces under a view-anchored headlight, in the same scene graph as the points, lines and splats. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Gaussian splats. Additive or volumetric blending, switched live on a Drosophila embryo. Volumetric gives absorption and depth; additive gives the glow. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

🧠 The HCP-1065 white-matter atlas (Yeh 2022), 87 tracts as polylines, each tract its own colour, orbited. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

A clinical CT coloured by 117 TotalSegmentator labels. Hovering names the structure; skeleton, organs, vessels, nerves and muscles toggle as layers. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Scenes can carry guided tours: camera flights, overlays and narration, authored in Python. Sound on. This is the hemoglobin story on the protein landscape. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

The same viewer on an iPad: one finger rotates, two pan, a pinch zooms, a twist rolls, a double tap recentres. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Ship it your way: a link, an offline folder, a native app, or the viewer embedded in any web page from npm. luxar export does all of it, and the recording panel writes a movie.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

It plays with the tools you have: luxar gsplat napari opens a fit in @napari_imaging, any NumPy array is one call from a scene, and every scene is a Zarr store you can read back.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

How a volume becomes a scene: luxar gsplat fit turns the light-sheet stack into Gaussians, fifteen lines of Python compile the scene, a link opens it. The GPU rasterises the splats directly. No voxel grid, ever. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

How many Gaussians? Training PSNR keeps rising, so it cannot say. Score the fit on voxels it never saw instead: held-out PSNR rises, peaks, then falls as the splats start memorising noise. The peak is the budget. Kidney nuclei: 64K splats, and a 14.6 dB gap by 512K.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

The how, from self-supervised denoising (Noise2Self, Batson & Royer 2019; Noise2Void): mask 5% of voxels, fill each from its neighbours, fit, score only at the masked voxels. Noise is independent between voxels, so only structure predicts them. One command: luxar gsplat cal.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Fits stop at the noise ceiling. A volume's noise sets a PSNR ceiling; at the cross-validated budget every fit sits below it (median 4.4 dB). Push on and training PSNR crosses it on the noisy confocal volumes: memorised noise. Only the signal is stored; the compression follows.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Not every curve turns down. Fusion and deconvolution correlate the noise, so on light-sheet volumes held-out PSNR plateaus or rises to 2M splats; Tribolium is still 5.5 dB under its 61 dB ceiling. Where a ceiling is estimable the curve stays below it: the ceiling is the stop.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

🐟 Channels and timepoints are just axes. A zebrafish neuromast (iSIM, 100 timepoints) recorded by Adrian Jacobo's lab: membranes and nuclei as two layers, each with its own window, colormap and blending. #lightsheet Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

🪰 MultiColor FlpOut neurons of a Drosophila brain (FlyLight, @HHMIJanelia) as Gaussian splats, flown through. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

The same brain, framed: a raw 63x FlyLight tile, every labelled neurite a splat, in its bounding box. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

A cryo-EM density map, the PBCV-1 giant-virus capsid (EMD-5384), as Gaussian splats. Splats are for any scalar field, fluorescence or otherwise. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

🐟 Zebrahub: a zebrafish single-cell UMAP with RNA-velocity streamlines, cells as points and flow as lines in one scene. #SingleCell Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

🌌 Cosmicflows-4: the Laniakea supercluster with its velocity streamlines. #dataviz Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

A cumulus cloud evolving: a 4D Gaussian-splat volume played in time. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

A Mandelbulb, because a fractal is a volume too. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

A particle collision, animated through time. Demo:

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Every line of Luxar was designed, generated, debugged, tested with the help of Claude Code: 33,000+ tests across Python, CUDA, TypeScript, Rust and Go, 91% coverage. Pre-AI this was thirty person-years. Analyses, figures and this thread too, under my direction and verification.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Thanks to Merlin Lange for the zebrafish recording, to Adrian Jacobo and his lab for the neuromast, and to Philipp Keller and @HHMIJanelia for permission to share the Drosophila time-lapse as splats.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Thanks to the teams behind the public data: OpenCell, FlyLight at Janelia and FISBe, the Cell Tracking Challenge, the Image Data Resource, the Allen Institute for Cell Science, scikit-image, Tabula Sapiens, DESI, Cosmicflows, PDB, HYCOM, Kaggle and more, credited in every scene.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Thanks to Kyle Harrington for discussions, to the communities behind Zarr, @threejs, @PyTorch, @numpy_team and @napari_imaging, and to @biohub for funding this work, with the generous support of Priscilla Chan and Mark Zuckerberg.

Profilbild von Loïc A. Royer 💻🔬⚗️
Loïc A. Royer 💻🔬⚗️vor 8 Tagen

Try it, break it, tell me what you would build with it. pip install luxar Code: Demos: Preprint: #microscopy #opensource #bioimaging #dataviz

Profilbild von ǝɥǝ⅄
ǝɥǝ⅄vor 8 Tagen

@biohub Just want to say this is more than awesome work to bring gaussian splatting to bioimage.

Profilbild von Anindyadeep
Anindyadeepvor 8 Tagen

@biohub This is awesome can i DM you?

Profilbild von Sabyasachi Ghosh
Sabyasachi Ghoshvor 8 Tagen

@biohub 🤯

Ähnliche Videos