正在加载视频...

视频加载失败

返回首页

🚀 Introducing scGPT-spatial! 🧬🌍 A game-changing spatial-omic foundation model, built on the powerful scGPT framework with MoE (mixture of experts) and continually pretrained on a massive 30 million spatial single-cell profiles! 🧠 What’s the challenge? Spatial transcriptomics is next-level complex—not only must we model single-cell/spot profiles, but we also...

BoWang87's profile picture

Bo Wang

30,344 subscribers

58,976 次观看 • 1 年前 •via X (Twitter)

科学技术教育健康养生
LIVE

Anya Rossi• Live Now

Private livecam show

11 条评论

Derya Unutmaz, MD 的头像
Derya Unutmaz, MD1 年前

Amazing as always!

AndaSeat 的头像
AndaSeat1 年前

🍀 Mercury retrograde-proof your gaming life! ⚔️ Kaiser 3's anti-bad-luck features: 💪 5,000+ durability cycles (tested!) 💧 Water-resistant surface 🛡️ Anti-spill protection ✨ Murphy's Law defying design 🌟 Because life happens, gaming shouldn't stop! 🪑⚡ Guard your gameplay: 🎲 Lucky deal: Chase away bad luck with $20 off! #AndaSeat #GamerLife #DurabilityGoals #Kaiser3 🎮✨

Tanishq Mathew Abraham, Ph.D. 的头像
Tanishq Mathew Abraham, Ph.D.1 年前

Looks interesting, congrats!

Gökbörü 的头像
Gökbörü1 年前

awesome

Aiden 的头像
Aiden1 年前

Awesome, looks interesting

Steven ten Holder — e/bio 的头像
Steven ten Holder — e/bio1 年前

This + RegFormer insights…

Patricia Cano 的头像
Patricia Cano1 年前

scGPT-spatial’s integration of MoE and 30M spatial single-cell profiles could transform spatial biology, especially in zoonotic disease research. A huge step for public health and One Health!

steven hoffman 的头像
steven hoffman1 年前

@qicy11 Awesome

Greg Cook 的头像
Greg Cook1 年前

Hey @DeryaTR_ FYI

Hao Yin 的头像
Hao Yin1 年前

Wowo 🤯 Absolutely a must read 🫡

Vivek Das 的头像
Vivek Das1 年前

Looks interesting. Congratulations to your team @BoWang87 .

相关视频

🔥😀Super-super excited to report the publication of "Single-cell multi-omic and spatial profiling of human kidneys implicates the fibrotic microenvironment in kidney disease progression" in Nature Genetics by Amin Abedini and Jon Levinsohn #kidney #diabetes #spatial transcriptomics
1:21

🔥😀Super-super excited to report the publication of "Single-cell multi-omic and spatial profiling of human kidneys implicates the fibrotic microenvironment in kidney disease progression" in Nature Genetics by Amin Abedini and Jon Levinsohn #kidney #diabetes #spatial transcriptomics

Katalin Susztak

16,259 次观看 • 1 年前

PantheonOS allows any biologist to perform complex data analyses of emerging single cell, multi-omics and spatial transcriptomics datasets end to end through AI agent and human collaboration. We are releasing six replays of use cases trajectories. Each "trajectory" is a complete end-to-end run, from prompt to analysis, to figures, and finally to report, and in fully inspectable and reproducible manner. See the six user cases in our PantheonOS Gallery: 1. 3D mouse embryo analysis: Tangram deconvolution and PyVista-based 3D visualization of E6 mouse embryo spatial transcriptomics data 2. 3D human fetal heart analysis: Spatial mapping of heart disease gene patterns using MERFISH data 3. Multi-omics spatial integration: Single cell multi-omics-to-spatial mapping with MOSCOT optimal transport 4. Gene panel design: 1000-plex immune-oncology MERFISH gene panel optimization 5. Cell segmentation benchmarking: Comparative evaluation of Cellpose-SAM, InstanSeg, StarDist, and other tools 6. Spatial disease biology: Ligand–receptor analysis of disease-associated tissue microenvironments We would love to hear how you can use PantheonOS for your research! Got an interesting agent run of your own and want to share? In the Pantheon UI, click Export Bundle (top-right of any chat) to package the full trajectory — chat history, code, figures, report — then submit it here:
0:58

PantheonOS allows any biologist to perform complex data analyses of emerging single cell, multi-omics and spatial transcriptomics datasets end to end through AI agent and human collaboration. We are releasing six replays of use cases trajectories. Each "trajectory" is a complete end-to-end run, from prompt to analysis, to figures, and finally to report, and in fully inspectable and reproducible manner. See the six user cases in our PantheonOS Gallery: 1. 3D mouse embryo analysis: Tangram deconvolution and PyVista-based 3D visualization of E6 mouse embryo spatial transcriptomics data 2. 3D human fetal heart analysis: Spatial mapping of heart disease gene patterns using MERFISH data 3. Multi-omics spatial integration: Single cell multi-omics-to-spatial mapping with MOSCOT optimal transport 4. Gene panel design: 1000-plex immune-oncology MERFISH gene panel optimization 5. Cell segmentation benchmarking: Comparative evaluation of Cellpose-SAM, InstanSeg, StarDist, and other tools 6. Spatial disease biology: Ligand–receptor analysis of disease-associated tissue microenvironments We would love to hear how you can use PantheonOS for your research! Got an interesting agent run of your own and want to share? In the Pantheon UI, click Export Bundle (top-right of any chat) to package the full trajectory — chat history, code, figures, report — then submit it here:

evo-devo

12,799 次观看 • 1 个月前

🔬 Exciting News! Our manuscript, "scGPT: toward building a foundation model for single-cell multi-omics using generative AI" is now finally published in Nature Methods (Nature Methods) 🎉 !!! (Re-)Introducing scGPT: A transformative foundation model engineered for single-cell omics analysis. Developed through the analysis of over 33 million human cells, scGPT sets a new benchmark for application versatility, offering both fine-tuning and zero-shot capabilities. Since its preprint in May 2023, scGPT has significantly impacted the field, evidenced by 13K+ installations, 600+ GitHub stars 🌟, and 40+ citations before its official publication! scGPT has been validated by numerous benchmark studies as a leading foundation model in single-cell analysis. Its pre-trained embeddings extend its utility beyond single-cell studies, enhancing a variety of downstream tasks including protein enrichment and genetic perturbation predictions. Some key updates lately: ---Expanded zero-shot applications for efficient reference mapping and integration, now with CellXGene census integration. ---Advanced perturbation analysis capabilities, including genome-scale perturb-seq data analysis and bulk sequencing data generalization. ---Upgraded scGPT package, offering versatile model loading compatible with PyTorch and flash-attn, for both GPU and CPU. ---Cloud-based scGPT applications for reference mapping, cell annotation, and gene regulatory network inference are available on ---Integration with Hugging Face for easier model training. Limitations: scGPT is an early foray into foundation models for single-cell omics, facing challenges like limited zero-shot learning in some tasks, pretraining constraints, data quality issues, and evaluation limitations. See our Supplementary Notes for details. 🚀 Future Work? Short-Term Goals: 1. Releasing a Mouse Model for broader analysis. 2. Developing a comprehensive evaluation suite for foundation models in single-cell analysis. 3. Creating a foundation model for single-cell spatial omics. 4. Enhancing zero-shot capacity by integrating scGPT with RAG (e.g., knowledge graphs). Long-Term Goals: 1. Expanding scGPT for comprehensive single-cell multi-omics analysis. 2. Developing an in-silico perturbation model for predicting genetic perturbation effects. 3. Merging scGPT with multi-modal genomic sequence models for a deeper understanding of cell biology. 📚 Access the paper on Nature Methods: 🔬Preprint in Bioarixv: 💻 All our codes/data/weights are open source: Wholehearted congratulations to all the authors, especially the two co-first authors, Haotian (Haotian Cui ) and Chloe (ChloeXWang), who are really the emerging superstars in AI and biology! Vector Institute Peter Munk Cardiac Centre AI U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology University Health Network University of Toronto #scGPT #GenerativeAI #AI4Science #Combio #opensource
0:52

🔬 Exciting News! Our manuscript, "scGPT: toward building a foundation model for single-cell multi-omics using generative AI" is now finally published in Nature Methods (Nature Methods) 🎉 !!! (Re-)Introducing scGPT: A transformative foundation model engineered for single-cell omics analysis. Developed through the analysis of over 33 million human cells, scGPT sets a new benchmark for application versatility, offering both fine-tuning and zero-shot capabilities. Since its preprint in May 2023, scGPT has significantly impacted the field, evidenced by 13K+ installations, 600+ GitHub stars 🌟, and 40+ citations before its official publication! scGPT has been validated by numerous benchmark studies as a leading foundation model in single-cell analysis. Its pre-trained embeddings extend its utility beyond single-cell studies, enhancing a variety of downstream tasks including protein enrichment and genetic perturbation predictions. Some key updates lately: ---Expanded zero-shot applications for efficient reference mapping and integration, now with CellXGene census integration. ---Advanced perturbation analysis capabilities, including genome-scale perturb-seq data analysis and bulk sequencing data generalization. ---Upgraded scGPT package, offering versatile model loading compatible with PyTorch and flash-attn, for both GPU and CPU. ---Cloud-based scGPT applications for reference mapping, cell annotation, and gene regulatory network inference are available on ---Integration with Hugging Face for easier model training. Limitations: scGPT is an early foray into foundation models for single-cell omics, facing challenges like limited zero-shot learning in some tasks, pretraining constraints, data quality issues, and evaluation limitations. See our Supplementary Notes for details. 🚀 Future Work? Short-Term Goals: 1. Releasing a Mouse Model for broader analysis. 2. Developing a comprehensive evaluation suite for foundation models in single-cell analysis. 3. Creating a foundation model for single-cell spatial omics. 4. Enhancing zero-shot capacity by integrating scGPT with RAG (e.g., knowledge graphs). Long-Term Goals: 1. Expanding scGPT for comprehensive single-cell multi-omics analysis. 2. Developing an in-silico perturbation model for predicting genetic perturbation effects. 3. Merging scGPT with multi-modal genomic sequence models for a deeper understanding of cell biology. 📚 Access the paper on Nature Methods: 🔬Preprint in Bioarixv: 💻 All our codes/data/weights are open source: Wholehearted congratulations to all the authors, especially the two co-first authors, Haotian (Haotian Cui ) and Chloe (ChloeXWang), who are really the emerging superstars in AI and biology! Vector Institute Peter Munk Cardiac Centre AI U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology University Health Network University of Toronto #scGPT #GenerativeAI #AI4Science #Combio #opensource

Bo Wang

199,592 次观看 • 2 年前

We NOETIK built a new foundation model for spatial biology: OCTO-VirtualCell. This model is prompted to simulate single cell gene expression in "virtual cells," which are placed within real, intact patient tissue. Read about & explore these simulations below:
0:26

We NOETIK built a new foundation model for spatial biology: OCTO-VirtualCell. This model is prompted to simulate single cell gene expression in "virtual cells," which are placed within real, intact patient tissue. Read about & explore these simulations below:

Daniel Bear

39,303 次观看 • 1 年前

New research with Tsinghua University: Spatial-TTT. A framework for streaming visual-based spatial intelligence with test-time training (TTT). Spatial-TTT adapts fast weights to capture and organize spatial evidence from long video streams, enabling models to build structured 3D spatial memory over time. Highlights: 🔹Efficient streaming memory. Fast weights act as compact spatial memory with sublinear memory growth over 7000+ frames and more than 40% lower compute. 🔹Spatial-predictive mechanism. TTT layers with 3D spatiotemporal convolution capture geometric correspondence and temporal continuity. 🔹SOTA results on long-horizon video spatial understanding (VSI-Bench). The paper ranked #1 on Hugging Face Daily Papers on March 13. Project page: GitHub: Paper: Model & Data:
0:47

New research with Tsinghua University: Spatial-TTT. A framework for streaming visual-based spatial intelligence with test-time training (TTT). Spatial-TTT adapts fast weights to capture and organize spatial evidence from long video streams, enabling models to build structured 3D spatial memory over time. Highlights: 🔹Efficient streaming memory. Fast weights act as compact spatial memory with sublinear memory growth over 7000+ frames and more than 40% lower compute. 🔹Spatial-predictive mechanism. TTT layers with 3D spatiotemporal convolution capture geometric correspondence and temporal continuity. 🔹SOTA results on long-horizon video spatial understanding (VSI-Bench). The paper ranked #1 on Hugging Face Daily Papers on March 13. Project page: GitHub: Paper: Model & Data:

Tencent Hy

20,792 次观看 • 2 个月前

"Spatial interactions modulate tumor growth and immune infiltration" Our latest preprint: Sadegh Marzban in collab w/ Amelio Lab! We apply methods from artificial life, Bert Chan's #Lenia, as a model of tumor-immune: - spatial growth dynamics - cell-cell competition & - cell migration Application to head & neck squamous cell carcinomas to elucidate the role of collagen on immune infiltration
0:20

"Spatial interactions modulate tumor growth and immune infiltration" Our latest preprint: Sadegh Marzban in collab w/ Amelio Lab! We apply methods from artificial life, Bert Chan's #Lenia, as a model of tumor-immune: - spatial growth dynamics - cell-cell competition & - cell migration Application to head & neck squamous cell carcinomas to elucidate the role of collagen on immune infiltration

Jeffrey West

12,297 次观看 • 2 年前

We’re thrilled to share that our MERFISH+ preprint is now live on bioRxiv!👉 In this work, the Bintu and Zhu labs (UCSD) developed MERFISH+, a next-generation spatial genomics platform that combines genome-wide RNA and epigenetic imaging over a large field of view. By introducing acrydite-modified probes covalently anchored to hydrogels, MERFISH+ achieves remarkable imaging stability and enables >1,800-gene, multi-modal, and multi-month experiments. With this platform, they, together with the Chi lab at UCSD, profiled a whole developing human heart at 12 post-conception week with merely two slides, resulting in a total of 53 slides, 3.1 million single cells and more than 30 cell types. Building upon our previous 3D reconstruction and modeling framework, Spateo ( we reconstruct the 3D human heart that nicely captures the anatomical structure of the heart, including the intricate vasculature network. Sophisticated analyses provide a holistic view of an entire organ and enable systematic characterization of 3D cellular neighborhoods and transcriptional gradients of substructures such as the descending arteries. Furthermore, using a generative integration framework for spatial multimodal data (Spateo-VI), we harmonized these MERFISH+ transcriptomic and chromatin data to reconstruct a 3D spatially-resolved multi-omics atlas of the developing human heart, shared at and MERFISH+ thus sets a new standard for large-format, multi-omic spatial profiling, enabling holistic, 3D characterization of organs at subcellular resolution. Huge congratulations to first authors Colin Kern, qingquan Zhang, Yifan Lu , and Jacqueline Eschbach, and to all collaborators from the Bintu, Zhu, Chi, and Qiu labs for this amazing team effort. Thanks for your diligence, creativity, and hard work on this project. We’re grateful for support from Arc Institute and our generous donors. Our lab is expanding—if you’re excited about building the next generation of single-cell and spatial genomics techniques and predictive single cell and spatial foundation models, we’re hiring! If you are interested, please reach out to me via direct message or email at xiaojie@stanford.edu. We are excited for any potential collaborations along this line of research in Stanford, UCSF and Berkeley and other labs as well.
0:42

We’re thrilled to share that our MERFISH+ preprint is now live on bioRxiv!👉 In this work, the Bintu and Zhu labs (UCSD) developed MERFISH+, a next-generation spatial genomics platform that combines genome-wide RNA and epigenetic imaging over a large field of view. By introducing acrydite-modified probes covalently anchored to hydrogels, MERFISH+ achieves remarkable imaging stability and enables >1,800-gene, multi-modal, and multi-month experiments. With this platform, they, together with the Chi lab at UCSD, profiled a whole developing human heart at 12 post-conception week with merely two slides, resulting in a total of 53 slides, 3.1 million single cells and more than 30 cell types. Building upon our previous 3D reconstruction and modeling framework, Spateo ( we reconstruct the 3D human heart that nicely captures the anatomical structure of the heart, including the intricate vasculature network. Sophisticated analyses provide a holistic view of an entire organ and enable systematic characterization of 3D cellular neighborhoods and transcriptional gradients of substructures such as the descending arteries. Furthermore, using a generative integration framework for spatial multimodal data (Spateo-VI), we harmonized these MERFISH+ transcriptomic and chromatin data to reconstruct a 3D spatially-resolved multi-omics atlas of the developing human heart, shared at and MERFISH+ thus sets a new standard for large-format, multi-omic spatial profiling, enabling holistic, 3D characterization of organs at subcellular resolution. Huge congratulations to first authors Colin Kern, qingquan Zhang, Yifan Lu , and Jacqueline Eschbach, and to all collaborators from the Bintu, Zhu, Chi, and Qiu labs for this amazing team effort. Thanks for your diligence, creativity, and hard work on this project. We’re grateful for support from Arc Institute and our generous donors. Our lab is expanding—if you’re excited about building the next generation of single-cell and spatial genomics techniques and predictive single cell and spatial foundation models, we’re hiring! If you are interested, please reach out to me via direct message or email at [email protected]. We are excited for any potential collaborations along this line of research in Stanford, UCSF and Berkeley and other labs as well.

evo-devo

42,087 次观看 • 7 个月前

Today in nature, we report a spatial single-cell atlas of human cortical development, revealing surprisingly early specification of human cortical layers and areas. We built an interactive browser to explore the spatial data: Paper link below👇
0:57

Today in nature, we report a spatial single-cell atlas of human cortical development, revealing surprisingly early specification of human cortical layers and areas. We built an interactive browser to explore the spatial data: Paper link below👇

Xuyu Qian

53,272 次观看 • 1 年前

Inspired by some gesture-based point cloud controllers I've seen on here, I vibe coded a similar web app to explore the relationship between spatial, UMAP, and PCA embeddings for spatial transcriptomics data. Next level interactivity via🖐️ Try it out:
0:42

Inspired by some gesture-based point cloud controllers I've seen on here, I vibe coded a similar web app to explore the relationship between spatial, UMAP, and PCA embeddings for spatial transcriptomics data. Next level interactivity via🖐️ Try it out:

Dr. Jean Fan

17,620 次观看 • 4 个月前

We are thrilled to share that our first paper from my new lab, Spateo ( for spatiotemporal modeling of molecular holograms, is now online in Cell: Spateo is a comprehensive analytical framework for 3D whole-embryo spatiotemporal modeling. Its advanced features include: • 3D alignment and reconstruction at the whole-mouse-embryo scale (see the animation). • 3D spatial domain digitization and cell-cell communication analysis to understand spatial gene expression gradients and both inter- and intracellular communication. • 3D morphometric and volumetric analyses along with 3D morphogenesis vector field modeling to quantify dynamics such as surface area, volume, and cell density across organs, and to dissect the interplay between morphogenesis factors and cell migration. • A “Google Earth”-like browser, Spateo-viewer ( and for interactive and intuitive exploration of 3D spatial data. • Additional features, such as RNA signal-based single-cell segmentation. We are also honored that Nature “News and Views” has highlighted this work as well: This is really an amazing outcome after two years' heroic revision process that rewrite the entire paper using a new data ( for whole mouse embryos.
1:24

We are thrilled to share that our first paper from my new lab, Spateo ( for spatiotemporal modeling of molecular holograms, is now online in Cell: Spateo is a comprehensive analytical framework for 3D whole-embryo spatiotemporal modeling. Its advanced features include: • 3D alignment and reconstruction at the whole-mouse-embryo scale (see the animation). • 3D spatial domain digitization and cell-cell communication analysis to understand spatial gene expression gradients and both inter- and intracellular communication. • 3D morphometric and volumetric analyses along with 3D morphogenesis vector field modeling to quantify dynamics such as surface area, volume, and cell density across organs, and to dissect the interplay between morphogenesis factors and cell migration. • A “Google Earth”-like browser, Spateo-viewer ( and for interactive and intuitive exploration of 3D spatial data. • Additional features, such as RNA signal-based single-cell segmentation. We are also honored that Nature “News and Views” has highlighted this work as well: This is really an amazing outcome after two years' heroic revision process that rewrite the entire paper using a new data ( for whole mouse embryos.

evo-devo

58,252 次观看 • 1 年前

Thanks to the authors for sharing all components of this mouse embryo spatial transcriptomics data from cell gene counts to per-molecule coordinates: 🥳 So I vibe coded an app to explore the 3D subcellular transcript organization:
0:20

Thanks to the authors for sharing all components of this mouse embryo spatial transcriptomics data from cell gene counts to per-molecule coordinates: 🥳 So I vibe coded an app to explore the 3D subcellular transcript organization:

Dr. Jean Fan

50,502 次观看 • 4 个月前

1/n: 🔬We introduce NicheCompass (NC) a graph learning model to build spatial atlases across millions of cells, quantitatively characterize cell niches, infer cellular communication, and map new data into these atlases. NC can be used with various spatial technologies, ranging
1:04

1/n: 🔬We introduce NicheCompass (NC) a graph learning model to build spatial atlases across millions of cells, quantitatively characterize cell niches, infer cellular communication, and map new data into these atlases. NC can be used with various spatial technologies, ranging

Mo Lotfollahi

61,747 次观看 • 2 年前

Spatial-omics goes 3D🤗! Out at Cell Cell 👉🏼 We developed DISCO-MS with Matthias Mann Lab, a spatial proteomics technology for specimens fully imaged in 3D. DISCO-MS is aided by robotics and enables the study of diseases at their early stages.
2:19

Spatial-omics goes 3D🤗! Out at Cell Cell 👉🏼 We developed DISCO-MS with Matthias Mann Lab, a spatial proteomics technology for specimens fully imaged in 3D. DISCO-MS is aided by robotics and enables the study of diseases at their early stages.

Ali Max Erturk

217,640 次观看 • 3 年前

Announcing the Video Editor in Spatial Media Toolkit. Effortlessly create spatial videos or photo slideshows with music, all privately on-device. Seamlessly integrates with our powerful 2D-to-3D conversion for stunning spatial storytelling.
0:47

Announcing the Video Editor in Spatial Media Toolkit. Effortlessly create spatial videos or photo slideshows with music, all privately on-device. Seamlessly integrates with our powerful 2D-to-3D conversion for stunning spatial storytelling.

Ryan Gordon

12,353 次观看 • 1 年前

Apple and Meta are still missing the point when it comes to spatial productivity. Going monitor-free or showing the weather in 3d space isn't enough. There's not enough value in replacing a frictionless setup like a laptop and mouse unless the spatial environment unlocks something radically more powerful: a creative work space so fluid and enabling that you wouldn't want to work any other way. I created this "Spatial AI Workbench" mockup a while ago to explore that exact idea.
1:49

Apple and Meta are still missing the point when it comes to spatial productivity. Going monitor-free or showing the weather in 3d space isn't enough. There's not enough value in replacing a frictionless setup like a laptop and mouse unless the spatial environment unlocks something radically more powerful: a creative work space so fluid and enabling that you wouldn't want to work any other way. I created this "Spatial AI Workbench" mockup a while ago to explore that exact idea.

Greg Madisᯅn

75,882 次观看 • 1 年前

WE DID IT! +100,000 OVRMAPS 🚀 A major milestone in decentralized spatial computing. A huge thank you to our incredible community of mappers and our technical team! ↳ 30M sqm of 3D-mapped terrain ↳ 55TB of image data processed with cutting-edge AI models ↳ Powered by DePIN and a global network of mappers This is the future of Spatial Computing. Let’s break it down: 🧵👇
0:28

WE DID IT! +100,000 OVRMAPS 🚀 A major milestone in decentralized spatial computing. A huge thank you to our incredible community of mappers and our technical team! ↳ 30M sqm of 3D-mapped terrain ↳ 55TB of image data processed with cutting-edge AI models ↳ Powered by DePIN and a global network of mappers This is the future of Spatial Computing. Let’s break it down: 🧵👇

Over the Reality 🌐

493,928 次观看 • 1 年前

For those attending AACR26 in San Diego, I will demonstrate how CODA can map whole tumors, organs, and organisms in 3D and at single-cell resolution. I will also show how CODA can integrate spatial transcriptomics, spatial proteomics, IHC, IMC, etc. April 17, 3:30 pm
0:42

For those attending AACR26 in San Diego, I will demonstrate how CODA can map whole tumors, organs, and organisms in 3D and at single-cell resolution. I will also show how CODA can integrate spatial transcriptomics, spatial proteomics, IHC, IMC, etc. April 17, 3:30 pm

Denis Wirtz

27,644 次观看 • 1 个月前

this is a must have app on the Apple Vision Pro Spatial Media Toolkit turns your normal photos and videos into spatial photos and videos. It’s hypnotic
0:30

this is a must have app on the Apple Vision Pro Spatial Media Toolkit turns your normal photos and videos into spatial photos and videos. It’s hypnotic

Justin Ryan ᯅ

48,754 次观看 • 2 年前

We were thrilled to have Niantic Spatial 🌎's Project Jade demo on Spectacles at Lens Fest 2025 to show the bleeding edge of what spatial, location-based experiences can be. Voices of VR host Kent Bye (Voices of VR) sat down with the team to talk how it came together, the role Snap’s tech stack plays in development, and what’s next for XR.
1:38

We were thrilled to have Niantic Spatial 🌎's Project Jade demo on Spectacles at Lens Fest 2025 to show the bleeding edge of what spatial, location-based experiences can be. Voices of VR host Kent Bye (Voices of VR) sat down with the team to talk how it came together, the role Snap’s tech stack plays in development, and what’s next for XR.

Spectacles

15,259 次观看 • 6 个月前

GS^3: Efficient Relighting with Triple Gaussian Splatting Abstract: We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex ap pearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance.
0:40

GS^3: Efficient Relighting with Triple Gaussian Splatting Abstract: We present a spatial and angular Gaussian based representation and a triple splatting process, for real-time, high-quality novel lighting-and-view synthesis from multi-view point-lit input images. To describe complex ap pearance, we employ a Lambertian plus a mixture of angular Gaussians as an effective reflectance function for each spatial Gaussian. To generate self-shadow, we splat all spatial Gaussians towards the light source to obtain shadow values, which are further refined by a small multi-layer perceptron. To compensate for other effects like global illumination, another network is trained to compute and add a per-spatial-Gaussian RGB tuple. The effectiveness of our representation is demonstrated on 30 samples with a wide variation in geometry (from solid to fluffy) and appearance (from translucent to anisotropic), as well as using different forms of input data, including rendered images of synthetic/reconstructed objects, photographs captured with a handheld camera and a flash, or from a professional lightstage. We achieve a training time of 40-70 minutes and a rendering speed of 90 fps on a single commodity GPU. Our results compare favorably with state-of-the-art techniques in terms of quality/performance.

MrNeRF

17,759 次观看 • 1 年前