Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Excited to share Spotiflow, our new spot detection method for image-based spatial transcriptomics that facilitates the analysis of large iST images. Lead by Albert Dominguez Mantes jointly w Gioele La Manno EPFL Life Sciences EPFL Center for Imaging EPFL Institute of Bioengineering

Martin Weigert (maweigert.bsky.social)

1,729 subscribers

25,715 views • 2 years ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

3 Comments

Martin Weigert (maweigert.bsky.social)2 years ago

Spotiflow uses multiscale Gaussian heatmap prediction alongside a unique stereographic flow regression, which smoothly embeds the local closest-point vector field onto the unit sphere, thereby avoiding the ambiguity at infinity

Martin Weigert (maweigert.bsky.social)2 years ago

Spotiflow achieves SOTA results on 7 synthetic and real datasets while being faster than commonly used iST spot detection methods. It comes with pre-trained models, a python library, and a plugin for @napari_imaging

Martin Weigert (maweigert.bsky.social)2 years ago

Thanks to the many amazing people at @epflSV that contributed: Toni Herrera, Irina Khven, @CanAztekin, @Anjalie_Sch, Georgios Tsissios, Eftychia Kyriacou, Joachim Liger. Feedback welcome!

Related Videos

EPFL researchers have built a drone that can walk, hop, and jump into flight with the aid of birdlike legs, greatly expanding the range of potential environments accessible to unmanned aerial vehicles. RAVEN (Robotic Avian-inspired Vehicle for multiple ENvironments), designed based on perching birds like ravens and crows that frequently switch between air and land, the multifunctional robotic legs allow it to take off autonomously in environments previously inaccessible to winged drones. Check out the full article here: Image/video credits: EPFL / Ulrich M. / Akshet Patel #drones #robotics #technology #engineering #stem #innovation

EPFL researchers have built a drone that can walk, hop, and jump into flight with the aid of birdlike legs, greatly expanding the range of potential environments accessible to unmanned aerial vehicles. RAVEN (Robotic Avian-inspired Vehicle for multiple ENvironments), designed based on perching birds like ravens and crows that frequently switch between air and land, the multifunctional robotic legs allow it to take off autonomously in environments previously inaccessible to winged drones. Check out the full article here: Image/video credits: EPFL / Ulrich M. / Akshet Patel #drones #robotics #technology #engineering #stem #innovation

Wevolver

37,377 views • 1 year ago

EPFL researchers have built a drone that can walk, hop, and jump into flight with the aid of birdlike legs, greatly expanding the range of potential environments accessible to unmanned aerial vehicles. RAVEN (Robotic Avian-inspired Vehicle for multiple ENvironments), designed based on perching birds like ravens and crows that frequently switch between air and land, the multifunctional robotic legs allow it to take off autonomously in environments previously inaccessible to winged drones. Image/video credits: EPFL / Ulrich M. / Akshet Patel #drones #robotics #technology #engineering #stem #innovation -------------------------------- Stay ahead of the curve! Follow us now on our WhatsApp ( and Telegram ( channels and stay updated about the cutting edge.

EPFL researchers have built a drone that can walk, hop, and jump into flight with the aid of birdlike legs, greatly expanding the range of potential environments accessible to unmanned aerial vehicles. RAVEN (Robotic Avian-inspired Vehicle for multiple ENvironments), designed based on perching birds like ravens and crows that frequently switch between air and land, the multifunctional robotic legs allow it to take off autonomously in environments previously inaccessible to winged drones. Image/video credits: EPFL / Ulrich M. / Akshet Patel #drones #robotics #technology #engineering #stem #innovation -------------------------------- Stay ahead of the curve! Follow us now on our WhatsApp ( and Telegram ( channels and stay updated about the cutting edge.

Wevolver

22,140 views • 1 year ago

Collaborative Score Distillation for Consistent Visual Synthesis paper page: Generative priors of large-scale text-to-image diffusion models enable a wide range of new generation and editing applications on diverse visual modalities. However, when adapting these priors to complex visual modalities, often represented as multiple images (e.g., video), achieving consistency across a set of images is challenging. In this paper, we address this challenge with a novel method, Collaborative Score Distillation (CSD). CSD is based on the Stein Variational Gradient Descent (SVGD). Specifically, we propose to consider multiple samples as "particles" in the SVGD update and combine their score functions to distill generative priors over a set of images synchronously. Thus, CSD facilitates seamless integration of information across 2D images, leading to a consistent visual synthesis across multiple samples. We show the effectiveness of CSD in a variety of tasks, encompassing the visual editing of panorama images, videos, and 3D scenes. Our results underline the competency of CSD as a versatile method for enhancing inter-sample consistency, thereby broadening the applicability of text-to-image diffusion models.

Collaborative Score Distillation for Consistent Visual Synthesis paper page: Generative priors of large-scale text-to-image diffusion models enable a wide range of new generation and editing applications on diverse visual modalities. However, when adapting these priors to complex visual modalities, often represented as multiple images (e.g., video), achieving consistency across a set of images is challenging. In this paper, we address this challenge with a novel method, Collaborative Score Distillation (CSD). CSD is based on the Stein Variational Gradient Descent (SVGD). Specifically, we propose to consider multiple samples as "particles" in the SVGD update and combine their score functions to distill generative priors over a set of images synchronously. Thus, CSD facilitates seamless integration of information across 2D images, leading to a consistent visual synthesis across multiple samples. We show the effectiveness of CSD in a variety of tasks, encompassing the visual editing of panorama images, videos, and 3D scenes. Our results underline the competency of CSD as a versatile method for enhancing inter-sample consistency, thereby broadening the applicability of text-to-image diffusion models.

AK

33,513 views • 3 years ago

PantheonOS allows any biologist to perform complex data analyses of emerging single cell, multi-omics and spatial transcriptomics datasets end to end through AI agent and human collaboration. We are releasing six replays of use cases trajectories. Each "trajectory" is a complete end-to-end run, from prompt to analysis, to figures, and finally to report, and in fully inspectable and reproducible manner. See the six user cases in our PantheonOS Gallery: 1. 3D mouse embryo analysis: Tangram deconvolution and PyVista-based 3D visualization of E6 mouse embryo spatial transcriptomics data 2. 3D human fetal heart analysis: Spatial mapping of heart disease gene patterns using MERFISH data 3. Multi-omics spatial integration: Single cell multi-omics-to-spatial mapping with MOSCOT optimal transport 4. Gene panel design: 1000-plex immune-oncology MERFISH gene panel optimization 5. Cell segmentation benchmarking: Comparative evaluation of Cellpose-SAM, InstanSeg, StarDist, and other tools 6. Spatial disease biology: Ligand–receptor analysis of disease-associated tissue microenvironments We would love to hear how you can use PantheonOS for your research! Got an interesting agent run of your own and want to share? In the Pantheon UI, click Export Bundle (top-right of any chat) to package the full trajectory — chat history, code, figures, report — then submit it here:

PantheonOS allows any biologist to perform complex data analyses of emerging single cell, multi-omics and spatial transcriptomics datasets end to end through AI agent and human collaboration. We are releasing six replays of use cases trajectories. Each "trajectory" is a complete end-to-end run, from prompt to analysis, to figures, and finally to report, and in fully inspectable and reproducible manner. See the six user cases in our PantheonOS Gallery: 1. 3D mouse embryo analysis: Tangram deconvolution and PyVista-based 3D visualization of E6 mouse embryo spatial transcriptomics data 2. 3D human fetal heart analysis: Spatial mapping of heart disease gene patterns using MERFISH data 3. Multi-omics spatial integration: Single cell multi-omics-to-spatial mapping with MOSCOT optimal transport 4. Gene panel design: 1000-plex immune-oncology MERFISH gene panel optimization 5. Cell segmentation benchmarking: Comparative evaluation of Cellpose-SAM, InstanSeg, StarDist, and other tools 6. Spatial disease biology: Ligand–receptor analysis of disease-associated tissue microenvironments We would love to hear how you can use PantheonOS for your research! Got an interesting agent run of your own and want to share? In the Pantheon UI, click Export Bundle (top-right of any chat) to package the full trajectory — chat history, code, figures, report — then submit it here:

evo-devo

13,444 views • 4 months ago

Happy New Year! For EMBL, 2024 is an important milestone: our 50th anniversary! This year we’ll share stories from our past, and, importantly, report on our work that advances life sciences & continues to push the frontiers of molecular biology. Join us in celebrating #EMBL50!

Happy New Year! For EMBL, 2024 is an important milestone: our 50th anniversary! This year we’ll share stories from our past, and, importantly, report on our work that advances life sciences & continues to push the frontiers of molecular biology. Join us in celebrating #EMBL50!

EMBL

21,002 views • 2 years ago

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,681 views • 3 years ago

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

TurboEdit Instant text-based image editing discuss: We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.

AK

16,062 views • 2 years ago

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

TokenFlow: Consistent Diffusion Features for Consistent Video Editing paper page: The generative AI revolution has recently expanded to videos. Nevertheless, current state-of-the-art video models are still lagging behind image models in terms of visual quality and user control over the generated content. In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing. Specifically, given a source video and a target text-prompt, our method generates a high-quality video that adheres to the target text, while preserving the spatial layout and motion of the input video. Our method is based on a key observation that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. We achieve this by explicitly propagating diffusion features based on inter-frame correspondences, readily available in the model. Thus, our framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method. We demonstrate state-of-the-art editing results on a variety of real-world videos.

AK

143,469 views • 3 years ago

🚀 We are hiring! 𝗣𝗵𝗗 𝗦𝘁𝘂𝗱𝗲𝗻𝘁 𝗶𝗻 𝗖𝗔𝗥-𝗧 𝗖𝗲𝗹𝗹𝘂𝗹𝗮𝗿 𝗧𝗵𝗲𝗿𝗮𝗽𝘆 𝗮𝗻𝗱 𝗖𝗮𝗻𝗰𝗲𝗿 𝗡𝗲𝘂𝗿𝗼𝗶𝗺𝗺𝘂𝗻𝗼𝗹𝗼𝗴𝘆 We are looking for a passionate and motivated PhD student to join an exciting project focused on how nerve, immune and cancer cell interactions shape metastatic tumor behavior and resistance to cellular therapies, especially CAR-T cells. The project combines: • Whole-body tissue clearing • Light-sheet microscopy • wildDISCO-based immunolabeling • CAR-T and immune cell tracking in metastatic cancer • Spatial proteomics and 3D spatial biology • AI-based analysis of large-scale imaging datasets This is a great opportunity for someone with a background in cancer biology, immunology, neuroscience, biomedical sciences, biomedical engineering or a related field who wants to work at the interface of experimental biology, advanced imaging and AI. Apply here: Please share with excellent candidates who may be interested. #Hiring #PhDPosition #CARTcells #CellularTherapy #CancerResearch #Neuroimmunology #Immunotherapy #Metastasis #SpatialBiology #AIinBiology #Microscopy #LMUMunich #HelmholtzMunich #ErturkLab

🚀 We are hiring! 𝗣𝗵𝗗 𝗦𝘁𝘂𝗱𝗲𝗻𝘁 𝗶𝗻 𝗖𝗔𝗥-𝗧 𝗖𝗲𝗹𝗹𝘂𝗹𝗮𝗿 𝗧𝗵𝗲𝗿𝗮𝗽𝘆 𝗮𝗻𝗱 𝗖𝗮𝗻𝗰𝗲𝗿 𝗡𝗲𝘂𝗿𝗼𝗶𝗺𝗺𝘂𝗻𝗼𝗹𝗼𝗴𝘆 We are looking for a passionate and motivated PhD student to join an exciting project focused on how nerve, immune and cancer cell interactions shape metastatic tumor behavior and resistance to cellular therapies, especially CAR-T cells. The project combines: • Whole-body tissue clearing • Light-sheet microscopy • wildDISCO-based immunolabeling • CAR-T and immune cell tracking in metastatic cancer • Spatial proteomics and 3D spatial biology • AI-based analysis of large-scale imaging datasets This is a great opportunity for someone with a background in cancer biology, immunology, neuroscience, biomedical sciences, biomedical engineering or a related field who wants to work at the interface of experimental biology, advanced imaging and AI. Apply here: Please share with excellent candidates who may be interested. #Hiring #PhDPosition #CARTcells #CellularTherapy #CancerResearch #Neuroimmunology #Immunotherapy #Metastasis #SpatialBiology #AIinBiology #Microscopy #LMUMunich #HelmholtzMunich #ErturkLab

Ali Max Erturk

14,832 views • 4 months ago

📢 IntrinsiX: High-Quality PBR Generation using Image Priors 📢 From text input, we generate renderable PBR maps! Next to editable image generation, our predictions can be distilled into room-scale scenes using SDS for large-scale PBR texture generation. We first train separate LoRA modules for the intrinsic properties of albedo, rough/metal, normal. Then, we introduce cross-intrinsic attention using a rerendering loss with importance-weighted light sampling to enable coherent PBR generation. Our method outperforms text -> image -> PBR methods both in generalization and quality, since directly generating PBR maps does not suffer from the inherent ambiguity of intrinsic image decomposition. In addition, our design choice facilitates SDS-based PBR texture distillation. 🌍 🎥 Great work by Peter Kocsis, Lukas Höllein

📢 IntrinsiX: High-Quality PBR Generation using Image Priors 📢 From text input, we generate renderable PBR maps! Next to editable image generation, our predictions can be distilled into room-scale scenes using SDS for large-scale PBR texture generation. We first train separate LoRA modules for the intrinsic properties of albedo, rough/metal, normal. Then, we introduce cross-intrinsic attention using a rerendering loss with importance-weighted light sampling to enable coherent PBR generation. Our method outperforms text -> image -> PBR methods both in generalization and quality, since directly generating PBR maps does not suffer from the inherent ambiguity of intrinsic image decomposition. In addition, our design choice facilitates SDS-based PBR texture distillation. 🌍 🎥 Great work by Peter Kocsis, Lukas Höllein

Matthias Niessner

21,921 views • 1 year ago

Excited to announce AlphaEvolve A powerful AI coding agent developed by our team in Google DeepMind that is able to discover impactful new algorithms for important problems in Maths and Computing by combining the creativity of large language models with automated evaluators.

Excited to announce AlphaEvolve A powerful AI coding agent developed by our team in Google DeepMind that is able to discover impactful new algorithms for important problems in Maths and Computing by combining the creativity of large language models with automated evaluators.

Pushmeet Kohli

203,589 views • 1 year ago

🩺🚀 SUI DeSci Agents is excited to introduce the #AI Medical Image Diagnosis Agent, an integral part of our #AI Longevity Assistant! This cutting-edge tool, within our wide product suite, is designed to onboard millions of longevity enthusiasts into the SUI $DeSci ecosystem while driving organic growth for the AI agents launched on our platform. 📊 How it works: - Upload X-rays, CT scans, MRIs, ultrasounds, etc. - Let #AI deliver precise analysis with actionable insights. Accurate medical imaging analysis is key to longevity — and we’re leading the charge. Data will be sold to longevity initiatives and research institutions with $DESCI token lying at the center of it all.

🩺🚀 SUI DeSci Agents is excited to introduce the #AI Medical Image Diagnosis Agent, an integral part of our #AI Longevity Assistant! This cutting-edge tool, within our wide product suite, is designed to onboard millions of longevity enthusiasts into the SUI $DeSci ecosystem while driving organic growth for the AI agents launched on our platform. 📊 How it works: - Upload X-rays, CT scans, MRIs, ultrasounds, etc. - Let #AI deliver precise analysis with actionable insights. Accurate medical imaging analysis is key to longevity — and we’re leading the charge. Data will be sold to longevity initiatives and research institutions with $DESCI token lying at the center of it all.

SUI Desci Agents

160,680 views • 1 year ago

Boston Dynamics has released a new video of their next-generation humanoid robot, Atlas. "We are excited to share some of our progress on developing Large Behavior Models for Atlas. This work is part of a collaboration between AI research teams at Toyota Research Institute (TRI) and Boston Dynamics. We have been building end-to-end language-conditioned policies that enable Atlas to accomplish long-horizon manipulation tasks."

Boston Dynamics has released a new video of their next-generation humanoid robot, Atlas. "We are excited to share some of our progress on developing Large Behavior Models for Atlas. This work is part of a collaboration between AI research teams at Toyota Research Institute (TRI) and Boston Dynamics. We have been building end-to-end language-conditioned policies that enable Atlas to accomplish long-horizon manipulation tasks."

Sawyer Merritt

200,744 views • 1 year ago

Check out our #ECCV2026 paper "Low-latency Event-based Object Detection with Spatially-Sparse Linear Attention", where we make linear attention sparse in space, recurrent in time, and parallel in training, enabling the first purely-linear-attention-based neural network for asynchronous object detection with #EventCameras, outperforming the previous best asynchronous method with 20x less computation with truly event-by-event inference on CPU! Code released! Paper: Code: Video: Event cameras promise extremely low-latency vision, but to fully exploit them, the neural network must be low-latency too. We introduce #SpatiallySparseLinearAttention (#SSLA) for asynchronous object detection directly from raw events. Linear attention is particularly appealing for event cameras: it can be trained efficiently in parallel on long event sequences, while at inference it operates recurrently, updating its prediction every time a new event arrives. The problem is that conventional linear attention updates its entire state for every event. For object detection, where fine spatial resolution matters, this quickly becomes expensive. Our key idea is simple: an event only carries information about a small spatial region, so why update the entire spatial state? SSLA updates only the relevant parts of the state, enabling fine-grained spatial representations while keeping per-event computation low. We achieve: - >20× lower per-event computation than the strongest prior asynchronous baseline - State-of-the-art accuracy among asynchronous object detection methods - Truly event-by-event inference on CPU, designed to preserve the latency advantage of event cameras Come to our poster on Friday September 11, 2026 from 4-6pm at ExHall #389 Reference: Haiqing Hao, Zhipeng Sui, Rong Zou, Zijia Dai, Nikola Zubić, Davide Scaramuzza, Wenhui Wang Low-latency Event-based Object Detection with Spatially-Sparse Linear Attention ECCV, 2026 Prophesee SynSense University of Zurich UZH Science European Research Council (ERC) UZHai UZH IfI Tesla BYD #EventCameras #ComputerVision #Robotics #DeepLearning #NeuromorphicVision #AI

Check out our #ECCV2026 paper "Low-latency Event-based Object Detection with Spatially-Sparse Linear Attention", where we make linear attention sparse in space, recurrent in time, and parallel in training, enabling the first purely-linear-attention-based neural network for asynchronous object detection with #EventCameras, outperforming the previous best asynchronous method with 20x less computation with truly event-by-event inference on CPU! Code released! Paper: Code: Video: Event cameras promise extremely low-latency vision, but to fully exploit them, the neural network must be low-latency too. We introduce #SpatiallySparseLinearAttention (#SSLA) for asynchronous object detection directly from raw events. Linear attention is particularly appealing for event cameras: it can be trained efficiently in parallel on long event sequences, while at inference it operates recurrently, updating its prediction every time a new event arrives. The problem is that conventional linear attention updates its entire state for every event. For object detection, where fine spatial resolution matters, this quickly becomes expensive. Our key idea is simple: an event only carries information about a small spatial region, so why update the entire spatial state? SSLA updates only the relevant parts of the state, enabling fine-grained spatial representations while keeping per-event computation low. We achieve: - >20× lower per-event computation than the strongest prior asynchronous baseline - State-of-the-art accuracy among asynchronous object detection methods - Truly event-by-event inference on CPU, designed to preserve the latency advantage of event cameras Come to our poster on Friday September 11, 2026 from 4-6pm at ExHall #389 Reference: Haiqing Hao, Zhipeng Sui, Rong Zou, Zijia Dai, Nikola Zubić, Davide Scaramuzza, Wenhui Wang Low-latency Event-based Object Detection with Spatially-Sparse Linear Attention ECCV, 2026 Prophesee SynSense University of Zurich UZH Science European Research Council (ERC) UZHai UZH IfI Tesla BYD #EventCameras #ComputerVision #Robotics #DeepLearning #NeuromorphicVision #AI

Davide Scaramuzza

52,727 views • 1 month ago

🔬🤖Excited to share our new preprint on robust virtual staining of key organelles! ✨ Virtual staining and segmentation of nuclei and cell membranes in phase images can significantly accelerate image-based screens and developmental studies. 🔍 Current virtual staining models often struggle with nuisance variations in label-free input and don't generalize well to new cell types. We’re releasing our training protocols that lead to robust models and enable few-shot generalization. 🔧 Stay tuned for our revamped codebase with weights and example data, coming soon! 👏🧫🐟 Huge shoutout to Ziwen Liu, Eduardo Hirata-Miyasaki, my team Chan Zuckerberg Biohub Network (Soorya, Johanna, Christian, Talon Chandler, Ivan Ivanov), and the teams of Manuel Leonetti, Carolina Arias, and Adrian Jacobo for this fun collaboration! #VirtualStaining #DL #CellBiology #Zebrafish

🔬🤖Excited to share our new preprint on robust virtual staining of key organelles! ✨ Virtual staining and segmentation of nuclei and cell membranes in phase images can significantly accelerate image-based screens and developmental studies. 🔍 Current virtual staining models often struggle with nuisance variations in label-free input and don't generalize well to new cell types. We’re releasing our training protocols that lead to robust models and enable few-shot generalization. 🔧 Stay tuned for our revamped codebase with weights and example data, coming soon! 👏🧫🐟 Huge shoutout to Ziwen Liu, Eduardo Hirata-Miyasaki, my team Chan Zuckerberg Biohub Network (Soorya, Johanna, Christian, Talon Chandler, Ivan Ivanov), and the teams of Manuel Leonetti, Carolina Arias, and Adrian Jacobo for this fun collaboration! #VirtualStaining #DL #CellBiology #Zebrafish

Shalin Mehta

20,190 views • 2 years ago

4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models Contributions: • We introduce 4D LangSplat for open-vocabulary 4D spatial-temporal queries. To the best of our knowledge, we are the first to construct 4D language fields with object textual captions generated by MLLMs. • To model smooth transitions across states for objects in 4D scenes, we propose a status deformable network to capture continuous temporal changes. • Experiential results show that our method attains state-of-the-art performance for both time-agnostic and time-sensitive open-vocabulary queries.

4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large Language Models Contributions: • We introduce 4D LangSplat for open-vocabulary 4D spatial-temporal queries. To the best of our knowledge, we are the first to construct 4D language fields with object textual captions generated by MLLMs. • To model smooth transitions across states for objects in 4D scenes, we propose a status deformable network to capture continuous temporal changes. • Experiential results show that our method attains state-of-the-art performance for both time-agnostic and time-sensitive open-vocabulary queries.

MrNeRF

10,968 views • 1 year ago

🚀 Introducing scGPT-spatial! 🧬🌍 A game-changing spatial-omic foundation model, built on the powerful scGPT framework with MoE (mixture of experts) and continually pretrained on a massive 30 million spatial single-cell profiles! 🧠 What’s the challenge? Spatial transcriptomics is next-level complex—not only must we model single-cell/spot profiles, but we also need to capture intricate spatial relationships while handling diverse sequencing protocols (imaging-based vs. sequencing-based). 🔥 Why scGPT-spatial? ✨ A Spatial-omic Foundation Model with Continual Pretraining – Built on scGPT’s robust initialization, it unlocks spatial context in tissues. ✨ SpatialHuman30M Dataset – The largest curated dataset: 30M profiles from Visium, Visium HD, Xenium, and MERFISH across 821 slides. ✨ Revolutionary MoE Decoders – A cutting-edge Mixture of Experts (MoE) architecture for protocol-aware gene expression decoding. ✨ Spatially-Aware Training Strategy – A neighborhood-based masked reconstruction approach to capture complex cell-type colocalization. ✨ Multi-Modal & Multi-Slide Integration – Seamless clustering & spatial domain identification across slides and modalities. ✨ Cell-Type Deconvolution & Gene Imputation – Unlocks cross-resolution & cross-modality harmonization with fine-tuned embeddings. 📄 Read the preprint: 💻 Explore the code/weights: #SpatialTranscriptomics #SingleCell #AIResearch #MachineLearning #SpatialData Huge shoutout to the incredible PHD students Chloe (ChloeXWang) and Haotian (Haotian Cui) for leading this groundbreaking project! 🎉 Massive thanks to our amazing co-authors Andrew, Ronald, and Hani (Hani Goodarzi) from Arc Institute—this work wouldn't have been possible without you! 👏

🚀 Introducing scGPT-spatial! 🧬🌍 A game-changing spatial-omic foundation model, built on the powerful scGPT framework with MoE (mixture of experts) and continually pretrained on a massive 30 million spatial single-cell profiles! 🧠 What’s the challenge? Spatial transcriptomics is next-level complex—not only must we model single-cell/spot profiles, but we also need to capture intricate spatial relationships while handling diverse sequencing protocols (imaging-based vs. sequencing-based). 🔥 Why scGPT-spatial? ✨ A Spatial-omic Foundation Model with Continual Pretraining – Built on scGPT’s robust initialization, it unlocks spatial context in tissues. ✨ SpatialHuman30M Dataset – The largest curated dataset: 30M profiles from Visium, Visium HD, Xenium, and MERFISH across 821 slides. ✨ Revolutionary MoE Decoders – A cutting-edge Mixture of Experts (MoE) architecture for protocol-aware gene expression decoding. ✨ Spatially-Aware Training Strategy – A neighborhood-based masked reconstruction approach to capture complex cell-type colocalization. ✨ Multi-Modal & Multi-Slide Integration – Seamless clustering & spatial domain identification across slides and modalities. ✨ Cell-Type Deconvolution & Gene Imputation – Unlocks cross-resolution & cross-modality harmonization with fine-tuned embeddings. 📄 Read the preprint: 💻 Explore the code/weights: #SpatialTranscriptomics #SingleCell #AIResearch #MachineLearning #SpatialData Huge shoutout to the incredible PHD students Chloe (ChloeXWang) and Haotian (Haotian Cui) for leading this groundbreaking project! 🎉 Massive thanks to our amazing co-authors Andrew, Ronald, and Hani (Hani Goodarzi) from Arc Institute—this work wouldn't have been possible without you! 👏

Bo Wang

59,097 views • 1 year ago

We’re thrilled to share that our MERFISH+ preprint is now live on bioRxiv!👉 In this work, the Bintu and Zhu labs (UCSD) developed MERFISH+, a next-generation spatial genomics platform that combines genome-wide RNA and epigenetic imaging over a large field of view. By introducing acrydite-modified probes covalently anchored to hydrogels, MERFISH+ achieves remarkable imaging stability and enables >1,800-gene, multi-modal, and multi-month experiments. With this platform, they, together with the Chi lab at UCSD, profiled a whole developing human heart at 12 post-conception week with merely two slides, resulting in a total of 53 slides, 3.1 million single cells and more than 30 cell types. Building upon our previous 3D reconstruction and modeling framework, Spateo ( we reconstruct the 3D human heart that nicely captures the anatomical structure of the heart, including the intricate vasculature network. Sophisticated analyses provide a holistic view of an entire organ and enable systematic characterization of 3D cellular neighborhoods and transcriptional gradients of substructures such as the descending arteries. Furthermore, using a generative integration framework for spatial multimodal data (Spateo-VI), we harmonized these MERFISH+ transcriptomic and chromatin data to reconstruct a 3D spatially-resolved multi-omics atlas of the developing human heart, shared at and MERFISH+ thus sets a new standard for large-format, multi-omic spatial profiling, enabling holistic, 3D characterization of organs at subcellular resolution. Huge congratulations to first authors Colin Kern, qingquan Zhang, Yifan Lu , and Jacqueline Eschbach, and to all collaborators from the Bintu, Zhu, Chi, and Qiu labs for this amazing team effort. Thanks for your diligence, creativity, and hard work on this project. We’re grateful for support from Arc Institute and our generous donors. Our lab is expanding—if you’re excited about building the next generation of single-cell and spatial genomics techniques and predictive single cell and spatial foundation models, we’re hiring! If you are interested, please reach out to me via direct message or email at xiaojie@stanford.edu. We are excited for any potential collaborations along this line of research in Stanford, UCSF and Berkeley and other labs as well.

We’re thrilled to share that our MERFISH+ preprint is now live on bioRxiv!👉 In this work, the Bintu and Zhu labs (UCSD) developed MERFISH+, a next-generation spatial genomics platform that combines genome-wide RNA and epigenetic imaging over a large field of view. By introducing acrydite-modified probes covalently anchored to hydrogels, MERFISH+ achieves remarkable imaging stability and enables >1,800-gene, multi-modal, and multi-month experiments. With this platform, they, together with the Chi lab at UCSD, profiled a whole developing human heart at 12 post-conception week with merely two slides, resulting in a total of 53 slides, 3.1 million single cells and more than 30 cell types. Building upon our previous 3D reconstruction and modeling framework, Spateo ( we reconstruct the 3D human heart that nicely captures the anatomical structure of the heart, including the intricate vasculature network. Sophisticated analyses provide a holistic view of an entire organ and enable systematic characterization of 3D cellular neighborhoods and transcriptional gradients of substructures such as the descending arteries. Furthermore, using a generative integration framework for spatial multimodal data (Spateo-VI), we harmonized these MERFISH+ transcriptomic and chromatin data to reconstruct a 3D spatially-resolved multi-omics atlas of the developing human heart, shared at and MERFISH+ thus sets a new standard for large-format, multi-omic spatial profiling, enabling holistic, 3D characterization of organs at subcellular resolution. Huge congratulations to first authors Colin Kern, qingquan Zhang, Yifan Lu , and Jacqueline Eschbach, and to all collaborators from the Bintu, Zhu, Chi, and Qiu labs for this amazing team effort. Thanks for your diligence, creativity, and hard work on this project. We’re grateful for support from Arc Institute and our generous donors. Our lab is expanding—if you’re excited about building the next generation of single-cell and spatial genomics techniques and predictive single cell and spatial foundation models, we’re hiring! If you are interested, please reach out to me via direct message or email at [email protected]. We are excited for any potential collaborations along this line of research in Stanford, UCSF and Berkeley and other labs as well.

evo-devo

42,340 views • 10 months ago