Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

What if a foundation model could align histology, spatial biology & clinical data to reveal latent biomedical insights? 🚀 Introducing Haiku — a tri-modal foundation model trained on 26.7M+ spatial proteomics patches with matched H&E and clinical text, aligned in one shared embedding space. 📄 🧵👇

Zhi Huang

1,559 subscribers

13,358 görüntüleme • 3 ay önce •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

🚀 Introducing scGPT-spatial! 🧬🌍 A game-changing spatial-omic foundation model, built on the powerful scGPT framework with MoE (mixture of experts) and continually pretrained on a massive 30 million spatial single-cell profiles! 🧠 What’s the challenge? Spatial transcriptomics is next-level complex—not only must we model single-cell/spot profiles, but we also need to capture intricate spatial relationships while handling diverse sequencing protocols (imaging-based vs. sequencing-based). 🔥 Why scGPT-spatial? ✨ A Spatial-omic Foundation Model with Continual Pretraining – Built on scGPT’s robust initialization, it unlocks spatial context in tissues. ✨ SpatialHuman30M Dataset – The largest curated dataset: 30M profiles from Visium, Visium HD, Xenium, and MERFISH across 821 slides. ✨ Revolutionary MoE Decoders – A cutting-edge Mixture of Experts (MoE) architecture for protocol-aware gene expression decoding. ✨ Spatially-Aware Training Strategy – A neighborhood-based masked reconstruction approach to capture complex cell-type colocalization. ✨ Multi-Modal & Multi-Slide Integration – Seamless clustering & spatial domain identification across slides and modalities. ✨ Cell-Type Deconvolution & Gene Imputation – Unlocks cross-resolution & cross-modality harmonization with fine-tuned embeddings. 📄 Read the preprint: 💻 Explore the code/weights: #SpatialTranscriptomics #SingleCell #AIResearch #MachineLearning #SpatialData Huge shoutout to the incredible PHD students Chloe (ChloeXWang) and Haotian (Haotian Cui) for leading this groundbreaking project! 🎉 Massive thanks to our amazing co-authors Andrew, Ronald, and Hani (Hani Goodarzi) from Arc Institute—this work wouldn't have been possible without you! 👏

🚀 Introducing scGPT-spatial! 🧬🌍 A game-changing spatial-omic foundation model, built on the powerful scGPT framework with MoE (mixture of experts) and continually pretrained on a massive 30 million spatial single-cell profiles! 🧠 What’s the challenge? Spatial transcriptomics is next-level complex—not only must we model single-cell/spot profiles, but we also need to capture intricate spatial relationships while handling diverse sequencing protocols (imaging-based vs. sequencing-based). 🔥 Why scGPT-spatial? ✨ A Spatial-omic Foundation Model with Continual Pretraining – Built on scGPT’s robust initialization, it unlocks spatial context in tissues. ✨ SpatialHuman30M Dataset – The largest curated dataset: 30M profiles from Visium, Visium HD, Xenium, and MERFISH across 821 slides. ✨ Revolutionary MoE Decoders – A cutting-edge Mixture of Experts (MoE) architecture for protocol-aware gene expression decoding. ✨ Spatially-Aware Training Strategy – A neighborhood-based masked reconstruction approach to capture complex cell-type colocalization. ✨ Multi-Modal & Multi-Slide Integration – Seamless clustering & spatial domain identification across slides and modalities. ✨ Cell-Type Deconvolution & Gene Imputation – Unlocks cross-resolution & cross-modality harmonization with fine-tuned embeddings. 📄 Read the preprint: 💻 Explore the code/weights: #SpatialTranscriptomics #SingleCell #AIResearch #MachineLearning #SpatialData Huge shoutout to the incredible PHD students Chloe (ChloeXWang) and Haotian (Haotian Cui) for leading this groundbreaking project! 🎉 Massive thanks to our amazing co-authors Andrew, Ronald, and Hani (Hani Goodarzi) from Arc Institute—this work wouldn't have been possible without you! 👏

Bo Wang

59,073 görüntüleme • 1 yıl önce

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.

Google presents Still-Moving Customized Video Generation without Customized Video Data Customizing text-to-image (T2I) models has seen tremendous progress recently, particularly in areas such as personalization, stylization, and conditional generation. However, expanding this progress to video generation is still in its infancy, primarily due to the lack of customized video data. In this work, we introduce Still-Moving, a novel generic framework for customizing a text-to-video (T2V) model, without requiring any customized video data. The framework applies to the prominent T2V design where the video model is built over a text-to-image (T2I) model (e.g., via inflation). We assume access to a customized version of the T2I model, trained only on still image data (e.g., using DreamBooth or StyleDrop). Naively plugging in the weights of the customized T2I model into the T2V model often leads to significant artifacts or insufficient adherence to the customization data. To overcome this issue, we train lightweight Spatial Adapters that adjust the features produced by the injected T2I layers. Importantly, our adapters are trained on "frozen videos" (i.e., repeated images), constructed from image samples generated by the customized T2I model. This training is facilitated by a novel Motion Adapter module, which allows us to train on such static videos while preserving the motion prior of the video model. At test time, we remove the Motion Adapter modules and leave in only the trained Spatial Adapters. This restores the motion prior of the T2V model while adhering to the spatial prior of the customized T2I model. We demonstrate the effectiveness of our approach on diverse tasks including personalized, stylized, and conditional generation. In all evaluated scenarios, our method seamlessly integrates the spatial prior of the customized T2I model with a motion prior supplied by the T2V model.

AK

40,485 görüntüleme • 2 yıl önce

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI

Zhiyang (Frank) Dou

14,610 görüntüleme • 1 yıl önce

Humanoid robots shouldn't just follow pre-defined movements—they should perform! 🤖✨ Introducing UniAct: A unified model for multimodal motion generation and action streaming. Most humanoids are limited to pre-designed moves. UniAct changes the game by allowing robots to generate live action sequences from: 📝 Text instructions 🎶 Music rhythms 📉 Spatial trajectories 🔄 Cross-modal signals Whether it’s dancing to a beat or following a complex path, UniAct brings humanoid robots to life in real-time. 🚀 🔗 Project: 📄 Paper:

Humanoid robots shouldn't just follow pre-defined movements—they should perform! 🤖✨ Introducing UniAct: A unified model for multimodal motion generation and action streaming. Most humanoids are limited to pre-designed moves. UniAct changes the game by allowing robots to generate live action sequences from: 📝 Text instructions 🎶 Music rhythms 📉 Spatial trajectories 🔄 Cross-modal signals Whether it’s dancing to a beat or following a complex path, UniAct brings humanoid robots to life in real-time. 🚀 🔗 Project: 📄 Paper:

Siyuan Huang

10,592 görüntüleme • 7 ay önce

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

AK

684,372 görüntüleme • 2 yıl önce

Images, video, audio, actions — generative modeling has converged on one recipe: compress into continuous latents, generate in latent space, decode. Language is the lone exception: still generated token by token, as a long discrete stream. Latent Thought Flows: We compress 256 text tokens into 8 continuous latents, generate them with a one-step flow model, and read them out as text using an autoregressive decoder. Result: a better inference-compute vs. generation-quality Pareto frontier than a tuned autoregressive baseline — thanks to compression and one-step generation. Co-led w/ Zhengyang Geng 🧵

Images, video, audio, actions — generative modeling has converged on one recipe: compress into continuous latents, generate in latent space, decode. Language is the lone exception: still generated token by token, as a long discrete stream. Latent Thought Flows: We compress 256 text tokens into 8 continuous latents, generate them with a one-step flow model, and read them out as text using an autoregressive decoder. Result: a better inference-compute vs. generation-quality Pareto frontier than a tuned autoregressive baseline — thanks to compression and one-step generation. Co-led w/ Zhengyang Geng 🧵

Mihir Prabhudesai

80,875 görüntüleme • 1 ay önce

Introducing VL-JEPA: Vision-Language Joint Embedding Predictive Architecture for streaming, live action recognition, retrieval, VQA, and classification tasks with better performance and higher efficiency than large VLMs. • VL-JEPA is the first non-generative model that can perform general-domain vision-language tasks in real-time, built on a joint embedding predictive architecture. • We demonstrate in controlled experiments that VL-JEPA, trained with latent space embedding prediction, outperforms VLMs that rely on data space token prediction. • We show that VL-JEPA delivers significant efficiency gains over VLMs for online video streaming applications, thanks to its non-autoregressive design and native support for selective decoding. • We highlight that our VL-JEPA model, with an unified model architecture, can effectively handle a wide range of classification, retrieval, and VQA tasks at the same time. by Delong Chen (陈德龙) Mustafa Shukor Théo Moutakanni Willy Jade Lei Yu Tejaswi Kasarla Allen Bolourchi Yann LeCun Pascale Fung

Introducing VL-JEPA: Vision-Language Joint Embedding Predictive Architecture for streaming, live action recognition, retrieval, VQA, and classification tasks with better performance and higher efficiency than large VLMs. • VL-JEPA is the first non-generative model that can perform general-domain vision-language tasks in real-time, built on a joint embedding predictive architecture. • We demonstrate in controlled experiments that VL-JEPA, trained with latent space embedding prediction, outperforms VLMs that rely on data space token prediction. • We show that VL-JEPA delivers significant efficiency gains over VLMs for online video streaming applications, thanks to its non-autoregressive design and native support for selective decoding. • We highlight that our VL-JEPA model, with an unified model architecture, can effectively handle a wide range of classification, retrieval, and VQA tasks at the same time. by Delong Chen (陈德龙) Mustafa Shukor Théo Moutakanni Willy Jade Lei Yu Tejaswi Kasarla Allen Bolourchi Yann LeCun Pascale Fung

Pascale Fung

90,144 görüntüleme • 8 ay önce

🔬 Exciting News! Our manuscript, "scGPT: toward building a foundation model for single-cell multi-omics using generative AI" is now finally published in Nature Methods (Nature Methods) 🎉 !!! (Re-)Introducing scGPT: A transformative foundation model engineered for single-cell omics analysis. Developed through the analysis of over 33 million human cells, scGPT sets a new benchmark for application versatility, offering both fine-tuning and zero-shot capabilities. Since its preprint in May 2023, scGPT has significantly impacted the field, evidenced by 13K+ installations, 600+ GitHub stars 🌟, and 40+ citations before its official publication! scGPT has been validated by numerous benchmark studies as a leading foundation model in single-cell analysis. Its pre-trained embeddings extend its utility beyond single-cell studies, enhancing a variety of downstream tasks including protein enrichment and genetic perturbation predictions. Some key updates lately: ---Expanded zero-shot applications for efficient reference mapping and integration, now with CellXGene census integration. ---Advanced perturbation analysis capabilities, including genome-scale perturb-seq data analysis and bulk sequencing data generalization. ---Upgraded scGPT package, offering versatile model loading compatible with PyTorch and flash-attn, for both GPU and CPU. ---Cloud-based scGPT applications for reference mapping, cell annotation, and gene regulatory network inference are available on ---Integration with Hugging Face for easier model training. Limitations: scGPT is an early foray into foundation models for single-cell omics, facing challenges like limited zero-shot learning in some tasks, pretraining constraints, data quality issues, and evaluation limitations. See our Supplementary Notes for details. 🚀 Future Work? Short-Term Goals: 1. Releasing a Mouse Model for broader analysis. 2. Developing a comprehensive evaluation suite for foundation models in single-cell analysis. 3. Creating a foundation model for single-cell spatial omics. 4. Enhancing zero-shot capacity by integrating scGPT with RAG (e.g., knowledge graphs). Long-Term Goals: 1. Expanding scGPT for comprehensive single-cell multi-omics analysis. 2. Developing an in-silico perturbation model for predicting genetic perturbation effects. 3. Merging scGPT with multi-modal genomic sequence models for a deeper understanding of cell biology. 📚 Access the paper on Nature Methods: 🔬Preprint in Bioarixv: 💻 All our codes/data/weights are open source: Wholehearted congratulations to all the authors, especially the two co-first authors, Haotian (Haotian Cui ) and Chloe (ChloeXWang), who are really the emerging superstars in AI and biology! Vector Institute Peter Munk Cardiac Centre AI U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology University Health Network University of Toronto #scGPT #GenerativeAI #AI4Science #Combio #opensource

🔬 Exciting News! Our manuscript, "scGPT: toward building a foundation model for single-cell multi-omics using generative AI" is now finally published in Nature Methods (Nature Methods) 🎉 !!! (Re-)Introducing scGPT: A transformative foundation model engineered for single-cell omics analysis. Developed through the analysis of over 33 million human cells, scGPT sets a new benchmark for application versatility, offering both fine-tuning and zero-shot capabilities. Since its preprint in May 2023, scGPT has significantly impacted the field, evidenced by 13K+ installations, 600+ GitHub stars 🌟, and 40+ citations before its official publication! scGPT has been validated by numerous benchmark studies as a leading foundation model in single-cell analysis. Its pre-trained embeddings extend its utility beyond single-cell studies, enhancing a variety of downstream tasks including protein enrichment and genetic perturbation predictions. Some key updates lately: ---Expanded zero-shot applications for efficient reference mapping and integration, now with CellXGene census integration. ---Advanced perturbation analysis capabilities, including genome-scale perturb-seq data analysis and bulk sequencing data generalization. ---Upgraded scGPT package, offering versatile model loading compatible with PyTorch and flash-attn, for both GPU and CPU. ---Cloud-based scGPT applications for reference mapping, cell annotation, and gene regulatory network inference are available on ---Integration with Hugging Face for easier model training. Limitations: scGPT is an early foray into foundation models for single-cell omics, facing challenges like limited zero-shot learning in some tasks, pretraining constraints, data quality issues, and evaluation limitations. See our Supplementary Notes for details. 🚀 Future Work? Short-Term Goals: 1. Releasing a Mouse Model for broader analysis. 2. Developing a comprehensive evaluation suite for foundation models in single-cell analysis. 3. Creating a foundation model for single-cell spatial omics. 4. Enhancing zero-shot capacity by integrating scGPT with RAG (e.g., knowledge graphs). Long-Term Goals: 1. Expanding scGPT for comprehensive single-cell multi-omics analysis. 2. Developing an in-silico perturbation model for predicting genetic perturbation effects. 3. Merging scGPT with multi-modal genomic sequence models for a deeper understanding of cell biology. 📚 Access the paper on Nature Methods: 🔬Preprint in Bioarixv: 💻 All our codes/data/weights are open source: Wholehearted congratulations to all the authors, especially the two co-first authors, Haotian (Haotian Cui ) and Chloe (ChloeXWang), who are really the emerging superstars in AI and biology! Vector Institute Peter Munk Cardiac Centre AI U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology University Health Network University of Toronto #scGPT #GenerativeAI #AI4Science #Combio #opensource

Bo Wang

199,747 görüntüleme • 2 yıl önce

4 billion payments. 3 trillion data points. One model trained on all of it. Meet Razorpay Vulcan - India’s first transformer-based AI foundation model for payments. Built, trained and hosted in India, in partnership with NVIDIA and Amazon Web Services. Until now, every payments problem was solved separately: routing, fraud, risk, personalisation and more. We asked: What if one model could understand how money moves? And like LLMs are trained on text to understand language, Vulcan is trained on payments to understand how money moves. Already running in beta across 51,000+ businesses, Vulcan is delivering: - 8–10% improvement in payment success rates - the ultimate measure of whether a payment simply works. - 8x more international card fraud detected. - 5x more fraudulent or disputed transactions identified. - 1–2 lakh more purchases completed every month through better checkout personalisation. And we’re just getting started. The best part? Every payment Vulcan sees makes the next one smarter. We’ve spent years building the infrastructure that moves money for India. Now, we’re building the intelligence that understands and improves it. 🔥

4 billion payments. 3 trillion data points. One model trained on all of it. Meet Razorpay Vulcan - India’s first transformer-based AI foundation model for payments. Built, trained and hosted in India, in partnership with NVIDIA and Amazon Web Services. Until now, every payments problem was solved separately: routing, fraud, risk, personalisation and more. We asked: What if one model could understand how money moves? And like LLMs are trained on text to understand language, Vulcan is trained on payments to understand how money moves. Already running in beta across 51,000+ businesses, Vulcan is delivering: - 8–10% improvement in payment success rates - the ultimate measure of whether a payment simply works. - 8x more international card fraud detected. - 5x more fraudulent or disputed transactions identified. - 1–2 lakh more purchases completed every month through better checkout personalisation. And we’re just getting started. The best part? Every payment Vulcan sees makes the next one smarter. We’ve spent years building the infrastructure that moves money for India. Now, we’re building the intelligence that understands and improves it. 🔥

Harshil Mathur

556,882 görüntüleme • 10 gün önce

Do Vision-Language Models represent space, and how? Spatial terms like "left" or "right" may not be enough to match images with spatial descriptions, as we often overlook the different frames of reference (FoR) used by speakers and listeners. See Figure 1 for examples! Introducing the COnsistent Multilingual Frame Of Reference Test (COMFORT), an evaluation protocol to assess the spatial reasoning capabilities of VLMs. COMFORT includes systematically designed datasets and metrics that evaluate model performance, and their deeper linguistic competence, specifically the spatial knowledge encoded in their internal representations. Find out more in the video teaser! Almost all VLMs prefer the egocentric relative FoR with reflected transform, similar to English. Yet, we reveal significant shortcomings of VLMs: notably, the models (1) exhibit poor robustness and consistency, (2) lack the flexibility to accommodate multiple FoRs, and (3) fail to adhere to language-specific or culture-specific conventions in cross-lingual tests, as English tends to dominate other languages. A shortened version will appear in Pluralistic Alignment Workshop Pluralistic Alignment Workshop #NeurIPS2024. It seems that the ArXiv moderators put it on hold and are eager to give it a thorough read first🤣! So here is the Paper/Code/Data: This collaboration turns out to be amazing, jointly led by Brian Zheyuan Zhang, @Hu_FY_ Jayjun Lee, with so many contributions and insights from Freda Shi, Parisa Kordjamshidi Michigan SLED Lab. With a growing effort to align vision-language models with human cognitive intuitions, we call for more attention to the ambiguous nature and cross-cultural diversity of spatial reasoning!

Do Vision-Language Models represent space, and how? Spatial terms like "left" or "right" may not be enough to match images with spatial descriptions, as we often overlook the different frames of reference (FoR) used by speakers and listeners. See Figure 1 for examples! Introducing the COnsistent Multilingual Frame Of Reference Test (COMFORT), an evaluation protocol to assess the spatial reasoning capabilities of VLMs. COMFORT includes systematically designed datasets and metrics that evaluate model performance, and their deeper linguistic competence, specifically the spatial knowledge encoded in their internal representations. Find out more in the video teaser! Almost all VLMs prefer the egocentric relative FoR with reflected transform, similar to English. Yet, we reveal significant shortcomings of VLMs: notably, the models (1) exhibit poor robustness and consistency, (2) lack the flexibility to accommodate multiple FoRs, and (3) fail to adhere to language-specific or culture-specific conventions in cross-lingual tests, as English tends to dominate other languages. A shortened version will appear in Pluralistic Alignment Workshop Pluralistic Alignment Workshop #NeurIPS2024. It seems that the ArXiv moderators put it on hold and are eager to give it a thorough read first🤣! So here is the Paper/Code/Data: This collaboration turns out to be amazing, jointly led by Brian Zheyuan Zhang, @Hu_FY_ Jayjun Lee, with so many contributions and insights from Freda Shi, Parisa Kordjamshidi Michigan SLED Lab. With a growing effort to align vision-language models with human cognitive intuitions, we call for more attention to the ambiguous nature and cross-cultural diversity of spatial reasoning!

Martin Ziqiao Ma

35,565 görüntüleme • 1 yıl önce

Spatial understanding is important to moving around in complex environments and is a huge part of the challenge of generalizing to new scenes. Most world models, however, largely ignore this spatial dimension, focusing on 2D images. Not PointWorld, though. PointWorld is a 3D world model trained from real and simulated data which can perform a wide variety of manipulation tasks on a real robot, including grasping or handling articulated objects, all without any additional fine tuning. Wenlong Huang joins us to tell us more about what makes this work and how it’s different from other world models. Watch Episode #83 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

Spatial understanding is important to moving around in complex environments and is a huge part of the challenge of generalizing to new scenes. Most world models, however, largely ignore this spatial dimension, focusing on 2D images. Not PointWorld, though. PointWorld is a 3D world model trained from real and simulated data which can perform a wide variety of manipulation tasks on a real robot, including grasping or handling articulated objects, all without any additional fine tuning. Wenlong Huang joins us to tell us more about what makes this work and how it’s different from other world models. Watch Episode #83 of RoboPapers, with Chris Paxton and Jiafei Duan, to learn more!

RoboPapers

16,197 görüntüleme • 3 ay önce

$Introducing HRM-Text. An ultra-lean 1B-parameter reasoning language model designed to deliver strong general performance with a fraction of the data, compute, and infrastructure. Trained on just 40B structured tokens, HRM-Text achieves competitive performance while using ~1/1000 of the training data of comparable models. The kicker? The full model trains in roughly one day on a $1,000 budget. This opens the door to a new generation of AI that is powerful, accessible, and radically easier to adapt. Theories and research concepts once deemed too expensive to test are officially back in the game. Sapient Intelligence invites you to help us shape a new paradigm for general intelligence.$

Introducing HRM-Text. An ultra-lean 1B-parameter reasoning language model designed to deliver strong general performance with a fraction of the data, compute, and infrastructure. Trained on just 40B structured tokens, HRM-Text achieves competitive performance while using ~1/1000 of the training data of comparable models. The kicker? The full model trains in roughly one day on a $1,000 budget. This opens the door to a new generation of AI that is powerful, accessible, and radically easier to adapt. Theories and research concepts once deemed too expensive to test are officially back in the game. Sapient Intelligence invites you to help us shape a new paradigm for general intelligence.

Sapient Intelligence

516,561 görüntüleme • 3 ay önce

Welcome to the Lab of the Future! 🧬🤖 Excited to share LUMI-lab, out today in Cell — a self-driving platform that pairs an AI foundation model with a robotic lab to autonomously discover ionizable lipids (LNPs) for mRNA delivery. The core problem: Designing lipid nanoparticles (LNPs) is hard. The chemical space of ionizable lipids is vast, experimental cycles are slow, and — critically — historical LNP datasets are far too small to train a predictive model from scratch. Most AI approaches in this space hit a wall immediately: not enough data to learn from. Our solution: lab-in-the-loop foundation model learning. Instead of training on LNP data alone, LUMI starts as a transformer-based foundation model pretrained across broad chemical space, building rich molecular representations before it ever sees a single LNP experiment. Then it enters a closed loop with a robotic synthesis platform: predict → synthesize → assay → update. Each round of real wet-lab experiments fine-tunes the model, which then proposes smarter candidates for the next round. The lab isn't just validating AI predictions — it's actively teaching the model, continuously. What happened when we let it run: LUMI-lab autonomously synthesized and screened 1,700+ ionizable lipids in human bronchial epithelial cells. The top candidate — LUMI-6 — features a brominated lipid tail, a structural motif that had been largely overlooked in LNP design. LUMI found it without being told where to look. When formulated into LNPs and delivered intratracheally to mice, LUMI-6 achieved 20.3% gene editing efficiency in lung epithelial cells — a compelling result for one of the hardest-to-reach therapeutic targets, directly relevant to diseases like cystic fibrosis and alpha-1 antitrypsin deficiency. Why this matters beyond LNPs: This is a proof of concept for a broader thesis — that foundation model pretraining + active learning + robotic experimentation can overcome the data scarcity bottleneck that plagues AI-driven discovery in biology. You don't need a massive domain-specific dataset to start. You need a model that can generalize, a lab that can generate the right data, and a loop that connects them. Huge congratulations to first authors Yue Xu, Haotian Cui, and Kuan Pang, and to the entire Bowen LI team. Grateful to our collaborators at University Health Network and Leslie Dan Faculty of Pharmacy, and to Princess Margaret Cancer Centre Research Princess Margaret Cancer Centre Research. 📄 Paper:

Welcome to the Lab of the Future! 🧬🤖 Excited to share LUMI-lab, out today in Cell — a self-driving platform that pairs an AI foundation model with a robotic lab to autonomously discover ionizable lipids (LNPs) for mRNA delivery. The core problem: Designing lipid nanoparticles (LNPs) is hard. The chemical space of ionizable lipids is vast, experimental cycles are slow, and — critically — historical LNP datasets are far too small to train a predictive model from scratch. Most AI approaches in this space hit a wall immediately: not enough data to learn from. Our solution: lab-in-the-loop foundation model learning. Instead of training on LNP data alone, LUMI starts as a transformer-based foundation model pretrained across broad chemical space, building rich molecular representations before it ever sees a single LNP experiment. Then it enters a closed loop with a robotic synthesis platform: predict → synthesize → assay → update. Each round of real wet-lab experiments fine-tunes the model, which then proposes smarter candidates for the next round. The lab isn't just validating AI predictions — it's actively teaching the model, continuously. What happened when we let it run: LUMI-lab autonomously synthesized and screened 1,700+ ionizable lipids in human bronchial epithelial cells. The top candidate — LUMI-6 — features a brominated lipid tail, a structural motif that had been largely overlooked in LNP design. LUMI found it without being told where to look. When formulated into LNPs and delivered intratracheally to mice, LUMI-6 achieved 20.3% gene editing efficiency in lung epithelial cells — a compelling result for one of the hardest-to-reach therapeutic targets, directly relevant to diseases like cystic fibrosis and alpha-1 antitrypsin deficiency. Why this matters beyond LNPs: This is a proof of concept for a broader thesis — that foundation model pretraining + active learning + robotic experimentation can overcome the data scarcity bottleneck that plagues AI-driven discovery in biology. You don't need a massive domain-specific dataset to start. You need a model that can generalize, a lab that can generate the right data, and a loop that connects them. Huge congratulations to first authors Yue Xu, Haotian Cui, and Kuan Pang, and to the entire Bowen LI team. Grateful to our collaborators at University Health Network and Leslie Dan Faculty of Pharmacy, and to Princess Margaret Cancer Centre Research Princess Margaret Cancer Centre Research. 📄 Paper:

Bo Wang

57,510 görüntüleme • 6 ay önce

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models

AK

62,719 görüntüleme • 1 yıl önce

Love seeing Silico (Goodfire ) used to probe our EchoJEPA's representations! this is exactly the kind of interpretability work that's been missing for JEPA-style models. One thing that makes EchoJEPA particularly interesting to interpret: unlike MAE-based approaches, it never reconstructs pixels. The model learns entirely in latent space through masked prediction, so you can't just look at decoder outputs to understand what it captured. Attribution onto a temporally aligned 3D mesh is a much more honest probe of what the representations actually encode. What we found in building EchoJEPA: training on 18M echo videos across 300K patients, the model learns to disentangle cardiac anatomy from ultrasound noise (speckle, reverberation artifacts) almost entirely through self-supervision. With 1% labeled data it already outperforms supervised baselines trained on 100%. The latent space is doing real anatomical work, but until you can visualize it like this, "real anatomical work" is mostly a claim. Paper + code: |

Love seeing Silico (Goodfire ) used to probe our EchoJEPA's representations! this is exactly the kind of interpretability work that's been missing for JEPA-style models. One thing that makes EchoJEPA particularly interesting to interpret: unlike MAE-based approaches, it never reconstructs pixels. The model learns entirely in latent space through masked prediction, so you can't just look at decoder outputs to understand what it captured. Attribution onto a temporally aligned 3D mesh is a much more honest probe of what the representations actually encode. What we found in building EchoJEPA: training on 18M echo videos across 300K patients, the model learns to disentangle cardiac anatomy from ultrasound noise (speckle, reverberation artifacts) almost entirely through self-supervision. With 1% labeled data it already outperforms supervised baselines trained on 100%. The latent space is doing real anatomical work, but until you can visualize it like this, "real anatomical work" is mostly a claim. Paper + code: |

Bo Wang

29,611 görüntüleme • 4 ay önce

Building an Artificial Mechanical Engineer: All of the "AI CAD" demos you've seen have been essentially the same thing, a frontier lab LLM writing code that is translated by an existing geometry kernel into a 3d B-rep model. I think this approach is fine, but generally lossy, and lacks true spatial understanding and geometric reasoning that humans naturally posses and apply when encoding their designs in a CAD system. So I attempted to do something unique in the generative design space, building off of other generative 3d research, and trained my own (two tiny, 16M Parameter) AI models on a hyper specific set of data. Introducing FieldGeo - a set of models that encode geometry, physics (FEA, Stress) and DFM into the same latent space, and reproduces (assisted with an additional design agent) geometries to meet the stated cases. Today, it's just for L shaped brackets.

Building an Artificial Mechanical Engineer: All of the "AI CAD" demos you've seen have been essentially the same thing, a frontier lab LLM writing code that is translated by an existing geometry kernel into a 3d B-rep model. I think this approach is fine, but generally lossy, and lacks true spatial understanding and geometric reasoning that humans naturally posses and apply when encoding their designs in a CAD system. So I attempted to do something unique in the generative design space, building off of other generative 3d research, and trained my own (two tiny, 16M Parameter) AI models on a hyper specific set of data. Introducing FieldGeo - a set of models that encode geometry, physics (FEA, Stress) and DFM into the same latent space, and reproduces (assisted with an additional design agent) geometries to meet the stated cases. Today, it's just for L shaped brackets.

Connor Kapoor

82,679 görüntüleme • 5 gün önce

🚀 We're thrilled to introduce Orthrus 🧬🐕—a groundbreaking mature RNA foundation model designed to push the boundaries of RNA property prediction! 🔬 What is Orthrus? Orthrus is a Mamba-based RNA foundation model, pre-trained using a novel self-supervised contrastive learning objective with biologically inspired augmentations. It optimizes the similarity between splicing isoforms and orthologous transcripts, capturing functional and evolutionary relationships to enhance mature RNA property prediction accuracy. 📑 Preprint: 💻 Code: 🌐 Project Page: 📦 Model Weights: 🧠 Why Orthrus? Decoding the RNA regulatory code is key to understanding biology, but traditional experimental approaches are slow and costly. Existing genomic foundation models rely on techniques like masked language modeling or next-token prediction, which aren't fully aligned with the complexities of genomic data—leading to suboptimal results. 🌟 Orthrus Highlights: - Biologically-Informed Contrastive Learning 🧪: A novel contrastive learning objective designed specifically for genomics, maximizing similarity between splicing isoforms and orthologous transcripts across species. - Extensive Pre-training 📊: Trained on splicing annotations from 10 species and orthologous alignments from 400+ mammalian species (Zoonomia Project), with a focus on sequences of high functional importance. - Superior Representations🏅: Orthrus outperforms existing genomic models on 5 mRNA property prediction tasks, often surpassing supervised methods with just a simple linear transformation. - Efficiency in Low-Data Settings📉: Orthrus excels in low-data regimes, achieving state-of-the-art results with as few as 45 labeled examples for fine-tuning on RNA half-life prediction. Shoutout to the amazing leading authors Phil (Phil Fradkin) and Ian (Ian Shi)! Also the work is impossible without an outstanding collaboration by Karina (Karin(a) Isaev), Brendan (Brendan Frey) , Quaid (Quaid Morris), Leo J. Lee! Vector Institute University Health Network U of T Department of Computer Science Temerty Centre for AI in Medicine (T-CAIREM) Department of Laboratory Medicine & Pathobiology

🚀 We're thrilled to introduce Orthrus 🧬🐕—a groundbreaking mature RNA foundation model designed to push the boundaries of RNA property prediction! 🔬 What is Orthrus? Orthrus is a Mamba-based RNA foundation model, pre-trained using a novel self-supervised contrastive learning objective with biologically inspired augmentations. It optimizes the similarity between splicing isoforms and orthologous transcripts, capturing functional and evolutionary relationships to enhance mature RNA property prediction accuracy. 📑 Preprint: 💻 Code: 🌐 Project Page: 📦 Model Weights: 🧠 Why Orthrus? Decoding the RNA regulatory code is key to understanding biology, but traditional experimental approaches are slow and costly. Existing genomic foundation models rely on techniques like masked language modeling or next-token prediction, which aren't fully aligned with the complexities of genomic data—leading to suboptimal results. 🌟 Orthrus Highlights: - Biologically-Informed Contrastive Learning 🧪: A novel contrastive learning objective designed specifically for genomics, maximizing similarity between splicing isoforms and orthologous transcripts across species. - Extensive Pre-training 📊: Trained on splicing annotations from 10 species and orthologous alignments from 400+ mammalian species (Zoonomia Project), with a focus on sequences of high functional importance. - Superior Representations🏅: Orthrus outperforms existing genomic models on 5 mRNA property prediction tasks, often surpassing supervised methods with just a simple linear transformation. - Efficiency in Low-Data Settings📉: Orthrus excels in low-data regimes, achieving state-of-the-art results with as few as 45 labeled examples for fine-tuning on RNA half-life prediction. Shoutout to the amazing leading authors Phil (Phil Fradkin) and Ian (Ian Shi)! Also the work is impossible without an outstanding collaboration by Karina (Karin(a) Isaev), Brendan (Brendan Frey) , Quaid (Quaid Morris), Leo J. Lee! Vector Institute University Health Network U of T Department of Computer Science Temerty Centre for AI in Medicine (T-CAIREM) Department of Laboratory Medicine & Pathobiology

Bo Wang

114,983 görüntüleme • 1 yıl önce

We’re excited to announce the release and open-source of HunyuanImage 3.0 — the largest and most powerful open-source text-to-image model to date, with over 80 billion total parameters, of which 13 billion are activated per token during inference.The effect is completely comparable to the industry’s flagship closed-source model.🚀🚀🚀 HunyuanImage 3.0 originates from our internally developed native multimodal large language model, with fine-tuning and post-training focused on text-to-image generation. This unique foundation gives the model a powerful set of capabilities: ✅Reason with world knowledge ✅Understand complex, thousand-word prompts ✅Generate precise text within images Different from traditional DiT architecture image generation models, HunyuanImage 3.0’s MoE architecture uses a Transfusion-based approach to deeply couple Diffusion and LLM training for a single, powerful system. Built on Hunyuan-A13B, HunyuanImage 3.0 was trained on a massive dataset: 5 billion image-text pairs, video frames, interleaved image-text data, and 6 trillion tokens of text corpora. This hybrid training across multimodal generation, understanding, and LLM capabilities allows the model to seamlessly integrate multiple tasks. Whether you're an illustrator, designer, or creator, this is built to slash your workflow from hours to minutes. HunyuanImage 3.0 can generate intricate text, detailed comics, expressive emojis, and lively, engaging illustrations for educational content. The current release focuses solely on text-to-image generation and future updates will include image-to-image, image editing, multi-turn interaction, and more. 👉🏻Try it now: 🔗GitHub: 🤗Hugging Face:

We’re excited to announce the release and open-source of HunyuanImage 3.0 — the largest and most powerful open-source text-to-image model to date, with over 80 billion total parameters, of which 13 billion are activated per token during inference.The effect is completely comparable to the industry’s flagship closed-source model.🚀🚀🚀 HunyuanImage 3.0 originates from our internally developed native multimodal large language model, with fine-tuning and post-training focused on text-to-image generation. This unique foundation gives the model a powerful set of capabilities: ✅Reason with world knowledge ✅Understand complex, thousand-word prompts ✅Generate precise text within images Different from traditional DiT architecture image generation models, HunyuanImage 3.0’s MoE architecture uses a Transfusion-based approach to deeply couple Diffusion and LLM training for a single, powerful system. Built on Hunyuan-A13B, HunyuanImage 3.0 was trained on a massive dataset: 5 billion image-text pairs, video frames, interleaved image-text data, and 6 trillion tokens of text corpora. This hybrid training across multimodal generation, understanding, and LLM capabilities allows the model to seamlessly integrate multiple tasks. Whether you're an illustrator, designer, or creator, this is built to slash your workflow from hours to minutes. HunyuanImage 3.0 can generate intricate text, detailed comics, expressive emojis, and lively, engaging illustrations for educational content. The current release focuses solely on text-to-image generation and future updates will include image-to-image, image editing, multi-turn interaction, and more. 👉🏻Try it now: 🔗GitHub: 🤗Hugging Face:

Tencent Hy

412,880 görüntüleme • 11 ay önce

Google DeepMind just dropped 'Genie', an AI that can generate interactive video games. This is a huge deal. Genie is trained on 200,000 hours of unsupervised public internet gaming videos and can generate video games from a single prompt or image. But here's what's insane: Despite not being trained on action or text annotations, the foundation model can determine who the main character is and enable a user to control that character in the generated world. It does this through its Latent Action Model, Video Tokenizer, and Dynamics Model (will go more in-depth on this in tomorrow's newsletter for those interested). And for those asking, yes, it's research-only and not publicly available (here come the Google memes), and it does come with some limitations, like only currently creating games at 1FPS. But this is the worst AI will ever be. Anyone will be able to create their own entirely imagined virtual worlds soon, and that's a wild sentence to say out loud.

Google DeepMind just dropped 'Genie', an AI that can generate interactive video games. This is a huge deal. Genie is trained on 200,000 hours of unsupervised public internet gaming videos and can generate video games from a single prompt or image. But here's what's insane: Despite not being trained on action or text annotations, the foundation model can determine who the main character is and enable a user to control that character in the generated world. It does this through its Latent Action Model, Video Tokenizer, and Dynamics Model (will go more in-depth on this in tomorrow's newsletter for those interested). And for those asking, yes, it's research-only and not publicly available (here come the Google memes), and it does come with some limitations, like only currently creating games at 1FPS. But this is the worst AI will ever be. Anyone will be able to create their own entirely imagined virtual worlds soon, and that's a wild sentence to say out loud.

Rowan Cheung

3,377,539 görüntüleme • 2 yıl önce

Excited to share TERRA, a tissue world model 🧬 Over ~1.5 years we ran a large data-generation + modelling effort to build a world model for human tissues, pretrained on 112M cells from spatial transcriptomics (mostly Xenium 5000-plex + public data). It's built on one of the largest human spatial transcriptomics corpora assembled to date, spanning 20 tissues across development, health and 26 disease conditions, ~two-thirds newly generated in-house. Why a "world model" for tissue? Images have universal representations (ViT/DINOv3), so do proteins (ESM, Alex Rives) and pathology (UNI, Faisal Mahmood). We've worked hard to build something similar for human tissue: one model that captures its multi-scale logic, genes → cells → their native microenvironments. Like the JEPA approach Yann LeCun has championed, TERRA learns by prediction in embedding space, but for human tissue. How it works: it tokenises each cell together with its nearest neighbours into one sequence while keeping every gene's identity, then masks part of a neighbourhood and predicts the representation of the hidden part, not raw noisy counts. From one backbone it reads out three scales, gene embeddings (what a gene is doing in a cell and its niche), cell embeddings (cell type and state) and neighbourhood embeddings (the niche), and because it keeps gene-level resolution it can knock a gene out in silico and predict the response. Applied entirely zero-shot, TERRA maps and perturbs human tissue across unseen organs, diseases and technologies, outperforming existing spatial approaches. Three take-homes: 1️⃣ One model, any tissue. A single pretrained backbone provides tissue representations zero-shot, handling genes, cells and niches across organs and platforms, off the shelf. 2️⃣ New biology, development to clinic. We built a new spatial atlas of the developing human pancreas and found an islet-associated capillary state that looks like a precursor of mature islet vasculature. In kidney, TERRA's in silico knockouts predicted the tissue-injury programme from cancer immunotherapy (checkpoint blockade), confirmed in treated kidneys, detected in blood, and linked to declining kidney function. 3️⃣ A grammar of tissue architecture. By coupling each cell's state to its niche, TERRA defines recurring cross-organ "archetypes" of macrophage neighbourhoods, including a tumour-boundary niche that tracks poor survival in kidney cancer. TERRA is already in use: it powered our recent skin atlas of hidden immune-memory niches ( with more studies coming soon. This was an amazing collaboration between clinicians, machine-learning scientists and cell biologists 🙏 Led by Sebastian Birk, Vali Sanian Amirhosein Vahidi, Samuel Ogden, Daniyal Jafree, Adib Miraki, Carlo Leonardi and Arpit Merchant, with Lassi Paavolainen, Menna Clatworthy, Omer Ali Bayraktar, Muzlifah Haniffa, Tom Mitchell and Mostafa Bakhti. Huge thanks too to everyone who shared data and helped along the way. What excites me most is seeing how the community builds on this. The model, code and tutorials are all public, so anyone can run TERRA on their own tissues, extend it, or build new models on top. Huge thanks to the whole team across Wellcome Sanger Institute and our many collaborators. 📄 Paper: 💻 Code: 🤗 Model: #SpatialTranscriptomics #SpatialGenomics #FoundationModels #AI4Science #MachineLearning #ComputationalBiology #SingleCell #WorldModels

Excited to share TERRA, a tissue world model 🧬 Over ~1.5 years we ran a large data-generation + modelling effort to build a world model for human tissues, pretrained on 112M cells from spatial transcriptomics (mostly Xenium 5000-plex + public data). It's built on one of the largest human spatial transcriptomics corpora assembled to date, spanning 20 tissues across development, health and 26 disease conditions, ~two-thirds newly generated in-house. Why a "world model" for tissue? Images have universal representations (ViT/DINOv3), so do proteins (ESM, Alex Rives) and pathology (UNI, Faisal Mahmood). We've worked hard to build something similar for human tissue: one model that captures its multi-scale logic, genes → cells → their native microenvironments. Like the JEPA approach Yann LeCun has championed, TERRA learns by prediction in embedding space, but for human tissue. How it works: it tokenises each cell together with its nearest neighbours into one sequence while keeping every gene's identity, then masks part of a neighbourhood and predicts the representation of the hidden part, not raw noisy counts. From one backbone it reads out three scales, gene embeddings (what a gene is doing in a cell and its niche), cell embeddings (cell type and state) and neighbourhood embeddings (the niche), and because it keeps gene-level resolution it can knock a gene out in silico and predict the response. Applied entirely zero-shot, TERRA maps and perturbs human tissue across unseen organs, diseases and technologies, outperforming existing spatial approaches. Three take-homes: 1️⃣ One model, any tissue. A single pretrained backbone provides tissue representations zero-shot, handling genes, cells and niches across organs and platforms, off the shelf. 2️⃣ New biology, development to clinic. We built a new spatial atlas of the developing human pancreas and found an islet-associated capillary state that looks like a precursor of mature islet vasculature. In kidney, TERRA's in silico knockouts predicted the tissue-injury programme from cancer immunotherapy (checkpoint blockade), confirmed in treated kidneys, detected in blood, and linked to declining kidney function. 3️⃣ A grammar of tissue architecture. By coupling each cell's state to its niche, TERRA defines recurring cross-organ "archetypes" of macrophage neighbourhoods, including a tumour-boundary niche that tracks poor survival in kidney cancer. TERRA is already in use: it powered our recent skin atlas of hidden immune-memory niches ( with more studies coming soon. This was an amazing collaboration between clinicians, machine-learning scientists and cell biologists 🙏 Led by Sebastian Birk, Vali Sanian Amirhosein Vahidi, Samuel Ogden, Daniyal Jafree, Adib Miraki, Carlo Leonardi and Arpit Merchant, with Lassi Paavolainen, Menna Clatworthy, Omer Ali Bayraktar, Muzlifah Haniffa, Tom Mitchell and Mostafa Bakhti. Huge thanks too to everyone who shared data and helped along the way. What excites me most is seeing how the community builds on this. The model, code and tutorials are all public, so anyone can run TERRA on their own tissues, extend it, or build new models on top. Huge thanks to the whole team across Wellcome Sanger Institute and our many collaborators. 📄 Paper: 💻 Code: 🤗 Model: #SpatialTranscriptomics #SpatialGenomics #FoundationModels #AI4Science #MachineLearning #ComputationalBiology #SingleCell #WorldModels

Mo Lotfollahi

50,443 görüntüleme • 23 gün önce