Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🚀 Introducing scGPT-spatial! 🧬🌍 A game-changing spatial-omic foundation model, built on the powerful scGPT framework with MoE (mixture of experts) and continually pretrained on a massive 30 million spatial single-cell profiles! 🧠 What’s the challenge? Spatial transcriptomics is next-level complex—not only must we model single-cell/spot profiles, but we also...

BoWang87's profile picture

Bo Wang

30,541 subscribers

59,008 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieBildungGesundheit & Wellness
LIVE

Anya Rossi• Live Now

Private livecam show

11 Kommentare

Profilbild von Derya Unutmaz, MD
Derya Unutmaz, MDvor 1 Jahr

Amazing as always!

Profilbild von AndaSeat
AndaSeatvor 1 Jahr

🍀 Mercury retrograde-proof your gaming life! ⚔️ Kaiser 3's anti-bad-luck features: 💪 5,000+ durability cycles (tested!) 💧 Water-resistant surface 🛡️ Anti-spill protection ✨ Murphy's Law defying design 🌟 Because life happens, gaming shouldn't stop! 🪑⚡ Guard your gameplay: 🎲 Lucky deal: Chase away bad luck with $20 off! #AndaSeat #GamerLife #DurabilityGoals #Kaiser3 🎮✨

Profilbild von Tanishq Mathew Abraham, Ph.D.
Tanishq Mathew Abraham, Ph.D.vor 1 Jahr

Looks interesting, congrats!

Profilbild von Gökbörü
Gökbörüvor 1 Jahr

awesome

Profilbild von Aiden
Aidenvor 1 Jahr

Awesome, looks interesting

Profilbild von Steven ten Holder — e/bio
Steven ten Holder — e/biovor 1 Jahr

This + RegFormer insights…

Profilbild von Patricia Cano
Patricia Canovor 1 Jahr

scGPT-spatial’s integration of MoE and 30M spatial single-cell profiles could transform spatial biology, especially in zoonotic disease research. A huge step for public health and One Health!

Profilbild von steven hoffman
steven hoffmanvor 1 Jahr

@qicy11 Awesome

Profilbild von Greg Cook
Greg Cookvor 1 Jahr

Hey @DeryaTR_ FYI

Profilbild von Hao Yin
Hao Yinvor 1 Jahr

Wowo 🤯 Absolutely a must read 🫡

Profilbild von Vivek Das
Vivek Dasvor 1 Jahr

Looks interesting. Congratulations to your team @BoWang87 .

Ähnliche Videos

🔬 Exciting News! Our manuscript, "scGPT: toward building a foundation model for single-cell multi-omics using generative AI" is now finally published in Nature Methods (Nature Methods) 🎉 !!! (Re-)Introducing scGPT: A transformative foundation model engineered for single-cell omics analysis. Developed through the analysis of over 33 million human cells, scGPT sets a new benchmark for application versatility, offering both fine-tuning and zero-shot capabilities. Since its preprint in May 2023, scGPT has significantly impacted the field, evidenced by 13K+ installations, 600+ GitHub stars 🌟, and 40+ citations before its official publication! scGPT has been validated by numerous benchmark studies as a leading foundation model in single-cell analysis. Its pre-trained embeddings extend its utility beyond single-cell studies, enhancing a variety of downstream tasks including protein enrichment and genetic perturbation predictions. Some key updates lately: ---Expanded zero-shot applications for efficient reference mapping and integration, now with CellXGene census integration. ---Advanced perturbation analysis capabilities, including genome-scale perturb-seq data analysis and bulk sequencing data generalization. ---Upgraded scGPT package, offering versatile model loading compatible with PyTorch and flash-attn, for both GPU and CPU. ---Cloud-based scGPT applications for reference mapping, cell annotation, and gene regulatory network inference are available on ---Integration with Hugging Face for easier model training. Limitations: scGPT is an early foray into foundation models for single-cell omics, facing challenges like limited zero-shot learning in some tasks, pretraining constraints, data quality issues, and evaluation limitations. See our Supplementary Notes for details. 🚀 Future Work? Short-Term Goals: 1. Releasing a Mouse Model for broader analysis. 2. Developing a comprehensive evaluation suite for foundation models in single-cell analysis. 3. Creating a foundation model for single-cell spatial omics. 4. Enhancing zero-shot capacity by integrating scGPT with RAG (e.g., knowledge graphs). Long-Term Goals: 1. Expanding scGPT for comprehensive single-cell multi-omics analysis. 2. Developing an in-silico perturbation model for predicting genetic perturbation effects. 3. Merging scGPT with multi-modal genomic sequence models for a deeper understanding of cell biology. 📚 Access the paper on Nature Methods: 🔬Preprint in Bioarixv: 💻 All our codes/data/weights are open source: Wholehearted congratulations to all the authors, especially the two co-first authors, Haotian (Haotian Cui ) and Chloe (ChloeXWang), who are really the emerging superstars in AI and biology! Vector Institute Peter Munk Cardiac Centre AI U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology University Health Network University of Toronto #scGPT #GenerativeAI #AI4Science #Combio #opensource
0:52

🔬 Exciting News! Our manuscript, "scGPT: toward building a foundation model for single-cell multi-omics using generative AI" is now finally published in Nature Methods (Nature Methods) 🎉 !!! (Re-)Introducing scGPT: A transformative foundation model engineered for single-cell omics analysis. Developed through the analysis of over 33 million human cells, scGPT sets a new benchmark for application versatility, offering both fine-tuning and zero-shot capabilities. Since its preprint in May 2023, scGPT has significantly impacted the field, evidenced by 13K+ installations, 600+ GitHub stars 🌟, and 40+ citations before its official publication! scGPT has been validated by numerous benchmark studies as a leading foundation model in single-cell analysis. Its pre-trained embeddings extend its utility beyond single-cell studies, enhancing a variety of downstream tasks including protein enrichment and genetic perturbation predictions. Some key updates lately: ---Expanded zero-shot applications for efficient reference mapping and integration, now with CellXGene census integration. ---Advanced perturbation analysis capabilities, including genome-scale perturb-seq data analysis and bulk sequencing data generalization. ---Upgraded scGPT package, offering versatile model loading compatible with PyTorch and flash-attn, for both GPU and CPU. ---Cloud-based scGPT applications for reference mapping, cell annotation, and gene regulatory network inference are available on ---Integration with Hugging Face for easier model training. Limitations: scGPT is an early foray into foundation models for single-cell omics, facing challenges like limited zero-shot learning in some tasks, pretraining constraints, data quality issues, and evaluation limitations. See our Supplementary Notes for details. 🚀 Future Work? Short-Term Goals: 1. Releasing a Mouse Model for broader analysis. 2. Developing a comprehensive evaluation suite for foundation models in single-cell analysis. 3. Creating a foundation model for single-cell spatial omics. 4. Enhancing zero-shot capacity by integrating scGPT with RAG (e.g., knowledge graphs). Long-Term Goals: 1. Expanding scGPT for comprehensive single-cell multi-omics analysis. 2. Developing an in-silico perturbation model for predicting genetic perturbation effects. 3. Merging scGPT with multi-modal genomic sequence models for a deeper understanding of cell biology. 📚 Access the paper on Nature Methods: 🔬Preprint in Bioarixv: 💻 All our codes/data/weights are open source: Wholehearted congratulations to all the authors, especially the two co-first authors, Haotian (Haotian Cui ) and Chloe (ChloeXWang), who are really the emerging superstars in AI and biology! Vector Institute Peter Munk Cardiac Centre AI U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology University Health Network University of Toronto #scGPT #GenerativeAI #AI4Science #Combio #opensource

Bo Wang

199,708 Aufrufe • vor 2 Jahren

We’re thrilled to share that our MERFISH+ preprint is now live on bioRxiv!👉 In this work, the Bintu and Zhu labs (UCSD) developed MERFISH+, a next-generation spatial genomics platform that combines genome-wide RNA and epigenetic imaging over a large field of view. By introducing acrydite-modified probes covalently anchored to hydrogels, MERFISH+ achieves remarkable imaging stability and enables >1,800-gene, multi-modal, and multi-month experiments. With this platform, they, together with the Chi lab at UCSD, profiled a whole developing human heart at 12 post-conception week with merely two slides, resulting in a total of 53 slides, 3.1 million single cells and more than 30 cell types. Building upon our previous 3D reconstruction and modeling framework, Spateo ( we reconstruct the 3D human heart that nicely captures the anatomical structure of the heart, including the intricate vasculature network. Sophisticated analyses provide a holistic view of an entire organ and enable systematic characterization of 3D cellular neighborhoods and transcriptional gradients of substructures such as the descending arteries. Furthermore, using a generative integration framework for spatial multimodal data (Spateo-VI), we harmonized these MERFISH+ transcriptomic and chromatin data to reconstruct a 3D spatially-resolved multi-omics atlas of the developing human heart, shared at and MERFISH+ thus sets a new standard for large-format, multi-omic spatial profiling, enabling holistic, 3D characterization of organs at subcellular resolution. Huge congratulations to first authors Colin Kern, qingquan Zhang, Yifan Lu , and Jacqueline Eschbach, and to all collaborators from the Bintu, Zhu, Chi, and Qiu labs for this amazing team effort. Thanks for your diligence, creativity, and hard work on this project. We’re grateful for support from Arc Institute and our generous donors. Our lab is expanding—if you’re excited about building the next generation of single-cell and spatial genomics techniques and predictive single cell and spatial foundation models, we’re hiring! If you are interested, please reach out to me via direct message or email at xiaojie@stanford.edu. We are excited for any potential collaborations along this line of research in Stanford, UCSF and Berkeley and other labs as well.
0:42

We’re thrilled to share that our MERFISH+ preprint is now live on bioRxiv!👉 In this work, the Bintu and Zhu labs (UCSD) developed MERFISH+, a next-generation spatial genomics platform that combines genome-wide RNA and epigenetic imaging over a large field of view. By introducing acrydite-modified probes covalently anchored to hydrogels, MERFISH+ achieves remarkable imaging stability and enables >1,800-gene, multi-modal, and multi-month experiments. With this platform, they, together with the Chi lab at UCSD, profiled a whole developing human heart at 12 post-conception week with merely two slides, resulting in a total of 53 slides, 3.1 million single cells and more than 30 cell types. Building upon our previous 3D reconstruction and modeling framework, Spateo ( we reconstruct the 3D human heart that nicely captures the anatomical structure of the heart, including the intricate vasculature network. Sophisticated analyses provide a holistic view of an entire organ and enable systematic characterization of 3D cellular neighborhoods and transcriptional gradients of substructures such as the descending arteries. Furthermore, using a generative integration framework for spatial multimodal data (Spateo-VI), we harmonized these MERFISH+ transcriptomic and chromatin data to reconstruct a 3D spatially-resolved multi-omics atlas of the developing human heart, shared at and MERFISH+ thus sets a new standard for large-format, multi-omic spatial profiling, enabling holistic, 3D characterization of organs at subcellular resolution. Huge congratulations to first authors Colin Kern, qingquan Zhang, Yifan Lu , and Jacqueline Eschbach, and to all collaborators from the Bintu, Zhu, Chi, and Qiu labs for this amazing team effort. Thanks for your diligence, creativity, and hard work on this project. We’re grateful for support from Arc Institute and our generous donors. Our lab is expanding—if you’re excited about building the next generation of single-cell and spatial genomics techniques and predictive single cell and spatial foundation models, we’re hiring! If you are interested, please reach out to me via direct message or email at [email protected]. We are excited for any potential collaborations along this line of research in Stanford, UCSF and Berkeley and other labs as well.

evo-devo

42,208 Aufrufe • vor 8 Monaten

Excited to share our new work on building a multimodal atlas of human skin in health and inflammatory disease — a project I’m especially proud of, bringing together AI, high-throughput genomics, and clinical science to accelerate discovery. Over the past decade, single-cell genomics has transformed how we map cells in human tissues. But a major challenge remains: can we systematically decode how cells organize into functional niches in situ — including those invisible to standard histopathology? To address this, we integrated large-scale scRNA-seq, spatial transcriptomics, histopathology, and AI-driven modeling frameworks to build an in situ atlas of human skin across health and disease. Led by Lloyd Steele, an MD/PhD student working between Haniffa Lab and my lab at Wellcome Sanger Institute and Cambridge University . Another amazing collaboration with Muzz Haniffa, the mastermind behind the work as part of Human Cell Atlas. A key part of this study is that we didn’t build everything from scratch — we leveraged and combined AI methods that actually work! and showed how they can be used together to extract biological insight at scale. We used: • scArches to build and map into a reference scRNA-seq atlas of human skin: • NicheCompass to identify and characterize spatial niches: • MINT-Flow to extract microenvironment-induced cell states and gene programs: Together, these enabled an end-to-end workflow from atlas construction to spatial mapping, niche discovery, and cell state decoding. At scale, we integrated ~5 million cells and 100+ spatial sections, enabling a systematic view of tissue organization. Using this framework, we identified 26 niches in skin, including known histopathologic structures as well as hidden disease-associated niches not visible on H&E. Among the most striking findings were a resident memory T cell-rich sebaceous gland niche and a plasma cell-rich sweat gland niche, suggesting that appendageal structures act as active immunological microenvironments and may contribute to inflammatory memory and disease persistence. Importantly, this atlas is not just descriptive — it is usable. It can support mapping of new datasets, resolve finer cell types and niches, extract microenvironment-driven programs, and enable predictive analyses at scale. More broadly, this work shows what becomes possible when AI, spatial genomics, and atlas-scale data are integrated end-to-end: not just mapping tissues, but systematically decoding them. This was a massive collaboration, and I’m very grateful to the amazing scientists April Foster, Kenny Roberts, and Chloe Admane. Lloyd is an amazing scientist, and I’m especially excited for the community to see more of his work soon — stay tuned. The data and pre-trained models will be released soon. Preprint:
1:09

Excited to share our new work on building a multimodal atlas of human skin in health and inflammatory disease — a project I’m especially proud of, bringing together AI, high-throughput genomics, and clinical science to accelerate discovery. Over the past decade, single-cell genomics has transformed how we map cells in human tissues. But a major challenge remains: can we systematically decode how cells organize into functional niches in situ — including those invisible to standard histopathology? To address this, we integrated large-scale scRNA-seq, spatial transcriptomics, histopathology, and AI-driven modeling frameworks to build an in situ atlas of human skin across health and disease. Led by Lloyd Steele, an MD/PhD student working between Haniffa Lab and my lab at Wellcome Sanger Institute and Cambridge University . Another amazing collaboration with Muzz Haniffa, the mastermind behind the work as part of Human Cell Atlas. A key part of this study is that we didn’t build everything from scratch — we leveraged and combined AI methods that actually work! and showed how they can be used together to extract biological insight at scale. We used: • scArches to build and map into a reference scRNA-seq atlas of human skin: • NicheCompass to identify and characterize spatial niches: • MINT-Flow to extract microenvironment-induced cell states and gene programs: Together, these enabled an end-to-end workflow from atlas construction to spatial mapping, niche discovery, and cell state decoding. At scale, we integrated ~5 million cells and 100+ spatial sections, enabling a systematic view of tissue organization. Using this framework, we identified 26 niches in skin, including known histopathologic structures as well as hidden disease-associated niches not visible on H&E. Among the most striking findings were a resident memory T cell-rich sebaceous gland niche and a plasma cell-rich sweat gland niche, suggesting that appendageal structures act as active immunological microenvironments and may contribute to inflammatory memory and disease persistence. Importantly, this atlas is not just descriptive — it is usable. It can support mapping of new datasets, resolve finer cell types and niches, extract microenvironment-driven programs, and enable predictive analyses at scale. More broadly, this work shows what becomes possible when AI, spatial genomics, and atlas-scale data are integrated end-to-end: not just mapping tissues, but systematically decoding them. This was a massive collaboration, and I’m very grateful to the amazing scientists April Foster, Kenny Roberts, and Chloe Admane. Lloyd is an amazing scientist, and I’m especially excited for the community to see more of his work soon — stay tuned. The data and pre-trained models will be released soon. Preprint:

Mo Lotfollahi

11,759 Aufrufe • vor 3 Monaten

Single-cell technologies now let us profile entire transcriptomes in individual cells. But how do we make sense of this complexity in a biologically meaningful way? Many methods summarise cells into a single embedding, but this often comes at the cost of interpretability, especially when multiple gene programs are active at once. We developed Tripso, a self-supervised transformer model that represents cells through multiple gene program-specific embeddings, while also uncovering new programs directly from the data. Instead of collapsing biology into a single vector, Tripso decomposes cell state into multiple representations, each reflecting a different gene program. We explored this across multiple systems. In human hematopoiesis, spanning development to aging, Tripso identified distinct age-associated program activity, including stronger JAK-STAT signalling in early life and dynamic IKZF1-related changes during B cell maturation. By comparing in vitro culture conditions with in vivo hematopoietic stem cell states, Tripso suggested that targeting the SEC61 translocon could enhance stem cell maintenance ex vivo, a prediction that we subsequently validated experimentally. In parallel, we identified a previously uncharacterised tissue-resident memory T-cell program associated with atopic dermatitis and mapped it to distinct spatial immune niches Together, these results show how modelling cells through gene programs can lead to interpretable and experimentally testable insights. More broadly, this work points toward a more interpretable and biologically grounded models of cell state. As single-cell datasets continue to grow, we hope approaches like Tripso will help bridge the gap between data-driven representations and biological insight. This work wouldn’t have been possible without the contributions of an amazing team. Thank you to co-first authors Marie, Tomoya Isobe, Amirhosein Vahidi, Carlo Leonardi, and everyone from roser's Lab, Haniffa Lab, Nicola Wilson and Bertie Gottgens's Lab, bringing together expertise across Cambridge Stem Cell Institute, Open Targets, Wellcome Sanger Institute and Cambridge University. Marie is one of the very best PhD students I have ever supervised. She is truly a force of nature, exceptionally resourceful, deeply innovative, and one of the most impressive scientists I have worked with. I am immensely proud of her and all that she has accomplished. As she begins her internship at Genentech , I have no doubt she will do amazing work there and continue to make her mark. paper: code:
1:25

Single-cell technologies now let us profile entire transcriptomes in individual cells. But how do we make sense of this complexity in a biologically meaningful way? Many methods summarise cells into a single embedding, but this often comes at the cost of interpretability, especially when multiple gene programs are active at once. We developed Tripso, a self-supervised transformer model that represents cells through multiple gene program-specific embeddings, while also uncovering new programs directly from the data. Instead of collapsing biology into a single vector, Tripso decomposes cell state into multiple representations, each reflecting a different gene program. We explored this across multiple systems. In human hematopoiesis, spanning development to aging, Tripso identified distinct age-associated program activity, including stronger JAK-STAT signalling in early life and dynamic IKZF1-related changes during B cell maturation. By comparing in vitro culture conditions with in vivo hematopoietic stem cell states, Tripso suggested that targeting the SEC61 translocon could enhance stem cell maintenance ex vivo, a prediction that we subsequently validated experimentally. In parallel, we identified a previously uncharacterised tissue-resident memory T-cell program associated with atopic dermatitis and mapped it to distinct spatial immune niches Together, these results show how modelling cells through gene programs can lead to interpretable and experimentally testable insights. More broadly, this work points toward a more interpretable and biologically grounded models of cell state. As single-cell datasets continue to grow, we hope approaches like Tripso will help bridge the gap between data-driven representations and biological insight. This work wouldn’t have been possible without the contributions of an amazing team. Thank you to co-first authors Marie, Tomoya Isobe, Amirhosein Vahidi, Carlo Leonardi, and everyone from roser's Lab, Haniffa Lab, Nicola Wilson and Bertie Gottgens's Lab, bringing together expertise across Cambridge Stem Cell Institute, Open Targets, Wellcome Sanger Institute and Cambridge University. Marie is one of the very best PhD students I have ever supervised. She is truly a force of nature, exceptionally resourceful, deeply innovative, and one of the most impressive scientists I have worked with. I am immensely proud of her and all that she has accomplished. As she begins her internship at Genentech , I have no doubt she will do amazing work there and continue to make her mark. paper: code:

Mo Lotfollahi

22,402 Aufrufe • vor 3 Monaten

Welcome to the Lab of the Future! 🧬🤖 Excited to share LUMI-lab, out today in Cell — a self-driving platform that pairs an AI foundation model with a robotic lab to autonomously discover ionizable lipids (LNPs) for mRNA delivery. The core problem: Designing lipid nanoparticles (LNPs) is hard. The chemical space of ionizable lipids is vast, experimental cycles are slow, and — critically — historical LNP datasets are far too small to train a predictive model from scratch. Most AI approaches in this space hit a wall immediately: not enough data to learn from. Our solution: lab-in-the-loop foundation model learning. Instead of training on LNP data alone, LUMI starts as a transformer-based foundation model pretrained across broad chemical space, building rich molecular representations before it ever sees a single LNP experiment. Then it enters a closed loop with a robotic synthesis platform: predict → synthesize → assay → update. Each round of real wet-lab experiments fine-tunes the model, which then proposes smarter candidates for the next round. The lab isn't just validating AI predictions — it's actively teaching the model, continuously. What happened when we let it run: LUMI-lab autonomously synthesized and screened 1,700+ ionizable lipids in human bronchial epithelial cells. The top candidate — LUMI-6 — features a brominated lipid tail, a structural motif that had been largely overlooked in LNP design. LUMI found it without being told where to look. When formulated into LNPs and delivered intratracheally to mice, LUMI-6 achieved 20.3% gene editing efficiency in lung epithelial cells — a compelling result for one of the hardest-to-reach therapeutic targets, directly relevant to diseases like cystic fibrosis and alpha-1 antitrypsin deficiency. Why this matters beyond LNPs: This is a proof of concept for a broader thesis — that foundation model pretraining + active learning + robotic experimentation can overcome the data scarcity bottleneck that plagues AI-driven discovery in biology. You don't need a massive domain-specific dataset to start. You need a model that can generalize, a lab that can generate the right data, and a loop that connects them. Huge congratulations to first authors Yue Xu, Haotian Cui, and Kuan Pang, and to the entire Bowen LI team. Grateful to our collaborators at University Health Network and Leslie Dan Faculty of Pharmacy, and to Princess Margaret Cancer Centre Research Princess Margaret Cancer Centre Research. 📄 Paper:
1:05

Welcome to the Lab of the Future! 🧬🤖 Excited to share LUMI-lab, out today in Cell — a self-driving platform that pairs an AI foundation model with a robotic lab to autonomously discover ionizable lipids (LNPs) for mRNA delivery. The core problem: Designing lipid nanoparticles (LNPs) is hard. The chemical space of ionizable lipids is vast, experimental cycles are slow, and — critically — historical LNP datasets are far too small to train a predictive model from scratch. Most AI approaches in this space hit a wall immediately: not enough data to learn from. Our solution: lab-in-the-loop foundation model learning. Instead of training on LNP data alone, LUMI starts as a transformer-based foundation model pretrained across broad chemical space, building rich molecular representations before it ever sees a single LNP experiment. Then it enters a closed loop with a robotic synthesis platform: predict → synthesize → assay → update. Each round of real wet-lab experiments fine-tunes the model, which then proposes smarter candidates for the next round. The lab isn't just validating AI predictions — it's actively teaching the model, continuously. What happened when we let it run: LUMI-lab autonomously synthesized and screened 1,700+ ionizable lipids in human bronchial epithelial cells. The top candidate — LUMI-6 — features a brominated lipid tail, a structural motif that had been largely overlooked in LNP design. LUMI found it without being told where to look. When formulated into LNPs and delivered intratracheally to mice, LUMI-6 achieved 20.3% gene editing efficiency in lung epithelial cells — a compelling result for one of the hardest-to-reach therapeutic targets, directly relevant to diseases like cystic fibrosis and alpha-1 antitrypsin deficiency. Why this matters beyond LNPs: This is a proof of concept for a broader thesis — that foundation model pretraining + active learning + robotic experimentation can overcome the data scarcity bottleneck that plagues AI-driven discovery in biology. You don't need a massive domain-specific dataset to start. You need a model that can generalize, a lab that can generate the right data, and a loop that connects them. Huge congratulations to first authors Yue Xu, Haotian Cui, and Kuan Pang, and to the entire Bowen LI team. Grateful to our collaborators at University Health Network and Leslie Dan Faculty of Pharmacy, and to Princess Margaret Cancer Centre Research Princess Margaret Cancer Centre Research. 📄 Paper:

Bo Wang

57,430 Aufrufe • vor 4 Monaten

🚀 Introducing PantheonOS ( A Fully Open-Source Agent OS for Science PantheonOS began as a research project in my Stanford lab and has since evolved into a vision to redefine data science in the era of AI—starting with computational biology, especially single-cell and spatial genomics. PantheonOS is a general agent platform built from the ground up. It is arguably the first distributed agent framework designed for scientific data analysis. 🔑 Key Features 1. Multi-Agent Collaboration – Built-in paradigms for distributed, cross-machine cooperation among agents and toolsets. 2. Native Toolset Support – Python, R, Julia, LaTeX, and more—designed for real scientific workflows. 3. Modular & Extensible – Developer-friendly design with shallow wrappers, plus LLM-driven toolset generation. 4. Evolvable Agents – Capable of evolving large-scale code projects to achieve superhuman performance (e.g., evolving upon the original Harmony [I Korsunsky, 2019, Nature Biotechnology] and Scanorama [BL Hie, 2019, Nature Biotechnology] implementations), and even evolving the system itself to adapt to new fields. 🎉 Stepwise Release Strategy We’re releasing PantheonOS in stages: Pantheon-CLI (today!), followed by Pantheon-Lab, Pantheon-Notebook, Pantheon-Slack, and more. 🌟 Pantheon-CLI Highlights - We're not just building another CLI tool. We're defining how scientists will interact with data in the AI era. - Open, Powerful, Python-First – The first fully open-source, endlessly extendable scientific “vibe analysis” framework. - Mixed Programming Magic – Combine Python, natural language, R, or Julia—seamlessly in the same environment. - PhD-Level Assistant – A command-line agent for complex real-world genomics and beyond, handling workflows at the PhD level. - Privacy by Design – Run entirely offline with local LLMs—your data never leaves your computer. ✅ Proven Applications (10 Demonstrations) Computational biology: 1. ATAC-seq: From raw reads to peak matrix 2. RNA-seq: From raw reads to expression matrix 3. Complex single-cell workflows (PhD-level) 4. Hybrid natural language + R for Seurat annotation 5. Learning from web tutorials + invoking single-cell foundation models 6. Cell segmentation on 10x Genomics HD Visium data And beyond: 7. Mixed Python & R programming examples 8. Molecular docking & structural analysis 9. Exploratory factor analysis for behavioral survey data 10. Customer segmentation & finance analytics 🌐 Learn More & Get Started Website: Pantheon-CLI Documentation: GitHub Repo: 💬 Join our community: PantheonOS Slack: PantheonOS Discord:
1:16

🚀 Introducing PantheonOS ( A Fully Open-Source Agent OS for Science PantheonOS began as a research project in my Stanford lab and has since evolved into a vision to redefine data science in the era of AI—starting with computational biology, especially single-cell and spatial genomics. PantheonOS is a general agent platform built from the ground up. It is arguably the first distributed agent framework designed for scientific data analysis. 🔑 Key Features 1. Multi-Agent Collaboration – Built-in paradigms for distributed, cross-machine cooperation among agents and toolsets. 2. Native Toolset Support – Python, R, Julia, LaTeX, and more—designed for real scientific workflows. 3. Modular & Extensible – Developer-friendly design with shallow wrappers, plus LLM-driven toolset generation. 4. Evolvable Agents – Capable of evolving large-scale code projects to achieve superhuman performance (e.g., evolving upon the original Harmony [I Korsunsky, 2019, Nature Biotechnology] and Scanorama [BL Hie, 2019, Nature Biotechnology] implementations), and even evolving the system itself to adapt to new fields. 🎉 Stepwise Release Strategy We’re releasing PantheonOS in stages: Pantheon-CLI (today!), followed by Pantheon-Lab, Pantheon-Notebook, Pantheon-Slack, and more. 🌟 Pantheon-CLI Highlights - We're not just building another CLI tool. We're defining how scientists will interact with data in the AI era. - Open, Powerful, Python-First – The first fully open-source, endlessly extendable scientific “vibe analysis” framework. - Mixed Programming Magic – Combine Python, natural language, R, or Julia—seamlessly in the same environment. - PhD-Level Assistant – A command-line agent for complex real-world genomics and beyond, handling workflows at the PhD level. - Privacy by Design – Run entirely offline with local LLMs—your data never leaves your computer. ✅ Proven Applications (10 Demonstrations) Computational biology: 1. ATAC-seq: From raw reads to peak matrix 2. RNA-seq: From raw reads to expression matrix 3. Complex single-cell workflows (PhD-level) 4. Hybrid natural language + R for Seurat annotation 5. Learning from web tutorials + invoking single-cell foundation models 6. Cell segmentation on 10x Genomics HD Visium data And beyond: 7. Mixed Python & R programming examples 8. Molecular docking & structural analysis 9. Exploratory factor analysis for behavioral survey data 10. Customer segmentation & finance analytics 🌐 Learn More & Get Started Website: Pantheon-CLI Documentation: GitHub Repo: 💬 Join our community: PantheonOS Slack: PantheonOS Discord:

evo-devo

17,356 Aufrufe • vor 11 Monaten

Hyperspace: The Agentic OS Apple Should Have Built On December 19th, 2024, we announced the world’s first Agentic Browser. What followed was a movement — a new category was born which led to many early products in this space and recently the hundreds of people lining up outside the The Agentic Browser Summit in San Francisco underscored that. Silicon Valley instinctively gets it, from students to tech executives, people can feel a revolutionary new change in computing is in the air. Past year taught us why such a product was inevitable, a hard engineering effort, and also the last mover in the entire software world this decade if and when done right. All paths are headed in the same direction: one tool which orchestrates them all. At Hyperspace we showed that path with essays and products we launched in earlier months: from a spatial UI of orchestrating agents, to showcasing transparent activity in how the AI system operates which leads to user trust, to presenting the software end-game, which massively improves human productivity. We also built the world’s largest AI network, drawing participation from people in almost 6000 cities around the world contributing their machines as nodes in the network. Think Uber, but for AI. That is, planetary-scale. And now we are stretching this industry ambition further with our end-to-end vision of the Agentic Supercomputer, the first breakthrough new AI OS, and an effort which spans from AI research to distributed systems to inventing a new UI to inventing a new business model to complement it. All of this together helps us in serving our mission, of delivering “Everyone’s Personal Supercomputer”. While others have built AI-native browsers, no one though has built something agentic from the ground up — with AI as the foundation, not a feature. How do you fundamentally improve the lives’ of billions around the world ? We believe that requires building a native environment for agents to be viewed, created, deployed, executed, discovered and priced in. That is a world where we move on from static apps, to dynamic agents. But, as my 2 year old niece likes to ask: “but why ?” The issue is that the world of software today is fragmented, and everyone is sprinkling on AI as a feature and charging a subscription fees for it. From browser makers, to IDEs, to design and other productivity tools. This leads to a fragmented UX, where people have to learn to use AI in each app, their memory and other context is not shared between all these apps, and they also have to pay separately for compute for each such AI-enhanced app. Each app maker has to figure out basics such as compute, and leads to the issues we saw with Cursor pricing recently. This is not the future. What if AI was the foundation instead of a feature ? What if Apple had built a fundamentally new AI OS from the ground up and what would it have looked like ? At Hyperspace, that is what we did. On July 15th we introduced three breakthrough key pillars of our AI OS: 1. Agentic Browser 2. Agentic Memory 3. Agentic Payments And we didn’t stop there. We also introduced a breakthrough new user interface called the Spatial AI which is inspired both from the spreadsheet and the HyperCard - each card is an agent, with it’s own inputs and outputs, endlessly extensible and pluggable with others, just like cells of a spreadsheet. Update one cell and all the dependents update, like a spreadsheet formula. It goes beyond a static linear workflow to being able to operate in all directions. This revolutionary new interface helps manage all of the below: 1. Multiple websites being browsed in parallel 2. Multiple desktop apps being browsed in parallel 3. Multiple server tools being used in parallel 4. Multiple smartphone apps streamed to your device or opened via an emulator All the software which you need comes together in this one seamless, agent-native interface. This interface provides you access to the largest network of models, vectors, agents and compute on the planet. The Browser. The IDE. The Notepad… they are not separate products: they are all in one, the Agentic Browser. As Steve Jobs famously said at the iPhone announcement, “are you getting it ?” And beneath this UI lies a new intelligence routing layer — leveraging both swarms of specialized models to the Hyperspace Matrix model that recalls thousands of tools in real-time, not by context window hacks, but through retrieval, ranking, and reuse. To many, this will feel like AGI. Not one big system by one big company, but an intelligent network. Now lets talk about privacy… Are you comfortable with one company owning all your memory forever ? I am not. So we have invented Agentic Memory as a new open protocol which provides full power over memory to you, the user. Your memory is yours, encrypted, on your device, and portable if and how you want. Anyone can build on it without our permission, but not without your permission. This protocol, and the decentralized vector database spread out across the world, would enable apps and agents to share context and memory. Think copy-paste, but for the AI world. It doesn’t just remember — it knows what matters. VectorRank helps your AI weigh your life’s most relevant moments over time, just like the way our minds elevate memories. Now each time you use an agent, your experience with other agents will also continuously improve: you don’t have to keep repeating the same things about yourself, while fully preserving your privacy. Agentic Memory is accessible within the Agentic Browser to manage. And there is one more thing… AI as the foundation requires compute to be available at the base layer, but this base layer spans models running on your own device, to cloud APIs, to also running across the peer-to-peer distributed network. Agentic Payments provides a singular interface to all of that compute, running a spot auction clearing marketplace every second to determine the fair price of compute. This results in price transparency, and you as the user paying the lowest possible cost. If you want predictability, you can reserve compute in advance. This end-to-end system provides the most streamlined world for agents to operate in. In order to enable this world and the world of agents being able to pay each other in sub-cent increments millions of times a second, we had to also invent a fundamentally new agentic micropayments blockchain. All of this together would enable a world where you as a user, or the agent itself, can efficiently call and utilize other agents built by others and also pay for content which is unique and useful. This enables a move away from the current AI exploitative economy for bloggers and other content creators, to a web with a fundamental new business model. Earlier we didn’t have the right infrastructure to enable such a world. Now, all the dots connect. The Hyperspace AI OS would give the power of a supercomputer in everyone’s hands. This isn’t a browser, or an IDE or limited to any device or cloud. It’s an entire AI operating system — with a breakthrough new spatial UI, local and distributed compute, agentic memory, agentic payments, and orchestration built into the foundation. As a user, we move the choice back in your hands with an experience you will love and find delightful. You get to choose the level of privacy, cost, and utility you want. And while Apple should have done it, we could not wait, and we feel this just required a new level of passion and DNA which we bring here. We are just getting started. Thank you, Varun Mathur Cofounder and CEO, Hyperspace cc Naval Marc Andreessen 🇺🇸 Vinod Khosla Andrej Karpathy Sam Altman
2:52

Hyperspace: The Agentic OS Apple Should Have Built On December 19th, 2024, we announced the world’s first Agentic Browser. What followed was a movement — a new category was born which led to many early products in this space and recently the hundreds of people lining up outside the The Agentic Browser Summit in San Francisco underscored that. Silicon Valley instinctively gets it, from students to tech executives, people can feel a revolutionary new change in computing is in the air. Past year taught us why such a product was inevitable, a hard engineering effort, and also the last mover in the entire software world this decade if and when done right. All paths are headed in the same direction: one tool which orchestrates them all. At Hyperspace we showed that path with essays and products we launched in earlier months: from a spatial UI of orchestrating agents, to showcasing transparent activity in how the AI system operates which leads to user trust, to presenting the software end-game, which massively improves human productivity. We also built the world’s largest AI network, drawing participation from people in almost 6000 cities around the world contributing their machines as nodes in the network. Think Uber, but for AI. That is, planetary-scale. And now we are stretching this industry ambition further with our end-to-end vision of the Agentic Supercomputer, the first breakthrough new AI OS, and an effort which spans from AI research to distributed systems to inventing a new UI to inventing a new business model to complement it. All of this together helps us in serving our mission, of delivering “Everyone’s Personal Supercomputer”. While others have built AI-native browsers, no one though has built something agentic from the ground up — with AI as the foundation, not a feature. How do you fundamentally improve the lives’ of billions around the world ? We believe that requires building a native environment for agents to be viewed, created, deployed, executed, discovered and priced in. That is a world where we move on from static apps, to dynamic agents. But, as my 2 year old niece likes to ask: “but why ?” The issue is that the world of software today is fragmented, and everyone is sprinkling on AI as a feature and charging a subscription fees for it. From browser makers, to IDEs, to design and other productivity tools. This leads to a fragmented UX, where people have to learn to use AI in each app, their memory and other context is not shared between all these apps, and they also have to pay separately for compute for each such AI-enhanced app. Each app maker has to figure out basics such as compute, and leads to the issues we saw with Cursor pricing recently. This is not the future. What if AI was the foundation instead of a feature ? What if Apple had built a fundamentally new AI OS from the ground up and what would it have looked like ? At Hyperspace, that is what we did. On July 15th we introduced three breakthrough key pillars of our AI OS: 1. Agentic Browser 2. Agentic Memory 3. Agentic Payments And we didn’t stop there. We also introduced a breakthrough new user interface called the Spatial AI which is inspired both from the spreadsheet and the HyperCard - each card is an agent, with it’s own inputs and outputs, endlessly extensible and pluggable with others, just like cells of a spreadsheet. Update one cell and all the dependents update, like a spreadsheet formula. It goes beyond a static linear workflow to being able to operate in all directions. This revolutionary new interface helps manage all of the below: 1. Multiple websites being browsed in parallel 2. Multiple desktop apps being browsed in parallel 3. Multiple server tools being used in parallel 4. Multiple smartphone apps streamed to your device or opened via an emulator All the software which you need comes together in this one seamless, agent-native interface. This interface provides you access to the largest network of models, vectors, agents and compute on the planet. The Browser. The IDE. The Notepad… they are not separate products: they are all in one, the Agentic Browser. As Steve Jobs famously said at the iPhone announcement, “are you getting it ?” And beneath this UI lies a new intelligence routing layer — leveraging both swarms of specialized models to the Hyperspace Matrix model that recalls thousands of tools in real-time, not by context window hacks, but through retrieval, ranking, and reuse. To many, this will feel like AGI. Not one big system by one big company, but an intelligent network. Now lets talk about privacy… Are you comfortable with one company owning all your memory forever ? I am not. So we have invented Agentic Memory as a new open protocol which provides full power over memory to you, the user. Your memory is yours, encrypted, on your device, and portable if and how you want. Anyone can build on it without our permission, but not without your permission. This protocol, and the decentralized vector database spread out across the world, would enable apps and agents to share context and memory. Think copy-paste, but for the AI world. It doesn’t just remember — it knows what matters. VectorRank helps your AI weigh your life’s most relevant moments over time, just like the way our minds elevate memories. Now each time you use an agent, your experience with other agents will also continuously improve: you don’t have to keep repeating the same things about yourself, while fully preserving your privacy. Agentic Memory is accessible within the Agentic Browser to manage. And there is one more thing… AI as the foundation requires compute to be available at the base layer, but this base layer spans models running on your own device, to cloud APIs, to also running across the peer-to-peer distributed network. Agentic Payments provides a singular interface to all of that compute, running a spot auction clearing marketplace every second to determine the fair price of compute. This results in price transparency, and you as the user paying the lowest possible cost. If you want predictability, you can reserve compute in advance. This end-to-end system provides the most streamlined world for agents to operate in. In order to enable this world and the world of agents being able to pay each other in sub-cent increments millions of times a second, we had to also invent a fundamentally new agentic micropayments blockchain. All of this together would enable a world where you as a user, or the agent itself, can efficiently call and utilize other agents built by others and also pay for content which is unique and useful. This enables a move away from the current AI exploitative economy for bloggers and other content creators, to a web with a fundamental new business model. Earlier we didn’t have the right infrastructure to enable such a world. Now, all the dots connect. The Hyperspace AI OS would give the power of a supercomputer in everyone’s hands. This isn’t a browser, or an IDE or limited to any device or cloud. It’s an entire AI operating system — with a breakthrough new spatial UI, local and distributed compute, agentic memory, agentic payments, and orchestration built into the foundation. As a user, we move the choice back in your hands with an experience you will love and find delightful. You get to choose the level of privacy, cost, and utility you want. And while Apple should have done it, we could not wait, and we feel this just required a new level of passion and DNA which we bring here. We are just getting started. Thank you, Varun Mathur Cofounder and CEO, Hyperspace cc Naval Marc Andreessen 🇺🇸 Vinod Khosla Andrej Karpathy Sam Altman

Varun

169,177 Aufrufe • vor 1 Jahr

Excited to share our new work. Over the past decade, single-cell genomics has transformed our ability to map cellular systems. But a major question remains: Can we predict how perturbations reshape cellular trajectories over time? In 2018, we first showed that it is possible to predict cellular responses to perturbations — ranging from disease signals to chemical treatments — even in unseen contexts. In 2022, we introduced CPA (MSB 2022; NeurIPS 2022), extending this idea to predict responses to unseen chemical and genetic perturbations, including their combinations. Since then, the field of perturbation modeling has grown enormously. The community has pushed the space forward with many creative ideas and powerful models. It’s exciting to see how fast things are moving — even though many fundamental challenges remain. One of the biggest is that cells are not static. They move through trajectories during development, immune responses, and disease. Yet most current models still predict perturbation effects within a single state, rather than how early perturbations propagate across future states and reshape downstream outcomes. To address this, we developed PerturbGen, a trajectory-aware generative AI model that predicts how genetic perturbations reshape downstream cellular states. Huge credit to the people who made this work possible. Thanks to co-first authors Kevin Ly, Adib Miraki, Tomoya Isobe, AmirHoss3in Vahidi, Delshad Vaghari & Anthony Rostron. Special recognition to Kevin Ly and Adib Miraki for driving this work over the finish line. Grateful for our outstanding collaborators from Haniffa Lab, Bertie Gottgens lab Gosia Trynka and many others — a true cross-institute effort across Cambridge Stem Cell Institute, Open Targets ,Wellcome Sanger Institute and Cambridge University.🎉 PerturbGen learns transcriptional dynamics across cellular trajectories. By introducing perturbations at an early source state, it can simulate how these effects propagate into future states along differentiation trajectories. Scaling this across genes enables the creation of dynamic in silico perturbation atlases — maps of how perturbations reshape biological trajectories over time. We explored this idea across three biological questions. First, in a human in vivo LPS immune challenge, PerturbGen predicted that perturbing a transient IL1B signal dampens downstream inflammatory programs in myeloid cells, with pathway changes reversing signatures observed in an independent IL-1β stimulation experiment. Second, in human hematopoiesis, PerturbGen predicted transcriptional responses to CRISPR transcription factor knockouts and enabled construction of perturbation atlases revealing lineage- and age-specific regulatory programs. These programs could also be linked to human genetics and blood diseases, including recapitulation of signatures associated with ETV6-related thrombocytopenia. Finally, we asked whether perturbation modeling could help improve complex tissue models. We built a dynamic perturbation atlas of human skin organoids to identify perturbations that could guideorganoid cells towardhuman fetal skin states. PerturbGen prioritized activation of Wnt signaling via GSK3β inhibition. Experimental validation confirmed the prediction: treatment with CHIR99021 induced stromal gene programs and shifted organoid fibroblasts toward transcriptional states observed in fetal skin stroma. Together, these results show how trajectory-aware perturbation modeling can connect gene perturbations to developmental programs, human genetics, disease mechanisms, and experimental interventions. More broadly, we think these point toward a future where single-cell atlases become predictive systems. As atlases expand across tissues, developmental windows, and modalities, models like PerturbGen could enable dynamic, virtual perturbation atlases— allowing us to simulate interventions, generate hypotheses, and design experiments before stepping into the lab. Preprint Code Excited to see how the community builds on this work.
2:46

Excited to share our new work. Over the past decade, single-cell genomics has transformed our ability to map cellular systems. But a major question remains: Can we predict how perturbations reshape cellular trajectories over time? In 2018, we first showed that it is possible to predict cellular responses to perturbations — ranging from disease signals to chemical treatments — even in unseen contexts. In 2022, we introduced CPA (MSB 2022; NeurIPS 2022), extending this idea to predict responses to unseen chemical and genetic perturbations, including their combinations. Since then, the field of perturbation modeling has grown enormously. The community has pushed the space forward with many creative ideas and powerful models. It’s exciting to see how fast things are moving — even though many fundamental challenges remain. One of the biggest is that cells are not static. They move through trajectories during development, immune responses, and disease. Yet most current models still predict perturbation effects within a single state, rather than how early perturbations propagate across future states and reshape downstream outcomes. To address this, we developed PerturbGen, a trajectory-aware generative AI model that predicts how genetic perturbations reshape downstream cellular states. Huge credit to the people who made this work possible. Thanks to co-first authors Kevin Ly, Adib Miraki, Tomoya Isobe, AmirHoss3in Vahidi, Delshad Vaghari & Anthony Rostron. Special recognition to Kevin Ly and Adib Miraki for driving this work over the finish line. Grateful for our outstanding collaborators from Haniffa Lab, Bertie Gottgens lab Gosia Trynka and many others — a true cross-institute effort across Cambridge Stem Cell Institute, Open Targets ,Wellcome Sanger Institute and Cambridge University.🎉 PerturbGen learns transcriptional dynamics across cellular trajectories. By introducing perturbations at an early source state, it can simulate how these effects propagate into future states along differentiation trajectories. Scaling this across genes enables the creation of dynamic in silico perturbation atlases — maps of how perturbations reshape biological trajectories over time. We explored this idea across three biological questions. First, in a human in vivo LPS immune challenge, PerturbGen predicted that perturbing a transient IL1B signal dampens downstream inflammatory programs in myeloid cells, with pathway changes reversing signatures observed in an independent IL-1β stimulation experiment. Second, in human hematopoiesis, PerturbGen predicted transcriptional responses to CRISPR transcription factor knockouts and enabled construction of perturbation atlases revealing lineage- and age-specific regulatory programs. These programs could also be linked to human genetics and blood diseases, including recapitulation of signatures associated with ETV6-related thrombocytopenia. Finally, we asked whether perturbation modeling could help improve complex tissue models. We built a dynamic perturbation atlas of human skin organoids to identify perturbations that could guideorganoid cells towardhuman fetal skin states. PerturbGen prioritized activation of Wnt signaling via GSK3β inhibition. Experimental validation confirmed the prediction: treatment with CHIR99021 induced stromal gene programs and shifted organoid fibroblasts toward transcriptional states observed in fetal skin stroma. Together, these results show how trajectory-aware perturbation modeling can connect gene perturbations to developmental programs, human genetics, disease mechanisms, and experimental interventions. More broadly, we think these point toward a future where single-cell atlases become predictive systems. As atlases expand across tissues, developmental windows, and modalities, models like PerturbGen could enable dynamic, virtual perturbation atlases— allowing us to simulate interventions, generate hypotheses, and design experiments before stepping into the lab. Preprint Code Excited to see how the community builds on this work.

Mo Lotfollahi

17,035 Aufrufe • vor 4 Monaten

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!
0:27

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!

Akshay 🚀

46,404 Aufrufe • vor 8 Monaten

I've had many engineers ask me why its worth their time and effort to learn biology in response to this post. Why should they be excited? We are poised for a revolution in biotech that will be uniquely enabled by computers. Convince yourself by digging into the examples I link: - The tooling is getting better. Assays are able to measure a broad array of molecules at a falling cost and increasing throughput. Look to ScaleBio, Curio Bioscience, AtlasXOmics for inspiration. We are sequencing millions of single cells and building spatial maps of the molecular state of tumors. After two generations of "next-generation sequencing", and stagnating DNA read + write costs under the monopoly of Illumina, this wave of the new assays will have a profound impact on the iteration speed and scale of experimentation. Little needs to be said about the impact of compounding trends in core tooling over a sufficient period of time. - Biotechs are using data to guide decisions and are incorporating domain informed machine learning as a core part of the molecular design process. There is great synergy here with better tooling as a means of abundant, cheap data. Look to Recursion, Manifold Bio, Dyno Tx, Asimov. There is also the cross pollination of biology informed architectures with the recent explosion in new machine learning techniques. These models are starting to do useful things, like generate functional gene editing proteins and entire prokaryotic organisms. Look to ESM, AlphaFold3, RFdiffusion. - New classes of therapies - genetic medicines and engineered immune cells - are having real success in the clinic. One dose cures for cardiovascular disease (Verve w ACSD), vaccines for cancer (Moderna w mRNA 4157), in vivo gene editing proteins (CRISPR Tx, Beam, Ensoma), metabolic disease (Novartis, Eli Lily w GLP-1/GIP modulators) are being dosed in real people right now + transforming lives. - The AI craze is commoditizing accelerated hardware and fast storage devices like NVMes, improving developer frameworks for writing code against these devices and maturing the systems tooling for moving around lots of data between computers for distributed training. One happy accident of this bubble will be the reuse of these components to build a new systems stack for the large scale processing of molecular data. This will be very important to construct a 1 billion single cell atlas and beyond. (For reference, the state of the art is ScaleBio's 2M cell kit, dubbed QuantumScale, and it is pushing things with the hardware + software we have today.) - Language models might be the perfect tool to distill the unstructured corpus of public data, literature, and methods sitting around on the Internet into real biological insights for scientist asking questions in natural language. It will also allow them to install, configure and run the slew of useful but poorly maintained academic computational tools to explore and hypothesize new biology on their own. Increasing the productivity of each scientist will do much to reverse Eroom's law. - There is an appetite from the market for new applications of biotech beyond drug development. Bacteria driven lithium mining (maverick), cell agriculture (growing cows in vats), early signs of consumer biologics (Geltor), biofoundries (Ginkgo). My guess is some of the greatest minds of our generation will want to do more than perturb the human body with therapies. I think it is also important to recognize that the need for computers and software is a secular trend in the progression of biotech independent of the interests of Silicon Valley. Clusters of computers, the information they store and the software that runs on them are precisely the technology needed by this field as it transforms into a discipline of information management, towards reducing living things into well characterized building blocks we can rebuild in our image. Software companies, as some local and hyper efficient structure in the arc of capitalism, with established methods and well trod rails to attract resources, talent and easily distribute product to an entire market, are the perfect place to incubate and disseminate these tools. There will probably be many, very large computer companies in biology in the next century.
0:52

I've had many engineers ask me why its worth their time and effort to learn biology in response to this post. Why should they be excited? We are poised for a revolution in biotech that will be uniquely enabled by computers. Convince yourself by digging into the examples I link: - The tooling is getting better. Assays are able to measure a broad array of molecules at a falling cost and increasing throughput. Look to ScaleBio, Curio Bioscience, AtlasXOmics for inspiration. We are sequencing millions of single cells and building spatial maps of the molecular state of tumors. After two generations of "next-generation sequencing", and stagnating DNA read + write costs under the monopoly of Illumina, this wave of the new assays will have a profound impact on the iteration speed and scale of experimentation. Little needs to be said about the impact of compounding trends in core tooling over a sufficient period of time. - Biotechs are using data to guide decisions and are incorporating domain informed machine learning as a core part of the molecular design process. There is great synergy here with better tooling as a means of abundant, cheap data. Look to Recursion, Manifold Bio, Dyno Tx, Asimov. There is also the cross pollination of biology informed architectures with the recent explosion in new machine learning techniques. These models are starting to do useful things, like generate functional gene editing proteins and entire prokaryotic organisms. Look to ESM, AlphaFold3, RFdiffusion. - New classes of therapies - genetic medicines and engineered immune cells - are having real success in the clinic. One dose cures for cardiovascular disease (Verve w ACSD), vaccines for cancer (Moderna w mRNA 4157), in vivo gene editing proteins (CRISPR Tx, Beam, Ensoma), metabolic disease (Novartis, Eli Lily w GLP-1/GIP modulators) are being dosed in real people right now + transforming lives. - The AI craze is commoditizing accelerated hardware and fast storage devices like NVMes, improving developer frameworks for writing code against these devices and maturing the systems tooling for moving around lots of data between computers for distributed training. One happy accident of this bubble will be the reuse of these components to build a new systems stack for the large scale processing of molecular data. This will be very important to construct a 1 billion single cell atlas and beyond. (For reference, the state of the art is ScaleBio's 2M cell kit, dubbed QuantumScale, and it is pushing things with the hardware + software we have today.) - Language models might be the perfect tool to distill the unstructured corpus of public data, literature, and methods sitting around on the Internet into real biological insights for scientist asking questions in natural language. It will also allow them to install, configure and run the slew of useful but poorly maintained academic computational tools to explore and hypothesize new biology on their own. Increasing the productivity of each scientist will do much to reverse Eroom's law. - There is an appetite from the market for new applications of biotech beyond drug development. Bacteria driven lithium mining (maverick), cell agriculture (growing cows in vats), early signs of consumer biologics (Geltor), biofoundries (Ginkgo). My guess is some of the greatest minds of our generation will want to do more than perturb the human body with therapies. I think it is also important to recognize that the need for computers and software is a secular trend in the progression of biotech independent of the interests of Silicon Valley. Clusters of computers, the information they store and the software that runs on them are precisely the technology needed by this field as it transforms into a discipline of information management, towards reducing living things into well characterized building blocks we can rebuild in our image. Software companies, as some local and hyper efficient structure in the arc of capitalism, with established methods and well trod rails to attract resources, talent and easily distribute product to an entire market, are the perfect place to incubate and disseminate these tools. There will probably be many, very large computer companies in biology in the next century.

Kenny Workman

113,534 Aufrufe • vor 1 Jahr

PHOTON COUNTING CT is NOT a better CT It is a NEW imaging modality Photon Counting CT (PCCT) represents a transformative leap in medical imaging, not only as a molecular imaging modality but also as a technology offering ultra-high resolution and functional imaging capabilities. It is fundamentally more than just an enhanced version of traditional CT—PCCT introduces new ways of seeing and understanding the human body, providing critical insights at the molecular, structural, and functional levels. This positions PCCT as a unique imaging modality that requires a fresh approach to technical implementation, operational workflows, and financial planning. Despite the larger upfront investment, PCCT’s ability to drastically reduce downstream healthcare costs makes it a highly valuable investment in the long run. 1. Technical Innovations • Molecular Imaging and Energy Discrimination: Unlike traditional CT, which simply measures the total absorbed energy, PCCT counts individual X-ray photons and differentiates their energy levels. This allows for precise molecular imaging, revealing the composition of tissues and materials at a biochemical level. By distinguishing between different tissue types and contrast agents, PCCT opens up new diagnostic possibilities, such as identifying molecular biomarkers in tumors or distinguishing between stable and unstable plaque in coronary arteries. This capability shifts the focus of imaging from purely anatomical to both anatomical and molecular, offering more comprehensive diagnostic information. • Ultra-High Spatial Resolution: PCCT features significantly smaller detector elements compared to conventional CT scanners, allowing for ultra-high resolution imaging. This means clinicians can visualize fine structures such as microcalcifications in arteries, small lesions in soft tissues, or the intricate architecture of bones. This level of detail was previously unattainable with traditional CT. When combined with molecular imaging, this ultra-high resolution allows for the precise localization and characterization of disease at very early stages, which is essential for early diagnosis and intervention. • Functional Imaging Capabilities: PCCT also excels as a functional imaging modality. By capturing energy-resolved information, PCCT can provide insights into tissue functionality and dynamic physiological processes. For instance, it can detect changes in blood flow, tissue perfusion, and oxygenation without the need for additional contrast agents or scans. This functionality allows for real-time assessment of physiological processes, making it particularly valuable in cardiology, oncology, and neurology for evaluating organ function and monitoring disease progression. • Reduced Noise and Artifact Reduction: Photon-counting technology dramatically reduces electronic noise and imaging artifacts, such as beam hardening, resulting in clearer and more accurate images. The ability to deliver ultra-high resolution images with minimal artifacts improves diagnostic accuracy, reducing the need for repeat scans and ensuring that even subtle abnormalities are detected. 2. Operational Considerations • New Workflow for Molecular, High-Resolution, and Functional Imaging: The integration of molecular, ultra-high resolution, and functional imaging into routine clinical workflows introduces complexity that requires adaptation. Radiologists and technicians need specialized training to interpret and analyze multi-energy datasets that include molecular and functional information. PCCT produces a vast amount of detailed data, requiring clinicians to adopt new imaging protocols and refine their diagnostic approaches to fully leverage its capabilities. • Post-Processing and Data Management: PCCT generates richer, more complex datasets, which necessitates advanced post-processing tools and data management systems. Existing PACS and imaging software may not be equipped to handle such large volumes of data or to process functional and molecular information effectively. This means healthcare institutions must invest in robust IT infrastructure, including upgraded software and storage solutions, as well as provide additional training for staff on new imaging analysis techniques. • Revised Clinical Protocols: The molecular, functional, and ultra-high resolution imaging capabilities of PCCT will likely prompt changes in clinical protocols. For instance, the need for contrast agents may be reduced, simplifying patient preparation and decreasing the risk of adverse reactions. Additionally, the ability to monitor physiological functions in real-time through functional imaging could lead to more dynamic diagnostic procedures, such as assessing the effectiveness of interventions or treatments in real-time. 3. Financial Impact • Higher Initial Investment: PCCT systems are more expensive than traditional CT scanners due to their advanced technology, which includes photon-counting detectors and the computational power required for high-resolution, molecular, and functional imaging. While this upfront cost is significant, it is crucial to view it in the broader context of the downstream benefits and cost reductions that PCCT offers. • Downstream Cost Reductions: Although the initial capital investment is higher, PCCT’s ability to combine molecular, functional, and ultra-high resolution imaging leads to substantial reductions in downstream healthcare costs. Its superior diagnostic accuracy minimizes the need for follow-up tests, repeat scans, or invasive diagnostic procedures, such as diagnostic coronary angiographies. For example, in cardiology, PCCT can precisely differentiate between types of coronary plaque, reducing the need for invasive procedures to assess risk. • Lower Overall Healthcare Expenditures: By enabling earlier, more accurate diagnoses, PCCT can reduce the overall cost of patient care. Early detection of disease, particularly through its molecular and functional imaging capabilities, allows for more targeted treatments, potentially preventing the need for more aggressive and expensive interventions down the line. For instance, early-stage tumor detection via molecular imaging could lead to less invasive treatments, reducing hospital stays and improving patient outcomes, ultimately driving down healthcare costs. • Increased ROI Through Enhanced Patient Outcomes: Over time, the combination of molecular, functional, and ultra-high resolution imaging enhances diagnostic precision, which translates into better patient outcomes. Improved diagnostic accuracy reduces the incidence of unnecessary procedures, minimizes treatment delays, and results in more personalized and effective care. This leads to increased patient satisfaction, better healthcare outcomes, and greater patient throughput—all factors that improve the institution’s return on investment (ROI). • Competitive Advantage and New Revenue Streams: By adopting PCCT, healthcare institutions position themselves at the forefront of advanced imaging technologies. The ability to offer molecular, functional, and ultra-high resolution imaging creates a competitive advantage, attracting more complex and high-value cases. This can boost the institution’s reputation for excellence in diagnostics, leading to increased referrals, new patient populations, and expanded revenue opportunities. Summary Photon Counting CT (PCCT) is not just an evolution of existing CT technology—it is a molecular, ultra-high resolution, and functional imaging modality that fundamentally transforms the diagnostic landscape. Its ability to capture detailed molecular data, visualize minute anatomical structures with ultra-high resolution, and provide real-time functional imaging opens new possibilities for earlier and more precise diagnoses. While the financial investment in PCCT is larger, the reduction in downstream healthcare costs through improved diagnostic accuracy, fewer unnecessary interventions, and earlier disease detection far outweighs the initial expense. For institutions committed to advancing patient care and improving long-term financial outcomes, PCCT is an essential investment in the future of medical imaging. The video attached shows a patient accessing the Hospital for ACS. PCCT can provide ALL the imaging information of the concurrent imaging modalities (CXR, CAG, Echo, CMR) that you see around it... that's a lot! #PhotonCountingCT #MolecularImaging #UltraHighResolution #FunctionalImaging #FutureOfImaging #AdvancedMedicalImaging #EarlyDiseaseDetection #InnovativeCT #CuttingEdgeHealthcare #PrecisionDiagnostics #HealthcareInnovation #MedicalTechnology #CostEffectiveImaging #NextGenCT #PatientCareRevolution
1:10

PHOTON COUNTING CT is NOT a better CT It is a NEW imaging modality Photon Counting CT (PCCT) represents a transformative leap in medical imaging, not only as a molecular imaging modality but also as a technology offering ultra-high resolution and functional imaging capabilities. It is fundamentally more than just an enhanced version of traditional CT—PCCT introduces new ways of seeing and understanding the human body, providing critical insights at the molecular, structural, and functional levels. This positions PCCT as a unique imaging modality that requires a fresh approach to technical implementation, operational workflows, and financial planning. Despite the larger upfront investment, PCCT’s ability to drastically reduce downstream healthcare costs makes it a highly valuable investment in the long run. 1. Technical Innovations • Molecular Imaging and Energy Discrimination: Unlike traditional CT, which simply measures the total absorbed energy, PCCT counts individual X-ray photons and differentiates their energy levels. This allows for precise molecular imaging, revealing the composition of tissues and materials at a biochemical level. By distinguishing between different tissue types and contrast agents, PCCT opens up new diagnostic possibilities, such as identifying molecular biomarkers in tumors or distinguishing between stable and unstable plaque in coronary arteries. This capability shifts the focus of imaging from purely anatomical to both anatomical and molecular, offering more comprehensive diagnostic information. • Ultra-High Spatial Resolution: PCCT features significantly smaller detector elements compared to conventional CT scanners, allowing for ultra-high resolution imaging. This means clinicians can visualize fine structures such as microcalcifications in arteries, small lesions in soft tissues, or the intricate architecture of bones. This level of detail was previously unattainable with traditional CT. When combined with molecular imaging, this ultra-high resolution allows for the precise localization and characterization of disease at very early stages, which is essential for early diagnosis and intervention. • Functional Imaging Capabilities: PCCT also excels as a functional imaging modality. By capturing energy-resolved information, PCCT can provide insights into tissue functionality and dynamic physiological processes. For instance, it can detect changes in blood flow, tissue perfusion, and oxygenation without the need for additional contrast agents or scans. This functionality allows for real-time assessment of physiological processes, making it particularly valuable in cardiology, oncology, and neurology for evaluating organ function and monitoring disease progression. • Reduced Noise and Artifact Reduction: Photon-counting technology dramatically reduces electronic noise and imaging artifacts, such as beam hardening, resulting in clearer and more accurate images. The ability to deliver ultra-high resolution images with minimal artifacts improves diagnostic accuracy, reducing the need for repeat scans and ensuring that even subtle abnormalities are detected. 2. Operational Considerations • New Workflow for Molecular, High-Resolution, and Functional Imaging: The integration of molecular, ultra-high resolution, and functional imaging into routine clinical workflows introduces complexity that requires adaptation. Radiologists and technicians need specialized training to interpret and analyze multi-energy datasets that include molecular and functional information. PCCT produces a vast amount of detailed data, requiring clinicians to adopt new imaging protocols and refine their diagnostic approaches to fully leverage its capabilities. • Post-Processing and Data Management: PCCT generates richer, more complex datasets, which necessitates advanced post-processing tools and data management systems. Existing PACS and imaging software may not be equipped to handle such large volumes of data or to process functional and molecular information effectively. This means healthcare institutions must invest in robust IT infrastructure, including upgraded software and storage solutions, as well as provide additional training for staff on new imaging analysis techniques. • Revised Clinical Protocols: The molecular, functional, and ultra-high resolution imaging capabilities of PCCT will likely prompt changes in clinical protocols. For instance, the need for contrast agents may be reduced, simplifying patient preparation and decreasing the risk of adverse reactions. Additionally, the ability to monitor physiological functions in real-time through functional imaging could lead to more dynamic diagnostic procedures, such as assessing the effectiveness of interventions or treatments in real-time. 3. Financial Impact • Higher Initial Investment: PCCT systems are more expensive than traditional CT scanners due to their advanced technology, which includes photon-counting detectors and the computational power required for high-resolution, molecular, and functional imaging. While this upfront cost is significant, it is crucial to view it in the broader context of the downstream benefits and cost reductions that PCCT offers. • Downstream Cost Reductions: Although the initial capital investment is higher, PCCT’s ability to combine molecular, functional, and ultra-high resolution imaging leads to substantial reductions in downstream healthcare costs. Its superior diagnostic accuracy minimizes the need for follow-up tests, repeat scans, or invasive diagnostic procedures, such as diagnostic coronary angiographies. For example, in cardiology, PCCT can precisely differentiate between types of coronary plaque, reducing the need for invasive procedures to assess risk. • Lower Overall Healthcare Expenditures: By enabling earlier, more accurate diagnoses, PCCT can reduce the overall cost of patient care. Early detection of disease, particularly through its molecular and functional imaging capabilities, allows for more targeted treatments, potentially preventing the need for more aggressive and expensive interventions down the line. For instance, early-stage tumor detection via molecular imaging could lead to less invasive treatments, reducing hospital stays and improving patient outcomes, ultimately driving down healthcare costs. • Increased ROI Through Enhanced Patient Outcomes: Over time, the combination of molecular, functional, and ultra-high resolution imaging enhances diagnostic precision, which translates into better patient outcomes. Improved diagnostic accuracy reduces the incidence of unnecessary procedures, minimizes treatment delays, and results in more personalized and effective care. This leads to increased patient satisfaction, better healthcare outcomes, and greater patient throughput—all factors that improve the institution’s return on investment (ROI). • Competitive Advantage and New Revenue Streams: By adopting PCCT, healthcare institutions position themselves at the forefront of advanced imaging technologies. The ability to offer molecular, functional, and ultra-high resolution imaging creates a competitive advantage, attracting more complex and high-value cases. This can boost the institution’s reputation for excellence in diagnostics, leading to increased referrals, new patient populations, and expanded revenue opportunities. Summary Photon Counting CT (PCCT) is not just an evolution of existing CT technology—it is a molecular, ultra-high resolution, and functional imaging modality that fundamentally transforms the diagnostic landscape. Its ability to capture detailed molecular data, visualize minute anatomical structures with ultra-high resolution, and provide real-time functional imaging opens new possibilities for earlier and more precise diagnoses. While the financial investment in PCCT is larger, the reduction in downstream healthcare costs through improved diagnostic accuracy, fewer unnecessary interventions, and earlier disease detection far outweighs the initial expense. For institutions committed to advancing patient care and improving long-term financial outcomes, PCCT is an essential investment in the future of medical imaging. The video attached shows a patient accessing the Hospital for ACS. PCCT can provide ALL the imaging information of the concurrent imaging modalities (CXR, CAG, Echo, CMR) that you see around it... that's a lot! #PhotonCountingCT #MolecularImaging #UltraHighResolution #FunctionalImaging #FutureOfImaging #AdvancedMedicalImaging #EarlyDiseaseDetection #InnovativeCT #CuttingEdgeHealthcare #PrecisionDiagnostics #HealthcareInnovation #MedicalTechnology #CostEffectiveImaging #NextGenCT #PatientCareRevolution

Dr. Filippo Cademartiri

11,820 Aufrufe • vor 1 Jahr

The $AEGIS DApp portal is now open to all: 🛡️ At Aegis, we believe in empowering the blockchain full of security, transparency and innovation. The Aegis Dapp has been under development for several months prior to the launch of $AEGIS and with that we have been able to build what we believe has the potential to change how users go about their day to day security. We are thrilled to share our progress and truly exciting news with you all. 🎯 First things first, at Aegis, we want to make it clear that the value of what we seek to bring to security across the blockchain, comes from our big vision, our strong team, and our commitment to long-term goals. ℹ️ Let’s kick this off with some information that is constantly happening, which is behind the scenes. Our full team is dedicated to the opportunity that lays ahead of us with becoming the leading voice/name for security, grasping every aspect with innovation, hard work, passion and commitment to see this sector grow. Everyone is aware of how important security is, a heartwarming mention to Messari for including us on how they see this sector growing rapidly and pushing a 10 Billion evaluation. We take that recognition with full responsibility and gratitude as we've been working hard on some really powerful stuff that could change the game for our industry. If you read the title and report itself, I’m sure that’ll give you some insight to what’s coming, and to the vast extent of what you can expect Aegis to be working towards. —> 🤝 This comes from teaming up with others within this sector and coming up with new tech to projects driven by our community, within the pipeline you can be confident that what we are building will push the cryptocurrency industry as a whole into a better future, the magnitude to what Aegis brings will not stop until we can confidently say, “Negative security reports across the blockchain are at an all time low, thousands of users are satisfied that Aegis is protecting them and their assets.” We're sticking to our vision no matter what the market does or whatever else comes our way. We plan to build what we set out to and we will see to it that our ecosystem is met. We've been working on some pretty amazing products that will be available within our Dapp, let’s go over what we offer: * AI Audits * Live Monitoring * Penetration Testing * Bug Bounties * Live Watchdog * Token analytics for everyday users, developers, teams, auditors, institutions, investors. ⬇️ Let’s break it down for you in some simple steps: AI AUDITS: We have trained our LLM models as AI AGENTS, these consist of 3 people ( AI AGENTS ) for the audits that are performed. - Audit - Reviewer - Judge Each one analyzes with a different personality, let’s check what personalities our AI AGENTS consist of: 3 different perspective auditors. 1 - Fine-tuned model x amount reads the code and generates the audit. ✅ 2 - Model x amount reviews the code and fact checks thoroughly. ✅ 3 - Model x amount ranks the code based on the severity outcome. ✅ ⌚️ Live Monitoring/Watchdog: The Live Monitoring/Watchdog system is designed to provide real-time surveillance of smart contracts, ensuring the detection and prevention of any potentially harmful transactions or malicious activities. Through the utilization of an AI Agent model, the system is trained to proactively identify and thwart suspicious behavior, thereby safeguarding the integrity of the smart contracts. Also, a paid sophisticated threat detection model is available for more intricate protocols and Dapps, offering an advanced level of protection against potential threats. This proactive approach is crucial in mitigating the risk of exploitation and ensuring the security of the smart contract ecosystem. 🖊️ Pen Testing: Our platform offers Pen Testing services to developers, providing a controlled environment for whitehat hackers to simulate attacks and identify vulnerabilities in smart contracts and protocols. In addition to human whitehat hackers, our AI Agents function as Red and Blue teams, actively engaging in simulated attacks to stress-test protocols and identify potential weaknesses. This comprehensive approach allows developers to proactively identify and address security issues, ultimately enhancing the robustness and resilience of their projects. 🕷️ Bug Bounties: Our Bug Bounty listing platform provides developers with the opportunity to list their protocols and offer bounties to white hat hackers for identifying vulnerabilities. By aggregating millions of bounties from various platforms and utilizing AI tools, we streamline the testing process, reducing up to 80% of the workload typically associated with security testing. This allows developers to efficiently identify and address potential vulnerabilities in their protocols, ultimately enhancing the overall security and resilience of their projects. 🪙 And lot more token analytics features for regular users, this will give you the opportunity to explore our Dapp for yourself and have some fun diving into the security platform of the future! I’m sure you’re excited to try it all out yourself, which is why we have some exciting news to bring to the #Guardians of the blockchain! But just before you continue the read and see the beans have been spilled, we have to take this opportunity to share with you that this large step to becoming a security leader is but only 20% of what we have revealed. This will be at the core of what Aegis stands for and hopes to achieve. The focus here is upon our Dapp, and in time we will slowly bring forward information/updates regarding segments of what makes Aegis a force to be reckoned with. Now that you’re fired up and excited to all of the announcements to come, let’s get to the news you’ve been waiting for! 🎉 We’re spilling the good news, and are happy to say we are now set for public release! The team at Aegis are overwhelmed with the development, support from teams, community, partners and more on what we believe to be an institutional-grade product. But the fun doesn’t stop there, this marks the start of what we aim to become, as it will take time and cycles to become better and better. Constant advancements will be set in place to attain the goal of achieving blockchain security. A statement from our CEO- Brian Hunt: “I can confirm from the security conferences I attended with Centralized security firms Peckshield, Hacken, Certik, BlockSec presentations, they are trying to achieve something similar and it will take them years. Decentralized AI for Security!” This initial drop of our dapp will be to get users signed up to gain access, in which we’ll whitelist users to get the ball rolling. 📣 To end this segment, let’s get the party started with the long awaited Aegis Ai Security Dapp and sign up now!
2:06

The $AEGIS DApp portal is now open to all: 🛡️ At Aegis, we believe in empowering the blockchain full of security, transparency and innovation. The Aegis Dapp has been under development for several months prior to the launch of $AEGIS and with that we have been able to build what we believe has the potential to change how users go about their day to day security. We are thrilled to share our progress and truly exciting news with you all. 🎯 First things first, at Aegis, we want to make it clear that the value of what we seek to bring to security across the blockchain, comes from our big vision, our strong team, and our commitment to long-term goals. ℹ️ Let’s kick this off with some information that is constantly happening, which is behind the scenes. Our full team is dedicated to the opportunity that lays ahead of us with becoming the leading voice/name for security, grasping every aspect with innovation, hard work, passion and commitment to see this sector grow. Everyone is aware of how important security is, a heartwarming mention to Messari for including us on how they see this sector growing rapidly and pushing a 10 Billion evaluation. We take that recognition with full responsibility and gratitude as we've been working hard on some really powerful stuff that could change the game for our industry. If you read the title and report itself, I’m sure that’ll give you some insight to what’s coming, and to the vast extent of what you can expect Aegis to be working towards. —> 🤝 This comes from teaming up with others within this sector and coming up with new tech to projects driven by our community, within the pipeline you can be confident that what we are building will push the cryptocurrency industry as a whole into a better future, the magnitude to what Aegis brings will not stop until we can confidently say, “Negative security reports across the blockchain are at an all time low, thousands of users are satisfied that Aegis is protecting them and their assets.” We're sticking to our vision no matter what the market does or whatever else comes our way. We plan to build what we set out to and we will see to it that our ecosystem is met. We've been working on some pretty amazing products that will be available within our Dapp, let’s go over what we offer: * AI Audits * Live Monitoring * Penetration Testing * Bug Bounties * Live Watchdog * Token analytics for everyday users, developers, teams, auditors, institutions, investors. ⬇️ Let’s break it down for you in some simple steps: AI AUDITS: We have trained our LLM models as AI AGENTS, these consist of 3 people ( AI AGENTS ) for the audits that are performed. - Audit - Reviewer - Judge Each one analyzes with a different personality, let’s check what personalities our AI AGENTS consist of: 3 different perspective auditors. 1 - Fine-tuned model x amount reads the code and generates the audit. ✅ 2 - Model x amount reviews the code and fact checks thoroughly. ✅ 3 - Model x amount ranks the code based on the severity outcome. ✅ ⌚️ Live Monitoring/Watchdog: The Live Monitoring/Watchdog system is designed to provide real-time surveillance of smart contracts, ensuring the detection and prevention of any potentially harmful transactions or malicious activities. Through the utilization of an AI Agent model, the system is trained to proactively identify and thwart suspicious behavior, thereby safeguarding the integrity of the smart contracts. Also, a paid sophisticated threat detection model is available for more intricate protocols and Dapps, offering an advanced level of protection against potential threats. This proactive approach is crucial in mitigating the risk of exploitation and ensuring the security of the smart contract ecosystem. 🖊️ Pen Testing: Our platform offers Pen Testing services to developers, providing a controlled environment for whitehat hackers to simulate attacks and identify vulnerabilities in smart contracts and protocols. In addition to human whitehat hackers, our AI Agents function as Red and Blue teams, actively engaging in simulated attacks to stress-test protocols and identify potential weaknesses. This comprehensive approach allows developers to proactively identify and address security issues, ultimately enhancing the robustness and resilience of their projects. 🕷️ Bug Bounties: Our Bug Bounty listing platform provides developers with the opportunity to list their protocols and offer bounties to white hat hackers for identifying vulnerabilities. By aggregating millions of bounties from various platforms and utilizing AI tools, we streamline the testing process, reducing up to 80% of the workload typically associated with security testing. This allows developers to efficiently identify and address potential vulnerabilities in their protocols, ultimately enhancing the overall security and resilience of their projects. 🪙 And lot more token analytics features for regular users, this will give you the opportunity to explore our Dapp for yourself and have some fun diving into the security platform of the future! I’m sure you’re excited to try it all out yourself, which is why we have some exciting news to bring to the #Guardians of the blockchain! But just before you continue the read and see the beans have been spilled, we have to take this opportunity to share with you that this large step to becoming a security leader is but only 20% of what we have revealed. This will be at the core of what Aegis stands for and hopes to achieve. The focus here is upon our Dapp, and in time we will slowly bring forward information/updates regarding segments of what makes Aegis a force to be reckoned with. Now that you’re fired up and excited to all of the announcements to come, let’s get to the news you’ve been waiting for! 🎉 We’re spilling the good news, and are happy to say we are now set for public release! The team at Aegis are overwhelmed with the development, support from teams, community, partners and more on what we believe to be an institutional-grade product. But the fun doesn’t stop there, this marks the start of what we aim to become, as it will take time and cycles to become better and better. Constant advancements will be set in place to attain the goal of achieving blockchain security. A statement from our CEO- Brian Hunt: “I can confirm from the security conferences I attended with Centralized security firms Peckshield, Hacken, Certik, BlockSec presentations, they are trying to achieve something similar and it will take them years. Decentralized AI for Security!” This initial drop of our dapp will be to get users signed up to gain access, in which we’ll whitelist users to get the ball rolling. 📣 To end this segment, let’s get the party started with the long awaited Aegis Ai Security Dapp and sign up now!

AEGIS AI

127,936 Aufrufe • vor 2 Jahren

Fast Company just published a great piece on World Labs , Fei-Fei Li , Marble, and the idea that spatial intelligence / world models may be one of the next big shifts in AI. I was happy to be quoted in the article, but I also wanted to share more context about my own experience with World Labs and Marble, and why this direction is especially interesting to me. My starting point: volumetric capture — For the past few years I’ve been exploring and using volumetric capture and reconstruction (photogrammetry, NeRFs, 3D Gaussian Splats) mostly capturing locations around Montreal. Alleys, museums, urban interiors. I love every step of it: the capture itself, the pipeline, and what can be done with the output. Turning real spaces into real-time explorable systems. I do this personally, sharing explorations here, and professionally as chief technologist, and co-founder of Dpt. Physical reality + generative manipulation — In my work I’m especially drawn to mixing physical reality with generative and digital manipulation: using physical interfaces (light, clay, ink, ... ) to drive generative AI pipelines, building mixed reality prototypes that reshape your surroundings, or starting from real captured spaces and transforming them using tools like Marble. Like many people, I saw the World Labs announcement on Twitter in September 2024, and Marble when it surfaced in early December. But by then, I already had a sense something was coming. The first conversation — As someone deep into volumetric capture and radiance fields, I obviously knew about Ben Mildenhall and his pioneering work on NeRF. To my surprise, Ben reached out to me in late June 2024. He’d been following some of my experiments and wanted to chat about my process and workflows and how I was using this “stuff” creatively. At that point he didn’t share what he was building, but we had a genuinely great conversation about radiance fields, AI, and my work. He was curious about the creative perspective, not just the technical one. When the World Labs announcement dropped a few months later, it all made sense. I understood what Ben had been working on, and why the creative angle mattered to them. Then in August 2025, he invited me to try the Marble beta, and I’ve been experimenting with it since. Experimenting with Marble — The first thing I used Marble for was materializing scene and world concepts during ideation at the studio, and seeing if and how it could fit into our production pipeline. In parallel, I dove into a series of experiments focused on world manipulation: starting from real captured spaces and transforming them using Marble. I’d already been exploring that idea using img2img diffusion with ControlNet on NeRF renders, real-time video streams, and even mixed reality using headset camera feeds. But Marble brings something different. It generates persistent, spatially cohesive 3D worlds that can be rendered in real time across a wide range of devices. That’s a real shift. Experiment 01: Parallel Realities — The first experiment, Parallel Realities, starts from a volumetric capture of a real location, reconstructed as 3D Gaussian Splats. Using Marble, I generate an alternate version of that same space, something informed by the original architecture: abandoned, nature-reclaimed, alternate era. Then, using Spark (World Labs’ 3D Gaussian Splatting renderer for THREE.js) I make both realities coexist in the same spatial coordinate system. From there, I use a portal UX mechanic to let the user step between the real reconstruction and the Marble-generated version. Experiment 02: Hidden Depth The second experiment, Hidden Depth, does not transform a space as much as expand it. A captured location has a visual boundary (a mural, a doorway, a dark corridor) and Marble generates what exists beyond it. For example: a Montreal alley has a painted mural; step through it and you’re inside a world informed by what is actually depicted there. World Labs showcased part of this work here: And in their Spark 2.0 post: The project page is here: Why this matters to me — Being able to start from a real 3D Gaussian Splat scene and manipulate it with Marble opens up a lot of ideas. The 3DGS pipeline is becoming an increasingly compelling foundation for exploration, experimentation, and storytelling. What matters most to me right now is more control. The more I can steer the generated scene or world, the more useful the tool becomes. I want more features like the already existing multiple input images and Chisel, the blockout-based approach. I would like better local control, the ability to expand a generated world more and more while preserving coherence, and the ability to directly import 3D Gaussian Splat scenes to be used as a starting point. I want more ways to shape the result, not just a “prompt and hope” approach. — It is exciting to see this field moving from research and demos toward actual creative workflows.
1:04

Fast Company just published a great piece on World Labs , Fei-Fei Li , Marble, and the idea that spatial intelligence / world models may be one of the next big shifts in AI. I was happy to be quoted in the article, but I also wanted to share more context about my own experience with World Labs and Marble, and why this direction is especially interesting to me. My starting point: volumetric capture — For the past few years I’ve been exploring and using volumetric capture and reconstruction (photogrammetry, NeRFs, 3D Gaussian Splats) mostly capturing locations around Montreal. Alleys, museums, urban interiors. I love every step of it: the capture itself, the pipeline, and what can be done with the output. Turning real spaces into real-time explorable systems. I do this personally, sharing explorations here, and professionally as chief technologist, and co-founder of Dpt. Physical reality + generative manipulation — In my work I’m especially drawn to mixing physical reality with generative and digital manipulation: using physical interfaces (light, clay, ink, ... ) to drive generative AI pipelines, building mixed reality prototypes that reshape your surroundings, or starting from real captured spaces and transforming them using tools like Marble. Like many people, I saw the World Labs announcement on Twitter in September 2024, and Marble when it surfaced in early December. But by then, I already had a sense something was coming. The first conversation — As someone deep into volumetric capture and radiance fields, I obviously knew about Ben Mildenhall and his pioneering work on NeRF. To my surprise, Ben reached out to me in late June 2024. He’d been following some of my experiments and wanted to chat about my process and workflows and how I was using this “stuff” creatively. At that point he didn’t share what he was building, but we had a genuinely great conversation about radiance fields, AI, and my work. He was curious about the creative perspective, not just the technical one. When the World Labs announcement dropped a few months later, it all made sense. I understood what Ben had been working on, and why the creative angle mattered to them. Then in August 2025, he invited me to try the Marble beta, and I’ve been experimenting with it since. Experimenting with Marble — The first thing I used Marble for was materializing scene and world concepts during ideation at the studio, and seeing if and how it could fit into our production pipeline. In parallel, I dove into a series of experiments focused on world manipulation: starting from real captured spaces and transforming them using Marble. I’d already been exploring that idea using img2img diffusion with ControlNet on NeRF renders, real-time video streams, and even mixed reality using headset camera feeds. But Marble brings something different. It generates persistent, spatially cohesive 3D worlds that can be rendered in real time across a wide range of devices. That’s a real shift. Experiment 01: Parallel Realities — The first experiment, Parallel Realities, starts from a volumetric capture of a real location, reconstructed as 3D Gaussian Splats. Using Marble, I generate an alternate version of that same space, something informed by the original architecture: abandoned, nature-reclaimed, alternate era. Then, using Spark (World Labs’ 3D Gaussian Splatting renderer for THREE.js) I make both realities coexist in the same spatial coordinate system. From there, I use a portal UX mechanic to let the user step between the real reconstruction and the Marble-generated version. Experiment 02: Hidden Depth The second experiment, Hidden Depth, does not transform a space as much as expand it. A captured location has a visual boundary (a mural, a doorway, a dark corridor) and Marble generates what exists beyond it. For example: a Montreal alley has a painted mural; step through it and you’re inside a world informed by what is actually depicted there. World Labs showcased part of this work here: And in their Spark 2.0 post: The project page is here: Why this matters to me — Being able to start from a real 3D Gaussian Splat scene and manipulate it with Marble opens up a lot of ideas. The 3DGS pipeline is becoming an increasingly compelling foundation for exploration, experimentation, and storytelling. What matters most to me right now is more control. The more I can steer the generated scene or world, the more useful the tool becomes. I want more features like the already existing multiple input images and Chisel, the blockout-based approach. I would like better local control, the ability to expand a generated world more and more while preserving coherence, and the ability to directly import 3D Gaussian Splat scenes to be used as a starting point. I want more ways to shape the result, not just a “prompt and hope” approach. — It is exciting to see this field moving from research and demos toward actual creative workflows.

Hugues Bruyère

65,393 Aufrufe • vor 1 Monat

A new mechanism for “RNA memory”! 😱 Thrilled to share another crazy paper from the lab (can’t believe we posted 2 in 2 days!), summarizing >10 years of research: Work on transgenerational inheritance of small RNAs in the powerful model organism C. elegans changed how we think about what’s possible in inheritance and evolution, because it allows the most heretical thing: inheritance of parental responses to the environment! However, it’s still unclear whether RNAs are inherited across generations in other animals, largely because the RNA-dependent RNA polymerases that amplify heritable small RNAs and prevent their dilution in C. elegans are not conserved in mammals. In this new work, an amazing collaboration with the Rink and Wurtzel labs, we show that planarians establish long-lasting and heritable small RNA–based gene regulatory states despite lacking canonical RNA-dependent RNA polymerases and nuclear RNAi machinery (that are required in C. elegans). You might say “they are both worms…” BUT planarians are evolutionarily very distant from C. elegans (flatworms vs. roundworms, diverged more than 500 million years ago), making this particularly surprising. These are totally different animals. We find that ingestion of double-stranded RNA induces sequence-specific silencing that persists for months and survives repeated cycles of whole-body regeneration. Even more strikingly, RNAi can be transferred between animals, echoing James V. McConnell’s controversial “RNA memory” experiments from the 1970s (his lab was targeted by the Unabomber terrorist Ted Kaczynski, who sent McConnell a bomb. This and other controversies ended this line of experiments…) Mechanistically, we find that the response transitions from a transient systemic dsRNA-triggered phase to a stable, cell-autonomous post-transcriptional “memory phase” maintained by antisense small RNAs. Using a new luminescence reporter (transgenesis is currently impossible in planarians), we show that silencing spreads along the targeted gene and identify a weird type of planarian small RNAs with untemplated polyA tails. RNAi inheritance without canonical RdRPs establishes planarians as a powerful system for studying RNA-based regulatory inheritance beyond C. elegans and raises the possibility that RNA-mediated inheritance may be more broadly conserved in animals, potentially even in mammals. Here’s a video of a planarian that is treated by RNAi against β-catenin and develops multiple heads instead of just one. This is one of the phenotypes that is inherited. Another phenotype is “loss of eyes” (which we show is not only inherited across multiple regeneration cycles, but can also be transmitted between animals in transplantation experiments). Amazing work led by first authors Prakash Cherian and Idit Aviram (co-supervised by Omri and me). Please read the preprint, the link is in the next tweet, and share!
0:44

A new mechanism for “RNA memory”! 😱 Thrilled to share another crazy paper from the lab (can’t believe we posted 2 in 2 days!), summarizing >10 years of research: Work on transgenerational inheritance of small RNAs in the powerful model organism C. elegans changed how we think about what’s possible in inheritance and evolution, because it allows the most heretical thing: inheritance of parental responses to the environment! However, it’s still unclear whether RNAs are inherited across generations in other animals, largely because the RNA-dependent RNA polymerases that amplify heritable small RNAs and prevent their dilution in C. elegans are not conserved in mammals. In this new work, an amazing collaboration with the Rink and Wurtzel labs, we show that planarians establish long-lasting and heritable small RNA–based gene regulatory states despite lacking canonical RNA-dependent RNA polymerases and nuclear RNAi machinery (that are required in C. elegans). You might say “they are both worms…” BUT planarians are evolutionarily very distant from C. elegans (flatworms vs. roundworms, diverged more than 500 million years ago), making this particularly surprising. These are totally different animals. We find that ingestion of double-stranded RNA induces sequence-specific silencing that persists for months and survives repeated cycles of whole-body regeneration. Even more strikingly, RNAi can be transferred between animals, echoing James V. McConnell’s controversial “RNA memory” experiments from the 1970s (his lab was targeted by the Unabomber terrorist Ted Kaczynski, who sent McConnell a bomb. This and other controversies ended this line of experiments…) Mechanistically, we find that the response transitions from a transient systemic dsRNA-triggered phase to a stable, cell-autonomous post-transcriptional “memory phase” maintained by antisense small RNAs. Using a new luminescence reporter (transgenesis is currently impossible in planarians), we show that silencing spreads along the targeted gene and identify a weird type of planarian small RNAs with untemplated polyA tails. RNAi inheritance without canonical RdRPs establishes planarians as a powerful system for studying RNA-based regulatory inheritance beyond C. elegans and raises the possibility that RNA-mediated inheritance may be more broadly conserved in animals, potentially even in mammals. Here’s a video of a planarian that is treated by RNAi against β-catenin and develops multiple heads instead of just one. This is one of the phenotypes that is inherited. Another phenotype is “loss of eyes” (which we show is not only inherited across multiple regeneration cycles, but can also be transmitted between animals in transplantation experiments). Amazing work led by first authors Prakash Cherian and Idit Aviram (co-supervised by Omri and me). Please read the preprint, the link is in the next tweet, and share!

Oded Rechavi

194,366 Aufrufe • vor 4 Monaten

🎉 new skill unlocked: 20s uninterrupted, unstitched, single render from our new ai video engine: Nami. This is my birb (#7531) from the Moonbirds collection, idling in the library. patent: "Intra-Latent Semantic Injection via Cross-Spatial Encoding and Decoding during Multi-Pass Inference for Generative AI Video Creation" At Scrypted we've been quietly working on an agentic generative AI stack for two years: • integrating and testing w/ partners across the games & entertainment sectors • stealthily building a community of early believers through AVB • showcasing some of what we're doing with amazing projects like H011yw00d Agent. -- about Nami -- Nami is an agentic orchestration layer for AI video models: it unlocks their inner superpowers without making them rely on custom LoRAs or fine-tunings. Instead of throwing raw training power and tens of millions of dollars at training yet another ai video model: we figured out new ways to use what we have. Nami harnesses a multi-agent system to perform the work needed in taking a simple prompt or image and turning it into something bigger - much bigger. The agentic steps are allowed to manipulate latent space, digging into tensors, yet doing so in semantically aware chunks - meaning that Nami inherently supports video generation of arbitrary length, though it's bound to O(n) rendering time. (We do have some cool sharding tech that allows us to cut the generative time in half for a reference pose idle-animation like this demo). It's also fairly agnostic, picking and choosing the right tools for the job, and plays really well with emerging tech like FLUX Kontext, FramePack, or <- without being limited by any of them. -- use cases -- Even just a year or two ago the 20 second render below would cost a company, paying an agency, around $10k start-to-finish. This one cost me $6.25 on our dev hardware in an unoptimized environment. There's something mind-blowing about the state-of-the-art when we reduce costs to 0.0625% - less than 1% - of what we used to pay. It's also empowering. For creators. Game developers. Content influencers: you name it. -- superpowers -- 1. it does the things you ask for, in the order you asked for it 2. consistency is king 3. single-shot text or image-to-video 4. future videos can reference previous ones to seamlessly maintain style 5. semantic stitching: can't wait to showcase this -- gtm -- We think Generative AI Video, like image generation, like text, like games, should be a publicly accessible common good. We believe democratizing access to Nami in web3, via x402 payments proposed by Drew Coffman, or in World's mini-apps, is a bold step forward for digital freedom. Permissionless, decentralized, generative ai video. Naturally, we'll also soon release a web platform for using Nami in a traditionally SaaSy way: bring your own images, videos, or prompts and we'll take care of the rest. In the mid-term, Scrypted is building a stack of agentic skills (we call it AVB) and making them available to projects like H011yw00d Agent on Virtuals Protocol and other platforms. -- long-term vision -- Scrypted's mission is to decentralize the things that can't be decentralized. We participated in a16z crypto's CSX (London 2024) during our pre-seed specifically to research a new consensus protocol for hard things like AI video and AI agents: where there's no "one right answer". When Zero-Knowledge Proofs (ZKP) can't secure it, and Trusted Execution Environments (TEEs) are too small, we've got you covered with our upcoming Inori Network. -- how you can help -- 1. Are you a GPU farm? We're gonna need more flops. 2. Do you represent an L1 or L2? We want to build bridges. 3. Do you represent a Wallet or App creator? Let's get an endpoint exposed. 4. Are you an investor? Let's chat. 5. Like, repost, share! -- team background -- We come from a background of AI in the Video Game industry with each founder having over 20 years of experience at companies like Electronic Arts & Square Enix. -- contact -- DMs are open, reach out if you want to be an early tester for your site, game, collection, or project! -- try it out -- Go anywhere on X and tag H011yw00d Agent with a prompt and she'll give you a free 2 second render. Have fun making cinematic shorts or meme videos! -- thanks -- AWS Startups has been an incredible help scaling our prototypes. Also, shout out to all loyal beans 🫘 in the Autonomous Virtuals Beings (AVB) community. Nami has a very important role in the upcoming XP agent platform, can't wait to show you all. AVbeings
0:23

🎉 new skill unlocked: 20s uninterrupted, unstitched, single render from our new ai video engine: Nami. This is my birb (#7531) from the Moonbirds collection, idling in the library. patent: "Intra-Latent Semantic Injection via Cross-Spatial Encoding and Decoding during Multi-Pass Inference for Generative AI Video Creation" At Scrypted we've been quietly working on an agentic generative AI stack for two years: • integrating and testing w/ partners across the games & entertainment sectors • stealthily building a community of early believers through AVB • showcasing some of what we're doing with amazing projects like H011yw00d Agent. -- about Nami -- Nami is an agentic orchestration layer for AI video models: it unlocks their inner superpowers without making them rely on custom LoRAs or fine-tunings. Instead of throwing raw training power and tens of millions of dollars at training yet another ai video model: we figured out new ways to use what we have. Nami harnesses a multi-agent system to perform the work needed in taking a simple prompt or image and turning it into something bigger - much bigger. The agentic steps are allowed to manipulate latent space, digging into tensors, yet doing so in semantically aware chunks - meaning that Nami inherently supports video generation of arbitrary length, though it's bound to O(n) rendering time. (We do have some cool sharding tech that allows us to cut the generative time in half for a reference pose idle-animation like this demo). It's also fairly agnostic, picking and choosing the right tools for the job, and plays really well with emerging tech like FLUX Kontext, FramePack, or <- without being limited by any of them. -- use cases -- Even just a year or two ago the 20 second render below would cost a company, paying an agency, around $10k start-to-finish. This one cost me $6.25 on our dev hardware in an unoptimized environment. There's something mind-blowing about the state-of-the-art when we reduce costs to 0.0625% - less than 1% - of what we used to pay. It's also empowering. For creators. Game developers. Content influencers: you name it. -- superpowers -- 1. it does the things you ask for, in the order you asked for it 2. consistency is king 3. single-shot text or image-to-video 4. future videos can reference previous ones to seamlessly maintain style 5. semantic stitching: can't wait to showcase this -- gtm -- We think Generative AI Video, like image generation, like text, like games, should be a publicly accessible common good. We believe democratizing access to Nami in web3, via x402 payments proposed by Drew Coffman, or in World's mini-apps, is a bold step forward for digital freedom. Permissionless, decentralized, generative ai video. Naturally, we'll also soon release a web platform for using Nami in a traditionally SaaSy way: bring your own images, videos, or prompts and we'll take care of the rest. In the mid-term, Scrypted is building a stack of agentic skills (we call it AVB) and making them available to projects like H011yw00d Agent on Virtuals Protocol and other platforms. -- long-term vision -- Scrypted's mission is to decentralize the things that can't be decentralized. We participated in a16z crypto's CSX (London 2024) during our pre-seed specifically to research a new consensus protocol for hard things like AI video and AI agents: where there's no "one right answer". When Zero-Knowledge Proofs (ZKP) can't secure it, and Trusted Execution Environments (TEEs) are too small, we've got you covered with our upcoming Inori Network. -- how you can help -- 1. Are you a GPU farm? We're gonna need more flops. 2. Do you represent an L1 or L2? We want to build bridges. 3. Do you represent a Wallet or App creator? Let's get an endpoint exposed. 4. Are you an investor? Let's chat. 5. Like, repost, share! -- team background -- We come from a background of AI in the Video Game industry with each founder having over 20 years of experience at companies like Electronic Arts & Square Enix. -- contact -- DMs are open, reach out if you want to be an early tester for your site, game, collection, or project! -- try it out -- Go anywhere on X and tag H011yw00d Agent with a prompt and she'll give you a free 2 second render. Have fun making cinematic shorts or meme videos! -- thanks -- AWS Startups has been an incredible help scaling our prototypes. Also, shout out to all loyal beans 🫘 in the Autonomous Virtuals Beings (AVB) community. Nami has a very important role in the upcoming XP agent platform, can't wait to show you all. AVbeings

Tim Cotten

12,617 Aufrufe • vor 1 Jahr

Thermodynamic computing is here There is a new computing paradigm emerging from the noise, and its arrival may be as significant as the dawn of deep learning or the advent of cloud virtualization. A new company, Extropic, has just launched its first thermodynamic computer, a device they call a TSU, or Thermal Sampling Unit. While the web is already filling with deep technical dives, what’s more important for most of us is building a clear intuition for what this technology is, how it’s fundamentally different from anything that’s come before, and why it’s generating so much excitement. This isn’t just another chip; it’s a new way to think about computation itself. Seeing is Believing: Solving Puzzles in One Shot To understand what a TSU does, let’s look at two classic, notoriously difficult computer science problems: Sudoku and the Eight Queens problem. When you or I solve a Sudoku, we use a process of sequential logic, guess-and-check, and backtracking. We make an assumption, follow its logical conclusion, and if we hit a dead end, we erase and try again. A classical computer does the same, just much faster. A TSU, however, approaches this in a completely different way. Using a TSU simulator, one can “program” the problem by first clamping the known values—the clues already on the board. Then, you program in the constraints: no duplicate numbers in any row, column, or 3x3 square. With the problem thus defined, the TSU doesn’t “search” for a solution; it anneals one. In a single computational step, the solution simply emerges, backfilling all the empty squares correctly. The same principle applies to the Eight Queens problem, a challenge to place eight queens on a chessboard so that none can attack any other. This is a complex combinatorial problem with 92 distinct solutions. A classical computer would have to iteratively search for these. A TSU, by contrast, can be programmed with the constraints (the “anti-affinity” between queens on the same row, column, or diagonal) and then set to sample the “solution space.” In this context, a valid solution is one with a “problem energy” of zero. The TSU’s physical nature allows it to naturally find these zero-energy states. A simulation of this process shows the TSU discovering all 92 unique solutions, demonstrating its ability to not just find an answer, but to explore the entire landscape of all correct answers. This is a fundamentally new approach, one that bypasses the brute-force, iterative methods we’ve relied on for decades. The Physics of Computation: Using Noise, Not Fighting It This new power comes from a radical design philosophy. For the last 70 years, computing has been about one thing: order. We build chips that are deterministic, logical, and precise. The great enemy has always been noise, heat, and randomness. We spend billions on cooling and error correction to eliminate these very things. Quantum computing, in many ways, is the ultimate expression of this, requiring temperatures near absolute zero to eliminate all thermal noise and achieve quantum coherence. Thermodynamic computing is the polar opposite. It doesn’t fight the noise; it uses it. The TSU is built on the understanding that the natural, stochastic noise from “leaky” transistors—the very randomness we’ve tried to engineer out of existence—is itself a powerful computational resource. Think of it this way: a GPU, which is central to today’s AI, has to simulate noise. When a generative AI model creates a new image or sentence, it’s using complex algorithms to fake randomness. The TSU doesn’t need to fake it; it harnesses the actual physical randomness of thermodynamics. It is a piece of hardware that directly computes with probability. This makes it a hybrid, sitting somewhere between a purely analog computer (which might use light or sound waves to compute) and a digital GPU. It’s a physical device that leverages the laws of physics itself to find solutions, rather than just using logic gates to simulate them. From a Lost Hiker to a Million Bouncy Balls Perhaps the best way to build intuition is with a metaphor. Imagine that solving a complex optimization problem is like trying to find the lowest point of altitude in a 100-square-mile mountainous landscape. Classical computing, using an algorithm like gradient descent, is like being a single hiker dropped into this landscape at night. You have no map or satellite view. All you have is an altimeter and the sensation of the slope under your feet. You can only take one step at a time, always walking downhill, hoping you don’t get stuck in a small local valley when the true, lowest canyon is miles away. Thermodynamic computing is a completely different approach. It’s like having a million bouncy balls and a helicopter. You drop all million balls simultaneously across the entire 100-square-mile landscape. Then, you “turn on an earthquake,” shaking the entire system. The balls bounce and jostle, but as the shaking (the “annealing”) subsides, where do they all end up? They naturally settle into the lowest points. The balls that collect in the deepest valley represent the optimal solution. The TSU is, in essence, a physical device for dropping those million balls at once and letting the laws of thermodynamics find the lowest “energy” state for you, all at the same time. Beyond Puzzles: The Real-World Impact This is far more than just a clever way to solve brain teasers. This ability to instantly find the lowest energy state for a complex, constrained system has staggering real-world applications. One of the most immediate is protein folding. Companies like Google’s DeepMind have made incredible progress with AI like AlphaFold, which predicts protein structures. But this is still a predictive model trained on existing data. A TSU could potentially solve the folding problem directly, treating the protein as a system of atomic affinities and repulsions and finding its most stable, lowest-energy configuration almost instantaneously. This could revolutionize drug discovery and materials science. An even more profound possibility lies in nuclear fusion. One of the greatest engineering challenges in history is controlling the superheated plasma within a tokamak reactor. This requires shaping unimaginably complex magnetic containment fields in real-time to prevent the plasma from touching the reactor walls. This is a real-time optimization problem so complex it’s currently beyond our capabilities. A TSU, however, could be fast enough. Its ability to compute with electricity itself, rather than abstracting the problem through layers of software, might allow it to update the magnetic fields fast enough to stabilize the fusion reaction. One could even imagine a future where thermodynamic computing elements are built directly into the tokamak’s walls, allowing the reactor to physically and intelligently react to the plasma’s state in real time. A ‘GPT-2 Moment’ for a New Era It’s easy to become numb to hype, but what we are witnessing with the TSU feels different. This is what you might call a “GPT-2 moment.” For those who were there, GPT-2 was the first generative AI model that wasn’t just a toy; it was the first time you could play with it at home and see the spark of true generative intelligence. It was the precursor that pointed directly to the GPT-3 and ChatGPT revolution that has since changed the world. This TSU has that same feel. It’s the “SDK” for a new computing paradigm. This technology is as different from classical computing as quantum computing is, but with a critical difference: a team of 15 built this in two years, and it runs at room temperature on your desk. Quantum computing has seen decades of work and billions in funding, and it still hasn’t produced a commercially viable, scalable machine. The TSU is here now. Based on a two-decade-long career at the cutting edge of technology—from seeing the obvious future of virtualization in 2007 to an early conviction in deep learning and GPT—this has all the same hallmarks of a fundamental, world-changing shift. We are not just building faster calculators; we are learning to compute with the universe itself. Pay close attention to this. This is the next big thing.
18:01

Thermodynamic computing is here There is a new computing paradigm emerging from the noise, and its arrival may be as significant as the dawn of deep learning or the advent of cloud virtualization. A new company, Extropic, has just launched its first thermodynamic computer, a device they call a TSU, or Thermal Sampling Unit. While the web is already filling with deep technical dives, what’s more important for most of us is building a clear intuition for what this technology is, how it’s fundamentally different from anything that’s come before, and why it’s generating so much excitement. This isn’t just another chip; it’s a new way to think about computation itself. Seeing is Believing: Solving Puzzles in One Shot To understand what a TSU does, let’s look at two classic, notoriously difficult computer science problems: Sudoku and the Eight Queens problem. When you or I solve a Sudoku, we use a process of sequential logic, guess-and-check, and backtracking. We make an assumption, follow its logical conclusion, and if we hit a dead end, we erase and try again. A classical computer does the same, just much faster. A TSU, however, approaches this in a completely different way. Using a TSU simulator, one can “program” the problem by first clamping the known values—the clues already on the board. Then, you program in the constraints: no duplicate numbers in any row, column, or 3x3 square. With the problem thus defined, the TSU doesn’t “search” for a solution; it anneals one. In a single computational step, the solution simply emerges, backfilling all the empty squares correctly. The same principle applies to the Eight Queens problem, a challenge to place eight queens on a chessboard so that none can attack any other. This is a complex combinatorial problem with 92 distinct solutions. A classical computer would have to iteratively search for these. A TSU, by contrast, can be programmed with the constraints (the “anti-affinity” between queens on the same row, column, or diagonal) and then set to sample the “solution space.” In this context, a valid solution is one with a “problem energy” of zero. The TSU’s physical nature allows it to naturally find these zero-energy states. A simulation of this process shows the TSU discovering all 92 unique solutions, demonstrating its ability to not just find an answer, but to explore the entire landscape of all correct answers. This is a fundamentally new approach, one that bypasses the brute-force, iterative methods we’ve relied on for decades. The Physics of Computation: Using Noise, Not Fighting It This new power comes from a radical design philosophy. For the last 70 years, computing has been about one thing: order. We build chips that are deterministic, logical, and precise. The great enemy has always been noise, heat, and randomness. We spend billions on cooling and error correction to eliminate these very things. Quantum computing, in many ways, is the ultimate expression of this, requiring temperatures near absolute zero to eliminate all thermal noise and achieve quantum coherence. Thermodynamic computing is the polar opposite. It doesn’t fight the noise; it uses it. The TSU is built on the understanding that the natural, stochastic noise from “leaky” transistors—the very randomness we’ve tried to engineer out of existence—is itself a powerful computational resource. Think of it this way: a GPU, which is central to today’s AI, has to simulate noise. When a generative AI model creates a new image or sentence, it’s using complex algorithms to fake randomness. The TSU doesn’t need to fake it; it harnesses the actual physical randomness of thermodynamics. It is a piece of hardware that directly computes with probability. This makes it a hybrid, sitting somewhere between a purely analog computer (which might use light or sound waves to compute) and a digital GPU. It’s a physical device that leverages the laws of physics itself to find solutions, rather than just using logic gates to simulate them. From a Lost Hiker to a Million Bouncy Balls Perhaps the best way to build intuition is with a metaphor. Imagine that solving a complex optimization problem is like trying to find the lowest point of altitude in a 100-square-mile mountainous landscape. Classical computing, using an algorithm like gradient descent, is like being a single hiker dropped into this landscape at night. You have no map or satellite view. All you have is an altimeter and the sensation of the slope under your feet. You can only take one step at a time, always walking downhill, hoping you don’t get stuck in a small local valley when the true, lowest canyon is miles away. Thermodynamic computing is a completely different approach. It’s like having a million bouncy balls and a helicopter. You drop all million balls simultaneously across the entire 100-square-mile landscape. Then, you “turn on an earthquake,” shaking the entire system. The balls bounce and jostle, but as the shaking (the “annealing”) subsides, where do they all end up? They naturally settle into the lowest points. The balls that collect in the deepest valley represent the optimal solution. The TSU is, in essence, a physical device for dropping those million balls at once and letting the laws of thermodynamics find the lowest “energy” state for you, all at the same time. Beyond Puzzles: The Real-World Impact This is far more than just a clever way to solve brain teasers. This ability to instantly find the lowest energy state for a complex, constrained system has staggering real-world applications. One of the most immediate is protein folding. Companies like Google’s DeepMind have made incredible progress with AI like AlphaFold, which predicts protein structures. But this is still a predictive model trained on existing data. A TSU could potentially solve the folding problem directly, treating the protein as a system of atomic affinities and repulsions and finding its most stable, lowest-energy configuration almost instantaneously. This could revolutionize drug discovery and materials science. An even more profound possibility lies in nuclear fusion. One of the greatest engineering challenges in history is controlling the superheated plasma within a tokamak reactor. This requires shaping unimaginably complex magnetic containment fields in real-time to prevent the plasma from touching the reactor walls. This is a real-time optimization problem so complex it’s currently beyond our capabilities. A TSU, however, could be fast enough. Its ability to compute with electricity itself, rather than abstracting the problem through layers of software, might allow it to update the magnetic fields fast enough to stabilize the fusion reaction. One could even imagine a future where thermodynamic computing elements are built directly into the tokamak’s walls, allowing the reactor to physically and intelligently react to the plasma’s state in real time. A ‘GPT-2 Moment’ for a New Era It’s easy to become numb to hype, but what we are witnessing with the TSU feels different. This is what you might call a “GPT-2 moment.” For those who were there, GPT-2 was the first generative AI model that wasn’t just a toy; it was the first time you could play with it at home and see the spark of true generative intelligence. It was the precursor that pointed directly to the GPT-3 and ChatGPT revolution that has since changed the world. This TSU has that same feel. It’s the “SDK” for a new computing paradigm. This technology is as different from classical computing as quantum computing is, but with a critical difference: a team of 15 built this in two years, and it runs at room temperature on your desk. Quantum computing has seen decades of work and billions in funding, and it still hasn’t produced a commercially viable, scalable machine. The TSU is here now. Based on a two-decade-long career at the cutting edge of technology—from seeing the obvious future of virtualization in 2007 to an early conviction in deep learning and GPT—this has all the same hallmarks of a fundamental, world-changing shift. We are not just building faster calculators; we are learning to compute with the universe itself. Pay close attention to this. This is the next big thing.

David Shapiro (L/0)

83,649 Aufrufe • vor 8 Monaten

People are undoubtedly a little alarmed at having unwittingly helped build a 3D map of the world for Niantic by contributing 30 billion crowdsourced images. I interviewed Niantic's CTO Brian McClendon about exactly this in a TED interview last year -- he's also the guy who co-created Google Earth. But let's put it in perspective. Pokestop data isn't what you think it is. It's not a surveillance panopticon of your neighborhood. These are static captures of parks, statues, murals, landmarks -- the places people congregate. Brian described it as "building the map from the bottom up, from the locations where people spend time." Think of these 20 million waypoints as basically the inverse of what Google mapped with Street View. Google mapped the drivable streets. Niantic mapped where people actually hang out. Cool data, genuinely useful for visual positioning -- but very different from what the headlines imply. And lest we forget that Niantic is just one of many companies quietly building their own map of the world right now -- and they're all capturing different facets of reality: >🚶 person-level: Axon body cams on hundreds of thousands of officers. Meta Ray-Ban glasses capturing first-person POV at scale -- overseas operators reviewing images every time someone says "Hey Meta." > 🚗 vehicle-level: Tesla dashcams on every car in the fleet, massive onboard compute extracting and distilling data to the cloud. Waymo with cm-accurate 3D maps of every city they operate in. Fleet telematics cameras on delivery vehicles globally. > 🏠 street & home-level: Flock Safety deploying CCTV across neighborhoods and cities. Amazon with Ring cameras on every doorstep and mailroom (recently got dragged over that Super Bowl commercial about fusing all these cams together to find your dog) plus dashcams on every Prime delivery van. Roomba mapping your floor plan every time it vacuums -- Amazon wanted that data badly enough to try acquiring iRobot for $1.7B before regulators shut it down. > 🥽 headset-level: Apple Vision Pro and Meta Quest build a 3D model of whatever room you're in every time you put them on. Between Ring, Roomba, and your headset, your entire home is being spatially understood by at least three different companies. >📍platform-level: Google with Street View cars, aerial planes, satellite imagery, and live location from every Android phone in your pocket. Apple doing the same with mapping cars AND every LiDAR iPhone is quietly a 3D scanner. And yeah, despite the "Apple is too privacy-conscious" narrative, they're collecting location data too. >🏃 trajectory-level: Strava mapped every running and cycling trail on Earth -- and accidentally exposed secret military bases in Afghanistan and Syria because soldiers logged their jogs. When you aggregate enough individual trajectories, patterns emerge that were never supposed to be visible. > 🛰️ space-level: Planet Labs imaging the entire Earth's landmass every single day from orbit. Vantor capturing it in higher detail. Iceye doing it in 3D using SAR. If something changes anywhere on the planet -- a building goes up, a forest burns down, a military convoy moves -- before-and-after imagery within 24 hours. Fused together -- we have everything from body cam to dashcam to doorbell to phone to satellite -- every layer of physical reality is being mapped by somebody right now. Different sensors, different angles, different purposes. Same pattern. The interesting part is how they incentivize it. Google spends billions. Mapillary tried altruism. Hivemapper grinds with crypto. Pokémon GO cracked something none of them could: a game mechanic that subsidizes the scanning behavior. You're not building a map. You're catching pokemon. The map is just a side effect. 3D scanning is still a niche hobby for reality capture nerds like me. The moment somebody gamifies dense 3D capture at scale -- not posed photos but actual geometry -- that's when this blows wide open. Niantic sold the games for $3.5B but kept the spatial platform, with a data-sharing agreement in place. One team makes the game great, the other builds the spatial infrastructure underneath. Incentives finally aligned. Gaming is becoming a way for humans to contribute real-world trajectories that help physical AI learn about the real world. Google does it with live traffic. Tesla does it with autopilot. The mechanic is different but the pattern is identical -- and most people are already part of at least one -- if not a majority -- of these datasets whether they realize it or not.
0:37

People are undoubtedly a little alarmed at having unwittingly helped build a 3D map of the world for Niantic by contributing 30 billion crowdsourced images. I interviewed Niantic's CTO Brian McClendon about exactly this in a TED interview last year -- he's also the guy who co-created Google Earth. But let's put it in perspective. Pokestop data isn't what you think it is. It's not a surveillance panopticon of your neighborhood. These are static captures of parks, statues, murals, landmarks -- the places people congregate. Brian described it as "building the map from the bottom up, from the locations where people spend time." Think of these 20 million waypoints as basically the inverse of what Google mapped with Street View. Google mapped the drivable streets. Niantic mapped where people actually hang out. Cool data, genuinely useful for visual positioning -- but very different from what the headlines imply. And lest we forget that Niantic is just one of many companies quietly building their own map of the world right now -- and they're all capturing different facets of reality: >🚶 person-level: Axon body cams on hundreds of thousands of officers. Meta Ray-Ban glasses capturing first-person POV at scale -- overseas operators reviewing images every time someone says "Hey Meta." > 🚗 vehicle-level: Tesla dashcams on every car in the fleet, massive onboard compute extracting and distilling data to the cloud. Waymo with cm-accurate 3D maps of every city they operate in. Fleet telematics cameras on delivery vehicles globally. > 🏠 street & home-level: Flock Safety deploying CCTV across neighborhoods and cities. Amazon with Ring cameras on every doorstep and mailroom (recently got dragged over that Super Bowl commercial about fusing all these cams together to find your dog) plus dashcams on every Prime delivery van. Roomba mapping your floor plan every time it vacuums -- Amazon wanted that data badly enough to try acquiring iRobot for $1.7B before regulators shut it down. > 🥽 headset-level: Apple Vision Pro and Meta Quest build a 3D model of whatever room you're in every time you put them on. Between Ring, Roomba, and your headset, your entire home is being spatially understood by at least three different companies. >📍platform-level: Google with Street View cars, aerial planes, satellite imagery, and live location from every Android phone in your pocket. Apple doing the same with mapping cars AND every LiDAR iPhone is quietly a 3D scanner. And yeah, despite the "Apple is too privacy-conscious" narrative, they're collecting location data too. >🏃 trajectory-level: Strava mapped every running and cycling trail on Earth -- and accidentally exposed secret military bases in Afghanistan and Syria because soldiers logged their jogs. When you aggregate enough individual trajectories, patterns emerge that were never supposed to be visible. > 🛰️ space-level: Planet Labs imaging the entire Earth's landmass every single day from orbit. Vantor capturing it in higher detail. Iceye doing it in 3D using SAR. If something changes anywhere on the planet -- a building goes up, a forest burns down, a military convoy moves -- before-and-after imagery within 24 hours. Fused together -- we have everything from body cam to dashcam to doorbell to phone to satellite -- every layer of physical reality is being mapped by somebody right now. Different sensors, different angles, different purposes. Same pattern. The interesting part is how they incentivize it. Google spends billions. Mapillary tried altruism. Hivemapper grinds with crypto. Pokémon GO cracked something none of them could: a game mechanic that subsidizes the scanning behavior. You're not building a map. You're catching pokemon. The map is just a side effect. 3D scanning is still a niche hobby for reality capture nerds like me. The moment somebody gamifies dense 3D capture at scale -- not posed photos but actual geometry -- that's when this blows wide open. Niantic sold the games for $3.5B but kept the spatial platform, with a data-sharing agreement in place. One team makes the game great, the other builds the spatial infrastructure underneath. Incentives finally aligned. Gaming is becoming a way for humans to contribute real-world trajectories that help physical AI learn about the real world. Google does it with live traffic. Tesla does it with autopilot. The mechanic is different but the pattern is identical -- and most people are already part of at least one -- if not a majority -- of these datasets whether they realize it or not.

Bilawal Sidhu

203,550 Aufrufe • vor 4 Monaten

✨ I spent the last 48 hours making GPT-4 read the entire Solana validator codebase and write documentation, so doesn't have to. Introducing — an AI-powered chatbot trained on nothing but code that can answer deep technical questions. How it works 👇 But first... A huge shoutout to , Zahid Khawaja, and Sean. Their hard work made prototyping this thing a breeze. Without further ado... Devs like to write code, not documentation. Tribal knowledge is lost when devs move on to other projects, leaving future devs to sort through mountains of code and figure out not just how it works, but why it works that way. This is all about to change. GPT-4's ability to write code is stunning. It seems to understand something fundamental about writing software that previous models just didn't. This comprehension of the principles that drive the design behind a complex system carries over into its ability to document existing codebases in a truly impressive way. With the enlarged context window(s), it's now feasible to feed GPT-4 entire files of code and ask it to write documentation about how the code works. Taking this as a starting point, the process looks something like this: 1. Download repo. 2. Depth-first traversal of repo contents, ignoring things like package-lock and binary files. 3. For each file, feed to GPT-4 and ask it to write documentation in markdown. 4. Save the output in a separate location as [outputRoot]/[inputFilepath][inputFilename].md 5. For each folder, we ask GPT-4 to write a summary of the folder, taking the newly generated documentation for all files in the folder and the summaries from each of its subfolders as context. Write this to the filesystem as markdown. Now we have a filesystem that matches the structure of the input repo, but all files in the tree are markdown documentation of the corresponding code file. From here, we: 1. Load markdown documents into LangChain. 2. Embed all documents via OpenAI embeddings. 3. Store embeddings in Pinecone. When a user sends a query: 1. Embed query. 2. Find k-nearest markdown files. 3. Feed to GPT-4 with a prompt asking to answer the query based on k-nearest markdown documents provided. The craziest part of all this? GPT-4 actually wrote ~30% of the code. The results are pretty good for 2 days of work. There is certainly room for improvement. Some items that are top of mind: 1. TolyGPT will occasionally hallucinate answers. It is especially bad with links to external sources, like GitHub. The base model seems to know a bit about Solana already, and sometimes this creeps in. Fine-tuning the prompt can solve some of this. 2. Context selection is difficult in a codebase this large. For example, sometimes it will pull in details about the Solana SDK when asked about transaction processing. The SDK files can seem relevant depending on the phrasing of the question. It may be worth breaking the documentation into subsystems to limit this. 3. Not all files fit into the 32k token window. As of now, there are 23 (out of ~1,100) files that cannot be documented in their entirety. Some of these files are very important to how Solana works. Final thoughts: 1. GPT-4 is super powerful, and we're going to see a ton of tools that supercharge the entire software development lifecycle. This is not 12 months away. For the people that can afford it, these tools are here now. And they're only getting better. Act accordingly. 2. The price of inference has to come down for this to go mainstream. I spent about $300 prototyping this project, and the final crawl cost about the same. The high cost of GPT-4 will push developers to other, cheaper alternatives with similar performance. This is coming very soon. If you have a large software project and you're interested in something like this for your codebase, fill out this form and we'll be in touch this week. Or just DM me :)
0:51

✨ I spent the last 48 hours making GPT-4 read the entire Solana validator codebase and write documentation, so doesn't have to. Introducing — an AI-powered chatbot trained on nothing but code that can answer deep technical questions. How it works 👇 But first... A huge shoutout to , Zahid Khawaja, and Sean. Their hard work made prototyping this thing a breeze. Without further ado... Devs like to write code, not documentation. Tribal knowledge is lost when devs move on to other projects, leaving future devs to sort through mountains of code and figure out not just how it works, but why it works that way. This is all about to change. GPT-4's ability to write code is stunning. It seems to understand something fundamental about writing software that previous models just didn't. This comprehension of the principles that drive the design behind a complex system carries over into its ability to document existing codebases in a truly impressive way. With the enlarged context window(s), it's now feasible to feed GPT-4 entire files of code and ask it to write documentation about how the code works. Taking this as a starting point, the process looks something like this: 1. Download repo. 2. Depth-first traversal of repo contents, ignoring things like package-lock and binary files. 3. For each file, feed to GPT-4 and ask it to write documentation in markdown. 4. Save the output in a separate location as [outputRoot]/[inputFilepath][inputFilename].md 5. For each folder, we ask GPT-4 to write a summary of the folder, taking the newly generated documentation for all files in the folder and the summaries from each of its subfolders as context. Write this to the filesystem as markdown. Now we have a filesystem that matches the structure of the input repo, but all files in the tree are markdown documentation of the corresponding code file. From here, we: 1. Load markdown documents into LangChain. 2. Embed all documents via OpenAI embeddings. 3. Store embeddings in Pinecone. When a user sends a query: 1. Embed query. 2. Find k-nearest markdown files. 3. Feed to GPT-4 with a prompt asking to answer the query based on k-nearest markdown documents provided. The craziest part of all this? GPT-4 actually wrote ~30% of the code. The results are pretty good for 2 days of work. There is certainly room for improvement. Some items that are top of mind: 1. TolyGPT will occasionally hallucinate answers. It is especially bad with links to external sources, like GitHub. The base model seems to know a bit about Solana already, and sometimes this creeps in. Fine-tuning the prompt can solve some of this. 2. Context selection is difficult in a codebase this large. For example, sometimes it will pull in details about the Solana SDK when asked about transaction processing. The SDK files can seem relevant depending on the phrasing of the question. It may be worth breaking the documentation into subsystems to limit this. 3. Not all files fit into the 32k token window. As of now, there are 23 (out of ~1,100) files that cannot be documented in their entirety. Some of these files are very important to how Solana works. Final thoughts: 1. GPT-4 is super powerful, and we're going to see a ton of tools that supercharge the entire software development lifecycle. This is not 12 months away. For the people that can afford it, these tools are here now. And they're only getting better. Act accordingly. 2. The price of inference has to come down for this to go mainstream. I spent about $300 prototyping this project, and the final crawl cost about the same. The high cost of GPT-4 will push developers to other, cheaper alternatives with similar performance. This is coming very soon. If you have a large software project and you're interested in something like this for your codebase, fill out this form and we'll be in touch this week. Or just DM me :)

Sam Hogan 🇺🇸

374,577 Aufrufe • vor 3 Jahren

The 118,000% Alpha: Building a High-Frequency AI Trading Floor with Claude Code if you think claude code is just for writing simple scripts then you are already losing to the bots that are hunting your liquidity right now. most traders are still clicking buttons while i have an ai employee running backtests on twenty eight different data sources simultaneously. i am going to show you how a strategy that returned over four hundred thousand percent was built in minutes using a secret sub agent workflow most people treat ai like a chatbot but i treat it like a quant architect that builds systems better than the devs i used to pay hundreds of thousands of dollars. there is one specific indicator combo that actually survived a stress test across tesla and bitcoin at the same time and i will reveal that logic further down. we have to talk about why your current backtests are probably lying to you before we get into the code my name is moon dev and i truly believe that code is the great equalizer in this world. for years i was the guy getting liquidated and overtrading because i was letting my emotions drive the wheel. i spent an insane amount of money hiring developers to build apps for me because i thought i was not smart enough to code myself. through that pain i realized that if i wanted to win i had to automate everything and learn to do it live on youtube for the world to see the secret to trading with claude code is not asking it for a strategy but using it to build a backtest architect. this sub agent acts as a consistent employee that understands how to test against massive datasets without getting tired. it allows me to iterate through hundreds of ideas in the time it used to take me to write one single line of python. this is how i found the strategy that hit a one hundred and eighteen thousand percent return on a single run there is a massive trap that almost every beginner falls into when they start using ai for trading. they find a strategy that looks amazing on one chart and they think they found the holy grail of wealth. that is usually just a lucky fluke or a curve fit mess that will blow up your account next week. the real secret to staying alive is the multi data testing system that claude built for me today we test every single idea against bitcoin and ethereum and solana but we also throw in apple and tesla and nvidia. if a strategy only works on crypto it is probably just riding a trend that is already over. i want to find the logic that is robust enough to handle the volatility of a meme coin and the steady grind of a blue chip stock. this is the only way to prove that the code actually has an edge in the market before we dive into the kalman filter logic i have to tell you about the dca bot i have running on solana right now. it is called housecoin and the thesis behind it is either going to make me a genius or leave me with nothing. it is buying every time we are under the five minute sma and i have been checking the transactions live. i will explain the risk management behind this "all or nothing" play shortly but first we need to look at the winners the winner of today was the acceleration bands combined with a kalman filter. the kalman filter is incredible because it helps remove the noise and lag that you get with standard moving averages. most indicators repaint which means they change their past values to look better after the price has already moved. the way i have implemented this filter prevents that trap so the results you see in the backtest are actually tradable when we ran the acceleration bands across the hourly nvidia chart it returned over two hundred percent while the underlying asset was down forty percent. that is a massive alpha gap that most people will never see because they are stuck using standard rsi settings. i have found that adding a volatility breakout with atr to this setup helps catch the moves that the banks are trying to hide. the math behind the atr breakout is what kept me from getting chopped up in the sideway ranges you might be wondering why i am giving all this code away for free on github instead of keeping it in a vault. it is because i remember what it felt like to be on the other side of the trade losing money every single day. i want to build a community of quads that are all researching and backtesting together. the goal is to chase the legacy of jim simons who proved that math and code are the only things that matter in the long run the rbi system is the framework that i follow every single day without exception. it stands for research and backtest and implement. most traders skip the middle step because they are too impatient to see the results. they hear a rumor on twitter and they buy the top only to get liquidated when the whales decide to take profits. if you do not backtest your ideas then you are just gambling with your life savings i am spending around forty to one hundred dollars a day on claude opus tokens because it is a drop in the bucket compared to what a developer would charge. this ai does not need a lunch break and it does not get bored when i ask it to create sixty different variations of a strategy. we just created five different parabolic sar versions today and found that the long only setup was the only one worth keeping. it returned sixteen thousand percent on the soul data set because it stayed out of the short side traps shorting crypto is extremely dangerous and usually not worth the stress for most people. i have found that focusing on long only strategies with a tight trail stop is the most consistent way to grow an account. the sub agent architect allowed me to verify this across twenty five data sources in less than ten minutes. this speed of iteration is the only way to stay ahead of the curve in an industry that changes every few seconds the dca bot i mentioned earlier is still grinding away and buying the dips as we speak. i have built it to be a long term play where i am slowly accumulating a position in housecoin based on smas. if the price stays under the moving average the bot keeps buying and if it goes above then it sits on its hands. it is a simple logic but it removes the human desire to "buy the moon" when the price is already overextended i found that the camarilla pivot indicator was mostly trash today when we ran the numbers. even though it looks fancy on a chart the backtest showed negative expectancy across almost every asset we tried. this is why backtesting is so important because it kills the "indicator porn" that influencers use to sell you courses. i would much rather know that a strategy is a loser now than find out after i put real money on the line the true secret to using claude code is to treat it like a partner and not just a tool. i ask it to find anomalies and then i ask it to prove me wrong by testing it against the worst market conditions in history. if a strategy can survive the 2022 crypto crash and the 2020 stock market dip then i might consider it for a live run. we are stepping on the gas every single day because there are always new anomalies popping up if you are fast enough to find them i have uploaded over twenty five new backtests to the github today for everyone to use. code is the equalizer because it does not care about your background or how much money you started with. if you can write the logic and prove the edge then the market has to pay you. i am going to keep building in public and showing the wins and the losses because that is the only way to stay real in this space the final piece of the puzzle is the mindset of iteration over perfection. i would rather run a hundred messy backtests today than spend a month trying to write one perfect script. the ai allows me to fail fast so that i can find the winners that actually move the needle. my housecoin dca bot is a testament to that philosophy of just building and letting the systems do the heavy lifting for me if you are still trading by hand you are playing a game that is rigged against you by the biggest firms in the world. they have the best servers and the best data and the best phds but they do not have your specific creativity. when you combine your ideas with the power of claude code you are creating a custom weapon that they have never seen before. i will see you in the code and we will keep chasing the goat until we find that ultimate edge
1:55:47

The 118,000% Alpha: Building a High-Frequency AI Trading Floor with Claude Code if you think claude code is just for writing simple scripts then you are already losing to the bots that are hunting your liquidity right now. most traders are still clicking buttons while i have an ai employee running backtests on twenty eight different data sources simultaneously. i am going to show you how a strategy that returned over four hundred thousand percent was built in minutes using a secret sub agent workflow most people treat ai like a chatbot but i treat it like a quant architect that builds systems better than the devs i used to pay hundreds of thousands of dollars. there is one specific indicator combo that actually survived a stress test across tesla and bitcoin at the same time and i will reveal that logic further down. we have to talk about why your current backtests are probably lying to you before we get into the code my name is moon dev and i truly believe that code is the great equalizer in this world. for years i was the guy getting liquidated and overtrading because i was letting my emotions drive the wheel. i spent an insane amount of money hiring developers to build apps for me because i thought i was not smart enough to code myself. through that pain i realized that if i wanted to win i had to automate everything and learn to do it live on youtube for the world to see the secret to trading with claude code is not asking it for a strategy but using it to build a backtest architect. this sub agent acts as a consistent employee that understands how to test against massive datasets without getting tired. it allows me to iterate through hundreds of ideas in the time it used to take me to write one single line of python. this is how i found the strategy that hit a one hundred and eighteen thousand percent return on a single run there is a massive trap that almost every beginner falls into when they start using ai for trading. they find a strategy that looks amazing on one chart and they think they found the holy grail of wealth. that is usually just a lucky fluke or a curve fit mess that will blow up your account next week. the real secret to staying alive is the multi data testing system that claude built for me today we test every single idea against bitcoin and ethereum and solana but we also throw in apple and tesla and nvidia. if a strategy only works on crypto it is probably just riding a trend that is already over. i want to find the logic that is robust enough to handle the volatility of a meme coin and the steady grind of a blue chip stock. this is the only way to prove that the code actually has an edge in the market before we dive into the kalman filter logic i have to tell you about the dca bot i have running on solana right now. it is called housecoin and the thesis behind it is either going to make me a genius or leave me with nothing. it is buying every time we are under the five minute sma and i have been checking the transactions live. i will explain the risk management behind this "all or nothing" play shortly but first we need to look at the winners the winner of today was the acceleration bands combined with a kalman filter. the kalman filter is incredible because it helps remove the noise and lag that you get with standard moving averages. most indicators repaint which means they change their past values to look better after the price has already moved. the way i have implemented this filter prevents that trap so the results you see in the backtest are actually tradable when we ran the acceleration bands across the hourly nvidia chart it returned over two hundred percent while the underlying asset was down forty percent. that is a massive alpha gap that most people will never see because they are stuck using standard rsi settings. i have found that adding a volatility breakout with atr to this setup helps catch the moves that the banks are trying to hide. the math behind the atr breakout is what kept me from getting chopped up in the sideway ranges you might be wondering why i am giving all this code away for free on github instead of keeping it in a vault. it is because i remember what it felt like to be on the other side of the trade losing money every single day. i want to build a community of quads that are all researching and backtesting together. the goal is to chase the legacy of jim simons who proved that math and code are the only things that matter in the long run the rbi system is the framework that i follow every single day without exception. it stands for research and backtest and implement. most traders skip the middle step because they are too impatient to see the results. they hear a rumor on twitter and they buy the top only to get liquidated when the whales decide to take profits. if you do not backtest your ideas then you are just gambling with your life savings i am spending around forty to one hundred dollars a day on claude opus tokens because it is a drop in the bucket compared to what a developer would charge. this ai does not need a lunch break and it does not get bored when i ask it to create sixty different variations of a strategy. we just created five different parabolic sar versions today and found that the long only setup was the only one worth keeping. it returned sixteen thousand percent on the soul data set because it stayed out of the short side traps shorting crypto is extremely dangerous and usually not worth the stress for most people. i have found that focusing on long only strategies with a tight trail stop is the most consistent way to grow an account. the sub agent architect allowed me to verify this across twenty five data sources in less than ten minutes. this speed of iteration is the only way to stay ahead of the curve in an industry that changes every few seconds the dca bot i mentioned earlier is still grinding away and buying the dips as we speak. i have built it to be a long term play where i am slowly accumulating a position in housecoin based on smas. if the price stays under the moving average the bot keeps buying and if it goes above then it sits on its hands. it is a simple logic but it removes the human desire to "buy the moon" when the price is already overextended i found that the camarilla pivot indicator was mostly trash today when we ran the numbers. even though it looks fancy on a chart the backtest showed negative expectancy across almost every asset we tried. this is why backtesting is so important because it kills the "indicator porn" that influencers use to sell you courses. i would much rather know that a strategy is a loser now than find out after i put real money on the line the true secret to using claude code is to treat it like a partner and not just a tool. i ask it to find anomalies and then i ask it to prove me wrong by testing it against the worst market conditions in history. if a strategy can survive the 2022 crypto crash and the 2020 stock market dip then i might consider it for a live run. we are stepping on the gas every single day because there are always new anomalies popping up if you are fast enough to find them i have uploaded over twenty five new backtests to the github today for everyone to use. code is the equalizer because it does not care about your background or how much money you started with. if you can write the logic and prove the edge then the market has to pay you. i am going to keep building in public and showing the wins and the losses because that is the only way to stay real in this space the final piece of the puzzle is the mindset of iteration over perfection. i would rather run a hundred messy backtests today than spend a month trying to write one perfect script. the ai allows me to fail fast so that i can find the winners that actually move the needle. my housecoin dca bot is a testament to that philosophy of just building and letting the systems do the heavy lifting for me if you are still trading by hand you are playing a game that is rigged against you by the biggest firms in the world. they have the best servers and the best data and the best phds but they do not have your specific creativity. when you combine your ideas with the power of claude code you are creating a custom weapon that they have never seen before. i will see you in the code and we will keep chasing the goat until we find that ultimate edge

Moon Dev

18,390 Aufrufe • vor 5 Monaten

War Diary Day 1,391 Blaise Metreweli, the Chief of Britain's Secret Intelligence Service, sticks it to the Killer in The Kremlin. And all his creepy helpers. I agree with every fucking word. VPDFO! (Transcript of the speech, exactly as it was delivered) 📷 Welcome inside MI6. This iconic building, familiar to movie fans everywhere, is the home of Britain’s foreign intelligence agency. But whilst hundreds of my team pass through the entry pods each day, the truth is that most of our work happens many miles away from this place - out of sight, hidden from the world, undercover, recruiting and running agents who choose to place their trust in us, sharing secrets to make the UK and the world safer. You might pass one of our officers on the street or sit next to them on a plane when you’re about to set off on an adventure of your own, or in a foreign city taking selfies by the sights. Whether it’s in seemingly everyday places, or on the front line embedded with our military, MI6 is there. In my first few weeks, I’ve heard repeatedly that MI6 is trusted and respected globally, two things that we never take for granted. We are seen as a source of hard power, soft influence and rapid innovation. I’ve also heard that people want to believe in MI6. It’s my job to make sure they can. Today, I want to talk about human agency. We all have choices to make about how we deal with the undercurrents shaping our world. About how, in our new, faster, more dangerous and technology-mediated world, it will be our rediscovery of our shared humanity, our ability to listen, and our courage that will determine how our future unfolds. Conflict is not inevitable. Understanding human nature is in my bones. From a family shaped by devastating conflict, I grew up with a deep sense of gratitude for the UK’s precious democracy and freedom. I spent much of my childhood overseas, which is where my passion for travel and adventure began. I studied anthropology, and later psychology and AI, exploring how we make sense of the world and each other. It’s why I was drawn to MI6: it offers strong purpose, a chance to serve and a belief in the positive power of human connection. Like the Service, I’m operational to my very core. Over nearly three decades, my career has involved recruiting and running agents in hostile territory; and leading operations in warzones to defuse threats and support peace. Always in teams, always learning from others. Over the years, I’ve worked with hundreds of brilliant partners – and indeed occasionally those we’d label as adversaries – across dozens of countries, tackling weapons proliferation and terrorism. During my time at MI5, I saw close up what it takes to defend Britain from being targeted by hostile states. You’ll find many like me in my organisation: powerfully motivated to protect our precious country; curious about how our world is changing, joining dots and taking action, across domains. But it was in my last role as ‘Q’, where it was my job to turn emerging technologies from threats to opportunities that I could most see the world changing. As I dug deep into data and extraordinary innovation, I could see how technology was rapidly reshaping not just our capabilities but also conflict and trust, truth and global power. Let me lay out how I see the global issues MI6 must tackle. Because the greatest danger we face is to misunderstand the nature of the problem. Let’s be in no doubt. Our world is more dangerous and contested now than it has been for decades. Conflict is evolving and trust eroding, just as new technologies spur both competition and dependence. We are being contested from sea to space, from the battlefield to the boardroom. And even our brains, as disinformation manipulates our understanding of each other and ourselves. Across the globe, we are now confronting not one single danger, but an interlocking web of security challenges – military, technological, social, ethical even – each shaping the other in complex ways. We are now operating in a space between peace and war. This is not a temporary state or a gradual, inevitable evolution. Our world is being actively remade, with profound implications for national and international security. Institutions which were designed in the ashes of the Second World War are being challenged. New blocs and identities forming and alliances reshaping. Multipolar competition in tension with multilateral cooperation. But there’s something distinctive that will make this change unlike any other: the impact of advanced technologies, which will accelerate the pace and scale of every threat and opportunity, and increasingly, individualise them too. Advances in artificial intelligence, biotechnology, and quantum computing are not only revolutionising economies but rewriting the reality of conflict, as they ‘converge’ to create science-fiction-like tools. There’s incredible promise in all this for all of us, from green technologies to hyper-personalised medicine. But also peril. AI-powered robots and drones are brilliant for scaled manufacturing but devastating on the battlefield. Discoveries that cure disease can also create new weapons. And as states race for tech supremacy, or as some algorithms become as powerful as states, those hyper-personalised tools could become a new vector for conflict and control. Power itself is becoming more diffuse, more unpredictable as control over these technologies is shifting from states to corporations, and sometimes to individuals. And at the same time, the foundations of trust in our societies are eroding. Information, once a unifying force, is increasingly weaponised. Falsehood spreads faster than fact, dividing communities and distorting reality. We live in an age of hyper-connection yet profound isolation. The algorithms flatter our biases and fracture our public squares. And as trust collapses, so does our shared sense of truth – one of the greatest losses a society can suffer. The defining challenge of the twenty-first century is not simply who wields the most powerful technologies, but who guides them with the greatest wisdom. Our security, our prosperity, and our humanity depend on it. Our world is being remade. And for the first time, we are all at the heart of it. My Service must now operate in this new context too: not just expert on hostile states, terrorism, proliferation and more, but also fluent in technology, able to anticipate the second and third order effects of advances that reshape the world in minutes not months. And as China will be a central part of the global transformation taking place this century, it is essential that we, as MI6, continue to inform the government’s understanding of China’s rise and the implications for UK national security. I’m going to break with tradition and won’t give you a global threat tour, but will focus here on Putin’s Russia. We all continue to face the menace of an aggressive, expansionist and revisionist Russia, seeking to subjugate Ukraine and harass NATO. I find it harrowing that hundreds of thousands have died, with the toll mounting every day, because of Putin’s historical distortions and his compromised desire for respect. He is dragging out negotiations and shifting the cost of war onto his own population. But Putin should be in no doubt, our support is enduring. The pressure we apply on Ukraine’s behalf will be sustained. Because it is fundamental not just to European sovereignty and security but to global stability. Alongside the grinding war, Russia is testing us in the grey zone with tactics that are just below the threshold of war. It’s important to understand their attempts to bully, fearmonger and manipulate, because it affects us all. I am talking about: Cyberattacks on critical infrastructure. Drones buzzing airports and bases. Aggressive activity in our seas, above and below the waves. State-sponsored arson and sabotage. Propaganda and influence operations that crack open and exploit fractures within societies. Countering this activity is the work of intelligence and security services across Europe and the globe. And as the Foreign Secretary made clear in a speech last week, the UK is defending itself against this Russian information warfare – sanctioning Russian media outlets pushing Kremlin narratives. The export of chaos is a feature not a bug in this Russian approach to international engagement; and we should be ready for this to continue until Putin is forced to change his calculus. So, how should we respond? It’s not enough now just to understand the world. We must shape it too. MI6 is well-positioned to respond to these threats and wider global instability. And we will continue to evolve, just as we have throughout our long history. The UK government has invested in our intelligence agencies and we are all using our unique powers to keep the British people safe. Our ‘open and connected’ partnerships across the UK Intelligence Community, with HMGCC, NSSIF and the wider tech ecosystem in the UK will become even more important – because in the digital battleground, no single organisation can prevail alone. As a global agency, MI6’s inbuilt strength is our partners and our people. The risks I have set out require us to work ever more closely with our colleagues in MI5, GCHQ and in defence and diplomacy. But also with our Five Eyes partners, with the E3, the EU, NATO, those across the Middle East, the Indo-Pacific and beyond. And with many valued partners whose identity needs to remain secret. Together, we integrate our diverse talent, data and tools to meet the threat. AI is a domain in which we will excel, using the technology to augment, not replace, our human skills. Every digital trace, every byte of data, every algorithmic decision has implications for the safety of the lives of the courageous people who work with us as officers and agents, and for the UK’s strategic advantage. Mastery of technology will infuse everything we do. Not just in our labs, but in the field, in our tradecraft, and even more importantly, in the mindset of every officer. We will become as comfortable with lines of code as we are with human sources, as fluent in Python as we are in multiple other languages. Under my leadership, MI6 will continue to attract Britain’s best and most creative minds: linguists and data scientists, case officers and engineers, behavioural experts and technologists. We need people who walk in the shoes and get in the heads of our adversaries. We need people who think differently, challenge assumptions, and act decisively. All can thrive and make a difference at MI6. At an operational level, we will sharpen our edge and impact with audacity, tapping into – if you like – our historical SOE instincts. We’re at our best when we’re hustling to make things happen, because our intelligence is most valuable when it changes reality on the ground. We will take calculated risks, where the prize is significant and the national interest clear. We will never stoop to the tactics of our opponents. But we must seek to outplay them. In every domain. In every way. So intelligence must drive action. Action must deliver advantage. And advantage must serve Britain’s security and prosperity. But at the core, our deeper contribution is also our simplest – how we unlock human agency. Our fast-paced, tech and threat-infused world now generates more heat than light. As nations retrench and rearm, we are losing opportunities to listen to what’s really going on. I’ve seen time and again throughout my career, that this is where MI6 matters most: we listen and we hear. We understand, because we take time to learn languages and cultures, complex technical and historical detail, immerse ourselves in what’s really driving the situation. Across the globe, right now, our officers are finding people with the courage to step forward, and they are taking time to sit and listen to break these tightening cycles of violence. They listen for nuance, for connection, for opportunity. Over the years, I’ve listened to terrorists who have told us how to defuse the bomb because they know that more violence won’t help. To proliferators and smugglers who’ve told us where to find the dangerous material, motivated to protect their children’s future. To people trapped in authoritarian regimes who know, deep down, that their humanity is being chipped away – and that telling us what’s really going on is an important release, allowing us all to find better ways to navigate our changing world. So, we will work with our agents. And we will continue to engage directly, and with respect, with states and organisation currently working against us. Away from the glare of the media, we will use MI6’s convening power wherever we can to make a material difference, bringing parties together to defuse tensions. But the response to the increasing risks we face won’t be delivered by the UK intelligence community alone. Wider society has a role to play too. That includes work taking place in schools across the country so our children don’t get duped by information manipulation. Let’s all check sources, consider evidence, and be alive to those algorithms that trigger intense reactions, like fear. It also means everyone in society really understanding the world we are in – a world where terrorists plot against us, where our enemies fearmonger, bully and manipulate, and the front line is everywhere. Online, on our streets, in our supply chains, in the minds and on the screens of our citizens. We must all stand together against this. As we do today with our friends in Australia after the shocking antisemitic terrorist attack this weekend. My thoughts -and those of my whole organisation – are with the family, friends and loved ones of the victims. Light will always win over darkness. In rising to meet these challenges we, in MI6, will remain anchored to our values: courage, creativity, respect and integrity. And to our principles: accountability and trust are not constraints on our work; they are the foundations of our legitimacy with the British public. Recently, I had the privilege of meeting and thanking a foreign agent who has worked with us for decades, taking extraordinary risks to help keep the UK safe. I asked why. They said simply, ‘Your values. Your integrity and respect. None of us have a future without them’. This moment reinforced to me that we must remain a very human agency. And so, to sustain that trust, MI6 will continue to be more open. Not for the sake of visibility, but because it matters – and as my MI5 counterpart Sir Ken McCallum said recently - because it is a strength. We will continue the practice of speaking publicly, broaden our channels of engagement, and sustain our focus on attracting the most diverse talent to join our Service. Transparency does not mean revealing what must remain secret. It means showing the British people who we are, what we stand for, and why our work matters. We need your trust and support for the difficult and often dangerous work our agents pursue, every day of the year. In an age of uncertainty, one constant remains: the choices made by human beings still determine the shape of the world. Yes, technology can illuminate possibilities: but information requires judgement; complexity demands clarity; and only people can decide which path to follow. The United Kingdom’s global voice has never rested solely on strength – it has rested on trust, principle, and the ability to understand others as well as ourselves. That is also the essence of intelligence: not simply knowing the world, but interpreting it through a uniquely human lens. Ours is the quiet service, the hidden service. It is one rooted in a profound belief that when human beings act with purpose and integrity, they can steady a faltering world. When the Berlin Wall fell, it was our shared belief in freedom that carried Europe forward. When acts of terror targeted open societies, it was intelligence, cooperation and resolve that preserved them. And when adversaries blur fact and falsehood, our task is to defend the space where truth can still stand. As we step into the future, the tools at our disposal will evolve. But what will always matter most is the human element – the person who stands in the shadows and says: this is right, and that is wrong. That choice – the exercise of human agency – has shaped our world before, and it will shape it again. Because in the end, it is not what we can do that defines us, but what we choose to do. Thank you. Published 15 December 2025
0:59

War Diary Day 1,391 Blaise Metreweli, the Chief of Britain's Secret Intelligence Service, sticks it to the Killer in The Kremlin. And all his creepy helpers. I agree with every fucking word. VPDFO! (Transcript of the speech, exactly as it was delivered) 📷 Welcome inside MI6. This iconic building, familiar to movie fans everywhere, is the home of Britain’s foreign intelligence agency. But whilst hundreds of my team pass through the entry pods each day, the truth is that most of our work happens many miles away from this place - out of sight, hidden from the world, undercover, recruiting and running agents who choose to place their trust in us, sharing secrets to make the UK and the world safer. You might pass one of our officers on the street or sit next to them on a plane when you’re about to set off on an adventure of your own, or in a foreign city taking selfies by the sights. Whether it’s in seemingly everyday places, or on the front line embedded with our military, MI6 is there. In my first few weeks, I’ve heard repeatedly that MI6 is trusted and respected globally, two things that we never take for granted. We are seen as a source of hard power, soft influence and rapid innovation. I’ve also heard that people want to believe in MI6. It’s my job to make sure they can. Today, I want to talk about human agency. We all have choices to make about how we deal with the undercurrents shaping our world. About how, in our new, faster, more dangerous and technology-mediated world, it will be our rediscovery of our shared humanity, our ability to listen, and our courage that will determine how our future unfolds. Conflict is not inevitable. Understanding human nature is in my bones. From a family shaped by devastating conflict, I grew up with a deep sense of gratitude for the UK’s precious democracy and freedom. I spent much of my childhood overseas, which is where my passion for travel and adventure began. I studied anthropology, and later psychology and AI, exploring how we make sense of the world and each other. It’s why I was drawn to MI6: it offers strong purpose, a chance to serve and a belief in the positive power of human connection. Like the Service, I’m operational to my very core. Over nearly three decades, my career has involved recruiting and running agents in hostile territory; and leading operations in warzones to defuse threats and support peace. Always in teams, always learning from others. Over the years, I’ve worked with hundreds of brilliant partners – and indeed occasionally those we’d label as adversaries – across dozens of countries, tackling weapons proliferation and terrorism. During my time at MI5, I saw close up what it takes to defend Britain from being targeted by hostile states. You’ll find many like me in my organisation: powerfully motivated to protect our precious country; curious about how our world is changing, joining dots and taking action, across domains. But it was in my last role as ‘Q’, where it was my job to turn emerging technologies from threats to opportunities that I could most see the world changing. As I dug deep into data and extraordinary innovation, I could see how technology was rapidly reshaping not just our capabilities but also conflict and trust, truth and global power. Let me lay out how I see the global issues MI6 must tackle. Because the greatest danger we face is to misunderstand the nature of the problem. Let’s be in no doubt. Our world is more dangerous and contested now than it has been for decades. Conflict is evolving and trust eroding, just as new technologies spur both competition and dependence. We are being contested from sea to space, from the battlefield to the boardroom. And even our brains, as disinformation manipulates our understanding of each other and ourselves. Across the globe, we are now confronting not one single danger, but an interlocking web of security challenges – military, technological, social, ethical even – each shaping the other in complex ways. We are now operating in a space between peace and war. This is not a temporary state or a gradual, inevitable evolution. Our world is being actively remade, with profound implications for national and international security. Institutions which were designed in the ashes of the Second World War are being challenged. New blocs and identities forming and alliances reshaping. Multipolar competition in tension with multilateral cooperation. But there’s something distinctive that will make this change unlike any other: the impact of advanced technologies, which will accelerate the pace and scale of every threat and opportunity, and increasingly, individualise them too. Advances in artificial intelligence, biotechnology, and quantum computing are not only revolutionising economies but rewriting the reality of conflict, as they ‘converge’ to create science-fiction-like tools. There’s incredible promise in all this for all of us, from green technologies to hyper-personalised medicine. But also peril. AI-powered robots and drones are brilliant for scaled manufacturing but devastating on the battlefield. Discoveries that cure disease can also create new weapons. And as states race for tech supremacy, or as some algorithms become as powerful as states, those hyper-personalised tools could become a new vector for conflict and control. Power itself is becoming more diffuse, more unpredictable as control over these technologies is shifting from states to corporations, and sometimes to individuals. And at the same time, the foundations of trust in our societies are eroding. Information, once a unifying force, is increasingly weaponised. Falsehood spreads faster than fact, dividing communities and distorting reality. We live in an age of hyper-connection yet profound isolation. The algorithms flatter our biases and fracture our public squares. And as trust collapses, so does our shared sense of truth – one of the greatest losses a society can suffer. The defining challenge of the twenty-first century is not simply who wields the most powerful technologies, but who guides them with the greatest wisdom. Our security, our prosperity, and our humanity depend on it. Our world is being remade. And for the first time, we are all at the heart of it. My Service must now operate in this new context too: not just expert on hostile states, terrorism, proliferation and more, but also fluent in technology, able to anticipate the second and third order effects of advances that reshape the world in minutes not months. And as China will be a central part of the global transformation taking place this century, it is essential that we, as MI6, continue to inform the government’s understanding of China’s rise and the implications for UK national security. I’m going to break with tradition and won’t give you a global threat tour, but will focus here on Putin’s Russia. We all continue to face the menace of an aggressive, expansionist and revisionist Russia, seeking to subjugate Ukraine and harass NATO. I find it harrowing that hundreds of thousands have died, with the toll mounting every day, because of Putin’s historical distortions and his compromised desire for respect. He is dragging out negotiations and shifting the cost of war onto his own population. But Putin should be in no doubt, our support is enduring. The pressure we apply on Ukraine’s behalf will be sustained. Because it is fundamental not just to European sovereignty and security but to global stability. Alongside the grinding war, Russia is testing us in the grey zone with tactics that are just below the threshold of war. It’s important to understand their attempts to bully, fearmonger and manipulate, because it affects us all. I am talking about: Cyberattacks on critical infrastructure. Drones buzzing airports and bases. Aggressive activity in our seas, above and below the waves. State-sponsored arson and sabotage. Propaganda and influence operations that crack open and exploit fractures within societies. Countering this activity is the work of intelligence and security services across Europe and the globe. And as the Foreign Secretary made clear in a speech last week, the UK is defending itself against this Russian information warfare – sanctioning Russian media outlets pushing Kremlin narratives. The export of chaos is a feature not a bug in this Russian approach to international engagement; and we should be ready for this to continue until Putin is forced to change his calculus. So, how should we respond? It’s not enough now just to understand the world. We must shape it too. MI6 is well-positioned to respond to these threats and wider global instability. And we will continue to evolve, just as we have throughout our long history. The UK government has invested in our intelligence agencies and we are all using our unique powers to keep the British people safe. Our ‘open and connected’ partnerships across the UK Intelligence Community, with HMGCC, NSSIF and the wider tech ecosystem in the UK will become even more important – because in the digital battleground, no single organisation can prevail alone. As a global agency, MI6’s inbuilt strength is our partners and our people. The risks I have set out require us to work ever more closely with our colleagues in MI5, GCHQ and in defence and diplomacy. But also with our Five Eyes partners, with the E3, the EU, NATO, those across the Middle East, the Indo-Pacific and beyond. And with many valued partners whose identity needs to remain secret. Together, we integrate our diverse talent, data and tools to meet the threat. AI is a domain in which we will excel, using the technology to augment, not replace, our human skills. Every digital trace, every byte of data, every algorithmic decision has implications for the safety of the lives of the courageous people who work with us as officers and agents, and for the UK’s strategic advantage. Mastery of technology will infuse everything we do. Not just in our labs, but in the field, in our tradecraft, and even more importantly, in the mindset of every officer. We will become as comfortable with lines of code as we are with human sources, as fluent in Python as we are in multiple other languages. Under my leadership, MI6 will continue to attract Britain’s best and most creative minds: linguists and data scientists, case officers and engineers, behavioural experts and technologists. We need people who walk in the shoes and get in the heads of our adversaries. We need people who think differently, challenge assumptions, and act decisively. All can thrive and make a difference at MI6. At an operational level, we will sharpen our edge and impact with audacity, tapping into – if you like – our historical SOE instincts. We’re at our best when we’re hustling to make things happen, because our intelligence is most valuable when it changes reality on the ground. We will take calculated risks, where the prize is significant and the national interest clear. We will never stoop to the tactics of our opponents. But we must seek to outplay them. In every domain. In every way. So intelligence must drive action. Action must deliver advantage. And advantage must serve Britain’s security and prosperity. But at the core, our deeper contribution is also our simplest – how we unlock human agency. Our fast-paced, tech and threat-infused world now generates more heat than light. As nations retrench and rearm, we are losing opportunities to listen to what’s really going on. I’ve seen time and again throughout my career, that this is where MI6 matters most: we listen and we hear. We understand, because we take time to learn languages and cultures, complex technical and historical detail, immerse ourselves in what’s really driving the situation. Across the globe, right now, our officers are finding people with the courage to step forward, and they are taking time to sit and listen to break these tightening cycles of violence. They listen for nuance, for connection, for opportunity. Over the years, I’ve listened to terrorists who have told us how to defuse the bomb because they know that more violence won’t help. To proliferators and smugglers who’ve told us where to find the dangerous material, motivated to protect their children’s future. To people trapped in authoritarian regimes who know, deep down, that their humanity is being chipped away – and that telling us what’s really going on is an important release, allowing us all to find better ways to navigate our changing world. So, we will work with our agents. And we will continue to engage directly, and with respect, with states and organisation currently working against us. Away from the glare of the media, we will use MI6’s convening power wherever we can to make a material difference, bringing parties together to defuse tensions. But the response to the increasing risks we face won’t be delivered by the UK intelligence community alone. Wider society has a role to play too. That includes work taking place in schools across the country so our children don’t get duped by information manipulation. Let’s all check sources, consider evidence, and be alive to those algorithms that trigger intense reactions, like fear. It also means everyone in society really understanding the world we are in – a world where terrorists plot against us, where our enemies fearmonger, bully and manipulate, and the front line is everywhere. Online, on our streets, in our supply chains, in the minds and on the screens of our citizens. We must all stand together against this. As we do today with our friends in Australia after the shocking antisemitic terrorist attack this weekend. My thoughts -and those of my whole organisation – are with the family, friends and loved ones of the victims. Light will always win over darkness. In rising to meet these challenges we, in MI6, will remain anchored to our values: courage, creativity, respect and integrity. And to our principles: accountability and trust are not constraints on our work; they are the foundations of our legitimacy with the British public. Recently, I had the privilege of meeting and thanking a foreign agent who has worked with us for decades, taking extraordinary risks to help keep the UK safe. I asked why. They said simply, ‘Your values. Your integrity and respect. None of us have a future without them’. This moment reinforced to me that we must remain a very human agency. And so, to sustain that trust, MI6 will continue to be more open. Not for the sake of visibility, but because it matters – and as my MI5 counterpart Sir Ken McCallum said recently - because it is a strength. We will continue the practice of speaking publicly, broaden our channels of engagement, and sustain our focus on attracting the most diverse talent to join our Service. Transparency does not mean revealing what must remain secret. It means showing the British people who we are, what we stand for, and why our work matters. We need your trust and support for the difficult and often dangerous work our agents pursue, every day of the year. In an age of uncertainty, one constant remains: the choices made by human beings still determine the shape of the world. Yes, technology can illuminate possibilities: but information requires judgement; complexity demands clarity; and only people can decide which path to follow. The United Kingdom’s global voice has never rested solely on strength – it has rested on trust, principle, and the ability to understand others as well as ourselves. That is also the essence of intelligence: not simply knowing the world, but interpreting it through a uniquely human lens. Ours is the quiet service, the hidden service. It is one rooted in a profound belief that when human beings act with purpose and integrity, they can steady a faltering world. When the Berlin Wall fell, it was our shared belief in freedom that carried Europe forward. When acts of terror targeted open societies, it was intelligence, cooperation and resolve that preserved them. And when adversaries blur fact and falsehood, our task is to defend the space where truth can still stand. As we step into the future, the tools at our disposal will evolve. But what will always matter most is the human element – the person who stands in the shadows and says: this is right, and that is wrong. That choice – the exercise of human agency – has shaped our world before, and it will shape it again. Because in the end, it is not what we can do that defines us, but what we choose to do. Thank you. Published 15 December 2025

John Sweeney

42,257 Aufrufe • vor 7 Monaten