1/8 Does text-to-image generation need language pretraining? Today, we... share Halcyon, a family of early-fusion, base models trained from scratch. Halcyon outperforms strong late-fusion baselines with up to 275× less pretraining compute. Models and recipes are open. Halcyon is part of a research program at Radical Numerics on the science of multimodal pretraining and scaling.show more

Switti -- a new scale-wise transformer for text-to-image generation... 🦾 🔥 Improved generation of fine-grained details. Outperforms existing T2I AR models and competes with state-of-the-art T2I diffusion models while being up to 7x faster.show more

STEVE-1: A Generative Model for Text-to-Behavior in Minecraft paper... page: Constructing AI models that respond to text instructions is challenging, especially for sequential decision-making tasks. This work introduces an instruction-tuned Video Pretraining (VPT) model for Minecraft called STEVE-1, demonstrating that the unCLIP approach, utilized in DALL-E 2, is also effective for creating instruction-following sequential decision-making agents. STEVE-1 is trained in two steps: adapting the pretrained VPT model to follow commands in MineCLIP's latent space, then training a prior to predict latent codes from text. This allows us to finetune VPT through self-supervised behavioral cloning and hindsight relabeling, bypassing the need for costly human text annotations. By leveraging pretrained models like VPT and MineCLIP and employing best practices from text-conditioned image generation, STEVE-1 costs just $60 to train and can follow a wide range of short-horizon open-ended text and visual instructions in Minecraft. STEVE-1 sets a new bar for open-ended instruction following in Minecraft with low-level controls (mouse and keyboard) and raw pixel inputs, far outperforming previous baselines. We provide experimental evidence highlighting key factors for downstream performance, including pretraining, classifier-free guidance, and data scaling. All resources, including our model weights, training scripts, and evaluation tools are made available for further research.show more

A major question in multimodal modeling is how to... leverage strong pre-trained models, such as industry-scale LLMs and VLMs, during training to avoid starting from scratch. This is important as the demand for rich and domain-specific multimodal models continues to increase, while training them from scratch is obviously impractical due to limited data and compute. The so-called “any-to-any” multimodal models can model a large diverse dictionary of modalities. That’s good. Their downside is that their architectures are often not decoder-only, which has limited their performance in practice and prevents them from leveraging strong pre-trained decoder-only models as priors. We are releasing MODUS (ICML Conference '26) a decoder-only any-to-any multimodal model to address some of these questions. A single transformer decoder predicts any modality from any others with no modality-specific heads, losses, or task pipelines. We show efficient adaptation of established models, e.g., BAGEL, to rich any-to-any multimodal modeling. We are releasing 14B to 77B parameter models. All materials are open-source. The download links and demos here 🧵show more

Pokémon Mystery Dungeon: Halcyon is available now! Recapturing the... vibes of PMD Explorers of Sky! In Halcyon, you've turned into a Pokémon! A brand new story, meet a fresh new cast of pals and go on amazing adventures! Check out + Palika #EeveeExpo3show more

Exciting news from Jasper Research! 🚀 We built an... internal 4B text-to-image model from scratch that turned out to be one of the fastest models out there! 1k image generation in less than a second! Our technical report shares everything we learned along the way. 👇show more

1/ With Omnii Diagnostics, we asked whether a multimodal... genome language model could help find cancers whose signals in blood are too faint for current classifiers. We adapted Omnii-1 to a biomarker never seen in pretraining. In a test cohort, Omnii's features improved early-stage cancer detectionshow more

📢Pix2NPHM: Learning to Regress NPHM Reconstructions From a Single... Image📢 We directly regress neural parametric head models (NPHMs) from a single image — fast, stable, and significantly more expressive than classical 3DMMs such as FLAME. Face tracking & 3D reconstruction are often limited by the representational capacity of PCA-based face models. By lifting NPHMs to a first-class reconstruction primitive, we enable more accurate geometry, richer expressions, and finer animation control. Pix2NPHM obtains fast and reliable NPHM reconstructions on real-world data. Inference-time optimization against surface normals and canonical point maps can further increase fidelity. Key to successful and generalized training of our ViT-based network are: (1) large-scale registration of existing 3D head datasets, and (2) self-supervised training on vast in-the-wild 2D video datasets using pseudo ground-truth surface normals. Finally, we show that geometry-aware pretraining on pixel-aligned reconstruction tasks significantly outperforms generic visual pretraining (e.g., DINO-style features) in terms of generalization. 🌍 🎥 Great work by Simon Giebenhain, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Zhe Chenshow more

Massive update for AI Engineers! Training diffusion models just... got a lot easier. dLLM is an open-source library that does for diffusion models what Hugging Face did for transformers. Here's why this matters: Traditional autoregressive models generate text left-to-right, one token at a time. Diffusion models work differently - they refine the entire sequence iteratively, giving you better control over generation quality and more flexible editing capabilities. The problem? Building and training these models required stitching together scattered tools and reimplementing research papers from scratch. Most of the tooling has been scattered and is hard to reproduce. dLLM changes this: It unifies everything you need to train, evaluate, and deploy diffusion language models: ↳ Scalable training with LoRA, DeepSpeed, and FSDP support ↳ Unified evaluation that abstracts away inference complexity ↳ Ready-to-use recipes for pretraining, finetuning, and evaluation The library includes implementations of models like LLaDA and Dream, plus training algorithms like Edit Flows that enable insertion, deletion, and substitution operations. The team just released ModernBERT-Chat models showing you can turn BERT into lightweight chatbots through masked instruction tuning. This is practical and worth exploring. The setup is straightforward. Run locally with Accelerate or scale to multi-node clusters with Slurm. If you're working with language models and want to explore diffusion-based approaches without rebuilding infrastructure, dLLM gives you a production-ready starting point. Link to the repo in the next tweet.show more

Introducing Muse Image and Muse Video, the first media... generation models developed by Meta Superintelligence Labs. Muse Image is our most advanced image generation model yet. It follows instructions faithfully, edits with precision, composes from multiple references, and draws on Instagram for social context. It also brings agentic tool use capabilities to image generation and integrates with Muse Spark. You can try Muse Image in the Meta AI app and web, as well as in Instagram Stories and WhatsApp – starting in limited countries with more locations on the way. Today we’re also previewing Muse Video, which is built upon the same pretraining base as Muse Image to deliver exceptional visual fidelity with native audio support. Learn more about both models:show more

Show-o One Single Transformer to Unify Multimodal Understanding and... Generation discuss: We present a unified transformer, i.e., Show-o, that unifies multimodal understanding and generation. Unlike fully autoregressive models, Show-o unifies autoregressive and (discrete) diffusion modeling to adaptively handle inputs and outputs of various and mixed modalities. The unified model flexibly supports a wide range of vision-language tasks including visual question-answering, text-to-image generation, text-guided inpainting/extrapolation, and mixed-modality generation. Across various benchmarks, it demonstrates comparable or superior performance to existing individual models with an equivalent or larger number of parameters tailored for understanding or generation. This significantly highlights its potential as a next-generation foundation model.show more

Training Wan 2.1-1.3B to use Qwen3-VL-2B text encoder. Doing... 33% text only, 33% VL only, and 33% both. Just pretraining the two text input linear layers for now. It is amazing how quickly a model can adapt to a different text encoder. This is 25,750 steps, BS of 10.show more

Introducing ✨Aya Expanse ✨ – an open-weights state-of-art family... of models to help close the language gap with AI. Aya Expanse is both global and local. Driven by a multi-year commitment to multilingual research.show more

Backprop has been the only credit assignment algorithm capable... of training large neural nets. Introducing Dust, the first zeroth-order method to pretrain transformers to approach and sometimes even *exceed* backprop with large amounts of computation (bitter lesson!) - Dust is on the order of 1,000 to 10,000x more compute efficient than EGGROLL, the state-of-the-art ES method, for training transformers. - We chose pretraining transformers because it's arguably the hardest task possible for zeroth-order optimization. - Search in high dimensions is really misunderstood: zeroth-order usually gets better with overparameterization, not worse, i.e. at a fixed population size, larger models (up to 120x) reach a lower loss than smaller ones. - Backprop-like gradients emerge from a large population of activation perturbations, and the alignment holds up at every scale we tested, up to 1B tokens, which is critical for scaling. The core idea is a *virtual population*, which helps scale to large populations in transformers by perturbing activations in parallel. w/ Bishwas, serdarml, Akshayshow more

Purple Halcyon🥵

Introducing benchmark tracking across all of arXiv With thousands... of models and benchmarks out there, it's impossible to track what's actually state-of-the-art We now index results from millions of papers to surface which models are SOTA according to open source research!show more

STRESSED IN THE SALLOWLANDS 😅 💀 There are only... a few hours left until #AgeOfUmbra: Sallowlands returns on YouTube, Twitch - and Zachery Renauldo is feeling the pressure! Uncover the mysteries of the Halcyon Domain's haunted desert at 7PM PT tonight! 🗡️show more

HEAD BACK TO THE HALCYON DOMAIN - TONIGHT! 🏜️... 💀 You heard Game Master Matthew Mercer! Our shaken survivors recover from a riptide and a hang glider ride as we return to the Sallowlands for more #Daggerheart adventure TONIGHT at 7PM PT on YouTube, and Twitch! 🗡️show more

You already know who Halcyon is. So... what are... you waiting for?show more

Purple Halcyon🥵🔥

NEW RELEASE: Today we're releasing CortexMAE: a family of... fMRI foundation models trained on 2.1K hours of open fMRI data. We're also releasing Brainmarks: an open benchmark suite for evaluating fMRI foundation models. Full paper is on arXiv (accepted to ICML 2026) A thread:show more