Most recent diffusion language model research (that I’ve seen)... seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.show more

The Hidden Language of Diffusion Models paper page: tackle... the challenge of understanding concept representations in text-to-image models by decomposing an input text prompt into a small set of interpretable elements. This is achieved by learning a pseudo-token that is a sparse weighted combination of tokens from the model's vocabulary, with the objective of reconstructing the images generated for the given concept. Applied over the state-of-the-art Stable Diffusion model, this decomposition reveals non-trivial and surprising structures in the representations of concepts. For example, we find that some concepts such as "a president" or "a composer" are dominated by specific instances (e.g., "Obama", "Biden") and their interpolations. Other concepts, such as "happiness" combine associated terms that can be concrete ("family", "laughter") or abstract ("friendship", "emotion"). In addition to peering into the inner workings of Stable Diffusion, our method also enables applications such as single-image decomposition to tokens, bias detection and mitigation, and semantic image manipulationshow more

Selected as a best paper finalist at #CVPR2026: PixelDiT... from NVIDIA Research In most image generation models, a pretrained autoencoder compresses the image before any diffusion happens, causing quality loss that accumulates across the entire pipeline. PixelDiT, or Pixel Diffusion Transformers, removes this step entirely. It's a single-stage model that learns the diffusion process directly in pixel space, end-to-end.show more

This weekend I invented something new, a new type... of token using Uniswap v4 hooks that I call a Pair Token. A Pair Token is a new type of ERC-20. It is a fused, auto-compounding asset, a fusion of two tokens. For example, the stock tokens NVDA and SPCX fused into a single tokenized version of the two, with the ticker NVDASPCX, or ETH with NVDA, creating ETHNVDA. Each Pair Token represents a full-range Uniswap v4 liquidity pool. So why mint these tokens? These tokens are self-multiplying in terms of value, as at any moment anyone can call the compound function to compound the LP fees generated by the pair’s trading activity back into the asset. The token represents a new financial primitive, as it represents a proportional value and moves as an LP position NFT would, except that it is auto-compounding and, thanks to its v4 hook, allows for compounding and self-growth. Since it is an ERC-20, this token can be freely traded onchain, LPed, or even paired with a launchpad. A new type of token for a new type of exposure and strategy. Starting tonight, you’ll be able to mint or redeem Pair Tokens for any pair on Robinhood Chain via the StonkBrokers website. Minting and redeeming are free, and the compounding process on the tokens incurs a 10% fee on the LP fees generated, which is used for $STONKBROKER buybacks. I look forward to a new line of products and protocols on Robinhood Chain using this new financial primitive. Only on Robinhood Chain, powered by Uniswap. Hookrshow more

Transformer and Mixture of Experts, explained visually! Mixture of... Experts (MoE) is a popular architecture that uses different experts to improve Transformer models. Transformer and MoE differ in the decoder block: - Transformer uses a feed-forward network. - MoE uses experts, which are feed-forward networks but smaller compared to those Transformer. During inference, a subset of experts are selected. This makes inference faster in MoE. Also, since the network has multiple decoder layers: - The text passes through different experts across layers. - The chosen experts also differ between tokens. But how does the model decide which experts should be ideal? The router does that. It is a multi-class classifier that produces softmax scores over experts to select the top K experts. The router is trained with the network, and it learns to select the best experts. But it isn't straightforward. There are challenges! Challenge 1) Notice this pattern at the start of training: - Say, the model selects "Expert 2" - This expert gets a bit better - It may get selected again since it's the "best" - It learns more - It gets selected again in the next iteration - It learns more, and so on! This means many experts can go under-trained due to the overselection of a few experts! We solve this in two steps: - Add noise to the feed-forward output of the router so that other experts can get higher logits. - Set all but the top K logits to -infinity. After softmax, these scores become zero. This way, other experts also get the opportunity to train. Challenge 2) Some experts may get exposed to more tokens than others, leading to under-trained experts. We prevent this by limiting the number of tokens an expert can process. If an expert reaches the limit, the token is passed to the next best expert. Overall, MoEs have more parameters to load. But a fraction of them are activated during inference. This leads to faster inference. Mixtral 8x7B and Llama 4 are two popular MoE-based LLMs. Have you used MoEs in production yet? To dive deeper into how MoE inference works in production, we wrote a full article covering token dispatch, grouped expert computation, model-weight memory, multi-GPU communication, expert placement, load imbalance, and performance diagnosis. Read it below.show more

Cuckoo Network announced today: In order to accelerate the... listing process of CK tokens, we will increase the output of CK tokens in June this year to provide users with more abundant income opportunities. At the same time, we will continue to optimize the application scenarios of CK tokens and integrate them into more services and platforms to provide users with a more convenient and richer experience. We believe that CK tokens will become an important engine for future cryptocurrency development, creating more value and opportunities for users. Join Cuckoo Network and embrace the future of cryptocurrency together! #Cuckoo_Network #BNBChain #BNB #Binance #crypto #web3 #TheOpenNetwork #cryptocurrency #Blockchain #CryptoNewsshow more

Smart Remesh 2.1.7, on the way.... ** Cloth Remeshing... ** coming soon.... Cloth Remesh · Progress 01: As I continue posting about cloth remeshing, I want to share some of the challenges I faced during development, as well as how I approached and solved them. This is Part 1. Remeshing clothing is more challenging than remeshing most other types of 3D models. Because clothing contains folds, wrinkles, and complex surface forms, it is very important to preserve as much detail as possible throughout the remeshing process. Standard methods that work well for organic models don't always give good results here, because they often require a much higher polygon count to preserve those details properly. To address this, I decided to use the garment's existing UV layout as part of the remeshing process. Each UV island is remeshed independently, and once the process is complete, all of the parts are combined back into a single mesh. This approach helps achieve better detail preservation with a lower polygon count, while also making the remeshing process faster and producing a more acceptable final result. #blender #b3dshow more

Massive update for AI Engineers! Training diffusion models just... got a lot easier. dLLM is an open-source library that does for diffusion models what Hugging Face did for transformers. Here's why this matters: Traditional autoregressive models generate text left-to-right, one token at a time. Diffusion models work differently - they refine the entire sequence iteratively, giving you better control over generation quality and more flexible editing capabilities. The problem? Building and training these models required stitching together scattered tools and reimplementing research papers from scratch. Most of the tooling has been scattered and is hard to reproduce. dLLM changes this: It unifies everything you need to train, evaluate, and deploy diffusion language models: ↳ Scalable training with LoRA, DeepSpeed, and FSDP support ↳ Unified evaluation that abstracts away inference complexity ↳ Ready-to-use recipes for pretraining, finetuning, and evaluation The library includes implementations of models like LLaDA and Dream, plus training algorithms like Edit Flows that enable insertion, deletion, and substitution operations. The team just released ModernBERT-Chat models showing you can turn BERT into lightweight chatbots through masked instruction tuning. This is practical and worth exploring. The setup is straightforward. Run locally with Accelerate or scale to multi-node clusters with Slurm. If you're working with language models and want to explore diffusion-based approaches without rebuilding infrastructure, dLLM gives you a production-ready starting point. Link to the repo in the next tweet.show more

High-resolution image and video generation is hitting a wall... because attention in DiTs scales quadratically with token count. But does every pixel need to be in full resolution? Introducing Foveated Diffusion: a new approach for efficient diffusion-based generation that allocates compute where it matters most. 1/7🧵show more

LongWriter Unleashing 10,000+ Word Generation from Long Context LLMs... discuss: Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.show more

Chop the gradients ✂️! We found that truncating decoder... gradients in latent video diffusion to a fixed window allows us to finetune on videos with pixel-wise perceptual losses without running out of memory. Pixel losses have been essential for image generation and reconstruction, but until now, they haven't scaled to long-duration, high-resolution video diffusion due to recursive activation accumulation in causal decoders, leading to OOM during training 💥📉. Project: Video diffusion models can do a lot more 🚀 when you can backprop the decoder! Post-process neural rendered scenes, super-resolve videos, harmonize lighting in controlled synthetic driving scenes, and inpaint videos — all in a single step ⚡ with a quick finetune from a standard diffusion model.show more

🔥 2 BILLION $ATLA BURNED 🔥 Major milestone for... the ATLA ecosystem. 2 BILLION tokens permanently removed from supply — cutting total allocation by nearly 2/3. 👇 Why it happened As ATLA demand and value grew, the team reviewed the original token allocation and decided to burn a large portion of undistributed tokens to strengthen the long-term token model. ⚠️ Important: This burn only affects undistributed tokens — current holders are not affected. Result: 📉 Lower future supply 📈 Stronger price fundamentals This makes $ATLA a much more limited and stronger asset🚀show more

DimensionX: Create Any 3D and 4D Scenes from a... Single Image with Controllable Video Diffusion TL;DR: Create 3/4DGS from Video Diffusion Note: Some first inference code released (not all yet). Contributions (cited): • We present DimensionX, a novel framework for generating photorealistic 3D and 4D scenes from only a single image using controllable video diffusion. • We propose ST-Director, which decouples the spatial and temporal priors in video diffusion models by learning (spatial and temporal) dimension-aware modules with our curated datasets. We further enhance the hybriddimension control with a training-free composition approach according to the essence of video diffusion denoising process. • To bridge the gap between video diffusion and real-world scenes, we design a trajectory-aware mechanism for 3D generation and an identity-preserving denoising approach for 4D generation, enabling more realistic and controllable scene synthesis. • Extensive experiments manifest that our DimensionX delivers superior performance in video, 3D, and 4D generation compared with baseline methods.show more

World Models are the path for some AI Models... in the future. But how can we efficiently train these models to not only see the world the way humans do but to see the world in a new and unique way. By visualizing, what is normally sequenced audio patterns, we can derive much more insights. Here we see Paganini in a visual form that can than be described and transcribed into a World Model. We can observe connections in a manner that may not have been clear prior to the digitalization of music and sound in this way. The company with the most valuable potential in building a World Model is Tesla. Not that this type of visualization is being used, but that the mechanisms are in place, and the technology is in place for the company to thrive in this new form of AI.show more

🚨Well-Known Crypto Expert Claims ‘#XRP WILL UNAVOIDABLY REPLACE SWIFT’!... TO PUT IT INTO CONTEXT, SWIFT HANDLED A MASSIVE $150 TRILLION IN 2022 ALONE! FOR XRP TO PROCESS THAT VOLUME, EACH TOKEN WOULD NEED TO REACH $2,670.00!! THE DEFI SECTOR ON XRPL IS ABOUT TO EXPLODE!! CTF TOKEN, THE TOP DEFI TOKEN ON XRPL, NEEDS JUST A $20 BILLION MARKET CAP TO SOAR FROM $0.80 TO $748.50!! CTF TOKEN HAS ALREADY SECURED PARTNERSHIPS WITH AMAZON AND WALMART!! A MAJOR REVOLUTION IS UNDERWAY!! TRADE CTF TOKEN HERE: TRADE CTF TOKEN ON MEXC: Official Website:show more

LLaDA (the first Large Language Diffusion Model) is *just*... out 💥 and I've built a demo, try out now 👨‍💻 It's mesmerizing to watch the diffusion process 🌀, and it being a diffusion model gives you superpowers like "the 4th word has to be pineapple" 🦸 Demo and weights 👇show more

Crypto narratives tend to move in cycles. 2020 was... DeFi. 2021 became NFTs. 2023 turned into the AI boom. 2024–2025 were dominated by memecoins and attention tokens. But markets eventually rotate back to something simple: real revenue. That’s why some people are starting to look at iGaming tokens as a potential emerging narrative in 2026. Unlike many hype driven tokens, the iGaming sector already runs large cash flow businesses. Many platforms generate hundreds of millions of dollars in monthly revenue, yet their tokens often trade with far less volume than projects that barely produce revenue at all. In other words, there’s a visible mismatch between actual business activity and token market valuation. One ecosystem that sits right inside this discussion is 1win Token, which already operates as one of the top 10 online casinos globally by scale and user activity. The upcoming $1win Token is designed to connect that existing business with on chain incentives. Its token model includes buybacks and burns funded directly from casino revenue, tying token supply mechanics to real cash flow. There’s also an interesting structural difference compared to previous gaming tokens. For example, $RLB (Rollbit) saw a massive post launch rally, but the product and revenue scale at launch were significantly smaller than what 1win operates today. Another notable point is the launch design: instead of only farming an airdrop, 1win Token plans a public sale model, allowing broader participation from the start. If Web3 narratives are indeed shifting away from pure attention cycles and back toward revenue generating platforms, sectors like iGaming may start attracting more analytical focus and 1win could emerge as the biggest winner.show more

AI has had exactly two scaling axes that worked... so far, and the second one is starting to look finite too the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before you’re just polishing RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and that’s much closer to actual RL and it’s what bought us planning and reasoning (arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing) so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- it’s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet... and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inferenceshow more