Continuous Diffusion Language Models (CDLM's)

Let’s be honest: autoregressive language models have had the mic for way too long. They’re brilliant, sure, but they’re also basically playing an endless game of Telephone with themselves—one token at a time, no take-backs. Sander’s article over at https://sander.ai/2026/08/24/continuous-dlms.html makes the case that continuous diffusion could break that loop. Instead of predicting the next word, you corrupt a sequence, learn to denoise it, and sample in parallel. That’s not just a tweak; that’s a different religion. The big appeal is obvious: diffusion models already crush it in image and audio domains because they can refine a whole sample at once. Language, though, is stubbornly discrete and syntactic. You can’t just “denoise” a sentence the way you’d sharpen a blurry cat photo. Sander’s piece wrestles with exactly that tension—how to represent text continuously without losing the categorical cliff that makes words, well, words. It’s the kind of problem that makes engineers reach for a drink and theorists reach for a whiteboard. What I find genuinely interesting here is the potential to escape the autoregressive bottleneck. Parallel generation could mean faster inference, better long-range coherence, and maybe a way to sidestep the compounding error that plagues long outputs. But the trade-offs are real: discrete text has no natural “noise schedule,” and evaluation gets murky when your model outputs a distribution over continuous embeddings instead of clean tokens. The HN thread (https://news.ycombinator.com/item?id=49502611) is predictably split between “this is the future” and “good luck sampling that.” None of this is settled science. The article is a map of a frontier, not a victory lap. But it’s the kind of thinking that matters—questioning the architectural default rather than just scaling it. If diffusion language models ever go mainstream, this is the kind of post people will point to and say, “Yeah, I was there when the idea started getting legs.” And if it fizzles? At least it’s a smarter failure than another 100-billion-parameter transformer doing the same thing with more compute.