Google has been quietly working on an skunkworks research project: to create the first ever massively-parallel 2D renderer! Interested? I got an exclusive interview with Raph Levien, a lead engineer on the project, and the world's foremost expert on parallel 2D graphics. 🧵

You heard him mention that he's going to "SIMD accelerate the hell out of that". SIMD (Single Instruction, Multiple Data) refers to a newer class of CPU instructions that work on multiple data points simultaneously. It's the secret sauce behind the new "hybrid" renderer

SIMD is powerful: it allows you to exploit GPU-like parallelism on the CPU. Yet unlike the GPU, you can switch back to regular "scalar" instructions when it's appropriate. This makes it ideal for certain steps of the 2D graphics pipeline.

But why does Google need a new 2D graphics renderer? It's a little-known fact that 2D graphics is traditionally very slow compared to 3D graphics. Most 2D graphics rendering is done serially, which make it too slow to render a page at 120fps. Ever notice how choppy this looks?

Since browsers can't render pages at 120fps, they have to cache different parts of the page to a persistent texture, and then "composite" those textures to render the page. This is why electron-based apps are never capable of "smooth" resizing, and why they use so much RAM.

But if you could make 2D rendering fast like 3D rendering, you wouldn't need that. You could re-render the whole page every frame, and that would eliminate the jitter we tolerate on webpages and electron-apps. Every app on your machine could feel snappier, and SIMD is the key.

The 2D graphics pipeline is inherently CPU-bound, so anything that allows us to make the CPU side as fast as possible makes it faster. But to write SIMD code, you'll have to understand a very important concept: predication.

Predication is what allows selective execution of SIMD instructions based on per-element conditions. It uses a "mask" vector to control which data lanes actually execute the operation and which remain unaffected, allowing conditional SIMD execution.

Predication is important - it's how you implement `if` statements in SIMD! But it's only recently that CPUs have started having good support for it. CPUs have been slow to support new SIMD instructions, in part because they make the instruction set far more complicated:

But on CPUs that support it, you can do amazing things. We've all used `to_upper`, to convert all the lowercase letters in a string to uppercase. But with new SIMD hardware, we can do that IN PARALLEL! First, we need a mask selecting the lowercase characters:

Once we have a predication mask that selects just the lowercase letters in the string, we then use SIMD instructions to subtract 0x20 from the characters in the mask. You want to to make sure you never load any bytes past the end of the string, but predication helps there too

It's not often that `to_lower` is the bottleneck in your program, but it's worth understanding the tech because it's also what enables things like this recent (utterly insane) demo of 21 GB/s CSV parsing

The really cool thing is that optimizing compilers like LLVM know about SIMD instructions, and are capable of using them to optimize your code, even without you explicitly telling them to. This is called "autovectorization"

Fun fact: the main obstacle with autovectorization is that it requires aliasing analysis, which is very difficult in languages like C++. But Rust's memory model makes aliasing analysis trivial, which allows LLVM to perform much more aggressive autovectorization

But there's a major problem looming over all SIMD code. Only brand-new hardware supports these fancy new predicated SIMD instructions. Ship a binary that uses them to someone running on slightly older hardware, and your code will definitely not do what it's supposed to.

So in practice, compilers will only generate code for your compile target's "least common denominator". To fix this, you implement "dispatch" or "multiversioning", where you have multiple versions of the code for different hardware and dispatch the appropriate version at runtime.

This actually gives Apple a bit of an advantage, because the compile target for Apple Silicon assumes support for this whole new era of SIMD instructions. This means that on Apple Sillicon, you don't have to do any dispatch or multiversioning to get all the new SIMD goodies

We talked about so many more topics, including video encoding, GPU-like parallelism using fp16, and Raph's vision for the next generation of SIMD APIs. All of that and more is in the full interview, free on Youtube or any podcast app!

I also write lots of interesting threads, follow me if you'd like to see more like this!