Know how long every operation takes on a computer. Once you do, identifying good vs bad vs acceptable performance becomes way easier. It all boils down to knowing your napkin math.

And btw @Sirupsen is the expert here. Longer video on youtube:

Ayy I was just thinking about this today! So cool.

/napkin-math when?

Already done: :)

napkin math is underrated. how do you keep those numbers in your head without writing them down

🧠

good stuff! for GPUs the bottleneck is bandwidth because we need to feed all compute units in parallel. per GPU bandwidth: HBM (GPU ↔ GPU memory): TB/s (~5-8 for HBM3) NVLink / scale-up (GPU ↔ GPU): low TB/s InfiniBand / scale-out (rack ↔ rack): 100s GB/s

We need the inference napkin math

if only there was a way to visualize this intuitively with balls

Flame Graphs 🔥

jus saw this yest while havin dinner....trus me i frgt what i had

I like when math and number is used in the right way