Ben Dicken
@BenjDicken • 35,240 subscribers
databases → @planetscale videos → https://t.co/J5soMyfENa writing → https://t.co/YpfkF0JIKG
Shorts
Videos
FYI physics is still physics. NVMe is fast. Felt like a good day to update you all on this
Ben Dicken685,311 Aufrufe • vor 27 Tagen
I am once again begging you to put your database servers and application servers in the same region.
Ben Dicken1,420,944 Aufrufe • vor 9 Monaten
*Finally* read through Sam Rose's blog on LLM quantization. It's incredible. For many (even in tech) the understanding of how LLMs work stops at the surface level. Sam is helping us all go deeper, digging into the interesting facets of how AI models truly work. Read it!
Ben Dicken271,695 Aufrufe • vor 1 Monat
Important to know that Postgres/MySQL on local SSD is *the* best way to get great performance. The reason it could get *even faster* boils down to abstraction boundaries. SSDs are built to be workload agnostic. They have their own controllers and garbage collectors meant to balance performance of many different I/O patterns. DBMS are written to support arbitrary backing I/O devices: SSD, HDD, or even a remote storage system. It's good that they're written to be broadly performant! The point of the paper was that, if you're willing to build something that specifically targets local SSDs (and even specific models of local SSDs) you can optimize the full DBMS -> OS -> hardware stack for the exact characteristics of a workload and hardware specifications. This is a classic tradeoff. You can usually get better performance as you narrow scope for specific hardware, architectures, etc. Maintainers of OSS must walk a careful balance of optimizing for performance while also maintaining wide compatibility. For the curious-minded, I wrote an interactive article on the history of I/O devices, and how that interplays with database performance:
Ben Dicken79,264 Aufrufe • vor 17 Tagen
Database table size impacts performance in more ways than one: a) B-tree depth. Using 8k pages and a 16b uuid: 1 level = ~370 rows 2 levels = ~138k rows 3 levels = ~50m rows 4 levels = ~20b rows The lookup cost on a table with 100k rows is not the same as one with 1b rows. This can apply both to the table itself (MySQL cluster index) as well as the indexes. Sometimes a single query requires many of them. b) Small table → fits in RAM → fast reads. The larger the table, the more likely to read from disk plus churn the cache. c) # of indexes. Each adds maintenance overhead for insertions, and for Postgres vacuum overhead as well. Keep an eye on this! It's useful to take regular stock of your tables + indexes. Clean bloat. Remove unused indexes. Partition if needed.
Ben Dicken211,360 Aufrufe • vor 1 Monat
Over the past 8 days, I received over 100 PRs on the languages repo with additions and improvements. - A bunch of languages were added - Some implementations got tweaks to modify performance - The run script now uses hyperfine for timing Thanks to all the contributors.
Ben Dicken1,200,016 Aufrufe • vor 1 Jahr
I implemented an ssd, hdd, and tape device. In javascript. For a blog.
Ben Dicken681,069 Aufrufe • vor 1 Jahr
