Ben Dicken's banner
Ben Dicken's profile picture

Ben Dicken

@BenjDicken • 46,032 subscribers

databases → @planetscale videos → https://t.co/J5soMyf6XC writing → https://t.co/YpfkF0JaV8

Shorts

Whoah he's right.

Whoah he's right.

2,446,206 views

oops did it again

oops did it again

4,026,107 views

How do you make 768 servers look like 1? The answer is sharding. I wrote about *why* it's needed and *how* it's achieved for Postgres and MySQL. It's a fun one. Link below.

How do you make 768 servers look like 1? The answer is sharding. I wrote about *why* it's needed and *how* it's achieved for Postgres and MySQL. It's a fun one. Link below.

262,905 views

You asked for it, so here it is. Visualizing CPU cache speeds relative to RAM. Cache optimization is important too!

You asked for it, so here it is. Visualizing CPU cache speeds relative to RAM. Cache optimization is important too!

1,474,564 views

Fast backups at-scale are made possible by sharding. These require an amazing amount of engineering to nail the infrastructure, orchestration, and reliability. I wrote all about it... with the help of some PlanetScale database geniuses, of course!

Fast backups at-scale are made possible by sharding. These require an amazing amount of engineering to nail the infrastructure, orchestration, and reliability. I wrote all about it... with the help of some PlanetScale database geniuses, of course!

36,771 views

Been working on this for 2 months, and it's finally ready to share! This piece not only *describes* how B-trees and database indexes work, but gives you the opportunity to interact with them, reinforcing the concepts, and providing a bit of fun along the way. Link in thread.

Been working on this for 2 months, and it's finally ready to share! This piece not only *describes* how B-trees and database indexes work, but gives you the opportunity to interact with them, reinforcing the concepts, and providing a bit of fun along the way. Link in thread.

340,040 views

1 of N reasons why sharding is a great way to scale a database? Backup speed. This is one of the fun little animations I worked on in my recent interactive sharding article. In case you missed it, link below.

1 of N reasons why sharding is a great way to scale a database? Backup speed. This is one of the fun little animations I worked on in my recent interactive sharding article. In case you missed it, link below.

13,264 views

Videos

LIVE
1.2k

Anya Rossi

sweetdream.ai

SweetDream.aiSponsored•Livecam
Streaming Now

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

HD live stream
Exclusive private shows
1.2k viewers online

Free preview available • Premium content

Everyone should know how WALs work. Not sure where to start? Write-ahead logs 101:
11:10
BenjDicken's profile picture

Everyone should know how WALs work. Not sure where to start? Write-ahead logs 101:

Ben Dicken

356,920 views • 2 days ago

So many S3 performance misunderstandings. Let me clear this up for you all.
0:49
BenjDicken's profile picture

So many S3 performance misunderstandings. Let me clear this up for you all.

Ben Dicken

957,380 views • 6 days ago

Breaking: DHH follows Theo - t3.gg. World peace ensues. But seriously Lex Fridman consider this my vote for getting Theo on the show 👀
0:43
BenjDicken's profile picture

Breaking: DHH follows Theo - t3.gg. World peace ensues. But seriously Lex Fridman consider this my vote for getting Theo on the show 👀

Ben Dicken

98,396 views • 1 day ago

I don't think you understand the magnitude of this situation. Let me explain.
0:38
BenjDicken's profile picture

I don't think you understand the magnitude of this situation. Let me explain.

Ben Dicken

737,139 views • 12 days ago

Daniel clearly hasn't seen the language balls.
0:51
BenjDicken's profile picture

Daniel clearly hasn't seen the language balls.

Ben Dicken

8,639,165 views • 1 year ago

This is one of the best engineering blogs you'll ever read. Not *just* because you'll learn a lot, but because Vicent Martí sprinkles in a bit of everything. A great story. History of git infrastructure. Distributed systems. Benchmarks. All written by someone who can speak to every point from deep, personal experience. It's also 5,300 words, so grab a drink and enjoy.
0:32
BenjDicken's profile picture

This is one of the best engineering blogs you'll ever read. Not *just* because you'll learn a lot, but because Vicent Martí sprinkles in a bit of everything. A great story. History of git infrastructure. Distributed systems. Benchmarks. All written by someone who can speak to every point from deep, personal experience. It's also 5,300 words, so grab a drink and enjoy.

Ben Dicken

229,527 views • 15 days ago

In case you forgot how good we have it with M.2 NVMe SSD.
0:54
BenjDicken's profile picture

In case you forgot how good we have it with M.2 NVMe SSD.

Ben Dicken

1,372,858 views • 4 months ago

FYI physics is still physics. NVMe is fast. Felt like a good day to update you all on this
0:49
BenjDicken's profile picture

FYI physics is still physics. NVMe is fast. Felt like a good day to update you all on this

Ben Dicken

686,841 views • 3 months ago

I am once again begging you to put your database servers and application servers in the same region.
0:53
BenjDicken's profile picture

I am once again begging you to put your database servers and application servers in the same region.

Ben Dicken

1,422,067 views • 1 year ago

There's a million reasons why your app can be slow. The stack of software and hardware beneath are complex. Pinning down performance issues is challenging (but also, fun!) A clip from the chapter 1 stream. More to come this week.
6:53
BenjDicken's profile picture

There's a million reasons why your app can be slow. The stack of software and hardware beneath are complex. Pinning down performance issues is challenging (but also, fun!) A clip from the chapter 1 stream. More to come this week.

Ben Dicken

236,756 views • 2 months ago

tuitter - twitter for your terminal
0:36
BenjDicken's profile picture

tuitter - twitter for your terminal

Ben Dicken

389,579 views • 5 months ago

Jeff is the reason for my first ever viral tweet. 2 years ago, with probably ~150 followers, I shared about my B-tree visualization tool (eventually turning into the PlanetScale blog). He saw it, liked it, and shared with his audience. Sounds silly, but a little thing like that can change your trajectory. How could one of the most intelligent people in the world think something I built was actually cool? At the time, sounded crazy. Build something useful. Share it. Who knows whose desk it will come across. Oh, and congrats on Discovery Loop!
0:20
BenjDicken's profile picture

Jeff is the reason for my first ever viral tweet. 2 years ago, with probably ~150 followers, I shared about my B-tree visualization tool (eventually turning into the PlanetScale blog). He saw it, liked it, and shared with his audience. Sounds silly, but a little thing like that can change your trajectory. How could one of the most intelligent people in the world think something I built was actually cool? At the time, sounded crazy. Build something useful. Share it. Who knows whose desk it will come across. Oh, and congrats on Discovery Loop!

Ben Dicken

75,261 views • 28 days ago

Ensuring everyone really understands.
0:43
BenjDicken's profile picture

Ensuring everyone really understands.

Ben Dicken

502,700 views • 6 months ago

Glad OpenAI bought the fast Python package manager.
0:28
BenjDicken's profile picture

Glad OpenAI bought the fast Python package manager.

Ben Dicken

395,548 views • 5 months ago

Over the past 8 days, I received over 100 PRs on the languages repo with additions and improvements. - A bunch of languages were added - Some implementations got tweaks to modify performance - The run script now uses hyperfine for timing Thanks to all the contributors.
0:37
BenjDicken's profile picture

Over the past 8 days, I received over 100 PRs on the languages repo with additions and improvements. - A bunch of languages were added - Some implementations got tweaks to modify performance - The run script now uses hyperfine for timing Thanks to all the contributors.

Ben Dicken

1,200,501 views • 1 year ago

1 billion loop iterations. 4 languages. I wrote the same code in js, python, go, and c. Timed executions on a digital ocean dedicated cpu vm, and here are the visualized results. Not all programming languages are created equal!
0:22
BenjDicken's profile picture

1 billion loop iterations. 4 languages. I wrote the same code in js, python, go, and c. Timed executions on a digital ocean dedicated cpu vm, and here are the visualized results. Not all programming languages are created equal!

Ben Dicken

1,202,975 views • 1 year ago

Remember that time Brendan Gregg shouted at hard drives to make them slower? Someone please confirm nvme ssd is immune.
0:48
BenjDicken's profile picture

Remember that time Brendan Gregg shouted at hard drives to make them slower? Someone please confirm nvme ssd is immune.

Ben Dicken

164,327 views • 2 months ago

*Finally* read through Sam Rose's blog on LLM quantization. It's incredible. For many (even in tech) the understanding of how LLMs work stops at the surface level. Sam is helping us all go deeper, digging into the interesting facets of how AI models truly work. Read it!
0:23
BenjDicken's profile picture

*Finally* read through Sam Rose's blog on LLM quantization. It's incredible. For many (even in tech) the understanding of how LLMs work stops at the surface level. Sam is helping us all go deeper, digging into the interesting facets of how AI models truly work. Read it!

Ben Dicken

273,219 views • 4 months ago

Database table size impacts performance in more ways than one: a) B-tree depth. Using 8k pages and a 16b uuid: 1 level = ~370 rows 2 levels = ~138k rows 3 levels = ~50m rows 4 levels = ~20b rows The lookup cost on a table with 100k rows is not the same as one with 1b rows. This can apply both to the table itself (MySQL cluster index) as well as the indexes. Sometimes a single query requires many of them. b) Small table → fits in RAM → fast reads. The larger the table, the more likely to read from disk plus churn the cache. c) # of indexes. Each adds maintenance overhead for insertions, and for Postgres vacuum overhead as well. Keep an eye on this! It's useful to take regular stock of your tables + indexes. Clean bloat. Remove unused indexes. Partition if needed.
0:58
BenjDicken's profile picture

Database table size impacts performance in more ways than one: a) B-tree depth. Using 8k pages and a 16b uuid: 1 level = ~370 rows 2 levels = ~138k rows 3 levels = ~50m rows 4 levels = ~20b rows The lookup cost on a table with 100k rows is not the same as one with 1b rows. This can apply both to the table itself (MySQL cluster index) as well as the indexes. Sometimes a single query requires many of them. b) Small table → fits in RAM → fast reads. The larger the table, the more likely to read from disk plus churn the cache. c) # of indexes. Each adds maintenance overhead for insertions, and for Postgres vacuum overhead as well. Keep an eye on this! It's useful to take regular stock of your tables + indexes. Clean bloat. Remove unused indexes. Partition if needed.

Ben Dicken

212,291 views • 4 months ago

I implemented an ssd, hdd, and tape device. In javascript. For a blog.
0:41
BenjDicken's profile picture

I implemented an ssd, hdd, and tape device. In javascript. For a blog.

Ben Dicken

705,700 views • 1 year ago