Ben Dicken's banner

Ben Dicken

@BenjDicken • 45,981 subscribers

databases → @planetscale videos → https://t.co/J5soMyf6XC writing → https://t.co/YpfkF0JaV8

Shorts

Whoah he's right.

Whoah he's right.

2,446,206 просмотров

oops did it again

oops did it again

4,026,107 просмотров

How do you make 768 servers look like 1? The answer is sharding. I wrote about *why* it's needed and *how* it's achieved for Postgres and MySQL. It's a fun one. Link below.

How do you make 768 servers look like 1? The answer is sharding. I wrote about why it's needed and how it's achieved for Postgres and MySQL. It's a fun one. Link below.

262,905 просмотров

You asked for it, so here it is. Visualizing CPU cache speeds relative to RAM. Cache optimization is important too!

You asked for it, so here it is. Visualizing CPU cache speeds relative to RAM. Cache optimization is important too!

1,474,564 просмотров

Fast backups at-scale are made possible by sharding. These require an amazing amount of engineering to nail the infrastructure, orchestration, and reliability. I wrote all about it... with the help of some PlanetScale database geniuses, of course!

Fast backups at-scale are made possible by sharding. These require an amazing amount of engineering to nail the infrastructure, orchestration, and reliability. I wrote all about it... with the help of some PlanetScale database geniuses, of course!

36,771 просмотров

Been working on this for 2 months, and it's finally ready to share! This piece not only *describes* how B-trees and database indexes work, but gives you the opportunity to interact with them, reinforcing the concepts, and providing a bit of fun along the way. Link in thread.

Been working on this for 2 months, and it's finally ready to share! This piece not only describes how B-trees and database indexes work, but gives you the opportunity to interact with them, reinforcing the concepts, and providing a bit of fun along the way. Link in thread.

340,040 просмотров

1 of N reasons why sharding is a great way to scale a database? Backup speed. This is one of the fun little animations I worked on in my recent interactive sharding article. In case you missed it, link below.

1 of N reasons why sharding is a great way to scale a database? Backup speed. This is one of the fun little animations I worked on in my recent interactive sharding article. In case you missed it, link below.

13,264 просмотров

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Breaking: DHH follows Theo - t3.gg. World peace ensues. But seriously Lex Fridman consider this my vote for getting Theo on the show 👀

Breaking: DHH follows Theo - t3.gg. World peace ensues. But seriously Lex Fridman consider this my vote for getting Theo on the show 👀

98,396 просмотров • 21 часов назад

Everyone should know how WALs work. Not sure where to start? Write-ahead logs 101:

Everyone should know how WALs work. Not sure where to start? Write-ahead logs 101:

356,920 просмотров • 2 дней назад

So many S3 performance misunderstandings. Let me clear this up for you all.

So many S3 performance misunderstandings. Let me clear this up for you all.

957,380 просмотров • 6 дней назад

I don't think you understand the magnitude of this situation. Let me explain.

I don't think you understand the magnitude of this situation. Let me explain.

737,139 просмотров • 12 дней назад

This is one of the best engineering blogs you'll ever read. Not *just* because you'll learn a lot, but because Vicent Martí sprinkles in a bit of everything. A great story. History of git infrastructure. Distributed systems. Benchmarks. All written by someone who can speak to every point from deep, personal experience. It's also 5,300 words, so grab a drink and enjoy.

This is one of the best engineering blogs you'll ever read. Not just because you'll learn a lot, but because Vicent Martí sprinkles in a bit of everything. A great story. History of git infrastructure. Distributed systems. Benchmarks. All written by someone who can speak to every point from deep, personal experience. It's also 5,300 words, so grab a drink and enjoy.

229,527 просмотров • 14 дней назад

Daniel clearly hasn't seen the language balls.

Daniel clearly hasn't seen the language balls.

8,639,165 просмотров • 1 год назад

In case you forgot how good we have it with M.2 NVMe SSD.

In case you forgot how good we have it with M.2 NVMe SSD.

1,372,858 просмотров • 4 месяцев назад

FYI physics is still physics. NVMe is fast. Felt like a good day to update you all on this

FYI physics is still physics. NVMe is fast. Felt like a good day to update you all on this

686,841 просмотров • 3 месяцев назад

I am once again begging you to put your database servers and application servers in the same region.

I am once again begging you to put your database servers and application servers in the same region.

1,422,067 просмотров • 1 год назад

There's a million reasons why your app can be slow. The stack of software and hardware beneath are complex. Pinning down performance issues is challenging (but also, fun!) A clip from the chapter 1 stream. More to come this week.

There's a million reasons why your app can be slow. The stack of software and hardware beneath are complex. Pinning down performance issues is challenging (but also, fun!) A clip from the chapter 1 stream. More to come this week.

236,756 просмотров • 2 месяцев назад

Jeff is the reason for my first ever viral tweet. 2 years ago, with probably ~150 followers, I shared about my B-tree visualization tool (eventually turning into the PlanetScale blog). He saw it, liked it, and shared with his audience. Sounds silly, but a little thing like that can change your trajectory. How could one of the most intelligent people in the world think something I built was actually cool? At the time, sounded crazy. Build something useful. Share it. Who knows whose desk it will come across. Oh, and congrats on Discovery Loop!

Jeff is the reason for my first ever viral tweet. 2 years ago, with probably ~150 followers, I shared about my B-tree visualization tool (eventually turning into the PlanetScale blog). He saw it, liked it, and shared with his audience. Sounds silly, but a little thing like that can change your trajectory. How could one of the most intelligent people in the world think something I built was actually cool? At the time, sounded crazy. Build something useful. Share it. Who knows whose desk it will come across. Oh, and congrats on Discovery Loop!

75,261 просмотров • 27 дней назад

tuitter - twitter for your terminal

tuitter - twitter for your terminal

389,579 просмотров • 5 месяцев назад

Ensuring everyone really understands.

Ensuring everyone really understands.

502,700 просмотров • 6 месяцев назад

Glad OpenAI bought the fast Python package manager.

Glad OpenAI bought the fast Python package manager.

395,548 просмотров • 5 месяцев назад

Over the past 8 days, I received over 100 PRs on the languages repo with additions and improvements. - A bunch of languages were added - Some implementations got tweaks to modify performance - The run script now uses hyperfine for timing Thanks to all the contributors.

Over the past 8 days, I received over 100 PRs on the languages repo with additions and improvements. - A bunch of languages were added - Some implementations got tweaks to modify performance - The run script now uses hyperfine for timing Thanks to all the contributors.

1,200,501 просмотров • 1 год назад

1 billion loop iterations. 4 languages. I wrote the same code in js, python, go, and c. Timed executions on a digital ocean dedicated cpu vm, and here are the visualized results. Not all programming languages are created equal!

1 billion loop iterations. 4 languages. I wrote the same code in js, python, go, and c. Timed executions on a digital ocean dedicated cpu vm, and here are the visualized results. Not all programming languages are created equal!

1,202,975 просмотров • 1 год назад

Remember that time Brendan Gregg shouted at hard drives to make them slower? Someone please confirm nvme ssd is immune.

Remember that time Brendan Gregg shouted at hard drives to make them slower? Someone please confirm nvme ssd is immune.

164,327 просмотров • 2 месяцев назад

*Finally* read through Sam Rose's blog on LLM quantization. It's incredible. For many (even in tech) the understanding of how LLMs work stops at the surface level. Sam is helping us all go deeper, digging into the interesting facets of how AI models truly work. Read it!

Finally read through Sam Rose's blog on LLM quantization. It's incredible. For many (even in tech) the understanding of how LLMs work stops at the surface level. Sam is helping us all go deeper, digging into the interesting facets of how AI models truly work. Read it!

273,219 просмотров • 4 месяцев назад

Database table size impacts performance in more ways than one: a) B-tree depth. Using 8k pages and a 16b uuid: 1 level = ~370 rows 2 levels = ~138k rows 3 levels = ~50m rows 4 levels = ~20b rows The lookup cost on a table with 100k rows is not the same as one with 1b rows. This can apply both to the table itself (MySQL cluster index) as well as the indexes. Sometimes a single query requires many of them. b) Small table → fits in RAM → fast reads. The larger the table, the more likely to read from disk plus churn the cache. c) # of indexes. Each adds maintenance overhead for insertions, and for Postgres vacuum overhead as well. Keep an eye on this! It's useful to take regular stock of your tables + indexes. Clean bloat. Remove unused indexes. Partition if needed.

Database table size impacts performance in more ways than one: a) B-tree depth. Using 8k pages and a 16b uuid: 1 level = ~370 rows 2 levels = ~138k rows 3 levels = ~50m rows 4 levels = ~20b rows The lookup cost on a table with 100k rows is not the same as one with 1b rows. This can apply both to the table itself (MySQL cluster index) as well as the indexes. Sometimes a single query requires many of them. b) Small table → fits in RAM → fast reads. The larger the table, the more likely to read from disk plus churn the cache. c) # of indexes. Each adds maintenance overhead for insertions, and for Postgres vacuum overhead as well. Keep an eye on this! It's useful to take regular stock of your tables + indexes. Clean bloat. Remove unused indexes. Partition if needed.

212,291 просмотров • 4 месяцев назад

I implemented an ssd, hdd, and tape device. In javascript. For a blog.

I implemented an ssd, hdd, and tape device. In javascript. For a blog.

705,700 просмотров • 1 год назад