Everyone should know how WALs work. Not sure where to start? Write-ahead logs 101:

YouTube, if you're into that kinda thing:

From the queryyyyyy to WAL (to the WAL) til the data's pulled off the log, all your clients can call.

Have you considered a career in entertainment Will?

Not quick witted enough! This was my one for the quarter. It took all my juice.

After going through Ben’s explanation, if you want to dive deeper into how WAL is actually used for database recovery, I wrote a blog covering the recovery process and ARIES from first principles:

This is a great article

@Hi_Mrinal If you get a chance, check out the article. Would love to hear your thoughts and get your feedback!

Great explanation. Also a good WAL is uniform, bcache does this quite nicely in its journal (which can be uniform, as the btrees are also uniform). Check out 9.7 and 9.8 in the principles of operation docs You want everything in your system to use the same infra and no special cases

Ask a candidate to explain WAL recovery in an interview and watch the room go quiet.

Excellent interview Q

Such a good refresher, you rock at explaining things

Imagine this on a physical blackboard 😮

Did you hand-draw this on Excalidraw?

True i built simple wal for 2 months ago for document writes using and also watched your video on that

instead of a cyclical buffer, does a using queue work as well?

Every new architecture blog post seems to be "we added a WAL to S3" so this is very timely, thanks!

one time my etcd database got corrupted I was trying to do a backup restore that's when I stumbled on WAL, still a rabbit hole tho

What software was used to make this diagram?

excalidraw

That was really well explained. I'll have to save this for later. it's a helpful recap.

Great vid as always is the case

I’m working on the db branching software . For syncing the main replica with the prod , i had three options : 1) create a pub sub slot and apply changes instantly 2) create a pub slot but apply wal periodically 3) push wal to s3 and apply changes later What should i use ?

Bro, it is just a file where you append data to it

Chỗ ghi log trước khi ghi dữ liệu chính đúng không, nhìn cái sơ đồ này là dễ hiểu hẳn ra đó 😄

How btree gets stored on the disk

data stored in disk in databases is basically just a materialized view of the event log (WAL in this case)

WAL is one of those things that suddenly makes half of database internals click.

Open a school, I will make sure I pay the school fees 😭😭🤣🤣

Karpathy-tier info density and understandability! Great video thanks!

Yeah

Mình mới tìm hiểu về WAL hôm qua, đọc xong thấy dễ hiểu hơn nhiều luôn 👍

no balls?

6:24 b ga.i, lol

Great watch

from an application architecture POV, processing the WAL is also the nicest way to react to state change events in the application.

It’s useful to see it explained with a drawing. Interestingly the WAL is also used for replication and for backups, is quite an important part on many databases.

warum?

They also unlock point-in-time recovery: you replay the log up to a chosen timestamp instead of restoring a whole snapshot.

@readwise save

I have implemented a Toy-WAL in pure Python. You guys can check out. Repo Link: