we’ve all got at least one DNS horror story

all “DNS problems” are really just routing/switching/application level problems on the backend & DNS is innocent. change my mind

This is an awesome explanation of one of the most complicated DNS failure for causes I ever heard. What was the troubleshooting and diagnosis for that? I imagine it was tricky to get from the people reporting to you (or someone else) who could figure out the problem and fix it.

Never downed a network but I did connect a 110V surge protector into a 220V socket and tripped the Access Control to the Data Center which also disabled the boom to enter the parking garage and therefore locking myself in on a Saturday , was only my first week . Fun times!

I love these kinda stories so much 🙌

First of all, that’s my favorite shirt lol. My DNS quick story was using DNS trace to show why a school district was having slow response times. Turned out a backup went and restored in a bad/old IP address and every resolution round robin’d to the old IP.

One of my biggest professional whiffs was assuming web browsers would respect SRV records for port 443

ah, the thousand eyes shirt, great shirt, I have that one too, love to wear it to meetings with other teams.

With the amount of DNS problems Comcast has historically had I'm surprised he asked that. They've gotten much better in the last ~5 years, but had a really rough patch before that.

Similar horror story here. Whole router pathed out via ISIS overload bit. But when returning live template fuckup had router go narrow metrics, rest of network on wides. Device pulled traffic from the entire nation to itself with obviously bad consequences.

That time I typed 8.8.8.7! Silly me! Good times.