"Of course it was DNS!"
But this time, it was my fault. Again.
I upgraded the BSD Cafe instances to Ruby 4.0.10 (installed via rbenv). Everything seemed fine, until a friendly message arrived: "Hey, the media aren't working properly!"
I checked, and @nuintari was right: HTTP 503.
I started investigating, but couldn't really figure out WHY it wasn't working.
And yes, it was DNS.
For media delivery, we have several remote Varnish caches around the world, to improve latency and reduce the load on the origin SeaweedFS server.
On the Mastodon servers, I also have an /etc/hosts entry pointing media.bsd.cafe directly to SeaweedFS, bypassing those caches.
Except... I'd only added the IPv4 address!
Apparently, after the Ruby upgrade, connections started using IPv6 instead, resolving media.bsd.cafe to one of the remote Varnish caches. And that's where things went wrong.
I fixed the mapping, and... we're back!
Sometimes the problem isn't DNS itself. It's the person who configured it.
That would be me. 😉
@stefano vedo che ti diverti a cazziarti 😜
A parte gli scherzi è sempre bello poter fare mea culpa quando si fa una svista o altro.... pazienza nessuno è infallibile 😉 per cui benvenuto nel club 😜