What 20 users visiting a site?

For the fifth fucking time, this is not the same thing .

Its not at all the same thing

Or get the preview on the first user seeing the post in that instance.

For sure there is more users than instances. So the amount of requests would be grater but at least they wouldn’t happen at the same time.

But if fediverse ever grow to be as big as other social media, then a single post/link can be viewed by thousands of users at the same time causing the same effect.

Another option could be to have the originating Mastodon server provide the preview image in content-adressable way, and have a standard allowing websites to provide the same: if the hash of the file(s) provided by the first / a Mastodon server to another matches the one from the target website, no need to download from the target website. Only if they differ then you’d actual download the data from the target website.

Each Mastodon server from the federation could quickly decide whether to trust that preview data from the peer it gets the rest of the post from or not.

This is basically taking ideas from IPFS but applying it to just the preview data that ends up being downloaded over and over again. You could also make that part a blob of data stored directly in IPFS. But IPFS was pretty slow to resolve content, when I last tested.

Well you can’t really trust the original instance to be truthful with its preview.

I’m not exactly sure why a web server can’t handle legit web traffic, just because it’s a lot?

Its not because it’s a lot, it’s because it all happens in the same instant, which does not resemble any sort of normal web traffic.

but solving it on the other side is bad architecture

Solving it on the side that’s creating the problem is bad architecture? 🤔

But it would never be truly solved that way.

Several potential solutions have been posited, if you just look through the comments.

Sure would.

i’m feeling less ambivalent about those reddit repost bots for a moment

Similar, except those are caused by users visiting a link, whereas the Fediverse issue is federated servers gathering info before users even see a link.

Or being slashdotted.

Yeah, thats how it should be: one download, everyone else downloads from that server, not the original source.

Seems abusable though

Either publishing server should serve the metadata - although it would be prone to bad actors, so a refetch in the next 0-180 seconds would be appropriate. Or just add a random 0-60 second delay before fetching the data.

To be honest, if a website has no caching of their server-rendered parts, they are also doing something wrong! Very little sites really need to re-render a public page on every view.

that don’t honour robots.txt (or aren’t yet listed).

Throw’em in the tar pit.

that don’t honour robots.txt

That’s essentially all of them in my experience.

Oh wow GNUNet I havent heard of that in a while. How is it nowadays?

Or i2p.

is there something that makes this different to 17 other proposals for a new internet or is it just a case of

Now this is interesting. Thanks for sharing

GNUnet helps building a new Internet

GNUnet is a network protocol stack for building secure, distributed, and privacy-preserving applications.

With strong roots in academic research, our goal is to replace the old insecure Internet protocol stack.

GNUnet provides privacy-by design

It provides [improving addressing, routing, naming and content distribution](https://www.w3.org/2014/strint/papers/65.pdf) in a technically robust manner - as opposed to ad-hoc designs in place today.