https://github.com/LemmyNet/lemmy/issues/3245

I posted far more details on the issue then I am putting here-

But, just to bring some math in- with the current full-mesh federation model, assuming 10,000 instances-

That will require nearly 50 million connections.

Each comment. Each vote. Each post, will have to be sent 50 million seperate times.

In the purposed hub-spoke model, We can reduce that by over 99%, so that each post/vote/comment/etc, only has to be sent 10,000 times (plus n*(n-1)/2 times, where n = number of hub servers).

The current full mesh architecture will not scale. I predict, exponential growth will continue to occur.

Let’s work on a solution to this problem together.

  • Hazelnoot [she/her]@beehaw.org
    link
    fedilink
    English
    arrow-up
    8
    ·
    1 year ago

    The same problem can also be solved with signed messages, like the HTTP Signatures used by Mastodon and most of the other microblogging fedi servers. Signatures allow a message to flow peer-to-peer instead of requiring a direct connection. You would only need a connection when actively interacting with a post on another instance, and its very unlikely that all 10K instances would be interacting with each other. Most likely, the network will consist of smallish groups of loosely-related instances plus a few giant servers that can handle the load of being popular.

    • HTTP_404_NotFound@lemmyonline.comOP
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 year ago

      That, honestly, wouldn’t be a bad idea either. That should in theory help break up a lot of the load which is currently overly centralized.

      The implementation should be a lot easier then my purposed idea as well, and it also has side effects of potentially improving security.