I’m looking for experiences and opinions on kubernetes storage.

I want to create a highly available homelab that spans 3 locations where the pods have a preferred locations but can move if necessary.

I’ve looked at linstore or seaweedfs/garage with juicefs but I’m not sure how well the performance of those options is across the internet and how well they last in long term operation. Is anyone else hosting k3s across the internet in their homelab?

Edit: fixed wording

  • F04118F@feddit.nlEnglish
    2·
    13 hours ago

    The problem with non-PLP drives is that Rook-Ceph will insist that its writes get done in a way that is safe wrt power loss.

    For regular consumer drives, that means it has to wait for the cache to be flushed, which takes aaaages (milliseconds!!) and that can cause all kinds of issues. PLP drives have a cache that is safe in the event of power loss, and thus Rook-Ceph is happy to write to cache and consider the operation done.

    Again, 1Gb network is not a big deal, not using PLP drives could cause issues.

    If you don’t need volsync and don’t need ReadWriteMany, just use Longhorn with its builtin backup system and call it a day.