I am the admin of the newly created instance https://indie-ver.se/, and it’s been going great. But as I was looking to migrate all images to a S3 bucket, I noticed, the images take up more space than I had anticipated, a bit over 1 GB. And I know for a fact, that there has not been many images uploaded to this instance directly.

My best guess is, that it’s because lemmy by default creates local thumbnails for external posts, via. the setting “image_mode: StoreLinkPreviews”. Lemmy does allow me to disable this behavior, i.e. it would not cache the thumbnails on my instance, and instead have browsers request the image from the other instances.

The question is, what impact would that have on my users, and what impact would it have on the fedi-verse in general? I believe it would save me a lot on future storage costs, but I don’t know if it’s worth it.

- ssnoer

  • Admiral Patrick@dubvee.org
    link
    fedilink
    English
    arrow-up
    8
    ·
    11 hours ago

    I disabled local thumbnail generation almost a year ago, and things mostly work the same.

    Instead of a local thumbnail image URL for things like news articles that get posted, it will be the direct URL value from the og:image metadata from the source. Usually those load fine, but sometimes they don’t due to CORS settings on their side. Probably only 1-2% of posts have issues, though.

    For image posts that come in via federation, (memes, pics, etc), the thumbnail image URL is the same as the post URL. In other words, you’re loading the full res version in the feed. Since I use a web client that has “card view”, this actually works out better, visually. YMMV whether that’s a drawback for you.

    The only pitfall is that you will lose thumbnails for image posts if an instance goes offline or shuts down.

    I’m sure that does increase load slightly on other instances, but no more than if the remote instance had image proxying turned on. And the full-res version always has to load from the remote instance (even if you have local thumbnail generation enabled). All in all, I’d say the additional load is acceptable given the benefits of disabling local thumbnail generation.

    To mitigate that, in my case anyway, I have my own image proxy/cache in place. My default UI is Tesseract and it’s configured with the image proxy/cache on by default… (I think I saw that Photon is also working on something similar). In this configuration, the first person to scroll past a remote image fetches it directly (via the proxy/cache) and it’s now available locally in the cache for everyone else (unless they’re connecting with a different client that doesn’t use Tesseract’s proxy). Granted, I shutdown my instance last year and it’s just now a private testbed for development, but when I did have daily active users (plural), the proxy cache helped.

    Now the only images my instance stores are ones that are uploaded locally.

    Why did I disable local thumbnails?

    • I closed up my instance and didn’t want potentially problematic thumbnails being generated while I wasn’t actively modding it
    • Generated thumbnails go in, but they don’t go out. There’s no way to clean them up later other than the ephemerally generated ones (if someone requests a version in a custom size, for example)
    • Increasing storage costs. Like, I’d be scrolling the feed and see some of the dumbest shitposts while constantly thinking “Ugh, this is costing me money to store a copy”.
    • ssnoer@indie-ver.seOP
      link
      fedilink
      English
      arrow-up
      3
      ·
      11 hours ago

      Yup. This is my thought process exactly. One gigabyte, for 20 days of being properly online, is not much. My cost is around 6€ pr. month pr. terabyte (pretty nice), but I have no idea how my usage will grow.