I am the admin of the newly created instance https://indie-ver.se/, and it’s been going great. But as I was looking to migrate all images to a S3 bucket, I noticed, the images take up more space than I had anticipated, a bit over 1 GB. And I know for a fact, that there has not been many images uploaded to this instance directly.
My best guess is, that it’s because lemmy by default creates local thumbnails for external posts, via. the setting “image_mode: StoreLinkPreviews”. Lemmy does allow me to disable this behavior, i.e. it would not cache the thumbnails on my instance, and instead have browsers request the image from the other instances.
The question is, what impact would that have on my users, and what impact would it have on the fedi-verse in general? I believe it would save me a lot on future storage costs, but I don’t know if it’s worth it.
- ssnoer
You must log in or # to comment.
I’m a small instance.
In 3 years my pict-rs S3 bucket has grown to over 1.2TB
Yeah disable it. I’m not sure why I didn’t before.
Turning it off completely means that theoretically I could expose a users IP. Say I DM them a unique photo URL. Zero click and I got that. Lemmy’s default UI blocks that but some apps like Voyager don’t. Or you could reply to an old comment which makes it very likely they’ll be the only ones to see it.
I turned on proxy-ing.
I mean, 1.2TB isn’t that much in terms of price, but it’s still a heck of a lot. I’m gonna look into proxying, and disabling the caching. Thanks!
On S3 storage it’s a monthly cost, so it’ll add up over the years.
1.2 TB is a shitload
That’s about 1/8 my current total storage - in today’s drive costs that’s about $40 for just primary storage, and then 2x - 3x that for backup storage.
I just set up our S3 image storage, which costs us 6.27€ pr. TB. (with Hetzner)
Actually, how do you backup your images? I am bit unsure of what to do here, since the objects aren’t really managed by me.
Related question: I want to start self-hosting. How easy is it to completely disable images?
Well. You probably want some pictures. For a site icon, for your pfp etc. which necessitates setting up the image server. You can quite easily stop your server from caching external images, as I’ve just done, so only your local images are stored. I don’t know about disallowing user uploaded images entirely. Maybe you can set an upload limit of 0 or smth?
I disabled local thumbnail generation almost a year ago, and things mostly work the same.
Instead of a local thumbnail image URL for things like news articles that get posted, it will be the direct URL value from the
og:imagemetadata from the source. Usually those load fine, but sometimes they don’t due to CORS settings on their side. Probably only 1-2% of posts have issues, though.For image posts that come in via federation, (memes, pics, etc), the thumbnail image URL is the same as the post URL. In other words, you’re loading the full res version in the feed. Since I use a web client that has “card view”, this actually works out better, visually. YMMV whether that’s a drawback for you.
The only pitfall is that you will lose thumbnails for image posts if an instance goes offline or shuts down.
I’m sure that does increase load slightly on other instances, but no more than if the remote instance had image proxying turned on. And the full-res version always has to load from the remote instance (even if you have local thumbnail generation enabled). All in all, I’d say the additional load is acceptable given the benefits of disabling local thumbnail generation.
To mitigate that, in my case anyway, I have my own image proxy/cache in place. My default UI is Tesseract and it’s configured with the image proxy/cache on by default… (I think I saw that Photon is also working on something similar). In this configuration, the first person to scroll past a remote image fetches it directly (via the proxy/cache) and it’s now available locally in the cache for everyone else (unless they’re connecting with a different client that doesn’t use Tesseract’s proxy). Granted, I shutdown my instance last year and it’s just now a private testbed for development, but when I did have daily active users (plural), the proxy cache helped.
Now the only images my instance stores are ones that are uploaded locally.
Why did I disable local thumbnails?
- I closed up my instance and didn’t want potentially problematic thumbnails being generated while I wasn’t actively modding it
- Generated thumbnails go in, but they don’t go out. There’s no way to clean them up later other than the ephemerally generated ones (if someone requests a version in a custom size, for example)
- Increasing storage costs. Like, I’d be scrolling the feed and see some of the dumbest shitposts while constantly thinking “Ugh, this is costing me money to store a copy”.
Yup. This is my thought process exactly. One gigabyte, for 20 days of being properly online, is not much. My cost is around 6€ pr. month pr. terabyte (pretty nice), but I have no idea how my usage will grow.
