OG post in case my instance dies, which is not unexpected:

The last couple of weeks have been truly eventful for lemmy.fan.

tl;dr (and first paragraph):

My plan was to wholeheartedly go all in on PieFed for lemmy.fan. I planned a cut-over date, started moving some things, and I was on track to get everything up and running solely on PieFed[.lemmy].fan¹. Then lemmy.fan died. It’s back now.

/slash tl;dr

The reasons behind the migration were varied, but essentially boiled down to a lack of development from two full-time, donation-sponsored-but-still-underpaid developers, nutomic and dessalines from lemmy.ml.

The lifecycle of open-source software development is well-established in lore if not in fact: under- or unpaid developers work on a project that started as a labor of love. The love disappears, and the labor quickly turns to animosity and dread, as Git repos devolve into loud, angry people demanding this or that, reporting bugs but not contributing to fixing existing ones, and always the politics, politics, politics.

Then, PostgreSQL made the decision to utterly shit itself. Lemmy.fan suffered a catastrophic database failure; from what, who knows.

Lemmy.fan, as I once knew and loved, now lies in a pile of corrupted Postgres garbage files, gnashed angrily together by some destructive, demonic, database daemon.

I know little to nothing about PostgreSQL, and that is why I absolutely despise it. My life has been spent using, manipulating, troubleshooting, and migrating MariaDB and MySQL, two very sane and easy-to-use database systems that Just WorkTM. I do not want to learn something new. I fear I now have no choice but to learn this garbage database system and adopt the same relationship with it as I have with so many other things in my life: Don’t fuck with me, and I will not fuck with you. Cross me once, though, and you best be prepared for total annihilation.

I think Postgres has that sorted now, as we’re circling the saloon old-west style, revolvers pointed at one another, shaking slightly in unsteady hands, and eyeing one another for the moment one of us so much as blinks.

Also, it’s good to have backups. I did, and still do, but I decided not to restore them, and here’s why:

A long time ago, when lemmy.fan was but a tiny baby Docker container nestled snugly in a NASsinette in a suburban basement, I created the very first federated lemmy.fan using Yunohost to test things out. I was new to ActivityPub and had little idea as to how the federation worked in broad terms, so I set about this and that. Before long, lemmy.fan was puttering along, populated by the lemmycommunity bot that would dutifully scrape and subscribe to popular communities and instances across the fediverse. I had no local communities and was very new to Docker, so lemmy.fan and I expanded our knowledge: I by learning Docker, Portainer, and other tools, and lemmy.fan sucking in content from across the fediverse, growing and becoming better in its own self.

Then, I broke something, or lemmy.fan itself broke, or something happened that resulted in me having to destroy the instance.

I mistakenly then thought that the domain, lemmy.fan, was no longer available to federate with because I had used it already, exchanging messages using the ActivityPub protocol. Now that the domain was established in the greater fediverse, I thought that I had to go to a subdomain.

So I added real.lemmy.fan as the federation source, CNAMED it back to lemmy.fan, and believed that perhaps everything would just work.

And it did, for about a year or so.

As I trudged along keeping lemmy.fan mostly running, I grew and fostered weirdnews, a community that surpassed 1,000 subscribers. A few friends joined, I added a few other communities, and I kept the small instance chugging along splendidly.

Then something changed.

Lemmy.fan became slow, unreliable. Server errors were pretty commonplace and, while restarting the Docker containers fixed the problem, the underlying cause was a mystery to me. Lemmy.fan’s performance began dragging down other containers on the NAS, the PostgreSQL and lemmy frontend containers putting load averages in the 20s. Logs were useless and showed no particular fault.

I decided to migrate the instance from the basement NAS to a VPS, my thought process being that allocating more RAM and throwing a few more processor cores at lemmy.fan would fix things. For us, it was the vacation preceding the divorce; we being the hypothetical couple who tried to save a failing marriage by going to Hawaii and renewing our vows. Instead, lemmy.fan fell asleep on the couch while I gamed and watched reruns of old Star Trek episodes. Year of Hell Parts 1 and 2 back-to-back on Pluto?

Yes, please!

This is a death knell for any relationship, be it human and human or human and silicon/electron.

I’ll back up a moment. I am not naive. I will ever conflate lemmy, or really any open-source software written by a small handful of volunteer or underpaid developers, with stability. And that’s OK. I accepted the fact that I would be in for a few bumps and scrapes here and there: like the time a new lemmy UI version was released that cocked up any form fields, resulting in a shitty UI experience. Or the time that the lemmy backend would just fuck around and die, taking others down with it in a spectacular blaze of error messages, all cryptic to me. Or the time when never-ending scrolling was dismissed because one person who happens to be the main developer just does not want it.

Concurrently, as lemmy.fan slowly grew and went through its adolescent phase, development on lemmy became less predictable and eventually stalled to the point where significant bugs and other issues were, and still are, being neglected as lemy version 1 is developed. I will NOT be that loud, vocal, open-source criticizer who laments the lack of work and progress from underpaid developers not giving into my demands and wants, so I began to research other options.

Some medium-sized instances had come to similar conclusions; whispers of moving to PieFed and PyFedi were becoming louder. Intrigued, I began looking into the PieFed project. I started doing some research, dodging thinly-veiled threats from my VPN provider that I was responsible for slowing down their shared server and I Better Fix It Soon Or Else. After waiting more than two minutes for lemmy.fan to crank out a single page of content as the server load averages climbed into the high twenties, I made the heartbreaking decision that lemmy.fan, the real.lemmy.fan, needed to be put out to pasture, and right soon. A deadline was set, I put some more popular communities (double-digit user counts, baby!) to read-only or migrated them to the new PieFed.lemmy.fan instance, and started the countdown clock. Archive.org has a cache of it if you are interested.

Then, lemmy.fan had its stroke.

To be continued…

¹ The original URL for the new PieFed instance I wanted to bring up was piefed.lemmy.fan. In the time between the plan to migrate and spin up PieFed Fan I purchased the piefed.fan domain. Because I have not yet worked much on PieFed after resuscitating lemmy.fan, I have not changed the federation URL in PieFed so the instance is not running yet. My goal is to have PieFed.fan up and running by the time part 2 of this post is published.

Thanks for the head’s up. Is it timing out or 404?

Edit: It’s not a federated community so of course it will not appear unless you are a lemmy.fan user. That’s my bad. I put the post below.

Master of Sharts in Colonial Studies

I hadn’t heard of a few of these and they sound really enjoyable. Thanks for sharing!

One Week by Barenaked Ladies is a mindfuck, as is REMs It’s the End of the World As We Know It (And I Feel Fine).

Not from US, but why do you wanna move out of US?

I’m guessing it’s politics. Our “religious conservatives” feel they can force women to have babies against their express desires while limiting birth control, sex education, and free school lunches.

It’s a party dominated by rich old white men who can’t rule over slaves anymore so they are going after all minorities and women.

I really felt cheated that no one was insulted, no rants were frothed, no theories conspiracied in this nfo. All we got was relevant information and kindness.

I’m kidding obvs

This is great advice, and to the OP, don’t feel bad. You’re really not an IT person of any caliber until you have experienced when I like to call the “Production Incident Experience”, or PIE. IT work is a job with unforseen consequences and hurdles, and we’ve all run into them at one point or another.

This being a learning experience, do what we’ve all done and learn from it. Now you can set up logging, whatif, sandbox instance, whatever you have to do.

You’re on the road to becoming a good programmer - just learn from your mistakes, do your research on best practices, ask intelligent questions, and in no time at all you’ll be writing one of these posts yourself.

I got lemmy.fan just a few months ago. I wanted a short, memorable TLD that stood out a little.

Blinking 12:00 intensifies

I have bought two laser printers in fifteen years. I got a Canon to replace the slow Brother MFC. I’ve lasted years on the toners I bought after the starter toners ran out. Toners never dry out and they don’t have the same print quality issues that ink jets have.

Anyone using an ink jet printer for anything other than printing photos onto photo paper is wasting tons and tons of money.

Oh, I understand now. Yes, that is strange.

It’s showing that it was cross posted. It may have just taken a few minutes.

Yep, looking like it’s expiring now.

I am running it in a VM now, using Linux and Docker.

Thanks for sharing this!

The Lemmy Easy Deploy script available at GitHub supports Arm64 in theory. In practice, some required binary refused to run on my Pi 4 so I think it’s still a WIP.

Hello fellow midwesterner?

This girl I knew had a fake seizure in front of me at a public museum. She had a crush on me, which I already was aware of, but I liked her as a friend. Even so, the flag was pretty red on that friendship.

Signal