☆ Yσɠƚԋσʂ ☆@lemmy.ml to Open Source@lemmy.mlEnglish · 19 days agoOpen Source Infrastructure is Breaking Down Due to Corporate Freeloadingnews.itsfoss.comexternal-linkmessage-square17fedilinkarrow-up1146arrow-down12cross-posted to: [email protected]
arrow-up1144arrow-down1external-linkOpen Source Infrastructure is Breaking Down Due to Corporate Freeloadingnews.itsfoss.com☆ Yσɠƚԋσʂ ☆@lemmy.ml to Open Source@lemmy.mlEnglish · 19 days agomessage-square17fedilinkcross-posted to: [email protected]
minus-squarerenegadespork@lemmy.jelliefrontier.netlinkfedilinkEnglisharrow-up70arrow-down1·19 days agoThere’s no excuse for this crap. Even if they insist on scraping every FOSS repo, there needs to be some logic to it (caches, diffs, longer intervals). These AI scrapers are so poorly thought out they are indistinguishable from DOS attacks.
minus-squareDigitalDilemma@lemmy.mllinkfedilinkEnglisharrow-up28·edit-219 days agoYou’re not wrong. Claudebot took down one of my sites repeatedly, hammering it for the same pages over and over at horrendous rates. I ended up spending several days having to convert it to a SSG and hosted it on Cloudflare pages. A lot of work I didn’t need to do.
There’s no excuse for this crap. Even if they insist on scraping every FOSS repo, there needs to be some logic to it (caches, diffs, longer intervals). These AI scrapers are so poorly thought out they are indistinguishable from DOS attacks.
You’re not wrong.
Claudebot took down one of my sites repeatedly, hammering it for the same pages over and over at horrendous rates.
I ended up spending several days having to convert it to a SSG and hosted it on Cloudflare pages. A lot of work I didn’t need to do.