Recent reporting by Nieman Lab describes how some major news organizations—including The Guardian, The New York Times, and Reddit—are limiting or blocking access to their content in the Internet Archive’s Wayback Machine. As stated in the article, these organizations are blocking access largely out of concern that generative AI companies are using the Wayback Machine as a backdoor for large-scale scraping.
These concerns are understandable, but unfounded. The Wayback Machine is not intended to be a backdoor for large-scale commercial scraping and, like others on the web today, we expend significant time and effort working to prevent such abuse. Whatever legitimate concerns people may have about generative AI, libraries are not the problem, and blocking access to web archives is not the solution; doing so risks serious harm to the public record.
I’m on board with wanting this :)
Not from the side of them gaining more knowledge but from the side of companies creating them monetizing and otherwise enshittifying them.
If we had a competitive open-source LLM…
So you’re not wrong, I agree; but I was speaking of a different angle. heh
Ah, in that dimension what I see seems similar to oil processing, again. They are generally all similar. Better datasets - better output. A natural curve of expenses and results.
A competitive open-source LLM makes sense ; but the real asset is data. So said LLM will be hosted (or provided with computing power) commercially to work on said processed data, usually. There are no anarchist free gas stations, and just like that it will be a building block of businesses.
I suppose the real issue is paying for the servers. There’s already pushback against the datacenters needed to power LLMs as it is. I suppose the capital to build would have to come from somewhere.
It’s a pity we don’t have a good government for a project like that. That would truly be a public service.
Did some calculations recently. If we took the cropland on which we grow corn strictly for ethanol production and put solar on it, something like 5% IIRC could power enough EVs to replace ALL vehicles in the US. Which means we could use a little more land for solar to power datacenters designed to be as environmentally friendly as possible. A government-run LLM run for the public.
It’s a pipe dream because in our current reality, it could never happen. But like universal health care and a living minimum wage, it should exist.
I know, I’m straying from the topic again. ADHD gonna ADHD. heh
I suppose as long as we were able to regulate AI companies to make sure they were forced to be upfront, honest, useful… it would be a sufficient compromise. But I’m sure we can’t even have that little.
Well, if we continue my analogy, government-run oil processing plants and gasoline subsidies have not historically worked well.
It’s a device of investing hard power into computing.
That cropland will repurpose itself by market laws if the change is so dramatic, I think it is. I don’t like the AI hype, but the major change of converting hard power into data and data into answers to questions is potent enough. It’s not just the difference in energy volumes between ethanol and solar power, it’s also that liquid fuel is easier to store. It’s not an equal comparison you’re making. But if the energy demand is skewed enough on the side of grid-connected datacenters, then economically solar power might become more attractive.
I think oligopoly on data is the main threat to this. Datacenters and hosters providing power to run whatever you want with whatever data you want are not the bottleneck for competition and good evolution.
Various data harvesting farms in which users roam are.
It’s funny, I’m optimistic lately and feel like this family of technologies is slowly killing the oligopolies of previous generation. Well, not themselves, but the mechanisms that brought them into existence. Of course they too have moved on past those, but it’s sort of an improvement.