Don’t all devs rewrite their commit history before pushing? I have a hundred commits in my branch, and I rebase that shit into a single-digit number of commits that actually capture my intent.
(j/k, I know they don’t, I have read my colleagues’ MRs)
- 0 Posts
- 68 Comments
Joined 3 years ago
Cake day: July 5th, 2023
You are not logged in. If you use a Fediverse account that is able to follow users, you can follow this user.
I always have to look to see whether I’m in /c/[email protected] or /c/[email protected]
4·2 months agoI believe that you’re thinking of the Suez Canal. The Panama Canal allows ship traffic to go from the Atlantic to the Pacific without going around all of South America.
They’re Pinky and The Brain… they’re Pinky and The Brain…
3·2 months agoI think it’s in reference to
18·3 months agoThis vastly underestimates how big and interconnected modern LLM networks are. They are 100s of GBs in size, and transmit GBs of data between layers 100s of times per second. Having to transmit any part of the intermediate state of an LLM over a 1 Gb/s link would degrade the performance of an LLM by over 1000x. Current top-of-the-line datacenter GPUs are running about 4 TB/s total memory bandwidth, a limit LLMs are already pushing up against.
The kind of clustering you’re talking about is being done: between multiple GPUs on the same machine. The slowdown of dropping to the measly 64 Gb/s speeds of the PCIe 5.0x16 bus is a huge performance hit and has to be done very carefully. The various parts of the computation are broken up and shuttled around.
It’s the combination of massively-parallel compute (in the GPU cores) and insanely-fast, fully interconnected RAM that makes generative AI possible. The load characteristics aren’t conducive to parallelization.
3·4 months ago“Kill dash 9! No more CPU time!” — Monzy
I also use CodeFloe. While smaller, they have fewer guidelines around what is allowed to be there. While Codeberg is generally okay with people putting small private repos there, I don’t feel comfortable using what I view as a public resource for my private stuff.
(TL;DR: It would take >10,000 of the satellites described in the video just to move the two data centres studied in this paper to space)
I remember that video (been watching Scott Manley on-and-off since his KSP Interstellar series)
He’s right that you can cool 20kW just fine and I agree that 100kW is still very doable with today’s engineering. Let’s assume that a MW is also within reason, though I think we’re starting to stretch practicality there, as we’re now talking about about 2500m2 of radiator if I’m remembering right. That would be 25 radiator groups, each one 5 times the size of an ISS group. I bet we could manage that with a few years of development.
The two datacenters that were studied in the linked article were 36MW and 169MW. So just to replace those two you would need 200 of those pushing-the-boundaries-of-human-ability satellites. Or, if you look at the Starlink-sized satellites that Scott Manley was referencing, you’d need OVER 10,000. And that’s just two data centres in one state in one country in the world.
I don’t think its “impossible”, or that “it can’t be cooled”. I think that focusing on the possibility of space data centres takes attention away from the harm that terrestrial data centres are causing today. “It’s okay if we build these on Earth right now, because we’ll move them into space later”? There’s nothing as permanent as a temporary solution.
Let’s force these companies to go to space by charging them exorbitant amounts of money to build terrestrial data centres to compensate for the effects that they have here. What would it cost to cool the areas around those data centres back down again? 100 million? A billion?
(And BTW, I’m a software engineer that’s been working in the AI space since 2018, before LLMs went crazy. I’m optimistic about AI in general. I’m pessimistic about companies that are clearly dumping externalities out into the general public.)
7·5 months agoI mean, he thinks he’s the mayor of Toronto, and really only wants to be mayor of Toronto, so…
The issue with space-based data centres is dissipating that heat, though. The ISS radiators can dissipate less than 100kW and they are the largest in space today, IIRC. Current land-based data centres already generate 100s of MW of heat. US Datacentres alone already consume multiple TWh of electricity/year.
I’m all for space-based data centres. But I don’t believe anyone who says they’re coming soon. One small space data centre would be 10 ISSs—the largest space architecture project to date.
I think what people who are pooh-poohing on space data centres are concerned about isn’t the literal heat issue, but that it serves the same purpose as the “Hyperloop”: not a real practicality, but serves to focus lawmakers attention in a direction that ignores a practical issue (with Hyperloop it was away from California HSR, which now has its own problems, but at least it was feasible)
The primary issue is that there’s a limit to how much energy you can get out based on the difference in temperature between the cold fluid (liquid or gas) and the hot fluid. With data centres it’s maybe 20°C? Based on that assumption and the Carnot Theorem you get a maximum work extraction efficiency of about 6-7%.
Unfortunately, in the data centres they obey the laws of thermodynamics.
It would work better in places that get colder, but unfortunately places like that don’t tend to have as much available electricity (or infrastructure).
An aside:
We are starting to run up against fundamental laws of how much energy is required to do a certain amount of computation. i.e. In order to do a computation that moves a system from a state X to another state Y, there is a minimum amount of entropy change. That entropy change requires a certain amount of energy based on thermodynamics, known as the Landauer Limit.
We were already only about a billion times less efficient than the limit in 2012. I would wager we’ve improved computation per watt by 1-2 orders of magnitude since then. Which means we might only be 107 or so off of the limit. That sounds like a lot, but when you think about how fast we’re improving…
Remember when people started wearing V masks to protest Scientology, and Anonymous showed up, and it was being coordinated on 4chan, and 4chan was somehow a source for good for a while?
Peppridge Farm remembers…
3·5 months agoList of fonts is in there.
When I clicked, I thought it was a cat/car typo joke because it’s a picture of a cat in a car.
F the bots. Would like to be able to have nice things. Happy that at least this is the 🇨🇦-made solution (at least the primary dev, anyways).
Does Fedecan have the budget to throw a couple of bucks a month to Xe? Completely understand if not, I’ve done not-for-profit corps before and I know what it’s like. But if the budget is there, spending it on a Canadian dev would be a nice choice, IMO.
Common pattern - the acqui-hire.
“These people are working in a problem area that we want to do better in. We’ll buy their company for their expertise.”
Whether they keep existing products or not is not a major factor in the decision and gets evaluated later. Often, because they want the people working on something new the existing products are put into maintenance mode or shut down.
Source: Have been acquired for both talent and for product. Seen both.
While imprisoned on Cardassia, yeah. :p
Chief Miles O’Brien would like a word…
That is exactly what the “dark” in “dark matter” and “dark energy” means: that our theories require it to exist to explain our observations, but we have no way to detect it, ergo it is “dark”.
It’s like the infamous cosmological constant that Einstein initially added to the theory of relativity. Just like the “dark” things it had no physical analogue, and was there so that his equations held for a universe of constant size. When Hubble demonstrated that (contrary to the belief at the time) the universe was expanding the cosmological constant was no longer needed.
It is entirely possible that a new theory will explain what the dark stuff is and we will be able to remove those terms from the equations that need them in order to explain our observations. It is possible that we will find some kind of actual matter that interacts with the gravitational force but not the strong, weak, or electromagnetic forces, which I think is what would be needed for a “dark matter particle”.
In my personal opinion, the best explanation I’ve heard is that the error terms are best explained by the expansion of space itself. Specifically, we have some very recent evidence that the expansion of space isn’t uniform, and that might explain a significant amount of the dark energy. PBS Space Time does a good job with relevant videos and keeping up with the science.
No physicist will tell you that we know what “dark matter” or “dark energy” is, because we don’t. I think most would agree with you that ultimately we will find multiple explanations that add up to “dark matter + dark energy”.
I think the one thing that I disagree with you on is that science is “bogged down” with theories and maybes. The theories and maybes are where the most exciting research is happening!
Science isn’t the body of knowledge that results from experimentation; it’s the cycle of theory and experimentation that generates new knowledge. We don’t remember the hundreds and hundreds of false theories that had to be argued about and discarded to get to relativity or quantum theory. Today we only read the papers that were proven right in hindsight. If you look at “dark matter/energy” science of today we’re in that churn, so it looks different. In time, all these disproved theories will be forgotten, too.
“Dark matter” and “dark energy” are the “errors in the prediction of Mercury’s orbit” (relativity) or “errors in the predictions of black-body radiation” (quantum) of our time.