I find this paper false/misleading. They just translated one algorithm in many languages, without using the language constructs or specificities to make the algorithm decent performant wise.
Also it doesn’t mean anything, as you aren’t just running your code. You are compiling/transpiling it, testing it, deploying it… and all those operations consume even more energy.
I’d argue that C/C++ projects use the most energy in term of testing due to the quantity of bugs it can present, and the amount of CPU time needed just to compile your 10-20k lines program. Just my 2 cents
The amount of CPU time compiling code is usually negligible compared to CPU time at runtime. Your comparison only really works if you are comparing against something like Rust, where less bugs are introduced due to certain guarantees by the language.
Regarding “language constructs” it really depends on what you mean. For example using numpy in python is kind of cheating because numpy is implemented in C. However using something like the algorithm libraries in Rust woulf be considered fair game since they are likely written in Rust itself.
They presented their methodology in an open and clear way and provide their data for everyone to interpret. You can disagree with conclusions but it’s pretty harsh to say it’s “misleading” simply because you don’t like the results.
They just translated one algorithm in many languages, without using the language constructs or specificities to make the algorithm decent performant wise.
They used two datasets, if you read the paper… It wasn’t “one algorithm” it was several from publicly available implementations of those algorithms. They chose an “optimized” set of algorithms from “The Computer Language Benchmarks Game” to produce results for well-optimized code in each language. They then used implementations of various algorithms from Rosetta Code which contained more… typical implementations that don’t have a heavy focus on performance.
In fact - using “typical language constructs or specificities” hurt the Java language implementations since List is slower than using arrays. It performed much better (surprisingly well actually) in the optimized tests than in the Rosetta Code tests.
I find this paper false/misleading. They just translated one algorithm in many languages, without using the language constructs or specificities to make the algorithm decent performant wise.
Also it doesn’t mean anything, as you aren’t just running your code. You are compiling/transpiling it, testing it, deploying it… and all those operations consume even more energy.
I’d argue that C/C++ projects use the most energy in term of testing due to the quantity of bugs it can present, and the amount of CPU time needed just to compile your 10-20k lines program. Just my 2 cents
The amount of CPU time compiling code is usually negligible compared to CPU time at runtime. Your comparison only really works if you are comparing against something like Rust, where less bugs are introduced due to certain guarantees by the language.
Regarding “language constructs” it really depends on what you mean. For example using numpy in python is kind of cheating because numpy is implemented in C. However using something like the algorithm libraries in Rust woulf be considered fair game since they are likely written in Rust itself.
They presented their methodology in an open and clear way and provide their data for everyone to interpret. You can disagree with conclusions but it’s pretty harsh to say it’s “misleading” simply because you don’t like the results.
They used two datasets, if you read the paper… It wasn’t “one algorithm” it was several from publicly available implementations of those algorithms. They chose an “optimized” set of algorithms from “The Computer Language Benchmarks Game” to produce results for well-optimized code in each language. They then used implementations of various algorithms from Rosetta Code which contained more… typical implementations that don’t have a heavy focus on performance.
In fact - using “typical language constructs or specificities” hurt the Java language implementations since List is slower than using arrays. It performed much better (surprisingly well actually) in the optimized tests than in the Rosetta Code tests.
Honestly that’s all you need to know to throw this paper away.
Why?
It’s a very heavily gamed benchmark. The most frequent issues I’ve seen are:
They’ve finally started labelling stupid submissions with “contentious” labels at least, but not when this study was done.