A bit old but still interesting

    • FizzyOrange@programming.dev
      link
      fedilink
      arrow-up
      1
      ·
      19 hours ago

      It’s a very heavily gamed benchmark. The most frequent issues I’ve seen are:

      • Different uses of multi-threading - some submissions use it, some don’t.
      • Different algorithms for the same problem.
      • Calling into C libraries to do the actual work. Lots of the Python submissions do this.

      They’ve finally started labelling stupid submissions with “contentious” labels at least, but not when this study was done.