Writing tests is a good example. It’s not great at writing tests, but it is definitely better than the average developer when you take the probability of them writing tests in the first place into account.
Outside of everything else discussed here, this is something I disagree with on a fundamental level, flawed tests are worse than no tests, IMO.
Not to get too deep in to the very contentious space of testing in development, but when it comes to automated testing, I think we’re better off with more rigorous[1] testing instead of just chasing test coverage metrics.
Validating tests through chaos/mutagen testing; or model verification (e.g. Kani) ↩︎
I never said you should use flawed tests. You ask AI to write some tests. You READ THEM and probably tweak them a little. You think "this test is basic but better than nothing and it took me 30 seconds. You commit it.
Outside of everything else discussed here, this is something I disagree with on a fundamental level, flawed tests are worse than no tests, IMO.
Not to get too deep in to the very contentious space of testing in development, but when it comes to automated testing, I think we’re better off with more rigorous[1] testing instead of just chasing test coverage metrics.
Validating tests through chaos/mutagen testing; or model verification (e.g. Kani) ↩︎
I never said you should use flawed tests. You ask AI to write some tests. You READ THEM and probably tweak them a little. You think "this test is basic but better than nothing and it took me 30 seconds. You commit it.
I guess, it would be useful as a kind of fuzzer with a high price to performance ratio.
Making tests to try and find vulnerabilities.
For normal functionality testing though, better off making it yourself.