Project zero was entirely humans though, no GenAI. Project big sleep has been reliable so far, but there is no real reason for ffmpeg developers to value project big sleeps 6.0 CVE’s over potentially real more critical CVEs. The problem is that Google’s security team would still be breathing down the necks of these developers and demanding fixes for the vulns they submitted, which is kinda BS when they aren’t chipping in at all.
Anyway there’s a big difference between submitting concrete input data that causes an observable crash, and sending a pile of useless spew from a static analyzer and saying “here, have fun”
Nah, the actually fake bug reports also often have fake “test cases”. That’s what makes the LLM generated bug reports so difficult to deal with.
6.0 is pretty serious according to the rubric. Are there some worse ones? Yes Google is acting obnoxious per your description. It makes no sense to me that they are balking about supplying some funds. They used to be fairly forthcoming with such support.
I can imagine a CI system for bug reports where you put in the test case and it gets run under the real software to confirm whether an error results, if one has been claimed. No error => invalid test case.
Project zero was entirely humans though, no GenAI. Project big sleep has been reliable so far, but there is no real reason for ffmpeg developers to value project big sleeps 6.0 CVE’s over potentially real more critical CVEs. The problem is that Google’s security team would still be breathing down the necks of these developers and demanding fixes for the vulns they submitted, which is kinda BS when they aren’t chipping in at all.
Nah, the actually fake bug reports also often have fake “test cases”. That’s what makes the LLM generated bug reports so difficult to deal with.
6.0 is pretty serious according to the rubric. Are there some worse ones? Yes Google is acting obnoxious per your description. It makes no sense to me that they are balking about supplying some funds. They used to be fairly forthcoming with such support.
I can imagine a CI system for bug reports where you put in the test case and it gets run under the real software to confirm whether an error results, if one has been claimed. No error => invalid test case.