AI Slop and OSS

One of the downsides of the LLMs right now is the amount of AI slop that is generated by them. I discussed already the sycophancy problem and how it caused a fake proof and a failed bet. But I also want to talk about the scale of the problem: with AI people can generate slop at scale and drown reviewers.

This is relevant in scientific conferences, for example. It is so much easy to produce a paper, or a talk proposal. But these at most will be acceptable, never good. Usually, they are bad. They need a lot of human intervention to clean and fix.

But, this article is about open source contributions. Over the past few months I have reviewed multiple PRs where it was evident that the author used LLMs to generate the PR: overly verbose PR description, comments which actually harmed readability, coding style that was the average of the world, but not fitting the coding style of the project, useless try-catch patterns, etc. In most of these cases, the author just wanted to get a PR to the project, to bump up their resume. They spent at most 2 minutes to write the prompt but wasted tens of minutes, if not hours, of human review time – if the PR was not desk rejected. As an example, consider the imbalance in this PR.

At least,now I can point them to Simon’s article about how their job is to deliver code that works, to provide the accountability. And it’s nice to see that more and more communities are starting to add AI policies.

What is an unsolved problem is that of sock puppet accounts. It used to be recommended to have a minimum of two pairs of eyes reviewing critical code (and always having code to the main branch reviewed by someone). This was a good measure to prevent supply chain attacks. But now it is too easy to create a new fake persona by using an LLM, so I’m afraid that attacks such as xz would occur at some point in the future, if we are not finding a control.