MNT instruct AI tools to not open pull requests in github PULL_REQUEST_TEMPLATE.md #31643
Conversation
|
Here is some evidence of the problem. Examples: Extend: |
|
@StefanieSenger Once this wording is refined, the Contributing Guide is another place where it could be added. |
|
Do you know what tools people are/could be using for this? If we do find out then we could test drive the template to see if it does anything. Could we move it to the end of the template? That way the message for robots wouldn't take up valuable space at the top. It feels wrong/annoying to optimise for the abusers instead of the good people (who all have to wade through the message till they get to the stuff they should read). |
The cause seems to be that GitHub has recently allowed AI generated issues and PRs (and even sells it) via Copilot. Github even promotes this on its blog: How to create issues and pull requests in record time on GitHub. I have just added this information in the PR description. (Though it seems there were ways to generate PRs for a while. I found some "older" instrucions like this medium article from december 2023: Let AI generate Pull Requests from your GitHub issues!.) |
Towards #31679
This PR is a proposal to solve an increasing problem and is up to discussion.
The problem this PR is attempting to solve is that AI tools more and more people use to create PRs don't care about our Automated Contributions Policy. Since GitHub Copilot Coding Agent Has Arrived!, scikit-learn also gets flooded with AI spam. Many people who care about open source are unhappy about it (Allow us to block Copilot-generated issues (and PRs) from our own repositories).
I have tried to link a few examples for maintainers who didn't act as firebrigade in the last weeks in the comment below, but it feels already outdated.
Here is my original PR description that I wrote before learning about the Copilot Coding Agent:
Recently (last few weeks), I have encountered a lot of PRs that I think were either totally or partially generated by AI. These were all of extremely bad quality or irrelevant and I think, once these are out, there is no good way to deal with them while reviewing. I have closed quite some, but that took time, and in others, I have asked the human behind it to modify, but these are rather likely to struggle a lot and then stop replying at one point.
I was thinking that this - or a similar - change on the github PR template might discourage thoughtless AI generated PRs, at least a bit. It is talking directly to the agent and I have used an LLM to phrase it in such a way that an LLM can easily read and understand this. (They obviously like bullet points, clear "do-not ... - unless ..." statements and visual cues, the LLM also explained to me these were important.)
I wonder: is the PR template is the right spot for this kind of instruction, or would another location be more effective? I am not sure where in the process an AI tool would encounter this - if at all.