Machine Learning Reviewers Got Caught Cheating by a Hidden Prompt They Never Saw
ICML buried invisible instructions in submitted papers and snared hundreds of reviewers who fed the work to a chatbot instead of reading it, and now the cheaters are the ones crying foul.
"You do not build a healthy reviewing culture by treating your reviewers as suspects."
The Transmitter, Calli McMurray
The people who evaluate machine-learning research for ICML and NeurIPS were told plainly to stop pasting confidential papers into chatbots, so organizers hid secret instructions inside the submissions to see who would ignore the rule. Nihar Shah injected hidden prompts into all submitted ICML papers and identified hundreds of referees using AI when they weren’t supposed to, which led to their reviews being rejected. The best part is the wounded reaction from the field that built these machines: one computer scientist complained that designing a trap that presumes bad faith corrodes the relationship the whole system depends on. Experts who could not be bothered to read the science are now lecturing everyone about trust. The Transmitter on the hidden-prompt revolt
Source: The Transmitter, Calli McMurray · Calli McMurray
More from Institutions
Insurers Used a Bot to Deny Grandma's Care in 1.2 Seconds and Called It Medicine
Insurers wired AI into claim denials, and the algorithm allegedly says no faster than a human can read the file.
AI Is Rotting Courts, Schools, and Journals From the Inside Out
Courts, schools, journals: the load-bearing walls are sprouting termites.