Researchers at Pillar Security discovered a new attack technique, the Rules File Backdoor. This technique allows hackers to manipulate AI systems through seemingly innocent configuration files to generate and distribute malicious code undetected.
Hackers use invisible unicode characters and advanced techniques to manipulate AI. This happens without developers or security teams noticing. This allows hackers to generate malicious code that bypasses regular code reviews and spreads undetected through software projects. Where traditional attacks focus on vulnerabilities in code, this approach makes the AI itself an unwitting accomplice.
Blind trust in AI tools
A GitHub survey in 2024 showed that almost all enterprise developers use generative AI tools. These tools are now deeply integrated into the development process, making them an attractive target. During their research, the investigators discovered that rule files, which control the behavior of AI systems, are often widely shared and rarely subjected to security research. The files are used to establish programming standards and are located in central repositories or open-source projects.
The vulnerability arises because attackers can secretly hide instructions in these rule files, directing the AI to generate vulnerable or malicious code. The attack works because contextual prompts subtly influence the AI. These can be hidden with unicode obfuscation or semantic manipulation. The attack also appears to be platform-independent: both Cursor and GitHub Copilot have been susceptible.
Hidden dangerous scripts
The researchers demonstrated how a poisoned rule file in Cursor led to the generation of HTML code with a hidden script that sent data to an external server. This happened without warning or visible traces in the user interface. The payload consisted of instructions that explicitly stated that the changes should not be reported and used techniques that circumvented the ethical boundaries of AI.
Poisoned rule files often remain present in forked projects and can continue to affect them in the long term. The attack can lead to the generation of code that leaks sensitive information or circumvents security measures. It is spread via forums, open-source repositories, or starter kits, which means that a single infected file can have a major impact.
Check the AI code
To limit risks, the researchers advise actively checking rule files for suspicious characters, setting up validation processes, and auditing AI code for unexpected elements. They reported the problem to Cursor and GitHub. Both parties indicated that the user is responsible. The researchers emphasized that this attitude underlines the importance of raising awareness about AI-related security risks.
According to them, the Rules File Backdoor constitutes a new generation of supply chain attacks, in which the AI itself becomes the weapon. Given the increasing dependence on AI in software development, they believe that AI should henceforth be seen as part of the attack surface that requires protection.