Fine-tuning AdvPrompter: A Novel AI Method to Generate Human-Readable Adversarial Prompt

Large Language Models (LLMs) have succeeded greatly and are widely used in various fields. LLMs are sensitive to input prompts, and this behavior has led to multiple research studies to understand and exploit this characteristic. This helps to create prompts for learning tasks like zero-shot and in-context. For instance, AutoPrompt recognizes task-specific tokens for zero-shot text classification and fact retrieval. This approach uses gradient-based scoring of tokens considering task-specific loss evaluation to find the optimal probability distributions over discrete tokens.

Despite showing great capability, LLMs sometimes become vulnerable to certain jailbreaking attacks due to which irrelevant or toxic contents are generated. The main cause of jailbreaking attacks is the requirement of adversarial prompts by manual re-teaming, and one of its examples is inserting a suffix to a given instruction, which is inadequate and time-consuming. However, the automated generation of adversarial prompts frequently results in attacks that lack semantic meaning, can be easily identified by filters based on perplexity, and may need gradient information from the TargetLLM. 

Researchers from AI at Meta, and Max-Planck-Institute for Intelligent Systems, Tubingen, Germany, introduced a novel method that uses another LLM, AdvPrompter, to generate human-readable adversarial prompts in seconds. Compared to other optimized approaches, this method is ∼ 800× faster. The AdvPrompter is trained by utilizing an AdvPromterTrain algorithm that does not need access to the TargetLLM gradients. The trained AdvPrompter can generate suffixes and veil the input instruction, keeping its meaning intact. This tactic lures the TargetLLM into providing a harmful response. 

The approach proposed by researchers has the following key advantages:

• It enhances human readability with the help of AdvPromter, which generates clear human-readable adversarial prompts.
• Researchers’ experiments on multiple open-source LLMs have demonstrated excellent attack success rates (ASR) compared to previous approaches such as GCG and AutoDAN.
• The trained AdvPrompter can generate adversarial suffixes using next-token prediction, unlike previous methods such as GCG and AutoDAN, which need to solve new optimization problems for every generated suffix. 

Generated adversarial suffixes with the help of trained AdvPromter are random with a non-zero temperature that allows users to sample a diverse set of adversarial prompts rapidly. Evaluation of more samples leads to better performance and a successful outcome. It further stabilizes at around k = 10, where k is the number of candidates of a score vector. Moreover, researchers found that the initial version of Llama2-7b constantly improves without fine-tuning, which means that generated suffixes with diversity are helpful for a successful attack.

In conclusion, researchers proposed a novel method for automated red-teaming of LLMs. The main approach includes training AdvPromter using an algorithm called AdvPromterTrain to generate human-readable adversarial prompts. Further, a novel algorithm called AdvPromterOpt is useful for automatically generating adversarial prompts. It is also used in the training loop to fine-tune the AdvPrompter predictions. Future work includes a detailed analysis of safety fine-tuning from automatically generated data, which is motivated by the robust increase of the TargetLLM via AdvPrompter.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our Telegram Channel, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our 40k+ ML SubReddit