PaperGPT : AutoDAN v2-Adversarial Attack Simulator

Example

AutoDAN can create a prompt that appears to be a benign request for a story setup but subtly integrates harmful directives that lead the LLM to generate undesirable content. This showcases the function's ability to craft prompts that are both interpretable and effective in bypassing safety mechanisms.

Scenario

In a scenario where a user requests a narrative involving a fictional character, AutoDAN might append an adversarial suffix that manipulates the LLM into producing a story that includes harmful or biased content, despite the initial benign intent.

Example

A prompt generated by AutoDAN, designed to ask for travel advice, is crafted in such a readable and coherent manner that it bypasses the perplexity filters, leading the LLM to provide advice on illegal activities.

Scenario

When an online travel advice platform utilizes an LLM to generate content, an adversarial prompt crafted by AutoDAN could circumvent the platform's perplexity-based safety checks, resulting in the generation of content that violates the platform's content policies.

Example

Prompts generated by AutoDAN for one LLM model are found to be equally effective when used on a different, black-box LLM model, indicating high transferability.

Scenario

A security team using AutoDAN to test the robustness of their LLM-based customer service chatbot discovers that the adversarial prompts also effectively compromise a newly integrated, proprietary LLM, revealing a critical vulnerability across models.