This project contains the official implementation of the NeurIPS 2025 poster paper Exploring Semantic-constrained Adversarial Example with Instruction Uncertainty Reduction.

Project Website: https://semanticae.github.io.

1. Setup environment:

   # install torch based on your cuda version
   pip install torch==2.4.1 torchvision==0.19.1 --index-url https://download.pytorch.org/whl/cu124 # (modify to your specific cuda version)
   
   # install other requirements
   pip install -r requirements.txt

2. Demo run:

   export PYTHONPATH=./:./sources_root
   python ./sources_root/resadv_ddim/masked_2d_generation.py

   Results:

   	Exemplar Image	SemanticAE
   Content
   Resnet50	Jellyfish (0.7942)	Goldfish(0.9992)
   ViT-B/16 (Transfer Attack)	Jellyfish (0.7103)	Goldfish(0.4447)

3. Generate and evaluate ImageNet adversarial examples:

   cd sources_root/resadv_ddim
   bash run.sh

   Please refer to run.sh for details. In addition, we use surrogate & targe models defined in BlackboxBench, see ./sources_root/surrogate_models/models_blackboxbench for details.

This projects follows the structure of https://github.com/hujinCN/aiworkflow/

1. resadv_ddim module: Contains core adversarial attack generation algorithms

   • model.py implements the basic diffusion model attack framework
   • masked_2d_generation.py extends the base model with mask mechanism & attack losses.
   • evaluation directory contains test scripts

2. image_evaluation module: Responsible for evaluating generated adversarial examples

   • datasets.py defines dataset loaders
   • metrics.py implements various evaluation metrics
   • evaluator.py is the main evaluation program

3. workflow module: Provides standardized project configuration and utilities

   • standarization.py handles configuration files and parameter parsing
   • Other auxiliary utility functions

4. imagenet_analytics module: Handles ImageNet labels and category information

   • Contains coarse-grained label definition files

5. configs/semanticae module: YAML format configuration files

   • Defines configuration parameters for different models and evaluation tasks

Our code references the following projects:

• Diffusion Guided Adversarial Attacks: SD-NAE, Adv-Diff, VENOM
• Benchmarks: BlackboxBench, TransferAttack
• The code in sources_root/surrogate_models/models_blackboxbench is for evaluation purposes only and is licensed under CC BY-NC 4.0, see LICENSE.
• The code for 3D adversarial example Demo is based on the Trellis project, under MIT License.

@inproceedings{
  hu2025exploring,
  title={Exploring Semantic-constrained Adversarial  Example with Instruction Uncertainty Reduction},
  author={Jin Hu and Jiakai Wang and Linna Jing and Haolin Li and Haodong Liu and Haotong Qin and Aishan Liu and Ke Xu and Xianglong Liu},
  booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
  year={2025}
}

This project is licensed under the MIT License - see the LICENSE file for details.