This project is an AI experiment framework focused on data and model storage and interaction, primarily designed for research scenarios in adversarial machine learning, collaborative visual generation, and perceptual foundation models. Key features include:

1. Configuration parsing with support for YAML and command-line arguments coordination.
2. Experiment data management with automatic caching of current code and experiment directories, integrated with frameworks like TensorBoard.
3. Standardized experiment ID management using a three-tier Project-Config-ID system. Logs are automatically overwritten under the same ID, and the same YAML configuration file is used under the same Config. Each Project corresponds to a sub-project code repository and configuration repository.
4. Standardized file path management for multi-project integration, providing standardized path access interfaces for Config files, Log files, and data storage files.
5. Recursive search-based aggregation of experimental data in CSV format.

• sources_root contains the current project code. Can be managed using git submodule or linked to external repositories with ln -s.

• configs stores experiment configurations.

• logs stores experiment data.

• The locations of logs and dataset directories can be configured via path_cfg.yml.

• The experiment data structure uses a three-layer directory structure for easier review and incremental experiments:

  • First layer: Project level, defining project_name.
  • Second layer: Config level, defining config_name. During experiments, configurations are read from configs/{project_name}/{config_name}(.yml).
  • Third layer: Experiment ID level, defining experiment_name. Experiment records are stored in logs/{project_name}/{config_name}/{experiment_name}, with checkpoint restart support for experiments with the same ID.

• The workflow package defines path specifications and automatic configuration saving scripts (standarization.py), mainly including:

  • Automatic reading of configurations defined in the configs directory
  • Automatic copying of corresponding files from sources_root/{project_name}/* folders to the experiment record directory
  • Quick methods for accessing experiment record paths
  • Automatic initialization functions that read parameters from Python startup commands. Sample startup command shown below, see standarization.py for detailed parameter descriptions:

  • python script.py (-n project_name) -c config_name -i experiment_name ( -d experiment_notes)  (... config args)

  • script.py must be located in the sources_root/{project_name} directory. Before starting, add the framework root directory, sources_root/, and sources_root/{project_name} directories to the PYTHON PATH environment variable.

• In remote development scenarios with file synchronization, exclude dataset and log folders from synchronization (configured in PyCharm Pro Tools->Deployment->Configuration->Excluded Paths).

cifar_adv_test is a simple adversarial attack demo, entirely generated by a large model.

This experiment is based on the CIFAR-10 dataset, using a simple CNN model for image classification, and evaluates the model's robustness against FGSM (Fast Gradient Sign Method) adversarial attacks. The experiment generates adversarial samples with different perturbation strengths (epsilon) and tests the model's accuracy on these samples.

sources_root/cifar_adv_test/
├── __init__.py
├── main.py              # Main program
├── run.sh               # Run script
└── README.md            # Experiment documentation

configs/cifar_adv_test/
└── base.yaml            # Base configuration file

The base configuration file base.yaml contains the following parameters:

• batch_size: Training batch size
• learning_rate: Learning rate
• epochs: Number of training epochs
• epsilons: List of perturbation strengths for FGSM attack

1. Ensure required dependencies are installed in the conda environment:

   pip install omegaconf easydict torch torchvision

2. Run the experiment:

   PYTHONPATH=./:./sources_root:./sources_root/cifar_adv_test python sources_root/cifar_adv_test/main.py -c base -i experiment_id -d "实验说明"

   Or run using the script:

   cd sources_root/cifar_adv_test
   ./run.sh

The experiment results are saved in CSV format in the log directory, containing the following fields:

• project_id: Project identifier (cifar_adv_test)
• config_id: Configuration identifier (base)
• experiment_id: Experiment identifier
• metric: Metric name (clean_accuracy or fgsm_epsilon_x)
• value: Metric value

Additionally, the following files are saved:

• Model weights file (model.pth)
• Configuration file copy (config.yaml)
• Source code copy (source_code/)
• Experiment notes (exp_notes.txt)

Use the publish_compiler.py script to package and publish specific subprojects and their associated resources.

1. Create a publish configuration file:

   python workflow/publish_compiler.py --create-config

2. Edit the generated publish_config.yaml file to set the project to be published and related options

3. Run the publish script:

   python workflow/publish_compiler.py

The published package will include source code, configuration files, experimental results, and license files, making it suitable for paper publication.