Evaluating a model performance for CIFAR10-C (PyTorch)

Preparation

Download data

Download CIFAR10-C dataset from here.

(Optional) Set your model

A default model is ResNet56. it's model file is already contained in src/models/.
If you use your custom model, please do the following procedure.

• Add your model file to src/models/.
• Add a new model option to test.py.

Evaluation

# ResNet56 (default)
python src/test.py --weight_path <path/to/weight/file> --data_root <path/to/parent/directory/of/cifar10-c>

# Your custom model
python src/test.py --arch <your/model/name> --weight_path <path/to/weight/file> --data_root <path/to/parent/directory/of/cifar10-c>

Output

All corruption accuracy.
This figure will be saved in figs/.

(Optional) Other Useful Options

• fig_dir : Specify the directory path to save an output figure.
• weight_dir : Execute an evaluation for all weight files in the specified directory.
• corruptions : Evaluate a model performance for the specified corruption types.