Extension of Semantic Image Synthesis with SPADE for Style Transfer

Acknowledgments

This code borrows heavily from SPADE

Installation

Clone this repo.

git clone https://github.com/NVlabs/SPADE.git
cd SPADE/

This code requires PyTorch 1.0 and python 3+. Please install dependencies by

pip install -r requirements.txt

This code also requires the Synchronized-BatchNorm-PyTorch rep.

cd models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../

Dataset Preparation

Preparing ADE20K-Outdoors Dataset. The ADE20K dataset can be downloaded here, which is from MIT Scene Parsing BenchMark. And we need the meta data to extract outdoor subset.

# Downlaod datasets
cd ~
wget your_ade20k_downlaod_url
wget http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip
# Mirror the directory of ADEChallengeData2016
find ~/ADEChallengeData2016 -type d -exec mkdir -p ~/ADEChallengeData2016_outdoors/{} \;
# extract subset 
python3 prepare_ade_outdoors.py
ln -s ~/ADEChallengeData2016_outdoors ~/SPADE_style_extention/datasets/ade20k_outdoors

Generating Images Using Pretrained Model

Once the dataset is ready, the result images can be generated using pretrained models. Download the pretrained models from Baseline, AdaIN Extension, save in 'checkpoints/', and run (change the how_many / batchSize for your own use. )

# baseline
python3 test_style2.py --name baseline_ade --dataset_mode ade20k --dataroot ./datasets/ade20k_outdoors/ --how_many 6 --batchSize 4  --use_vae

# extended model
python3 test_style2.py --name adain_ade --dataset_mode ade20k --dataroot ./datasets/ade20k_outdoors/ --how_many 6 --batchSize 4  --use_vae

The outputs images are stored at ./results/ by default. You can view them using the autogenerated HTML file in the directory.

Training New Models

Please refer to SPADE's README

Citation

@inproceedings{park2019SPADE,
  title={Semantic Image Synthesis with Spatially-Adaptive Normalization},
  author={Park, Taesung and Liu, Ming-Yu and Wang, Ting-Chun and Zhu, Jun-Yan},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2019}
}