Diffusion-RSCC: Diffusion Probabilistic Model for Change Captioning in Remote Sensing Images

Diffusion-RSCC: Diffusion Probabilistic Model for Change Captioning in Remote Sensing Images
Xiaofei Yu, Yitong Li, Jie Ma* [paper]

Diffusion-RSCC

Here we provide the structure of our model:

LEVIR-CC Dataset

Download Source: -Thanks for the Dataset by Liu et. al:[GitHub]. Put the content of downloaded dataset under the folder 'data'

path to ./data:
                ├─LevirCCcaptions.json
                ├─images
                  ├─train
                  │  ├─A
                  │  ├─B
                  ├─val
                  │  ├─A
                  │  ├─B
                  ├─test
                  │  ├─A
                  │  ├─B

Installation and Dependencies

git clone https://github.com/Fay-Y/Diffusion-RSCC
cd Diffusion-RSCC
conda create -n DiffusionRSCC_env python=3.8
conda activate DiffusionRSCC_env
pip install -r requirements.txt

Preparation

Preprocess the raw captions and image pairs:

python word_encode.py
python img_preprocess.py

Training

To train the proposed Diffusion-RSCC, run the following command:

sh demo.sh

Testing

To test, evaluate and visualize on the test dataset, run the following command

sh testlm.sh

Visualization

cd result

In the paper, the predicted captions are saved in folder "result".

Prediction samples

Prediction results in test set with 5 Ground Truth captions are partly shown below, proving the effectiveness of our model. For each image pair, the left part is the before image, the righ part is the after image.