remove depre code. add code to convert old models.

README.md

@@ -4,7 +4,7 @@ A Tensorflow implementation of faster RCNN detection framework by Xinlei Chen (x
 **Note**: Several minor modifications are made when reimplementing the framework, which give potential improvements. For details about the modifications and ablative analysis, please refer to the technical report [An Implementation of Faster RCNN with Study for Region Sampling](https://arxiv.org/pdf/1702.02138.pdf). If you are seeking to reproduce the results in the original paper, please use the [official code](https://github.com/ShaoqingRen/faster_rcnn) or maybe the [semi-official code](https://github.com/rbgirshick/py-faster-rcnn). For details about the faster RCNN architecture please refer to the paper [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](http://arxiv.org/pdf/1506.01497.pdf). 
 
 ### Detection Performance
-We only tested it on plain VGG16 and Resnet101 (thank you @philokey!) architecture so far. As the baseline, we report numbers using a single model on a single convolution layer, so no multi-scale, no multi-stage bounding box regression, no skip-connection, no extra input is used. The only data augmentation technique is left-right flipping during training following the original Faster RCNN. All models are released.
+The current code support **VGG16** and **Resnet V1** models. We tested it on plain VGG16 and Resnet101 (thank you @philokey!) architecture so far. As the baseline, we report numbers using a single model on a single convolution layer, so no multi-scale, no multi-stage bounding box regression, no skip-connection, no extra input is used. The only data augmentation technique is left-right flipping during training following the original Faster RCNN. All models are released.
 
 With VGG16 (``conv5_3``):
   - Train on VOC 2007 trainval and test on VOC 2007 test, **71.2**.
@@ -57,69 +57,46 @@ Additional features not mentioned in the [report](https://arxiv.org/pdf/1702.021
   make
   cd ..
   ```
-
-4. Download pre-trained models and weights. The current code support VGG16 and Resnet V1 models. Pre-trained models are provided by slim, you can get the pre-trained models [here](https://github.com/tensorflow/models/tree/master/slim#pre-trained-models) and set them in the ``data/imagenet_weights`` folder. For example for VGG16 model, you can set up like:
-   ```Shell
-   mkdir -p data/imagenet_weights
-   cd data/imagenet_weights
-   wget -v http://download.tensorflow.org/models/vgg_16_2016_08_28.tar.gz
-   tar -xzvf vgg_16_2016_08_28.tar.gz
-   mv vgg_16.ckpt vgg16.ckpt
-   cd ../..
-   ```
-   For Resnet101, you can set up like:
-   ```Shell
-   mkdir -p data/imagenet_weights
-   cd data/imagenet_weights
-   wget -v http://download.tensorflow.org/models/resnet_v1_101_2016_08_28.tar.gz
-   tar -xzvf resnet_v1_101_2016_08_28.tar.gz
-   mv resnet_v1_101.ckpt res101.ckpt
-   cd ../..
-   ```
 
-5. Install the [Python COCO API](https://github.com/pdollar/coco). The code requires the API to access COCO dataset.
-   ```Shell
-   cd data
-   git clone https://github.com/pdollar/coco.git
-   cd ..
-   ```
+4. Install the [Python COCO API](https://github.com/pdollar/coco). The code requires the API to access COCO dataset.
+  ```Shell
+  cd data
+  git clone https://github.com/pdollar/coco.git
+  cd ..
+  ```
 
 ### Setup data
 Please follow the instructions of py-faster-rcnn [here](https://github.com/rbgirshick/py-faster-rcnn#beyond-the-demo-installation-for-training-and-testing-models) to setup VOC and COCO datasets (Part of COCO is done). The steps involve downloading data and optionally creating softlinks in the ``data`` folder. Since faster RCNN does not rely on pre-computed proposals, it is safe to ignore the steps that setup proposals.
 
 If you find it useful, the ``data/cache`` folder created on my side is also shared [here](http://ladoga.graphics.cs.cmu.edu/xinleic/tf-faster-rcnn/cache.tgz). 
 
-### Demo and Test with (old) pre-trained models
-1. Download pre-trained models and weights (VGG16)
+### Demo and Test with pre-trained models
+1. Download pre-trained model
   ```Shell
-  # return to the repository root
-  cd ..
-  # VGG16 for both voc and coco using default training scheme
+  # Resnet101 for voc pretrained on 07+12 set
   ./data/scripts/fetch_faster_rcnn_models.sh
-  # VGG16 weights for imagenet pretrained model, extracted from released caffe model
-  ./data/scripts/fetch_imagenet_weights.sh
   ```
-  **Note**: if you cannot download the models through the link. You can check out the following solutions:
+  **Note**: if you cannot download the models through the link, or you want to try more models, you can check out the following solutions and optionally update the downloading script:
   - Another server [here](http://gs11655.sp.cs.cmu.edu/xinleic/tf-faster-rcnn/).
   - Google drive [here](https://drive.google.com/open?id=0B1_fAEgxdnvJSmF3YUlZcHFqWTQ).
 
 2. Create a folder and a softlink to use the pretrained model
   ```Shell
-  NET=vgg16_depre
-  mkdir -p output/${NET}
-  cd output/${NET}
-  ln -s ../../data/faster_rcnn_models/voc_2007_trainval ./
-  ln -s ../../data/faster_rcnn_models/coco_2014_train+coco_2014_valminusminival ./
-  cd ../..
+  NET=res101
+  TRAIN_IMDB=voc_2007_trainval+voc_2012_trainval
+  mkdir -p output/${NET}/${TRAIN_IMDB}
+  cd output/${NET}/${TRAIN_IMDB}
+  ln -s ../../../data/voc_2007_trainval+voc_2012_trainval ./default
+  cd ../../..
   ```
 
-3. Demo for testing on custom images (VGG16, VOC)
+3. Demo for testing on custom images
   ```Shell
   # at reposistory root
   GPU_ID=0
-  CUDA_VISIBLE_DEVICES=${GPU_ID} ./tools/demo_depre.py 
+  CUDA_VISIBLE_DEVICES=${GPU_ID} ./tools/demo.py 
   ```
-  **Note**: VGG16 testing probably requires 4G memory, so if you are using GPUs with a smaller memory capacity, please install it with CPU support only. Refer to [Issue 25](https://github.com/endernewton/tf-faster-rcnn/issues/25).  
+  **Note**: Resnet101 testing probably requires 4G memory, so if you are using GPUs with a smaller memory capacity, please install it with CPU support only. Refer to [Issue 25](https://github.com/endernewton/tf-faster-rcnn/issues/25).  
 
   Demo with Resnet101 if you have downloaded those and placed them in the proper locations:
    ```Shell
@@ -137,7 +114,26 @@ If you find it useful, the ``data/cache`` folder created on my side is also shar
   **Note**: If you cannot get the reported numbers, then probabaly the NMS function is compiled improperly, refer to [Issue 5](https://github.com/endernewton/tf-faster-rcnn/issues/5). 
 
 ### Train your own model
-1. Train (and test, evaluation)
+1. Download pre-trained models and weights. The current code support VGG16 and Resnet V1 models. Pre-trained models are provided by slim, you can get the pre-trained models [here](https://github.com/tensorflow/models/tree/master/slim#pre-trained-models) and set them in the ``data/imagenet_weights`` folder. For example for VGG16 model, you can set up like:
+   ```Shell
+   mkdir -p data/imagenet_weights
+   cd data/imagenet_weights
+   wget -v http://download.tensorflow.org/models/vgg_16_2016_08_28.tar.gz
+   tar -xzvf vgg_16_2016_08_28.tar.gz
+   mv vgg_16.ckpt vgg16.ckpt
+   cd ../..
+   ```
+   For Resnet101, you can set up like:
+   ```Shell
+   mkdir -p data/imagenet_weights
+   cd data/imagenet_weights
+   wget -v http://download.tensorflow.org/models/resnet_v1_101_2016_08_28.tar.gz
+   tar -xzvf resnet_v1_101_2016_08_28.tar.gz
+   mv resnet_v1_101.ckpt res101.ckpt
+   cd ../..
+   ```
+
+2. Train (and test, evaluation)
   ```Shell
   ./experiments/scripts/train_faster_rcnn.sh [GPU_ID] [DATASET] [NET]
   # GPU_ID is the GPU you want to test on
@@ -148,13 +144,13 @@ If you find it useful, the ``data/cache`` folder created on my side is also shar
   ./experiments/scripts/train_faster_rcnn.sh 1 coco res101
   ```
 
-2. Visualization with Tensorboard
+3. Visualization with Tensorboard
   ```Shell
   tensorboard --logdir=tensorboard/vgg16/voc_2007_trainval/ --port=7001 &
   tensorboard --logdir=tensorboard/vgg16/coco_2014_train+coco_2014_valminusminival/ --port=7002 &
   ```
 
-3. Test and evaluate
+4. Test and evaluate
   ```Shell
   ./experiments/scripts/test_faster_rcnn.sh [GPU_ID] [DATASET] [NET]
   # GPU_ID is the GPU you want to test on
@@ -165,7 +161,7 @@ If you find it useful, the ``data/cache`` folder created on my side is also shar
   ./experiments/scripts/test_faster_rcnn.sh 1 coco res101
   ```
 
-4. You can use ``tools/reval.sh`` for re-evaluation
+5. You can use ``tools/reval.sh`` for re-evaluation
 
 
 By default, trained networks are saved under:

data/scripts/fetch_faster_rcnn_models.sh

@@ -3,10 +3,11 @@
 DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/../" && pwd )"
 cd $DIR
 
-FILE=faster_rcnn_models.tgz
+NET=res101
+FILE=voc_0712_80k-110k.tgz
 # replace it with gs11655.sp.cs.cmu.edu if ladoga.graphics.cs.cmu.edu does not work
-URL=http://ladoga.graphics.cs.cmu.edu/xinleic/tf-faster-rcnn/$FILE
-CHECKSUM=865cdf7350a87ef41d6476e6e33b7212
+URL=http://ladoga.graphics.cs.cmu.edu/xinleic/tf-faster-rcnn/$NET/$FILE
+CHECKSUM=cb32e9df553153d311cc5095b2f8c340
 
 if [ -f $FILE ]; then
   echo "File already exists. Checking md5..."
@@ -24,7 +25,7 @@ if [ -f $FILE ]; then
   fi
 fi
 
-echo "Downloading Faster R-CNN models (2G)..."
+echo "Downloading Resnet 101 Faster R-CNN models Pret-trained on VOC 07+12 (340M)..."
 
 wget $URL -O $FILE
 

experiments/scripts/convert_vgg16.sh

@@ -0,0 +1,70 @@
+#!/bin/bash
+
+set -x
+set -e
+
+export PYTHONUNBUFFERED="True"
+
+GPU_ID=$1
+DATASET=$2
+NET=vgg16
+
+array=( $@ )
+len=${#array[@]}
+EXTRA_ARGS=${array[@]:2:$len}
+EXTRA_ARGS_SLUG=${EXTRA_ARGS// /_}
+
+case ${DATASET} in
+  pascal_voc)
+    TRAIN_IMDB="voc_2007_trainval"
+    TEST_IMDB="voc_2007_test"
+    STEPSIZE=50000
+    ITERS=70000
+    ANCHORS="[8,16,32]"
+    RATIOS="[0.5,1,2]"
+    ;;
+  pascal_voc_0712)
+    TRAIN_IMDB="voc_2007_trainval+voc_2012_trainval"
+    TEST_IMDB="voc_2007_test"
+    STEPSIZE=80000
+    ITERS=110000
+    ANCHORS="[8,16,32]"
+    RATIOS="[0.5,1,2]"
+    ;;
+  coco)
+    TRAIN_IMDB="coco_2014_train+coco_2014_valminusminival"
+    TEST_IMDB="coco_2014_minival"
+    STEPSIZE=350000
+    ITERS=490000
+    ANCHORS="[4,8,16,32]"
+    RATIOS="[0.5,1,2]"
+    ;;
+  *)
+    echo "No dataset given"
+    exit
+    ;;
+esac
+
+set +x
+NET_FINAL=${NET}_faster_rcnn_iter_${ITERS}
+set -x
+
+if [ ! -f ${NET_FINAL}.index ]; then
+    if [[ ! -z  ${EXTRA_ARGS_SLUG}  ]]; then
+        CUDA_VISIBLE_DEVICES=${GPU_ID} time python ./tools/convert_from_depre.py \
+            --snapshot ${NET_FINAL} \
+            --imdb ${TRAIN_IMDB} \
+            --iters ${ITERS} \
+            --cfg experiments/cfgs/${NET}.yml \
+            --tag ${EXTRA_ARGS_SLUG} \
+            --set ANCHOR_SCALES ${ANCHORS} ANCHOR_RATIOS ${RATIOS} TRAIN.STEPSIZE ${STEPSIZE} ${EXTRA_ARGS}
+    else
+        CUDA_VISIBLE_DEVICES=${GPU_ID} time python ./tools/convert_from_depre.py \
+            --snapshot ${NET_FINAL} \
+            --imdb ${TRAIN_IMDB} \
+            --iters ${ITERS} \
+            --cfg experiments/cfgs/${NET}.yml \
+            --set ANCHOR_SCALES ${ANCHORS} ANCHOR_RATIOS ${RATIOS} TRAIN.STEPSIZE ${STEPSIZE} ${EXTRA_ARGS}
+    fi
+fi
+