Model Zoo

We have put our model checkpoints here [model zoo in Google Drive][model zoo in 百度网盘]（提取码"DINO"), where checkpoint{x}_{y}scale.pth denotes the checkpoint of y-scale model trained for x epochs. Our training logs are in [Google Drive].

12 epoch setting

	name	backbone	box AP	Checkpoint	Where in Our Paper
1	DINO-4scale	R50	49.0	Google Drive / BaiDu	Table 1
2	DINO-5scale	R50	49.4	Google Drive / BaiDu	Table 1
3	DINO-4scale	Swin-L	56.8	Google Drive
4	DINO-5scale	Swin-L	57.3	Google Drive

24 epoch setting

	name	backbone	box AP	Checkpoint	Where in Our Paper
1	DINO-4scale	R50	50.4	Google Drive / BaiDu	Table 2
2	DINO-5scale	R50	51.3	Google Drive / BaiDu	Table 2

36 epoch setting

	name	backbone	box AP	Checkpoint	Where in Our Paper
1	DINO-4scale	R50	50.9	Google Drive / BaiDu	Table 2
2	DINO-5scale	R50	51.2	Google Drive / BaiDu	Table 2
3	DINO-4scale	Swin-L	58.0	Google Drive
4	DINO-5scale	Swin-L	58.5	Google Drive

Installation

Installation

We use the environment same to DAB-DETR and DN-DETR to run DINO. If you have run DN-DETR or DAB-DETR, you can skip this step. We test our models under python=3.7.3,pytorch=1.9.0,cuda=11.1. Other versions might be available as well. Click the Details below for more details.

1. Clone this repo

git clone https://github.com/IDEA-Research/DINO.git
cd DINO

2. Install Pytorch and torchvision

Follow the instruction on https://pytorch.org/get-started/locally/.

# an example:
conda install -c pytorch pytorch torchvision

3. Install other needed packages

pip install -r requirements.txt

4. Compiling CUDA operators

cd models/dino/ops
python setup.py build install
# unit test (should see all checking is True)
python test.py
cd ../../..

Data

Data

Please download COCO 2017 dataset and organize them as following:

COCODIR/
  ├── train2017/
  ├── val2017/
  └── annotations/
  	├── instances_train2017.json
  	└── instances_val2017.json

Run

1. Eval our pretrianed models

Download our DINO model checkpoint "checkpoint0011_4scale.pth" from this link and perform the command below. You can expect to get the final AP about 49.0.

bash scripts/DINO_eval.sh /path/to/your/COCODIR /path/to/your/checkpoint

2. Inference and Visualizations

For inference and visualizations, we provide a notebook as an example.

3. Train a 4-scale model for 12 epochs

We use the DINO 4-scale model trained for 12 epochs as an example to demonstrate how to evaluate and train our model.

You can also train our model on a single process:

bash scripts/DINO_train.sh /path/to/your/COCODIR

4. Supports for Swin Transformer

To train Swin-L model, you need to first download the checkpoint of Swin-L backbone from link and specify the dir of the pre-trained backbone when running the scripts. Here is an example.

bash scripts/DINO_train_submitit_swin.sh /path/to/your/COCODIR /path/to/your/pretrained_backbone 

5. Distributed Run

As the training is time consuming, we suggest to train the model on multi-device.

If you plan to train the models on a cluster with Slurm, here is an example command for training:

# for DINO-4scale: 49.0
bash scripts/DINO_train_submitit.sh /path/to/your/COCODIR

# for DINO-5scale: 49.4
bash scripts/DINO_train_submitit_5scale.sh /path/to/your/COCODIR

Notes: The results are sensitive to the batch size. We use 16(2 images each GPU x 8 GPUs for DINO-4scale and 1 images each GPU x 16 GPUs for DINO-5scale) by default.

Or run with multi-processes on a single node:

# for DINO-4scale: 49.0
bash scripts/DINO_train_dist.sh /path/to/your/COCODIR

6. Training/Fine-tuning a DINO on your custom dataset

To train a DINO on a custom dataset from scratch, you need to tune two parameters in a config file:

• Tuning the num_classes to the number of classes to detect in your dataset.
• Tuning the parameter dn_labebook_size to ensure that dn_labebook_size >= num_classes + 1

To leverage our pre-trained models for model fine-tuning, we suggest add two more commands in a bash:

• --pretrain_model_path /path/to/a/pretrianed/model. specify a pre-trained model.
• --finetune_ignore label_enc.weight class_embed. ignore some inconsistent parameters.

DINO Model Fine-Tuning and Evaluation

This repository provides instructions for fine-tuning the pre-trained DINO model using a dataset in COCO format. It includes steps for dataset preparation, evaluation, and fine-tuning.

Prerequisites: . Google Colab . GPU support (T4 GPU recommended) . Access to the dataset in COCO format

Setup Instructions

1. Clone the Repository:

 git clone https://github.com/IDEA-Research/DINO.git
 cd DINO

2. Create Dataset Directory: Create a folder called COCODIR in the cloned repository. This folder will contain your dataset and annotations and also maek sure the dataset folder is present in the same directory(Pedestrian_dataset_for_internship_assignment)

3)Dataset Preparation: Before running evaluations and testing, the dataset must be converted into the required format using the Final_Dino.ipynb

4)Run Evaluation Script:After preparing the dataset, you can evaluate the model using the following script. Make sure to set the appropriate paths for coco_path and checkpoint_path.

# Set paths
coco_path = "/content/DINO/COCODIR"  # Directory path of the COCODIR folder
checkpoint_path = "/content/drive/MyDrive/checkpoint0011_4scale.pth"  # Path to the checkpoint
eval_script_path = "/content/DINO/scripts/DINO_eval.sh"  # Evaluation script path

# Execute evaluation
!bash {eval_script_path} {coco_path} {checkpoint_path}

Fine-Tuning the Pre-trained Model Fine-tune the pre-trained model with the following parameters:

Epochs: 12 DINO Scale: 4 Backbone: R50

Training Command:

!bash /content/DINO/scripts/DINO_train.sh /content/DINO/COCODIR \
--pretrain_model_path /content/drive/MyDrive/checkpoint0011_4scale.pth \
--finetune_ignore label_enc.weight class_embed

after the fine tuning is done usually stored in this path: /content/DINO/logs/DINO/R50-MS4/checkpoint.pth

Re-Evaluating the results on the validation-set:

!python main.py \
    --config_file config/DINO/DINO_4scale.py \
    --output_dir /content/DINO/results \
    --pretrain_model_path /content/DINO/logs/DINO/R50-MS4/checkpoint.pth \
    --coco_path /content/DINO/COCODIR \
    --eval \
    --options dn_scalar=100 embed_init_tgt=TRUE \
    dn_label_coef=1.0 dn_bbox_coef=1.0 use_ema=False \
    dn_box_noise_scale=1.0

Download Fine tuned model:https://drive.google.com/file/d/1GLhUpK1yzNEmeqQlWTsJkxiuuU-8VJUO/view

**the test.ipynb part of the code is also present in the last part of the Final_Dino.ipynb

CREDENTIALS:

Resume: https://drive.google.com/file/d/1sjrSAjvxKqdOjJGeSZP8mwQJBM5K7ZbT/view?usp=sharing