MODEL_ZOO.md

Model Zoo

Pre-trained model download links

Method	Config	Remarks	Download link
ImageNet	-	torchvision	imagenet_r50-21352794.pth
Random	-	kaiming	random_r50-5d0fa71b.pth
Relative-Loc	selfsup/relative_loc/r50.py	default	relative_loc_r50-342c9097.pth
Rotation-Pred	selfsup/rotation_pred/r50.py	default	rotation_r50-cfab8ebb.pth
DeepCluster	selfsup/deepcluster/r50.py	default	deepcluster_r50-bb8681e2.pth
NPID	selfsup/npid/r50.py	default	npid_r50-dec3df0c.pth
ODC	selfsup/odc/r50_v1.py	default	odc_r50_v1-5af5dd0c.pth
MoCo	selfsup/moco/r50_v1.py	default	moco_r50_v1-4ad89b5c.pth
MoCo v2	selfsup/moco/r50_v2.py	default	moco_r50_v2-58f10cfe.pth
	selfsup/moco/r50_v2_simclr_neck.py	-> SimCLR neck	moco_r50_v2_simclr_neck-70379356.pth
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	simclr_r50_bs256_ep200-4577e9a6.pth
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	simclr_r50_bs256_ep200_mocov2_neck-0d6e5ff2.pth
BYOL	selfsup/byol/r50.py	default

Benchmarks

VOC07 SVM & SVM Low-shot

Method	Config	Remarks	Best layer	VOC07 SVM	VOC07 SVM Low-shot
					1	2	4	8	16	32	64	96
ImageNet	-	torchvision	feat5	87.17	52.99	63.55	73.7	78.79	81.76	83.75	85.18	85.97
Random	-	kaiming	feat2	30.22
Relative-Loc		default	feat4	65.06
Rotation-Pred	selfsup/rotation_pred/r50.py	default	feat4	67.38
DeepCluster	selfsup/deepcluster/r50.py	default	feat5	74.26
NPID	selfsup/npid/r50.py	default	feat5	74.50
	selfsup/npid/r50_ensure_neg.py	ensure_neg=True	feat5	75.70
ODC	selfsup/odc/r50_v1.py	default	feat5	78.42
MoCo	selfsup/moco/r50_v1.py	default	feat5	79.18
MoCo v2	selfsup/moco/r50_v2.py	default	feat5	84.05
	selfsup/moco/r50_v2_simclr_neck.py	-> SimCLR neck	feat5	84.00
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	feat5	78.95
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	feat5	77.65
BYOL	selfsup/byol/r50.py	default

ImageNet Linear Classification

Note

• Config: configs/benchmarks/linear_classification/imagenet/r50_multihead.py for ImageNet (Multi) and configs/benchmarks/linear_classification/imagenet/r50_last.py for ImageNet (Last).
• For DeepCluster, use the corresponding one with _sobel.
• ImageNet (Multi) evaluates features in around 9k dimensions from different layers. Top-1 result of the last epoch is reported.
• ImageNet (Last) evaluates the last feature after global average pooling, e.g., 2048 dimensions for resnet50. The best top-1 result among all epochs is reported.

Method	Config	Remarks	ImageNet (Multi)					ImageNet (Last)
			feat1	feat2	feat3	feat4	feat5	avgpool
ImageNet	-	torchvision	15.18	33.96	47.86	67.56	76.17	74.12
Random	-	kaiming	11.37	16.21	13.47	9.07	6.54	4.35
Relative-Loc	selfsup/relative_loc/r50.py	default	14.76	31.29	45.77	49.31	40.20	38.83
Rotation-Pred	selfsup/rotation_pred/r50.py	default	12.89	34.30	44.91	54.99	49.09	47.01
DeepCluster	selfsup/deepcluster/r50.py	default	12.78	30.81	43.88	57.71	51.68	46.92
NPID	selfsup/npid/r50.py	default	14.28	31.20	40.68	54.46	56.61	56.60
ODC	selfsup/odc/r50_v1.py	default	14.76	31.82	42.44	55.76	57.70	53.42
MoCo	selfsup/moco/r50_v1.py	default	15.32	33.08	44.68	57.27	60.60	61.02
MoCo v2	selfsup/moco/r50_v2.py	default	15.35	34.57	45.81	60.96	66.72	67.02
	selfsup/moco/r50_v2_simclr_neck.py	-> SimCLR neck	15.19	32.54	43.12	60.36	67.08	65.39
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	17.09	31.37	41.38	54.35	61.57	60.06
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	16.97	31.88	41.73	54.33	59.94	58.00
BYOL	selfsup/byol/r50.py	default

Places205 Linear Classification

Coming soon.

ImageNet Semi-Supervised Classification

Note

• In this benchmark, the necks or heads are removed and only the backbone CNN is evaluated by appending a linear classification head. All parameters are fine-tuned.
• Config: under configs/benchmarks/semi_classification/imagenet_1percent/ for 1% data, and configs/benchmarks/semi_classification/imagenet_10percent/ for 10% data.
• When training with 1% ImageNet, we find hyper-parameters especially the learning rate greatly influence the performance. Hence, we prepare a list of settings with the base learning rate from {0.001, 0.01, 0.1} and the learning rate multiplier for the head from {1, 10, 100}. We choose the best performing setting for each method.
• Please use --deterministic in this benchmark.

Method	Config	Remarks	Optimal setting for ImageNet 1%	ImageNet 1%
				top-1	top-5
ImageNet	-	torchvision	r50_lr0_001_head100.py	68.68	88.87
Random	-	kaiming	r50_lr0_01_head1.py	1.56	4.99
Relative-Loc	selfsup/relative_loc/r50.py	default	r50_lr0_01_head100.py	16.48	40.37
Rotation-Pred	selfsup/rotation_pred/r50.py	default	r50_lr0_01_head100.py	18.98	44.05
DeepCluster	selfsup/deepcluster/r50.py	default	r50_lr0_01_head1_sobel.py	33.44	58.62
NPID	selfsup/npid/r50.py	default	r50_lr0_01_head100.py	27.95	54.37
ODC	selfsup/odc/r50_v1.py	default	r50_lr0_1_head100.py	32.39	61.02
MoCo	selfsup/moco/r50_v1.py	default	r50_lr0_01_head100.py	33.15	61.30
MoCo v2	selfsup/moco/r50_v2.py	default	r50_lr0_01_head100.py	38.71	67.90
	selfsup/moco/r50_v2_simclr_neck.py	-> SimCLR neck	r50_lr0_01_head100.py	31.37	59.65
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	r50_lr0_01_head100.py	36.09	64.50
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck	r50_lr0_01_head100.py	36.31	64.68
BYOL	selfsup/byol/r50.py	default

Method	Config	Remarks	Optimal setting for ImageNet 10%	ImageNet 10%
				top-1	top-5
ImageNet	-	torchvision	r50_lr0_001_head10.py	74.53	92.19
Random	-	kaiming	r50_lr0_01_head1.py	21.78	44.24
Relative-Loc	selfsup/relative_loc/r50.py	default	r50_lr0_01_head100.py	53.86	79.62
Rotation-Pred	selfsup/rotation_pred/r50.py	default	r50_lr0_01_head100.py	54.75	80.21
DeepCluster	selfsup/deepcluster/r50.py	default	r50_lr0_01_head1_sobel.py	52.94	77.96
NPID	selfsup/npid/r50.py	default	r50_lr0_01_head100.py	57.22	81.39
ODC	selfsup/odc/r50_v1.py	default	r50_lr0_1_head10.py	58.15	82.55
MoCo	selfsup/moco/r50_v1.py	default	r50_lr0_01_head100.py	60.08	84.02
MoCo v2	selfsup/moco/r50_v2.py	default	r50_lr0_01_head100.py	61.64	84.90
	selfsup/moco/r50_v2_simclr_neck.py	-> SimCLR neck		60.60	84.29
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	r50_lr0_01_head100.py	58.46	82.60
	selfsup/simclr/r50_bs256_ep200_mocov2_neck.py	-> MoCo v2 neck		58.38	82.53
BYOL	selfsup/byol/r50.py	default

PASCAL VOC07+12 Object Detection

Note

• This benchmark follows the evluation protocols set up by MoCo.
• Config: benchmarks/detection/configs/pascal_voc_R_50_C4_24k_moco.yaml.
• Please follow here to run the evaluation.

Method	Config	Remarks	VOC07+12
			AP50	AP	AP75
ImageNet	-	torchvision	81.58	54.19	59.80
Random	-	kaiming	59.02	32.83	31.60
Relative-Loc	selfsup/relative_loc/r50.py	default
Rotation-Pred	selfsup/rotation_pred/r50.py	default	80.91	55.52	61.39
NPID	selfsup/npid/r50.py	default	80.03	54.11	59.50
MoCo	selfsup/moco/r50_v1.py	default	81.38	55.95	62.23
MoCo v2	selfsup/moco/r50_v2.py	default	81.96	56.63	62.90
SimCLR	selfsup/simclr/r50_bs256_ep200.py	default	79.41	51.54	55.63
BYOL	selfsup/byol/r50.py	default