Every run is a failed run

[thelcrysis]

When starting sample_run.py as python sample_coco.py --coco_path "coco/" --save_format "json" --sample_image_count 1000 --run_count 100, every run is a failed run.

Did I get something wrong or?

If I add print('failed_run) inside failed run check, this is the output:

loading annotations into memory...
Done (t=12.92s)
creating index...
index created!
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:06<00:00, 15.12it/s]
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
failed_run
{"info": {"description": "COCO 2017 Dataset", "url": "http://cocodataset.org", "version": "1.0", "year": 2017, "contributor": "COCO Consortium", "date_created": "2017/09/01"}, "licenses": [{"url": "http://creativecommons.org/licenses/by-nc-sa/2.0/", "id": 1, "name": "Attribution-NonCommercial-ShareAlike License"}, {"url": "http://creativecommons.org/licenses/by-nc/2.0/", "id": 2, "name": "Attribution-NonCommercial License"}, {"url": "http://creativecommons.org/licenses/by-nc-nd/2.0/", "id": 3, "name": "Attribution-NonCommercial-NoDerivs License"}, {"url": "http://creativecommons.org/licenses/by/2.0/", "id": 4, "name": "Attribution License"}, {"url": "http://creativecommons.org/licenses/by-sa/2.0/", "id": 5, "name": "Attribution-ShareAlike License"}, {"url": "http://creativecommons.org/licenses/by-nd/2.0/", "id": 6, "name": "Attribution-NoDerivs License"}, {"url": "http://flickr.com/commons/usage/", "id": 7, "name": "No known copyright restrictions"}, {"url": "http://www.usa.gov/copyright.shtml", "id": 8, "name": "United States Government Work"}], "categories": [{"supercategory": "person", "id": 1, "name": "person"}, {"supercategory": "vehicle", "id": 2, "name": "bicycle"}, {"supercategory": "vehicle", "id": 3, "name": "car"}, {"supercategory": "vehicle", "id": 4, "name": "motorcycle"}, {"supercategory": "vehicle", "id": 5, "name": "airplane"}, {"supercategory": "vehicle", "id": 6, "name": "bus"}, {"supercategory": "vehicle", "id": 7, "name": "train"}, {"supercategory": "vehicle", "id": 8, "name": "truck"}, {"supercategory": "vehicle", "id": 9, "name": "boat"}, {"supercategory": "outdoor", "id": 10, "name": "traffic light"}, {"supercategory": "outdoor", "id": 11, "name": "fire hydrant"}, {"supercategory": "outdoor", "id": 13, "name": "stop sign"}, {"supercategory": "outdoor", "id": 14, "name": "parking meter"}, {"supercategory": "outdoor", "id": 15, "name": "bench"}, {"supercategory": "animal", "id": 16, "name": "bird"}, {"supercategory": "animal", "id": 17, "name": "cat"}, {"supercategory": "animal", "id": 18, "name": "dog"}, {"supercategory": "animal", "id": 19, "name": "horse"}, {"supercategory": "animal", "id": 20, "name": "sheep"}, {"supercategory": "animal", "id": 21, "name": "cow"}, {"supercategory": "animal", "id": 22, "name": "elephant"}, {"supercategory": "animal", "id": 23, "name": "bear"}, {"supercategory": "animal", "id": 24, "name": "zebra"}, {"supercategory": "animal", "id": 25, "name": "giraffe"}, {"supercategory": "accessory", "id": 27, "name": "backpack"}, {"supercategory": "accessory", "id": 28, "name": "umbrella"}, {"supercategory": "accessory", "id": 31, "name": "handbag"}, {"supercategory": "accessory", "id": 32, "name": "tie"}, {"supercategory": "accessory", "id": 33, "name": "suitcase"}, {"supercategory": "sports", "id": 34, "name": "frisbee"}, {"supercategory": "sports", "id": 35, "name": "skis"}, {"supercategory": "sports", "id": 36, "name": "snowboard"}, {"supercategory": "sports", "id": 37, "name": "sports ball"}, {"supercategory": "sports", "id": 38, "name": "kite"}, {"supercategory": "sports", "id": 39, "name": "baseball bat"}, {"supercategory": "sports", "id": 40, "name": "baseball glove"}, {"supercategory": "sports", "id": 41, "name": "skateboard"}, {"supercategory": "sports", "id": 42, "name": "surfboard"}, {"supercategory": "sports", "id": 43, "name": "tennis racket"}, {"supercategory": "kitchen", "id": 44, "name": "bottle"}, {"supercategory": "kitchen", "id": 46, "name": "wine glass"}, {"supercategory": "kitchen", "id": 47, "name": "cup"}, {"supercategory": "kitchen", "id": 48, "name": "fork"}, {"supercategory": "kitchen", "id": 49, "name": "knife"}, {"supercategory": "kitchen", "id": 50, "name": "spoon"}, {"supercategory": "kitchen", "id": 51, "name": "bowl"}, {"supercategory": "food", "id": 52, "name": "banana"}, {"supercategory": "food", "id": 53, "name": "apple"}, {"supercategory": "food", "id": 54, "name": "sandwich"}, {"supercategory": "food", "id": 55, "name": "orange"}, {"supercategory": "food", "id": 56, "name": "broccoli"}, {"supercategory": "food", "id": 57, "name": "carrot"}, {"supercategory": "food", "id": 58, "name": "hot dog"}, {"supercategory": "food", "id": 59, "name": "pizza"}, {"supercategory": "food", "id": 60, "name": "donut"}, {"supercategory": "food", "id": 61, "name": "cake"}, {"supercategory": "furniture", "id": 62, "name": "chair"}, {"supercategory": "furniture", "id": 63, "name": "couch"}, {"supercategory": "furniture", "id": 64, "name": "potted plant"}, {"supercategory": "furniture", "id": 65, "name": "bed"}, {"supercategory": "furniture", "id": 67, "name": "dining table"}, {"supercategory": "furniture", "id": 70, "name": "toilet"}, {"supercategory": "electronic", "id": 72, "name": "tv"}, {"supercategory": "electronic", "id": 73, "name": "laptop"}, {"supercategory": "electronic", "id": 74, "name": "mouse"}, {"supercategory": "electronic", "id": 75, "name": "remote"}, {"supercategory": "electronic", "id": 76, "name": "keyboard"}, {"supercategory": "electronic", "id": 77, "name": "cell phone"}, {"supercategory": "appliance", "id": 78, "name": "microwave"}, {"supercategory": "appliance", "id": 79, "name": "oven"}, {"supercategory": "appliance", "id": 80, "name": "toaster"}, {"supercategory": "appliance", "id": 81, "name": "sink"}, {"supercategory": "appliance", "id": 82, "name": "refrigerator"}, {"supercategory": "indoor", "id": 84, "name": "book"}, {"supercategory": "indoor", "id": 85, "name": "clock"}, {"supercategory": "indoor", "id": 86, "name": "vase"}, {"supercategory": "indoor", "id": 87, "name": "scissors"}, {"supercategory": "indoor", "id": 88, "name": "teddy bear"}, {"supercategory": "indoor", "id": 89, "name": "hair drier"}, {"supercategory": "indoor", "id": 90, "name": "toothbrush"}], "images": [], "annotations": []}

[thelcrysis]

There is a restriction for sampled annotations so it contains at least one annotation per size (S,M,L) per class which is a hard restriction for small samples to satisfy. This can be bypassed by replacing a few lines in sample_coco.py:

failed_run = False
for k, v in size_dict.items():
	if k in annot_sampled:
		failed_run = True
		break
	ratios_obj_count[k] = annot_sampled[k] / float(v)
if failed_run:
	continue

with

number_of_annots = None
for k, v in size_dict.items():
    if k not in annot_sampled.keys():
        number_of_annots = 0
    else:
        number_of_annots = annot_sampled[k]

As this is author's decision and not a bug, I'll be closing this issue.

[hzy-del]

There is a restriction for sampled annotations so it contains at least one annotation per size (S,M,L) per class which is a hard restriction for small samples to satisfy. This can be bypassed by replacing a few lines in sample_coco.py:

failed_run = False
for k, v in size_dict.items():
	if k in annot_sampled:
		failed_run = True
		break
	ratios_obj_count[k] = annot_sampled[k] / float(v)
if failed_run:
	continue

with

number_of_annots = None
for k, v in size_dict.items():
    if k not in annot_sampled.keys():
        number_of_annots = 0
    else:
        number_of_annots = annot_sampled[k]

As this is author's decision and not a bug, I'll be closing this issue.

but

min_ratio = min(number_of_annots.values())
AttributeError: 'int' object has no attribute 'values'

[thelcrysis]

I don't really remember what was happening here, but I don't think you should be calculating min_ratio like that. You can see what I did in my fork of this project to circumvent this. Here it is: sample_coco.py. I originally opted for calculating KL divergence instead of min and max ratios, so you'll find lines where ratios is calculated commented out, but they still might work.