WiderFace import does not work

[amjadsaadeh]

Actions before raising this issue

• I searched the existing issues and did not find anything similar.
• I read/searched the docs

Steps to Reproduce

1. Create an empty project
2. Put the example dataset into a zip file
3. Goto the new project
4. Click on the "Actions" button and click on "Import Dataset"
5. Select "WiderFace 1.0" as Import Format
6. Upload the created zip file

I am getting an error with the following message:

cvat_server                    | 2023-12-12 12:50:08,113 DEBG 'uvicorn-0' stderr output:
cvat_server                    | [2023-12-12 12:50:08,113] ERROR cvat.server.engine: Traceback (most recent call last):
cvat_server                    |   File "/home/django/cvat/apps/dataset_manager/bindings.py", line 1950, in import_dm_annotations
cvat_server                    |     raise CvatImportError("annotation has no label")
cvat_server                    | cvat.apps.dataset_manager.bindings.CvatImportError: annotation has no label
cvat_server                    |
cvat_server                    | The above exception was the direct cause of the following exception:
cvat_server                    |
cvat_server                    | Traceback (most recent call last):
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/rq/worker.py", line 936, in perform_job
cvat_server                    |     rv = job.perform()
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/rq/job.py", line 684, in perform
cvat_server                    |     self._result = self._execute()
cvat_server                    |   File "/opt/venv/lib/python3.10/site-packages/rq/job.py", line 690, in _execute
cvat_server                    |     return self.func(*self.args, **self.kwargs)
cvat_server                    |   File "/home/django/cvat/apps/engine/utils.py", line 260, in import_resource_with_clean_up_after
cvat_server                    |     result = func(filename, *args, **kwargs)
cvat_server                    |   File "/usr/lib/python3.10/contextlib.py", line 79, in inner
cvat_server                    |     return func(*args, **kwds)
cvat_server                    |   File "/home/django/cvat/apps/dataset_manager/project.py", line 176, in import_dataset_as_project
cvat_server                    |     project.import_dataset(f, importer, conv_mask_to_poly=conv_mask_to_poly)
cvat_server                    |   File "/home/django/cvat/apps/dataset_manager/project.py", line 155, in import_dataset
cvat_server                    |     importer(dataset_file, temp_dir, project_data, self.load_dataset_data, **options)
cvat_server                    |   File "/home/django/cvat/apps/dataset_manager/formats/registry.py", line 36, in __call__
cvat_server                    |     f_or_cls(*args, **kwargs)
cvat_server                    |   File "/home/django/cvat/apps/dataset_manager/formats/widerface.py", line 33, in _import
cvat_server                    |     import_dm_annotations(dataset, instance_data)
cvat_server                    |   File "/home/django/cvat/apps/dataset_manager/bindings.py", line 1890, in import_dm_annotations
cvat_server                    |     import_dm_annotations(sub_dataset, task_data)
cvat_server                    |   File "/home/django/cvat/apps/dataset_manager/bindings.py", line 2104, in import_dm_annotations
cvat_server                    |     raise CvatImportError("Image {}: can't import annotation "
cvat_server                    | cvat.apps.dataset_manager.bindings.CvatImportError: Image 0_Parade_image_01: can't import annotation #1 (bbox): annotation has no label

Expected Behavior

The dataset gets imported.

Possible Solution

No response

Context

Well, I started with the WiderFace dataset from huggingface, but unfortunatly it didn't work, even after considering CVAT docs. When using VGGFace2 1.0 with the corresponding example dataset works fine.

Environment

- Git commit 3c05e8479d4113470c1164349253a67e8767b1ec
- 
Client: Docker Engine - Community
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:07:41 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.10
  Git commit:       311b9ff
  Built:            Thu Oct 26 09:07:41 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.25
  GitCommit:        d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

• OS: Windows 10, with WSL 2.0

[amjadsaadeh]

Ok, I did some more investigation and found out that the root cause of the problem is within the dataset_meta.json, which contains a list of labels. Unfortunatly, the face label is missing, which is required by CVAT's WiderFace import routine. So, the main question is where should this be fixed? Should CVAT modified to cope with this or should the dataset and the test in datumaro be fixed? I would advocate for the latter, since adding the face label to the dataset_meta.json would make sense to me.

[zhiltsov-max]

Hi, if I understand correctly, fixing the dataset_meta.json file resolved the import problem, including this part:

I started with the WiderFace dataset from huggingface, but unfortunatly it didn't work, even after considering CVAT docs

I think it's a problem in the Datumaro example, and it should be fixed there.

[teja1412-hub]

@nmanovic and @zhiltsov-max, I've reproduced the error in my local development and according to the error there seems to be problem with the annotations label in the dataset, specifically, one of the bbox doesn't have a corresponding label as mentioned above. As this issue is still open, can you give me a hint on the work to be done here? If there is any, I would like to take it work on it.

[zhiltsov-max]

@teja1412-hub, it seems like the problem can be resolved just by updating the format example in our Datumaro fork, as it's being referenced by CVAT documentation. So, to fix this issue, please clone Datumaro locally and setup the dev environment, update the test dataset, update relevant tests in this file, and send us a PR in that repo. Please also check if the CVAT documentation on this format can be improved.

[teja1412-hub]

@nmanovic @zhiltsov-max Okay. Can you assign this to me please.

[teja1412-hub]

@nmanovic & @zhiltsov-max Could you review this and give me comments.

As the error was occurring due to absence of a label named "face" which is being looked up by the CVAT in the dataset_meta.json file and as according to the format of WiderFace, the labels.txt file is optional but if dataset_meta.json is present, it looks up to the label named "face", so adding "labels": ["face] inside the object resolves the error.

Although in the dataset_meta.json file, the labels_map which was previously mentioned as
"labels_map": {"0":"Parade", "1":"Handshaking"} was not being utilized anywhere while importing, still, I prefer to leave it as it is and add labels instead of deleting the file (as it's optional and import works without it) as might be used elsewhere (in datumaro test_widerface_format.py test_can_import function). So a PR with this change would be sufficient to solve the problem.

Additionally,

2) Task frame error while importing
Error: Although the above error was resolved, I discovered that if more than 1 folders under images as 0--label0, 1--label1 etc, the CVAT gives a task frame error while assigning the itemIDs to task frames from wider_face_train_bbx_gt.txt to images under 1--label1/ in images.
Reason: This is occurring due to the import process looking for the image in the 0--label0/ file only as root hint being set to 0--label0/. More specifically in the import_dm_annotations function in bindings.py (at cvat/apps/dataset_manager/) the root_hint is calculated and sent as a parameter while calling match_dm_item function at line 1967. And this function, when it reads the 1--label1/1_label1_image2.jpg annotation in wider_face_train_bbx_gt.txt file and it can't find the corresponding image in the folder 0--Parade0/ (as it is the root hint) it generated error.

taskname.zip/
├── labels.txt # optional
├── wider_face_split/
│ └── wider_face_<any_subset_name>bbx_gt.txt
└── WIDER<any_subset_name>/
└── images/
├── 0--label0/
│ └── 0_label0_image1.jpg
└── 1--label1/
└── 1_label1_image2.jpg

[zhiltsov-max]

@teja1412-hub, thank you for the investigation. I suggest that we start from the first problem. Regarding the second problem, do you have any ideas on how it can be resolved?

[teja1412-hub]

@nmanovic @zhiltsov-max For the first problem, addition of "labels": ["face"] in the dataset_meta.json file would suffice. I was able to import the dataset successfully once this change was made. I wrote the above message so I could inform you before sending a PR incase you suggest anything after reviewing.

Also, I forgot to mention that the images in the WideFace dataset in datumaro repository are erroneous. The images have a problem, need to look into them.

[teja1412-hub]

@nmanovic @zhiltsov-max I've fixed the first issue being encountered in the widerface dataset of datumaro repository. Pls find the link for the PR #36 and can you please review it.