Issue with creating task with annotatios #6287

thaikoh · 2023-06-12T07:46:12Z

My actions before raising this issue

Read/searched the docs
Searched past issues

Steps to Reproduce (for bugs)

I have two CVAT instances, one on local machine, another on remote. Task creation on local machine works fine, but on remote I getting exception when I try to create task with annotations.

Expected Behaviour

Task created on both local and remote hosts.

Current Behaviour

Exception raised while creating task on remote machine:

cvat_sdk.api_client.exceptions.ApiException: Status Code: 400
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Allow': 'GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS', 'Content-Length': '28', 'Content-Type': 'application/vnd.cvat+json', 'Cross-Origin-Opener-Policy': 'same-origin', 'Date': 'Fri, 09 Jun 2023 09:38:52 GMT', 'Referrer-Policy': 'same-origin', 'Server': 'nginx/1.18.0 (Ubuntu)', 'Vary': 'Accept, Origin', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'DENY', 'X-Request-Id': 'bf291e39-535a-48b8-bad8-20353761fa12'})
HTTP response body: b'"No such file were uploaded"'

Tasks with no annotations were created.

Possible Solution

Context

Your Environment

Local: Mac Intel
Docker version 24.0.2, build cb74dfc

Remote: Ubuntu
Docker version 23.0.5, build bc4487a

CVAT are the same on both hosts.
Server version: 2.5
Core version: 9.1.1

SDK version: 2.4.4

My code:

    def create_task_(self,
                     name: str,
                     resources: List[str],
                     labels: Optional[list] = None,
                     an_file: Optional[str] = None,
                     an_format: Optional[str] = None,
                     project_id: Optional[int] = None):
        task_spec = {
            "name": name,
            "labels": labels,
            "project_id": project_id,
        }
        self._client.tasks.create_from_data(
            spec=task_spec,
            resource_type=ResourceType.LOCAL,
            resources=resources,
            annotation_format=an_format,
            annotation_path=an_file,
        )

Annotation formats I've tried to upload: PASCAL VOC 1.1 and CVAT 1.1.

Posted here #5781 as well.

The text was updated successfully, but these errors were encountered:

zhiltsov-max · 2023-06-12T09:16:29Z

Hi, could you please specify server and SDK versions?

thaikoh · 2023-06-12T09:23:56Z

CVAT are the same on both hosts.
Server version: 2.5
Core version: 9.1.1

SDK version: 2.4.4

zhiltsov-max · 2023-06-12T09:33:28Z

Could you please check if you can upload annotations separately, after the task is created?

thaikoh · 2023-06-12T11:55:41Z

I've tried to upload them after task creation, but result was the same.

def import_annotations(task: Task, annotations: str, format_name: str):
        task.import_annotations(filename=annotations, format_name=format_name)

zhiltsov-max · 2023-06-12T11:59:36Z

Could you please tell how the annotation file path looks like? Is it relative or absolute?

thaikoh · 2023-06-12T12:39:58Z

CVAT file for task with video looks like this:

<?xml version="1.0" encoding="utf-8"?>
<annotations>
  <version>1.1</version>
  <meta>
    <task>
      <id>729</id>
      <name>some_name</name>
      <size>1944</size>
      <mode>interpolation</mode>
      <overlap>5</overlap>
      <bugtracker></bugtracker>
      <created>2020-01-10 11:01:33.881107+00:00</created>
      <updated>2020-01-10 13:09:14.734967+00:00</updated>
      <start_frame>0</start_frame>
      <stop_frame>1943</stop_frame>
      <frame_filter></frame_filter>
      <z_order>False</z_order>
      <labels>
        <label>
          <name>cart</name>
          <attributes>
            <attribute>
              <name>type</name>
              <mutable>True</mutable>
              <input_type>radio</input_type>
              <default_value>empty</default_value>
              <values>empty
full
unclear</values>
            </attribute>
            <attribute>
              <name>difficult</name>
              <mutable>True</mutable>
              <input_type>checkbox</input_type>
              <default_value>false</default_value>
              <values>false</values>
            </attribute>
          </attributes>
        </label>
      </labels>
      <segments>
        <segment>
          <id>443</id>
          <start>0</start>
          <stop>1943</stop>
          <url>http://localhost:8080/?id=443</url>
        </segment>
      </segments>
      <owner>
        <username>sally</username>
        <email></email>
      </owner>
      <assignee>
        <username>bot</username>
        <email></email>
      </assignee>
      <original_size>
        <width>1280</width>
        <height>720</height>
      </original_size>
    </task>
    <dumped>2020-01-10 13:47:42.853681+00:00</dumped>
    <source>some_name.mp4</source>
  </meta>
  <track id="0" label="cart">
    <box frame="48" outside="0" occluded="0" keyframe="1" xtl="415.53" ytl="0.00" xbr="613.87" ybr="212.29">
      <attribute name="difficult">false</attribute>
      <attribute name="type">empty</attribute>
    </box>
    <box frame="49" outside="0" occluded="0" keyframe="0" xtl="409.18" ytl="0.00" xbr="612.55" ybr="222.15">
      <attribute name="difficult">false</attribute>
      <attribute name="type">empty</attribute>
    </box>
    <box frame="50" outside="0" occluded="0" keyframe="1" xtl="402.83" ytl="0.00" xbr="611.24" ybr="232.02">
      <attribute name="difficult">false</attribute>
      <attribute name="type">empty</attribute>
    </box>

    ...

  </track>
</annotations>

thaikoh · 2023-06-13T09:39:37Z

It looks like issue is related to traffic route. Uploading to localhost works fine, but for DNS name it doesn't.
May be some traefic settings need to be fixed?

zhiltsov-max · 2023-06-13T12:27:52Z

If the annotation file is uploaded successfully and you're using the latest develop branch, I feel the problem may be related to how uploaded files are stored on the server, as it was changed in #5909 . Please tell how the uploaded file name looks like and try to use the same SDK version as the server version. Please also check if uploading with UI works normally.

@Marishka17, please take a look.

thaikoh · 2023-06-13T13:19:45Z

Uploaded file names are:

for PVOC files - archive.zip
for CVAT files - cvat_01.xml

Manual uploading via UI works fine.

thaikoh · 2023-06-14T08:25:02Z

I discovered that despite of errors annotations is uploaded to tasks and they are visible in jobs.
Sorry, haven't checked that earlier.

bennet-cv · 2023-06-14T12:25:47Z

I am experiencing a similar Issue, I will add my information in the hope that the context may help with finding a solution.

My actions before raising this issue

Read/searched the docs
Searched past issues

Steps to Reproduce (for bugs)

I have one running version of CVAT on my local machine which I access via localhost:8080. I tried to use the High Level Python API to upload a task with annotations. The code looks like:

with make_client(host="localhost:8080", credentials=('User_Email', 'User_Password')) as client:

task_spec = {
    "name": "Review",
    "project_id": 1,
}

task = client.tasks.create_from_data(
    spec = task_spec,
    resource_type = ResourceType.LOCAL,
    resources = ['surgery.jpg'],
    annotation_path = 'annotations.zip',
    annotation_format = 'COCO 1.0',
    status_check_period = 5,
)

Expected Behaviour

Upload Task with annotations without exception.

Current Behaviour

Uploading in general works fine, the task gets created and the annoations are added but the following Exception is thrown:
Traceback (most recent call last): File "/home/dev108/Desktop/review.py", line 11, in <module> task = client.tasks.create_from_data( File "/home/dev108/.local/lib/python3.10/site-packages/cvat_sdk/core/proxies/tasks.py", line 381, in create_from_data task.import_annotations(annotation_format, annotation_path, pbar=pbar) File "/home/dev108/.local/lib/python3.10/site-packages/cvat_sdk/core/proxies/tasks.py", line 173, in import_annotations AnnotationUploader(self._client).upload_file_and_wait( File "/home/dev108/.local/lib/python3.10/site-packages/cvat_sdk/core/uploading.py", line 280, in upload_file_and_wait self._wait_for_completion( File "/home/dev108/.local/lib/python3.10/site-packages/cvat_sdk/core/uploading.py", line 198, in _wait_for_completion return self._client.wait_for_completion( File "/home/dev108/.local/lib/python3.10/site-packages/cvat_sdk/core/client.py", line 226, in wait_for_completion response = self.api_client.rest_client.request( File "/home/dev108/.local/lib/python3.10/site-packages/cvat_sdk/api_client/rest.py", line 211, in request raise ApiException(http_resp=r) cvat_sdk.api_client.exceptions.ApiException: Status Code: 400 Reason: Bad Request HTTP response headers: HTTPHeaderDict({'Allow': 'GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS', 'Content-Length': '28', 'Content-Type': 'application/vnd.cvat+json', 'Cross-Origin-Opener-Policy': 'same-origin', 'Date': 'Wed, 14 Jun 2023 10:56:50 GMT', 'Referrer-Policy': 'same-origin', 'Server': 'nginx/1.18.0 (Ubuntu)', 'Vary': 'Accept, Origin', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'DENY', 'X-Request-Id': 'a5cd258b-92f7-4dea-b78a-c7e9a924196e'})7

Possible Solution

I tried to test the code with the versions 2.4.5 and 2.4.4 of the cvat_server, but the server container does not start.
I tried to create a task first and upload the annotations with the task.import_annotations function, but then the same exception occurs just at another part of the code.
Creating a task without annotation_path and annotation_format works fine.
I also tried to change up the annotations, the exception occurs for zip and json files.
I tried to upload annotations manually via the web ui, this works fine.

Your Environment

Docker version: 24.0.2, build cb74dfc
Operating System and version: Ubuntu 22.04.2 LTS
CVAT Sdk: 2.4.4
CVAT Server: 2.4.6 (as mentioned before v2.4.5/v2.4.4 do not start up the cvat_server container)
Other diagnostic information / logs:
I attached the occuring exception from the cvat_server container.
cvat_log.txt

zhiltsov-max · 2023-06-14T13:30:37Z

Hi. We were able to reproduce the problem. It seems that there is no public released version of SDK for v2.4.5 / 2.4.6, while there are some API changes in 2.4.6, so v2.4.4 doesn't work. You can either use a self-compiled version of SDK from sources (read here how to do this), or wait for the public package to arrive in pip.

bennet-cv · 2023-06-15T17:50:55Z

Since today it is possible to install v2.4.6 via pip. Updating to this version fixed he problem for me. Thanks a lot @zhiltsov-max !

fshaker · 2024-06-18T20:12:57Z

I had problem uploading the annotations to a job that I had already created using the example codes that was provided in the cvat_sdk documentations. The code that I used was as follows:

    with ApiClient(configuration) as client:
            response = client.tasks_api.create_annotations(
                id=1688,
                filename="annotations.xml",
                format="CVAT 1.1",
                _parse_response=False,
            )
            print("response", response.data)

Using this code I got the following error:

      cvat_sdk.api_client.exceptions.ApiException: Status Code: 400 Reason: Bad Request HTTP response headers: HTTPHeaderDict({'Allow': 'GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS', 'Content-Length': '28', 'Content-Type': 'application/vnd.cvat+json', 'Cross-Origin-Opener-Policy': 'same-origin', 'Date': 'Tue, 18 Jun 2024 19:43:54 GMT', 'Referrer-Policy': 'same-origin', 'Server': 'nginx/1.18.0 (Ubuntu)', 'Vary': 'Accept, Origin, Cookie', 'X-Content-Type-Options': 'nosniff', 'X-Frame-Options': 'DENY', 'X-Request-Id': '931c845e-80e5-4c31-add6-b5118ab2f571'}) 
      HTTP response body: "No such file were uploaded" `

After searching the internet and reading any document that I could find, eventually I tried digging into the cvat codes. I finally found out what the solution is an present it here for others who might have the same problem. The following code worked for me:

    credentials = ('username','password',)
    host = "https://the_host.com"
    with make_client(host=host, credentials=credentials) as client:
            client.tasks.retrieve(1688).import_annotations( format_name="CVAT 1.1", filename="./annotations.xml")

I hope that it helps.

zhiltsov-max added bug Something isn't working infra labels Jun 14, 2023

zhiltsov-max closed this as completed Jun 16, 2023

NeBTrumpf mentioned this issue Aug 29, 2023

Tasks API POST request does not accept csrf token #6742

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue with creating task with annotatios #6287

Issue with creating task with annotatios #6287

thaikoh commented Jun 12, 2023 •

edited

Loading

zhiltsov-max commented Jun 12, 2023

thaikoh commented Jun 12, 2023

zhiltsov-max commented Jun 12, 2023 •

edited

Loading

thaikoh commented Jun 12, 2023 •

edited

Loading

zhiltsov-max commented Jun 12, 2023

thaikoh commented Jun 12, 2023 •

edited

Loading

thaikoh commented Jun 13, 2023

zhiltsov-max commented Jun 13, 2023 •

edited

Loading

thaikoh commented Jun 13, 2023

thaikoh commented Jun 14, 2023

bennet-cv commented Jun 14, 2023 •

edited

Loading

zhiltsov-max commented Jun 14, 2023 •

edited

Loading

bennet-cv commented Jun 15, 2023

fshaker commented Jun 18, 2024

Issue with creating task with annotatios #6287

Issue with creating task with annotatios #6287

Comments

thaikoh commented Jun 12, 2023 • edited Loading

My actions before raising this issue

Steps to Reproduce (for bugs)

Expected Behaviour

Current Behaviour

Possible Solution

Context

Your Environment

zhiltsov-max commented Jun 12, 2023

thaikoh commented Jun 12, 2023

zhiltsov-max commented Jun 12, 2023 • edited Loading

thaikoh commented Jun 12, 2023 • edited Loading

zhiltsov-max commented Jun 12, 2023

thaikoh commented Jun 12, 2023 • edited Loading

thaikoh commented Jun 13, 2023

zhiltsov-max commented Jun 13, 2023 • edited Loading

thaikoh commented Jun 13, 2023

thaikoh commented Jun 14, 2023

bennet-cv commented Jun 14, 2023 • edited Loading

My actions before raising this issue

Steps to Reproduce (for bugs)

Expected Behaviour

Current Behaviour

Possible Solution

Your Environment

zhiltsov-max commented Jun 14, 2023 • edited Loading

bennet-cv commented Jun 15, 2023

fshaker commented Jun 18, 2024

thaikoh commented Jun 12, 2023 •

edited

Loading

zhiltsov-max commented Jun 12, 2023 •

edited

Loading

thaikoh commented Jun 12, 2023 •

edited

Loading

thaikoh commented Jun 12, 2023 •

edited

Loading

zhiltsov-max commented Jun 13, 2023 •

edited

Loading

bennet-cv commented Jun 14, 2023 •

edited

Loading

zhiltsov-max commented Jun 14, 2023 •

edited

Loading