Intelligent scissors with OpenCV javascript

[bsekachev]

Motivation and context

Integration of OpenCV.js library into CVAT. Now Intelligent Scissors have been added (resolved #336), but we expect more functions in the future.

Threshold maintenance using ctrl + mousewheel

How has this been tested?

Manual testing

Checklist

• I submit my changes into the develop branch
• I have added description of my changes into CHANGELOG file
• I have updated the documentation accordingly
• I have added tests to cover my changes
• I have linked related issues (read github docs)
• I have increased versions of npm packages if it is necessary (cvat-canvas,
  cvat-core, cvat-data and cvat-ui)

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below)

[nmanovic]

Great!
/cc @dkurt @alvoron @SimpleVlad

cvat/apps/opencv/static/opencv/js/opencv.js

It would be nice to store somewhere some instructions how to update this file in the future

@alalek , do you think we can place information about the integration somewhere on OpenCV site? For example, tutorial or demo page? Let's say a statement like "the feature is integrated into CVAT annotation tool and you can try it online on cvat.org".

[nmanovic]

Should we consider it as server API and add into server-proxy?

[nmanovic]

?

[nmanovic]

@bsekachev , build bs/opencv and got the following error:

[nmanovic]

commit b20f5028dfa74b8e63bc7a4180aa064e5ed4dda7 (HEAD -> bs/opencv, origin/bs/opencv)
Author: Boris Sekachev <boris.sekachev@intel.com>
Date:   Thu Jan 28 12:22:59 2021 +0300

    Fixed corner case

[ActiveChooN]

Most works really fine, but have some thoughts.

1. If we have no script on related path, click on load button will be completed with success icon.
2. Threshold can be configurated with consts or state for larger working area for example.
3. Sometimes i've got an error message when move pointer fast enough. It quite rare and i connot record this case.
4. Guess we can improve interactivity by starting interactor if only previous interaction has finished. Without it you can only achieve interaction result when mouse pointer stops moving.

[bsekachev]

@bsekachev , build bs/opencv and got the following error:

@nmanovic Forgot to setup server & proxy, fixed.

[bsekachev]

@ActiveChooN

If we have no script on related path, click on load button will be completed with success icon.

Fixed.

Threshold can be configurated with consts or state for larger working area for example.

Ideally it should be configurated in UI, but it would be difficult to unify UI for all possible thresholds with different tools from OpenCV. Now we also do not have thresholds for automatic annotation, maybe it will be done in the future.

Guess we can improve interactivity by starting interactor if only previous interaction has finished. Without it you can only achieve interaction result when mouse pointer stops moving.

I am not sure I understand you correct

[nmanovic]

@bsekachev , an alternative way to implement the feature https://www.photoshopessentials.com/basics/selections/magnetic-lasso-tool/. What do you think?

Looks like it is a polygon feature with "shift" pressed but you don't need move the mouse along the boundary precisely.

[nmanovic]

Server part looks good for me. I think we can merge it as is for now.

[bsekachev]

Okay, let's merge it as an initial version and look how we can improve it in the future patches.

[jahaniam]

@bsekachev @nmanovic Hi guys,
I've tested this feature. It looks great although it mostly works well when there is a clear boundary between background and foreground.

I have these two suggestions:

1. removing points is done using ctrl + clicking. It is not consistent with normal polygon drawing which is right-clicking. It would have been nicer to be consistent.
2. It would have been great to automatically draw a point when it is at the end of the rectangle (like how magnetic lasso tool does in photoshop. or how it is done in current cvat when holding shift.)

Thanks again for your work.

[bsekachev]

@jahaniam

Hi, thanks for your feedback. Please, address your comment to dedicated issue #2800