Run generates massive amount of channels, causing QGIS to crash

[fredrikbacke92]

I don't have experience with Python so i've tried an easier way to predict an orthophoto with custom model but haven't gotten it to work like when i test your test models. See workflow below.

1. Export 640 px tiles of orthophoto with deepness (works like a charm).

2. Upload dataset to Roboflow. Annotate tiles there, using their simplistic interface.

• I've tried annotating both a instance segmentation and object detection project with no difference.

4. Export annotated dataset to Ultralytics and train model there.

• First trained a converted segmentation dataset (to object detection) with Yolov8m, and then tried Yolov5mu.
• Then tried a new dataset with native object detection. I trained this with Yolo5mu.

5. From Ultralytics i exported the trained model to .onnx

6. Load .onnx model to deepness and run with orthophoto

7. Creates tons of "channels" and crashes.

I've searched around and tried a bunch of things. I want to try to export to tensorflow/pytorch and from there export to .onnx but cant figure out how to point to a local path. I don't understand the checkpoint/hub thing. I've tried changing metadata on the .onnx using python script but it didn't change the amount of channels. Cant find the script mentioned in Docs and not 100% on the instructions.

My orthophoto is RGBA and been wondering if that's a problem. Though, when i use included model I can detect cars on a orthophoto with RGBA.

Would love some advice what to try. Maybe needs a metadata/config adjustment? Help on how to convert exported local tensorflow (.pb) and pytorch (.pt) to .onnx would also be appreciated.

Thanks for your work!

[przemyslaw-aszkowski]

Hi,
your workflow sounds reasonable. Can you share your model and part of the orthophoto so we can investigate?
Cheers
Przemek

[fredrikbacke92]

Absolutely! Had to upload to google drive. Trained on Yolov5mu, using pre-trained model on COCO 2017 dataset. Exported with settings FP16 Quantization "No", Dynamic Axes "No", Simplify Model "No", Opset 15.

https://drive.google.com/file/d/1vG3bPbrAjCJY0giUYcgtJGJQj77z1MUt/view?usp=sharing

[przemyslaw-aszkowski]

Hi,

1. Thanks for providing the details. The model you trained has an inverted output shape and is missing one value column. I'm unsure if it is some variation of YOLO model or just something with exporting the model.

Anyway, fortunately, I managed to extend the plugin code to handle your model output.

And your model works quite well! It found 375 trees on your orthophoto!

2. I see you set the default resolution in the model parameter as 2 cm per pixel. Though you wrote "Cant find the script mentioned in Docs and not 100% on the instructions." - can you please point to this place in the documentation so we can update it?

3. It will take time to release a new Deepness version, but you can download a zip with my latest fix. It should allow you to run your model without any issues. You need to install the zip file manually in QGIS (see instructions in the documentation). Here is a link to zip: https://github.com/PUTvision/qgis-plugin-deepness/suites/14894481743/artifacts/848534588

4. I'm pretty impressed you trained your model quite well from scratch without Python expertise :) I'm also not familiar with some tools you used. If you feel like preparing a short tutorial for others on how to achieve what you did, it would be really amazing!

Cheers
Przemek

[fredrikbacke92]

Wow! Amazing that you could fix it. Thank you SO much!

I've done a little testing and this fix also worked great for a model trained with Yolov8m. The only bug i found so far was that restricting input layer with polygon caused the inference to not find any trees. This is no biggie, but maybe worth mentioning.

2. I've tried alot and honestly not sure but i think i tried writing some code in notepad++ according to the text in the Docs. When it didn't fix it i just moved on. The script that i was referering to that i couldn't find was "tools/add_model_metadata.py".

I'm gonna try and see if I can change metadata again. Is it correct that you paste the text table from the docs into a notepad++, adjust settings, save as .py and then run it? And i couldn't figure out if m2 key and value be left as default?

4. I absolutely can put together a tutorial :) Should i do a post here on discussion board or rather a word document that can be ported to the Docs?

Trying to inference with a sliding window on your own without python experience wasn't going to good. This plugin put ends to that and everyone who wants to deploy in orthophoto environment can with some computer experience.

Again, thank you! Gotta love open source

[fredrikbacke92]

Bumping the thread because I found another error. I trained a multi class object detector with the same tools and architecture which creates the same issue with massive amount of channels. Model + .tif from google drive below.

https://drive.google.com/file/d/1GMZzCz9CGaC_N6axl5NvVAIzliEaOw70/view?usp=sharing

Would love some tips on how to change metadata. I've tried to understand the docs but i fail. What am I doing wrong?

>>> import json
>>> import onnx
>>> model = onnx.load('C:/Users/fredr/Downloads/model_provytor.onnx')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'onnx' has no attribute 'load'

[przemyslaw-aszkowski]

Hi again,
It is a similar situation as it was for the previous model, but now with multiple classes. Again the model is missing the confidence output. I did a quick rework to handle this case, but we may need to consider creating a separate types of Detectors in Deepnsess.

Again, please try the latest plugin from Github Actions: https://github.com/PUTvision/qgis-plugin-deepness/suites/15183894606/artifacts/867642208

Cheers,
Przemek

[przemyslaw-aszkowski]

To be refactored in #105