Use exported .onnx model on QGIS "Deepness" plugin

[fredrikbacke92]

Search before asking

• I have searched the HUB issues and discussions and found no similar questions.

Question

When running the plugin with exported model it generates a massive amount of 'channels' (polygon layers) causing QGIS to crash, due to running out of memory. I've tried a included .onnx model which only generate 1 channel which is the class 'car' and that works without problems. I've uploaded to models to netron.app, hopefully that helps.

Exported ultralytics model .onnx (Full graph)

Included model in plugin .onnx (Full graph)

Cheers!

Additional

No response

[github-actions]

👋 Hello @fredrikbacke92, thank you for raising an issue about Ultralytics HUB 🚀! Please visit our HUB Docs to learn more:

• Quickstart. Start training and deploying YOLO models with HUB in seconds.
• Datasets: Preparing and Uploading. Learn how to prepare and upload your datasets to HUB in YOLO format.
• Projects: Creating and Managing. Group your models into projects for improved organization.
• Models: Training and Exporting. Train YOLOv5 and YOLOv8 models on your custom datasets and export them to various formats for deployment.
• Integrations. Explore different integration options for your trained models, such as TensorFlow, ONNX, OpenVINO, CoreML, and PaddlePaddle.
• Ultralytics HUB App. Learn about the Ultralytics App for iOS and Android, which allows you to run models directly on your mobile device.
  • iOS. Learn about YOLO CoreML models accelerated on Apple's Neural Engine on iPhones and iPads.
  • Android. Explore TFLite acceleration on mobile devices.
• Inference API. Understand how to use the Inference API for running your trained models in the cloud to generate predictions.

If this is a 🐛 Bug Report, please provide screenshots and steps to reproduce your problem to help us get started working on a fix.

If this is a ❓ Question, please provide as much information as possible, including dataset, model, environment details etc. so that we might provide the most helpful response.

We try to respond to all issues as promptly as possible. Thank you for your patience!

[glenn-jocher]

@fredrikbacke92 hello, and thank you for bringing this to our attention!

From what you've described, it appears you're experiencing an issue with the exported ONNX model when using it with the QGIS "Deepness" plugin. The problem arises when the generated polygon layers or 'channels' become too numerous, leading to a memory overflow and consequently making QGIS crash.

In comparing the .onnx models, it sounds like the key difference is the number of classes they are trained to recognize. The model included in the plugin, which is working as expected, is trained to detect only a single class - 'car'. This means it exclusively creates polygon layers for this one class, hence only one channel is generated.

On the other hand, when exporting models with the Ultralytics HUB, they will by default be trained to detect numerous classes based on the COCO dataset. This expanded class detection is likely leading to the creation of many polygon layers, or 'channels', thereby exceeding the memory on QGIS and causing it to crash.

To resolve this, one possible solution would be to re-train your model to detect only the specific classes you are interested in. This would reduce the number of channels, and we expect it would prevent the memory overload for QGIS. You could perform your training with a dataset that contains only your desired classes to ensure the model will only generate the corresponding channels.

I hope this helps guide you in investigating and addressing this issue. If you have any further queries or need further clarification, please do not hesitate to reach out! We're here to help, and appreciate you being part of the Ultralytics and YOLO community.

[fredrikbacke92]

Thanks you for the answer!

I should mention i use roboflow for annotation then use the API to train model on ultralytics. So far i've tried instance segmentation and object detection project (you choose before annotating), so i've tried 2 datasets. Both of these i've only annotated one class, "tree". I've used magic select in roboflow, but when seeing the preview is converts to bounding boxes so i dont know i that would be a problem.

I tried training a new model without COCO pretraining. Used the object detection dataset and YOLOv8m. The only difference when trying this in QGIS was that it didn't visually generate layers in the layers tab. But it still crashes and you can see all channels still remain in the plugin options.

[fredrikbacke92]

Solved!

There was some compatible issue that was solved by the creator of deepness. See link.

PUTvision/qgis-plugin-deepness#101 (reply in thread)

This workflow now works if you install fixed version of deepness provided in the linked thread.

[UltralyticsAssistant]

Hello @fredrikbacke92,

I'm really glad to hear that you were able to resolve the issue with the help of deepness' creator! Updates and patches like the one you've applied are crucial in maintaining compatibility and performance across different software and plugins. They enable us to continue our work smoothly and effectively.

Now, the workflow with the updated version of the deepness plugin is performing as expected on your end, which is fantastic news. Thank you for sharing this resolution back to the Ultralytics and YOLO community. It might help others who are facing a similar issue.

Also, I'd like to extend our gratitude towards the creator of the deepness plugin for their fast response and solution. This is a great example of the open-source community working together to solve problems and improve the ecosystem.

Feel free to reach out if you have any more questions or need further assistance. We're always here to help!