Add Detectron2 Support to TorchServe Object Detection Examples

[Mudassar-MLE]

🚀 The feature

Implement support for Detectron2 models within the TorchServe object detection examples. This includes:

1. Developing a custom handler that works seamlessly with both CPU and GPU environments.
2. Providing sample configuration files (YAML files) as examples for model deployment.
3. Ensuring compatibility with both pre-trained models from the Detectron2 Model Zoo and custom fine-tuned models.
4. Updating documentation to guide users on serving Detectron2 models using TorchServe.

Motivation:

Detectron2 is a widely used open-source object detection library developed by Facebook AI Research, licensed under the permissive Apache-2.0 license. This makes it highly suitable for commercial use without the legal complexities associated with more restrictive licenses. In contrast, competitor libraries like Ultralytics' YOLO models are licensed under the AGPL-3.0 license, which imposes stricter requirements and can be less favorable for commercial applications.

Companies seeking free and commercially friendly resources for object detection often prefer Detectron2. Currently, TorchServe provides examples for YOLOv8, Mask R-CNN, and Faster R-CNN but lacks support for Detectron2 models. While TorchServe offers an easy way to deploy models, developers unfamiliar with its internals or with integrating Detectron2 may find the process challenging.

Moreover, there is a lack of optimized scripts for serving Detectron2 models:

• Absence in Official Repository: The official TorchServe repository does not include examples or handlers for Detectron2 models.
• ** Community Scripts Issues:** While one community-contributed script exist in issue thread, that is not properly coded and have several issues.
• Device Compatibility Problems: These scripts often fail to handle models fine-tuned on GPU but deployed on CPU, leading to errors during prediction.
• Predictive Errors: Without appropriate device management, Detectron2 raises errors if a model trained on GPU is run in a CPU environment.
• Lack of Documentation: Insufficient code documentation makes it difficult for developers to understand and adapt these scripts.

Pitch:

By adding support for Detectron2 to TorchServe:

• Enhanced Capabilities: Users can easily deploy Detectron2 models, expanding the range of object detection models available through TorchServe.
• Commercial Viability: Provides a commercially friendly option for companies needing object detection solutions without licensing constraints.
• Robust Deployment: The custom handler will intelligently manage device settings (CPU or GPU), preventing errors associated with mismatched training and deployment environments.
• Versatility: Supports both pre-trained models from the Detectron2 Model Zoo and custom fine-tuned models, offering flexibility to users regardless of their needs.
• Ease of Use: Simplifies the deployment process by providing sample configuration files (YAML files) as examples, reducing the learning curve for developers.
• Community Benefit: Assists a significant portion of the PyTorch community that relies on Detectron2, making model deployment more accessible.
• Increased Adoption: Facilitating the use of Detectron2 with TorchServe can encourage more users to adopt TorchServe for their model serving needs.

I have already implemented a custom handler that addresses these issues, ensuring smooth deployment of Detectron2 models regardless of the training environment (CPU or GPU) and without requiring extensive configuration from the user. This contribution will be highly beneficial for users who wish to deploy Detectron2 models using TorchServe but may be unfamiliar with creating custom handlers or managing device compatibility.

Alternatives

While users can manually create custom handlers for Detectron2 models, this requires additional effort and can be a significant barrier for those unfamiliar with TorchServe's internals. New developers or individuals who are self-learning may find the process confusing and overwhelming. By adding official support and examples, we can greatly assist many developers, making it easier for them to deploy Detectron2 models without getting discouraged.

Existing Limitations:

• No Official Support: The official TorchServe repository lacks examples or handlers for Detectron2 models.
• Community Solutions Are Inadequate: Scripts shared by community members in issue threads are often not properly coded and have multiple issues.
• Device Handling Issues: These scripts frequently fail to manage device compatibility, causing errors when models fine-tuned on GPU are deployed on CPU.
• Prediction Failures: Without correct device management, deploying a model trained on GPU in a CPU environment leads to prediction failures due to Detectron2 errors.
• Insufficient Documentation: The lack of thorough code documentation makes it challenging for developers to understand and modify the scripts according to their needs.

Contribution:

I am willing to contribute my existing code to the TorchServe project and collaborate with the maintainers to ensure it meets all guidelines and quality standards.
My goal is to make it easier for developers to deploy Detectron2 models using TorchServe, thereby enhancing the utility of both projects.

[mreso]

Hi @Mudassar-MLE thanks for your proposal, we're always happy to integrate community contributions to the examples. Please go ahead and create a PR. I would suggest placing the example under: examples/object_detector/detectron2

[Mudassar-MLE]

Hi @mreso Thanks for the opportunity! I’ll create the PR and place the example under examples/object_detector/detectron2 as suggested. Could you please assign this issue to me?