-
-
Notifications
You must be signed in to change notification settings - Fork 15.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding polygon detection alongside bounding boxes #6506
Conversation
and polygon yolo
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👋 Hello @ahmad4633, thank you for submitting a YOLOv5 🚀 PR! To allow your work to be integrated as seamlessly as possible, we advise you to:
- ✅ Verify your PR is up-to-date with upstream/master. If your PR is behind upstream/master an automatic GitHub Actions merge may be attempted by writing /rebase in a new comment, or by running the following code, replacing 'feature' with the name of your local branch:
git remote add upstream https://github.com/ultralytics/yolov5.git
git fetch upstream
# git checkout feature # <--- replace 'feature' with local branch name
git merge upstream/master
git push -u origin -f
- ✅ Verify all Continuous Integration (CI) checks are passing.
- ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." -Bruce Lee
for more information, see https://pre-commit.ci
remove static path
for more information, see https://pre-commit.ci
… into test/polygon_yolo
for more information, see https://pre-commit.ci
… into test/polygon_yolo
Merhaba Uygar, please refer to Colab Tutorial. |
for more information, see https://pre-commit.ci
Hi, @RoudyES
I figured out that you can just comment Also I have a question about Also I don't know if this changes are correct, but I'll leave that question to you 🤷 |
@tujh2 Careful of removing that As for the Also, if you're on C++ I recommend swapping your inference engine to TensorRT, your pre and post processing steps will remain almost the same and you'll get a significant speedup in your pipeline. |
@RoudyES Hi, I've converted the model to onnx and when I want to do inference in C++ I get this error: (-2:Unspecified error) Can't create layer "416" of type "Range" in function 'cv::dnn::dnn4_v20211004::LayerData::getLayerInstance' Do you have a sample to use in C++ on onnx or TensorRT? Thanks |
@sctrueew it seems to me that you tried to run inference using OpenCV's DNN module. If so, please refer to @tujh2's comment above. They made some slight changes and were able to run the model with OpenCV's DNN. However, I recommend running ONNX models using ONNX Runtime. Also, if you compile onnx runtime from source with TensorRT execution provider you will be able to run the same .onnx model with TensorRT backend without changing your code (the runtime will automatically convert the onnx model to .engine). Or you can just export the .pt model into .engine using As for the sample, I'm afraid we don't have a C++ snippet. We usually run everything in C#. |
@sctrueew |
@RoudyES Hi, |
Thanks, I've exported successfully the model to onnx but I still get the error when I want to readNet. OpenCV version: 4.5.4 => built with GPU What changes did you in the C++ side? |
Nothing at all. Actually I use GoCV - golang bindings for OpenCV, so I didn't modify code inside OpenCV library at all |
I've followed your tips and finally, I could readNet but I get an error in forward():
|
This is a problem with def forward(self, x):
# x = x.copy() # for profiling
z = [] # inference output
for i in range(self.nl):
x[i] = self.m[i](x[i]) # conv
bs, _, ny, nx = x[i].shape # x(bs,267,20,20) to x(bs,3,20,20,89)
x[i] = x[i].view(bs, self.na, self.no, ny, nx).permute(0, 1, 3, 4, 2).contiguous()
if not self.training: # inference
if self.onnx_dynamic or self.grid[i].shape[2:4] != x[i].shape[2:4]:
self.grid[i] = self._make_grid(nx, ny, i)
y = x[i].clone()
#y[..., 8:] = y[..., 8:].sigmoid()
# y[..., 8], y[..., 9:] = y[..., 8].sigmoid(), y[...,
# 9:].softmax(dim=-1) # softmax loss for classes
if self.inplace:
y[..., :8] = (y[..., :8] + self.grid[i].repeat((1, 1, 1, 1, 4))) * self.stride[i] # xyxyxyxy
else:
xyxyxyxy = (y[..., :8] + self.grid[i].repeat((1, 1, 1, 1, 4))) * self.stride[i] # xyxyxyxy
y = torch.cat((xyxyxyxy, y[..., 8:]), -1)
z.append(y.view(bs, -1, self.no))
return x if self.training else (torch.cat(z, 1), x)
def _make_grid(self, nx=20, ny=20, i=0):
d = self.anchors[i].device
if check_version(torch.__version__, '1.10.0'): # torch>=1.10.0 meshgrid workaround for torch>=0.7 compatibility
yv, xv = torch.meshgrid([torch.arange(ny, device=d), torch.arange(nx, device=d)], indexing='ij')
else:
yv, xv = torch.meshgrid([torch.arange(ny, device=d), torch.arange(nx, device=d)])
return torch.stack((xv, yv), 2).expand((1, self.na, ny, nx, 2)).float() |
@tujh2 Thank you very much. it works for me but I have to change the result of forwarding. `
|
@tujh2 Do you have any ideas on how to resolve this issue? |
I can publish my Go-inference code. Just modify this for C++.
|
Thanks for sharing, If we have points more than 4 points how can we set (netWidth += 9). I don't know how many points are there? for example, one detection has 4 (x,y) and another one has 12(x,y) |
This is my code but not working.
` |
@sctrueew Please note that the model can only predict 4 points polygons. You will never have a result with more than 4 points (if you did your post processing isn't correct). |
@sctrueew This can't be achieved in this PR. The repo can only detect bounding boxes (such as the rotated boxes of the persons in this example). The image you shared seems to me the result of an Instance Segmentation model, where you get an object's bounding box along with it's contour polygon, this is not yet supported in this PR (although we might look into it in the future if this PR gets merged with master branch). |
There is a bug when I want to save the predicted text. If I run the inference command |
It seems that the size of |
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions YOLOv5 🚀 and Vision AI ⭐. |
Hi, no, i have missed nms step cause my photo context contains only 1 object. So i just choose polygon by max score. Also, as i know, polygon nms is an expensive step |
@tujh2 how about the preprocessing letterbox resize? can i have look on your code anyway , will be greatful. |
I would like to thank XinzeLee for his amazing work, that inspired this pull request.
This PR adds 4 points polygon detection capability to this repo. Original bounding box detection is not modified. The new head will output 9 values (index x1 y1 x2 y2 x3 y3 x4 y4), the labels for training are of the same format as the mentioned output.
Also, for training we need to install polygon IoU Cuda to speed up the validation process and we need set "--polygon True", a full tutorial for training is provided in this notebook Polygon Tutorial.
🛠️ PR Summary
Made with ❤️ by Ultralytics Actions
🌟 Summary
Enhanced support for polygon labels in YOLOv5.
📊 Key Changes
🎯 Purpose & Impact