Release v2.5.0 #6425
Merged
Release v2.5.0 #6425
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The current mitigation approach (resolving the IP address and checking if it's in the private range) is insufficient for a few reasons: * It is susceptible to DNS rebinding (an attacker-controlled DNS name resolving to a public IP address when queried during the check, and to a private IP address afterwards). * It is susceptible to redirect-based attacks (a server with a public address redirecting to a server with a private address). * It is only applied when downloading remote files of tasks (and is not easily reusable). Replace it with an approach based on smokescreen, a proxy that blocks connections to private IP addresses. In addition, use this proxy for webhooks, since they also make requests to untrusted URLs. The benefits of smokescreen are as follows: * It's not susceptible to the problems listed above. * It's configurable, so system administrators can allow certain private IP ranges if necessary. This configurability is exposed via the `SMOKESCREEN_OPTS` environment variable. * It doesn't require much code to use. The drawbacks of smokescreen are: * It's not as clear when the request fails due to smokescreen (compared to manual IP validation). To compensate, make the error message in `_download_data` more verbose. * The smokescreen project seems to be in early development (judging by the 0.0.x version numbers). Still, Stripe itself uses it, so it should be good enough. It's also not very convenient to set up (on account of not providing binaries), so disable it in development environments. Keep the scheme check from `_validate_url`. I don't think this check prevents any attacks (as requests only supports http/https to begin with), but it provides a friendly error message in case the user tries to use an unsupported scheme.
<!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - ~~[ ] I have added a description of my changes into the [CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md) file~~ - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - ~~[ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~ - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern.
<!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> For unclear reasons, when this setting is enabled, the OpenAPI Generator runtime becomes unable to parse `AnnotationsRead` values from JSON. Specifically, it thinks that the JSON value is ambiguous, because it can be parsed both as `LabeledData` and as `IOBase`. The former makes sense, but the latter does not. I think it's just a bug in the runtime. This line is why the issue only occurs when `discard_unknown_keys` is set: https://github.com/OpenAPITools/openapi-generator/blob/v6.0.1/modules/openapi-generator/src/main/resources/python/model_utils.mustache#L1139 The CLI currently doesn't parse `AnnotationRead` anywhere, so this patch doesn't have any user-visible effects. However, I'm prototyping a new CLI command which will need to parse annotations. Change `tasks_create` so that it routes kwargs to the appropriate destinations instead of passing all of them both to `TaskWriteRequest` and `create_from_data`. I don't think this is _really_ necessary (the extra arguments should just be ignored), but it makes the code more illustrative. ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> Unit tests. ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - ~~[ ] I have added a description of my changes into the [CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md) file~~ - ~~[ ] I have updated the documentation accordingly~~ - ~~[ ] I have added tests to cover my changes~~ - ~~[ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~ - ~~[ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern.
…#6350) <!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> Fixes #6319 - Fixed invalid dataset root search, leading to invalid file matching - Restored detailed dataset import error messages - Added tests ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [ ] I have added a description of my changes into the [CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md) file - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern.
<!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch - [ ] I have added a description of my changes into the [CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md) file - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) - [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning)) ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: Boris Sekachev <boris@cvat.ai>
<!-- Raise an issue to propose your change (https://github.com/opencv/cvat/issues). It helps to avoid duplication of efforts from multiple independent contributors. Discuss your ideas with maintainers to be sure that changes will be approved and merged. Read the [Contribution guide](https://opencv.github.io/cvat/docs/contributing/). --> <!-- Provide a general summary of your changes in the Title above --> ### Motivation and context <!-- Why is this change required? What problem does it solve? If it fixes an open issue, please link to the issue here. Describe your changes in detail, add screenshots. --> Before  After   ### How has this been tested? <!-- Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> Manually ### Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply. If an item isn't applicable for some reason, then ~~explicitly strikethrough~~ the whole line. If you don't do that, GitHub will show incorrect progress for the pull request. If you're unsure about any of these, don't hesitate to ask. We're here to help! --> - [x] I submit my changes into the `develop` branch ~~- [ ] I have added a description of my changes into the [CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md) file~~ ~~- [ ] I have updated the documentation accordingly~~ ~~- [ ] I have added tests to cover my changes~~ ~~- [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~ ~~- [ ] I have increased versions of npm packages if it is necessary ([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning), [cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning), [cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning) and [cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~ ### License - [x] I submit _my code changes_ under the same [MIT License]( https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: Boris Sekachev <boris@cvat.ai>
azhavoro
requested review from
sizov-kirill,
zhiltsov-max,
bsekachev,
mdacoca,
SpecLad,
Marishka17 and
nmanovic
as code owners
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| request, view, AnnotationGuidePermission.Scopes.UPDATE, iam_context, obj=obj) | ||
| ) | ||
| except AnnotationGuide.DoesNotExist as ex: | ||
| raise ValidationError(str(ex)) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
| try: | ||
| db_target = Project.objects.get(id=self.project_id) | ||
| except Project.DoesNotExist as ex: | ||
| raise ValidationError(str(ex)) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
| try: | ||
| db_target = Task.objects.get(id=self.task_id) | ||
| except Task.DoesNotExist as ex: | ||
| raise ValidationError(str(ex)) |
Check warning
Code scanning / CodeQL
Information exposure through an exception Medium
Codecov Report
@@ Coverage Diff @@
## master #6425 +/- ##
==========================================
+ Coverage 79.73% 80.90% +1.16%
==========================================
Files 320 332 +12
Lines 37249 38426 +1177
Branches 6747 6924 +177
==========================================
+ Hits 29701 31088 +1387
+ Misses 7270 7023 -247
- Partials 278 315 +37
|
azhavoro
removed request for
SpecLad,
zhiltsov-max,
mdacoca,
bsekachev,
sizov-kirill and
Marishka17
|
Hi @nmanovic, |
|
Thanks for the link @bsekachev. But, it is a tag not a branch like release-2.4.9. To use this tag I need to create a branch from that tag. I only asked because all other releases have a branch. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[2.5.0] - 2023-07-05
Added
(png, jpeg, gif, webp images and pdf files) (Markdown documentation for tasks/projects #6191)
Fixed
(Can't import dataset into Pytorch (saved locally). #6047)
(Fix file matching in annotation import for multiple dots in filenames #6350)
Security
Previously, on task creation it was prohibited to specify remote data URLs
with hosts that resolved to IP addresses in the private ranges.
Now, redirects to such URLs are also prohibited.
In addition, this restriction is now also applied to webhook URLs.
System administrators can allow or deny custom IP address ranges
with the
SMOKESCREEN_OPTSenvironment variable.(Implement more comprehensive SSRF mitigation #6362).