Remove support for redundant request media types in the API #5874
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/check |
|
❌ Some checks failed |
SpecLad
force-pushed
the
remove-redundant-parsers
branch
from
56c99da
to
9507d7b
Compare
|
/check |
|
❌ Some checks failed |
SpecLad
force-pushed
the
remove-redundant-parsers
branch
2 times, most recently
from
e51bbff
to
33f4ae6
Compare
|
/check |
|
❌ Some checks failed |
SpecLad
force-pushed
the
remove-redundant-parsers
branch
from
aa63bbb
to
7bf6885
Compare
|
/check |
|
✔️ All checks completed successfully |
Currently, every API endpoint that takes a request body supports (or at least declares to support) 4 media types: * `application/json` * `application/offset+octet-stream` * `application/x-www-form-urlencoded` * `multipart/form-data` Supporting multiple media types has a cost. We need to test that the various media types actually work, and we need to document their use (e.g., providing examples for each supported type). In practice, we mostly don't... but we still need to. In addition, the user, seeing the list of supported types, has to decide which one to use. Now, the cost could be worthwhile if the multiple type support provided value. However, for the most part, it doesn't: * `application/offset+octet-stream` only makes sense for the TUS endpoints. Moreover, for those endpoints it's the only type that makes sense. * `application/x-www-form-urlencoded` is strictly inferior to JSON. It doesn't support compound values, and it doesn't carry type information, so you can't, for example, distinguish a string from a null. It's potentially susceptible to CSRF attacks (we have protections against those, but we could accidentally disable them and not notice). Its main use is for form submissions, but we don't use HTML-based submissions. * `multipart/form-data` shares the downsides of `application/x-www-form-urlencoded`, however it does have a redeeming quality: it allows to efficiently upload binary files. Therefore, it has legitimate uses in endpoints that accept such files. Therefore, I believe it is justified to reduce the API surface area as follows: * Restrict `application/offset+octet-stream` to TUS endpoints and remove support for other types from those endpoints. * Remove `application/x-www-form-urlencoded` support entirely. * Restrict `multipart/form-data` support to endpoints dealing with file uploads. Note that I had to keep `multipart/form-data` support for `POST /api/cloudstorages` and `PATCH /api/cloudstorages/<id>`. That's because they accept a file-type parameter (`key_file`). I don't especially like this. Key files are not big, so the efficiency benefits of `multipart/form-data` don't matter. Therefore, I don't think we really need to support this type here; it would be more elegant to just use JSON and Base64-encode the key file contents. However, I don't have time to make that change right now, so I'm leaving it for another time.
SpecLad
force-pushed
the
remove-redundant-parsers
branch
from
7bf6885
to
43f1e9e
Compare
SpecLad
changed the title
zhiltsov-max
approved these changes
Mar 29, 2023
mikhail-treskin
pushed a commit
to retailnext/cvat
that referenced
this pull request
Jul 1, 2023
…5874) Currently, every API endpoint that takes a request body supports (or at least declares to support) 4 media types: * `application/json` * `application/offset+octet-stream` * `application/x-www-form-urlencoded` * `multipart/form-data` Supporting multiple media types has a cost. We need to test that the various media types actually work, and we need to document their use (e.g., providing examples for each supported type). In practice, we mostly don't... but we still need to. In addition, the user, seeing the list of supported types, has to decide which one to use. Now, the cost could be worthwhile if the multiple type support provided value. However, for the most part, it doesn't: * `application/offset+octet-stream` only makes sense for the TUS endpoints. Moreover, for those endpoints it's the only type that makes sense. * `application/x-www-form-urlencoded` is strictly inferior to JSON. It doesn't support compound values, and it doesn't carry type information, so you can't, for example, distinguish a string from a null. It's potentially susceptible to CSRF attacks (we have protections against those, but we could accidentally disable them and not notice). Its main use is for form submissions, but we don't use HTML-based submissions. * `multipart/form-data` shares the downsides of `application/x-www-form-urlencoded`, however it does have a redeeming quality: it allows to efficiently upload binary files. Therefore, it has legitimate uses in endpoints that accept such files. Therefore, I believe it is justified to reduce the API surface area as follows: * Restrict `application/offset+octet-stream` to TUS endpoints and remove support for other types from those endpoints. * Remove `application/x-www-form-urlencoded` support entirely. * Restrict `multipart/form-data` support to endpoints dealing with file uploads. Note that I had to keep `multipart/form-data` support for `POST /api/cloudstorages` and `PATCH /api/cloudstorages/<id>`. That's because they accept a file-type parameter (`key_file`). I don't especially like this. Key files are not big, so the efficiency benefits of `multipart/form-data` don't matter. Therefore, I don't think we really need to support this type here; it would be more elegant to just use JSON and Base64-encode the key file contents. However, I don't have time to make that change right now, so I'm leaving it for another time.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and context
Currently, every API endpoint that takes a request body supports (or at least declares to support) 4 media types:
application/jsonapplication/offset+octet-streamapplication/x-www-form-urlencodedmultipart/form-dataSupporting multiple media types has a cost. We need to test that the various media types actually work, and we need to document their use (e.g., providing examples for each supported type). In practice, we mostly don't... but we still need to. In addition, the user, seeing the list of supported types, has to decide which one to use.
Now, the cost could be worthwhile if the multiple type support provided value. However, for the most part, it doesn't:
application/offset+octet-streamonly makes sense for the TUS endpoints. Moreover, for those endpoints it's the only type that makes sense.application/x-www-form-urlencodedis strictly inferior to JSON. It doesn't support compound values, and it doesn't carry type information, so you can't, for example, distinguish a string from a null. It's potentially susceptible to CSRF attacks (we have protections against those, but we could accidentally disable them and not notice). Its main use is for form submissions, but we don't use HTML-based submissions.multipart/form-datashares the downsides ofapplication/x-www-form-urlencoded, however it does have a redeeming quality: it allows to efficiently upload binary files. Therefore, it has legitimate uses in endpoints that accept such files.Therefore, I believe it is justified to reduce the API surface area as follows:
Restrict
application/offset+octet-streamto TUS endpoints and remove support for other types from those endpoints.Remove
application/x-www-form-urlencodedsupport entirely.Restrict
multipart/form-datasupport to endpoints dealing with file uploads.Note that I had to keep
multipart/form-datasupport forPOST /api/cloudstoragesandPATCH /api/cloudstorages/<id>. That's because they accept a file-type parameter (key_file). I don't especially like this. Key files are not big, so the efficiency benefits ofmultipart/form-datadon't matter. Therefore, I don't think we really need to support this type here; it would be more elegant to just use JSON and Base64-encode the key file contents. However, I don't have time to make that change right now, so I'mleaving it for another time.
How has this been tested?
CI.
Checklist
developbranch[ ] I have updated the documentation accordingly[ ] I have added tests to cover my changes[ ] I have linked related issues (see GitHub docs)[ ] I have increased versions of npm packages if it is necessary(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)
License
Feel free to contact the maintainers if that's a concern.