Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

"Invalid type for parameter ContentType" on uploading Scorm package #16

Closed
regisb opened this issue Dec 30, 2020 · 0 comments
Closed

"Invalid type for parameter ContentType" on uploading Scorm package #16

regisb opened this issue Dec 30, 2020 · 0 comments

Comments

@regisb
Copy link
Contributor

regisb commented Dec 30, 2020 •
edited
Loading

See discussion: https://discuss.overhang.io/t/issue-with-scorm/1126

The tutor-minio package is enabled. When uploading a Scorm package, I sometimes get the following error:

cms_1            | 2020-12-30 10:10:28,212 ERROR 8 [django.request] [user 3] [ip 172.18.0.1] log.py:222 - Internal Server Error: /xblock/block-v1:overhangio+scorm101+alpha+type@scorm+block@4
570ce0de76d40158507b19ea58f8d69/handler/studio_submit                                                                                                                                         
cms_1            | Traceback (most recent call last):                                                                                                                                         
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner                                                                    
cms_1            |     response = get_response(request)                                                                                                                                       
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response                                                                
cms_1            |     response = self.process_exception_by_middleware(e, request)                                                                                                            
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response                                                                
cms_1            |     response = wrapped_callback(request, *callback_args, **callback_kwargs)                                                                                                
cms_1            |   File "/opt/pyenv/versions/3.8.6/lib/python3.8/contextlib.py", line 75, in inner                                                                                          
cms_1            |     return func(*args, **kwds)                                                                                                                                             
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view                                                            
cms_1            |     return view_func(request, *args, **kwargs)                                                                                                                             
cms_1            |   File "./cms/djangoapps/contentstore/views/component.py", line 479, in component_handler                                                                                  
cms_1            |     resp = handler_descriptor.handle(handler, req, suffix)
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/xblock/mixins.py", line 85, in handle            
cms_1            |     return self.runtime.handle(self, handler_name, request, suffix)
cms_1            |   File "/openedx/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1453, in handle                
cms_1            |     return super(MetricsMixin, self).handle(block, handler_name, request, suffix=suffix)
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/xblock/runtime.py", line 1060, in handle               
cms_1            |     results = handler(request, suffix)     
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/openedxscorm/scormxblock.py", line 217, in studio_submit
cms_1            |     self.extract_package(package_file)
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/openedxscorm/scormxblock.py", line 275, in extract_package
cms_1            |     self.storage.save(               
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/files/storage.py", line 52, in save
cms_1            |     return self._save(name, content)
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 495, in _save
cms_1            |     self._save_content(obj, content, parameters=parameters)
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 510, in _save_content
cms_1            |     obj.upload_fileobj(content, ExtraArgs=put_parameters)
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/boto3/s3/inject.py", line 511, in object_upload_fileobj
cms_1            |     return self.meta.client.upload_fileobj(                           
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/boto3/s3/inject.py", line 431, in upload_fileobj
cms_1            |     return future.result()                            
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/s3transfer/futures.py", line 73, in result      
cms_1            |     return self._coordinator.result()            
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/s3transfer/futures.py", line 233, in result               
cms_1            |     raise self._exception                                
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/s3transfer/tasks.py", line 126, in __call__             
cms_1            |     return self._execute_main(kwargs)                          
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/s3transfer/tasks.py", line 150, in _execute_main
cms_1            |     return_value = self._main(**kwargs)                                                                             
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 692, in _main           
cms_1            |     client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/botocore/client.py", line 317, in _api_call
cms_1            |     return self._make_api_call(operation_name, kwargs)
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/botocore/client.py", line 588, in _make_api_call
cms_1            |     request_dict = self._convert_to_request_dict(
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/botocore/client.py", line 643, in _convert_to_request_dict
cms_1            |     request_dict = self._serializer.serialize_to_request(
cms_1            |   File "/openedx/venv/lib/python3.8/site-packages/botocore/validate.py", line 291, in serialize_to_request
cms_1            |     raise ParamValidationError(report=report.generate_report())
cms_1            | botocore.exceptions.ParamValidationError: Parameter validation failed:
cms_1            | Invalid type for parameter ContentType, value: b'text/javascript', type: <class 'bytes'>, valid types: <class 'str'>

This is not consistently reproduced. The error occurs in the unzipping step, on js assets.

Further debugging shows that when this error occurs, the mimetypes.guess_type function returns a bytes object, and not a str:

import mimetypes
# This returns b'text/javascript'
mimetypes.guess_type("somefile.js")[0]

This in turns causes s3boto3 request validation to fail.

The b'text/javascript' mimetype is introduced by the django-pipeline package: https://github.com/jazzband/django-pipeline/blob/1.7.0/pipeline/conf.py#L80

The bug was introduced in this PR: jazzband/django-pipeline#297
It turns out that other people face the exact same bug: jazzband/django-pipeline#297 (comment)
This bug was later fixed in v2.0.3: jazzband/django-pipeline#715

However, edx-platform depends on django-pipeline<2.0.0: https://github.com/edx/edx-platform/blob/open-release/koa.1/requirements/constraints.txt#L26
It looks like this constraint was introduced for compatibility withg python 3.5, but we don't need this anymore in Koa (python 3.8): https://github.com/edx/edx-platform/pull/24056

Thus, the fix would consist in upgrading django-pipeline to 2.0.3+ upstream.
@regisb regisb mentioned this issue Dec 30, 2020
Merged
regisb added a commit to overhangio/edx-platform that referenced this issue Dec 30, 2020
@regisb
fix: "Invalid type for parameter ContentType" error on js upload
1c733e3 
We are affected by this issue:
jazzband/django-pipeline#297 (comment)

In particular, this occurs when trying to upload js assets to s3, such
as with the Scorm xblock:
overhangio/openedx-scorm-xblock#16

This issue is resolved by upgrading django-pipeline to 2.0.3+, as the
fix was introduced here:
jazzband/django-pipeline#715
regisb added a commit to overhangio/tutor that referenced this issue Dec 30, 2020
@regisb
v11.0.5 (2020-12-30)
75af640 
- [Bugfix] Fix "Invalid type for parameter ContentType" error on js
upload in Scorm xblock ([openedx-scorm-xblock #16](overhangio/openedx-scorm-xblock#16))
regisb added a commit to regisb/edx-platform that referenced this issue Jan 29, 2021
@regisb
fix: "Invalid type for parameter ContentType" error on js upload
c656ce0 
We are affected by this issue:
jazzband/django-pipeline#297 (comment)

In particular, this occurs when trying to upload js assets to s3, such
as with the Scorm xblock:
overhangio/openedx-scorm-xblock#16

This issue is resolved by upgrading django-pipeline to 2.0.3+, as the
fix was introduced here:
jazzband/django-pipeline#715
jmbowman pushed a commit to openedx/edx-platform that referenced this issue Feb 1, 2021
@regisb
fix: "Invalid type for parameter ContentType" error on js upload (#25957
d89868b 
)

We are affected by this issue:
jazzband/django-pipeline#297 (comment)

In particular, this occurs when trying to upload js assets to s3, such
as with the Scorm xblock:
overhangio/openedx-scorm-xblock#16

This issue is resolved by upgrading django-pipeline to 2.0.3+, as the
fix was introduced here:
jazzband/django-pipeline#715
@OmarIthawi OmarIthawi mentioned this issue Aug 19, 2021
Merged
johanseto pushed a commit to eduNEXT/edx-platform that referenced this issue Jun 16, 2022
@regisb @johanseto
fix: "Invalid type for parameter ContentType" error on js upload (ope…
6150b03 
…nedx#25957)

We are affected by this issue:
jazzband/django-pipeline#297 (comment)

In particular, this occurs when trying to upload js assets to s3, such
as with the Scorm xblock:
overhangio/openedx-scorm-xblock#16

This issue is resolved by upgrading django-pipeline to 2.0.3+, as the
fix was introduced here:
jazzband/django-pipeline#715

(cherry picked from commit d89868b)
@johanseto johanseto mentioned this issue Jun 16, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@regisb