Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADD] external-request-timeout: It could wait for a long time #370

Merged
merged 1 commit into from
May 24, 2022

Conversation

moylop260
Copy link
Collaborator

@moylop260 moylop260 commented May 20, 2022

Calling external request needs to use timeout in order to avoid waiting for a long time
It could be even worse if you have one or many records locked
then calling a requests it could accumulate a lot of locks and
the workers could be used for more time and the system could be down
waiting for release

You can even reproduce the case using deelay.me
e.g.

import requests
response = requests.get("https://deelay.me/5000/https://www.google.com")  # It will spend 5s

# Now using timeout
response = requests.get("https://deelay.me/5000/https://www.google.com", timeout=2)

# After 2s if the request doesn't have response it raises the following exception
# requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='deelay.me', port=443): Read timed out. (read timeout=2)

# But if it responses <=1s it is fine
response = requests.get("https://deelay.me/1000/https://www.google.com", timeout=2)

Now you can test the same but using a bigger delay

You can define your custom methods using the parameter: --external_request_timeout_methods

The default methods are:
- http.client.HTTPConnection
- http.client.HTTPSConnection
- odoo.addons.iap.models.iap.jsonrpc
- requests.delete
- requests.get
- requests.head
- requests.options
- requests.patch
- requests.post
- requests.put
- requests.request
- serial.Serial
- smtplib.SMTP
- suds.client.Client

@moylop260 moylop260 self-assigned this May 20, 2022
@moylop260 moylop260 force-pushed the oca-requests-timeout-moy branch 4 times, most recently from 0a18c9b to 15931d2 Compare May 24, 2022 17:44
@moylop260 moylop260 changed the title DRAFT: [ADD] request-without-timeout: It could wait for a long time [ADD] request-without-timeout: It could wait for a long time May 24, 2022
@moylop260 moylop260 force-pushed the oca-requests-timeout-moy branch 2 times, most recently from 11da8c4 to 509444e Compare May 24, 2022 18:07
@moylop260 moylop260 changed the title [ADD] request-without-timeout: It could wait for a long time [ADD] requests-without-timeout: It could wait for a long time May 24, 2022
@moylop260 moylop260 force-pushed the oca-requests-timeout-moy branch 9 times, most recently from a665b20 to 8141378 Compare May 24, 2022 20:55
@moylop260 moylop260 force-pushed the oca-requests-timeout-moy branch 3 times, most recently from fe67ce3 to 2face89 Compare May 24, 2022 23:02
Calling external request needs to use timeout in order to avoid waiting for a long time
It could be even worse if you have one or many records locked
then calling a requests it could accumulate a lot of locks and
the workers could be used for more time and the system could be down
waiting for release

You can define your custom methods using the parameter: --external_request_timeout_methods

The default methods are:
    - http.client.HTTPConnection
    - http.client.HTTPSConnection
    - odoo.addons.iap.models.iap.jsonrpc
    - requests.delete
    - requests.get
    - requests.head
    - requests.options
    - requests.patch
    - requests.post
    - requests.put
    - requests.request
    - serial.Serial
    - smtplib.SMTP
    - suds.client.Client
    - urllib.request.urlopen
@moylop260 moylop260 merged commit 4b6fcbe into OCA:master May 24, 2022
@moylop260 moylop260 deleted the oca-requests-timeout-moy branch May 24, 2022 23:41
moylop260 added a commit to Vauxoo/pylint-odoo that referenced this pull request May 24, 2022
* [FIX] missing-return: Incorrect check for tearDownClass (OCA#362)

* [FIX] Pinned lazy-object-proxy for Python 2.7 (OCA#365)

lazy-object-proxy is a library which astroids depends on. Astroid
is one of pylint's dependencies. Version 1.7.0 of lazy-object-proxy
fails to install on Python 2.7, therefore it was pinned to 1.6.0.

* [IMP] sql-injection: Detect possible sql injections when using f-strings (OCA#364)

* [FIX] CI: Fix pluggy error on py3.7 and running lint only in one build (OCA#366)

Travis has pre-installed a version of pluggy reproducing the following error when you try to install any pip package:

    Preparing metadata (setup.py) ... error
    error: subprocess-exited-with-error

    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [22 lines of output]
        Traceback (most recent call last):
            File "<string>", line 36, in <module>
            File "<pip-setuptools-caller>", line 14, in <module>
            File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/__init__.py", line 18, in <module>
            from setuptools.dist import Distribution
            File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/dist.py", line 34, in <module>
            from ._importlib import metadata
            File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/_importlib.py", line 28, in <module>
            disable_importlib_metadata_finder(metadata)
            File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/_importlib.py", line 12, in disable_importlib_metadata_finder
            import importlib_metadata
            File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 471, in <module>
            __version__ = version(__name__)
            File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 438, in version
            return distribution(package).version
            File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 411, in distribution
            return Distribution.from_name(package)
            File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 179, in from_name
            dists = resolver(name)
            File "/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 886, in find_distributions
            found = self._search_paths(context.name, context.path)
        AttributeError: 'str' object has no attribute 'name'

It is related to the following thread:
 - https://stackoverflow.com/a/59549323/3753497

It was already fixed running the following command:
 - `pip install -U --force-reinstall pluggy`

Another change in the CI was running lints only for one build instead of all them

* [REF] requirements: Bump pylint package from 2.11.1 to 2.13.5 for py3.x (OCA#367)

Changelog

Run the following command in pylint repo:
 - `git diff v2.11.1..v2.13.5`

* [ADD] pre-commit-hooks: Add pylint_odoo hook to be used from pre-commit.yaml directly (OCA#368)

It will allow to add in your pre-commit.yaml directly an entry for pylint_odoo instead of pylint

  .pre-commit-config.yaml

    - repo: https://github.com/OCA/pylint-odoo
      rev: 5.0.5
      hooks:
        - id: pylint_odoo

You can get the current `rev` in the current tags:
 - https://github.com/OCA/pylint-odoo/tags

* [REF] README: Use new travis URL (OCA#369)

* [FIX] .travis.yml: npm v18.0.0 is not compatible with ubuntu used from travis (OCA#371)

More info about:
 - https://travis-ci.community/t/the-command-npm-config-set-spin-false-failed-and-exited-with-1-during/12909/4

Summary it requires GLIBC_2.28 but it is not available in ubuntu used from travis

* [ADD] external-request-timeout: It could wait for a long time (OCA#370)

Calling external request needs to use a timeout in order to avoid waiting for a long time
It could be even worse if you have one or many records locked
then calling a request it could accumulate a lot of locks and
the workers could be used for more time and the system could be down
waiting for release

You can define your custom methods using the parameter: 
`--external_request_timeout_methods`

The default methods are:
    - http.client.HTTPConnection
    - http.client.HTTPSConnection
    - odoo.addons.iap.models.iap.jsonrpc
    - requests.delete
    - requests.get
    - requests.head
    - requests.options
    - requests.patch
    - requests.post
    - requests.put
    - requests.request
    - serial.Serial
    - smtplib.SMTP
    - suds.client.Client
    - urllib.request.urlopen

* [REF] README: Auto-update Using pylint_odoo.messages2rst()

Co-authored-by: Raf Ven <raf.ven@dynapps.be>
Co-authored-by: [Vauxoo] Antonio <101678296+antonag32@users.noreply.github.com>
@moylop260 moylop260 changed the title [ADD] requests-without-timeout: It could wait for a long time [ADD] external-request-timeout: It could wait for a long time May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants