Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

conda smithy rerender -c auto hangs for >1min in get_most_recent_version("conda-forge-pinning") function #1973

Open
1 task done
corneliusroemer opened this issue Jul 11, 2024 · 1 comment · May be fixed by #1972
Open
1 task done

conda smithy rerender -c auto hangs for >1min in get_most_recent_version("conda-forge-pinning") function #1973

corneliusroemer opened this issue Jul 11, 2024 · 1 comment · May be fixed by #1972
Labels
bug

Comments

@corneliusroemer
Copy link
Member

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

I locally ran conda smithy rerender -c auto and nothing happened for >1m.

I aborted after 1m4s, at which point the traceback showed that the program was stuck in the function get_most_recent_version("conda-forge-pinning")

Full traceback:

$ micromamba run -n smithy conda smithy rerender -c auto
^CTraceback (most recent call last):
  File "/Users/corneliusromer/micromamba/envs/smithy/bin/conda-smithy", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/cli.py", line 737, in main
    args.subcommand_func(args)
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/cli.py", line 584, in __call__
    self._call(args, tmpdir)
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/cli.py", line 589, in _call
    configure_feedstock.main(
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/configure_feedstock.py", line 2714, in main
    exclusive_config_file, cf_pinning_ver = get_cached_cfp_file_path(
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/configure_feedstock.py", line 2535, in get_cached_cfp_file_path
    get_cfp_file_path(smithy_cache)
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/configure_feedstock.py", line 2459, in get_cfp_file_path
    pkg = get_most_recent_version("conda-forge-pinning")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/configure_feedstock.py", line 2395, in get_most_recent_version
    request = requests.get(
              ^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/urllib3/connection.py", line 464, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/http/client.py", line 1428, in getresponse
    response.begin()
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/http/client.py", line 331, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/http/client.py", line 292, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/socket.py", line 708, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/ssl.py", line 1252, in recv_into
    return self.read(nbytes, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/ssl.py", line 1104, in read
    return self._sslobj.read(len, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

Installed packages

# packages in environment at /Users/corneliusromer/micromamba/envs/smithy:
#
# Name                    Version                   Build  Channel
anyio                     4.4.0              pyhd8ed1ab_0    conda-forge
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
attrs                     23.2.0             pyh71513ae_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.strenum         1.2.8              pyhd8ed1ab_0    conda-forge
backports.tarfile         1.0.0              pyhd8ed1ab_1    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
blinker                   1.8.2              pyhd8ed1ab_0    conda-forge
boltons                   24.0.0             pyhd8ed1ab_0    conda-forge
boolean.py                4.0                pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py312h9f69965_1    conda-forge
bzip2                     1.0.8                h93a5062_5    conda-forge
c-ares                    1.28.1               h93a5062_0    conda-forge
ca-certificates           2024.6.2             hf0a4a13_0    conda-forge
cctools                   986                  hf67d63f_0    conda-forge
cctools_osx-arm64         986                  h998149b_0    conda-forge
certifi                   2024.6.2           pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py312h8e38eb3_0    conda-forge
chardet                   5.2.0           py312h81bd7bf_1    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
cirun                     0.30               pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     24.5.0          py312h81bd7bf_0    conda-forge
conda-build               24.5.1          py312h81bd7bf_0    conda-forge
conda-forge-pinning       2024.07.01.19.34.18      hd8ed1ab_0    conda-forge
conda-index               0.5.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     24.1.0             pyhd8ed1ab_0    conda-forge
conda-package-handling    2.3.0              pyh7900ff3_0    conda-forge
conda-package-streaming   0.10.0             pyhd8ed1ab_0    conda-forge
conda-smithy              3.36.2          unix_pyh707e725_0    conda-forge
cryptography              42.0.8          py312had01cb0_0    conda-forge
deprecated                1.2.14             pyh1a96a4e_0    conda-forge
distlib                   0.3.8              pyhd8ed1ab_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
editables                 0.5                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
filelock                  3.15.4             pyhd8ed1ab_0    conda-forge
fmt                       10.2.1               h2ffa867_0    conda-forge
frozendict                2.4.4           py312h7e5086c_0    conda-forge
git                       2.45.2          pl5321h41514c7_1    conda-forge
gitdb                     4.0.11             pyhd8ed1ab_0    conda-forge
gitpython                 3.1.43             pyhd8ed1ab_0    conda-forge
gmp                       6.3.0                h7bae524_2    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hatch                     1.12.0             pyhd8ed1ab_0    conda-forge
hatchling                 1.25.0             pyhd8ed1ab_0    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  1.0.5              pyhd8ed1ab_0    conda-forge
httpx                     0.27.0             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
hyperlink                 21.0.0             pyhd3deb0d_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
idna                      3.7                pyhd8ed1ab_0    conda-forge
importlib-metadata        8.0.0              pyha770c72_0    conda-forge
importlib_metadata        8.0.0                hd8ed1ab_0    conda-forge
importlib_resources       6.4.0              pyhd8ed1ab_0    conda-forge
isodate                   0.6.1              pyhd8ed1ab_0    conda-forge
jaraco.classes            3.4.0              pyhd8ed1ab_1    conda-forge
jaraco.context            5.3.0              pyhd8ed1ab_1    conda-forge
jaraco.functools          4.0.0              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               3.0.0           py312h81bd7bf_0    conda-forge
jsonschema                4.22.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
keyring                   25.2.1             pyh534df25_0    conda-forge
krb5                      1.21.3               h237132a_0    conda-forge
ld64                      711                  h39a299f_0    conda-forge
ld64_osx-arm64            711                  h6a44367_0    conda-forge
libarchive                3.7.4                h83d404f_0    conda-forge
libcurl                   8.8.0                h7b6f9a7_1    conda-forge
libcxx                    17.0.6               he7857fb_2    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.6.2                hebf3989_0    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libiconv                  1.17                 h0d3ecfb_2    conda-forge
libintl                   0.22.5               h8fbad5d_2    conda-forge
liblief                   0.14.1               h3f3aa29_1    conda-forge
libllvm17                 17.0.6               h5090b49_2    conda-forge
libmamba                  1.5.8                h90c426b_0    conda-forge
libmambapy                1.5.8           py312h344e357_0    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libsodium                 1.0.18               h27ca646_1    conda-forge
libsolv                   0.7.29               h1efcc80_0    conda-forge
libsqlite                 3.46.0               hfb93653_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libxml2                   2.12.7               ha661575_1    conda-forge
libzlib                   1.3.1                hfb2fe0b_1    conda-forge
license-expression        30.1.1             pyhd8ed1ab_0    conda-forge
lz4-c                     1.9.4                hb7217d7_0    conda-forge
lzo                       2.10              h93a5062_1001    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5           py312he37b823_0    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
menuinst                  2.1.1           py312h81bd7bf_0    conda-forge
more-itertools            10.3.0             pyhd8ed1ab_0    conda-forge
msrest                    0.6.21             pyh44b312d_0    conda-forge
ncurses                   6.5                  hb89a1cb_0    conda-forge
oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
openssl                   3.3.1                hfb2fe0b_1    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
patch                     2.7.6             h27ca646_1002    conda-forge
pathspec                  0.12.1             pyhd8ed1ab_0    conda-forge
pcre2                     10.44                h297a79d_0    conda-forge
perl                      5.32.1          7_h4614cfb_perl5    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pkginfo                   1.11.1             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.2.2              pyhd8ed1ab_0    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_0    conda-forge
psutil                    6.0.0           py312h7e5086c_0    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
py-lief                   0.14.1          py312h2721eaf_1    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6           py312h02f2b3b_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pycryptodome              3.20.0          py312h332fddf_0    conda-forge
pygithub                  2.3.0              pyhd8ed1ab_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyjwt                     2.8.0              pyhd8ed1ab_1    conda-forge
pynacl                    1.5.0           py312h02f2b3b_3    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.12.4          h30c5eda_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-libarchive-c       5.1             py312h81bd7bf_0    conda-forge
python_abi                3.12                    4_cp312    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1           py312h02f2b3b_1    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
reproc                    14.2.4.post0         h93a5062_1    conda-forge
reproc-cpp                14.2.4.post0         h965bd2d_1    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
requests-oauthlib         2.0.0              pyhd8ed1ab_0    conda-forge
rich                      13.7.1             pyhd8ed1ab_0    conda-forge
ripgrep                   14.1.0               h5ef7bb8_0    conda-forge
rpds-py                   0.18.1          py312h552d48e_0    conda-forge
ruamel.yaml               0.18.6          py312he37b823_0    conda-forge
ruamel.yaml.clib          0.2.8           py312he37b823_0    conda-forge
scrypt                    0.8.24          py312h25fc620_0    conda-forge
setuptools                70.1.1             pyhd8ed1ab_0    conda-forge
shellingham               1.5.4              pyhd8ed1ab_0    conda-forge
sigtool                   0.1.3                h44b9a77_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smmap                     5.0.0              pyhd8ed1ab_0    conda-forge
sniffio                   1.3.1              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
tapi                      1100.0.11            he4954df_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomli-w                   1.0.0              pyhd8ed1ab_0    conda-forge
tomlkit                   0.12.5             pyha770c72_0    conda-forge
toolz                     0.12.1             pyhd8ed1ab_0    conda-forge
tqdm                      4.66.4             pyhd8ed1ab_0    conda-forge
trove-classifiers         2024.5.22          pyhd8ed1ab_0    conda-forge
truststore                0.8.0              pyhd8ed1ab_0    conda-forge
typer                     0.12.3             pyhd8ed1ab_0    conda-forge
typer-slim                0.12.3             pyhd8ed1ab_0    conda-forge
typer-slim-standard       0.12.3               hd8ed1ab_0    conda-forge
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
urllib3                   2.2.2              pyhd8ed1ab_1    conda-forge
userpath                  1.7.0              pyhd8ed1ab_0    conda-forge
uv                        0.2.18               hc069d6b_0    conda-forge
virtualenv                20.26.3            pyhd8ed1ab_0    conda-forge
vsts-python-api           0.1.25             pyhd8ed1ab_1    conda-forge
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
wrapt                     1.16.0          py312he37b823_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
yaml-cpp                  0.8.0                h13dd4ca_0    conda-forge
zipp                      3.19.2             pyhd8ed1ab_0    conda-forge
zstandard                 0.22.0          py312h721a963_1    conda-forge
zstd                      1.5.6                hb46c0d2_0    conda-forge

Environment info

active environment : base
    active env location : /Users/corneliusromer/micromamba/envs/smithy
            shell level : 3
       user config file : /Users/corneliusromer/.condarc
 populated config files : /Users/corneliusromer/.condarc
          conda version : 24.5.0
    conda-build version : 24.5.1
         python version : 3.12.4.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=m1
                          __conda=24.5.0=0
                          __osx=14.5=0
                          __unix=0=0
       base environment : /Users/corneliusromer/micromamba/envs/smithy  (writable)
      conda av data dir : /Users/corneliusromer/micromamba/envs/smithy/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/corneliusromer/micromamba/envs/smithy/pkgs
                          /Users/corneliusromer/.conda/pkgs
       envs directories : /Users/corneliusromer/micromamba/envs/smithy/envs
                          /Users/corneliusromer/.conda/envs
               platform : osx-arm64
             user-agent : conda/24.5.0 requests/2.32.3 CPython/3.12.4 Darwin/23.5.0 OSX/14.5 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8
                UID:GID : 501:20
             netrc file : None
           offline mode : False
@corneliusroemer corneliusroemer linked a pull request Jul 11, 2024 that will close this issue
Open
2 tasks
@jaimergp jaimergp linked a pull request Jul 16, 2024 that will close this issue
feat: add timeout to get_most_recent_version to prevent hang/deadlock #1972
Open
2 tasks
@jakirkham
Copy link
Member

Wonder if we could just save the commit used for the last download and use that to check whether it is up-to-date before downloading it again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: add timeout to get_most_recent_version to prevent hang/deadlock corneliusroemer/conda-smithy
2 participants
@jakirkham @corneliusroemer