Poetry incorrectly resolves netmiko dependencies

[dmfigol]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Docker python:3.6-alpine
• Poetry version: 0.12.11
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/dmfigol/ec86438a2e46b25b53f196f471caffbc

Issue

It seems that poetry incorrectly resolves netmiko dependencies. Here is the output with poetry in Docker (Python 3.6.7):

/ # poetry add netmiko
Skipping virtualenv creation, as specified in config file.
Using version ^2.3 for netmiko

Updating dependencies
Resolving dependencies... (26.2s)


Package operations: 15 installs, 0 updates, 0 removals

Writing lock file

  - Installing pycparser (2.19)
  - Installing asn1crypto (0.24.0)
  - Installing cffi (1.11.5)
  - Installing bcrypt (3.1.6)
  - Installing cryptography (2.5)
  - Installing pyasn1 (0.4.5)
  - Installing pynacl (1.3.0)
  - Installing paramiko (2.4.2)
  - Installing enum34 (1.1.6)   # enum34 is installed when it is not supposed to
  - Installing ipaddress (1.0.22)
  - Installing pyserial (3.4)
  - Installing pyyaml (3.13)
  - Installing scp (0.13.0)
  - Installing textfsm (0.4.1)
  - Installing netmiko (2.3.0)

Here is the output from pip when installing netmiko (enum34 is not installed):

(.test-venv) / # pip install netmiko
Collecting netmiko
...
Successfully installed asn1crypto-0.24.0 bcrypt-3.1.6 cffi-1.11.5 cryptography-2.5 netmiko-2.3.0 paramiko-2.4.2 pyasn1-0.4.5 pycparser-2.19 pynacl-1.3.0 pyserial-3.4 pyyaml-3.13 scp-0.13.0 six-1.12.0 textfsm-0.4.1
You are using pip version 10.0.1, however version 19.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
(.test-venv) / # pip freeze
asn1crypto==0.24.0
bcrypt==3.1.6
cffi==1.11.5
cryptography==2.5
netmiko==2.3.0
paramiko==2.4.2
pyasn1==0.4.5
pycparser==2.19
PyNaCl==1.3.0
pyserial==3.4
PyYAML==3.13
scp==0.13.0
six==1.12.0
textfsm==0.4.1

[sdispater]

This is on netmiko's end. They declare the enum34 dependency like so:

if sys.version_info < (3,):
    requirements.append("enum34")
    requirements.append("ipaddress")

at this line: https://github.com/ktbyers/netmiko/blob/develop/setup.py#L12

This is not the recommended way of doing it. They should use environment markers.

Basically, when they package netmiko with the sdist command it creates a requires.txt file like the following:

paramiko>=2.4.1
scp>=0.10.0
pyyaml
pyserial
textfsm
enum34
ipaddress

[test]
pytest>=3.2.5

where the enum34 and ipaddress dependencies are not dependent on the Python version. They would be with environment markers.

[dmfigol]

Thanks for the analysis @sdispater
I thought that because pip does not install it, it is a problem on poetry end. But it seems that the wheel still contains enum34 in requires, but the reason it is not getting installed is because setup.py is actually run by setuptools when the pip install netmiko is run.

@ktbyers would you consider adding environment marker?
Thanks!

[ktbyers]

Yes, I am fine doing that.

[matt2000]

I keep running into this issue (which is the same issue as #881) in various third-party modules; I found it in 2 just today (confluent_kafka and requests-futures, FWIW). It seems that comparing sys.version_info is a very common pattern among package developers, and its a shame that poetry is mostly unusable until these other developers fix their setup scripts. Is there anyway Poetry could work around this?

[matt2000]

Alternatively, if there were a way to tell poetry to explicitly prevent the installation of some package, that would help, at least.

[ktbyers]

This was fixed in Netmiko:

ktbyers/netmiko#1099

We should close this issue.

[b-long]

Poetry is great, thanks for building it!

I ran into the enum34 issue as well. If I understand correctly, in my case it was due to my grpcio "=1.19.0" dependency (in pyproject.toml). Here you can see grpc's setup.py , in that version: https://github.com/grpc/grpc/blob/v1.19.0/setup.py#L304-L305

As a workaround, I declared a dependency on grpcio "=1.20.0.rc3 in my pyproject.toml , and after running poetry install, I see enum34 (1.1.6) and futures (3.1.1) removed from my poetry.lock file

There's some discussion of using environment markers, but I'm afraid I don't know enough to comment on this PR or provide other support to the Poetry / the grpc project: grpc/grpc#12976

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

Closing this issue automatically because it has not had any activity since it has been marked as stale. If you think it is still relevant and should be addressed, feel free to open a new one.

[zupo]

I have exactly the same problem, openapi-core 0.11.1 is using sys.version_info to declare requirements and I get enum34 installed in Python 3.7 environment. Which fails spectacularly.

I know that the openapi-core package author should fix this by using Environmental Markers. But there hasn't been a release in months.

Ideally, poetry would support temporarily fixing such broken packages by overriding their dependency declarations. Basically what was proposed in #1122 (comment) sounds absolutely epic.

Here's a workaround for anyone interested:

1. run poetry lock
2. manually remove all mentions of enum34 from poetry.lock
3. run poetry install
4. repeat every time you add a new package, or update an existing one

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.