Failed to install packages that target a range of python versions if local python version is not fixed #1413
Comments
|
I had the same problem trying to install |
|
I have the same when I want to add PySide2. $ poetry add pyside2
[SolverProblemError]
The current project's Python requirement (^3.7) is not compatible with some of the required packages Python requirement:
- pyside2 requires Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <3.9
Because no versions of pyside2 match >5.14.0,<6.0.0
and pyside2 (5.14.0) requires Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <3.9, pyside2 is forbidden.
So, because tester depends on pyside2 (^5.14.0), version solving failed.When I use the 'fix' from @MarcBoissonneault (changing to |
|
I have the same with |
|
I am having this same problem with dataclasses in Python 3.6: |
|
I'm pretty sure the bug is here: poetry/poetry/puzzle/provider.py Lines 500 to 507 in 7d5934e I'm testing using this test, but not quite sure what to change in order to fix it. poetry/tests/mixology/version_solver/test_python_constraint.py Lines 5 to 19 in 7d5934e FWIW I'm testing with |
|
Same problem when trying to add the Poetry version: 1.0.3 |
|
I looked at this a bit deeper and I think that the It seems that it’s evaluating properly, but it also seems that the expected behavior from the end-user differs from how the solver deals with packages that have upper bounds which conflict with project python constraints. The workaround discovered here is to pin the python version, but it would also work to set an upper bound on the python version which matches that of the package. I’m not sure what the best solution here is, but if this special case happens, it might be helpful to log a suggestion to set a strict upper bound on the current project to match that of the package if you want to keep the solver behavior the way it is. Otherwise, I think the solution is to code this particular edge case into the solver. However, this somewhat invalidates the project python restrictions as the actual restriction then becomes the intersection of the project and its dependencies, but is no longer manifested explicitly in the |
|
I just stumbled upon this myself when using a dependency that depends on the backport of Installing version |
|
Just experienced the same strange behavior. Seems like when you use tilde requirement for Python: IMO, the caret requirements |
|
Also receive similar error with |
|
Hello, the "issue" here is one of the most misunderstood of poetry's version resolving. So, if one define that the project is compatible with python Let's have a look at one of the examples above:
Or this one: The python requirement for the project has no upper boundary but A further requirement is, that there is one major version of a package that is compatible with all python version. If this is not the case, the cli is not able to resolve this and we have to help by defining the version in the fin swimmer |
|
@finswimmer I will argue that, since this many people got confused, this is actually a bug that needs to be resolved somehow. We'd at least appreciate the proper way to do this, as currently no solution is posted. Maybe a solution could be included in the docs? |
|
@skorokithakis This won't be treated as a bug since it's not one but the intended behavior. That being said I agree that, since it's confusing for some people, we should give guidance to resolve the issue. That's something that is in progress and will be available in the next feature release (
|
|
That looks great, offering a solution in the output is a great idea. I would add something to make clear that all versions need to be satisfied, otherwise you will get text like:
Which is confusing to a user, as above. The problem is in "must support the following Python versions", which users interpret as "any of", but is actually "all of". Maybe changing it to "The current project must support all of the following Python versions" would be acceptable. |
|
I also ran into this problem just now. I hear that this is intentional, but I'm not sure why.
I think this is the essence. Why do all python versions given in the Or, to make it more specific, look at this example:
I can see that installing taskipy would limit the python versions that the project works with to < 4.0. And apparently the solution here, would be to modify the project's python requirement to be >=3.7,<4.0, making it a subset of the taskipy allowed python versions (did I get that right?). But why does one need to do this manually? Isn't this something that poetry could sort out? If every library, and the project, declares what Python versions it works with, then poetry can figure out what the resulting supported Python range is? If this results in an empty set of Python versions, that's of course problematic, but that does not seem to be the case at hand. I can see that there is some documentation value in a project's If this is is indeed the rationale for this requirement, I guess that the proper solution could be to have two python version requirements for the project: One that shows the requirements of the project itself, and one that shows the requirements of the project and all its requirements combined (where the former is manually configured and the latter is automatically determined by poetry whenever the lock file changes). Another thing I wondered: Is this same policy applied to normal dependencies, or just to the python version? In other words, if a project depends on package |
|
Hello @matthijskooijman,
at first place Once you've answer this question and write it down to your
Might be possible, but that's not how should manage your python project. As I said, you first have to decide what python versions you want to support, because this is essential for the python syntax you will use. Having valid dependencies is only the second step. Not the other way round.
No, this will not produce an error. Because the version constraints for the package tell poetry from which versions it can choose. Which one it will choose depends on the other packages in the project and there dependencies. fin swimmer |
|
That makes sense to me, it should probably just be a bit better documented because many users (including me) assume the opposite. |
|
I @finswimmer, thanks for the explanation. You have given a few examples for why we should decide/pin the Python version first, and then install dependencies. However, can't you just use these as arguments against version deciding/pinning, too?
That's exactly my thought when starting out: What versions do I/does this library support? Most likely it is
Syntax features are available in a wide range of Python versions. If a user wants to use a specific syntax, say
I am not saying I should not care, but why exactly do I have to know this? Say in a scenario where I want to use Django 3, shouldn't Poetry figure out that (Sidenote: the fact that poetry wants me to switch Python versions using I absolutely get that the Python version should be pinned during deployment, but that's what The mental model I have: Python is just another dependency, just like any package from PyPI. In my mental model I should not have to treat it specially. If I and all my dependencies define their compatible version range (and I don't have a conflict somewhere), Poetry should be able to find a valid Python version that suits all. It should be flexible in |
|
@nils-werner I know it might be confusing but there is actually a good and logical reason behind the way Poetry behaves in those cases. As an example, I will take the
At this point, you have two choices, either change the taskipy = {version = "^1.2.1", python = ">=3.7,<4.0"}It will now work properly. This will be better explained when the
On another note
No, the lock file is system and Python version agnostic. |
Exactly.
That does not really seem right either: Aren't you now specifying that for Python >= 4.0, |
@matthijskooijman Exactly. The only thing we know is that the two projects aren't totally compatible. So we need to either restrict the possible python versions for our project or restrict the dependency usage. I agree this isn't satisfying, but I'd say that with @sdispater's solution we wait for the dependency to be compatible with our vision of our project 💪. We don't bend our vision to match those of our dependencies (though we have to keep in mind what you were saying about missing dependencies 😢). |
But we do bend our vision, because we specify that I would greatly prefer to just state the dependencies as they are in reality, so It might still be useful if this information is also shared with the current package maintainer, but I do not think it should prevent adding the dependencie. i.e. I can imagine that running |
Apparently, Poetry 1.0 does not handle dependencies on specific versions of Python correctly: python-poetry/poetry#1413 While waiting for Poetry 1.1, this commit tries to fix the problem.
|
Please, you must change the python property as shown below: [tool.poetry.dependencies] to [tool.poetry.dependencies] In my case, I had an error with pyside6 with python 3.8. |
@sdispater Why can't poetry add this in the |
|
we fixed the problem in test-automation toml repo with "^3.6" > "3.6.4" (we use a fixed version)... In this case, better to define python version "^3.6.0" than "^3.6" ///// I guess, issue is in comparing 3.x string format with 3.x.x |
|
Was having this problem with scipy 1.7.2 as well. "so it will not be satisfied for Python >=3.11,<4.0". I had ^3.9 in [tool.poetry.dependencies]. Changed that to >=3.9,<3.11 and it worked. Looks like there's definitely some kind of bug here. |
|
@finswimmer why is Poetry trying to satisfy all constraints rather than some constraint? For example, if one component requires |
|
Hello @skorokithakis, poetry is a tool for developing python packages and a such a tool it makes sure, that you don't define invalid dependency definitions. poetry is not a tool to install packages into a given environment like fin swimmer |
|
@finswimmer ah okay, that makes sense, thanks! If the docs mentioned that I think it would clarify a lot for users. |
@finswimmer Can I ask for a clarification, please? So you would not recommend that poetry is being used for, say a django website project that is getting deployed in kubernetes via containers (so it's less of a package, but more an app as pipenv used to distinguish this in the past)? I am confused, because e.g. docs say:
and that sounds a lot like using it to install stuff into a given environment. Would you use something else for this purpose then (like pip-tools or what)? Thanks for clarification. |
|
Hey @hnykda, sorry if this was not clear enough. So here is another try :)
The important thing I wanted to point out here, was "like pip".
To reach this goal, Poetry needs to make sure that your project definition is valid. And that's the main task. Poetry thinks in "projects" and not in "current environment". And this is why it complains, if the current environments fits to the needs of a dependency, but the project definitions doesn't fit. Is it somehow clearer now? fin swimmer |
I make app containers with poetry, specifically to use the lock file, such that what i'm developing and testing with is what i deploy with. I'm relatively happy with my setup now. It does mean having poetry in the containers to install them (I had no luck with micropipenv back when, might be better now. I install into a virtual environment in the container, which is a bit matryoshka, but I got over that as well. Maybe at some point in the future there will be a plugin that can turn a pyproject.toml and a poetry.lock into an app-locked.whl and i'll use that, but for now i'm happy. |
|
@clintonroy all this affects is the way dependencies are resolved (all solutions need to be solvable, rather than just one solution). If you specify your Python version, I think you'll mostly be fine, and the lock file locks things very exactly anyway. |
|
@finswimmer Thanks, I wanted to make sure I am not entirely off by using poetry in a similar way to @clintonroy 😌 |
|
changing 'python = "^3.8"' to 'python = "~3.8"' allowed to get around the erroneous dependency constraint resulting in an error (as per above) |
This is really counter-intuitive, a bad API design example tbh |
|
Why does Poetry not try to find the latest release of a package that DOES fulfill the constraints? This is so frustrating. I have a brand new project that I have defined as such: [tool.poetry]
name = "newproject"
version = "1.0.0"
description = "A project"
authors = ["Your Name <you@example.com>"]
[tool.poetry.dependencies]
python = "^3.7"
Super basic. Nothing special. Created it with init, setup the env with Then I run this: At what point did I say that newproject depends on pyproj? I am adding it right now. It chose that version, and then complains that the version isn't compatible. If we look at the pypi for pyproj, it's clear to see that while 3.3.0 is Python 3.8 or above, 3.2.1 works with Python 3.7. So why won't poetry use it instead when I have not asked for a specific version of it? If the argument is that Poetry is not a package installer but a dependency manager, it should absolutely be able to find the correct dependency given the constraints. |
|
Could I ask a very simple question as someone who came from C# .NET and can not understand what is going on here? Why by any means author of a library says something like: "Hey my lib is compatible with Python starting from 3.7 but only up to 3.10, so sorry but 3.11 is not supported"? What is the most important, as an author of my own code I personally know that my target language version of python is Python 3.10 and I expect that any client code that is targeting 3.10 or newer could use my lib. Why I can not say that I'm writing a library written in a Python 3.10 version and that's all the contract for anyone who is using my code. It basically a very straightforward definition. I'm not supporting any clients who are writing their code via an earlier version of Python. Sorry but yeah, it is my decision as an author. However, everyone who is using 3.10, 3.11, 3.12 or any other future versions of Python in their code, could use my library with no issue. I expect that Python 3 is backward compatible, yep? So why I should be punished when a dependency package tells me that it is compatible with Python 3.7, but, by some unimaginable for me reason, it is not compatible with Python 3.11? |
Since external teams are using the checker tool in this package, it makes sense to use the most "standard" packaging tools and not require them to learn poetry. Also specify a lower bound for the dataclasses backport to work around the issue linked below. The package should be installable by pip and poetry in environments with both python 3.6 and 3.7. python-poetry/poetry#1413 Approved-by: Tomas Svarovsky Approved-by: Pavlo Bashmakov
This is also my question, which is how I ended up here...
|
|
After having been stuck in this rabbit hole for several hours, I would like to add my thoughts to this very old thread. I think the rationale behind the current behaviour is very inconsistent. Basically, poetry is first lying on our behalf, and a few moments later, it is calling us out on that lie. What is the lie? But we can't know that for certain. Future minor versions of Python 3 may break our own code. Or the packages we depend on may not be upgraded to support those Python versions. Is the lie acceptable? However, for many of us, the lie is fully acceptable. We understand that the creator of an imaginary and now abandoned my_hello_world package can't know with 100% certainty if Python 3.47 is going to deprecate the We also understand that other packages, which our package depends on, are not yet released for Python minor versions, which do not exist yet. There is no pandas package for Python 3.13, because Python 3.13 has not yet been released. But if we include pandas as a dependency in our project, we can be quite confident that there will be a pandas package for Python 3.13 if someone downloads our package to his python 3.13 environment a few years from now. And if there isn't a pandas package, his package manager should tell him to downgrade his python version anyway. Does poetry think that the lie is acceptable? Consequently, when I try to include pandas as a dependency in my package, my screen will be filled with red text stating: "Hey! What are you doing? You have said that your package is compatible with all future minor versions of Python 3, and now you try to install pandas, which have no packages for Python 3.13." Well, duh! Python 3.13 isn't released yet (as least not in a stable version), so of course there is no pandas package for Python 3.13. Who made the lie? When I use poetry for creating a new environment with the command That is not really consistent with poetry's hardcore approach to dependencies, is it? Why not let the default entry in pyproject.toml be this: That would be in line with poetry's hardcore approach to dependency solving, and then it would be up to the developer to slacken the dependencies if he has reason for doing that. TL;DR
With the current behaviour, I first have to create the environment with poetry, then manually edit the pyproject.toml file to make the change from ^3.11 to ~3.11, and then I can start adding packages. |
|
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
and others
-vvvoption).pyproject.toml
Issue
When I try to install a dependency that target a range of python versions, that seems to contains the python version I target in my
pyproject.tomlit fails with the following error.As you can see from the error and from the kedro pypi release page, the target python version is
Python >=3.5, <3.8.I specify python
^3.7in mypyprojet.tomland as you can see when I runpoetry debug:infomy python version is* Python: 3.7.0.Other info, if I change the
pyproject.toml:To:
It's working
The text was updated successfully, but these errors were encountered: