Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix editable installations using setup.py. #335

Closed

Conversation

matthewwardrop
Copy link
Contributor

@matthewwardrop matthewwardrop commented Apr 28, 2022

Hi! I have really come to love poetry, but cannot currently use it as is for my project that uses compiled modules. The issues I've encountered (and fixed in this PR) are detailed below. Hopefully these small patches are helpful to others also.

When you have configured poetry to generate a setup file, for example by using:

[tool.poetry.build]
script = "build.py"
generate-setup-file = true

and then attempt an editable install, poetry will include the module in the generated wheel. This shouldn't happen according to PEP 660 (you should only include the files which are necessary for the editable installation to work), and it gets in the way of the editable module being imported from the "editable" path. This is fixed in the commit labelled: "fix: Do not include library in wheel when performing an editable install".

Additionally, when your project uses extension modules, to be usable in an editable installation they need to be built in-place during the editable installation. This is added in the commit labelled: "enh: Build extension modules inplace during editable install." PEP 660 also mentions that this is an optional thing to do, but for modules with compiled dependencies, it is required to achieve an editable installation.

  • Added tests for changed code. (I've tested things locally, but will wait for your blessing before worrying about unit tests).
  • Updated documentation for changed code. (Based on comments in Stabilize 'build.py' poetry#2740, I don't think you are documenting this functionality yet).

@sonarcloud
Copy link

sonarcloud bot commented Apr 28, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@matthewwardrop
Copy link
Contributor Author

@dimbleby @branchvincent @radoering Sorry to ping directly, but wanted to see whether this is something that you would be interested in merging, or whether I should commit to maintaining my own fork for my repositories?

@neersighted
Copy link
Member

If I'm not mistaken, python-poetry/poetry#5590 should have fixed this in a more proper way. Could you please test against the latest master and report back?

@matthewwardrop
Copy link
Contributor Author

@neersighted Nope. Unfortunately the master branch of poetry has the same pathologies that I tried to fix here. It doesn't compile the modules in the local directory and it installs all the package files into site-packages. :(.

@abn
Copy link
Member

abn commented May 13, 2022

I'm a bit unclear on what is being solved here. Poetry's editable builder has nothing to do with poetry-core's wheel builder. No wheel is built during that process. A temporary setup file is created and pip install -e . is called when setup file generation is enabled.

Can you provide an example project with the issue you are detailing? That could help.

Also do note that PEP 662 is rejected.

@matthewwardrop
Copy link
Contributor Author

Hi @abn,

Thanks for your reply, and yes... apologies. I should have referred to PEP 660 (will update in the PR description).

I've put together a minimum working example here.

Let me know if you need more context/etc.

@matthewwardrop
Copy link
Contributor Author

matthewwardrop commented Jul 13, 2022

@abn Any additional thoughts given the example code? I'd still love to adopt poetry, but need this editable behaviour.

@robbotorigami
Copy link
Contributor

Is it possible to get a response on this thread? Unfortunately, especially with the changes made in python-poetry/poetry#7975, this is now a significant issue for doing development in any project that uses compiled extension modules. There's no way I've found to get an editable install to work properly, either with poetry install or pip install -e ..

@radoering
Copy link
Member

I'll take a closer look if the conflicts are resolved and a unit test is added.

@robbotorigami
Copy link
Contributor

@matthewwardrop If you'd like to work on this let me know, otherwise I'm happy to make a fork and work on @radoering 's requests.

@matthewwardrop
Copy link
Contributor Author

@matthewwardrop If you'd like to work on this let me know, otherwise I'm happy to make a fork and work on @radoering 's requests.

@robbotorigami you can go ahead (as I see you already have!). Most of my projects have moved on from poetry in any case due to this and other reasons. Thanks for doing the work to get this in!

I'll close this one out in favour of your PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants