Allow editable packages in requirements.in when using --generate-hashes

[jdufresne]

Describe the changes here.

Use case:

Editable packages are useful to install private company packages from
VCS. It is also useful to use --generate-hashes to help verify that 3rd
party packages are correct and not tampered with. Previously, combining
these two features would result in the error:

TypeError: Expected pinned requirement, not unpinned or editable, got ...

Can now use both editable packages and --generate-hashes features when
compiling a requirements.in file.

Changelog friendly entry: Allow editable packages in requirements.in with --generate-hashes

Contributor checklist

• Provided the tests for the changes
• Added the changes to CHANGELOG.md
• Requested (or received) a review from another contributor

[jdufresne]

Any idea why using editable packages fail on Windows with Python 2.7 with the error:

WindowsError(123, 'The filename, directory name, or volume label syntax is incorrect')

I do not have a Windows box handy to easily test.

[vphilippon]

I just tried with the head of master, and it seems to work. Try rebasing on master to see if it works now.

[jdufresne]

Rebased but and still receiving appveyor only issues. The error is different tho, I now see:

Exception ignored in: <finalize object at 0x11a3530; dead>
Traceback (most recent call last):
  File "C:\projects\pip-tools\.tox\py34-pip8\lib\weakref.py", line 519, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "C:\projects\pip-tools\.tox\py34-pip8\lib\tempfile.py", line 698, in _cleanup
    _shutil.rmtree(name)
  File "C:\projects\pip-tools\.tox\py34-pip8\lib\shutil.py", line 482, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\projects\pip-tools\.tox\py34-pip8\lib\shutil.py", line 372, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\projects\pip-tools\.tox\py34-pip8\lib\shutil.py", line 372, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\projects\pip-tools\.tox\py34-pip8\lib\shutil.py", line 372, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\projects\pip-tools\.tox\py34-pip8\lib\shutil.py", line 372, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\projects\pip-tools\.tox\py34-pip8\lib\shutil.py", line 377, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\projects\pip-tools\.tox\py34-pip8\lib\shutil.py", line 375, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\appveyor\\AppData\\Local\\Temp\\1\\tmp7ih6qhnesource\\django\\.git\\objects\\pack\\pack-3e97656eababafe662700fb821cea4092a0ee735.idx'

Looks like appveyor is failing to clean a temporary directory. I'm unable to investigate this as I don't have a Windows box to easily test with. Any thoughts?

[vphilippon]

I'll try to setup a Windows+Py3 environment and investigate this.
If I haven't replied in 2 days, ping me.

[vphilippon]

(Catching up) (like #424)
A bunch of thing has moved. If you're still interested and willing to, get this rebased on master.
You can also remove then changelog entry and just put a nice one-liner in the PR description, maintainers will take care of it in due time.

[jdufresne]

Updated PR.

• Rebased on latest master
• Dropped changelog entry
• Simplified commit message to one line

Thanks. Ready for review.

[vphilippon]

Dang, still a py3 error on appveyor. I'll try to reproduce.
Edit: I was able to reproduce, I'll investigate.

[vphilippon]

This is not only an appveyor issue: This really happens on windows with python 3.
Its simply ignored.
requirements.in:

-e git+https://github.com/django/django.git@1.11.1#egg=django

Then:

$ pip-compile
Exception ignored in: <finalize object at 0x41f1ba0; dead>
Traceback (most recent call last):
  File "c:\users\vphilippon\projects\pip-tools\venv\lib\weakref.py", line 548, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "c:\users\vphilippon\projects\pip-tools\venv\lib\tempfile.py", line 797, in _cleanup
    _shutil.rmtree(name)
  File "c:\users\vphilippon\projects\pip-tools\venv\lib\shutil.py", line 494, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\vphilippon\projects\pip-tools\venv\lib\shutil.py", line 384, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\vphilippon\projects\pip-tools\venv\lib\shutil.py", line 384, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\vphilippon\projects\pip-tools\venv\lib\shutil.py", line 384, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\vphilippon\projects\pip-tools\venv\lib\shutil.py", line 389, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "c:\users\vphilippon\projects\pip-tools\venv\lib\shutil.py", line 387, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'F:\\tmp\\tmpqgb_vo8dsource\\django\\.git\\objects\\pack\\pack-b0575a33cf48a7cf219e39160c194330f3d05391.idx'
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile --output-file requirements.txt requirements.in
#
-e git+https://github.com/django/django.git@1.11.1#egg=django
pytz==2017.3

$ echo $?
0  # The operation is still considered as a success, but the cache still contains the django directory.

[vphilippon]

This is not part of your change though. You can work around this (in the context of this PR) by using in checks instead of == on the output.

You can also remove test_editable as have some tests are already covering this (test_editable_package_vcs)

[jdufresne]

Thanks for the investigation! I really appreciate it. I've made the suggested updates:

• Removed test_editable
• Use in instead of ==

[vphilippon]

Thank you for your contribution, and for your time 👍