Use the OEM encoding for bat files

[pfmoore]

No description provided.

[pfmoore]

Note: if the path to the virtualenv contains characters that can't be encoded in the OEM characterset, this PR will cause the creation of the virtualenv to fail, because of an encoding error in the creation of activate.bat. This needs to be fixed - but in this case activate.bat simply can't work, as bat files must be encoded in the OEM code page.

I suspect the best we can do in this case would be to write the bat files with the "replace" error handler, and issue a warning that the user will have to manually handle the PATH, as the activate scripts won't work.

[yan12125]

There's a more generic approach in my PR #900. I use sys.getfilesystemencoding(), while I'm not sure whether its value is always the same as sys.stdout.encoding or not.

[pfmoore]

Thanks for the pointer, I hadn't spotted your PR. I've added comments there, but no, you can't use sys.getfilesystemencoding() for batch files on Windows.

[Ivoz]

@pfmoore do you know if the same logic should apply to .ps1 files, or are they ok as UTF-8?

[pfmoore]

.ps1 files can be UTF-8, but they need a BOM.

[Ivoz]

Since all 3 references to content in the function body are to call it with encode(encoding), why not write binary_content = content.encode(encoding) at the top of the function and then reference binary_content in those 3 places. Just bit of basic getting-rid-of-repetition.

[techtonik]

if the path to the path to the virtualenv contains characters that can't be encoded in the OEM characterset

If those characters are invalid, then virtualenv dir won't be created before any files are created there.

virtualenv/virtualenv.py

Line 960 in 263283b

if is_win:

If directory was created, then the file path is right and should be written as-is. If it is not in bytes, then I guess any encoding can damage it if it remaps bytes. Does cp437 remap bytes?

But it may be that #950 is more important and people simple don't create virtualenvs with non-ascii names on Windows.

Use any character in the current code page for a name, including Unicode characters and characters in the extended character set (128–255), except for the following:
The following reserved characters:
< (less than)
> (greater than)
: (colon)
" (double quote)
/ (forward slash)
\ (backslash)
| (vertical bar or pipe)
? (question mark)
* (asterisk)
* Integer value zero, sometimes referred to as the ASCII NUL character.
* Characters whose integer representations are in the range from 1 through 31, except for alternate data streams where these characters are allowed.

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#file_and_directory_names

[yan12125]

@techtonik For #950, my pull request #900 may work. It's messy and imperfect, but it should work in simple cases.

#900 contains lots of fixes, and #943 is part of it. It would be great if #943 is merged first. My pull request can be cleaner.

[techtonik]

It's messy and imperfect, but it should work in simple cases.

If that's the reason it won't be merged, then it doesn't help. =)

[yan12125]

I guess it's because my PR should be separated into multiple parts.

[yan12125]

By the way, this PR looks quite fine. Just replace sys.stdout.encoding with ctypes.windll.kernel32.GetOEMCP(). The former exists only if stdout is a tty.

[gaborbernat]

This will require a rebase on master, and some test to validate the changes. Thanks!

[gaborbernat]

Closing due to inactivity.