Fixes to build and install 4.1.1 on Windows with Python 3.10 (64 bit) and Visual Studio 2019

[psam44]

Note: this procedure was initiated because of #2128.

context:

>>> sys.version
'3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64 bit (AMD64)]'

>"C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe" -products * -legacy
Visual Studio Locator version 2.7.1+180c706d56 [query version 2.7.3111.17308]
Copyright (C) Microsoft Corporation. All rights reserved.
	instanceId: b0bd1622
	installDate: 15/09/2021 22:28:16
	installationName: VisualStudio/16.11.3+31702.278
	installationPath: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
	installationVersion: 16.11.31702.278

1. Help the Python tooling to find the compiler

First, it is highly recommended to set up a Python virtual environment, as some changes will be done in the setuptools package.
For example:

• create a folder for virtual environments, let's say C:\venvs\
• create a virtual environment for our task, let's choose the name 'py310wx':
  >py -m venv --system-site-packages \venvs\py310wx
• activate the new environment:
  >\venvs\py310wx\Scripts\activate

A first basic attempt of "pip.exe install wxPython" fails with:
distutils.errors.DistutilsPlatformError: Microsoft Visual C++ 14.2 or greater is required.

The problem is probably related to the fact denoted by this comment in setuptools\msvc.py:
# Except for VS15+, VC version is aligned with VS version

A way (may be dirty) to let the finders locate the compiler is to not impose a required version, so an autodetect will be done.
in setuptools\msvc.py, inside msvc9_query_vcvarsall(), change this line:
return EnvironmentInfo(arch, ver).return_env()
by this one:
return EnvironmentInfo(arch).return_env()

The problem on a next attempt is the search for redistributable dlls:

    File "C:\venvs\py310wx\lib\site-packages\setuptools\msvc.py", line 1774, in return_env
      if self.vs_ver >= 14 and isfile(self.VCRuntimeRedist):
    File "C:\Program Files\Python310\lib\genericpath.py", line 30, in isfile
      st = os.stat(path)
  TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

My installation is like this:

>dir "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Redist\MSVC"
15/09/2021  22:39    <DIR>          14.29.30133
15/09/2021  22:39    <DIR>          v142

Unfortunately, the second folder may be detected instead of the correct first one, which leads self.VCRuntimeRedist to be None.

in setuptools\msvc.py, inside VCRuntimeRedist(), change this line:
redist_path = join(redist_path, listdir(redist_path)[-1])
by this one:
redist_path = join(redist_path, [x for x in listdir(redist_path) if x[0].isdigit()][-1])

2. Get a local copy of the sources and do some changes.

Next trial gives:

   Traceback (most recent call last):
      File "C:\Users\Patrick\AppData\Local\Temp\pip-install-ykfhf964\wxpython_f72145775988443db9724f61fe154879\build.py", line 676, in getMSWebView2
        import requests

OK, known but not well documented, we need to install some packages for the building work.
Download the source tarball (or find it in PIP cache, something like C:\Users\Patrick\AppData\Local\pip\cache\http\0\f\4\a\f\0f4af8d32a9cc03262951107296474923f79fa06483b3d3ae695a257)
Extract it somewhere, for example in C:\tmp\wxPython-4.1.1

Extra packages for the build are mentioned in requirements\devel.txt
As it is, with cython==0.29.21, the build will later failed with this error:

    building 'wx.svg._nanosvg' extension
    wx/svg\_nanosvg.c(20946): warning C4013: '_PyGen_Send' non défini(e) ; extern retournant int pris par défaut

The fix was introduced in version 0.29.22 (source cython/cython#3876)
Note: This problem has already been fixed by this pull request: #2017, with a 0.29.24 version.

Choose to apply either option A or option B in this sequence:

• Option A (before): update the requirement file to have at least cython==0.29.22
• install the requirements:
  (py310wx) C:\tmp>pip.exe install --no-compile -r \tmp\wxPython-4.1.1\requirements\devel.txt
• Option B (after): upgrade the Cython installation with a newer version:
  (py310wx) C:\tmp>pip.exe install --no-compile cython==0.29.22

3. Help the WAF tool to find the compiler

We need to set the expected code page, according to the instruction at https://github.com/wxWidgets/Phoenix/
(py310wx) C:\tmp>chcp 1252

Next failure is related to:
msvc: Impossible to find a valid architecture for building ['msvc 14.0'] - ['msvc 16.11']

So, if VS is indeed a 16.11 version, its compiler should be a 14.29 version according to the presence of:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133

The problem with WAF is that the tool unpacks its embedded library dynamically when run, so we can not change the code.
Another way to influence its behaviour is to use an environment variable (seen in wxPython-4.1.1\build.py):
(py310wx) C:\tmp>set WXPYTHON_BUILD_ARGS=--dump_waf_log --extra_waf="--msvc_version=\"msvc 16.11\""

--extra_waf is the useful option, --dump_waf_log is not required but set as we try to get a maximum of visibility.
Notice the quotes escaping, to allow a correct unpacking in each successive steps.

Next failure:

    File "C:\Users\Patrick\AppData\Local\Temp\pip-req-build-x2rxidlm\buildtools\config.py", line 174, in finishSetup
      if int(getVisCVersion()) > 100:
  ValueError: invalid literal for int() with base 10: 'FIXME'

The problem is how the CL.exe command outputs its identification:

(py310wx) C:\tmp\wxPython-4.1.1>py -c "from buildtools.config import getVisCVersion; print(getVisCVersion())"
Compilateur d'optimisation Microsoft (R) C/C++ version 19.29.30133 pour x64

Issue 1: it says 'version', not 'Version'
Issue 2: the space character after 'version' is not the standard one (Unicode U+0020) but a non-breaking space (Unicode U+00A0)

in \tmp\wxPython-4.1.1\buildtools\config.py, inside getVisCVersion(), change this line:
if 'Version 19' in text:
by this one:
if 'Version 19' in text or 'version\u00A019' in text:

Next failure:

    [ 27/958] Linking build\waf\3.1\x64\release\siplib.cp310-win_amd64.pyd
    LINK : fatal error LNK1181: impossible d'ouvrir le fichier en entrée 'wxbase31u.lib'

The problem is exactly the same as the previous one, in another file.
in \tmp\wxPython-4.1.1\buildtools\build_wxwidgets.py, inside getVisCVersion(), change this line:
if 'Version 19' in text:
by this one:
if 'Version 19' in text or 'version\u00A019' in text:

4. Apply a fix, even if it doesn't prevent the build, to avoid a crash with wx.Region(points)
   This is related to v4.1.1: Crash in wx.Region(points) constructor on MacOS #1878 (impact on Windows as well)
   Disable the lines marked here as comments (putting them in comment or deleting them):
   wxPython-4.1.1\sip\cpp\sip_corewxRegion.cpp

        // PyErr_Clear();
        // Py_BEGIN_ALLOW_THREADS
        size_t count;
        wxPoint* array = wxPoint_array_helper(points, &count);
        if ( array != NULL ) {
            sipCpp = new wxRegion(count, array, fillStyle);
            delete [] array;
        }
        if (PyErr_Occurred()) sipIsErr = 1;
        // Py_END_ALLOW_THREADS
        // if (PyErr_Occurred()) sipIsErr = 1;

wxPython-4.1.1\sip\gen\region.sip (not sure if it's really necessary)

    %MethodCode
        // PyErr_Clear();
        // Py_BEGIN_ALLOW_THREADS
        size_t count;
        wxPoint* array = wxPoint_array_helper(points, &count);
        if ( array != NULL ) {
            sipCpp = new wxRegion(count, array, fillStyle);
            delete [] array;
        }
        if (PyErr_Occurred()) sipIsErr = 1;
        // Py_END_ALLOW_THREADS
        // if (PyErr_Occurred()) sipIsErr = 1;
    %End

5. Run finally a successful installation

(py310wx) C:\tmp>pip.exe install --no-compile C:\tmp\wxPython-4.1.1
Processing c:\tmp\wxpython-4.1.1
Requirement already satisfied: pillow in c:\venvs\py310wx\lib\site-packages (from wxPython==4.1.1) (9.1.1)
Requirement already satisfied: six in c:\venvs\py310wx\lib\site-packages (from wxPython==4.1.1) (1.16.0)
Requirement already satisfied: numpy in c:\venvs\py310wx\lib\site-packages (from wxPython==4.1.1) (1.23.0)
Building wheels for collected packages: wxPython
  Building wheel for wxPython (setup.py) ... done
  Created wheel for wxPython: filename=wxPython-4.1.1-cp310-cp310-win_amd64.whl size=15784627 sha256=405e4553c28bc65a353bbb2e24c5bfd8882ebaf15b1e625a831e8b779ff26d45
  Stored in directory: c:\users\patrick\appdata\local\pip\cache\wheels\df\4f\0b\8b9e2f72f99a11e897f221884bb5b936f55244920647a186bd
Successfully built wxPython
Installing collected packages: wxPython
Successfully installed wxPython-4.1.1