Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Packaging] Bump MSI embedded Python to 3.10.3 #21746

Merged
merged 2 commits into from
Mar 24, 2022
Merged

[Packaging] Bump MSI embedded Python to 3.10.3 #21746

merged 2 commits into from
Mar 24, 2022

Conversation

jiasli
Copy link
Member

@jiasli jiasli commented Mar 22, 2022

Description

Fix #21734

#7633 says:

Use regular python instead of embed version. This should unblock ML extension

This PR switched from using Python from https://www.python.org/ftp/python/3.6.5/python-3.6.5-embed-win32.zip to https://azurecliprod.blob.core.windows.net/util/Python366-32.zip. We don't exactly know what the original ML extension problem is and why the change was made.

But according to our internal knowledge base, Python366-32.zip from the Storage Account is created following

3.6.9
  1. Download a Windows embeddable zip file in https://www.python.org/downloads/windows/.
  2. Unzip the file, you'll see a python*.zip file, continue to unzip it and rename the directory to Lib.
  3. Delete the python*._pth file.
  4. Install pip with this embeddable python as instruction in https://pip.pypa.io/en/stable/installing/
  5. Delete all the __pychache__ folder under site-packages.

3.8.9
  1. Download a Windows embeddable zip file in https://www.python.org/downloads/windows/.
  2. Unzip the file. Keep the inner python*.zip file zipped after unzipping the outer zip file.
  3. Delete the python*._pth file.
  4. Install pip with this embeddable python as instruction in https://pip.pypa.io/en/stable/installing/
  5. Delete all the __pychache__ folder under site-packages.

Embedded python are then brought back by

and we didn't receive any issue for ML extension.

This PR somehow reverts #7633 and installs embedded Python from official URL.

@ghost ghost requested review from wangzelin007, yonzhan and kairu-ms March 22, 2022 11:30
@ghost ghost added the Auto-Assign Auto assign by bot label Mar 22, 2022
@ghost ghost assigned jiasli Mar 22, 2022
@ghost ghost added this to the Mar 2022 (2022-04-06) milestone Mar 22, 2022
@ghost ghost requested a review from jsntcy March 22, 2022 11:31
Comment on lines +4 to +5
REM Double colon :: should not be used in parentheses blocks, so we use REM.
REM See https://stackoverflow.com/a/12407934/2199657
Copy link
Member Author

@jiasli jiasli Mar 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using double colons in a block results in failure:

test.cmd:

@echo off
SetLocal EnableDelayedExpansion

for /f %%f in ('dir /b /s *.pyc') do (
     ::echo test
     ::echo test
)
>test.cmd
The system cannot find the drive specified.

mkdir %PYTHON_DIR%
pushd %PYTHON_DIR%
echo Downloading Python.
curl -o Python389-32.zip %PYTHON_DOWNLOAD_URL% -k
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-k, --insecure was introduced by #4523 allowing non-secure HTTPS, which we don't know why either.

Comment on lines +93 to +96
REM Delete _pth file so that Lib\site-packages is included in sys.path
REM https://github.com/pypa/pip/issues/4207#issuecomment-297396913
REM https://docs.python.org/3.10/using/windows.html#finding-modules
del python*._pth
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yonzhan
Copy link
Collaborator

yonzhan commented Mar 22, 2022

Packaging related bug

@jiasli
Copy link
Member Author

jiasli commented Mar 23, 2022

During my tests, even without this PR, I found edge build MSIs with the same version can be installed multiple times side by side for unknown reason:

image

From the UI there is no way to tell which is which.

For example, you may install these MSIs:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bump MSI embedded Python to 3.10.3 or 3.9.11 to fix CVE-2022-0778
3 participants