cibuildwheel: Add macOS arm64 support

[xavier2k6]

Closes #7308.
Closes #7536.
Closes #7650.
Closes #7683.
Closes #7729.

[xavier2k6]

Note/Review: pypa/cibuildwheel#1926 (comment)

[xavier2k6]

@qstokkink Can you give the created wheels a test?

[xavier2k6]

Upcoming changes:

• Support for musllinux_1_1 images will be dropped on 2024-11-01
• feat: use SPEC 0 schedule for cibuildwheel

[qstokkink]

@qstokkink Can you give the created wheels a test?

Works on my machine. 👍

On behalf of the Tribler team, I would like to request you to keep {"os": "macos-12", "CIBW_BUILD": "cp39-*", "CIBW_ARCHS": "x86_64"} in the workflow dispatch matrix though.

[xavier2k6]

@qstokkink

1. I've added macOS-12/x86_x64 architecture with Python 3.9 to pull_request matrix for testing purposes (this can probably be removed again, once you've tested)
2. Have included macOS-12/x86_x64 architecture with Python * to workflow dispatch unless of course You/Tribler team only want Python 3.9/x86_x64 on macOS-12??

[qstokkink]

Thanks 👍 and, "correct" to 2: I copied the wrong string.

[xavier2k6]

@qstokkink Just so the Tribler team are aware macOS-12 images are considered Deprecated & will be Removed by end of year!
See: actions/runner-images#9255 or screenshot below.

---

[qstokkink]

🤔 that's sad. It would be nice to get a PyPI release before that gets killed though.

[qstokkink]

@xavier2k6 considering the other closed issues, it looks like you can add #7729 to your "Closes" list.

[xavier2k6]

manylinux2014 image may be dropped in a upcoming PR too by "cibuildwheel" repo., so may need to change to manylinux_2_28 but this may also drop older python versions but that is already on the cards by "cibuildwheel" repo.

[xavier2k6]

@arvidn Regarding to OpenSSL 1.1.1 End Of Life

---

1. Should wheels for Python <3.10 be dropped?

• Python 3.11: - Windows builds and macOS installers from python.org now use OpenSSL 3.0.
• feat: use SPEC 0 schedule for cibuildwheel

---

2. What minimum OpenSSL 3.x.x version should be declared for below?

libtorrent/tools/cibuildwheel/manylinux/build-openssl.sh

Line 19 in 2ab8fed

OPENSSL_MIN_VERSION=1.1.1

libtorrent/tools/cibuildwheel/manylinux/openssl-version.sh

Lines 3 to 4 in 2ab8fed

	export OPENSSL_ROOT=openssl-1.1.1l
	export OPENSSL_HASH=0b7a3e5e59c34827fe0c3a74b7ec8baef302b98fa80088d7f9153aa16fa76bd1

---

3. FORTIFY_SOURCE may be bumped to 3 if images are newer/all conditions/requirements are met.

libtorrent/tools/cibuildwheel/manylinux/build_utils.sh

Line 8 in 2ab8fed

MANYLINUX_CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2"

[DjLegolas]

@xavier2k6 please re-add MacOS to the pull request matrix (MATRIX_PULL_REQUEST) as the libtorrent must be also validated on it.

[xavier2k6]

@DjLegolas I only removed macOS from pull request matrix to reduce building time while I test as I know they work.

When I am done, they will be re-added - don't worry.

[arvidn]

using windows-latest means that CI will probably break when that version changes. Is there a good reason to use this rather than the specific version of the runner?

[xavier2k6]

GHA CI use the latest (2) server editions....so 2019/2022 - latest = 2022, latest will eventually become 2025 a few months after being GA & 2019 will become obsolete/removed....at some stage the CI will eventually have to migrate away from obsolete/deprecated OS runners provided by github.

This will be the same for macOS-12 & also your linux CI will have to be changed as Ubuntu-20.04 will be removed soon too.

[xavier2k6]

The only real show stopper here is the windows 32bit failure & to then re-instate the builds I took out while testing.

[qstokkink]

C:\Program Files\OpenSSL\lib\libcrypto.lib : warning LNK4272: library machine type 'x64' conflicts with target machine type 'x86'
C:\Program Files\OpenSSL\lib\libssl.lib : warning LNK4272: library machine type 'x64' conflicts with target machine type 'x86'

Apparently, the 64-bit OpenSSL is found. AFAIK, GitHub Actions runners do not have 32-bit OpenSSL by default. So, you would need to make sure that the 32-bit version is downloaded and installed from somewhere.

Judging from the paths in /Jamfile (in rule openssl-lib-path and rule openssl-include-path), installing a 32-bit OpenSSL in C:/Program Files (x86)/OpenSSL-Win32 could magically fix this. If not, you would need to conditionally update the OPENSSL_LIB and OPENSSL_INCLUDE paths in the /Jamfile.

[xavier2k6]

@arvidn I'll leave the OpenSSL 32bit installation to you, do you still want to support 32bit??

We also now have a definitive timeline for the deprecation of the macOS-12 runner
The macOS 12 Actions runner image will begin deprecation on 10/7/24 and will be fully unsupported by 12/3/24

[qstokkink]

@xavier2k6 Do you have time to experiment a bit with the win32 build? If we're lucky, it could be as simple as inserting the following at line 106 of cibuildwheel.yml:

- name: Install OpenSSL
  if: ${{ endsWith(matrix.CIBW_BUILD, 'win32') }}
  run: choco install openssl

If all checks are green/passing, it would be easier to merge this PR, I guess.

[xavier2k6]

@qstokkink i'll be able to give that a try in about 3hrs (got a spare few mins now to do it), action needs to be updated too & python 3.13 is gone GA.

[xavier2k6]

vcpkg is on the windows runners, this could be used to install openssl perhaps.....

[qstokkink]

@xavier2k6 Thanks for putting some time into this. Using vcpkg should also work, but the current setup was made for chocolatey:

libtorrent/Jamfile

Lines 442 to 446 in 790b662

	# the de-facto windows installer is https://slproweb.com/products/Win32OpenSSL.html, which installs to c:\Program Files\OpenSSL-Win{32,64}.
	# chocolatey appears to use this installer.
	local address_model = [ feature.get-values <address-model> : $(properties) ] ;
	OPENSSL_LIB += "C:/Program Files/OpenSSL-Win$(address_model)/lib" ;
	OPENSSL_LIB += "C:/Program Files (x86)/OpenSSL-Win$(address_model)/lib" ;

libtorrent/Jamfile

Lines 473 to 474 in 790b662

	OPENSSL_INCLUDE += "C:/Program Files/OpenSSL-Win$(address_model)/include" ;
	OPENSSL_INCLUDE += "C:/Program Files (x86)/OpenSSL-Win$(address_model)/include" ;

---

Using your log output, switching the four paths to this should work:

if address_model = 32
{
    OPENSSL_LIB += "C:/vcpkg/packages/openssl_x86-windows-static-md/lib" ;
} else {
    OPENSSL_LIB += "C:/vcpkg/packages/openssl_x64-windows-static-md/lib" ;
}

if address_model = 32
{
    OPENSSL_INCLUDE += "C:/vcpkg/packages/openssl_x86-windows-static-md/include" ;
}  else {
    OPENSSL_INCLUDE += "C:/vcpkg/packages/openssl_x64-windows-static-md/include" ;
}

My Jam language is a bit rusty. So, this might need to be address_model = "32".

[xavier2k6]

@qstokkink Will make some changes later, will also need to make sure we use the right package as there's 3x release options I believe:
openssl:x86-windows-static
openssl:x86-windows-static-md
openssl:x86-windows

I'll have to look at vcpkg port/json files etc.

vcpkg gets updated each time there's a newer windows runner image released so that will bring newer versions of OpenSSL.