Use specific Android NDK version in CI builds. #12350

edgchen1 · 2022-07-27T21:57:39Z

Description
Current builds use a NDK version that happens to be on the build machine. The build machine environment may change in ways that are outside of our control.
This change installs a specific version of NDK (the current LTS version 25.0.8775105) and uses it.

Motivation and Context
Fix Android CI build, be explicit about the NDK version.
Related PRs: #12341 and #12345

…ackaging pipeline too.

snnn · 2022-07-27T22:10:18Z

tools/ci_build/github/azure-pipelines/templates/use-android-ndk.yml

+  - bash: |
+      set -e
+
+      "${ANDROID_SDK_ROOT}/cmdline-tools/latest/bin/sdkmanager" --install "ndk;${{ parameters.AndroidNdkVersion }}"


What if the sdk is already installed? Would it cause any error?

tested locally, was able to run it multiple times without error

Is it possible to check if the version is already installed? This way when version 25 is added to the hosted agents we dont need to make any changes.

I think the installation is a hot fix. It should be removed while the upstream image become stable.

we may not need to make changes when 25 is already installed on the hosted agent if running the install command again succeeds.
we can update it to not attempt to install anything at all eventually if that is desired, but I would like to have the NDK version that we actually use be more visible.

snnn · 2022-07-27T22:20:58Z

tools/ci_build/github/azure-pipelines/templates/use-android-ndk.yml

+  parameters:
+  - name: AndroidNdkVersion
+    type: string
+    default: "25.0.8775105"  # LTS version


I don't understand: in my test change, when it failed I got an error message with "No version of NDK matched the requested version 21.0.6113669. Versions available locally: 21.4.7075529, 23.2.8568313, 24.0.8215888". So, 25.0 isn't in the list. How does it work for you?

we install NDK 25.0 and set the relevant environment variables in this template. it's done for both Linux and Mac jobs

If we're using a version that is not already available in the env (which I'm guessing might be the case based on Changming's list) is there any concern about network usage? The zipped windows NDK is 446MB, so a lot to download if every time the CI runs it has to do that once.

right, it's not available in the build machine env at the moment. in the Android CI pipeline, it takes about 20 seconds to install on the Linux machines and around a minute on the Mac machines.

As a hot fix, it looks good to me. I still hope we don't need to download the SDK every time. It's not only about network spendings, the more we download, the more likely network errors could occur.

we can pick another version too, as long as we use the same one consistently. I just picked 25.0.8775105 because it sounds like it is supported.
https://developer.android.com/ndk/downloads

actions/runner-images#5930
looks like 25 will be added soon to the hosted agents

Current builds use a NDK version that happens to be on the build machine. The build machine environment may change in ways that are outside of our control. This change installs a specific version of NDK (the current LTS version 25.0.8775105) and uses it.

* update package version * Prevent unbounded growth of command allocator memory (#12114) Prevent unbounded growth of command allocator memory * Update supported ops md for NNAPI/CoreML EP (#12245) * update supported ops md * address pr comments * address pr comments * wording * Change native folder name for java macos arm64 (#12335) * Bump async from 2.6.3 to 2.6.4 in /js/react_native/e2e (#11280) Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4. - [Release notes](https://github.com/caolan/async/releases) - [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md) - [Commits](caolan/async@v2.6.3...v2.6.4) --- updated-dependencies: - dependency-name: async dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [js/rn] upgrade dependencies for e2e test (#11863) * [js/rn] upgrade dependencies for e2e test * use JDK11 only for gradle * expand variable * [js/rn] upgrade package react-native@^0.69.1 (#12155) * [js/rn] upgrade package react-native@^0.69.1 * upgrade compile sdk to v31 * update ios version requirement * update pod path for onnxruntime-react-native * add missing build_java in Android testing stage. (#12187) add missing build_java in testing * Use specific Android NDK version in CI builds. (#12350) Current builds use a NDK version that happens to be on the build machine. The build machine environment may change in ways that are outside of our control. This change installs a specific version of NDK (the current LTS version 25.0.8775105) and uses it. * Remove preview keyword from DirectML pacakge (#12368) Remove preview keyword Co-authored-by: Sumit Agarwal <sumitagarwal@microsoft.com> * Scope CreateFileMapping2 to valid API partitions (#12374) * Fix TRT custom op issue (#12283) * Pass schema registry on CreateModel. * Fix ORT_MINIMAL_BUILD. * Fix build issue. * Manually add optimization flag for Android Release builds. (#12390) With recent versions of NDK (since 23), the `-O` optimization level compile flag is not being passed when building in the "Release" configuration. More details here: android/ndk#1740 Our "Release" Android builds have been built without the optimization flag since we upgraded from NDK 21. This change is a workaround to manually add `-O3` for "Release" Android builds. * resolve conflicts in tensorRT related changes * Enable support of multi-level nested control flow ops model for TRT EP (#12147) * Make multiple-level nested control flow op model work * find correct input index * find correct input index (cont.) * enable nested layer unit tests for TRT EP * add comment * add Scan op to current workaround support of control flow op Co-authored-by: Jeff Bloomfield <38966965+jeffbloo@users.noreply.github.com> Co-authored-by: Rachel Guo <35738743+YUNQIUGUO@users.noreply.github.com> Co-authored-by: Changming Sun <chasun@microsoft.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com> Co-authored-by: Yi Zhang <zhanyi@microsoft.com> Co-authored-by: Edward Chen <18449977+edgchen1@users.noreply.github.com> Co-authored-by: sumitsays <sumitagarwal330@gmail.com> Co-authored-by: Sumit Agarwal <sumitagarwal@microsoft.com> Co-authored-by: Justin Stoecker <justoeck@microsoft.com> Co-authored-by: Yateng Hong <yatengh@microsoft.com> Co-authored-by: Chi Lo <54722500+chilo-ms@users.noreply.github.com>

edgchen1 added 2 commits July 27, 2022 13:11

Add installation of specific NDK version to Android CI build.

a1ae347

Add default value for version parameter, rename template, use it in p…

8e411fe

…ackaging pipeline too.

edgchen1 requested review from snnn, mszhanyi, skottmckay and YUNQIUGUO July 27, 2022 21:57

snnn reviewed Jul 27, 2022

View reviewed changes

Add NDK template to packaging pipeline testing job.

023067d

snnn approved these changes Jul 28, 2022

View reviewed changes

mszhanyi approved these changes Jul 28, 2022

View reviewed changes

edgchen1 merged commit 6e892a9 into master Jul 28, 2022

edgchen1 deleted the edgchen1/specify_ndk_version branch July 28, 2022 18:01

edgchen1 added the release:1.12.1 label Aug 1, 2022

mszhanyi mentioned this pull request Aug 8, 2022

Remove temp downloading NDK 25 #12503

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use specific Android NDK version in CI builds. #12350

Use specific Android NDK version in CI builds. #12350

edgchen1 commented Jul 27, 2022

snnn Jul 27, 2022

edgchen1 Jul 27, 2022

askhade Jul 28, 2022

mszhanyi Jul 28, 2022

edgchen1 Jul 28, 2022

snnn Jul 27, 2022

edgchen1 Jul 27, 2022

skottmckay Jul 28, 2022

edgchen1 Jul 28, 2022

snnn Jul 28, 2022

edgchen1 Jul 28, 2022

edgchen1 Jul 28, 2022

Use specific Android NDK version in CI builds. #12350

Use specific Android NDK version in CI builds. #12350

Conversation

edgchen1 commented Jul 27, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment