Android cross-compiling from Windows error "cargo:warning=The command line is too long"

[repi]

When building locally on Windows to Android I've been running into Cargo errors about the cc command-line being too long which fails the build.

Windows has a max of 8192 characters, and got over 10k characters in some of the compile invocations, mostly due to long paths for all the include paths.

Example:

running: "c:\\Users\\repi\\Downloads\\android-ndk-r21d\\toolchains\\llvm\\prebuilt\\windows-x86_64\\bin\\aarch64-linux-android30-clang++.cmd" "-O3" "-DANDROID" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=aarch64-linux-android" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\common\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/common/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\fastxml\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/fastxml/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/filebuf/include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/foundation/include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/foundation/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/foundation/src/unix" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/ccd" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/common" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/contact" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/convex" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/distance" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/gjk" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/hf" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/intersection" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/mesh" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/pcm" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/geomutils/src/sweep" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/include\\cooking" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/lowlevel\\api/include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/lowlevel\\common/include/collision" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/lowlevel\\common/include/pipeline" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/lowlevel\\common/include/utils" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/lowlevel/common/src/pipeline" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/lowlevel\\software/include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/lowlevel/software/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\lowlevelaabb\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/lowlevelaabb/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\lowleveldynamics\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/lowleveldynamics/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physx/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physx/src/buffering" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\physxcharacterkinematic\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxcharacterkinematic/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxcooking/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxcooking/src/convex" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxcooking/src/mesh" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\physxextensions\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxextensions/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxextensions/src/serialization" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxextensions/src/serialization/Binary" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxextensions/src/serialization\\File" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxextensions/src/serialization/Xml" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\physxmetadata/core/include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxmetadata/core/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxmetadata/extensions\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxmetadata/extensions/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\physxvehicle\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxvehicle/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxvehicle/src/physxmetadata\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/physxvehicle/src/physxmetadata/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\pvd\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/pvd/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\scenequery\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/scenequery/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\simulationcontroller\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/simulationcontroller/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source\\task\\include" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/task/src" "-I" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX\\pxshared\\include" "-Wall" "-Wextra" "--sysroot=c:\\Users\\repi\\Downloads\\android-ndk-r21d\\toolchains/llvm/prebuilt\\windows-x86_64\\sysroot" "-std=c++14" "-ferror-limit=0" "-Wall" "-Wextra" "-Werror" "-Wstrict-aliasing=2" "-Weverything" "-Wno-alloca" "-Wno-anon-enum-enum-conversion" "-Wno-documentation-deprecated-sync" "-Wno-documentation-unknown-command" "-Wno-gnu-anonymous-struct" "-Wno-undef" "-Wno-unused-function" "-Wno-nested-anon-types" "-Wno-float-equal" "-Wno-padded" "-Wno-weak-vtables" "-Wno-cast-align" "-Wno-conversion" "-Wno-missing-noreturn" "-Wno-missing-variable-declarations" "-Wno-shift-sign-overflow" "-Wno-covered-switch-default" "-Wno-exit-time-destructors" "-Wno-global-constructors" "-Wno-missing-prototypes" "-Wno-unreachable-code" "-Wno-unused-macros" "-Wno-unused-member-function" "-Wno-used-but-marked-unused" "-Wno-weak-template-vtables" "-Wno-deprecated" "-Wno-non-virtual-dtor" "-Wno-invalid-noreturn" "-Wno-return-type-c-linkage" "-Wno-reserved-id-macro" "-Wno-c++98-compat-pedantic" "-Wno-unused-local-typedef" "-Wno-old-style-cast" "-Wno-newline-eof" "-Wno-unused-private-field" "-Wno-format-nonliteral" "-Wno-implicit-fallthrough" "-Wno-undefined-reinterpret-cast" "-Wno-disabled-macro-expansion" "-Wno-zero-as-null-pointer-constant" "-Wno-shadow" "-Wno-unknown-warning-option" "-Wno-atomic-implicit-seq-cst" "-Wno-extra-semi-stmt" "-Wno-gcc-compat" "-Wno-gnu-include-next" "-Wno-class-varargs" "-Wno-implicit-exception-spec-mismatch" "-Wno-macro-redefined" "-Wno-zero-length-array" "-Wno-undefined-func-template" "-Wno-c99-extensions" "-DANDROID" "-DPX_PHYSX_STATIC_LIB" "-DDISABLE_CUDA_PHYSX" "-DPX_SUPPORT_PVD=1" "-DNDEBUG=1" "-DPX_COOKING" "-o" "C:\\git\\embark\\ark\\target\\aarch64-linux-android\\release\\build\\physx-sys-b1c2471c131819bb\\out\\TaskManager.o" "-c" "C:\\Users\\repi\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\physx-sys-0.4.6\\PhysX/physx\\source/task/src\\TaskManager.cpp"
cargo:warning=The command line is too long.
exit code: 1

cc @Jasper-Bekkers @VZout

[repi]

@Jasper-Bekkers you were looking into this right?

[Jasper-Bekkers]

I am, patching up cc to get this to work.

…

On Wed, 1 Jul 2020, 17:40 Johan Andersson, ***@***.***> wrote: Assigned #75 <#75> to @Jasper-Bekkers <https://github.com/Jasper-Bekkers>. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#75 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAMDOR3TBMRJ7GO573UFV3RZNKFZANCNFSM4ONRB6PQ> .

[VZout]

Reduced to roughly 8257 chars when using -no-everything. So close yet so far...

[repi]

@Jasper-Bekkers any further progress on this?

[VZout]

rust-lang/cc-rs#547 solves this.

[repi]

Awesome!

[maxded]

Alright, I've been looking into this issue quite a bit this week and am majorly confused.

The error @repi shows above shouldn't actually error at all. The reason being, yes the command is 10k+ lines of code but cc spawns the command as a new process, whose command limit is 32767 characters according to this blog. I've compared the command length of that same command on a msvc -> msvc cl.exe invocation and it was also over 10k+ characters long but doesn't error.

Linking would actually fail on msvc -> Android as cc tried to link every .o file in one go, which would result in a command over 32k+ characters. When targeting msvc this was solved by using a response file but not when targeting Android. This issue I fixed by progressive linking, which has been merged in cc.

TLDR: no clue why we get this error :(

[repi]

@maxded ouch. you are able to reproduce the error locally on Windows right though?

[maxded]

Yes

[maxded]

Figured it out! Fix is on it's way rust-lang/cc-rs#572 :)

[maxded]

Would it be possible to reduce the number of include directories when compiling physx-sys itself instead of relying on a fix in cc-rs? My fix mentioned above reintroduces response files to cc-rs when they just recently got completely removed and the maintainer would prefer if they stayed removed.

[Jake-Shadle]

You can try, but the way the physx code works means a lot of includes are needed.

[maxded]

The latest version of cc fixes this issue!

[repi]

Awesome!