Swift host tools compiled with wrong cpu on Apple Silicon

[thii]

Host tools are being compiled for x86_64 cpu when compiling a target that uses them on an arm Mac, with an arm Bazel.

# BUILD
load("@build_bazel_rules_swift//swift:swift.bzl", "swift_binary")

genrule(
    name = "hello_swift",
    outs = ["hello.swift"],
    cmd = """echo 'print("Hello")' > $@""",
)

swift_binary(
    name = "hello_swift_bin",
    srcs = [":hello_swift"],
)

genrule(
    name = "a",
    outs = ["a.out"],
    cmd = """$(execpath :hello_swift_bin) > $@""",
    tools = [":hello_swift_bin"],
)

$ bazel build a
...
SUBCOMMAND: # //:hello [action 'Compiling Swift module hello [for host]', configuration: 130cff6c5881621671f254bf93eadfab04237d1f78aac0f4e2233c4b12d50eee, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_user/c65ad10e7591af2b858f41477e1f1a20/execroot/build_bazel_rules_swift && \
  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=11.0 \
    XCODE_VERSION_OVERRIDE=12.2.0.12B45b \
  bazel-out/host/bin/tools/worker/worker swiftc @bazel-out/host/bin/hello.swiftmodule-0.params)
SUBCOMMAND: # //:hello [action 'Linking hello [for host]', configuration: 130cff6c5881621671f254bf93eadfab04237d1f78aac0f4e2233c4b12d50eee, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_user/c65ad10e7591af2b858f41477e1f1a20/execroot/build_bazel_rules_swift && \
  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=11.0 \
    PATH=/bin:/usr/bin \
    XCODE_VERSION_OVERRIDE=12.2.0.12B45b \
  external/local_config_cc/cc_wrapper.sh -lc++ -fobjc-link-runtime -Wl,-S -o bazel-out/host/bin/hello bazel-out/host/bin/hello_objs/hello.swift.o -headerpad_max_install_names -Wl,-rpath,/usr/lib/swift -Wl,-rpath,__BAZEL_XCODE_DEVELOPER_DIR__/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx -F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks -L__BAZEL_XCODE_DEVELOPER_DIR__/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx -ObjC -Wl,-objc_abi_version,2 -L/usr/lib/swift -no-canonical-prefixes -target arm64-apple-macosx '-mmacosx-version-min=11.0')
ERROR: /Users/user/forks/rules_swift/BUILD:26:13: Linking hello [for host] failed: (Exit 1): cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -lc++ -fobjc-link-runtime -Wl,-S -o bazel-out/host/bin/hello bazel-out/host/bin/hello_objs/hello.swift.o -headerpad_max_install_names -Wl,-rpath,/usr/lib/swift ... (remaining 10 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -lc++ -fobjc-link-runtime -Wl,-S -o bazel-out/host/bin/hello bazel-out/host/bin/hello_objs/hello.swift.o -headerpad_max_install_names -Wl,-rpath,/usr/lib/swift ... (remaining 10 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
ld: warning: ignoring file bazel-out/host/bin/hello_objs/hello.swift.o, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
Undefined symbols for architecture arm64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Target //:a failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 52.627s, Critical Path: 15.98s
INFO: 214 processes: 7 internal, 206 darwin-sandbox, 1 worker.
FAILED: Build did NOT complete successfully

$ cat bazel-out/host/bin/hello_swift_bin.swiftmodule-0.params
-target
x86_64-apple-macosx11.0
-sdk
__BAZEL_XCODE_SDKROOT__
-F__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks
-I__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/usr/lib
-emit-object
-output-file-map
bazel-out/host/bin/hello_swift_bin.output_file_map.json
-emit-module-path
bazel-out/host/bin/hello_swift_bin.swiftmodule
-emit-objc-header-path
bazel-out/host/bin/hello_swift_bin-Swift.h
-DNDEBUG
-O
-Xwrapped-swift=-ephemeral-module-cache
-Xcc
-iquote.
-Xcc
-iquotebazel-out/host/bin
-Xfrontend
-color-diagnostics
-enable-batch-mode
-module-name
hello_swift_bin
-Xcc
-Os
-Xcc
-DNDEBUG=1
-Xcc
-Wno-unused-variable
-Xcc
-Winit-self
-Xcc
-Wno-extra
bazel-out/host/bin/hello.swift

It works fine if we build the tool directly with --macos_cpus=arm64.

Tools written in C/C++ work fine.

I'm using a custom build for darwin arm64 at https://github.com/thii/bazel/tree/4.0.0-darwin_arm64 (based on 4.0.0rc2), so there’s a possibility that it’s a problem with my build.

[keith]

Looks like the cpu we use for the triple comes from the apple fragment:

https://github.com/bazelbuild/rules_swift/blob/0f773324d3778e21a6a3c57ef8d3cdd8e4e5a645/swift/internal/xcode_swift_toolchain.bzl#L572

[keith]

Seems like the issue is coming from here

bazel/src/main/java/com/google/devtools/build/lib/rules/apple/AppleConfiguration.java

Lines 99 to 101 in 92a9e2f

	this.macosCpus = (options.macosCpus == null || options.macosCpus.isEmpty())
	? ImmutableList.of(AppleCommandLineOptions.DEFAULT_MACOS_CPU)
	: ImmutableList.copyOf(options.macosCpus);

[keith]

It makes sense to me that you building directly with --macos_cpus works, but it doesn't work as a tool because in that case what you'd really need would be --host_macos_cpus.

[thii]

Getting the same result with --host_cpu=darwin_arm64.

[keith]

I'm not sure how host_cpu works for the apple configuration so I'm not sure if that makes a difference

[thii]

cc_binary tools work because they're not being passed a -target at all, so they'd just be built for the default target. Maybe we can do the same with Swift?

[keith]

I think we need to fix this in bazel itself otherwise I imagine many other things will also be incorrect

[keith]

This is actually probably the same case as if you try to build for the simulator on a apple silicon machine, it will build a x86_64 simulator binary, I assume the fix for both will be similar

[thii]

Our current workaround is invoking the tool with swift run.

[keith]

@susinmotion can you move this issue to the bazel repo?

[susinmotion]

I'm afraid I don't have permissions in rules_swift, so I can't see the option to move this issue. @allevato, any chance you have permissions to transfer this issue?

[allevato]

@susinmotion Transferred!

[aiuto]

cc: @susinmotion

[keith]

#13440