Auto merge of #40018 - japaric:ld, r=alexcrichton

-Z linker-flavor (Please read the commit message first) This PR is an alternative to #36120 (internal lld linker). The main goal of this PR is to make it *possible* to use LLD as a linker to allow out of tree experimentation. Now that LLD is going to be shipped with LLVM 4.0, it should become easier to get a hold of LLD (hopefully, it will be packaged by Linux distros soon). Since LLD is a multiarch linker, it has the potential to make cross compilation easier (less tools need to be installed). Supposedly, LLD is also faster than the gold linker so LLD may improve build times where link times are significant (e.g. 100% incremental compilation reuse). The place where LLD shines is at linking Rust programs that don't depend on system libraries. For example, here's how you would link a bare metal ARM Cortex-M program: ``` $ xargo rustc --target thumbv7m-none-eabi -- -Z linker-flavor=ld -C linker=ld.lld -Z print-link-args "ld.lld" \ "-L" \ "$XARGO_HOME/lib/rustlib/thumbv7m-none-eabi/lib" \ "$PWD/target/thumbv7m-none-eabi/debug/deps/app-de1f86df314ad68c.0.o" \ "-o" \ "$PWD/target/thumbv7m-none-eabi/debug/deps/app-de1f86df314ad68c" \ "--gc-sections" \ "-L" \ "$PWD/target/thumbv7m-none-eabi/debug/deps" \ "-L" \ "$PWD/target/debug/deps" \ "-L" \ "$XARGO_HOME/lib/rustlib/thumbv7m-none-eabi/lib" \ "-Bstatic" \ "-Bdynamic" \ "$XARGO_HOME/lib/rustlib/thumbv7m-none-eabi/lib/libcore-11670d2bd4951fa7.rlib" $ file target/thumbv7m-none-eabi/debug/app app: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, not stripped, with debug_info ``` This doesn't require installing the `arm-none-eabi-gcc` toolchain. Even cooler (but I'm biased) is that you can link Rust programs that use [`steed`] (`steed` is a `std` re-implementation free of C dependencies for Linux systems) instead of `std` for a bunch of different architectures without having to install a single cross toolchain. [`steed`]: https://github.com/japaric/steed ``` $ xargo rustc --target aarch64-unknown-linux-steed --example hello --release -- -Z print-link-args "ld.lld" \ "-L" \ "$XARGO_HOME/lib/rustlib/aarch64-unknown-linux-steed/lib" \ "$PWD/target/aarch64-unknown-linux-steed/release/examples/hello-80c130ad884c0f8f.0.o" \ "-o" \ "$PWD/target/aarch64-unknown-linux-steed/release/examples/hello-80c130ad884c0f8f" \ "--gc-sections" \ "-L" \ "$PWD/target/aarch64-unknown-linux-steed/release/deps" \ "-L" \ "$PWD/target/release/deps" \ "-L" \ "$XARGO_HOME/lib/rustlib/aarch64-unknown-linux-steed/lib" \ "-Bstatic" \ "-Bdynamic" \ "/tmp/rustc.lAybk9Ltx93Q/libcompiler_builtins-589aede02de78434.rlib" $ file target/aarch64-unknown-linux-steed/release/examples/hello hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, not stripped, with debug_info ``` All these targets (architectures) worked with LLD: - [aarch64-unknown-linux-steed](https://github.com/japaric/steed/blob/lld/docker/aarch64-unknown-linux-steed.json) - [arm-unknown-linux-steedeabi](https://github.com/japaric/steed/blob/lld/docker/arm-unknown-linux-steedeabi.json) - [arm-unknown-linux-steedeabihf](https://github.com/japaric/steed/blob/lld/docker/arm-unknown-linux-steedeabihf.json) - [armv7-unknown-linux-steedeabihf](https://github.com/japaric/steed/blob/lld/docker/armv7-unknown-linux-steedeabihf.json) - [i686-unknown-linux-steed](https://github.com/japaric/steed/blob/lld/docker/i686-unknown-linux-steed.json) - [mips-unknown-linux-steed](https://github.com/japaric/steed/blob/lld/docker/mips-unknown-linux-steed.json) - [mipsel-unknown-linux-steed](https://github.com/japaric/steed/blob/lld/docker/mipsel-unknown-linux-steed.json) - [powerpc-unknown-linux-steed](https://github.com/japaric/steed/blob/lld/docker/powerpc-unknown-linux-steed.json) - [powerpc64-unknown-linux-steed](https://github.com/japaric/steed/blob/lld/docker/powerpc64-unknown-linux-steed.json) - [x86_64-unknown-linux-steed](https://github.com/japaric/steed/blob/lld/docker/x86_64-unknown-linux-steed.json) --- The case where lld is unergonomic is linking binaries that depend on system libraries. Like "Hello, world" for `x86_64-unknown-linux-gnu`. Because you have to pass as linker arguments: the path to the startup objects, the path to the dynamic linker and the library search paths. And all those are system specific so they can't be encoded in the target itself. ``` $ cargo \ rustc \ --release \ -- \ -C \ linker=ld.lld \ -Z \ linker-flavor=ld \ -C \ link-args='-dynamic-linker /lib64/ld-linux-x86-64.so.2 -L/usr/lib -L/usr/lib/gcc/x86_64-pc-linux-gnu/6.3.1 /usr/lib/Scrt1.o /usr/lib/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/6.3.1/crtbeginS.o /usr/lib/gcc/x86_64-pc-linux-gnu/6.3.1/crtendS.o /usr/lib/crtn.o' ``` --- Another case where `-Z linker-flavor` may come in handy is directly calling Solaris' linker which is also a multiarch linker (or so I have heard). cc @binarycrusader cc @alexcrichton Heads up: [breaking-change] due to changes in the target specification format.
rust-lang · Apr 10, 2017 · 3b5754e · 3b5754e
2 parents 8493dd6 + e192fb3
commit 3b5754e
Show file tree

Hide file tree

Showing 92 changed files with 682 additions and 240 deletions.
diff --git a/src/doc/unstable-book/src/SUMMARY.md b/src/doc/unstable-book/src/SUMMARY.md
@@ -108,6 +108,7 @@
 - [link_llvm_intrinsics](link-llvm-intrinsics.md)
 - [linkage](linkage.md)
 - [linked_list_extras](linked-list-extras.md)
+- [linker_flavor](linker-flavor.md)
 - [log_syntax](log-syntax.md)
 - [lookup_host](lookup-host.md)
 - [loop_break_value](loop-break-value.md)

diff --git a/src/doc/unstable-book/src/linker-flavor.md b/src/doc/unstable-book/src/linker-flavor.md
@@ -0,0 +1,61 @@
+# `linker-flavor`
+
+The tracking issue for this feature is: None
+
+------------------------
+
+Every `rustc` target defaults to some linker. For example, Linux targets default
+to gcc. In some cases, you may want to override the default; you can do that
+with the unstable CLI argument: `-Z linker-flavor`.
+
+Here how you would use this flag to link a Rust binary for the
+`thumbv7m-none-eabi` using LLD instead of GCC.
+
+``` text
+$ xargo rustc --target thumbv7m-none-eabi -- \
+    -C linker=ld.lld \
+    -Z linker-flavor=ld \
+    -Z print-link-args | tr ' ' '\n'
+"ld.lld"
+"-L"
+"$SYSROOT/lib/rustlib/thumbv7m-none-eabi/lib"
+"$PWD/target/thumbv7m-none-eabi/debug/deps/app-512e9dbf385f233c.0.o"
+"-o"
+"$PWD/target/thumbv7m-none-eabi/debug/deps/app-512e9dbf385f233c"
+"--gc-sections"
+"-L"
+"$PWD/target/thumbv7m-none-eabi/debug/deps"
+"-L"
+"$PWD/target/debug/deps"
+"-L"
+"$SYSROOT/lib/rustlib/thumbv7m-none-eabi/lib"
+"-Bstatic"
+"$SYSROOT/lib/rustlib/thumbv7m-none-eabi/lib/libcore-e1ccb7dfb1cb9ebb.rlib"
+"-Bdynamic"
+```
+
+Whereas the default is:
+
+``` text
+$ xargo rustc --target thumbv7m-none-eabi -- \
+    -C link-arg=-nostartfiles \
+    -Z print-link-args | tr ' ' '\n'
+"arm-none-eabi-gcc"
+"-L"
+"$SYSROOT/lib/rustlib/thumbv7m-none-eabi/lib"
+"$PWD/target/thumbv7m-none-eabi/debug/deps/app-961e39416baa38d9.0.o"
+"-o"
+"$PWD/target/thumbv7m-none-eabi/debug/deps/app-961e39416baa38d9"
+"-Wl,--gc-sections"
+"-nodefaultlibs"
+"-L"
+"$PWD/target/thumbv7m-none-eabi/debug/deps"
+"-L"
+"$PWD/target/debug/deps"
+"-L"
+"$SYSROOT/lib/rustlib/thumbv7m-none-eabi/lib"
+"-Wl,-Bstatic"
+"$SYSROOT/lib/rustlib/thumbv7m-none-eabi/lib/libcore-e1ccb7dfb1cb9ebb.rlib"
+"-nostartfiles"
+"-Wl,-Bdynamic"
+```
diff --git a/src/librustc/session/config.rs b/src/librustc/session/config.rs
@@ -19,7 +19,7 @@ pub use self::DebugInfoLevel::*;
 use session::{early_error, early_warn, Session};
 use session::search_paths::SearchPaths;
 
-use rustc_back::PanicStrategy;
+use rustc_back::{LinkerFlavor, PanicStrategy};
 use rustc_back::target::Target;
 use lint;
 use middle::cstore;
@@ -641,12 +641,14 @@ macro_rules! options {
             Some("either `panic` or `abort`");
         pub const parse_sanitizer: Option<&'static str> =
             Some("one of: `address`, `leak`, `memory` or `thread`");
+        pub const parse_linker_flavor: Option<&'static str> =
+            Some(::rustc_back::LinkerFlavor::one_of());
     }
 
     #[allow(dead_code)]
     mod $mod_set {
         use super::{$struct_name, Passes, SomePasses, AllPasses, Sanitizer};
-        use rustc_back::PanicStrategy;
+        use rustc_back::{LinkerFlavor, PanicStrategy};
 
         $(
             pub fn $opt(cg: &mut $struct_name, v: Option<&str>) -> bool {
@@ -777,6 +779,14 @@ macro_rules! options {
             }
             true
         }
+
+        fn parse_linker_flavor(slote: &mut Option<LinkerFlavor>, v: Option<&str>) -> bool {
+            match v.and_then(LinkerFlavor::from_str) {
+                Some(lf) => *slote = Some(lf),
+                _ => return false,
+            }
+            true
+        }
     }
 ) }
 
@@ -979,6 +989,8 @@ options! {DebuggingOptions, DebuggingSetter, basic_debugging_options,
           "pass `-install_name @rpath/...` to the macOS linker"),
     sanitizer: Option<Sanitizer> = (None, parse_sanitizer, [TRACKED],
                                    "Use a sanitizer"),
+    linker_flavor: Option<LinkerFlavor> = (None, parse_linker_flavor, [UNTRACKED],
+                                           "Linker flavor"),
 }
 
 pub fn default_lib_output() -> CrateType {

diff --git a/src/librustc/session/mod.rs b/src/librustc/session/mod.rs
@@ -36,7 +36,7 @@ use syntax::{ast, codemap};
 use syntax::feature_gate::AttributeType;
 use syntax_pos::{Span, MultiSpan};
 
-use rustc_back::PanicStrategy;
+use rustc_back::{LinkerFlavor, PanicStrategy};
 use rustc_back::target::Target;
 use rustc_data_structures::flock;
 use llvm;
@@ -363,6 +363,9 @@ impl Session {
     pub fn panic_strategy(&self) -> PanicStrategy {
         self.opts.cg.panic.unwrap_or(self.target.target.options.panic_strategy)
     }
+    pub fn linker_flavor(&self) -> LinkerFlavor {
+        self.opts.debugging_opts.linker_flavor.unwrap_or(self.target.target.linker_flavor)
+    }
     pub fn no_landing_pads(&self) -> bool {
         self.opts.debugging_opts.no_landing_pads || self.panic_strategy() == PanicStrategy::Abort
     }

diff --git a/src/librustc_back/lib.rs b/src/librustc_back/lib.rs
@@ -52,6 +52,48 @@ pub mod dynamic_lib;
 
 use serialize::json::{Json, ToJson};
 
+macro_rules! linker_flavor {
+    ($(($variant:ident, $string:expr),)+) => {
+        #[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd, Hash,
+                 RustcEncodable, RustcDecodable)]
+        pub enum LinkerFlavor {
+            $($variant,)+
+        }
+
+        impl LinkerFlavor {
+            pub const fn one_of() -> &'static str {
+                concat!("one of: ", $($string, " ",)+)
+            }
+
+            pub fn from_str(s: &str) -> Option<Self> {
+                Some(match s {
+                    $($string => LinkerFlavor::$variant,)+
+                    _ => return None,
+                })
+            }
+
+            pub fn desc(&self) -> &str {
+                match *self {
+                    $(LinkerFlavor::$variant => $string,)+
+                }
+            }
+        }
+
+        impl ToJson for LinkerFlavor {
+            fn to_json(&self) -> Json {
+                self.desc().to_json()
+            }
+        }
+    }
+}
+
+linker_flavor! {
+    (Em, "em"),
+    (Gcc, "gcc"),
+    (Ld, "ld"),
+    (Msvc, "msvc"),
+}
+
 #[derive(Clone, Copy, Debug, PartialEq, Hash, RustcEncodable, RustcDecodable)]
 pub enum PanicStrategy {
     Unwind,

diff --git a/src/librustc_back/target/aarch64_apple_ios.rs b/src/librustc_back/target/aarch64_apple_ios.rs
@@ -8,6 +8,7 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use target::{Target, TargetOptions, TargetResult};
 use super::apple_ios_base::{opts, Arch};
 
@@ -22,6 +23,7 @@ pub fn target() -> TargetResult {
         target_os: "ios".to_string(),
         target_env: "".to_string(),
         target_vendor: "apple".to_string(),
+        linker_flavor: LinkerFlavor::Gcc,
         options: TargetOptions {
             features: "+neon,+fp-armv8,+cyclone".to_string(),
             eliminate_frame_pointer: false,

diff --git a/src/librustc_back/target/aarch64_linux_android.rs b/src/librustc_back/target/aarch64_linux_android.rs
@@ -8,6 +8,7 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use target::{Target, TargetOptions, TargetResult};
 
 // See https://developer.android.com/ndk/guides/abis.html#arm64-v8a
@@ -28,6 +29,7 @@ pub fn target() -> TargetResult {
         target_os: "android".to_string(),
         target_env: "".to_string(),
         target_vendor: "unknown".to_string(),
+        linker_flavor: LinkerFlavor::Gcc,
         options: TargetOptions {
             abi_blacklist: super::arm_base::abi_blacklist(),
             .. base

diff --git a/src/librustc_back/target/aarch64_unknown_freebsd.rs b/src/librustc_back/target/aarch64_unknown_freebsd.rs
@@ -8,6 +8,7 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use target::{Target, TargetOptions, TargetResult};
 
 pub fn target() -> TargetResult {
@@ -26,6 +27,7 @@ pub fn target() -> TargetResult {
         target_os: "freebsd".to_string(),
         target_env: "".to_string(),
         target_vendor: "unknown".to_string(),
+        linker_flavor: LinkerFlavor::Gcc,
         options: TargetOptions {
             abi_blacklist: super::arm_base::abi_blacklist(),
             .. base

diff --git a/src/librustc_back/target/aarch64_unknown_fuchsia.rs b/src/librustc_back/target/aarch64_unknown_fuchsia.rs
@@ -8,6 +8,7 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use target::{Target, TargetOptions, TargetResult};
 
 pub fn target() -> TargetResult {
@@ -23,6 +24,7 @@ pub fn target() -> TargetResult {
         target_os: "fuchsia".to_string(),
         target_env: "".to_string(),
         target_vendor: "unknown".to_string(),
+        linker_flavor: LinkerFlavor::Gcc,
         options: TargetOptions {
             abi_blacklist: super::arm_base::abi_blacklist(),
             .. base

diff --git a/src/librustc_back/target/aarch64_unknown_linux_gnu.rs b/src/librustc_back/target/aarch64_unknown_linux_gnu.rs
@@ -8,6 +8,7 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use target::{Target, TargetOptions, TargetResult};
 
 pub fn target() -> TargetResult {
@@ -26,6 +27,7 @@ pub fn target() -> TargetResult {
         arch: "aarch64".to_string(),
         target_os: "linux".to_string(),
         target_vendor: "unknown".to_string(),
+        linker_flavor: LinkerFlavor::Gcc,
         options: TargetOptions {
             abi_blacklist: super::arm_base::abi_blacklist(),
             .. base

diff --git a/src/librustc_back/target/android_base.rs b/src/librustc_back/target/android_base.rs
@@ -8,13 +8,15 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use target::TargetOptions;
 
 pub fn opts() -> TargetOptions {
     let mut base = super::linux_base::opts();
     // Many of the symbols defined in compiler-rt are also defined in libgcc.
     // Android's linker doesn't like that by default.
-    base.pre_link_args.push("-Wl,--allow-multiple-definition".to_string());
+    base.pre_link_args
+        .get_mut(&LinkerFlavor::Gcc).unwrap().push("-Wl,--allow-multiple-definition".to_string());
     base.is_like_android = true;
     base.position_independent_executables = true;
     base.has_elf_tls = false;

diff --git a/src/librustc_back/target/apple_base.rs b/src/librustc_back/target/apple_base.rs
@@ -10,7 +10,7 @@
 
 use std::env;
 
-use target::TargetOptions;
+use target::{LinkArgs, TargetOptions};
 
 pub fn opts() -> TargetOptions {
     // ELF TLS is only available in macOS 10.7+. If you try to compile for 10.6
@@ -43,7 +43,7 @@ pub fn opts() -> TargetOptions {
         dll_prefix: "lib".to_string(),
         dll_suffix: ".dylib".to_string(),
         archive_format: "bsd".to_string(),
-        pre_link_args: Vec::new(),
+        pre_link_args: LinkArgs::new(),
         exe_allocation_crate: super::maybe_jemalloc(),
         has_elf_tls: version >= (10, 7),
         .. Default::default()

diff --git a/src/librustc_back/target/apple_ios_base.rs b/src/librustc_back/target/apple_ios_base.rs
@@ -8,9 +8,10 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use std::io;
 use std::process::Command;
-use target::TargetOptions;
+use target::{LinkArgs, TargetOptions};
 
 use self::Arch::*;
 
@@ -60,7 +61,7 @@ pub fn get_sdk_root(sdk_name: &str) -> Result<String, String> {
     }
 }
 
-fn build_pre_link_args(arch: Arch) -> Result<Vec<String>, String> {
+fn build_pre_link_args(arch: Arch) -> Result<LinkArgs, String> {
     let sdk_name = match arch {
         Armv7 | Armv7s | Arm64 => "iphoneos",
         I386 | X86_64 => "iphonesimulator"
@@ -70,8 +71,14 @@ fn build_pre_link_args(arch: Arch) -> Result<Vec<String>, String> {
 
     let sdk_root = get_sdk_root(sdk_name)?;
 
-    Ok(vec!["-arch".to_string(), arch_name.to_string(),
-         "-Wl,-syslibroot".to_string(), sdk_root])
+    let mut args = LinkArgs::new();
+    args.insert(LinkerFlavor::Gcc,
+                vec!["-arch".to_string(),
+                     arch_name.to_string(),
+                     "-Wl,-syslibroot".to_string(),
+                     sdk_root]);
+
+    Ok(args)
 }
 
 fn target_cpu(arch: Arch) -> String {

diff --git a/src/librustc_back/target/arm_linux_androideabi.rs b/src/librustc_back/target/arm_linux_androideabi.rs
@@ -8,6 +8,7 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use target::{Target, TargetOptions, TargetResult};
 
 pub fn target() -> TargetResult {
@@ -24,6 +25,7 @@ pub fn target() -> TargetResult {
         target_os: "android".to_string(),
         target_env: "".to_string(),
         target_vendor: "unknown".to_string(),
+        linker_flavor: LinkerFlavor::Gcc,
         options: TargetOptions {
             abi_blacklist: super::arm_base::abi_blacklist(),
             .. base

diff --git a/src/librustc_back/target/arm_unknown_linux_gnueabi.rs b/src/librustc_back/target/arm_unknown_linux_gnueabi.rs
@@ -8,6 +8,7 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use target::{Target, TargetOptions, TargetResult};
 
 pub fn target() -> TargetResult {
@@ -22,6 +23,7 @@ pub fn target() -> TargetResult {
         target_os: "linux".to_string(),
         target_env: "gnu".to_string(),
         target_vendor: "unknown".to_string(),
+        linker_flavor: LinkerFlavor::Gcc,
 
         options: TargetOptions {
             features: "+v6".to_string(),

diff --git a/src/librustc_back/target/arm_unknown_linux_gnueabihf.rs b/src/librustc_back/target/arm_unknown_linux_gnueabihf.rs
@@ -8,6 +8,7 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
+use LinkerFlavor;
 use target::{Target, TargetOptions, TargetResult};
 
 pub fn target() -> TargetResult {
@@ -22,6 +23,7 @@ pub fn target() -> TargetResult {
         target_os: "linux".to_string(),
         target_env: "gnu".to_string(),
         target_vendor: "unknown".to_string(),
+        linker_flavor: LinkerFlavor::Gcc,
 
         options: TargetOptions {
             features: "+v6,+vfp2".to_string(),