std: Add a new env module

[alexcrichton]

This is an implementation of RFC 578 which adds a new std::env module
to replace most of the functionality in the current std::os module. More
details can be found in the RFC itself, but as a summary the following methods
have all been deprecated:

• os::args_as_bytes => env::args
• os::args => env::args
• os::consts => env::consts
• os::dll_filename => no replacement, use env::consts directly
• os::page_size => env::page_size
• os::make_absolute => use env::current_dir + join instead
• os::getcwd => env::current_dir
• os::change_dir => env::set_current_dir
• os::homedir => env::home_dir
• os::tmpdir => env::temp_dir
• os::join_paths => env::join_paths
• os::split_paths => env::split_paths
• os::self_exe_name => env::current_exe
• os::self_exe_path => use env::current_exe + pop
• os::set_exit_status => env::set_exit_status
• os::get_exit_status => env::get_exit_status
• os::env => env::vars
• os::env_as_bytes => env::vars
• os::getenv => env::var or env::var_string
• os::getenv_as_bytes => env::var
• os::setenv => env::set_var
• os::unsetenv => env::remove_var

Many function signatures have also been tweaked for various purposes, but the
main changes were:

• Vec-returning APIs now all return iterators instead
• All APIs are now centered around OsString instead of Vec<u8> or String.
  There is currently on convenience API, env::var_string, which can be used to
  get the value of an environment variable as a unicode String.

All old APIs are #[deprecated] in-place and will remain for some time to allow
for migrations. The semantics of the APIs have been tweaked slightly with regard
to dealing with invalid unicode (panic instead of replacement).

The new std::env module is all contained within the env feature, so crates
must add the following to access the new APIs:

#![feature(env)]

[breaking-change]

[rust-highfive]

r? @pcwalton

(rust_highfive has picked a reviewer for you, use r? to override)

[alexcrichton]

r? @aturon

note that rust-lang/rfcs#578 has not yet been merged and this should hold off on approval until that time.

[bors]

⌛ Testing commit b05dbc6 with merge 48a923d...

[bors]

💔 Test failed - auto-mac-64-nopt-t

[alexcrichton]

@bors: r=aturon 630df23

[bors]

⌛ Testing commit 630df23 with merge a097829...

[bors]

💔 Test failed - auto-win-64-nopt-t

[alexcrichton]

@bors: r+ 70ed3a4

[John-Nagle]

This is great! With this, to get the command line arguments as a vector of strings, you simply write

let args: Vec<String> = env::args().map(|s| s.into_string().unwrap()).collect();

Now, new programmers learning Rust will have to understand closures, iterators, functional programming, and UTF8 vs OS strings just to get past "Hello World". That will keep out the Code Academy riff-raff!

Put in a convenience function in env::args for this.

For a sense of the pain caused by this, see this Stack Overflow question:

http://stackoverflow.com/questions/15619320/how-to-access-command-line-parameters-in-rust/28356205#28356205

[nagisa]

@John-Nagle your example panics if any of the arguments contain non-unicode data which is not that uncommon. Let’s not allow sloppy programming practices proliferate.

[John-Nagle]

I know. "pub fn into_string_lossy(Self) -> String;" for OsString doesn't seem to have been implemented yet.

If you have non-Unicode data from OsString conversion of command line args, something is badly broken.

[nagisa]

If you have non-Unicode data from OsString conversion of command line args, something is badly broken.

Are you implying any system which uses non utf-8 locale is broken?

[aturon]

@John-Nagle

I'm sympathetic to your original comment; the prominence of OsString was debated as part of this API, and what landed in this PR was a minimalistic, initial cut.

An alternative API that pushes stronger for Unicode-by-default might be:

pub fn args() -> Args // Iterator<Item = String>
pub fn args_os() -> ArgsOs // Iterator<Item = OsString>

pub fn var<K: ?Sized>(key: &K) -> Result<String, VarError> where K: AsOsStr
pub fn var_os<K: ?Sized>(key: &K) -> Option<OsString> where K: AsOsStr

pub fn vars() -> Vars // Iterator<Item = (String, String)>
pub fn vars_os() -> VarsOs // Iterator<Item = (OsString, OsString)>

where the non-OS versions will panic on non-Unicode data, just as your example does. We're likely to grow such variants regardless; the main question is what the un-decorated names like args should map to.

While I'm also sympathetic to worries about how much newcomers have to learn at once, I don't particularly buy the point about Vec versus iterators. For one, iterators are an extremely common concept found in many (these days, most) languages. It's a concept you need to know to understand for in Rust. For another, if you're thinking about very simple scripts that take e.g. a single argument, then args().nth(1) will do the trick (and give you an Option value that you can use to display an error for example).

I know. "pub fn into_string_lossy(Self) -> String;" for OsString doesn't seem to have been implemented yet.

The to_string_lossy method is probably what you want (http://static.rust-lang.org/doc/master/std/ffi/struct.OsStr.html#method.to_string_lossy), but discoverability is a bit poor right now because rustdoc doesn't yet hint about Deref.

[John-Nagle]

Are you implying any system which uses non utf-8 locale is broken?

No, that's handled. OsString is an opaque type. See
https://github.com/rust-lang/rfcs/blob/master/text/0517-io-os-reform.md#the-design-os_str
especially where it says "but CRUCIALLY NOT as_bytes". If OsString values are locale sensitive, the OS support package for that platform must deal with it invisibly to the Rust program.

This whole area seems way overdesigned. The old approach was much simpler, and the use case for all this complexity seems to be corrupted UCS-2 on Windows. Anything Windows can represent as UCS-2, even with surrogate pairs, can be represented as a UTF-8 Rust string. Some conversions in the other direction won't work, and they should raise exceptions. Yes, this may mean that you can't put emoji into a Windows file name.

[aturon]

This whole area seems way overdesigned. The old approach was much simpler, and the use case for all this complexity seems to be corrupted UCS-2 on Windows.

The other side of OsString, on Unix, is dealing with byte sequences that are not UTF-8. Since paths, arguments, environment variables, etc are just C strings (and in many systems are not required to follow any particular encoding), we need a way to represent the full space of values.

Note that "the old approach" already had things like env_as_bytes to tackle the above. This has just been packaged into a more robust, cross-platform OsString.

That said, as I wrote above, I think we probably want to consider making the "default" variants work with String and have a separate set of more "advanced" forms that robust, cross-platform code would use.

[nagisa]

@John-Nagle yes, this is handled by OsString, but conversion of the data inside OsString into String will fail with panic in many cases.

Which is exactly why I argue against any methods converting to String implicitly.

[aturon]

@John-Nagle

Anything Windows can represent as UCS-2, even with surrogate pairs, can be represented as a UTF-8 Rust string. Some conversions in the other direction won't work, and they should raise exceptions.

This appears to be backwards. UCS-2 is a superset of UTF-16. That means that e.g. Rust strings can be converted to UCS-2 without loss, but not vice versa.

Regardless, the motivation for OsString is to provide some means of faithfully representing the platform's "native" string type (usually UCS-2 on Windows, byte sequences on Unixen) without loss and in a way that supports cross-platform programming.

[John-Nagle]

That said, as I wrote above, I think we probably want to consider making the "default" variants work with String and have a separate set of more "advanced" forms that robust, cross-platform code would use.

That seems to be the most practical approach. New Rust programmers should not have to comprehend all the topics covered in this discussion just to access the command line arguments.