Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[libc] Support configurable errno modes #98287

Merged
merged 10 commits into from
Jul 13, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions libc/config/baremetal/config.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
{
"errno": {
"LIBC_CONF_ERRNO_MODE": {
"value": "LIBC_ERRNO_MODE_EXTERNAL"
frobtech marked this conversation as resolved.
Show resolved Hide resolved
}
},
"printf": {
"LIBC_CONF_PRINTF_DISABLE_FLOAT": {
"value": true
Expand Down
6 changes: 6 additions & 0 deletions libc/config/config.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
{
"errno": {
"LIBC_CONF_ERRNO_MODE": {
"value": "",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would probably be best to make the default value LIBC_ERRNO_MODE_THREAD_LOCAL

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree but the problem is that we currently don't have any mechanism to distinguish the overlay from the full build mode, and in the overlay mode we want to default to LIBC_ERRNO_MODE_SYSTEM.

"doc": "The implementation used for errno, acceptable values are LIBC_ERRNO_MODE_NONE, LIBC_ERRNO_MODE_INTERNAL, LIBC_ERRNO_MODE_EXTERNAL, LIBC_ERRNO_MODE_THREAD_LOCAL, LIBC_ERRNO_MODE_GLOBAL, and LIBC_ERRNO_MODE_SYSTEM."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really think this needs to not only enumerate the modes but document what each one means. That would make this spot the right place to bikeshed the names, none of which I really like so far.

Perhaps:

  • UNDEFINED: libc never stores a value; errno macro uses get link-time failure
  • EXTERNAL_ABI: embedder must define int *__llvm_libc_errno(void); C function
  • THREAD_LOCAL: libc maintains per-thread state (requires C++ thread_local support for libc implementation code)
  • SHARED_ATOMIC: libc maintains an atomic_int used by all threads, contrary to standard C semantics (requires use of atomics in libc implementation code, may produce libcalls an embedder must supply)
  • SHARED_THREAD_UNSAFE: libc maintains a plain int used by all threads, contrary to standard C semantics unless always single-threaded; nothing prevents data races
  • SYSTEM_HEADER: In overlay mode, system <errno.h> errno macro is used directly by libc. In fullbuild mode, effectively the same as EXTERNAL_ABI (or perhaps we make it an error to choose this in fullbuild mode).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it makes sense to distinguish the "shared atomic" and "shared thread unsafe" modes? Right now these are only used by the GPU port which uses the atomic version. Specifically the only reason for the "shared thread unsafe" mode seems to be avoiding the use of atomics, but we already use those elsewhere so I'm not sure how useful that is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Today's "baremetal" is a very specific set of choices about an embedded-style use case. I'm confident that there will in the long run be embedded-style use cases that are purely single-threaded and want to avoid the overhead (or need for useless libcall stubs that do nothing) of atomics. I think it will certainly be the right thing for all libc code that might ever be useful in some minimal embedded context to be configurable to avoid atomics at the cost of any kind of thread safety. I don't think it will ever be wise enough to be acceptable to "achieve" that via hacks like -DLIBC_THREAD_LOCAL= or making cpp::Atomic configurable to itself not actually be atomic. Instead, I think each and every case (so far, rand and errno) merits its own intentional and explicit configuration choice so in each context someone has to decide what semantics, and tradeoffs between formally standards-compliant and/or de facto norm semantics and other context-specific factors, are right for them.

For today, it doesn't matter if we support everything that we should in principle support in terms of configuration options. The taxonomy above I think covers all the options for errno in particular that I anticipate we will support long term. The subset of THREAD_LOCAL, SHARED_ATOMIC, and SHARED_THREAD_UNSAFE is probably a common vocabulary that makes sense for the broader general class of cases that includes rand, while the other options here are idiosyncratic to errno.

}
},
"printf": {
"LIBC_CONF_PRINTF_DISABLE_FLOAT": {
"value": false,
Expand Down
5 changes: 5 additions & 0 deletions libc/config/gpu/config.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
{
"errno": {
"LIBC_CONF_ERRNO_MODE": {
"value": "LIBC_ERRNO_MODE_GLOBAL"
}
},
"printf": {
"LIBC_CONF_PRINTF_DISABLE_FLOAT": {
"value": true
Expand Down
2 changes: 2 additions & 0 deletions libc/docs/configure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ to learn about the defaults for your platform and target.
* **"codegen" options**
- ``LIBC_CONF_ENABLE_STRONG_STACK_PROTECTOR``: Enable -fstack-protector-strong to defend against stack smashing attack.
- ``LIBC_CONF_KEEP_FRAME_POINTER``: Keep frame pointer in functions for better debugging experience.
* **"errno" options**
- ``LIBC_CONF_ERRNO_MODE``: The implementation used for errno, acceptable values are LIBC_ERRNO_MODE_NONE, LIBC_ERRNO_MODE_INTERNAL, LIBC_ERRNO_MODE_EXTERNAL, LIBC_ERRNO_MODE_THREAD_LOCAL, LIBC_ERRNO_MODE_GLOBAL, and LIBC_ERRNO_MODE_SYSTEM.
* **"malloc" options**
- ``LIBC_CONF_FREELIST_MALLOC_BUFFER_SIZE``: Default size for the constinit freelist buffer used for the freelist malloc implementation (default 1o 1GB).
* **"math" options**
Expand Down
18 changes: 6 additions & 12 deletions libc/include/errno.h.def
Original file line number Diff line number Diff line change
Expand Up @@ -25,18 +25,12 @@
#include "llvm-libc-macros/generic-error-number-macros.h"
#endif

#if defined(__AMDGPU__) || defined(__NVPTX__)
extern int __llvmlibc_errno; // Not thread_local!
#else
#ifdef __cplusplus
extern "C" {
extern thread_local int __llvmlibc_errno;
}
#else
extern _Thread_local int __llvmlibc_errno;
#endif // __cplusplus
#endif
__BEGIN_C_DECLS

int *__llvm_libc_errno(void) __NOEXCEPT;

__END_C_DECLS

#define errno __llvmlibc_errno
#define errno (*__llvm_libc_errno())

#endif // LLVM_LIBC_ERRNO_H
5 changes: 5 additions & 0 deletions libc/src/errno/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ if(LLVM_LIBC_FULL_BUILD)
set(full_build_flag "-DLIBC_FULL_BUILD")
endif()

if(LIBC_CONF_ERRNO_MODE)
set(errno_config_copts "-DLIBC_ERRNO_MODE=${LIBC_CONF_ERRNO_MODE}")
endif()

add_entrypoint_object(
errno
SRCS
Expand All @@ -17,6 +21,7 @@ add_entrypoint_object(
libc_errno.h # Include this
COMPILE_OPTIONS
${full_build_flag}
${errno_config_copts}
DEPENDS
libc.hdr.errno_macros
libc.src.__support.common
Expand Down
99 changes: 73 additions & 26 deletions libc/src/errno/libc_errno.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,44 +9,91 @@
#include "libc_errno.h"
#include "src/__support/CPP/atomic.h"

#ifdef LIBC_TARGET_ARCH_IS_GPU
// LIBC_THREAD_LOCAL on GPU currently does nothing. So essentially this is just
// a global errno for gpu to use for now.
#define LIBC_ERRNO_MODE_UNDEFINED 0x01
petrhosek marked this conversation as resolved.
Show resolved Hide resolved
#define LIBC_ERRNO_MODE_THREAD_LOCAL 0x02
#define LIBC_ERRNO_MODE_GLOBAL 0x04
#define LIBC_ERRNO_MODE_EXTERNAL 0x08
#define LIBC_ERRNO_MODE_SYSTEM 0x10

#ifndef LIBC_ERRNO_MODE
#ifndef LIBC_COPT_PUBLIC_PACKAGING
// This mode is for unit testing. We just use our internal errno.
#define LIBC_ERRNO_MODE LIBC_ERRNO_MODE_INTERNAL
#elif defined(LIBC_FULL_BUILD)
// In full build mode, we provide the errno storage ourselves.
#define LIBC_ERRNO_MODE LIBC_ERRNO_MODE_THREAD_LOCAL
#else
#define LIBC_ERRNO_MODE LIBC_ERRNO_MODE_SYSTEM
#endif
#endif // LIBC_ERRNO_MODE

#if LIBC_ERRNO_MODE != LIBC_ERRNO_MODE_UNDEFINED && \
michaelrj-google marked this conversation as resolved.
Show resolved Hide resolved
LIBC_ERRNO_MODE != LIBC_ERRNO_MODE_THREAD_LOCAL && \
LIBC_ERRNO_MODE != LIBC_ERRNO_MODE_GLOBAL && \
LIBC_ERRNO_MODE != LIBC_ERRNO_MODE_EXTERNAL && \
LIBC_ERRNO_MODE != LIBC_ERRNO_MODE_SYSTEM
#error LIBC_ERRNO_MODE must be one of the following values: \
LIBC_ERRNO_MODE_NONE, \
LIBC_ERRNO_MODE_THREAD_LOCAL, \
LIBC_ERRNO_MODE_GLOBAL, \
LIBC_ERRNO_MODE_EXTERNAL, \
LIBC_ERRNO_MODE_SYSTEM
#endif

namespace LIBC_NAMESPACE {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to update to LIBC_NAMESPACE_DECL when that stuff lands. (I'm not sure it technically matters as long as it was done in libc_errno.h, but I think uniformity it what we're going for regardless.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, I expect this change is going to land after that one.


// Define the global `libc_errno` instance.
Errno libc_errno;

#if LIBC_ERRNO_MODE == LIBC_ERRNO_MODE_UNDEFINED

void Errno::operator=(int) {}
Errno::operator int() { return 0; }

#elif LIBC_ERRNO_MODE == LIBC_ERRNO_MODE_THREAD_LOCAL
petrhosek marked this conversation as resolved.
Show resolved Hide resolved

namespace {
LIBC_THREAD_LOCAL int __libc_errno;
petrhosek marked this conversation as resolved.
Show resolved Hide resolved
}

extern "C" {
LIBC_THREAD_LOCAL LIBC_NAMESPACE::cpp::Atomic<int> __llvmlibc_errno;
int *__llvm_libc_errno(void) { return &__libc_errno; }
petrhosek marked this conversation as resolved.
Show resolved Hide resolved
}

void LIBC_NAMESPACE::Errno::operator=(int a) {
__llvmlibc_errno.store(a, cpp::MemoryOrder::RELAXED);
void Errno::operator=(int a) { __libc_errno = a; }
Errno::operator int() { return __libc_errno; }

#elif LIBC_ERRNO_MODE == LIBC_ERRNO_MODE_GLOBAL
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this mode should be called "SHARED_ATOMIC" or something like that. ISTM there should be another "SHARED_THREAD_UNSAFE" or the like that's just a plain variable.

Again, these variables should have names appropriate for a local linkage implementation detail, and it doesn't hurt to use different names specific to each different implementation here.


namespace {
LIBC_NAMESPACE::cpp::Atomic<int> __libc_errno;
petrhosek marked this conversation as resolved.
Show resolved Hide resolved
}
LIBC_NAMESPACE::Errno::operator int() {
return __llvmlibc_errno.load(cpp::MemoryOrder::RELAXED);

extern "C" {
int *__llvm_libc_errno(void) { return &__libc_errno; }
}

#elif !defined(LIBC_COPT_PUBLIC_PACKAGING)
// This mode is for unit testing. We just use our internal errno.
LIBC_THREAD_LOCAL int __llvmlibc_internal_errno;
void Errno::operator=(int a) {
__libc_errno.store(a, cpp::MemoryOrder::RELAXED);
}
Errno::operator int() {
return __libc_errno.load(cpp::MemoryOrder::RELAXED);
}

void LIBC_NAMESPACE::Errno::operator=(int a) { __llvmlibc_internal_errno = a; }
LIBC_NAMESPACE::Errno::operator int() { return __llvmlibc_internal_errno; }
#elif LIBC_ERRNO_MODE == LIBC_ERRNO_MODE_EXTERNAL

#elif defined(LIBC_FULL_BUILD)
// This mode is for public libc archive, hermetic, and integration tests.
// In full build mode, we provide the errno storage ourselves.
extern "C" {
LIBC_THREAD_LOCAL int __llvmlibc_errno;
int *__llvm_libc_errno(void);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need a declaration here. It's already in <errno.h>.

Copy link
Member Author

@petrhosek petrhosek Jul 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a policy about using a hdr/errno_macros.h wrapper instead of doing it directly, but yeah. In some fashion, the authoritative header file declaration of any public ABI function you're defining really ought to be in scope in the implementation file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there may be a case to be made that there should be an __llvm_libc_errno.h in this directory to declare the function like we have for other public (ABI) functions. But then that would go with the __llvm_libc_errno.cpp implementation file for just that function, which isn't what we want to do since we want it and the Errno internal class implemented together.

I'm not sure what the precedent is (or if there is any yet) for other non-functions in the API, i.e. having an errno.h that's responsible for errno inside the libc implementation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the precedent is (or if there is any yet) for other non-functions in the API, i.e. having an errno.h that's responsible for errno inside the libc implementation.

We use https://github.com/llvm/llvm-project/blob/0b15f89182a4b2a4c46ad207fa2e282ad35f12ee/libc/src/errno/libc_errno.h for that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That header has that name because it's implementing LIBC_NAMESPACE::libc_errno. It's true that today it does include hdr/errno_macros.h, but that only defines the E* constants (necessarily, i.e. in all build modes).

That's a different issue from whether when there is a variable (or variable-like API, such as errno) in the public API, we have a symbol_name.h header for it in src/something/ like we do for functions in the public API.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I introduced src/errno/errno.h in the latest revision.

}

void LIBC_NAMESPACE::Errno::operator=(int a) { __llvmlibc_errno = a; }
LIBC_NAMESPACE::Errno::operator int() { return __llvmlibc_errno; }
void Errno::operator=(int a) { *__llvm_libc_errno() = a; }
Errno::operator int() { return *__llvm_libc_errno(); }

#else
void LIBC_NAMESPACE::Errno::operator=(int a) { errno = a; }
LIBC_NAMESPACE::Errno::operator int() { return errno; }
#elif LIBC_ERRNO_MODE == LIBC_ERRNO_MODE_SYSTEM

#endif // LIBC_FULL_BUILD
void Errno::operator=(int a) { errno = a; }
Errno::operator int() { return errno; }

#endif

namespace LIBC_NAMESPACE {
// Define the global `libc_errno` instance.
Errno libc_errno;
} // namespace LIBC_NAMESPACE
1 change: 1 addition & 0 deletions libc/src/errno/libc_errno.h
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
// - Still depend on libc.src.errno.errno

namespace LIBC_NAMESPACE {

struct Errno {
void operator=(int);
operator int();
Expand Down
Loading