subclass distutils UnixCCompiler for more control over building native extensions in setup.py invocation

[cosmicexplorer]

It's really hard to invoke a setup.py and have it "just work" when providing a toolchain as in #5490. See this FIXME -- it looks like I may have restarted all the failing builds that spawned this comment, so the gist is that setup.py will fail because it cannot be told not to use x86_64-linux-gnu-gcc to link native extensions (even with CC=clang, which makes compilation to object files succeed). The FIXME describes subclassing distutils's UnixCCompiler (also see parent CCompiler) to control the compiler that is invoked.

The implementation of this would be simple: just compile all the files and link into shared libraries. You can see that's just what UnixCCompiler is doing in the link above. This would need to be provided to a setup.py subject of a python_dist() target, and we should probably try to make it so the setup.py doesn't have to be modified.

[cosmicexplorer]

Thinking about implementing this -- was struck that it might be useful to spin this off into its own package on pypi that doesn't depend on the pants package (something like pantsbuild.pants.distutils_extensions maybe?). The idea would be that this code would just expose a better interface to invoke native compilation in setup.py for a build tool, and the build tool would take care of making pantsbuild.pants.distutils_extensions available for import. It could also not be exposed as its own pypi package, and just generated by Pants and added to the PYTHONPATH when invoking setup.py (which is probably a good first step). Does this sound reasonable?

[cosmicexplorer]

Discussed with @stuhood offline and decided that decoupling this from its usage in Pants would make this harder than necessary at first, but I definitely think that makes sense after we hammer it out here.