aws/core/Aws.h - anyway of building without the dependency #11333
Labels
question
Further information is requested
Comments
sononix
added
Needs Triage
|
The S3 support can be disabled using the |
|
Thanks @sononix for the question and @upsj for the answer. Setting the CMake variable |
rapids-bot bot
pushed a commit
that referenced
this issue
Aug 8, 2022
This PR is a breaking change that disables Arrow S3 support by default. Enabling this feature by default has caused build issues for many downstream consumers, all of whom (to my knowledge) manually disable support for this feature. Most commonly, that build error appears as `fatal error: aws/core/Aws.h: No such file or directory`. In my understanding, several downstream consumers of cudf no longer rely on Arrow S3 support from this library and instead get S3 access via fsspec. I am not aware of any users of libcudf who rely on this being enabled by default (or enabled at all). See related issues and discussions: #8617, #11333, #8867, #10644 (comment), NVIDIA/spark-rapids#2827. Build errors caused by this default behavior have also been reported internally. cc: @rjzamora @beckernick @jdye64 @randerzander @robertmaynard @jlowe @quasiben if you have comments following our previous discussion. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Nghia Truong (https://github.com/ttnghia) - GALI PREM SAGAR (https://github.com/galipremsagar) - Vyas Ramasubramani (https://github.com/vyasr) - AJ Schmidt (https://github.com/ajschmidt8) URL: #11470
|
Just FYI. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
After looking through the documentation on building libcudf - I'm still hitting a block on
/workspace/build/_deps/arrow-src/cpp/src/arrow/filesystem/s3fs.cc:42:10: fatal error: aws/core/Aws.h: No such file or directory 42 | #include | ^~~~~~~~~~~~~~~~I have managed to build libcudf locally, yes after jumping through a number of hopes in particular the above error where I built the AWS sdk - https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/setup-linux.html from source code.
Im trying to stream line my build process by docker-ising the build procedure - Im using https://github.com/isVoid/linkto_libcudf,where I added some additional libraries and including the latest version of build tools
The error is displaying again when I build within a docker container, so I was wondering is there anyway of installing a pre-built binary etc - I have tried using environment variable to skip an S3 build, but that doesnt work, building from AWS source was how I got past it locally but the build takes a very long time !!!!!
From multiple treads that are here - Im wondering how everyone else is building from source as it doesnt appear to be straight forward, there are multiple dependencies on external modules, for example:
./build.sh cudfbuilding cudf you are required to have skbuild install to progress the build.
The text was updated successfully, but these errors were encountered: