bump NCCL floor to 2.18.1.1, include nccl.h where it's needed

[jameslamb]

Description

Contributes to rapidsai/build-planning#102

Some RAPIDS libraries are using ncclCommSplit(), which was introduced in nccl==2.18.1.1. This is part of a series of PRs across RAPIDS updating libraries' pins to nccl>=2.18.1.1 to ensure they get a new-enough version that supports that.

[vyasr]

cugraph also ought to be able to rely on raft for its NCCL handling rather than doing anything on its own AFAICT. It does use some raft APIs that accept NCCL types directly as arguments, but I still think that's OK.

[jameslamb]

I'm not convinced that cugraph could/should drop its NCCL dependency and only get it transitively through raft.

It has some direct uses, like these:

cugraph/cpp/include/cugraph/mtmg/instance_manager.hpp

Line 52 in 7e058e2

RAFT_NCCL_TRY(ncclCommDestroy(*nccl_comms_[i]));

cugraph/cpp/include/cugraph/mtmg/resource_manager.hpp

Line 200 in 7e058e2

ncclCommInitRank(nccl_comms[i].get(), ranks_to_include.size(), instance_manager_id, rank));

As well as others in tests and examples:

cugraph/cpp/tests/mtmg/threaded_test.cu

Line 41 in 7e058e2

#include <nccl.h>

cugraph/cpp/tests/mtmg/threaded_test_jaccard.cu

Line 41 in 7e058e2

#include <nccl.h>

cugraph/cpp/tests/mtmg/threaded_test_louvain.cu

Line 41 in 7e058e2

#include <nccl.h>

cugraph/cpp/examples/developers/graph_operations/CMakeLists.txt

Line 28 in 7e058e2

include(../../../cmake/thirdparty/get_nccl.cmake)

cugraph/cpp/examples/developers/graph_operations/CMakeLists.txt

Line 38 in 7e058e2

target_link_libraries(graph_operations PRIVATE cugraph::cugraph NCCL::NCCL MPI::MPI_CXX)

Those look like direct-enough uses that cugraph should have its own pinnings, in my opinion.

[vyasr]

I agree, those are direct uses. In that case, though, the files should be including nccl.h rather than relying on it being transitively included by raft's std_comms.h.

[jameslamb]

the files should be including nccl.h rather than relying on it being transitively included by raft's std_comms.h.

Agreed! Pushed 7fca13f adding that.

Maybe in the future we can explore using include-what-you-use (https://github.com/include-what-you-use/include-what-you-use) in CI. I found from search that you'd tried that last year: rapidsai/cudf#581 (comment), maybe at some point we can look into it again.

[jameslamb]

/merge