Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Migrating cuml comms to raft::comms #7

Merged
merged 51 commits into from
Jun 3, 2020
Merged
Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
fd38dbb
Initial commit of raft comms
cjnolet May 7, 2020
8a6e054
Removing cuml verbiage
cjnolet May 7, 2020
2bfc0dc
Renaming to comms_t, removing logs for now, adding to handle
cjnolet May 8, 2020
f129710
Looking like it's building!
cjnolet May 8, 2020
a0c13af
Adding NCCL and UCX build for tests
cjnolet May 8, 2020
7ad2053
Merge branch 'fea-ext-migrate-cumlHandle_impl-into-raft' into fea-ext…
cjnolet May 8, 2020
cb19482
Consolidating injection functions
cjnolet May 8, 2020
1d66637
Merge branch 'fea-ext-migrate-cumlHandle_impl-into-raft' into fea-ext…
cjnolet May 12, 2020
1548177
Making progress on Python. Need to refactor the comms interface a li…
cjnolet May 12, 2020
5d37e9e
Cython is building!
cjnolet May 13, 2020
bd1b876
The comms tests pass!!!
cjnolet May 13, 2020
98496d3
Fixing flake8 style
cjnolet May 13, 2020
fb52f79
Running clang format and removing getDataType
cjnolet May 13, 2020
d666314
Merge remote-tracking branch 'github/branch-0.14' into fea-ext-comms
cjnolet May 13, 2020
46a27eb
Cleaning up
cjnolet May 13, 2020
fc6ba2e
Fixing python style
cjnolet May 13, 2020
842f533
Fixing cpp style
cjnolet May 13, 2020
43f0d69
Adding copyright headers
cjnolet May 13, 2020
ac6e699
Adding init py for tests
cjnolet May 13, 2020
c442d87
Adding license headers and consistent namespacing
cjnolet May 13, 2020
32e63c1
More cleanup
cjnolet May 13, 2020
d377347
Cleaning up raft.dask.common.Comms
cjnolet May 13, 2020
e1b4ea7
Ignoring raft egg artifacts
cjnolet May 13, 2020
736696b
Cleaning up raft dask utils
cjnolet May 13, 2020
55d9dfd
More cleanup, copyright headers, and docs
cjnolet May 13, 2020
6eb0325
Removing the last of references to cuml
cjnolet May 13, 2020
b65e20d
Fixing python style
cjnolet May 13, 2020
7a845cb
Fixing c++ style
cjnolet May 13, 2020
c269300
Updating changelog
cjnolet May 13, 2020
d4aa5c5
Testing non-ucx cluster for pytests
cjnolet May 13, 2020
1ee1363
Implementing review feedback
cjnolet May 18, 2020
c165a36
More review feedback
cjnolet May 19, 2020
12f3db7
Fixing style
cjnolet May 19, 2020
aae4625
FIX Use relative imports
dantegd May 19, 2020
bc39321
Adding compile-time templates for comms_t to make interaction more st…
cjnolet May 19, 2020
e9c0995
Merge pull request #1 from dantegd/fea-ext-comms-pr1
cjnolet May 19, 2020
3150fbd
Using std::this_thread::yield instead of pthread_yield()
cjnolet May 19, 2020
5628ad2
Adding python tests for collective functions
cjnolet May 19, 2020
488d0d5
Running cpp style
cjnolet May 19, 2020
3d362d0
Updating tabbing for pytests
cjnolet May 19, 2020
417a4bd
Following clang tidy standards
cjnolet May 19, 2020
cb92349
Moving get_type out of comms_t
cjnolet May 19, 2020
6e9025d
More review feedback
cjnolet May 26, 2020
28f8101
Running cpp style check
cjnolet May 26, 2020
1a18553
Nccl red op
cjnolet May 27, 2020
a22dd62
Raising an exception to get around gcc issue
cjnolet May 27, 2020
fbd12aa
Using static for functions for now
cjnolet May 27, 2020
cc71ccb
Fixing style
cjnolet May 27, 2020
dc123b2
Fixing more relative imports
cjnolet May 28, 2020
60597f3
final updates based on feedback
cjnolet Jun 3, 2020
5bbb8be
Merge branch 'branch-0.15' into fea-ext-comms
afender Jun 3, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ log
.ipynb_checkpoints
.DS_Store
dask-worker-space/
*.egg-info/
## eclipse
.project
.cproject
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
## New Features
- Initial RAFT version
- PR #3: defining raft::handle_t, device_buffer, host_buffer, allocator classes
- PR #7: Migrating cuml comms -> raft comms_t

## Improvements

Expand Down
3 changes: 3 additions & 0 deletions cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ find_package(CUDA 10.0 REQUIRED)

set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake)

message("HELLO!")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is probably a remnant of a debug session?


##############################################################################
# - Compiler Options --------------------------------------------------------

Expand Down Expand Up @@ -161,6 +163,7 @@ set(CMAKE_CUDA_FLAGS
# - dependencies -------------------------------------------------------------

include(cmake/Dependencies.cmake)
include(cmake/comms.cmake)

##############################################################################
# - include paths ------------------------------------------------------------
Expand Down
37 changes: 37 additions & 0 deletions cpp/cmake/comms.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#
# Copyright (c) 2019-2020, NVIDIA CORPORATION.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

cmake_minimum_required(VERSION 3.14 FATAL_ERROR)
project(comms LANGUAGES CXX CUDA)

set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this module-path set command here, in a module?


if(NOT NCCL_PATH)
find_package(NCCL REQUIRED)
else()
message("-- Manually set NCCL PATH to ${NCCL_PATH}")
set(NCCL_INCLUDE_DIRS ${NCCL_PATH}/include)
set(NCCL_LIBRARIES ${NCCL_PATH}/lib/libnccl.so)
endif(NOT NCCL_PATH)

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

find_package(UCX)
include_directories(${UCX_INCLUDE_DIRS})

include_directories( ${NCCL_INCLUDE_DIRS} )
list(APPEND RAFT_LINK_LIBRARIES ${NCCL_LIBRARIES})
Loading