From cefd678e029e6a493296a9b439aa23b6d644a2e6 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Tue, 24 Sep 2024 13:47:22 -0400 Subject: [PATCH 01/11] Updating deprecation info --- README.md | 111 ++---------------------------------------------------- 1 file changed, 3 insertions(+), 108 deletions(-) diff --git a/README.md b/README.md index 25ce059630..af2219fdd1 100755 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ #
 RAFT: Reusable Accelerated Functions and Tools for Vector Search and More
> [!IMPORTANT] -> The vector search and clustering algorithms in RAFT are being migrated to a new library dedicated to vector search called [cuVS](https://github.com/rapidsai/cuvs). We will continue to support the vector search algorithms in RAFT during this move, but will no longer update them after the RAPIDS 24.06 (June) release. We plan to complete the migration by RAPIDS 24.08 (August) release. +> The vector search and clustering algorithms in RAFT are being migrated to a new library dedicated to vector search called [cuVS](https://github.com/rapidsai/cuvs). We will continue to support the vector search algorithms in RAFT during this move, but will no longer update them after the RAPIDS 24.06 (June) release. We plan to complete the migration by RAPIDS 24.10 (October) release and will be removing them altogether in the 24.12 (December) release. ![RAFT tech stack](img/raft-tech-stack-vss.png) @@ -36,7 +36,7 @@ ## What is RAFT? -RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications. +RAFT contains fundamental widely-used algorithms and primitives for machine learning and data mining. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications. By taking a primitives-based approach to algorithm development, RAFT - accelerates algorithm construction time @@ -47,12 +47,10 @@ While not exhaustive, the following general categories help summarize the accele ##### | Category | Accelerated Functions in RAFT | |-----------------------|-----------------------------------------------------------------------------------------------------------------------------------| -| **Nearest Neighbors** | vector search, neighborhood graph construction, epsilon neighborhoods, pairwise distances | -| **Basic Clustering** | spectral clustering, hierarchical clustering, k-means | -| **Solvers** | combinatorial optimization, iterative solvers | | **Data Formats** | sparse & dense, conversions, data generation | | **Dense Operations** | linear algebra, matrix and vector operations, reductions, slicing, norms, factorization, least squares, svd & eigenvalue problems | | **Sparse Operations** | linear algebra, eigenvalue problems, slicing, norms, reductions, factorization, symmetrization, components & labeling | +| **Solvers** | combinatorial optimization, iterative solvers | | **Statistics** | sampling, moments and summary statistics, metrics, model evaluation | | **Tools & Utilities** | common tools and utilities for developing CUDA applications, multi-node multi-gpu infrastructure | @@ -67,42 +65,6 @@ In addition being a C++ library, RAFT also provides 2 Python libraries: ![RAFT is a C++ header-only template library with optional shared library and lightweight Python wrappers](img/arch.png) -## Use cases - -### Vector Similarity Search - -RAFT contains state-of-the-art implementations of approximate nearest neighbors search (ANNS) algorithms on the GPU, such as: - -* [Brute force](https://docs.rapids.ai/api/raft/nightly/pylibraft_api/neighbors/#brute-force). Performs a brute force nearest neighbors search without an index. -* [IVF-Flat](https://docs.rapids.ai/api/raft/nightly/pylibraft_api/neighbors/#ivf-flat) and [IVF-PQ](https://docs.rapids.ai/api/raft/nightly/pylibraft_api/neighbors/#ivf-pq). Use an inverted file index structure to map contents to their locations. IVF-PQ additionally uses product quantization to reduce the memory usage of vectors. These methods were originally popularized by the [FAISS](https://github.com/facebookresearch/faiss) library. -* [CAGRA](https://docs.rapids.ai/api/raft/nightly/pylibraft_api/neighbors/#cagra) (Cuda Anns GRAph-based). Uses a fast ANNS graph construction and search implementation optimized for the GPU. CAGRA outperforms state-of-the art CPU methods (i.e. HNSW) for large batch queries, single queries, and graph construction time. - -Projects that use the RAFT ANNS algorithms for accelerating vector search include: [Milvus](https://milvus.io/), [Redis](https://redis.io/), and [Faiss](https://github.com/facebookresearch/faiss). - -Please see the example [Jupyter notebook](https://github.com/rapidsai/raft/blob/HEAD/notebooks/VectorSearch_QuestionRetrieval.ipynb) to get started RAFT for vector search in Python. - - - -### Information Retrieval - -RAFT contains a catalog of reusable primitives for composing algorithms that require fast neighborhood computations, such as - -1. Computing distances between vectors and computing kernel gramm matrices -2. Performing ball radius queries for constructing epsilon neighborhoods -3. Clustering points to partition a space for smaller and faster searches -4. Constructing neighborhood "connectivities" graphs from dense vectors - -### Machine Learning - -RAFT's primitives are used in several RAPIDS libraries, including [cuML](https://github.com/rapidsai/cuml), [cuGraph](https://github.com/rapidsai/cugraph), and [cuOpt](https://github.com/rapidsai/cuopt) to build many end-to-end machine learning algorithms that span a large spectrum of different applications, including -- data generation -- model evaluation -- classification and regression -- clustering -- manifold learning -- dimensionality reduction. - -RAFT is also used by the popular collaborative filtering library [implicit](https://github.com/benfred/implicit) for recommender systems. ## Is RAFT right for me? @@ -327,70 +289,3 @@ When citing RAFT generally, please consider referencing this Github project. year={2022} } ``` -If citing the sparse pairwise distances API, please consider using the following bibtex: -```bibtex -@article{nolet2021semiring, - title={Semiring primitives for sparse neighborhood methods on the gpu}, - author={Nolet, Corey J and Gala, Divye and Raff, Edward and Eaton, Joe and Rees, Brad and Zedlewski, John and Oates, Tim}, - journal={arXiv preprint arXiv:2104.06357}, - year={2021} -} -``` - -If citing the single-linkage agglomerative clustering APIs, please consider the following bibtex: -```bibtex -@misc{nolet2023cuslink, - title={cuSLINK: Single-linkage Agglomerative Clustering on the GPU}, - author={Corey J. Nolet and Divye Gala and Alex Fender and Mahesh Doijade and Joe Eaton and Edward Raff and John Zedlewski and Brad Rees and Tim Oates}, - year={2023}, - eprint={2306.16354}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - -If citing CAGRA, please consider the following bibtex: -```bibtex -@misc{ootomo2023cagra, - title={CAGRA: Highly Parallel Graph Construction and Approximate Nearest Neighbor Search for GPUs}, - author={Hiroyuki Ootomo and Akira Naruse and Corey Nolet and Ray Wang and Tamas Feher and Yong Wang}, - year={2024}, - series = {ICDE '24} -} -``` - -If citing the k-selection routines, please consider the following bibtex: - -```bibtex -@proceedings{10.1145/3581784, - title = {Parallel Top-K Algorithms on GPU: A Comprehensive Study and New Methods}, - author={Jingrong Zhang, Akira Naruse, Xipeng Li, and Yong Wang}, - year = {2023}, - isbn = {9798400701092}, - publisher = {Association for Computing Machinery}, - address = {New York, NY, USA}, - location = {Denver, CO, USA}, - series = {SC '23} -} -``` - -If citing the nearest neighbors descent API, please consider the following bibtex: -```bibtex -@inproceedings{10.1145/3459637.3482344, - author = {Wang, Hui and Zhao, Wan-Lei and Zeng, Xiangxiang and Yang, Jianye}, - title = {Fast K-NN Graph Construction by GPU Based NN-Descent}, - year = {2021}, - isbn = {9781450384469}, - publisher = {Association for Computing Machinery}, - address = {New York, NY, USA}, - url = {https://doi.org/10.1145/3459637.3482344}, - doi = {10.1145/3459637.3482344}, - abstract = {NN-Descent is a classic k-NN graph construction approach. It is still widely employed in machine learning, computer vision, and information retrieval tasks due to its efficiency and genericness. However, the current design only works well on CPU. In this paper, NN-Descent has been redesigned to adapt to the GPU architecture. A new graph update strategy called selective update is proposed. It reduces the data exchange between GPU cores and GPU global memory significantly, which is the processing bottleneck under GPU computation architecture. This redesign leads to full exploitation of the parallelism of the GPU hardware. In the meantime, the genericness, as well as the simplicity of NN-Descent, are well-preserved. Moreover, a procedure that allows to k-NN graph to be merged efficiently on GPU is proposed. It makes the construction of high-quality k-NN graphs for out-of-GPU-memory datasets tractable. Our approach is 100-250\texttimes{} faster than the single-thread NN-Descent and is 2.5-5\texttimes{} faster than the existing GPU-based approaches as we tested on million as well as billion scale datasets.}, - booktitle = {Proceedings of the 30th ACM International Conference on Information \& Knowledge Management}, - pages = {1929–1938}, - numpages = {10}, - keywords = {high-dimensional, nn-descent, gpu, k-nearest neighbor graph}, - location = {Virtual Event, Queensland, Australia}, - series = {CIKM '21} -} -``` From 4b10bf651cc9dda013e42930413a0ddd0e15b470 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Tue, 24 Sep 2024 13:51:12 -0400 Subject: [PATCH 02/11] Deprecating --- cpp/include/raft/neighbors/ball_cover.cuh | 1 + cpp/include/raft/neighbors/ball_cover_types.hpp | 8 ++++---- cpp/include/raft/neighbors/brute_force_types.hpp | 2 +- cpp/include/raft/neighbors/cagra_types.hpp | 2 +- cpp/include/raft/neighbors/epsilon_neighborhood.cuh | 7 ++++--- cpp/include/raft/neighbors/hnsw_types.hpp | 3 +-- cpp/include/raft/neighbors/ivf_flat_types.hpp | 2 +- cpp/include/raft/neighbors/ivf_pq_types.hpp | 2 +- cpp/include/raft/neighbors/nn_descent_types.hpp | 2 +- cpp/include/raft/neighbors/refine-ext.cuh | 10 ++++++++-- 10 files changed, 23 insertions(+), 16 deletions(-) diff --git a/cpp/include/raft/neighbors/ball_cover.cuh b/cpp/include/raft/neighbors/ball_cover.cuh index 20c88f3318..09938020b9 100644 --- a/cpp/include/raft/neighbors/ball_cover.cuh +++ b/cpp/include/raft/neighbors/ball_cover.cuh @@ -13,6 +13,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ + #pragma once #ifndef RAFT_EXPLICIT_INSTANTIATE_ONLY #include "ball_cover-inl.cuh" diff --git a/cpp/include/raft/neighbors/ball_cover_types.hpp b/cpp/include/raft/neighbors/ball_cover_types.hpp index e3ea5f0005..476dc2ea17 100644 --- a/cpp/include/raft/neighbors/ball_cover_types.hpp +++ b/cpp/include/raft/neighbors/ball_cover_types.hpp @@ -42,10 +42,10 @@ namespace raft::neighbors::ball_cover { * @tparam value_t * @tparam value_int */ -template +[[deprecated("Use cuVS instead")]] template class BallCoverIndex { public: explicit BallCoverIndex(raft::resources const& handle_, diff --git a/cpp/include/raft/neighbors/brute_force_types.hpp b/cpp/include/raft/neighbors/brute_force_types.hpp index a8f073edc6..4aafe374b7 100644 --- a/cpp/include/raft/neighbors/brute_force_types.hpp +++ b/cpp/include/raft/neighbors/brute_force_types.hpp @@ -45,7 +45,7 @@ using ann::search_params; * * @tparam T data element type */ -template +[[deprecated("Use cuVS instead")]] template struct index : ann::index { public: /** Distance metric used for retrieval */ diff --git a/cpp/include/raft/neighbors/cagra_types.hpp b/cpp/include/raft/neighbors/cagra_types.hpp index 97c9c0d098..ad017bc854 100644 --- a/cpp/include/raft/neighbors/cagra_types.hpp +++ b/cpp/include/raft/neighbors/cagra_types.hpp @@ -138,7 +138,7 @@ static_assert(std::is_aggregate_v); * @tparam IdxT type of the vector indices (represent dataset.extent(0)) * */ -template +[[deprecated("Use cuVS instead")]] template struct index : ann::index { static_assert(!raft::is_narrowing_v, "IdxT must be able to represent all values of uint32_t"); diff --git a/cpp/include/raft/neighbors/epsilon_neighborhood.cuh b/cpp/include/raft/neighbors/epsilon_neighborhood.cuh index bade4385fb..9a2e82cf5e 100644 --- a/cpp/include/raft/neighbors/epsilon_neighborhood.cuh +++ b/cpp/include/raft/neighbors/epsilon_neighborhood.cuh @@ -1,5 +1,5 @@ /* - * Copyright (c) 2020-2023, NVIDIA CORPORATION. + * Copyright (c) 2020-2024, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -45,7 +45,7 @@ namespace raft::neighbors::epsilon_neighborhood { * squared as we compute L2-squared distance in this method) * @param[in] stream cuda stream */ -template +[[deprecated("Use cuVS instead")]] template void epsUnexpL2SqNeighborhood(bool* adj, idx_t* vd, const value_t* x, @@ -97,7 +97,8 @@ void epsUnexpL2SqNeighborhood(bool* adj, * @param[in] eps defines epsilon neighborhood radius (should be passed as * squared as we compute L2-squared distance in this method) */ -template +[[deprecated( + "Use cuVS instead")]] template void eps_neighbors_l2sq(raft::resources const& handle, raft::device_matrix_view x, raft::device_matrix_view y, diff --git a/cpp/include/raft/neighbors/hnsw_types.hpp b/cpp/include/raft/neighbors/hnsw_types.hpp index f90de6f01b..6718810e1d 100644 --- a/cpp/include/raft/neighbors/hnsw_types.hpp +++ b/cpp/include/raft/neighbors/hnsw_types.hpp @@ -38,8 +38,7 @@ struct search_params : ann::search_params { int num_threads = 0; // number of host threads to use for concurrent searches. Value of 0 // automatically maximizes parallelism }; - -template +[[deprecated("Use cuVS instead")]] template struct index : ann::index { public: /** diff --git a/cpp/include/raft/neighbors/ivf_flat_types.hpp b/cpp/include/raft/neighbors/ivf_flat_types.hpp index 7605bd82a3..a2266cb9c8 100644 --- a/cpp/include/raft/neighbors/ivf_flat_types.hpp +++ b/cpp/include/raft/neighbors/ivf_flat_types.hpp @@ -125,7 +125,7 @@ using list_data = ivf::list; * @tparam IdxT type of the indices in the source dataset * */ -template +[[deprecated("Use cuVS instead")]] template struct index : ann::index { static_assert(!raft::is_narrowing_v, "IdxT must be able to represent all values of uint32_t"); diff --git a/cpp/include/raft/neighbors/ivf_pq_types.hpp b/cpp/include/raft/neighbors/ivf_pq_types.hpp index 3ee350c6fb..0ec75fb354 100644 --- a/cpp/include/raft/neighbors/ivf_pq_types.hpp +++ b/cpp/include/raft/neighbors/ivf_pq_types.hpp @@ -289,7 +289,7 @@ using list_data = ivf::list; * @tparam IdxT type of the indices in the source dataset * */ -template +[[deprecated("Use cuVS instead")]] template struct index : ann::index { static_assert(!raft::is_narrowing_v, "IdxT must be able to represent all values of uint32_t"); diff --git a/cpp/include/raft/neighbors/nn_descent_types.hpp b/cpp/include/raft/neighbors/nn_descent_types.hpp index eb01a423be..7bcb18166d 100644 --- a/cpp/include/raft/neighbors/nn_descent_types.hpp +++ b/cpp/include/raft/neighbors/nn_descent_types.hpp @@ -86,7 +86,7 @@ struct index_params : ann::index_params { * * @tparam IdxT dtype to be used for constructing knn-graph */ -template +[[deprecated("Use cuVS instead")]] template struct index : ann::index { public: /** diff --git a/cpp/include/raft/neighbors/refine-ext.cuh b/cpp/include/raft/neighbors/refine-ext.cuh index 7948a0e4f2..580d807828 100644 --- a/cpp/include/raft/neighbors/refine-ext.cuh +++ b/cpp/include/raft/neighbors/refine-ext.cuh @@ -28,7 +28,10 @@ namespace raft::neighbors { -template +[[deprecated("Use cuVS instead")]] template void refine(raft::resources const& handle, raft::device_matrix_view dataset, raft::device_matrix_view queries, @@ -38,7 +41,10 @@ void refine(raft::resources const& handle, raft::distance::DistanceType metric = distance::DistanceType::L2Unexpanded) RAFT_EXPLICIT; -template +[[deprecated("Use cuVS instead")]] template void refine(raft::resources const& handle, raft::host_matrix_view dataset, raft::host_matrix_view queries, From a40ac62e460e00f7cec0ddf06722582db86ebec1 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Tue, 24 Sep 2024 13:55:14 -0400 Subject: [PATCH 03/11] Deprecating cluster APIs --- cpp/include/raft/cluster/kmeans.cuh | 63 ++++++++++---------- cpp/include/raft/cluster/kmeans_balanced.cuh | 49 ++++++++------- cpp/include/raft/cluster/single_linkage.cuh | 12 ++-- 3 files changed, 65 insertions(+), 59 deletions(-) diff --git a/cpp/include/raft/cluster/kmeans.cuh b/cpp/include/raft/cluster/kmeans.cuh index eb28cc1626..e4f5fe322d 100644 --- a/cpp/include/raft/cluster/kmeans.cuh +++ b/cpp/include/raft/cluster/kmeans.cuh @@ -85,7 +85,7 @@ using KeyValueIndexOp = detail::KeyValueIndexOp; * closest cluster center. * @param[out] n_iter Number of iterations run. */ -template +[[deprecated("Use cuVS instead")]] template void fit(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -149,7 +149,7 @@ void fit(raft::resources const& handle, * @param[out] inertia Sum of squared distances of samples to * their closest cluster center. */ -template +[[deprecated("Use cuVS instead")]] template void predict(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -212,7 +212,7 @@ void predict(raft::resources const& handle, * closest cluster center. * @param[out] n_iter Number of iterations run. */ -template +[[deprecated("Use cuVS instead")]] template void fit_predict(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -241,7 +241,7 @@ void fit_predict(raft::resources const& handle, * @param[out] X_new X transformed in the new space. * [dim = n_samples x n_features] */ -template +[[deprecated("Use cuVS instead")]] template void transform(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -251,7 +251,7 @@ void transform(raft::resources const& handle, detail::kmeans_transform(handle, params, X, centroids, X_new); } -template +[[deprecated("Use cuVS instead")]] template void transform(raft::resources const& handle, const KMeansParams& params, const DataT* X, @@ -304,7 +304,7 @@ void transform(raft::resources const& handle, * @param maxiter maximum number of iterations to run * @param tol tolerance for early stopping convergence */ -template +[[deprecated("Use cuVS instead")]] template void find_k(raft::resources const& handle, raft::device_matrix_view X, raft::host_scalar_view best_k, @@ -337,7 +337,7 @@ void find_k(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -template +[[deprecated("Use cuVS instead")]] template void sample_centroids(raft::resources const& handle, raft::device_matrix_view X, raft::device_vector_view minClusterDistance, @@ -364,7 +364,7 @@ void sample_centroids(raft::resources const& handle, * @param[in] reduction_op The reduction operation used for the cost * */ -template +[[deprecated("Use cuVS instead")]] template void cluster_cost(raft::resources const& handle, raft::device_vector_view minClusterDistance, rmm::device_uvector& workspace, @@ -390,7 +390,8 @@ void cluster_cost(raft::resources const& handle, * @param[out] weight_per_cluster: sum of sample weights per cluster (size n_clusters) * @param[out] new_centroids: output matrix of updated centroids (size n_clusters, n_features) */ -template +[[deprecated( + "Use cuVS instead")]] template void update_centroids(raft::resources const& handle, raft::device_matrix_view X, raft::device_vector_view sample_weights, @@ -431,7 +432,7 @@ void update_centroids(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -template +[[deprecated("Use cuVS instead")]] template void min_cluster_distance(raft::resources const& handle, raft::device_matrix_view X, raft::device_matrix_view centroids, @@ -481,7 +482,7 @@ void min_cluster_distance(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -template +[[deprecated("Use cuVS instead")]] template void min_cluster_and_distance( raft::resources const& handle, raft::device_matrix_view X, @@ -522,7 +523,7 @@ void min_cluster_and_distance( * @param[in] seed Seed for the shuffle * */ -template +[[deprecated("Use cuVS instead")]] template void shuffle_and_gather(raft::resources const& handle, raft::device_matrix_view in, raft::device_matrix_view out, @@ -551,7 +552,7 @@ void shuffle_and_gather(raft::resources const& handle, * [dim = n_cluster] * */ -template +[[deprecated("Use cuVS instead")]] template void count_samples_in_cluster(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -581,7 +582,7 @@ void count_samples_in_cluster(raft::resources const& handle, * [dim = n_cluster x n_features] * @param[in] workspace Temporary workspace buffer which can get resized */ -template +[[deprecated("Use cuVS instead")]] template void init_plus_plus(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -614,7 +615,7 @@ void init_plus_plus(raft::resources const& handle, * @param[out] n_iter Number of iterations run. * @param[in] workspace Temporary workspace buffer which can get resized */ -template +[[deprecated("Use cuVS instead")]] template void fit_main(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -661,7 +662,7 @@ namespace raft::cluster { * closest cluster center. * @param[out] n_iter Number of iterations run. */ -template +[[deprecated("Use cuVS instead")]] template void kmeans_fit(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -673,7 +674,7 @@ void kmeans_fit(raft::resources const& handle, kmeans::fit(handle, params, X, sample_weight, centroids, inertia, n_iter); } -template +[[deprecated("Use cuVS instead")]] template void kmeans_fit(raft::resources const& handle, const KMeansParams& params, const DataT* X, @@ -708,7 +709,7 @@ void kmeans_fit(raft::resources const& handle, * @param[out] inertia Sum of squared distances of samples to * their closest cluster center. */ -template +[[deprecated("Use cuVS instead")]] template void kmeans_predict(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -722,7 +723,7 @@ void kmeans_predict(raft::resources const& handle, handle, params, X, sample_weight, centroids, labels, normalize_weight, inertia); } -template +[[deprecated("Use cuVS instead")]] template void kmeans_predict(raft::resources const& handle, const KMeansParams& params, const DataT* X, @@ -773,7 +774,7 @@ void kmeans_predict(raft::resources const& handle, * closest cluster center. * @param[out] n_iter Number of iterations run. */ -template +[[deprecated("Use cuVS instead")]] template void kmeans_fit_predict(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -787,7 +788,7 @@ void kmeans_fit_predict(raft::resources const& handle, handle, params, X, sample_weight, centroids, labels, inertia, n_iter); } -template +[[deprecated("Use cuVS instead")]] template void kmeans_fit_predict(raft::resources const& handle, const KMeansParams& params, const DataT* X, @@ -818,7 +819,7 @@ void kmeans_fit_predict(raft::resources const& handle, * @param[out] X_new X transformed in the new space. * [dim = n_samples x n_features] */ -template +[[deprecated("Use cuVS instead")]] template void kmeans_transform(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -828,7 +829,7 @@ void kmeans_transform(raft::resources const& handle, kmeans::transform(handle, params, X, centroids, X_new); } -template +[[deprecated("Use cuVS instead")]] template void kmeans_transform(raft::resources const& handle, const KMeansParams& params, const DataT* X, @@ -865,7 +866,7 @@ using KeyValueIndexOp = kmeans::KeyValueIndexOp; * @param[in] workspace Temporary workspace buffer which can get resized * */ -template +[[deprecated("Use cuVS instead")]] template void sampleCentroids(raft::resources const& handle, raft::device_matrix_view X, raft::device_vector_view minClusterDistance, @@ -892,7 +893,7 @@ void sampleCentroids(raft::resources const& handle, * @param[in] reduction_op The reduction operation used for the cost * */ -template +[[deprecated("Use cuVS instead")]] template void computeClusterCost(raft::resources const& handle, raft::device_vector_view minClusterDistance, rmm::device_uvector& workspace, @@ -923,7 +924,7 @@ void computeClusterCost(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -template +[[deprecated("Use cuVS instead")]] template void minClusterDistanceCompute(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -969,7 +970,7 @@ void minClusterDistanceCompute(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -template +[[deprecated("Use cuVS instead")]] template void minClusterAndDistanceCompute( raft::resources const& handle, const KMeansParams& params, @@ -1008,7 +1009,7 @@ void minClusterAndDistanceCompute( * @param[in] seed Seed for the shuffle * */ -template +[[deprecated("Use cuVS instead")]] template void shuffleAndGather(raft::resources const& handle, raft::device_matrix_view in, raft::device_matrix_view out, @@ -1037,7 +1038,7 @@ void shuffleAndGather(raft::resources const& handle, * [dim = n_cluster] * */ -template +[[deprecated("Use cuVS instead")]] template void countSamplesInCluster(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -1068,7 +1069,7 @@ void countSamplesInCluster(raft::resources const& handle, * [dim = n_cluster x n_features] * @param[in] workspace Temporary workspace buffer which can get resized */ -template +[[deprecated("Use cuVS instead")]] template void kmeansPlusPlus(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -1101,7 +1102,7 @@ void kmeansPlusPlus(raft::resources const& handle, * @param[out] n_iter Number of iterations run. * @param[in] workspace Temporary workspace buffer which can get resized */ -template +[[deprecated("Use cuVS instead")]] template void kmeans_fit_main(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, diff --git a/cpp/include/raft/cluster/kmeans_balanced.cuh b/cpp/include/raft/cluster/kmeans_balanced.cuh index a1a182608b..45039dd8b8 100644 --- a/cpp/include/raft/cluster/kmeans_balanced.cuh +++ b/cpp/include/raft/cluster/kmeans_balanced.cuh @@ -72,7 +72,10 @@ namespace raft::cluster::kmeans_balanced { * @param[in] mapping_op (optional) Functor to convert from the input datatype to the arithmetic * datatype. If DataT == MathT, this must be the identity. */ -template +[[deprecated("Use cuVS instead")]] template void fit(const raft::resources& handle, kmeans_balanced_params const& params, raft::device_matrix_view X, @@ -126,11 +129,11 @@ void fit(const raft::resources& handle, * @param[in] mapping_op (optional) Functor to convert from the input datatype to the arithmetic * datatype. If DataT == MathT, this must be the identity. */ -template +[[deprecated("Use cuVS instead")]] template void predict(const raft::resources& handle, kmeans_balanced_params const& params, raft::device_matrix_view X, @@ -191,11 +194,11 @@ void predict(const raft::resources& handle, * @param[in] mapping_op (optional) Functor to convert from the input datatype to the arithmetic * datatype. If DataT and MathT are the same, this must be the identity. */ -template +[[deprecated("Use cuVS instead")]] template void fit_predict(const raft::resources& handle, kmeans_balanced_params const& params, raft::device_matrix_view X, @@ -249,12 +252,12 @@ namespace helpers { * arithmetic datatype. If DataT == MathT, this must be the identity. * @param[in] X_norm (optional) Dataset's row norms [dim = n_samples] */ -template +[[deprecated("Use cuVS instead")]] template void build_clusters(const raft::resources& handle, const kmeans_balanced_params& params, raft::device_matrix_view X, @@ -328,12 +331,12 @@ void build_clusters(const raft::resources& handle, * @param[in] mapping_op (optional) Functor to convert from the input datatype to the * arithmetic datatype. If DataT == MathT, this must be the identity. */ -template +[[deprecated("Use cuVS instead")]] template void calc_centers_and_sizes(const raft::resources& handle, raft::device_matrix_view X, raft::device_vector_view labels, diff --git a/cpp/include/raft/cluster/single_linkage.cuh b/cpp/include/raft/cluster/single_linkage.cuh index d9eba6edc5..92cd413379 100644 --- a/cpp/include/raft/cluster/single_linkage.cuh +++ b/cpp/include/raft/cluster/single_linkage.cuh @@ -1,5 +1,5 @@ /* - * Copyright (c) 2021-2023, NVIDIA CORPORATION. + * Copyright (c) 2021-2024, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -47,9 +47,9 @@ namespace raft::cluster { * of k. The algorithm will set `k = log(n) + c` * @param[in] n_clusters number of clusters to assign data samples */ -template +[[deprecated("Use cuVS instead")]] template void single_linkage(raft::resources const& handle, const value_t* X, size_t m, @@ -86,7 +86,9 @@ constexpr int DEFAULT_CONST_C = 15; * @param[in] c a constant used when constructing connectivities from knn graph. Allows the indirect control of k. The algorithm will set `k = log(n) + c` */ -template +[[deprecated("Use cuVS instead")]] template void single_linkage(raft::resources const& handle, raft::device_matrix_view X, raft::device_matrix_view dendrogram, From 8ddcd3eb446a6fb3fa9064469f9d31b14be66be0 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Tue, 24 Sep 2024 14:14:55 -0400 Subject: [PATCH 04/11] Fixing deprecationsy --- cpp/include/raft/cluster/kmeans.cuh | 126 +++++++++--------- cpp/include/raft/cluster/kmeans_balanced.cuh | 117 ++++++++-------- cpp/include/raft/cluster/single_linkage.cuh | 41 +++--- .../raft/cluster/single_linkage_types.hpp | 4 +- .../raft/neighbors/ball_cover_types.hpp | 25 ++-- .../raft/neighbors/brute_force_types.hpp | 34 ++--- cpp/include/raft/neighbors/cagra_types.hpp | 16 ++- .../raft/neighbors/epsilon_neighborhood.cuh | 36 ++--- cpp/include/raft/neighbors/hnsw_types.hpp | 7 +- cpp/include/raft/neighbors/ivf_flat_types.hpp | 18 +-- cpp/include/raft/neighbors/ivf_pq_types.hpp | 22 +-- .../raft/neighbors/nn_descent_types.hpp | 18 ++- cpp/include/raft/neighbors/refine-ext.cuh | 42 +++--- 13 files changed, 260 insertions(+), 246 deletions(-) diff --git a/cpp/include/raft/cluster/kmeans.cuh b/cpp/include/raft/cluster/kmeans.cuh index e4f5fe322d..38318e8ec8 100644 --- a/cpp/include/raft/cluster/kmeans.cuh +++ b/cpp/include/raft/cluster/kmeans.cuh @@ -85,14 +85,15 @@ using KeyValueIndexOp = detail::KeyValueIndexOp; * closest cluster center. * @param[out] n_iter Number of iterations run. */ -[[deprecated("Use cuVS instead")]] template -void fit(raft::resources const& handle, - const KMeansParams& params, - raft::device_matrix_view X, - std::optional> sample_weight, - raft::device_matrix_view centroids, - raft::host_scalar_view inertia, - raft::host_scalar_view n_iter) +template +[[deprecated("Use cuVS instead")]] void fit( + raft::resources const& handle, + const KMeansParams& params, + raft::device_matrix_view X, + std::optional> sample_weight, + raft::device_matrix_view centroids, + raft::host_scalar_view inertia, + raft::host_scalar_view n_iter) { detail::kmeans_fit(handle, params, X, sample_weight, centroids, inertia, n_iter); } @@ -149,15 +150,16 @@ void fit(raft::resources const& handle, * @param[out] inertia Sum of squared distances of samples to * their closest cluster center. */ -[[deprecated("Use cuVS instead")]] template -void predict(raft::resources const& handle, - const KMeansParams& params, - raft::device_matrix_view X, - std::optional> sample_weight, - raft::device_matrix_view centroids, - raft::device_vector_view labels, - bool normalize_weight, - raft::host_scalar_view inertia) +template +[[deprecated("Use cuVS instead")]] void predict( + raft::resources const& handle, + const KMeansParams& params, + raft::device_matrix_view X, + std::optional> sample_weight, + raft::device_matrix_view centroids, + raft::device_vector_view labels, + bool normalize_weight, + raft::host_scalar_view inertia) { detail::kmeans_predict( handle, params, X, sample_weight, centroids, labels, normalize_weight, inertia); @@ -212,15 +214,16 @@ void predict(raft::resources const& handle, * closest cluster center. * @param[out] n_iter Number of iterations run. */ -[[deprecated("Use cuVS instead")]] template -void fit_predict(raft::resources const& handle, - const KMeansParams& params, - raft::device_matrix_view X, - std::optional> sample_weight, - std::optional> centroids, - raft::device_vector_view labels, - raft::host_scalar_view inertia, - raft::host_scalar_view n_iter) +template +[[deprecated("Use cuVS instead")]] void fit_predict( + raft::resources const& handle, + const KMeansParams& params, + raft::device_matrix_view X, + std::optional> sample_weight, + std::optional> centroids, + raft::device_vector_view labels, + raft::host_scalar_view inertia, + raft::host_scalar_view n_iter) { detail::kmeans_fit_predict( handle, params, X, sample_weight, centroids, labels, inertia, n_iter); @@ -241,7 +244,7 @@ void fit_predict(raft::resources const& handle, * @param[out] X_new X transformed in the new space. * [dim = n_samples x n_features] */ -[[deprecated("Use cuVS instead")]] template +template void transform(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -251,14 +254,14 @@ void transform(raft::resources const& handle, detail::kmeans_transform(handle, params, X, centroids, X_new); } -[[deprecated("Use cuVS instead")]] template -void transform(raft::resources const& handle, - const KMeansParams& params, - const DataT* X, - const DataT* centroids, - IndexT n_samples, - IndexT n_features, - DataT* X_new) +template +[[deprecated("Use cuVS instead")]] void transform(raft::resources const& handle, + const KMeansParams& params, + const DataT* X, + const DataT* centroids, + IndexT n_samples, + IndexT n_features, + DataT* X_new) { detail::kmeans_transform( handle, params, X, centroids, n_samples, n_features, X_new); @@ -304,7 +307,7 @@ void transform(raft::resources const& handle, * @param maxiter maximum number of iterations to run * @param tol tolerance for early stopping convergence */ -[[deprecated("Use cuVS instead")]] template +template void find_k(raft::resources const& handle, raft::device_matrix_view X, raft::host_scalar_view best_k, @@ -337,7 +340,7 @@ void find_k(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -[[deprecated("Use cuVS instead")]] template +template void sample_centroids(raft::resources const& handle, raft::device_matrix_view X, raft::device_vector_view minClusterDistance, @@ -364,7 +367,7 @@ void sample_centroids(raft::resources const& handle, * @param[in] reduction_op The reduction operation used for the cost * */ -[[deprecated("Use cuVS instead")]] template +template void cluster_cost(raft::resources const& handle, raft::device_vector_view minClusterDistance, rmm::device_uvector& workspace, @@ -390,8 +393,7 @@ void cluster_cost(raft::resources const& handle, * @param[out] weight_per_cluster: sum of sample weights per cluster (size n_clusters) * @param[out] new_centroids: output matrix of updated centroids (size n_clusters, n_features) */ -[[deprecated( - "Use cuVS instead")]] template +template void update_centroids(raft::resources const& handle, raft::device_matrix_view X, raft::device_vector_view sample_weights, @@ -432,7 +434,7 @@ void update_centroids(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -[[deprecated("Use cuVS instead")]] template +template void min_cluster_distance(raft::resources const& handle, raft::device_matrix_view X, raft::device_matrix_view centroids, @@ -482,7 +484,7 @@ void min_cluster_distance(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -[[deprecated("Use cuVS instead")]] template +template void min_cluster_and_distance( raft::resources const& handle, raft::device_matrix_view X, @@ -523,7 +525,7 @@ void min_cluster_and_distance( * @param[in] seed Seed for the shuffle * */ -[[deprecated("Use cuVS instead")]] template +template void shuffle_and_gather(raft::resources const& handle, raft::device_matrix_view in, raft::device_matrix_view out, @@ -552,7 +554,7 @@ void shuffle_and_gather(raft::resources const& handle, * [dim = n_cluster] * */ -[[deprecated("Use cuVS instead")]] template +template void count_samples_in_cluster(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -582,7 +584,7 @@ void count_samples_in_cluster(raft::resources const& handle, * [dim = n_cluster x n_features] * @param[in] workspace Temporary workspace buffer which can get resized */ -[[deprecated("Use cuVS instead")]] template +template void init_plus_plus(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -615,7 +617,7 @@ void init_plus_plus(raft::resources const& handle, * @param[out] n_iter Number of iterations run. * @param[in] workspace Temporary workspace buffer which can get resized */ -[[deprecated("Use cuVS instead")]] template +template void fit_main(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -662,7 +664,7 @@ namespace raft::cluster { * closest cluster center. * @param[out] n_iter Number of iterations run. */ -[[deprecated("Use cuVS instead")]] template +template void kmeans_fit(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -674,7 +676,7 @@ void kmeans_fit(raft::resources const& handle, kmeans::fit(handle, params, X, sample_weight, centroids, inertia, n_iter); } -[[deprecated("Use cuVS instead")]] template +template void kmeans_fit(raft::resources const& handle, const KMeansParams& params, const DataT* X, @@ -709,7 +711,7 @@ void kmeans_fit(raft::resources const& handle, * @param[out] inertia Sum of squared distances of samples to * their closest cluster center. */ -[[deprecated("Use cuVS instead")]] template +template void kmeans_predict(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -723,7 +725,7 @@ void kmeans_predict(raft::resources const& handle, handle, params, X, sample_weight, centroids, labels, normalize_weight, inertia); } -[[deprecated("Use cuVS instead")]] template +template void kmeans_predict(raft::resources const& handle, const KMeansParams& params, const DataT* X, @@ -774,7 +776,7 @@ void kmeans_predict(raft::resources const& handle, * closest cluster center. * @param[out] n_iter Number of iterations run. */ -[[deprecated("Use cuVS instead")]] template +template void kmeans_fit_predict(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -788,7 +790,7 @@ void kmeans_fit_predict(raft::resources const& handle, handle, params, X, sample_weight, centroids, labels, inertia, n_iter); } -[[deprecated("Use cuVS instead")]] template +template void kmeans_fit_predict(raft::resources const& handle, const KMeansParams& params, const DataT* X, @@ -819,7 +821,7 @@ void kmeans_fit_predict(raft::resources const& handle, * @param[out] X_new X transformed in the new space. * [dim = n_samples x n_features] */ -[[deprecated("Use cuVS instead")]] template +template void kmeans_transform(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -829,7 +831,7 @@ void kmeans_transform(raft::resources const& handle, kmeans::transform(handle, params, X, centroids, X_new); } -[[deprecated("Use cuVS instead")]] template +template void kmeans_transform(raft::resources const& handle, const KMeansParams& params, const DataT* X, @@ -866,7 +868,7 @@ using KeyValueIndexOp = kmeans::KeyValueIndexOp; * @param[in] workspace Temporary workspace buffer which can get resized * */ -[[deprecated("Use cuVS instead")]] template +template void sampleCentroids(raft::resources const& handle, raft::device_matrix_view X, raft::device_vector_view minClusterDistance, @@ -893,7 +895,7 @@ void sampleCentroids(raft::resources const& handle, * @param[in] reduction_op The reduction operation used for the cost * */ -[[deprecated("Use cuVS instead")]] template +template void computeClusterCost(raft::resources const& handle, raft::device_vector_view minClusterDistance, rmm::device_uvector& workspace, @@ -924,7 +926,7 @@ void computeClusterCost(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -[[deprecated("Use cuVS instead")]] template +template void minClusterDistanceCompute(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -970,7 +972,7 @@ void minClusterDistanceCompute(raft::resources const& handle, * @param[in] workspace Temporary workspace buffer which can get resized * */ -[[deprecated("Use cuVS instead")]] template +template void minClusterAndDistanceCompute( raft::resources const& handle, const KMeansParams& params, @@ -1009,7 +1011,7 @@ void minClusterAndDistanceCompute( * @param[in] seed Seed for the shuffle * */ -[[deprecated("Use cuVS instead")]] template +template void shuffleAndGather(raft::resources const& handle, raft::device_matrix_view in, raft::device_matrix_view out, @@ -1038,7 +1040,7 @@ void shuffleAndGather(raft::resources const& handle, * [dim = n_cluster] * */ -[[deprecated("Use cuVS instead")]] template +template void countSamplesInCluster(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -1069,7 +1071,7 @@ void countSamplesInCluster(raft::resources const& handle, * [dim = n_cluster x n_features] * @param[in] workspace Temporary workspace buffer which can get resized */ -[[deprecated("Use cuVS instead")]] template +template void kmeansPlusPlus(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, @@ -1102,7 +1104,7 @@ void kmeansPlusPlus(raft::resources const& handle, * @param[out] n_iter Number of iterations run. * @param[in] workspace Temporary workspace buffer which can get resized */ -[[deprecated("Use cuVS instead")]] template +template void kmeans_fit_main(raft::resources const& handle, const KMeansParams& params, raft::device_matrix_view X, diff --git a/cpp/include/raft/cluster/kmeans_balanced.cuh b/cpp/include/raft/cluster/kmeans_balanced.cuh index 45039dd8b8..7479047fce 100644 --- a/cpp/include/raft/cluster/kmeans_balanced.cuh +++ b/cpp/include/raft/cluster/kmeans_balanced.cuh @@ -72,15 +72,12 @@ namespace raft::cluster::kmeans_balanced { * @param[in] mapping_op (optional) Functor to convert from the input datatype to the arithmetic * datatype. If DataT == MathT, this must be the identity. */ -[[deprecated("Use cuVS instead")]] template -void fit(const raft::resources& handle, - kmeans_balanced_params const& params, - raft::device_matrix_view X, - raft::device_matrix_view centroids, - MappingOpT mapping_op = raft::identity_op()) +template +[[deprecated("Use cuVS instead")]] void fit(const raft::resources& handle, + kmeans_balanced_params const& params, + raft::device_matrix_view X, + raft::device_matrix_view centroids, + MappingOpT mapping_op = raft::identity_op()) { RAFT_EXPECTS(X.extent(1) == centroids.extent(1), "Number of features in dataset and centroids are different"); @@ -129,17 +126,18 @@ void fit(const raft::resources& handle, * @param[in] mapping_op (optional) Functor to convert from the input datatype to the arithmetic * datatype. If DataT == MathT, this must be the identity. */ -[[deprecated("Use cuVS instead")]] template -void predict(const raft::resources& handle, - kmeans_balanced_params const& params, - raft::device_matrix_view X, - raft::device_matrix_view centroids, - raft::device_vector_view labels, - MappingOpT mapping_op = raft::identity_op()) +template +[[deprecated("Use cuVS instead")]] void predict( + const raft::resources& handle, + kmeans_balanced_params const& params, + raft::device_matrix_view X, + raft::device_matrix_view centroids, + raft::device_vector_view labels, + MappingOpT mapping_op = raft::identity_op()) { RAFT_EXPECTS(X.extent(0) == labels.extent(0), "Number of rows in dataset and labels are different"); @@ -194,17 +192,18 @@ void predict(const raft::resources& handle, * @param[in] mapping_op (optional) Functor to convert from the input datatype to the arithmetic * datatype. If DataT and MathT are the same, this must be the identity. */ -[[deprecated("Use cuVS instead")]] template -void fit_predict(const raft::resources& handle, - kmeans_balanced_params const& params, - raft::device_matrix_view X, - raft::device_matrix_view centroids, - raft::device_vector_view labels, - MappingOpT mapping_op = raft::identity_op()) +template +[[deprecated("Use cuVS instead")]] void fit_predict( + const raft::resources& handle, + kmeans_balanced_params const& params, + raft::device_matrix_view X, + raft::device_matrix_view centroids, + raft::device_vector_view labels, + MappingOpT mapping_op = raft::identity_op()) { auto centroids_const = raft::make_device_matrix_view( centroids.data_handle(), centroids.extent(0), centroids.extent(1)); @@ -252,20 +251,21 @@ namespace helpers { * arithmetic datatype. If DataT == MathT, this must be the identity. * @param[in] X_norm (optional) Dataset's row norms [dim = n_samples] */ -[[deprecated("Use cuVS instead")]] template -void build_clusters(const raft::resources& handle, - const kmeans_balanced_params& params, - raft::device_matrix_view X, - raft::device_matrix_view centroids, - raft::device_vector_view labels, - raft::device_vector_view cluster_sizes, - MappingOpT mapping_op = raft::identity_op(), - std::optional> X_norm = std::nullopt) +template +[[deprecated("Use cuVS instead")]] void build_clusters( + const raft::resources& handle, + const kmeans_balanced_params& params, + raft::device_matrix_view X, + raft::device_matrix_view centroids, + raft::device_vector_view labels, + raft::device_vector_view cluster_sizes, + MappingOpT mapping_op = raft::identity_op(), + std::optional> X_norm = std::nullopt) { RAFT_EXPECTS(X.extent(0) == labels.extent(0), "Number of rows in dataset and labels are different"); @@ -331,19 +331,20 @@ void build_clusters(const raft::resources& handle, * @param[in] mapping_op (optional) Functor to convert from the input datatype to the * arithmetic datatype. If DataT == MathT, this must be the identity. */ -[[deprecated("Use cuVS instead")]] template -void calc_centers_and_sizes(const raft::resources& handle, - raft::device_matrix_view X, - raft::device_vector_view labels, - raft::device_matrix_view centroids, - raft::device_vector_view cluster_sizes, - bool reset_counters = true, - MappingOpT mapping_op = raft::identity_op()) +template +[[deprecated("Use cuVS instead")]] void calc_centers_and_sizes( + const raft::resources& handle, + raft::device_matrix_view X, + raft::device_vector_view labels, + raft::device_matrix_view centroids, + raft::device_vector_view cluster_sizes, + bool reset_counters = true, + MappingOpT mapping_op = raft::identity_op()) { RAFT_EXPECTS(X.extent(0) == labels.extent(0), "Number of rows in dataset and labels are different"); diff --git a/cpp/include/raft/cluster/single_linkage.cuh b/cpp/include/raft/cluster/single_linkage.cuh index 92cd413379..067445c542 100644 --- a/cpp/include/raft/cluster/single_linkage.cuh +++ b/cpp/include/raft/cluster/single_linkage.cuh @@ -47,17 +47,17 @@ namespace raft::cluster { * of k. The algorithm will set `k = log(n) + c` * @param[in] n_clusters number of clusters to assign data samples */ -[[deprecated("Use cuVS instead")]] template -void single_linkage(raft::resources const& handle, - const value_t* X, - size_t m, - size_t n, - raft::distance::DistanceType metric, - linkage_output* out, - int c, - size_t n_clusters) +template +[[deprecated("Use cuVS instead")]] void single_linkage(raft::resources const& handle, + const value_t* X, + size_t m, + size_t n, + raft::distance::DistanceType metric, + linkage_output* out, + int c, + size_t n_clusters) { detail::single_linkage( handle, X, m, n, metric, out, c, n_clusters); @@ -86,16 +86,15 @@ constexpr int DEFAULT_CONST_C = 15; * @param[in] c a constant used when constructing connectivities from knn graph. Allows the indirect control of k. The algorithm will set `k = log(n) + c` */ -[[deprecated("Use cuVS instead")]] template -void single_linkage(raft::resources const& handle, - raft::device_matrix_view X, - raft::device_matrix_view dendrogram, - raft::device_vector_view labels, - raft::distance::DistanceType metric, - size_t n_clusters, - std::optional c = std::make_optional(DEFAULT_CONST_C)) +template +[[deprecated("Use cuVS instead")]] void single_linkage( + raft::resources const& handle, + raft::device_matrix_view X, + raft::device_matrix_view dendrogram, + raft::device_vector_view labels, + raft::distance::DistanceType metric, + size_t n_clusters, + std::optional c = std::make_optional(DEFAULT_CONST_C)) { linkage_output out_arrs; out_arrs.children = dendrogram.data_handle(); diff --git a/cpp/include/raft/cluster/single_linkage_types.hpp b/cpp/include/raft/cluster/single_linkage_types.hpp index cd815622bf..1defc3df46 100644 --- a/cpp/include/raft/cluster/single_linkage_types.hpp +++ b/cpp/include/raft/cluster/single_linkage_types.hpp @@ -1,5 +1,5 @@ /* - * Copyright (c) 2021-2023, NVIDIA CORPORATION. + * Copyright (c) 2021-2024, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -23,7 +23,7 @@ namespace raft::cluster::hierarchy { /** * Determines the method for computing the minimum spanning tree (MST) */ -enum LinkageDistance { +[[deprecated("Use cuVS instead")]] enum LinkageDistance { /** * Use a pairwise distance matrix as input to the mst. This diff --git a/cpp/include/raft/neighbors/ball_cover_types.hpp b/cpp/include/raft/neighbors/ball_cover_types.hpp index 476dc2ea17..8a6d2e7e9e 100644 --- a/cpp/include/raft/neighbors/ball_cover_types.hpp +++ b/cpp/include/raft/neighbors/ball_cover_types.hpp @@ -42,17 +42,17 @@ namespace raft::neighbors::ball_cover { * @tparam value_t * @tparam value_int */ -[[deprecated("Use cuVS instead")]] template +template class BallCoverIndex { public: - explicit BallCoverIndex(raft::resources const& handle_, - const value_t* X_, - value_int m_, - value_int n_, - raft::distance::DistanceType metric_) + [[deprecated("Use cuVS instead")]] explicit BallCoverIndex(raft::resources const& handle_, + const value_t* X_, + value_int m_, + value_int n_, + raft::distance::DistanceType metric_) : handle(handle_), X(raft::make_device_matrix_view(X_, m_, n_)), m(m_), @@ -75,9 +75,10 @@ class BallCoverIndex { { } - explicit BallCoverIndex(raft::resources const& handle_, - raft::device_matrix_view X_, - raft::distance::DistanceType metric_) + [[deprecated("Use cuVS instead")]] explicit BallCoverIndex( + raft::resources const& handle_, + raft::device_matrix_view X_, + raft::distance::DistanceType metric_) : handle(handle_), X(X_), m(X_.extent(0)), diff --git a/cpp/include/raft/neighbors/brute_force_types.hpp b/cpp/include/raft/neighbors/brute_force_types.hpp index 4aafe374b7..4511f8d8ba 100644 --- a/cpp/include/raft/neighbors/brute_force_types.hpp +++ b/cpp/include/raft/neighbors/brute_force_types.hpp @@ -45,7 +45,7 @@ using ann::search_params; * * @tparam T data element type */ -[[deprecated("Use cuVS instead")]] template +template struct index : ann::index { public: /** Distance metric used for retrieval */ @@ -94,12 +94,14 @@ struct index : ann::index { * the dataset. If the dataset is in host memory, it will be copied to the device and the * index will own the device memory. */ + template - index(raft::resources const& res, - mdspan, row_major, data_accessor> dataset, - std::optional>&& norms, - raft::distance::DistanceType metric, - T metric_arg = 0.0) + [[deprecated("Use cuVS instead")]] index( + raft::resources const& res, + mdspan, row_major, data_accessor> dataset, + std::optional>&& norms, + raft::distance::DistanceType metric, + T metric_arg = 0.0) : ann::index(), metric_(metric), dataset_(make_device_matrix(res, 0, 0)), @@ -116,11 +118,12 @@ struct index : ann::index { * This class stores a non-owning reference to the dataset and norms here. * Having precomputed norms gives us a performance advantage at query time. */ - index(raft::resources const& res, - raft::device_matrix_view dataset_view, - std::optional> norms_view, - raft::distance::DistanceType metric, - T metric_arg = 0.0) + [[deprecated("Use cuVS instead")]] index( + raft::resources const& res, + raft::device_matrix_view dataset_view, + std::optional> norms_view, + raft::distance::DistanceType metric, + T metric_arg = 0.0) : ann::index(), metric_(metric), dataset_(make_device_matrix(res, 0, 0)), @@ -131,10 +134,11 @@ struct index : ann::index { } template - index(raft::resources const& res, - index_params const& params, - mdspan, row_major, data_accessor> dataset, - std::optional>&& norms = std::nullopt) + [[deprecated("Use cuVS instead")]] index( + raft::resources const& res, + index_params const& params, + mdspan, row_major, data_accessor> dataset, + std::optional>&& norms = std::nullopt) : ann::index(), metric_(params.metric), dataset_(make_device_matrix(res, 0, 0)), diff --git a/cpp/include/raft/neighbors/cagra_types.hpp b/cpp/include/raft/neighbors/cagra_types.hpp index ad017bc854..bc7c380db1 100644 --- a/cpp/include/raft/neighbors/cagra_types.hpp +++ b/cpp/include/raft/neighbors/cagra_types.hpp @@ -138,7 +138,7 @@ static_assert(std::is_aggregate_v); * @tparam IdxT type of the vector indices (represent dataset.extent(0)) * */ -[[deprecated("Use cuVS instead")]] template +template struct index : ann::index { static_assert(!raft::is_narrowing_v, "IdxT must be able to represent all values of uint32_t"); @@ -201,8 +201,9 @@ struct index : ann::index { ~index() = default; /** Construct an empty index. */ - index(raft::resources const& res, - raft::distance::DistanceType metric = raft::distance::DistanceType::L2Expanded) + [[deprecated("Use cuVS instead")]] index( + raft::resources const& res, + raft::distance::DistanceType metric = raft::distance::DistanceType::L2Expanded) : ann::index(), metric_(metric), graph_(make_device_matrix(res, 0, 0)), @@ -265,10 +266,11 @@ struct index : ann::index { * */ template - index(raft::resources const& res, - raft::distance::DistanceType metric, - mdspan, row_major, data_accessor> dataset, - mdspan, row_major, graph_accessor> knn_graph) + [[deprecated("Use cuVS instead")]] index( + raft::resources const& res, + raft::distance::DistanceType metric, + mdspan, row_major, data_accessor> dataset, + mdspan, row_major, graph_accessor> knn_graph) : ann::index(), metric_(metric), graph_(make_device_matrix(res, 0, 0)), diff --git a/cpp/include/raft/neighbors/epsilon_neighborhood.cuh b/cpp/include/raft/neighbors/epsilon_neighborhood.cuh index 9a2e82cf5e..7c66272051 100644 --- a/cpp/include/raft/neighbors/epsilon_neighborhood.cuh +++ b/cpp/include/raft/neighbors/epsilon_neighborhood.cuh @@ -45,16 +45,16 @@ namespace raft::neighbors::epsilon_neighborhood { * squared as we compute L2-squared distance in this method) * @param[in] stream cuda stream */ -[[deprecated("Use cuVS instead")]] template -void epsUnexpL2SqNeighborhood(bool* adj, - idx_t* vd, - const value_t* x, - const value_t* y, - idx_t m, - idx_t n, - idx_t k, - value_t eps, - cudaStream_t stream) +template +[[deprecated("Use cuVS instead")]] void epsUnexpL2SqNeighborhood(bool* adj, + idx_t* vd, + const value_t* x, + const value_t* y, + idx_t m, + idx_t n, + idx_t k, + value_t eps, + cudaStream_t stream) { spatial::knn::detail::epsUnexpL2SqNeighborhood( adj, vd, x, y, m, n, k, eps, stream); @@ -97,14 +97,14 @@ void epsUnexpL2SqNeighborhood(bool* adj, * @param[in] eps defines epsilon neighborhood radius (should be passed as * squared as we compute L2-squared distance in this method) */ -[[deprecated( - "Use cuVS instead")]] template -void eps_neighbors_l2sq(raft::resources const& handle, - raft::device_matrix_view x, - raft::device_matrix_view y, - raft::device_matrix_view adj, - raft::device_vector_view vd, - value_t eps) +template +[[deprecated("Use cuVS instead")]] void eps_neighbors_l2sq( + raft::resources const& handle, + raft::device_matrix_view x, + raft::device_matrix_view y, + raft::device_matrix_view adj, + raft::device_vector_view vd, + value_t eps) { epsUnexpL2SqNeighborhood(adj.data_handle(), vd.data_handle(), diff --git a/cpp/include/raft/neighbors/hnsw_types.hpp b/cpp/include/raft/neighbors/hnsw_types.hpp index 6718810e1d..f78571f491 100644 --- a/cpp/include/raft/neighbors/hnsw_types.hpp +++ b/cpp/include/raft/neighbors/hnsw_types.hpp @@ -38,7 +38,7 @@ struct search_params : ann::search_params { int num_threads = 0; // number of host threads to use for concurrent searches. Value of 0 // automatically maximizes parallelism }; -[[deprecated("Use cuVS instead")]] template +template struct index : ann::index { public: /** @@ -50,7 +50,10 @@ struct index : ann::index { * @param[in] dim dimensions of the training dataset * @param[in] metric distance metric to search. Supported metrics ("L2Expanded", "InnerProduct") */ - index(int dim, raft::distance::DistanceType metric) : dim_{dim}, metric_{metric} {} + [[deprecated("Use cuVS instead")]] index(int dim, raft::distance::DistanceType metric) + : dim_{dim}, metric_{metric} + { + } /** @brief Get underlying index diff --git a/cpp/include/raft/neighbors/ivf_flat_types.hpp b/cpp/include/raft/neighbors/ivf_flat_types.hpp index a2266cb9c8..2cafceb512 100644 --- a/cpp/include/raft/neighbors/ivf_flat_types.hpp +++ b/cpp/include/raft/neighbors/ivf_flat_types.hpp @@ -125,7 +125,7 @@ using list_data = ivf::list; * @tparam IdxT type of the indices in the source dataset * */ -[[deprecated("Use cuVS instead")]] template +template struct index : ann::index { static_assert(!raft::is_narrowing_v, "IdxT must be able to represent all values of uint32_t"); @@ -261,12 +261,12 @@ struct index : ann::index { ~index() = default; /** Construct an empty index. It needs to be trained and then populated. */ - index(raft::resources const& res, - raft::distance::DistanceType metric, - uint32_t n_lists, - bool adaptive_centers, - bool conservative_memory_allocation, - uint32_t dim) + [[deprecated("Use cuVS instead")]] index(raft::resources const& res, + raft::distance::DistanceType metric, + uint32_t n_lists, + bool adaptive_centers, + bool conservative_memory_allocation, + uint32_t dim) : ann::index(), veclen_(calculate_veclen(dim)), metric_(metric), @@ -285,7 +285,9 @@ struct index : ann::index { } /** Construct an empty index. It needs to be trained and then populated. */ - index(raft::resources const& res, const index_params& params, uint32_t dim) + [[deprecated("Use cuVS instead")]] index(raft::resources const& res, + const index_params& params, + uint32_t dim) : index(res, params.metric, params.n_lists, diff --git a/cpp/include/raft/neighbors/ivf_pq_types.hpp b/cpp/include/raft/neighbors/ivf_pq_types.hpp index 0ec75fb354..d5906d621c 100644 --- a/cpp/include/raft/neighbors/ivf_pq_types.hpp +++ b/cpp/include/raft/neighbors/ivf_pq_types.hpp @@ -289,7 +289,7 @@ using list_data = ivf::list; * @tparam IdxT type of the indices in the source dataset * */ -[[deprecated("Use cuVS instead")]] template +template struct index : ann::index { static_assert(!raft::is_narrowing_v, "IdxT must be able to represent all values of uint32_t"); @@ -361,14 +361,14 @@ struct index : ann::index { ~index() = default; /** Construct an empty index. It needs to be trained and then populated. */ - index(raft::resources const& handle, - raft::distance::DistanceType metric, - codebook_gen codebook_kind, - uint32_t n_lists, - uint32_t dim, - uint32_t pq_bits = 8, - uint32_t pq_dim = 0, - bool conservative_memory_allocation = false) + [[deprecated("Use cuVS instead")]] index(raft::resources const& handle, + raft::distance::DistanceType metric, + codebook_gen codebook_kind, + uint32_t n_lists, + uint32_t dim, + uint32_t pq_bits = 8, + uint32_t pq_dim = 0, + bool conservative_memory_allocation = false) : ann::index(), metric_(metric), codebook_kind_(codebook_kind), @@ -391,7 +391,9 @@ struct index : ann::index { } /** Construct an empty index. It needs to be trained and then populated. */ - index(raft::resources const& handle, const index_params& params, uint32_t dim) + [[deprecated("Use cuVS instead")]] index(raft::resources const& handle, + const index_params& params, + uint32_t dim) : index(handle, params.metric, params.codebook_kind, diff --git a/cpp/include/raft/neighbors/nn_descent_types.hpp b/cpp/include/raft/neighbors/nn_descent_types.hpp index 7bcb18166d..9decf47f39 100644 --- a/cpp/include/raft/neighbors/nn_descent_types.hpp +++ b/cpp/include/raft/neighbors/nn_descent_types.hpp @@ -86,7 +86,7 @@ struct index_params : ann::index_params { * * @tparam IdxT dtype to be used for constructing knn-graph */ -[[deprecated("Use cuVS instead")]] template +template struct index : ann::index { public: /** @@ -101,7 +101,10 @@ struct index : ann::index { * @param n_cols number of cols in knn-graph * @param return_distances whether to allocate and get distances information */ - index(raft::resources const& res, int64_t n_rows, int64_t n_cols, bool return_distances = false) + [[deprecated("Use cuVS instead")]] index(raft::resources const& res, + int64_t n_rows, + int64_t n_cols, + bool return_distances = false) : ann::index(), res_{res}, metric_{raft::distance::DistanceType::L2Expanded}, @@ -128,11 +131,12 @@ struct index : ann::index { * storing knn-graph distances * @param return_distances whether to allocate and get distances information */ - index(raft::resources const& res, - raft::host_matrix_view graph_view, - std::optional> distances_view = - std::nullopt, - bool return_distances = false) + [[deprecated("Use cuVS instead")]] index( + raft::resources const& res, + raft::host_matrix_view graph_view, + std::optional> distances_view = + std::nullopt, + bool return_distances = false) : ann::index(), res_{res}, metric_{raft::distance::DistanceType::L2Expanded}, diff --git a/cpp/include/raft/neighbors/refine-ext.cuh b/cpp/include/raft/neighbors/refine-ext.cuh index 580d807828..216e1b9ab5 100644 --- a/cpp/include/raft/neighbors/refine-ext.cuh +++ b/cpp/include/raft/neighbors/refine-ext.cuh @@ -28,31 +28,25 @@ namespace raft::neighbors { -[[deprecated("Use cuVS instead")]] template -void refine(raft::resources const& handle, - raft::device_matrix_view dataset, - raft::device_matrix_view queries, - raft::device_matrix_view neighbor_candidates, - raft::device_matrix_view indices, - raft::device_matrix_view distances, - raft::distance::DistanceType metric = distance::DistanceType::L2Unexpanded) - RAFT_EXPLICIT; +template +[[deprecated("Use cuVS instead")]] void refine( + raft::resources const& handle, + raft::device_matrix_view dataset, + raft::device_matrix_view queries, + raft::device_matrix_view neighbor_candidates, + raft::device_matrix_view indices, + raft::device_matrix_view distances, + raft::distance::DistanceType metric = distance::DistanceType::L2Unexpanded) RAFT_EXPLICIT; -[[deprecated("Use cuVS instead")]] template -void refine(raft::resources const& handle, - raft::host_matrix_view dataset, - raft::host_matrix_view queries, - raft::host_matrix_view neighbor_candidates, - raft::host_matrix_view indices, - raft::host_matrix_view distances, - raft::distance::DistanceType metric = distance::DistanceType::L2Unexpanded) - RAFT_EXPLICIT; +template +[[deprecated("Use cuVS instead")]] void refine( + raft::resources const& handle, + raft::host_matrix_view dataset, + raft::host_matrix_view queries, + raft::host_matrix_view neighbor_candidates, + raft::host_matrix_view indices, + raft::host_matrix_view distances, + raft::distance::DistanceType metric = distance::DistanceType::L2Unexpanded) RAFT_EXPLICIT; } // namespace raft::neighbors From a8b44f8517b11ba045628bcd1f22434eddd68ddc Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Tue, 24 Sep 2024 14:18:01 -0400 Subject: [PATCH 05/11] Deprecating distance APIs --- cpp/include/raft/distance/distance-ext.cuh | 133 +++++++++++---------- 1 file changed, 68 insertions(+), 65 deletions(-) diff --git a/cpp/include/raft/distance/distance-ext.cuh b/cpp/include/raft/distance/distance-ext.cuh index 2d41e029fe..dcbfbfdbc3 100644 --- a/cpp/include/raft/distance/distance-ext.cuh +++ b/cpp/include/raft/distance/distance-ext.cuh @@ -35,42 +35,43 @@ template -void distance(raft::resources const& handle, - const DataT* x, - const DataT* y, - OutT* dist, - IdxT m, - IdxT n, - IdxT k, - void* workspace, - size_t worksize, - FinalLambda fin_op, - bool isRowMajor = true, - DataT metric_arg = 2.0f) RAFT_EXPLICIT; +[[deprecated("Use cuVS instead")]] void distance(raft::resources const& handle, + const DataT* x, + const DataT* y, + OutT* dist, + IdxT m, + IdxT n, + IdxT k, + void* workspace, + size_t worksize, + FinalLambda fin_op, + bool isRowMajor = true, + DataT metric_arg = 2.0f) RAFT_EXPLICIT; template -void distance(raft::resources const& handle, - const DataT* x, - const DataT* y, - OutT* dist, - IdxT m, - IdxT n, - IdxT k, - void* workspace, - size_t worksize, - bool isRowMajor = true, - DataT metric_arg = 2.0f) RAFT_EXPLICIT; +[[deprecated("Use cuVS instead")]] void distance(raft::resources const& handle, + const DataT* x, + const DataT* y, + OutT* dist, + IdxT m, + IdxT n, + IdxT k, + void* workspace, + size_t worksize, + bool isRowMajor = true, + DataT metric_arg = 2.0f) RAFT_EXPLICIT; template -size_t getWorkspaceSize(const DataT* x, const DataT* y, IdxT m, IdxT n, IdxT k) RAFT_EXPLICIT; +[[deprecated("Use cuVS instead")]] size_t getWorkspaceSize( + const DataT* x, const DataT* y, IdxT m, IdxT n, IdxT k) RAFT_EXPLICIT; template -void distance(raft::resources const& handle, - const DataT* x, - const DataT* y, - OutT* dist, - IdxT m, - IdxT n, - IdxT k, - bool isRowMajor = true, - DataT metric_arg = 2.0f) RAFT_EXPLICIT; +[[deprecated("Use cuVS instead")]] void distance(raft::resources const& handle, + const DataT* x, + const DataT* y, + OutT* dist, + IdxT m, + IdxT n, + IdxT k, + bool isRowMajor = true, + DataT metric_arg = 2.0f) RAFT_EXPLICIT; template -void pairwise_distance(raft::resources const& handle, - const Type* x, - const Type* y, - Type* dist, - IdxT m, - IdxT n, - IdxT k, - rmm::device_uvector& workspace, - raft::distance::DistanceType metric, - bool isRowMajor = true, - Type metric_arg = 2.0f) RAFT_EXPLICIT; +[[deprecated("Use cuVS instead")]] void pairwise_distance(raft::resources const& handle, + const Type* x, + const Type* y, + Type* dist, + IdxT m, + IdxT n, + IdxT k, + rmm::device_uvector& workspace, + raft::distance::DistanceType metric, + bool isRowMajor = true, + Type metric_arg = 2.0f) RAFT_EXPLICIT; template -void pairwise_distance(raft::resources const& handle, - const Type* x, - const Type* y, - Type* dist, - IdxT m, - IdxT n, - IdxT k, - raft::distance::DistanceType metric, - bool isRowMajor = true, - Type metric_arg = 2.0f) RAFT_EXPLICIT; +[[deprecated("Use cuVS instead")]] void pairwise_distance(raft::resources const& handle, + const Type* x, + const Type* y, + Type* dist, + IdxT m, + IdxT n, + IdxT k, + raft::distance::DistanceType metric, + bool isRowMajor = true, + Type metric_arg = 2.0f) RAFT_EXPLICIT; template -void distance(raft::resources const& handle, - raft::device_matrix_view const x, - raft::device_matrix_view const y, - raft::device_matrix_view dist, - DataT metric_arg = 2.0f) RAFT_EXPLICIT; +[[deprecated("Use cuVS instead")]] void distance( + raft::resources const& handle, + raft::device_matrix_view const x, + raft::device_matrix_view const y, + raft::device_matrix_view dist, + DataT metric_arg = 2.0f) RAFT_EXPLICIT; template -void pairwise_distance(raft::resources const& handle, - device_matrix_view const x, - device_matrix_view const y, - device_matrix_view dist, - raft::distance::DistanceType metric, - Type metric_arg = 2.0f) RAFT_EXPLICIT; +[[deprecated("Use cuVS instead")]] void pairwise_distance( + raft::resources const& handle, + device_matrix_view const x, + device_matrix_view const y, + device_matrix_view dist, + raft::distance::DistanceType metric, + Type metric_arg = 2.0f) RAFT_EXPLICIT; }; // namespace distance }; // namespace raft From 4a98b5df951b76bcffa369e768a16c6041a0e2a6 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Thu, 26 Sep 2024 12:01:50 -0400 Subject: [PATCH 06/11] Removing deprecations from epsilon neighborhood prims --- .../raft/neighbors/epsilon_neighborhood.cuh | 33 +++++++++---------- 1 file changed, 16 insertions(+), 17 deletions(-) diff --git a/cpp/include/raft/neighbors/epsilon_neighborhood.cuh b/cpp/include/raft/neighbors/epsilon_neighborhood.cuh index 7c66272051..bade4385fb 100644 --- a/cpp/include/raft/neighbors/epsilon_neighborhood.cuh +++ b/cpp/include/raft/neighbors/epsilon_neighborhood.cuh @@ -1,5 +1,5 @@ /* - * Copyright (c) 2020-2024, NVIDIA CORPORATION. + * Copyright (c) 2020-2023, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -46,15 +46,15 @@ namespace raft::neighbors::epsilon_neighborhood { * @param[in] stream cuda stream */ template -[[deprecated("Use cuVS instead")]] void epsUnexpL2SqNeighborhood(bool* adj, - idx_t* vd, - const value_t* x, - const value_t* y, - idx_t m, - idx_t n, - idx_t k, - value_t eps, - cudaStream_t stream) +void epsUnexpL2SqNeighborhood(bool* adj, + idx_t* vd, + const value_t* x, + const value_t* y, + idx_t m, + idx_t n, + idx_t k, + value_t eps, + cudaStream_t stream) { spatial::knn::detail::epsUnexpL2SqNeighborhood( adj, vd, x, y, m, n, k, eps, stream); @@ -98,13 +98,12 @@ template * squared as we compute L2-squared distance in this method) */ template -[[deprecated("Use cuVS instead")]] void eps_neighbors_l2sq( - raft::resources const& handle, - raft::device_matrix_view x, - raft::device_matrix_view y, - raft::device_matrix_view adj, - raft::device_vector_view vd, - value_t eps) +void eps_neighbors_l2sq(raft::resources const& handle, + raft::device_matrix_view x, + raft::device_matrix_view y, + raft::device_matrix_view adj, + raft::device_vector_view vd, + value_t eps) { epsUnexpL2SqNeighborhood(adj.data_handle(), vd.data_handle(), From bafff63b2eae0249bf8e75d4d5d0edd8266b8e40 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Thu, 26 Sep 2024 12:05:51 -0400 Subject: [PATCH 07/11] Fixing docs accordingly --- docs/source/conf.py | 2 +- docs/source/raft_ann_benchmarks.md | 8 ++++++-- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index 8b2040baa2..7a287b689f 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -71,7 +71,7 @@ .. attention:: - The vector search and clustering algorithms in RAFT are being migrated to a new library dedicated to vector search called `cuVS `_. We will continue to support the vector search algorithms in RAFT during this move, but will no longer update them after the RAPIDS 24.06 (June) release. We plan to complete the migration by RAPIDS 24.08 (August) release. + The vector search and clustering algorithms in RAFT are being migrated to a new library dedicated to vector search called `cuVS `_. We will continue to support the vector search algorithms in RAFT during this move, but will no longer update them after the RAPIDS 24.06 (June) release. We plan to complete the migration by RAPIDS 24.10 (October) release and they will be removed from RAFT altogether in the 24.12 (December) release. """ diff --git a/docs/source/raft_ann_benchmarks.md b/docs/source/raft_ann_benchmarks.md index fc11a56ac8..addf6b5a72 100644 --- a/docs/source/raft_ann_benchmarks.md +++ b/docs/source/raft_ann_benchmarks.md @@ -2,6 +2,10 @@ This project provides a benchmark program for various ANN search implementations. It's especially suitable for comparing GPU implementations as well as comparing GPU against CPU. +> [!IMPORTANT] +> The vector search and clustering algorithms in RAFT are being migrated to a new library dedicated to vector search called [cuVS](https://github.com/rapidsai/cuvs). As a result, `raft-ann-bench` is being migrated to `cuvs-bench` and will be removed from RAFT altogether in the 24.12 (December) release. + + ## Table of Contents - [Installing the benchmarks](#installing-the-benchmarks) @@ -279,7 +283,7 @@ python -m raft_ann_bench.plot --dataset deep-image-96-inner Configuration files already exist for the following list of the million-scale datasets. Please refer to [ann-benchmarks datasets](https://github.com/erikbern/ann-benchmarks/#data-sets) for more information, including actual train and sizes. These all work out-of-the-box with the `--dataset` argument. Other million-scale datasets from `ann-benchmarks.com` will work, but will require a json configuration file to be created in `$CONDA_PREFIX/lib/python3.xx/site-packages/raft_ann_bench/run/conf`, or you can specify the `--configuration` option to use a specific file. -| Dataset Name | Train Rows | Columns | Test Rows | Distance | +**| Dataset Name | Train Rows | Columns | Test Rows | Distance | |-----|------------|----|----------------|------------| | `deep-image-96-angular` | 10M | 96 | 10K | Angular | | `fashion-mnist-784-euclidean` | 60K | 784 | 10K | Euclidean | @@ -287,7 +291,7 @@ Configuration files already exist for the following list of the million-scale da | `glove-100-angular` | 1.1M | 100 | 10K | Angular | | `mnist-784-euclidean` | 60K | 784 | 10K | Euclidean | | `nytimes-256-angular` | 290K | 256 | 10K | Angular | -| `sift-128-euclidean` | 1M | 128 | 10K | Euclidean| +| `sift-128-euclidean` | 1M | 128 | 10K | Euclidean|** All of the datasets above contain ground test datasets with 100 neighbors. Thus `k` for these datasets must be less than or equal to 100. From a24fc0f1d2580f2149d3651abd07804a74f9667b Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Thu, 26 Sep 2024 12:23:50 -0400 Subject: [PATCH 08/11] Removing deprecation from random ball cover --- cpp/include/raft/neighbors/ball_cover_types.hpp | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/cpp/include/raft/neighbors/ball_cover_types.hpp b/cpp/include/raft/neighbors/ball_cover_types.hpp index 8a6d2e7e9e..e3ea5f0005 100644 --- a/cpp/include/raft/neighbors/ball_cover_types.hpp +++ b/cpp/include/raft/neighbors/ball_cover_types.hpp @@ -48,11 +48,11 @@ template class BallCoverIndex { public: - [[deprecated("Use cuVS instead")]] explicit BallCoverIndex(raft::resources const& handle_, - const value_t* X_, - value_int m_, - value_int n_, - raft::distance::DistanceType metric_) + explicit BallCoverIndex(raft::resources const& handle_, + const value_t* X_, + value_int m_, + value_int n_, + raft::distance::DistanceType metric_) : handle(handle_), X(raft::make_device_matrix_view(X_, m_, n_)), m(m_), @@ -75,10 +75,9 @@ class BallCoverIndex { { } - [[deprecated("Use cuVS instead")]] explicit BallCoverIndex( - raft::resources const& handle_, - raft::device_matrix_view X_, - raft::distance::DistanceType metric_) + explicit BallCoverIndex(raft::resources const& handle_, + raft::device_matrix_view X_, + raft::distance::DistanceType metric_) : handle(handle_), X(X_), m(X_.extent(0)), From 8c98d17f9eaf255fdc747048721e0a6250740f22 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Thu, 26 Sep 2024 13:01:53 -0400 Subject: [PATCH 09/11] Fixing single linkage deprecations --- cpp/include/raft/cluster/single_linkage_types.hpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cpp/include/raft/cluster/single_linkage_types.hpp b/cpp/include/raft/cluster/single_linkage_types.hpp index 1defc3df46..cd815622bf 100644 --- a/cpp/include/raft/cluster/single_linkage_types.hpp +++ b/cpp/include/raft/cluster/single_linkage_types.hpp @@ -1,5 +1,5 @@ /* - * Copyright (c) 2021-2024, NVIDIA CORPORATION. + * Copyright (c) 2021-2023, NVIDIA CORPORATION. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -23,7 +23,7 @@ namespace raft::cluster::hierarchy { /** * Determines the method for computing the minimum spanning tree (MST) */ -[[deprecated("Use cuVS instead")]] enum LinkageDistance { +enum LinkageDistance { /** * Use a pairwise distance matrix as input to the mst. This From f776e220071937e06a3486808ce958cade1dc4e3 Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Thu, 26 Sep 2024 13:47:50 -0400 Subject: [PATCH 10/11] Fixing a typo --- docs/source/raft_ann_benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/raft_ann_benchmarks.md b/docs/source/raft_ann_benchmarks.md index addf6b5a72..1212cfb5c7 100644 --- a/docs/source/raft_ann_benchmarks.md +++ b/docs/source/raft_ann_benchmarks.md @@ -283,7 +283,7 @@ python -m raft_ann_bench.plot --dataset deep-image-96-inner Configuration files already exist for the following list of the million-scale datasets. Please refer to [ann-benchmarks datasets](https://github.com/erikbern/ann-benchmarks/#data-sets) for more information, including actual train and sizes. These all work out-of-the-box with the `--dataset` argument. Other million-scale datasets from `ann-benchmarks.com` will work, but will require a json configuration file to be created in `$CONDA_PREFIX/lib/python3.xx/site-packages/raft_ann_bench/run/conf`, or you can specify the `--configuration` option to use a specific file. -**| Dataset Name | Train Rows | Columns | Test Rows | Distance | +| Dataset Name | Train Rows | Columns | Test Rows | Distance | |-----|------------|----|----------------|------------| | `deep-image-96-angular` | 10M | 96 | 10K | Angular | | `fashion-mnist-784-euclidean` | 60K | 784 | 10K | Euclidean | From 8c0090be63a51648e1e581f3224c2dad305daeeb Mon Sep 17 00:00:00 2001 From: "Corey J. Nolet" Date: Thu, 26 Sep 2024 16:35:17 -0400 Subject: [PATCH 11/11] Doh! Removing rest of typo --- docs/source/raft_ann_benchmarks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/raft_ann_benchmarks.md b/docs/source/raft_ann_benchmarks.md index 1212cfb5c7..7bac2047fc 100644 --- a/docs/source/raft_ann_benchmarks.md +++ b/docs/source/raft_ann_benchmarks.md @@ -291,7 +291,7 @@ Configuration files already exist for the following list of the million-scale da | `glove-100-angular` | 1.1M | 100 | 10K | Angular | | `mnist-784-euclidean` | 60K | 784 | 10K | Euclidean | | `nytimes-256-angular` | 290K | 256 | 10K | Angular | -| `sift-128-euclidean` | 1M | 128 | 10K | Euclidean|** +| `sift-128-euclidean` | 1M | 128 | 10K | Euclidean| All of the datasets above contain ground test datasets with 100 neighbors. Thus `k` for these datasets must be less than or equal to 100.