Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Add transform_output_iterator and transform_input_output_iterator default constructors #1805

Merged

Conversation

harrism
Copy link
Contributor

@harrism harrism commented Oct 5, 2022

Fixes #1804

This PR adds default constructors to transform_output_iterator and transform_input_output_iterator. It also adds a unit test of transform_output_iterator with reduce_by_key which fails to compile before this change. With this change it compiles and runs correctly.

All other Thrust fancy iterators have default constructors, so I think this change improves consistency.

Sorry about all the formatting changes -- Thrust includes a .clang-format, but apparently doesn't use it to format files, so when I saved these files, clang-format reformatted them (I have format-on-save enabled). If this is a problem, I can resubmit without clang-formatting. To review just the functional changes, look for TestTransformOutputIteratorReduceByKey, and in the headers look for = default

Copy link
Collaborator

@miscco miscco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracking the bug down!

I have some minor comment about defaulting instead of defining but besides that it looks good

thrust/iterator/transform_input_output_iterator.h Outdated Show resolved Hide resolved
thrust/iterator/transform_output_iterator.h Outdated Show resolved Hide resolved
@miscco
Copy link
Collaborator

miscco commented Oct 5, 2022

run tests

@davidwendt
Copy link

I would recommend undoing the formatting if only to make it easier to create a patch for this change.

@miscco
Copy link
Collaborator

miscco commented Oct 5, 2022

I would recommend undoing the formatting if only to make it easier to create a patch for this change.

I believe that is fine, one can ignore whitespace changes in the github UI so the number of changes to review is rather small

@miscco
Copy link
Collaborator

miscco commented Oct 5, 2022

Something is strange here

09:59:32 CMake Error at /workspace/cmake/ThrustRunTest.cmake:7 (message):
09:59:32   /workspace/build/bin/thrust.cpp.tbb.cpp11.test.transform_output_iterator
09:59:32   failed (No such file or directory)

@miscco
Copy link
Collaborator

miscco commented Oct 5, 2022

Yeah it seesm that tbb is cranky:

testing/CMakeFiles/thrust.cpp.tbb.cpp11.test.transform_output_iterator.dir/thrust.cpp.tbb.cpp11/transform_output_iterator.cu.cpp.o -c /workspace/build/testing/thrust.cpp.tbb.cpp11/transform_output_iterator.cu.cpp
09:09:07 /workspace/thrust/cmake/../../thrust/system/tbb/detail/reduce_by_key.inl(329): error: function "thrust::plus<T>::operator() [with T=signed char]" cannot be called with the given argument list
09:09:07             argument types are: (thrust::detail::transform_output_iterator_proxy<thrust::negate<signed char>, thrust::detail::normal_iterator<thrust::device_ptr<signed char>>>, thrust::reference<signed char, thrust::pointer<signed char, thrust::system::tbb::detail::par_t, thrust::use_default, thrust::use_default>, thrust::use_default>)
09:09:07             object type is: thrust::plus<signed char>
09:09:07         values_result[output_idx] = binary_op(values_result[output_idx], carries[i]);
09:09:07                                     ^
09:09:07 /workspace/thrust/cmake/../../thrust/functional.h(231): note: this candidate was rejected because arguments do not match
09:09:07     constexpr T operator()(const T &lhs, const T &rhs) const
09:09:07                 ^
09:09:07           detected during:
09:09:07             instantiation of "thrust::pair<OutputIterator1, OutputIterator2> thrust::system::tbb::detail::reduce_by_key(thrust::system::tbb::detail::execution_policy<DerivedPolicy> &, InputIterator1, InputIterator1, InputIterator2, OutputIterator1, OutputIterator2, BinaryPredicate, BinaryFunction) [with DerivedPolicy=thrust::system::tbb::detail::par_t, InputIterator1=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>,
09:09:07                       InputIterator2=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>, OutputIterator1=thrust::discard_iterator<signed char>, OutputIterator2=thrust::transform_output_iterator<thrust::negate<signed char>, thrust::detail::normal_iterator<thrust::device_ptr<signed char>>>, BinaryPredicate=thrust::equal_to<signed char>, BinaryFunction=thrust::plus<signed char>]" at line 135 of "/workspace/thrust/cmake/../../thrust/detail/reduce.inl"
09:09:07             instantiation of "thrust::pair<OutputIterator1, OutputIterator2> thrust::reduce_by_key(const thrust::detail::execution_policy_base<DerivedPolicy> &, InputIterator1, InputIterator1, InputIterator2, OutputIterator1, OutputIterator2, BinaryPredicate, BinaryFunction) [with DerivedPolicy=thrust::system::tbb::detail::par_t, InputIterator1=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>, InputIterator2=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>,
09:09:07                       OutputIterator1=thrust::discard_iterator<signed char>, OutputIterator2=thrust::transform_output_iterator<thrust::negate<signed char>, thrust::detail::normal_iterator<thrust::device_ptr<signed char>>>, BinaryPredicate=thrust::equal_to<signed char>, BinaryFunction=thrust::plus<signed char>]" at line 185 of "/workspace/thrust/cmake/../../thrust/system/detail/generic/reduce_by_key.inl"
09:09:07             instantiation of "thrust::pair<OutputIterator1, OutputIterator2> thrust::system::detail::generic::reduce_by_key(thrust::execution_policy<DerivedPolicy> &, InputIterator1, InputIterator1, InputIterator2, OutputIterator1, OutputIterator2, BinaryPredicate) [with DerivedPolicy=thrust::system::tbb::detail::par_t, InputIterator1=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>, InputIterator2=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>,
09:09:07                       OutputIterator1=thrust::discard_iterator<signed char>, OutputIterator2=thrust::transform_output_iterator<thrust::negate<signed char>, thrust::detail::normal_iterator<thrust::device_ptr<signed char>>>, BinaryPredicate=thrust::equal_to<signed char>]" at line 111 of "/workspace/thrust/cmake/../../thrust/detail/reduce.inl"
09:09:07             instantiation of "thrust::pair<OutputIterator1, OutputIterator2> thrust::reduce_by_key(const thrust::detail::execution_policy_base<DerivedPolicy> &, InputIterator1, InputIterator1, InputIterator2, OutputIterator1, OutputIterator2, BinaryPredicate) [with DerivedPolicy=thrust::system::tbb::detail::par_t, InputIterator1=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>, InputIterator2=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>,
09:09:07                       OutputIterator1=thrust::discard_iterator<signed char>, OutputIterator2=thrust::transform_output_iterator<thrust::negate<signed char>, thrust::detail::normal_iterator<thrust::device_ptr<signed char>>>, BinaryPredicate=thrust::equal_to<signed char>]" at line 152 of "/workspace/thrust/cmake/../../thrust/system/detail/generic/reduce_by_key.inl"
09:09:07             instantiation of "thrust::pair<OutputIterator1, OutputIterator2> thrust::system::detail::generic::reduce_by_key(thrust::execution_policy<DerivedPolicy> &, InputIterator1, InputIterator1, InputIterator2, OutputIterator1, OutputIterator2) [with DerivedPolicy=thrust::system::tbb::detail::par_t, InputIterator1=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>, InputIterator2=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>,
09:09:07                       OutputIterator1=thrust::discard_iterator<signed char>, OutputIterator2=thrust::transform_output_iterator<thrust::negate<signed char>, thrust::detail::normal_iterator<thrust::device_ptr<signed char>>>]" at line 89 of "/workspace/thrust/cmake/../../thrust/detail/reduce.inl"
09:09:07             instantiation of "thrust::pair<OutputIterator1, OutputIterator2> thrust::reduce_by_key(const thrust::detail::execution_policy_base<DerivedPolicy> &, InputIterator1, InputIterator1, InputIterator2, OutputIterator1, OutputIterator2) [with DerivedPolicy=thrust::system::tbb::detail::par_t, InputIterator1=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>, InputIterator2=thrust::detail::normal_iterator<thrust::device_ptr<signed char>>,
09:09:07                       OutputIterator1=thrust::discard_iterator<signed char>, OutputIterator2=thrust::transform_output_iterator<thrust::negate<signed char>, thrust::detail::normal_iterator<thrust::device_ptr<signed char>>>]" at line 127 of "/workspace/testing/transform_output_iterator.cu"
09:09:07             instantiation of "void TestTransformOutputIteratorReduceByKey<T>::operator()(size_t={unsigned long}) [with T=signed char]" at line 56 of "/workspace/testing/unittest/meta.h"
09:09:07             instantiation of "void unittest::for_each_type<TypeList, Function, T, i>::operator()(U) [with TypeList=SignedIntegralTypes, Function=TestTransformOutputIteratorReduceByKey, T=signed char, i=0U, U=size_t={unsigned long}]" at line 567 of "/workspace/testing/unittest/testframework.h"
09:09:07             instantiation of "void VariableUnitTest<TestName, TypeList>::run() [with TestName=TestTransformOutputIteratorReduceByKey, TypeList=SignedIntegralTypes]" 
09:09:07 

09:09:07 /workspace/thrust/cmake/../../thrust/system/tbb/detail/reduce_by_key.inl(329): error: function "thrust::plus<T>::operator() [with T=short]" cannot be called with the given argument list
09:09:07             argument types are: (thrust::detail::transform_output_iterator_proxy<thrust::negate<signed short>, thrust::detail::normal_iterator<thrust::device_ptr<short>>>, thrust::reference<short, thrust::pointer<short, thrust::system::tbb::detail::par_t, thrust::use_default, thrust::use_default>, thrust::use_default>)
09:09:07             object type is: thrust::plus<short>
09:09:07         values_result[output_idx] = binary_op(values_result[output_idx], carries[i]);

@harrism
Copy link
Contributor Author

harrism commented Oct 5, 2022

I have no idea how to investigate TBB failures. Where do I start? How do I build with TBB locally?

@harrism
Copy link
Contributor Author

harrism commented Oct 11, 2022

I removed the clang-format formatting changes so now the changes are only the functional ones. I still don't know how to solve the TBB failures -- I think my new test may have exposed an existing problem with the TBB backend...

@harrism
Copy link
Contributor Author

harrism commented Oct 11, 2022

OK I split the new test into its own source file and disabled it on TBB for now. I filed NVIDIA/cccl#821 to report the TBB backend bug.

@griwes
Copy link
Collaborator

griwes commented Oct 11, 2022

run tests

@harrism
Copy link
Contributor Author

harrism commented Oct 11, 2022

Cool, we're passing now.

@miscco
Copy link
Collaborator

miscco commented Oct 11, 2022

@wmaxey AFAIK we already have the 2.0.X branch, so I believe we should be able to merge this to main?

rapids-bot bot pushed a commit to rapidsai/cudf that referenced this pull request Oct 13, 2022
Adds fix from NVIDIA/thrust#1805 to libcudf's `thrust.patch`

Authors:
  - David Wendt (https://github.com/davidwendt)

Approvers:
  - Mark Harris (https://github.com/harrism)
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #11900
robertmaynard added a commit to robertmaynard/rapids-cmake that referenced this pull request Oct 13, 2022
rapids-bot bot pushed a commit to rapidsai/rapids-cmake that referenced this pull request Oct 14, 2022
Backports  NVIDIA/thrust#1805 to thrust 1.17.2 since it looks to be slated for thrust 2.1 . Adding the patch to ensure that all RAPIDS projects won't be hit by this issue

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #291
@miscco
Copy link
Collaborator

miscco commented Nov 1, 2022

@senior-zero can we merge this?

@gevtushenko
Copy link
Collaborator

It seems that I've fixed the initial issue with reduce by key here. In any way, having a defaulted default constructor wouldn't hurt. As I understand, the defaulted default constructors will be deleted if subobjects (output iterator and invocable) are not default constructible.

@gevtushenko gevtushenko merged commit fb758b8 into NVIDIA:main Dec 2, 2022
github-actions bot pushed a commit that referenced this pull request Dec 2, 2022
…form_output_iterator-default-ctor

Add transform_output_iterator and transform_input_output_iterator default constructors fb758b8
github-actions bot pushed a commit to gevtushenko/thrust that referenced this pull request Dec 2, 2022
…-transform_output_iterator-default-ctor

Add transform_output_iterator and transform_input_output_iterator default constructors fb758b8
github-actions bot pushed a commit to clayne/thrust that referenced this pull request Dec 2, 2022
…-transform_output_iterator-default-ctor

Add transform_output_iterator and transform_input_output_iterator default constructors fb758b8
@alliepiper alliepiper added this to the 2.1.0 milestone Mar 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Archived in project
6 participants