[BUG] cudf concat / append must not materialize contiguous rangeIndex objects #6872
Assignees
Milestone
Comments
galipremsagar
added
bug
|
This issue has been marked stale due to no recent activity in the past 30d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be marked rotten if there is no activity in the next 60d. |
|
This issue has been labeled |
rapids-bot bot
pushed a commit
that referenced
this issue
Aug 5, 2021
Fixes: #6872 In cudf, we have been concatenating a collection of `RangeIndex`'s by materializing each one of them, but instead we should rather be introspecting each RangeIndex to decide whether to materialize of not. This PR fixes it. <!-- Thank you for contributing to cuDF :) Here are some guidelines to help the review process go smoothly. 1. Please write a description in this text box of the changes that are being made. 2. Please ensure that you have written units tests for the changes made/features added. 3. There are CI checks in place to enforce that committed code follows our style and syntax standards. Please see our contribution guide in `CONTRIBUTING.MD` in the project root for more information about the checks we perform and how you can run them locally. 4. If you are closing an issue please use one of the automatic closing words as noted here: https://help.github.com/articles/closing-issues-using-keywords/ 5. If your pull request is not ready for review but you want to make use of the continuous integration testing facilities please mark your pull request as Draft. https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-stage-of-a-pull-request#converting-a-pull-request-to-a-draft 6. If your pull request is ready to be reviewed without requiring additional work on top of it, then remove it from "Draft" and make it "Ready for Review". https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-stage-of-a-pull-request#marking-a-pull-request-as-ready-for-review If assistance is required to complete the functionality, for example when the C/C++ code of a feature is complete but Python bindings are still required, then add the label `help wanted` so that others can triage and assist. The additional changes then can be implemented on top of the same PR. If the assistance is done by members of the rapidsAI team, then no additional actions are required by the creator of the original PR for this, otherwise the original author of the PR needs to give permission to the person(s) assisting to commit to their personal fork of the project. If that doesn't happen then a new PR based on the code of the original PR can be opened by the person assisting, which then will be the PR that will be merged. 7. Once all work has been done and review has taken place please do not add features or make changes out of the scope of those requested by the reviewer (doing this just add delays as already reviewed code ends up having to be re-reviewed/it is hard to tell what is new etc!). Further, please do not rebase your branch on the target branch, force push, or rewrite history. Doing any of these causes the context of any comments made by reviewers to be lost. If conflicts occur against the target branch they should be resolved by merging the target branch into the branch used for making the pull request. Many thanks in advance for your cooperation! --> Authors: - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - Benjamin Zaitlen (https://github.com/quasiben) - Vyas Ramasubramani (https://github.com/vyasr) URL: #8970
galipremsagar
added a commit
to galipremsagar/cudf
that referenced
this issue
Aug 6, 2021
Fixes: rapidsai#6872 In cudf, we have been concatenating a collection of `RangeIndex`'s by materializing each one of them, but instead we should rather be introspecting each RangeIndex to decide whether to materialize of not. This PR fixes it. <!-- Thank you for contributing to cuDF :) Here are some guidelines to help the review process go smoothly. 1. Please write a description in this text box of the changes that are being made. 2. Please ensure that you have written units tests for the changes made/features added. 3. There are CI checks in place to enforce that committed code follows our style and syntax standards. Please see our contribution guide in `CONTRIBUTING.MD` in the project root for more information about the checks we perform and how you can run them locally. 4. If you are closing an issue please use one of the automatic closing words as noted here: https://help.github.com/articles/closing-issues-using-keywords/ 5. If your pull request is not ready for review but you want to make use of the continuous integration testing facilities please mark your pull request as Draft. https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-stage-of-a-pull-request#converting-a-pull-request-to-a-draft 6. If your pull request is ready to be reviewed without requiring additional work on top of it, then remove it from "Draft" and make it "Ready for Review". https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-stage-of-a-pull-request#marking-a-pull-request-as-ready-for-review If assistance is required to complete the functionality, for example when the C/C++ code of a feature is complete but Python bindings are still required, then add the label `help wanted` so that others can triage and assist. The additional changes then can be implemented on top of the same PR. If the assistance is done by members of the rapidsAI team, then no additional actions are required by the creator of the original PR for this, otherwise the original author of the PR needs to give permission to the person(s) assisting to commit to their personal fork of the project. If that doesn't happen then a new PR based on the code of the original PR can be opened by the person assisting, which then will be the PR that will be merged. 7. Once all work has been done and review has taken place please do not add features or make changes out of the scope of those requested by the reviewer (doing this just add delays as already reviewed code ends up having to be re-reviewed/it is hard to tell what is new etc!). Further, please do not rebase your branch on the target branch, force push, or rewrite history. Doing any of these causes the context of any comments made by reviewers to be lost. If conflicts occur against the target branch they should be resolved by merging the target branch into the branch used for making the pull request. Many thanks in advance for your cooperation! --> Authors: - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - Benjamin Zaitlen (https://github.com/quasiben) - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#8970
shwina
pushed a commit
to shwina/cudf
that referenced
this issue
Aug 9, 2021
Fixes: rapidsai#6872 In cudf, we have been concatenating a collection of `RangeIndex`'s by materializing each one of them, but instead we should rather be introspecting each RangeIndex to decide whether to materialize of not. This PR fixes it. <!-- Thank you for contributing to cuDF :) Here are some guidelines to help the review process go smoothly. 1. Please write a description in this text box of the changes that are being made. 2. Please ensure that you have written units tests for the changes made/features added. 3. There are CI checks in place to enforce that committed code follows our style and syntax standards. Please see our contribution guide in `CONTRIBUTING.MD` in the project root for more information about the checks we perform and how you can run them locally. 4. If you are closing an issue please use one of the automatic closing words as noted here: https://help.github.com/articles/closing-issues-using-keywords/ 5. If your pull request is not ready for review but you want to make use of the continuous integration testing facilities please mark your pull request as Draft. https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-stage-of-a-pull-request#converting-a-pull-request-to-a-draft 6. If your pull request is ready to be reviewed without requiring additional work on top of it, then remove it from "Draft" and make it "Ready for Review". https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/changing-the-stage-of-a-pull-request#marking-a-pull-request-as-ready-for-review If assistance is required to complete the functionality, for example when the C/C++ code of a feature is complete but Python bindings are still required, then add the label `help wanted` so that others can triage and assist. The additional changes then can be implemented on top of the same PR. If the assistance is done by members of the rapidsAI team, then no additional actions are required by the creator of the original PR for this, otherwise the original author of the PR needs to give permission to the person(s) assisting to commit to their personal fork of the project. If that doesn't happen then a new PR based on the code of the original PR can be opened by the person assisting, which then will be the PR that will be merged. 7. Once all work has been done and review has taken place please do not add features or make changes out of the scope of those requested by the reviewer (doing this just add delays as already reviewed code ends up having to be re-reviewed/it is hard to tell what is new etc!). Further, please do not rebase your branch on the target branch, force push, or rewrite history. Doing any of these causes the context of any comments made by reviewers to be lost. If conflicts occur against the target branch they should be resolved by merging the target branch into the branch used for making the pull request. Many thanks in advance for your cooperation! --> Authors: - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - Benjamin Zaitlen (https://github.com/quasiben) - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#8970
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
While we try appending/concatenating multiple RangeIndex objects if they are contiguous we should not be materializing RangeIndex objects.
Steps/Code to reproduce bug
Expected behavior
We should avoid materializing RangeIndex if there are contiguous RangeIndex inputs.
Environment overview (please complete the following information)
Environment details
Please run and paste the output of the
cudf/print_env.shscript here, to gather any other relevant environment detailsClick here to see environment details
Additional context
This would help avoid performance issues while appending contiguous rangeIndex objects in : :#6771
The text was updated successfully, but these errors were encountered: