Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Optimizations for cudf.concat when axis=1 #9333

Merged
merged 64 commits into from
Oct 19, 2021
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
826cd6c
add tests
galipremsagar Sep 21, 2021
50b8850
multiindex union
galipremsagar Sep 21, 2021
61b56cc
merge
galipremsagar Sep 21, 2021
910e682
add number of index apis
galipremsagar Sep 29, 2021
d252aae
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Sep 29, 2021
35406f6
cleanup
galipremsagar Sep 29, 2021
160093d
cleanup
galipremsagar Sep 29, 2021
9d5f7df
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Sep 29, 2021
60daeaf
cover all tests for mulitIndex.union
galipremsagar Sep 29, 2021
a43de79
add MultiIndex.intersections tests
galipremsagar Sep 29, 2021
28c13ff
add Index.union tests
galipremsagar Sep 29, 2021
d4c1ebd
add index intersection tests
galipremsagar Sep 29, 2021
ea32e41
remove print
galipremsagar Sep 29, 2021
205d947
add union docstring
galipremsagar Sep 29, 2021
e6f0ea5
add intersection docs
galipremsagar Sep 29, 2021
a943842
add docstrings
galipremsagar Sep 30, 2021
abecd07
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Sep 30, 2021
fcf2664
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Sep 30, 2021
c702396
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Sep 30, 2021
838a34c
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 1, 2021
aaab3a5
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 4, 2021
3adab76
add caching to distinct_count
galipremsagar Oct 5, 2021
a56dbfa
fix union
galipremsagar Oct 5, 2021
f7d9a8f
reorganize
galipremsagar Oct 5, 2021
5d57597
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 5, 2021
d897e1d
cleanup
galipremsagar Oct 5, 2021
a136bf1
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 7, 2021
7e26d25
add sort validation
galipremsagar Oct 7, 2021
1ef2b02
Apply suggestions from code review
galipremsagar Oct 7, 2021
c44b099
use BaseIndex
galipremsagar Oct 7, 2021
bf399db
Merge branch '9223' of https://github.com/galipremsagar/cudf into 9223
galipremsagar Oct 7, 2021
8d46320
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 7, 2021
53ef02b
cache end value
galipremsagar Oct 7, 2021
fcf5f6c
address reviews
galipremsagar Oct 8, 2021
dabb543
cleanup
galipremsagar Oct 8, 2021
346fe0e
simplify
galipremsagar Oct 8, 2021
29df86a
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 8, 2021
7a5301a
make cache value a dict
galipremsagar Oct 8, 2021
eab7194
make is_* methods specific to classes
galipremsagar Oct 8, 2021
340d2dc
make _extended_gcd free function
galipremsagar Oct 8, 2021
ce53913
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 8, 2021
6f389da
change Index call
galipremsagar Oct 8, 2021
04ac7fc
remove class name
galipremsagar Oct 11, 2021
838fe88
pass names in multiIndex construction itself
galipremsagar Oct 11, 2021
26564d3
remove comments
galipremsagar Oct 11, 2021
b4ea5a3
add todo
galipremsagar Oct 11, 2021
c20dfe6
add comment
galipremsagar Oct 11, 2021
fba6d78
add comment explaining materializations
galipremsagar Oct 11, 2021
ae25694
fix res_name
galipremsagar Oct 11, 2021
d595dc3
add comments
galipremsagar Oct 11, 2021
0873d39
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 11, 2021
92a9a40
add todo
galipremsagar Oct 11, 2021
49cabf9
Apply suggestions from code review
galipremsagar Oct 12, 2021
1a75299
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 12, 2021
5e1c7cc
style
galipremsagar Oct 12, 2021
0221386
remove paranthesis
galipremsagar Oct 12, 2021
897c25a
add todo
galipremsagar Oct 12, 2021
21f5a97
add more test coverage
galipremsagar Oct 12, 2021
975566d
refactor if/elif blocks
galipremsagar Oct 12, 2021
048ec0a
Update python/cudf/cudf/core/index.py
galipremsagar Oct 13, 2021
b783de0
address reviews
galipremsagar Oct 13, 2021
1365f36
merge
galipremsagar Oct 13, 2021
46d54fc
Merge remote-tracking branch 'upstream/branch-21.12' into 9223
galipremsagar Oct 13, 2021
47ce5d1
add tests for is_* methods
galipremsagar Oct 13, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion python/cudf/cudf/_lib/column.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

from __future__ import annotations

from typing import Optional, Tuple, TypeVar, Union
from typing import Dict, Optional, Tuple, TypeVar

from cudf._typing import Dtype, DtypeObj, ScalarLike
from cudf.core.buffer import Buffer
Expand All @@ -21,6 +21,7 @@ class Column:
_null_count: int
_children: Tuple[ColumnBase, ...]
_base_children: Tuple[ColumnBase, ...]
_distinct_count: Dict[bool, int]

def __init__(
self,
Expand Down
8 changes: 7 additions & 1 deletion python/cudf/cudf/_lib/column.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ cdef class Column:

self._size = size
self._cached_sizeof = None
self._distinct_count = None
galipremsagar marked this conversation as resolved.
Show resolved Hide resolved
self._dtype = dtype
self._offset = offset
self._null_count = null_count
Expand Down Expand Up @@ -203,9 +204,14 @@ cdef class Column:
raise ValueError(error_msg)

self._mask = None
self._null_count = None
galipremsagar marked this conversation as resolved.
Show resolved Hide resolved
self._children = None
self._base_mask = value
self._clear_cache()

def _clear_cache(self):
self._distinct_count = None
galipremsagar marked this conversation as resolved.
Show resolved Hide resolved
self._cached_sizeof = None
self._null_count = None

def set_mask(self, value):
"""
Expand Down
Loading