-
Notifications
You must be signed in to change notification settings - Fork 68
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Align
extract_partitioning_index
logic with upstream shuffling (#60)
* update extract_partitioning_index with compat code Signed-off-by: rjzamora <rzamora217@gmail.com> * [Tutorials] Add a tutorial for PEFT data curation (#45) This PR adds a new tutorial to demonstrate data curation for PEFT use-cases. Signed-off-by: Mehran Maghoumi <Maghoumi@users.noreply.github.com> Signed-off-by: rjzamora <rzamora217@gmail.com> * move compat code to _compat file Signed-off-by: rjzamora <rzamora217@gmail.com> * Only import PII constants during Curator import (#61) * Move PII constants to a seperate file that does not import presidio/spacy and other GPU dependencies Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com> * Add comment around import, move constant import to global scope Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com> --------- Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com> * add unit test Signed-off-by: rjzamora <rzamora217@gmail.com> * add pytest.mark.gpu Signed-off-by: rjzamora <rzamora217@gmail.com> * move extract_partitioning_index import for now Signed-off-by: rjzamora <rzamora217@gmail.com> * test both cudf and pandas Signed-off-by: rjzamora <rzamora217@gmail.com> * spelling Signed-off-by: rjzamora <rzamora217@gmail.com> --------- Signed-off-by: rjzamora <rzamora217@gmail.com> Signed-off-by: Mehran Maghoumi <Maghoumi@users.noreply.github.com> Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com> Co-authored-by: Mehran Maghoumi <Maghoumi@users.noreply.github.com> Co-authored-by: Ayush Dattagupta <ayushdg95@gmail.com>
- Loading branch information
1 parent
38d8ce7
commit ecd4f4b
Showing
3 changed files
with
98 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters