Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Align extract_partitioning_index logic with upstream shuffling #60

Merged
merged 10 commits into from
May 15, 2024
Merged

Align extract_partitioning_index logic with upstream shuffling #60

merged 10 commits into from
May 15, 2024

Commits on May 13, 2024

  1. update extract_partitioning_index with compat code 

    Signed-off-by: rjzamora <rzamora217@gmail.com>
    @rjzamora
    rjzamora committed May 13, 2024
    Configuration menu
    Copy the full SHA
    68af3f9 View commit details
    Browse the repository at this point in the history

  2. [Tutorials] Add a tutorial for PEFT data curation (NVIDIA#45) 

    This PR adds a new tutorial to demonstrate data curation for PEFT
use-cases.

Signed-off-by: Mehran Maghoumi <Maghoumi@users.noreply.github.com>
Signed-off-by: rjzamora <rzamora217@gmail.com>
    @Maghoumi @rjzamora
    Maghoumi authored and rjzamora committed May 13, 2024
    Configuration menu
    Copy the full SHA
    644739b View commit details
    Browse the repository at this point in the history

  3. move compat code to _compat file 

    Signed-off-by: rjzamora <rzamora217@gmail.com>
    @rjzamora
    rjzamora committed May 13, 2024
    Configuration menu
    Copy the full SHA
    1f28a35 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. Only import PII constants during Curator import (NVIDIA#61) 

    * Move PII constants to a seperate file that does not import presidio/spacy and other GPU dependencies

Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>

* Add comment around import, move constant import to global scope

Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>

---------

Signed-off-by: Ayush Dattagupta <ayushdg95@gmail.com>
    @ayushdg @rjzamora
    ayushdg authored and rjzamora committed May 14, 2024
    Configuration menu
    Copy the full SHA
    2f5678b View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/main' into revise-extract_part… 

    …itioning_index
    @rjzamora
    rjzamora committed May 14, 2024
    Configuration menu
    Copy the full SHA
    33064e8 View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. add unit test 

    Signed-off-by: rjzamora <rzamora217@gmail.com>
    @rjzamora
    rjzamora committed May 15, 2024
    Configuration menu
    Copy the full SHA
    5da12ce View commit details
    Browse the repository at this point in the history

  2. add pytest.mark.gpu 

    Signed-off-by: rjzamora <rzamora217@gmail.com>
    @rjzamora
    rjzamora committed May 15, 2024
    Configuration menu
    Copy the full SHA
    25c1eb2 View commit details
    Browse the repository at this point in the history

  3. move extract_partitioning_index import for now 

    Signed-off-by: rjzamora <rzamora217@gmail.com>
    @rjzamora
    rjzamora committed May 15, 2024
    Configuration menu
    Copy the full SHA
    647406f View commit details
    Browse the repository at this point in the history

  4. test both cudf and pandas 

    Signed-off-by: rjzamora <rzamora217@gmail.com>
    @rjzamora
    rjzamora committed May 15, 2024
    Configuration menu
    Copy the full SHA
    b48a37c View commit details
    Browse the repository at this point in the history

  5. spelling 

    Signed-off-by: rjzamora <rzamora217@gmail.com>
    @rjzamora
    rjzamora committed May 15, 2024
    Configuration menu
    Copy the full SHA
    9ef4aa3 View commit details
    Browse the repository at this point in the history