Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for writing deletion vectors in Delta Lake 2.4 #8674

Closed
wants to merge 19 commits into from
Closed

Add support for writing deletion vectors in Delta Lake 2.4 #8674

wants to merge 19 commits into from

Commits on Jul 8, 2023

  1. Add support for deletion vector writes for Delta Lake 2.4 

    Signed-off-by: Andy Grove <andygrove@nvidia.com>
    @andygrove
    andygrove committed Jul 8, 2023
    Configuration menu
    Copy the full SHA
    1221464 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2023

  1. Merge branch 'branch-23.08' into delta-lake-deletion-vector

    @andygrove
    andygrove committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    ae8ab26 View commit details
    Browse the repository at this point in the history

  2. refactor test

    @andygrove
    andygrove committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    9a55f40 View commit details
    Browse the repository at this point in the history

  3. improve test

    @andygrove
    andygrove committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    d428545 View commit details
    Browse the repository at this point in the history

  4. revert accidental variable rename

    @andygrove
    andygrove committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    082a092 View commit details
    Browse the repository at this point in the history

  5. fix db failure

    @razajafri
    razajafri committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    c72693c View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2023

  1. addressed review comments

    @razajafri
    razajafri committed Jul 18, 2023
    Configuration menu
    Copy the full SHA
    4296d88 View commit details
    Browse the repository at this point in the history

  2. signing off

    @razajafri
    razajafri committed Jul 18, 2023
    Configuration menu
    Copy the full SHA
    899de4f View commit details
    Browse the repository at this point in the history

  3. signing off 

    Signed-off-by: Raza Jafri <raza.jafri@gmail.com>
    @razajafri
    razajafri committed Jul 18, 2023
    Configuration menu
    Copy the full SHA
    b91ab52 View commit details
    Browse the repository at this point in the history

  4. Update integration_tests/src/main/python/parquet_write_test.py 

    Co-authored-by: Jason Lowe <jlowe@nvidia.com>
    @razajafri @jlowe
    razajafri and jlowe authored Jul 18, 2023
    Configuration menu
    Copy the full SHA
    817e853 View commit details
    Browse the repository at this point in the history

  5. Fall back to CPU for Parquet scans referencing internal Delta Lake me… 

    …tadata fields
    @andygrove
    andygrove committed Jul 18, 2023
    Configuration menu
    Copy the full SHA
    5d49c70 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2023

  1. fix for db 12.2

    @razajafri
    razajafri committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    47b8ff3 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.08' into delta-lake-deletion-vector

    @andygrove
    andygrove committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    aed7eb7 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'raza/SP-8726-db-fix' into delta-lake-de… 

    …letion-vector
    @andygrove
    andygrove committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    8466f28 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'nvidia/branch-23.08' into delta-lake-de… 

    …letion-vector
    @andygrove
    andygrove committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    19b5ea4 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2023

  1. Merge remote-tracking branch 'nvidia/branch-23.08' into delta-lake-de… 

    …letion-vector
    @andygrove
    andygrove committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    1f56bc3 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2023

  1. change test from xfail to skip for Databricks 12.2

    @andygrove
    andygrove committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    6f89535 View commit details
    Browse the repository at this point in the history

  2. skip test on earlier databricks versions

    @andygrove
    andygrove committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    f1ba32c View commit details
    Browse the repository at this point in the history

  3. adjust skip rules

    @andygrove
    andygrove committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    6609431 View commit details
    Browse the repository at this point in the history