Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extract parquet statistics to its own module, add tests #8294

Merged
merged 21 commits into from
Nov 29, 2023
Merged

Extract parquet statistics to its own module, add tests #8294

merged 21 commits into from
Nov 29, 2023

Commits on Nov 21, 2023

  1. Extract parquet statistics to its own module, add tests

    @alamb
    alamb committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    d187e36 View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. Merge remote-tracking branch 'apache/main' into alamb/extract_parquet… 

    …_statistics
    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    7512c8b View commit details
    Browse the repository at this point in the history

  2. Update datafusion/core/src/datasource/physical_plan/parquet/statistic… 

    …s.rs

Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
    @alamb @tustvold
    alamb and tustvold authored Nov 27, 2023
    Configuration menu
    Copy the full SHA
    d4e660a View commit details
    Browse the repository at this point in the history

  3. rename enum

    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    fd2aebc View commit details
    Browse the repository at this point in the history

  4. Merge branch 'alamb/extract_parquet_statistics' of github.com:alamb/a… 

    …rrow-datafusion into alamb/extract_parquet_statistics
    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    96a42f9 View commit details
    Browse the repository at this point in the history

  5. Improve API

    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    a128a20 View commit details
    Browse the repository at this point in the history

  6. Add test for reading struct array statistics

    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    b4009c2 View commit details
    Browse the repository at this point in the history

  7. Add test for column after statistics

    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    ef79c42 View commit details
    Browse the repository at this point in the history

  8. improve tests

    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    9b914db View commit details
    Browse the repository at this point in the history

  9. simplify

    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    b95dea9 View commit details
    Browse the repository at this point in the history

  10. clippy

    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    cd3c042 View commit details
    Browse the repository at this point in the history

  11. Update datafusion/core/src/datasource/physical_plan/parquet/statistic… 

    …s.rs
    @alamb
    alamb authored Nov 27, 2023
    Configuration menu
    Copy the full SHA
    ab95453 View commit details
    Browse the repository at this point in the history

  12. Update datafusion/core/src/datasource/physical_plan/parquet/statistic… 

    …s.rs
    @alamb
    alamb authored Nov 27, 2023
    Configuration menu
    Copy the full SHA
    0235a9e View commit details
    Browse the repository at this point in the history

  13. Add test showing incorrect statistics

    @alamb
    alamb committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    a601fbf View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. Merge remote-tracking branch 'upstream/main' into alamb/extract_parqu… 

    …et_statistics
    @tustvold
    tustvold committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    5c55302 View commit details
    Browse the repository at this point in the history

  2. Rework statistics

    @tustvold
    tustvold committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    06b5201 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #16 from tustvold/tustvold/extract_parquet_statistics 

    Tustvold/extract parquet statistics
    @alamb
    alamb authored Nov 28, 2023
    Configuration menu
    Copy the full SHA
    641142b View commit details
    Browse the repository at this point in the history

  4. Fix clippy

    @alamb
    alamb committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    e5cd8cf View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'apache/main' into alamb/extract_parquet… 

    …_statistics
    @alamb
    alamb committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    b1666c2 View commit details
    Browse the repository at this point in the history

  6. Update documentation and make it clear the statistics are not publica… 

    …lly accessable
    @alamb
    alamb committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    7022691 View commit details
    Browse the repository at this point in the history

  7. Add link to upstream arrow ticket

    @alamb
    alamb committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    a5e235a View commit details
    Browse the repository at this point in the history