Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Profile/qualification tool error handling improvements and support spark < 3.1.1 #2604

Merged
merged 47 commits into from
Jun 6, 2021
Merged

Profile/qualification tool error handling improvements and support spark < 3.1.1 #2604

merged 47 commits into from
Jun 6, 2021

Commits on Jun 3, 2021

  1. Qualification tool 

    Signed-off-by: Thomas Graves <tgraves@apache.org>
    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    9ffe53e View commit details
    Browse the repository at this point in the history

  2. remove unused func

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    56c8cc2 View commit details
    Browse the repository at this point in the history

  3. Add missing files

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    ef5cf4c View commit details
    Browse the repository at this point in the history

  4. Add checks for format option

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    6f5271c View commit details
    Browse the repository at this point in the history

  5. cast columsn to string to write to text

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    81dc6a4 View commit details
    Browse the repository at this point in the history

  6. Revert "Add checks for format option" 

    This reverts commit 6f5271c.
    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    ee8c1dd View commit details
    Browse the repository at this point in the history

  7. cleanup 

    Signed-off-by: Thomas Graves <tgraves@nvidia.com>
    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    bc0acd1 View commit details
    Browse the repository at this point in the history

  8. update output dir

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    e9dc1ce View commit details
    Browse the repository at this point in the history

  9. formating

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    aa298b2 View commit details
    Browse the repository at this point in the history

  10. Update help messages

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    f3f35f4 View commit details
    Browse the repository at this point in the history

  11. update app name

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    10f6678 View commit details
    Browse the repository at this point in the history

  12. cleanup

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    9e0d104 View commit details
    Browse the repository at this point in the history

  13. put test functions back

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    50b4c29 View commit details
    Browse the repository at this point in the history

  14. fix typo

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    f2162f1 View commit details
    Browse the repository at this point in the history

  15. add printSQLPlanMetrics and printRapidsJar

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    37b7225 View commit details
    Browse the repository at this point in the history

  16. use opt

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    208baf4 View commit details
    Browse the repository at this point in the history

  17. Add Analysis

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    30bf946 View commit details
    Browse the repository at this point in the history

  18. format output

    @tgravescs
    tgravescs committed Jun 3, 2021
    Configuration menu
    Copy the full SHA
    e04e926 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2021

  1. Merge remote-tracking branch 'origin/branch-21.06' into profilingToolImp

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    10cfc28 View commit details
    Browse the repository at this point in the history

  2. more tests 

    Signed-off-by: Thomas Graves <tgraves@apache.org>
    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    19d804f View commit details
    Browse the repository at this point in the history

  3. tests working

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    a4e6095 View commit details
    Browse the repository at this point in the history

  4. test rearrange utils

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    21fc1e6 View commit details
    Browse the repository at this point in the history

  5. move test file

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    3fd1864 View commit details
    Browse the repository at this point in the history

  6. move test file right location

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    2507bd5 View commit details
    Browse the repository at this point in the history

  7. add Analysis Suite

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    d3418b9 View commit details
    Browse the repository at this point in the history

  8. update test analysis

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    bd3f4d9 View commit details
    Browse the repository at this point in the history

  9. add 

    rapids-4-spark-tools/src/test/resources/ProfilingExpectations/rapids_join_eventlog_jobandstagemetrics_expectation.csv
    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    065ff96 View commit details
    Browse the repository at this point in the history

  10. add more tests

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    ec845ee View commit details
    Browse the repository at this point in the history

  11. more tests 

    Signed-off-by: Thomas Graves <tgraves@nvidia.com>
    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    66a4e58 View commit details
    Browse the repository at this point in the history

  12. remove unneeded expectation file

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    08c8f4d View commit details
    Browse the repository at this point in the history

  13. Add more analysis tests

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    97cd725 View commit details
    Browse the repository at this point in the history

  14. comment

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    3fdeb2a View commit details
    Browse the repository at this point in the history

  15. cleanup

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    3308e65 View commit details
    Browse the repository at this point in the history

  16. Start handling ResourceProfile with reflection 

    Signed-off-by: Thomas Graves <tgraves@nvidia.com>
    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    6859bf5 View commit details
    Browse the repository at this point in the history

  17. allow spark 3.0 and 3.1.1 to parse logs and fix bug with missing table

    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    341d998 View commit details
    Browse the repository at this point in the history

  18. Merge remote-tracking branch 'origin/branch-21.06' into profilefixspa… 

    …rkcompat
    @tgravescs
    tgravescs committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    b6ad6f5 View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2021

  1. Add more test files

    @tgravescs
    tgravescs committed Jun 5, 2021
    Configuration menu
    Copy the full SHA
    7cb222b View commit details
    Browse the repository at this point in the history

  2. Add more tests and error handling

    @tgravescs
    tgravescs committed Jun 5, 2021
    Configuration menu
    Copy the full SHA
    6193fe1 View commit details
    Browse the repository at this point in the history

  3. improve error handling to skip bad event logs

    @tgravescs
    tgravescs committed Jun 5, 2021
    Configuration menu
    Copy the full SHA
    3295bc9 View commit details
    Browse the repository at this point in the history

  4. update readme

    @tgravescs
    tgravescs committed Jun 5, 2021
    Configuration menu
    Copy the full SHA
    dca1292 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin/branch-21.06' into profilefixspa… 

    …rkcompat
    @tgravescs
    tgravescs committed Jun 5, 2021
    Configuration menu
    Copy the full SHA
    ba2098a View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2021

  1. remove unneeded temp dir

    @tgravescs
    tgravescs committed Jun 6, 2021
    Configuration menu
    Copy the full SHA
    828b302 View commit details
    Browse the repository at this point in the history

  2. Revert "remove unneeded temp dir" 

    This reverts commit 828b302.
    @tgravescs
    tgravescs committed Jun 6, 2021
    Configuration menu
    Copy the full SHA
    f9fb166 View commit details
    Browse the repository at this point in the history

  3. close file writer on exception 

    Signed-off-by: Thomas Graves <tgraves@nvidia.com>
    @tgravescs
    tgravescs committed Jun 6, 2021
    Configuration menu
    Copy the full SHA
    94cc5b6 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/branch-21.06' into profilefixspa… 

    …rkcompat
    @tgravescs
    tgravescs committed Jun 6, 2021
    Configuration menu
    Copy the full SHA
    0fbe989 View commit details
    Browse the repository at this point in the history

  5. move test files

    @tgravescs
    tgravescs committed Jun 6, 2021
    Configuration menu
    Copy the full SHA
    1a71d26 View commit details
    Browse the repository at this point in the history

  6. fix move of test files

    @tgravescs
    tgravescs committed Jun 6, 2021
    Configuration menu
    Copy the full SHA
    40eeead View commit details
    Browse the repository at this point in the history