Dependency info

[NvTimLiu]

List the latest SNAPSHOT jar file in local maven repo

Get cudf/spark dependency from the correct .m2 dir (#1062)

* Get cudf/spark dependency from the correct .m2 dir

'WORKSPACE' & 'M2DIR' vars are needed for shims to gen the correct cudf/spark dependency info in shims.

Below error in 'spark*-info.properties' is due to unset of 'WORKSPACE' & 'M2DIR':
    build/dependency-info.sh: line 30: /jenkins/printJarVersion.sh: No such file or directory
    build/dependency-info.sh: line 33: /jenkins/printJarVersion.sh: No such file or directory

To fix the error, we set the default values for them in 'build/dependency-info.sh':
    'M2DIR=$HOME/.m2/repository'
    'WORKSPACE=../..'

We also need to explicitly set the correct 'M2DIR' path, in case we change it by '-Dmaven.repo.local=$M2DIR'.
Already updated Jenkins scripts to set the correct 'M2DIR'.

[NvTimLiu]

build

[NvTimLiu]

@jlowe @revans2
It is not a good way to parse the dependency info from maven-metadata-[REPO_ID].xml file, because there are a list of maven-metadata-[REPO_ID].xml files

Mvn will cmp the timestamp among these .xml file and determine whether need to update SNAPSHTO jars.

I buildt spark-rapids from Jenkins maven package -s jenkins/settings, the REPO_ID is snapshots：https://github.com/NVIDIA/spark-rapids/blob/branch-0.3/jenkins/settings.xml#L49

@jlowe Your case REPO_ID is snapshots-repo by mvn package, defined here https://github.com/NVIDIA/spark-rapids/blob/branch-0.3/pom.xml#L579

So I suppose we can just find out the newest jar file in SNAPSHOT folder, as below change shows. Could you please help to review? Thanks!
05cd80c

[NvTimLiu]

build

[NvTimLiu]

build

[revans2]

The build failed not sure why it didn't update the PR.

https://blossom.nvidia.com/sw-gpu-spark-jenkins/job/rapids_premerge-github/391/

09:34:32  [INFO] ------------------------------------------------------------------------
09:34:32  [INFO] Reactor Summary:
09:34:32  [INFO] 
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark Root Project ... FAILURE [  3.511 s]
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark SQL Plugin ..... SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark Shuffle Plugin . SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark SQL Plugin Shims SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark SQL Plugin Spark 3.0.0 Shim SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark SQL Plugin Spark 3.0.0 EMR Shim SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark SQL Plugin Spark 3.0.1 Shim SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark SQL Plugin Spark 3.0.1 EMR Shim SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark SQL Plugin Shim Aggregator SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark Scala UDF Plugin SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark Distribution ... SKIPPED
09:34:32  [INFO] RAPIDS Accelerator for Apache Spark Tests .......... SKIPPED
09:34:32  [INFO] rapids-4-spark-integration-tests_2.12 .............. SKIPPED
09:34:32  [INFO] rapids-4-spark-api-validation ...................... SKIPPED
09:34:32  [INFO] ------------------------------------------------------------------------
09:34:32  [INFO] BUILD FAILURE
09:34:32  [INFO] ------------------------------------------------------------------------
09:34:32  [INFO] Total time: 3.803 s
09:34:32  [INFO] Finished at: 2020-11-17T15:34:28+00:00
09:34:32  [INFO] Final Memory: 23M/494M
09:34:32  [INFO] ------------------------------------------------------------------------
09:34:32  [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.8:run (default) on project rapids-4-spark-parent: An Ant BuildException has occured: exec returned: 127
09:34:32  [ERROR] around Ant part ...<exec output="/home/jenkins/agent/workspace/jenkins-rapids_premerge-github-391/target/extra-resources/rapids4spark-version-info.properties" failonerror="true" executable="bash">... @ 6:180 in /home/jenkins/agent/workspace/jenkins-rapids_premerge-github-391/target/antrun/build-main.xml
09:34:32  [ERROR] -> [Help 1]
09:34:32  [ERROR] 
09:34:32  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
09:34:32  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
09:34:32  [ERROR] 
09:34:32  [ERROR] For more information about the errors and possible solutions, please read the following articles:
09:34:32  [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

[NvTimLiu]

build

[NvTimLiu]

rebase PR source to fix merge conflicts

[NvTimLiu]

build