[BUG] Project fails to build on some environments due to invalid path to build/build-info

[andygrove]

Describe the bug
Multiple people have reported seeing this error on a fresh build on a new environment:

Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.0.20/plexus-utils-3.0.20.jar (243 kB at 737 kB/s)
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/ant/ant/1.9.4/ant-1.9.4.jar (2.0 MB at 1.8 MB/s)
[INFO] Executing tasks
main:
    [mkdir] Created dir: /Users/USERNAME/Documents/GitHub/NVIDIA/spark-rapids/sql-plugin/target/extra-resources
    [mkdir] Created dir: /Users/USERNAME/Documents/GitHub/NVIDIA/spark-rapids/sql-plugin/target/tmp
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:22 min
[INFO] Finished at: 2020-12-03T14:23:58-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.8:run (default) on project rapids-4-spark-sql_2.12: An Ant BuildException has occured: exec returned: 127
[ERROR] around Ant part ...<exec output="/Users/USERNAME/Documents/GitHub/NVIDIA/spark-rapids/sql-plugin/target/extra-resources/rapids4spark-version-info.properties" failonerror="true" executable="bash">... @ 6:178 in /Users/USERNAME/Documents/GitHub/NVIDIA/spark-rapids/sql-plugin/target/antrun/build-main.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

The issue is that the generated Ant file sql-plugin/target/antrun/build-main.xml has the wrong path to the build/build-info script.

<?xml version="1.0" encoding="UTF-8" ?>
<project name="maven-antrun-" default="main"  >
<target name="main">
  <mkdir dir="/home/rladmin/tpc-ds-benchmark/spark-rapids/sql-plugin/target/extra-resources"/>
  <mkdir dir="/home/rladmin/tpc-ds-benchmark/spark-rapids/sql-plugin/target/tmp"/>
  <exec output="/home/rladmin/tpc-ds-benchmark/spark-rapids/sql-plugin/target/extra-resources/rapids4spark-version-info.properties" failonerror="true" executable="bash">
    <arg value="/home/rladmin/tpc-ds-benchmark/spark-rapids/sql-plugin/build/build-info"/>
    <arg value="0.3.0-SNAPSHOT"/>
    <arg value="0.17-SNAPSHOT"/>
  </exec>
</target>
</project>

The script is looking for ../spark-rapids/sql-plugin/build/build-info but it should just be ../spark-rapids/build/build-info.

Steps/Code to reproduce bug

Clone the repo on a fresh environment and run mvn clean package -DskipTests but the issue is likely specific to certain environments although it has been reported on Ubuntu as well as Mac + Homebrew.

Expected behavior
Build should work.

Additional context

This part of the maven file was chaged recently [1] to use ${user.dir} instead of ${project.basedir} and that seems pretty relevant.

[1] #1095

[jlowe]

The change to user.dir was made because the common code is in the parent pom.xml but project.basedir changes for each project, so the project.dir approach only works for projects at a certain directory depth.

I'm assuming this fails in environments that are trying to build without the current working directory being the top of the repo.

We may need to move this common build code into each relevant pom, using project.basedir and adjusting the relative path to the build script in each, or at least update a path-to-script property in each sub-project.

[jlowe]

Note that I build on Ubuntu and Mac all the time without issue, so we're going to need more info on how to reproduce.

[andygrove]

I could not reproduce this either. My understanding is that the issue happens for others when running mvn clean package -DskipTests from the root of the repo (those were the instrucutions I gave, at least). I will see what other information I can gather.