Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure log4j version 2.x for test cases #5822

Merged
merged 1 commit into from
Jun 15, 2022

Conversation

res-life
Copy link
Collaborator

Closes #5145

Fix: Running Scala tests against Spark 3.3 results in lots of Spark logging output

problem:

Log4j changed to version 2.x from 1.x on Spark330+, log4j2.properties is needed instead of log4j.properties

cd spark-rapids/sql-plugin
mvn dependency:tree -Dbuildver=330 | grep log4j
[INFO] |  |  +- org.apache.logging.log4j:log4j-slf4j-impl:jar:2.17.2:provided
[INFO] |  |  +- org.apache.logging.log4j:log4j-api:jar:2.17.2:provided
[INFO] |  |  +- org.apache.logging.log4j:log4j-core:jar:2.17.2:provided
[INFO] |  |  +- org.apache.logging.log4j:log4j-1.2-api:jar:2.17.2:provided

mvn dependency:tree -Dbuildver=320 | grep log4j
[INFO] |  |  +- log4j:log4j:jar:1.2.17:provided
[INFO] |  |  +- org.slf4j:slf4j-log4j12:jar:1.7.30:provided

changes:

find . -name log4j.properties prints the log4j files add a coresponding log4j2.properties

./api_validation/src/main/resources/log4j.properties
./integration_tests/src/test/resources/log4j.properties
./tests/src/test/resources/log4j.properties
./tools/src/test/resources/log4j.properties
./udf-compiler/src/test/resources/log4j.properties
./sql-plugin/src/test/resources/log4j.properties

Did not configure for the api_validation module, because it's only compiled for buildver=31x
Note the log path is in target/surefire-reports as before, not target/spark33x/surefire-reports

Signed-off-by: Chong Gao res_life@163.com

Signed-off-by: Chong Gao <res_life@163.com>
@jlowe jlowe added the test Only impacts tests label Jun 14, 2022
@res-life
Copy link
Collaborator Author

build

@res-life res-life merged commit ed7de6e into NVIDIA:branch-22.08 Jun 15, 2022
@res-life res-life deleted the test-log branch June 15, 2022 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Only impacts tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Running Scala tests against Spark 3.3 results in lots of Spark logging output
2 participants