You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Running the Scala tests against Spark 3.3 will result in a lot of logging messages to the console.
Steps/Code to reproduce bug mvn clean package -Dbuildver=330
When the build gets to the point of running the Scala tests with a Spark session, note how many Spark logging messages appear, e.g.:
[INFO] --- scalatest-maven-plugin:2.0.2:test (test) @ rapids-4-spark-udf_2.12 ---
Discovery starting.
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
22/04/05 15:15:28 INFO SparkContext: Running Spark version 3.3.0-SNAPSHOT
22/04/05 15:15:28 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
22/04/05 15:15:28 INFO ResourceUtils: ==============================================================
22/04/05 15:15:28 INFO ResourceUtils: No custom resources configured for spark.driver.
22/04/05 15:15:28 INFO ResourceUtils: ==============================================================
22/04/05 15:15:28 INFO SparkContext: Submitted application: OpcodeSuite
22/04/05 15:15:28 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(cores -> name: cores, amount: 1, script: , vendor: , memory -> name: memory, amount: 1024, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)
22/04/05 15:15:28 INFO ResourceProfile: Limiting resource is cpu
22/04/05 15:15:28 INFO ResourceProfileManager: Added ResourceProfile id: 0
[...]
Expected behavior
Logging messages should be redirected to a file in surefire-reports rather than sent to the console when running the Scala tests (i.e.: the behavior when running with Spark 3.2.x or earlier)
Environment details (please complete the following information)
Spark 3.3.0
The text was updated successfully, but these errors were encountered:
Describe the bug
Running the Scala tests against Spark 3.3 will result in a lot of logging messages to the console.
Steps/Code to reproduce bug
mvn clean package -Dbuildver=330
When the build gets to the point of running the Scala tests with a Spark session, note how many Spark logging messages appear, e.g.:
Expected behavior
Logging messages should be redirected to a file in
surefire-reports
rather than sent to the console when running the Scala tests (i.e.: the behavior when running with Spark 3.2.x or earlier)Environment details (please complete the following information)
Spark 3.3.0
The text was updated successfully, but these errors were encountered: