Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable implicit JDK profile activation [databricks] #9591

Merged
merged 25 commits into from
Nov 2, 2023
Merged

Enable implicit JDK profile activation [databricks] #9591

merged 25 commits into from
Nov 2, 2023

Conversation

gerashegalov
Copy link
Collaborator

@gerashegalov gerashegalov commented Oct 31, 2023 •
edited
Loading

  • creates a generic pom to activate for all JDK9+
  • by default enforces usage of Java 8, 11, 17 tested by CI
  • documents how to experiment with other Java versions
  • removes hardcoding of -target/release in Scala 2.13 builds
  • removes explicit -Pjdk11,17 from github workflows

Test

JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 ./build/buildall --profile=330,350 --scala213
JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 TEST_PARALLEL=0 SPARK_HOME=~/dist/spark-3.5.0-bin-hadoop3-scala2.13 ./integration_tests/run_pyspark_from_build.sh -s -k array_exists

Closes #9589

Signed-off-by: Gera Shegalov gera@apache.org

@gerashegalov
Consolidate deps switching in an intermediate pom
093e31d 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Merge remote-tracking branch 'origin/branch-23.12' into shimDepsSwitc…
824cc80 
…herParent
@gerashegalov
Merge remote-tracking branch 'origin/branch-23.12' into shimDepsSwitc…
e0b5384 
…herParent
@gerashegalov
Revert inclusion of shim-deps module
225e517 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Merge remote-tracking branch 'origin/branch-23.12' into shimDepsSwitc…
956d952 
…herParent
@gerashegalov
Merge remote-tracking branch 'origin/branch-23.12' into shimDepsSwitc…
1410c12 
…herParent
@gerashegalov
merge
addaccb
@gerashegalov
Merge remote-tracking branch 'origin/branch-23.12' into shimDepsSwitc…
53258d8 
…herParent
@gerashegalov
shim-deps module as a build order function for a decendant
b881e30 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
regenerate 2.13 poms
7ae26ba 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
make shim-deps 2.12/2.13 sensitive
30646ec 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Make JDK profiles implicit
5034895 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Merge remote-tracking branch 'origin/branch-23.12' into implicitJDKPr…
d990638 
…ofiles2
@gerashegalov
Remove explicit JDK profiles from github workflow
8456c52 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Enforce Java and Scala 2.13 buildvers
183aa14 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Merge remote-tracking branch 'origin/branch-23.12' into shimDepsSwitc…
50254a2 
…herParent
@gerashegalov
Merge branch 'shimDepsSwitcherParent' into implicitJDKProfiles2
2cba130
@gerashegalov
Document skipping JDK enforcement
0ea0357 
- single profile to cover JDK9+

Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Update README
b355ae9 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Merge remote-tracking branch 'origin/branch-23.12' into implicitJDKPr…
07fdfe1 
…ofiles2
@gerashegalov
Merge remote-tracking branch 'origin/branch-23.12' into implicitJDKPr…
b44c921 
…ofiles2
@gerashegalov
separate fake modules
985f60a 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Remove hardcoding of -target/release
6dc4088 
Fixes NVIDIA#9589

Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov
Remove scala2.13 Java target hardcoding
a7a4a40 
Signed-off-by: Gera Shegalov <gera@apache.org>
@gerashegalov gerashegalov self-assigned this Oct 31, 2023
@gerashegalov gerashegalov added build Related to CI / CD or cleanly building Scala 2.13 Issues related to Scala 2.13 binaries labels Oct 31, 2023
@gerashegalov
Copy link
Collaborator Author

build

@gerashegalov
Copy link
Collaborator Author

build

jlowe
jlowe reviewed Oct 31, 2023
View reviewed changes
jdk-profiles/pom.xml Show resolved Hide resolved
pom.xml Show resolved Hide resolved
@pxLi pxLi merged commit 525c73e into NVIDIA:branch-23.12 Nov 2, 2023
37 checks passed
This was referenced Nov 2, 2023
Open
Open
@jlowe jlowe mentioned this pull request Nov 8, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jlowe jlowe jlowe approved these changes

@pxLi pxLi pxLi approved these changes

@revans2 revans2 Awaiting requested review from revans2 revans2 is a code owner

@tgravescs tgravescs Awaiting requested review from tgravescs tgravescs is a code owner

@GaryShen2008 GaryShen2008 Awaiting requested review from GaryShen2008 GaryShen2008 is a code owner

@NvTimLiu NvTimLiu Awaiting requested review from NvTimLiu NvTimLiu is a code owner

@NVnavkumar NVnavkumar Awaiting requested review from NVnavkumar

Assignees

@gerashegalov gerashegalov

Labels
build Related to CI / CD or cleanly building Scala 2.13 Issues related to Scala 2.13 binaries
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Scala 2.13 build hard-codes Java 8 target
4 participants
@gerashegalov @jlowe @pxLi @NVnavkumar