Skip to content

Use JDK 17 runtime#1471

Merged
wisechengyi merged 2 commits intomasterfrom
jdk17
Nov 17, 2025
Merged

Use JDK 17 runtime#1471
wisechengyi merged 2 commits intomasterfrom
jdk17

Conversation

@wisechengyi
Copy link
Collaborator

@wisechengyi wisechengyi commented Oct 31, 2025

Summary

Upgrades Polynote to use Java 17 (from Java 8) for CI builds, adding necessary JVM compatibility flags for Spark.

Changes

Java Version Upgrade

  • Updated CI workflows to use Java 17:
    • ci-backend-2.12.yml: Java 8 → 17
    • ci-backend-2.13.yml: Java 8 → 17
    • dist.yml: Java 8 → 17

Spark Java 17 Compatibility

  • Added JVM flags to build.sbt for Spark tests to work with Java 17+
  • Flags open internal Java modules that Spark's unsafe code requires
  • Based on Spark's JavaModuleOptions
  • Includes --add-opens for: java.lang, java.io, java.net, java.nio, java.util, sun.nio, sun.security, and more

Script Improvements

  • Updated polynote.py to use JAVA_HOME if set (previously always used java from PATH)

Compatibility

✅ Benefits

  • Modern Java runtime (Java 17 LTS)
  • Better performance and security
  • Continued Spark 3.3.4 compatibility

📋 Requirements

  • Java 17+ required for building and running Polynote
  • Existing Java 8 users need to upgrade

Technical Details

JVM Flags Added:

  • -XX:+IgnoreUnrecognizedVMOptions - Gracefully handle unknown options
  • --add-opens=java.base/*=ALL-UNNAMED - Opens internal Java modules for Spark's reflection/unsafe usage
  • Applied to test scope only in build.sbt

Why needed:

  • Java 17 restricts access to internal APIs
  • Spark uses sun.misc.Unsafe and reflection on internal classes
  • Without these flags, Spark fails at runtime with InaccessibleObjectException

@wisechengyi wisechengyi force-pushed the jdk17 branch 2 times, most recently from f6e5270 to 6f4602f Compare November 4, 2025 22:30
@wisechengyi wisechengyi changed the base branch from master to fixrace3 November 4, 2025 22:31
@wisechengyi wisechengyi changed the base branch from fixrace3 to shade-guava-fixrace3 November 4, 2025 22:31
@wisechengyi wisechengyi changed the base branch from shade-guava-fixrace3 to testextract November 4, 2025 22:34
@wisechengyi wisechengyi force-pushed the jdk17 branch 3 times, most recently from a70125f to 952fd82 Compare November 5, 2025 00:46
@wisechengyi wisechengyi changed the base branch from testextract to bumpsparkto3.3 November 5, 2025 00:46
@wisechengyi wisechengyi marked this pull request as ready for review November 5, 2025 18:53
Copy link
Collaborator Author

wisechengyi commented Nov 5, 2025

classpath = ":".join([":".join([ f'"{d}"' for d in deps ]), ":".join([ f'"{p}"' for p in plugins ])])
cmd = f"java -cp {classpath} -Djava.library.path={jep_path} polynote.Main {' '.join(sys.argv[1:])}"
java_executable = os.path.join(os.environ['JAVA_HOME'], 'bin', 'java') if os.environ.get('JAVA_HOME') else 'java'
cmd = f"{java_executable} -cp {classpath} -Djava.library.path={jep_path} polynote.Main {' '.join(sys.argv[1:])}"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would give us flexibility to launch polynote with desired jdk during apt-get install polynote

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR upgrades the Java runtime from JDK 8 to JDK 17 across the project's build and execution environments.

  • Updates all GitHub Actions workflows to use Java 17 instead of Java 8
  • Modifies the Python launcher script to support custom JAVA_HOME configurations
  • Ensures consistent Java version across CI pipelines and distribution builds

Reviewed Changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.

File Description
scripts/polynote.py Adds JAVA_HOME environment variable support for custom JDK paths
.github/workflows/dist.yml Updates distribution workflow to use Java 17
.github/workflows/ci-backend-2.13.yml Updates Scala 2.13 CI pipeline to use Java 17
.github/workflows/ci-backend-2.12.yml Updates Scala 2.12 CI pipeline to use Java 17

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Collaborator

@jonathanindig jonathanindig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my understanding this isn't changing anything compatibility-wise as it'll still be generating Java 8 compatible bytecode (which is fine though we will probably want to update that eventually). Please lmk if that's not the case - otherwise, LGTM.

Please add a description before merging, thanks!

@wisechengyi wisechengyi force-pushed the bumpsparkto3.3 branch 2 times, most recently from 2daacfd to 7922bc2 Compare November 7, 2025 19:35
@wisechengyi wisechengyi force-pushed the jdk17 branch 2 times, most recently from 202847b to 6f4c37a Compare November 11, 2025 19:45
@wisechengyi wisechengyi marked this pull request as draft November 12, 2025 02:00
@wisechengyi wisechengyi marked this pull request as ready for review November 12, 2025 02:01
@wisechengyi
Copy link
Collaborator Author

Based on my understanding this isn't changing anything compatibility-wise as it'll still be generating Java 8 compatible bytecode (which is fine though we will probably want to update that eventually). Please lmk if that's not the case - otherwise, LGTM.

Please add a description before merging, thanks!

That is correct. Scala 2.12.x can only produce JDK 8 bytecode anyway.

For 2.13.x, not sure, but may not matter too much because there is no prod usage yet.

@wisechengyi wisechengyi force-pushed the bumpsparkto3.3 branch 2 times, most recently from af9954b to ffbddaf Compare November 14, 2025 00:24
@wisechengyi wisechengyi force-pushed the jdk17 branch 3 times, most recently from f0ee547 to 65b6b10 Compare November 14, 2025 03:14
@wisechengyi wisechengyi changed the base branch from bumpsparkto3.3 to graphite-base/1471 November 14, 2025 23:45
@graphite-app graphite-app bot changed the base branch from graphite-base/1471 to master November 17, 2025 18:54
@graphite-app
Copy link

graphite-app bot commented Nov 17, 2025

Merge activity

  • Nov 17, 6:54 PM UTC: Graphite rebased this pull request, because this pull request is set to merge when ready.

@wisechengyi wisechengyi merged commit 0f4a417 into master Nov 17, 2025
7 checks passed
@wisechengyi wisechengyi deleted the jdk17 branch November 17, 2025 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants