RFR: 8340840: jshell ClassFormatError when making inner class static

[Jan Lahoda]

Consider a JShell interaction like:

jshell> class O { class I {} }
|  created class O

jshell> var i = new O().new I();
i ==> O$I at 77caeb3e

jshell> class O { static class I {} }
Exception in thread "main" java.lang.ClassFormatError: class not in class file format
        at jdk.jdi/com.sun.tools.jdi.VirtualMachineImpl.redefineClasses(VirtualMachineImpl.java:396)
        at jdk.jshell/jdk.jshell.execution.JdiExecutionControl.redefine(JdiExecutionControl.java:90)
        at jdk.jshell/jdk.jshell.Unit.doRedefines(Unit.java:312)
        at jdk.jshell/jdk.jshell.Eval.lambda$compileAndLoad$27(Eval.java:1120)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
        at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
        at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
        at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
        at java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
        at java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
        at jdk.jshell/jdk.jshell.Eval.lambda$compileAndLoad$29(Eval.java:1121)
        at jdk.jshell/jdk.jshell.TaskFactory.lambda$runTask$4(TaskFactory.java:213)
        at jdk.compiler/com.sun.tools.javac.api.JavacTaskPool.getTask(JavacTaskPool.java:193)
        at jdk.jshell/jdk.jshell.TaskFactory.runTask(TaskFactory.java:206)
        at jdk.jshell/jdk.jshell.TaskFactory.compile(TaskFactory.java:186)
        at jdk.jshell/jdk.jshell.Eval.compileAndLoad(Eval.java:1100)
        at jdk.jshell/jdk.jshell.Eval.declare(Eval.java:901)
        at jdk.jshell/jdk.jshell.Eval.eval(Eval.java:140)
        at jdk.jshell/jdk.jshell.JShell.eval(JShell.java:513)
        at jdk.jshell/jdk.internal.jshell.tool.JShellTool.processSource(JShellTool.java:3633)
        at jdk.jshell/jdk.internal.jshell.tool.JShellTool.processSourceCatchingReset(JShellTool.java:1353)
        at jdk.jshell/jdk.internal.jshell.tool.JShellTool.processInput(JShellTool.java:1251)
        at jdk.jshell/jdk.internal.jshell.tool.JShellTool.run(JShellTool.java:1222)
        at jdk.jshell/jdk.internal.jshell.tool.JShellTool.start(JShellTool.java:1005)
        at jdk.jshell/jdk.internal.jshell.tool.JShellToolBuilder.start(JShellToolBuilder.java:261)
        at jdk.jshell/jdk.internal.jshell.tool.JShellToolProvider.main(JShellToolProvider.java:120)


There are two problems here (although the stack trace immediately only shows one of them):

Redefining Classes
---

When a snippet is redefined, JShell first tries to redefine it using JDI (`VirtualMachine.redefineClasses). This usually throws `UnsupportedOperationException` is the redefine cannot happen, and `JdiExecutionControl` handles that gracefully. JShell will recompile and reload the given snippet, and dependent snippets, under different names.

But the `redefineClasses` method can also throw various `LinkageError`s. These are properly documented  for the method: https://docs.oracle.com/en/java/javase/25/docs/api/jdk.jdi/com/sun/jdi/VirtualMachine.html#redefineClasses(java.util.Map)

But `JdiExecutionControl` is not handling these `LinkageError`s.

The proposed solution herein is to simply catch and handle the `LinkageError`s in the same way as the `UnsupportedOperationException` (and other exceptions).

InnerClasses attribute data overriding information from sources
---

Consider situation when compiling the third input: `class O { static class I {} }`.

When this is being compiled, a classfile for `var i = new O().new I();` exists, and its `InnerClasses` attribute records `O.I` to be a non-`static` inner class.

While compiling the code for `class O { static class I {} }`, the classfile for `var i` is also read from the classfile, and its `InnerClasses` attribute is read as well. And as a consequence, the `O.I` class will be marked as non-`static`, which conflicts with what is in the source file.

This is not related to JShell as such, it can be reproduced with javac. Please see the `test/langtools/tools/javac/recovery/SourceAndInnerClassInconsistency.java` test.

The proposal herein is to not use the information from the `InnerClasses` classfile attribute to manipulate classes that originate in a source file. More generally, I think the information from the source should always prevail over information from other/unrelated classfiles. Note the `InnerClasses` attribute is in a classfile that has no real relation to the source code that is being compiled, it is simply an classfile on the classpath.

-------------

Commit messages:
 - Updating copyright year.
 - Fixing test.
 - 8340840: jshell ClassFormatError when making inner class static

Changes: https://git.openjdk.org/jdk/pull/27665/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=27665&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8340840
  Stats: 141 lines in 4 files changed: 130 ins; 0 del; 11 mod
  Patch: https://git.openjdk.org/jdk/pull/27665.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/27665/head:pull/27665

PR: https://git.openjdk.org/jdk/pull/27665