Permitted Subclasses verification issue when redefining class (InstanceKlass::has_as_permitted_subclass) in jdk-16+28

[Michael Rasmussen]

OK, managed to make a very simple reproduceable test:

package com.test;
import net.bytebuddy.agent.ByteBuddyAgent;
import java.lang.instrument.Instrumentation;

public class Main {
  public static void main(String[] args) throws Exception {
    final Instrumentation instrumentation = ByteBuddyAgent.install();
    Foo.class.isSealed();
    instrumentation.retransformClasses(Bar.class);
  }

  static sealed interface Foo permits Bar {}
  static final class Bar implements Foo {}
}

--

java.exe -showversion --enable-preview -Xlog:class+redefine+load=trace "-Xlog:sealed*=trace" -classpath target\classes;lib\asm-9.0.jar;lib\byte-buddy-agent-1.10.19.jar com.test.Main
openjdk version "16-ea" 2021-03-16
OpenJDK Runtime Environment (build 16-ea+32-2190)
OpenJDK 64-Bit Server VM (build 16-ea+32-2190, mixed mode, sharing)
[0.271s][trace][class,sealed] Checking for permitted subclass of com.test.Main$Bar in com.test.Main$Foo
[0.271s][trace][class,sealed] - Found it at permitted_subclasses[0] => cp[11]
[0.273s][debug][redefine,class,load] loading name=com.test.Main$Bar kind=101 (avail_mem=22046284K)
[0.273s][trace][class,sealed       ] Checking for permitted subclass of com.test.Main$Bar in com.test.Main$Foo
[0.273s][trace][class,sealed       ] - class is NOT a permitted subclass!
Exception in thread "main" java.lang.VerifyError
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)
at java.instrument/sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:168)
at com.test.Main.main(Main.java:13)

/Michael

________________________________
From: Harold Seigel <harold.seigel at oracle.com>
Sent: 19 January 2021 22:52
To: Michael Rasmussen <MRasmussen at perforce.com>; hotspot-dev at openjdk.java.net <hotspot-dev at openjdk.java.net>
Subject: Re: Permitted Subclasses verification issue when redefining class (InstanceKlass::has_as_permitted_subclass) in jdk-16+28

Hi Michael,
I entered bug https://bugs.openjdk.java.net/browse/JDK-8260009 for this problem.

Thank you for reporting it.

Harold
________________________________
From: hotspot-dev <hotspot-dev-retn at openjdk.java.net> on behalf of Michael Rasmussen <MRasmussen at perforce.com>
Sent: Tuesday, January 19, 2021 3:27 PM
To: hotspot-dev at openjdk.java.net <hotspot-dev at openjdk.java.net>
Subject: Permitted Subclasses verification issue when redefining class (InstanceKlass::has_as_permitted_subclass) in jdk-16+28

Hi,

I noticed a regression in the verification of sealed classes, that I've narrowed down to happening between jdk-16+27 and jdk-16+28.

Retransforming or redefining a sealed class can lead to a JVMTI_ERROR_FAILS_VERIFICATION, with Exception <a 'java/lang/IncompatibleClassChangeError'{0x00000000feb820a0}: class autotest.TransparentRectangle cannot inherit from sealed class autotest.Rectangle> (0x00000000feb820a0), even though the class actually does that.

I've tried to investigate the reason, and from what I can tell, it boils down to the check in InstanceKlass::has_as_permitted_subclass.

In jdk-16+27, this method always seems to take the Symbol branch (_constants->tag_at(cp_index).is_klass() is false).
But for jdk-16+28, it takes the Klass branch after the classes are loaded, and from what I can tell the (k2 == k) comparison can then fail during the redefinition since the Klass stored in the _permitted_subclasses is not the same as the one being verified, as a new InstanceKlass is allocated for the stream parser.

Unfortunately, I haven't been able to produce a small self-contained test example, but I can reliably reproduce it while developing JRebel (which utilizes class retransformation), but hopefully the above is at least helpful in finding the issue and fixing it.

/Michael

This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.




CAUTION: This email originated from outside of the organization. Do not click on links or open attachments unless you recognize the sender and know the content is safe.


This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.