RFR (S) 8178870: instrumentation.retransformClasses cause coredump

[Jiangli Zhou]

Hi Coleen,

The update looks correct. I also ran your test locally on linux-64 and verified it passed with your fix.

Thanks,
Jiangli

> On Oct 5, 2017, at 7:56 PM, coleen.phillimore at oracle.com wrote:
> 
> Thank you for the code review!
> 
> I had to fix the .c file slightly for Solaris because in solaris C you still can't declare variables before statements.  It's a minor change.  I also added -lc to the makefile for solaris because the other .c files had it.
> 
> http://cr.openjdk.java.net/~coleenp/8178870.02/webrev/index.html
> 
> Thanks,
> Coleen
> 
> On 10/5/17 9:39 PM, Jiangli Zhou wrote:
>> Hi Coleen,
>> 
>> Not an expert in JVMTI area. The bug analysis, fix and test look good to me!
>> 
>> Thanks,
>> Jiangli
>> 
>>> On Oct 5, 2017, at 11:15 AM, coleen.phillimore at oracle.com wrote:
>>> 
>>> Summary: Don't double-free cached class bytes on redefinition loading failure.
>>> 
>>> See bug for more description.  Tested with test/jdk/java/lang/instrument, test/hotspot/jtreg/runtime/RedefineTests, internal CDS tests, internal nsk.jvmti tests, and added test.
>>> 
>>> open webrev at http://cr.openjdk.java.net/~coleenp/8178870.01/webrev
>>> bug link https://bugs.openjdk.java.net/browse/JDK-8178870
>>> 
>>> Thanks,
>>> Coleen
>