hang when using -XX:-UseCompilerSafepoints

[David Holmes]

Hi Tom,

On 14/11/2014 2:43 AM, Deneau, Tom wrote:
> Just as an aside, I got involved in this because I wanted to see the effect of the CompilerSafepoints poll instruction when comparing performance of small JMH benchmarks across a couple of different architectures.

Assuming you are interested in the cost of accessing the page, not the 
cost of the trap, you've probably already deduced that the flag simply 
disabled the arming of the page, as opposed to disabling generation of 
the polling instructions - and so would be of no help.

> But I'm fine with getting rid of the flag.

Thanks for confirming.

David

> -- Tom
>
> -----Original Message-----
> From: David Holmes [mailto:david.holmes at oracle.com]
> Sent: Wednesday, November 12, 2014 11:40 PM
> To: Vladimir Kozlov; hotspot-runtime-dev at openjdk.java.net
> Cc: Deneau, Tom
> Subject: Re: hang when using -XX:-UseCompilerSafepoints
>
> Hi Vladimir,
>
> On 13/11/2014 1:39 PM, Vladimir Kozlov wrote:
>> I agrer that workaround is -Xint. But if we disable compilation with
>> -UseCompilerSafepoints, the flag becomes useless. You can get the same
>> result with just -Xint.
>>
>> The history shows that it was added at the very beginning of Hotspot
>> development, at the day one. I can only speculate that it was used to
>> find performance effects of safepoints in compiled code . It could be
>> the case that we removed safepoints from Counted loops as result of that
>> investigation. I think it was never intended to be used in production.
>>
>> Although we can fix compilers to generate a runtime call which does
>> safepoint when -UseCompilerSafepoints is specified, it will be useless
>> work, I think.
>
> There is some history in JDK-4974572 (which is non-public I'm afraid).
> To all intents and purposes the flag at that point was used to enable
> testing of workarounds if problems were suspected in the "new"
> safepointing code. I think it has outlived its usefulness by a few major
> releases so I'm happy to see it go.
>
> Cheers,
> David
>
>> thanks,
>> Vladimir
>>
>> On 11/12/14 6:57 PM, David Holmes wrote:
>>> On 13/11/2014 9:38 AM, Vladimir Kozlov wrote:
>>>> On 11/12/14 12:13 PM, Aleksey Shipilev wrote:
>>>>> Hi,
>>>>>
>>>>> Still not sure if this is a runtime bug: stripping safepoints from the
>>>>> non-counted loop seems to be a recipe for disaster.
>>>>
>>>> This flag does not affect compiled code - so it is not compiler issue.
>>>
>>> Well, it disables the mechanism that the compiler inserts for checking
>>> if a safepoint has been requested. As I've added to the bug report,
>>> disabling compiler safepoints should go hand-in-hand with disabling the
>>> compilers (ie run with -Xint) - otherwise you have to know that the
>>> compiled code will eventually hit a non-compiler safepoint check.
>>>
>>>> It is only used in runtime/safepoint.cpp and it guards the code which
>>>> protects a polling page.
>>>>
>>>> There are many bugs which shows current problem. For example:
>>>>
>>>> https://bugs.openjdk.java.net/browse/JDK-6873333
>>>>
>>>> I would say that we have to remove it or at least make it experimental
>>>> flag if we want to do experiments with it.
>>>>
>>>> We definitely should not allow to use it in production!
>>>
>>> If we assume there is a reason it was made a product flag then the
>>> correct fix in my opinion would be to fall back to intepreter-only mode
>>> when this flag is turned off.
>>>
>>> If we don't make that assumption then we could still tie it to
>>> interpreter-only mode, but we definitely should not make it configurable
>>> in product mode without some effort.
>>>
>>> Or if we can't ascertain a valid reason for ever wanting to do this, we
>>> could simply delete the flag altogether. :)
>>>
>>> Cheers,
>>> David
>>>
>>>> Regards,
>>>> Vladimir
>>>>
>>>>>
>>>>> Anyhow, I think it deserves a simpler example. Submitted the bug and
>>>>> attached a simple test there:
>>>>>    https://bugs.openjdk.java.net/browse/JDK-8064749
>>>>>
>>>>> Thanks,
>>>>> -Aleksey.
>>>>>
>>>>> On 12.11.2014 19:52, Deneau, Tom wrote:
>>>>>> Hi all --
>>>>>>
>>>>>> Forwarding a thread which came about on the jmh-dev mail list, as
>>>>>> recommended by Aleksey Shipilev (see below).  The JMH framework has a
>>>>>> timing control thread which sleeps for a certain period, then sets a
>>>>>> volatile isDone variable.  Meanwhile, the benchmark thread loops
>>>>>> doing its benchmark code and also checking the isDone field.   A hang
>>>>>> occurs if -XX:-UseCompilerSafepoints is used.
>>>>>>
>>>>>> The original issue can be reproduced by the following steps
>>>>>>
>>>>>>      hg clone http://hg.openjdk.java.net/code-tools/jmh
>>>>>>      cd jmh
>>>>>>      mvn clean install -DskipTests=true
>>>>>>      cd jmh-samples
>>>>>>      java  -server -XX:-UseCompilerSafepoints -jar
>>>>>> target/benchmarks.jar 'JMHSample_01_.*' -t 1 -wi 5 -i 5 -f 0
>>>>>>
>>>>>> -- Tom Deneau
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Aleksey Shipilev [mailto:aleksey.shipilev at oracle.com]
>>>>>> Sent: Wednesday, November 12, 2014 6:09 AM
>>>>>> To: Deneau, Tom; jmh-dev at openjdk.java.net
>>>>>> Subject: Re: using -XX:-UseCompilerSafepoints
>>>>>>
>>>>>> Hi Tom,
>>>>>>
>>>>>> On 11/11/2014 07:34 PM, Deneau, Tom wrote:
>>>>>>> It looks like a thread that calls Thread.sleep (as the timing control
>>>>>>> thread does in the harness) will eventually go thru
>>>>>>> SafepointSynchonize::block (as part of the ThreadBlockInVM
>>>>>>> destructor).  So if there is a looping benchmark thread compiled
>>>>>>> without Compiler Safepoints, the control thread will be blocked and
>>>>>>> will never set the isDone flag.
>>>>>>
>>>>>> So, you are saying that without the safepoint in the while(!isDone)
>>>>>> loop in workload, control thread and workload thread will never
>>>>>> rendezvous on safepoint? I believe this is a bug with
>>>>>> -XX:-CompilerSafepoints, because the comment in safepoint.cpp calls
>>>>>> this
>>>>>> out specifically for VMThread vs. Mutator threads:
>>>>>>
>>>>>>    // In a pathological scenario such as that described in CR6415670
>>>>>>    // the VMthread may sleep just before the mutator(s) become safe.
>>>>>>    // In that case the mutators will be stalled waiting for the
>>>>>> safepoint
>>>>>>    // to complete and the the VMthread will be sleeping, waiting for
>>>>>> the
>>>>>>    // mutators to rendezvous. The VMthread will eventually wake up and
>>>>>>    // detect that all mutators are safe, at which point we'll again
>>>>>> make
>>>>>>    // progress.
>>>>>>
>>>>>> If this is a case, you probably need to report this to runtime guys.
>>>>>>
>>>>>>> This is probably OK, just need to document that CompilerSafepoints
>>>>>>> cannot be turned off.
>>>>>>
>>>>>> I think it is safe to presume something will go hairy if you are using
>>>>>> any special VM flag, therefore I am not inclined to document this.
>>>>>>
>>>>>> Thanks,
>>>>>> -Aleksey.
>>>>>>
>>>>>
>>>>>