RFR 8073354: TimSortStackSize2.java: requires 64*2^20*sizeof(reference) continues heap space

[Lev Priima]

After Jespers comments I removed catch of OOME and added minimum heap 
size required for test(-Xms385) to overcome default ergonomics for client.
Please review: http://cr.openjdk.java.net/~lpriima/8073354/webrev.01/

Lev

On 02/19/2015 03:00 PM, Lev Priima wrote:
> There is also a problem, if the memory on host is highly fragmented, 
> then we can't allocate continuous amount of heap for creating such big 
> array. I've added catch for OOME and treat this case as skip-pass to 
> make test more robust. Also I've removed explicit -Xmx385 from @run 
> tag and made it start with a single argument.
>
> Please review and push: 
> http://cr.openjdk.java.net/~lpriima/8073354/webrev.00/
> Issue: https://bugs.openjdk.java.net/browse/JDK-8073354
>
> Lev
>
> On 02/17/2015 09:55 AM, David Holmes wrote:
>> On 17/02/2015 3:43 PM, Lev Priima wrote:
>>> Thanks David!
>>> Is this expected behavior of this annotation ?
>>
>> Yes that is the way jtreg defines tags:
>>
>> http://openjdk.java.net/jtreg/tag-spec.html
>>
>> "The argument tokens of a tag extend from the first token after the 
>> tag token to the end of the comment, the end of the file, or the next 
>> tag token, whichever comes first."
>>
>> So everything between @run and @summary are taken to be the @run 
>> commands. And there's no @comment tag unfortunately.
>>
>> David
>>
>>> Lev
>>>
>>> On 02/17/2015 03:20 AM, David Holmes wrote:
>>>> On 16/02/2015 9:20 PM, David Holmes wrote:
>>>>> On 16/02/2015 6:59 PM, Lev Priima wrote:
>>>>>> Thanks, David,
>>>>>> Could you please push it ?
>>>>>
>>>>> I will if Roger doesn't get to it first. It'll be 11 hours before 
>>>>> I can
>>>>> push it.
>>>>
>>>> This has been pushed but note there is a minor issue with the test.
>>>> The jtreg tag specification doesn't terminate tags on newlines, they
>>>> continue until the next tag is encountered or the end of the comment.
>>>> Consequently this:
>>>>
>>>>  * @run main/othervm -Xmx385m TimSortStackSize2 67108864
>>>>  * not for regular execution on all platforms:
>>>>  * run main/othervm -Xmx8g TimSortStackSize2 1073741824
>>>>  * run main/othervm -Xmx16g TimSortStackSize2 2147483644
>>>>
>>>> is processed as:
>>>>
>>>> @run main/othervm -Xmx385m TimSortStackSize2 67108864 not for regular
>>>> execution on all platforms: run main/othervm -Xmx8g TimSortStackSize2
>>>> 1073741824 run main/othervm -Xmx16g TimSortStackSize2 2147483644
>>>>
>>>> and so TimSortStackSize2 is invoked with 18 arguments.
>>>>
>>>> David
>>>> -----
>>>>
>>>>> David
>>>>>
>>>>>> Lev
>>>>>>
>>>>>> On 02/16/2015 08:55 AM, David Holmes wrote:
>>>>>>> On 14/02/2015 12:03 AM, Lev Priima wrote:
>>>>>>>> Please review and push:
>>>>>>>> http://cr.openjdk.java.net/~lpriima/8073124/webrev.00/
>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8073124
>>>>>>>
>>>>>>> I hadn't realized 8072909 had been pushed without final reviewer
>>>>>>> comments being addressed. :(
>>>>>>>
>>>>>>> These changes seem okay. I hope they get promoted at the same 
>>>>>>> time as
>>>>>>> the original changeset so we don't get test failures.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>> Lev
>>>>>>>>
>>>>>>>> On 02/13/2015 05:20 AM, David Holmes wrote:
>>>>>>>>> Hi Lev,
>>>>>>>>>
>>>>>>>>> On 13/02/2015 2:56 AM, Lev Priima wrote:
>>>>>>>>>> Christos,
>>>>>>>>>>
>>>>>>>>>> Test may fail on shorter arrays(page 8 of paper). For 
>>>>>>>>>> instance, on
>>>>>>>>>> worst
>>>>>>>>>> case, generated by test, it starts to fail on length 67108864.
>>>>>>>>>> After increasing stack size of runs to merge, Arrays.sort(T[])
>>>>>>>>>> works
>>>>>>>>>> also on maximum possible array for HotSpot JVM.
>>>>>>>>>
>>>>>>>>> I'd also like to see this documented somewhere in the code.
>>>>>>>>> Presently
>>>>>>>>> there is a reference to listsort.txt but then you have to go and
>>>>>>>>> find
>>>>>>>>> it on the web. :( At a minimum could we please add:
>>>>>>>>>
>>>>>>>>>  175          * computation below must be changed if MIN_MERGE is
>>>>>>>>> decreased.  See
>>>>>>>>>  176          * the MIN_MERGE declaration above for more
>>>>>>>>> information.
>>>>>>>>> +             * The maximum value of 49 allows for an array up to
>>>>>>>>> length
>>>>>>>>> +             * Integer.MAX_VALUE-4.
>>>>>>>>>
>>>>>>>>>> Roger, David,
>>>>>>>>>> I've updated the test (
>>>>>>>>>> http://cr.openjdk.java.net/~lpriima/8072909/webrev.01/test/java/util/Arrays/TimSortStackSize2.java.html 
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ) to make it more suitable for regular execution:
>>>>>>>>>>
>>>>>>>>>>    27  * @run main/othervm TimSortStackSize2 67108864
>>>>>>>>>
>>>>>>>>> This will still fail on small memory devices:
>>>>>>>>>
>>>>>>>>> :~> java TimSortStackSize2 67108864
>>>>>>>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
>>>>>>>>> space
>>>>>>>>>
>>>>>>>>> as the default heap ergonomics may not be large enough. I had to
>>>>>>>>> add a
>>>>>>>>> minimum heap of -Xmx385M to get it to run.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>>    28  * not for regular execution on all platforms:
>>>>>>>>>>    29  * run main/othervm -Xmx8g TimSortStackSize2 1073741824
>>>>>>>>>>    30  * run main/othervm -Xmx32g TimSortStackSize2 2147483644
>>>>>>>>>>
>>>>>>>>>> Could you please push this:
>>>>>>>>>> http://cr.openjdk.java.net/~lpriima/8072909/webrev.01/
>>>>>>>>>> ?
>>>>>>>>>>
>>>>>>>>>> Lev
>>>>>>>>>>
>>>>>>>>>> On 02/12/2015 12:54 PM, christos at zoulas.com wrote:
>>>>>>>>>>> On Feb 12, 9:57pm,david.holmes at oracle.com  (David Holmes) 
>>>>>>>>>>> wrote:
>>>>>>>>>>> -- Subject: Re: 8072909: TimSort fails with
>>>>>>>>>>> ArrayIndexOutOfBoundsException on
>>>>>>>>>>>
>>>>>>>>>>> | Ok - thanks Lev!
>>>>>>>>>>> |
>>>>>>>>>>> | David
>>>>>>>>>>>
>>>>>>>>>>> For posterity can someone document this, and also the value for
>>>>>>>>>>> which
>>>>>>>>>>> Integer.MAX_VALUE-4 fails?
>>>>>>>>>>>
>>>>>>>>>>> christos
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>
>