RFR 8073354: TimSortStackSize2.java: requires 64*2^20*sizeof(reference) continues heap space

[David Holmes]

On 19/02/2015 11:23 PM, Lev Priima wrote:
> After Jespers comments I removed catch of OOME and added minimum heap
> size required for test(-Xms385) to overcome default ergonomics for client.
> Please review: http://cr.openjdk.java.net/~lpriima/8073354/webrev.01/

Didn't see this before sending my previous reply.

I'm confused now. What functional change is now being made here ??

Thanks,
David

> Lev
>
> On 02/19/2015 03:00 PM, Lev Priima wrote:
>> There is also a problem, if the memory on host is highly fragmented,
>> then we can't allocate continuous amount of heap for creating such big
>> array. I've added catch for OOME and treat this case as skip-pass to
>> make test more robust. Also I've removed explicit -Xmx385 from @run
>> tag and made it start with a single argument.
>>
>> Please review and push:
>> http://cr.openjdk.java.net/~lpriima/8073354/webrev.00/
>> Issue: https://bugs.openjdk.java.net/browse/JDK-8073354
>>
>> Lev
>>
>> On 02/17/2015 09:55 AM, David Holmes wrote:
>>> On 17/02/2015 3:43 PM, Lev Priima wrote:
>>>> Thanks David!
>>>> Is this expected behavior of this annotation ?
>>>
>>> Yes that is the way jtreg defines tags:
>>>
>>> http://openjdk.java.net/jtreg/tag-spec.html
>>>
>>> "The argument tokens of a tag extend from the first token after the
>>> tag token to the end of the comment, the end of the file, or the next
>>> tag token, whichever comes first."
>>>
>>> So everything between @run and @summary are taken to be the @run
>>> commands. And there's no @comment tag unfortunately.
>>>
>>> David
>>>
>>>> Lev
>>>>
>>>> On 02/17/2015 03:20 AM, David Holmes wrote:
>>>>> On 16/02/2015 9:20 PM, David Holmes wrote:
>>>>>> On 16/02/2015 6:59 PM, Lev Priima wrote:
>>>>>>> Thanks, David,
>>>>>>> Could you please push it ?
>>>>>>
>>>>>> I will if Roger doesn't get to it first. It'll be 11 hours before
>>>>>> I can
>>>>>> push it.
>>>>>
>>>>> This has been pushed but note there is a minor issue with the test.
>>>>> The jtreg tag specification doesn't terminate tags on newlines, they
>>>>> continue until the next tag is encountered or the end of the comment.
>>>>> Consequently this:
>>>>>
>>>>>  * @run main/othervm -Xmx385m TimSortStackSize2 67108864
>>>>>  * not for regular execution on all platforms:
>>>>>  * run main/othervm -Xmx8g TimSortStackSize2 1073741824
>>>>>  * run main/othervm -Xmx16g TimSortStackSize2 2147483644
>>>>>
>>>>> is processed as:
>>>>>
>>>>> @run main/othervm -Xmx385m TimSortStackSize2 67108864 not for regular
>>>>> execution on all platforms: run main/othervm -Xmx8g TimSortStackSize2
>>>>> 1073741824 run main/othervm -Xmx16g TimSortStackSize2 2147483644
>>>>>
>>>>> and so TimSortStackSize2 is invoked with 18 arguments.
>>>>>
>>>>> David
>>>>> -----
>>>>>
>>>>>> David
>>>>>>
>>>>>>> Lev
>>>>>>>
>>>>>>> On 02/16/2015 08:55 AM, David Holmes wrote:
>>>>>>>> On 14/02/2015 12:03 AM, Lev Priima wrote:
>>>>>>>>> Please review and push:
>>>>>>>>> http://cr.openjdk.java.net/~lpriima/8073124/webrev.00/
>>>>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8073124
>>>>>>>>
>>>>>>>> I hadn't realized 8072909 had been pushed without final reviewer
>>>>>>>> comments being addressed. :(
>>>>>>>>
>>>>>>>> These changes seem okay. I hope they get promoted at the same
>>>>>>>> time as
>>>>>>>> the original changeset so we don't get test failures.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> David
>>>>>>>>
>>>>>>>>> Lev
>>>>>>>>>
>>>>>>>>> On 02/13/2015 05:20 AM, David Holmes wrote:
>>>>>>>>>> Hi Lev,
>>>>>>>>>>
>>>>>>>>>> On 13/02/2015 2:56 AM, Lev Priima wrote:
>>>>>>>>>>> Christos,
>>>>>>>>>>>
>>>>>>>>>>> Test may fail on shorter arrays(page 8 of paper). For
>>>>>>>>>>> instance, on
>>>>>>>>>>> worst
>>>>>>>>>>> case, generated by test, it starts to fail on length 67108864.
>>>>>>>>>>> After increasing stack size of runs to merge, Arrays.sort(T[])
>>>>>>>>>>> works
>>>>>>>>>>> also on maximum possible array for HotSpot JVM.
>>>>>>>>>>
>>>>>>>>>> I'd also like to see this documented somewhere in the code.
>>>>>>>>>> Presently
>>>>>>>>>> there is a reference to listsort.txt but then you have to go and
>>>>>>>>>> find
>>>>>>>>>> it on the web. :( At a minimum could we please add:
>>>>>>>>>>
>>>>>>>>>>  175          * computation below must be changed if MIN_MERGE is
>>>>>>>>>> decreased.  See
>>>>>>>>>>  176          * the MIN_MERGE declaration above for more
>>>>>>>>>> information.
>>>>>>>>>> +             * The maximum value of 49 allows for an array up to
>>>>>>>>>> length
>>>>>>>>>> +             * Integer.MAX_VALUE-4.
>>>>>>>>>>
>>>>>>>>>>> Roger, David,
>>>>>>>>>>> I've updated the test (
>>>>>>>>>>> http://cr.openjdk.java.net/~lpriima/8072909/webrev.01/test/java/util/Arrays/TimSortStackSize2.java.html
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ) to make it more suitable for regular execution:
>>>>>>>>>>>
>>>>>>>>>>>    27  * @run main/othervm TimSortStackSize2 67108864
>>>>>>>>>>
>>>>>>>>>> This will still fail on small memory devices:
>>>>>>>>>>
>>>>>>>>>> :~> java TimSortStackSize2 67108864
>>>>>>>>>> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
>>>>>>>>>> space
>>>>>>>>>>
>>>>>>>>>> as the default heap ergonomics may not be large enough. I had to
>>>>>>>>>> add a
>>>>>>>>>> minimum heap of -Xmx385M to get it to run.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>>>    28  * not for regular execution on all platforms:
>>>>>>>>>>>    29  * run main/othervm -Xmx8g TimSortStackSize2 1073741824
>>>>>>>>>>>    30  * run main/othervm -Xmx32g TimSortStackSize2 2147483644
>>>>>>>>>>>
>>>>>>>>>>> Could you please push this:
>>>>>>>>>>> http://cr.openjdk.java.net/~lpriima/8072909/webrev.01/
>>>>>>>>>>> ?
>>>>>>>>>>>
>>>>>>>>>>> Lev
>>>>>>>>>>>
>>>>>>>>>>> On 02/12/2015 12:54 PM, christos at zoulas.com wrote:
>>>>>>>>>>>> On Feb 12, 9:57pm,david.holmes at oracle.com  (David Holmes)
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> -- Subject: Re: 8072909: TimSort fails with
>>>>>>>>>>>> ArrayIndexOutOfBoundsException on
>>>>>>>>>>>>
>>>>>>>>>>>> | Ok - thanks Lev!
>>>>>>>>>>>> |
>>>>>>>>>>>> | David
>>>>>>>>>>>>
>>>>>>>>>>>> For posterity can someone document this, and also the value for
>>>>>>>>>>>> which
>>>>>>>>>>>> Integer.MAX_VALUE-4 fails?
>>>>>>>>>>>>
>>>>>>>>>>>> christos
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>
>>
>