Excessive memory usage

[Hannes Wallnoefer]

Hi Michel,

Thanks for letting us know that this fix improves memory usage for you. 
The bug has already been backported to 8u, the fix version is 8u72.

https://bugs.openjdk.java.net/browse/JDK-8138634

Hannes

Am 2015-10-20 um 07:37 schrieb Michel Krämer:
> Hi!
>
> I can confirm that metaspace usage is much better with JDK 9 Build 
> b85. My little sample program now only needs 1 GB of resident memory 
> and not 1.5 GB like before. That's a huge improvement! Thank you so much!
>
> Still I'm wondering - if the heap usage is only at 150 MB, why does 
> Nashorn need 850 MB of metaspace after all just to execute a 
> JavaScript program. OK, the TypeScript compiler is not really small, 
> but 850 MB seems to be a lot compared to what the V8 engine needs for 
> example (just a few MB). Maybe it's worth looking deeper into this issue?
>
> Is there any chance these fixes might be backported to Java 8 or is it 
> too much work?
>
> Cheers,
> Michel
>
>
> Am 05.10.2015 um 22:38 schrieb Michel Krämer:
>> (Ooops. I didn't press the reply-to-list button.)
>>
>> Thanks for the link to the bug report. I will keep observing the issue
>> and let you know if I find something. If possible I'll also do a test
>> with a snapshot build of JDK9 and see if things have changed.
>>
>> Thanks,
>> Michel
>>
>>
>> Am 05.10.2015 um 22:19 schrieb Hannes Wallnoefer:
>>> Well maybe there is a problem if the process is killed because it uses
>>> too much memory. I just think it's not heap space, at least not in that
>>> particular code in your github repository. Is that the same code 
>>> that is
>>> running in the CI environments?
>>>
>>> I must say I did see increased GC activity and a slowdown after half 
>>> the
>>> iterations.
>>>
>>> We did fix some problems recently that may cause memory problems under
>>> certain circumstances, such as
>>> https://bugs.openjdk.java.net/browse/JDK-8137333
>>>
>>> So I'm not saying there isn't a problem - there may well be one. I just
>>> think that the observed memory numbers on Linux are not that
>>> extraordinary. Please keep observing and let us know if you find
>>> anything new and noteworthy.
>>>
>>> Hannes
>>>
>>>
>>> Am 2015-10-05 um 21:46 schrieb Michel Krämer:
>>>> Hannes,
>>>>
>>>> Thanks for the quick response. I see...
>>>>
>>>> The reason why I'm asking all this is because I tried to run the unit
>>>> tests for vertx-lang-typescript on Travis CI and Circle CI and they
>>>> both killed the process after it had reached 4GB (in fact very quickly
>>>> after the process had started). I had to play with environment
>>>> variables such as MALLOC_ARENA_MAX to get it down to 2.5 GB.
>>>>
>>>> At least it runs now, but I figured it might be something of interest
>>>> for you as I've never experienced this behaviour with any other code
>>>> that I ran on Travis CI or Circle CI (and I have a couple of open
>>>> source projects running there). The memory usage increased quickly
>>>> when I ran JavaScript code so I thought it might have something to do
>>>> with Nashorn.
>>>>
>>>> I was able to reproduce the issue on my own Ubuntu machine so it's not
>>>> related to Travis or Circle.
>>>>
>>>> As far as I can tell heap memory is not the problem. It seems
>>>> metaspace is used a lot or maybe direct memory buffers or maybe even
>>>> something else!?
>>>>
>>>> Anyway, if you say this much of memory is normal under Linux then I'm
>>>> completely fine with it. I just thought if it wasn't the case you
>>>> might want to know.
>>>>
>>>> Cheers,
>>>> Michel
>>>>
>>>>
>>>> Am 05.10.2015 um 21:34 schrieb Hannes Wallnoefer:
>>>>> Hi Michel,
>>>>>
>>>>> Thanks for the quick and easy github repository to reproduce this.
>>>>>
>>>>> Looking at the code running with jvisualvm I can't see any excessive
>>>>> memory usage. Both with JDK 8u60 and 9 heap usage is around 150 MB.
>>>>>
>>>>> The fact that Java reserves a lot of memory on Linux is a well-known
>>>>> fact. It's not related to Nashorn, but rather to how Java interacts
>>>>> with
>>>>> the Linux memory system. It is a bit annoying, but usually it's not a
>>>>> problem and does not reflect Java heap usage.
>>>>>
>>>>> Hannes
>>>>>
>>>>>
>>>>> Am 2015-10-05 um 20:02 schrieb Michel Krämer:
>>>>>> Hi!
>>>>>>
>>>>>> I'm currently working on TypeScript support for Vert.x. I'm 
>>>>>> trying to
>>>>>> run the TypeScript compiler on Nashorn. It works well, but the 
>>>>>> process
>>>>>> uses a lot of memory. I'm wondering if there is a bug in Nashorn 
>>>>>> or if
>>>>>> I'm doing something wrong.
>>>>>>
>>>>>> I uploaded a small example project demonstrating the issue:
>>>>>> https://github.com/michel-kraemer/nashorn-memory-test
>>>>>>
>>>>>> I tested it under Ubuntu 14.04 with JDK 8u60. I run
>>>>>>
>>>>>> javac Main.java && java -Xmx256M -cp . Main
>>>>>>
>>>>>> and watch the process with top. The resident memory quickly goes 
>>>>>> up to
>>>>>> 1 GB and after about 20 iterations grows even further to 1.2 GB. 
>>>>>> After
>>>>>> about 500 iterations it sometimes even goes up to 1.5 GB where it
>>>>>> stays until the end of the program. That's 1.3 GB more than I
>>>>>> specified with -Xmx. I know this is metaspace, but I wonder why
>>>>>> Nashorn needs so much of it. By the way, even if I do only 1 
>>>>>> iteration
>>>>>> the process still goes up to 400-500 MB.
>>>>>>
>>>>>> Any ideas?
>>>>>>
>>>>>> Cheers,
>>>>>> Michel
>>>>>
>>>>>
>>>
>>>
>>