Excessive memory usage

[Michel Krämer]

Hi!

I can confirm that metaspace usage is much better with JDK 9 Build b85. 
My little sample program now only needs 1 GB of resident memory and not 
1.5 GB like before. That's a huge improvement! Thank you so much!

Still I'm wondering - if the heap usage is only at 150 MB, why does 
Nashorn need 850 MB of metaspace after all just to execute a JavaScript 
program. OK, the TypeScript compiler is not really small, but 850 MB 
seems to be a lot compared to what the V8 engine needs for example (just 
a few MB). Maybe it's worth looking deeper into this issue?

Is there any chance these fixes might be backported to Java 8 or is it 
too much work?

Cheers,
Michel


Am 05.10.2015 um 22:38 schrieb Michel Krämer:
> (Ooops. I didn't press the reply-to-list button.)
>
> Thanks for the link to the bug report. I will keep observing the issue
> and let you know if I find something. If possible I'll also do a test
> with a snapshot build of JDK9 and see if things have changed.
>
> Thanks,
> Michel
>
>
> Am 05.10.2015 um 22:19 schrieb Hannes Wallnoefer:
>> Well maybe there is a problem if the process is killed because it uses
>> too much memory. I just think it's not heap space, at least not in that
>> particular code in your github repository. Is that the same code that is
>> running in the CI environments?
>>
>> I must say I did see increased GC activity and a slowdown after half the
>> iterations.
>>
>> We did fix some problems recently that may cause memory problems under
>> certain circumstances, such as
>> https://bugs.openjdk.java.net/browse/JDK-8137333
>>
>> So I'm not saying there isn't a problem - there may well be one. I just
>> think that the observed memory numbers on Linux are not that
>> extraordinary. Please keep observing and let us know if you find
>> anything new and noteworthy.
>>
>> Hannes
>>
>>
>> Am 2015-10-05 um 21:46 schrieb Michel Krämer:
>>> Hannes,
>>>
>>> Thanks for the quick response. I see...
>>>
>>> The reason why I'm asking all this is because I tried to run the unit
>>> tests for vertx-lang-typescript on Travis CI and Circle CI and they
>>> both killed the process after it had reached 4GB (in fact very quickly
>>> after the process had started). I had to play with environment
>>> variables such as MALLOC_ARENA_MAX to get it down to 2.5 GB.
>>>
>>> At least it runs now, but I figured it might be something of interest
>>> for you as I've never experienced this behaviour with any other code
>>> that I ran on Travis CI or Circle CI (and I have a couple of open
>>> source projects running there). The memory usage increased quickly
>>> when I ran JavaScript code so I thought it might have something to do
>>> with Nashorn.
>>>
>>> I was able to reproduce the issue on my own Ubuntu machine so it's not
>>> related to Travis or Circle.
>>>
>>> As far as I can tell heap memory is not the problem. It seems
>>> metaspace is used a lot or maybe direct memory buffers or maybe even
>>> something else!?
>>>
>>> Anyway, if you say this much of memory is normal under Linux then I'm
>>> completely fine with it. I just thought if it wasn't the case you
>>> might want to know.
>>>
>>> Cheers,
>>> Michel
>>>
>>>
>>> Am 05.10.2015 um 21:34 schrieb Hannes Wallnoefer:
>>>> Hi Michel,
>>>>
>>>> Thanks for the quick and easy github repository to reproduce this.
>>>>
>>>> Looking at the code running with jvisualvm I can't see any excessive
>>>> memory usage. Both with JDK 8u60 and 9 heap usage is around 150 MB.
>>>>
>>>> The fact that Java reserves a lot of memory on Linux is a well-known
>>>> fact. It's not related to Nashorn, but rather to how Java interacts
>>>> with
>>>> the Linux memory system. It is a bit annoying, but usually it's not a
>>>> problem and does not reflect Java heap usage.
>>>>
>>>> Hannes
>>>>
>>>>
>>>> Am 2015-10-05 um 20:02 schrieb Michel Krämer:
>>>>> Hi!
>>>>>
>>>>> I'm currently working on TypeScript support for Vert.x. I'm trying to
>>>>> run the TypeScript compiler on Nashorn. It works well, but the process
>>>>> uses a lot of memory. I'm wondering if there is a bug in Nashorn or if
>>>>> I'm doing something wrong.
>>>>>
>>>>> I uploaded a small example project demonstrating the issue:
>>>>> https://github.com/michel-kraemer/nashorn-memory-test
>>>>>
>>>>> I tested it under Ubuntu 14.04 with JDK 8u60. I run
>>>>>
>>>>> javac Main.java && java -Xmx256M -cp . Main
>>>>>
>>>>> and watch the process with top. The resident memory quickly goes up to
>>>>> 1 GB and after about 20 iterations grows even further to 1.2 GB. After
>>>>> about 500 iterations it sometimes even goes up to 1.5 GB where it
>>>>> stays until the end of the program. That's 1.3 GB more than I
>>>>> specified with -Xmx. I know this is metaspace, but I wonder why
>>>>> Nashorn needs so much of it. By the way, even if I do only 1 iteration
>>>>> the process still goes up to 400-500 MB.
>>>>>
>>>>> Any ideas?
>>>>>
>>>>> Cheers,
>>>>> Michel
>>>>
>>>>
>>
>>
>