[External] : Re: Virtual thread memory leak because TrackingRootContainer keeps threads

[Robert Engels]

I created another more real-world test based on experience in debugging systems/memory leaks. The code is here https://github.com/robaho/closablequeue/blob/sample/src/main/java/robaho/queue/sample/SampleOrigWithRef.java <https://github.com/robaho/closablequeue/blob/sample/src/main/java/robaho/queue/sample/SampleOrigWithRef.java>

At least for me, I have found it useful to take heap dumps / snapshots and check instance count invariants - e.g. number of reader threads should equal the number of Producer objects, etc.

When the lifecycle is not correctly implemented using platform threads, you will get a continual increase in threads and the number of consumer objects. (A)

With VT and tracking turned off, the number of threads and consumer objects will remain constant, but the behavior is not as expected (there is no printing of the stats) (B)

With the lifecycle correctly implemented, the instance count invariant holds and the stats are printed. (C)

You can argue that clearly B is broken because the stats line is not observable but this is only a trivial example and it isn’t always that easy.

I find memory leaks - or lack there of - a decent check of correctness. With this VT change, that will no longer be the case.

I also don’t like reviewing a piece of code - consumer() in this case - and having to determine if it is correct based on a system property value. This seems against Java’s explicitness (although I realize some system properties already alter behavior greatly).

As an aside, based on your previous comments, it is curious that the objects are not collected in A. It is in the same state as B - the system knows that queue.take() can never advance - so if the references are not being held by stack entries then they should be collected, right - regardless that a Thread is a GC root?

-----

A) With platform thread consumer:

0 ThirdActor: Queue created
2 Producer: no queue
4 Producer: enqueued message
4 Consumer 0: message received
6 Producer: enqueued message
6 Consumer 0: message received
6 Consumer 0: close shop
8 Producer: enqueued message
8 Consumer 0: message received
10 Producer: no queue
12 ThirdActor: Queue created
12 Producer: no queue
14 Producer: enqueued message
14 Consumer 12: message received
16 Producer: enqueued message
16 Consumer 12: message received
16 Consumer 12: close shop
18 Producer: enqueued message
18 Consumer 12: message received
20 Producer: no queue
22 Producer: no queue
24 ThirdActor: Queue created
24 Producer: no queue
26 Producer: enqueued message
26 Consumer 24: message received
28 Producer: enqueued message
28 Consumer 24: message received
28 Consumer 24: close shop

B) and with virtual thread consumer and tracking turned off:

0 ThirdActor: Queue created
2 Producer: no queue
4 Producer: enqueued message
4 Consumer 0: message received
6 Producer: enqueued message
6 Consumer 0: message received
6 Consumer 0: close shop
8 Producer: enqueued message
8 Consumer 0: message received
10 Producer: no queue
12 Producer: no queue
12 ThirdActor: Queue created
consumer object collected: java.lang.ref.PhantomReference at 7d52c835, thread count 13
queue collected: java.lang.ref.PhantomReference at 4a02d483, thread count 13
14 Producer: enqueued message
14 Consumer 12: message received
16 Producer: enqueued message
16 Consumer 12: message received
16 Consumer 12: close shop
18 Producer: enqueued message
18 Consumer 12: message received
20 Producer: no queue
22 Producer: no queue
24 ThirdActor: Queue created
consumer object collected: java.lang.ref.PhantomReference at 26e0c418, thread count 13
queue collected: java.lang.ref.PhantomReference at 60aece00, thread count 13
24 Producer: no queue
26 Producer: enqueued message

C) and with either type and correct lifecycle code:

0 ThirdActor: Queue created
2 Producer: no queue
4 Producer: enqueued message
4 Consumer 0: message received
6 Producer: enqueued message
6 Consumer 0: message received
6 Consumer 0: close shop
8 Producer: enqueued message
8 Consumer 0: message received
10 Producer: no queue
12 Producer: no queue
consumer 0 interrupted
consumer ended, stats = 3
12 ThirdActor: Queue created
queue collected: java.lang.ref.PhantomReference at 4a02d483, thread count 13
consumer object collected: java.lang.ref.PhantomReference at 7d52c835, thread count 13
14 Producer: no queue
16 Producer: enqueued message
16 Consumer 12: message received
18 Producer: enqueued message
18 Consumer 12: message received
18 Consumer 12: close shop
20 Producer: enqueued message
20 Consumer 12: message received
22 Producer: no queue
24 Producer: no queue
consumer 12 interrupted
consumer ended, stats = 3
24 ThirdActor: Queue created
queue collected: java.lang.ref.PhantomReference at 79a8a792, thread count 13
consumer object collected: java.lang.ref.PhantomReference at 4af7de55, thread count 13
26 Producer: no queue
28 Producer: enqueued message
28 Consumer 24: message received

> On Aug 1, 2024, at 12:11 PM, Ron Pressler <ron.pressler at oracle.com> wrote:
> 
> 
> 
>> On 1 Aug 2024, at 10:41, robert engels <robaho at icloud.com> wrote:
>> 
>> I was not aware of that! So are you saying that current JVMs will GC or not even create A in the following:
>> 
>> A a = …
>> while(true) {
>> …never reference A…
>> }
>> a.doSomething();
>> 
> 
> That depends on whether or not the VM is able to prove that the loop can never be exited (or if it is, then only through an exception that won’t be caught before a.doSomething, which also depends on the inference power of the implementation. But if the VM could prove that, then the object referenced by A is allowed to be collected (that is exactly what ReachabilityFence is for, although that API has some problems we won’t get into here).
> 
> The rule is that when you access an object through a strong reference it will be there. The converse is that the VM may collect an object that will provably never be accessed through a strong reference. The rest is implementation details (and ReachabilityFence, which is problematic in multiple ways, including in how it’s specified).
> 
> — Ron

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/loom-dev/attachments/20240801/e4d80062/attachment.htm>