IIUC, all of the replacements for finalizers (Cleaners, PhantomReferences) still have the problem that they can't easily share a single thread across separately-developed libraries to actually run the cleanups? It seems to me that has been the only even halfway-convincing reason to still use finalizers for a long time, but it seems to be a pretty good one in practice. On Fri, Mar 10, 2017 at 5:36 PM, Kim Barrett <kim.barrett@oracle.com> wrote:

...

On Mar 10, 2017, at 4:40 PM, Roger Riggs <roger.riggs@oracle.com> wrote:

Finalizers are inherently problematic and their use can lead to performance issues, deadlocks, hangs, and other problematic behavior.

The problems have been accumulating for many years and the first step to deprecate Object.finalize and the overrides in the JDK to communicate the issues, recommend alternatives, and motivate changes where finalization is currently used.

The behavior of finalization nor any uses of finalize are not modified by this change. Most of the changes are to suppress compilation warnings within the JDK.

Please review and comment.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/

Issue: https://bugs.openjdk.java.net/browse/JDK-8165641

Thanks, Roger

looks good.

How does that scale to independently developed libraries? In my world, finalizer deadlocks are relatively easy to diagnose, and a relatively minor annoyance. Stepping back, I'm a little surprised at the deprecation. In my opinion, the problems with finalizers are, in rough order of significance: 1) Premature finalization issues in the absence of reachabilityFence. And I would be pleasantly surprised if reachabilityFence saw much use. 2) Lack of ordering guarantees. 3) Some less significant issues, like runFinalization() synchronization, difficulty in properly triggering GC for non-memory resource exhaustion, slow and deadlocked finalizers, inability to finalize more than once, potential need for finalizer guardians, etc. The first two appear on top since they can cause memory corruption in a world with JNI. The problems with PhantomReference or Cleaner appear to be: 1) Premature cleanup issues in the absence of reachabilityFence. 2) Too many cleanup threads in large multi-author systems. (Can't cause memory corruption, but seems to be a serious issue for complex systems running on resource-constrained hardware.) 3) A similar, though perhaps slightly shorter list of less significant issues. At least it seems shorter at the moment; most real experience in my world still seems to be with finalizers. Some of the old problems, like runFinalization() seem to get worse with the number of clients, since each client needs to invent its own mechanism. Although I agree we'd be marginally better off without finalizers, it's unclear to me the improvement is substantial enough to bother without more of a replacement story. On Sat, Mar 11, 2017 at 9:24 AM, Roger Riggs <Roger.Riggs@oracle.com> wrote:

Hi,

A common cleaner was considered but it would be susceptible to one of the same problems as finalizers. A bug in the cleanup code could block the thread due to locks or deadlock, and hold up cleanup for all of its clients. It would be an attractive nuisance. Library or modules can create a cleaner shared and any issues can be resolved within that context.

Roger

On 3/10/2017 9:15 PM, Hans Boehm wrote:

IIUC, all of the replacements for finalizers (Cleaners, PhantomReferences) still have the problem that they can't easily share a single thread across separately-developed libraries to actually run the cleanups? It seems to me that has been the only even halfway-convincing reason to still use finalizers for a long time, but it seems to be a pretty good one in practice.

On Fri, Mar 10, 2017 at 5:36 PM, Kim Barrett <kim.barrett@oracle.com> wrote:

...

...

On Mar 10, 2017, at 4:40 PM, Roger Riggs <roger.riggs@oracle.com> wrote:

Finalizers are inherently problematic and their use can lead to performance issues, deadlocks, hangs, and other problematic behavior.

The problems have been accumulating for many years and the first step to deprecate Object.finalize and the overrides in the JDK to communicate the issues, recommend alternatives, and motivate changes where finalization is currently used.

The behavior of finalization nor any uses of finalize are not modified by this change. Most of the changes are to suppress compilation warnings within the JDK.

Please review and comment.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/

Issue: https://bugs.openjdk.java.net/browse/JDK-8165641

Thanks, Roger

looks good.

On 3/12/17, 3:07 AM, Andrew Haley wrote:

I guess "difficulty in properly triggering GC for non-memory resource exhaustion" is closest to what I just described.

This has long been an issue in the Java client area where native desktop resources or the like need to be managed carefully, but it is not just native window handles but any resource not managed by GC. So more precisely "non-Java Heap" resource exhaustion as you could allocate 2GB of native heap and the GC would not care at all if it were tied solely to a 32 byte finalizable Java object. -phil.

Hi,

...

...

I guess "difficulty in properly triggering GC for non-memory resource exhaustion" is closest to what I just described.

This has long been an issue in the Java client area where native desktop resources or the like need to be managed carefully, but it is not just native window handles but any resource not managed by GC.

So more precisely "non-Java Heap" resource exhaustion as you could allocate 2GB of native heap and the GC would not care at all if it were tied solely to a 32 byte finalizable Java object.

Indeed. For what it's worth, I have a plan to design and implement a mechanism to allow native memory to be safely deallocated without using finalization, but it's proving rather difficult to find a nice way to design the API and make it fit the JMM. I hope to get it done by JDK 10. :-)

Yeah, make it work! If you have some time, don't mind to share updated ideas with Robert Muir and me! Thanks, Uwe (for Apache Lucene MMapDirectory where a few bytes of heap space bind up to terabytes of mmap fulltext index data 😊)

In regards to the optimized-out "this" stuff - remember that a while ago Paul Sandoz introduced java.lang.ref.Reference#reachabilityFence()... On 03/12/2017 06:55 PM, Hans Boehm wrote:

I agree that's indeed a solvable problem, perhaps not terribly elegantly, but without too much user-visible complication. Cf. Android's (internal-use-only) NativeAllocationRegistry. Basically the GC triggering heuristic needs to consider other resources as well as Java memory. And "other resources" seems to mean "native heap" in 90+% of cases.

The premature cleanup issues (due to e.g. optimized out "this" pointers) scare me much more, since they require either significant changes to compiler guarantees and the language spec, or a really difficult user education effort. It seems to take on the order of an hour one-on-one conversation to convince the typical developer that their finalizer or java.lang.ref use is broken, which it seems to be about 80% of the time.

But I think we agree that it doesn't matter for this discussion; neither of these problems are addressed by deprecating finalizers. PhantomReferences have exactly the same issues. And in my experience it's unfortunately unrealistic to say we're going to use neither. There will be Java wrappers for native objects. And they will be embedded in Java data structures. Requiring explicit management for those amounts to mostly throwing out Java garbage collection.

On Sun, Mar 12, 2017 at 12:02 PM, Uwe Schindler <uschindler@apache.org> wrote:

...

Hi,

...

...

...

I guess "difficulty in properly triggering GC for non-memory resource exhaustion" is closest to what I just described.

This has long been an issue in the Java client area where native desktop resources or the like need to be managed carefully, but it is not just native window handles but any resource not managed by GC.

So more precisely "non-Java Heap" resource exhaustion as you could allocate 2GB of native heap and the GC would not care at all if it were tied solely to a 32 byte finalizable Java object.

Indeed. For what it's worth, I have a plan to design and implement a mechanism to allow native memory to be safely deallocated without using finalization, but it's proving rather difficult to find a nice way to design the API and make it fit the JMM. I hope to get it done by JDK 10. :-)

Yeah, make it work! If you have some time, don't mind to share updated ideas with Robert Muir and me!

Thanks, Uwe (for Apache Lucene MMapDirectory where a few bytes of heap space bind up to terabytes of mmap fulltext index data 😊)

-- - DML

Hi, A reasonably complete treatment of finalizers and suggestions for correct use are in Josh Bloch' Effective Java. It may be a bit dated but is a good start for the casual reader. Josh provided an excerpt: http://www.informit.com/articles/article.aspx?p=1216151&seqNum=7 There is a lot more to do to design better APIs and mechanisms for the most difficult cases. Project Panama is working on some of them and more than a few ideas are being developed (as noted in earlier emails). Roger On 3/13/2017 9:07 AM, Doug Lea wrote:

On 03/12/2017 07:55 PM, Hans Boehm wrote:

...

But I think we agree that it doesn't matter for this discussion; neither of these problems are addressed by deprecating finalizers. PhantomReferences have exactly the same issues. ...

The motivation seems more along the lines of "denigration" vs deprecation. But even if so, I agree with Hans's implicit claim that it might be better to thoroughly document all the issues encountered when using finalization and/or PhantomReferences and/or cleaners, etc. This coverage will necessarily convey the high expertise required to implement and use them effectively, and to not use them otherwise. Currently no single authoritative account exists, so a lot of errors are due to confusion or ignorance. Doing better would seem to be the highest priority. But maybe adding deprecation warnings on top of this will have some beneficial effect. So, like Hans, I'm not completely opposed to this, but not excited either.

-Doug

On 12/03/17 23:55, Hans Boehm wrote:

But I think we agree that it doesn't matter for this discussion; neither of these problems are addressed by deprecating finalizers. PhantomReferences have exactly the same issues. And in my experience it's unfortunately unrealistic to say we're going to use neither. There will be Java wrappers for native objects. And they will be embedded in Java data structures. Requiring explicit management for those amounts to mostly throwing out Java garbage collection.

Not exactly: Java garbage collection is great for what it was intended to do, i.e. managing memory. It's terrible for managing other scarce resources such as file handles. There are much better ways to do that: explicit resource acquisition, good old-fashioned reference counting, etc. Andrew.

On 14/03/17 14:01, Timo Kinnunen wrote:

File handles aren’t that scarce of a resource, really, at least on Windows.

I've seen processes running out of file handles. It is possible to change the per-process limit. And, of course, there is a lot of hidden context in the kernel and device drivers.

On Windows threads are a lot scarcer resource than file handles, and I don’t recall anyone suggesting Java’s GC wasn’t suitable for managing that limited but crucially important resource.

Java's GC isn't used for managing threads.

The question should then be, what makes threads so much easier to manage than file handles and how can we make file handles be more like threads?

They're not.

Food for thought: threads need a big stack which means a lot of memory, but a file handle might be just 8 bytes which is hard to keep track of. So, change the storage of file handles to use slot-0 of new long[65536];

I did try that, and it does work, but it's an ugly waste of memory. Andrew.

On Mar 14, 2017, at 12:25 PM, Timo Kinnunen <timo.kinnunen@gmail.com> wrote:

Hi,

Maybe rather than “an ugly waste of memory” which sounds somewhat negative, we could instead call it “an honestly assessed true resource cost”?

Yuck! I'm imagining opening 32k file descriptors that represent sockets in e.g. a high performance asynchronous Jetty web server and finding that it consumes new long[65536] = 65536 * 8 bytes * 32768 ~= 18GB just in "fake garbage" to pressure out file descriptors. Sure sounds negative to me!

Besides, this amount wouldn’t have to be set in stone. It could be easily increased or decreased in response to resource pressures. Lots of ergonomics options here. In fact you’d never need more memory for this than what the maximum amount is that’s available for new allocations. And it turns out you’ll always have that much memory available by definition, so really no memory would have to be wasted at all.

Sent from Mail for Windows 10

From: Andrew Haley Sent: Tuesday, March 14, 2017 19:03 To: Timo Kinnunen; Hans Boehm; Uwe Schindler Cc: core-libs-dev Subject: Re: RFR 9: 8165641 : Deprecate Object.finalize

On 14/03/17 14:01, Timo Kinnunen wrote:

...

File handles aren’t that scarce of a resource, really, at least on Windows.

I've seen processes running out of file handles. It is possible to change the per-process limit. And, of course, there is a lot of hidden context in the kernel and device drivers.

...

On Windows threads are a lot scarcer resource than file handles, and I don’t recall anyone suggesting Java’s GC wasn’t suitable for managing that limited but crucially important resource.

Java's GC isn't used for managing threads.

...

The question should then be, what makes threads so much easier to manage than file handles and how can we make file handles be more like threads?

They're not.

...

Food for thought: threads need a big stack which means a lot of memory, but a file handle might be just 8 bytes which is hard to keep track of. So, change the storage of file handles to use slot-0 of new long[65536];

I did try that, and it does work, but it's an ugly waste of memory.

Andrew.

On 14/03/17 20:44, Hans Boehm wrote:

It's not hard to add an API to tell the garbage collector to pretend that you're holding onto those 18GB of additional memory when it decides whether to collect, without actually actually holding onto all of that memory.

It may not be hard to add the API, but actually adding a check for such a flag in the hot path of the collector would be very unpleasant. But what would we do instead? I guess the best trick would simply be never to promote them. Then they'd get freed at minor collections, rather than only at full collections. The main problem seems to be generational collectors; non-generational ones, in practice, seem to work just fine. I'm sticking to my position: it's no more appropriate to manage such native resources with GC than it is file handles. If Java had never had finalizers people would have figured out better ways to manage such resources: what we have here is a legacy issue. Andrew.

Hi, If we are to come up with a good alternative to finalization, having to compare disparate things seems unavoidable. Comparing threads and Threads, suppose there’s a subclass of Thread which holds a native resource that’s not reachable from any other Thread and it has a finalize method. Let’s say this Thread’s resource is a handle to its own native thread and the finalize method calls CloseHandle on the handle and ExitThread to signal an A-OK exit code. The thread will not be removed from the system while the handle remains open. Could this Thread’s thread invoke its own finalize method once it’s done or would it have to wait for the Finalizer to call it instead, and if so, why? Being able to say “this resource will be finalized by the Thread from which it was last reachable” would be quite useful for this case and augment the current finalization without having to replace it. Sent from Mail for Windows 10 From: David Holmes Sent: Wednesday, March 15, 2017 00:57 To: Timo Kinnunen; Andrew Haley; Hans Boehm; Uwe Schindler Cc: core-libs-dev Subject: Re: RFR 9: 8165641 : Deprecate Object.finalize On 15/03/2017 12:01 AM, Timo Kinnunen wrote:

Hi,

File handles aren’t that scarce of a resource, really, at least on Windows. On Windows threads are a lot scarcer resource than file handles, and I don’t recall anyone suggesting Java’s GC wasn’t suitable for managing that limited but crucially important resource. The question should then be, what makes threads so much easier to manage than file handles and how can we make file handles be more like threads?

Native thread resources are directly tied to the lifetime of the thread. Once it reaches the end of execution then all native resources with it are reclaimed. It's lifecycle is very specific and well-defined and not related directly to any other entity in the system. Comparing threads to GC managed objects, or file handles, is trying to compare completely disparate things. David -----

Food for thought: threads need a big stack which means a lot of memory, but a file handle might be just 8 bytes which is hard to keep track of. So, change the storage of file handles to use slot-0 of new long[65536];

Sent from Mail for Windows 10

From: Andrew Haley Sent: Tuesday, March 14, 2017 11:14 To: Hans Boehm; Uwe Schindler Cc: core-libs-dev Subject: Re: RFR 9: 8165641 : Deprecate Object.finalize

On 12/03/17 23:55, Hans Boehm wrote:

...

But I think we agree that it doesn't matter for this discussion; neither of these problems are addressed by deprecating finalizers. PhantomReferences have exactly the same issues. And in my experience it's unfortunately unrealistic to say we're going to use neither. There will be Java wrappers for native objects. And they will be embedded in Java data structures. Requiring explicit management for those amounts to mostly throwing out Java garbage collection.

Not exactly: Java garbage collection is great for what it was intended to do, i.e. managing memory. It's terrible for managing other scarce resources such as file handles. There are much better ways to do that: explicit resource acquisition, good old-fashioned reference counting, etc.

Andrew.

Hi, Yes, indeed it does. I find it intriguing that finalization as currently specified is so unsuited for managing the one native resource that from a Java programmers point-of-view the JVM is managing perfectly. This resource being of course not memory, but thread handles. I think by using a cleanup thread pinned to the same processor as the thread about to exit it would be possible to make it seem like the thread was cleaning up after it has exited, but this point isn’t really important. I now realize that “this resource will be finalized by the Thread from which it was last reachable” is too strong for what I had in mind. I should have said something more along the lines of “this resource will be finalized before the Thread from which it was last reachable or other Threads after it fail to acquire a new one”. The operative words “before” and “after” and no mention of synchronization making it a per-processor core rather than a per-thread function. As for what comes after, I’d like to propose that any proposed alternative to finalization be evaluated for both improvements in functionality and ease of use of its interface. Specifically, compared to finalization the alternative should show the same level of improved functionality as GC in Java 8 shows compared to GC of Java 1.0. Similarly, its ease of use should not be significantly worse than what the ease of use of GC in Java 8 is compared to what using GC in Java 1.0 was. This sets a high bar, perhaps, but as Java 8 demonstrates it’s a bar that’s attainable. Sent from Mail for Windows 10 From: David Holmes Sent: Wednesday, March 15, 2017 11:38 To: Timo Kinnunen; Andrew Haley; Hans Boehm; Uwe Schindler Cc: core-libs-dev Subject: Re: RFR 9: 8165641 : Deprecate Object.finalize On 15/03/2017 8:21 PM, Timo Kinnunen wrote:

Hi,

If we are to come up with a good alternative to finalization, having to compare disparate things seems unavoidable. Comparing threads and Threads, suppose there’s a subclass of Thread which holds a native resource that’s not reachable from any other Thread and it has a finalize method. Let’s say this Thread’s resource is a handle to its own native thread and the finalize method calls CloseHandle on the handle and ExitThread to signal an A-OK exit code. The thread will not be removed from the system while the handle remains open. Could this Thread’s thread invoke its own finalize method once it’s done or would it have to wait for the Finalizer to call it instead, and if so, /why?/

ExitThread applies to the current thread, which means it would terminate the finalization thread. The java.lang.Thread object can not be GC'd before the native thread has terminated from the VMs perspective (which is earlier than when the thread terminates from the OS perspective). Once it has terminated the finalizer may run and do whatever it does - it has no affect on the native thread from the VMs perspective.

Being able to say “this resource will be finalized by the Thread from which it was last reachable” would be quite useful for this case and augment the current finalization without having to replace it.

For a thread to claim ownership of a resource such that the thread would be responsible for "finalizing" that resource, you would have to implement a reliable means to ensure the resource can only be transferred amongst threads and never actually shared. But yes this would be another form of thread-centric resource management if a means were provided to register such resources with the thread and to clean them up as part of thread termination. Of course you can implement a simple/crude form of this directly in a custom thread class. David -----

Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

*From: *David Holmes <mailto:david.holmes@oracle.com> *Sent: *Wednesday, March 15, 2017 00:57 *To: *Timo Kinnunen <mailto:timo.kinnunen@gmail.com>; Andrew Haley <mailto:aph@redhat.com>; Hans Boehm <mailto:hboehm@google.com>; Uwe Schindler <mailto:uschindler@apache.org> *Cc: *core-libs-dev <mailto:core-libs-dev@openjdk.java.net> *Subject: *Re: RFR 9: 8165641 : Deprecate Object.finalize

On 15/03/2017 12:01 AM, Timo Kinnunen wrote:

...

Hi,

...

...

File handles aren’t that scarce of a resource, really, at least on Windows. On Windows threads are a lot scarcer resource than file handles, and I don’t recall anyone suggesting Java’s GC wasn’t suitable for managing that limited but crucially important resource. The question should then be, what makes threads so much easier to manage than file handles and how can we make file handles be more like threads?

Native thread resources are directly tied to the lifetime of the thread.

Once it reaches the end of execution then all native resources with it

are reclaimed. It's lifecycle is very specific and well-defined and not

related directly to any other entity in the system. Comparing threads to

GC managed objects, or file handles, is trying to compare completely

disparate things.

David

-----

...

Food for thought: threads need a big stack which means a lot of memory, but a file handle might be just 8 bytes which is hard to keep track of. So, change the storage of file handles to use slot-0 of new long[65536];

...

...

...

...

...

...

...

Sent from Mail for Windows 10

...

...

From: Andrew Haley

...

Sent: Tuesday, March 14, 2017 11:14

...

To: Hans Boehm; Uwe Schindler

...

Cc: core-libs-dev

...

Subject: Re: RFR 9: 8165641 : Deprecate Object.finalize

...

...

On 12/03/17 23:55, Hans Boehm wrote:

...

...

...

But I think we agree that it doesn't matter for this discussion;

...

...

neither of these problems are addressed by deprecating

...

...

finalizers. PhantomReferences have exactly the same issues. And in

...

...

my experience it's unfortunately unrealistic to say we're going to

...

...

use neither. There will be Java wrappers for native objects. And

...

...

they will be embedded in Java data structures. Requiring explicit

...

...

management for those amounts to mostly throwing out Java garbage

...

...

collection.

...

...

Not exactly: Java garbage collection is great for what it was intended

...

to do, i.e. managing memory. It's terrible for managing other scarce

...

resources such as file handles. There are much better ways to do

...

that: explicit resource acquisition, good old-fashioned reference

...

counting, etc.

...

...

Andrew.

...

Hi, The design decision to manage native threads on behalf of the Java programmer was made a long time ago. One consequence of that decision is that today there is no alternative threading library for Java that would allow me to terminate a thread that’s gone into an infinite loop, to give an example. Looking into the future where Object.finalize has been deprecated and then removed from the JVM after an orderly migration period to its replacement, it’s worth considering what facilities would this future JVM provide that would make managing the resources used by your own implementation of threads as trivial as it should be. And who on earth could want to implement their own threading? Project Sumatra might. Sent from Mail for Windows 10 From: David Holmes Sent: Thursday, March 16, 2017 06:41 To: Timo Kinnunen; Andrew Haley; Hans Boehm; Uwe Schindler Cc: core-libs-dev Subject: Re: RFR 9: 8165641 : Deprecate Object.finalize On 16/03/2017 3:06 PM, Timo Kinnunen wrote:

Hi,

Yes, indeed it does. I find it intriguing that finalization as currently specified is so unsuited for managing the one native resource that from a Java programmers point-of-view the JVM is managing perfectly. This resource being of course not memory, but thread handles.

I'm afraid you've lost me. Threads have a very clear and well-defined lifecycle and so are trivially managed. Finalization was never intended to manage threads as they don't need managing. David -----

I think by using a cleanup thread pinned to the same processor as the thread about to exit it would be possible to make it seem like the thread was cleaning up after it has exited, but this point isn’t really important.

I now realize that “this resource will be finalized by the Thread from which it was last reachable” is too strong for what I had in mind. I should have said something more along the lines of “this resource will be finalized before the Thread from which it was last reachable or other Threads after it fail to acquire a new one”. The operative words “before” and “after” and no mention of synchronization making it a per-processor core rather than a per-thread function.

As for what comes after, I’d like to propose that any proposed alternative to finalization be evaluated for both improvements in functionality and ease of use of its interface. Specifically, compared to finalization the alternative should show the same level of improved functionality as GC in Java 8 shows compared to GC of Java 1.0. Similarly, its ease of use should not be significantly worse than what the ease of use of GC in Java 8 is compared to what using GC in Java 1.0 was. This sets a high bar, perhaps, but as Java 8 demonstrates it’s a bar that’s attainable.

Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

*From: *David Holmes <mailto:david.holmes@oracle.com> *Sent: *Wednesday, March 15, 2017 11:38 *To: *Timo Kinnunen <mailto:timo.kinnunen@gmail.com>; Andrew Haley <mailto:aph@redhat.com>; Hans Boehm <mailto:hboehm@google.com>; Uwe Schindler <mailto:uschindler@apache.org> *Cc: *core-libs-dev <mailto:core-libs-dev@openjdk.java.net> *Subject: *Re: RFR 9: 8165641 : Deprecate Object.finalize

On 15/03/2017 8:21 PM, Timo Kinnunen wrote:

...

Hi,

...

...

If we are to come up with a good alternative to finalization, having to

...

compare disparate things seems unavoidable. Comparing threads and

...

Threads, suppose there’s a subclass of Thread which holds a native

...

resource that’s not reachable from any other Thread and it has a

...

finalize method. Let’s say this Thread’s resource is a handle to its own

...

native thread and the finalize method calls CloseHandle on the handle

...

and ExitThread to signal an A-OK exit code. The thread will not be

...

removed from the system while the handle remains open. Could this

...

Thread’s thread invoke its own finalize method once it’s done or would

...

it have to wait for the Finalizer to call it instead, and if so, /why?/

ExitThread applies to the current thread, which means it would terminate

the finalization thread.

The java.lang.Thread object can not be GC'd before the native thread has

terminated from the VMs perspective (which is earlier than when the

thread terminates from the OS perspective). Once it has terminated the

finalizer may run and do whatever it does - it has no affect on the

native thread from the VMs perspective.

...

Being able to say “this resource will be finalized by the Thread from

...

which it was last reachable” would be quite useful for this case and

...

augment the current finalization without having to replace it.

For a thread to claim ownership of a resource such that the thread would

be responsible for "finalizing" that resource, you would have to

implement a reliable means to ensure the resource can only be

transferred amongst threads and never actually shared. But yes this

would be another form of thread-centric resource management if a means

were provided to register such resources with the thread and to clean

them up as part of thread termination. Of course you can implement a

simple/crude form of this directly in a custom thread class.

David

-----

...

...

...

...

...

...

...

Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for

...

Windows 10

...

...

...

...

*From: *David Holmes <mailto:david.holmes@oracle.com>

...

*Sent: *Wednesday, March 15, 2017 00:57

...

*To: *Timo Kinnunen <mailto:timo.kinnunen@gmail.com>; Andrew Haley

...

<mailto:aph@redhat.com>; Hans Boehm <mailto:hboehm@google.com>; Uwe

...

Schindler <mailto:uschindler@apache.org>

...

*Cc: *core-libs-dev <mailto:core-libs-dev@openjdk.java.net>

...

*Subject: *Re: RFR 9: 8165641 : Deprecate Object.finalize

...

...

...

...

On 15/03/2017 12:01 AM, Timo Kinnunen wrote:

...

...

...

Hi,

...

...

...

...

...

...

File handles aren’t that scarce of a resource, really, at least on

...

Windows. On Windows threads are a lot scarcer resource than file

...

handles, and I don’t recall anyone suggesting Java’s GC wasn’t suitable

...

for managing that limited but crucially important resource. The question

...

should then be, what makes threads so much easier to manage than file

...

handles and how can we make file handles be more like threads?

...

...

...

...

Native thread resources are directly tied to the lifetime of the thread.

...

...

Once it reaches the end of execution then all native resources with it

...

...

are reclaimed. It's lifecycle is very specific and well-defined and not

...

...

related directly to any other entity in the system. Comparing threads to

...

...

GC managed objects, or file handles, is trying to compare completely

...

...

disparate things.

...

...

...

...

David

...

...

-----

...

...

...

...

...

Food for thought: threads need a big stack which means a lot of

...

memory, but a file handle might be just 8 bytes which is hard to keep

...

track of. So, change the storage of file handles to use slot-0 of new

...

long[65536];

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

...

Sent from Mail for Windows 10

...

...

...

...

...

...

From: Andrew Haley

...

...

...

Sent: Tuesday, March 14, 2017 11:14

...

...

...

To: Hans Boehm; Uwe Schindler

...

...

...

Cc: core-libs-dev

...

...

...

Subject: Re: RFR 9: 8165641 : Deprecate Object.finalize

...

...

...

...

...

...

On 12/03/17 23:55, Hans Boehm wrote:

...

...

...

...

...

...

...

But I think we agree that it doesn't matter for this discussion;

...

...

...

...

neither of these problems are addressed by deprecating

...

...

...

...

finalizers. PhantomReferences have exactly the same issues. And in

...

...

...

...

my experience it's unfortunately unrealistic to say we're going to

...

...

...

...

use neither. There will be Java wrappers for native objects. And

...

...

...

...

they will be embedded in Java data structures. Requiring explicit

...

...

...

...

management for those amounts to mostly throwing out Java garbage

...

...

...

...

collection.

...

...

...

...

...

...

Not exactly: Java garbage collection is great for what it was intended

...

...

...

to do, i.e. managing memory. It's terrible for managing other scarce

...

...

...

resources such as file handles. There are much better ways to do

...

...

...

that: explicit resource acquisition, good old-fashioned reference

...

...

...

counting, etc.

...

...

...

...

...

...

Andrew.

...

...

...

...

...

...

Hi, yes, the thread should use try...finally to cleanup its resources explicitly. The method to cleanup and release resources should *not* be called finalize and should be idempotent. Finalization is needed as a last resort it should call the cleanup method (and nothing else). Not all resources are lexically scoped but encapsulating the resource in a class, naming the method close and implementing AutoCloseable are good conventions. As you probably know, calling the finalize method *does not* trigger the finalization process and so finalize could be called again. $02, Roger On 3/15/17 6:43 AM, Andrew Haley wrote:

On 15/03/17 10:21, Timo Kinnunen wrote:

...

If we are to come up with a good alternative to finalization, having to compare disparate things seems unavoidable. Comparing threads and Threads, suppose there’s a subclass of Thread which holds a native resource that’s not reachable from any other Thread and it has a finalize method. Let’s say this Thread’s resource is a handle to its own native thread and the finalize method calls CloseHandle on the handle and ExitThread to signal an A-OK exit code. The thread will not be removed from the system while the handle remains open. Could this Thread’s thread invoke its own finalize method once it’s done or would it have to wait for the Finalizer to call it instead, and if so, why? Isn't this a classic example of Finalizer abuse? A Thread's run() method could be enclosed in a try ... finally region which closes the handle. If, for some odd reason, that doesn't happen, then the Thread's finalizer is a last resort, but it shouldn't normally be needed. Whether it's safe for a native thread to invoke its Thread's finalize method depends on the OS-specific details of what CloseHandle does.

Andrew.

I agree that's also an issue. But it's clearly fixable with smallish API additions. It's less important in my context, since more than 90% of finalizers I see are for Java wrappers of native objects, where it actually seems to be mildly painful to have to keep the associated reference live. On Mon, Mar 13, 2017 at 12:25 PM, Kim Barrett <kim.barrett@oracle.com> wrote:

...

On Mar 11, 2017, at 1:56 PM, Hans Boehm <hboehm@google.com> wrote: Stepping back, I'm a little surprised at the deprecation. In my opinion, the problems with finalizers are, in rough order of significance:

I think an important difference between finalize and other j.l.r.Reference-based cleanups that hasn't been mentioned is that finalization is not cancellable. If one creates creates a finalizable object and explicitly "closes" it when done with it, there is no way to avoid the later finalization processing that will ultimately be a very expensive nop. For an example of this problem, see https://issues.apache.org/jira/browse/HDFS-8562 Extending finalization as suggested there seems to me to be feature creep on a facility that we already heavily denigrate and would prefer didn't exist.

On 11/03/2017 07:59, Andrew Haley wrote:

: Wow. I totally agree with the sentiment, but it's a bit late in the JDK 9 timeframe to be talking about this. Has this been discussed internally within Oracle for some time? For years, long before SUNW was finalized :-) Note that Roger's patch just deprecates it, the only implication is warnings at compile-time, nothing else.

-Alan

On 12/03/2017 09:55, Andrew Haley wrote:

: I have no problem with the deprecation, BTW. It's what happens next that is more problematic.

I think it has to be baby steps and I would expect going further will involve years and several major releases. That should give time to look at use-cases that aren't adequately handled by the Cleaner API so that the support in place long before there is any proposal to remove anything. -Alan

On 13/03/17 12:15, Andrew Dinn wrote:

On 12/03/17 09:55, Andrew Haley wrote:

...

Oh, absolutely, I know about that. I was just wondering why now, and is this something you just came up with, and are we going to have the compatibility discussion?

Perhaps now is the chosen moment (and a good choice at that, I say) because deprecation in release N is necessary to allow removal or modification of semantics in release N+k, k >= 1. So, rather N == 10 than N == 9 (n.b. in this case, I strongly suspect that inequality on k will not actually encompass the equals case).

Oops, obviously I meant: So, rather N == 9 than N == 10 regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in England and Wales under Company Registration No. 03798903 Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander

If we're going to keep runFinalization(), could we please document clearly whether it may block indefinitely or not? It seems to me that the answer to this question fundamentally determines the usage model. If not, I can call runFinalization() (and possibly gc()) from a library when I'm out of resources managed by finalizers. But runFinalization() is really only a hint that needs to be careful about blocking. If yes, then runFinalization() is likely to deadlock if I call it from any setting in which I hold a lock, and this kind of usage is unsafe, unless I run it in a separate thread, and refuse to wait for it indefinitely. I can't decide whether the current javadoc says "yes" or "no", making any usage suspect. "No" is probably a more useful answer, but any clear answer seems like it would be a major improvement. On Tue, Mar 14, 2017 at 1:13 PM, Mandy Chung <mandy.chung@oracle.com> wrote:

...

On Mar 14, 2017, at 10:37 AM, Stuart Marks <stuart.marks@oracle.com> wrote:

Tagir Valeev asked on Twitter whether System.runFinalization() and Runtime.runFinalization() should also be deprecated. The obvious answer is "yes" since we're deprecating finalization, but maybe it's not so obvious.

One view is that we're not deprecating the entire finalization mechanism, we're simply deprecating Object.finalize() and its overrides. The point of doing this is to encourage code to migrate away from these methods. Code that calls runFinalization() can't rely on it having many semantics, so such calls are harmless. (Well, mostly harmless.) Encouraging migration away from runFinalization() thus isn't necessary.

I agree deprecating runFinalization() isn’t necessary in this proposed patch. Deprecating Object.finalize would encourage existing code to migrate away from finalizers as well as discourage new code to add any new finalizer.

...

Another point is that "finalization" in a generic sense can be applied to reference processing, not just calls to the finalize() method.

Hmm.. I am not sure what this exactly means here. But there would be lot to figure out what happens next and what baby steps to take.

...

Offhand I'm not sure what runFinalization() does today, but perhaps in the future it could be modified to have stronger semantics regarding reference processing -- which is one of the big unresolved issues in this area.

I don’t know about this. Certainly it would be something to look into.

Mandy

...

What do you think?

s'marks

On 3/10/17 1:40 PM, Roger Riggs wrote:

...

Finalizers are inherently problematic and their use can lead to performance issues, deadlocks, hangs, and other problematic behavior.

The problems have been accumulating for many years and the first step to deprecate Object.finalize and the overrides in the JDK to communicate the issues, recommend alternatives, and motivate changes where finalization is currently used.

The behavior of finalization nor any uses of finalize are not modified by this change. Most of the changes are to suppress compilation warnings within the JDK.

Please review and comment.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/

Issue: https://bugs.openjdk.java.net/browse/JDK-8165641

Thanks, Roger

If runFinalization() waits unconditionally for the other thread to complete, then I think it would be great to add a warning. Conversely, if it times out quickly enough to consider it non-blocking, that's useful information, too. The fact that finalizers run in a new thread is also important, but I think already follows from JLS 12.6. On Tue, Mar 14, 2017 at 2:54 PM, Mandy Chung <mandy.chung@oracle.com> wrote:

Since nothing is proposed to be removed in this patch, Object.finalize and runFinalization will stay.

runFinalization is a hint. The spec states that it suggests the JVM expend effort toward running the finalizer methods of objects pending for finalization. It’s best effort and no guarantee. The current implementation of runFinalization invokes the finalizers in a new thread so that it provides some kind of insulation but it would still be prone to deadlock. We could add a note about such warning.

Mandy

On Mar 14, 2017, at 2:02 PM, Hans Boehm <hboehm@google.com> wrote:

If we're going to keep runFinalization(), could we please document clearly whether it may block indefinitely or not? It seems to me that the answer to this question fundamentally determines the usage model. If not, I can call runFinalization() (and possibly gc()) from a library when I'm out of resources managed by finalizers. But runFinalization() is really only a hint that needs to be careful about blocking. If yes, then runFinalization() is likely to deadlock if I call it from any setting in which I hold a lock, and this kind of usage is unsafe, unless I run it in a separate thread, and refuse to wait for it indefinitely. I can't decide whether the current javadoc says "yes" or "no", making any usage suspect. "No" is probably a more useful answer, but any clear answer seems like it would be a major improvement.

On Tue, Mar 14, 2017 at 1:13 PM, Mandy Chung <mandy.chung@oracle.com> wrote:

...

...

On Mar 14, 2017, at 10:37 AM, Stuart Marks <stuart.marks@oracle.com> wrote:

Tagir Valeev asked on Twitter whether System.runFinalization() and Runtime.runFinalization() should also be deprecated. The obvious answer is "yes" since we're deprecating finalization, but maybe it's not so obvious.

One view is that we're not deprecating the entire finalization mechanism, we're simply deprecating Object.finalize() and its overrides. The point of doing this is to encourage code to migrate away from these methods. Code that calls runFinalization() can't rely on it having many semantics, so such calls are harmless. (Well, mostly harmless.) Encouraging migration away from runFinalization() thus isn't necessary.

I agree deprecating runFinalization() isn’t necessary in this proposed patch. Deprecating Object.finalize would encourage existing code to migrate away from finalizers as well as discourage new code to add any new finalizer.

...

Another point is that "finalization" in a generic sense can be applied to reference processing, not just calls to the finalize() method.

Hmm.. I am not sure what this exactly means here. But there would be lot to figure out what happens next and what baby steps to take.

...

Offhand I'm not sure what runFinalization() does today, but perhaps in the future it could be modified to have stronger semantics regarding reference processing -- which is one of the big unresolved issues in this area.

I don’t know about this. Certainly it would be something to look into.

Mandy

...

What do you think?

s'marks

On 3/10/17 1:40 PM, Roger Riggs wrote:

...

Finalizers are inherently problematic and their use can lead to performance issues, deadlocks, hangs, and other problematic behavior.

The problems have been accumulating for many years and the first step to deprecate Object.finalize and the overrides in the JDK to communicate the issues, recommend alternatives, and motivate changes where finalization is currently used.

The behavior of finalization nor any uses of finalize are not modified by this change. Most of the changes are to suppress compilation warnings within the JDK.

Please review and comment.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/

Issue: https://bugs.openjdk.java.net/browse/JDK-8165641

Thanks, Roger

The problem with runFinalizers() is that it only executes one additional (second) finalized-queue processing thread. It will not trigger the Gc mechanism needed to actually detect and enqueue the unreachable candidates. So it might help with slow or stalled finalize() calls, but by doing so it may block or take long to terminate itself (the invocation joins the system thread). I think that's JEP132 related. Gruss Bernd BTW: For me having a guarantee max-Intervall for tenured generation reference processing (trigger a mixed collection) is something which could improve the robustness in face of oversized old generations. Especially when off heap resources wait for it. Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: core-libs-dev <core-libs-dev-bounces@openjdk.java.net> on behalf of Hans Boehm <hboehm@google.com> Sent: Tuesday, March 14, 2017 10:02:01 PM To: Mandy Chung Cc: core-libs-dev Subject: Re: RFR 9: 8165641 : Deprecate Object.finalize If we're going to keep runFinalization(), could we please document clearly whether it may block indefinitely or not? It seems to me that the answer to this question fundamentally determines the usage model. If not, I can call runFinalization() (and possibly gc()) from a library when I'm out of resources managed by finalizers. But runFinalization() is really only a hint that needs to be careful about blocking. If yes, then runFinalization() is likely to deadlock if I call it from any setting in which I hold a lock, and this kind of usage is unsafe, unless I run it in a separate thread, and refuse to wait for it indefinitely. I can't decide whether the current javadoc says "yes" or "no", making any usage suspect. "No" is probably a more useful answer, but any clear answer seems like it would be a major improvement. On Tue, Mar 14, 2017 at 1:13 PM, Mandy Chung <mandy.chung@oracle.com> wrote:

...

On Mar 14, 2017, at 10:37 AM, Stuart Marks <stuart.marks@oracle.com> wrote:

Tagir Valeev asked on Twitter whether System.runFinalization() and Runtime.runFinalization() should also be deprecated. The obvious answer is "yes" since we're deprecating finalization, but maybe it's not so obvious.

One view is that we're not deprecating the entire finalization mechanism, we're simply deprecating Object.finalize() and its overrides. The point of doing this is to encourage code to migrate away from these methods. Code that calls runFinalization() can't rely on it having many semantics, so such calls are harmless. (Well, mostly harmless.) Encouraging migration away from runFinalization() thus isn't necessary.

I agree deprecating runFinalization() isn’t necessary in this proposed patch. Deprecating Object.finalize would encourage existing code to migrate away from finalizers as well as discourage new code to add any new finalizer.

...

Another point is that "finalization" in a generic sense can be applied to reference processing, not just calls to the finalize() method.

Hmm.. I am not sure what this exactly means here. But there would be lot to figure out what happens next and what baby steps to take.

...

Offhand I'm not sure what runFinalization() does today, but perhaps in the future it could be modified to have stronger semantics regarding reference processing -- which is one of the big unresolved issues in this area.

I don’t know about this. Certainly it would be something to look into.

Mandy

...

What do you think?

s'marks

On 3/10/17 1:40 PM, Roger Riggs wrote:

...

Finalizers are inherently problematic and their use can lead to performance issues, deadlocks, hangs, and other problematic behavior.

The problems have been accumulating for many years and the first step to deprecate Object.finalize and the overrides in the JDK to communicate the issues, recommend alternatives, and motivate changes where finalization is currently used.

The behavior of finalization nor any uses of finalize are not modified by this change. Most of the changes are to suppress compilation warnings within the JDK.

Please review and comment.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/

Issue: https://bugs.openjdk.java.net/browse/JDK-8165641

Thanks, Roger

Hi Mandy, Webrev updated in place with your and Stuart's review comments. http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/ On 3/14/17 4:53 PM, Mandy Chung wrote:

This patch looks fine in general.

Does @inheritDoc inherit @deprecated? What does the javadoc of javax.imageio.stream.* look like? Yes, the deprecation is inherited, see:

http://cr.openjdk.java.net/~rriggs/finalize-doc/api/javax/imageio/stream/Fil...

+ * @deprecated Finalization is deprecated and should not be used. + * See {@link Object#finalize} for the explanation and recommended replacement.

Also, the second line is rather long and might be good to break into two lines?

The style guide allows a somewhat longer line. I moved the "See" to the previous line. it didn't seem worth adding more vertical space for a couple of words. Thanks, Roger

Mandy

...

On Mar 10, 2017, at 1:40 PM, Roger Riggs <Roger.Riggs@oracle.com> wrote:

Finalizers are inherently problematic and their use can lead to performance issues, deadlocks, hangs, and other problematic behavior.

The problems have been accumulating for many years and the first step to deprecate Object.finalize and the overrides in the JDK to communicate the issues, recommend alternatives, and motivate changes where finalization is currently used.

The behavior of finalization nor any uses of finalize are not modified by this change. Most of the changes are to suppress compilation warnings within the JDK.

Please review and comment.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/

Issue: https://bugs.openjdk.java.net/browse/JDK-8165641

Thanks, Roger

Hi Mandy, I re-checked and @deprecated is not inherited; and if it was it would include a generic message from the super so the @deprecated content is needed. Sample of javax.imageio.stream.FileImageInputStream.[1] Only minor cleanup to the webrev: http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/ Thanks, Roger [1] http://cr.openjdk.java.net/~rriggs/finalize-doc/api/javax/imageio/stream/Fil... On 3/14/2017 10:19 PM, Mandy Chung wrote:

...

On Mar 14, 2017, at 7:06 PM, Roger Riggs <roger.riggs@oracle.com> wrote:

On 3/14/17 4:53 PM, Mandy Chung wrote:

...

This patch looks fine in general.

Does @inheritDoc inherit @deprecated? What does the javadoc of javax.imageio.stream.* look like? Yes, the deprecation is inherited, see:

http://cr.openjdk.java.net/~rriggs/finalize-doc/api/javax/imageio/stream/Fil...

If so, @deprecated is not needed in the method that does have {@inheritDoc} e.g. line 158-159, is it?

156 /** 157 * {@inheritDoc} 158 * @deprecated Finalization is deprecated and should not be used. See 159 * {@link Object#finalize} for the explanation and recommended replacement. 160 */

...

...

+ * @deprecated Finalization is deprecated and should not be used. + * See {@link Object#finalize} for the explanation and recommended replacement.

Also, the second line is rather long and might be good to break into two lines? The style guide allows a somewhat longer line. I moved the "See" to the previous line. it didn't seem worth adding more vertical space for a couple of words. Maybe trim the spaces before {@link Object#finalize}.

Mandy

On 31/03/17 18:05, Hans Boehm wrote:

Can we add a warning that it is almost never safe to override finalize() without use of reachabilityFence()?

We'll have to be careful not to suggest to some people that if you don't override finalize() then you don't need reachabilityFence(). Careful wording needed. Andrew.

There was some observation (in an off-line conversation) that since this is being deprecated it is probably a bit late to start documenting it with lots of advice along these and other lines .. -phil. On 03/31/2017 10:25 AM, Roger Riggs wrote:

Hi Hans,

Can you suggest how that would be expressed? The advice would seem to have some complex conditions.

Would it be the case that a caller using a reference to any resource held by a finalizable object must use reachabilityFence to ensure that the resource can not be cleaned up while it is planning to use the resource?

Is there a citation that can supply the context and background for a warning or is the explanation in reachabilityFence sufficient?

Roger

On 3/31/2017 1:05 PM, Hans Boehm wrote:

...

In my view, the primary reason to prefer Cleaner or PhantomReference is safety (no accidental access to previously finalized objects, which usually turn into dangling pointer dereferences with JNI). Flexibility and efficiency also matter, but I would put them second and third.

Can we add a warning that it is almost never safe to override finalize() without use of reachabilityFence()?

On Fri, Mar 31, 2017 at 6:55 AM, Roger Riggs <Roger.Riggs@oracle.com <mailto:Roger.Riggs@oracle.com>> wrote:

The webrev for deprecating finalize has been updated: - to improve the advice existing JDK subclasses overriding finalize provides in @deprecated javadoc, - to expand Object.finalize() javadoc to reinforce the correct use of super.finalize() by subclasses, - and to remove redundant @SuppressWarnings annotations.

Please review and comment.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/ <http://cr.openjdk.java.net/%7Erriggs/webrev-finalize-deprecate-8165641/>

Issue: https://bugs.openjdk.java.net/browse/JDK-8165641 <https://bugs.openjdk.java.net/browse/JDK-8165641>

Thanks, Roger

Hi Stuart, ok, will switch to @SuppressWarnings; it will avoid the deprecation related clutter in the javadoc. Thanks, Roger On 3/31/2017 1:29 PM, Stuart Marks wrote:

On 3/31/17 6:55 AM, Roger Riggs wrote:

...

The webrev for deprecating finalize has been updated: - to improve the advice existing JDK subclasses overriding finalize provides in @deprecated javadoc, - to expand Object.finalize() javadoc to reinforce the correct use of super.finalize() by subclasses, - and to remove redundant @SuppressWarnings annotations.

Please review and comment.

Webrev: http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641/

Issue: https://bugs.openjdk.java.net/browse/JDK-8165641

Hi Roger,

The main text in Object.finalize(), and the text for each of the occurrences of an overriding finalize() method, look good.

I don't think j.l.Enum.finalize() needs to be deprecated. The main point of deprecating finalize() is to inform subclassers. But Enum.finalize() can't be overriden. I suppose somebody could *call* it from an enum, and deprecation would serve to generate a warning, but this is pretty obscure.

I'd suggest replacing the deprecation of Enum.finalize() with @SuppressWarnings.

s'marks

Hi Hans, On 3/31/2017 2:56 PM, Hans Boehm wrote:

Agree entirely with Andrew's concern. I would mostly point to the java.lang.Reference documentation. One possible approach follows.

Add the following to the apiNote. I would add it at the beginning, since I've rarely encountered the inheritance issues, and they're probably easier to debug:

As with all reachability-based cleanup mechanisms, it is important to use java.lang.reReference.reachabilityFence to ensure that objects remain reachable while resources embedded in the object are in use.

Good suggestion for the Object.finalize deprecation doc, I'll see how to work that in.

However, looking at the reachabilityFence documentation, I think it's quite misleading, and greatly understates the need for using it. In particular the following paragraph seems most misleading:

"This method is designed for use in uncommon situations of premature finalization where using synchronized blocks or methods, or using other synchronization facilities are not possible or do not provide the desired control. This method is applicable only when reclamation may have visible effects, which is possible for objects with finalizers (See Section 12.6 17 of The Java™ Language Specification) that are implemented in ways that rely on ordering control for correctness."

These seem completely misleading to me.

I would replace this with

This method should be used when cleanup actions (finalize() calls, or Reference enqueuing, Cleaner invocation) could otherwise be triggered while a resource under control of the object is still in use. This method should normally be called when these cleanup facilities are used to perform actions other than simply issuing a warning.

(Specific complaints about the original: In my experience, this is needed for the large majority of finalization/ReferenceQueue uses. What's the point of a finalizer without visible effects? The object is otherwise dead: Why bother? In practice, synchronization almost never provides the correct guarantees. Finalizers should indeed usually synchronize, but NOT on the object itself. It's not clear what "ordering control" here means. The order in which program actions are performed generally does matter.)

The section starting with "It is sometimes possible to better encapsulate use of reachabilityFence..." just seems misleading. Delete it. I can't imagine a scenario in which that could work. Using files as a (bad for other reasons) example, assume the external resource is a file descriptor, and I'm about to read it. Keeping the file open until I retrieve the descriptor isn't good enough. It needs to be kept open for the duration of the read.

I would replace the last paragraph with the following, or delete it entirely. I don't believe I've ever seen code to which it applies:

ReachabilityFence is not needed when all uses of the embedded resource (including in the constructors or the finalize() method) are entirely enclosed in synchronized(this) blocks.

(Note again that surrounding all accesses to the object itself in synchronized blocks does not suffice.)

Right, good suggestions. Though I think that's out of scope for the current deprecation issue. I created a new Issue [1] to track that and reference that/this email. Thanks, Roger [1] 8177915 Clarify Reference.reachabilityFence use <https://bugs.openjdk.java.net/browse/JDK-8177915>

Thanks! I think that helps a lot. On Mon, Apr 3, 2017 at 7:44 AM, Roger Riggs <Roger.Riggs@oracle.com> wrote:

The api note for Object.finalize is not an optimal location for this more general advice, but is good as far as it can go in relation to the current changes to deprecate Object.finalize().

The webrev[1] is updated to add a paragraph to the apiNote[2]:

* @apiNote * Classes that embed non-heap resources have a number of options * for cleanup of those resources. The class must ensure that the * lifetime of each instance is longer than that of any resource it embeds. * {@link java.lang.ref.Reference#reachabilityFence} can be used to ensure that * objects remain reachable while resources embedded in the object are in use.

Regards, Roger

[1] http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate-8165641 [2] http://cr.openjdk.java.net/~rriggs/webrev-finalize-deprecate -8165641/src/java.base/share/classes/java/lang/Object.java.sdiff.html

(I would not have deprecated finalize; I don't think we have a good enough answer for what to use instead. But) Looks good. Typo: an finalizable object

That also sounds fine to me. The difficulty with applying this, especially in the JNI case, is that you may need several reachability fences at the end of each method, guarding parameters and temporaries, not all of which may be naturally in scope at the end. But we're not going to be able to include a full discussion here. In the interest of full disclosure, we currently, at least temporarily, dodge this issue by inhibiting dead variable elimination for references. On Apr 1, 2017 01:55, "Andrew Haley" <aph@redhat.com> wrote: On 31/03/17 19:56, Hans Boehm wrote:

This method should be used when cleanup actions (finalize() calls, or Reference enqueuing, Cleaner invocation) could otherwise be triggered while a resource under control of the object is still in use. This method should normally be called when these cleanup facilities are used to perform actions other than simply issuing a warning.

It's still pretty confusing. It would be simpler to say that reachabilityFence should be used at the end of every method of a class which has cleanup actions if you don't want those cleanup actions to be run before the end of that method. In that case, reachabilityFence only makes any difference in cases where otherwise you'd have a bug. Andrew.

The approach Gil is advocating certainly has its attraction. The reason I'm not really enthusiastic, and don't really view the language design problem as solved anymore, is that I'm no longer certain that the majority of programmers will ever be able to use reachabilityFence correctly. The rules are really, really subtle and unintuitive, especially for JNI-intensive code. I don't have a termination proof for Gil's step 4. Android's BigInt.java is a good illustration of the problem. This is a wrapper around a native (BoringSSL) bignum package, which generally forwards most operations directly to native code. But functions routinely take BigInt arguments in addition to "this". Those each need reachabilityFences, even for those that currently have one-liner implementations. Often temporary BigInts are allocated, filled in, and possibly returned as results. Those need reachabilityFences, just in case the result is not used. The rules for constructors are not 100% clear to me. But I think those, counterintuitively, also generally need reachabilityFences to deal with the case in which the protected field is not subsequently used. I have little confidence I could get this right without a number of iterations. I agree that it would be good to make the JDK reachabilityFence-correct. It's the only available tool, so it's necessary for correctness. But I would also view it as a feasibility experiment, keeping in mind that the hardest code to deal with is probably in third-party libraries. My guess is that we haven't gotten to the final solution here. On Sat, Apr 15, 2017 at 8:51 AM, Gil Tene <gil@azul.com> wrote:

IMO, so long as the JDK's own performance-sensitive methods do not follow this advice, it will be hard to "educate" people to use reachabilityFence responsibly.

To be specific:

As long as java.nioDirectByteBuffer.get() is effectively coded like this (after the templates are applied):

public byte get() { return (unsafe.getByte(ix(nextGetIndex()))); }

Instead of like this:

public byte get() { try { return (unsafe.getByte(ix(nextGetIndex()))); } finally { Reference.reachabilityFence(this); } }

People will copy it and extend on it, replicating the reachability bug and the use-after-free problem in the current code.

It is hard to ask people to code "responsibly" when the JDK itself doesn't. This is especially true when the bug in the above code is not obvious, and (as Hans points out earlier) it takes a long time to convince people it actually exists. This is especially true since the "well, when was the last time someone complained about DirectByteBuffer.get() doing getting a SEGV on access to a freed buffer?" argument holds some water. And that's where you actually end up usually when discussing the above missing reachabilityFence…

A few things are keeping the JDK code from evolving to properly use Reference.reachabilityFence(this):

a. It didn't exist until recently. [That's a very good excuse]

b. While Reference.reachabilityFence exists now, it's implementation is sub-optimal. IUUC it is modeled [currently] as an opaque method call, which prohibits enough useful optimizations to significantly hurt DirectByteBuffer.get() if it were actually included in the code right now.

c. As Hans notes below, the compilers may temporarily dodge this issue by inhibiting dead variable elimination for references. They do so to allow the actual (semantically buggy) current JDK code to execute safely and performantly.

As long as the compilers paper over this by temporarily dodging the issue and inhibiting dead variable elimination for references, we are giving people a continued excuse to ignore the need for adding reachabilityFences to their code. And this situation will probably continue (for practical purposes) at least as long as using reachabilityFence in the above common form is not optimized in practice to be equivalent to just inhibiting dead variable elimination of (this) across the same scope [and not just implemented as a much more expensive opaque method effect].

This makes the subject somewhat premature for wide public education IMO. A practical set of steps might look like this:

0. Add Reference.reachabilityFence() to the platform [this is basically the existing "collect underpants" step]

1. make try { } finally { Reference.reachabilityFence(this); } optimize well (to be no more expensive than inhibiting dead variable elimination for 'this' across the try block).

2. Change JDK code to actually make use of this code idiom in the various places where it is actually needed for correctness.

3. Relax compiler limitation crutches that currently help make the correctness bugs not trigger often. But do so first with a non-default -XX flag, and later transition to default with a backout flag (since a lot of user code will start breaking when the crutch is taken away).

4. Educate and evangelize for proper use of Reference.reachabilityFence(this); (maybe even add a method annotation for people who like that sort of thing? E.g. @AlwaysReachableThis)

5. Profit

...

On Apr 1, 2017, at 10:19 AM, Hans Boehm <hboehm@google.com> wrote:

That also sounds fine to me.

The difficulty with applying this, especially in the JNI case, is that you may need several reachability fences at the end of each method, guarding parameters and temporaries, not all of which may be naturally in scope at the end. But we're not going to be able to include a full discussion here.

In the interest of full disclosure, we currently, at least temporarily, dodge this issue by inhibiting dead variable elimination for references.

On Apr 1, 2017 01:55, "Andrew Haley" <aph@redhat.com> wrote:

On 31/03/17 19:56, Hans Boehm wrote:

...

This method should be used when cleanup actions (finalize() calls, or Reference enqueuing, Cleaner invocation) could otherwise be triggered while a resource under control of the object is still in use. This method should normally be called when these cleanup facilities are used to perform actions other than simply issuing a warning.

It's still pretty confusing. It would be simpler to say that reachabilityFence should be used at the end of every method of a class which has cleanup actions if you don't want those cleanup actions to be run before the end of that method. In that case, reachabilityFence only makes any difference in cases where otherwise you'd have a bug.

Andrew.