Finalizer being run while class still in use (escape analysis bug)
Volker Simonis
volker.simonis at gmail.com
Wed Oct 10 07:31:48 UTC 2018
Hi Luke,
I think the behaviour you see in not related to Escape Analysis at
all. It should also occur if you run with -XX:-DoEscapeAnalysis (note
that -XX:+DoEscapeAnalysis is the default). I also think that this
behaviour doesn't break the Java specification. You have to carefully
read the section cited by David.
Consider the following small, stand alone program which should
simulate your use case quite well:
=================================================
import java.util.ArrayList;
public class EAFinalizer {
private static ArrayList<Integer> cal = new ArrayList<Integer>(10_000);
static {
for (int i = 0; i < 10_000; i++) cal.add(i);
}
private static ArrayList<Object> sink = new ArrayList<Object>();
private ArrayList<Integer> al;
public EAFinalizer(ArrayList<Integer> al) {
this.al = new ArrayList<Integer>(al);
}
public static void main(String[] args) {
while(true) {
EAFinalizer eaf = new EAFinalizer(cal);
System.err.println("Iterating " + eaf);
for (Integer i : eaf.al) {
// Create some garbage to trigger GC and finalization
sink.add(new int[i.toString().length() * 1_000]);
}
// System.err.println("Iterated " + eaf);
}
}
@Override
protected void finalize() throws Throwable {
System.err.println("Finalizing " + this);
al.clear();
}
}
==================================================
Running this program (even with -XX:-DoEscapeAnalysis) will give you:
$ java -XX:-DoEscapeAnalysis EAFinalizer
Iterating EAFinalizer at 5ecddf8f
Iterating EAFinalizer at 3f102e87
Iterating EAFinalizer at 27abe2cd
Iterating EAFinalizer at 5f5a92bb
Iterating EAFinalizer at 6fdb1f78
Iterating EAFinalizer at 51016012
Iterating EAFinalizer at 29444d75
Iterating EAFinalizer at 2280cdac
Finalizing EAFinalizer at 2280cdac
Finalizing EAFinalizer at 29444d75
Exception in thread "main" java.util.ConcurrentModificationException
at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1042)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:996)
at EAFinalizer.main(EAFinalizer.java:21)
The problem with the program is that in the JIT compiled version of
"test()" the EAFinalizer object "eaf" is not reachable in the for loop
any more (because the JIT compiler is smart enough to detect that
after extracting the AraryList "al" from "eaf", "eaf" is no longer
needed). So when a GC is being triggered in the loop because the
allocation of the integer array fails, the VM can safely garbage
collect the "eaf" object and call its finalizer (notice that within
the loop we only reference "eaf"s ArrayList field "al", but not the
containing EAFinalizer object).
You can easily fix this little toy example by uncommenting the second
System.println statement in the "main()" method. This outputs "eaf"
and thus keeps it alive. You can also run the program in interpreted
mode ("-Xint") in which case you won't see the problem as well. That's
because the Interpreter doesn't optimizes as eagerly as the JIT
compiler and keeps "eaf" alive for the whole life time of the
corresponding method invocation. That's actually what most people
naivly expect to be the "guaranteed" life time of an object, but as
the specification tells us, that's not required by the standard. If it
would, the JIT would miss a whole lot of optimization opportunities.
Regards,
Volker
On Wed, Oct 10, 2018 at 2:36 AM David Holmes <david.holmes at oracle.com> wrote:
>
> Hi Luke,
>
> Moving this over to hotspot-dev. If it is an issue with the JIT escape
> analysis then hotspot-compiler-dev would be the place to raise this. But
> a general finalization issue may hit GC and runtime as well, so I
> redirected to the broader hotspot-dev.
>
> Also note that finalizers can run while methods of a class instance are
> still in progress [1]
>
> "Optimizing transformations of a program can be designed that reduce the
> number of objects that are reachable to be less than those which would
> naively be considered reachable."
>
> - this is one of the evil things about finalizers. So it may not be a
> bug, just surprising.
>
> Cheers,
> David
>
> [1] https://docs.oracle.com/javase/specs/jls/se11/html/jls-12.html#jls-12.6
>
> On 10/10/2018 10:27 AM, Luke Hutchison wrote:
> > A user of a library I maintain, ClassGraph, has reported that a finalizer
> > is being called on a class while methods of the class are still running,
> > indicating that the class, even though it overrides finalize(), is not
> > being correctly marked as GlobalEscape, so that escape analysis is trying
> > to garbage-collect the class too early. Is this mailing list the correct
> > place to report this bug?
> >
> > Here is the Maven rule for ClassGraph (you will also need to add Scala
> > deps) -- you will need version 4.2.8, since I put a mitigation in place in
> > version 4.2.9:
> >
> > <dependency>
> > <groupId>io.github.classgraph</groupId>
> > <artifactId>classgraph</artifactId>
> > <version>4.2.8</version>
> > </dependency>
> >
> > Here is the Scala code that reproduces the problem 100% of the time:
> >
> > import io.github.classgraph.{ClassGraph, ClassInfo}
> > import scala.collection.JavaConverters._
> > import scala.compat.Platform.ConcurrentModificationException
> >
> > while (true) {
> > try {
> > new ClassGraph().scan()
> > .getResourcesMatchingPattern(".*".r.pattern)
> > .asScala
> > .map(_.getURL)
> > } catch {
> > case cme: ConcurrentModificationException =>
> > cme.printStackTrace()
> > throw cme
> > }
> > }
> >
> > Then start the JVM with -XX:+DoEscapeAnalysis
> >
> > The code new ClassGraph().scan() returns a ScanResult, and then as soon as
> > ScanResult::getResourcesMatchingPattern is called with the ScanResult as
> > the invocation target, escape analysis sees that the ScanResult has "gone
> > out of scope" (despite the fact that the ScanResult is the invocation
> > target for a currently-running method), and immediately tries to garbage
> > collect it. This should not happen (1) since an invocation target should
> > never be marked for garbage collection while its methods are being run, and
> > (2) since ScanResult overrides Object::finalize, so should have been marked
> > as "GlobalEscape" by escape analysis.
> >
> > The finalizer for the ScanResult class calls ScanResult::close, which
> > clears the allResources list:
> >
> > https://github.com/classgraph/classgraph/blob/b9e8165d34575378697a727888457abb164311b8/src/main/java/io/github/classgraph/ScanResult.java#L940
> >
> > This results in a ConcurrentModificationException since
> > ScanResult::getResourcesMatchingPattern is still iterating through
> > allResources:
> >
> > https://github.com/classgraph/classgraph/blob/b9e8165d34575378697a727888457abb164311b8/src/main/java/io/github/classgraph/ScanResult.java#L383
> >
> > I read somewhere (and it makes sense) that escape analysis is performed by
> > the compiler, so maybe this is a Scala bug, but this behavior is triggered
> > by the JVM switch -XX:+DoEscapeAnalysis , and I couldn't find any
> > references to escape analysis information in the Java classfile format
> > spec, so that leads me to think this may be a JVM bug (which is why I am
> > asking on this list).
> >
> > Obviously having a garbage collector run a finalizer run while methods of
> > the object are still running is a very serious bug. Any pointers as to
> > where to report this issue would be appreciated. Thanks!
> >
More information about the jdk-dev
mailing list