IllegalStateException in CharsetDecoder when inspecting JarFile contents on Java 15

[Vitaly Davidovich]

Hi all,

I observed the following stacktrace when inspecting a JarFile's contents
using a parallel stream (i.e. FJ pool):

Exception in thread "main" java.lang.IllegalStateException:
java.lang.IllegalStateException: Current state = CODING_END, new state =
FLUSHED

              at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)

              at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64)

              at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

              at
java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)

              at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)

              at
java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:600)

              at
java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:678)

              at
java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:737)

              at
java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)

              at
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)

              at
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)

              at
java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)

              at
java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:575)

              at <stripped>.ClassVersions.conflicts(ClassVersions.java:703)

              at <stripped>.ClassVersions.check(ClassVersions.java:743)

              at <stripped>.ClassVersions.main(ClassVersions.java:839)

Caused by: java.lang.IllegalStateException: Current state = CODING_END, new
state = FLUSHED

              at
java.base/java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:998)

              at
java.base/java.nio.charset.CharsetDecoder.flush(CharsetDecoder.java:681)

              at
java.base/java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:810)

              at
java.base/java.util.zip.ZipCoder.normalizedHashDecode(ZipCoder.java:136)

              at
java.base/java.util.zip.ZipCoder$UTF8ZipCoder.normalizedHash(ZipCoder.java:228)

              at
java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1527)

              at
java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:1249)

              at
java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1211)

              at
java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:701)

              at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:240)

              at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:171)

              at java.base/java.util.jar.JarFile.<init>(JarFile.java:347)

              at java.base/java.util.jar.JarFile.<init>(JarFile.java:318)

              at java.base/java.util.jar.JarFile.<init>(JarFile.java:284)

              at <stripped>.ClassVersions.lookupJar(ClassVersions.java:665)

              at
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)

              at
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)

              at
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)

              at
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)

              at
java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)

              at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)

              at
java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)

              at
java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)

              at
java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)

              at
java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1016)

              at
java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1665)

              at
java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1598)

              at
java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)


(Note: I stripped off some package names to protect the innocent).


While investigating this a bit, I noticed some code that looks a bit
suspect.  Namely:

   - ZipCoder has a singleton UTF8ZipCoder
   <https://github.com/openjdk/jdk/blob/05bcd67e6501f3824020edd1ab036c87888fa229/src/java.base/share/classes/java/util/zip/ZipCoder.java#L48>
   instance it uses for UTF8 decoding.
   - Source.get() creates a new Source
   <https://github.com/openjdk/jdk/blob/05bcd67e6501f3824020edd1ab036c87888fa229/src/java.base/share/classes/java/util/zip/ZipFile.java#L1225>
   instance outside of any synchronization

The links above are to tip, but I think this code is the same in Java 15.

It seems possible that multiple threads can end up in Source.get(), new up
a Source, and proceed to use the decoder of UTF8ZipCoder racily.  Per my
understanding (and CharsetDecoder javadoc), CharsetDecoders are not
threadsafe.

Any thoughts? Did I miss anything that would suggest the stacktrace above
is due to something else entirely?

Thanks