IllegalStateException in CharsetDecoder when inspecting JarFile contents on Java 15

[Claes Redestad]

Hi,

yes, this seems like a bug introduced by JDK-8243469 where the 
previously thread-safe UTF8ZipCoder is now using a thread-unsafe decoder 
to calculate the normalized hash. I'll file a bug and take a look.

/Claes

On 2021-01-19 20:19, Vitaly Davidovich wrote:
> Hi all,
>
> I observed the following stacktrace when inspecting a JarFile's contents
> using a parallel stream (i.e. FJ pool):
>
> Exception in thread "main" java.lang.IllegalStateException:
> java.lang.IllegalStateException: Current state = CODING_END, new state =
> FLUSHED
>
>                at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>
>                at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:64)
>
>                at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>
>                at
> java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
>
>                at
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
>
>                at
> java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:600)
>
>                at
> java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:678)
>
>                at
> java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:737)
>
>                at
> java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
>
>                at
> java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
>
>                at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
>
>                at
> java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
>
>                at
> java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:575)
>
>                at <stripped>.ClassVersions.conflicts(ClassVersions.java:703)
>
>                at <stripped>.ClassVersions.check(ClassVersions.java:743)
>
>                at <stripped>.ClassVersions.main(ClassVersions.java:839)
>
> Caused by: java.lang.IllegalStateException: Current state = CODING_END, new
> state = FLUSHED
>
>                at
> java.base/java.nio.charset.CharsetDecoder.throwIllegalStateException(CharsetDecoder.java:998)
>
>                at
> java.base/java.nio.charset.CharsetDecoder.flush(CharsetDecoder.java:681)
>
>                at
> java.base/java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:810)
>
>                at
> java.base/java.util.zip.ZipCoder.normalizedHashDecode(ZipCoder.java:136)
>
>                at
> java.base/java.util.zip.ZipCoder$UTF8ZipCoder.normalizedHash(ZipCoder.java:228)
>
>                at
> java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1527)
>
>                at
> java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:1249)
>
>                at
> java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1211)
>
>                at
> java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:701)
>
>                at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:240)
>
>                at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:171)
>
>                at java.base/java.util.jar.JarFile.<init>(JarFile.java:347)
>
>                at java.base/java.util.jar.JarFile.<init>(JarFile.java:318)
>
>                at java.base/java.util.jar.JarFile.<init>(JarFile.java:284)
>
>                at <stripped>.ClassVersions.lookupJar(ClassVersions.java:665)
>
>                at
> java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
>
>                at
> java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
>
>                at
> java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
>
>                at
> java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
>
>                at
> java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
>
>                at
> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
>
>                at
> java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
>
>                at
> java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
>
>                at
> java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
>
>                at
> java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1016)
>
>                at
> java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1665)
>
>                at
> java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1598)
>
>                at
> java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
>
>
> (Note: I stripped off some package names to protect the innocent).
>
>
> While investigating this a bit, I noticed some code that looks a bit
> suspect.  Namely:
>
>     - ZipCoder has a singleton UTF8ZipCoder
>     <https://github.com/openjdk/jdk/blob/05bcd67e6501f3824020edd1ab036c87888fa229/src/java.base/share/classes/java/util/zip/ZipCoder.java#L48>
>     instance it uses for UTF8 decoding.
>     - Source.get() creates a new Source
>     <https://github.com/openjdk/jdk/blob/05bcd67e6501f3824020edd1ab036c87888fa229/src/java.base/share/classes/java/util/zip/ZipFile.java#L1225>
>     instance outside of any synchronization
>
> The links above are to tip, but I think this code is the same in Java 15.
>
> It seems possible that multiple threads can end up in Source.get(), new up
> a Source, and proceed to use the decoder of UTF8ZipCoder racily.  Per my
> understanding (and CharsetDecoder javadoc), CharsetDecoders are not
> threadsafe.
>
> Any thoughts? Did I miss anything that would suggest the stacktrace above
> is due to something else entirely?
>
> Thanks