RFR: 8364655: Loading class with nested annotations causes stack overflow in VM

[Johan Sjölen]

Hi,

`skip_annotation` and `skip_annotation_value` are two mutually recursive functions calling each other in order to skip over classfile annotations. If a classfile contains a highly nested annotation, then this will lead to a stack overflow and a subsequent crash of the JVM. I propose that we insert a recursion limit to prevent this from happening.

This recursion limit will make the annotation parsing to bail out on the JVM side, skipping the rest of the annotations present. This is fine! The reason that the JVM parses these annotations is to gain access to a select few JDK-internal annotations, and we trust our own code to construct classfiles without egregious nesting. All classfile annotations are also parsed by Java-code, so we're not skipping general annotation parsing.

A regression test has been added.

Thanks!

-------------

Commit messages:
 - Untabify
 - Spelling mistake
 - Add a recursion limit

Changes: https://git.openjdk.org/jdk/pull/28674/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=28674&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8364655
  Stats: 90 lines in 2 files changed: 84 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/28674.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/28674/head:pull/28674

PR: https://git.openjdk.org/jdk/pull/28674