RFR: 8296426: x86: Narrow UseAVX and UseSSE flags

[Claes Redestad]

This patch narrows down the UseAVX and UseSSE flags to their actual supported range and uses int rather than intx for their type. This avoids need for some silly casts, and surprisingly has a small beneficial effect to binary size (-4kb libjvm on linux-x64)

This changes behavior of previously in-range values: `-XX:UseAVX=4` would emit a strongly worded warning, but with the proposed change we'll instead terminate the JVM with an error similar to `-XX:UseAVX=100`. I believe this is too trivial for a CSR, since it only changes behavior for unsupported values.

-------------

Commit messages:
 - Merge
 - Narrow UseAVX and UseSSE flags

Changes: https://git.openjdk.org/jdk/pull/10997/files
 Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=10997&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8296426
  Stats: 11 lines in 2 files changed: 0 ins; 0 del; 11 mod
  Patch: https://git.openjdk.org/jdk/pull/10997.diff
  Fetch: git fetch https://git.openjdk.org/jdk pull/10997/head:pull/10997

PR: https://git.openjdk.org/jdk/pull/10997