RFR: 8234003: Improve IndexSet iteration

[Claes Redestad]

Hi,

a significant portion of work done during register allocation in C2 is
iterating over IndexSets.

A few small optimizations show a ~4% decrease in instructions retired by
register allocation when instrumenting, and up to 3% fewer instructions
retired in total on startup tests.

Bug:    https://bugs.openjdk.java.net/browse/JDK-8234003
Webrev: http://cr.openjdk.java.net/~redestad/8234003/open.00/

The biggest improvement comes from avoiding iterating over empty sets
altogether. A smaller improvement from adding a water mark to avoid
iterating over all the blocks in the IndexSet.

Testing: tier1-3, verified improvements on large and tiny startup tests,
checked that any increased inlining is footprint neutral on Linux.

Thanks!

/Claes