RFC: Dynamic groups support

[Aleksey Shipilev]

On 11/19/2013 05:46 PM, Dmitry Chuyko wrote:
> It works. And results with dynamic groups are the same as with generated
> groups.

Good, thanks for checking!

> Single threads distribution for all groups matching specified pattern is
> handy but may be confusing if there are different number of methods in
> groups.

Indeed. In fact, we should prohibit executing the benchmarks when there
is discrepancy between what is supplied in -tg, and what benchmark accepts.

> Groups multiplication is a powerful but not obvious feature along with
> rounding. But for me it would be more clear to write also @Threads(32),
> @GroupThreads(.25), @GroupThreads(.75) (or -t 32 -tg 1,4 or -t 32 -tg
> 16.0,48.0) and get 8 and 24.

Passing ratios instead of absolute counts works great for the single
group only. There are cases when we need several groups. Also, the
numbers in annotations and options should be consistent with each other.
(Note that we don't have the easy property overload in annotations, and
hence can't have both int/double parameters to assign different meanings
to them).

Here's the illustration. Let's consider a few use cases.

Suppose I have the benchmark like this:

  class T {
    @GMB @Group @GroupThreads(1)
    void testA() {}

    @GMB @Group @GroupThreads(4)
    void testB() {}
  }

Then, the options I have to run with:
  a) 1 group, {1,4} threads:
         (leave the default)
         (-tg 1,4)
  b) 1 group, {2,8} threads:
         (-tg 2,8)
  c) 2 groups, {1,4} threads:
         (-t 10)

Note the rounding up brings in the great feature: I can skip setting -t
at all, and only use thread distribution to drive the thread counts.

Now, with your suggestion it seems to require:

  class T {
    @GMB @Group @GroupThreads(0.2)
    void testA() {}

    @GMB @Group @GroupThreads(0.8)
    void testB() {}
  }

Then, the options I have to run with:
  a) 1 group, {1,4} threads:
         (default)
         (-tg 0.2, 0.8}
         (-tg (1.0*N),(4.0*N))
         (-tg 1,4)
  b) 1 group, {2,8} threads:
         (-t 10)
  c) 2 groups, {1,4} threads:
         (whoops, what do I do here?!)

On these grounds, I'm still standing on my initial suggestion.
Additional point of frustration is the rounding itself. What is the
semantics of this code?

   @Threads(1)
   class T {
    @GMB @Group @GroupThreads(0.2)
    void testA() {}

    @GMB @Group @GroupThreads(0.8)
    void testB() {}
  }

Should it run only the single thread, associated randomly with testA or
testB? Should it be biased to become testB? What if there are two
threads? Three? Four? Six? Should we search for some thread number which
fits these ratios exactly? Absolute counts evade these tricky questions.

Thanks,
-Aleksey.