Optimal place for inserting calls to the VM from interpreted java methods
Is there a place to encode calls to the VM in the interpreted methods other than the method prologs? I've recently discovered the interpreted method adaptors which seems like a candidate, but there's no examples of the adaptors calling into the VM. Follows is the rational as to why I'm looking for an alternative to the method prologs The AOTEndTrainingOnMethodEntry feature [1] introduces calls from java methods into the VM (upcalls) when specific methods are entered. The methods are identified via a pattern in a similar manner to the CompileOnly option. Following the initial PR review we’re looking at removing the knowledge of this AOT feature from the compilers/interpreter and introducing a more generic system (RuntimeUpcalls) that can be used by other parts of the VM [2]. In building out the RuntimeUpcalls system we've come across an inefficiency that isn't an immediate problem for this feature, but should another feature use this new system then it's less than optimal. Interpreted code uses a shared method prolog (there are 8 variants for 'regular' methods [more for some special math/zip methods]), the AOTEndTrainingOnMethodEntry feature introduces a further 5 prolog types. When there is a single upcall (eg. AOTEndTrainingOnMethodEntry) to the VM everything is efficient. The inefficiency issue arises as soon as there are two or more upcalls; which upcalls relate to which methods is contained within the RuntimeUpcalls system. When the interpreter examines the method flags they only indicate whether there are any upcalls (but not how many or which ones). As the interpreter can't encode which upcalls should be called in the prolog (without an explosion of new runtime generated prologs), it needs to call the RuntimeUpcalls system which in turn iterates over the upcalls and calls the appropriate ones; the problem is that during that iteration the methods need to be compared against the pattern. So we either pay a memory cost to cache the method to upcall relationships or we pay a performance cost to repeatedly test the method against the pattern. This is not a problem for C1 and C2 as we pay this cost only when the methods are compiled and create the multiple upcalls in those methods, eliminating the need for pattern matching by the RuntimeUpcalls system during method execution Thanks in advance Mat [1] https://github.com/openjdk/leyden/pull/21 [2] https://github.com/macarte/leyden/pull/2
Generally speaking, when trading off complexity against performance in interpreted code paths, performance always loses. We only add complexity after the less complex solution has proven itself to be a measurable problem for real workloads. On Oct 9, 2024, at 6:43 PM, Mat Carter <Matthew.Carter@microsoft.com> wrote: Is there a place to encode calls to the VM in the interpreted methods other than the method prologs? I've recently discovered the interpreted method adaptors which seems like a candidate, but there's no examples of the adaptors calling into the VM. Follows is the rational as to why I'm looking for an alternative to the method prologs The AOTEndTrainingOnMethodEntry feature [1] introduces calls from java methods into the VM (upcalls) when specific methods are entered. The methods are identified via a pattern in a similar manner to the CompileOnly option. Following the initial PR review we’re looking at removing the knowledge of this AOT feature from the compilers/interpreter and introducing a more generic system (RuntimeUpcalls) that can be used by other parts of the VM [2]. In building out the RuntimeUpcalls system we've come across an inefficiency that isn't an immediate problem for this feature, but should another feature use this new system then it's less than optimal. Interpreted code uses a shared method prolog (there are 8 variants for 'regular' methods [more for some special math/zip methods]), the AOTEndTrainingOnMethodEntry feature introduces a further 5 prolog types. When there is a single upcall (eg. AOTEndTrainingOnMethodEntry) to the VM everything is efficient. The inefficiency issue arises as soon as there are two or more upcalls; which upcalls relate to which methods is contained within the RuntimeUpcalls system. When the interpreter examines the method flags they only indicate whether there are any upcalls (but not how many or which ones). As the interpreter can't encode which upcalls should be called in the prolog (without an explosion of new runtime generated prologs), it needs to call the RuntimeUpcalls system which in turn iterates over the upcalls and calls the appropriate ones; the problem is that during that iteration the methods need to be compared against the pattern. So we either pay a memory cost to cache the method to upcall relationships or we pay a performance cost to repeatedly test the method against the pattern. This is not a problem for C1 and C2 as we pay this cost only when the methods are compiled and create the multiple upcalls in those methods, eliminating the need for pattern matching by the RuntimeUpcalls system during method execution Thanks in advance Mat [1] https://github.com/openjdk/leyden/pull/21 [2] https://github.com/macarte/leyden/pull/2 <image.png>
Hi Mat. It sounds like you could use multiple entry points based on MethodKind to accomplish this. Add new entries to the MethodKind enum for the upcall versions. Generate both the upcall and non-upcall entry points, and map to appropriate MethodKind values. Then use your runtime pattern match to decide with MethodKind to use for each method in AbstractInterpreter <https://java.se.oracle.com/source/s?defs=AbstractInterpreter&project=jdk-cpu-git>::method_kind <https://java.se.oracle.com/source/s?defs=method_kind&project=jdk-cpu-git>(*const* methodHandle <https://java.se.oracle.com/source/s?defs=methodHandle&project=jdk-cpu-git>& m <https://java.se.oracle.com/source/s?defs=m&project=jdk-cpu-git>). dl On 10/9/24 6:42 PM, Mat Carter wrote:
Is there a place to encode calls to the VM in the interpreted methods other than the method prologs?
I've recently discovered the interpreted method adaptors which seems like a candidate, but there's
no examples of the adaptors calling into the VM.
Follows is the rational as to why I'm looking for an alternative to the method prologs
The AOTEndTrainingOnMethodEntry feature [1] introduces calls from java methods into the
VM (upcalls) when specific methods are entered. The methods are identified via a pattern in a
similar manner to the CompileOnly option.
Following the initial PR review we’re looking at removing the knowledge of this AOT feature from
the compilers/interpreter and introducing a more generic system (RuntimeUpcalls) that can be
used by other parts of the VM [2]. In building out the RuntimeUpcalls system we've come across
an inefficiency that isn't an immediate problem for this feature, but should another feature use this
new system then it's less than optimal.
Interpreted code uses a shared method prolog (there are 8 variants for 'regular' methods [more
for some special math/zip methods]), the AOTEndTrainingOnMethodEntry feature introduces a
further 5 prolog types. When there is a single upcall (eg. AOTEndTrainingOnMethodEntry) to the VM
everything is efficient.
The inefficiency issue arises as soon as there are two or more upcalls; which upcalls relate to
which methods is contained within the RuntimeUpcalls system. When the interpreter examines
the method flags they only indicate whether there are any upcalls (but not how many or which ones).
As the interpreter can't encode which upcalls should be called in the prolog (without an explosion
of new runtime generated prologs), it needs to call the RuntimeUpcalls system which in turn iterates
over the upcalls and calls the appropriate ones; the problem is that during that iteration the methods
need to be compared against the pattern. So we either pay a memory cost to cache the method to
upcall relationships or we pay a performance cost to repeatedly test the method against the pattern.
This is not a problem for C1 and C2 as we pay this cost only when the methods are compiled and
create the multiple upcalls in those methods, eliminating the need for pattern matching by the
RuntimeUpcalls system during method execution
Thanks in advance
Mat
Thanks Dean That is indeed the way I ended up going. I have a draft PR [1] up with these changes Cheers Mat [1] https://github.com/openjdk/leyden/pull/21 From: leyden-dev <leyden-dev-retn@openjdk.org> on behalf of dean.long@oracle.com <dean.long@oracle.com> Date: Friday, October 18, 2024 at 11:39 AM To: leyden-dev@openjdk.org <leyden-dev@openjdk.org> Subject: Re: Optimal place for inserting calls to the VM from interpreted java methods Hi Mat. It sounds like you could use multiple entry points based on MethodKind to accomplish this. Add new entries to the MethodKind enum for the upcall versions. Generate both the upcall and non-upcall entry points, and map to appropriate MethodKind values. Then use your runtime pattern match to decide with MethodKind to use for each method in AbstractInterpreter<https://java.se.oracle.com/source/s?defs=AbstractInterpreter&project=jdk-cpu-git>::method_kind<https://java.se.oracle.com/source/s?defs=method_kind&project=jdk-cpu-git>(const methodHandle<https://java.se.oracle.com/source/s?defs=methodHandle&project=jdk-cpu-git>& m<https://java.se.oracle.com/source/s?defs=m&project=jdk-cpu-git>). dl On 10/9/24 6:42 PM, Mat Carter wrote: Is there a place to encode calls to the VM in the interpreted methods other than the method prologs? I've recently discovered the interpreted method adaptors which seems like a candidate, but there's no examples of the adaptors calling into the VM. Follows is the rational as to why I'm looking for an alternative to the method prologs The AOTEndTrainingOnMethodEntry feature [1] introduces calls from java methods into the VM (upcalls) when specific methods are entered. The methods are identified via a pattern in a similar manner to the CompileOnly option. Following the initial PR review we’re looking at removing the knowledge of this AOT feature from the compilers/interpreter and introducing a more generic system (RuntimeUpcalls) that can be used by other parts of the VM [2]. In building out the RuntimeUpcalls system we've come across an inefficiency that isn't an immediate problem for this feature, but should another feature use this new system then it's less than optimal. Interpreted code uses a shared method prolog (there are 8 variants for 'regular' methods [more for some special math/zip methods]), the AOTEndTrainingOnMethodEntry feature introduces a further 5 prolog types. When there is a single upcall (eg. AOTEndTrainingOnMethodEntry) to the VM everything is efficient. The inefficiency issue arises as soon as there are two or more upcalls; which upcalls relate to which methods is contained within the RuntimeUpcalls system. When the interpreter examines the method flags they only indicate whether there are any upcalls (but not how many or which ones). As the interpreter can't encode which upcalls should be called in the prolog (without an explosion of new runtime generated prologs), it needs to call the RuntimeUpcalls system which in turn iterates over the upcalls and calls the appropriate ones; the problem is that during that iteration the methods need to be compared against the pattern. So we either pay a memory cost to cache the method to upcall relationships or we pay a performance cost to repeatedly test the method against the pattern. This is not a problem for C1 and C2 as we pay this cost only when the methods are compiled and create the multiple upcalls in those methods, eliminating the need for pattern matching by the RuntimeUpcalls system during method execution Thanks in advance Mat [1] https://github.com/openjdk/leyden/pull/21 [2] https://github.com/macarte/leyden/pull/2
participants (3)
-
dean.long＠oracle.com
-
John Rose
-
Mat Carter