From serkanozal86 at hotmail.com Mon Sep 8 17:56:39 2014 From: serkanozal86 at hotmail.com (=?windows-1254?B?c2Vya2FuIPZ6YWw=?=) Date: Mon, 8 Sep 2014 20:56:39 +0300 Subject: More reliable compressed-oops patch for JOL and some simple improvements Message-ID: From aleksey.shipilev at oracle.com Mon Sep 8 18:01:15 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 08 Sep 2014 22:01:15 +0400 Subject: More reliable compressed-oops patch for JOL and some simple improvements In-Reply-To: References: Message-ID: <540DEEEB.4060009@oracle.com> Hi Serkan, I think the attachment was scrubbed due to the size limit. I adjusted the limit now, please try again? Thanks, -Aleksey. On 09/08/2014 09:56 PM, serkan ?zal wrote: > > From serkanozal86 at hotmail.com Mon Sep 8 18:03:17 2014 From: serkanozal86 at hotmail.com (=?windows-1254?B?c2Vya2FuIPZ6YWw=?=) Date: Mon, 8 Sep 2014 21:03:17 +0300 Subject: More reliable compressed-oops patch for JOL and some simple improvements [Patch reattached] Message-ID: From serkanozal86 at hotmail.com Mon Sep 8 18:20:08 2014 From: serkanozal86 at hotmail.com (=?windows-1254?B?c2Vya2FuIPZ6YWw=?=) Date: Mon, 8 Sep 2014 21:20:08 +0300 Subject: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] Message-ID: Patch Link: https://raw.githubusercontent.com/serkan-ozal/dropbox/master/compressed-oops.patch From aleksey.shipilev at oracle.com Tue Sep 9 15:48:43 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 09 Sep 2014 19:48:43 +0400 Subject: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] In-Reply-To: References: Message-ID: <540F215B.6020801@oracle.com> Hi Serkan, On 09/08/2014 10:20 PM, serkan ?zal wrote: > Patch Link: https://raw.githubusercontent.com/serkan-ozal/dropbox/master/compressed-oops.patch Thanks for doing this! I think we would need a few rounds of code reviews to see what is going on there. Comments: * I think the code is in a dire need of "Implementation notes" section explaining what it tries to do. Previous code was also scarce on that, but with this change, it is really over the top. * Are we sure the magic offsets in HotspotJvmClassAddressFinder are stable across JDKs? I would not be that sure, but let's talk about that later. * Do we need to have the AbstractHotspotJvmClassAddressFinder hierarchy? It seems you can just turn the Abstract* class into the concrete one, and accept the offsets there. * JvmInfo and JavaVersionInfo enums feel redundant. Would you like to just make JAVA*.equals() calls in the getters like isJava_6()? * Please use shorter identifiers. Identifiers like CLASS_DEFINITION_POINTER_OFFSET_IN_CLASS_INSTANCE_64_BIT_WITHOUT_COMPRESSED_REF_FOR_JAVA_8 are actually hard to read. CLASSDEF_CI_OFFSET_64_JAVA8 seems better. This applies to field/local names as well. * We use 4 spaces as the indentation units, not tabs. Please also auto-format the source, you have the indentation gone wrong a few times. * Do we actually need these two guys? private static final byte ADDRESS_SIZE_4_BYTE = 4; private static final byte ADDRESS_SIZE_8_BYTE = 8; * "////////////////////////" is not a good delimiter in our codebase. Use "/* ------------------------- */"? Thanks, -Aleksey. From serkanozal86 at hotmail.com Tue Sep 9 18:30:34 2014 From: serkanozal86 at hotmail.com (=?windows-1254?B?c2Vya2FuIPZ6YWw=?=) Date: Tue, 9 Sep 2014 21:30:34 +0300 Subject: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] In-Reply-To: References: , <540F215B.6020801@oracle.com>, Message-ID: Hi Aleksey, Thanks for your comments. My comments are inline. > * I think the code is in a dire need of "Implementation notes" section > explaining what it tries to do. Previous code was also scarce on that, > but with this change, it is really over the top. Right, I can write an implementation details comments at last to describe what we did for compressed-oops. > * Are we sure the magic offsets in HotspotJvmClassAddressFinder are > stable across JDKs? I would not be that sure, but let's talk about that > later. Currenly, I got out these magic offsets from native codes of JDK. In my opinion, to be sure about them, * Test as much as all published JDK binaries :) (for JDK 7, such as jdk1.7_u17, jdk1.7_u21, etc ...) * Looking commit logs of these files if ltheir ayout is changed or not. What do you think about how we can be sure for these magic offsets? > * Do we need to have the AbstractHotspotJvmClassAddressFinder > hierarchy? It seems you can just turn the Abstract* class into the > concrete one, and accept the offsets there. Yeah, we can use just concrete "HotspotJvmClassAddressFinder" by giving offsets as argument. > * JvmInfo and JavaVersionInfo enums feel redundant. Would you like to > just make JAVA*.equals() calls in the getters like isJava_6()? Ok, will replace enums with "JAVA*.equals()" like calls. > * Please use shorter identifiers. Identifiers like > CLASS_DEFINITION_POINTER_OFFSET_IN_CLASS_INSTANCE_64_BIT_WITHOUT_COMPRESSED_REF_FOR_JAVA_8 > are actually hard to read. CLASSDEF_CI_OFFSET_64_JAVA8 seems better. > This applies to field/local names as well. Right, they are too long :) will trim them. > * We use 4 spaces as the indentation units, not tabs. Please also > auto-format the source, you have the indentation gone wrong a few times. sure, will format with spaces instead of tabs. > * Do we actually need these two guys? > private static final byte ADDRESS_SIZE_4_BYTE = 4; > private static final byte ADDRESS_SIZE_8_BYTE = 8; yes, can use just 4 or 8 instead of constant > * "////////////////////////" is not a good delimiter in our codebase. > Use "/* ------------------------- */"? Ok Regards. -- Serkan ?ZAL > Date: Tue, 9 Sep 2014 19:48:43 +0400 > From: aleksey.shipilev at oracle.com > To: serkanozal86 at hotmail.com; jol-dev at openjdk.java.net > Subject: Re: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] > > Hi Serkan, > > On 09/08/2014 10:20 PM, serkan ?zal wrote: > > Patch Link: https://raw.githubusercontent.com/serkan-ozal/dropbox/master/compressed-oops.patch > > Thanks for doing this! I think we would need a few rounds of code > reviews to see what is going on there. > > Comments: > > * I think the code is in a dire need of "Implementation notes" section > explaining what it tries to do. Previous code was also scarce on that, > but with this change, it is really over the top. > > * Are we sure the magic offsets in HotspotJvmClassAddressFinder are > stable across JDKs? I would not be that sure, but let's talk about that > later. > > * Do we need to have the AbstractHotspotJvmClassAddressFinder > hierarchy? It seems you can just turn the Abstract* class into the > concrete one, and accept the offsets there. > > * JvmInfo and JavaVersionInfo enums feel redundant. Would you like to > just make JAVA*.equals() calls in the getters like isJava_6()? > > * Please use shorter identifiers. Identifiers like > CLASS_DEFINITION_POINTER_OFFSET_IN_CLASS_INSTANCE_64_BIT_WITHOUT_COMPRESSED_REF_FOR_JAVA_8 > are actually hard to read. CLASSDEF_CI_OFFSET_64_JAVA8 seems better. > This applies to field/local names as well. > > * We use 4 spaces as the indentation units, not tabs. Please also > auto-format the source, you have the indentation gone wrong a few times. > > * Do we actually need these two guys? > private static final byte ADDRESS_SIZE_4_BYTE = 4; > private static final byte ADDRESS_SIZE_8_BYTE = 8; > > * "////////////////////////" is not a good delimiter in our codebase. > Use "/* ------------------------- */"? > > Thanks, > -Aleksey. > From aleksey.shipilev at oracle.com Wed Sep 10 06:55:53 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 10 Sep 2014 10:55:53 +0400 Subject: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] In-Reply-To: References: , <540F215B.6020801@oracle.com>, Message-ID: <540FF5F9.6010908@oracle.com> On 09/09/2014 10:30 PM, serkan ?zal wrote: > Currenly, I got out these magic offsets from native codes of JDK. In > my opinion, to be sure about them, * Test as much as all > published JDK binaries :) (for JDK 7, such as jdk1.7_u17, jdk1.7_u21, > etc ...) * Looking commit logs of these files if ltheir ayout is > changed or not. What do you think about how we can be sure for these > magic offsets? This does add significant maintenance burden. I am thinking: this is why we need to involve Serviceability Agent. Hard-coding the offsets seems very wrong. Can we instead ask user to supply the relevant sa-jdi.jar, and pull the offsets from there? -Aleksey. From serkanozal86 at hotmail.com Wed Sep 10 07:08:26 2014 From: serkanozal86 at hotmail.com (=?windows-1254?B?c2Vya2FuIPZ6YWw=?=) Date: Wed, 10 Sep 2014 10:08:26 +0300 Subject: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] In-Reply-To: References: , , <540F215B.6020801@oracle.com>, , , <540FF5F9.6010908@oracle.com>, Message-ID: Hi, As I seen, there are two problems/disadvantages with Hotspot SA: 1. Process attach problem on MacOSX2. "sa-jdi.jar" version must be same with current JVM version. For example, if the program is run on Java 7, "sa-jdi.jar" for Java 7 must be used. "sa-jdi.jar" of Java 6 is not work on Java 7. But to be sure, I will check it again. However there are two advantages of Hotspot SA: 1. We can always be sure that we get right compressed-oops values since they are directly taken by JVM2. For future works, Hotspot SA gives us more opportunities to play on JVM :)3. Less complex code What do you think ? > Date: Wed, 10 Sep 2014 10:55:53 +0400 > From: aleksey.shipilev at oracle.com > To: serkanozal86 at hotmail.com; jol-dev at openjdk.java.net > Subject: Re: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] > > On 09/09/2014 10:30 PM, serkan ?zal wrote: > > Currenly, I got out these magic offsets from native codes of JDK. In > > my opinion, to be sure about them, * Test as much as all > > published JDK binaries :) (for JDK 7, such as jdk1.7_u17, jdk1.7_u21, > > etc ...) * Looking commit logs of these files if ltheir ayout is > > changed or not. What do you think about how we can be sure for these > > magic offsets? > > This does add significant maintenance burden. > > I am thinking: this is why we need to involve Serviceability Agent. > Hard-coding the offsets seems very wrong. Can we instead ask user to > supply the relevant sa-jdi.jar, and pull the offsets from there? > > -Aleksey. > > > From serkanozal86 at hotmail.com Wed Sep 10 07:18:13 2014 From: serkanozal86 at hotmail.com (=?windows-1254?B?c2Vya2FuIPZ6YWw=?=) Date: Wed, 10 Sep 2014 10:18:13 +0300 Subject: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] In-Reply-To: References: , , <540F215B.6020801@oracle.com>, , , , <540FF5F9.6010908@oracle.com>, , Message-ID: Hi all, Sorry for my previous bad formatted mail :( As I seen, there are two problems/disadvantages with Hotspot SA: * Process attach problem on MacOSX * "sa-jdi.jar" version must be same with current JVM version. For example, if the program is run on Java 7, "sa-jdi.jar" for Java 7 must be used. "sa-jdi.jar" of Java 6 is not work on Java 7. But to be sure, I will check it again. However there are three advantages of Hotspot SA: * We can always be sure that we get right compressed-oops values since they are directly taken by JVM * For future works, Hotspot SA gives us more opportunities to play on JVM :) * Less complex code What do you think ? -- Serkan ?ZAL > Date: Wed, 10 Sep 2014 10:55:53 +0400 > From: aleksey.shipilev at oracle.com > To: serkanozal86 at hotmail.com; jol-dev at openjdk.java.net > Subject: Re: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] > > On 09/09/2014 10:30 PM, serkan ?zal wrote: > > Currenly, I got out these magic offsets from native codes of JDK. In > > my opinion, to be sure about them, * Test as much as all > > published JDK binaries :) (for JDK 7, such as jdk1.7_u17, jdk1.7_u21, > > etc ...) * Looking commit logs of these files if ltheir ayout is > > changed or not. What do you think about how we can be sure for these > > magic offsets? > > This does add significant maintenance burden. > > I am thinking: this is why we need to involve Serviceability Agent. > Hard-coding the offsets seems very wrong. Can we instead ask user to > supply the relevant sa-jdi.jar, and pull the offsets from there? > > -Aleksey. > > > From aleksey.shipilev at oracle.com Wed Sep 10 07:28:27 2014 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 10 Sep 2014 11:28:27 +0400 Subject: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] In-Reply-To: References: , , <540F215B.6020801@oracle.com>, , , , <540FF5F9.6010908@oracle.com>, , Message-ID: <540FFD9B.4090002@oracle.com> On 09/10/2014 11:18 AM, serkan ?zal wrote: > As I seen, there are two problems/disadvantages with Hotspot SA: > * Process attach problem on MacOSX Do we really need to attach? Can we just poke the named constant from SA jar? Asking seriously since I haven't looked what's in there. > * "sa-jdi.jar" version must be same with current JVM version. For > example, if the program is run on Java 7, "sa-jdi.jar" for Java 7 > must be used. "sa-jdi.jar" of Java 6 is not work on Java 7. But to be > sure, I will check it again. Well, that is the whole point of SA, right? We pack the internal VM information in a separate JAR, which we can then reference from Java. > However there are three advantages of Hotspot SA: > * We can always be sure that we get right compressed-oops values since they are directly taken by JVM > * For future works, Hotspot SA gives us more opportunities to play on JVM :) > * Less complex code Yes. -Aleksey. From serkanozal86 at hotmail.com Wed Sep 10 07:41:32 2014 From: serkanozal86 at hotmail.com (=?iso-8859-1?Q?Serkan_=D6ZAL?=) Date: Wed, 10 Sep 2014 10:41:32 +0300 Subject: More reliable compressed-oops patch for JOL and some simple improvements [Patch Linked] In-Reply-To: <540FFD9B.4090002@oracle.com> References: , , <540F215B.6020801@oracle.com>, , , , <540FF5F9.6010908@oracle.com>, , <540FFD9B.4090002@oracle.com> Message-ID: Yes, we need to attach to the process. As far as I know and seen from the code there is no way. Because these offsets are not defined as public member and they are getting from JVM process by reading symbols on dynamically loaded jvm library (dll, so, etc ?) I had many previous attempts to get them without attaching but I couldn?t find a way :(. But I will look it again to find if there is another way without attaching. For current Hotspot SA solution, on my application, I create a new process with same classpath of current process and attach this process to caller process as Hotspot SA agent. Then I get compressed-oops informations and then back to caller process via pipeline between processes. https://github.com/serkan-ozal/jillegal/blob/master/src/main/java/tr/com/serkanozal/jillegal/util/HotspotJvmInfoUtil.java By the way, when a Hotspot SA agent attaches to JVM process, attached application is suspended. So instead of attaching to itself, I create another process for attaching to current process. Currently I have no other solution idea and all your suggestions are welcome. ? Serkan ?ZAL On 10 Sep 2014, at 10:28, Aleksey Shipilev wrote: > On 09/10/2014 11:18 AM, serkan ?zal wrote: >> As I seen, there are two problems/disadvantages with Hotspot SA: >> * Process attach problem on MacOSX > > Do we really need to attach? Can we just poke the named constant from SA > jar? Asking seriously since I haven't looked what's in there. > >> * "sa-jdi.jar" version must be same with current JVM version. For >> example, if the program is run on Java 7, "sa-jdi.jar" for Java 7 >> must be used. "sa-jdi.jar" of Java 6 is not work on Java 7. But to be >> sure, I will check it again. > > Well, that is the whole point of SA, right? We pack the internal VM > information in a separate JAR, which we can then reference from Java. > >> However there are three advantages of Hotspot SA: >> * We can always be sure that we get right compressed-oops values since they are directly taken by JVM >> * For future works, Hotspot SA gives us more opportunities to play on JVM :) >> * Less complex code > > Yes. > > -Aleksey. > >