From Y.S.Ramakrishna at Sun.COM Tue Jan 1 11:19:50 2008 From: Y.S.Ramakrishna at Sun.COM (Y Srinivas Ramakrishna) Date: Tue, 01 Jan 2008 11:19:50 -0800 Subject: How "evil" is GetPrimitiveArrayCritical? In-Reply-To: <194f62550712310258p3cff8c17i98fb7561bec9d02b@mail.gmail.com> References: <194f62550712301231n4570872fj6b11680dffd39095@mail.gmail.com> <87sl1jdfrh.fsf@mid.deneb.enyo.de> <194f62550712310258p3cff8c17i98fb7561bec9d02b@mail.gmail.com> Message-ID: By the way ... > > or lock out. Thus, for example, in your case, GC > > may be delayed by as much as 2 ms. During this time, allocation requests ... that is true assuming single-threaded calls to the CS's, but of course, as you noted, this approach does not scale as one increases the number of such threads or the size of the array chunk that you process during a JNI/CS. > So, to make it short - which implementation should I prefer? > The one which copies the data into a DirectByteBuffer or the > implementation which uses Get/ReleaseCritical and holds the lock until > it processes a small piece of data? (for max 2ms, but more likely > arround 500us). Honestly, i am not sure, but perhaps someone else on this (or the core libraries) list may have direct experience. > > Copying means two additional JNI call per stride, two > Get/ReleaseCritical pair which are held only short, and also managing > the native memory. The best approach might be (if that is possible, avoiding the copying and keeping all data in the direct byte buffer during its entire tenure). That avoids the critical section entirely and makes the solution more scalable, i guess. > I benchmarked a bit and it seems both implementations perform almost > equally, however the code for the ByteBuffer-copying approach is more > complex and means a lot more offset/lenght management. The > OpenJDK-Corelibs guys seem a bit affraid of the Get/ReleaseCritical > approach, thats why I am nerving that much here ;) > If you have only a few threads which each process relatively small chunks of the buffer in each JNI/CS, then you should stick with the Get/ReleaseCritical approach and avoid then copying entirely. If on the other hand you expect to have lots of threads and expect lots of concurrency, then i'd go for a direct byte buffer approach but avoid copying (or at least amortize the cost of any copying over much larger and heavyweight processing of the copied buffer). In the end it's a trade-off, and it's true that Get/ReleaseCritrical on the one hand as well as copying on the other will both be inhibitors to scaling. On the third hand, direct byte buffer management itself is, as you stated, extra work and messy. Sorry, the above is more armchair theorization than borne out of any direct experience with either approach so i will defer to those with (recent, post-6186200) experience with both. (Re 6186200, see also http://bugs.sun.com/view_bug.do?bug_id=6539517 whose fix has not yet been committed, and which closes a hole in 6186200.) over and out. -- ramki From Y.S.Ramakrishna at Sun.COM Tue Jan 1 12:51:34 2008 From: Y.S.Ramakrishna at Sun.COM (Y Srinivas Ramakrishna) Date: Tue, 01 Jan 2008 12:51:34 -0800 Subject: How "evil" is GetPrimitiveArrayCritical? In-Reply-To: References: <194f62550712301231n4570872fj6b11680dffd39095@mail.gmail.com> <87sl1jdfrh.fsf@mid.deneb.enyo.de> <194f62550712310258p3cff8c17i98fb7561bec9d02b@mail.gmail.com> Message-ID: Good point! Yes, that appears a very reasonable change (when it's the case that the target is in the old generation collected by the CMS collector). thanks! -- ramki ----- Original Message ----- From: Ram?n Garc?a Date: Monday, December 31, 2007 3:16 pm Subject: Re: How "evil" is GetPrimitiveArrayCritical? To: hotspot-runtime-dev at openjdk.java.net > Y Srinivas Ramakrishna, you said that the the CMS is not a moving GC, > so GetPrimitiveArrayCritical does not block garbage collection. If I > understand well, the functions GetXXXArrayElements > (XXX=Boolean,Short,Chart,...) should not perform a copy of the > elements when a non moving garbage collector is used. But, from the > implementation (hotspot/src/share/vm/prims/jni.cpp, macro > jni_Get##Result##ArrayElements) this is not the case: a copy is always > made. Is this the logical behaviour? If not, would a contribution > changing this behaviour be accepted (not making a copy if the > collector is not a moving collector)? > > Ramon > > > On Dec 31, 2007 11:58 AM, Clemens Eisserer wrote: > > Hello, > > > > Thanks a lot for the detailed in-depth explanation :) > > > > > Yes, it may potentially stall GC by as long as it might take the threads > > > to release the critical section (although in all honesty, this > could be > > > dynamically improved). Basically, the way things work now is that > once > > > Eden fills up, and a JNI critical section is in use, further > entries into > > > the critical section are delayed -- thus there is not a danger of > starvation > > > or lock out. Thus, for example, in your case, GC > > > may be delayed by as much as 2 ms. During this time, allocation requests > > > may either go slow path (allocating out of the old generation), or > in > > > extreme cases, get stalled until the JNI CS releases. > > > > So, to make it short - which implementation should I prefer? > > The one which copies the data into a DirectByteBuffer or the > > implementation which uses Get/ReleaseCritical and holds the lock until > > it processes a small piece of data? (for max 2ms, but more likely > > arround 500us). > > > > Copying means two additional JNI call per stride, two > > Get/ReleaseCritical pair which are held only short, and also managing > > the native memory. > > I benchmarked a bit and it seems both implementations perform almost > > equally, however the code for the ByteBuffer-copying approach is more > > complex and means a lot more offset/lenght management. The > > OpenJDK-Corelibs guys seem a bit affraid of the Get/ReleaseCritical > > approach, thats why I am nerving that much here ;) > > > > Thanks a lot, lg Clemens > > From Stephen.Bohne at Sun.COM Wed Jan 2 08:02:21 2008 From: Stephen.Bohne at Sun.COM (Steve Bohne) Date: Wed, 02 Jan 2008 11:02:21 -0500 Subject: Advice of how to implement ahead of time compilation. In-Reply-To: References: Message-ID: <477BB58D.9080404@sun.com> Hi Ram?n, Ram?n Garc?a wrote: > Hello, > > I am interested in implementing ahead of time compilation. Testing > with Mono (a free .NET implementation) I see that ahead of time > compilation greatly improves startup time, turning about 0.7 s of a > compilation (equivalent to javac) into 0.08 s. Therefore, I think that > it would be useful to have this option for interactive applications, > that could be compiled to native during installation, and then > executed directly. This would not harm the cross platform nature of > Java, because applications would be still distributed in bytecode > format. > > I need advice about what path should I follow. > > - One way would be to extend the current support of class data sharing > to user supplied code, so that the virtual machine could load several > data sharing archives, one for the system classes, and another for the > current application. First you'd have to extend class data sharing to cache compiled system code, which it doesn't do. (Maybe that was already implied in what you wrote.) FWIW, a while back we did some internal experiments to determine the benefit of caching compiled code for client/desktop type applications. At the time there wasn't much benefit. The consensus explanation was that the HotSpot client compiler is actually very fast to compile code, and the initial interpreter + compilation overhead doesn't cost all that much for real applications. That is not to say that a well optimized AOT scheme couldn't have benefit - just some anecdotal evidence. > > - An alternative method, the virtual machine would load a shared > library containing the compiled application code. Tools must be > provided to compile a collection of classes (and JARS) into a shared > library, where all internal references are resolved. It seems like the first option would be easier to port and maintain. The class data sharing archive file format is the same across all platforms we support, while shared libraries aren't. Steve > > I see that unit of compilation must be the classloader, that is, > classes intended to be loaded in the same classloader should be > compiled together. Otherwise, it would be incorrect to compile a call > to a class into a concrete call, because one cannot know if the class > referenced will be overridden by some other class earlier in the > classpath; and it seems that linking is an important part of compiling > time. In fact, the JSR 294 of Java Superpackages subtitle says > "Language extensions in support of information hiding and separate > compilation". I understand that the mechanism of superpackages, by > modifying the access rules, allow one class to be compiled knowing > which concrete classes is referenced, and thus make it possible ahead > of time compilation of the superpackage into a shared library. > > Ramon From ramon.garcia.f+java at gmail.com Thu Jan 3 12:17:04 2008 From: ramon.garcia.f+java at gmail.com (=?UTF-8?Q?Ram=C3=B3n_Garc=C3=ADa?=) Date: Thu, 3 Jan 2008 21:17:04 +0100 Subject: Advice of how to implement ahead of time compilation. In-Reply-To: <477BB58D.9080404@sun.com> References: <477BB58D.9080404@sun.com> Message-ID: Thanks for the tips. I will explore this method. From Y.S.Ramakrishna at Sun.COM Thu Jan 3 15:56:46 2008 From: Y.S.Ramakrishna at Sun.COM (Y.S.Ramakrishna at Sun.COM) Date: Thu, 03 Jan 2008 15:56:46 -0800 Subject: Negative durations? In-Reply-To: <304E9E55F6A4BE4B910C2437D4D1B49608FC799162@MERCMBX14.na.sas.com> References: <9BADD5B8-F9DA-4656-843B-7D44FF36963A@mugfu.com> <477C09ED.9020400@Sun.COM> <22133221-306B-41D3-AE57-155876104354@mugfu.com> <477C1718.70004@Sun.COM> <304E9E55F6A4BE4B910C2437D4D1B49608FC799162@MERCMBX14.na.sas.com> Message-ID: <477D763E.9000607@Sun.COM> If on windows, you could try disabling NTP sync to see if the "problem" goes away... Cross-posting to the runtime list for possible further comment. You clearly see time-stamps also stepping back (for example between line n-1 and n-2 below). -- ramki Keith Holdaway wrote: > I have Google'd, but found no explanation for negative GC pauses: > > 8829.092: [GC 426933K->375154K(517760K), 0.0117606 secs] > 8830.373: [GC 427634K->376403K(517760K), 0.0833584 secs] > 8830.569: [GC 428883K->377941K(517760K), -0.6576383 secs] > 8831.110: [GC 430421K->379175K(517760K), 0.0161026 secs] > 8831.548: [GC 431628K->379968K(517760K), 0.0134666 secs] > 8831.942: [GC 432448K->379701K(517760K), -0.1804611 secs] > 8832.718: [GC 432180K->382348K(517760K), 0.0836352 secs] > 8832.303: [GC 434828K->382927K(517760K), 0.6898266 secs] > 8833.482: [GC 435407K->384775K(517760K), -0.1111267 secs] > > > Keith R Holdaway > Java Development Technologies > > SAS... The Power to Know > > Carpe Diem ... > > _______________________________________________ > hotspot-gc-use mailing list > hotspot-gc-use at openjdk.java.net > http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use From David.Holmes at Sun.COM Thu Jan 3 16:17:28 2008 From: David.Holmes at Sun.COM (David Holmes - Sun Microsystems) Date: Fri, 04 Jan 2008 10:17:28 +1000 Subject: Negative durations? In-Reply-To: <477D763E.9000607@Sun.COM> References: <9BADD5B8-F9DA-4656-843B-7D44FF36963A@mugfu.com> <477C09ED.9020400@Sun.COM> <22133221-306B-41D3-AE57-155876104354@mugfu.com> <477C1718.70004@Sun.COM> <304E9E55F6A4BE4B910C2437D4D1B49608FC799162@MERCMBX14.na.sas.com> <477D763E.9000607@Sun.COM> Message-ID: <477D7B18.4010304@sun.com> Possible problems depend on the OS, as the elapsed_counter used underneath these "timings" is totally different on each platform: Solaris: uses gethrtime() via getTimeNanos() which has code to ensure the returned value is monotonic even if gethrtime() is not (it should be but it has had bugs due to TSC usage on x86 MP systems). So negative values should not been seen on Solaris, but you might see zero values. Linux: uses gettimeofday() - so if you have a bad clock and are running (x)ntpd then you might see lots of adjustments that cause apparent negative intervals. I don't recall if gettimeofday can be subject to TSC problems, but you might try booting with the notsc boot option. (It all depends on which Linux kernel version you have.) Windows: uses QueryPerformanceCounter if available, else GetSystemTimeAsFileTime. If QPC is used then you need to ensure Windows is using a stable time source for your platform eg. not using the TSC on MP systems; and not using the TSC if you have "speed-step" or equivalent dynamic CPU frequency adjusting mechanisms. Add /usepmtimer to the boot options in boot.ini to avoid TSC use. I don't know what problems GetSystemTimeAsFileTime might encounter - I suspect it would be susceptible to NTP adjustments as well. (For gory details of clocks and timers on Windows see: http://blogs.sun.com/dholmes/entry/inside_the_hotspot_vm_clocks) Cheers, David Holmes Y.S.Ramakrishna at Sun.COM said the following on 4/01/08 09:56 AM: > If on windows, you could try disabling NTP sync to see if > the "problem" goes away... Cross-posting to the runtime > list for possible further comment. > > You clearly see time-stamps also stepping back (for example > between line n-1 and n-2 below). > > -- ramki > > Keith Holdaway wrote: >> I have Google'd, but found no explanation for negative GC pauses: >> >> 8829.092: [GC 426933K->375154K(517760K), 0.0117606 secs] >> 8830.373: [GC 427634K->376403K(517760K), 0.0833584 secs] >> 8830.569: [GC 428883K->377941K(517760K), -0.6576383 secs] >> 8831.110: [GC 430421K->379175K(517760K), 0.0161026 secs] >> 8831.548: [GC 431628K->379968K(517760K), 0.0134666 secs] >> 8831.942: [GC 432448K->379701K(517760K), -0.1804611 secs] >> 8832.718: [GC 432180K->382348K(517760K), 0.0836352 secs] >> 8832.303: [GC 434828K->382927K(517760K), 0.6898266 secs] >> 8833.482: [GC 435407K->384775K(517760K), -0.1111267 secs] >> >> >> Keith R Holdaway >> Java Development Technologies >> >> SAS... The Power to Know >> >> Carpe Diem ... >> >> _______________________________________________ >> hotspot-gc-use mailing list >> hotspot-gc-use at openjdk.java.net >> http://mail.openjdk.java.net/mailman/listinfo/hotspot-gc-use > From peter.helfer.java at gmail.com Tue Jan 8 14:43:57 2008 From: peter.helfer.java at gmail.com (Peter Helfer) Date: Tue, 8 Jan 2008 23:43:57 +0100 Subject: Resolving a methodOop from a constantPool-index Message-ID: Quick question: how to resolve a methodOop, given a constantPool-Index, and the constantPoolHandle.. ? Thanks, Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20080108/82f8970f/attachment.html From Keith.McGuigan at Sun.COM Tue Jan 8 15:00:30 2008 From: Keith.McGuigan at Sun.COM (Keith McGuigan) Date: Tue, 08 Jan 2008 18:00:30 -0500 Subject: Resolving a methodOop from a constantPool-index In-Reply-To: References: Message-ID: <4784008E.9060108@sun.com> Something like this: (?) Use constantPoolOop::field_or_method_at() to get the the class/name_and_type index pair. Use constantPoolOop::klass_at() to get the instanceKlass. Use constantPoolOop::name_and_type_at() to get the indices for the method name and the method signature, then get the corresponding symbolOops for the name and signature (with symbol_at()). You can then call instanceKlass::find_method() on the retrieved class with the method name and signature to get your methodOop. Does that help? This kind-of assumes that initialization is done, the class is loaded, etc., etc. It can be trickier if that isn't the case. -- - Keith Peter Helfer wrote: > Quick question: how to resolve a methodOop, given a constantPool-Index, > and the constantPoolHandle.. ? > > Thanks, Peter From peter.helfer.java at gmail.com Wed Jan 9 10:11:55 2008 From: peter.helfer.java at gmail.com (Peter Helfer) Date: Wed, 9 Jan 2008 19:11:55 +0100 Subject: Tool to track Oops ? Message-ID: Is there a tool to track the allocation of the oops ? I'm seeing some SIGSEVs in oopDesc::verify_on, as part of GC/VMThread... I do know there is HAT/JHAT & friends, but those are tracking the objects that are seen from the outside(userspace)... but as I am throwing dirt around within the JVM, it would be really helpful to see where that oop was allocated ... Regards, Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20080109/7adcc783/attachment.html From peter.helfer.java at gmail.com Fri Jan 11 08:55:49 2008 From: peter.helfer.java at gmail.com (Peter Helfer) Date: Fri, 11 Jan 2008 17:55:49 +0100 Subject: x86 address generation / jump-targets Message-ID: Hi all I need to generate some return address, which is within a RuntimeStub I'm creating. But the return address created as InternalAddress does show up as 0x00009BE9 - this seems not to be ok.. Is this the right approach ? Regards, Peter [do some stuff] __ jmp(done) // jumps over next segment //============================================== Label exitadapter; address returnpc_for_interpreter = __ pc(); __ jmp(exitadapter); // jump over next segment //============================================== __ bind(done); __ mov32(rcx, InternalAddress(returnpc_for_interpreter)); // lets hope we've got the right one __ xchg(rcx, Address(rbp, frame::return_addr_offset * wordSize) // replace current returnpc with new one ... stackdump done here __ jmp(rcx); // but still go once into the old entry.. //============================================== __ bind(exitadapter); // stack dump [AD759F30] = 0817BC00.%edi [AD759F34] = 00000000.%esi [AD759F38] = AD759F74.%ebp [AD759F3C] = AD759F50.%esp before pushad [AD759F40] = B7F7AFF4.%ebx [AD759F44] = 0817BC00.%edx [AD759F48] = B5C002B0.%ecx [AD759F4C] = D00DFACE.%eax [AD759F50] = ADBB36E8. [AD759F54] = AD759F54. [AD759F58] = B1D5506F. [AD759F5C] = AD759FA0. [AD759F60] = B1D55380. [AD759F64] = 00000000. [AD759F68] = B1D55188. [AD759F6C] = 00000000. [AD759F70] = B5C06752. [AD759F74] = AD759FA8 <-- rbp [AD759F78] = 00009BE9 <-- returnpc .. and zeros after that.. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20080111/b2f3840f/attachment.html From peter.helfer.java at gmail.com Sat Jan 12 01:43:04 2008 From: peter.helfer.java at gmail.com (Peter Helfer) Date: Sat, 12 Jan 2008 10:43:04 +0100 Subject: the connection between JVM buffer and java heap? In-Reply-To: <13432ab00801111816g23f3c3f5x16bdd5bc4b06627f@mail.gmail.com> References: <13432ab00801111816g23f3c3f5x16bdd5bc4b06627f@mail.gmail.com> Message-ID: Hi Jackie I guess you need to clarify which buffer you mean - from your words, it looks like you mean the java.io.* / java.nio.* subsystem. I assume you have access to the source code, check openjdk/j2se/src/share/classes/java/(n)io/*.java and their respective native methods in openjdk/j2se/src/share/native/java/(n)io/*.c. Now afai can tell, every call from Java into C (~= native function), even from the Hotspot itself has to be handled, and the objects need to be unmarshalled, so the native code can work on them safely. The generator for a native method is at openjdk/hotspot/src/share/vm/runtime/sharedRuntime.cpp in function AdapterHandlerLibrary::create_native_wrapper(...) and in your architecture-dependent file openjdk/hotspot/src/cpu//vm/sharedRuntime_.cpp in function SharedRuntime::generate_native_wrapper(...) Now for type T_OBJECT and T_ARRAY, a function is called object_move(....), now on inspection, I don't see any copying of a buffer, it is only a reworking of the reference to the data. So my conclusion would be that the buffer remains the same. However, I'm not a JVM professional, so you might have to wait till Monday 9am PST, before anyone is having a look into your question. hth, Peter 2008/1/12, zhang Jackie : > > hi > are these two the same,or different? When sending data from Java , the > data is copied from java heap to JVM buffer and then give it to OS . is this > right? > > Thanks > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20080112/115ed719/attachment.html From Keith.McGuigan at Sun.COM Tue Jan 22 10:01:05 2008 From: Keith.McGuigan at Sun.COM (Keith McGuigan) Date: Tue, 22 Jan 2008 13:01:05 -0500 Subject: request for review (XS) Message-ID: <47962F61.3080409@sun.com> Fixed 6631248: Memory problem when doing invalid type cast The message created for ClassCastException was incorrectly allocated out of C-heap, instead of as resource memory. Webrev: http://web-east.east/~km88527/webrev/2008/6631248 Testing: + JPRT + NSK runtime + JCK vm tests + runThese full Reviewed by: Fix verified (y/n)? y Verified by: + CR-attached test case -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: patch.cdiff Url: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20080122/e904cb68/attachment.ksh From Keith.McGuigan at Sun.COM Fri Jan 25 11:21:32 2008 From: Keith.McGuigan at Sun.COM (Keith McGuigan) Date: Fri, 25 Jan 2008 14:21:32 -0500 Subject: request for code review (S) Message-ID: <479A36BC.8010807@sun.com> Would appreciate any review for this little fix: Fixed 6615981: JVM class file parser incorrectly rejects class files with version < 45.2 A check on the Code attribute length in the class file parser did not take into account the old sizes of the max_stack, max_locals, and code_length. Webrev: http://web-east.east/~km88527/webrev/2008/6615981 Testing: + JPRT + NSK runtime + JCK vm tests + runThese full Reviewed by: Fix verified (y/n)? y Verified by: + Class file with version 45.0 -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 6615981.cdiff Url: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20080125/0fd90218/attachment.ksh From Keith.McGuigan at Sun.COM Fri Jan 25 11:24:38 2008 From: Keith.McGuigan at Sun.COM (Keith McGuigan) Date: Fri, 25 Jan 2008 14:24:38 -0500 Subject: another request for review (S) Message-ID: <479A3776.6060404@sun.com> Another small fix, but this one may have larger ramifications, so any export on access controls please take a look: Fixed 6622385: Accessing protected static methods A protected member accessible from the class is accessible even if it is referred to via a sibling class -- if that member is static. Webrev: http://web-east.east/~km88527/webrev/2008/6622385 Testing: + JPRT + NSK runtime + JCK vm tests + runThese full Reviewed by: Fix verified (y/n)? y Verified by: + CR-attached test case From Keith.McGuigan at Sun.COM Fri Jan 25 11:26:28 2008 From: Keith.McGuigan at Sun.COM (Keith McGuigan) Date: Fri, 25 Jan 2008 14:26:28 -0500 Subject: another request for review (S) In-Reply-To: <479A3776.6060404@sun.com> References: <479A3776.6060404@sun.com> Message-ID: <479A37E4.4060107@sun.com> Apologies - forgot to attach the diff. Keith McGuigan wrote: > Another small fix, but this one may have larger ramifications, so any > export on access controls please take a look: > > Fixed 6622385: Accessing protected static methods > > A protected member accessible from the class is accessible > even if it is referred to via a sibling class -- if that member is static. > > Webrev: http://web-east.east/~km88527/webrev/2008/6622385 > > Testing: > + JPRT > + NSK runtime > + JCK vm tests > + runThese full > > Reviewed by: > > Fix verified (y/n)? y > Verified by: > + CR-attached test case > > -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 6622385.cdiff Url: http://mail.openjdk.java.net/pipermail/hotspot-runtime-dev/attachments/20080125/a9e816ba/attachment.ksh