RFR(L): 8031581: PPC64: Addons and fixes for AIX to pass the jdk regression tests

Wed Jan 15 19:03:01 UTC 2014

On 15 jan 2014, at 18:27, Volker Simonis <volker.simonis at gmail.com> wrote:

> On Wed, Jan 15, 2014 at 5:34 PM, Volker Simonis
> <volker.simonis at gmail.com> wrote:
>> Hi Staffan,
>> 
>> thanks for the review. Please find my comments inline:
>> 
>> On Wed, Jan 15, 2014 at 9:57 AM, Staffan Larsen <staffan.larsen at oracle.com>
>> wrote:
>>> 
>>> Volker,
>>> 
>>> I’ve look at the following files:
>>> 
>>> src/share/native/sun/management/DiagnosticCommandImpl.c:
>>> nit: “legel” -> “legal” (two times)
>>> In Java_sun_management_DiagnosticCommandImpl_getDiagnosticCommandInfo() if
>>> you allow dcmd_info_array to become NULL, then
>>> jmm_interface->GetDiagnosticCommandInfo() will throw an NPE and you need to
>>> check that.
>> 
>> 
>> Good catch. I actually had problems with malloc returning NULL in
>> 'getDiagnosticCommandArgumentInfoArray()' and then changed all other
>> potentially dangerous locations which used the same pattern.
>> 
>> However I think if the 'dcmd_info_array' has zero length it would be
>> perfectly fine to return a zero length array. So what about the following
>> solution:
>> 
>>  dcmdInfoCls = (*env)->FindClass(env,
>>                                  "sun/management/DiagnosticCommandInfo");
>>  num_commands = (*env)->GetArrayLength(env, commands);
> 
> Sorry, of course I wanted to say "if (num_commands == 0)" here!

I understood as much :-)

> 
>>  if (num_commands = 0) {
>>      result = (*env)->NewObjectArray(env, 0, dcmdInfoCls, NULL);
>>      if (result == NULL) {
>>          JNU_ThrowOutOfMemoryError(env, 0);
>>      }
>>      else {
>>          return result;
>>      }
>>  }
>>  dcmd_info_array = (dcmdInfo*) malloc(num_commands * sizeof(dcmdInfo));
>>  if (dcmd_info_array == NULL) {
>>      JNU_ThrowOutOfMemoryError(env, NULL);
>>  }
>>  jmm_interface->GetDiagnosticCommandInfo(env, commands, dcmd_info_array);
>>  result = (*env)->NewObjectArray(env, num_commands, dcmdInfoCls, NULL);
>> 
>> That seems easier and saves me from handling the exception.
>> 
>> What do you think?
>> 
>>> src/solaris/native/sun/management/OperatingSystemImpl.c
>>> No comments.
>>> 
>>> src/share/transport/socket/socketTransport.c
>>> No comments.
>>> 
>>> 
>>> src/share/classes/sun/tools/attach/META-INF/services/com.sun.tools.attach.spi.AttachProvider
>>> No comments.
>>> 
>>> 
>>> Thanks,
>>> /Staffan
>>> 
>>> 
>>> 
>>> On 14 jan 2014, at 09:40, Volker Simonis <volker.simonis at gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> could you please review the following changes for the ppc-aix-port
>>> stage/stage-9 repositories (the changes are planned for integration into
>>> ppc-aix-port/stage-9 and subsequent backporting to ppc-aix-port/stage):
>>> 
>>> http://cr.openjdk.java.net/~simonis/webrevs/8031581/
>>> 
>>> I've build and smoke tested without any problems on Linux/x86_64 and
>>> PPC64, Windows/x86_64, MacOSX, Solaris/SPARC64 and AIX7PPC64.
>>> 
>>> With these changes (and together with the changes from "8028537: PPC64:
>>> Updated jdk/test scripts to understand the AIX os and environment" and
>>> "8031134 : PPC64: implement printing on AIX") our port passes all but the
>>> following 7 jtreg regression tests on AIX (compared to the Linux/x86_64
>>> baseline from www.java.net/download/jdk8/testresults/testresults.html‎):
>>> 
>>> java/net/Inet6Address/B6558853.java
>>> java/nio/channels/AsynchronousChannelGroup/Basic.java (sporadically)
>>> java/nio/channels/AsynchronousChannelGroup/GroupOfOne.java
>>> java/nio/channels/AsynchronousChannelGroup/Unbounded.java (sporadically)
>>> java/nio/channels/Selector/RacyDeregister.java
>>> sun/security/krb5/auto/Unreachable.java (only on IPv6)
>>> 
>>> Thank you and best regards,
>>> Volker
>>> 
>>> 
>>> Following a detailed description of the various changes:
>>> 
>>> src/share/native/java/util/zip/zip_util.c
>>> src/share/native/sun/management/DiagnosticCommandImpl.c
>>> 
>>> According to ISO C it is perfectly legal for malloc to return zero if
>>> called with a zero argument. Fix various places where malloc can potentially
>>> correctly return zero because it was called with a zero argument.
>>> Also fixed DiagnosticCommandImpl.c to include stdlib.h. This only fixes a
>>> compiler warning on Linux, but on AIX it prevents a VM crash later on
>>> because the return value of malloc() will be casted to int which is
>>> especially bad if that pointer was bigger than 32-bit.
>>> 
>>> make/CompileJavaClasses.gmk
>>> 
>>> Also use PollingWatchService on AIX.
>>> 
>>> make/lib/NioLibraries.gmk
>>> src/aix/native/sun/nio/ch/AixNativeThread.c
>>> 
>>> Put the implementation for the native methods of NativeThread into
>>> AixNativeThread.c on AIX.
>>> 
>>> src/solaris/native/sun/nio/ch/PollArrayWrapper.c
>>> src/solaris/native/sun/nio/ch/Net.c
>>> src/aix/classes/sun/nio/ch/AixPollPort.java
>>> src/aix/native/sun/nio/ch/AixPollPort.c
>>> src/aix/native/java/net/aix_close.c
>>> 
>>> On AIX, the constants used for the polling events (i.e. POLLIN, POLLOUT,
>>> ...) are defined to different values than on other operating systems. The
>>> problem is however, that these constants are hardcoded as public final
>>> static members of various, shared Java classes. We therefore have to map
>>> them from Java to native every time before calling one of the native poll
>>> functions and back to Java after the call on AIX in order to get the right
>>> semantics.
>>> 
>>> src/share/classes/java/nio/file/CopyMoveHelper.java
>>> 
>>> As discussed on the core-libs mailing list (see
>>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-December/024119.html)
>>> it is not necessary to call Files.getFileAttributeView() with any
>>> linkOptions because at that place we've already checked that the target file
>>> can not be a symbolic link. This change makes the implementation more robust
>>> on platforms which support symbolic links but do not support the O_NOFOLLOW
>>> flag to the open system call. It also makes the JDK pass the
>>> demo/zipfs/basic.sh test on AIX.
>>> 
>>> src/share/classes/sun/nio/cs/ext/ExtendedCharsets.java
>>> 
>>> Support "compound text" on AIX in the same way like on other Unix
>>> platforms.
>>> 
>>> 
>>> src/share/classes/sun/tools/attach/META-INF/services/com.sun.tools.attach.spi.AttachProvider
>>> 
>>> Define the correct attach provider for AIX.
>>> 
>>> src/solaris/native/java/net/net_util_md.h
>>> src/solaris/native/sun/nio/ch/FileDispatcherImpl.c
>>> src/solaris/native/sun/nio/ch/ServerSocketChannelImpl.c
>>> 
>>> AIX needs a workaround for I/O cancellation (see:
>>> http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.basetechref/doc/basetrf1/close.htm).
>>> "..The close() subroutine is blocked until all subroutines which use the
>>> file descriptor return to usr space. For example, when a thread is calling
>>> close and another thread is calling select with the same file descriptor,
>>> the close subroutine does not return until the select call returns...". To
>>> fix this problem, we have to use the various NET_ wrappers which are
>>> declared in net_util_md.h and defined in aix_close.c and we also need some
>>> additional wrappers for fcntl(), read() and write() on AIX.
>>> While the current solution isn't really nice because it introduces some
>>> more AIX-specifc sections in shared code, I think it is the best way to go
>>> for JDK 8 because it imposes the smallest possible changes and risks for the
>>> existing platforms. I'm ready to change the code to unconditionally use the
>>> wrappers for all platforms and implement the wrappers empty on platforms
>>> which don't need any wrapping. I think it would also be nice to clean up the
>>> names (e.g. NET_Read() is currently a wrapper for recv() and the NET_ prefix
>>> is probably not appropriate any more so maybe change it to something like
>>> IO_). But again, I'll prefer to keep that as a follow up change for JDK9.
>>> Calling fsync() on a "read-only" file descriptor on AIX will result in an
>>> error (i.e. "EBADF: The FileDescriptor parameter is not a valid file
>>> descriptor open for writing."). To prevent this error we have to query if
>>> the corresponding file descriptor is writeable. Notice that at that point we
>>> can not access the writable attribute of the corresponding file channel so
>>> we have to use fcntl().
>>> 
>>> src/solaris/classes/java/lang/UNIXProcess.java.aix
>>> 
>>> On AIX the implementation is especially tricky, because the close() system
>>> call will block if another thread is at the same time blocked in a file
>>> operation (e.g. 'read()') on the same file descriptor. We therefore combine
>>> the AIX ProcessPipeInputStream implemenatation with the
>>> DeferredCloseInputStream approach used on Solaris (see
>>> UNIXProcess.java.solaris). This means that every potentially blocking
>>> operation on the file descriptor increments a counter before it is executed
>>> and decrements it once it finishes. The 'close()' operation will only be
>>> executed if there are no pending operations. Otherwise it is deferred after
>>> the last pending operation has finished.
>>> 
>>> src/share/transport/socket/socketTransport.c
>>> 
>>> On AIX we have to call shutdown() on a socket descriptor before closing
>>> it, otherwise the close() call may be blocked. This is the same problem as
>>> described before. Unfortunately the JDI framework doesn't use the same IO
>>> wrappers like other class library components so we can not easily use the
>>> NET_ abstractions from aix_close.c here.
>>> Without this small change all JDI regression tests will fail on AIX
>>> because of the way how the tests act as a "debugger" which launches another
>>> VM (the "debugge") which connects itself back to the debugger. In this
>>> scenario the "debugge" can not shut down itself because one thread will
>>> always be blocked in the close() call on one of the communication sockets.
>>> 
>>> src/solaris/native/java/net/NetworkInterface.c
>>> 
>>> Set the scope identifier for IPv6 addresses on AIX.
>>> 
>>> src/solaris/native/java/net/net_util_md.c
>>> 
>>> It turns out that we do not always have to replace SO_REUSEADDR on AIX by
>>> SO_REUSEPORT. Instead we can simply use the same approach like BSD and only
>>> use SO_REUSEPORT additionally, if several datagram sockets try to bind to
>>> the same port.
>>> Also fixed a comment and removed unused local variables.
>>> Fixed the obviously inverted assignment newTime = prevTime; which should
>>> read prevTime = newTime;. Otherwise prevTime will never change and the
>>> timeout will be potential reached too fast.
>>> 
>>> src/solaris/native/sun/management/OperatingSystemImpl.c
>>> 
>>> AIX does not understand /proc/self so we have to query the real process ID
>>> to access the proc file system.
>>> 
>>> src/solaris/native/sun/nio/ch/DatagramChannelImpl.c
>>> 
>>> On AIX, connect() may legally return EAFNOSUPPORT if called on a socket
>>> with the address family set to AF_UNSPEC.