RFR: JDK-8200178 Remove mapfiles for JDK native libraries

[Phil Race]

http://cr.openjdk.java.net/~ihse/JDK-8200178-remove-mapfiles/webrev.01/src/java.desktop/share/native/libmlib_image/mlib_image_proto.h.udiff.html

The variable definitions here are now misaligned.

..and added 2d-dev since many of these native changes are in 2d.

-phil.

On 03/23/2018 10:33 AM, Phil Race wrote:
> There are a lot of changes in the desktop libraries.
> Doing mach5 tier1/2/3 testing is not nearly sufficient to cover those 
> since
> only tier3 has any UI tests and it barely uses anything that's touched 
> here.
> So since testing seems to be wise, then I think you should do a
> jtreg desktop group run on Linux & Windows.
> You can probably skip Mac since it is unaffected and I think Linux 
> will cover Solaris here.
> You should also do some headless testing.
>
> It could take some time to review this properly and decide what 
> changes are OK,
> what changes are something we should clean up later, and what changes 
> are something
> that ought to be addressed now ..
>
> I think I'd be mainly concerned that something fails due to a missing 
> symbol, or
> that for newly exported symbols if we ended up with duplicate symbols 
> as a result.
>
> The results of a test run will add confidence here.
>
> BTW I don't think you are right that
> java.desktop:/libawt_headless: The following symbols are now also 
> exported due to JNIEXPORT (they were previously
> ..
>  X11SurfaceData_GetOps
>
> It looks to me like it was previously exported.
>
>
> -phil.
>
>
>
> On 03/23/2018 06:56 AM, Magnus Ihse Bursie wrote:
>> With modern compilers, we can use compiler directives (such as 
>> _attribute__((visibility("default"))), or __declspec(dllexport)) to 
>> control symbol visibility, directly in the source code. This has 
>> historically not been present on all compilers, so we had to resort 
>> to using mapfiles (also known as linker scripts).
>>
>> This is no longer the case. Now all compilers we use support symbol 
>> visibility directives, in one form or another. We should start using 
>> this. Since this has been the only way to control symbol visibility 
>> on Windows, for most of the shared code, we already have proper 
>> JNIEXPORT decorations in place.
>>
>> If we fix the remaining platform-specific files to have proper 
>> JNIEXPORT tagging, then we can finally get rid of mapfiles.
>>
>> This fix removed mapfiles for all JDK libraries. It does not touch 
>> hotspot libraries nor JDK executables; they will have to wait for a 
>> future fix -- this was complex enough. This change will not have any 
>> impact on macosx, since we do not use mapfiles there, but instead 
>> export all symbols. (This is not a good idea, but I'll address that 
>> separately.) This change will also have a minimal impact on Windows. 
>> The only reason Windows is impacted at all, is that some changes 
>> needed by Solaris and Linux were simpler to fix for all platforms.
>>
>> I have strived for this change to have no impact on the actual 
>> generated code. Unfortunately, this was not possible to fully 
>> achieve. I do not believe that these changes will have any actual 
>> impact on the product, though. I will present the differences more in 
>> detail further down. Those who are not interested can probably skip 
>> that.
>>
>> The patch has passed tier1 testing and is currently running tier2 and 
>> tier3. Since the running code is more or less (see caveat below) 
>> unmodified, I don't expect any testing issues.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8200178
>> WebRev: 
>> http://cr.openjdk.java.net/~ihse/JDK-8200178-remove-mapfiles/webrev.01
>>
>> Details on changes:
>> Most of the source code changes are (unsurprisingly) in java.base and 
>> java.desktop. Remaining changes are in jdk.crypto.ucrypto, 
>> jdk.hotspot.agent, jdk.jdi and jdk.jdwp.agent.
>>
>> Source code changes does almost to 100% consists in decorating an 
>> exported function with JNIEXPORT. I have also followed the 
>> long-standing convention of adding JNICALL. This is a no-op on 
>> non-Windows platforms, so for most of the changes this is purely 
>> cosmetic (and possibly adding in robustness, should the function ever 
>> be used on Windows in the future). I have also followed the stylistic 
>> convention of putting "JNIEXPORT <return type> JNICALL" on a separate 
>> line. For some functions, however, this might cause a change in 
>> calling convention on Windows. Since this can not apply to exported 
>> functions on Windows (otherwise they would already have had 
>> JNIEXPORT), I do not think this matters anything.
>>
>> A few libraries did not have a mapfile, on Linux and/or Solaris. This 
>> actually meant that all symbols were exported. It is highly unclear 
>> if this was known and intended by the original make rule writer. I 
>> have emulated this by adding the flag $(EXPORT_ALL_SYMBOLS) to these 
>> libraries. Hopefully, we can remove this flag and fix proper exported 
>> symbols in the future.
>>
>> I have run the complete build using COMPARE_BUILD, and made a 
>> thourough analysis of the differences for Linux and Solaris. All 
>> native libraries have symbol differences, but most of them are 
>> trivial and/or harmless. As a result, most libraries have disasm 
>> differences as well, but these too seem trivial and harmless. The 
>> differences in symbols that are common to all libraries include:
>>  * Internal symbols such as __bss_start, _edata, _end and _fini are 
>> now global. (They are imported as such from the compiler 
>> libraries/archives, and we have no linker script to override this 
>> behavior).
>>  * The versioning tag SUNWprivate_1.1 is not included, and thus 
>> neither the .gnu.version_d symbol.
>>  * There are a few differences in the symbol and/or mangling of some 
>> local functions. I'm not sure what's causing this,
>> but it's unlikely to have any effect on the product.
>>
>> Another common source for change in symbols is due to previous 
>> platform differences. For instance, if we had "JNIEXPORT int JNICALL 
>> do_foo() { ... }", but do_foo was not in the mapfile, the symbol was 
>> exported on Windows but not on Linux and Solaris. (Presumable since 
>> it was not needed there, even though it was compiled for those 
>> platforms as well.) Now, with the mapfiles gone, do_foo() will be 
>> exported on all platforms. And contrary, functions that are compiled 
>> on all platforms, and were exported in mapfiles, but now have gotten 
>> an JNIEXPORT decoration, will now be visible even on Windows. (This 
>> accounts for half of the noticed symbol differences on Windows.) I 
>> could have made the JNIEXPORT conditional on OS, but I didn't think 
>> the mess in source code were worth the keeping of binary confidence 
>> with the old build.
>>
>> A third common source for change in symbols is due to exported 
>> functions "leaking" across library borders. For instance, some 
>> functions in java.desktop is compiled in both libawt_xawt and 
>> libawt_headless, but they were previously only included in the 
>> mapfile for one of these libraries. Now, since the visibility is 
>> determined by the source code itself, it gets exported in both 
>> libraries. A variant of this is when a library depends on another JDK 
>> library, and includes the header file from that other library, which 
>> in turn declares a function as JNIEXPORT. This will cause the 
>> including library to also export the function. This accounts for the 
>> other half of the changes on Windows. A typical example of this is 
>> that multiple libraries now re-export hotspot symbols from libjvm.so, 
>> like jio_fprintf. (I have not listed the libjvm re-exports below.)
>>
>> Note that  Java_java_io_FileOutputStream_close0 in 
>> java.base/unix/native/libjava/FileOutputStream_md.c is no longer 
>> exported,
>> and can probably be removed.
>>
>> Here is a detailed table showing and accounting for all the remaining 
>> differences found on Linux and Solaris:
>> java.base/unix/native/libjava: Java_java_io_FileOutputStream_close0 
>> is now also exported on unix platforms due to JNIEXPORT.
>>
>> java.base/jspawnlauncher: On solaris, we also include 
>> libjava/childproc.o, which
>> now exports less functions than it used to (it used to export all 
>> functions, now it is compiled with visibility=hidden).
>>
>> java.base/java(w).exe: Is now also exporting the following symbols 
>> due to added JNIEXPORT in libjli on Windows:
>> (Yes, executables can export symbols on Windows. Confusing, I know.)
>>  JLI_AddArgsFromEnvVar
>>  JLI_CmdToArgs
>>  JLI_GetAppArgIndex
>>  JLI_GetStdArgc
>>  JLI_GetStdArgs
>>  JLI_InitArgProcessing
>>  JLI_Launch
>>  JLI_List_add
>>  JLI_List_new
>>  JLI_ManifestIterate
>>  JLI_MemAlloc
>>  JLI_MemFree
>>  JLI_PreprocessArg
>>  JLI_ReportErrorMessage
>>  JLI_ReportErrorMessageSys
>>  JLI_ReportExceptionDescription
>>  JLI_ReportMessage
>>  JLI_SetTraceLauncher
>>  JLI_StringDup
>>
>> java.desktop:/libawt_xawt: The following symbols are now also 
>> exported on linux and solaris due to JNIEXPORT:
>>  awt_DrawingSurface_FreeDrawingSurfaceInfo
>>  awt_DrawingSurface_GetDrawingSurfaceInfo
>>  awt_DrawingSurface_Lock
>>  awt_DrawingSurface_Unlock
>>  awt_GetColor
>>
>> The following symbols are now also exported on linux and solaris due 
>> to JNIEXPORT (they were previously
>>  exported only in libawt):
>>  Java_sun_awt_DebugSettings_setCTracingOn__Z
>>  Java_sun_awt_DebugSettings_setCTracingOn__ZLjava_lang_String_2
>>  Java_sun_awt_DebugSettings_setCTracingOn__ZLjava_lang_String_2I
>>  Java_sun_awt_X11GraphicsConfig_getNumColors
>>
>> java.desktop:/libawt_headless: The following symbols are now also 
>> exported due to JNIEXPORT (they were previously
>>  exported only in libawt_xawt and/or libawt):
>>  Java_sun_java2d_opengl_GLXGraphicsConfig_getGLXConfigInfo
>>  Java_sun_java2d_opengl_GLXGraphicsConfig_getOGLCapabilities
>>  Java_sun_java2d_x11_X11PMBlitLoops_updateBitmask
>>  Java_sun_java2d_x11_X11SurfaceData_isShmPMAvailable
>>  X11SurfaceData_GetOps
>>
>> java.desktop/libawt: The following symbols are now also exported on 
>> Windows, due to added
>> JNIEXPORT:
>>  SurfaceData_InitOps
>>  mul8table
>>  div8table
>>  doDrawPath
>>  doFillPath
>>  g_CMpDataID
>>  initInverseGrayLut
>>  make_dither_arrays
>>  make_uns_ordered_dither_array
>>  path2DFloatCoordsID
>>  path2DNumTypesID
>>  path2DTypesID
>>  path2DWindingRuleID
>>  sg2dStrokeHintID
>>  std_img_oda_blue
>>  std_img_oda_green
>>  std_img_oda_red
>>  std_odas_computed
>>  sunHints_INTVAL_STROKE_PURE
>>
>> java.desktop/libawt on solaris:
>> A number of "#pragma weak" directives was previously overridden by 
>> the mapfile.
>> Now these directives are respected, so these symbols are now weak 
>> instead of local:
>>  ByteGrayToIntArgbPreConvert_F
>>  ByteGrayToIntArgbPreScaleConvert_F
>>  IntArgbBmToFourByteAbgrPreScaleXparOver_F
>>  IntArgbToIntRgbXorBlit_F
>>  IntBgrToIntBgrAlphaMaskBlit_F
>>
>> java.desktop/libawt on solaris: These are now also exported due to 
>> JNIEXPORT in libmlib_image.
>>  j2d_mlib_ImageCreate
>>  j2d_mlib_ImageCreateStruct
>>  j2d_mlib_ImageDelete
>>
>> java.desktop/libawt on solaris: This is now also exported due to 
>> JNIEXPORT:
>>  GrPrim_CompGetXorColor
>>  SurfaceData_GetOpsNoSetup
>>  SurfaceData_IntersectBoundsXYWH
>>  SurfaceData_SetOps
>>  Transform_GetInfo
>>  Transform_transform
>>
>> java.desktop/libsplashscreen: JNI_OnLoad is now exported on linux and 
>> solaris due to JNIEXPORT.
>> libspashscreen also had JNIEXPORT (actually a pure 
>> _declspec(dllexport)) but no JNICALL, which I added as
>> a part of converting to JNIEXPORT. The same goes for libmlib_image .
>>
>> jdk.sctp/libsctp: handleSocketError is now exported on linux and 
>> solaris due to JNIEXPORT in libnio.
>>
>> java.instrument:/libinstrument: Agent_OnUnload is now also exported 
>> on linux and solaris platforms due to JNIEXPORT.
>> JLI_ManifestIterate is now also exported on Windows, due to added 
>> JNIEXPORT in libjli.
>>
>> jdk.management/libmanagement_ext: 
>> Java_com_sun_management_internal_Flag_setDoubleValue is now also 
>> exported on linux and solaris platforms due to JNIEXPORT.
>>
>> /Magnus
>>
>>
>