RFR: JDK-8200178 Remove mapfiles for JDK native libraries

[Erik Joelsson]

I have looked at the build changes and they look good.

Will you file followups for each component team to look over their 
exported symbols, at least for the libraries with $(EXPORT_ALL_SYMBOLS)? 
It sure looks like there is some technical debt laying around here.

/Erik


On 2018-03-23 06:56, Magnus Ihse Bursie wrote:
> With modern compilers, we can use compiler directives (such as 
> _attribute__((visibility("default"))), or __declspec(dllexport)) to 
> control symbol visibility, directly in the source code. This has 
> historically not been present on all compilers, so we had to resort to 
> using mapfiles (also known as linker scripts).
>
> This is no longer the case. Now all compilers we use support symbol 
> visibility directives, in one form or another. We should start using 
> this. Since this has been the only way to control symbol visibility on 
> Windows, for most of the shared code, we already have proper JNIEXPORT 
> decorations in place.
>
> If we fix the remaining platform-specific files to have proper 
> JNIEXPORT tagging, then we can finally get rid of mapfiles.
>
> This fix removed mapfiles for all JDK libraries. It does not touch 
> hotspot libraries nor JDK executables; they will have to wait for a 
> future fix -- this was complex enough. This change will not have any 
> impact on macosx, since we do not use mapfiles there, but instead 
> export all symbols. (This is not a good idea, but I'll address that 
> separately.) This change will also have a minimal impact on Windows. 
> The only reason Windows is impacted at all, is that some changes 
> needed by Solaris and Linux were simpler to fix for all platforms.
>
> I have strived for this change to have no impact on the actual 
> generated code. Unfortunately, this was not possible to fully achieve. 
> I do not believe that these changes will have any actual impact on the 
> product, though. I will present the differences more in detail further 
> down. Those who are not interested can probably skip that.
>
> The patch has passed tier1 testing and is currently running tier2 and 
> tier3. Since the running code is more or less (see caveat below) 
> unmodified, I don't expect any testing issues.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8200178
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8200178-remove-mapfiles/webrev.01
>
> Details on changes:
> Most of the source code changes are (unsurprisingly) in java.base and 
> java.desktop. Remaining changes are in jdk.crypto.ucrypto, 
> jdk.hotspot.agent, jdk.jdi and jdk.jdwp.agent.
>
> Source code changes does almost to 100% consists in decorating an 
> exported function with JNIEXPORT. I have also followed the 
> long-standing convention of adding JNICALL. This is a no-op on 
> non-Windows platforms, so for most of the changes this is purely 
> cosmetic (and possibly adding in robustness, should the function ever 
> be used on Windows in the future). I have also followed the stylistic 
> convention of putting "JNIEXPORT <return type> JNICALL" on a separate 
> line. For some functions, however, this might cause a change in 
> calling convention on Windows. Since this can not apply to exported 
> functions on Windows (otherwise they would already have had 
> JNIEXPORT), I do not think this matters anything.
>
> A few libraries did not have a mapfile, on Linux and/or Solaris. This 
> actually meant that all symbols were exported. It is highly unclear if 
> this was known and intended by the original make rule writer. I have 
> emulated this by adding the flag $(EXPORT_ALL_SYMBOLS) to these 
> libraries. Hopefully, we can remove this flag and fix proper exported 
> symbols in the future.
>
> I have run the complete build using COMPARE_BUILD, and made a 
> thourough analysis of the differences for Linux and Solaris. All 
> native libraries have symbol differences, but most of them are trivial 
> and/or harmless. As a result, most libraries have disasm differences 
> as well, but these too seem trivial and harmless. The differences in 
> symbols that are common to all libraries include:
>  * Internal symbols such as __bss_start, _edata, _end and _fini are 
> now global. (They are imported as such from the compiler 
> libraries/archives, and we have no linker script to override this 
> behavior).
>  * The versioning tag SUNWprivate_1.1 is not included, and thus 
> neither the .gnu.version_d symbol.
>  * There are a few differences in the symbol and/or mangling of some 
> local functions. I'm not sure what's causing this,
> but it's unlikely to have any effect on the product.
>
> Another common source for change in symbols is due to previous 
> platform differences. For instance, if we had "JNIEXPORT int JNICALL 
> do_foo() { ... }", but do_foo was not in the mapfile, the symbol was 
> exported on Windows but not on Linux and Solaris. (Presumable since it 
> was not needed there, even though it was compiled for those platforms 
> as well.) Now, with the mapfiles gone, do_foo() will be exported on 
> all platforms. And contrary, functions that are compiled on all 
> platforms, and were exported in mapfiles, but now have gotten an 
> JNIEXPORT decoration, will now be visible even on Windows. (This 
> accounts for half of the noticed symbol differences on Windows.) I 
> could have made the JNIEXPORT conditional on OS, but I didn't think 
> the mess in source code were worth the keeping of binary confidence 
> with the old build.
>
> A third common source for change in symbols is due to exported 
> functions "leaking" across library borders. For instance, some 
> functions in java.desktop is compiled in both libawt_xawt and 
> libawt_headless, but they were previously only included in the mapfile 
> for one of these libraries. Now, since the visibility is determined by 
> the source code itself, it gets exported in both libraries. A variant 
> of this is when a library depends on another JDK library, and includes 
> the header file from that other library, which in turn declares a 
> function as JNIEXPORT. This will cause the including library to also 
> export the function. This accounts for the other half of the changes 
> on Windows. A typical example of this is that multiple libraries now 
> re-export hotspot symbols from libjvm.so, like jio_fprintf. (I have 
> not listed the libjvm re-exports below.)
>
> Note that  Java_java_io_FileOutputStream_close0 in 
> java.base/unix/native/libjava/FileOutputStream_md.c is no longer 
> exported,
> and can probably be removed.
>
> Here is a detailed table showing and accounting for all the remaining 
> differences found on Linux and Solaris:
> java.base/unix/native/libjava: Java_java_io_FileOutputStream_close0 is 
> now also exported on unix platforms due to JNIEXPORT.
>
> java.base/jspawnlauncher: On solaris, we also include 
> libjava/childproc.o, which
> now exports less functions than it used to (it used to export all 
> functions, now it is compiled with visibility=hidden).
>
> java.base/java(w).exe: Is now also exporting the following symbols due 
> to added JNIEXPORT in libjli on Windows:
> (Yes, executables can export symbols on Windows. Confusing, I know.)
>  JLI_AddArgsFromEnvVar
>  JLI_CmdToArgs
>  JLI_GetAppArgIndex
>  JLI_GetStdArgc
>  JLI_GetStdArgs
>  JLI_InitArgProcessing
>  JLI_Launch
>  JLI_List_add
>  JLI_List_new
>  JLI_ManifestIterate
>  JLI_MemAlloc
>  JLI_MemFree
>  JLI_PreprocessArg
>  JLI_ReportErrorMessage
>  JLI_ReportErrorMessageSys
>  JLI_ReportExceptionDescription
>  JLI_ReportMessage
>  JLI_SetTraceLauncher
>  JLI_StringDup
>
> java.desktop:/libawt_xawt: The following symbols are now also exported 
> on linux and solaris due to JNIEXPORT:
>  awt_DrawingSurface_FreeDrawingSurfaceInfo
>  awt_DrawingSurface_GetDrawingSurfaceInfo
>  awt_DrawingSurface_Lock
>  awt_DrawingSurface_Unlock
>  awt_GetColor
>
> The following symbols are now also exported on linux and solaris due 
> to JNIEXPORT (they were previously
>  exported only in libawt):
>  Java_sun_awt_DebugSettings_setCTracingOn__Z
>  Java_sun_awt_DebugSettings_setCTracingOn__ZLjava_lang_String_2
>  Java_sun_awt_DebugSettings_setCTracingOn__ZLjava_lang_String_2I
>  Java_sun_awt_X11GraphicsConfig_getNumColors
>
> java.desktop:/libawt_headless: The following symbols are now also 
> exported due to JNIEXPORT (they were previously
>  exported only in libawt_xawt and/or libawt):
>  Java_sun_java2d_opengl_GLXGraphicsConfig_getGLXConfigInfo
>  Java_sun_java2d_opengl_GLXGraphicsConfig_getOGLCapabilities
>  Java_sun_java2d_x11_X11PMBlitLoops_updateBitmask
>  Java_sun_java2d_x11_X11SurfaceData_isShmPMAvailable
>  X11SurfaceData_GetOps
>
> java.desktop/libawt: The following symbols are now also exported on 
> Windows, due to added
> JNIEXPORT:
>  SurfaceData_InitOps
>  mul8table
>  div8table
>  doDrawPath
>  doFillPath
>  g_CMpDataID
>  initInverseGrayLut
>  make_dither_arrays
>  make_uns_ordered_dither_array
>  path2DFloatCoordsID
>  path2DNumTypesID
>  path2DTypesID
>  path2DWindingRuleID
>  sg2dStrokeHintID
>  std_img_oda_blue
>  std_img_oda_green
>  std_img_oda_red
>  std_odas_computed
>  sunHints_INTVAL_STROKE_PURE
>
> java.desktop/libawt on solaris:
> A number of "#pragma weak" directives was previously overridden by the 
> mapfile.
> Now these directives are respected, so these symbols are now weak 
> instead of local:
>  ByteGrayToIntArgbPreConvert_F
>  ByteGrayToIntArgbPreScaleConvert_F
>  IntArgbBmToFourByteAbgrPreScaleXparOver_F
>  IntArgbToIntRgbXorBlit_F
>  IntBgrToIntBgrAlphaMaskBlit_F
>
> java.desktop/libawt on solaris: These are now also exported due to 
> JNIEXPORT in libmlib_image.
>  j2d_mlib_ImageCreate
>  j2d_mlib_ImageCreateStruct
>  j2d_mlib_ImageDelete
>
> java.desktop/libawt on solaris: This is now also exported due to 
> JNIEXPORT:
>  GrPrim_CompGetXorColor
>  SurfaceData_GetOpsNoSetup
>  SurfaceData_IntersectBoundsXYWH
>  SurfaceData_SetOps
>  Transform_GetInfo
>  Transform_transform
>
> java.desktop/libsplashscreen: JNI_OnLoad is now exported on linux and 
> solaris due to JNIEXPORT.
> libspashscreen also had JNIEXPORT (actually a pure 
> _declspec(dllexport)) but no JNICALL, which I added as
> a part of converting to JNIEXPORT. The same goes for libmlib_image .
>
> jdk.sctp/libsctp: handleSocketError is now exported on linux and 
> solaris due to JNIEXPORT in libnio.
>
> java.instrument:/libinstrument: Agent_OnUnload is now also exported on 
> linux and solaris platforms due to JNIEXPORT.
> JLI_ManifestIterate is now also exported on Windows, due to added 
> JNIEXPORT in libjli.
>
> jdk.management/libmanagement_ext: 
> Java_com_sun_management_internal_Flag_setDoubleValue is now also 
> exported on linux and solaris platforms due to JNIEXPORT.
>
> /Magnus
>
>