From erik.joelsson at oracle.com  Thu Jan  2 07:57:07 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 2 Jan 2020 08:57:07 +0100
Subject: <AWT Dev> [15] Review Request: 8235975 Update copyright year to
 match last edit in jdk repository for 2014/15/16/17/18
In-Reply-To: <3460a6f6-6178-cc45-5840-0f215eebc53f@oracle.com>
References: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com>
 <3460a6f6-6178-cc45-5840-0f215eebc53f@oracle.com>
Message-ID: <7b67e39d-752c-50a8-8e18-9a8f86bd641c@oracle.com>

Build files look good.

/Erik

On 2019-12-24 19:22, Sergey Bylokhov wrote:
> Hello.
>
> Here is an updated version:
> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8235975
> ? Patch (2 Mb): 
> http://cr.openjdk.java.net/~serb/8235975/webrev.03/open.patch
> ? Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.03/
>
> ?- "jdk.internal.vm.compiler" is removed from the patch.
> ?- "Aes128CtsHmacSha2EType.java" is updated to "Copyright (c) 2018"
>
> On 12/22/19 11:24 pm, Sergey Bylokhov wrote:
>> Hello.
>> Please review the fix for JDK 15.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235975
>> Patch (2 Mb): 
>> http://cr.openjdk.java.net/~serb/8235975/webrev.02/open.patch
>> Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.02
>>
>> I have updated the source code copyrights by the 
>> "update_copyright_year.sh"
>> script for 2014/15/16/18/19 years, unfortunately, cannot run it for 2017
>> because of: "JDK-8187443: Forest Consolidation: Move files to unified 
>> layout"
>> which touched all files.
>>
>>
>
>


From erik.joelsson at oracle.com  Thu Jan  2 08:09:01 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 2 Jan 2020 09:09:01 +0100
Subject: RFR: 8236488: Support for configure option
 --with-native-debug-symbols=internal is impossible on Windows
In-Reply-To: <AM6PR02MB49849E5E1C5A015B67175BA88A2F0@AM6PR02MB4984.eurprd02.prod.outlook.com>
References: <AM6PR02MB49849E5E1C5A015B67175BA88A2F0@AM6PR02MB4984.eurprd02.prod.outlook.com>
Message-ID: <eb54bc6b-a37e-0fd2-a8ae-b8bd5aa14a95@oracle.com>

Hello Christoph,

I think the change makes sense and follows the established behavior on 
AIX for invalid settings.

In GtestImage.gmk, instead of repeating the SetupCopy call, you can do 
something like this:

41 $(foreach v, $(JVM_VARIANTS), \
42 $(eval $(call SetupCopyFiles, COPY_GTEST_MSVCR_$v, \
43 DEST := $(TEST_IMAGE_DIR)/hotspot/gtest/$v, \
44 FILES := $(MSVCR_DLL) $(MSVCP_DLL), \
45 FLATTEN := true, \
46 )) \
47 $(eval TARGETS += $$(COPY_GTEST_MSVCR_$v)) \
          $(if $(call equals, $(COPY_DEBUG_SYMBOLS), true), \
   48 $(eval $(call SetupCopyFiles, COPY_GTEST_PDB_$v, \
49 SRC := $(HOTSPOT_OUTPUTDIR)/variant-$v/libjvm/gtest, \
50 DEST := $(TEST_IMAGE_DIR)/hotspot/gtest/$v, \
51 FILES := jvm.pdb gtestLauncher.pdb, \
52 )) \   53 $(eval TARGETS += $$(COPY_GTEST_PDB_$v)) \
) \ 54 )

Otherwise this looks good.

/Erik

On 2019-12-23 00:24, Langer, Christoph wrote:
> Hi,
>
> please review the following change as a follow up to an earlier discussion here: http://mail.openjdk.java.net/pipermail/build-dev/2019-December/026408.html
>
> On Windows it's not possible to support configure option "--with-native-debug-symbols=internal" in a way that one would expect. E.g. native debug information should be part of the binaries while on Windows it's only possible to have the debug information in external .pdb files. So it seems natural to reject "--with-native-debug-symbols=internal" on Windows.
>
> In my proposed patch I change this, together with some minor correction for AIX and a little cleanup. I also repair a build error that would currently happen in gtest image if one is to build target 'images' on Windows while having set "--with-native-debug-symbols=internal". Note that this path could not be taken any more, though. I only added this for completeness since I ran over it during my experiments.
>
> An alternative to rejecting "--with-native-debug-symbols=internal" would be to implicitly convert it to "--with-native-debug-symbols=external" but I think it is better if the user gets the feedback that the option he intended to use won't work.
>
> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8236488.0/
> Bug: https://bugs.openjdk.java.net/browse/JDK-8236488
>
> Thanks
> Christoph
>


From Sergey.Bylokhov at oracle.com  Thu Jan  2 12:02:14 2020
From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov)
Date: Thu, 2 Jan 2020 15:02:14 +0300
Subject: <AWT Dev> [15] Review Request: 8235975 Update copyright year to
 match last edit in jdk repository for 2014/15/16/17/18
In-Reply-To: <7b67e39d-752c-50a8-8e18-9a8f86bd641c@oracle.com>
References: <1e7d0395-fc57-4d5b-9cfa-c33e0f6462d5@oracle.com>
 <3460a6f6-6178-cc45-5840-0f215eebc53f@oracle.com>
 <7b67e39d-752c-50a8-8e18-9a8f86bd641c@oracle.com>
Message-ID: <0caab700-28e3-16e7-db00-b698557443f0@oracle.com>

I guess it is too late to fix it, will need to update the files at the end of 2020.

On 1/2/20 10:57 am, Erik Joelsson wrote:
> Build files look good.
> 
> /Erik
> 
> On 2019-12-24 19:22, Sergey Bylokhov wrote:
>> Hello.
>>
>> Here is an updated version:
>> ? Bug: https://bugs.openjdk.java.net/browse/JDK-8235975
>> ? Patch (2 Mb): http://cr.openjdk.java.net/~serb/8235975/webrev.03/open.patch
>> ? Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.03/
>>
>> ?- "jdk.internal.vm.compiler" is removed from the patch.
>> ?- "Aes128CtsHmacSha2EType.java" is updated to "Copyright (c) 2018"
>>
>> On 12/22/19 11:24 pm, Sergey Bylokhov wrote:
>>> Hello.
>>> Please review the fix for JDK 15.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8235975
>>> Patch (2 Mb): http://cr.openjdk.java.net/~serb/8235975/webrev.02/open.patch
>>> Fix: http://cr.openjdk.java.net/~serb/8235975/webrev.02
>>>
>>> I have updated the source code copyrights by the "update_copyright_year.sh"
>>> script for 2014/15/16/18/19 years, unfortunately, cannot run it for 2017
>>> because of: "JDK-8187443: Forest Consolidation: Move files to unified layout"
>>> which touched all files.
>>>
>>>
>>
>>


-- 
Best regards, Sergey.


From joe.darcy at oracle.com  Fri Jan  3 02:46:13 2020
From: joe.darcy at oracle.com (Joe Darcy)
Date: Thu, 2 Jan 2020 18:46:13 -0800
Subject: JDK 15 RFR of JDK-8235532: Update --release 14 symbol information
 for JDK 14 b27
In-Reply-To: <e7171de8-6dcb-a4c6-4144-5c0763ebe668@oracle.com>
References: <e7171de8-6dcb-a4c6-4144-5c0763ebe668@oracle.com>
Message-ID: <c25f2ab6-62ca-3b32-6778-86e666df124a@oracle.com>

Reminder for the review; happy new year,

-Joe

On 12/14/2019 3:22 PM, Joe Darcy wrote:
> Hello,
>
> The latest promoted build of JDK 14 before the fork was b26; 
> therefore, the 26 API was used to create the --release 14 symbol 
> information for JDK 15 b01 (coming rsn). Please review updating this 
> information in JDK 15 for JDK 14 b27:
>
> ??? http://cr.openjdk.java.net/~darcy/8235532.0/
>
> Eye-balling the changes, they look plausible for the JDK 14 changes 
> included in b27, including records, jpackage, and the removal of pack200.
>
> Thanks,
>
> -Joe
>


From christoph.langer at sap.com  Sun Jan  5 16:47:57 2020
From: christoph.langer at sap.com (Langer, Christoph)
Date: Sun, 5 Jan 2020 16:47:57 +0000
Subject: RFR: 8236488: Support for configure option
 --with-native-debug-symbols=internal is impossible on Windows
In-Reply-To: <eb54bc6b-a37e-0fd2-a8ae-b8bd5aa14a95@oracle.com>
References: <AM6PR02MB49849E5E1C5A015B67175BA88A2F0@AM6PR02MB4984.eurprd02.prod.outlook.com>
 <eb54bc6b-a37e-0fd2-a8ae-b8bd5aa14a95@oracle.com>
Message-ID: <AM6PR02MB498452650C8C5BD80E6306938A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>

Hi Erik,



thanks for your review and the hint about GtestImage.gmk.



That would be the next version then: http://cr.openjdk.java.net/~clanger/webrevs/8236488.1/ I tested it and seems to work as intended.



Good to go?



Thanks

Christoph





From: Erik Joelsson <erik.joelsson at oracle.com>
Sent: Donnerstag, 2. Januar 2020 09:09
To: Langer, Christoph <christoph.langer at sap.com>; build-dev <build-dev at openjdk.java.net>
Subject: Re: RFR: 8236488: Support for configure option --with-native-debug-symbols=internal is impossible on Windows



Hello Christoph,

I think the change makes sense and follows the established behavior on AIX for invalid settings.

In GtestImage.gmk, instead of repeating the SetupCopy call, you can do something like this:

  41   $(foreach v, $(JVM_VARIANTS), \
  42     $(eval $(call SetupCopyFiles, COPY_GTEST_MSVCR_$v, \
  43         DEST := $(TEST_IMAGE_DIR)/hotspot/gtest/$v, \
  44         FILES := $(MSVCR_DLL) $(MSVCP_DLL), \
  45         FLATTEN := true, \
  46     )) \
  47     $(eval TARGETS += $$(COPY_GTEST_MSVCR_$v)) \
         $(if $(call equals, $(COPY_DEBUG_SYMBOLS), true), \
  48       $(eval $(call SetupCopyFiles, COPY_GTEST_PDB_$v, \
  49           SRC := $(HOTSPOT_OUTPUTDIR)/variant-$v/libjvm/gtest, \
  50           DEST := $(TEST_IMAGE_DIR)/hotspot/gtest/$v, \
  51           FILES := jvm.pdb gtestLauncher.pdb, \
  52       )) \
  53       $(eval TARGETS += $$(COPY_GTEST_PDB_$v)) \
         ) \
  54   )

Otherwise this looks good.



/Erik



On 2019-12-23 00:24, Langer, Christoph wrote:

   Hi,

   please review the following change as a follow up to an earlier discussion here: http://mail.openjdk.java.net/pipermail/build-dev/2019-December/026408.html

   On Windows it's not possible to support configure option "--with-native-debug-symbols=internal" in a way that one would expect. E.g. native debug information should be part of the binaries while on Windows it's only possible to have the debug information in external .pdb files. So it seems natural to reject "--with-native-debug-symbols=internal" on Windows.

   In my proposed patch I change this, together with some minor correction for AIX and a little cleanup. I also repair a build error that would currently happen in gtest image if one is to build target 'images' on Windows while having set "--with-native-debug-symbols=internal". Note that this path could not be taken any more, though. I only added this for completeness since I ran over it during my experiments.

   An alternative to rejecting "--with-native-debug-symbols=internal" would be to implicitly convert it to "--with-native-debug-symbols=external" but I think it is better if the user gets the feedback that the option he intended to use won't work.

   Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8236488.0/
   Bug: https://bugs.openjdk.java.net/browse/JDK-8236488

   Thanks
   Christoph



From jan.lahoda at oracle.com  Mon Jan  6 12:21:15 2020
From: jan.lahoda at oracle.com (Jan Lahoda)
Date: Mon, 6 Jan 2020 13:21:15 +0100
Subject: JDK 15 RFR of JDK-8235532: Update --release 14 symbol information
 for JDK 14 b27
In-Reply-To: <c25f2ab6-62ca-3b32-6778-86e666df124a@oracle.com>
References: <e7171de8-6dcb-a4c6-4144-5c0763ebe668@oracle.com>
 <c25f2ab6-62ca-3b32-6778-86e666df124a@oracle.com>
Message-ID: <a780b149-73ca-83ca-e92f-e8160d598a06@oracle.com>

Looks good to me.

Jan

On 03. 01. 20 3:46, Joe Darcy wrote:
> Reminder for the review; happy new year,
> 
> -Joe
> 
> On 12/14/2019 3:22 PM, Joe Darcy wrote:
>> Hello,
>>
>> The latest promoted build of JDK 14 before the fork was b26; 
>> therefore, the 26 API was used to create the --release 14 symbol 
>> information for JDK 15 b01 (coming rsn). Please review updating this 
>> information in JDK 15 for JDK 14 b27:
>>
>> ??? http://cr.openjdk.java.net/~darcy/8235532.0/
>>
>> Eye-balling the changes, they look plausible for the JDK 14 changes 
>> included in b27, including records, jpackage, and the removal of pack200.
>>
>> Thanks,
>>
>> -Joe
>>


From philip.race at oracle.com  Mon Jan  6 17:41:18 2020
From: philip.race at oracle.com (Philip Race)
Date: Mon, 06 Jan 2020 09:41:18 -0800
Subject: [14] Review Request: 8233827 Enable screenshots in the enhanced
 failure handler on Linux/macOS
In-Reply-To: <4ea1a4a8-9d7a-4b4b-804c-b22c4bc0cbe7@oracle.com>
References: <931be5f0-085a-37e7-c188-18128d92227d@oracle.com>
 <57e519a8-a4a1-02f4-d06e-674a10fe9cdd@oracle.com>
 <4ea1a4a8-9d7a-4b4b-804c-b22c4bc0cbe7@oracle.com>
Message-ID: <5E13713E.1020607@oracle.com>

That didn't answer all my questions, at least not in a way that I can 
understand.

How is this useful given that we disable jtreg failure handlers for the 
headful tests ?

-phil.

On 12/30/19, 11:33 AM, Sergey Bylokhov wrote:
> On 12/23/19 9:15 pm, Phil Race wrote:
>> I am not sure what the right mailing list(s) are for this change.
>> It definitely isn't a core-libs change. I think build-dev may be better.
>
> Previous changes to these configs were discussed here, so I have send 
> it here as well.
>
>>
>> I am also unclear when this failure handler is invoked and how all 
>> this machinery works.
>>
>> It is only useful for headful tests and so I'd only want it enabled 
>> in such a case.
>> And we disable the failure handlers in the headful test jobs anyway 
>> because they seem
>> focused on taking pointless core dumps ...> So we need something that 
>> can be used with headful tests only and doesn't involve
>> re-enabling the other handlers.
> It could be useful for other tests as well and may be able to identify 
> problems such as:
>  - Suggestions "to open under debugger" from the native asserts
>  - Various error dialogs from the OS
> And it does not spend much resources compared to current handlers.
>
>>
>> Also why exclude Windows ? No easy way to get the screenshot ?
>
> There is no command-line program that can take a screenshot on windows 
> by default
>
>>
>> -phil.
>>
>> On 12/11/19 1:00 AM, Sergey Bylokhov wrote:
>>> Hello.
>>> Please review the fix for JDK 14.
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8233827
>>> Fix: http://cr.openjdk.java.net/~serb/8233827/webrev.01
>>>
>>> This change adds the "screen capture on the test failure" feature on 
>>> macOS and Linux.
>>>  - On Linux, it is implemented by the "gnome-screenshot" command(in 
>>> case of
>>>    multiscreen+xinerama    the whole big screen will be saved to the 
>>> "screen.png" file).
>>>  - On macOS it is implemented by the "screencapture" command, note 
>>> that I have
>>>    used 1 file per screen, if the number of screens less than 5, 
>>> other files will be ignored.
>>>
>>
>
>


From Sergey.Bylokhov at oracle.com  Mon Jan  6 20:34:30 2020
From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov)
Date: Mon, 6 Jan 2020 23:34:30 +0300
Subject: [14] Review Request: 8233827 Enable screenshots in the enhanced
 failure handler on Linux/macOS
In-Reply-To: <5E13713E.1020607@oracle.com>
References: <931be5f0-085a-37e7-c188-18128d92227d@oracle.com>
 <57e519a8-a4a1-02f4-d06e-674a10fe9cdd@oracle.com>
 <4ea1a4a8-9d7a-4b4b-804c-b22c4bc0cbe7@oracle.com>
 <5E13713E.1020607@oracle.com>
Message-ID: <9ed4eec6-b1ca-8c62-9b50-5c94db384795@oracle.com>

On 1/6/20 8:41 pm, Philip Race wrote:
> How is this useful given that we disable jtreg failure handlers for the headful tests ?

It is disabled only in mach5 for headful nightly and CI builds, but it is enabled for
other builds, also it is enabled if the headful tests are executed standalone via
"make test", and it could easily be enabled in mach5 for personal jobs(I do this time to time)

> 
> -phil.
> 
> On 12/30/19, 11:33 AM, Sergey Bylokhov wrote:
>> On 12/23/19 9:15 pm, Phil Race wrote:
>>> I am not sure what the right mailing list(s) are for this change.
>>> It definitely isn't a core-libs change. I think build-dev may be better.
>>
>> Previous changes to these configs were discussed here, so I have send it here as well.
>>
>>>
>>> I am also unclear when this failure handler is invoked and how all this machinery works.
>>>
>>> It is only useful for headful tests and so I'd only want it enabled in such a case.
>>> And we disable the failure handlers in the headful test jobs anyway because they seem
>>> focused on taking pointless core dumps ...> So we need something that can be used with headful tests only and doesn't involve
>>> re-enabling the other handlers.
>> It could be useful for other tests as well and may be able to identify problems such as:
>> ?- Suggestions "to open under debugger" from the native asserts
>> ?- Various error dialogs from the OS
>> And it does not spend much resources compared to current handlers.
>>
>>>
>>> Also why exclude Windows ? No easy way to get the screenshot ?
>>
>> There is no command-line program that can take a screenshot on windows by default
>>
>>>
>>> -phil.
>>>
>>> On 12/11/19 1:00 AM, Sergey Bylokhov wrote:
>>>> Hello.
>>>> Please review the fix for JDK 14.
>>>>
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8233827
>>>> Fix: http://cr.openjdk.java.net/~serb/8233827/webrev.01
>>>>
>>>> This change adds the "screen capture on the test failure" feature on macOS and Linux.
>>>> ?- On Linux, it is implemented by the "gnome-screenshot" command(in case of
>>>> ?? multiscreen+xinerama??? the whole big screen will be saved to the "screen.png" file).
>>>> ?- On macOS it is implemented by the "screencapture" command, note that I have
>>>> ?? used 1 file per screen, if the number of screens less than 5, other files will be ignored.
>>>>
>>>
>>
>>


-- 
Best regards, Sergey.


From erik.joelsson at oracle.com  Wed Jan  8 16:09:54 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 8 Jan 2020 17:09:54 +0100
Subject: RFR: 8236488: Support for configure option
 --with-native-debug-symbols=internal is impossible on Windows
In-Reply-To: <AM6PR02MB498452650C8C5BD80E6306938A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>
References: <AM6PR02MB49849E5E1C5A015B67175BA88A2F0@AM6PR02MB4984.eurprd02.prod.outlook.com>
 <eb54bc6b-a37e-0fd2-a8ae-b8bd5aa14a95@oracle.com>
 <AM6PR02MB498452650C8C5BD80E6306938A3D0@AM6PR02MB4984.eurprd02.prod.outlook.com>
Message-ID: <d2cec056-abe6-2c5e-faac-5fdd01a62be7@oracle.com>

Looks good.

/Erik

On 2020-01-05 17:47, Langer, Christoph wrote:
>
> Hi Erik,
>
> thanks for your review and the hint about GtestImage.gmk.
>
> That would be the next version then: 
> http://cr.openjdk.java.net/~clanger/webrevs/8236488.1/ I tested it and 
> seems to work as intended.
>
> Good to go?
>
> Thanks
>
> Christoph
>
> *From:*Erik Joelsson <erik.joelsson at oracle.com>
> *Sent:* Donnerstag, 2. Januar 2020 09:09
> *To:* Langer, Christoph <christoph.langer at sap.com>; build-dev 
> <build-dev at openjdk.java.net>
> *Subject:* Re: RFR: 8236488: Support for configure option 
> --with-native-debug-symbols=internal is impossible on Windows
>
> Hello Christoph,
>
> I think the change makes sense and follows the established behavior on 
> AIX for invalid settings.
>
> In GtestImage.gmk, instead of repeating the SetupCopy call, you can do 
> something like this:
>
> ? 41?? $(foreach v, $(JVM_VARIANTS), \
> ? 42???? $(eval $(call SetupCopyFiles, COPY_GTEST_MSVCR_$v, \
> ? 43???????? DEST := $(TEST_IMAGE_DIR)/hotspot/gtest/$v, \
> ? 44???????? FILES := $(MSVCR_DLL) $(MSVCP_DLL), \
> ? 45???????? FLATTEN := true, \
> ? 46???? )) \
> ? 47???? $(eval TARGETS += $$(COPY_GTEST_MSVCR_$v)) \
>  ???????? $(if $(call equals, $(COPY_DEBUG_SYMBOLS), true), \
>    48?????? $(eval $(call SetupCopyFiles, COPY_GTEST_PDB_$v, \
> ? 49?????????? SRC := $(HOTSPOT_OUTPUTDIR)/variant-$v/libjvm/gtest, \
> ? 50?????????? DEST := $(TEST_IMAGE_DIR)/hotspot/gtest/$v, \
> ? 51?????????? FILES := jvm.pdb gtestLauncher.pdb, \
> ? 52?????? )) \
>   53?????? $(eval TARGETS += $$(COPY_GTEST_PDB_$v)) \
> ???????? ) \
> ? 54?? )
>
> Otherwise this looks good.
>
> /Erik
>
> On 2019-12-23 00:24, Langer, Christoph wrote:
>
>     Hi,
>
>     please review the following change as a follow up to an earlier discussion here:http://mail.openjdk.java.net/pipermail/build-dev/2019-December/026408.html
>
>     On Windows it's not possible to support configure option "--with-native-debug-symbols=internal" in a way that one would expect. E.g. native debug information should be part of the binaries while on Windows it's only possible to have the debug information in external .pdb files. So it seems natural to reject "--with-native-debug-symbols=internal" on Windows.
>
>     In my proposed patch I change this, together with some minor correction for AIX and a little cleanup. I also repair a build error that would currently happen in gtest image if one is to build target 'images' on Windows while having set "--with-native-debug-symbols=internal". Note that this path could not be taken any more, though. I only added this for completeness since I ran over it during my experiments.
>
>     An alternative to rejecting "--with-native-debug-symbols=internal" would be to implicitly convert it to "--with-native-debug-symbols=external" but I think it is better if the user gets the feedback that the option he intended to use won't work.
>
>     Webrev:http://cr.openjdk.java.net/~clanger/webrevs/8236488.0/
>
>     Bug:https://bugs.openjdk.java.net/browse/JDK-8236488
>
>     Thanks
>
>     Christoph
>


From dms at samersoff.net  Thu Jan  9 08:29:41 2020
From: dms at samersoff.net (Dmitry Samersoff)
Date: Thu, 9 Jan 2020 11:29:41 +0300
Subject: [APPCDS] JDK-8236847: JDK built on machine with 4K pages refused to
 start on machine with 64k pages
Message-ID: <2d6a9132-7aee-fb72-557f-580f4cdee91f@samersoff.net>

Hello Ioi,

*Story*

If you build JDK-14 with the changes for

  JDK-8231610 Relocate the CDS archive if it cannot be mapped
  to the requested address

on a machine with 4k pages (typical Jenkins machine) then move the
bundle to a huge server with 64k pages, it refuses to start with

[0.015s][error][cds] Unable to map CDS archive --
os::vm_allocation_granularity() expected: 4096 actual: 65536

*Question*

I'm not sure if just removing an assert is the correct fix. What would
you recommend?

Also @build-dev - is it possible to add configure option to disable
AppCDS archive generation?


-Dmitry\S





From david.holmes at oracle.com  Thu Jan  9 08:47:21 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 9 Jan 2020 18:47:21 +1000
Subject: [APPCDS] JDK-8236847: JDK built on machine with 4K pages refused
 to start on machine with 64k pages
In-Reply-To: <2d6a9132-7aee-fb72-557f-580f4cdee91f@samersoff.net>
References: <2d6a9132-7aee-fb72-557f-580f4cdee91f@samersoff.net>
Message-ID: <0c06ae6b-e4f9-4fc5-2c37-8e66a0fa1aee@oracle.com>

On 9/01/2020 6:29 pm, Dmitry Samersoff wrote:
> Hello Ioi,
> 
> *Story*
> 
> If you build JDK-14 with the changes for
> 
>    JDK-8231610 Relocate the CDS archive if it cannot be mapped
>    to the requested address
> 
> on a machine with 4k pages (typical Jenkins machine) then move the
> bundle to a huge server with 64k pages, it refuses to start with
> 
> [0.015s][error][cds] Unable to map CDS archive --
> os::vm_allocation_granularity() expected: 4096 actual: 65536
> 
> *Question*
> 
> I'm not sure if just removing an assert is the correct fix. What would
> you recommend?

I thought that failing to map the archive was a non-fatal error that 
just results in falling back to not using CDS. Even in a debug build 
this should not be an assertion unless perhaps the user has explicitly 
requested CDS to be enabled.

> Also @build-dev - is it possible to add configure option to disable
> AppCDS archive generation?

Already there I think: --disable-cds-archive

Cheers,
David
-----


> 
> -Dmitry\S
> 
> 
> 


From ioi.lam at oracle.com  Thu Jan  9 17:02:09 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Thu, 9 Jan 2020 09:02:09 -0800
Subject: [APPCDS] JDK-8236847: JDK built on machine with 4K pages refused
 to start on machine with 64k pages
In-Reply-To: <0c06ae6b-e4f9-4fc5-2c37-8e66a0fa1aee@oracle.com>
References: <2d6a9132-7aee-fb72-557f-580f4cdee91f@samersoff.net>
 <0c06ae6b-e4f9-4fc5-2c37-8e66a0fa1aee@oracle.com>
Message-ID: <25aeebef-52e6-0ff5-1dee-d8be91af137e@oracle.com>



On 1/9/20 12:47 AM, David Holmes wrote:
> On 9/01/2020 6:29 pm, Dmitry Samersoff wrote:
>> Hello Ioi,
>>
>> *Story*
>>
>> If you build JDK-14 with the changes for
>>
>> ?? JDK-8231610 Relocate the CDS archive if it cannot be mapped
>> ?? to the requested address
>>
>> on a machine with 4k pages (typical Jenkins machine) then move the
>> bundle to a huge server with 64k pages, it refuses to start with
>>
>> [0.015s][error][cds] Unable to map CDS archive --
>> os::vm_allocation_granularity() expected: 4096 actual: 65536
>>
>> *Question*
>>
>> I'm not sure if just removing an assert is the correct fix. What would
>> you recommend?
>
> I thought that failing to map the archive was a non-fatal error that 
> just results in falling back to not using CDS. Even in a debug build 
> this should not be an assertion unless perhaps the user has explicitly 
> requested CDS to be enabled.
>
Dmitry, are you running with -Xshare:on? If you use -Xshare:off (which 
is the default), when the CDS archive cannot be mapped, the JVM will 
continue execution.

Are you having problems running with the jtreg test cases? I can think 
of a way to fix this, so that CDS can continue to work under this 
circumstance, but I want to understand your scenarios more.

Thanks
- Ioi

>> Also @build-dev - is it possible to add configure option to disable
>> AppCDS archive generation?
>
> Already there I think: --disable-cds-archive
>
> Cheers,
> David
> -----
>
>
>>
>> -Dmitry\S
>>
>>
>>



From dms at samersoff.net  Fri Jan 10 09:40:26 2020
From: dms at samersoff.net (Dmitry Samersoff)
Date: Fri, 10 Jan 2020 12:40:26 +0300
Subject: [APPCDS] JDK-8236847: JDK built on machine with 4K pages refused
 to start on machine with 64k pages
In-Reply-To: <25aeebef-52e6-0ff5-1dee-d8be91af137e@oracle.com>
References: <2d6a9132-7aee-fb72-557f-580f4cdee91f@samersoff.net>
 <0c06ae6b-e4f9-4fc5-2c37-8e66a0fa1aee@oracle.com>
 <25aeebef-52e6-0ff5-1dee-d8be91af137e@oracle.com>
Message-ID: <293f8626-c40e-89a7-8c45-fea82bfad19b@samersoff.net>

Ioi,

> Dmitry, are you running with -Xshare:on? If you use -Xshare:off (which
> is the default), when the CDS archive cannot be mapped, the JVM will
> continue execution.

I use default parameters for java. i.e. ever simple

java -version

refuses to start with the message below.

> Are you having problems running with the jtreg test cases? I can think
> of a way to fix this, so that CDS can continue to work under this
> circumstance, but I want to understand your scenarios more.

Typical scenario is:
   We have AArch64 Jenkins build server with 4k pages that automatically

      - builds JDK
      - deploy it to a large server with 64k pages
      - run set of tests
          (regression and performance, not all of them are jtreg one)

-Dmitry




On 09.01.20 20:02, Ioi Lam wrote:
> 
> 
> On 1/9/20 12:47 AM, David Holmes wrote:
>> On 9/01/2020 6:29 pm, Dmitry Samersoff wrote:
>>> Hello Ioi,
>>>
>>> *Story*
>>>
>>> If you build JDK-14 with the changes for
>>>
>>> ?? JDK-8231610 Relocate the CDS archive if it cannot be mapped
>>> ?? to the requested address
>>>
>>> on a machine with 4k pages (typical Jenkins machine) then move the
>>> bundle to a huge server with 64k pages, it refuses to start with
>>>
>>> [0.015s][error][cds] Unable to map CDS archive --
>>> os::vm_allocation_granularity() expected: 4096 actual: 65536
>>>
>>> *Question*
>>>
>>> I'm not sure if just removing an assert is the correct fix. What would
>>> you recommend?
>>
>> I thought that failing to map the archive was a non-fatal error that
>> just results in falling back to not using CDS. Even in a debug build
>> this should not be an assertion unless perhaps the user has explicitly
>> requested CDS to be enabled.
>>
> Dmitry, are you running with -Xshare:on? If you use -Xshare:off (which
> is the default), when the CDS archive cannot be mapped, the JVM will
> continue execution.
> 
> Are you having problems running with the jtreg test cases? I can think
> of a way to fix this, so that CDS can continue to work under this
> circumstance, but I want to understand your scenarios more.
> 
> Thanks
> - Ioi
> 
>>> Also @build-dev - is it possible to add configure option to disable
>>> AppCDS archive generation?
>>
>> Already there I think: --disable-cds-archive
>>
>> Cheers,
>> David
>> -----
>>
>>
>>>
>>> -Dmitry\S
>>>
>>>
>>>
> 


From sgehwolf at redhat.com  Fri Jan 10 15:45:45 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Fri, 10 Jan 2020 16:45:45 +0100
Subject: RFR: 8236921: Add build target to produce a JDK image suitable for
 a Graal/SVM build
Message-ID: <4b34adfde790ee699e3cdf0ee113b835139320f1.camel@redhat.com>

Hi,

Currently there is no easy way to produce an OpenJDK build which could
subsequently be used to build Substrate VM (part of Graal CE). Basic
building blocks are there, but no actual JDK image is getting produced
with the static libs along side their dynamic counterparts. This patch
addresses this by adding a new target called 'graal-builder-image'
which produces a jdk image in $IMAGES/graal-builder-jdk. It basically
copies over libs from static-libs folder and the jdk folder to assemble
the desired result.

Bug: https://bugs.openjdk.java.net/browse/JDK-8236921
webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8236921/02/webrev/

I've tested this works for some snapshot of JDK 14 (rev 3009b27660be)
and graal at revision d9fb0b7dc35a9a7eb559a5e468bcc4c99e918449.

Thoughts?

Thanks,
Severin



From bob.vandette at oracle.com  Fri Jan 10 17:28:01 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Fri, 10 Jan 2020 12:28:01 -0500
Subject: RFR: 8236921: Add build target to produce a JDK image suitable
 for a Graal/SVM build
In-Reply-To: <4b34adfde790ee699e3cdf0ee113b835139320f1.camel@redhat.com>
References: <4b34adfde790ee699e3cdf0ee113b835139320f1.camel@redhat.com>
Message-ID: <40C2C62E-9612-431D-BF87-F72C442DAC07@oracle.com>

I like it!  

I have to manually copy the libs each time.

BTW:  I have a PR in our internal queue to update the graal repo so it can build with the latest JDK 14.

Bob.


> On Jan 10, 2020, at 10:45 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> 
> Hi,
> 
> Currently there is no easy way to produce an OpenJDK build which could
> subsequently be used to build Substrate VM (part of Graal CE). Basic
> building blocks are there, but no actual JDK image is getting produced
> with the static libs along side their dynamic counterparts. This patch
> addresses this by adding a new target called 'graal-builder-image'
> which produces a jdk image in $IMAGES/graal-builder-jdk. It basically
> copies over libs from static-libs folder and the jdk folder to assemble
> the desired result.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8236921
> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8236921/02/webrev/
> 
> I've tested this works for some snapshot of JDK 14 (rev 3009b27660be)
> and graal at revision d9fb0b7dc35a9a7eb559a5e468bcc4c99e918449.
> 
> Thoughts?
> 
> Thanks,
> Severin
> 



From erik.joelsson at oracle.com  Fri Jan 10 21:19:01 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 10 Jan 2020 13:19:01 -0800
Subject: RFR: 8236921: Add build target to produce a JDK image suitable
 for a Graal/SVM build
In-Reply-To: <40C2C62E-9612-431D-BF87-F72C442DAC07@oracle.com>
References: <4b34adfde790ee699e3cdf0ee113b835139320f1.camel@redhat.com>
 <40C2C62E-9612-431D-BF87-F72C442DAC07@oracle.com>
Message-ID: <fd1e8082-4527-74ef-2304-d84911af8e58@oracle.com>

Hello Severin,

Looks good.

/Erik

On 2020-01-10 09:28, Bob Vandette wrote:
> I like it!
>
> I have to manually copy the libs each time.
>
> BTW:  I have a PR in our internal queue to update the graal repo so it can build with the latest JDK 14.
>
> Bob.
>
>
>> On Jan 10, 2020, at 10:45 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
>>
>> Hi,
>>
>> Currently there is no easy way to produce an OpenJDK build which could
>> subsequently be used to build Substrate VM (part of Graal CE). Basic
>> building blocks are there, but no actual JDK image is getting produced
>> with the static libs along side their dynamic counterparts. This patch
>> addresses this by adding a new target called 'graal-builder-image'
>> which produces a jdk image in $IMAGES/graal-builder-jdk. It basically
>> copies over libs from static-libs folder and the jdk folder to assemble
>> the desired result.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8236921
>> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8236921/02/webrev/
>>
>> I've tested this works for some snapshot of JDK 14 (rev 3009b27660be)
>> and graal at revision d9fb0b7dc35a9a7eb559a5e468bcc4c99e918449.
>>
>> Thoughts?
>>
>> Thanks,
>> Severin
>>


From ioi.lam at oracle.com  Sat Jan 11 18:31:53 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Sat, 11 Jan 2020 10:31:53 -0800
Subject: [APPCDS] JDK-8236847: JDK built on machine with 4K pages refused
 to start on machine with 64k pages
In-Reply-To: <293f8626-c40e-89a7-8c45-fea82bfad19b@samersoff.net>
References: <2d6a9132-7aee-fb72-557f-580f4cdee91f@samersoff.net>
 <0c06ae6b-e4f9-4fc5-2c37-8e66a0fa1aee@oracle.com>
 <25aeebef-52e6-0ff5-1dee-d8be91af137e@oracle.com>
 <293f8626-c40e-89a7-8c45-fea82bfad19b@samersoff.net>
Message-ID: <6ef82ae3-2ce2-1256-2a5e-28fbc9dabe13@oracle.com>

Hi Dmitry,

I tried to simulate the scenario on x64 and I found out that the VM will 
continue to execute after printing out the "error" message, but with CDS 
disabled. Can you confirm that this is also the case in your environment?

I am working on a fix now (see proposed fix in the bug report).

Thanks
- Ioi

On 1/10/20 1:40 AM, Dmitry Samersoff wrote:
> Ioi,
>
>> Dmitry, are you running with -Xshare:on? If you use -Xshare:off (which
>> is the default), when the CDS archive cannot be mapped, the JVM will
>> continue execution.
> I use default parameters for java. i.e. ever simple
>
> java -version
>
> refuses to start with the message below.
>
>> Are you having problems running with the jtreg test cases? I can think
>> of a way to fix this, so that CDS can continue to work under this
>> circumstance, but I want to understand your scenarios more.
> Typical scenario is:
>     We have AArch64 Jenkins build server with 4k pages that automatically
>
>        - builds JDK
>        - deploy it to a large server with 64k pages
>        - run set of tests
>            (regression and performance, not all of them are jtreg one)
>
> -Dmitry
>
>
>
>
> On 09.01.20 20:02, Ioi Lam wrote:
>>
>> On 1/9/20 12:47 AM, David Holmes wrote:
>>> On 9/01/2020 6:29 pm, Dmitry Samersoff wrote:
>>>> Hello Ioi,
>>>>
>>>> *Story*
>>>>
>>>> If you build JDK-14 with the changes for
>>>>
>>>>  ?? JDK-8231610 Relocate the CDS archive if it cannot be mapped
>>>>  ?? to the requested address
>>>>
>>>> on a machine with 4k pages (typical Jenkins machine) then move the
>>>> bundle to a huge server with 64k pages, it refuses to start with
>>>>
>>>> [0.015s][error][cds] Unable to map CDS archive --
>>>> os::vm_allocation_granularity() expected: 4096 actual: 65536
>>>>
>>>> *Question*
>>>>
>>>> I'm not sure if just removing an assert is the correct fix. What would
>>>> you recommend?
>>> I thought that failing to map the archive was a non-fatal error that
>>> just results in falling back to not using CDS. Even in a debug build
>>> this should not be an assertion unless perhaps the user has explicitly
>>> requested CDS to be enabled.
>>>
>> Dmitry, are you running with -Xshare:on? If you use -Xshare:off (which
>> is the default), when the CDS archive cannot be mapped, the JVM will
>> continue execution.
>>
>> Are you having problems running with the jtreg test cases? I can think
>> of a way to fix this, so that CDS can continue to work under this
>> circumstance, but I want to understand your scenarios more.
>>
>> Thanks
>> - Ioi
>>
>>>> Also @build-dev - is it possible to add configure option to disable
>>>> AppCDS archive generation?
>>> Already there I think: --disable-cds-archive
>>>
>>> Cheers,
>>> David
>>> -----
>>>
>>>
>>>> -Dmitry\S
>>>>
>>>>
>>>>



From sgehwolf at redhat.com  Mon Jan 13 08:42:06 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Mon, 13 Jan 2020 09:42:06 +0100
Subject: RFR: 8236921: Add build target to produce a JDK image suitable
 for a Graal/SVM build
In-Reply-To: <fd1e8082-4527-74ef-2304-d84911af8e58@oracle.com>
References: <4b34adfde790ee699e3cdf0ee113b835139320f1.camel@redhat.com>
 <40C2C62E-9612-431D-BF87-F72C442DAC07@oracle.com>
 <fd1e8082-4527-74ef-2304-d84911af8e58@oracle.com>
Message-ID: <9d220fd3c3ca1c3eccf57476cc6b0d6a11fc3301.camel@redhat.com>

On Fri, 2020-01-10 at 13:19 -0800, Erik Joelsson wrote:
> Hello Severin,
> 
> Looks good.

Thanks for the review, Erik!

> /Erik
> 
> On 2020-01-10 09:28, Bob Vandette wrote:
> > I like it!
> > 
> > I have to manually copy the libs each time.
> > 
> > BTW:  I have a PR in our internal queue to update the graal repo so
> > it can build with the latest JDK 14.

I'd love to see that patch pushed to graal master! :)

Thanks,
Severin

> > Bob.
> > 
> > 
> > > On Jan 10, 2020, at 10:45 AM, Severin Gehwolf <
> > > sgehwolf at redhat.com> wrote:
> > > 
> > > Hi,
> > > 
> > > Currently there is no easy way to produce an OpenJDK build which
> > > could
> > > subsequently be used to build Substrate VM (part of Graal CE).
> > > Basic
> > > building blocks are there, but no actual JDK image is getting
> > > produced
> > > with the static libs along side their dynamic counterparts. This
> > > patch
> > > addresses this by adding a new target called 'graal-builder-
> > > image'
> > > which produces a jdk image in $IMAGES/graal-builder-jdk. It
> > > basically
> > > copies over libs from static-libs folder and the jdk folder to
> > > assemble
> > > the desired result.
> > > 
> > > Bug: https://bugs.openjdk.java.net/browse/JDK-8236921
> > > webrev: 
> > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8236921/02/webrev/
> > > 
> > > I've tested this works for some snapshot of JDK 14 (rev
> > > 3009b27660be)
> > > and graal at revision d9fb0b7dc35a9a7eb559a5e468bcc4c99e918449.
> > > 
> > > Thoughts?
> > > 
> > > Thanks,
> > > Severin
> > > 



From bob.vandette at oracle.com  Mon Jan 13 17:46:41 2020
From: bob.vandette at oracle.com (Bob Vandette)
Date: Mon, 13 Jan 2020 12:46:41 -0500
Subject: RFR: 8236921: Add build target to produce a JDK image suitable
 for a Graal/SVM build
In-Reply-To: <9d220fd3c3ca1c3eccf57476cc6b0d6a11fc3301.camel@redhat.com>
References: <4b34adfde790ee699e3cdf0ee113b835139320f1.camel@redhat.com>
 <40C2C62E-9612-431D-BF87-F72C442DAC07@oracle.com>
 <fd1e8082-4527-74ef-2304-d84911af8e58@oracle.com>
 <9d220fd3c3ca1c3eccf57476cc6b0d6a11fc3301.camel@redhat.com>
Message-ID: <CEFFD4FE-CC39-422A-A2F1-98C8F364163A@oracle.com>



> On Jan 13, 2020, at 3:42 AM, Severin Gehwolf <sgehwolf at redhat.com> wrote:
> 
> On Fri, 2020-01-10 at 13:19 -0800, Erik Joelsson wrote:
>> Hello Severin,
>> 
>> Looks good.
> 
> Thanks for the review, Erik!
> 
>> /Erik
>> 
>> On 2020-01-10 09:28, Bob Vandette wrote:
>>> I like it!
>>> 
>>> I have to manually copy the libs each time.
>>> 
>>> BTW:  I have a PR in our internal queue to update the graal repo so
>>> it can build with the latest JDK 14.
> 
> I'd love to see that patch pushed to graal master! :)

My changes have been integrated into the internal graal repo.  

[GR-20168] Add new JDK14 methods and bump JDK version support to JDK 15. MERGED 

Hopefully it will be in the github mirror soon.  I?m not sure how long that takes.

Bob.


> 
> Thanks,
> Severin
> 
>>> Bob.
>>> 
>>> 
>>>> On Jan 10, 2020, at 10:45 AM, Severin Gehwolf <
>>>> sgehwolf at redhat.com> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> Currently there is no easy way to produce an OpenJDK build which
>>>> could
>>>> subsequently be used to build Substrate VM (part of Graal CE).
>>>> Basic
>>>> building blocks are there, but no actual JDK image is getting
>>>> produced
>>>> with the static libs along side their dynamic counterparts. This
>>>> patch
>>>> addresses this by adding a new target called 'graal-builder-
>>>> image'
>>>> which produces a jdk image in $IMAGES/graal-builder-jdk. It
>>>> basically
>>>> copies over libs from static-libs folder and the jdk folder to
>>>> assemble
>>>> the desired result.
>>>> 
>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8236921
>>>> webrev: 
>>>> http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8236921/02/webrev/
>>>> 
>>>> I've tested this works for some snapshot of JDK 14 (rev
>>>> 3009b27660be)
>>>> and graal at revision d9fb0b7dc35a9a7eb559a5e468bcc4c99e918449.
>>>> 
>>>> Thoughts?
>>>> 
>>>> Thanks,
>>>> Severin
>>>> 
> 



From alexandre.iline at oracle.com  Mon Jan 13 18:48:12 2020
From: alexandre.iline at oracle.com (Alexandre (Shura) Iline)
Date: Mon, 13 Jan 2020 10:48:12 -0800
Subject: RFR JDK-8237042: Switch to JCov build which supports byte code
 version 59
Message-ID: <B6A99599-E081-4B65-ACFA-170E9225927D@oracle.com>

Hi.

Can you please review this simple change:
---------------------------------------------------------------
$ hg diff
diff -r f7edb9ca045c make/conf/jib-profiles.js
--- a/make/conf/jib-profiles.js Fri Jan 10 15:38:25 2020 +0100
+++ b/make/conf/jib-profiles.js Mon Jan 13 10:42:27 2020 -0800
@@ -1066,7 +1066,7 @@
             // build_number: "b07",
             // file: "bundles/jcov-3_0.zip",
             organization: common.organization,
-            revision: "3.0-58-support+1.0",
+            revision: "3.0-59-support+1.0",
             ext: "zip",
             environment_name: "JCOV_HOME",
         },
---------------------------------------------------------------


Thank you.

Shura

From erik.joelsson at oracle.com  Mon Jan 13 19:27:21 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Mon, 13 Jan 2020 11:27:21 -0800
Subject: RFR JDK-8237042: Switch to JCov build which supports byte code
 version 59
In-Reply-To: <B6A99599-E081-4B65-ACFA-170E9225927D@oracle.com>
References: <B6A99599-E081-4B65-ACFA-170E9225927D@oracle.com>
Message-ID: <aec6d2f8-7725-1927-91c7-77b41f1a2ed0@oracle.com>

Looks good.

/Erik

On 2020-01-13 10:48, Alexandre (Shura) Iline wrote:
> Hi.
>
> Can you please review this simple change:
> ---------------------------------------------------------------
> $ hg diff
> diff -r f7edb9ca045c make/conf/jib-profiles.js
> --- a/make/conf/jib-profiles.js Fri Jan 10 15:38:25 2020 +0100
> +++ b/make/conf/jib-profiles.js Mon Jan 13 10:42:27 2020 -0800
> @@ -1066,7 +1066,7 @@
>               // build_number: "b07",
>               // file: "bundles/jcov-3_0.zip",
>               organization: common.organization,
> -            revision: "3.0-58-support+1.0",
> +            revision: "3.0-59-support+1.0",
>               ext: "zip",
>               environment_name: "JCOV_HOME",
>           },
> ---------------------------------------------------------------
>
>
> Thank you.
>
> Shura


From magnus.ihse.bursie at oracle.com  Tue Jan 14 11:27:52 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 14 Jan 2020 12:27:52 +0100
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>

On 2020-01-10 11:01, Baesken, Matthias wrote:
> Hello,   I recently looked into  the  gcc  lto  optimization mode (see for some details https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html  and  http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html  ).
> This mode can lead to more compact binaries (~10% smaller)  , it also might bring  small performance improvements  but that wasn't my (main)  goal  .
>
> The changes for this are rather small , one needs to use a recent gcc  , add  -flto   to the compile flags  , for example
>
> --- a/make/autoconf/flags-cflags.m4      Wed Jan 01 03:08:45 2020 +0100
> +++ b/make/autoconf/flags-cflags.m4   Wed Jan 08 17:39:10 2020 +0100
> @@ -530,8 +530,13 @@
>     fi
>     if test "x$TOOLCHAIN_TYPE" = xgcc; then
> -    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector"
> -    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
> +    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector -flto"
> +    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"
>
>    .... and you have to make sure  to use  gcc-ar  and  gcc-nm instead   of  ar / nm .
> Build and test(s)  work,  however with  one exception.
> The  serviceability   tests like  serviceability/sa   seems to rely   heavily  on the "normal"   structure  of   libjvm.so   (from what I   understand  e.g. in  LinuxVtblAccess  it is attempted to access  internal symbols  like  _ZTV ).
>
> Errors in the sa  tests look like :
>
>
> java.lang.InternalError: Metadata does not appear to be polymorphic
>           at jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)
>           at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
>           at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)
>           at jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
>           at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
>           at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)
>           at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>
> Has anyone experimented with LTO optimization ?

Hi Matthias,

We used to have LTO enabled on the old, closed-source Oracle arm-32 
builds. There is still a "link-time-opt" JVM feature present; afaik it 
still works and adds the -flto flag. The main drawback of this is the 
*extremely* long link times of libjvm.so.

I don't think servicability was ever supported for that platform, so I'm 
not surprised this does not work.

/Magnus

>
> And to the  serviceability   agent experts -  any idea  how to make the  jdk.hotspot.agent   more independent from  optimization settings ?
>
>
> Best regards, Matthias



From matthias.baesken at sap.com  Tue Jan 14 12:49:43 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 14 Jan 2020 12:49:43 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
Message-ID: <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Magnus, thanks for the info , I already noticed yesterday the setting for arm-32 in the minimal build .
Do you think  we could set it too for the other Linux platforms  in the minimal build  ( serviceability agent is not supported there as well so the  observed issue wouldn?t be a problem).

Best regards, Matthias




On 2020-01-10 11:01, Baesken, Matthias wrote:

Hello,   I recently looked into  the  gcc  lto  optimization mode (see for some details https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html  and  http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html  ).

This mode can lead to more compact binaries (~10% smaller)  , it also might bring  small performance improvements  but that wasn't my (main)  goal  .



The changes for this are rather small , one needs to use a recent gcc  , add  -flto   to the compile flags  , for example



--- a/make/autoconf/flags-cflags.m4      Wed Jan 01 03:08:45 2020 +0100

+++ b/make/autoconf/flags-cflags.m4   Wed Jan 08 17:39:10 2020 +0100

@@ -530,8 +530,13 @@

   fi

   if test "x$TOOLCHAIN_TYPE" = xgcc; then

-    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector"

-    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"

+    TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector -flto"

+    TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"



  .... and you have to make sure  to use  gcc-ar  and  gcc-nm instead   of  ar / nm .

Build and test(s)  work,  however with  one exception.

The  serviceability   tests like  serviceability/sa   seems to rely   heavily  on the "normal"   structure  of   libjvm.so   (from what I   understand  e.g. in  LinuxVtblAccess  it is attempted to access  internal symbols  like  _ZTV ).



Errors in the sa  tests look like :





java.lang.InternalError: Metadata does not appear to be polymorphic

         at jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)

         at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)

         at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)

         at jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)

         at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)

         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)

         at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)



Has anyone experimented with LTO optimization ?

Hi Matthias,

We used to have LTO enabled on the old, closed-source Oracle arm-32 builds. There is still a "link-time-opt" JVM feature present; afaik it still works and adds the -flto flag. The main drawback of this is the *extremely* long link times of libjvm.so.

I don't think servicability was ever supported for that platform, so I'm not surprised this does not work.

/Magnus







And to the  serviceability   agent experts -  any idea  how to make the  jdk.hotspot.agent   more independent from  optimization settings ?





Best regards, Matthias


From magnus.ihse.bursie at oracle.com  Tue Jan 14 14:04:11 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 14 Jan 2020 15:04:11 +0100
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>

On 2020-01-14 13:49, Baesken, Matthias wrote:
>
> Hi Magnus, thanks for the info , I already noticed yesterday the 
> setting for arm-32 in the minimal build .
>
> Do you think? we could set it too for the other Linux platforms? in 
> the minimal build? ( serviceability agent is not supported there as 
> well so the? observed issue wouldn?t be a problem).
>

You mean if you could enable it on your builds without any issues? I'd 
guess so, but I don't know. Just try it: 
--with-jvm-features="link-time-opt".

If you mean that it should be turned on by default on minimal builds for 
all platforms? No, I don't think that's a good idea. The link time is 
really a killer. I remember arm-32 going from like a couple of minutes 
to half an hour for linking libjvm.so.

Things might be different with gold, though. I know they have done work 
with at least some kind of "lightweight" LTO, that might be worth at 
least looking into.

/Magnus


> Best regards, Matthias
>
> On 2020-01-10 11:01, Baesken, Matthias wrote:
>
>     Hello,?? I recently looked into? the? gcc? lto? optimization mode (see for some detailshttps://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html ? andhttp://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html ? ).
>
>     This mode can lead to more compact binaries (~10% smaller)? , it also might bring? small performance improvements? but that wasn't my (main)? goal? .
>
>     The changes for this are rather small , one needs to use a recent gcc? , add? -flto?? to the compile flags? , for example
>
>     --- a/make/autoconf/flags-cflags.m4????? Wed Jan 01 03:08:45 2020 +0100
>
>     +++ b/make/autoconf/flags-cflags.m4?? Wed Jan 08 17:39:10 2020 +0100
>
>     @@ -530,8 +530,13 @@
>
>      ?? fi
>
>      ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then
>
>     -??? TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector"
>
>     -??? TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
>
>     +??? TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new -fstack-protector -flto"
>
>     +??? TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"
>
>      ? .... and you have to make sure? to use? gcc-ar? and? gcc-nm instead?? of? ar / nm .
>
>     Build and test(s)? work,? however with? one exception.
>
>     The? serviceability?? tests like? serviceability/sa?? seems to rely?? heavily? on the "normal"?? structure? of?? libjvm.so?? (from what I?? understand? e.g. in? LinuxVtblAccess? it is attempted to access? internal symbols? like? _ZTV ).
>
>     Errors in the sa? tests look like :
>
>     java.lang.InternalError: Metadata does not appear to be polymorphic
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
>
>      ???? ????at jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
>
>      ??? ?????at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)
>
>      ???????? at jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>
>     Has anyone experimented with LTO optimization ?
>
>
> Hi Matthias,
>
> We used to have LTO enabled on the old, closed-source Oracle arm-32 
> builds. There is still a "link-time-opt" JVM feature present; afaik it 
> still works and adds the -flto flag. The main drawback of this is the 
> *extremely* long link times of libjvm.so.
>
> I don't think servicability was ever supported for that platform, so 
> I'm not surprised this does not work.
>
> /Magnus
>
>
>     And to the? serviceability?? agent experts -? any idea? how to make the? jdk.hotspot.agent?? more independent from? optimization settings ?
>
>     Best regards, Matthias
>



From matthias.baesken at sap.com  Tue Jan 14 14:07:16 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 14 Jan 2020 14:07:16 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove unused
 code
Message-ID: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello,  the following change enables the  link-time section-gc for linux .

gcc and ld support enabling "garbage collection" of unused input sections.
This can be used to eliminate unused coding from native libraries (especially when already compiling the objects with compiler flags -ffunction-sections -fdata-sections .
See for details the --gc-sections and --print-gc-sections parts of the ld documentation :

https://linux.die.net/man/1/ld


We had this enabled already  for  linux s390x ,  with https://bugs.openjdk.java.net/browse/JDK-8234525
8234525: enable link-time section-gc for linux s390x to remove unused code .

This  time we enable it too for  the other linux platforms .

For the other platforms I do not enable it for JVM, just for the JDK libs.  The reason is that the serviceability agent  (not supported on linux s390x )    is not  (yet) ready for the optimization .
Below you see the results , for some libraries  a significant  size reduction can be achieved .


Results from linux x86_64 product builds :

without / with ltgc

320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
36K  / 36K     /images/jdk/lib/libdt_socket.so
280K / 276K   /images/jdk/lib/libjdwp.so
23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so for x86_64
16K  / 16K    /images/jdk/lib/server/libjsig.so
72K  / 72M    /images/jdk/lib/libverify.so
84K  / 84M   /images/jdk/lib/libjli.so
16K  / 16K    /images/jdk/lib/libjsig.so
196K / 196K   /images/jdk/lib/libjava.so
44K  / 44K    /images/jdk/lib/libzip.so
144K / 136K   /images/jdk/lib/libjimage.so
112K / 112K   /images/jdk/lib/libnet.so
100K / 100K   /images/jdk/lib/libnio.so
36K  / 36K    /images/jdk/lib/libsctp.so
576K / 556K   /images/jdk/lib/libmlib_image.so
752K / 752K   /images/jdk/lib/libawt.so
260K / 252K   /images/jdk/lib/libjavajpeg.so
784K / 784K   /images/jdk/lib/libfreetype.so
368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
88K / 88K    /images/jdk/lib/libjsound.so
472K / 468K    /images/jdk/lib/libawt_xawt.so
564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
48K / 48K    /images/jdk/lib/libawt_headless.so
12K / 12K    /images/jdk/lib/libjawt.so
1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
12K / 12K    /images/jdk/lib/libjaas.so
92K / 92K    /images/jdk/lib/libj2pkcs11.so
16K / 16K    /images/jdk/lib/libattach.so
8.0K / 8.0K   /images/jdk/lib/librmi.so
56K / 56K    /images/jdk/lib/libinstrument.so
16K / 16K    /images/jdk/lib/libprefs.so
52K / 52K    /images/jdk/lib/libj2gss.so
12K / 12K    /images/jdk/lib/libmanagement_agent.so
36K / 32K    /images/jdk/lib/libmanagement.so
16K / 16K    /images/jdk/lib/libextnet.so
20K / 20K    /images/jdk/lib/libj2pcsc.so
40K / 40K    /images/jdk/lib/libmanagement_ext.so
60K / 60K    /images/jdk/lib/libsaproc.so


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8236714

http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/


Thanks, Matthias


From aleksei.voitylov at bell-sw.com  Tue Jan 14 15:15:14 2020
From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov)
Date: Tue, 14 Jan 2020 18:15:14 +0300
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
Message-ID: <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>

Magnus, Matthias,

for me, lto is a little heavyweight for development. x86_64 build time
with gcc 7:

Server 1m32.484s
Server+Minimal 1m42.166s
Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s

If the change to enable lto by default is proposed, what would be the
recommended strategy for development?

For ARM32 Minimal, please keep in mind that it's not uncommon to disable
LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
does not matter what settings we have in OpenJDK. I believe there could
be other reasons for that on top of build time (bugs?).

-Aleksei

On 14/01/2020 17:04, Magnus Ihse Bursie wrote:
> On 2020-01-14 13:49, Baesken, Matthias wrote:
>>
>> Hi Magnus, thanks for the info , I already noticed yesterday the
>> setting for arm-32 in the minimal build .
>>
>> Do you think? we could set it too for the other Linux platforms? in
>> the minimal build? ( serviceability agent is not supported there as
>> well so the? observed issue wouldn?t be a problem).
>>
>
> You mean if you could enable it on your builds without any issues? I'd
> guess so, but I don't know. Just try it:
> --with-jvm-features="link-time-opt".
>
> If you mean that it should be turned on by default on minimal builds
> for all platforms? No, I don't think that's a good idea. The link time
> is really a killer. I remember arm-32 going from like a couple of
> minutes to half an hour for linking libjvm.so.
>
> Things might be different with gold, though. I know they have done
> work with at least some kind of "lightweight" LTO, that might be worth
> at least looking into.
>
> /Magnus
>
>
>> Best regards, Matthias
>>
>> On 2020-01-10 11:01, Baesken, Matthias wrote:
>>
>> ??? Hello,?? I recently looked into? the? gcc? lto? optimization mode
>> (see for some
>> detailshttps://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html ?
>> andhttp://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html
>> ? ).
>>
>> ??? This mode can lead to more compact binaries (~10% smaller)? , it
>> also might bring? small performance improvements? but that wasn't my
>> (main)? goal? .
>>
>> ??? The changes for this are rather small , one needs to use a recent
>> gcc? , add? -flto?? to the compile flags? , for example
>>
>> ??? --- a/make/autoconf/flags-cflags.m4????? Wed Jan 01 03:08:45 2020
>> +0100
>>
>> ??? +++ b/make/autoconf/flags-cflags.m4?? Wed Jan 08 17:39:10 2020 +0100
>>
>> ??? @@ -530,8 +530,13 @@
>>
>> ???? ?? fi
>>
>> ???? ?? if test "x$TOOLCHAIN_TYPE" = xgcc; then
>>
>> ??? -??? TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
>> -fstack-protector"
>>
>> ??? -??? TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector"
>>
>> ??? +??? TOOLCHAIN_CFLAGS_JVM="$TOOLCHAIN_CFLAGS_JVM -fcheck-new
>> -fstack-protector -flto"
>>
>> ??? +??? TOOLCHAIN_CFLAGS_JDK="-pipe -fstack-protector -flto"
>>
>> ???? ? .... and you have to make sure? to use? gcc-ar? and? gcc-nm
>> instead?? of? ar / nm .
>>
>> ??? Build and test(s)? work,? however with? one exception.
>>
>> ??? The? serviceability?? tests like? serviceability/sa?? seems to
>> rely?? heavily? on the "normal"?? structure? of?? libjvm.so?? (from
>> what I?? understand? e.g. in? LinuxVtblAccess? it is attempted to
>> access? internal symbols? like? _ZTV ).
>>
>> ??? Errors in the sa? tests look like :
>>
>> ??? java.lang.InternalError: Metadata does not appear to be polymorphic
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.types.basic.BasicTypeDataBase.findDynamicTypeForAddress(BasicTypeDataBase.java:279)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.runtime.VirtualBaseConstructor.instantiateWrapperFor(VirtualBaseConstructor.java:102)
>>
>> ???? ???? ????at
>> jdk.hotspot.agent/sun.jvm.hotspot.oops.Metadata.instantiateWrapperFor(Metadata.java:74)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.memory.SystemDictionary.getClassLoaderKlass(SystemDictionary.java:96)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.printClassLoaderStatistics(ClassLoaderStats.java:93)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.ClassLoaderStats.run(ClassLoaderStats.java:78)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.run(JMap.java:115)
>>
>> ???? ??? ?????at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.startInternal(Tool.java:262)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.start(Tool.java:225)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.tools.JMap.main(JMap.java:176)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.runJMAP(SALauncher.java:321)
>>
>> ???? ???????? at
>> jdk.hotspot.agent/sun.jvm.hotspot.SALauncher.main(SALauncher.java:406)
>>
>> ??? Has anyone experimented with LTO optimization ?
>>
>>
>> Hi Matthias,
>>
>> We used to have LTO enabled on the old, closed-source Oracle arm-32
>> builds. There is still a "link-time-opt" JVM feature present; afaik
>> it still works and adds the -flto flag. The main drawback of this is
>> the *extremely* long link times of libjvm.so.
>>
>> I don't think servicability was ever supported for that platform, so
>> I'm not surprised this does not work.
>>
>> /Magnus
>>
>>
>> ??? And to the? serviceability?? agent experts -? any idea? how to
>> make the? jdk.hotspot.agent?? more independent from? optimization
>> settings ?
>>
>> ??? Best regards, Matthias
>>
>

From erik.joelsson at oracle.com  Tue Jan 14 16:12:24 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Tue, 14 Jan 2020 08:12:24 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>

(adding core-libs-dev)

Change looks good to me, but would like input from at least someone in 
core-libs.

/Erik

On 2020-01-14 06:07, Baesken, Matthias wrote:
> Hello,  the following change enables the  link-time section-gc for linux .
>
> gcc and ld support enabling "garbage collection" of unused input sections.
> This can be used to eliminate unused coding from native libraries (especially when already compiling the objects with compiler flags -ffunction-sections -fdata-sections .
> See for details the --gc-sections and --print-gc-sections parts of the ld documentation :
>
> https://linux.die.net/man/1/ld
>
>
> We had this enabled already  for  linux s390x ,  with https://bugs.openjdk.java.net/browse/JDK-8234525
> 8234525: enable link-time section-gc for linux s390x to remove unused code .
>
> This  time we enable it too for  the other linux platforms .
>
> For the other platforms I do not enable it for JVM, just for the JDK libs.  The reason is that the serviceability agent  (not supported on linux s390x )    is not  (yet) ready for the optimization .
> Below you see the results , for some libraries  a significant  size reduction can be achieved .
>
>
> Results from linux x86_64 product builds :
>
> without / with ltgc
>
> 320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
> 36K  / 36K     /images/jdk/lib/libdt_socket.so
> 280K / 276K   /images/jdk/lib/libjdwp.so
> 23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so for x86_64
> 16K  / 16K    /images/jdk/lib/server/libjsig.so
> 72K  / 72M    /images/jdk/lib/libverify.so
> 84K  / 84M   /images/jdk/lib/libjli.so
> 16K  / 16K    /images/jdk/lib/libjsig.so
> 196K / 196K   /images/jdk/lib/libjava.so
> 44K  / 44K    /images/jdk/lib/libzip.so
> 144K / 136K   /images/jdk/lib/libjimage.so
> 112K / 112K   /images/jdk/lib/libnet.so
> 100K / 100K   /images/jdk/lib/libnio.so
> 36K  / 36K    /images/jdk/lib/libsctp.so
> 576K / 556K   /images/jdk/lib/libmlib_image.so
> 752K / 752K   /images/jdk/lib/libawt.so
> 260K / 252K   /images/jdk/lib/libjavajpeg.so
> 784K / 784K   /images/jdk/lib/libfreetype.so
> 368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
> 88K / 88K    /images/jdk/lib/libjsound.so
> 472K / 468K    /images/jdk/lib/libawt_xawt.so
> 564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
> 48K / 48K    /images/jdk/lib/libawt_headless.so
> 12K / 12K    /images/jdk/lib/libjawt.so
> 1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
> 12K / 12K    /images/jdk/lib/libjaas.so
> 92K / 92K    /images/jdk/lib/libj2pkcs11.so
> 16K / 16K    /images/jdk/lib/libattach.so
> 8.0K / 8.0K   /images/jdk/lib/librmi.so
> 56K / 56K    /images/jdk/lib/libinstrument.so
> 16K / 16K    /images/jdk/lib/libprefs.so
> 52K / 52K    /images/jdk/lib/libj2gss.so
> 12K / 12K    /images/jdk/lib/libmanagement_agent.so
> 36K / 32K    /images/jdk/lib/libmanagement.so
> 16K / 16K    /images/jdk/lib/libextnet.so
> 20K / 20K    /images/jdk/lib/libj2pcsc.so
> 40K / 40K    /images/jdk/lib/libmanagement_ext.so
> 60K / 60K    /images/jdk/lib/libsaproc.so
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8236714
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/
>
>
> Thanks, Matthias


From matthias.baesken at sap.com  Tue Jan 14 16:57:33 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 14 Jan 2020 16:57:33 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
Message-ID: <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello  Magnus and Aleksei,  thanks for the input .

The times you  provided really look like they make a big difference  at least for people  often  building   minimal-vm  .
Guess I have to measure myself a bit  (maybe the difference is not that big on our linux s390x / ppc64(le) ) .

>
> If the change to enable lto by default is proposed, what would be the
> recommended strategy for development?
>

Probably  we should a)   do not enable it by default but just make sure it can be enabled easily and works  for  the minimal-vm     or  b)  take it easy to disable it for local development.

Best regards, Matthias



> 
> Magnus, Matthias,
> 
> for me, lto is a little heavyweight for development. x86_64 build time
> with gcc 7:
> 
> Server 1m32.484s
> Server+Minimal 1m42.166s
> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
> 
> If the change to enable lto by default is proposed, what would be the
> recommended strategy for development?
> 
> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
> does not matter what settings we have in OpenJDK. I believe there could
> be other reasons for that on top of build time (bugs?).
> 


From aleksei.voitylov at bell-sw.com  Tue Jan 14 17:54:36 2020
From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov)
Date: Tue, 14 Jan 2020 20:54:36 +0300
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>


On 14/01/2020 19:57, Baesken, Matthias wrote:
> Hello  Magnus and Aleksei,  thanks for the input .
>
> The times you  provided really look like they make a big difference  at least for people  often  building   minimal-vm  .
> Guess I have to measure myself a bit  (maybe the difference is not that big on our linux s390x / ppc64(le) ) .
>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
> Probably  we should a)   do not enable it by default but just make sure it can be enabled easily and works  for  the minimal-vm   
That would be welcome. I have high hopes to LTO the VM some time by
default, and the tendency observed is that the compiler time overhead
for GCC becomes smaller. At the same time there is no reason why vendors
that invested in testing and can absorb the build time hit could provide
binaries with LTO built VMs by passing an additional option flag.
>   or  b)  take it easy to disable it for local development.
>
> Best regards, Matthias
>
>
>
>> Magnus, Matthias,
>>
>> for me, lto is a little heavyweight for development. x86_64 build time
>> with gcc 7:
>>
>> Server 1m32.484s
>> Server+Minimal 1m42.166s
>> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
>>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
>> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
>> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
>> does not matter what settings we have in OpenJDK. I believe there could
>> be other reasons for that on top of build time (bugs?).
>>



From jonathan.gibbons at oracle.com  Tue Jan 14 21:27:52 2020
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Tue, 14 Jan 2020 13:27:52 -0800
Subject: RFR: docs JDK-8237058 Update all nroff manpages for JDK 14 release
Message-ID: <0436ad70-bebb-0b26-dab3-c181b31b513f@oracle.com>

Please review a change to refresh the content of the man pages (*.1 
files) in the main repo,
to contain essentially the exact same content as found in the Oracle JDK 
man pages.

This email is to build-dev at ojn, even though there are no Makefile 
changes, because that
list includes the folk mostly likely to make direct use of these files.? 
It is also cc jdk-dev at ojn
so that the OpenJDK community is generally aware of this change.

For many/most files, the change is just to update the JDK version number 
in the header.

-- Jon

JBS: https://bugs.openjdk.java.net/browse/JDK-8237058
Webrev: http://cr.openjdk.java.net/~jjg/8237058/webrev/



From david.holmes at oracle.com  Tue Jan 14 21:57:10 2020
From: david.holmes at oracle.com (David Holmes)
Date: Wed, 15 Jan 2020 07:57:10 +1000
Subject: RFR: docs JDK-8237058 Update all nroff manpages for JDK 14 release
In-Reply-To: <0436ad70-bebb-0b26-dab3-c181b31b513f@oracle.com>
References: <0436ad70-bebb-0b26-dab3-c181b31b513f@oracle.com>
Message-ID: <b592cc61-13c4-50df-f809-ef3dec114952@oracle.com>

Thanks for doing this update Jon.

The changes seem fine to me.

David

On 15/01/2020 7:27 am, Jonathan Gibbons wrote:
> Please review a change to refresh the content of the man pages (*.1 
> files) in the main repo,
> to contain essentially the exact same content as found in the Oracle JDK 
> man pages.
> 
> This email is to build-dev at ojn, even though there are no Makefile 
> changes, because that
> list includes the folk mostly likely to make direct use of these files. 
> It is also cc jdk-dev at ojn
> so that the OpenJDK community is generally aware of this change.
> 
> For many/most files, the change is just to update the JDK version number 
> in the header.
> 
> -- Jon
> 
> JBS: https://bugs.openjdk.java.net/browse/JDK-8237058
> Webrev: http://cr.openjdk.java.net/~jjg/8237058/webrev/
> 


From matthias.baesken at sap.com  Wed Jan 15 08:27:09 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 08:27:09 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>
Message-ID: <AM6PR02MB420099D1C75E508D1BE1CD8D93370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, thanks for the review and for forwarding ,  you are correct  corelibs-dev is probably  interested in this as well .

Best regards, Matthias


> (adding core-libs-dev)
> 
> Change looks good to me, but would like input from at least someone in
> core-libs.
> 
> /Erik
> 
> On 2020-01-14 06:07, Baesken, Matthias wrote:
> > Hello,  the following change enables the  link-time section-gc for linux .
> >
> > gcc and ld support enabling "garbage collection" of unused input sections.
> > This can be used to eliminate unused coding from native libraries
> (especially when already compiling the objects with compiler flags -ffunction-
> sections -fdata-sections .
> > See for details the --gc-sections and --print-gc-sections parts of the ld
> documentation :
> >
> > https://linux.die.net/man/1/ld
> >
> >
> > We had this enabled already  for  linux s390x ,  with
> https://bugs.openjdk.java.net/browse/JDK-8234525
> > 8234525: enable link-time section-gc for linux s390x to remove unused code
> .
> >
> > This  time we enable it too for  the other linux platforms .
> >
> > For the other platforms I do not enable it for JVM, just for the JDK libs.  The
> reason is that the serviceability agent  (not supported on linux s390x )    is not
> (yet) ready for the optimization .
> > Below you see the results , for some libraries  a significant  size reduction
> can be achieved .
> >
> >
> > Results from linux x86_64 product builds :
> >
> > without / with ltgc
> >
> > 320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
> > 36K  / 36K     /images/jdk/lib/libdt_socket.so
> > 280K / 276K   /images/jdk/lib/libjdwp.so
> > 23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so
> for x86_64
> > 16K  / 16K    /images/jdk/lib/server/libjsig.so
> > 72K  / 72M    /images/jdk/lib/libverify.so
> > 84K  / 84M   /images/jdk/lib/libjli.so
> > 16K  / 16K    /images/jdk/lib/libjsig.so
> > 196K / 196K   /images/jdk/lib/libjava.so
> > 44K  / 44K    /images/jdk/lib/libzip.so
> > 144K / 136K   /images/jdk/lib/libjimage.so
> > 112K / 112K   /images/jdk/lib/libnet.so
> > 100K / 100K   /images/jdk/lib/libnio.so
> > 36K  / 36K    /images/jdk/lib/libsctp.so
> > 576K / 556K   /images/jdk/lib/libmlib_image.so
> > 752K / 752K   /images/jdk/lib/libawt.so
> > 260K / 252K   /images/jdk/lib/libjavajpeg.so
> > 784K / 784K   /images/jdk/lib/libfreetype.so
> > 368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
> > 88K / 88K    /images/jdk/lib/libjsound.so
> > 472K / 468K    /images/jdk/lib/libawt_xawt.so
> > 564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
> > 48K / 48K    /images/jdk/lib/libawt_headless.so
> > 12K / 12K    /images/jdk/lib/libjawt.so
> > 1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
> > 12K / 12K    /images/jdk/lib/libjaas.so
> > 92K / 92K    /images/jdk/lib/libj2pkcs11.so
> > 16K / 16K    /images/jdk/lib/libattach.so
> > 8.0K / 8.0K   /images/jdk/lib/librmi.so
> > 56K / 56K    /images/jdk/lib/libinstrument.so
> > 16K / 16K    /images/jdk/lib/libprefs.so
> > 52K / 52K    /images/jdk/lib/libj2gss.so
> > 12K / 12K    /images/jdk/lib/libmanagement_agent.so
> > 36K / 32K    /images/jdk/lib/libmanagement.so
> > 16K / 16K    /images/jdk/lib/libextnet.so
> > 20K / 20K    /images/jdk/lib/libj2pcsc.so
> > 40K / 40K    /images/jdk/lib/libmanagement_ext.so
> > 60K / 60K    /images/jdk/lib/libsaproc.so
> >
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8236714
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/
> >
> >
> > Thanks, Matthias

From volker.simonis at gmail.com  Wed Jan 15 13:40:02 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 15 Jan 2020 05:40:02 -0800
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
Message-ID: <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>

While we are speaking about all the drawbacks of LTO, it's still not clear
what the benefits are? In the very first mail Matthias mentioned that there
might be performance improvements but that performance is not the main
driving factor behind this initiative. So is it the reduced code size
(Matthias mentioned something around ~10%)?

It would be nice to see some real numbers on various platform for both, the
performance improvements for native parts like JIT/GC as well as for the
size reduction.

Aleksei Voitylov <aleksei.voitylov at bell-sw.com> schrieb am Di., 14. Jan.
2020, 09:54:

>
> On 14/01/2020 19:57, Baesken, Matthias wrote:
> > Hello  Magnus and Aleksei,  thanks for the input .
> >
> > The times you  provided really look like they make a big difference  at
> least for people  often  building   minimal-vm  .
> > Guess I have to measure myself a bit  (maybe the difference is not that
> big on our linux s390x / ppc64(le) ) .
> >
> >> If the change to enable lto by default is proposed, what would be the
> >> recommended strategy for development?
> >>
> > Probably  we should a)   do not enable it by default but just make sure
> it can be enabled easily and works  for  the minimal-vm
> That would be welcome. I have high hopes to LTO the VM some time by
> default, and the tendency observed is that the compiler time overhead
> for GCC becomes smaller. At the same time there is no reason why vendors
> that invested in testing and can absorb the build time hit could provide
> binaries with LTO built VMs by passing an additional option flag.
> >   or  b)  take it easy to disable it for local development.
> >
> > Best regards, Matthias
> >
> >
> >
> >> Magnus, Matthias,
> >>
> >> for me, lto is a little heavyweight for development. x86_64 build time
> >> with gcc 7:
> >>
> >> Server 1m32.484s
> >> Server+Minimal 1m42.166s
> >> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
> >>
> >> If the change to enable lto by default is proposed, what would be the
> >> recommended strategy for development?
> >>
> >> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
> >> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
> >> does not matter what settings we have in OpenJDK. I believe there could
> >> be other reasons for that on top of build time (bugs?).
> >>
>
>


From erik.joelsson at oracle.com  Wed Jan 15 14:06:59 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 15 Jan 2020 06:06:59 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB420099D1C75E508D1BE1CD8D93370@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>
 <AM6PR02MB420099D1C75E508D1BE1CD8D93370@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <998c4a08-5670-f51d-4625-9c7e984b4b5d@oracle.com>

Given the discussion regarding lto on hotspot and the extreme increased 
build time, have you noticed any difference in build times with this patch?

/Erik

On 2020-01-15 00:27, Baesken, Matthias wrote:
> Hi Erik, thanks for the review and for forwarding ,  you are correct  corelibs-dev is probably  interested in this as well .
>
> Best regards, Matthias
>
>
>> (adding core-libs-dev)
>>
>> Change looks good to me, but would like input from at least someone in
>> core-libs.
>>
>> /Erik
>>
>> On 2020-01-14 06:07, Baesken, Matthias wrote:
>>> Hello,  the following change enables the  link-time section-gc for linux .
>>>
>>> gcc and ld support enabling "garbage collection" of unused input sections.
>>> This can be used to eliminate unused coding from native libraries
>> (especially when already compiling the objects with compiler flags -ffunction-
>> sections -fdata-sections .
>>> See for details the --gc-sections and --print-gc-sections parts of the ld
>> documentation :
>>> https://linux.die.net/man/1/ld
>>>
>>>
>>> We had this enabled already  for  linux s390x ,  with
>> https://bugs.openjdk.java.net/browse/JDK-8234525
>>> 8234525: enable link-time section-gc for linux s390x to remove unused code
>> .
>>> This  time we enable it too for  the other linux platforms .
>>>
>>> For the other platforms I do not enable it for JVM, just for the JDK libs.  The
>> reason is that the serviceability agent  (not supported on linux s390x )    is not
>> (yet) ready for the optimization .
>>> Below you see the results , for some libraries  a significant  size reduction
>> can be achieved .
>>>
>>> Results from linux x86_64 product builds :
>>>
>>> without / with ltgc
>>>
>>> 320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
>>> 36K  / 36K     /images/jdk/lib/libdt_socket.so
>>> 280K / 276K   /images/jdk/lib/libjdwp.so
>>> 23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so
>> for x86_64
>>> 16K  / 16K    /images/jdk/lib/server/libjsig.so
>>> 72K  / 72M    /images/jdk/lib/libverify.so
>>> 84K  / 84M   /images/jdk/lib/libjli.so
>>> 16K  / 16K    /images/jdk/lib/libjsig.so
>>> 196K / 196K   /images/jdk/lib/libjava.so
>>> 44K  / 44K    /images/jdk/lib/libzip.so
>>> 144K / 136K   /images/jdk/lib/libjimage.so
>>> 112K / 112K   /images/jdk/lib/libnet.so
>>> 100K / 100K   /images/jdk/lib/libnio.so
>>> 36K  / 36K    /images/jdk/lib/libsctp.so
>>> 576K / 556K   /images/jdk/lib/libmlib_image.so
>>> 752K / 752K   /images/jdk/lib/libawt.so
>>> 260K / 252K   /images/jdk/lib/libjavajpeg.so
>>> 784K / 784K   /images/jdk/lib/libfreetype.so
>>> 368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
>>> 88K / 88K    /images/jdk/lib/libjsound.so
>>> 472K / 468K    /images/jdk/lib/libawt_xawt.so
>>> 564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
>>> 48K / 48K    /images/jdk/lib/libawt_headless.so
>>> 12K / 12K    /images/jdk/lib/libjawt.so
>>> 1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
>>> 12K / 12K    /images/jdk/lib/libjaas.so
>>> 92K / 92K    /images/jdk/lib/libj2pkcs11.so
>>> 16K / 16K    /images/jdk/lib/libattach.so
>>> 8.0K / 8.0K   /images/jdk/lib/librmi.so
>>> 56K / 56K    /images/jdk/lib/libinstrument.so
>>> 16K / 16K    /images/jdk/lib/libprefs.so
>>> 52K / 52K    /images/jdk/lib/libj2gss.so
>>> 12K / 12K    /images/jdk/lib/libmanagement_agent.so
>>> 36K / 32K    /images/jdk/lib/libmanagement.so
>>> 16K / 16K    /images/jdk/lib/libextnet.so
>>> 20K / 20K    /images/jdk/lib/libj2pcsc.so
>>> 40K / 40K    /images/jdk/lib/libmanagement_ext.so
>>> 60K / 60K    /images/jdk/lib/libsaproc.so
>>>
>>>
>>> Bug/webrev :
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8236714
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/
>>>
>>>
>>> Thanks, Matthias


From aleksei.voitylov at bell-sw.com  Wed Jan 15 14:57:11 2020
From: aleksei.voitylov at bell-sw.com (Aleksei Voitylov)
Date: Wed, 15 Jan 2020 17:57:11 +0300
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
Message-ID: <58b2c73c-c49e-49cf-71fc-7d6c2225b880@bell-sw.com>

Volker,

not a full answer, but here is some static size stats:

Server ??? x86_64? AArch64
regular ??? 23M ?? ?? 20M
lto ??? ??? ?? 17M ? ??? 14M

Minimal?? x86_64? AArch64
regular???? 4.9M????? 3.9M
lto??????????? 4.7M????? 3.6M

-Aleksei

On 15/01/2020 16:40, Volker Simonis wrote:
> While we are speaking about all the drawbacks of LTO, it's still not
> clear what the benefits are? In the very first mail Matthias mentioned
> that there might be performance improvements but that performance is
> not the main driving factor behind this initiative. So is it the
> reduced code size (Matthias mentioned something around ~10%)?
>
> It would be nice to see some real numbers on various platform for
> both, the performance improvements for native parts like JIT/GC as
> well as for the size reduction.
>
> Aleksei Voitylov <aleksei.voitylov at bell-sw.com
> <mailto:aleksei.voitylov at bell-sw.com>> schrieb am Di., 14. Jan. 2020,
> 09:54:
>
>
>     On 14/01/2020 19:57, Baesken, Matthias wrote:
>     > Hello? Magnus and Aleksei,? thanks for the input .
>     >
>     > The times you? provided really look like they make a big
>     difference? at least for people? often? building? ?minimal-vm? .
>     > Guess I have to measure myself a bit? (maybe the difference is
>     not that big on our linux s390x / ppc64(le) ) .
>     >
>     >> If the change to enable lto by default is proposed, what would
>     be the
>     >> recommended strategy for development?
>     >>
>     > Probably? we should a)? ?do not enable it by default but just
>     make sure it can be enabled easily and works? for? the minimal-vm? ?
>     That would be welcome. I have high hopes to LTO the VM some time by
>     default, and the tendency observed is that the compiler time overhead
>     for GCC becomes smaller. At the same time there is no reason why
>     vendors
>     that invested in testing and can absorb the build time hit could
>     provide
>     binaries with LTO built VMs by passing an additional option flag.
>     >? ?or? b)? take it easy to disable it for local development.
>     >
>     > Best regards, Matthias
>     >
>     >
>     >
>     >> Magnus, Matthias,
>     >>
>     >> for me, lto is a little heavyweight for development. x86_64
>     build time
>     >> with gcc 7:
>     >>
>     >> Server 1m32.484s
>     >> Server+Minimal 1m42.166s
>     >> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
>     >>
>     >> If the change to enable lto by default is proposed, what would
>     be the
>     >> recommended strategy for development?
>     >>
>     >> For ARM32 Minimal, please keep in mind that it's not uncommon
>     to disable
>     >> LTO plugin in commodity ARM32 gcc compiler distributions, so
>     for some it
>     >> does not matter what settings we have in OpenJDK. I believe
>     there could
>     >> be other reasons for that on top of build time (bugs?).
>     >>
>


From matthias.baesken at sap.com  Wed Jan 15 15:02:37 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 15:02:37 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
Message-ID: <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello , I can comment on   the  code size .  This is what I get when comparing  a build  without  and  with  -flto .

gcc7 linux x86_64  product build, normal / with -flto
----------------------------------------------------------------------------------

du -sh on the *.so files gives :

16K / 16K      ./lib/libattach.so
48K / 44K      ./lib/libawt_headless.so
752K / 760K    ./lib/libawt.so            <------------------ this one gets a bit larger with flto
472K / 456K    ./lib/libawt_xawt.so       <------------------ small gain
36K / 32K      ./lib/libdt_socket.so
16K /16K       ./lib/libextnet.so
1.5M / 824K    ./lib/libfontmanager.so     <------------------ HUGE gain
784K / 792K    ./lib/libfreetype.so        <------------------ this one gets a bit larger with flto
56K / 56K      ./lib/libinstrument.so
52K / 52K      ./lib/libj2gss.so
20K / 20K      ./lib/libj2pcsc.so
92K / 84K      ./lib/libj2pkcs11.so
12K / 12k      ./lib/libjaas.so
260K / 244K    ./lib/libjavajpeg.so         <----------------- small gain
196K / 188K    ./lib/libjava.so
12K / 12K      ./lib/libjawt.so
280K / 256K    ./lib/libjdwp.so             <----------------- small gain
144K / 140K    ./lib/libjimage.so
84K / 76K      ./lib/libjli.so
16K / 16K      ./lib/libjsig.so
88K / 80K      ./lib/libjsound.so
564K / 420K    ./lib/liblcms.so             <----------------- large gain
12K / 12K      ./lib/libmanagement_agent.so
40K / 36K      ./lib/libmanagement_ext.so
36K / 32K      ./lib/libmanagement.so
576K / 496K    ./lib/libmlib_image.so       <----------------- large gain
112K / 108K    ./lib/libnet.so
100K / 100K    ./lib/libnio.so
16K  / 16K     ./lib/libprefs.so
8.0K / 8.0K    ./lib/librmi.so
60K / 60K      ./lib/libsaproc.so
36K / 32K      ./lib/libsctp.so
368K / 212K    ./lib/libsplashscreen.so     <----------------- large gain
320K / 296K    ./lib/libsunec.so            <----------------- medium gain
72K / 72K      ./lib/libverify.so
44K / 44K      ./lib/libzip.so
16K / 16K      ./lib/server/libjsig.so
23M / 17M      ./lib/server/libjvm.so        <----------------- big gain maybe because it is C++ ?


So  for  some libs  you see  10% and more , but not for all .   But  most  large  libs  like   libjvm.so,  libfontmanager.so      or   liblcms.so    we see good results regarding reduced code size.

I Cannot say much about performance improvements , probably it would be small .

For SPEC  you find something at

http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html

(not that these results would say too much about  JVM performance ).


Best regards, Matthias

From: Volker Simonis <volker.simonis at gmail.com>
Sent: Mittwoch, 15. Januar 2020 14:40
To: Aleksei Voitylov <aleksei.voitylov at bell-sw.com>
Cc: Baesken, Matthias <matthias.baesken at sap.com>; Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; serviceability-dev at openjdk.java.net; build-dev <build-dev at openjdk.java.net>; hotspot-dev at openjdk.java.net
Subject: Re: serviceability agent : problems when using gcc LTO (link time optimization)

While we are speaking about all the drawbacks of LTO, it's still not clear what the benefits are? In the very first mail Matthias mentioned that there might be performance improvements but that performance is not the main driving factor behind this initiative. So is it the reduced code size (Matthias mentioned something around ~10%)?

It would be nice to see some real numbers on various platform for both, the performance improvements for native parts like JIT/GC as well as for the size reduction.
Aleksei Voitylov <aleksei.voitylov at bell-sw.com<mailto:aleksei.voitylov at bell-sw.com>> schrieb am Di., 14. Jan. 2020, 09:54:

On 14/01/2020 19:57, Baesken, Matthias wrote:
> Hello  Magnus and Aleksei,  thanks for the input .
>
> The times you  provided really look like they make a big difference  at least for people  often  building   minimal-vm  .
> Guess I have to measure myself a bit  (maybe the difference is not that big on our linux s390x / ppc64(le) ) .
>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
> Probably  we should a)   do not enable it by default but just make sure it can be enabled easily and works  for  the minimal-vm
That would be welcome. I have high hopes to LTO the VM some time by
default, and the tendency observed is that the compiler time overhead
for GCC becomes smaller. At the same time there is no reason why vendors
that invested in testing and can absorb the build time hit could provide
binaries with LTO built VMs by passing an additional option flag.
>   or  b)  take it easy to disable it for local development.
>
> Best regards, Matthias
>
>
>
>> Magnus, Matthias,
>>
>> for me, lto is a little heavyweight for development. x86_64 build time
>> with gcc 7:
>>
>> Server 1m32.484s
>> Server+Minimal 1m42.166s
>> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
>>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
>> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
>> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
>> does not matter what settings we have in OpenJDK. I believe there could
>> be other reasons for that on top of build time (bugs?).
>>

From volker.simonis at gmail.com  Wed Jan 15 15:29:48 2020
From: volker.simonis at gmail.com (Volker Simonis)
Date: Wed, 15 Jan 2020 07:29:48 -0800
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
 <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>

Aleksei, Matthias,

thanks for the numbers. The size reduction on libjvm.so looks not bad,
indeed.

Do you know if newer versions of GCC use the gold linker by default? I
remember from some experiments which I did many years ago that gold was
considerably faster compared to the default ld linker.

Unfortunately, the documentation I found about LTO/ld/gold [1,2] seems to
be quite old and not very precise. Do you have gained any experience with
LTO/gold and know if gold could maybe improve linking times with LTO?

[1] https://gcc.gnu.org/wiki/LinkTimeOptimization
[2] https://stackoverflow.com/questions/31688069/requirements-to-use-flto


Baesken, Matthias <matthias.baesken at sap.com> schrieb am Mi., 15. Jan. 2020,
07:02:

> Hello , I can comment on   the  code size .  This is what I get when
> comparing  a build  without  and  with  -flto .
>
>
>
> gcc7 linux x86_64  product build, normal / with -flto
>
>
> ----------------------------------------------------------------------------------
>
>
>
> du -sh on the *.so files gives :
>
>
>
> 16K / 16K      ./lib/libattach.so
>
> 48K / 44K      ./lib/libawt_headless.so
>
> 752K / 760K    ./lib/libawt.so            <------------------ this one
> gets a bit larger with flto
>
> 472K / 456K    ./lib/libawt_xawt.so       <------------------ small gain
>
> 36K / 32K      ./lib/libdt_socket.so
>
> 16K /16K       ./lib/libextnet.so
>
> 1.5M / 824K    ./lib/libfontmanager.so     <------------------ HUGE gain
>
> 784K / 792K    ./lib/libfreetype.so        <------------------ this one
> gets a bit larger with flto
>
> 56K / 56K      ./lib/libinstrument.so
>
> 52K / 52K      ./lib/libj2gss.so
>
> 20K / 20K      ./lib/libj2pcsc.so
>
> 92K / 84K      ./lib/libj2pkcs11.so
>
> 12K / 12k      ./lib/libjaas.so
>
> 260K / 244K    ./lib/libjavajpeg.so         <----------------- small gain
>
> 196K / 188K    ./lib/libjava.so
>
> 12K / 12K      ./lib/libjawt.so
>
> 280K / 256K    ./lib/libjdwp.so             <----------------- small gain
>
> 144K / 140K    ./lib/libjimage.so
>
> 84K / 76K      ./lib/libjli.so
>
> 16K / 16K      ./lib/libjsig.so
>
> 88K / 80K      ./lib/libjsound.so
>
> 564K / 420K    ./lib/liblcms.so             <----------------- large gain
>
> 12K / 12K      ./lib/libmanagement_agent.so
>
> 40K / 36K      ./lib/libmanagement_ext.so
>
> 36K / 32K      ./lib/libmanagement.so
>
> 576K / 496K    ./lib/libmlib_image.so       <----------------- large gain
>
> 112K / 108K    ./lib/libnet.so
>
> 100K / 100K    ./lib/libnio.so
>
> 16K  / 16K     ./lib/libprefs.so
>
> 8.0K / 8.0K    ./lib/librmi.so
>
> 60K / 60K      ./lib/libsaproc.so
>
> 36K / 32K      ./lib/libsctp.so
>
> 368K / 212K    ./lib/libsplashscreen.so     <----------------- large gain
>
> 320K / 296K    ./lib/libsunec.so            <----------------- medium gain
>
> 72K / 72K      ./lib/libverify.so
>
> 44K / 44K      ./lib/libzip.so
>
> 16K / 16K      ./lib/server/libjsig.so
>
> 23M / 17M      ./lib/server/libjvm.so        <----------------- big gain
> maybe because it is C++ ?
>
>
>
>
>
> So  for  some libs  you see  10% and more , but not for all .   But  most
> large  libs  like   libjvm.so,  libfontmanager.so      or   liblcms.so
> we see good results regarding reduced code size.
>
>
>
> I Cannot say much about performance improvements , probably it would be
> small .
>
>
>
> For SPEC  you find something at
>
>
>
>
> http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html
>
>
>
> (not that these results would say too much about  JVM performance ).
>
>
>
>
>
> Best regards, Matthias
>
>
>
> *From:* Volker Simonis <volker.simonis at gmail.com>
> *Sent:* Mittwoch, 15. Januar 2020 14:40
> *To:* Aleksei Voitylov <aleksei.voitylov at bell-sw.com>
> *Cc:* Baesken, Matthias <matthias.baesken at sap.com>; Magnus Ihse Bursie <
> magnus.ihse.bursie at oracle.com>; serviceability-dev at openjdk.java.net;
> build-dev <build-dev at openjdk.java.net>; hotspot-dev at openjdk.java.net
> *Subject:* Re: serviceability agent : problems when using gcc LTO (link
> time optimization)
>
>
>
> While we are speaking about all the drawbacks of LTO, it's still not clear
> what the benefits are? In the very first mail Matthias mentioned that there
> might be performance improvements but that performance is not the main
> driving factor behind this initiative. So is it the reduced code size
> (Matthias mentioned something around ~10%)?
>
>
>
> It would be nice to see some real numbers on various platform for both,
> the performance improvements for native parts like JIT/GC as well as for
> the size reduction.
>
> Aleksei Voitylov <aleksei.voitylov at bell-sw.com> schrieb am Di., 14. Jan.
> 2020, 09:54:
>
>
> On 14/01/2020 19:57, Baesken, Matthias wrote:
> > Hello  Magnus and Aleksei,  thanks for the input .
> >
> > The times you  provided really look like they make a big difference  at
> least for people  often  building   minimal-vm  .
> > Guess I have to measure myself a bit  (maybe the difference is not that
> big on our linux s390x / ppc64(le) ) .
> >
> >> If the change to enable lto by default is proposed, what would be the
> >> recommended strategy for development?
> >>
> > Probably  we should a)   do not enable it by default but just make sure
> it can be enabled easily and works  for  the minimal-vm
> That would be welcome. I have high hopes to LTO the VM some time by
> default, and the tendency observed is that the compiler time overhead
> for GCC becomes smaller. At the same time there is no reason why vendors
> that invested in testing and can absorb the build time hit could provide
> binaries with LTO built VMs by passing an additional option flag.
> >   or  b)  take it easy to disable it for local development.
> >
> > Best regards, Matthias
> >
> >
> >
> >> Magnus, Matthias,
> >>
> >> for me, lto is a little heavyweight for development. x86_64 build time
> >> with gcc 7:
> >>
> >> Server 1m32.484s
> >> Server+Minimal 1m42.166s
> >> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
> >>
> >> If the change to enable lto by default is proposed, what would be the
> >> recommended strategy for development?
> >>
> >> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
> >> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
> >> does not matter what settings we have in OpenJDK. I believe there could
> >> be other reasons for that on top of build time (bugs?).
> >>
>
>


From erik.joelsson at oracle.com  Wed Jan 15 15:47:57 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 15 Jan 2020 07:47:57 -0800
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
 <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
Message-ID: <824316f1-43f6-b61b-c764-ad0c0996b325@oracle.com>

On 2020-01-15 07:29, Volker Simonis wrote:
> Do you know if newer versions of GCC use the gold linker by default? I
> remember from some experiments which I did many years ago that gold was
> considerably faster compared to the default ld linker.

The default linker is system configured so it depends on your Linux 
distro. The devkits generated by the current devkit makefiles configures 
gold as default.

/Erik




From matthias.baesken at sap.com  Wed Jan 15 16:00:38 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 16:00:38 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <824316f1-43f6-b61b-c764-ad0c0996b325@oracle.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
 <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
 <824316f1-43f6-b61b-c764-ad0c0996b325@oracle.com>
Message-ID: <AM6PR02MB42005E8E0E9B8FB4C10991AB93370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, I  used  the  "normal"  linker so I think  what 

https://stackoverflow.com/questions/31688069/requirements-to-use-flto

says is true ,  one can use  also the "normal"  linker .
Haven't checked  for any performance  (or other) improvements   when using gold  instead .  



Best regards, Matthias


> On 2020-01-15 07:29, Volker Simonis wrote:
> > Do you know if newer versions of GCC use the gold linker by default? I
> > remember from some experiments which I did many years ago that gold
> was
> > considerably faster compared to the default ld linker.
> 
> The default linker is system configured so it depends on your Linux
> distro. The devkits generated by the current devkit makefiles configures
> gold as default.
> 
> /Erik
> 


From matthias.baesken at sap.com  Wed Jan 15 16:11:03 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 16:11:03 +0000
Subject: serviceability agent : problems when using gcc LTO (link time
 optimization)
In-Reply-To: <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
References: <AM6PR02MB4200339A78CD451FDF566A3C93380@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <131b1189-d0e8-3d3e-6137-81ffc8eeeb84@oracle.com>
 <AM6PR02MB420025390B1BEE71F80DB8B693340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <4fdc39f0-9ed3-5878-93b2-b536b8779125@oracle.com>
 <249cba82-1469-03f8-d205-e5895eab2cc3@bell-sw.com>
 <AM6PR02MB4200457722633979CA9EC83093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <9908f346-f376-5bfc-8d5b-bc250e336032@bell-sw.com>
 <CA+3eh11mg+UgbwgF7qCHkq3hvr73H=3X4+sDTaZjdvZ3kTuejQ@mail.gmail.com>
 <AM6PR02MB42007C34069D56491E86D8B893370@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <CA+3eh12gmh2-xOhbPf9Gc=92mfWE1XcRJeUo32Aam01ipKx49w@mail.gmail.com>
Message-ID: <AM6PR02MB4200A3D1523DB8800DCC194893370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello,    here is another  comparison for the larger  JDK  shared libs,  this time  with the sizes  of   build with linktime-gc (--gc-sections)   added .
( just for the larger libs )
(  I had not  enabled  linktime-gc  for libjvm   in our  test build , just for the JDK libs . )

Linuxx86_64 / gcc7

normal / with -flto / with linktime-gc (--gc-sections)
-----------------------------------------------------------
752K / 760K / 752K   ./lib/libawt.so            <------------------ this one gets a bit larger but only with flto
472K / 456K / 468K   ./lib/libawt_xawt.so       <------------------ small gain
1.5M / 824K / 900K   ./lib/libfontmanager.so     <------------------ HUGE gain , not as good with ltgc but still good
784K / 792K / 784K  ./lib/libfreetype.so        <------------------ this one gets a bit larger  (but not with ltgc)
260K / 244K / 252K   ./lib/libjavajpeg.so         <----------------- small gain
196K / 188K / 196K   ./lib/libjava.so
280K / 256K / 276K   ./lib/libjdwp.so             <----------------- small gain
144K / 140K / 136K   ./lib/libjimage.so
564K / 420K / 404K   ./lib/liblcms.so             <----------------- large gain ,  even better with  ltgc
576K / 496K / 556K   ./lib/libmlib_image.so       <----------------- large gain with flto , small one with ltgc
368K / 212K / 236K   ./lib/libsplashscreen.so     <----------------- large gain
320K / 296K / 300K   ./lib/libsunec.so            <----------------- medium gain
23M / 17M   /  --not enabled---    ./lib/server/libjvm.so        <----------------- big gain maybe because it is C++ ?


So   one can see,  that   flto   is usually  a bit better  than link-time-gc  when it comes to  improving lib sizes, but not always .
However  linktime-gc   seems to be faster when comparing build times   , I did not really notice much  build  time slowdown because of it .
( we have it enabled  for linux  s390x  for some time in OpenJDK ).
The  linktime-gc   also offers a nice feature  to print out the eliminated stuff ,   that can be used  to remove  unused code cross-platform .
e.g.  the removed symbols  from   https://bugs.openjdk.java.net/browse/JDK-8234629    has been found this way .


Best regards, Matthias



Aleksei, Matthias,

thanks for the numbers. The size reduction on libjvm.so looks not bad, indeed.

Do you know if newer versions of GCC use the gold linker by default? I remember from some experiments which I did many years ago that gold was considerably faster compared to the default ld linker.

Unfortunately, the documentation I found about LTO/ld/gold [1,2] seems to be quite old and not very precise. Do you have gained any experience with LTO/gold and know if gold could maybe improve linking times with LTO?

[1] https://gcc.gnu.org/wiki/LinkTimeOptimization
[2] https://stackoverflow.com/questions/31688069/requirements-to-use-flto


Baesken, Matthias <matthias.baesken at sap.com<mailto:matthias.baesken at sap.com>> schrieb am Mi., 15. Jan. 2020, 07:02:
Hello , I can comment on   the  code size .  This is what I get when comparing  a build  without  and  with  -flto .

gcc7 linux x86_64  product build, normal / with -flto
----------------------------------------------------------------------------------

du -sh on the *.so files gives :

16K / 16K      ./lib/libattach.so
48K / 44K      ./lib/libawt_headless.so
752K / 760K    ./lib/libawt.so            <------------------ this one gets a bit larger with flto
472K / 456K    ./lib/libawt_xawt.so       <------------------ small gain
36K / 32K      ./lib/libdt_socket.so
16K /16K       ./lib/libextnet.so
1.5M / 824K    ./lib/libfontmanager.so     <------------------ HUGE gain
784K / 792K    ./lib/libfreetype.so        <------------------ this one gets a bit larger with flto
56K / 56K      ./lib/libinstrument.so
52K / 52K      ./lib/libj2gss.so
20K / 20K      ./lib/libj2pcsc.so
92K / 84K      ./lib/libj2pkcs11.so
12K / 12k      ./lib/libjaas.so
260K / 244K    ./lib/libjavajpeg.so         <----------------- small gain
196K / 188K    ./lib/libjava.so
12K / 12K      ./lib/libjawt.so
280K / 256K    ./lib/libjdwp.so             <----------------- small gain
144K / 140K    ./lib/libjimage.so
84K / 76K      ./lib/libjli.so
16K / 16K      ./lib/libjsig.so
88K / 80K      ./lib/libjsound.so
564K / 420K    ./lib/liblcms.so             <----------------- large gain
12K / 12K      ./lib/libmanagement_agent.so
40K / 36K      ./lib/libmanagement_ext.so
36K / 32K      ./lib/libmanagement.so
576K / 496K    ./lib/libmlib_image.so       <----------------- large gain
112K / 108K    ./lib/libnet.so
100K / 100K    ./lib/libnio.so
16K  / 16K     ./lib/libprefs.so
8.0K / 8.0K    ./lib/librmi.so
60K / 60K      ./lib/libsaproc.so
36K / 32K      ./lib/libsctp.so
368K / 212K    ./lib/libsplashscreen.so     <----------------- large gain
320K / 296K    ./lib/libsunec.so            <----------------- medium gain
72K / 72K      ./lib/libverify.so
44K / 44K      ./lib/libzip.so
16K / 16K      ./lib/server/libjsig.so
23M / 17M      ./lib/server/libjvm.so        <----------------- big gain maybe because it is C++ ?


So  for  some libs  you see  10% and more , but not for all .   But  most  large  libs  like   libjvm.so,  libfontmanager.so      or   liblcms.so    we see good results regarding reduced code size.

I Cannot say much about performance improvements , probably it would be small .

For SPEC  you find something at

http://hubicka.blogspot.com/2019/05/gcc-9-link-time-and-inter-procedural.html

(not that these results would say too much about  JVM performance ).


Best regards, Matthias

From: Volker Simonis <volker.simonis at gmail.com<mailto:volker.simonis at gmail.com>>
Sent: Mittwoch, 15. Januar 2020 14:40
To: Aleksei Voitylov <aleksei.voitylov at bell-sw.com<mailto:aleksei.voitylov at bell-sw.com>>
Cc: Baesken, Matthias <matthias.baesken at sap.com<mailto:matthias.baesken at sap.com>>; Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com<mailto:magnus.ihse.bursie at oracle.com>>; serviceability-dev at openjdk.java.net<mailto:serviceability-dev at openjdk.java.net>; build-dev <build-dev at openjdk.java.net<mailto:build-dev at openjdk.java.net>>; hotspot-dev at openjdk.java.net<mailto:hotspot-dev at openjdk.java.net>
Subject: Re: serviceability agent : problems when using gcc LTO (link time optimization)

While we are speaking about all the drawbacks of LTO, it's still not clear what the benefits are? In the very first mail Matthias mentioned that there might be performance improvements but that performance is not the main driving factor behind this initiative. So is it the reduced code size (Matthias mentioned something around ~10%)?

It would be nice to see some real numbers on various platform for both, the performance improvements for native parts like JIT/GC as well as for the size reduction.
Aleksei Voitylov <aleksei.voitylov at bell-sw.com<mailto:aleksei.voitylov at bell-sw.com>> schrieb am Di., 14. Jan. 2020, 09:54:

On 14/01/2020 19:57, Baesken, Matthias wrote:
> Hello  Magnus and Aleksei,  thanks for the input .
>
> The times you  provided really look like they make a big difference  at least for people  often  building   minimal-vm  .
> Guess I have to measure myself a bit  (maybe the difference is not that big on our linux s390x / ppc64(le) ) .
>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
> Probably  we should a)   do not enable it by default but just make sure it can be enabled easily and works  for  the minimal-vm
That would be welcome. I have high hopes to LTO the VM some time by
default, and the tendency observed is that the compiler time overhead
for GCC becomes smaller. At the same time there is no reason why vendors
that invested in testing and can absorb the build time hit could provide
binaries with LTO built VMs by passing an additional option flag.
>   or  b)  take it easy to disable it for local development.
>
> Best regards, Matthias
>
>
>
>> Magnus, Matthias,
>>
>> for me, lto is a little heavyweight for development. x86_64 build time
>> with gcc 7:
>>
>> Server 1m32.484s
>> Server+Minimal 1m42.166s
>> Server+Minimal (--with-jvm-features="link-time-opt") 5m29.422s
>>
>> If the change to enable lto by default is proposed, what would be the
>> recommended strategy for development?
>>
>> For ARM32 Minimal, please keep in mind that it's not uncommon to disable
>> LTO plugin in commodity ARM32 gcc compiler distributions, so for some it
>> does not matter what settings we have in OpenJDK. I believe there could
>> be other reasons for that on top of build time (bugs?).
>>

From matthias.baesken at sap.com  Wed Jan 15 16:14:54 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 15 Jan 2020 16:14:54 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <998c4a08-5670-f51d-4625-9c7e984b4b5d@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <b8f01c3f-27d1-f055-ba49-448a47ca4960@oracle.com>
 <AM6PR02MB420099D1C75E508D1BE1CD8D93370@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <998c4a08-5670-f51d-4625-9c7e984b4b5d@oracle.com>
Message-ID: <AM6PR02MB420070AEDB1159EC92DDE58493370@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, I did not notice  slowdowns in our night makes .

Looking at a specific  test machine  I use  (x86_64,  build JOBS hardwired  set  to 12 )   I  get around  6 minutes build time  with and without the feature .

( but you have to take into account  that  the  link-time section-gc   on  x86_64    in my patch is only enabled  for the smaller JDK libs and not libjvm.so  )  

Best regards, Matthias


> 
> Given the discussion regarding lto on hotspot and the extreme increased
> build time, have you noticed any difference in build times with this patch?
> 
> /Erik
> 


From joe.darcy at oracle.com  Wed Jan 15 21:14:34 2020
From: joe.darcy at oracle.com (Joe Darcy)
Date: Wed, 15 Jan 2020 13:14:34 -0800
Subject: JDK 15 RFR of JDK-8235959: Update --release 14 symbol information up
 to JDK 14 b31
Message-ID: <7c73a23e-aa3c-1f25-d66d-43ab1a1e7c9a@oracle.com>

Hello,

Since JDK 14 build 27,

 ??? JDK-8236695 "java.lang.Record should be declared with an explicit 
constructor" and
 ??? JDK-8234049 "Implementation of Memory Access API (Incubator)"

changed the API in JDK 14. The symbol information for 14 in JDK 15 
should be updated accordingly to bring it current with JDK 14 b31:

 ??? http://cr.openjdk.java.net/~darcy/8235959.0/

Thanks,

-Joe



From jonathan.gibbons at oracle.com  Wed Jan 15 21:47:40 2020
From: jonathan.gibbons at oracle.com (Jonathan Gibbons)
Date: Wed, 15 Jan 2020 13:47:40 -0800
Subject: JDK 15 RFR of JDK-8235959: Update --release 14 symbol information
 up to JDK 14 b31
In-Reply-To: <7c73a23e-aa3c-1f25-d66d-43ab1a1e7c9a@oracle.com>
References: <7c73a23e-aa3c-1f25-d66d-43ab1a1e7c9a@oracle.com>
Message-ID: <60ad69a9-bb92-412b-05c3-066b281ea6d6@oracle.com>

Looks good to me.

-- Jon

On 1/15/20 1:14 PM, Joe Darcy wrote:
> Hello,
>
> Since JDK 14 build 27,
>
> ??? JDK-8236695 "java.lang.Record should be declared with an explicit 
> constructor" and
> ??? JDK-8234049 "Implementation of Memory Access API (Incubator)"
>
> changed the API in JDK 14. The symbol information for 14 in JDK 15 
> should be updated accordingly to bring it current with JDK 14 b31:
>
> ??? http://cr.openjdk.java.net/~darcy/8235959.0/
>
> Thanks,
>
> -Joe
>


From david.holmes at oracle.com  Wed Jan 15 22:25:09 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Jan 2020 08:25:09 +1000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>

Hi Matthias,

I have reservations about turning this on by default and with no way to 
control it. This seems like it should be something you have to opt-in to 
initially while we gain some experience with it and ensure there are no 
unexpected side-effects. After that it could be enabled by default.

David

On 15/01/2020 12:07 am, Baesken, Matthias wrote:
> Hello,  the following change enables the  link-time section-gc for linux .
> 
> gcc and ld support enabling "garbage collection" of unused input sections.
> This can be used to eliminate unused coding from native libraries (especially when already compiling the objects with compiler flags -ffunction-sections -fdata-sections .
> See for details the --gc-sections and --print-gc-sections parts of the ld documentation :
> 
> https://linux.die.net/man/1/ld
> 
> 
> We had this enabled already  for  linux s390x ,  with https://bugs.openjdk.java.net/browse/JDK-8234525
> 8234525: enable link-time section-gc for linux s390x to remove unused code .
> 
> This  time we enable it too for  the other linux platforms .
> 
> For the other platforms I do not enable it for JVM, just for the JDK libs.  The reason is that the serviceability agent  (not supported on linux s390x )    is not  (yet) ready for the optimization .
> Below you see the results , for some libraries  a significant  size reduction can be achieved .
> 
> 
> Results from linux x86_64 product builds :
> 
> without / with ltgc
> 
> 320K / 300K    /images/jdk/lib/libsunec.so       <-------------------------
> 36K  / 36K     /images/jdk/lib/libdt_socket.so
> 280K / 276K   /images/jdk/lib/libjdwp.so
> 23M  / 23M    /images/jdk/lib/server/libjvm.so    <---- not set for libjvm.so for x86_64
> 16K  / 16K    /images/jdk/lib/server/libjsig.so
> 72K  / 72M    /images/jdk/lib/libverify.so
> 84K  / 84M   /images/jdk/lib/libjli.so
> 16K  / 16K    /images/jdk/lib/libjsig.so
> 196K / 196K   /images/jdk/lib/libjava.so
> 44K  / 44K    /images/jdk/lib/libzip.so
> 144K / 136K   /images/jdk/lib/libjimage.so
> 112K / 112K   /images/jdk/lib/libnet.so
> 100K / 100K   /images/jdk/lib/libnio.so
> 36K  / 36K    /images/jdk/lib/libsctp.so
> 576K / 556K   /images/jdk/lib/libmlib_image.so
> 752K / 752K   /images/jdk/lib/libawt.so
> 260K / 252K   /images/jdk/lib/libjavajpeg.so
> 784K / 784K   /images/jdk/lib/libfreetype.so
> 368K / 236K /images/jdk/lib/libsplashscreen.so   <-------------------------
> 88K / 88K    /images/jdk/lib/libjsound.so
> 472K / 468K    /images/jdk/lib/libawt_xawt.so
> 564K / 404K   /images/jdk/lib/liblcms.so         <--------------------------
> 48K / 48K    /images/jdk/lib/libawt_headless.so
> 12K / 12K    /images/jdk/lib/libjawt.so
> 1.5M / 900K   /images/jdk/lib/libfontmanager.so  <------------------------------
> 12K / 12K    /images/jdk/lib/libjaas.so
> 92K / 92K    /images/jdk/lib/libj2pkcs11.so
> 16K / 16K    /images/jdk/lib/libattach.so
> 8.0K / 8.0K   /images/jdk/lib/librmi.so
> 56K / 56K    /images/jdk/lib/libinstrument.so
> 16K / 16K    /images/jdk/lib/libprefs.so
> 52K / 52K    /images/jdk/lib/libj2gss.so
> 12K / 12K    /images/jdk/lib/libmanagement_agent.so
> 36K / 32K    /images/jdk/lib/libmanagement.so
> 16K / 16K    /images/jdk/lib/libextnet.so
> 20K / 20K    /images/jdk/lib/libj2pcsc.so
> 40K / 40K    /images/jdk/lib/libmanagement_ext.so
> 60K / 60K    /images/jdk/lib/libsaproc.so
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8236714
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.2/
> 
> 
> Thanks, Matthias
> 


From matthias.baesken at sap.com  Thu Jan 16 08:10:30 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 16 Jan 2020 08:10:30 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
Message-ID: <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi David, sure we can introduce a way to switch this on/off.
There is already something similar for the link-time optimization (flto) , see the feature 

JvmFeatures.gmk
180 ifeq ($(call check-jvm-feature, link-time-opt), true)
190 ifeq ($(call check-jvm-feature, link-time-opt), false)

hotspot.m4
29 static-build link-time-opt aot jfr"
502 JVM_FEATURES_link_time_opt="link-time-opt"


Should we have  "link-time-gc"  additionally to   " link-time-opt"  ?   (however it  would  be a  bit misleading   that it is a "JVM" feature , but except linux s390x  it is only changing the build of the JDK libs) .

Best regards, Matthias



> 
> Hi Matthias,
> 
> I have reservations about turning this on by default and with no way to
> control it. This seems like it should be something you have to opt-in to
> initially while we gain some experience with it and ensure there are no
> unexpected side-effects. After that it could be enabled by default.
> 


From david.holmes at oracle.com  Thu Jan 16 09:30:38 2020
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 16 Jan 2020 19:30:38 +1000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>

Hi Matthias,

On 16/01/2020 6:10 pm, Baesken, Matthias wrote:
> Hi David, sure we can introduce a way to switch this on/off.

Thanks.

> There is already something similar for the link-time optimization (flto) , see the feature
> 
> JvmFeatures.gmk
> 180 ifeq ($(call check-jvm-feature, link-time-opt), true)
> 190 ifeq ($(call check-jvm-feature, link-time-opt), false)
> 
> hotspot.m4
> 29 static-build link-time-opt aot jfr"
> 502 JVM_FEATURES_link_time_opt="link-time-opt"

Yep familiar with that from Minimal VM and SE Embedded days :)

> Should we have  "link-time-gc"  additionally to   " link-time-opt"  ?   (however it  would  be a  bit misleading   that it is a "JVM" feature , but except linux s390x  it is only changing the build of the JDK libs) .

I agree the definition of this as a "JVM" feature is a bit 
odd/misleading. Perhaps the build folk have a suggestion on how to 
refactor this kind of option into something more general? In the 
meantime having link-time-gc sit alongside link-time-opt seems 
acceptable to me.

Thanks,
David

> Best regards, Matthias
> 
> 
> 
>>
>> Hi Matthias,
>>
>> I have reservations about turning this on by default and with no way to
>> control it. This seems like it should be something you have to opt-in to
>> initially while we gain some experience with it and ensure there are no
>> unexpected side-effects. After that it could be enabled by default.
>>
> 


From matthias.baesken at sap.com  Thu Jan 16 10:18:39 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 16 Jan 2020 10:18:39 +0000
Subject: configuring with  --with-jvm-variants=minimal,server makes cds
 disappear in server
Message-ID: <AM6PR02MB42005B5EA3E77330E579354F93360@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, I noticed the following  strange "feature" (or is it a bug?) .
When  building 2 VM variants in one build  and  using

  --with-jvm-variants=minimal,server

For this, the build works nicely. But I notice that  in the server VM, cds  is removed.
Instead of

checking if cds should be enabled... yes

I get ( with the following patch that adds tracing)  :

configure: WARNING: ENABLE_CDS set to false because we found a minimal, core or zero JVM.
checking if cds should be enabled... no
  ...

* JVM features:   minimal: 'compiler1 minimal serialgc'       server: 'compiler1 compiler2 epsilongc g1gc jfr jni-check jvmti management nmt parallelgc serialgc services vm-structs'

(patch is

--- a/make/autoconf/hotspot.m4  Wed Jan 15 21:20:40 2020 -0800
+++ b/make/autoconf/hotspot.m4  Thu Jan 16 10:24:43 2020 +0100
@@ -528,6 +528,7 @@
  if HOTSPOT_CHECK_JVM_VARIANT(zero) || HOTSPOT_CHECK_JVM_VARIANT(minimal) || HOTSPOT_CHECK_JVM_VARIANT(core); then
     # ..except when the user explicitely requested it with --enable-jvm-features
     if ! HOTSPOT_CHECK_JVM_FEATURE(cds); then
       ENABLE_CDS="false"
+      AC_MSG_WARN([ENABLE_CDS set to false because we found a minimal, core or zero JVM.])
       if test "x$enable_cds" = "xyes"; then
         AC_MSG_ERROR([CDS not implemented for variants zero, minimal, core. Remove --enable-cds.])
       Fi


Is it expected that  cds goes away in "server"   when  configuring  "--with-jvm-variants=minimal,server"   ?   Looks like a bug to me,  should it be fixed   (so that cds stays in the server   jvm-feature list) ?


Thanks, Matthias



From erik.joelsson at oracle.com  Thu Jan 16 15:59:19 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 16 Jan 2020 07:59:19 -0800
Subject: configuring with --with-jvm-variants=minimal,server makes cds
 disappear in server
In-Reply-To: <AM6PR02MB42005B5EA3E77330E579354F93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42005B5EA3E77330E579354F93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <eed77ed5-9800-6a91-89f3-dc7c896cbd86@oracle.com>

This does indeed look like a bug to me. At least at Oracle, we no longer 
build any multi JVM configs regularly, so things like this falls through 
the cracks.

/Erik

On 2020-01-16 02:18, Baesken, Matthias wrote:
> Hello, I noticed the following  strange "feature" (or is it a bug?) .
> When  building 2 VM variants in one build  and  using
>
>    --with-jvm-variants=minimal,server
>
> For this, the build works nicely. But I notice that  in the server VM, cds  is removed.
> Instead of
>
> checking if cds should be enabled... yes
>
> I get ( with the following patch that adds tracing)  :
>
> configure: WARNING: ENABLE_CDS set to false because we found a minimal, core or zero JVM.
> checking if cds should be enabled... no
>    ...
>
> * JVM features:   minimal: 'compiler1 minimal serialgc'       server: 'compiler1 compiler2 epsilongc g1gc jfr jni-check jvmti management nmt parallelgc serialgc services vm-structs'
>
> (patch is
>
> --- a/make/autoconf/hotspot.m4  Wed Jan 15 21:20:40 2020 -0800
> +++ b/make/autoconf/hotspot.m4  Thu Jan 16 10:24:43 2020 +0100
> @@ -528,6 +528,7 @@
>    if HOTSPOT_CHECK_JVM_VARIANT(zero) || HOTSPOT_CHECK_JVM_VARIANT(minimal) || HOTSPOT_CHECK_JVM_VARIANT(core); then
>       # ..except when the user explicitely requested it with --enable-jvm-features
>       if ! HOTSPOT_CHECK_JVM_FEATURE(cds); then
>         ENABLE_CDS="false"
> +      AC_MSG_WARN([ENABLE_CDS set to false because we found a minimal, core or zero JVM.])
>         if test "x$enable_cds" = "xyes"; then
>           AC_MSG_ERROR([CDS not implemented for variants zero, minimal, core. Remove --enable-cds.])
>         Fi
>
>
> Is it expected that  cds goes away in "server"   when  configuring  "--with-jvm-variants=minimal,server"   ?   Looks like a bug to me,  should it be fixed   (so that cds stays in the server   jvm-feature list) ?
>
>
> Thanks, Matthias
>


From matthias.baesken at sap.com  Thu Jan 16 16:22:11 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 16 Jan 2020 16:22:11 +0000
Subject: configuring with --with-jvm-variants=minimal,server makes cds
 disappear in server
In-Reply-To: <eed77ed5-9800-6a91-89f3-dc7c896cbd86@oracle.com>
References: <AM6PR02MB42005B5EA3E77330E579354F93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <eed77ed5-9800-6a91-89f3-dc7c896cbd86@oracle.com>
Message-ID: <AM6PR02MB420000FAB535AAC907A9E16493360@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, thanks for looking into it .
Should I do a check just looking into single JVM configs,   something like

AC_DEFUN([HOTSPOT_IS_JVM_VARIANT],
[ [ [[ " $JVM_VARIANTS " == " $1 " ]] ] ])

if HOTSPOT_IS_JVM_VARIANT(zero) ||  HOTSPOT_IS_JVM_VARIANT(minimal) ||  HOTSPOT_IS_JVM_VARIANT(core); then

instead of

AC_DEFUN([HOTSPOT_CHECK_JVM_VARIANT],
[ [ [[ " $JVM_VARIANTS " =~ " $1 " ]] ] ])

if HOTSPOT_CHECK_JVM_VARIANT(zero) ||  HOTSPOT_CHECK_JVM_VARIANT(minimal) ||  HOTSPOT_CHECK_JVM_VARIANT(core); then


This should  remove the error in case of multi-JVM configs .
Or maybe  the whole check ( if HOTSPOT_CHECK_JVM_VARIANT(zero) ||  HOTSPOT_CHECK_JVM_VARIANT(minimal) ||  HOTSPOT_CHECK_JVM_VARIANT(core); then .... fi )
 might be removed , I see not so much value in it  because  in case one sets --enable-jvm-features   directly the check isn?t done .

I opened  bug :

https://bugs.openjdk.java.net/browse/JDK-8237374

JDK-8237374 :  configuring with --with-jvm-variants=minimal,server makes cds disappear in server


Best regards, Matthias




> 
> This does indeed look like a bug to me. At least at Oracle, we no longer
> build any multi JVM configs regularly, so things like this falls through
> the cracks.
> 
> /Erik
> 
> On 2020-01-16 02:18, Baesken, Matthias wrote:
> > Hello, I noticed the following  strange "feature" (or is it a bug?) .
> > When  building 2 VM variants in one build  and  using
> >
> >    --with-jvm-variants=minimal,server
> >
> > For this, the build works nicely. But I notice that  in the server VM, cds  is
> removed.
> > Instead of
> >
> > checking if cds should be enabled... yes
> >
> > I get ( with the following patch that adds tracing)  :
> >
> > configure: WARNING: ENABLE_CDS set to false because we found a
> minimal, core or zero JVM.
> > checking if cds should be enabled... no
> >    ...
> >
> > * JVM features:   minimal: 'compiler1 minimal serialgc'       server: 'compiler1
> compiler2 epsilongc g1gc jfr jni-check jvmti management nmt parallelgc
> serialgc services vm-structs'
> >
> > (patch is
> >
> > --- a/make/autoconf/hotspot.m4  Wed Jan 15 21:20:40 2020 -0800
> > +++ b/make/autoconf/hotspot.m4  Thu Jan 16 10:24:43 2020 +0100
> > @@ -528,6 +528,7 @@
> >    if HOTSPOT_CHECK_JVM_VARIANT(zero) ||
> HOTSPOT_CHECK_JVM_VARIANT(minimal) ||
> HOTSPOT_CHECK_JVM_VARIANT(core); then
> >       # ..except when the user explicitely requested it with --enable-jvm-
> features
> >       if ! HOTSPOT_CHECK_JVM_FEATURE(cds); then
> >         ENABLE_CDS="false"
> > +      AC_MSG_WARN([ENABLE_CDS set to false because we found a
> minimal, core or zero JVM.])
> >         if test "x$enable_cds" = "xyes"; then
> >           AC_MSG_ERROR([CDS not implemented for variants zero, minimal,
> core. Remove --enable-cds.])
> >         Fi
> >
> >
> > Is it expected that  cds goes away in "server"   when  configuring  "--with-
> jvm-variants=minimal,server"   ?   Looks like a bug to me,  should it be fixed
> (so that cds stays in the server   jvm-feature list) ?
> >
> >
> > Thanks, Matthias
> >

From erik.joelsson at oracle.com  Thu Jan 16 17:09:30 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 16 Jan 2020 09:09:30 -0800
Subject: configuring with --with-jvm-variants=minimal,server makes cds
 disappear in server
In-Reply-To: <AM6PR02MB420000FAB535AAC907A9E16493360@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42005B5EA3E77330E579354F93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <eed77ed5-9800-6a91-89f3-dc7c896cbd86@oracle.com>
 <AM6PR02MB420000FAB535AAC907A9E16493360@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <766f2d58-e8eb-3c89-b8f7-4294723135d9@oracle.com>

I would lean towards changing the check from "if any of the variants do 
not support CDS, disable it" to "if any of the variants support CDS, 
enable it". The cds jvm feature is only added to the variants that 
support it anyway.

If you do this, you probably need to figure out how to explicitly only 
generate the archive for the supported jvm variants in Images.gmk too. 
Not sure if the archive is jvm variant dependent or if the layout even 
supports multiple variants. It will likely get messy.

/Erik

On 2020-01-16 08:22, Baesken, Matthias wrote:
> Hello, thanks for looking into it .
> Should I do a check just looking into single JVM configs,   something like
>
> AC_DEFUN([HOTSPOT_IS_JVM_VARIANT],
> [ [ [[ " $JVM_VARIANTS " == " $1 " ]] ] ])
>
> if HOTSPOT_IS_JVM_VARIANT(zero) ||  HOTSPOT_IS_JVM_VARIANT(minimal) ||  HOTSPOT_IS_JVM_VARIANT(core); then
>
> instead of
>
> AC_DEFUN([HOTSPOT_CHECK_JVM_VARIANT],
> [ [ [[ " $JVM_VARIANTS " =~ " $1 " ]] ] ])
>
> if HOTSPOT_CHECK_JVM_VARIANT(zero) ||  HOTSPOT_CHECK_JVM_VARIANT(minimal) ||  HOTSPOT_CHECK_JVM_VARIANT(core); then
>
>
> This should  remove the error in case of multi-JVM configs .
> Or maybe  the whole check ( if HOTSPOT_CHECK_JVM_VARIANT(zero) ||  HOTSPOT_CHECK_JVM_VARIANT(minimal) ||  HOTSPOT_CHECK_JVM_VARIANT(core); then .... fi )
>   might be removed , I see not so much value in it  because  in case one sets --enable-jvm-features   directly the check isn?t done .
>
> I opened  bug :
>
> https://bugs.openjdk.java.net/browse/JDK-8237374
>
> JDK-8237374 :  configuring with --with-jvm-variants=minimal,server makes cds disappear in server
>
>
> Best regards, Matthias
>
>
>
>
>> This does indeed look like a bug to me. At least at Oracle, we no longer
>> build any multi JVM configs regularly, so things like this falls through
>> the cracks.
>>
>> /Erik
>>
>> On 2020-01-16 02:18, Baesken, Matthias wrote:
>>> Hello, I noticed the following  strange "feature" (or is it a bug?) .
>>> When  building 2 VM variants in one build  and  using
>>>
>>>     --with-jvm-variants=minimal,server
>>>
>>> For this, the build works nicely. But I notice that  in the server VM, cds  is
>> removed.
>>> Instead of
>>>
>>> checking if cds should be enabled... yes
>>>
>>> I get ( with the following patch that adds tracing)  :
>>>
>>> configure: WARNING: ENABLE_CDS set to false because we found a
>> minimal, core or zero JVM.
>>> checking if cds should be enabled... no
>>>     ...
>>>
>>> * JVM features:   minimal: 'compiler1 minimal serialgc'       server: 'compiler1
>> compiler2 epsilongc g1gc jfr jni-check jvmti management nmt parallelgc
>> serialgc services vm-structs'
>>> (patch is
>>>
>>> --- a/make/autoconf/hotspot.m4  Wed Jan 15 21:20:40 2020 -0800
>>> +++ b/make/autoconf/hotspot.m4  Thu Jan 16 10:24:43 2020 +0100
>>> @@ -528,6 +528,7 @@
>>>     if HOTSPOT_CHECK_JVM_VARIANT(zero) ||
>> HOTSPOT_CHECK_JVM_VARIANT(minimal) ||
>> HOTSPOT_CHECK_JVM_VARIANT(core); then
>>>        # ..except when the user explicitely requested it with --enable-jvm-
>> features
>>>        if ! HOTSPOT_CHECK_JVM_FEATURE(cds); then
>>>          ENABLE_CDS="false"
>>> +      AC_MSG_WARN([ENABLE_CDS set to false because we found a
>> minimal, core or zero JVM.])
>>>          if test "x$enable_cds" = "xyes"; then
>>>            AC_MSG_ERROR([CDS not implemented for variants zero, minimal,
>> core. Remove --enable-cds.])
>>>          Fi
>>>
>>>
>>> Is it expected that  cds goes away in "server"   when  configuring  "--with-
>> jvm-variants=minimal,server"   ?   Looks like a bug to me,  should it be fixed
>> (so that cds stays in the server   jvm-feature list) ?
>>>
>>> Thanks, Matthias
>>>


From ioi.lam at oracle.com  Thu Jan 16 18:48:38 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Thu, 16 Jan 2020 10:48:38 -0800
Subject: configuring with --with-jvm-variants=minimal,server makes cds
 disappear in server
In-Reply-To: <766f2d58-e8eb-3c89-b8f7-4294723135d9@oracle.com>
References: <AM6PR02MB42005B5EA3E77330E579354F93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <eed77ed5-9800-6a91-89f3-dc7c896cbd86@oracle.com>
 <AM6PR02MB420000FAB535AAC907A9E16493360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <766f2d58-e8eb-3c89-b8f7-4294723135d9@oracle.com>
Message-ID: <e8d7d7be-08f5-cdb9-1b76-a8a8c0c8a55b@oracle.com>



On 1/16/20 9:09 AM, Erik Joelsson wrote:
> I would lean towards changing the check from "if any of the variants 
> do not support CDS, disable it" to "if any of the variants support 
> CDS, enable it". The cds jvm feature is only added to the variants 
> that support it anyway.
>
> If you do this, you probably need to figure out how to explicitly only 
> generate the archive for the supported jvm variants in Images.gmk too. 
> Not sure if the archive is jvm variant dependent or if the layout even 
> supports multiple variants. It will likely get messy.

The CDS archive is dependent on the exact libjvm.so, so each JDK image 
should creates its own CDS archive during the build.

I think this can be handled emitting a different value for 
"BUILD_CDS_ARCHIVE" in the spec.gmk of each variant.

Thanks
- Ioi

>
> /Erik
>
> On 2020-01-16 08:22, Baesken, Matthias wrote:
>> Hello, thanks for looking into it .
>> Should I do a check just looking into single JVM configs, something like
>>
>> AC_DEFUN([HOTSPOT_IS_JVM_VARIANT],
>> [ [ [[ " $JVM_VARIANTS " == " $1 " ]] ] ])
>>
>> if HOTSPOT_IS_JVM_VARIANT(zero) || HOTSPOT_IS_JVM_VARIANT(minimal) || 
>> HOTSPOT_IS_JVM_VARIANT(core); then
>>
>> instead of
>>
>> AC_DEFUN([HOTSPOT_CHECK_JVM_VARIANT],
>> [ [ [[ " $JVM_VARIANTS " =~ " $1 " ]] ] ])
>>
>> if HOTSPOT_CHECK_JVM_VARIANT(zero) || 
>> HOTSPOT_CHECK_JVM_VARIANT(minimal) || 
>> HOTSPOT_CHECK_JVM_VARIANT(core); then
>>
>>
>> This should? remove the error in case of multi-JVM configs .
>> Or maybe? the whole check ( if HOTSPOT_CHECK_JVM_VARIANT(zero) ||? 
>> HOTSPOT_CHECK_JVM_VARIANT(minimal) || 
>> HOTSPOT_CHECK_JVM_VARIANT(core); then .... fi )
>> ? might be removed , I see not so much value in it? because? in case 
>> one sets --enable-jvm-features?? directly the check isn?t done .
>>
>> I opened? bug :
>>
>> https://bugs.openjdk.java.net/browse/JDK-8237374
>>
>> JDK-8237374 :? configuring with --with-jvm-variants=minimal,server 
>> makes cds disappear in server
>>
>>
>> Best regards, Matthias
>>
>>
>>
>>
>>> This does indeed look like a bug to me. At least at Oracle, we no 
>>> longer
>>> build any multi JVM configs regularly, so things like this falls 
>>> through
>>> the cracks.
>>>
>>> /Erik
>>>
>>> On 2020-01-16 02:18, Baesken, Matthias wrote:
>>>> Hello, I noticed the following strange "feature" (or is it a bug?) .
>>>> When? building 2 VM variants in one build? and? using
>>>>
>>>> ??? --with-jvm-variants=minimal,server
>>>>
>>>> For this, the build works nicely. But I notice that? in the server 
>>>> VM, cds? is
>>> removed.
>>>> Instead of
>>>>
>>>> checking if cds should be enabled... yes
>>>>
>>>> I get ( with the following patch that adds tracing)? :
>>>>
>>>> configure: WARNING: ENABLE_CDS set to false because we found a
>>> minimal, core or zero JVM.
>>>> checking if cds should be enabled... no
>>>> ??? ...
>>>>
>>>> * JVM features:?? minimal: 'compiler1 minimal serialgc'?????? 
>>>> server: 'compiler1
>>> compiler2 epsilongc g1gc jfr jni-check jvmti management nmt parallelgc
>>> serialgc services vm-structs'
>>>> (patch is
>>>>
>>>> --- a/make/autoconf/hotspot.m4? Wed Jan 15 21:20:40 2020 -0800
>>>> +++ b/make/autoconf/hotspot.m4? Thu Jan 16 10:24:43 2020 +0100
>>>> @@ -528,6 +528,7 @@
>>>> ??? if HOTSPOT_CHECK_JVM_VARIANT(zero) ||
>>> HOTSPOT_CHECK_JVM_VARIANT(minimal) ||
>>> HOTSPOT_CHECK_JVM_VARIANT(core); then
>>>> ?????? # ..except when the user explicitely requested it with 
>>>> --enable-jvm-
>>> features
>>>> ?????? if ! HOTSPOT_CHECK_JVM_FEATURE(cds); then
>>>> ???????? ENABLE_CDS="false"
>>>> +????? AC_MSG_WARN([ENABLE_CDS set to false because we found a
>>> minimal, core or zero JVM.])
>>>> ???????? if test "x$enable_cds" = "xyes"; then
>>>> ?????????? AC_MSG_ERROR([CDS not implemented for variants zero, 
>>>> minimal,
>>> core. Remove --enable-cds.])
>>>> ???????? Fi
>>>>
>>>>
>>>> Is it expected that? cds goes away in "server"?? when configuring? 
>>>> "--with-
>>> jvm-variants=minimal,server"?? ??? Looks like a bug to me, should it 
>>> be fixed
>>> (so that cds stays in the server?? jvm-feature list) ?
>>>>
>>>> Thanks, Matthias
>>>>



From erik.joelsson at oracle.com  Thu Jan 16 19:16:19 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 16 Jan 2020 11:16:19 -0800
Subject: configuring with --with-jvm-variants=minimal,server makes cds
 disappear in server
In-Reply-To: <e8d7d7be-08f5-cdb9-1b76-a8a8c0c8a55b@oracle.com>
References: <AM6PR02MB42005B5EA3E77330E579354F93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <eed77ed5-9800-6a91-89f3-dc7c896cbd86@oracle.com>
 <AM6PR02MB420000FAB535AAC907A9E16493360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <766f2d58-e8eb-3c89-b8f7-4294723135d9@oracle.com>
 <e8d7d7be-08f5-cdb9-1b76-a8a8c0c8a55b@oracle.com>
Message-ID: <f6ae4fc3-83f6-afa9-3247-39dbf9aa5794@oracle.com>

On 2020-01-16 10:48, Ioi Lam wrote:
>
>
> On 1/16/20 9:09 AM, Erik Joelsson wrote:
>> I would lean towards changing the check from "if any of the variants 
>> do not support CDS, disable it" to "if any of the variants support 
>> CDS, enable it". The cds jvm feature is only added to the variants 
>> that support it anyway.
>>
>> If you do this, you probably need to figure out how to explicitly 
>> only generate the archive for the supported jvm variants in 
>> Images.gmk too. Not sure if the archive is jvm variant dependent or 
>> if the layout even supports multiple variants. It will likely get messy.
>
> The CDS archive is dependent on the exact libjvm.so, so each JDK image 
> should creates its own CDS archive during the build.
>
> I think this can be handled emitting a different value for 
> "BUILD_CDS_ARCHIVE" in the spec.gmk of each variant.
>
Building with multiple JVM variants does not create separate JDK images, 
the libjvm.so files simply end up in:

lib/server/libjvm.so
lib/minimal/libjvm.so

I don't think the default archive takes that into account.

/Erik

> Thanks
> - Ioi
>
>>
>> /Erik
>>
>> On 2020-01-16 08:22, Baesken, Matthias wrote:
>>> Hello, thanks for looking into it .
>>> Should I do a check just looking into single JVM configs, something 
>>> like
>>>
>>> AC_DEFUN([HOTSPOT_IS_JVM_VARIANT],
>>> [ [ [[ " $JVM_VARIANTS " == " $1 " ]] ] ])
>>>
>>> if HOTSPOT_IS_JVM_VARIANT(zero) || HOTSPOT_IS_JVM_VARIANT(minimal) 
>>> || HOTSPOT_IS_JVM_VARIANT(core); then
>>>
>>> instead of
>>>
>>> AC_DEFUN([HOTSPOT_CHECK_JVM_VARIANT],
>>> [ [ [[ " $JVM_VARIANTS " =~ " $1 " ]] ] ])
>>>
>>> if HOTSPOT_CHECK_JVM_VARIANT(zero) || 
>>> HOTSPOT_CHECK_JVM_VARIANT(minimal) || 
>>> HOTSPOT_CHECK_JVM_VARIANT(core); then
>>>
>>>
>>> This should? remove the error in case of multi-JVM configs .
>>> Or maybe? the whole check ( if HOTSPOT_CHECK_JVM_VARIANT(zero) ||? 
>>> HOTSPOT_CHECK_JVM_VARIANT(minimal) || 
>>> HOTSPOT_CHECK_JVM_VARIANT(core); then .... fi )
>>> ? might be removed , I see not so much value in it? because in case 
>>> one sets --enable-jvm-features?? directly the check isn?t done .
>>>
>>> I opened? bug :
>>>
>>> https://bugs.openjdk.java.net/browse/JDK-8237374
>>>
>>> JDK-8237374 :? configuring with --with-jvm-variants=minimal,server 
>>> makes cds disappear in server
>>>
>>>
>>> Best regards, Matthias
>>>
>>>
>>>
>>>
>>>> This does indeed look like a bug to me. At least at Oracle, we no 
>>>> longer
>>>> build any multi JVM configs regularly, so things like this falls 
>>>> through
>>>> the cracks.
>>>>
>>>> /Erik
>>>>
>>>> On 2020-01-16 02:18, Baesken, Matthias wrote:
>>>>> Hello, I noticed the following strange "feature" (or is it a bug?) .
>>>>> When? building 2 VM variants in one build? and? using
>>>>>
>>>>> ??? --with-jvm-variants=minimal,server
>>>>>
>>>>> For this, the build works nicely. But I notice that? in the server 
>>>>> VM, cds? is
>>>> removed.
>>>>> Instead of
>>>>>
>>>>> checking if cds should be enabled... yes
>>>>>
>>>>> I get ( with the following patch that adds tracing)? :
>>>>>
>>>>> configure: WARNING: ENABLE_CDS set to false because we found a
>>>> minimal, core or zero JVM.
>>>>> checking if cds should be enabled... no
>>>>> ??? ...
>>>>>
>>>>> * JVM features:?? minimal: 'compiler1 minimal serialgc'?????? 
>>>>> server: 'compiler1
>>>> compiler2 epsilongc g1gc jfr jni-check jvmti management nmt parallelgc
>>>> serialgc services vm-structs'
>>>>> (patch is
>>>>>
>>>>> --- a/make/autoconf/hotspot.m4? Wed Jan 15 21:20:40 2020 -0800
>>>>> +++ b/make/autoconf/hotspot.m4? Thu Jan 16 10:24:43 2020 +0100
>>>>> @@ -528,6 +528,7 @@
>>>>> ??? if HOTSPOT_CHECK_JVM_VARIANT(zero) ||
>>>> HOTSPOT_CHECK_JVM_VARIANT(minimal) ||
>>>> HOTSPOT_CHECK_JVM_VARIANT(core); then
>>>>> ?????? # ..except when the user explicitely requested it with 
>>>>> --enable-jvm-
>>>> features
>>>>> ?????? if ! HOTSPOT_CHECK_JVM_FEATURE(cds); then
>>>>> ???????? ENABLE_CDS="false"
>>>>> +????? AC_MSG_WARN([ENABLE_CDS set to false because we found a
>>>> minimal, core or zero JVM.])
>>>>> ???????? if test "x$enable_cds" = "xyes"; then
>>>>> ?????????? AC_MSG_ERROR([CDS not implemented for variants zero, 
>>>>> minimal,
>>>> core. Remove --enable-cds.])
>>>>> ???????? Fi
>>>>>
>>>>>
>>>>> Is it expected that? cds goes away in "server"?? when configuring? 
>>>>> "--with-
>>>> jvm-variants=minimal,server"?? ??? Looks like a bug to me, should 
>>>> it be fixed
>>>> (so that cds stays in the server?? jvm-feature list) ?
>>>>>
>>>>> Thanks, Matthias
>>>>>
>


From matthias.baesken at sap.com  Fri Jan 17 06:47:36 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 06:47:36 +0000
Subject: RFR [XXS]: 8237382: Cleanup the OPT_SPEED_SRC file list in
 JvmFeatures.gmk
Message-ID: <AM6PR02MB420001BB07A53C75A12C2B2093310@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, please review this very small change .

It removes file that are not present any more from the  OPT_SPEED_SRC file list in JvmFeatures.gmk  .

( this is a list of files to be optimized for speed when we otherwise optimize for size  in the minimal-JVM build)


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8237382

http://cr.openjdk.java.net/~mbaesken/webrevs/8237382.0/

Thanks, Matthias



From matthias.baesken at sap.com  Fri Jan 17 08:07:49 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 08:07:49 +0000
Subject: RFR [XS]: 8237374: configuring with
 --with-jvm-variants=minimal,server makes cds disappear in server
Message-ID: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, please review  this small patch .

When building 2 VM variants   minimal and server  in one build and using

  --with-jvm-variants=minimal,server

to configure this setup  , the build works nicely. But I notice that in the server VM, cds is removed.
Instead of

checking if cds should be enabled... yes

I get (with some tracing added ) :

configure: WARNING: ENABLE_CDS set to false because we found a minimal, core or zero JVM.
checking if cds should be enabled... no

The checks in hotspot.m4 disables cds by error for the server v,  because it matches to minimal  in the string  "minimal,server"  .

The configure option  --with-jvm-variants=minimal,server   enables  a  multi-JVM variants build  (variable BUILDING_MULTIPLE_JVM_VARIANTS in hotspot.m4) .
This   special build is only  supported  for   VALID_MULTIPLE_JVM_VARIANTS="server client minimal" .
So  we  better do not  disable  cds  in  a  BUILDING_MULTIPLE_JVM_VARIANTS - build   (means minimal + server/client ) ;   minimal has cds disabled by default anyway.


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8237374


http://cr.openjdk.java.net/~mbaesken/webrevs/8237374.0/


Thanks, Matthias


From matthias.baesken at sap.com  Fri Jan 17 08:44:29 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 08:44:29 +0000
Subject: RFR:  8237192: Generate stripped/public pdbs on Windows for jdk images
Message-ID: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello, please review this change related to stripped/"public" pdb file generation on Windows .

Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).
This leads currently to bad native stack traces e.g. when crashes occur.
One reason not to deliver the full pdb files might be the large size of these files.

However there exist also "public" or stripped pdb files on Windows, see :

https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017

Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber" info in the native stacks but at least the function name+hex-offset is visible)
to delivering full pdbs in the JDK.

Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :
jvm.pdb : 73,1 MB / 9,46 MB
awt.pdb : 7,05 MB / 1,48 MB

The patch  adds  generation  of stripped  pdb files  to the Windows  build.
Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .


Bug/webrev :

https://bugs.openjdk.java.net/browse/JDK-8237192

http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/


Thanks, Matthias


From magnus.ihse.bursie at oracle.com  Fri Jan 17 09:22:15 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 17 Jan 2020 10:22:15 +0100
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <d4923478-38e3-60b5-9025-c9af9026c4d2@oracle.com>

On 2020-01-17 09:44, Baesken, Matthias wrote:
> Hello, please review this change related to stripped/"public" pdb file generation on Windows .
>
> Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).
> This leads currently to bad native stack traces e.g. when crashes occur.
> One reason not to deliver the full pdb files might be the large size of these files.
>
> However there exist also "public" or stripped pdb files on Windows, see :
>
> https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017
>
> Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber" info in the native stacks but at least the function name+hex-offset is visible)
> to delivering full pdbs in the JDK.
>
> Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :
> jvm.pdb : 73,1 MB / 9,46 MB
> awt.pdb : 7,05 MB / 1,48 MB
>
> The patch  adds  generation  of stripped  pdb files  to the Windows  build.
> Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8237192
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/

What is the extra payload of all the *.stripped.pdb files together?

/Magnus

>
>
> Thanks, Matthias



From matthias.baesken at sap.com  Fri Jan 17 09:25:01 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 09:25:01 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <d4923478-38e3-60b5-9025-c9af9026c4d2@oracle.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d4923478-38e3-60b5-9025-c9af9026c4d2@oracle.com>
Message-ID: <AM6PR02MB420098C02FEC93014868B68A93310@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello,  my example product build (64 bit Windows / VS2017) shows the following sizes for the uncompressed pdb files :
sum of size of all full pdbs : 117 MB (jvm.pdb is 73,1 MB )
sum of size of all stripped pdbs: 18,2 MB (jvm.pdb is 9,46 MB = ~ 50 % of all)

Best regards, Matthias


On 2020-01-17 09:44, Baesken, Matthias wrote:

Hello, please review this change related to stripped/"public" pdb file generation on Windows .



Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).

This leads currently to bad native stack traces e.g. when crashes occur.

One reason not to deliver the full pdb files might be the large size of these files.



However there exist also "public" or stripped pdb files on Windows, see :



https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017



Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber"<file://linenumber> info in the native stacks but at least the function name+hex-offset is visible)

to delivering full pdbs in the JDK.



Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :

jvm.pdb : 73,1 MB / 9,46 MB

awt.pdb : 7,05 MB / 1,48 MB



The patch  adds  generation  of stripped  pdb files  to the Windows  build.

Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .





Bug/webrev :



https://bugs.openjdk.java.net/browse/JDK-8237192



http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/

What is the extra payload of all the *.stripped.pdb files together?




From magnus.ihse.bursie at oracle.com  Fri Jan 17 09:31:06 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 17 Jan 2020 10:31:06 +0100
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
Message-ID: <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>

On 2020-01-16 10:30, David Holmes wrote:
> Hi Matthias,
>
> On 16/01/2020 6:10 pm, Baesken, Matthias wrote:
>> Hi David, sure we can introduce a way to switch this on/off.
>
> Thanks.
>
>> There is already something similar for the link-time optimization 
>> (flto) , see the feature
>>
>> JvmFeatures.gmk
>> 180 ifeq ($(call check-jvm-feature, link-time-opt), true)
>> 190 ifeq ($(call check-jvm-feature, link-time-opt), false)
>>
>> hotspot.m4
>> 29 static-build link-time-opt aot jfr"
>> 502 JVM_FEATURES_link_time_opt="link-time-opt"
>
> Yep familiar with that from Minimal VM and SE Embedded days :)
>
>> Should we have? "link-time-gc" additionally to?? " link-time-opt"? 
>> ??? (however it? would? be a? bit misleading?? that it is a "JVM" 
>> feature , but except linux s390x? it is only changing the build of 
>> the JDK libs) .
>
> I agree the definition of this as a "JVM" feature is a bit 
> odd/misleading. Perhaps the build folk have a suggestion on how to 
> refactor this kind of option into something more general? In the 
> meantime having link-time-gc sit alongside link-time-opt seems 
> acceptable to me.

We don't have the concept of "JDK features", akin to "JVM features". 
Maybe we should have. It's an idea worth exploring, anyway.

The way we currently do on/off features for the entire JDK is by using 
autoconf options. So, in this case, --enable-link-time-gc, or something 
like that. It might just as well be on by default, which gives us a 
--disable-link-time-gc instead. (I understand David's reservation not 
about this being the default, just that it is not possible to simply 
turn off.)

Matthias: Have a look at some recently added option to get an indication 
of the best practice in adding new options. There are some ways to 
easily make this incorrect, and our code is full of old examples that 
does this unnecessary complex, or downright wrong. (These should be 
fixed, and we should probably introduce a simpler API for doing this, 
and so on... I'll address those as soon as time permits.)

/Magnus


>
> Thanks,
> David
>
>> Best regards, Matthias
>>
>>
>>
>>>
>>> Hi Matthias,
>>>
>>> I have reservations about turning this on by default and with no way to
>>> control it. This seems like it should be something you have to 
>>> opt-in to
>>> initially while we gain some experience with it and ensure there are no
>>> unexpected side-effects. After that it could be enabled by default.
>>>
>>



From sgehwolf at redhat.com  Fri Jan 17 10:58:39 2020
From: sgehwolf at redhat.com (Severin Gehwolf)
Date: Fri, 17 Jan 2020 11:58:39 +0100
Subject: [8u] RFR: 8227397: Add --with-extra-asflags configure option
In-Reply-To: <c50d00dd-db86-4691-a6fc-ec6c85538f1e@oracle.com>
References: <62e03a27132a82b9f8f22c8fb16d21fba66a7bb6.camel@redhat.com>
 <c50d00dd-db86-4691-a6fc-ec6c85538f1e@oracle.com>
Message-ID: <2f0260a06a984d09e14fcef64d1b95036dbd3e23.camel@redhat.com>

Hi,

Could I get a second review from an JDK 8u Reviewer, please?

Thanks,
Severin

On Mon, 2019-09-30 at 11:36 +0200, Magnus Ihse Bursie wrote:
> On 2019-09-27 17:48, Severin Gehwolf wrote:
> > Hi,
> > 
> > Could I please get a review of this 8u build change backport which
> > adds
> > --with-extra-asflags to OpenJDK 8u. At Red Hat, we need to pass
> > certain
> > assembler only flags for some builds. For example "-Wa,--generate-
> > missing-build-notes=yes", to assembly files only. As the build
> > system
> > is different in 8u over 11u I've re-done the patch.
> > 
> > Bug: https://bugs.openjdk.java.net/browse/JDK-8227397
> > webrev: 
> > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8227397/jdk8/01/
>  Looks good to me.
> 
> /Magnus
> > Testing: Built with --with-extra-asflags=-Wa,--generate-missing-
> > build-
> > notes=yes on Linux x86_64, confirmed linux_x86_64.s gets assembled
> > with
> > the flag and only that file.
> > 
> > I've omitted the windows portion of passing as flags to the hotspot
> > build as I've no idea how. Contributions welcome if that should get
> > included.
> > 
> > Thoughts?
> > 
> > Thanks,
> > Severin
> > 
>  



From matthias.baesken at sap.com  Fri Jan 17 11:44:28 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 17 Jan 2020 11:44:28 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
Message-ID: <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>



  *   Matthias: Have a look at some recently added option to get an indication of the best practice in adding new options. There are some ways to easily make this incorrect

Hi Magnus, do you have a good/?best practice?  example  (not that I catch a bad one ? )  ?

Best regards, Matthias



On 2020-01-16 10:30, David Holmes wrote:
Hi Matthias,

On 16/01/2020 6:10 pm, Baesken, Matthias wrote:

Hi David, sure we can introduce a way to switch this on/off.

Thanks.


There is already something similar for the link-time optimization (flto) , see the feature

JvmFeatures.gmk
180 ifeq ($(call check-jvm-feature, link-time-opt), true)
190 ifeq ($(call check-jvm-feature, link-time-opt), false)

hotspot.m4
29 static-build link-time-opt aot jfr"
502 JVM_FEATURES_link_time_opt="link-time-opt"

Yep familiar with that from Minimal VM and SE Embedded days :)


Should we have  "link-time-gc"  additionally to   " link-time-opt"  ?   (however it  would  be a  bit misleading   that it is a "JVM" feature , but except linux s390x  it is only changing the build of the JDK libs) .

I agree the definition of this as a "JVM" feature is a bit odd/misleading. Perhaps the build folk have a suggestion on how to refactor this kind of option into something more general? In the meantime having link-time-gc sit alongside link-time-opt seems acceptable to me.

We don't have the concept of "JDK features", akin to "JVM features". Maybe we should have. It's an idea worth exploring, anyway.

The way we currently do on/off features for the entire JDK is by using autoconf options. So, in this case, --enable-link-time-gc, or something like that. It might just as well be on by default, which gives us a --disable-link-time-gc instead. (I understand David's reservation not about this being the default, just that it is not possible to simply turn off.)

Matthias: Have a look at some recently added option to get an indication of the best practice in adding new options. There are some ways to easily make this incorrect, and our code is full of old examples that does this unnecessary complex, or downright wrong. (These should be fixed, and we should probably introduce a simpler API for doing this, and so on... I'll address those as soon as time permits.)

/Magnus



From erik.joelsson at oracle.com  Fri Jan 17 14:09:05 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 17 Jan 2020 06:09:05 -0800
Subject: RFR [XXS]: 8237382: Cleanup the OPT_SPEED_SRC file list in
 JvmFeatures.gmk
In-Reply-To: <AM6PR02MB420001BB07A53C75A12C2B2093310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB420001BB07A53C75A12C2B2093310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <aaa794a1-80ac-41cf-880a-f9ae04bfb714@oracle.com>

Looks good.

/Erik

On 2020-01-16 22:47, Baesken, Matthias wrote:
> Hello, please review this very small change .
>
> It removes file that are not present any more from the  OPT_SPEED_SRC file list in JvmFeatures.gmk  .
>
> ( this is a list of files to be optimized for speed when we otherwise optimize for size  in the minimal-JVM build)
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8237382
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237382.0/
>
> Thanks, Matthias
>


From erik.joelsson at oracle.com  Fri Jan 17 14:17:34 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 17 Jan 2020 06:17:34 -0800
Subject: RFR [XS]: 8237374: configuring with
 --with-jvm-variants=minimal,server makes cds disappear in server
In-Reply-To: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <57f43f5a-ae37-5d98-e9e4-bb30c3607300@oracle.com>

Hello Matthias,

Using BUILDING_MULTIPLE_JVM_VARIANTS as condition is clever and happens 
to coincide with the set of variants that also support CDS, but I would 
say this correlation is incidental. I would still prefer an explicit 
test for if any of the variants that do support CDS is in the set of 
variants being built. This will make it much easier to read and 
understand the logic. Simply:

if ! HOTSPOT_CHECK_JVM_VARIANT(server) && ! 
HOTSPOT_CHECK_JVM_VARIANT(client); then
 ? ENABLE_CDS="false"
 ? ...

/Erik

On 2020-01-17 00:07, Baesken, Matthias wrote:
> Hello, please review  this small patch .
>
> When building 2 VM variants   minimal and server  in one build and using
>
>    --with-jvm-variants=minimal,server
>
> to configure this setup  , the build works nicely. But I notice that in the server VM, cds is removed.
> Instead of
>
> checking if cds should be enabled... yes
>
> I get (with some tracing added ) :
>
> configure: WARNING: ENABLE_CDS set to false because we found a minimal, core or zero JVM.
> checking if cds should be enabled... no
>
> The checks in hotspot.m4 disables cds by error for the server v,  because it matches to minimal  in the string  "minimal,server"  .
>
> The configure option  --with-jvm-variants=minimal,server   enables  a  multi-JVM variants build  (variable BUILDING_MULTIPLE_JVM_VARIANTS in hotspot.m4) .
> This   special build is only  supported  for   VALID_MULTIPLE_JVM_VARIANTS="server client minimal" .
> So  we  better do not  disable  cds  in  a  BUILDING_MULTIPLE_JVM_VARIANTS - build   (means minimal + server/client ) ;   minimal has cds disabled by default anyway.
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8237374
>
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237374.0/
>
>
> Thanks, Matthias


From erik.joelsson at oracle.com  Fri Jan 17 15:20:09 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 17 Jan 2020 07:20:09 -0800
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <eb0310f3-21e9-ae31-e797-542c17d9c858@oracle.com>

Hello Matthias,

Providing these stripped pdb files in the distribution is a good idea, 
but finding a good solution in the build is unfortunately more 
complicated than this. The JDK image we ship should (with very few 
exceptions) be the result of running jlink on all the jmods. If a user 
runs jlink and includes all the jmods we ship with the JDK, the result 
should be essentially equivalent to the original JDK image. The way the 
stripped pdb files are included in the bundles sort of at the last 
second of the build here breaks this property. Any user generated image 
would miss the stripped pdb files since they aren't packaged in the jmods.

To properly implement this, care will need to be taken to juggle the two 
sets of pdb files around, making sure each build and test use case has 
the correct one in place where and when it's needed. Quite possibly, we 
cannot cover all use cases with one build configuration. Developers 
needing the full debug symbols when debugging locally would likely need 
to disable the stripped symbols so they get the full symbols everywhere. 
Possibly this would need to be the default for debug builds and 
configurable for release builds.

/Erik

On 2020-01-17 00:44, Baesken, Matthias wrote:

> Hello, please review this change related to stripped/"public" pdb file generation on Windows .
>
> Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).
> This leads currently to bad native stack traces e.g. when crashes occur.
> One reason not to deliver the full pdb files might be the large size of these files.
>
> However there exist also "public" or stripped pdb files on Windows, see :
>
> https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017
>
> Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber" info in the native stacks but at least the function name+hex-offset is visible)
> to delivering full pdbs in the JDK.
>
> Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :
> jvm.pdb : 73,1 MB / 9,46 MB
> awt.pdb : 7,05 MB / 1,48 MB
>
> The patch  adds  generation  of stripped  pdb files  to the Windows  build.
> Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .
>
>
> Bug/webrev :
>
> https://bugs.openjdk.java.net/browse/JDK-8237192
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/
>
>
> Thanks, Matthias


From martinrb at google.com  Fri Jan 17 17:57:29 2020
From: martinrb at google.com (Martin Buchholz)
Date: Fri, 17 Jan 2020 09:57:29 -0800
Subject: [8u] RFR: 8227397: Add --with-extra-asflags configure option
In-Reply-To: <2f0260a06a984d09e14fcef64d1b95036dbd3e23.camel@redhat.com>
References: <62e03a27132a82b9f8f22c8fb16d21fba66a7bb6.camel@redhat.com>
 <c50d00dd-db86-4691-a6fc-ec6c85538f1e@oracle.com>
 <2f0260a06a984d09e14fcef64d1b95036dbd3e23.camel@redhat.com>
Message-ID: <CA+kOe08ZdgD=zxX9XnBThT0xpf4OBNKJeEQzmdR1C4cVhQ9nAw@mail.gmail.com>

LGTM

On Fri, Jan 17, 2020 at 2:59 AM Severin Gehwolf <sgehwolf at redhat.com> wrote:

> Hi,
>
> Could I get a second review from an JDK 8u Reviewer, please?
>
> Thanks,
> Severin
>
> On Mon, 2019-09-30 at 11:36 +0200, Magnus Ihse Bursie wrote:
> > On 2019-09-27 17:48, Severin Gehwolf wrote:
> > > Hi,
> > >
> > > Could I please get a review of this 8u build change backport which
> > > adds
> > > --with-extra-asflags to OpenJDK 8u. At Red Hat, we need to pass
> > > certain
> > > assembler only flags for some builds. For example "-Wa,--generate-
> > > missing-build-notes=yes", to assembly files only. As the build
> > > system
> > > is different in 8u over 11u I've re-done the patch.
> > >
> > > Bug: https://bugs.openjdk.java.net/browse/JDK-8227397
> > > webrev:
> > > http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8227397/jdk8/01/
> >  Looks good to me.
> >
> > /Magnus
> > > Testing: Built with --with-extra-asflags=-Wa,--generate-missing-
> > > build-
> > > notes=yes on Linux x86_64, confirmed linux_x86_64.s gets assembled
> > > with
> > > the flag and only that file.
> > >
> > > I've omitted the windows portion of passing as flags to the hotspot
> > > build as I've no idea how. Contributions welcome if that should get
> > > included.
> > >
> > > Thoughts?
> > >
> > > Thanks,
> > > Severin
> > >
> >
>
>


From david.holmes at oracle.com  Fri Jan 17 22:25:16 2020
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 18 Jan 2020 08:25:16 +1000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>

Hi Matthias,

This also needs to be a configurable option not one done by default as 
there can be non-technical issues relating to shipping symbol files in a 
product.

Thanks,
David

On 17/01/2020 6:44 pm, Baesken, Matthias wrote:
> Hello, please review this change related to stripped/"public" pdb file generation on Windows .
> 
> Currently the JDK bundle on Windows does not contain pdb files (full pdb files are in a separate symbols bundle).
> This leads currently to bad native stack traces e.g. when crashes occur.
> One reason not to deliver the full pdb files might be the large size of these files.
> 
> However there exist also "public" or stripped pdb files on Windows, see :
> 
> https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-private-symbols?view=vs-2017
> 
> Those are much smaller (often only 10-20% of the full pdb files) and they offer a good compromise (no "file:linenumber" info in the native stacks but at least the function name+hex-offset is visible)
> to delivering full pdbs in the JDK.
> 
> Example sizes for the currently built full pdbs / stripped pdbs from VS2017 based 64bit build of jdk/jdk :
> jvm.pdb : 73,1 MB / 9,46 MB
> awt.pdb : 7,05 MB / 1,48 MB
> 
> The patch  adds  generation  of stripped  pdb files  to the Windows  build.
> Additionally those files are put into the JDK  bundle    (while the symbols bundle still gets the full pdb files ) .
> 
> 
> Bug/webrev :
> 
> https://bugs.openjdk.java.net/browse/JDK-8237192
> 
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/
> 
> 
> Thanks, Matthias
> 


From gnu.andrew at redhat.com  Mon Jan 20 05:58:44 2020
From: gnu.andrew at redhat.com (Andrew John Hughes)
Date: Mon, 20 Jan 2020 05:58:44 +0000
Subject: [8u] RFR: 8227397: Add --with-extra-asflags configure option
In-Reply-To: <62e03a27132a82b9f8f22c8fb16d21fba66a7bb6.camel@redhat.com>
References: <62e03a27132a82b9f8f22c8fb16d21fba66a7bb6.camel@redhat.com>
Message-ID: <fa3df862-74c6-5395-66d1-cf492cea95f1@redhat.com>



On 27/09/2019 16:48, Severin Gehwolf wrote:
> Hi,
> 
> Could I please get a review of this 8u build change backport which adds
> --with-extra-asflags to OpenJDK 8u. At Red Hat, we need to pass certain
> assembler only flags for some builds. For example "-Wa,--generate-
> missing-build-notes=yes", to assembly files only. As the build system
> is different in 8u over 11u I've re-done the patch.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8227397
> webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8227397/jdk8/01/
> 
> Testing: Built with --with-extra-asflags=-Wa,--generate-missing-build-
> notes=yes on Linux x86_64, confirmed linux_x86_64.s gets assembled with
> the flag and only that file.
> 
> I've omitted the windows portion of passing as flags to the hotspot
> build as I've no idea how. Contributions welcome if that should get
> included.
> 
> Thoughts?
> 
> Thanks,
> Severin
> 

Looks ok to me.

As to the Windows side, it seems there are no ASFLAGS rules at all nor
any assembly files:

$ grep -r 'ASFLAGS' hotspot/make/
hotspot/make/aix/makefiles/xlc.make:ASFLAGS    += $(ARCHFLAG)
hotspot/make/aix/makefiles/rules.make:AS.S            = $(AS) $(ASFLAGS)
hotspot/make/solaris/makefiles/sparcWorks.make:ASFLAGS += $(AS_ARCHFLAG)
hotspot/make/solaris/makefiles/sparcv9.make:ASFLAGS += $(AS_ARCHFLAG)
hotspot/make/solaris/makefiles/sparc.make:ASFLAGS += $(AS_ARCHFLAG)
hotspot/make/solaris/makefiles/gcc.make:ASFLAGS    += $(ARCHFLAG)
hotspot/make/solaris/makefiles/gcc.make:ASFLAGS += -march=k8  -march=amd64
hotspot/make/solaris/makefiles/rules.make:AS.S            = $(AS) $(ASFLAGS)
hotspot/make/bsd/makefiles/sparcWorks.make:ASFLAGS    += $(ARCHFLAG)
hotspot/make/bsd/makefiles/gcc.make:ASFLAGS    += $(ARCHFLAG)
hotspot/make/bsd/makefiles/gcc.make:ASFLAGS += -x assembler-with-cpp
hotspot/make/bsd/makefiles/rules.make:AS.S            = $(AS) $(ASFLAGS)
hotspot/make/linux/makefiles/sparcWorks.make:ASFLAGS    += $(ARCHFLAG)
hotspot/make/linux/makefiles/gcc.make:ASFLAGS    += $(ARCHFLAG)
hotspot/make/linux/makefiles/gcc.make:  ASFLAGS    +=
$(ASFLAGS_DEBUG_SYMBOLS)
hotspot/make/linux/makefiles/rules.make:AS.S            = $(AS) $(ASFLAGS)

$ find hotspot/src/ -name '*.s'
hotspot/src/os_cpu/bsd_x86/vm/bsd_x86_64.s
hotspot/src/os_cpu/bsd_x86/vm/bsd_x86_32.s
hotspot/src/os_cpu/solaris_sparc/vm/solaris_sparc.s
hotspot/src/os_cpu/solaris_x86/vm/solaris_x86_32.s
hotspot/src/os_cpu/solaris_x86/vm/solaris_x86_64.s
hotspot/src/os_cpu/linux_x86/vm/linux_x86_64.s
hotspot/src/os_cpu/linux_x86/vm/linux_x86_32.s
hotspot/src/os_cpu/linux_sparc/vm/linux_sparc.s

so it looks like assembly support would need to be there first before
making this change on top.

Thanks,
-- 
Andrew :)

Senior Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net)
Fingerprint = 5132 579D D154 0ED2 3E04  C5A0 CFDA 0F9B 3596 4222
https://keybase.io/gnu_andrew



From martin.doerr at sap.com  Mon Jan 20 11:10:53 2020
From: martin.doerr at sap.com (Doerr, Martin)
Date: Mon, 20 Jan 2020 11:10:53 +0000
Subject: RFR [XXS]: 8237382: Cleanup the OPT_SPEED_SRC file list in
 JvmFeatures.gmk
In-Reply-To: <aaa794a1-80ac-41cf-880a-f9ae04bfb714@oracle.com>
References: <AM6PR02MB420001BB07A53C75A12C2B2093310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <aaa794a1-80ac-41cf-880a-f9ae04bfb714@oracle.com>
Message-ID: <AM0PR0202MB32977EE6D5E302FB82B4C0749A320@AM0PR0202MB3297.eurprd02.prod.outlook.com>

Hi Matthias,

thanks for removing no longer existing files from the list.
I guess the list will need further updates to become really useful, but your change looks good.

Best regards,
Martin


> -----Original Message-----
> From: hotspot-dev <hotspot-dev-bounces at openjdk.java.net> On Behalf Of
> Erik Joelsson
> Sent: Freitag, 17. Januar 2020 15:09
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'build-
> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR [XXS]: 8237382: Cleanup the OPT_SPEED_SRC file list in
> JvmFeatures.gmk
> 
> Looks good.
> 
> /Erik
> 
> On 2020-01-16 22:47, Baesken, Matthias wrote:
> > Hello, please review this very small change .
> >
> > It removes file that are not present any more from the  OPT_SPEED_SRC
> file list in JvmFeatures.gmk  .
> >
> > ( this is a list of files to be optimized for speed when we otherwise optimize
> for size  in the minimal-JVM build)
> >
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8237382
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8237382.0/
> >
> > Thanks, Matthias
> >

From matthias.baesken at sap.com  Tue Jan 21 10:02:48 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 21 Jan 2020 10:02:48 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
Message-ID: <AM6PR02MB4200C166288C01E34F192C93930D0@AM6PR02MB4200.eurprd02.prod.outlook.com>


Hi David ,  yes I think it makes sense to have a configure option for this .
Not everyone would like to have a larger JDK (even it is only a bit larger).

Best regards, Matthias


> 
> Hi Matthias,
> 
> This also needs to be a configurable option not one done by default as
> there can be non-technical issues relating to shipping symbol files in a
> product.
> 
> Thanks,
> David
> 
> On 17/01/2020 6:44 pm, Baesken, Matthias wrote:
> > Hello, please review this change related to stripped/"public" pdb file
> generation on Windows .
> >
> > Currently the JDK bundle on Windows does not contain pdb files (full pdb
> files are in a separate symbols bundle).
> > This leads currently to bad native stack traces e.g. when crashes occur.
> > One reason not to deliver the full pdb files might be the large size of these
> files.
> >
> > However there exist also "public" or stripped pdb files on Windows, see :
> >
> > https://docs.microsoft.com/en-us/cpp/build/reference/pdbstripped-strip-
> private-symbols?view=vs-2017
> >
> > Those are much smaller (often only 10-20% of the full pdb files) and they
> offer a good compromise (no "file:linenumber" info in the native stacks but
> at least the function name+hex-offset is visible)
> > to delivering full pdbs in the JDK.
> >
> > Example sizes for the currently built full pdbs / stripped pdbs from VS2017
> based 64bit build of jdk/jdk :
> > jvm.pdb : 73,1 MB / 9,46 MB
> > awt.pdb : 7,05 MB / 1,48 MB
> >
> > The patch  adds  generation  of stripped  pdb files  to the Windows  build.
> > Additionally those files are put into the JDK  bundle    (while the symbols
> bundle still gets the full pdb files ) .
> >
> >
> > Bug/webrev :
> >
> > https://bugs.openjdk.java.net/browse/JDK-8237192
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.0/
> >
> >
> > Thanks, Matthias
> >

From matthias.baesken at sap.com  Tue Jan 21 14:45:47 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Tue, 21 Jan 2020 14:45:47 +0000
Subject: RFR [XS]: 8237374: configuring with
 --with-jvm-variants=minimal,server makes cds disappear in server
In-Reply-To: <57f43f5a-ae37-5d98-e9e4-bb30c3607300@oracle.com>
References: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <57f43f5a-ae37-5d98-e9e4-bb30c3607300@oracle.com>
Message-ID: <AM6PR02MB4200CC0697112F35A6585DE6930D0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, new webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8237374.1/

Best regards, Matthias

-----Original Message-----
From: Erik Joelsson <erik.joelsson at oracle.com> 
Sent: Freitag, 17. Januar 2020 15:18
To: Baesken, Matthias <matthias.baesken at sap.com>; 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: Re: RFR [XS]: 8237374: configuring with --with-jvm-variants=minimal,server makes cds disappear in server

Hello Matthias,

Using BUILDING_MULTIPLE_JVM_VARIANTS as condition is clever and happens 
to coincide with the set of variants that also support CDS, but I would 
say this correlation is incidental. I would still prefer an explicit 
test for if any of the variants that do support CDS is in the set of 
variants being built. This will make it much easier to read and 
understand the logic. Simply:

if ! HOTSPOT_CHECK_JVM_VARIANT(server) && ! 
HOTSPOT_CHECK_JVM_VARIANT(client); then
 ? ENABLE_CDS="false"
 ? ...

/Erik



From erik.joelsson at oracle.com  Tue Jan 21 16:35:20 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Tue, 21 Jan 2020 08:35:20 -0800
Subject: RFR [XS]: 8237374: configuring with
 --with-jvm-variants=minimal,server makes cds disappear in server
In-Reply-To: <AM6PR02MB4200CC0697112F35A6585DE6930D0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200A055393D4CA7B3FA06E393310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <57f43f5a-ae37-5d98-e9e4-bb30c3607300@oracle.com>
 <AM6PR02MB4200CC0697112F35A6585DE6930D0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <2f32b930-79b2-d043-faf3-ba3975b72af1@oracle.com>

That looks good.

/Erik


On 2020-01-21 06:45, Baesken, Matthias wrote:
> Hi Erik, new webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237374.1/
>
> Best regards, Matthias
>
> -----Original Message-----
> From: Erik Joelsson <erik.joelsson at oracle.com>
> Sent: Freitag, 17. Januar 2020 15:18
> To: Baesken, Matthias <matthias.baesken at sap.com>; 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: Re: RFR [XS]: 8237374: configuring with --with-jvm-variants=minimal,server makes cds disappear in server
>
> Hello Matthias,
>
> Using BUILDING_MULTIPLE_JVM_VARIANTS as condition is clever and happens
> to coincide with the set of variants that also support CDS, but I would
> say this correlation is incidental. I would still prefer an explicit
> test for if any of the variants that do support CDS is in the set of
> variants being built. This will make it much easier to read and
> understand the logic. Simply:
>
> if ! HOTSPOT_CHECK_JVM_VARIANT(server) && !
> HOTSPOT_CHECK_JVM_VARIANT(client); then
>   ? ENABLE_CDS="false"
>   ? ...
>
> /Erik
>
>


From joe.darcy at oracle.com  Tue Jan 21 17:54:10 2020
From: joe.darcy at oracle.com (Joe Darcy)
Date: Tue, 21 Jan 2020 09:54:10 -0800
Subject: JDK-8237248: Update --release 14 symbol information for JDK 14 b32
Message-ID: <07b9876d-d771-bd3c-c105-124f2d181855@oracle.com>

Hello,

Another JDK 14 build, another update to the --release 14 information in 15:

 ??? http://cr.openjdk.java.net/~darcy/8237248.0/

Thanks,

-Joe



From jan.lahoda at oracle.com  Tue Jan 21 20:54:55 2020
From: jan.lahoda at oracle.com (Jan Lahoda)
Date: Tue, 21 Jan 2020 21:54:55 +0100
Subject: JDK-8237248: Update --release 14 symbol information for JDK 14 b32
In-Reply-To: <07b9876d-d771-bd3c-c105-124f2d181855@oracle.com>
References: <07b9876d-d771-bd3c-c105-124f2d181855@oracle.com>
Message-ID: <bc66d30a-c546-a372-4893-83e62a37f8b3@oracle.com>

Looks OK to me.

Jan

On 21. 01. 20 18:54, Joe Darcy wrote:
> Hello,
> 
> Another JDK 14 build, another update to the --release 14 information in 15:
> 
>  ??? http://cr.openjdk.java.net/~darcy/8237248.0/
> 
> Thanks,
> 
> -Joe
> 


From matthias.baesken at sap.com  Wed Jan 22 13:33:24 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 22 Jan 2020 13:33:24 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Magnus / David,  here is a new webrev :


http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/


it supports now a  configure switch  --enable-linktime-gc=yes  that needs to be set  to enable the link time section gc  .

Exception is linuxs390x  where we already have the  feature enabled  (and keep it enabled always for LIB_JVM).

Best regards, Matthias



From: Baesken, Matthias
Sent: Freitag, 17. Januar 2020 12:44
To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David Holmes <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
Subject: RE: RFR: 8236714: enable link-time section-gc for linux to remove unused code



  *   Matthias: Have a look at some recently added option to get an indication of the best practice in adding new options. There are some ways to easily make this incorrect

Hi Magnus, do you have a good/?best practice?  example  (not that I catch a bad one ? )  ?

Best regards, Matthias


From erik.joelsson at oracle.com  Wed Jan 22 14:38:55 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 22 Jan 2020 06:38:55 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>

Hello Matthias,

You can keep the setting up of all the flags in flags-cflags.m4 and 
flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can also 
default the value of this new parameter to true for s390x to keep the 
current behavior for that platform. As it is in this patch, the JVM 
flags for s390x are setup in configure while the JDK flags are in make, 
which gets confusing I think.

/Erik


On 2020-01-22 05:33, Baesken, Matthias wrote:
> Hi Magnus / David,  here is a new webrev :
>
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
>
>
> it supports now a  configure switch  --enable-linktime-gc=yes  that needs to be set  to enable the link time section gc  .
>
> Exception is linuxs390x  where we already have the  feature enabled  (and keep it enabled always for LIB_JVM).
>
> Best regards, Matthias
>
>
>
> From: Baesken, Matthias
> Sent: Freitag, 17. Januar 2020 12:44
> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David Holmes <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to remove unused code
>
>
>
>    *   Matthias: Have a look at some recently added option to get an indication of the best practice in adding new options. There are some ways to easily make this incorrect
>
> Hi Magnus, do you have a good/?best practice?  example  (not that I catch a bad one ? )  ?
>
> Best regards, Matthias
>


From matthias.baesken at sap.com  Wed Jan 22 15:46:48 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 22 Jan 2020 15:46:48 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com> 
Message-ID: <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hello,  here is an updated version  :

http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/

this one supports a configure switch  "--enable-stripped-pdbs"    to enable the feature .

Best regards, Matthias


> -----Original Message-----
> From: Baesken, Matthias
> Sent: Dienstag, 21. Januar 2020 11:03
> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for
> jdk images
> 
> 
> Hi David ,  yes I think it makes sense to have a configure option for this .
> Not everyone would like to have a larger JDK (even it is only a bit larger).
> 


From matthias.baesken at sap.com  Wed Jan 22 16:01:59 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Wed, 22 Jan 2020 16:01:59 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
Message-ID: <AM6PR02MB4200FB39C1F2AA0C1B6D8279930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>


Hi Erik, okay I'll check that .
I had the impression I would have ordering  issues of the m4 files and how they end up in the generated-configure.sh  but  looks like that?s not the case .

Best regards, Matthias


> 
> Hello Matthias,
> 
> You can keep the setting up of all the flags in flags-cflags.m4 and
> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can also
> default the value of this new parameter to true for s390x to keep the
> current behavior for that platform. As it is in this patch, the JVM
> flags for s390x are setup in configure while the JDK flags are in make,
> which gets confusing I think.
> 
> /Erik
> 


From erik.joelsson at oracle.com  Wed Jan 22 16:36:50 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 22 Jan 2020 08:36:50 -0800
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
 <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>

This still does not address anything in my objection.

/Erik

On 2020-01-22 07:46, Baesken, Matthias wrote:
> Hello,  here is an updated version  :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/
>
> this one supports a configure switch  "--enable-stripped-pdbs"    to enable the feature .
>
> Best regards, Matthias
>
>
>> -----Original Message-----
>> From: Baesken, Matthias
>> Sent: Dienstag, 21. Januar 2020 11:03
>> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
>> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
>> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for
>> jdk images
>>
>>
>> Hi David ,  yes I think it makes sense to have a configure option for this .
>> Not everyone would like to have a larger JDK (even it is only a bit larger).
>>


From matthias.baesken at sap.com  Thu Jan 23 08:03:10 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 23 Jan 2020 08:03:10 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
 <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>
Message-ID: <AM6PR02MB420064F6F67387E6596F0E67930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik,  yes true sorry for answering  your comments a bit late .

> If a user runs jlink and includes all the jmods we ship with the JDK, the result 
> should be essentially equivalent to the original JDK image. The way the 
> stripped pdb files are included in the bundles sort of at the last 
> second of the build here breaks this property.

I think we should address this in a separate bug/CR .
Looking for example  into a Linux  build, I see  a lot of debuginfo files  in the jdk image (more or less for every shared lib)  .
But when looking into the jmods  of that jdk image ,  no debuginfo files are in there ( I checked the java.base jmod).
So  putting  the  files with debug information into the jmods  seems to be something that was not done so far  cross platform  (or is there some build switch  for it that I did not check?) .
Maybe to keep the jmods as small as possible .


> To properly implement this, care will need to be taken to juggle the two 
> sets of pdb files around, making sure each build and test use case has 
> the correct one in place where and when it's needed. Quite possibly, we 
> cannot cover all use cases with one build configuration. Developers 
> needing the full debug symbols when debugging locally would likely need 
> to disable the stripped symbols so they get the full symbols everywhere. 
> Possibly this would need to be the default for debug builds and 
> configurable for release builds.

From my limited experience , the developers  do not work with the bundles (that  would contain now after my patch the stripped pds)  but with a "normal" jdk image that  is created my make all.

Best regards, Matthias



> 
> This still does not address anything in my objection.
> 
> /Erik
> 
> On 2020-01-22 07:46, Baesken, Matthias wrote:
> > Hello,  here is an updated version  :
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/
> >
> > this one supports a configure switch  "--enable-stripped-pdbs"    to enable
> the feature .
> >
> > Best regards, Matthias
> >
> >
> >> -----Original Message-----
> >> From: Baesken, Matthias
> >> Sent: Dienstag, 21. Januar 2020 11:03
> >> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
> >> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
> >> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> >> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for
> >> jdk images
> >>
> >>
> >> Hi David ,  yes I think it makes sense to have a configure option for this .
> >> Not everyone would like to have a larger JDK (even it is only a bit larger).
> >>

From matthias.baesken at sap.com  Thu Jan 23 13:15:28 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Thu, 23 Jan 2020 13:15:28 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
Message-ID: <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik,  new webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/

I moved the settings back into the  m4 files .

Best regards, Matthias

> 
> Hello Matthias,
> 
> You can keep the setting up of all the flags in flags-cflags.m4 and
> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can also
> default the value of this new parameter to true for s390x to keep the
> current behavior for that platform. As it is in this patch, the JVM
> flags for s390x are setup in configure while the JDK flags are in make,
> which gets confusing I think.
> 
> /Erik
> 
> 
> On 2020-01-22 05:33, Baesken, Matthias wrote:
> > Hi Magnus / David,  here is a new webrev :
> >
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
> >
> >
> > it supports now a  configure switch  --enable-linktime-gc=yes  that needs to
> be set  to enable the link time section gc  .
> >
> > Exception is linuxs390x  where we already have the  feature enabled  (and
> keep it enabled always for LIB_JVM).
> >
> > Best regards, Matthias
> >
> >
> >
> > From: Baesken, Matthias
> > Sent: Freitag, 17. Januar 2020 12:44
> > To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David Holmes
> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
> dev at openjdk.java.net>
> > Subject: RE: RFR: 8236714: enable link-time section-gc for linux to remove
> unused code
> >
> >
> >
> >    *   Matthias: Have a look at some recently added option to get an
> indication of the best practice in adding new options. There are some ways to
> easily make this incorrect
> >
> > Hi Magnus, do you have a good/?best practice?  example  (not that I catch a
> bad one ? )  ?
> >
> > Best regards, Matthias
> >

From erik.joelsson at oracle.com  Thu Jan 23 17:06:34 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 23 Jan 2020 09:06:34 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>

Hello,

That's better, but there are still some issues.

flags-cflags.m4

Code is repeated in both if and else block.

jdk-options.m4

The default is now true for all platforms. I would suggest moving the 
s390x conditional down into an elif after the elif for "no".

LibCommon.gmk

Please revert whole file.

/Erik

On 2020-01-23 05:15, Baesken, Matthias wrote:
> Hi Erik,  new webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/
>
> I moved the settings back into the  m4 files .
>
> Best regards, Matthias
>
>> Hello Matthias,
>>
>> You can keep the setting up of all the flags in flags-cflags.m4 and
>> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can also
>> default the value of this new parameter to true for s390x to keep the
>> current behavior for that platform. As it is in this patch, the JVM
>> flags for s390x are setup in configure while the JDK flags are in make,
>> which gets confusing I think.
>>
>> /Erik
>>
>>
>> On 2020-01-22 05:33, Baesken, Matthias wrote:
>>> Hi Magnus / David,  here is a new webrev :
>>>
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
>>>
>>>
>>> it supports now a  configure switch  --enable-linktime-gc=yes  that needs to
>> be set  to enable the link time section gc  .
>>> Exception is linuxs390x  where we already have the  feature enabled  (and
>> keep it enabled always for LIB_JVM).
>>> Best regards, Matthias
>>>
>>>
>>>
>>> From: Baesken, Matthias
>>> Sent: Freitag, 17. Januar 2020 12:44
>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David Holmes
>> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
>> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
>> dev at openjdk.java.net>
>>> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to remove
>> unused code
>>>
>>>
>>>     *   Matthias: Have a look at some recently added option to get an
>> indication of the best practice in adding new options. There are some ways to
>> easily make this incorrect
>>> Hi Magnus, do you have a good/?best practice?  example  (not that I catch a
>> bad one ? )  ?
>>> Best regards, Matthias
>>>


From erik.joelsson at oracle.com  Thu Jan 23 17:48:34 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Thu, 23 Jan 2020 09:48:34 -0800
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <AM6PR02MB420064F6F67387E6596F0E67930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
 <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>
 <AM6PR02MB420064F6F67387E6596F0E67930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <e9372084-eabc-8f57-981f-b95f9795582e@oracle.com>


On 2020-01-23 00:03, Baesken, Matthias wrote:
> Hi Erik,  yes true sorry for answering  your comments a bit late .
>
>> If a user runs jlink and includes all the jmods we ship with the JDK, the result
>> should be essentially equivalent to the original JDK image. The way the
>> stripped pdb files are included in the bundles sort of at the last
>> second of the build here breaks this property.
> I think we should address this in a separate bug/CR .
Maybe. I realize that my proposal below is quite a big task. But on the 
other hand, I don't think breaking the relationship between the jmods 
and the distribution bundles is on the table really.
> Looking for example  into a Linux  build, I see  a lot of debuginfo files  in the jdk image (more or less for every shared lib)  .
> But when looking into the jmods  of that jdk image ,  no debuginfo files are in there ( I checked the java.base jmod).
> So  putting  the  files with debug information into the jmods  seems to be something that was not done so far  cross platform  (or is there some build switch  for it that I did not check?) .
> Maybe to keep the jmods as small as possible .

No, we do not put the debuginfo files in the jmods nor the bundles 
because those are not intended to be shipped to customers. We are 
currently overlaying them into images/jdk in the build output to make 
them available for local debugging. (This is convoluted and I would very 
much like to get away from this practice at some point so that there is 
a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The 
stripped pdb files you are proposing are on the contrary intended for 
shipping to customers (as I understand your proposal) so comparing them 
with the debuginfo files is not relevant.

Now if MS had been kind enough to define a separate file type for the 
stripped pdbs, so that they could live alongside the full pdbs, we 
wouldn't have this issue. The heart of the problem is that only one set 
of files (either stripped or full) can be present and usable in 
images/jdk at a time. We have 2 main uses for images/jdk.

1. Developer running and debugging locally

2. Serve as the source for generating the distribution bundles

We currently have one image serving both of these purposes, which is 
already creating complicated and convoluted build steps. To properly 
solve this I would argue for splitting these two apart into two 
different images for the two different purposes. The build procedure 
would then be, first build the images for distribution, only containing 
what should go into each bundle. Then create the developer jdk image by 
copying files from the distribution images. On Windows, the first image 
would contain the stripped pdbs and when building the second, those 
would get overwritten with the full pdbs.

Now that I figured out a working model that would solve a bunch of other 
problems as well, I would love to implement it, but I doubt I will have 
time in the near future.

/Erik

>
>> To properly implement this, care will need to be taken to juggle the two
>> sets of pdb files around, making sure each build and test use case has
>> the correct one in place where and when it's needed. Quite possibly, we
>> cannot cover all use cases with one build configuration. Developers
>> needing the full debug symbols when debugging locally would likely need
>> to disable the stripped symbols so they get the full symbols everywhere.
>> Possibly this would need to be the default for debug builds and
>> configurable for release builds.
>  From my limited experience , the developers  do not work with the bundles (that  would contain now after my patch the stripped pds)  but with a "normal" jdk image that  is created my make all.
>
> Best regards, Matthias
>
>
>
>> This still does not address anything in my objection.
>>
>> /Erik
>>
>> On 2020-01-22 07:46, Baesken, Matthias wrote:
>>> Hello,  here is an updated version  :
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/
>>>
>>> this one supports a configure switch  "--enable-stripped-pdbs"    to enable
>> the feature .
>>> Best regards, Matthias
>>>
>>>
>>>> -----Original Message-----
>>>> From: Baesken, Matthias
>>>> Sent: Dienstag, 21. Januar 2020 11:03
>>>> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
>>>> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
>>>> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
>>>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows for
>>>> jdk images
>>>>
>>>>
>>>> Hi David ,  yes I think it makes sense to have a configure option for this .
>>>> Not everyone would like to have a larger JDK (even it is only a bit larger).
>>>>


From matthias.baesken at sap.com  Fri Jan 24 09:27:19 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 24 Jan 2020 09:27:19 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
Message-ID: <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, thanks for the comments, new webrev :

http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.7/

Best regards, Matthias



> Hello,
> 
> That's better, but there are still some issues.
> 
> flags-cflags.m4
> 
> Code is repeated in both if and else block.
> 
> jdk-options.m4
> 
> The default is now true for all platforms. I would suggest moving the
> s390x conditional down into an elif after the elif for "no".
> 
> LibCommon.gmk
> 
> Please revert whole file.
> 
> /Erik
> 
> On 2020-01-23 05:15, Baesken, Matthias wrote:
> > Hi Erik,  new webrev :
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/
> >
> > I moved the settings back into the  m4 files .
> >
> > Best regards, Matthias
> >
> >> Hello Matthias,
> >>
> >> You can keep the setting up of all the flags in flags-cflags.m4 and
> >> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can
> also
> >> default the value of this new parameter to true for s390x to keep the
> >> current behavior for that platform. As it is in this patch, the JVM
> >> flags for s390x are setup in configure while the JDK flags are in make,
> >> which gets confusing I think.
> >>
> >> /Erik
> >>
> >>
> >> On 2020-01-22 05:33, Baesken, Matthias wrote:
> >>> Hi Magnus / David,  here is a new webrev :
> >>>
> >>>
> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
> >>>
> >>>
> >>> it supports now a  configure switch  --enable-linktime-gc=yes  that needs
> to
> >> be set  to enable the link time section gc  .
> >>> Exception is linuxs390x  where we already have the  feature enabled
> (and
> >> keep it enabled always for LIB_JVM).
> >>> Best regards, Matthias
> >>>
> >>>
> >>>
> >>> From: Baesken, Matthias
> >>> Sent: Freitag, 17. Januar 2020 12:44
> >>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David
> Holmes
> >> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
> >> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
> >> dev at openjdk.java.net>
> >>> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to
> remove
> >> unused code
> >>>
> >>>
> >>>     *   Matthias: Have a look at some recently added option to get an
> >> indication of the best practice in adding new options. There are some
> ways to
> >> easily make this incorrect
> >>> Hi Magnus, do you have a good/?best practice?  example  (not that I
> catch a
> >> bad one ? )  ?
> >>> Best regards, Matthias
> >>>

From magnus.ihse.bursie at oracle.com  Fri Jan 24 10:40:13 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 24 Jan 2020 11:40:13 +0100
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
 <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <d4ee5f0f-ab52-e77e-a44a-b21f7e554743@oracle.com>

On 2020-01-24 10:27, Baesken, Matthias wrote:
> Hi Erik, thanks for the comments, new webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.7/
Looks good to me. Sorry for not being able to provide an example in a 
timely matter. Good thing you found one yourself. :)

/Magnus
>
> Best regards, Matthias
>
>
>
>> Hello,
>>
>> That's better, but there are still some issues.
>>
>> flags-cflags.m4
>>
>> Code is repeated in both if and else block.
>>
>> jdk-options.m4
>>
>> The default is now true for all platforms. I would suggest moving the
>> s390x conditional down into an elif after the elif for "no".
>>
>> LibCommon.gmk
>>
>> Please revert whole file.
>>
>> /Erik
>>
>> On 2020-01-23 05:15, Baesken, Matthias wrote:
>>> Hi Erik,  new webrev :
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/
>>>
>>> I moved the settings back into the  m4 files .
>>>
>>> Best regards, Matthias
>>>
>>>> Hello Matthias,
>>>>
>>>> You can keep the setting up of all the flags in flags-cflags.m4 and
>>>> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can
>> also
>>>> default the value of this new parameter to true for s390x to keep the
>>>> current behavior for that platform. As it is in this patch, the JVM
>>>> flags for s390x are setup in configure while the JDK flags are in make,
>>>> which gets confusing I think.
>>>>
>>>> /Erik
>>>>
>>>>
>>>> On 2020-01-22 05:33, Baesken, Matthias wrote:
>>>>> Hi Magnus / David,  here is a new webrev :
>>>>>
>>>>>
>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
>>>>>
>>>>>
>>>>> it supports now a  configure switch  --enable-linktime-gc=yes  that needs
>> to
>>>> be set  to enable the link time section gc  .
>>>>> Exception is linuxs390x  where we already have the  feature enabled
>> (and
>>>> keep it enabled always for LIB_JVM).
>>>>> Best regards, Matthias
>>>>>
>>>>>
>>>>>
>>>>> From: Baesken, Matthias
>>>>> Sent: Freitag, 17. Januar 2020 12:44
>>>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David
>> Holmes
>>>> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
>>>> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
>>>> dev at openjdk.java.net>
>>>>> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to
>> remove
>>>> unused code
>>>>>
>>>>>      *   Matthias: Have a look at some recently added option to get an
>>>> indication of the best practice in adding new options. There are some
>> ways to
>>>> easily make this incorrect
>>>>> Hi Magnus, do you have a good/?best practice?  example  (not that I
>> catch a
>>>> bad one ? )  ?
>>>>> Best regards, Matthias
>>>>>



From matthias.baesken at sap.com  Fri Jan 24 13:36:05 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 24 Jan 2020 13:36:05 +0000
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <d4ee5f0f-ab52-e77e-a44a-b21f7e554743@oracle.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
 <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d4ee5f0f-ab52-e77e-a44a-b21f7e554743@oracle.com>
Message-ID: <AM6PR02MB4200830D38B0FBF3F89347C0930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Thanks for the review !
Erik,  are you fine with the latest change too ?

Best regards, Matthias



> On 2020-01-24 10:27, Baesken, Matthias wrote:
> > Hi Erik, thanks for the comments, new webrev :
> >
> > http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.7/
> Looks good to me. Sorry for not being able to provide an example in a
> timely matter. Good thing you found one yourself. :)
> 
> /Magnus
> >
> > Best regards, Matthias


From matthias.baesken at sap.com  Fri Jan 24 13:48:16 2020
From: matthias.baesken at sap.com (Baesken, Matthias)
Date: Fri, 24 Jan 2020 13:48:16 +0000
Subject: RFR: 8237192: Generate stripped/public pdbs on Windows for jdk
 images
In-Reply-To: <e9372084-eabc-8f57-981f-b95f9795582e@oracle.com>
References: <AM6PR02MB4200E26AF7FD64841475148893310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <215dd116-d0f6-6e35-6b02-20d3986fa85a@oracle.com>
 <AM6PR02MB42007DB6FF70591EC151174A930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <df71f609-64a6-4997-0ff8-562a81be6134@oracle.com>
 <AM6PR02MB420064F6F67387E6596F0E67930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e9372084-eabc-8f57-981f-b95f9795582e@oracle.com>
Message-ID: <AM6PR02MB4200DAC4C1A1D363F8049D05930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>

Hi Erik, maybe we can just rename the configure option to

--enable-stripped-pdbs-for-bundle

AND make the default  = no/false .
Then without setting the configure flag,   everything stays as it is  for JDK vendors/distributors  who do not want  the stripped pdbs in the bundle.

Others who set the flag,  have to "teach"  the developers that the bundle already contains stripped pdbs that need to be replaced by full/"private" pdbs  in case better  symbols/stacks are wanted .
I think that?s a good compromise.

> Now if MS had been kind enough to define a separate file type for the
> stripped pdbs, so that they could live alongside the full pdbs, we
> wouldn't have this issue.

Unfortunately that seems not to work,  I tried to use the stripped pdb-files with another extension  but no success ?  .

An alternative could be to create 2 bundles  when  "--enable-stripped-pdbs-for-bundle" is set to yes , one with one without   stripped pdbs . 

Best regards, Matthias


> 
> On 2020-01-23 00:03, Baesken, Matthias wrote:
> > Hi Erik,  yes true sorry for answering  your comments a bit late .
> >
> >> If a user runs jlink and includes all the jmods we ship with the JDK, the
> result
> >> should be essentially equivalent to the original JDK image. The way the
> >> stripped pdb files are included in the bundles sort of at the last
> >> second of the build here breaks this property.
> > I think we should address this in a separate bug/CR .
> Maybe. I realize that my proposal below is quite a big task. But on the
> other hand, I don't think breaking the relationship between the jmods
> and the distribution bundles is on the table really.
> > Looking for example  into a Linux  build, I see  a lot of debuginfo files  in the
> jdk image (more or less for every shared lib)  .
> > But when looking into the jmods  of that jdk image ,  no debuginfo files are
> in there ( I checked the java.base jmod).
> > So  putting  the  files with debug information into the jmods  seems to be
> something that was not done so far  cross platform  (or is there some build
> switch  for it that I did not check?) .
> > Maybe to keep the jmods as small as possible .
> 
> No, we do not put the debuginfo files in the jmods nor the bundles
> because those are not intended to be shipped to customers. We are
> currently overlaying them into images/jdk in the build output to make
> them available for local debugging. (This is convoluted and I would very
> much like to get away from this practice at some point so that there is
> a 1-1 mapping between images/jdk and bundles/jdk*-bin.tar.gz.) The
> stripped pdb files you are proposing are on the contrary intended for
> shipping to customers (as I understand your proposal) so comparing them
> with the debuginfo files is not relevant.
> 
> Now if MS had been kind enough to define a separate file type for the
> stripped pdbs, so that they could live alongside the full pdbs, we
> wouldn't have this issue. The heart of the problem is that only one set
> of files (either stripped or full) can be present and usable in
> images/jdk at a time. We have 2 main uses for images/jdk.
> 
> 1. Developer running and debugging locally
> 
> 2. Serve as the source for generating the distribution bundles
> 
> We currently have one image serving both of these purposes, which is
> already creating complicated and convoluted build steps. To properly
> solve this I would argue for splitting these two apart into two
> different images for the two different purposes. The build procedure
> would then be, first build the images for distribution, only containing
> what should go into each bundle. Then create the developer jdk image by
> copying files from the distribution images. On Windows, the first image
> would contain the stripped pdbs and when building the second, those
> would get overwritten with the full pdbs.
> 
> Now that I figured out a working model that would solve a bunch of other
> problems as well, I would love to implement it, but I doubt I will have
> time in the near future.
> 
> /Erik
> 
> >
> >> To properly implement this, care will need to be taken to juggle the two
> >> sets of pdb files around, making sure each build and test use case has
> >> the correct one in place where and when it's needed. Quite possibly, we
> >> cannot cover all use cases with one build configuration. Developers
> >> needing the full debug symbols when debugging locally would likely need
> >> to disable the stripped symbols so they get the full symbols everywhere.
> >> Possibly this would need to be the default for debug builds and
> >> configurable for release builds.
> >  From my limited experience , the developers  do not work with the
> bundles (that  would contain now after my patch the stripped pds)  but with
> a "normal" jdk image that  is created my make all.
> >
> > Best regards, Matthias
> >
> >
> >
> >> This still does not address anything in my objection.
> >>
> >> /Erik
> >>
> >> On 2020-01-22 07:46, Baesken, Matthias wrote:
> >>> Hello,  here is an updated version  :
> >>>
> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8237192.3/
> >>>
> >>> this one supports a configure switch  "--enable-stripped-pdbs"    to
> enable
> >> the feature .
> >>> Best regards, Matthias
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Baesken, Matthias
> >>>> Sent: Dienstag, 21. Januar 2020 11:03
> >>>> To: 'David Holmes' <david.holmes at oracle.com>; 'build-
> >>>> dev at openjdk.java.net' <build-dev at openjdk.java.net>; 'hotspot-
> >>>> dev at openjdk.java.net' <hotspot-dev at openjdk.java.net>
> >>>> Subject: RE: RFR: 8237192: Generate stripped/public pdbs on Windows
> for
> >>>> jdk images
> >>>>
> >>>>
> >>>> Hi David ,  yes I think it makes sense to have a configure option for this .
> >>>> Not everyone would like to have a larger JDK (even it is only a bit
> larger).
> >>>>

From erik.joelsson at oracle.com  Fri Jan 24 14:22:53 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 24 Jan 2020 06:22:53 -0800
Subject: RFR: 8236714: enable link-time section-gc for linux to remove
 unused code
In-Reply-To: <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
References: <AM6PR02MB42009BDB19745CD5E05360D093340@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <ef93cf0e-1e88-5515-415f-54606ec2790f@oracle.com>
 <AM6PR02MB4200C52E6B4ADF803043D37D93360@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <172d4100-5cd2-2a99-f940-8aa25414c87a@oracle.com>
 <5d4de23f-a89d-7217-1776-51a1f3611d9a@oracle.com>
 <AM6PR02MB4200B721BD0C7728B21E2C4193310@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <AM6PR02MB4200BCFF64D8C46340B17A6F930C0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <d3e3641f-d6ae-979d-d53f-93136da6d62a@oracle.com>
 <AM6PR02MB42000810C8986AA738DB5EA3930F0@AM6PR02MB4200.eurprd02.prod.outlook.com>
 <e770b014-2562-ed73-b258-d017d6f603a4@oracle.com>
 <AM6PR02MB4200F337259362841A4360E3930E0@AM6PR02MB4200.eurprd02.prod.outlook.com>
Message-ID: <4067336a-cd36-7ec3-97e7-00cd3b8e86ab@oracle.com>

Looks good.

/Erik

On 2020-01-24 01:27, Baesken, Matthias wrote:
> Hi Erik, thanks for the comments, new webrev :
>
> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.7/
>
> Best regards, Matthias
>
>
>
>> Hello,
>>
>> That's better, but there are still some issues.
>>
>> flags-cflags.m4
>>
>> Code is repeated in both if and else block.
>>
>> jdk-options.m4
>>
>> The default is now true for all platforms. I would suggest moving the
>> s390x conditional down into an elif after the elif for "no".
>>
>> LibCommon.gmk
>>
>> Please revert whole file.
>>
>> /Erik
>>
>> On 2020-01-23 05:15, Baesken, Matthias wrote:
>>> Hi Erik,  new webrev :
>>>
>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.6/
>>>
>>> I moved the settings back into the  m4 files .
>>>
>>> Best regards, Matthias
>>>
>>>> Hello Matthias,
>>>>
>>>> You can keep the setting up of all the flags in flags-cflags.m4 and
>>>> flags-ldflags.m4 based on the value of ENABLE_LINKTIME_GC. You can
>> also
>>>> default the value of this new parameter to true for s390x to keep the
>>>> current behavior for that platform. As it is in this patch, the JVM
>>>> flags for s390x are setup in configure while the JDK flags are in make,
>>>> which gets confusing I think.
>>>>
>>>> /Erik
>>>>
>>>>
>>>> On 2020-01-22 05:33, Baesken, Matthias wrote:
>>>>> Hi Magnus / David,  here is a new webrev :
>>>>>
>>>>>
>>>>> http://cr.openjdk.java.net/~mbaesken/webrevs/8236714.4/
>>>>>
>>>>>
>>>>> it supports now a  configure switch  --enable-linktime-gc=yes  that needs
>> to
>>>> be set  to enable the link time section gc  .
>>>>> Exception is linuxs390x  where we already have the  feature enabled
>> (and
>>>> keep it enabled always for LIB_JVM).
>>>>> Best regards, Matthias
>>>>>
>>>>>
>>>>>
>>>>> From: Baesken, Matthias
>>>>> Sent: Freitag, 17. Januar 2020 12:44
>>>>> To: Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com>; David
>> Holmes
>>>> <david.holmes at oracle.com>; 'build-dev at openjdk.java.net' <build-
>>>> dev at openjdk.java.net>; 'hotspot-dev at openjdk.java.net' <hotspot-
>>>> dev at openjdk.java.net>
>>>>> Subject: RE: RFR: 8236714: enable link-time section-gc for linux to
>> remove
>>>> unused code
>>>>>
>>>>>      *   Matthias: Have a look at some recently added option to get an
>>>> indication of the best practice in adding new options. There are some
>> ways to
>>>> easily make this incorrect
>>>>> Hi Magnus, do you have a good/?best practice?  example  (not that I
>> catch a
>>>> bad one ? )  ?
>>>>> Best regards, Matthias
>>>>>


From sundararajan.athijegannathan at oracle.com  Mon Jan 27 08:37:27 2020
From: sundararajan.athijegannathan at oracle.com (sundararajan.athijegannathan at oracle.com)
Date: Mon, 27 Jan 2020 14:07:27 +0530
Subject: gnu make 4.3 breaks the build
In-Reply-To: <26ec1533-e3be-c045-9ab5-34be9bbe3972@gmail.com>
References: <26ec1533-e3be-c045-9ab5-34be9bbe3972@gmail.com>
Message-ID: <fc236d5c-7255-dd48-886f-43b197aee8fc@oracle.com>

CC: build-dev

On 27/01/20 4:55 am, Michael Zucchi wrote:
>
> I'm sure if it hasn't it'll come up on the other openjdk lists, but 
> this is the only one i'm on so i'll post it here.
>
> slackware-current updated gnu make to 4.3 a few days ago and it breaks 
> the build.? Make just fails immediately with some odd missing targets.
>
> It took me a while to track down so I thought I'd let everyone know.
>


From magnus.ihse.bursie at oracle.com  Mon Jan 27 13:10:12 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Mon, 27 Jan 2020 14:10:12 +0100
Subject: gnu make 4.3 breaks the build
In-Reply-To: <fc236d5c-7255-dd48-886f-43b197aee8fc@oracle.com>
References: <26ec1533-e3be-c045-9ab5-34be9bbe3972@gmail.com>
 <fc236d5c-7255-dd48-886f-43b197aee8fc@oracle.com>
Message-ID: <9958e60b-2137-6627-0acd-c5364ef6e494@oracle.com>

On 2020-01-27 09:37, sundararajan.athijegannathan at oracle.com wrote:
> CC: build-dev
>
> On 27/01/20 4:55 am, Michael Zucchi wrote:
>>
>> I'm sure if it hasn't it'll come up on the other openjdk lists, but 
>> this is the only one i'm on so i'll post it here.
>>
>> slackware-current updated gnu make to 4.3 a few days ago and it 
>> breaks the build.? Make just fails immediately with some odd missing 
>> targets.
>>
>> It took me a while to track down so I thought I'd let everyone know.
>>
Thanks Michael for the report and Sundararajan for cc:ing build-dev.

I have now opened https://bugs.openjdk.java.net/browse/JDK-8237879.

/Magnus


From ncopa at alpinelinux.org  Mon Jan 27 11:37:16 2020
From: ncopa at alpinelinux.org (Natanael Copa)
Date: Mon, 27 Jan 2020 12:37:16 +0100
Subject: gnu make 4.3 breaks the build
Message-ID: <20200127123716.4dc90778@ncopa-desktop.copa.dup.pw>

Hi,

this is a follow up on https://mail.openjdk.java.net/pipermail/build-dev/2020-January/026635.html

Alpine linux has bumped into the same issue and it sort of blocking the
security update. I would rather try fix the make issue than roll back
to make 4.2.1.

Downstream mergerequest/report:
https://gitlab.alpinelinux.org/alpine/aports/merge_requests/3224

It fails on different location even if I run it with JOBS=1, so to
track it down I had to run make clean between each run.

This is the command that fails (I have manually inserted the --debug --trace options):

$ (cd /home/ncopa/aports/community/openjdk11/src/
jdk11u-jdk-11.0.6+10/make && /usr/bin/make --debug --trace -s -r -R -I /home/ncopa/aports/community/openjdk11/src/jdk11u
-jdk-11.0.6+10/make/common SPEC=/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-norma
l-server-release/spec.gmk MAKE_LOG_FLAGS="-s"  -f ToolsLangtools.gmk)
GNU Make 4.3
Built for x86_64-alpine-linux-musl
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html> This is free software: you are free
to change and redistribute it. There is NO WARRANTY, to the extent
permitted by law. Reading makefiles...
Updating makefiles....
Updating goal targets....
 File 'default' does not exist.
   File 'all' does not exist.
     File
'/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties'
does not exist. Must remake target
'/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties'.
ToolsLangtools.gmk:37: update target
'/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties'
due to:
/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/resources/templates.properties
/bin/mkdir -p
/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources
/bin/cp
/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/resources/templates.properties
/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties
/bin/chmod -f ug+w
/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties
Successfully remade target file
'/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties'.
File
'/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch'
does not exist. make: *** No rule to make target
'/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps',
needed by
'/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch'.
 Stop.



If I re-run the command (without make clean) it will succeed:
GNU Make 4.3
Built for x86_64-alpine-linux-musl
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Updating makefiles....
Updating goal targets....
 File 'default' does not exist.
   File 'all' does not exist.
     File '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch' does not exist.
    Must remake target '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch'.
Compiling 8 files for BUILD_TOOLS_LANGTOOLS
ToolsLangtools.gmk:37: update target '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch' due to: /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/compileproperties/CompileProperties.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/PropertiesParser.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/gen/ClassGenerator.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/Message.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/MessageFile.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/MessageInfo.java /home/ncopa/aports/community
 /openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/MessageLine.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/MessageType.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps
( /bin/rm -f /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.log && /usr/lib/jvm/java-10-openjdk/bin/javac -g -J-XX:+UseSerialGC -J-Xms32M -J-Xmx512M -J-XX:TieredStopAtLevel=1 -source 9 -target 9 -XDignore.symbol.file=true -g -Xlint:all,-deprecation,-options -Werror -Xprefer:source -implicit:none -d /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes @/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.tmp > >(/usr/bin/tee -a /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.log) 2> >(/usr/bin/tee -a /home/ncopa/aports/community/openjd
 k11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.log >&2) || ( exitcode=$? && /bin/cp /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.log /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/make-support/failure-logs/buildtools_langtools_tools_classes__the.BUILD_TOOLS_LANGTOOLS_batch.log && /bin/cp /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.cmdline /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/make-support/failure-logs/buildtools_langtools_tools_classes__the.BUILD_TOOLS_LANGTOOLS_batch.cmdline && exit $exitcode ) ) && /bin/mv /home/ncop
 a/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.tmp /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch
    Successfully remade target file '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch'.
  Must remake target 'all'.
  Successfully remade target file 'all'.
Must remake target 'default'.
Successfully remade target file 'default'.



From claes.redestad at oracle.com  Mon Jan 27 17:13:19 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Mon, 27 Jan 2020 18:13:19 +0100
Subject: RFR: 8236272: Improve fidelity between contents of default CDS
 archive and classes loaded at runtime
Message-ID: <3e49ba9a-74a8-72cb-23bd-63256c23e65b@oracle.com>

Hi,

when generating the default classlist, doing an extra pass with a
-Xshare:dump between means the final classlist will be more in line
with what will be used at runtime.

Webrev: http://cr.openjdk.java.net/~redestad/8236272/open.00/
Bug:    https://bugs.openjdk.java.net/browse/JDK-8236272

This removes a number of classes from being dumped into the default
archive, effectively reducing size by ~170Kb (~1.5%).

The size reduction might be improved upon further in the future by
making sure that -Xshare:dump can be limited to not dump classes not
listed in the generated classlist, but this requires changes to the
runtime and needs to be thoroughly examined.

Such potential improvements will probably need this build-time fix to be
effective. In the mean-time, this patch provides a small but clear gain.

Thanks!

/Claes


From erik.joelsson at oracle.com  Mon Jan 27 17:31:46 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Mon, 27 Jan 2020 09:31:46 -0800
Subject: RFR: 8236272: Improve fidelity between contents of default CDS
 archive and classes loaded at runtime
In-Reply-To: <3e49ba9a-74a8-72cb-23bd-63256c23e65b@oracle.com>
References: <3e49ba9a-74a8-72cb-23bd-63256c23e65b@oracle.com>
Message-ID: <b3794dba-d895-27ee-8739-c2582eb9bbde@oracle.com>

Looks good.

/Erik

On 2020-01-27 09:13, Claes Redestad wrote:
> Hi,
>
> when generating the default classlist, doing an extra pass with a
> -Xshare:dump between means the final classlist will be more in line
> with what will be used at runtime.
>
> Webrev: http://cr.openjdk.java.net/~redestad/8236272/open.00/
> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8236272
>
> This removes a number of classes from being dumped into the default
> archive, effectively reducing size by ~170Kb (~1.5%).
>
> The size reduction might be improved upon further in the future by
> making sure that -Xshare:dump can be limited to not dump classes not
> listed in the generated classlist, but this requires changes to the
> runtime and needs to be thoroughly examined.
>
> Such potential improvements will probably need this build-time fix to be
> effective. In the mean-time, this patch provides a small but clear gain.
>
> Thanks!
>
> /Claes


From magnus.ihse.bursie at oracle.com  Mon Jan 27 20:26:17 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Mon, 27 Jan 2020 21:26:17 +0100
Subject: gnu make 4.3 breaks the build
In-Reply-To: <20200127123716.4dc90778@ncopa-desktop.copa.dup.pw>
References: <20200127123716.4dc90778@ncopa-desktop.copa.dup.pw>
Message-ID: <bd69751c-7c23-4cee-1dc6-47a766f719de@oracle.com>



On 2020-01-27 12:37, Natanael Copa wrote:
> Hi,
>
> this is a follow up on https://mail.openjdk.java.net/pipermail/build-dev/2020-January/026635.html
>
> Alpine linux has bumped into the same issue and it sort of blocking the
> security update. I would rather try fix the make issue than roll back
> to make 4.2.1.
>
> Downstream mergerequest/report:
> https://gitlab.alpinelinux.org/alpine/aports/merge_requests/3224
>
> It fails on different location even if I run it with JOBS=1, so to
> track it down I had to run make clean between each run.
>
> This is the command that fails (I have manually inserted the --debug --trace options):
Thanks for the debugging help.

I looked at the alpine bug report -- at this point I would not consider 
this a make bug, since it's more likely that we've got caught on one of 
their changes. But it might be a regression in make, too. (But even in 
that case, I'm afraid we'll have to work around it.)
>
> $ (cd /home/ncopa/aports/community/openjdk11/src/
> jdk11u-jdk-11.0.6+10/make && /usr/bin/make --debug --trace -s -r -R -I /home/ncopa/aports/community/openjdk11/src/jdk11u
> -jdk-11.0.6+10/make/common SPEC=/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-norma
> l-server-release/spec.gmk MAKE_LOG_FLAGS="-s"  -f ToolsLangtools.gmk)
> GNU Make 4.3
> Built for x86_64-alpine-linux-musl
> Copyright (C) 1988-2020 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html> This is free software: you are free
> to change and redistribute it. There is NO WARRANTY, to the extent
> permitted by law. Reading makefiles...
> Updating makefiles....
> Updating goal targets....
>   File 'default' does not exist.
>     File 'all' does not exist.
>       File
> '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties'
> does not exist. Must remake target
> '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties'.
> ToolsLangtools.gmk:37: update target
> '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties'
> due to:
> /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/resources/templates.properties
> /bin/mkdir -p
> /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources
> /bin/cp
> /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/resources/templates.properties
> /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties
> /bin/chmod -f ug+w
> /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties
> Successfully remade target file
> '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/propertiesparser/resources/templates.properties'.
> File
> '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch'
> does not exist. make: *** No rule to make target
> '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps',
> needed by
> '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch'.
>   Stop.
>
>
>
> If I re-run the command (without make clean) it will succeed:
My preliminary analysis indicated a problem with our MakeDir macro, 
which is basically a sophisticated way of calling mkdir -p. If a rerun 
succeeded, this definitely corroborates this story. It might be a couple 
of days before I have time to look further into this issue. If you want 
to delve deeper, have a look at make/common/MakeBase.gmk, especially 
DependOnVariable and MakeDir.

/Magnus

> GNU Make 4.3
> Built for x86_64-alpine-linux-musl
> Copyright (C) 1988-2020 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> Reading makefiles...
> Updating makefiles....
> Updating goal targets....
>   File 'default' does not exist.
>     File 'all' does not exist.
>       File '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch' does not exist.
>      Must remake target '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch'.
> Compiling 8 files for BUILD_TOOLS_LANGTOOLS
> ToolsLangtools.gmk:37: update target '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch' due to: /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/compileproperties/CompileProperties.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/PropertiesParser.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/gen/ClassGenerator.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/Message.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/MessageFile.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/MessageInfo.java /home/ncopa/aports/!
>   community
>   /openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/MessageLine.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/make/langtools/tools/propertiesparser/parser/MessageType.java /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps
> ( /bin/rm -f /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.log && /usr/lib/jvm/java-10-openjdk/bin/javac -g -J-XX:+UseSerialGC -J-Xms32M -J-Xmx512M -J-XX:TieredStopAtLevel=1 -source 9 -target 9 -XDignore.symbol.file=true -g -Xlint:all,-deprecation,-options -Werror -Xprefer:source -implicit:none -d /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes @/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.tmp > >(/usr/bin/tee -a /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.log) 2> >(/usr/bin/tee -a /home/ncopa/aports/communi!
>   ty/openjd
>   k11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.log >&2) || ( exitcode=$? && /bin/cp /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.log /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/make-support/failure-logs/buildtools_langtools_tools_classes__the.BUILD_TOOLS_LANGTOOLS_batch.log && /bin/cp /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.cmdline /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/make-support/failure-logs/buildtools_langtools_tools_classes__the.BUILD_TOOLS_LANGTOOLS_batch.cmdline && exit $exitcode ) ) && /bin/mv /!
>   home/ncop
>   a/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch.tmp /home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch
>      Successfully remade target file '/home/ncopa/aports/community/openjdk11/src/jdk11u-jdk-11.0.6+10/build/linux-x86_64-normal-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS_batch'.
>    Must remake target 'all'.
>    Successfully remade target file 'all'.
> Must remake target 'default'.
> Successfully remade target file 'default'.
>




From ncopa at alpinelinux.org  Mon Jan 27 22:01:56 2020
From: ncopa at alpinelinux.org (Natanael Copa)
Date: Mon, 27 Jan 2020 23:01:56 +0100
Subject: gnu make 4.3 breaks the build
In-Reply-To: <bd69751c-7c23-4cee-1dc6-47a766f719de@oracle.com>
References: <20200127123716.4dc90778@ncopa-desktop.copa.dup.pw>
 <bd69751c-7c23-4cee-1dc6-47a766f719de@oracle.com>
Message-ID: <20200127230156.103d1af1@ncopa-desktop.copa.dup.pw>

Hi!

On Mon, 27 Jan 2020 21:26:17 +0100
Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com> wrote:

> On 2020-01-27 12:37, Natanael Copa wrote:
> > Hi,
> >
> > this is a follow up on https://mail.openjdk.java.net/pipermail/build-dev/2020-January/026635.html
> >
> > Alpine linux has bumped into the same issue and it sort of blocking the
> > security update. I would rather try fix the make issue than roll back
> > to make 4.2.1.
> >
> > Downstream mergerequest/report:
> > https://gitlab.alpinelinux.org/alpine/aports/merge_requests/3224
> >
> > It fails on different location even if I run it with JOBS=1, so to
> > track it down I had to run make clean between each run.
> >
> > This is the command that fails (I have manually inserted the --debug --trace options):  
> Thanks for the debugging help.
> 
> I looked at the alpine bug report -- at this point I would not consider 
> this a make bug, since it's more likely that we've got caught on one of 
> their changes. But it might be a regression in make, too. (But even in 
> that case, I'm afraid we'll have to work around it.)

I filed a bug to make as well.

> > If I re-run the command (without make clean) it will succeed:  
> My preliminary analysis indicated a problem with our MakeDir macro, 
> which is basically a sophisticated way of calling mkdir -p. If a rerun 
> succeeded, this definitely corroborates this story. It might be a couple 
> of days before I have time to look further into this issue. If you want 
> to delve deeper, have a look at make/common/MakeBase.gmk, especially 
> DependOnVariable and MakeDir.

I think you are right. I was able to create a test case which I
submitted to gnu make bug tracker.

https://savannah.gnu.org/bugs/index.php?57676

Thank you for quick response.

-nc


From erik.joelsson at oracle.com  Mon Jan 27 23:13:57 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Mon, 27 Jan 2020 15:13:57 -0800
Subject: RFR: JDK-8237879: make 4.3 breaks build
Message-ID: <814332a3-5915-fd61-fbee-193c3993e6db@oracle.com>

Please review this fix to make the build compatible with GNU make 4.3. 
See bug for details on the fix, but this change basically adds an empty 
rule like this for the example failure:

/localhome/git/jdk-sandbox/open/build/linux-x86_64-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps:

With this change I'm able to build with make 4.3 and DependOnVariable is 
still behaving as expected.

Webrev: http://cr.openjdk.java.net/~erikj/8237879/webrev.01/index.html

Bug: https://bugs.openjdk.java.net/browse/JDK-8237879

/Erik



From tim.bell at oracle.com  Tue Jan 28 00:05:30 2020
From: tim.bell at oracle.com (Tim Bell)
Date: Mon, 27 Jan 2020 16:05:30 -0800
Subject: RFR: JDK-8237879: make 4.3 breaks build
In-Reply-To: <814332a3-5915-fd61-fbee-193c3993e6db@oracle.com>
References: <814332a3-5915-fd61-fbee-193c3993e6db@oracle.com>
Message-ID: <8e220b52-37a0-b05b-7635-7df49543ce23@oracle.com>

Erik:

> Please review this fix to make the build compatible with GNU make 4.3. 
> See bug for details on the fix, but this change basically adds an empty 
> rule like this for the example failure:
> 
> /localhome/git/jdk-sandbox/open/build/linux-x86_64-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps: 
> 
> 
> With this change I'm able to build with make 4.3 and DependOnVariable is 
> still behaving as expected.
> 
> Webrev: http://cr.openjdk.java.net/~erikj/8237879/webrev.01/index.html
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8237879

Looks good.

Tim


From ioi.lam at oracle.com  Tue Jan 28 01:09:39 2020
From: ioi.lam at oracle.com (Ioi Lam)
Date: Mon, 27 Jan 2020 17:09:39 -0800
Subject: RFR: 8236272: Improve fidelity between contents of default CDS
 archive and classes loaded at runtime
In-Reply-To: <3e49ba9a-74a8-72cb-23bd-63256c23e65b@oracle.com>
References: <3e49ba9a-74a8-72cb-23bd-63256c23e65b@oracle.com>
Message-ID: <fa37714d-b4fc-dd92-a823-741431e35b25@oracle.com>

Hi Claes,

This looks OK to me. One thing I noticed is when running 
-XX:DumpLoadedClassList from the "real" JDK image, with CDS enabled, I 
get more classes such as these:

 > java/lang/invoke/BoundMethodHandle$Species_LJ
 > java/lang/invoke/BoundMethodHandle$Species_LL
 > java/lang/invoke/BoundMethodHandle$Species_LLL
 > java/lang/invoke/BoundMethodHandle$Species_LLLL
 > java/lang/invoke/BoundMethodHandle$Species_LLLLL
 > java/lang/invoke/BoundMethodHandle$Species_LLLLLL
 > java/lang/invoke/BoundMethodHandle$Species_LLLLLLL
 > java/lang/invoke/BoundMethodHandle$Species_LLLLLLLL
 > java/lang/invoke/BoundMethodHandle$Species_LLLLLLLLL

I think these are generated later in the build process. Should we try to 
get these into the CDS archive as well? My unscientific measurement 
shows about 500K fewer instructions and up to half a percent elapsed 
time improvement (vs jdk/jdk, without your patch).

Also, these classes are not longer in the classlist, because they are 
used only when building the archived heap objects.

< java/lang/invoke/ClassSpecializer$Factory$1Var
< java/lang/module/ModuleDescriptor$Modifier
 > java/lang/Module$ReflectionData
< java/lang/NoSuchMethodError
< sun/nio/cs/StandardCharsets$Aliases
< sun/nio/cs/StandardCharsets$Cache
< sun/nio/cs/Surrogate
< sun/nio/cs/Surrogate$Parser
< sun/nio/cs/US_ASCII$Encoder
< sun/nio/cs/UTF_16
< sun/nio/cs/UTF_16BE
< sun/nio/cs/UTF_16LE
< ... more ...

That might cause a slight degradation when running without archived 
objects (such as -XX:+UseSerialGC) where these classes will be loaded. 
Have you measured the differences?

Thanks
- Ioi

On 1/27/20 9:13 AM, Claes Redestad wrote:
> Hi,
>
> when generating the default classlist, doing an extra pass with a
> -Xshare:dump between means the final classlist will be more in line
> with what will be used at runtime.
>
> Webrev: http://cr.openjdk.java.net/~redestad/8236272/open.00/
> Bug:??? https://bugs.openjdk.java.net/browse/JDK-8236272
>
> This removes a number of classes from being dumped into the default
> archive, effectively reducing size by ~170Kb (~1.5%).
>
> The size reduction might be improved upon further in the future by
> making sure that -Xshare:dump can be limited to not dump classes not
> listed in the generated classlist, but this requires changes to the
> runtime and needs to be thoroughly examined.
>
> Such potential improvements will probably need this build-time fix to be
> effective. In the mean-time, this patch provides a small but clear gain.
>
> Thanks!
>
> /Claes



From jianglizhou at google.com  Tue Jan 28 01:37:34 2020
From: jianglizhou at google.com (Jiangli Zhou)
Date: Mon, 27 Jan 2020 17:37:34 -0800
Subject: RFR: 8236272: Improve fidelity between contents of default CDS
 archive and classes loaded at runtime
In-Reply-To: <3e49ba9a-74a8-72cb-23bd-63256c23e65b@oracle.com>
References: <3e49ba9a-74a8-72cb-23bd-63256c23e65b@oracle.com>
Message-ID: <CALrW1jzHz7BpzrNSz8SQA8CK+mxsrkXqaxUgL9FxE7-G1KM2PA@mail.gmail.com>

Hi Claes,

Looks good to me. Nice improvement.

Best regards,
Jiangli

On Mon, Jan 27, 2020 at 9:10 AM Claes Redestad
<claes.redestad at oracle.com> wrote:
>
> Hi,
>
> when generating the default classlist, doing an extra pass with a
> -Xshare:dump between means the final classlist will be more in line
> with what will be used at runtime.
>
> Webrev: http://cr.openjdk.java.net/~redestad/8236272/open.00/
> Bug:    https://bugs.openjdk.java.net/browse/JDK-8236272
>
> This removes a number of classes from being dumped into the default
> archive, effectively reducing size by ~170Kb (~1.5%).
>
> The size reduction might be improved upon further in the future by
> making sure that -Xshare:dump can be limited to not dump classes not
> listed in the generated classlist, but this requires changes to the
> runtime and needs to be thoroughly examined.
>
> Such potential improvements will probably need this build-time fix to be
> effective. In the mean-time, this patch provides a small but clear gain.
>
> Thanks!
>
> /Claes


From claes.redestad at oracle.com  Tue Jan 28 09:59:09 2020
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 28 Jan 2020 10:59:09 +0100
Subject: RFR: 8236272: Improve fidelity between contents of default CDS
 archive and classes loaded at runtime
In-Reply-To: <fa37714d-b4fc-dd92-a823-741431e35b25@oracle.com>
References: <3e49ba9a-74a8-72cb-23bd-63256c23e65b@oracle.com>
 <fa37714d-b4fc-dd92-a823-741431e35b25@oracle.com>
Message-ID: <b53aa77b-9f5b-094e-9882-698b6b6853d3@oracle.com>

Hi Ioi,

On 2020-01-28 02:09, Ioi Lam wrote:
> Hi Claes,
> 
> This looks OK to me. One thing I noticed is when running 
> -XX:DumpLoadedClassList from the "real" JDK image, with CDS enabled, I 
> get more classes such as these:
> 
>  > java/lang/invoke/BoundMethodHandle$Species_LJ
>  > java/lang/invoke/BoundMethodHandle$Species_LL
>  > java/lang/invoke/BoundMethodHandle$Species_LLL
>  > java/lang/invoke/BoundMethodHandle$Species_LLLL
>  > java/lang/invoke/BoundMethodHandle$Species_LLLLL
>  > java/lang/invoke/BoundMethodHandle$Species_LLLLLL
>  > java/lang/invoke/BoundMethodHandle$Species_LLLLLLL
>  > java/lang/invoke/BoundMethodHandle$Species_LLLLLLLL
>  > java/lang/invoke/BoundMethodHandle$Species_LLLLLLLLL
> 
> I think these are generated later in the build process. Should we try to 
> get these into the CDS archive as well? My unscientific measurement 
> shows about 500K fewer instructions and up to half a percent elapsed 
> time improvement (vs jdk/jdk, without your patch).

right, these classes are generated by jlink, so they're not seen by the
invocation of HelloClasslist that generates the final classlist
(line 75 in GenerateLinkOpt). To get them into the classlist, we could
do another pass of creating an interim image, dump CDS again, generate a
new classlist - or we could add logic somewhere to append to the
classlist using the information we log to $JLI_TRACE_FILE.

Before we do that, I have a related RFE I'd like to deal with to clean
up what JLI classes jlink generates by default, since it's rather
outdated: https://bugs.openjdk.java.net/browse/JDK-8230301

> 
> Also, these classes are not longer in the classlist, because they are 
> used only when building the archived heap objects.
> 
> < java/lang/invoke/ClassSpecializer$Factory$1Var
> < java/lang/module/ModuleDescriptor$Modifier
>  > java/lang/Module$ReflectionData
> < java/lang/NoSuchMethodError
> < sun/nio/cs/StandardCharsets$Aliases
> < sun/nio/cs/StandardCharsets$Cache
> < sun/nio/cs/Surrogate
> < sun/nio/cs/Surrogate$Parser
> < sun/nio/cs/US_ASCII$Encoder
> < sun/nio/cs/UTF_16
> < sun/nio/cs/UTF_16BE
> < sun/nio/cs/UTF_16LE
> < ... more ...
> 
> That might cause a slight degradation when running without archived 
> objects (such as -XX:+UseSerialGC) where these classes will be loaded. 
> Have you measured the differences?

On my setup it adds 1-2M instructions to -XX:+UseSerialGC, but I think
rather than keeping these in the default archive we should fix
https://bugs.openjdk.java.net/browse/JDK-8234679 to get Serial and
other GCs see (most of) the benefits of heap archiving.

Thanks!

/Claes


From magnus.ihse.bursie at oracle.com  Tue Jan 28 11:02:36 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 28 Jan 2020 12:02:36 +0100
Subject: gnu make 4.3 breaks the build
In-Reply-To: <20200127230156.103d1af1@ncopa-desktop.copa.dup.pw>
References: <20200127123716.4dc90778@ncopa-desktop.copa.dup.pw>
 <bd69751c-7c23-4cee-1dc6-47a766f719de@oracle.com>
 <20200127230156.103d1af1@ncopa-desktop.copa.dup.pw>
Message-ID: <b6905504-7aa5-21d1-1064-47a4ba81ccde@oracle.com>

On 2020-01-27 23:01, Natanael Copa wrote:
> Hi!
>
> On Mon, 27 Jan 2020 21:26:17 +0100
> Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com> wrote:
>
>> On 2020-01-27 12:37, Natanael Copa wrote:
>>> Hi,
>>>
>>> this is a follow up on https://mail.openjdk.java.net/pipermail/build-dev/2020-January/026635.html
>>>
>>> Alpine linux has bumped into the same issue and it sort of blocking the
>>> security update. I would rather try fix the make issue than roll back
>>> to make 4.2.1.
>>>
>>> Downstream mergerequest/report:
>>> https://gitlab.alpinelinux.org/alpine/aports/merge_requests/3224
>>>
>>> It fails on different location even if I run it with JOBS=1, so to
>>> track it down I had to run make clean between each run.
>>>
>>> This is the command that fails (I have manually inserted the --debug --trace options):
>> Thanks for the debugging help.
>>
>> I looked at the alpine bug report -- at this point I would not consider
>> this a make bug, since it's more likely that we've got caught on one of
>> their changes. But it might be a regression in make, too. (But even in
>> that case, I'm afraid we'll have to work around it.)
> I filed a bug to make as well.
>
>>> If I re-run the command (without make clean) it will succeed:
>> My preliminary analysis indicated a problem with our MakeDir macro,
>> which is basically a sophisticated way of calling mkdir -p. If a rerun
>> succeeded, this definitely corroborates this story. It might be a couple
>> of days before I have time to look further into this issue. If you want
>> to delve deeper, have a look at make/common/MakeBase.gmk, especially
>> DependOnVariable and MakeDir.
> I think you are right. I was able to create a test case which I
> submitted to gnu make bug tracker.
>
> https://savannah.gnu.org/bugs/index.php?57676

I've found the culprit. The "-include" statement does not work properly. 
Manually checking for the existence of the file and then using "include" 
works fine. I would consider this a bug in make. Feel free to pass this 
information along to the upstream make project.

Nevertheless, since this is spotted "in the wild" in a released version 
of GNU make, we need to patch around it. I'll post a patch shortly.

/Magnus





From magnus.ihse.bursie at oracle.com  Tue Jan 28 11:05:52 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Tue, 28 Jan 2020 12:05:52 +0100
Subject: RFR: JDK-8237879: make 4.3 breaks build
In-Reply-To: <814332a3-5915-fd61-fbee-193c3993e6db@oracle.com>
References: <814332a3-5915-fd61-fbee-193c3993e6db@oracle.com>
Message-ID: <17695e75-c11b-ceca-3eb3-2314dcf45686@oracle.com>

On 2020-01-28 00:13, Erik Joelsson wrote:
> Please review this fix to make the build compatible with GNU make 4.3. 
> See bug for details on the fix, but this change basically adds an 
> empty rule like this for the example failure:
>
> /localhome/git/jdk-sandbox/open/build/linux-x86_64-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps: 
>
>
> With this change I'm able to build with make 4.3 and DependOnVariable 
> is still behaving as expected.
>
> Webrev: http://cr.openjdk.java.net/~erikj/8237879/webrev.01/index.html
Oh, we've solved this in parallel. :-&

Actually, I think I prefer my fix:

http://cr.openjdk.java.net/~ihse/JDK-8237879-make-4-3-include-is-broken/webrev.01

I think it more clearly captures the issue, rather than adding a "magic" 
workaround.

/Magnus


>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8237879
>
> /Erik
>



From ncopa at alpinelinux.org  Tue Jan 28 11:21:38 2020
From: ncopa at alpinelinux.org (Natanael Copa)
Date: Tue, 28 Jan 2020 12:21:38 +0100
Subject: gnu make 4.3 breaks the build
In-Reply-To: <b6905504-7aa5-21d1-1064-47a4ba81ccde@oracle.com>
References: <20200127123716.4dc90778@ncopa-desktop.copa.dup.pw>
 <bd69751c-7c23-4cee-1dc6-47a766f719de@oracle.com>
 <20200127230156.103d1af1@ncopa-desktop.copa.dup.pw>
 <b6905504-7aa5-21d1-1064-47a4ba81ccde@oracle.com>
Message-ID: <20200128122138.38777890@ncopa-desktop.copa.dup.pw>

On Tue, 28 Jan 2020 12:02:36 +0100
Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com> wrote:

> On 2020-01-27 23:01, Natanael Copa wrote:
> > Hi!
> >
> > On Mon, 27 Jan 2020 21:26:17 +0100
> > Magnus Ihse Bursie <magnus.ihse.bursie at oracle.com> wrote:
> >  
> >> On 2020-01-27 12:37, Natanael Copa wrote:  
> >>> Hi,
> >>>
> >>> this is a follow up on https://mail.openjdk.java.net/pipermail/build-dev/2020-January/026635.html

...

> > https://savannah.gnu.org/bugs/index.php?57676  
> 
> I've found the culprit. The "-include" statement does not work properly. 
> Manually checking for the existence of the file and then using "include" 
> works fine. I would consider this a bug in make. Feel free to pass this 
> information along to the upstream make project.
> 
> Nevertheless, since this is spotted "in the wild" in a released version 
> of GNU make, we need to patch around it. I'll post a patch shortly.

Awesome!

I will forward it upstream.

Good work! Thank you.

-nc


From erik.joelsson at oracle.com  Tue Jan 28 16:15:53 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Tue, 28 Jan 2020 08:15:53 -0800
Subject: RFR: JDK-8237879: make 4.3 breaks build
In-Reply-To: <17695e75-c11b-ceca-3eb3-2314dcf45686@oracle.com>
References: <814332a3-5915-fd61-fbee-193c3993e6db@oracle.com>
 <17695e75-c11b-ceca-3eb3-2314dcf45686@oracle.com>
Message-ID: <9592309e-9e49-6149-5c76-3e718c8bf4ee@oracle.com>

On 2020-01-28 03:05, Magnus Ihse Bursie wrote:
> On 2020-01-28 00:13, Erik Joelsson wrote:
>> Please review this fix to make the build compatible with GNU make 
>> 4.3. See bug for details on the fix, but this change basically adds 
>> an empty rule like this for the example failure:
>>
>> /localhome/git/jdk-sandbox/open/build/linux-x86_64-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps: 
>>
>>
>> With this change I'm able to build with make 4.3 and DependOnVariable 
>> is still behaving as expected.
>>
>> Webrev: http://cr.openjdk.java.net/~erikj/8237879/webrev.01/index.html
> Oh, we've solved this in parallel. :-&
>
> Actually, I think I prefer my fix:
>
> http://cr.openjdk.java.net/~ihse/JDK-8237879-make-4-3-include-is-broken/webrev.01
>
> I think it more clearly captures the issue, rather than adding a 
> "magic" workaround.
>
Ah, it's the -include that was the issue. I didn't get that far. Then 
yes, your solution is better (if you break that long line)!

/Erik

> /Magnus
>
>
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237879
>>
>> /Erik
>>
>


From magnus.ihse.bursie at oracle.com  Wed Jan 29 10:58:50 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Wed, 29 Jan 2020 11:58:50 +0100
Subject: RFR: JDK-8237879: make 4.3 breaks build
In-Reply-To: <9592309e-9e49-6149-5c76-3e718c8bf4ee@oracle.com>
References: <814332a3-5915-fd61-fbee-193c3993e6db@oracle.com>
 <17695e75-c11b-ceca-3eb3-2314dcf45686@oracle.com>
 <9592309e-9e49-6149-5c76-3e718c8bf4ee@oracle.com>
Message-ID: <685c5365-c298-0d1b-75ee-413db945fe5a@oracle.com>

On 2020-01-28 17:15, Erik Joelsson wrote:
> On 2020-01-28 03:05, Magnus Ihse Bursie wrote:
>> On 2020-01-28 00:13, Erik Joelsson wrote:
>>> Please review this fix to make the build compatible with GNU make 
>>> 4.3. See bug for details on the fix, but this change basically adds 
>>> an empty rule like this for the example failure:
>>>
>>> /localhome/git/jdk-sandbox/open/build/linux-x86_64-server-release/buildtools/langtools_tools_classes/_the.BUILD_TOOLS_LANGTOOLS.vardeps: 
>>>
>>>
>>> With this change I'm able to build with make 4.3 and 
>>> DependOnVariable is still behaving as expected.
>>>
>>> Webrev: http://cr.openjdk.java.net/~erikj/8237879/webrev.01/index.html
>> Oh, we've solved this in parallel. :-&
>>
>> Actually, I think I prefer my fix:
>>
>> http://cr.openjdk.java.net/~ihse/JDK-8237879-make-4-3-include-is-broken/webrev.01 
>>
>>
>> I think it more clearly captures the issue, rather than adding a 
>> "magic" workaround.
>>
> Ah, it's the -include that was the issue. I didn't get that far. Then 
> yes, your solution is better (if you break that long line)!
I got tired of the code duplication of "$(call DependOnVariableFileName, 
$1, $2)". Here's a new webrev with shorter lines,
and hopefully better readability overall:

http://cr.openjdk.java.net/~ihse/JDK-8237879-make-4-3-include-is-broken/webrev.02/make/common/MakeBase.gmk.udiff.html

/Magnus

>
> /Erik
>
>> /Magnus
>>
>>
>>>
>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8237879
>>>
>>> /Erik
>>>
>>



From erik.joelsson at oracle.com  Wed Jan 29 14:01:16 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 29 Jan 2020 06:01:16 -0800
Subject: RFR: JDK-8237879: make 4.3 breaks build
In-Reply-To: <685c5365-c298-0d1b-75ee-413db945fe5a@oracle.com>
References: <814332a3-5915-fd61-fbee-193c3993e6db@oracle.com>
 <17695e75-c11b-ceca-3eb3-2314dcf45686@oracle.com>
 <9592309e-9e49-6149-5c76-3e718c8bf4ee@oracle.com>
 <685c5365-c298-0d1b-75ee-413db945fe5a@oracle.com>
Message-ID: <991ff11a-ea97-e4bf-328f-b94ebc430021@oracle.com>

On 2020-01-29 02:58, Magnus Ihse Bursie wrote:
> I got tired of the code duplication of "$(call 
> DependOnVariableFileName, $1, $2)". Here's a new webrev with shorter 
> lines,
> and hopefully better readability overall:
>
> http://cr.openjdk.java.net/~ihse/JDK-8237879-make-4-3-include-is-broken/webrev.02/make/common/MakeBase.gmk.udiff.html
>
Looks even better!

/Erik




From tim.bell at oracle.com  Wed Jan 29 14:36:25 2020
From: tim.bell at oracle.com (Tim Bell)
Date: Wed, 29 Jan 2020 06:36:25 -0800
Subject: RFR: JDK-8237879: make 4.3 breaks build
In-Reply-To: <991ff11a-ea97-e4bf-328f-b94ebc430021@oracle.com>
References: <814332a3-5915-fd61-fbee-193c3993e6db@oracle.com>
 <17695e75-c11b-ceca-3eb3-2314dcf45686@oracle.com>
 <9592309e-9e49-6149-5c76-3e718c8bf4ee@oracle.com>
 <685c5365-c298-0d1b-75ee-413db945fe5a@oracle.com>
 <991ff11a-ea97-e4bf-328f-b94ebc430021@oracle.com>
Message-ID: <71c56f22-e64c-0603-50f8-a53bc5887957@oracle.com>

Hi Magnus:

On 1/29/20 6:01 AM, Erik Joelsson wrote:
> On 2020-01-29 02:58, Magnus Ihse Bursie wrote:
>> I got tired of the code duplication of "$(call 
>> DependOnVariableFileName, $1, $2)". Here's a new webrev with shorter 
>> lines,
>> and hopefully better readability overall:
>>
>> http://cr.openjdk.java.net/~ihse/JDK-8237879-make-4-3-include-is-broken/webrev.02/make/common/MakeBase.gmk.udiff.html 
>>
>>
> Looks even better!
> 
> /Erik

Looks good to me as well.

Tim




From erik.joelsson at oracle.com  Wed Jan 29 14:59:47 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Wed, 29 Jan 2020 06:59:47 -0800
Subject: RFR: JDK-8237914: The test-make target does not fail on test failure
Message-ID: <407ed552-d847-f7b9-f6e4-ccecc156fd3a@oracle.com>

When running the make tests for the build itself, RunTest.gmk currently 
does not attempt at parsing the results. Because of this, failures 
aren't detected and so the build does not fail if a test fails. This 
patch adds a very rudimentary parsing, defining the whole test run as 1 
test and success/fail based on the exit code. With this, a test failure 
will cause the top level make to exit with error. I've verified that the 
testmake tests are still clean on all relevant platforms (as our CI 
would have failed to notice failures for a while).

Webrev: http://cr.openjdk.java.net/~erikj/8237914/webrev.01/

Bug: https://bugs.openjdk.java.net/browse/JDK-8237914

/Erik



From tim.bell at oracle.com  Wed Jan 29 15:08:57 2020
From: tim.bell at oracle.com (Tim Bell)
Date: Wed, 29 Jan 2020 07:08:57 -0800
Subject: RFR: JDK-8237914: The test-make target does not fail on test
 failure
In-Reply-To: <407ed552-d847-f7b9-f6e4-ccecc156fd3a@oracle.com>
References: <407ed552-d847-f7b9-f6e4-ccecc156fd3a@oracle.com>
Message-ID: <4515f275-5f17-f816-41ef-0e045a631943@oracle.com>

Erik:

> When running the make tests for the build itself, RunTest.gmk currently 
> does not attempt at parsing the results. Because of this, failures 
> aren't detected and so the build does not fail if a test fails. This 
> patch adds a very rudimentary parsing, defining the whole test run as 1 
> test and success/fail based on the exit code. With this, a test failure 
> will cause the top level make to exit with error. I've verified that the 
> testmake tests are still clean on all relevant platforms (as our CI 
> would have failed to notice failures for a while).
> 
> Webrev: http://cr.openjdk.java.net/~erikj/8237914/webrev.01/
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8237914

Looks good.

Tim


From magnus.ihse.bursie at oracle.com  Thu Jan 30 10:07:38 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Thu, 30 Jan 2020 11:07:38 +0100
Subject: RFR: JDK-8237914: The test-make target does not fail on test
 failure
In-Reply-To: <407ed552-d847-f7b9-f6e4-ccecc156fd3a@oracle.com>
References: <407ed552-d847-f7b9-f6e4-ccecc156fd3a@oracle.com>
Message-ID: <ac37888e-96b4-9013-1bca-a4d82a4a107d@oracle.com>



On 2020-01-29 15:59, Erik Joelsson wrote:
> When running the make tests for the build itself, RunTest.gmk 
> currently does not attempt at parsing the results. Because of this, 
> failures aren't detected and so the build does not fail if a test 
> fails. This patch adds a very rudimentary parsing, defining the whole 
> test run as 1 test and success/fail based on the exit code. With this, 
> a test failure will cause the top level make to exit with error. I've 
> verified that the testmake tests are still clean on all relevant 
> platforms (as our CI would have failed to notice failures for a while).
>
> Webrev: http://cr.openjdk.java.net/~erikj/8237914/webrev.01/
Looks good to me. Clever and simple solution!

/Magnus
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8237914
>
> /Erik
>



From magnus.ihse.bursie at oracle.com  Thu Jan 30 15:37:30 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Thu, 30 Jan 2020 16:37:30 +0100
Subject: building libjvm with -Os for space optimization - was : RE: RFR:
 8234525: enable link-time section-gc for linux s390x to remove unused
 code
In-Reply-To: <FD4C63C7-0B7D-44CC-9F6F-334CC7DDFE55@gmail.com>
References: <FD4C63C7-0B7D-44CC-9F6F-334CC7DDFE55@gmail.com>
Message-ID: <3bc4c61e-9a59-d9d9-791d-258008322907@oracle.com>



On 2019-12-18 00:05, August Nagro wrote:
> I published some benchmarks of OpenJDK on Mac with Ofast and O3 [1]. Some microbenchmarks like Netty?s HttpObjectEncoder experienced >100% speedup with O3, and the more real-world Dacapo suite was ~15% improvement over O2 (which is exactly the same as Os). I did include a few other flags, however the speedup was primarily due to optimization level.
>
> Building with Os is the old wisdom. It used to be the case that many programs would be faster with the smaller binary size, but this is almost never the case nowadays.
>
> - August
>
> [1]: http://august.nagro.us/optimized-openjdk.html
Nice. :)

I noticed this on your blog post:
> I also tried building withIntel?s Compiler 
> <https://software.intel.com/en-us/c-compilers>, and patched the 
> configure scripts to allow it. However, the build failed cryptically. 
> I?d also be interested to see Linux results.

How did you patch configure, and what errors did you run into? In 
general, it's not a small task to get OpenJDK to compile with a new 
compiler, but Intel claims a high level of compatibility with gcc [1] 
and that increases chances that it should be possible to achieve with 
not too much effort. I'm slightly tempted to looking into it. :)

/Magnus

[1] 
https://software.intel.com/en-us/cpp-compiler-developer-guide-and-reference-gcc-compatibility-and-interoperability





From augustnagro at gmail.com  Thu Jan 30 20:38:31 2020
From: augustnagro at gmail.com (August Nagro)
Date: Thu, 30 Jan 2020 14:38:31 -0600
Subject: building libjvm with -Os for space optimization - was : RE: RFR:
 8234525: enable link-time section-gc for linux s390x to remove unused
 code
In-Reply-To: <3bc4c61e-9a59-d9d9-791d-258008322907@oracle.com>
References: <FD4C63C7-0B7D-44CC-9F6F-334CC7DDFE55@gmail.com>
 <3bc4c61e-9a59-d9d9-791d-258008322907@oracle.com>
Message-ID: <CAHNb0ZNvVJCOQtMWnUPazBrSw2Bovcz=3D68wGYE_SVX7iAP5w@mail.gmail.com>

Its been a while since I tried, but if I remember correctly I had to delete
the code that tested the compiler name / version in one of the sh files. I
set the compiler to icc via the environment variables.

The build ran for a while but then stopped with an error message. I
remember it not being very helpful.

I would think you'll have better luck on Linux + gcc than me on Mac,
although icc claims compatible with llvm too.
Let me know if you succeed!

On Thu, Jan 30, 2020, 9:37 AM Magnus Ihse Bursie <
magnus.ihse.bursie at oracle.com> wrote:

>
>
> On 2019-12-18 00:05, August Nagro wrote:
>
> I published some benchmarks of OpenJDK on Mac with Ofast and O3 [1]. Some microbenchmarks like Netty?s HttpObjectEncoder experienced >100% speedup with O3, and the more real-world Dacapo suite was ~15% improvement over O2 (which is exactly the same as Os). I did include a few other flags, however the speedup was primarily due to optimization level.
>
> Building with Os is the old wisdom. It used to be the case that many programs would be faster with the smaller binary size, but this is almost never the case nowadays.
>
> - August
>
> [1]: http://august.nagro.us/optimized-openjdk.html
>
> Nice. :)
>
> I noticed this on your blog post:
>
> I also tried building with Intel?s Compiler
> <https://software.intel.com/en-us/c-compilers>, and patched the configure
> scripts to allow it. However, the build failed cryptically. I?d also be
> interested to see Linux results.
>
>
> How did you patch configure, and what errors did you run into? In general,
> it's not a small task to get OpenJDK to compile with a new compiler, but
> Intel claims a high level of compatibility with gcc [1] and that increases
> chances that it should be possible to achieve with not too much effort. I'm
> slightly tempted to looking into it. :)
>
> /Magnus
>
> [1]
> https://software.intel.com/en-us/cpp-compiler-developer-guide-and-reference-gcc-compatibility-and-interoperability
>
>
>
>


From magnus.ihse.bursie at oracle.com  Fri Jan 31 09:37:32 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 31 Jan 2020 10:37:32 +0100
Subject: RFR: JDK-8196875 Update run-test instructions for TEST_MODE
Message-ID: <4bb7c66f-55d1-d02a-546d-434b3b0864a9@oracle.com>

As reported by Martin Buchholz <martinrb at google.com>:

#### TEST_MODE
The test mode (`-agentvm`, `-samevm` or `-othervm`).

but TEST_MODE=-agentvm is rejected - it must be TEST_MODE=agentvm.

and Jon Gibbons noted:
.. and samevm mode no longer exists.

Bug: https://bugs.openjdk.java.net/browse/JDK-8196875
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8196875-update-test-readme-for-TEST_MODE/webrev.01 



From magnus.ihse.bursie at oracle.com  Fri Jan 31 11:04:47 2020
From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie)
Date: Fri, 31 Jan 2020 12:04:47 +0100
Subject: building libjvm with -Os for space optimization - was : RE: RFR:
 8234525: enable link-time section-gc for linux s390x to remove unused
 code
In-Reply-To: <CAHNb0ZNvVJCOQtMWnUPazBrSw2Bovcz=3D68wGYE_SVX7iAP5w@mail.gmail.com>
References: <FD4C63C7-0B7D-44CC-9F6F-334CC7DDFE55@gmail.com>
 <3bc4c61e-9a59-d9d9-791d-258008322907@oracle.com>
 <CAHNb0ZNvVJCOQtMWnUPazBrSw2Bovcz=3D68wGYE_SVX7iAP5w@mail.gmail.com>
Message-ID: <b398d264-a8bf-0b48-d4be-399f3de310f8@oracle.com>



On 2020-01-30 21:38, August Nagro wrote:
> Its been a while since I tried, but if I remember correctly I had to 
> delete the code that tested the compiler name / version in one of the 
> sh files. I set the compiler to icc via the environment variables.
>
> The build ran for a while but then stopped with an error message. I 
> remember it not being very helpful.
>
> I would think you'll have better luck on Linux + gcc than me on Mac, 
> although icc claims compatible with llvm too.
> Let me know if you succeed!
I stopped before I even started. I'd just assumed the icc had a free 
download, but this was not the case. Possibly I could use the open 
source contributor version, but that required downloading a whole suit, 
not just the compiler. Oh well.

/Magnus
>
> On Thu, Jan 30, 2020, 9:37 AM Magnus Ihse Bursie 
> <magnus.ihse.bursie at oracle.com <mailto:magnus.ihse.bursie at oracle.com>> 
> wrote:
>
>
>
>     On 2019-12-18 00:05, August Nagro wrote:
>>     I published some benchmarks of OpenJDK on Mac with Ofast and O3 [1]. Some microbenchmarks like Netty?s HttpObjectEncoder experienced >100% speedup with O3, and the more real-world Dacapo suite was ~15% improvement over O2 (which is exactly the same as Os). I did include a few other flags, however the speedup was primarily due to optimization level.
>>
>>     Building with Os is the old wisdom. It used to be the case that many programs would be faster with the smaller binary size, but this is almost never the case nowadays.
>>
>>     - August
>>
>>     [1]:http://august.nagro.us/optimized-openjdk.html
>     Nice. :)
>
>     I noticed this on your blog post:
>>     I also tried building withIntel?s Compiler
>>     <https://software.intel.com/en-us/c-compilers>, and patched the
>>     configure scripts to allow it. However, the build failed
>>     cryptically. I?d also be interested to see Linux results.
>
>     How did you patch configure, and what errors did you run into? In
>     general, it's not a small task to get OpenJDK to compile with a
>     new compiler, but Intel claims a high level of compatibility with
>     gcc [1] and that increases chances that it should be possible to
>     achieve with not too much effort. I'm slightly tempted to looking
>     into it. :)
>
>     /Magnus
>
>     [1]
>     https://software.intel.com/en-us/cpp-compiler-developer-guide-and-reference-gcc-compatibility-and-interoperability
>
>
>



From erik.joelsson at oracle.com  Fri Jan 31 14:08:28 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 31 Jan 2020 06:08:28 -0800
Subject: RFR: JDK-8196875 Update run-test instructions for TEST_MODE
In-Reply-To: <4bb7c66f-55d1-d02a-546d-434b3b0864a9@oracle.com>
References: <4bb7c66f-55d1-d02a-546d-434b3b0864a9@oracle.com>
Message-ID: <26a7f62e-9a87-5ebd-c514-e25718e49691@oracle.com>

Looks good.

/Erik

On 2020-01-31 01:37, Magnus Ihse Bursie wrote:
> As reported by Martin Buchholz <martinrb at google.com>:
>
> #### TEST_MODE
> The test mode (`-agentvm`, `-samevm` or `-othervm`).
>
> but TEST_MODE=-agentvm is rejected - it must be TEST_MODE=agentvm.
>
> and Jon Gibbons noted:
> .. and samevm mode no longer exists.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8196875
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8196875-update-test-readme-for-TEST_MODE/webrev.01 
>


From erik.joelsson at oracle.com  Fri Jan 31 15:31:35 2020
From: erik.joelsson at oracle.com (Erik Joelsson)
Date: Fri, 31 Jan 2020 07:31:35 -0800
Subject: RFR: JDK-8238225: Issues reported after replacing symlink at
 Contents/MacOS/libjli.dylib with binary
Message-ID: <a8542200-585b-1643-db48-ada74f2eda22@oracle.com>

In JDK-8235687 the MacOS bundle distribution of the JDK was changed to 
conform to Apple requirements by changing Contents/MacOS/libjli.dylib 
from a symlink into ../Home/lib/libjli.dylib to a copy of that file. The 
problem with having a symlink there is that Contents/MacOS/libjli.dylib 
is the declared CFBundleExecutable of the bundle and that executable may 
not be a symlink. The history around why that particular library was put 
there seems lost in ancient history. All we know is that it was there 
when Apple donated the Mac port and according to this bug report, there 
are users out there relying on it.

When changing Contents/MacOS/libjli.dylib to a copy, loading that dylib 
and using it to launch a JVM no longer works. This patch fixes that by 
making libjli.dylib aware of potentially being located there and if so, 
finding the JDK home dir in ../Home.

I've also expanded the existing test for launching a JVM through 
libjli.dylib directly to also test this location when possible. In local 
testing, this will not be covered unless the user explicitly specifies 
that the JDK under test should be the bundle image on the command line 
like this:

$ make test-only TEST=open/test/jdk/tools/launcher/JliLaunchTest.java 
JDK_IMAGE_DIR=$PWD/build/macosx-x64/images/jdk-bundle/jdk-15.jdk/Contents/Home

But, at least in Oracle's distributed testing, the JDK on MacOS is 
distributed in the bundle layout, so there this functionality will be 
tested.

Bug: https://bugs.openjdk.java.net/browse/JDK-8238225

Webrev: http://cr.openjdk.java.net/~erikj/8238225/webrev.01/

/Erik