Using hard links for debug builds

[Volker Simonis]

Hi Ioi,

I think the problem with links (both, hard and soft) is that they are
not supported on all filesystems. The same applies to the proposed "cp
--reflink". So you would at least have to check if it works during the
configuration step.

Regards,
Volker


On Wed, Jan 11, 2017 at 1:57 AM, Ioi Lam <ioi.lam at oracle.com> wrote:
> Hi Eric,
>
> Thanks for the detailed explanation.
>
> Perhaps this one copy can be changed to a hard link?
>
> Copying jdk/lib/server/libjvm.so
> /bin/cp -fP
> '/home/iklam/jdk/bld/hs-debug/support/modules_libs/java.base/server/libjvm.so'
> '/home/iklam/jdk/bld/hs-debug/jdk/lib/server/libjvm.so'
>
> My script changed it to a hard link and I see no ill effects:
>
> '/home/iklam/jdk/bld/hs-debug/jdk/lib/server/libjvm.so' =>
> '/home/iklam/jdk/bld/hs-debug/support/modules_libs/java.base/server/libjvm.so'
> 15452588 -rwxr-xr-x 2 iklam dba 322793048 Jan 10 16:54
> /home/iklam/jdk/bld/hs-debug/jdk/lib/server/libjvm.so
> 15452588 -rwxr-xr-x 2 iklam dba 322793048 Jan 10 16:54
> /home/iklam/jdk/bld/hs-debug/support/modules_libs/java.base/server/libjvm.so
>
> Thanks
> - Ioi
>
>
>
> On 1/10/17 12:56 AM, Erik Joelsson wrote:
>>
>> Hello,
>>
>> I recently did work to reduce the number of copies. While doing that I did
>> investigate using softlinks, but that didn't work for .so files. I did not
>> consider hard links. All other files in the exploded image are however
>> softlinked when possible. For all the copies in images, any kind of linking
>> won't work because those are generated with jlink, not cp. This means you
>> would only be able to eliminate 2 copies by linking out of the below 8.
>>
>> I did put in more detailed top level targets to make it possible for an
>> active user to get fewer copies. For example, instead of building "images",
>> you can build just "jdk-image" and you will skip 2 of the copies (jre and
>> serverjre). If you don't need a real jdk image, you can build just
>> "exploded-image" and you skip 2 more (jdk and support/interim-image). I have
>> tried to optimize the "test*" targets to only depend on the appropriate
>> images as well.
>>
>> If you don't care for the gtest binaries, you can configure
>> --disable-hotspot-gtest to get rid of those.
>>
>> Doing all of this and you are down to 2 copies, which your trick takes
>> down to one.
>>
>> The gtest version of libjvm.so contains all the objects of the normal
>> libjvm.so + the gtest unit tests. This is used to run the gtest unit tests.
>>
>> In general, --with-native-debug-symbols=external is easier on disk usage
>> since the debug symbols aren't copied everywhere like the libraries are.
>> Also note that even if you do =zipped (the default), there will be a link to
>> an unzipped copy available in the exploded image.
>>
>> /Erik
>>
>> On 2017-01-09 21:40, Ioi Lam wrote:
>>>
>>> Hi,
>>>
>>> libjvm.so gets copied a few times during the build. I usually build
>>> slowdebug builds with --with-native-debug-symbols=internal (to make it
>>> easier with gdb). This gives me 8 libjvm.so files:
>>>
>>> ./support/interim-image/lib/server/libjvm.so
>>> ./support/modules_libs/java.base/server/libjvm.so
>>> ./hotspot/variant-server/libjvm/gtest/libjvm.so
>>> ./images/test/hotspot/gtest/server/libjvm.so
>>> ./images/serverjre/lib/server/libjvm.so
>>> ./images/jdk/lib/server/libjvm.so
>>> ./images/jre/lib/server/libjvm.so
>>> ./jdk/lib/server/libjvm.so
>>>
>>> Each of them is 320+MB. Even though I have a fast SSD, it's overwhelmed
>>> by the large churn and my machine becomes unresponsive for a long time.
>>>
>>> The 8 libjvm.so files have 2 variants -- the gtest version (2 of them) vs
>>> the normal version (6 of them). Would it make sense to change the 'cp' to
>>> hard links instead?
>>>
>>> For now, I am doing a hack by patching this in the generated spec.gmk
>>> file:
>>>
>>>     CP:=/bin/cp
>>>
>>> to point to my script, which uses 'ln' instead of /bin/cp when dealing
>>> with libjvm.so ....
>>>
>>> BTW, what's the use for the 'gtest' versions of libjvm.so?
>>>
>>> Thanks
>>> - Ioi
>>
>>
>