[External] : Re: RFC: Change CDS JAR file validation
Ioi Lam
ioi.lam at oracle.com
Mon Sep 12 17:52:11 UTC 2022
On 9/7/2022 9:07 PM, Jiangli Zhou wrote:
> On Thu, Sep 1, 2022 at 12:51 PM Jiangli Zhou <jianglizhou at google.com> wrote:
>> When utilizing CDS for tools in a cloud environment a few years back,
>> we ran into the path checking issue. One of the main problems that we
>> observed was that the mtime based check was not reliable. Internally,
>> we've explored a few potential solutions. One of suggested ideas was
>> to compute a checksum of the jar file and store the value in the zip
>> central directory. Runtime can then validate the checksum. That can be
>> reliable. It may require specification changes.
> Some additional details about the idea using the zip central directory
> for storing checksum (which was suggested by @martin):
>
> The checksum would be computed at JAR creation time and stored in the
> zip central directory. The checksum can be updated when the JAR file
> is updated. At CDS image creation time, the JAR checksum could be
> obtained and stored in the CDS image header. Runtime can then compare
> the checksums to validate compatibility. The advantage is that it
> avoids having to compute the checksum at both CDS creation time and
> runtime. However, it requires JAR tools to be updated to support that.
Hi Jiangli,
Is there a field in the central directory for storing the creation time?
I can't find that in the zip specification:
https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
By the way, I tested all version of JDK down to JDK 6, and all of them
would create a META-INF/ entry at the very beginning of the JAR file
that has the creation time of the JAR file.
So I think we can effectively use the first 128 bytes of the JAR file to
determine the creation time.
There's a catch. If the JAR file was not created using the official JAR
file (e.g., using zip.exe), we may not have the META-INF entry.Or the
META_INF entry may not have the actual creation time. E.g.,
zip foo.jar META-INF/MANIFEST.MF HelloWorld.class
/jdk/official$ date
Mon Sep 12 10:42:20 AM PDT 2022
/jdk/official$ for i in $(find . -name jar | sort); do echo $i; rm -f
foo.zip; $i cf foo.jar HelloWorld.class; unzip -lv foo.jar; head -c 128
foo.jar | hexdump -C; done
./jdk10/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
62 Defl:N 62 0% 2022-09-12 10:42 3a1538ab
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
647 456 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk11/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
62 Defl:N 62 0% 2022-09-12 10:42 dd089e3c
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
647 456 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk12/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
62 Defl:N 62 0% 2022-09-12 10:42 2f5f73c4
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
647 456 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk13/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
62 Defl:N 62 0% 2022-09-12 10:42 c842d553
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
647 456 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk14/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
62 Defl:N 62 0% 2022-09-12 10:42 1081ae75
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
647 456 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk15/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
62 Defl:N 62 0% 2022-09-12 10:42 f79c08e2
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
647 456 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk16/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
62 Defl:N 62 0% 2022-09-12 10:42 05cbe51a
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
647 456 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk17/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
62 Defl:N 62 0% 2022-09-12 10:42 e2d6438d
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
647 456 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk1.8.0_261/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
69 Defl:N 68 1% 2022-09-12 10:42 41397d72
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
654 462 29% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk18/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
62 Defl:N 62 0% 2022-09-12 10:42 6f3c1517
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
647 456 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk6u_211/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
72 Defl:N 72 0% 2022-09-12 10:42 27324655
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
657 466 29% 3 files
00000000 50 4b 03 04 14 00 08 00 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 00 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk7u_321/fastdebug/bin/jar
Error occurred during initialization of VM
java/lang/ClassNotFoundException: error in opening JAR file <Zip file
open error>
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
72 Defl:N 72 0% 2022-09-12 10:42 27324655
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
657 466 29% 3 files
00000000 50 4b 03 04 14 00 08 00 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 00 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk8_b132/linux_amd64/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
65 Defl:N 64 2% 2022-09-12 10:42 19255984
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
650 458 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk8_b132/linux_i586/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
65 Defl:N 64 2% 2022-09-12 10:42 19255984
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
650 458 30% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
./jdk9/bin/jar
Archive: foo.jar
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
61 Defl:N 62 -2% 2022-09-12 10:42 3912bcee
META-INF/MANIFEST.MF
585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
-------- ------- --- -------
646 456 29% 3 files
00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
|PK........HU,U..|
00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
|..............ME|
00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
|TA-INF/......PK.|
00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
|.............PK.|
00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
|.......HU,U.....|
00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
|...........META-|
00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
|INF/MANIFEST.MF.|
00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
|M..LK-...K-*....|
00000080
Thanks
- Ioi
>> Another approach is to provide a runtime flag (e.g.
>> -XX:+|-ValidateSharedClassPaths), which can be used to disable the
>> problematic jar path checking in use cases where it is safe to do so.
>> This is the approach that we have been using for tools that use CDS.
>> The tools' JAR files and CDS images are built and released together.
>> As the release progresses guarantee the compatibility between the JAR
>> file and the CDS archive in these kinds of usages, we can safely
>> disable the corresponding path checking. It also minimizes the related
>> overhead. I'd like to contribute the related patch in the short term.
> Created https://bugs.openjdk.org/browse/JDK-8293526 (apologizing for
> duplicating with https://bugs.openjdk.org/browse/JDK-8284692).
>
> Thanks,
> Jiangli
>
>> Thanks,
>> Jiangli
>>
>> On Wed, Aug 31, 2022 at 9:47 PM Ioi Lam <ioi.lam at oracle.com> wrote:
>>> Proposal Summary:
>>>
>>> Use a digest of a JAR file to detect if the file has changed
>>>
>>>
>>> Background
>>> ==========
>>>
>>> CDS is in effect a caching mechanism -- it needs to make sure that the
>>> InstanceKlasses stored in the archive are the same as those parsed from
>>> classfiles.
>>>
>>> To do this, we archive only the classes from (a) the JDK's modules image
>>> file and (b) JAR files. We don't archive classes in directories since
>>> it's difficult to check if the contents of a directory have changed.
>>>
>>> At runtime, we assume that (a) didn't change, since we require the exact
>>> same JDK build to be used.
>>>
>>> For (b) we currently do this:
>>>
>>> (1) Check that -classpath and -Xbootclasspath (absolute paths) are
>>> identical between run time and dump time.
>>> (2) For each JAR file in cp and bcp, check if its size and modification
>>> time has changed.
>>> (3) (Something similar happens with the module path ....)
>>>
>>> We have used this scheme for almost a decade. Note that we avoid reading
>>> the JAR files as that may slow down start-up time on old spinning disks.
>>> However, as most work-loads run on SSDs now, I believe this is no longer
>>> a concern.
>>>
>>> Recently, we are seeing problems when people deploy CDS inside containers:
>>>
>>> For (1) the file system structure may different between run time and
>>> dump time. We can kludge around this problem by using relative paths
>>> instead of absolute paths, but this will make the existing code even
>>> more complicated.
>>>
>>> For (2) when deploying the app, it may not be easy to keep the
>>> modification time unchanged (see JDK-8284692).
>>>
>>>
>>> Proposal
>>> ========
>>>
>>> For (1) - don't not compare directory name anymore. Only check that the
>>> filename is the same:
>>>
>>> E.g.
>>>
>>> Dump:
>>> java -Xshare:dump -cp dir1/Foo.jar:dir2/Bar.jar ..
>>>
>>> Run:
>>> java -cp dir1/Foo.jar:dir2/Bar.jar ... [OK]
>>> java -cp Foo.jar:Bar.jar ... [OK]
>>> java -cp Foo.jar:Bxx.jar ... [Fail - changed from Bar.jar to Bxx.jar]
>>>
>>> For (2)
>>>
>>> - Check that file size has not changed.
>>> - Compute a digest of the file. Check that this has not changed.
>>>
>>>
>>> Digest
>>> ======
>>>
>>> The purpose is not for security or (malicious) tamper detection. It's
>>> for guarding against innocent mistakes (forgot to regenerate CDS archive
>>> after JAR files have been updated). Therefore, we don't need to run an
>>> expensive digest like MD5.
>>>
>>> Instead, it should be enough to just do a quick XOR digest of the first
>>> 128 bytes of the JAR file. Since this part usually contains the
>>> META-INF/ directory and its modification time. So it effectively
>>> contains the time when this JAR file was created. The timestamp seems to
>>> have a 2 second resolution:
>>>
>>> $ while true; do jar cfm foo.jar MANIFEST.MF HelloWorld.class ; head -c
>>> 128 foo.jar | cksum; sleep 2; done
>>> 3803507028 128
>>> 1857545662 128
>>> 916098721 128
>>> 3740087168 128
>>> 2260752543 128
>>> 3257546723 128
>>> 2584173820 128
>>> ...
>>>
>>>
>>> Advantage:
>>>
>>> - Make it easier to deploy CDS archive (fewer false negatives)
>>> - Simplify logic in the CDS code
>>>
>>> Risks:
>>>
>>> - Opening every JAR file may cause a slow down if you have lots of JARs
>>> in the classpath running on a slow file system.
>>>
More information about the hotspot-runtime-dev
mailing list