[External] : Re: RFC: Change CDS JAR file validation
Jiangli Zhou
jianglizhou at google.com
Thu Sep 15 00:47:43 UTC 2022
On Mon, Sep 12, 2022 at 10:52 AM Ioi Lam <ioi.lam at oracle.com> wrote:
> On 9/7/2022 9:07 PM, Jiangli Zhou wrote:
> > On Thu, Sep 1, 2022 at 12:51 PM Jiangli Zhou <jianglizhou at google.com>
> wrote:
> >> When utilizing CDS for tools in a cloud environment a few years back,
> >> we ran into the path checking issue. One of the main problems that we
> >> observed was that the mtime based check was not reliable. Internally,
> >> we've explored a few potential solutions. One of suggested ideas was
> >> to compute a checksum of the jar file and store the value in the zip
> >> central directory. Runtime can then validate the checksum. That can be
> >> reliable. It may require specification changes.
> > Some additional details about the idea using the zip central directory
> > for storing checksum (which was suggested by @martin):
> >
> > The checksum would be computed at JAR creation time and stored in the
> > zip central directory. The checksum can be updated when the JAR file
> > is updated. At CDS image creation time, the JAR checksum could be
> > obtained and stored in the CDS image header. Runtime can then compare
> > the checksums to validate compatibility. The advantage is that it
> > avoids having to compute the checksum at both CDS creation time and
> > runtime. However, it requires JAR tools to be updated to support that.
>
> Hi Jiangli,
>
> Is there a field in the central directory for storing the creation time?
> I can't find that in the zip specification:
>
> https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
If I recall correctly, @martin probably suggested using the "extra field"
in a central directory header, during the brainstorming from a few years
back. That's a variable sized field. Could you please look into that? As
mentioned earlier, we didn't pursue this direction as the use cases that we
found (mostly likely are common use cases in practices) provided JAR<->JSA
compatibility at runtime. For cases that do not have such a guarantee
(possibly less common), runtime checks probably are still useful.
> By the way, I tested all version of JDK down to JDK 6, and all of them
> would create a META-INF/ entry at the very beginning of the JAR file
> that has the creation time of the JAR file.
>
> So I think we can effectively use the first 128 bytes of the JAR file to
> determine the creation time.
>
A zip file may have program code appended. Please see ZIP (file format)
<https://en.wikipedia.org/wiki/ZIP_(file_format)#:~:text=The%20ZIP%20format%20uses%20specific,byte%20central%20file%20header%20signature>
for
a brief mention. However, I guess it could always seek to the beginning of
the JAR content then check.
> There's a catch. If the JAR file was not created using the official JAR
> file (e.g., using zip.exe), we may not have the META-INF entry.Or the
> META_INF entry may not have the actual creation time. E.g.,
>
Yes. That could be fragile.
Thanks,
Jiangli
>
> zip foo.jar META-INF/MANIFEST.MF HelloWorld.class
>
>
> /jdk/official$ date
> Mon Sep 12 10:42:20 AM PDT 2022
> /jdk/official$ for i in $(find . -name jar | sort); do echo $i; rm -f
> foo.zip; $i cf foo.jar HelloWorld.class; unzip -lv foo.jar; head -c 128
> foo.jar | hexdump -C; done
> ./jdk10/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 62 Defl:N 62 0% 2022-09-12 10:42 3a1538ab
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 647 456 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk11/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 62 Defl:N 62 0% 2022-09-12 10:42 dd089e3c
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 647 456 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk12/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 62 Defl:N 62 0% 2022-09-12 10:42 2f5f73c4
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 647 456 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk13/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 62 Defl:N 62 0% 2022-09-12 10:42 c842d553
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 647 456 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk14/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 62 Defl:N 62 0% 2022-09-12 10:42 1081ae75
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 647 456 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk15/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 62 Defl:N 62 0% 2022-09-12 10:42 f79c08e2
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 647 456 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk16/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 62 Defl:N 62 0% 2022-09-12 10:42 05cbe51a
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 647 456 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk17/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 62 Defl:N 62 0% 2022-09-12 10:42 e2d6438d
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 647 456 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk1.8.0_261/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 69 Defl:N 68 1% 2022-09-12 10:42 41397d72
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 654 462 29% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk18/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 62 Defl:N 62 0% 2022-09-12 10:42 6f3c1517
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 647 456 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk6u_211/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 72 Defl:N 72 0% 2022-09-12 10:42 27324655
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 657 466 29% 3 files
> 00000000 50 4b 03 04 14 00 08 00 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 00 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk7u_321/fastdebug/bin/jar
> Error occurred during initialization of VM
> java/lang/ClassNotFoundException: error in opening JAR file <Zip file
> open error>
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 72 Defl:N 72 0% 2022-09-12 10:42 27324655
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 657 466 29% 3 files
> 00000000 50 4b 03 04 14 00 08 00 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 00 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk8_b132/linux_amd64/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 65 Defl:N 64 2% 2022-09-12 10:42 19255984
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 650 458 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk8_b132/linux_i586/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 65 Defl:N 64 2% 2022-09-12 10:42 19255984
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 650 458 30% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
> ./jdk9/bin/jar
> Archive: foo.jar
> Length Method Size Cmpr Date Time CRC-32 Name
> -------- ------ ------- ---- ---------- ----- -------- ----
> 0 Defl:N 2 0% 2022-09-12 10:42 00000000 META-INF/
> 61 Defl:N 62 -2% 2022-09-12 10:42 3912bcee
> META-INF/MANIFEST.MF
> 585 Defl:N 392 33% 2022-09-12 10:39 4af91087 HelloWorld.class
> -------- ------- --- -------
> 646 456 29% 3 files
> 00000000 50 4b 03 04 14 00 08 08 08 00 48 55 2c 55 00 00
> |PK........HU,U..|
> 00000010 00 00 00 00 00 00 00 00 00 00 09 00 04 00 4d 45
> |..............ME|
> 00000020 54 41 2d 49 4e 46 2f fe ca 00 00 03 00 50 4b 07
> |TA-INF/......PK.|
> 00000030 08 00 00 00 00 02 00 00 00 00 00 00 00 50 4b 03
> |.............PK.|
> 00000040 04 14 00 08 08 08 00 48 55 2c 55 00 00 00 00 00
> |.......HU,U.....|
> 00000050 00 00 00 00 00 00 00 14 00 00 00 4d 45 54 41 2d
> |...........META-|
> 00000060 49 4e 46 2f 4d 41 4e 49 46 45 53 54 2e 4d 46 f3
> |INF/MANIFEST.MF.|
> 00000070 4d cc cb 4c 4b 2d 2e d1 0d 4b 2d 2a ce cc cf b3
> |M..LK-...K-*....|
> 00000080
>
>
> Thanks
> - Ioi
>
> >> Another approach is to provide a runtime flag (e.g.
> >> -XX:+|-ValidateSharedClassPaths), which can be used to disable the
> >> problematic jar path checking in use cases where it is safe to do so.
> >> This is the approach that we have been using for tools that use CDS.
> >> The tools' JAR files and CDS images are built and released together.
> >> As the release progresses guarantee the compatibility between the JAR
> >> file and the CDS archive in these kinds of usages, we can safely
> >> disable the corresponding path checking. It also minimizes the related
> >> overhead. I'd like to contribute the related patch in the short term.
> > Created https://bugs.openjdk.org/browse/JDK-8293526 (apologizing for
> > duplicating with https://bugs.openjdk.org/browse/JDK-8284692).
> >
> > Thanks,
> > Jiangli
> >
> >> Thanks,
> >> Jiangli
> >>
> >> On Wed, Aug 31, 2022 at 9:47 PM Ioi Lam <ioi.lam at oracle.com> wrote:
> >>> Proposal Summary:
> >>>
> >>> Use a digest of a JAR file to detect if the file has changed
> >>>
> >>>
> >>> Background
> >>> ==========
> >>>
> >>> CDS is in effect a caching mechanism -- it needs to make sure that the
> >>> InstanceKlasses stored in the archive are the same as those parsed from
> >>> classfiles.
> >>>
> >>> To do this, we archive only the classes from (a) the JDK's modules
> image
> >>> file and (b) JAR files. We don't archive classes in directories since
> >>> it's difficult to check if the contents of a directory have changed.
> >>>
> >>> At runtime, we assume that (a) didn't change, since we require the
> exact
> >>> same JDK build to be used.
> >>>
> >>> For (b) we currently do this:
> >>>
> >>> (1) Check that -classpath and -Xbootclasspath (absolute paths) are
> >>> identical between run time and dump time.
> >>> (2) For each JAR file in cp and bcp, check if its size and modification
> >>> time has changed.
> >>> (3) (Something similar happens with the module path ....)
> >>>
> >>> We have used this scheme for almost a decade. Note that we avoid
> reading
> >>> the JAR files as that may slow down start-up time on old spinning
> disks.
> >>> However, as most work-loads run on SSDs now, I believe this is no
> longer
> >>> a concern.
> >>>
> >>> Recently, we are seeing problems when people deploy CDS inside
> containers:
> >>>
> >>> For (1) the file system structure may different between run time and
> >>> dump time. We can kludge around this problem by using relative paths
> >>> instead of absolute paths, but this will make the existing code even
> >>> more complicated.
> >>>
> >>> For (2) when deploying the app, it may not be easy to keep the
> >>> modification time unchanged (see JDK-8284692).
> >>>
> >>>
> >>> Proposal
> >>> ========
> >>>
> >>> For (1) - don't not compare directory name anymore. Only check that the
> >>> filename is the same:
> >>>
> >>> E.g.
> >>>
> >>> Dump:
> >>> java -Xshare:dump -cp dir1/Foo.jar:dir2/Bar.jar ..
> >>>
> >>> Run:
> >>> java -cp dir1/Foo.jar:dir2/Bar.jar ... [OK]
> >>> java -cp Foo.jar:Bar.jar ... [OK]
> >>> java -cp Foo.jar:Bxx.jar ... [Fail - changed from Bar.jar to
> Bxx.jar]
> >>>
> >>> For (2)
> >>>
> >>> - Check that file size has not changed.
> >>> - Compute a digest of the file. Check that this has not changed.
> >>>
> >>>
> >>> Digest
> >>> ======
> >>>
> >>> The purpose is not for security or (malicious) tamper detection. It's
> >>> for guarding against innocent mistakes (forgot to regenerate CDS
> archive
> >>> after JAR files have been updated). Therefore, we don't need to run an
> >>> expensive digest like MD5.
> >>>
> >>> Instead, it should be enough to just do a quick XOR digest of the first
> >>> 128 bytes of the JAR file. Since this part usually contains the
> >>> META-INF/ directory and its modification time. So it effectively
> >>> contains the time when this JAR file was created. The timestamp seems
> to
> >>> have a 2 second resolution:
> >>>
> >>> $ while true; do jar cfm foo.jar MANIFEST.MF HelloWorld.class ; head -c
> >>> 128 foo.jar | cksum; sleep 2; done
> >>> 3803507028 <(380)%20350-7028> 128
> >>> 1857545662 128
> >>> 916098721 128
> >>> 3740087168 128
> >>> 2260752543 128
> >>> 3257546723 <(325)%20754-6723> 128
> >>> 2584173820 128
> >>> ...
> >>>
> >>>
> >>> Advantage:
> >>>
> >>> - Make it easier to deploy CDS archive (fewer false negatives)
> >>> - Simplify logic in the CDS code
> >>>
> >>> Risks:
> >>>
> >>> - Opening every JAR file may cause a slow down if you have lots of JARs
> >>> in the classpath running on a slow file system.
> >>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.openjdk.org/pipermail/hotspot-runtime-dev/attachments/20220914/d4c98961/attachment-0001.htm>
More information about the hotspot-runtime-dev
mailing list