<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi,<br>
<br>
Recently, I tried to fix only bug 6202130 with the intention to fix
bug 6443578 later with the intention to get some opportunity for
feedback, but haven't got any, and propose now a fix for both
together which in my opinion makes more sense.<br>
<br>
See attached patch.<br>
<br>
Some considerations, assumptions, and explanations<br>
<ul>
<li>In my opinion, the code for writing manifests was distributed
in the two classes Attributes and Manifest in an elegant way but
somewhat difficult to explain the coherence. I chose to group
the code that writes manifests into a new class ManifestWriter.
The main incentive for that was to prevent or reduce duplicated
code I would have had to change twice otherwise. This also
results in a source file of a suitable size.</li>
<li>I could not support the assumption that the write and
writeMain methods in Attributes couldn't be referenced anywhere
so I deprecated them rather than having them removed.<br>
</li>
<li>I assumed the patch will not make it into JDK 10 and, hence,
the deprecated annotations are attributed with since = 11.</li>
<li>I could not figure out any reason for the use of
DataOutputStream and did not use it.</li>
<li>Performance-wise I assume that the code is approximately
comparable to the previous version. The biggest improvement in
this respect I hope comes from removing the String that contains
the byte array constructed with deprecated String(byte[], int,
int, int) and then copying it over again to a StringBuffer and
from there to a String again and then Characters. On the other
hand, keeping whole characters together when breaking lines
might make it slightly slower. I hope my changes are an overall
improvement, but I haven't measured it.</li>
<li>For telling first from continuation bytes of utf-8 characters
apart I re-used a method isNotUtfContinuationByte from either
StringCoding or UTF_8.Decoder. Unfortunately I found no way not
to duplicate it.</li>
<li>Where it said before "XXX Need to handle UTF8 values and break
up lines longer than 72 bytes" in Attributes#writeMain I did not
dare to remove the comment completely because it still does not
deal correctly with version headers longer than 72 bytes and the
set of allowed values. I changed it accordingly. Two similar
comments are removed in the patch.<br>
</li>
<li>I added two tests, WriteDeprecated and NullKeysAndValues, to
demonstrate compatibility as good as I could. Might however not
be desired to keep and having to maintain.</li>
<li>LineBrokenMultiByteCharacter for jarsigner should not be
removed or not so immediately because someone might attempt to
sign an older jarfile created without that patch with a newer
jarsigner that already contains it.<br>
</li>
</ul>
<br>
<br>
suggested changes or additions to the bug database: (i have no
permissions to edit it myself)<br>
<ul>
<li>Re-combine copies of isNotUtfContinuationByte (three by now).
Relates to 6184334. Worth to file another issue?<br>
</li>
<li>Manifest versions have specific specifications, cannot break
across lines and can contain a subset of characters only. Bug
6910466 relates but is not exactly the same. If someone else is
convinced that writing a manifest should issue a warning or any
other way to deal with a version that does not conform to the
specification, I'd suggest to create a separate bug for that.<br>
</li>
</ul>
<br>
Now, I would be glad if someone sponsored a review. This is only my
third attempt to submit a patch which is why I chose a lesser
important subject to fix in order to get familiar and now I
understand it's not the most attractive patch to review. Please
don't hesitate to suggest what I could do better or differently.<br>
<br>
As a bonus, with these changes, manifest files will always be
displayed correctly with just any utf capable viewer even if they
contain multi-byte utf characters that would have been broken across
a line break with the current/previous implementation and all
manifests will become also valid strings in Java.<br>
<br>
Regards,<br>
Philipp<br>
<br>
<br>
<br>
On 20.04.2018 00:58, Philipp Kunz wrote:<br>
<blockquote
cite="mid:ec9706e3-6549-a3c5-68fa-815fdca9b9ca@paratix.ch"
type="cite">Hi,
<br>
<br>
I tried to fix bug 6202130 about manifest utf support and come up
now with a test as suggested in the bug's comments that shows that
utf charset actually works before removing the comments from the
code.
<br>
<br>
When I wanted to remove the XXX comments about utf it occurred to
me that version attributes ("Signature-Version" and
"Manifest-Version") would never be broken across lines and should
anyway not support the whole utf character set which sounds more
like related to bugs 6910466 or 4935610 but it's not a real fit.
Therefore, I could not remove one such comment of
Attributes#writeMain but I changed it. The first comment in bug
6202130 mentions only two comments but there are three in
Attributes. In the attached patch I removed only two of three and
changed the remaining third to not mention utf anymore.
<br>
<br>
At the moment, at least until 6443578 is fixed, multi-byte utf
characters can be broken across lines. It might be worth a
consideration to test that explicitly as well but then I guess
there is not much of a point in testing the current behavior that
will change with 6443578, hopefully soon. There are in my opinion
enough characters broken across lines in the attached test that
demonstrate that this still works like it did before.
<br>
<br>
I would have preferred also to remove the calls to deprecated
String(byte[], int, int, int) but then figured it relates more to
bug 6443578 than 6202130 and now prefer to do that in another
separate patch.
<br>
<br>
Bug 6202130 also states that lines are broken by String.length not
by byte length. While it looks so at first glance, I could not
confirm. The combination of getBytes("UTF8"), String(byte[], int,
int, int), and then DataOutputStream.writeBytes(String) in that
combination does not drop high-bytes because every byte (whether a
whole character or only a part of a multi-byte character) becomes
a character in String(...) containing that byte in its low-byte
which will be read again from writeBytes(...). Or put in a
different way, every utf encoded byte becomes a character and
multi-byte utf characters are converted into multiple string
characters containing one byte each in their lower bytes. The
current solution is not nice, but at least works. With that
respect I'd like to suggest to deprecate
DataOutputStream.writeBytes(String) because it does something not
exactly expected when guessing from its name and that would suit a
byte[] parameter better very much like it has been done with
String(byte[], int, int, int). Any advice about the procedure to
deprecate something?
<br>
<br>
I was surprised that it was not trivial to list all valid utf
characters. If someone has a better idea than isValidUtfCharacter
in the attached test, let me know.
<br>
<br>
Altogether, I would not consider 6202130 resolved completely,
unless maybe all remaining points are copied to 6443578 and maybe
another bug about valid values for "Signature-Version" and
"Manifest-Version" if at all desired. But still I consider the
attached patch an improvement and most of the remainder can then
be solved in 6443578 and so far I am looking forward to any kind
of feedback.
<br>
<br>
Regards,
<br>
Philipp
<br>
<br>
</blockquote>
<br>
<br>
<br>
<br>
diff -r 2ace90aec488
src/java.base/share/classes/java/util/jar/Attributes.java<br>
--- a/src/java.base/share/classes/java/util/jar/Attributes.java
Mon Apr 30 21:56:54 2018 -0400<br>
+++ b/src/java.base/share/classes/java/util/jar/Attributes.java
Wed May 02 07:20:46 2018 +0200<br>
@@ -296,27 +296,13 @@<br>
<br>
/*<br>
* Writes the current attributes to the specified data output
stream.<br>
- * XXX Need to handle UTF8 values and break up lines longer
than 72 bytes<br>
+ *<br>
+ * @deprecated moved to<br>
+ * {@link ManifestWriter#writeSection(java.io.OutputStream)}<br>
*/<br>
- @SuppressWarnings("deprecation")<br>
- void write(DataOutputStream os) throws IOException {<br>
- for (Entry<Object, Object> e : entrySet()) {<br>
- StringBuffer buffer = new StringBuffer(<br>
- ((Name)
e.getKey()).toString());<br>
- buffer.append(": ");<br>
-<br>
- String value = (String) e.getValue();<br>
- if (value != null) {<br>
- byte[] vb = value.getBytes("UTF8");<br>
- value = new String(vb, 0, 0, vb.length);<br>
- }<br>
- buffer.append(value);<br>
-<br>
- Manifest.make72Safe(buffer);<br>
- buffer.append("\r\n");<br>
- os.writeBytes(buffer.toString());<br>
- }<br>
- os.writeBytes("\r\n");<br>
+ @Deprecated(since = "11")<br>
+ void write(DataOutputStream os) throws IOException {<br>
+ new ManifestWriter(os).writeSection(this);<br>
}<br>
<br>
/*<br>
@@ -324,50 +310,16 @@<br>
* make sure to write out the MANIFEST_VERSION or
SIGNATURE_VERSION<br>
* attributes first.<br>
*<br>
- * XXX Need to handle UTF8 values and break up lines longer
than 72 bytes<br>
+ * @deprecated moved to<br>
+ * {@link ManifestWriter#writeMain(java.io.OutputStream)}<br>
*/<br>
- @SuppressWarnings("deprecation")<br>
- void writeMain(DataOutputStream out) throws IOException<br>
- {<br>
- // write out the *-Version header first, if it exists<br>
- String vername = Name.MANIFEST_VERSION.toString();<br>
- String version = getValue(vername);<br>
- if (version == null) {<br>
- vername = Name.SIGNATURE_VERSION.toString();<br>
- version = getValue(vername);<br>
- }<br>
-<br>
- if (version != null) {<br>
- out.writeBytes(vername+": "+version+"\r\n");<br>
- }<br>
-<br>
- // write out all attributes except for the version<br>
- // we wrote out earlier<br>
- for (Entry<Object, Object> e : entrySet()) {<br>
- String name = ((Name) e.getKey()).toString();<br>
- if ((version != null) &&
!(name.equalsIgnoreCase(vername))) {<br>
-<br>
- StringBuffer buffer = new StringBuffer(name);<br>
- buffer.append(": ");<br>
-<br>
- String value = (String) e.getValue();<br>
- if (value != null) {<br>
- byte[] vb = value.getBytes("UTF8");<br>
- value = new String(vb, 0, 0, vb.length);<br>
- }<br>
- buffer.append(value);<br>
-<br>
- Manifest.make72Safe(buffer);<br>
- buffer.append("\r\n");<br>
- out.writeBytes(buffer.toString());<br>
- }<br>
- }<br>
- out.writeBytes("\r\n");<br>
+ @Deprecated(since = "11")<br>
+ void writeMain(DataOutputStream os) throws IOException {<br>
+ new ManifestWriter(os).writeMain(this);<br>
}<br>
<br>
/*<br>
* Reads attributes from the specified input stream.<br>
- * XXX Need to handle UTF8 values.<br>
*/<br>
@SuppressWarnings("deprecation")<br>
void read(Manifest.FastInputStream is, byte[] lbuf) throws
IOException {<br>
diff -r 2ace90aec488
src/java.base/share/classes/java/util/jar/Manifest.java<br>
--- a/src/java.base/share/classes/java/util/jar/Manifest.java Mon
Apr 30 21:56:54 2018 -0400<br>
+++ b/src/java.base/share/classes/java/util/jar/Manifest.java Wed
May 02 07:20:46 2018 +0200<br>
@@ -26,7 +26,6 @@<br>
package java.util.jar;<br>
<br>
import java.io.FilterInputStream;<br>
-import java.io.DataOutputStream;<br>
import java.io.InputStream;<br>
import java.io.OutputStream;<br>
import java.io.IOException;<br>
@@ -143,31 +142,19 @@<br>
* @exception IOException if an I/O error has occurred<br>
* @see #getMainAttributes<br>
*/<br>
- @SuppressWarnings("deprecation")<br>
public void write(OutputStream out) throws IOException {<br>
- DataOutputStream dos = new DataOutputStream(out);<br>
- // Write out the main attributes for the manifest<br>
- attr.writeMain(dos);<br>
- // Now write out the per-entry attributes<br>
- for (Map.Entry<String, Attributes> e :
entries.entrySet()) {<br>
- StringBuffer buffer = new StringBuffer("Name: ");<br>
- String value = e.getKey();<br>
- if (value != null) {<br>
- byte[] vb = value.getBytes("UTF8");<br>
- value = new String(vb, 0, 0, vb.length);<br>
- }<br>
- buffer.append(value);<br>
- make72Safe(buffer);<br>
- buffer.append("\r\n");<br>
- dos.writeBytes(buffer.toString());<br>
- e.getValue().write(dos);<br>
- }<br>
- dos.flush();<br>
+ new ManifestWriter(out).write(this);<br>
}<br>
<br>
/**<br>
* Adds line breaks to enforce a maximum 72 bytes per line.<br>
+ * <br>
+ * @see ManifestWriter#write72broken(String)<br>
+ * @deprecated replaced by<br>
+ * {@link ManifestWriter#write72broken(String)}<br>
+ * which keeps whole utf character together when breaking lines<br>
*/<br>
+ @Deprecated(since = "11")<br>
static void make72Safe(StringBuffer line) {<br>
int length = line.length();<br>
int index = 72;<br>
diff -r 2ace90aec488
src/java.base/share/classes/java/util/jar/ManifestWriter.java<br>
--- /dev/null Thu Jan 01 00:00:00 1970 +0000<br>
+++
b/src/java.base/share/classes/java/util/jar/ManifestWriter.java
Wed May 02 07:20:46 2018 +0200<br>
@@ -0,0 +1,224 @@<br>
+/*<br>
+ * Copyright (c) 2018, Oracle and/or its affiliates. All rights
reserved.<br>
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.<br>
+ *<br>
+ * This code is free software; you can redistribute it and/or
modify it<br>
+ * under the terms of the GNU General Public License version 2
only, as<br>
+ * published by the Free Software Foundation. Oracle designates
this<br>
+ * particular file as subject to the "Classpath" exception as
provided<br>
+ * by Oracle in the LICENSE file that accompanied this code.<br>
+ *<br>
+ * This code is distributed in the hope that it will be useful, but
WITHOUT<br>
+ * ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or<br>
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
License<br>
+ * version 2 for more details (a copy is included in the LICENSE
file that<br>
+ * accompanied this code).<br>
+ *<br>
+ * You should have received a copy of the GNU General Public
License version<br>
+ * 2 along with this work; if not, write to the Free Software
Foundation,<br>
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.<br>
+ *<br>
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA
94065 USA<br>
+ * or visit <a class="moz-txt-link-abbreviated" href="http://www.oracle.com">www.oracle.com</a> if you need additional information or
have any<br>
+ * questions.<br>
+ */<br>
+<br>
+package java.util.jar;<br>
+<br>
+import java.io.IOException;<br>
+import java.io.OutputStream;<br>
+import java.nio.charset.Charset;<br>
+import java.nio.charset.StandardCharsets;<br>
+import java.util.Map;<br>
+import java.util.Map.Entry;<br>
+import java.util.jar.Attributes.Name;<br>
+<br>
+/**<br>
+ * ManifestWriter writes manifests and takes care of their
structure, <br>
+ * correct encoding regarding character set, and maximum line
width.<br>
+ * <p><br>
+ * For information on the Manifest format, please see the<br>
+ * <a href=<a class="moz-txt-link-rfc2396E" href="mailto:{@docRoot}/../specs/jar/jar.html">"{@docRoot}/../specs/jar/jar.html"</a>><br>
+ * Manifest format specification</a>.<br>
+ * <br>
+ * @see Manifest#write(OutputStream)<br>
+ * @since 11<br>
+ */<br>
+class ManifestWriter {<br>
+<br>
+ /**<br>
+ * The utf-8 character set used for {@link Manifest}s.<br>
+ */<br>
+ private static final java.nio.charset.Charset UTF_8 =<br>
+ StandardCharsets.UTF_8;<br>
+<br>
+ private static final String<br>
+ INDIVIDUAL_SECTION_NAME_HEADER_KEY = "Name",<br>
+ KEY_VALUE_SEPARATOR = ": ",<br>
+ LINE_BREAK = "\r\n",<br>
+ CONTINUATION = " ";<br>
+<br>
+ /**<br>
+ * The <a
href=<a class="moz-txt-link-rfc2396E" href="mailto:{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files">"{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files"</a>><br>
+ * "Notes on Manifest and Signature Files"</a> in the "JAR
File<br>
+ * Specification" state that <q>no line may be longer than<br>
+ * <strong>72</strong> bytes (not characters), in
its utf8-encoded form.</q><br>
+ */<br>
+ private static final int MANIFEST_LINE_WIDTH = 72;<br>
+<br>
+ /**<br>
+ * The underlying output stream where to write a {@link
Manifest} to by<br>
+ * {@link #write(Manifest)}.<br>
+ */<br>
+ private final OutputStream out;<br>
+<br>
+ /**<br>
+ * Only constructor.<br>
+ *<br>
+ * @param out output stream where to write a {@link Manifest}
to by<br>
+ * {@link #write(Manifest)}<br>
+ */<br>
+ ManifestWriter(OutputStream out) {<br>
+ this.out = out;<br>
+ }<br>
+<br>
+ private static byte[] encode(String value) {<br>
+ return value.getBytes(UTF_8);<br>
+ }<br>
+<br>
+ private void write(String value) throws IOException {<br>
+ out.write(encode(value));<br>
+ }<br>
+<br>
+ private void write(byte[] value, int off, int len) throws
IOException {<br>
+ out.write(value, off, len);<br>
+ }<br>
+<br>
+ private static final byte[] LINE_BREAK_BUF =
encode(LINE_BREAK);<br>
+<br>
+ private void writeLineBreak() throws IOException {<br>
+ out.write(LINE_BREAK_BUF);<br>
+ }<br>
+<br>
+ private static final byte[] LINE_BREAK_CONTINUATION_BUF =<br>
+ encode(LINE_BREAK + CONTINUATION);<br>
+<br>
+ private void writeContinuationLineBreak() throws IOException {<br>
+ out.write(LINE_BREAK_CONTINUATION_BUF);<br>
+ }<br>
+<br>
+ /**<br>
+ * Writes a header line with added line breaks to enforce a
maximum of 72<br>
+ * bytes per line with respect only to breaks whole characters
and writes<br>
+ * the result to the current {@link ManifestWriter}'s
underlying output<br>
+ * stream.<br>
+ */<br>
+ private void write72broken(String header) throws IOException {<br>
+ byte[] buf = encode(header);<br>
+ int l = buf.length;<br>
+ int p = 0; // start position in buf of current line to
write<br>
+ int n = MANIFEST_LINE_WIDTH; // number of bytes going on
current line<br>
+ while (true) {<br>
+ if (p + n >= l) {<br>
+ n = l - p; // remainder fits on current (last) line<br>
+ } else {<br>
+ /*<br>
+ * break whole utf characters only:<br>
+ * put the current line end position on the next
character<br>
+ * start position / utf non-continuation byte left
of and<br>
+ * including n initial value / the maximum line
width by<br>
+ * decreasing the current line end position n until
not<br>
+ * followed by an utf continuation byte in order to
not break<br>
+ * utf characters across line breaks<br>
+ */<br>
+ while (!isNotUtfContinuationByte(buf[p + n])) n--;<br>
+ }<br>
+ write(buf, p, n);<br>
+ p += n;<br>
+ if (p == l) break;<br>
+ n = MANIFEST_LINE_WIDTH - 1;<br>
+ writeContinuationLineBreak();<br>
+ }<br>
+ }<br>
+<br>
+ /**<br>
+ * @see StringCoding#isNotUtfContinuationByte(int)<br>
+ * @see sun.nio.cs.UTF_8.Decoder#isNotUtfContinuationByte(int)<br>
+ */<br>
+ private static boolean isNotUtfContinuationByte(byte b) {<br>
+ return (b & 0xc0) != 0x80;<br>
+ }<br>
+<br>
+ private static String composeHeaderLine(String name, String
value) {<br>
+ return name + KEY_VALUE_SEPARATOR + value;<br>
+ }<br>
+<br>
+ private void writeHeader(String name, String value) throws
IOException {<br>
+ write72broken(composeHeaderLine(name, value));<br>
+ writeLineBreak();<br>
+ }<br>
+<br>
+ /**<br>
+ * Writes the specified attributes to the current output
stream.<br>
+ */<br>
+ void writeSection(Attributes attributes) throws IOException {<br>
+ for (Entry<Object, Object> e : attributes.entrySet())
{<br>
+ String name = ((Name) e.getKey()).toString();<br>
+ writeHeader(name, (String) e.getValue());<br>
+ }<br>
+ writeLineBreak();<br>
+ }<br>
+<br>
+ /**<br>
+ * Writes the specified attributes to the current output
stream,<br>
+ * make sure to write out the MANIFEST_VERSION or
SIGNATURE_VERSION<br>
+ * attributes first.<br>
+ */<br>
+ /*<br>
+ * XXX Need to handle version compliant to specification:<br>
+ * - only digits and periods but no period at the beginning or
end<br>
+ * - not more than 72-16-1=55 characters in order not to exceed
line width<br>
+ * - no space after colon or rather change specs and only 54
characters max<br>
+ */<br>
+ void writeMain(Attributes attributes) throws IOException {<br>
+ // write out the *-Version header first, if it exists<br>
+ String vername = Name.MANIFEST_VERSION.toString();<br>
+ String version = attributes.getValue(vername);<br>
+ if (version == null) {<br>
+ vername = Name.SIGNATURE_VERSION.toString();<br>
+ version = attributes.getValue(vername);<br>
+ }<br>
+<br>
+ if (version != null) {<br>
+ // version header cannot be continued on next line<br>
+ write(composeHeaderLine(vername, version));<br>
+ writeLineBreak();<br>
+ }<br>
+<br>
+ // write out all attributes except for the version we wrote
out earlier<br>
+ for (Entry<Object, Object> e : attributes.entrySet())
{<br>
+ String name = ((Name) e.getKey()).toString();<br>
+ if ((version != null) &&
!(name.equalsIgnoreCase(vername))) {<br>
+ writeHeader(name, (String) e.getValue());<br>
+ }<br>
+ }<br>
+ writeLineBreak();<br>
+ }<br>
+<br>
+ /**<br>
+ * Writes the given {@link Manifest} to the output stream of
this<br>
+ * {@link ManifestWriter} instance.<br>
+ * <br>
+ * @param mf the {@link Manifest} to write<br>
+ */<br>
+ void write(Manifest mf) throws IOException {<br>
+ // Write out the main attributes for the manifest<br>
+ writeMain(mf.getMainAttributes());<br>
+ // Now write out the per-entry attributes<br>
+ for (Entry<String, Attributes> e :
mf.getEntries().entrySet()) {<br>
+ writeHeader(INDIVIDUAL_SECTION_NAME_HEADER_KEY,
e.getKey());<br>
+ writeSection(e.getValue());<br>
+ }<br>
+ }<br>
+<br>
+}<br>
diff -r 2ace90aec488
test/jdk/java/util/jar/Attributes/WriteDeprecated.java<br>
--- /dev/null Thu Jan 01 00:00:00 1970 +0000<br>
+++ b/test/jdk/java/util/jar/Attributes/WriteDeprecated.java Wed
May 02 07:20:46 2018 +0200<br>
@@ -0,0 +1,123 @@<br>
+/*<br>
+ * Copyright (c) 2018, Oracle and/or its affiliates. All rights
reserved.<br>
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.<br>
+ *<br>
+ * This code is free software; you can redistribute it and/or
modify it<br>
+ * under the terms of the GNU General Public License version 2
only, as<br>
+ * published by the Free Software Foundation.<br>
+ *<br>
+ * This code is distributed in the hope that it will be useful, but
WITHOUT<br>
+ * ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or<br>
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
License<br>
+ * version 2 for more details (a copy is included in the LICENSE
file that<br>
+ * accompanied this code).<br>
+ *<br>
+ * You should have received a copy of the GNU General Public
License version<br>
+ * 2 along with this work; if not, write to the Free Software
Foundation,<br>
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.<br>
+ *<br>
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA
94065 USA<br>
+ * or visit <a class="moz-txt-link-abbreviated" href="http://www.oracle.com">www.oracle.com</a> if you need additional information or
have any<br>
+ * questions.<br>
+ */<br>
+<br>
+import static java.nio.charset.StandardCharsets.UTF_8;<br>
+<br>
+import java.io.ByteArrayInputStream;<br>
+import java.io.ByteArrayOutputStream;<br>
+import java.io.DataOutputStream;<br>
+import java.io.IOException;<br>
+import java.util.jar.Attributes;<br>
+import java.util.jar.Attributes.Name;<br>
+import java.util.jar.Manifest;<br>
+import java.lang.reflect.Method;<br>
+<br>
+import org.testng.annotations.Test;<br>
+import static org.testng.Assert.*;<br>
+<br>
+/**<br>
+ * @test<br>
+ * @run testng/othervm --illegal-access=warn WriteDeprecated<br>
+ * @summary Tests that the Attribute's write methods still work
despite not<br>
+ * being used any longer by Manifest.<br>
+ */<br>
+@Deprecated<br>
+/*<br>
+ * Note to future maintainer:<br>
+ * In order to actually being able to test Attributes' write
methods work<br>
+ * as before normal manifest and attributes manipulation through
their public<br>
+ * api is not sufficient but then these methods were there before
and this<br>
+ * way it's ensured that the behavior does not change with that
respect.<br>
+ * Once module isolation is enforced some test cases will not any
longer be<br>
+ * possible and those now tested situations will be guaranteed not
to occur<br>
+ * any longer at all at which point the corresponding tests can be
removed<br>
+ * safely without replacement unless of course another way is found
to inject<br>
+ * the tested null values.<br>
+ * Another trick to access package private class members could be
to use<br>
+ * deserialization or adding a new class to the same package on the
classpath.<br>
+ * Here is not important how the methods are invoked because it
shows that<br>
+ * the behavior remains unchanged.<br>
+ */<br>
+public class WriteDeprecated {<br>
+<br>
+ static final Name KEY = new Name("some-key");<br>
+ static final String VALUE = "value";<br>
+<br>
+ static class AccessibleAttributes extends Attributes {<br>
+<br>
+ void invoke(String name, DataOutputStream os) throws
IOException {<br>
+ try {<br>
+ Method method = Attributes.class.getDeclaredMethod(<br>
+ name, DataOutputStream.class);<br>
+ method.setAccessible(true);<br>
+ method.invoke(this, os);<br>
+ } catch (ReflectiveOperationException e) {<br>
+ fail(e.getMessage(), e);<br>
+ }<br>
+ }<br>
+<br>
+ /**<br>
+ * @see Attributes#write(DataOutputStream)<br>
+ */<br>
+ void write(DataOutputStream os) throws IOException {<br>
+ invoke("write", os);<br>
+ }<br>
+<br>
+ /**<br>
+ * @see Attributes#writeMain(DataOutputStream)<br>
+ */<br>
+ void writeMain(DataOutputStream os) throws IOException {<br>
+ invoke("writeMain", os);<br>
+ }<br>
+<br>
+ }<br>
+<br>
+ @Test<br>
+ public void testWriteMethods() throws Exception {<br>
+ ByteArrayOutputStream out = new ByteArrayOutputStream();<br>
+ AccessibleAttributes attributes = new
AccessibleAttributes();<br>
+ attributes.put(Name.MANIFEST_VERSION, "1.0");<br>
+ attributes.put(KEY, VALUE);<br>
+ DataOutputStream dos = new DataOutputStream(out);<br>
+ attributes.writeMain(dos);<br>
+ attributes.remove(Name.MANIFEST_VERSION);<br>
+ dos.writeBytes("Name: " + VALUE + "\r\n");<br>
+ attributes.write(dos);<br>
+ dos.flush();<br>
+<br>
+ byte[] mfBytes = out.toByteArray();<br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+ System.out.print(new String(mfBytes, UTF_8));<br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+ Manifest mf = new Manifest(new
ByteArrayInputStream(mfBytes));<br>
+ assertEquals(mf.getMainAttributes().getValue(KEY), VALUE, <br>
+ "main attributes header value");<br>
+ Attributes attributes2 = mf.getAttributes(VALUE);<br>
+ assertNotNull(attributes2, "named section not found");<br>
+ assertEquals(attributes2.getValue(KEY), VALUE,<br>
+ "named section attributes header value");<br>
+ }<br>
+<br>
+}<br>
diff -r 2ace90aec488
test/jdk/java/util/jar/Manifest/LineBreakCharacter.java<br>
--- /dev/null Thu Jan 01 00:00:00 1970 +0000<br>
+++ b/test/jdk/java/util/jar/Manifest/LineBreakCharacter.java Wed
May 02 07:20:46 2018 +0200<br>
@@ -0,0 +1,405 @@<br>
+/*<br>
+ * Copyright (c) 2018, Oracle and/or its affiliates. All rights
reserved.<br>
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.<br>
+ *<br>
+ * This code is free software; you can redistribute it and/or
modify it<br>
+ * under the terms of the GNU General Public License version 2
only, as<br>
+ * published by the Free Software Foundation.<br>
+ *<br>
+ * This code is distributed in the hope that it will be useful, but
WITHOUT<br>
+ * ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or<br>
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
License<br>
+ * version 2 for more details (a copy is included in the LICENSE
file that<br>
+ * accompanied this code).<br>
+ *<br>
+ * You should have received a copy of the GNU General Public
License version<br>
+ * 2 along with this work; if not, write to the Free Software
Foundation,<br>
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.<br>
+ *<br>
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA
94065 USA<br>
+ * or visit <a class="moz-txt-link-abbreviated" href="http://www.oracle.com">www.oracle.com</a> if you need additional information or
have any<br>
+ * questions.<br>
+ */<br>
+<br>
+import static java.nio.charset.StandardCharsets.UTF_8;<br>
+<br>
+import java.io.ByteArrayInputStream;<br>
+import java.io.ByteArrayOutputStream;<br>
+import java.io.DataOutputStream;<br>
+import java.io.IOException;<br>
+import java.util.jar.Attributes;<br>
+import java.util.jar.Manifest;<br>
+import java.util.jar.Attributes.Name;<br>
+import java.util.List;<br>
+import java.util.LinkedList;<br>
+<br>
+import org.testng.annotations.Test;<br>
+import org.testng.annotations.DataProvider;<br>
+import static org.testng.Assert.*;<br>
+<br>
+/**<br>
+ * @test<br>
+ * @bug 6443578<br>
+ * @run testng/othervm --illegal-access=warn LineBreakCharacter<br>
+ * @summary Tests reading and writing of jar manifests with headers
broken<br>
+ * across lines in conjunction with multi-byte utf
characters.<br>
+ * <p><br>
+ * Line breaks may or may not occur within utf encoded characters
that are<br>
+ * represented with more than one byte. Even though characters
should not be<br>
+ * broken across lines according to the specification the previous
Manifest<br>
+ * implementation did it and this test makes sure that no
multi-byte characters<br>
+ * are broken apart across a line break when writing manifests and
manifests<br>
+ * are read correctly no matter if multi-byte characters are
continued after a<br>
+ * line break.<br>
+ * <p><br>
+ * Correct support of all utf characters is the concern of<br>
+ * {@link ValueUtfEncoding}.<br>
+ */<br>
+public class LineBreakCharacter {<br>
+<br>
+ static final int MANIFEST_LINE_CONTENT_WIDTH_BYTES = 72;<br>
+<br>
+ static final String FILL = "x";<br>
+<br>
+ /**<br>
+ * By using names of four characters the same values can be
used for<br>
+ * testing line breaks at exact positions for both header and
section<br>
+ * names, header names in main and named sections, because a
named<br>
+ * section name is represented with a header with key "Name"
that occurs<br>
+ * after a blank line and also has four characters.<br>
+ */<br>
+ static final String NAME = FILL + FILL + FILL + FILL;<br>
+<br>
+ /**<br>
+ * Cover main attributes headers, section names, and headers in
named<br>
+ * sections because an implementation might make a difference.<br>
+ */<br>
+ enum PositionInManifest {<br>
+ MAIN_ATTRIBUTES, SECTION_NAME, NAMED_SECTION;<br>
+ }<br>
+<br>
+ /**<br>
+ * @see LineBreakLineWidth#numByteUtfCharacter(int, int)<br>
+ */<br>
+ static String numByteUtfCharacter(int numBytes, int seed) {<br>
+ seed = seed < 0 ? -seed : seed;<br>
+ if (numBytes == 1) {<br>
+ seed %= 0x5F;<br>
+ seed += 0x20; // exclude control characters (0..0x19)
here<br>
+ } else if (numBytes == 2) {<br>
+ seed %= 0x800 - 0x80;<br>
+ seed += 0x80;<br>
+ } else if (numBytes == 3) {<br>
+ seed %= 0x10000 - 0x800 + 0xFDD0 - 0xFDEF + 0xFFFE -
0xFFFF;<br>
+ seed += 0x800<br>
+ + (seed >= 0xFDD0 ? 0xFDEF - 0xFDD0 : 0) //
non-characters<br>
+ + (seed % 0x10000) * (0xFFFF - 0xFFFE); // byte
order marks<br>
+ } else {<br>
+ seed %= 0x110000 - (0x10000 - 0xFFFE);<br>
+ seed += 0x10000<br>
+ + (seed % 0x10000) * (0xFFFF - 0xFFFE); // byte
order marks<br>
+ }<br>
+<br>
+ String string = new String(Character.toChars(seed));<br>
+ assertEquals(string.getBytes(UTF_8).length, numBytes,<br>
+ "self-test failed: unexpected utf encoded character
length");<br>
+ return string;<br>
+ }<br>
+<br>
+ @DataProvider(name = "lineBreakParameters")<br>
+ public static Object[][] lineBreakParameters() {<br>
+ LinkedList<Object[]> params = new
LinkedList<>();<br>
+<br>
+ // b: number of line breaks before character under test<br>
+ for (int b = 0; b <= 3; b++) {<br>
+<br>
+ // c: character utf encoded length in bytes<br>
+ for (int c = 1; c <= 4; c++) {<br>
+<br>
+ // p: potential break position offset in bytes<br>
+ // p = 0 => before character<br>
+ // p = c => after character and<br>
+ // 0 < p < c => character potentially
broken across line break<br>
+ // within the character<br>
+ for (int p = c; p >= 0; p--) {<br>
+<br>
+ // a: no or one character following the one
under test<br>
+ // (a = 0 meaning the character under test is
followed by<br>
+ // end of value which is also represented as a
line break)<br>
+ for (int a = 0; a <= 1; a++) {<br>
+<br>
+ // offset: so many characters (actually
bytes here<br>
+ // filled with one byte characters) are
needed to place<br>
+ // the next character into a position
relative to the<br>
+ // maximum line width that it may or may
not have to be<br>
+ // broken onto the next line<br>
+ int offset =<br>
+ // number of lines; - 1 is
continuation space<br>
+ b *
(MANIFEST_LINE_CONTENT_WIDTH_BYTES - 1)<br>
+ // line length minus "Name:
".length()<br>
+ + MANIFEST_LINE_CONTENT_WIDTH_BYTES
- 6<br>
+ // position of maximum line width
relative to<br>
+ // beginning of encoded character<br>
+ - p;<br>
+ int seed = ((a * 2 + p) * c + c) * 4 + b;<br>
+ String value = "";<br>
+ for (int i = 0; i < offset - 1; i++) {<br>
+ value += numByteUtfCharacter(1,
seed++);<br>
+ }<br>
+ // character before the one to test the
break<br>
+ value += FILL;<br>
+ String character = numByteUtfCharacter(c,
seed);<br>
+ value += character;<br>
+ for (int i = 0; i < a; i++) {<br>
+ // character after the one to test the
break<br>
+ value += FILL;<br>
+ }<br>
+<br>
+ for (PositionInManifest i :<br>
+ PositionInManifest.values()) {<br>
+<br>
+ params.add(new Object[] {<br>
+ b, c, p, a, i, character,
value});<br>
+ }<br>
+ }<br>
+ }<br>
+ }<br>
+ }<br>
+<br>
+ return params.toArray(new Object[0][0]);<br>
+ }<br>
+<br>
+ /**<br>
+ * Checks that multi-byte utf characters work well with line
breaks and<br>
+ * continuation lines in jar manifests.<br>
+ *<br>
+ * Produces otherwise arbitrary manifests so that a specific
character<br>
+ * will not just fit on the same line and a line break and
continuation<br>
+ * line are required with:<ul><br>
+ * <li>different encoded sizes of characters: one, two,
three, and four<br>
+ * bytes per character (including also utf bmp and surrogate
pairs)</li><br>
+ * <li>different amount of space remaining on the same
line or relative<br>
+ * position of the latest possible line break position with
respect to the<br>
+ * character that can be continued on the same line or will
have to be<br>
+ * continued on the next line after a line break</li><br>
+ * <li>different number of preceding line
breaks</li><br>
+ * </ul><br>
+ * For each of these cases the break position is verified in
the binary<br>
+ * manifest.<br>
+ */<br>
+ @Test(dataProvider = "lineBreakParameters")<br>
+ public void testWriteLineBreaksKeepCharactersTogether(int b,
int c, int p,<br>
+ int a, PositionInManifest i, String character, String
value)<br>
+ throws IOException {<br>
+ byte[] mfBytes = writeManifest(i, NAME, value);<br>
+<br>
+ // expect the whole character on the next line unless it
fits<br>
+ // completely on the current line<br>
+ boolean breakExpected = p < c;<br>
+ String brokenPart = character;<br>
+ if (breakExpected) {<br>
+ brokenPart = "\r\n " + brokenPart;<br>
+ }<br>
+ // expect a line break before the next character if there
is a next<br>
+ // character and the previous not already broken on next
line<br>
+ if (a == 1) {<br>
+ if (!breakExpected) {<br>
+ brokenPart += "\r\n ";<br>
+ }<br>
+ brokenPart += FILL;<br>
+ }<br>
+ brokenPart = FILL + brokenPart + "\r\n";<br>
+ assertOccurrence(mfBytes, brokenPart.getBytes(UTF_8));<br>
+ readManifestAndAssertValue(mfBytes, i, NAME, value);<br>
+ }<br>
+<br>
+ static byte[] writeManifest(PositionInManifest i, String name,<br>
+ String value) throws IOException {<br>
+ Manifest mf = new Manifest();<br>
+ mf.getMainAttributes().put(Name.MANIFEST_VERSION, "1.0");<br>
+ Attributes attributes = new Attributes();<br>
+<br>
+ switch (i) {<br>
+ case MAIN_ATTRIBUTES:<br>
+ mf.getMainAttributes().put(new Name(name), value);<br>
+ break;<br>
+ case SECTION_NAME:<br>
+ mf.getEntries().put(value, attributes);<br>
+ break;<br>
+ case NAMED_SECTION:<br>
+ mf.getEntries().put(NAME, attributes);<br>
+ attributes.put(new Name(name), value);<br>
+ break;<br>
+ }<br>
+<br>
+ ByteArrayOutputStream out = new ByteArrayOutputStream();<br>
+ mf.write(out);<br>
+ byte[] mfBytes = out.toByteArray();<br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+ System.out.print(new String(mfBytes, UTF_8));<br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+ return mfBytes;<br>
+ }<br>
+<br>
+ static void assertOccurrence(byte[] mf, byte[] part) {<br>
+ List<Integer> matchPos = new LinkedList<>();<br>
+ for (int i = 0; i < mf.length; i++) {<br>
+ for (int j = 0; j < part.length && i + j
<= mf.length; j++) {<br>
+ if (part[j] == 0) {<br>
+ if (i + j != mf.length) {<br>
+ break; // expected eof not found<br>
+ }<br>
+ } else if (i + j == mf.length) {<br>
+ break;<br>
+ } else if (mf[i + j] != part[j]) {<br>
+ break;<br>
+ }<br>
+ if (j == part.length - 1) {<br>
+ matchPos.add(i);<br>
+ }<br>
+ }<br>
+ }<br>
+ assertEquals(matchPos.size(), 1, "not " <br>
+ + (matchPos.size() < 1 ? "found" : "unique") +
": '"<br>
+ + new String(part, UTF_8) + "'");<br>
+ }<br>
+<br>
+ static void readManifestAndAssertValue(<br>
+ byte[] mfBytes, PositionInManifest i, String name,
String value)<br>
+ throws IOException {<br>
+ Manifest mf = new Manifest(new
ByteArrayInputStream(mfBytes));<br>
+<br>
+ switch (i) {<br>
+ case MAIN_ATTRIBUTES:<br>
+ assertEquals(mf.getMainAttributes().getValue(name),
value,<br>
+ "main attributes header value");<br>
+ break;<br>
+ case SECTION_NAME:<br>
+ Attributes attributes = mf.getAttributes(value);<br>
+ assertNotNull(attributes, "named section not found");<br>
+ break;<br>
+ case NAMED_SECTION:<br>
+ attributes =<br>
+ mf.getAttributes(NAME);<br>
+ assertEquals(attributes.getValue(name), value,<br>
+ "named section attributes header value");<br>
+ break;<br>
+ }<br>
+ }<br>
+<br>
+ @Test(dataProvider = "lineBreakParameters")<br>
+ public void readContinuationLines(int b, int c, int p, int a,<br>
+ PositionInManifest i, String character, String value)<br>
+ throws IOException {<br>
+ byte[] mfBytes = writeManifestWithBrokenCharacters(i, NAME,
value);<br>
+<br>
+ ByteArrayOutputStream buf = new ByteArrayOutputStream();<br>
+ buf.write(FILL.getBytes(UTF_8));<br>
+ byte[] characterBytes = character.getBytes(UTF_8);<br>
+ // the portion of the character that fits on the current
line before<br>
+ // a break at 72 bytes, ranges from nothing (p == 0) to the
whole<br>
+ // character (p == c)<br>
+ for (int j = 0; j < p; j++) {<br>
+ buf.write(characterBytes, j, 1);<br>
+ }<br>
+ // expect a line break at exactly 72 bytes from the
beginning of the<br>
+ // line unless the whole character fits on that line<br>
+ boolean breakExpected = p < c;<br>
+ if (breakExpected) {<br>
+ buf.write("\r\n ".getBytes(UTF_8));<br>
+ }<br>
+ // the remaining portion of the character, if any<br>
+ for (int j = p; j < c; j++) {<br>
+ buf.write(characterBytes, j, 1);<br>
+ }<br>
+ // expect another linebreak if the whole character fitted
on the same<br>
+ // line and there is another character<br>
+ if (a == 1) {<br>
+ if (c == p) {<br>
+ buf.write("\r\n ".getBytes(UTF_8));<br>
+ }<br>
+ buf.write(FILL.getBytes(UTF_8));<br>
+ }<br>
+ buf.write("\r\n".getBytes(UTF_8));<br>
+ byte[] brokenPart = buf.toByteArray();<br>
+ assertOccurrence(mfBytes, brokenPart);<br>
+ readManifestAndAssertValue(mfBytes, i, NAME, value);<br>
+ }<br>
+<br>
+ /*<br>
+ * From previous Manifest implementation reduced to the minimum
required to<br>
+ * demonstrate compatibility<br>
+ */<br>
+ @SuppressWarnings("deprecation")<br>
+ static byte[] writeManifestWithBrokenCharacters(<br>
+ PositionInManifest i, String name, String value)<br>
+ throws IOException {<br>
+ byte[] vb = value.getBytes("UTF8");<br>
+ value = new String(vb, 0, 0, vb.length);<br>
+ ByteArrayOutputStream out = new ByteArrayOutputStream();<br>
+ DataOutputStream dos = new DataOutputStream(out);<br>
+ String vername = Name.MANIFEST_VERSION.toString();<br>
+ dos.writeBytes(vername + ": 0.1\r\n");<br>
+<br>
+ if (i == PositionInManifest.MAIN_ATTRIBUTES) {<br>
+ StringBuffer buffer = new StringBuffer(name);<br>
+ buffer.append(": ");<br>
+ buffer.append(value);<br>
+ make72Safe(buffer);<br>
+ buffer.append("\r\n");<br>
+ dos.writeBytes(buffer.toString());<br>
+ }<br>
+ dos.writeBytes("\r\n");<br>
+<br>
+ if (i == PositionInManifest.SECTION_NAME ||<br>
+ i == PositionInManifest.NAMED_SECTION) {<br>
+ StringBuffer buffer = new StringBuffer("Name: ");<br>
+ if (i == PositionInManifest.SECTION_NAME) {<br>
+ buffer.append(value);<br>
+ } else {<br>
+ buffer.append(NAME);<br>
+ }<br>
+ make72Safe(buffer);<br>
+ buffer.append("\r\n");<br>
+ dos.writeBytes(buffer.toString());<br>
+<br>
+ if (i == PositionInManifest.NAMED_SECTION) {<br>
+ buffer = new StringBuffer(name);<br>
+ buffer.append(": ");<br>
+ buffer.append(value);<br>
+ make72Safe(buffer);<br>
+ buffer.append("\r\n");<br>
+ dos.writeBytes(buffer.toString());<br>
+ }<br>
+<br>
+ dos.writeBytes("\r\n");<br>
+ }<br>
+<br>
+ dos.flush();<br>
+ byte[] mfBytes = out.toByteArray();<br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+ System.out.print(new String(mfBytes, UTF_8));<br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+ return mfBytes;<br>
+ }<br>
+<br>
+ /**<br>
+ * Adds line breaks to enforce a maximum 72 bytes per line.<br>
+ * From previous Manifest implementation.<br>
+ */<br>
+ static void make72Safe(StringBuffer line) {<br>
+ try {<br>
+ Method method = Manifest.class.getDeclaredMethod(<br>
+ "make72Safe", StringBuffer.class);<br>
+ method.setAccessible(true);<br>
+ method.invoke(null, line);<br>
+ } catch (ReflectiveOperationException e) {<br>
+ fail(e.getMessage(), e);<br>
+ }<br>
+ }<br>
+<br>
+}<br>
diff -r 2ace90aec488
test/jdk/java/util/jar/Manifest/LineBreakLineWidth.java<br>
--- a/test/jdk/java/util/jar/Manifest/LineBreakLineWidth.java Mon
Apr 30 21:56:54 2018 -0400<br>
+++ b/test/jdk/java/util/jar/Manifest/LineBreakLineWidth.java Wed
May 02 07:20:46 2018 +0200<br>
@@ -26,9 +26,13 @@<br>
import java.io.ByteArrayInputStream;<br>
import java.io.ByteArrayOutputStream;<br>
import java.io.IOException;<br>
+import java.util.Set;<br>
+import java.util.TreeSet;<br>
import java.util.jar.Manifest;<br>
import java.util.jar.Attributes;<br>
import java.util.jar.Attributes.Name;<br>
+import java.util.stream.Collectors;<br>
+import java.util.stream.IntStream;<br>
<br>
import org.testng.annotations.Test;<br>
import static org.testng.Assert.*;<br>
@@ -37,8 +41,10 @@<br>
* @test<br>
* @bug 6372077<br>
* @run testng LineBreakLineWidth<br>
- * @summary write valid manifests with respect to line breaks<br>
- * and read any line width<br>
+ * @summary Write valid manifests with respect to line widths
breaking longer<br>
+ * lines, especially for 6372077 that the lines are
sufficiently wide<br>
+ * to contain all possible valid header names, and read
any line<br>
+ * width.<br>
*/<br>
public class LineBreakLineWidth {<br>
<br>
@@ -49,6 +55,11 @@<br>
final static int MAX_HEADER_NAME_LENGTH = 70;<br>
<br>
/**<br>
+ * maximum line width not including the line break<br>
+ */<br>
+ final static int MAX_LINE_CONTENT_LENGTH = 72;<br>
+<br>
+ /**<br>
* range of one..{@link #TEST_WIDTH_RANGE} covered in this test
that<br>
* exceeds the range of allowed header name lengths or line
widths<br>
* in order to also cover invalid cases beyond the valid
boundaries<br>
@@ -60,20 +71,27 @@<br>
final static int TEST_WIDTH_RANGE = 123;<br>
<br>
/**<br>
- * tests if only valid manifest files can written depending on
the header<br>
+ * Tests if only valid manifest files can written depending on
the header<br>
* name length or that an exception occurs already on the
attempt to write<br>
* an invalid one otherwise and that no invalid manifest can be
written.<br>
* <p><br>
- * due to bug JDK-6372077 it was possible to write a manifest
that could<br>
- * not be read again. independent of the actual manifest line
width, such<br>
+ * Due to bug JDK-6372077 it was possible to write a manifest
that could<br>
+ * not be read again. Independent of the actual manifest line
width, such<br>
* a situation should never happen, which is the subject of
this test.<br>
+ * <p><br>
+ * While this test covers header names which can contain only
one-byte utf<br>
+ * encoded characters and the resulting manifest line widths,
multi-byte<br>
+ * utf encoded characters can occur in header values, which is
subject of<br>
+ * {@link #testManifestLineWidthMaximumRange()}.<br>
*/<br>
@Test<br>
public void testWriteValidManifestOrException() throws
IOException {<br>
+ assertTrue(TEST_WIDTH_RANGE > MAX_HEADER_NAME_LENGTH);<br>
+<br>
/*<br>
- * multi-byte utf-8 characters cannot occur in header
names,<br>
+ * Multi-byte utf-8 characters cannot occur in header
names,<br>
* only in values which are not subject of this test here.<br>
- * hence, each character in a header name uses exactly one
byte and<br>
+ * Hence, each character in a header name uses exactly one
byte and<br>
* variable length utf-8 character encoding doesn't affect
this test.<br>
*/<br>
<br>
@@ -111,18 +129,18 @@<br>
}<br>
<br>
/**<br>
- * tests that manifest files can be read even if the line
breaks are<br>
+ * Tests that manifest files can be read even if the line
breaks are<br>
* placed in different positions than where the current JDK's<br>
* {@link Manifest#write(java.io.OutputStream)} would have put
it provided<br>
* the manifest is valid otherwise.<br>
* <p><br>
- * the <a
href=<a class="moz-txt-link-rfc2396E" href="mailto:{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files">"{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files"</a>><br>
- * "Notes on Manifest and Signature Files" in the "JAR File<br>
- * Specification"</a> state that "no line may be longer
than 72 bytes<br>
- * (not characters), in its utf8-encoded form." but allows for
earlier or<br>
+ * The <a
href=<a class="moz-txt-link-rfc2396E" href="mailto:{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files">"{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files"</a>><br>
+ * "Notes on Manifest and Signature Files"</a> in the
"JAR File<br>
+ * Specification" state that <q>no line may be longer
than 72 bytes (not<br>
+ * characters), in its utf8-encoded form.</q> but allows
for earlier or<br>
* additional line breaks.<br>
* <p><br>
- * the most important purpose of this test case is probably to
make sure<br>
+ * The most important purpose of this test case is probably to
make sure<br>
* that manifest files broken at 70 bytes line width written
with the<br>
* previous version of {@link Manifest} before this fix still
work well.<br>
*/<br>
@@ -186,7 +204,8 @@<br>
* form.<br>
*/<br>
String lineBrokenSectionName = breakLines(lineWidth, "Name:
" + name);<br>
- String lineBrokenNameAndValue = breakLines(lineWidth, name
+ ": " + value);<br>
+ String lineBrokenNameAndValue = breakLines(lineWidth, name
+ ": "<br>
+ + value);<br>
<br>
ByteArrayOutputStream mfBuf = new ByteArrayOutputStream();<br>
mfBuf.write("Manifest-Version: 1.0".getBytes(UTF_8));<br>
@@ -264,7 +283,7 @@<br>
return mfBytes;<br>
}<br>
<br>
- private static void printManifest(byte[] mfBytes) {<br>
+ static void printManifest(byte[] mfBytes) {<br>
final String sepLine =
"----------------------------------------------"<br>
+ "---------------------|-|-|"; // |-positions:
---68-70-72<br>
System.out.println(sepLine);<br>
@@ -272,13 +291,179 @@<br>
System.out.println(sepLine);<br>
}<br>
<br>
- private static void assertMainAndSectionValues(Manifest mf,
String name,<br>
+ static void assertMainAndSectionValues(Manifest mf, String
name,<br>
String value) {<br>
String mainValue = mf.getMainAttributes().getValue(name);<br>
- String sectionValue =
mf.getAttributes(name).getValue(name);<br>
-<br>
assertEquals(value, mainValue, "value different in main
section");<br>
+ Attributes attributes = mf.getAttributes(name);<br>
+ assertNotNull(attributes, "named section not found");<br>
+ String sectionValue = attributes.getValue(name);<br>
assertEquals(value, sectionValue, "value different in named
section");<br>
}<br>
<br>
+ @Test<br>
+ public void testWriteManifestOnOneLine() throws IOException {<br>
+ testWriteVersionInfoLineWidth(Name.MANIFEST_VERSION);<br>
+ }<br>
+<br>
+ @Test<br>
+ public void testWriteSignatureVersionOnOneLine() throws
IOException {<br>
+ testWriteVersionInfoLineWidth(Name.SIGNATURE_VERSION);<br>
+ }<br>
+<br>
+ /**<br>
+ * Tests that manifest versions "Manifest-Version" and
"Signature-Version"<br>
+ * are not continued on a next line.<br>
+ */<br>
+ void testWriteVersionInfoLineWidth(Name version) throws
IOException {<br>
+ // e: number of characters of version header exceeding line
width<br>
+ for (int e = 0; e <= 1; e++) {<br>
+ Manifest mf = new Manifest();<br>
+ mf.getMainAttributes().put(version,<br>
+
"01234567890123456789012345678901234567890123456789012345"<br>
+ .substring(0, MAX_HEADER_NAME_LENGTH<br>
+ - version.toString().length() + e));<br>
+ ByteArrayOutputStream out = new
ByteArrayOutputStream();<br>
+ mf.write(out);<br>
+ byte[] mfBytes = out.toByteArray();<br>
+ printManifest(mfBytes);<br>
+ // maximum line width can only be exceeded if header
not broken<br>
+ assertEquals(MAX_LINE_CONTENT_LENGTH + e,<br>
+ findMaximumLineWidth(mfBytes));<br>
+ }<br>
+ }<br>
+<br>
+ static int findMaximumLineWidth(byte[] mfBytes) {<br>
+ int max = 0;<br>
+ int b = 0; // start position of current line<br>
+ for (int i = 0; i < mfBytes.length; i++) {<br>
+ switch (mfBytes[i]) {<br>
+ case 10:<br>
+ case 13:<br>
+ max = Math.max(max, i - b);<br>
+ b = i + 1;<br>
+ }<br>
+ }<br>
+ return max;<br>
+ }<br>
+<br>
+ /**<br>
+ * @see LineBreakCharacter#numByteUtfCharacter(int, int)<br>
+ */<br>
+ static String numByteUtfCharacter(int numBytes, int seed) {<br>
+ seed = seed < 0 ? -seed : seed;<br>
+ if (numBytes == 1) {<br>
+ seed %= 0x5F;<br>
+ seed += 0x20; // exclude control characters (0..0x19)
here<br>
+ } else if (numBytes == 2) {<br>
+ seed %= 0x800 - 0x80;<br>
+ seed += 0x80;<br>
+ } else if (numBytes == 3) {<br>
+ seed %= 0x10000 - 0x800 + 0xFDD0 - 0xFDEF + 0xFFFE -
0xFFFF;<br>
+ seed += 0x800<br>
+ + (seed >= 0xFDD0 ? 0xFDEF - 0xFDD0 : 0) //
non-characters<br>
+ + (seed % 0x10000) * (0xFFFF - 0xFFFE); // byte
order marks<br>
+ } else {<br>
+ seed %= 0x110000 - (0x10000 - 0xFFFE);<br>
+ seed += 0x10000<br>
+ + (seed % 0x10000) * (0xFFFF - 0xFFFE); // byte
order marks<br>
+ }<br>
+<br>
+ String string = new String(Character.toChars(seed));<br>
+ assertEquals(string.getBytes(UTF_8).length, numBytes,<br>
+ "self-test failed: unexpected utf encoded character
length");<br>
+ return string;<br>
+ }<br>
+<br>
+ /**<br>
+ * Tests that the manifest line content width is between
<em>69 (1 +<br>
+ * maximum line width (without line break) 72 - maximum utf
encoded<br>
+ * character length 4)</em> and <em>72</em>
before continued on the next<br>
+ * line, aka continuation line, depending only on the last
character<br>
+ * before the line break and not on those before or after in
order to<br>
+ * ensure the minimum number of line breaks and continuation
lines<br>
+ * possible.<br>
+ * <p><br>
+ * The <a
href=<a class="moz-txt-link-rfc2396E" href="mailto:{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files">"{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files"</a>><br>
+ * "Notes on Manifest and Signature Files"</a> in the
"JAR File<br>
+ * Specification" state that <q>no line may be longer
than 72 bytes (not<br>
+ * characters), in its utf8-encoded form.</q> and allows
for earlier or<br>
+ * additional line breaks.<br>
+ * <p><br>
+ * The number of characters a line may be less than the number
of bytes<br>
+ * but this is not explicitly verified because it can be
assumed if some<br>
+ * characters use more than one byte each and the number of
bytes is as<br>
+ * expected up to 72 that fewer than 72 characters were placed
on the same<br>
+ * line.<br>
+ * <p><br>
+ * While {@link #testWriteValidManifestOrException()} tests
header names<br>
+ * lengths, this test covers the values.<br>
+ */<br>
+ @Test<br>
+ public void testManifestLineWidthRange() throws Exception {<br>
+ final int MAX_UTF_CHARACTER_ENCODED_LENGTH = 4;<br>
+ final String TEST_NAME = "test";<br>
+<br>
+ // value will be filled with many possible sequences of utf<br>
+ // characters of different encoded numbers of bytes in the
form of <br>
+ // "k times i-bytes size character" + "k times i-bytes size
character"<br>
+ // and by prepending up to 72 "x" characters at every
possible position<br>
+ // relative to a possible line break.<br>
+ String value = "";<br>
+ int seed = 0;<br>
+ for (int i = 1; i <= MAX_UTF_CHARACTER_ENCODED_LENGTH;
i++) {<br>
+ for (int j = 1; j <=
MAX_UTF_CHARACTER_ENCODED_LENGTH; j++) {<br>
+ String valueI = "", valueJ = "";<br>
+ for (int k = 1; k < MAX_LINE_CONTENT_LENGTH;
k++) {<br>
+ valueI += numByteUtfCharacter(i, seed++);<br>
+ valueJ += numByteUtfCharacter(j, seed++);<br>
+ value += valueI + valueJ;<br>
+ }<br>
+ }<br>
+ }<br>
+<br>
+ String offset = "";<br>
+ for (int i = 0; i <= MAX_LINE_CONTENT_LENGTH; i++) {<br>
+ offset += "x";<br>
+ byte[] mfBytes = writeManifest(TEST_NAME, offset +
value);<br>
+ Manifest mf = new Manifest(new
ByteArrayInputStream(mfBytes));<br>
+ assertMainAndSectionValues(mf, TEST_NAME, offset +
value);<br>
+<br>
+ Set<Integer> allowedWidths =
IntStream.rangeClosed(<br>
+ MAX_LINE_CONTENT_LENGTH -
MAX_UTF_CHARACTER_ENCODED_LENGTH + 1,<br>
+
MAX_LINE_CONTENT_LENGTH).boxed().collect(Collectors.toSet());<br>
+ assertEquals(allowedWidths,
findContinuedLineWidths(mfBytes));<br>
+ }<br>
+ }<br>
+<br>
+ /**<br>
+ * Counts the length of lines (without the line breaks) which
are<br>
+ * continued, not the continuation lines but those before that
did not fit<br>
+ * on one line, whereby continuation lines may be continued
again on<br>
+ * another line, and returns their length in bytes, not
characters.<br>
+ */<br>
+ static Set<Integer> findContinuedLineWidths(byte[]
mfBytes) {<br>
+ Set<Integer> widths = new TreeSet<>();<br>
+ int previousLineWidth = -1;<br>
+ int b = 0; // start position of current line<br>
+ for (int i = 0; i < mfBytes.length; i++) {<br>
+ switch (mfBytes[i]) {<br>
+ case '\r':<br>
+ case '\n':<br>
+ previousLineWidth = i - b;<br>
+ if (mfBytes[i] == '\r' &&<br>
+ i + 1 < mfBytes.length &&
mfBytes[i + 1] == '\n') {<br>
+ i++;<br>
+ }<br>
+ b = i + 1;<br>
+ break;<br>
+ case ' ':<br>
+ if (i == b) {<br>
+ widths.add(previousLineWidth);<br>
+ }<br>
+ }<br>
+ }<br>
+ return widths;<br>
+ }<br>
+<br>
}<br>
diff -r 2ace90aec488
test/jdk/java/util/jar/Manifest/NullKeysAndValues.java<br>
--- /dev/null Thu Jan 01 00:00:00 1970 +0000<br>
+++ b/test/jdk/java/util/jar/Manifest/NullKeysAndValues.java Wed
May 02 07:20:46 2018 +0200<br>
@@ -0,0 +1,149 @@<br>
+/*<br>
+ * Copyright (c) 2018, Oracle and/or its affiliates. All rights
reserved.<br>
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.<br>
+ *<br>
+ * This code is free software; you can redistribute it and/or
modify it<br>
+ * under the terms of the GNU General Public License version 2
only, as<br>
+ * published by the Free Software Foundation.<br>
+ *<br>
+ * This code is distributed in the hope that it will be useful, but
WITHOUT<br>
+ * ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or<br>
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
License<br>
+ * version 2 for more details (a copy is included in the LICENSE
file that<br>
+ * accompanied this code).<br>
+ *<br>
+ * You should have received a copy of the GNU General Public
License version<br>
+ * 2 along with this work; if not, write to the Free Software
Foundation,<br>
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.<br>
+ *<br>
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA
94065 USA<br>
+ * or visit <a class="moz-txt-link-abbreviated" href="http://www.oracle.com">www.oracle.com</a> if you need additional information or
have any<br>
+ * questions.<br>
+ */<br>
+<br>
+import static java.nio.charset.StandardCharsets.UTF_8;<br>
+<br>
+import java.io.ByteArrayInputStream;<br>
+import java.io.ByteArrayOutputStream;<br>
+import java.io.IOException;<br>
+import java.util.jar.Attributes;<br>
+import java.util.jar.Manifest;<br>
+import java.util.jar.Attributes.Name;<br>
+import java.lang.reflect.Field;<br>
+<br>
+import org.testng.annotations.Test;<br>
+import static org.testng.Assert.*;<br>
+<br>
+/**<br>
+ * @test<br>
+ * @run testng/othervm --illegal-access=warn NullKeysAndValues<br>
+ * @summary Tests manifests with {@code null} values as section
name, header<br>
+ * name, or value in both main and named attributes
sections.<br>
+ */<br>
+/*<br>
+ * Note to future maintainer:<br>
+ * In order to actually being able to test all the cases where key
and values<br>
+ * are null normal manifest and attributes manipulation through
their public<br>
+ * api is not sufficient but then there were these null checks
there before<br>
+ * which may or may not have had their reason and this way it's
ensured that<br>
+ * the behavior does not change with that respect.<br>
+ * Once module isolation is enforced some test cases will not any
longer be<br>
+ * possible and those now tested situations will be guaranteed not
to occur<br>
+ * any longer at all at which point the corresponding tests can be
removed<br>
+ * safely without replacement unless of course another way is found
inject the<br>
+ * tested null values.<br>
+ * Another trick to access package private class members could be
to use<br>
+ * deserialization or adding a new class to the same package on the
classpath.<br>
+ * Here is not important how the values are set to null because it
shows that<br>
+ * the behavior remains unchanged. <br>
+ */<br>
+public class NullKeysAndValues {<br>
+<br>
+ static final String SOME_KEY = "some-key";<br>
+ static final String SOME_VALUE = "some value";<br>
+ static final String STRNULL = "null";<br>
+<br>
+ @Test<br>
+ public void testMainAttributesHeaderNameNull() throws Exception
{<br>
+ Manifest mf = new Manifest();<br>
+ Field attr = mf.getClass().getDeclaredField("attr");<br>
+ attr.setAccessible(true);<br>
+ Attributes mainAtts = new Attributes() {{<br>
+ super.put( /* in: */ null, SOME_VALUE);<br>
+ }};<br>
+ attr.set(mf, mainAtts);<br>
+ mf.getMainAttributes().put(Name.MANIFEST_VERSION, "1.0");<br>
+ try {<br>
+ mf = writeAndRead(mf);<br>
+ fail();<br>
+ } catch ( /* out: */ NullPointerException e) {<br>
+ return; // ok<br>
+ }<br>
+ }<br>
+<br>
+ @Test<br>
+ public void testMainAttributesHeaderValueNull() throws
Exception {<br>
+ Manifest mf = new Manifest();<br>
+ Field attr = mf.getClass().getDeclaredField("attr");<br>
+ attr.setAccessible(true);<br>
+ Attributes mainAtts = new Attributes() {{<br>
+ map.put(new Name(SOME_KEY), /* in: */ null);<br>
+ }};<br>
+ attr.set(mf, mainAtts);<br>
+ mf.getMainAttributes().put(Name.MANIFEST_VERSION, "1.0");<br>
+ mf = writeAndRead(mf);<br>
+ assertEquals(mf.getMainAttributes().getValue(SOME_KEY), <br>
+ /* out: */ STRNULL);<br>
+ }<br>
+<br>
+ @Test<br>
+ public void testSectionNameNull() throws IOException {<br>
+ Manifest mf = new Manifest();<br>
+ mf.getMainAttributes().put(Name.MANIFEST_VERSION, "1.0");<br>
+ mf.getEntries().put( /* in: */ null, new Attributes());<br>
+ mf = writeAndRead(mf);<br>
+ assertNotNull(mf.getEntries().get( /* out: */ STRNULL));<br>
+ }<br>
+<br>
+ @Test<br>
+ public void testNamedSectionHeaderNameNull() throws IOException
{<br>
+ Manifest mf = new Manifest();<br>
+ mf.getMainAttributes().put(Name.MANIFEST_VERSION, "1.0");<br>
+ mf.getEntries().put(SOME_KEY, new Attributes() {{<br>
+ map.put( /* in: */ null, SOME_VALUE);<br>
+ }});<br>
+ try {<br>
+ mf = writeAndRead(mf);<br>
+ fail();<br>
+ } catch ( /* out: */ NullPointerException e) {<br>
+ return; // ok<br>
+ }<br>
+ }<br>
+<br>
+ @Test<br>
+ public void testNamedSectionHeaderValueNull() throws
IOException {<br>
+ Manifest mf = new Manifest();<br>
+ mf.getMainAttributes().put(Name.MANIFEST_VERSION, "1.0");<br>
+ mf.getEntries().put(SOME_KEY, new Attributes() {{<br>
+ map.put(new Name(SOME_KEY), /* in: */ null);<br>
+ }});<br>
+ mf = writeAndRead(mf);<br>
+
assertEquals(mf.getEntries().get(SOME_KEY).getValue(SOME_KEY),<br>
+ /* out: */ STRNULL);<br>
+ }<br>
+<br>
+ static Manifest writeAndRead(Manifest mf) throws IOException {<br>
+ ByteArrayOutputStream out = new ByteArrayOutputStream();<br>
+ mf.write(out);<br>
+ byte[] mfBytes = out.toByteArray();<br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+ System.out.print(new String(mfBytes, UTF_8));<br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+<br>
+ ByteArrayInputStream in = new
ByteArrayInputStream(mfBytes);<br>
+ return new Manifest(in);<br>
+ }<br>
+<br>
+}<br>
diff -r 2ace90aec488
test/jdk/java/util/jar/Manifest/ValueUtfEncoding.java<br>
--- /dev/null Thu Jan 01 00:00:00 1970 +0000<br>
+++ b/test/jdk/java/util/jar/Manifest/ValueUtfEncoding.java Wed
May 02 07:20:46 2018 +0200<br>
@@ -0,0 +1,226 @@<br>
+/*<br>
+ * Copyright (c) 2018, Oracle and/or its affiliates. All rights
reserved.<br>
+ * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.<br>
+ *<br>
+ * This code is free software; you can redistribute it and/or
modify it<br>
+ * under the terms of the GNU General Public License version 2
only, as<br>
+ * published by the Free Software Foundation.<br>
+ *<br>
+ * This code is distributed in the hope that it will be useful, but
WITHOUT<br>
+ * ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or<br>
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
License<br>
+ * version 2 for more details (a copy is included in the LICENSE
file that<br>
+ * accompanied this code).<br>
+ *<br>
+ * You should have received a copy of the GNU General Public
License version<br>
+ * 2 along with this work; if not, write to the Free Software
Foundation,<br>
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.<br>
+ *<br>
+ * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA
94065 USA<br>
+ * or visit <a class="moz-txt-link-abbreviated" href="http://www.oracle.com">www.oracle.com</a> if you need additional information or
have any<br>
+ * questions.<br>
+ */<br>
+<br>
+import static java.nio.charset.StandardCharsets.UTF_8;<br>
+<br>
+import java.io.ByteArrayInputStream;<br>
+import java.io.ByteArrayOutputStream;<br>
+import java.io.IOException;<br>
+import java.util.jar.Attributes;<br>
+import java.util.jar.Attributes.Name;<br>
+import java.util.jar.Manifest;<br>
+import java.util.List;<br>
+import java.util.ArrayList;<br>
+<br>
+import org.testng.annotations.Test;<br>
+import static org.testng.Assert.*;<br>
+<br>
+/**<br>
+ * @test<br>
+ * @bug 6202130<br>
+ * @run testng ValueUtfEncoding<br>
+ * @summary Tests complete manifest values utf encoding<br>
+ * <p><br>
+ * This test writes and reads a manifest that contains every valid
utf<br>
+ * character (three times), grouped into manifest header values
with about<br>
+ * 65535 bytes each or slightly more, resulting in a single huge
manifest with<br>
+ * 3 * 67 + 1 values and 13703968 bytes in the manifest's encoded
form in<br>
+ * total. This way, all possible 1111995 utf characters are covered
in one<br>
+ * manifest.<br>
+ * <p><br>
+ * Every character occurs three times, once in a main attribute
value, once in<br>
+ * a section name, and once in a named section attribute value,
because<br>
+ * implementation of writing the main section headers differs from
the one<br>
+ * writing named section headers in<br>
+ * {@link Attributes#writeMain(java.io.DataOutputStream)} and<br>
+ * {@link Attributes#write(java.io.DataOutputStream)} due to
special order of<br>
+ * {@link Name#MANIFEST_VERSION} and {@link
Name#SIGNATURE_VERSION}.<br>
+ * and also {@link Manifest#read(java.io.InputStream)} treating
reading the<br>
+ * main section differently from reading named sections names
headers.<br>
+ * <p><br>
+ * Only header values are tested. Characters for header names are
much more<br>
+ * limited and very simple ones are used just to get valid and
different ones.<br>
+ * <p><br>
+ * Correct support of all utf characters also has some relation to
breaking<br>
+ * characters across lines, see {@link LineBreakCharacter}.<br>
+ */<br>
+public class ValueUtfEncoding {<br>
+<br>
+ /**<br>
+ * From the specifications:<br>
+ * <q>Implementations should support 65535-byte (not
character) header<br>
+ * values, and 65535 headers per file. They might run out of
memory,<br>
+ * but there should not be hard-coded limits below these
values.</q><br>
+ *<br>
+ * @see <a
href=<a class="moz-txt-link-rfc2396E" href="mailto:{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files">"{@docRoot}/../specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files"</a>>Notes
on Manifest and Signature Files</a><br>
+ */<br>
+ static final int MIN_VALUE_LENGTH_SUPPORTED = 2 << 16 -
1;<br>
+<br>
+ static final int MAX_UTF_CHARACTER_ENCODED_LENGTH = 4;<br>
+<br>
+ static boolean isValidUtfCharacter(int codePoint) {<br>
+ if (0xFDD0 <= codePoint && codePoint <=
0xFDEF) {<br>
+ return false; /* non-characters */<br>
+ }<br>
+ if ((codePoint & 0xFFFE) == 0xFFFE) {<br>
+ return false; /* byte order marks */<br>
+ }<br>
+ return true;<br>
+ }<br>
+<br>
+ /**<br>
+ * returns {@code true} if {@code codePoint} is explicitly
forbidden in<br>
+ * manifest values based on a statement from the specs:<br>
+ * <pre>otherchar: any UTF-8 character except NUL, CR and
LF<pre><br>
+ *<br>
+ * @see <a
href=<a class="moz-txt-link-rfc2396E" href="mailto:{@docRoot}/../specs/jar/jar.html#Section-Specification">"{@docRoot}/../specs/jar/jar.html#Section-Specification"</a>>Jar
File Specification</a><br>
+ */<br>
+ static boolean isInvalidManifestValueCharacter(int codePoint) {<br>
+ return codePoint == 0 /* NUL */<br>
+ || codePoint == '\r' /* CR */<br>
+ || codePoint == '\n' /* LF */;<br>
+ };<br>
+<br>
+ /**<br>
+ * Produces a list of strings with all known utf characters
except those<br>
+ * invalid in manifest header values with at least<br>
+ * {@link #MIN_VALUE_LENGTH_SUPPORTED} utf-8 encoded bytes each<br>
+ * except the last string which contains just the remaining
characters.<br>
+ */<br>
+ static List<String>
produceValidManifestUtfCharacterValues() {<br>
+ int maxLengthBytes = MIN_VALUE_LENGTH_SUPPORTED +<br>
+ // exceed the specified limit by at least one
character<br>
+ MAX_UTF_CHARACTER_ENCODED_LENGTH + 1;<br>
+<br>
+ int numberOfUsedCodePoints = 0;<br>
+ ArrayList<String> values = new ArrayList<>();<br>
+ byte[] valueBuf = new byte[maxLengthBytes];<br>
+ int pos = 0;<br>
+ for (int codePoint = Character.MIN_CODE_POINT;<br>
+ codePoint <= Character.MAX_CODE_POINT;
codePoint++) {<br>
+ if (!isValidUtfCharacter(codePoint)) {<br>
+ continue;<br>
+ }<br>
+ if (isInvalidManifestValueCharacter(codePoint)) {<br>
+ continue;<br>
+ }<br>
+ numberOfUsedCodePoints++;<br>
+<br>
+ byte[] charBuf =<br>
+ new
String(Character.toChars(codePoint)).getBytes(UTF_8);<br>
+ if (pos + charBuf.length > valueBuf.length) {<br>
+ values.add(new String(valueBuf, 0, pos, UTF_8));<br>
+ pos = 0;<br>
+ }<br>
+ System.arraycopy(charBuf, 0, valueBuf, pos,
charBuf.length);<br>
+ pos += charBuf.length;<br>
+ }<br>
+ if (pos > 0) {<br>
+ values.add(new String(valueBuf, 0, pos, UTF_8));<br>
+ }<br>
+<br>
+ if (numberOfUsedCodePoints !=<br>
+ (17 << 16) /* utf space */<br>
+ - 66 /* non-characters */<br>
+ - 3 /* nul, cr, lf */) {<br>
+ fail("self-test: utf character set not covered
exactly");<br>
+ }<br>
+<br>
+ return values;<br>
+ }<br>
+<br>
+ /**<br>
+ * returns simple, valid, short, and distinct manifest header
names.<br>
+ * The returned name cannot be "{@code Manifest-Version}"
because the<br>
+ * returned string does not contain "{@code -}".<br>
+ *<br>
+ * @param seed seed to produce distinct names<br>
+ */<br>
+ static String azName(int seed) {<br>
+ StringBuffer name = new StringBuffer();<br>
+ do {<br>
+ name.insert(0, (char) (seed % 26 + (seed < 26 ? 'A'
: 'a')));<br>
+ seed = seed / 26 - 1;<br>
+ } while (seed >= 0);<br>
+ return name.toString();<br>
+ }<br>
+<br>
+ /**<br>
+ * covers writing and reading of manifests with all known utf
characters.<br>
+ *<br>
+ * Because the implementation used different portions of code
depending on<br>
+ * where the value occurs to read or write in earlier versions,
each<br>
+ * character is tested in each of the three
positions:<ul><br>
+ * <li>main attribute header,</li><br>
+ * <li>named section name, which is in fact a header
value after a blank<br>
+ * line, and</li><br>
+ * <li>named sections header values</li><br>
+ * <ul><br>
+ */<br>
+ @Test<br>
+ public void testValueUtfEncoding() throws IOException {<br>
+ Manifest mf = new Manifest();<br>
+ mf.getMainAttributes().put(Name.MANIFEST_VERSION, "1.0");<br>
+<br>
+ List<String> values =
produceValidManifestUtfCharacterValues();<br>
+ for (int i = 0; i < values.size(); i++) {<br>
+ String name = azName(i);<br>
+ String value = values.get(i);<br>
+<br>
+ mf.getMainAttributes().put(new Name(name), value);<br>
+ Attributes attributes = new Attributes();<br>
+ mf.getEntries().put(value, attributes);<br>
+ attributes.put(new Name(name), value);<br>
+ }<br>
+<br>
+ mf = writeAndRead(mf);<br>
+<br>
+ for (int i = 0; i < values.size(); i++) {<br>
+ String value = values.get(i);<br>
+ String name = azName(i);<br>
+<br>
+ assertEquals(mf.getMainAttributes().getValue(name),
value,<br>
+ "main attributes header value");<br>
+ Attributes attributes = mf.getAttributes(value);<br>
+ assertNotNull(attributes, "named section not found");<br>
+ assertEquals(attributes.getValue(name), value,<br>
+ "named section attributes value");<br>
+ }<br>
+ }<br>
+<br>
+ static Manifest writeAndRead(Manifest mf) throws IOException {<br>
+ ByteArrayOutputStream out = new ByteArrayOutputStream();<br>
+ mf.write(out);<br>
+ byte[] mfBytes = out.toByteArray();<br>
+ <br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+ System.out.print(new String(mfBytes, UTF_8));<br>
+
System.out.println("-------------------------------------------"<br>
+ + "-----------------------------");<br>
+<br>
+ ByteArrayInputStream in = new
ByteArrayInputStream(mfBytes);<br>
+ return new Manifest(in);<br>
+ }<br>
+<br>
+}<br>
diff -r 2ace90aec488
test/jdk/sun/security/tools/jarsigner/LineBrokenMultiByteCharacter.java<br>
---
a/test/jdk/sun/security/tools/jarsigner/LineBrokenMultiByteCharacter.java
Mon Apr 30 21:56:54 2018 -0400<br>
+++
b/test/jdk/sun/security/tools/jarsigner/LineBrokenMultiByteCharacter.java
Wed May 02 07:20:46 2018 +0200<br>
@@ -21,35 +21,48 @@<br>
* questions.<br>
*/<br>
<br>
-/*<br>
- * @test<br>
- * @bug 6695402<br>
- * @summary verify signatures of jars containing classes with names<br>
- * with multi-byte unicode characters broken across lines<br>
- * @library /test/lib<br>
- */<br>
-<br>
import java.io.IOException;<br>
import java.io.InputStream;<br>
import java.nio.file.Files;<br>
import java.nio.file.Paths;<br>
+import java.util.HashMap;<br>
+import java.util.jar.Attributes.Name;<br>
import java.util.jar.JarFile;<br>
-import java.util.jar.Attributes.Name;<br>
import java.util.jar.JarEntry;<br>
+import java.util.zip.ZipEntry;<br>
+import java.util.zip.ZipFile;<br>
<br>
import static java.nio.charset.StandardCharsets.UTF_8;<br>
<br>
import jdk.test.lib.SecurityTools;<br>
import jdk.test.lib.util.JarUtils;<br>
<br>
+import org.testng.annotations.*;<br>
+import static org.testng.Assert.*;<br>
+<br>
+/**<br>
+ * @test<br>
+ * @bug 6695402<br>
+ * @library /test/lib<br>
+ * @run testng LineBrokenMultiByteCharacter<br>
+ * @summary verify signatures of jars containing classes with names<br>
+ * with multi-byte unicode characters broken across lines<br>
+ *<br>
+ * Before utf characters were kept in one piece by bug 6443578 and
partially<br>
+ * broken across a line break in manifests, bug 6695402 could occur
resulting<br>
+ * in a signature considered invalid. That can still happen after
bug 6443578<br>
+ * fixed when signing manifests that already contain entries with
characters<br>
+ * broken across a line break, presumably when an older manifest is
present<br>
+ * created with a different jdk.<br>
+ */<br>
public class LineBrokenMultiByteCharacter {<br>
<br>
/**<br>
- * this name will break across lines in MANIFEST.MF at the<br>
+ * This name will break across lines in MANIFEST.MF at the<br>
* middle of a two-byte utf-8 encoded character due to its e
acute letter<br>
* at its exact position.<br>
- *<br>
- * because no file with such a name exists {@link JarUtils}
will add the<br>
+ * <p><br>
+ * Because no file with such a name exists {@link JarUtils}
will add the<br>
* name itself as contents to the jar entry which would have
contained a<br>
* compiled class in the original bug. For this test, the
contents of the<br>
* files contained in the jar file is not important as long as
they get<br>
@@ -58,55 +71,157 @@<br>
* @see #verifyClassNameLineBroken(JarFile, String)<br>
*/<br>
static final String testClassName =<br>
-
"LineBrokenMultiByteCharacterA1234567890B1234567890C123456789D1234\u00E9xyz.class";<br>
+
"LineBrokenMultiByteCharacterA1234567890B1234567890C123456789"<br>
+ + "D1234\u00E9xyz.class";<br>
<br>
+ /**<br>
+ * Used for a test case where an entry / class file is added to
an existing<br>
+ * signed jar that already contains an entry with this name
which of course<br>
+ * has to be distinct from the one tested.<br>
+ */<br>
static final String anotherName =<br>
-
"LineBrokenMultiByteCharacterA1234567890B1234567890C123456789D1234567890.class";<br>
+
"LineBrokenMultiByteCharacterA1234567890B1234567890C123456789"<br>
+ + "D1234567890.class";<br>
<br>
static final String alias = "a";<br>
static final String keystoreFileName = "test.jks";<br>
- static final String manifestFileName = "MANIFEST.MF";<br>
+ static final String manifestFileName = JarFile.MANIFEST_NAME;<br>
<br>
- public static void main(String[] args) throws Exception {<br>
- prepare();<br>
+ byte[] manifestNoDigest = "Manifest-Version:
1.0\r\n\r\n".getBytes(UTF_8);<br>
+ byte[] manifestWithDigest; // with character broken across line
break<br>
<br>
- testSignJar("test.jar");<br>
- testSignJarNoManifest("test-no-manifest.jar");<br>
- testSignJarUpdate("test-update.jar", "test-updated.jar");<br>
+ /**<br>
+ * Simulate a jar manifest as it would have been created by an
earlier jdk<br>
+ * by re-arranging the line break at exactly 72 bytes content
thereby<br>
+ * breaking the multi-byte character under test.<br>
+ */<br>
+ static byte[] rebreak(byte[] mf0) {<br>
+ byte[] mf1 = new byte[mf0.length];<br>
+ int c0 = 0, c1 = 0; // bytes since last line start<br>
+ for (int i0 = 0, i1 = 0; i0 < mf0.length; i0++, i1++) {<br>
+ switch (mf0[i0]) {<br>
+ case '\r':<br>
+ if (i0 + 2 < mf0.length &&<br>
+ mf0[i0 + 1] == '\n' && mf0[i0 + 2]
== ' ') {<br>
+ // skip line break<br>
+ i0 += 2;<br>
+ i1 -= 1;<br>
+ } else {<br>
+ mf1[i1] = mf0[i0];<br>
+ c0 = c1 = 0;<br>
+ }<br>
+ break;<br>
+ case '\n':<br>
+ if (i0 + 1 < mf0.length && mf0[i0 + 1]
== ' ') {<br>
+ // skip line break<br>
+ i0 += 1;<br>
+ i1 -= 1;<br>
+ } else {<br>
+ mf1[i1] = mf0[i0];<br>
+ c0 = c1 = 0;<br>
+ }<br>
+ break;<br>
+ case ' ':<br>
+ if (c0 == 0) {<br>
+ continue;<br>
+ }<br>
+ default:<br>
+ c0++;<br>
+ if (c1 == 72) {<br>
+ mf1[i1++] = '\r';<br>
+ mf1[i1++] = '\n';<br>
+ mf1[i1++] = ' ';<br>
+ c1 = 1;<br>
+ } else {<br>
+ c1++;<br>
+ }<br>
+ mf1[i1] = mf0[i0];<br>
+ }<br>
+ }<br>
+ return mf1;<br>
}<br>
<br>
- static void prepare() throws Exception {<br>
+ @BeforeClass<br>
+ public void prepare() throws Exception {<br>
SecurityTools.keytool("-keystore", keystoreFileName,
"-genkeypair",<br>
"-storepass", "changeit", "-keypass", "changeit",
"-storetype",<br>
"JKS", "-alias", alias, "-dname", "CN=X",
"-validity", "366")<br>
.shouldHaveExitValue(0);<br>
<br>
- Files.write(Paths.get(manifestFileName), (Name.<br>
- MANIFEST_VERSION.toString() + ":
1.0\r\n").getBytes(UTF_8));<br>
+ String jarFileName = "reference-manifest.jar";<br>
+ createJarWithManifest(jarFileName, manifestNoDigest,<br>
+ testClassName, anotherName);<br>
+ SecurityTools.jarsigner("-keystore", keystoreFileName,
"-storetype",<br>
+ "JKS", "-storepass", "changeit", "-debug",
jarFileName, alias)<br>
+ .shouldHaveExitValue(0);<br>
+ try (ZipFile refJar = new ZipFile(jarFileName);) {<br>
+ ZipEntry mfEntry =
refJar.getEntry(JarFile.MANIFEST_NAME);<br>
+ manifestWithDigest =
refJar.getInputStream(mfEntry).readAllBytes();<br>
+ }<br>
+ System.out.println("manifestWithDigest before re-break = "
+<br>
+ new String(manifestWithDigest, UTF_8));<br>
+ manifestWithDigest = rebreak(manifestWithDigest);<br>
+ System.out.println("manifestWithDigest after re-break = " +<br>
+ new String(manifestWithDigest, UTF_8));<br>
+ // now, manifestWithDigest is accepted as unmodified by<br>
+ // jdk.security.jarsigner.JarSigner#updateDigests<br>
+ // (ZipEntry,ZipFile,MessageDigest[],Manifest) on line 985:<br>
+ // "if (!mfDigest.equalsIgnoreCase(base64Digests[i])) {"<br>
+ // and therefore left unchanged when the jar is signed and<br>
+ // signature verification will check it, which is the test
case.<br>
+<br>
+ Files.createDirectory(Paths.get("META-INF/"));<br>
}<br>
<br>
- static void testSignJar(String jarFileName) throws Exception {<br>
- JarUtils.createJar(jarFileName, manifestFileName,
testClassName);<br>
- verifyJarSignature(jarFileName);<br>
+ @Test<br>
+ public void testSignJarWithNoExistingClassEntry() throws
Exception {<br>
+ String jarFileName = "test-eacuteinonepiece.jar";<br>
+ createJarWithManifest(jarFileName, manifestNoDigest,
testClassName);<br>
+ signAndVerifyJarSignature(jarFileName, false);<br>
}<br>
<br>
- static void testSignJarNoManifest(String jarFileName) throws
Exception {<br>
- JarUtils.createJar(jarFileName, testClassName);<br>
- verifyJarSignature(jarFileName);<br>
+ @Test<br>
+ public void testSignJarWithBrokenEAcuteClassEntry() throws
Exception {<br>
+ String jarFileName = "test-brokeneacute.jar";<br>
+ createJarWithManifest(jarFileName, manifestWithDigest,
testClassName);<br>
+ signAndVerifyJarSignature(jarFileName, true);<br>
}<br>
<br>
- static void testSignJarUpdate(<br>
- String initialFileName, String updatedFileName) throws
Exception {<br>
- JarUtils.createJar(initialFileName, manifestFileName,
anotherName);<br>
+ @Test<br>
+ public void testSignJarNoManifest() throws Exception {<br>
+ String jarFileName = "test-no-manifest.jar";<br>
+ JarUtils.createJar(jarFileName, testClassName);<br>
+ signAndVerifyJarSignature(jarFileName, false);<br>
+ }<br>
+<br>
+ @Test<br>
+ public void testSignJarUpdateWithEAcuteClassEntryInOnePiece()<br>
+ throws Exception {<br>
+ String initialFileName =
"test-eacuteinonepiece-update.jar";<br>
+ String updatedFileName =
"test-eacuteinonepiece-updated.jar";<br>
+ createJarWithManifest(initialFileName, manifestNoDigest,
anotherName);<br>
SecurityTools.jarsigner("-keystore", keystoreFileName,
"-storetype",<br>
"JKS", "-storepass", "changeit", "-debug",
initialFileName,<br>
alias).shouldHaveExitValue(0);<br>
JarUtils.updateJar(initialFileName, updatedFileName,
testClassName);<br>
- verifyJarSignature(updatedFileName);<br>
+ signAndVerifyJarSignature(updatedFileName, false);<br>
}<br>
<br>
- static void verifyJarSignature(String jarFileName) throws
Exception {<br>
- // actually sign the jar<br>
+ @Test<br>
+ public void testSignJarUpdateWithBrokenEAcuteClassEntry()<br>
+ throws Exception {<br>
+ String initialFileName = "test-brokeneacute-update.jar";<br>
+ String updatedFileName = "test-brokeneacute-updated.jar";<br>
+ createJarWithManifest(initialFileName, manifestWithDigest,
anotherName);<br>
+ SecurityTools.jarsigner("-keystore", keystoreFileName,
"-storetype",<br>
+ "JKS", "-storepass", "changeit", "-debug",
initialFileName,<br>
+ alias).shouldHaveExitValue(0);<br>
+ JarUtils.updateJar(initialFileName, updatedFileName,
testClassName);<br>
+ signAndVerifyJarSignature(updatedFileName, true);<br>
+ }<br>
+<br>
+ void signAndVerifyJarSignature(String jarFileName,<br>
+ boolean expectBrokenEAcute) throws Exception {<br>
SecurityTools.jarsigner("-keystore", keystoreFileName,
"-storetype",<br>
"JKS", "-storepass", "changeit", "-debug",
jarFileName, alias)<br>
.shouldHaveExitValue(0);<br>
@@ -114,34 +229,34 @@<br>
try (<br>
JarFile jar = new JarFile(jarFileName);<br>
) {<br>
- verifyClassNameLineBroken(jar, testClassName);<br>
+ verifyClassNameLineBroken(jar, testClassName,
expectBrokenEAcute);<br>
verifyCodeSigners(jar, jar.getJarEntry(testClassName));<br>
}<br>
}<br>
<br>
/**<br>
- * it would be too easy to miss the actual test case by just
renaming an<br>
+ * It would be too easy to miss the actual test case by just
renaming an<br>
* identifier so that the multi-byte encoded character would
not any longer<br>
* be broken across a line break.<br>
*<br>
- * this check here verifies that the actual test case is tested
based on<br>
+ * This check here verifies that the actual test case is tested
based on<br>
* the manifest and not based on the signature file because at
the moment,<br>
* the signature file does not even contain the desired entry
at all.<br>
- *<br>
- * this relies on {@link java.util.jar.Manifest} breaking lines
unaware<br>
- * of bytes that belong to the same multi-byte utf characters.<br>
*/<br>
- static void verifyClassNameLineBroken(JarFile jar, String
className)<br>
- throws IOException {<br>
+ static void verifyClassNameLineBroken(JarFile jar, String
className,<br>
+ boolean expectBrokenEAcute) throws IOException {<br>
byte[] eAcute = "\u00E9".getBytes(UTF_8);<br>
byte[] eAcuteBroken =<br>
new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]};<br>
<br>
- if (jar.getManifest().getAttributes(className) == null) {<br>
- throw new AssertionError(className + " not found in
manifest");<br>
- }<br>
+ assertNotNull(jar.getManifest().getAttributes(className),<br>
+ className + " not found in manifest");<br>
<br>
JarEntry manifestEntry =
jar.getJarEntry(JarFile.MANIFEST_NAME);<br>
+ System.out.println("expectBrokenEAcute = " +
expectBrokenEAcute);<br>
+ System.out.println("Manifest = \n" + new String(<br>
+ jar.getInputStream(manifestEntry).readAllBytes(),
UTF_8));<br>
+<br>
try (<br>
InputStream manifestIs =
jar.getInputStream(manifestEntry);<br>
) {<br>
@@ -156,10 +271,12 @@<br>
bytesMatched = 0;<br>
}<br>
}<br>
- if (bytesMatched < eAcuteBroken.length) {<br>
- throw new AssertionError("self-test failed:
multi-byte "<br>
- + "utf-8 character not broken across
lines");<br>
- }<br>
+ assertEquals(expectBrokenEAcute,<br>
+ bytesMatched == eAcuteBroken.length,<br>
+ "self-test failed: multi-byte "<br>
+ + "utf-8 character "<br>
+ + (expectBrokenEAcute ? "not " : "")<br>
+ + "broken across lines");<br>
}<br>
}<br>
<br>
@@ -175,11 +292,16 @@<br>
// a check for the presence of code signers is sufficient
to check<br>
// bug JDK-6695402. no need to also verify the actual code
signers<br>
// attributes here.<br>
- if (jarEntry.getCodeSigners() == null<br>
- || jarEntry.getCodeSigners().length == 0) {<br>
- throw new AssertionError(<br>
- "no signing certificate found for " +
jarEntry.getName());<br>
- }<br>
+ assertTrue(jarEntry.getCodeSigners() != null<br>
+ && jarEntry.getCodeSigners().length > 0,<br>
+ "no signing certificate found for " +
jarEntry.getName());<br>
+ }<br>
+<br>
+ static void createJarWithManifest(String jarFileName, byte[]
manifest,<br>
+ String... files) throws IOException {<br>
+ JarUtils.createJar("yetwithoutmanifest-" + jarFileName,
files);<br>
+ JarUtils.updateJar("yetwithoutmanifest-" + jarFileName,
jarFileName,<br>
+ new HashMap<>() {{ put(manifestFileName,
manifest); }});<br>
}<br>
<br>
}<br>
<br>
</body>
</html>