From philipp.kunz at paratix.ch Mon Oct 2 17:24:46 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Mon, 2 Oct 2017 19:24:46 +0200 Subject: Manifest Related Bugs JDK-6372077, JDK-6202130, JDK-8176843, JDK-4842483, JDK-6443578, JDK-6910466, JDK-4935610, and JDK-4271239 Message-ID: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> Hi, While fixing JDK-6695402 I came across other related bugs to manifests such as JDK-6372077, JDK-6202130, JDK-8176843, JDK-4842483, and JDK-6443578 which all relate to manifest reading and writing. Somewhere bug 7071055 is mentioned but I cannot find anywhere. Another group of bugs, JDK-6910466, JDK-4935610, and JDK-4271239 concern the mandatory manifest main attributes Manifest-Version or Signature-Version and at first glance are duplicates. If you know of more known bugs, not necessarily present in jira, I'd be glad to get notified. There are also some comments about utf handling and line breaking in the code of Manifest: http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l299 http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l327 http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l370 Furthermore, Attributes.map should declare appropriate type parameters: http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l61 The specification would also require that `header names must not start with _, - or "From"` (http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html#Section-Specification) but I would opt not to add this as a hard restriction because I guess it can be assumed that such names are in use now after having been working for years. A warning to a logger might be conceivable such as in http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l424 Attribute values are never checked at all and invalid characters such as line breaks are never detected except that when reading the manifest again the values are cut off. The tab character (U+0008) does not work in manifest values. I suspect that there are also issues regarding the iteration order but I haven't got a prove yet unlike for the other points above: http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Manifest.java#l54 There is duplicated or very similar code in Attributes and Manifest: Attributes.write-Manifest.write-Attributes.writeMain and Attributes.read-Manifest.read. Resolving JDK-6202130 would have the additional benefit to be able to view manifests with any utf-8 capable editor even if multi-byte characters break across lines. Fixing these issues individually looks like more complicated work than fixing them all at once, I guess, also because of a very low current test coverage. So I'd suggest to add some thorough tests along with fixing these issues. But before I start I would like to get some guidance, assistance or at least confirmation on how to proceed. I'm new to open jdk and have submitted only one patch so far. Is it ok to add tests for things that have worked before? Is it ok to refactor duplicated code just for the added value to reduce effort for testing? I assume compatibility to and from existing manifests is the highest priority, correct? This would also be the hard part in adding as complete test coverage as possible. What would be acceptable criteria to meet? Why does Attributes not extend LinkedHashMap and why does Manifest not extend HashMap? Any objection? While I would not want to write code that looks slow or change more than necessary one can never know before having performance actually measured. Is there some way this is dealt with or should I wait for such feedback until after patch submission? Would someone sponsor? Regards, Philipp ------------------------------------------------------------------------ Paratix GmbH St Peterhofstatt 11 8001 Z?rich +41 (0)76 397 79 35 philipp.kunz at paratix.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 5134 bytes Desc: not available URL: From philipp.kunz at paratix.ch Tue Oct 3 05:48:56 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Tue, 3 Oct 2017 07:48:56 +0200 Subject: Request for Review: Attributes.map generic types Message-ID: <845306f0-7855-fe01-a017-53e96fdafb31@paratix.ch> Hi This has not been asked for and there is also no bug yet but nevertheless let me propose to change Attributes.map to specific generic types. It looks like the type parameters were added automatically with their introduction in java 5. I figure that no specific test is required in this particular case because it's a pure compile-time issue which is already sufficiently covered with existing code and there are also tests that already now involve manifests with attributes at run-time. Any feedback or thoughts? Would someone volunteer to sponsor it? Regards, Philipp ----- BEGIN PATCH ----- diff -r 2e947e1bd907 src/java.base/share/classes/java/util/jar/Attributes.java --- a/src/java.base/share/classes/java/util/jar/Attributes.java Mon Oct 02 10:04:22 2017 -0700 +++ b/src/java.base/share/classes/java/util/jar/Attributes.java Tue Oct 03 07:41:40 2017 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1997, 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -54,11 +54,11 @@ * @see Manifest * @since 1.2 */ -public class Attributes implements Map, Cloneable { +public class Attributes implements Map, Cloneable { /** * The attribute name-value mappings. */ - protected Map map; + protected Map map; /** * Constructs a new, empty Attributes object with default size. @@ -87,7 +87,6 @@ map = new LinkedHashMap<>(attr); } - /** * Returns the value of the specified attribute name, or null if the * attribute name was not found. @@ -96,7 +95,7 @@ * @return the value of the specified attribute name, or null if * not found. */ - public Object get(Object name) { + public String get(Object name) { return map.get(name); } @@ -116,7 +115,7 @@ * @throws IllegalArgumentException if the attribute name is invalid */ public String getValue(String name) { - return (String)get(new Attributes.Name(name)); + return get(new Attributes.Name(name)); } /** @@ -133,7 +132,7 @@ * not found. */ public String getValue(Name name) { - return (String)get(name); + return get(name); } /** @@ -144,11 +143,9 @@ * @param name the attribute name * @param value the attribute value * @return the previous value of the attribute, or null if none - * @exception ClassCastException if the name is not a Attributes.Name - * or the value is not a String */ - public Object put(Object name, Object value) { - return map.put((Attributes.Name)name, (String)value); + public String put(Attributes.Name name, String value) { + return map.put(name, value); } /** @@ -168,7 +165,7 @@ * @exception IllegalArgumentException if the attribute name is invalid */ public String putValue(String name, String value) { - return (String)put(new Name(name), value); + return put(new Name(name), value); } /** @@ -178,7 +175,7 @@ * @param name attribute name * @return the previous value of the attribute, or null if none */ - public Object remove(Object name) { + public String remove(Object name) { return map.remove(name); } @@ -208,15 +205,14 @@ * Copies all of the attribute name-value mappings from the specified * Attributes to this Map. Duplicate mappings will be replaced. * - * @param attr the Attributes to be stored in this map - * @exception ClassCastException if attr is not an Attributes + * @param attr the attributes to be stored in this map, may also be of + * type {@link Attributes} */ - public void putAll(Map attr) { - // ## javac bug? - if (!Attributes.class.isInstance(attr)) - throw new ClassCastException(); - for (Map.Entry me : (attr).entrySet()) + public void putAll(Map attr) { + for (Map.Entry me + : attr.entrySet()) { put(me.getKey(), me.getValue()); + } } /** @@ -243,14 +239,14 @@ /** * Returns a Set view of the attribute names (keys) contained in this Map. */ - public Set keySet() { + public Set keySet() { return map.keySet(); } /** * Returns a Collection view of the attribute values contained in this Map. */ - public Collection values() { + public Collection values() { return map.values(); } @@ -258,7 +254,7 @@ * Returns a Collection view of the attribute name-value mappings * contained in this Map. */ - public Set> entrySet() { + public Set> entrySet() { return map.entrySet(); } @@ -290,7 +286,7 @@ * the Attributes returned can be safely modified without affecting * the original. */ - public Object clone() { + public Attributes clone() { return new Attributes(this); } @@ -300,12 +296,12 @@ */ @SuppressWarnings("deprecation") void write(DataOutputStream os) throws IOException { - for (Entry e : entrySet()) { + for (Entry e : entrySet()) { StringBuffer buffer = new StringBuffer( - ((Name) e.getKey()).toString()); + e.getKey().toString()); buffer.append(": "); - String value = (String) e.getValue(); + String value = e.getValue(); if (value != null) { byte[] vb = value.getBytes("UTF8"); value = new String(vb, 0, 0, vb.length); @@ -343,14 +339,14 @@ // write out all attributes except for the version // we wrote out earlier - for (Entry e : entrySet()) { - String name = ((Name) e.getKey()).toString(); + for (Entry e : entrySet()) { + String name = e.getKey().toString(); if ((version != null) && !(name.equalsIgnoreCase(vername))) { StringBuffer buffer = new StringBuffer(name); buffer.append(": "); - String value = (String) e.getValue(); + String value = e.getValue(); if (value != null) { byte[] vb = value.getBytes("UTF8"); value = new String(vb, 0, 0, vb.length); diff -r 2e947e1bd907 src/java.base/share/classes/sun/net/www/protocol/jar/URLJarFile.java --- a/src/java.base/share/classes/sun/net/www/protocol/jar/URLJarFile.java Mon Oct 02 10:04:22 2017 -0700 +++ b/src/java.base/share/classes/sun/net/www/protocol/jar/URLJarFile.java Tue Oct 03 07:41:40 2017 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2001, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2001, 2017, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -148,14 +148,14 @@ Manifest man = new Manifest(); Attributes attr = man.getMainAttributes(); - attr.putAll((Map)superAttr.clone()); + attr.putAll(superAttr.clone()); // now deep copy the manifest entries if (superEntries != null) { Map entries = man.getEntries(); for (String key : superEntries.keySet()) { Attributes at = superEntries.get(key); - entries.put(key, (Attributes) at.clone()); + entries.put(key, at.clone()); } } @@ -261,7 +261,7 @@ if (e != null) { Attributes a = e.get(getName()); if (a != null) - return (Attributes)a.clone(); + return a.clone(); } } return null; diff -r 2e947e1bd907 src/java.base/share/classes/sun/security/util/ManifestEntryVerifier.java --- a/src/java.base/share/classes/sun/security/util/ManifestEntryVerifier.java Mon Oct 02 10:04:22 2017 -0700 +++ b/src/java.base/share/classes/sun/security/util/ManifestEntryVerifier.java Tue Oct 03 07:41:40 2017 +0200 @@ -30,6 +30,7 @@ import java.security.CodeSigner; import java.util.*; import java.util.jar.*; +import java.util.jar.Attributes.Name; import java.util.Base64; @@ -122,7 +123,7 @@ } } - for (Map.Entry se : attr.entrySet()) { + for (Map.Entry se : attr.entrySet()) { String key = se.getKey().toString(); if (key.toUpperCase(Locale.ENGLISH).endsWith("-DIGEST")) { @@ -146,7 +147,7 @@ digest.reset(); digests.add(digest); manifestHashes.add( - Base64.getMimeDecoder().decode((String)se.getValue())); + Base64.getMimeDecoder().decode(se.getValue())); } } } diff -r 2e947e1bd907 src/java.base/share/classes/sun/security/util/SignatureFileVerifier.java --- a/src/java.base/share/classes/sun/security/util/SignatureFileVerifier.java Mon Oct 02 10:04:22 2017 -0700 +++ b/src/java.base/share/classes/sun/security/util/SignatureFileVerifier.java Tue Oct 03 07:41:40 2017 +0200 @@ -46,6 +46,7 @@ import java.util.Locale; import java.util.Map; import java.util.jar.Attributes; +import java.util.jar.Attributes.Name; import java.util.jar.JarException; import java.util.jar.JarFile; import java.util.jar.Manifest; @@ -445,7 +446,7 @@ boolean validEntry = false; // go through all the attributes and process *-Digest-Manifest entries - for (Map.Entry se : mattr.entrySet()) { + for (Map.Entry se : mattr.entrySet()) { String key = se.getKey().toString(); @@ -469,7 +470,7 @@ if (digest != null) { byte[] computedHash = md.manifestDigest(digest); byte[] expectedHash = - Base64.getMimeDecoder().decode((String)se.getValue()); + Base64.getMimeDecoder().decode(se.getValue()); if (debug != null) { debug.println("Signature File: Manifest digest " + @@ -517,7 +518,7 @@ // go through all the attributes and process // digest entries for the manifest main attributes - for (Map.Entry se : mattr.entrySet()) { + for (Map.Entry se : mattr.entrySet()) { String key = se.getKey().toString(); if (key.toUpperCase(Locale.ENGLISH).endsWith(ATTR_DIGEST)) { @@ -540,7 +541,7 @@ md.get(ManifestDigester.MF_MAIN_ATTRS, false); byte[] computedHash = mde.digest(digest); byte[] expectedHash = - Base64.getMimeDecoder().decode((String)se.getValue()); + Base64.getMimeDecoder().decode(se.getValue()); if (debug != null) { debug.println("Signature File: " + @@ -620,7 +621,7 @@ //hex.encodeBuffer(data, System.out); // go through all the attributes and process *-Digest entries - for (Map.Entry se : sfAttr.entrySet()) { + for (Map.Entry se : sfAttr.entrySet()) { String key = se.getKey().toString(); if (key.toUpperCase(Locale.ENGLISH).endsWith("-DIGEST")) { @@ -643,7 +644,7 @@ boolean ok = false; byte[] expected = - Base64.getMimeDecoder().decode((String)se.getValue()); + Base64.getMimeDecoder().decode(se.getValue()); byte[] computed; if (workaround) { computed = mde.digestWorkaround(digest); diff -r 2e947e1bd907 src/jdk.jartool/share/classes/jdk/security/jarsigner/JarSigner.java --- a/src/jdk.jartool/share/classes/jdk/security/jarsigner/JarSigner.java Mon Oct 02 10:04:22 2017 -0700 +++ b/src/jdk.jartool/share/classes/jdk/security/jarsigner/JarSigner.java Tue Oct 03 07:41:40 2017 +0200 @@ -685,7 +685,7 @@ // Manifest exists. Read its raw bytes. mfRawBytes = zipFile.getInputStream(mfFile).readAllBytes(); manifest.read(new ByteArrayInputStream(mfRawBytes)); - oldAttr = (Attributes) (manifest.getMainAttributes().clone()); + oldAttr = manifest.getMainAttributes().clone(); } else { // Create new manifest Attributes mattr = manifest.getMainAttributes(); ----- END PATCH ----- ------------------------------------------------------------------------ Paratix GmbH St Peterhofstatt 11 8001 Z?rich +41 (0)76 397 79 35 philipp.kunz at paratix.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 5134 bytes Desc: not available URL: From sean.mullan at oracle.com Tue Oct 10 20:30:48 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 10 Oct 2017 16:30:48 -0400 Subject: Request for Review: Attributes.map generic types In-Reply-To: <845306f0-7855-fe01-a017-53e96fdafb31@paratix.ch> References: <845306f0-7855-fe01-a017-53e96fdafb31@paratix.ch> Message-ID: <1d212c50-7a35-0a2d-992c-edd1a36068fa@oracle.com> Cross-posting to core-libs-dev as this proposal is really more appropriate for review on that list. --Sean On 10/3/17 1:48 AM, Philipp Kunz wrote: > Hi > > This has not been asked for and there is also no bug yet but > nevertheless let me propose to change Attributes.map to specific generic > types. It looks like the type parameters were added automatically with > their introduction in java 5. I figure that no specific test is required > in this particular case because it's a pure compile-time issue which is > already sufficiently covered with existing code and there are also tests > that already now involve manifests with attributes at run-time. > > Any feedback or thoughts? Would someone volunteer to sponsor it? > > Regards, > Philipp > > > > ----- BEGIN PATCH ----- > diff -r 2e947e1bd907 > src/java.base/share/classes/java/util/jar/Attributes.java > --- a/src/java.base/share/classes/java/util/jar/Attributes.java Mon Oct > 02 10:04:22 2017 -0700 > +++ b/src/java.base/share/classes/java/util/jar/Attributes.java Tue Oct > 03 07:41:40 2017 +0200 > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 1997, 2015, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -54,11 +54,11 @@ > ? * @see???? Manifest > ? * @since?? 1.2 > ? */ > -public class Attributes implements Map, Cloneable { > +public class Attributes implements Map, > Cloneable { > ???? /** > ????? * The attribute name-value mappings. > ????? */ > -??? protected Map map; > +??? protected Map map; > > ???? /** > ????? * Constructs a new, empty Attributes object with default size. > @@ -87,7 +87,6 @@ > ???????? map = new LinkedHashMap<>(attr); > ???? } > > - > ???? /** > ????? * Returns the value of the specified attribute name, or null if the > ????? * attribute name was not found. > @@ -96,7 +95,7 @@ > ????? * @return the value of the specified attribute name, or null if > ????? *???????? not found. > ????? */ > -??? public Object get(Object name) { > +??? public String get(Object name) { > ???????? return map.get(name); > ???? } > > @@ -116,7 +115,7 @@ > ????? * @throws IllegalArgumentException if the attribute name is invalid > ????? */ > ???? public String getValue(String name) { > -??????? return (String)get(new Attributes.Name(name)); > +??????? return get(new Attributes.Name(name)); > ???? } > > ???? /** > @@ -133,7 +132,7 @@ > ????? *???????? not found. > ????? */ > ???? public String getValue(Name name) { > -??????? return (String)get(name); > +??????? return get(name); > ???? } > > ???? /** > @@ -144,11 +143,9 @@ > ????? * @param name the attribute name > ????? * @param value the attribute value > ????? * @return the previous value of the attribute, or null if none > -???? * @exception ClassCastException if the name is not a Attributes.Name > -???? *??????????? or the value is not a String > ????? */ > -??? public Object put(Object name, Object value) { > -??????? return map.put((Attributes.Name)name, (String)value); > +??? public String put(Attributes.Name name, String value) { > +??????? return map.put(name, value); > ???? } > > ???? /** > @@ -168,7 +165,7 @@ > ????? * @exception IllegalArgumentException if the attribute name is > invalid > ????? */ > ???? public String putValue(String name, String value) { > -??????? return (String)put(new Name(name), value); > +??????? return put(new Name(name), value); > ???? } > > ???? /** > @@ -178,7 +175,7 @@ > ????? * @param name attribute name > ????? * @return the previous value of the attribute, or null if none > ????? */ > -??? public Object remove(Object name) { > +??? public String remove(Object name) { > ???????? return map.remove(name); > ???? } > > @@ -208,15 +205,14 @@ > ????? * Copies all of the attribute name-value mappings from the specified > ????? * Attributes to this Map. Duplicate mappings will be replaced. > ????? * > -???? * @param attr the Attributes to be stored in this map > -???? * @exception ClassCastException if attr is not an Attributes > +???? * @param attr the attributes to be stored in this map, may also be of > +???? *???????????? type {@link Attributes} > ????? */ > -??? public void putAll(Map attr) { > -??????? // ## javac bug? > -??????? if (!Attributes.class.isInstance(attr)) > -??????????? throw new ClassCastException(); > -??????? for (Map.Entry me : (attr).entrySet()) > +??? public void putAll(Map attr) { > +??????? for (Map.Entry me > +??????? ??? ??? : attr.entrySet()) { > ???????????? put(me.getKey(), me.getValue()); > +??????? } > ???? } > > ???? /** > @@ -243,14 +239,14 @@ > ???? /** > ????? * Returns a Set view of the attribute names (keys) contained in > this Map. > ????? */ > -??? public Set keySet() { > +??? public Set keySet() { > ???????? return map.keySet(); > ???? } > > ???? /** > ????? * Returns a Collection view of the attribute values contained in > this Map. > ????? */ > -??? public Collection values() { > +??? public Collection values() { > ???????? return map.values(); > ???? } > > @@ -258,7 +254,7 @@ > ????? * Returns a Collection view of the attribute name-value mappings > ????? * contained in this Map. > ????? */ > -??? public Set> entrySet() { > +??? public Set> entrySet() { > ???????? return map.entrySet(); > ???? } > > @@ -290,7 +286,7 @@ > ????? * the Attributes returned can be safely modified without affecting > ????? * the original. > ????? */ > -??? public Object clone() { > +??? public Attributes clone() { > ???????? return new Attributes(this); > ???? } > > @@ -300,12 +296,12 @@ > ????? */ > ????? @SuppressWarnings("deprecation") > ????? void write(DataOutputStream os) throws IOException { > -???????? for (Entry e : entrySet()) { > +???????? for (Entry e : entrySet()) { > ????????????? StringBuffer buffer = new StringBuffer( > -???????????????????????????????????????? ((Name) e.getKey()).toString()); > +???????????????? e.getKey().toString()); > ????????????? buffer.append(": "); > > -???????????? String value = (String) e.getValue(); > +???????????? String value = e.getValue(); > ????????????? if (value != null) { > ????????????????? byte[] vb = value.getBytes("UTF8"); > ????????????????? value = new String(vb, 0, 0, vb.length); > @@ -343,14 +339,14 @@ > > ???????? // write out all attributes except for the version > ???????? // we wrote out earlier > -??????? for (Entry e : entrySet()) { > -??????????? String name = ((Name) e.getKey()).toString(); > +??????? for (Entry e : entrySet()) { > +??????????? String name = e.getKey().toString(); > ???????????? if ((version != null) && !(name.equalsIgnoreCase(vername))) { > > ???????????????? StringBuffer buffer = new StringBuffer(name); > ???????????????? buffer.append(": "); > > -??????????????? String value = (String) e.getValue(); > +??????????????? String value = e.getValue(); > ???????????????? if (value != null) { > ???????????????????? byte[] vb = value.getBytes("UTF8"); > ???????????????????? value = new String(vb, 0, 0, vb.length); > diff -r 2e947e1bd907 > src/java.base/share/classes/sun/net/www/protocol/jar/URLJarFile.java > --- > a/src/java.base/share/classes/sun/net/www/protocol/jar/URLJarFile.java > Mon Oct 02 10:04:22 2017 -0700 > +++ > b/src/java.base/share/classes/sun/net/www/protocol/jar/URLJarFile.java > Tue Oct 03 07:41:40 2017 +0200 > @@ -1,5 +1,5 @@ > ?/* > - * Copyright (c) 2001, 2016, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2001, 2017, Oracle and/or its affiliates. All rights > reserved. > ? * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > ? * > ? * This code is free software; you can redistribute it and/or modify it > @@ -148,14 +148,14 @@ > > ???????? Manifest man = new Manifest(); > ???????? Attributes attr = man.getMainAttributes(); > -??????? attr.putAll((Map)superAttr.clone()); > +??????? attr.putAll(superAttr.clone()); > > ???????? // now deep copy the manifest entries > ???????? if (superEntries != null) { > ???????????? Map entries = man.getEntries(); > ???????????? for (String key : superEntries.keySet()) { > ???????????????? Attributes at = superEntries.get(key); > -??????????????? entries.put(key, (Attributes) at.clone()); > +??????????????? entries.put(key, at.clone()); > ???????????? } > ???????? } > > @@ -261,7 +261,7 @@ > ???????????????? if (e != null) { > ???????????????????? Attributes a = e.get(getName()); > ???????????????????? if (a != null) > -??????????????????????? return? (Attributes)a.clone(); > +??????????????????????? return a.clone(); > ???????????????? } > ???????????? } > ???????????? return null; > diff -r 2e947e1bd907 > src/java.base/share/classes/sun/security/util/ManifestEntryVerifier.java > --- > a/src/java.base/share/classes/sun/security/util/ManifestEntryVerifier.java > Mon Oct 02 10:04:22 2017 -0700 > +++ > b/src/java.base/share/classes/sun/security/util/ManifestEntryVerifier.java > Tue Oct 03 07:41:40 2017 +0200 > @@ -30,6 +30,7 @@ > ?import java.security.CodeSigner; > ?import java.util.*; > ?import java.util.jar.*; > +import java.util.jar.Attributes.Name; > > ?import java.util.Base64; > > @@ -122,7 +123,7 @@ > ???????????? } > ???????? } > > -??????? for (Map.Entry se : attr.entrySet()) { > +??????? for (Map.Entry se : attr.entrySet()) { > ???????????? String key = se.getKey().toString(); > > ???????????? if (key.toUpperCase(Locale.ENGLISH).endsWith("-DIGEST")) { > @@ -146,7 +147,7 @@ > ???????????????????? digest.reset(); > ???????????????????? digests.add(digest); > ???????????????????? manifestHashes.add( > - Base64.getMimeDecoder().decode((String)se.getValue())); > + Base64.getMimeDecoder().decode(se.getValue())); > ???????????????? } > ???????????? } > ???????? } > diff -r 2e947e1bd907 > src/java.base/share/classes/sun/security/util/SignatureFileVerifier.java > --- > a/src/java.base/share/classes/sun/security/util/SignatureFileVerifier.java > Mon Oct 02 10:04:22 2017 -0700 > +++ > b/src/java.base/share/classes/sun/security/util/SignatureFileVerifier.java > Tue Oct 03 07:41:40 2017 +0200 > @@ -46,6 +46,7 @@ > ?import java.util.Locale; > ?import java.util.Map; > ?import java.util.jar.Attributes; > +import java.util.jar.Attributes.Name; > ?import java.util.jar.JarException; > ?import java.util.jar.JarFile; > ?import java.util.jar.Manifest; > @@ -445,7 +446,7 @@ > ???????? boolean validEntry = false; > > ???????? // go through all the attributes and process *-Digest-Manifest > entries > -??????? for (Map.Entry se : mattr.entrySet()) { > +??????? for (Map.Entry se : mattr.entrySet()) { > > ???????????? String key = se.getKey().toString(); > > @@ -469,7 +470,7 @@ > ???????????????? if (digest != null) { > ???????????????????? byte[] computedHash = md.manifestDigest(digest); > ???????????????????? byte[] expectedHash = > - Base64.getMimeDecoder().decode((String)se.getValue()); > + Base64.getMimeDecoder().decode(se.getValue()); > > ???????????????????? if (debug != null) { > ???????????????????????? debug.println("Signature File: Manifest digest " + > @@ -517,7 +518,7 @@ > > ???????? // go through all the attributes and process > ???????? // digest entries for the manifest main attributes > -??????? for (Map.Entry se : mattr.entrySet()) { > +??????? for (Map.Entry se : mattr.entrySet()) { > ???????????? String key = se.getKey().toString(); > > ???????????? if (key.toUpperCase(Locale.ENGLISH).endsWith(ATTR_DIGEST)) { > @@ -540,7 +541,7 @@ > ???????????????????????? md.get(ManifestDigester.MF_MAIN_ATTRS, false); > ???????????????????? byte[] computedHash = mde.digest(digest); > ???????????????????? byte[] expectedHash = > - Base64.getMimeDecoder().decode((String)se.getValue()); > + Base64.getMimeDecoder().decode(se.getValue()); > > ???????????????????? if (debug != null) { > ????????????????????? debug.println("Signature File: " + > @@ -620,7 +621,7 @@ > ???????????? //hex.encodeBuffer(data, System.out); > > ???????????? // go through all the attributes and process *-Digest entries > -??????????? for (Map.Entry se : sfAttr.entrySet()) { > +??????????? for (Map.Entry se : sfAttr.entrySet()) { > ???????????????? String key = se.getKey().toString(); > > ???????????????? if (key.toUpperCase(Locale.ENGLISH).endsWith("-DIGEST")) { > @@ -643,7 +644,7 @@ > ???????????????????????? boolean ok = false; > > ???????????????????????? byte[] expected = > - Base64.getMimeDecoder().decode((String)se.getValue()); > + Base64.getMimeDecoder().decode(se.getValue()); > ???????????????????????? byte[] computed; > ???????????????????????? if (workaround) { > ???????????????????????????? computed = mde.digestWorkaround(digest); > diff -r 2e947e1bd907 > src/jdk.jartool/share/classes/jdk/security/jarsigner/JarSigner.java > --- > a/src/jdk.jartool/share/classes/jdk/security/jarsigner/JarSigner.java > Mon Oct 02 10:04:22 2017 -0700 > +++ > b/src/jdk.jartool/share/classes/jdk/security/jarsigner/JarSigner.java > Tue Oct 03 07:41:40 2017 +0200 > @@ -685,7 +685,7 @@ > ???????????? // Manifest exists. Read its raw bytes. > ???????????? mfRawBytes = zipFile.getInputStream(mfFile).readAllBytes(); > ???????????? manifest.read(new ByteArrayInputStream(mfRawBytes)); > -??????????? oldAttr = (Attributes) (manifest.getMainAttributes().clone()); > +??????????? oldAttr = manifest.getMainAttributes().clone(); > ???????? } else { > ???????????? // Create new manifest > ???????????? Attributes mattr = manifest.getMainAttributes(); > ----- END PATCH ----- > > > ------------------------------------------------------------------------ > > > > Paratix GmbH > St Peterhofstatt 11 > 8001 Z?rich > > +41 (0)76 397 79 35 > philipp.kunz at paratix.ch From tom at quarendon.net Wed Oct 11 12:26:29 2017 From: tom at quarendon.net (tom at quarendon.net) Date: Wed, 11 Oct 2017 13:26:29 +0100 (BST) Subject: Trouble with SPNEGO Message-ID: <808499655.3685093.1507724794028@webmail.123-reg.co.uk> I'm having trouble with getting spnego to work, and was hoping someone might help. I'm trying to perform a "Negotiate" authentication from a browser. My browser is Chrome on Windows 10. The endpoint I'm connecting to us a standard jetty server (it's wrapped up in karaf), but I've implemented the actual Negotate handling manually. If I run the server on a Linux machine, using openjdk 1.8.141, it works, I can successfully authenticate. If I run the server on a Windows machine (same desktop machine as the browser is running on), using oracle jdk 1.8.144, I can't get it to work. I have researched around, and it looks like there are some relevant JDK issues in the past in this area (e.g, JDK-8078439, JDK-8048194), but I should have the fixes to those. However my symptoms seem very similar to these bug reports. However, since this seems very complicated to get exactly right, I suspect I've either got something wrong in my java code, or there's some issue in my machine Active Directory configuration. The sun.security.spnego.debug trace output in the case it doesn't work is this: SpNegoContext.acceptSecContext: receiving token = a0 74 30 72 a0 30 30 2e 06 0a 2b 06 01 04 01 82 37 02 02 0a 06 09 2a 86 48 82 f7 12 01 02 02 06 09 2a 86 48 86 f7 12 01 02 02 06 0a 2b 06 01 04 01 82 37 02 02 1e a2 3e 04 3c 4e 54 4c 4d 53 53 50 00 01 00 00 00 97 b2 08 e2 07 00 07 00 35 00 00 00 0d 00 0d 00 28 00 00 00 0a 00 d7 3a 00 00 00 0f 54 51 55 41 52 45 4e 44 4f 4e 2d 50 43 54 45 41 4d 57 50 43 SpNegoToken NegTokenInit: reading Mechanism Oid = 1.3.6.1.4.1.311.2.2.10 SpNegoToken NegTokenInit: reading Mechanism Oid = 1.2.840.48018.1.2.2 SpNegoToken NegTokenInit: reading Mechanism Oid = 1.2.840.113554.1.2.2 SpNegoToken NegTokenInit: reading Mechanism Oid = 1.3.6.1.4.1.311.2.2.30 SpNegoToken NegTokenInit: reading Mech Token SpNegoContext.acceptSecContext: received token of type = SPNEGO NegTokenInit SpNegoContext: negotiated mechanism = 1.2.840.113554.1.2.2 The underlying mechanism context has not been initialized SpNegoContext.acceptSecContext: mechanism wanted = 1.2.840.113554.1.2.2 SpNegoContext.acceptSecContext: negotiated result = ACCEPT_INCOMPLETE SpNegoContext.acceptSecContext: sending token of type = SPNEGO NegTokenTarg SpNegoContext.acceptSecContext: sending token = a1 14 30 12 a0 03 0a 01 01 a1 0b 06 09 2a 86 48 86 f7 12 01 02 02 I then get a second request comming in, and this fails quickly with GSSAPI exception GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag) at sun.security.jgss.GSSHeader.(Unknown Source) at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source) at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source) When it does work, to a linux machine, I see this: SpNegoContext.acceptSecContext: receiving token = SpNegoToken NegTokenInit: reading Mechanism Oid = 1.2.840.48018.1.2.2 SpNegoToken NegTokenInit: reading Mechanism Oid = 1.2.840.113554.1.2.2 SpNegoToken NegTokenInit: reading Mechanism Oid = 1.3.6.1.4.1.311.2.2.30 SpNegoToken NegTokenInit: reading Mechanism Oid = 1.3.6.1.4.1.311.2.2.10 SpNegoToken NegTokenInit: reading Mech Token SpNegoContext.acceptSecContext: received token of type = SPNEGO NegTokenInit SpNegoContext: negotiated mechanism = 1.2.840.113554.1.2.2 SpNegoContext.acceptSecContext: negotiated mech adjusted to 1.2.840.48018.1.2.2 Entered Krb5Context.acceptSecContext with state=STATE_NEW The obvious difference here is the order of the mechanism OIDs. My first question is why this is different? It's the same browser performing both requests. So does this difference indicate something different in the kerberos service tickets the client machine has for the two services? I have had difficulties getting this far, with there being some configuration changes needed at the Active Directory level related to my Windows machine, so maybe there's still some some configuration issue there? Tracing through the java code in SpNegoContext.java, it's clear that the condition: if (mechList[0].equals(mech_wanted) || (GSSUtil.isKerberosMech(mechList[0]) && GSSUtil.isKerberosMech(mech_wanted))) { isn't true, hence GSS_acceptSecContext is never called, hence isMechContextEstablished() when it is called generates the "The underlying mechanism context has not been initialized" message. However I've no idea why that's the case. Can someone give me more of an insight into why this isn't working for me? Given the previous bugs in this area that have been fixed, I suspect that this *can* work, which suggests to me that I've got some kind of configuration problem that is why I'm getting this. Thanks. From mbalao at redhat.com Wed Oct 11 13:31:30 2017 From: mbalao at redhat.com (Martin Balao) Date: Wed, 11 Oct 2017 10:31:30 -0300 Subject: RFR 6913047: SunPKCS11 memory leak Message-ID: Hi, I'd like to propose a fix for bug JDK-6913047: "Long term memory leak when using PKCS11 and JCE exceeds 32 bit process address space" [1]. This fix does not contain changes in the GC and is SunPKCS11 internal only. PROBLEM ........................................................ When using the SunPKCS11 crypto provider (for cipher, signature, mac, key generation or any other operation), multiple key objects may be created. I.e.: every time a TLS session is established, a unique master key (derived from the pre-master-key) has to be created and then used for encryption and decryption operations. This is a legitimate use case in which key caching does not make sense as each key is unique per session. These keys are of P11Key type and have a corresponding native key object created. In the case of NSS SunPKCS11 backend (PKCS11 software token), this native key object is temporarily stored in the process native heap. The interface is simple: a JNI call is done to create a native key object (C_CreateObject, C_CopyObject, C_DeriveKey, C_GenerateKeys, etc., according to the PKCS11 interface) and an integer handler is kept in the Java side (P11Key). When the P11Key object is destroyed, a finalizer code is executed to free the native key object (through C_DestroyObject JNI call). The problem is that finalizer code execution happens only if the JVM garbage-collector cleans up the P11Key object. That may be delayed or not done at all, depending on different GC algorithms, parameters and environment conditions. As a result, the native heap may be exhausted with not freed native key objects, and the JVM will then crash -this is particularly true for 32 bits VMs where the virtual address space can be exhausted-. SCOPE ........................................................ The fix is proposed for SunPKCS11 with NSS backend only. Other PKCS11 backends are not currently under scope. It's likely that hardware PKCS11 backends store native key objects in their own memory, preventing a native heap exhaustion and a JVM crash. However, it might be possible to cause an exhaustion on their own memory blocking key objects creation at some point. In any case, this is speculative as no tests were done on our side with real hardware. SOLUTION ........................................................ Assuming that native keys are extractable, the idea is to hold native key data in the Java heap while keys are not in use. When a P11Key is created, every CK_ATTRIBUTE (PKCS11) value for the native key is queried, data stored in an opaque Java byte[] (inside the P11Key object) and native key destroyed. Every time the P11Key is about to be used, the native key is created with the stored data. After usage, the native key is again destroyed. Thus, it's not necessary to wait for a finalizer execution to cleanup native resources: cleanup is done at deterministic and previously-known points. This comes with a resposibility for key users -which are all SunPKCS11 internal services like P11Signature, P11Cipher, P11KeyGenerator, etc.-: create and destroy native keys through a reference counting scheme exposed by P11Key class. There are two kind of usages: 1) stateless: the native key is "atomically" created, used and destroyed. I.e.: MAC calculation, getEncodedInternal operation (on P11Key objects), signature operations, TLS key derivation, etc. 2) statefull: the native key is created, one or multiple intermediate actions are performed by the key user, a final action is performed and finally the native key is destroyed. I.e.: cipher operations. For keys that are extractable but sensitive (CKA_SENSITIVE attribute is true), as the case when operating in FIPS mode, wrapping/unwrapping is used as a workaround to extract session keys. Wrapper key is global and lives forever. There are no interface changes for SunPKCS11 external users. If keys are not extractable or the feature cannot be enabled for any other reason, the previous finalizer scheme is used as a fallback. ADDITIONAL IMPLEMENTATION NOTES ........................................................ When a P11Key is created, a constructor parameter exists to indicate if the feature is enabled for that specific key. For this feature to be enabled, 2 additional conditions apply: 1) SunPKCS11 backend has to be NSS, and 2) key has to be extractable. If the feature is not enabled for a key, behavior is as previous to this patch (native key destruction relies on finalizer execution). The only P11Key user that explicitly does not use this feature is P11KeyStore. This is because these keys (token keys) are managed by alias names and makes no sense to remove them from the key store (they need to be accessible by an alias at any time). Because P11Key objects can be used by multiple threads at a time, there is a thread-safe reference counting scheme in order to decide when a native key object has to be created or destroyed. The SunPKCS11 internal API to use a P11Key is as follows: 1) increment the reference counter (which will eventually create the native key object if it doesn't exist), 2) use the key and 3) decrement the reference counter (which will eventually destroy the native key if there it's not being used by anyone else). The reason why an opaque byte[] is used in P11Key objects to store native keys data (instead of a CK_ATTRIBUTE[] Java objects, queried by Java's C_GetAttributeValue function) is performance. My prototypes show a difference of 4x in speed. 2 functions were added to libj2pkcs11 library: getNativeKeyInfo (to extract the opaque byte[] from a native key object) and createNativeKey (to create a native key object from an opaque byte[]). CHANGESET ........................................................ This changeset is JDK-10 (at jdk c8796a577885 rev) based: * http://people.redhat.com/mbalaoal/webrevs/jdk_6913047_sunpkcs11_nss_memory_leak/2017_10_06/6913047.webrev.04/ (browse online) * http://people.redhat.com/mbalaoal/webrevs/jdk_6913047_sunpkcs11_nss_memory_leak/2017_10_06/6913047.webrev.04.zip (download) TESTING ........................................................ Test suite for 32 bits JVMs only: http://people.redhat.com/mbalaoal/webrevs/jdk_6913047_sunpkcs11_nss_memory_leak/2017_10_06/Bug6913047.java * Suite (Bug6913047.java) * Tests JVM memory exhaustion while using keys for different services: P11Cipher, P11Signature, P11KeyAgreement, P11Mac, P11Digest, P11KeyGenerator, P11KeyFactory, etc. * Tests functional regression. * Including Key Stores (P11KeyStore) Parameters to run the reproducer (on JDK-10): * javac: --add-modules jdk.crypto.cryptoki --add-exports java.base/sun.security.internal.spec=ALL-UNNAMED * java: -XX:+UseParallelGC -Xmx3500m --add-modules jdk.crypto.cryptoki --add-opens java.base/javax.crypto=ALL-UNNAMED --add-opens jdk.crypto.cryptoki/sun.security.pkcs11=ALL-UNNAMED You can also use jtreg. PERFORMANCE ........................................................ For a quick reproducer previously developed (which looped 100000 times creating P11Cipher and P11Key objects to encrypt a plaintext), these are the figures I got: * real 1m11.328s (without fix) * real 1m12.795s (with fix) Performance penalty seems to be low in current state. OTHER ....................................................... My employer has an OCA agreement with Oracle and this work has been done in that context. Look forward to your comments. Kind regards, Martin.- -- [1] - https://bugs.openjdk.java.net/browse/JDK-6913047 -------------- next part -------------- An HTML attachment was scrubbed... URL: From weijun.wang at oracle.com Wed Oct 11 14:07:39 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 11 Oct 2017 22:07:39 +0800 Subject: Trouble with SPNEGO In-Reply-To: <808499655.3685093.1507724794028@webmail.123-reg.co.uk> References: <808499655.3685093.1507724794028@webmail.123-reg.co.uk> Message-ID: 1.3.6.1.4.1.311.2.2.10 is NTLM which has a much smaller token size. Java does not support NTLM as a GSS-API mechanism. I am not sure how this happens. Normally you need to configure the browser (For example, somewhere in about:config for Firefox) on which server you can use SPNEGO. Maybe you set up one and ignored the other? BTW, I assume you are using a single Windows KDC with 2 service principals HTTP/windows.server and HTTP/linux.server and different keytab used on those 2 servers. Is that correct? --Max > On Oct 11, 2017, at 8:26 PM, tom at quarendon.net wrote: > > I'm having trouble with getting spnego to work, and was hoping someone might help. > I'm trying to perform a "Negotiate" authentication from a browser. My browser is Chrome on Windows 10. > The endpoint I'm connecting to us a standard jetty server (it's wrapped up in karaf), but I've implemented the actual Negotate handling manually. > > If I run the server on a Linux machine, using openjdk 1.8.141, it works, I can successfully authenticate. > If I run the server on a Windows machine (same desktop machine as the browser is running on), using oracle jdk 1.8.144, I can't get it to work. > > I have researched around, and it looks like there are some relevant JDK issues in the past in this area (e.g, JDK-8078439, JDK-8048194), but I should have the fixes to those. However my symptoms seem very similar to these bug reports. However, since this seems very complicated to get exactly right, I suspect I've either got something wrong in my java code, or there's some issue in my machine Active Directory configuration. > > The sun.security.spnego.debug trace output in the case it doesn't work is this: > > SpNegoContext.acceptSecContext: receiving token = a0 74 30 72 a0 30 30 2e 06 0a 2b 06 01 04 01 82 37 02 02 0a 06 09 2a 86 48 82 f7 12 01 02 02 06 09 2a 86 48 86 f7 12 01 02 02 06 0a 2b 06 01 04 01 82 37 02 02 1e a2 3e 04 3c 4e 54 4c 4d 53 53 50 00 01 00 00 00 97 b2 08 e2 07 00 07 00 35 00 00 00 0d 00 0d 00 28 00 00 00 0a 00 d7 3a 00 00 00 0f 54 51 55 41 52 45 4e 44 4f 4e 2d 50 43 54 45 41 4d 57 50 43 > SpNegoToken NegTokenInit: reading Mechanism Oid = 1.3.6.1.4.1.311.2.2.10 > SpNegoToken NegTokenInit: reading Mechanism Oid = 1.2.840.48018.1.2.2 > SpNegoToken NegTokenInit: reading Mechanism Oid = 1.2.840.113554.1.2.2 > SpNegoToken NegTokenInit: reading Mechanism Oid = 1.3.6.1.4.1.311.2.2.30 > SpNegoToken NegTokenInit: reading Mech Token > SpNegoContext.acceptSecContext: received token of type = SPNEGO NegTokenInit > SpNegoContext: negotiated mechanism = 1.2.840.113554.1.2.2 > The underlying mechanism context has not been initialized > SpNegoContext.acceptSecContext: mechanism wanted = 1.2.840.113554.1.2.2 > SpNegoContext.acceptSecContext: negotiated result = ACCEPT_INCOMPLETE > SpNegoContext.acceptSecContext: sending token of type = SPNEGO NegTokenTarg > SpNegoContext.acceptSecContext: sending token = a1 14 30 12 a0 03 0a 01 01 a1 0b 06 09 2a 86 48 86 f7 12 01 02 02 > > I then get a second request comming in, and this fails quickly with > GSSAPI exception > GSSException: Defective token detected (Mechanism level: GSSHeader did not find the right tag) > at sun.security.jgss.GSSHeader.(Unknown Source) > at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source) > at sun.security.jgss.GSSContextImpl.acceptSecContext(Unknown Source) > > > When it does work, to a linux machine, I see this: > SpNegoContext.acceptSecContext: receiving token = > SpNegoToken NegTokenInit: reading Mechanism Oid = 1.2.840.48018.1.2.2 > SpNegoToken NegTokenInit: reading Mechanism Oid = 1.2.840.113554.1.2.2 > SpNegoToken NegTokenInit: reading Mechanism Oid = 1.3.6.1.4.1.311.2.2.30 > SpNegoToken NegTokenInit: reading Mechanism Oid = 1.3.6.1.4.1.311.2.2.10 > SpNegoToken NegTokenInit: reading Mech Token > SpNegoContext.acceptSecContext: received token of type = SPNEGO NegTokenInit > SpNegoContext: negotiated mechanism = 1.2.840.113554.1.2.2 > SpNegoContext.acceptSecContext: negotiated mech adjusted to 1.2.840.48018.1.2.2 > Entered Krb5Context.acceptSecContext with state=STATE_NEW > > > The obvious difference here is the order of the mechanism OIDs. My first question is why this is different? It's the same browser performing both requests. So does this difference indicate something different in the kerberos service tickets the client machine has for the two services? I have had difficulties getting this far, with there being some configuration changes needed at the Active Directory level related to my Windows machine, so maybe there's still some some configuration issue there? > > > Tracing through the java code in SpNegoContext.java, it's clear that the condition: > if (mechList[0].equals(mech_wanted) || > (GSSUtil.isKerberosMech(mechList[0]) && > GSSUtil.isKerberosMech(mech_wanted))) { > > isn't true, hence GSS_acceptSecContext is never called, hence isMechContextEstablished() when it is called generates the "The underlying mechanism context has not been initialized" message. However I've no idea why that's the case. > > Can someone give me more of an insight into why this isn't working for me? Given the previous bugs in this area that have been fixed, I suspect that this *can* work, which suggests to me that I've got some kind of configuration problem that is why I'm getting this. > > Thanks. From mbalao at redhat.com Wed Oct 11 14:27:20 2017 From: mbalao at redhat.com (Martin Balao) Date: Wed, 11 Oct 2017 11:27:20 -0300 Subject: RFR 6913047: SunPKCS11 memory leak In-Reply-To: References: Message-ID: SCOPE clarification: * The proposed patch should be solving the issue in HSMs also. Real hardware testing would be needed to check that, but it should be working. On Wed, Oct 11, 2017 at 10:31 AM, Martin Balao wrote: > Hi, > > I'd like to propose a fix for bug JDK-6913047: "Long term memory leak when > using PKCS11 and JCE exceeds 32 bit process address space" [1]. This fix > does not contain changes in the GC and is SunPKCS11 internal only. > > PROBLEM > ........................................................ > > When using the SunPKCS11 crypto provider (for cipher, signature, mac, key > generation or any other operation), multiple key objects may be created. > I.e.: every time a TLS session is established, a unique master key (derived > from the pre-master-key) has to be created and then used for encryption and > decryption operations. This is a legitimate use case in which key caching > does not make sense as each key is unique per session. These keys are of > P11Key type and have a corresponding native key object created. In the case > of NSS SunPKCS11 backend (PKCS11 software token), this native key object is > temporarily stored in the process native heap. The interface is simple: a > JNI call is done to create a native key object (C_CreateObject, > C_CopyObject, C_DeriveKey, C_GenerateKeys, etc., according to the PKCS11 > interface) and an integer handler is kept in the Java side (P11Key). When > the P11Key object is destroyed, a finalizer code is executed to free the > native key object (through C_DestroyObject JNI call). The problem is that > finalizer code execution happens only if the JVM garbage-collector cleans > up the P11Key object. That may be delayed or not done at all, depending on > different GC algorithms, parameters and environment conditions. As a > result, the native heap may be exhausted with not freed native key objects, > and the JVM will then crash -this is particularly true for 32 bits VMs > where the virtual address space can be exhausted-. > > > SCOPE > ........................................................ > > The fix is proposed for SunPKCS11 with NSS backend only. Other PKCS11 > backends are not currently under scope. It's likely that hardware PKCS11 > backends store native key objects in their own memory, preventing a native > heap exhaustion and a JVM crash. However, it might be possible to cause an > exhaustion on their own memory blocking key objects creation at some point. > In any case, this is speculative as no tests were done on our side with > real hardware. > > > SOLUTION > ........................................................ > > Assuming that native keys are extractable, the idea is to hold native key > data in the Java heap while keys are not in use. When a P11Key is created, > every CK_ATTRIBUTE (PKCS11) value for the native key is queried, data > stored in an opaque Java byte[] (inside the P11Key object) and native key > destroyed. Every time the P11Key is about to be used, the native key is > created with the stored data. After usage, the native key is again > destroyed. Thus, it's not necessary to wait for a finalizer execution to > cleanup native resources: cleanup is done at deterministic and > previously-known points. This comes with a resposibility for key users > -which are all SunPKCS11 internal services like P11Signature, P11Cipher, > P11KeyGenerator, etc.-: create and destroy native keys through a reference > counting scheme exposed by P11Key class. There are two kind of usages: > > 1) stateless: the native key is "atomically" created, used and destroyed. > I.e.: MAC calculation, getEncodedInternal operation (on P11Key objects), > signature operations, TLS key derivation, etc. > > 2) statefull: the native key is created, one or multiple intermediate > actions are performed by the key user, a final action is performed and > finally the native key is destroyed. I.e.: cipher operations. > > For keys that are extractable but sensitive (CKA_SENSITIVE attribute is > true), as the case when operating in FIPS mode, wrapping/unwrapping is used > as a workaround to extract session keys. Wrapper key is global and lives > forever. > > There are no interface changes for SunPKCS11 external users. > > If keys are not extractable or the feature cannot be enabled for any other > reason, the previous finalizer scheme is used as a fallback. > > > ADDITIONAL IMPLEMENTATION NOTES > ........................................................ > > When a P11Key is created, a constructor parameter exists to indicate if > the feature is enabled for that specific key. For this feature to be > enabled, 2 additional conditions apply: 1) SunPKCS11 backend has to be NSS, > and 2) key has to be extractable. If the feature is not enabled for a key, > behavior is as previous to this patch (native key destruction relies on > finalizer execution). > > The only P11Key user that explicitly does not use this feature is > P11KeyStore. This is because these keys (token keys) are managed by alias > names and makes no sense to remove them from the key store (they need to be > accessible by an alias at any time). > > Because P11Key objects can be used by multiple threads at a time, there is > a thread-safe reference counting scheme in order to decide when a native > key object has to be created or destroyed. The SunPKCS11 internal API to > use a P11Key is as follows: 1) increment the reference counter (which will > eventually create the native key object if it doesn't exist), 2) use the > key and 3) decrement the reference counter (which will eventually destroy > the native key if there it's not being used by anyone else). > > The reason why an opaque byte[] is used in P11Key objects to store native > keys data (instead of a CK_ATTRIBUTE[] Java objects, queried by Java's > C_GetAttributeValue function) is performance. My prototypes show a > difference of 4x in speed. 2 functions were added to libj2pkcs11 library: > getNativeKeyInfo (to extract the opaque byte[] from a native key object) > and createNativeKey (to create a native key object from an opaque byte[]). > > > CHANGESET > ........................................................ > > This changeset is JDK-10 (at jdk c8796a577885 rev) based: > > * http://people.redhat.com/mbalaoal/webrevs/jdk_6913047_ > sunpkcs11_nss_memory_leak/2017_10_06/6913047.webrev.04/ (browse online) > * http://people.redhat.com/mbalaoal/webrevs/jdk_6913047_ > sunpkcs11_nss_memory_leak/2017_10_06/6913047.webrev.04.zip (download) > > > TESTING > ........................................................ > > Test suite for 32 bits JVMs only: http://people.redhat.com/ > mbalaoal/webrevs/jdk_6913047_sunpkcs11_nss_memory_leak/ > 2017_10_06/Bug6913047.java > > * Suite (Bug6913047.java) > * Tests JVM memory exhaustion while using keys for different services: > P11Cipher, P11Signature, P11KeyAgreement, P11Mac, P11Digest, > P11KeyGenerator, P11KeyFactory, etc. > * Tests functional regression. > * Including Key Stores (P11KeyStore) > > Parameters to run the reproducer (on JDK-10): > * javac: --add-modules jdk.crypto.cryptoki --add-exports > java.base/sun.security.internal.spec=ALL-UNNAMED > * java: -XX:+UseParallelGC -Xmx3500m --add-modules jdk.crypto.cryptoki > --add-opens java.base/javax.crypto=ALL-UNNAMED --add-opens > jdk.crypto.cryptoki/sun.security.pkcs11=ALL-UNNAMED > > You can also use jtreg. > > > PERFORMANCE > ........................................................ > > For a quick reproducer previously developed (which looped 100000 times > creating P11Cipher and P11Key objects to encrypt a plaintext), these are > the figures I got: > > * real 1m11.328s (without fix) > * real 1m12.795s (with fix) > > Performance penalty seems to be low in current state. > > OTHER > ....................................................... > > My employer has an OCA agreement with Oracle and this work has been done > in that context. > > Look forward to your comments. > > Kind regards, > Martin.- > > -- > [1] - https://bugs.openjdk.java.net/browse/JDK-6913047 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom at quarendon.net Wed Oct 11 18:42:18 2017 From: tom at quarendon.net (tom at quarendon.net) Date: Wed, 11 Oct 2017 19:42:18 +0100 (BST) Subject: Trouble with SPNEGO In-Reply-To: <1867357988.430127.1507740335428@webmail.123-reg.co.uk> References: <808499655.3685093.1507724794028@webmail.123-reg.co.uk> <1867357988.430127.1507740335428@webmail.123-reg.co.uk> Message-ID: <983580809.434597.1507747338267@webmail.123-reg.co.uk> > 1.3.6.1.4.1.311.2.2.10 is NTLM which has a much smaller token size. Java does not support NTLM as a GSS-API mechanism. Yes. So it looks like the browser is preferring NTLM and pre-emptively sending a token for that. > I am not sure how this happens. Normally you need to configure the browser (For example, somewhere in about:config for Firefox) on which server you can use SPNEGO. Maybe you set up one and ignored the other? Chrome uses the normal Windows "internet options" configuration for this. You configure "zones" and one of the settings is "allow windows integrated authentication". Both machines in my case are in the same domain, so I *think* that it should be attempting the same kind of authentication. But I've no idea how I tell (well, other than the fact that the lists of mechanism OIDs it sends are different, so there must be *some* difference) I get exactly the same result if I try from Firefox. Adding my host into the network.negotiate-auth.trusted-uris configuration setting doesn't have any effect. Well, without it firefox doesn't seem to do anything, but with it I get exactly the same behaviour of mechanism OIDs being sent. I also get the same behaviour if I use a C++ HTTP client I have that just uses Windows SSPI "negotiate" package to do the same thing, but my guess is that's subject to the same basic configuration in terms of what mechanisms it will put in its list. > BTW, I assume you are using a single Windows KDC with 2 service principals HTTP/windows.server and HTTP/linux.server and different keytab used on those 2 servers. Is that correct? I am, yes. I have successfully run a "java to java" example using the same keytab I'm using for this experiment, so I've successfully used a java client to perform the same authentication (at a lower level, directly, not going over HTTP, just JGSS talking to JGSS if you know what I mean, as a "unit test" of the basic code that's accepting the token that comes out of the HTTP header and attempts to perform the authentication), so I believe that the keytab is OK, and I believe that the KDC will accept my requests to authenticate users. However I have had difficulties getting that far. The Linux set up worked easily, but the Windows setup was more complex, there needs to be an actual user principal as well as the HTTP service principal. I *believe* that I'm passed that now though. Issue JDK-8048194 has the same set of mechanism OIDs, and has the same basic symptom, but I'm unclear whether the issue is the same or not there. The fact that the client request contains Kerberos in the mechanism OIDs suggests to me that it would be happy to process a Kerberos authentication, it's just that its first choice would be NTLM. I'm assuming that the response that the java sends back is "I can't accept your first choice, but this OID is on your list and I will accept that, so send me a token for that mechanism please". The browser response to that though appears to be a token that the java code can't read, and the error appears to come before any of the debugging output that prints any of the token information. Naively, I assume that the fact that the client and the server are on the same machine isn't an issue? Thanks for any help you can give me! From magnus.ihse.bursie at oracle.com Thu Oct 12 09:19:44 2017 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 12 Oct 2017 11:19:44 +0200 Subject: Manifest Related Bugs JDK-6372077, JDK-6202130, JDK-8176843, JDK-4842483, JDK-6443578, JDK-6910466, JDK-4935610, and JDK-4271239 In-Reply-To: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> References: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> Message-ID: <86bc4261-fa00-ca56-45d1-8285bfa78038@oracle.com> On 2017-10-02 19:24, Philipp Kunz wrote: > Hi, > > While fixing JDK-6695402 I came across other related bugs to manifests > such as JDK-6372077, JDK-6202130, JDK-8176843, JDK-4842483, and > JDK-6443578 which all relate to manifest reading and writing. > Somewhere bug 7071055 is mentioned but I cannot find anywhere. You cannot find JDK-7071055 since it's classified Confidential. I'm not sure why, but it has been closed as a duplicate of JDK-6695402, so it does not really matter. I'm happy to see we've gotten such a thorough contributor! :-) Keep up the good work. /Magnus > Another group of bugs, JDK-6910466, JDK-4935610, and JDK-4271239 > concern the mandatory manifest main attributes Manifest-Version or > Signature-Version and at first glance are duplicates. If you know of > more known bugs, not necessarily present in jira, I'd be glad to get > notified. > > There are also some comments about utf handling and line breaking in > the code of Manifest: > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l299 > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l327 > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l370 > > Furthermore, Attributes.map should declare appropriate type parameters: > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l61 > The specification would also require that `header names must not start > with _, - or "From"` > (http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html#Section-Specification) > but I would opt not to add this as a hard restriction because I guess > it can be assumed that such names are in use now after having been > working for years. A warning to a logger might be conceivable such as in > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l424 > Attribute values are never checked at all and invalid characters such > as line breaks are never detected except that when reading the > manifest again the values are cut off. > The tab character (U+0008) does not work in manifest values. > I suspect that there are also issues regarding the iteration order but > I haven't got a prove yet unlike for the other points above: > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Manifest.java#l54 > There is duplicated or very similar code in Attributes and Manifest: > Attributes.write-Manifest.write-Attributes.writeMain and > Attributes.read-Manifest.read. > Resolving JDK-6202130 would have the additional benefit to be able to > view manifests with any utf-8 capable editor even if multi-byte > characters break across lines. > > Fixing these issues individually looks like more complicated work than > fixing them all at once, I guess, also because of a very low current > test coverage. So I'd suggest to add some thorough tests along with > fixing these issues. But before I start I would like to get some > guidance, assistance or at least confirmation on how to proceed. I'm > new to open jdk and have submitted only one patch so far. > > Is it ok to add tests for things that have worked before? > Is it ok to refactor duplicated code just for the added value to > reduce effort for testing? > I assume compatibility to and from existing manifests is the highest > priority, correct? This would also be the hard part in adding as > complete test coverage as possible. What would be acceptable criteria > to meet? > Why does Attributes not extend LinkedHashMap and why does Manifest not > extend HashMap? Any objection? > While I would not want to write code that looks slow or change more > than necessary one can never know before having performance actually > measured. Is there some way this is dealt with or should I wait for > such feedback until after patch submission? > > Would someone sponsor? > > Regards, > Philipp > > > > > ------------------------------------------------------------------------ > > > > Paratix GmbH > St Peterhofstatt 11 > 8001 Z?rich > > +41 (0)76 397 79 35 > philipp.kunz at paratix.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: lhjimfkdlcjkhmoc.png Type: image/png Size: 5134 bytes Desc: not available URL: From mbalao at redhat.com Thu Oct 12 13:00:09 2017 From: mbalao at redhat.com (Martin Balao) Date: Thu, 12 Oct 2017 10:00:09 -0300 Subject: RFR 6913047: SunPKCS11 memory leak In-Reply-To: References: Message-ID: Webrev 04 uploaded to cr.openjdk.java.net: * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK- 6913047/webrev.04/ (browse online) * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK- 6913047/webrev.04/6913047.webrev.04.zip (download) On Wed, Oct 11, 2017 at 10:31 AM, Martin Balao wrote: > Hi, > > I'd like to propose a fix for bug JDK-6913047: "Long term memory leak when > using PKCS11 and JCE exceeds 32 bit process address space" [1]. This fix > does not contain changes in the GC and is SunPKCS11 internal only. > > PROBLEM > ........................................................ > > When using the SunPKCS11 crypto provider (for cipher, signature, mac, key > generation or any other operation), multiple key objects may be created. > I.e.: every time a TLS session is established, a unique master key (derived > from the pre-master-key) has to be created and then used for encryption and > decryption operations. This is a legitimate use case in which key caching > does not make sense as each key is unique per session. These keys are of > P11Key type and have a corresponding native key object created. In the case > of NSS SunPKCS11 backend (PKCS11 software token), this native key object is > temporarily stored in the process native heap. The interface is simple: a > JNI call is done to create a native key object (C_CreateObject, > C_CopyObject, C_DeriveKey, C_GenerateKeys, etc., according to the PKCS11 > interface) and an integer handler is kept in the Java side (P11Key). When > the P11Key object is destroyed, a finalizer code is executed to free the > native key object (through C_DestroyObject JNI call). The problem is that > finalizer code execution happens only if the JVM garbage-collector cleans > up the P11Key object. That may be delayed or not done at all, depending on > different GC algorithms, parameters and environment conditions. As a > result, the native heap may be exhausted with not freed native key objects, > and the JVM will then crash -this is particularly true for 32 bits VMs > where the virtual address space can be exhausted-. > > > SCOPE > ........................................................ > > The fix is proposed for SunPKCS11 with NSS backend only. Other PKCS11 > backends are not currently under scope. It's likely that hardware PKCS11 > backends store native key objects in their own memory, preventing a native > heap exhaustion and a JVM crash. However, it might be possible to cause an > exhaustion on their own memory blocking key objects creation at some point. > In any case, this is speculative as no tests were done on our side with > real hardware. > > > SOLUTION > ........................................................ > > Assuming that native keys are extractable, the idea is to hold native key > data in the Java heap while keys are not in use. When a P11Key is created, > every CK_ATTRIBUTE (PKCS11) value for the native key is queried, data > stored in an opaque Java byte[] (inside the P11Key object) and native key > destroyed. Every time the P11Key is about to be used, the native key is > created with the stored data. After usage, the native key is again > destroyed. Thus, it's not necessary to wait for a finalizer execution to > cleanup native resources: cleanup is done at deterministic and > previously-known points. This comes with a resposibility for key users > -which are all SunPKCS11 internal services like P11Signature, P11Cipher, > P11KeyGenerator, etc.-: create and destroy native keys through a reference > counting scheme exposed by P11Key class. There are two kind of usages: > > 1) stateless: the native key is "atomically" created, used and destroyed. > I.e.: MAC calculation, getEncodedInternal operation (on P11Key objects), > signature operations, TLS key derivation, etc. > > 2) statefull: the native key is created, one or multiple intermediate > actions are performed by the key user, a final action is performed and > finally the native key is destroyed. I.e.: cipher operations. > > For keys that are extractable but sensitive (CKA_SENSITIVE attribute is > true), as the case when operating in FIPS mode, wrapping/unwrapping is used > as a workaround to extract session keys. Wrapper key is global and lives > forever. > > There are no interface changes for SunPKCS11 external users. > > If keys are not extractable or the feature cannot be enabled for any other > reason, the previous finalizer scheme is used as a fallback. > > > ADDITIONAL IMPLEMENTATION NOTES > ........................................................ > > When a P11Key is created, a constructor parameter exists to indicate if > the feature is enabled for that specific key. For this feature to be > enabled, 2 additional conditions apply: 1) SunPKCS11 backend has to be NSS, > and 2) key has to be extractable. If the feature is not enabled for a key, > behavior is as previous to this patch (native key destruction relies on > finalizer execution). > > The only P11Key user that explicitly does not use this feature is > P11KeyStore. This is because these keys (token keys) are managed by alias > names and makes no sense to remove them from the key store (they need to be > accessible by an alias at any time). > > Because P11Key objects can be used by multiple threads at a time, there is > a thread-safe reference counting scheme in order to decide when a native > key object has to be created or destroyed. The SunPKCS11 internal API to > use a P11Key is as follows: 1) increment the reference counter (which will > eventually create the native key object if it doesn't exist), 2) use the > key and 3) decrement the reference counter (which will eventually destroy > the native key if there it's not being used by anyone else). > > The reason why an opaque byte[] is used in P11Key objects to store native > keys data (instead of a CK_ATTRIBUTE[] Java objects, queried by Java's > C_GetAttributeValue function) is performance. My prototypes show a > difference of 4x in speed. 2 functions were added to libj2pkcs11 library: > getNativeKeyInfo (to extract the opaque byte[] from a native key object) > and createNativeKey (to create a native key object from an opaque byte[]). > > > CHANGESET > ........................................................ > > This changeset is JDK-10 (at jdk c8796a577885 rev) based: > > * http://people.redhat.com/mbalaoal/webrevs/jdk_6913047_ > sunpkcs11_nss_memory_leak/2017_10_06/6913047.webrev.04/ (browse online) > * http://people.redhat.com/mbalaoal/webrevs/jdk_6913047_ > sunpkcs11_nss_memory_leak/2017_10_06/6913047.webrev.04.zip (download) > > > TESTING > ........................................................ > > Test suite for 32 bits JVMs only: http://people.redhat.com/ > mbalaoal/webrevs/jdk_6913047_sunpkcs11_nss_memory_leak/ > 2017_10_06/Bug6913047.java > > * Suite (Bug6913047.java) > * Tests JVM memory exhaustion while using keys for different services: > P11Cipher, P11Signature, P11KeyAgreement, P11Mac, P11Digest, > P11KeyGenerator, P11KeyFactory, etc. > * Tests functional regression. > * Including Key Stores (P11KeyStore) > > Parameters to run the reproducer (on JDK-10): > * javac: --add-modules jdk.crypto.cryptoki --add-exports > java.base/sun.security.internal.spec=ALL-UNNAMED > * java: -XX:+UseParallelGC -Xmx3500m --add-modules jdk.crypto.cryptoki > --add-opens java.base/javax.crypto=ALL-UNNAMED --add-opens > jdk.crypto.cryptoki/sun.security.pkcs11=ALL-UNNAMED > > You can also use jtreg. > > > PERFORMANCE > ........................................................ > > For a quick reproducer previously developed (which looped 100000 times > creating P11Cipher and P11Key objects to encrypt a plaintext), these are > the figures I got: > > * real 1m11.328s (without fix) > * real 1m12.795s (with fix) > > Performance penalty seems to be low in current state. > > OTHER > ....................................................... > > My employer has an OCA agreement with Oracle and this work has been done > in that context. > > Look forward to your comments. > > Kind regards, > Martin.- > > -- > [1] - https://bugs.openjdk.java.net/browse/JDK-6913047 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mullan at oracle.com Thu Oct 12 20:32:46 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 12 Oct 2017 16:32:46 -0400 Subject: Manifest Related Bugs JDK-6372077, JDK-6202130, JDK-8176843, JDK-4842483, JDK-6443578, JDK-6910466, JDK-4935610, and JDK-4271239 In-Reply-To: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> References: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> Message-ID: <02abcb68-2294-c733-5048-b1f81ee99435@oracle.com> Hi Phillip, All of these bugs are in the core-libs area, so I am copying the core-libs-dev list since that is where they should be discussed and reviewed. I have -bcc-ed security-dev (where this was originally posted). --Sean On 10/2/17 1:24 PM, Philipp Kunz wrote: > Hi, > > While fixing JDK-6695402 I came across other related bugs to manifests > such as JDK-6372077, JDK-6202130, JDK-8176843, JDK-4842483, and > JDK-6443578 which all relate to manifest reading and writing. Somewhere > bug 7071055 is mentioned but I cannot find anywhere. Another group of > bugs, JDK-6910466, JDK-4935610, and JDK-4271239 concern the mandatory > manifest main attributes Manifest-Version or Signature-Version and at > first glance are duplicates. If you know of more known bugs, not > necessarily present in jira, I'd be glad to get notified. > > There are also some comments about utf handling and line breaking in the > code of Manifest: > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l299 > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l327 > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l370 > > Furthermore, Attributes.map should declare appropriate type parameters: > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l61 > The specification would also require that `header names must not start > with _, - or "From"` > (http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html#Section-Specification) > but I would opt not to add this as a hard restriction because I guess it > can be assumed that such names are in use now after having been working > for years. A warning to a logger might be conceivable such as in > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l424 > Attribute values are never checked at all and invalid characters such as > line breaks are never detected except that when reading the manifest > again the values are cut off. > The tab character (U+0008) does not work in manifest values. > I suspect that there are also issues regarding the iteration order but I > haven't got a prove yet unlike for the other points above: > http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Manifest.java#l54 > There is duplicated or very similar code in Attributes and Manifest: > Attributes.write-Manifest.write-Attributes.writeMain and > Attributes.read-Manifest.read. > Resolving JDK-6202130 would have the additional benefit to be able to > view manifests with any utf-8 capable editor even if multi-byte > characters break across lines. > > Fixing these issues individually looks like more complicated work than > fixing them all at once, I guess, also because of a very low current > test coverage. So I'd suggest to add some thorough tests along with > fixing these issues. But before I start I would like to get some > guidance, assistance or at least confirmation on how to proceed. I'm new > to open jdk and have submitted only one patch so far. > > Is it ok to add tests for things that have worked before? > Is it ok to refactor duplicated code just for the added value to reduce > effort for testing? > I assume compatibility to and from existing manifests is the highest > priority, correct? This would also be the hard part in adding as > complete test coverage as possible. What would be acceptable criteria to > meet? > Why does Attributes not extend LinkedHashMap and why does Manifest not > extend HashMap? Any objection? > While I would not want to write code that looks slow or change more than > necessary one can never know before having performance actually > measured. Is there some way this is dealt with or should I wait for such > feedback until after patch submission? > > Would someone sponsor? > > Regards, > Philipp > > > > > ------------------------------------------------------------------------ > > > > Paratix GmbH > St Peterhofstatt 11 > 8001 Z?rich > > +41 (0)76 397 79 35 > philipp.kunz at paratix.ch From tobias.wagner at n-design.de Tue Oct 17 08:55:03 2017 From: tobias.wagner at n-design.de (=?utf-8?Q?Tobias_Wagner?=) Date: Tue, 17 Oct 2017 10:55:03 +0200 Subject: Arithmetic error in SunEC Message-ID: Hi, we found an error in the GF(p)-arithmetics of SunEC, while adding support for brainpool-curves in ECDHE for TLS connections as suggested in RFC 7027. BrainpoolP256r1 and brainpoolP512r1 worked out of the box, but brainpoolP384r1 did not. The calculated public keys were not on that curve and thus we got handshake failures. After debugging the key generation during an TLS handshake, we came to the conclusion, that there must be a flaw in ec_GFp_sub_6 from ecl_gf.c. Using ec_GFp_sub instead worked with brainpoolP384r1. Researching further on that issue, we learned that SunEC originates from Mozilla's NSS library and that the same error was reported to Mozilla three years ago (https://bugzilla.mozilla.org/show_bug.cgi?id=1128140): > The functions ec_GFp_sub_5 and ec_GFp_sub_6 are missing an additional: > > MP_ADD_CARRY(b4, r4, r4, borrow, borrow) > > and > > MP_ADD_CARRY(b5, r5, r5, borrow, borrow) > > in the /* Do quick 'add' if we've gone under 0 > * (subtract the 2's complement of the curve field) * > check. It has eventually been fixed last year (https://hg.mozilla.org/projects/nss/rev/d81d6127781e). The attached patch would fix the issue on JDK9 in the same manner as it was done in Mozilla's NSS. The issue is also present in JDK8 and probably in earlier JDKs as well. Regards Tobias -- phone: +49 221 222896 17 fax: +49 221 222896 11 keybase: https://keybase.io/toebix n - d e s i g n?? G m b H www.n-design.de Alpenerstr. 16 D-50825 K?ln Amtsgericht K?ln HRB 33766 B Gesch?ftsf?hrer Andy Kohl -------------- next part -------------- A non-text attachment was scrubbed... Name: jdk9_jdk_17287.patch Type: application/octet-stream Size: 1217 bytes Desc: not available URL: From mbalao at redhat.com Tue Oct 17 18:45:47 2017 From: mbalao at redhat.com (Martin Balao) Date: Tue, 17 Oct 2017 15:45:47 -0300 Subject: Code Review Request: JDK-8148421 (Extended Master Secret TLS extension) In-Reply-To: <809c45dd-463d-479b-8194-b78440a22b3a@oracle.com> References: <809c45dd-463d-479b-8194-b78440a22b3a@oracle.com> Message-ID: Hi Xuelei, Webrev 03 -------------------- * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-8148421/webrev.03/ (browse online) * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-8148421/webrev.03/8148421.webrev.03.zip (download) Differences with previous webrev 02: * jsse.useExtendedMasterSecret system property was added to completely disable the feature on both client and server sides. * Bug fix: if the server does not support TLS 1.0+ version (thus, a previous TLS version was negotiated), the Extended Master Secret extension is not used even if the client sends the extension in its ClientHello message. * Bug fix: When TLS 1.0 or 1.1 are used, the session hash is calculated concatenating a MD5 with a SHA1 hash. * 2 regression testcases were fixed. * knownExtensions array (in ExtensionType class) has now an initial length of 16. Feature documentation (for JSSE Reference Guides [1]) ---------------------------------------------------- ## Extended Master Secret Extension The Extended Master Secret Extension is a feature that replaces the algorithm used to derive the *master secret* for a TLS session. The new algorithm provides a security enhancement to mitigate attacks such as the [Triple Handshake Attack](https://www.mitls.org/pages/attacks/3SHAKE). The Extension is defined by [RFC 7627](https://tools.ietf.org/html/rfc7627) (*Transport Layer Security (TLS) Session Hash and Extended Master Secret Extension*) and applies to TLS 1.0+. Clients and servers need to agree on the usage of this extension during the TLS full handshake, or the previous algorithm is used as a fallback. JSSE supports and enables the Extension by default on both client and server sides. However, for compatibility reasons, disabling is possible by setting *jsse.useExtendedMasterSecret* system property to *false* (i.e. through the *-Djsse.useExtendedMasterSecret="false"* command-line argument). ### How is the new algorithm different than the previous? The original algorithm uses a PRF function to derive the *master secret* from the following inputs: *pre-master secret* (result of a previous key exchange); "master secret" string; and, client and server random values. The new algorithm replaces the client and server random values with a hash of the previously exchanged handshake messages. As a result, the session *hash* contains information from certificates, key exchange parameters and other handshake-specific values; in addition to the client and server random numbers. Through binding *master secret* to the connection, an active man-in-the-middle attacker cannot force the generation of an identical value in a parallel connection. Thus, values that depend on the *master secret* for authentication (such as the TLS "tls-unique" binding value) can be trusted. ### Sessions resumption The *master secret* for a TLS session is established during a full handshake. When a session is resumed, the original *master secret* is used without any further negotiations. However, for checking purposes, Extended Master Secret Extension messages are exchanged during the abreviated handshake to indicate that the Extension was used when the original *master secret* was derived. JSSE raises an error if an incongruence is found here. Release note: Extended Master Secret Extension support was added to JSSE for both client and server sides --------------------------------------------------------------------------------------------------------- The Extended Master Secret Extension for TLS 1.0+ is now supported on JSSE for both client and server sides. By modifying the algorithm to derive the session *master secret* (during a full handshake) and binding it to connection-specific values, attacks such as the [Triple Handshake Attack]( https://www.mitls.org/pages/attacks/3SHAKE) are mitigated. The extension, enabled by default, can be turned off by setting *jsse.useExtendedMasterSecret* system property to *false*. See further information about the Extension in [RFC 7627]( https://tools.ietf.org/html/rfc7627) (*Transport Layer Security (TLS) Session Hash and Extended Master Secret Extension*). Testing ----------- I've run jtreg regression tests for the following categories: * javax/net/ssl * sun/security/ssl Results: * Found 2 bugs running regression tests (both fixed in webrev 03). * Found 2 tests broken because of this patch (both fixed in webrev 03). * The number of tests that pass is equal with the patch than without, so I assume that no regression was introduced given the current coverage. Kind regards, Martin.- -- [1] - http://docs.oracle.com/javase/9/security/java-secure-socket-extension-jsse-reference-guide.htm#JSSEC-GUID-93DEEE16-0B70-40E5-BBE7-55C3FD432345 On Sat, Aug 26, 2017 at 7:49 PM, Xuelei Fan wrote: > Hi Martin, > > Sorry for the delay. > > I like this no-API-change design. > > There may be some interoperbility/compatibility issues because of > implementation issues of the Extended Master Secret Extension. Maybe, we > want an approach to turn off the extension if there is a concern. It could > be a system property (for example, jsse.useExtendedMasterSecret="false"). > > Would you mind file a Compatibility & Specification Review (CSR) request > for this feature proposal? For more information, see the CSR wiki at > OpenJDK: > https://wiki.openjdk.java.net/display/csr/Main > > I may have some comments about the implementation if the CSR request get > approved. > > Thanks & Regards, > Xuelei > > On 8/4/2017 6:18 AM, Martin Balao wrote: > >> Hi, >> >> This is my proposal for JDK-8148421 (Support Transport Layer Security >> (TLS) Session Hash and Extended Master Secret Extension) [1]: >> >> * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-81 >> 48421/webrev.01/ > gehwolf/webrevs/mbalaoal/JDK-8148421/webrev.01/>(browse online) >> * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-81 >> 48421/webrev.01/8148421.webrev.01.zip (download) >> >> Notes: >> >> * There is no PKCS#11 support for Extended Master Secret key derivation >> at this moment. NSS supports it through a vendor-specific type definition >> (CKM_NSS_TLS_EXTENDED_MASTER_KEY_DERIVE and >> CKM_NSS_TLS_EXTENDED_MASTER_KEY_DERIVE_DH in pkcs11n.h file). Thus, >> P11TlsMasterSecretGenerator uses the legacy Master Key Derivation method >> only. >> >> Thanks in advanced, >> Martin.- >> >> -- >> [1] - https://bugs.openjdk.java.net/browse/JDK-8148421 >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.petcher at oracle.com Tue Oct 17 20:24:58 2017 From: adam.petcher at oracle.com (Adam Petcher) Date: Tue, 17 Oct 2017 16:24:58 -0400 Subject: Arithmetic error in SunEC In-Reply-To: References: Message-ID: On 10/17/2017 4:55 AM, Tobias Wagner wrote: > Hi, > > we found an error in the GF(p)-arithmetics of SunEC, while adding > support for brainpool-curves in ECDHE for TLS connections as > suggested in RFC 7027. Thanks! I created JDK-8189594[1] to track this issue. My understanding is that this error doesn't cause any bugs in the existing JDK code, but it may cause bugs if we add new curves that use this optimization. If I am wrong about this, please let me know. [1] https://bugs.openjdk.java.net/browse/JDK-8189594 > > Regards > Tobias > From sha.jiang at oracle.com Wed Oct 18 02:34:23 2017 From: sha.jiang at oracle.com (sha.jiang at oracle.com) Date: Wed, 18 Oct 2017 10:34:23 +0800 Subject: RFR[10] JDK-8189603: ProblemList sun/security/pkcs11/KeyStore/SecretKeysBasic.sh on Linux platform Message-ID: <136b7049-690a-4d66-4ae8-efb4a079d013@oracle.com> Hi, Test sun/security/pkcs11/KeyStore/SecretKeysBasic.sh failed frequently on Linux due to JDK-8186098. It has to ProblemList this test until JDK-8186098 is fixed. diff -r 74700c8e39e9 test/jdk/ProblemList.txt --- a/test/jdk/ProblemList.txt??? Tue Oct 17 10:32:01 2017 -0700 +++ b/test/jdk/ProblemList.txt??? Wed Oct 18 10:30:14 2017 +0800 @@ -207,6 +207,7 @@ ?# jdk_security ?sun/security/pkcs11/ec/TestKeyFactory.java 8026976 generic-all +sun/security/pkcs11/KeyStore/SecretKeysBasic.sh 8186098 linux-all ?sun/security/tools/keytool/ListKeychainStore.sh 8156889 macosx-all Best regards, John Jiang From xuelei.fan at oracle.com Wed Oct 18 09:03:03 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 18 Oct 2017 02:03:03 -0700 Subject: RFR[10] JDK-8189603: ProblemList sun/security/pkcs11/KeyStore/SecretKeysBasic.sh on Linux platform In-Reply-To: <136b7049-690a-4d66-4ae8-efb4a079d013@oracle.com> References: <136b7049-690a-4d66-4ae8-efb4a079d013@oracle.com> Message-ID: <1d0aba01-5ff3-6630-68db-4f63f72cc924@oracle.com> Looks fine to me. Xuelei On 10/17/2017 7:34 PM, sha.jiang at oracle.com wrote: > Hi, > Test sun/security/pkcs11/KeyStore/SecretKeysBasic.sh failed frequently > on Linux due to JDK-8186098. > It has to ProblemList this test until JDK-8186098 is fixed. > > diff -r 74700c8e39e9 test/jdk/ProblemList.txt > --- a/test/jdk/ProblemList.txt??? Tue Oct 17 10:32:01 2017 -0700 > +++ b/test/jdk/ProblemList.txt??? Wed Oct 18 10:30:14 2017 +0800 > @@ -207,6 +207,7 @@ > ?# jdk_security > > ?sun/security/pkcs11/ec/TestKeyFactory.java 8026976 generic-all > +sun/security/pkcs11/KeyStore/SecretKeysBasic.sh 8186098 linux-all > > ?sun/security/tools/keytool/ListKeychainStore.sh 8156889 macosx-all > > Best regards, > John Jiang > From tobias.wagner at n-design.de Wed Oct 18 09:58:15 2017 From: tobias.wagner at n-design.de (=?utf-8?Q?Tobias_Wagner?=) Date: Wed, 18 Oct 2017 11:58:15 +0200 Subject: AW: Arithmetic error in SunEC Message-ID: Hi, yes, from what we know your understanding is correct. The NIST curve secp384r1 is using these functions but seems not to be affected because of its prime. Any other curve will probably affected. Regards Tobias -----Urspr?ngliche Nachricht----- > Von:Adam Petcher > Gesendet: Die 17 Oktober 2017 22:26 > An: security-dev at openjdk.java.net > Betreff: Re: Arithmetic error in SunEC > > On 10/17/2017 4:55 AM, Tobias Wagner wrote: > > > Hi, > > > > we found an error in the GF(p)-arithmetics of SunEC, while adding > > support for brainpool-curves in ECDHE for TLS connections as > > suggested in RFC 7027. > > > Thanks! I created JDK-8189594[1] to track this issue. My understanding > is that this error doesn't cause any bugs in the existing JDK code, but > it may cause bugs if we add new curves that use this optimization. If I > am wrong about this, please let me know. > > [1] https://bugs.openjdk.java.net/browse/JDK-8189594 > > > > > Regards > > Tobias > > > > From xuelei.fan at oracle.com Wed Oct 18 12:54:09 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 18 Oct 2017 05:54:09 -0700 Subject: Code Review Request: JDK-8148421 (Extended Master Secret TLS extension) In-Reply-To: References: <809c45dd-463d-479b-8194-b78440a22b3a@oracle.com> Message-ID: <6fbd6a1c-ea69-cca3-27f3-0f4a41bee85b@oracle.com> On 10/17/2017 11:45 AM, Martin Balao wrote: > > http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-8148421/webrev.03/ TlsMasterSecretParameterSpec.java --------------------------------- This spec update impacts the PKCS11 implementation too. Please update jdk/src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs11/P11TlsMasterSecretGenerator.java as well. ClientHandshaker.java: ---------------------- Per RFC 7627: 1. For full handshaking, a client MUST send the "extended_master_secret" extension. 2. A client SHOULD NOT offer an abbreviated handshake to resume a session that that does not use an extended master secret. Instead, it SHOULD offer a full handshake. 3. When offering an abbreviated handshake, the client MUST send the "extended_master_secret" extension in its ClientHello. 4. For abbreviated handshake, if the original session did not use the "extended_master_secret" extension but the new ServerHello contains the extension, the client MUST abort the handshake. 5. For abbreviated handshake, if the original session used the extension but the new ServerHello does not contain the extension, the client MUST abort the handshake. If I'm reading correct, the update in ClientHandshaker.java implements #1, but missing #2, #4 and #5, and a conditional support of #3. For the missing of #4 and #5, it might be mainly caused by that the 800-828 lines are put in a place where full handshaking happens. Abbreviated handshake return at line 756 and cannot reach line 800. For #2, I may suggest combine the extension together with System property "jdk.tls.allowUnsafeServerCertChange". If not using extended master secret, and not allowUnsafeServerCertChange, and useExtendedMasterSecretExtension, do not offer abbreviated handshake. Using full handshake instead for TLS 1.0+. Besides, if using the extension, don't use the server certificate change checking any more. See allowUnsafeServerCertChange comments in ClientHandshaker.java. For #3, I may always send the "extended_master_secret" extension, the server side can handle it property, no matter the original session use the extension or not. SSLSessionImpl.java ------------------- 94 private boolean useExtendedMasterSecret; 200 void setUseExtendedMasterSecret() { 211 boolean getUseExtendedMasterSecret() { I may suggest use "final" useExtendedMasterSecret (set during construction), so that the set/get methods do not compete against each other. Using "final" may need to adjust some source code. Looks like it is doable. ServerHandshaker.java --------------------- For safer, as there is no compatibility impact as if the client request for the extension, I think we may want to always enable the extension in server side. It means the system property "jsse.useExtendedMasterSecret" disables the extension in client side only. And the property cannot be used to disable server acceptance of the extension. Per RFC 7627: A. For full handshaking, if a server implementing this document receives the "extended_master_secret" extension, it MUST include the extension in its ServerHello message. B. For abbreviated handshake request, If the original session did not use the "extended_master_secret" extension but the new ClientHello contains the extension, then the server MUST NOT perform the abbreviated handshake. Instead, it SHOULD continue with a full handshake. C. For abbreviated handshake request, if the original session used the "extended_master_secret" extension but the new ClientHello does not contain it, the server MUST abort the abbreviated handshake. D. For abbreviated handshake request, if neither the original session nor the new ClientHello uses the extension, the server SHOULD abort the handshake. E. For abbreviated handshake request, if the new ClientHello contains the extension and the server chooses to continue the handshake, then the server MUST include the "extended_master_secret" extension in its ServerHello message. If I'm reading correct, the update in ServerHandshaker.java implements #A, #C and #E, but missing #B and #C. For #B, I think it should be fine to follow the spec: continue with a full handshake. For #C, I was wondering we may need a new system property (jsse.allowLegacyResumption?) to turn on/off this behavior. If application want a strict mode, the server abort the abbreviated handshake for case #C. Otherwise, the server can continue with an abbreviated handshake in order to support legacy resumption. Hope it helps! Regards, Xuelei From mstjohns at comcast.net Wed Oct 18 17:01:30 2017 From: mstjohns at comcast.net (Michael StJohns) Date: Wed, 18 Oct 2017 13:01:30 -0400 Subject: AW: Arithmetic error in SunEC In-Reply-To: References: Message-ID: <003c1a2e-1dde-5533-3b0d-99f72de56abf@comcast.net> On 10/18/2017 5:58 AM, Tobias Wagner wrote: > Hi, > > yes, from what we know your understanding is correct. The NIST curve secp384r1 is using these functions but seems not to be affected because of its prime. Any other curve will probably affected. > > Regards > Tobias > Stupid question. Given that there this came over from NSS and that NSS had reported the error, has anyone checked the rest of NSS reported errors/fixes for EC for porting to the JDK? Mike > -----Urspr?ngliche Nachricht----- >> Von:Adam Petcher >> Gesendet: Die 17 Oktober 2017 22:26 >> An: security-dev at openjdk.java.net >> Betreff: Re: Arithmetic error in SunEC >> >> On 10/17/2017 4:55 AM, Tobias Wagner wrote: >> >>> Hi, >>> >>> we found an error in the GF(p)-arithmetics of SunEC, while adding >>> support for brainpool-curves in ECDHE for TLS connections as >>> suggested in RFC 7027. >> >> >> Thanks! I created JDK-8189594[1] to track this issue. My understanding >> is that this error doesn't cause any bugs in the existing JDK code, but >> it may cause bugs if we add new curves that use this optimization. If I >> am wrong about this, please let me know. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8189594 >> >>> Regards >>> Tobias >>> >> From weijun.wang at oracle.com Thu Oct 19 06:45:18 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Thu, 19 Oct 2017 14:45:18 +0800 Subject: RFR 8189657: LineBrokenMultiByteCharacter.java fails on some systems that does not accept a non-ASCII char in Path Message-ID: <2F4B6425-530C-48FF-9B77-C950C16EE699@oracle.com> Please review this patch: $ hg qdiff -wb diff --git a/test/lib/jdk/test/lib/util/JarUtils.java b/test/lib/jdk/test/lib/util/JarUtils.java --- a/test/lib/jdk/test/lib/util/JarUtils.java +++ b/test/lib/jdk/test/lib/util/JarUtils.java @@ -28,6 +28,7 @@ import java.io.FileOutputStream; import java.io.IOException; import java.nio.file.Files; +import java.nio.file.InvalidPathException; import java.nio.file.Path; import java.nio.file.Paths; import java.util.Enumeration; @@ -88,12 +89,17 @@ if (file.equals("-")) { update = false; } else if (update) { + try { Path p = Paths.get(file); if (Files.exists(p)) { changes.put(file, p); } else { changes.put(file, file); } + } catch (InvalidPathException e) { + // Fallback if file not a valid Path. + changes.put(file, file); + } } else { changes.put(file, Boolean.FALSE); } This is a part of the JarUtils::updateJar method. When it adds a new entry into a jar, it will add the actual file if a file with the entry name exists, and fabricate some content if not. Unfortunately just creating a Path object on some systems would fail. Noreg-self. Thanks Max From weijun.wang at oracle.com Thu Oct 19 07:11:14 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Thu, 19 Oct 2017 15:11:14 +0800 Subject: RFR 8180289: jarsigner treats timestamped signed jar invalid after the signer cert expires Message-ID: <3D43EBA3-EE5B-4FB3-83A6-6AC7790E56C5@oracle.com> Please review the fix at http://cr.openjdk.java.net/~weijun/8180289/webrev.00/ The code change contains: - Fix the bug by passing the timestamp to Validator::validate. - CertPath validation on timestamp signer cert and related warning messages - Output change: "chain not validated" -> "invalid chain". Otherwise it looks jarsigner has not validated them. Thanks Max From weijun.wang at oracle.com Fri Oct 20 03:56:09 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 20 Oct 2017 11:56:09 +0800 Subject: RFR 8159535: Mark deprecated javax.security.auth.Policy API with forRemoval=true Message-ID: <5AF8833D-352A-40A3-9EEA-02E79BBC8929@oracle.com> Please review this patch. A CSR [1] is also filed. diff --git a/src/java.base/share/classes/javax/security/auth/Policy.java b/src/java.base/share/classes/javax/security/auth/Policy.java --- a/src/java.base/share/classes/javax/security/auth/Policy.java +++ b/src/java.base/share/classes/javax/security/auth/Policy.java @@ -152,11 +152,12 @@ * * These two APIs provide callers the means to query the * Policy for Principal-based Permission entries. + * This class is subject to removal in a future version of Java SE. * * @since 1.4 * @see java.security.Security security properties */ - at Deprecated(since="1.4") + at Deprecated(since="1.4", forRemoval=true) public abstract class Policy { private static Policy policy; Thanks Max [1] https://bugs.openjdk.java.net/browse/JDK-8189621 From weijun.wang at oracle.com Fri Oct 20 05:44:30 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 20 Oct 2017 13:44:30 +0800 Subject: RFR 8189657: LineBrokenMultiByteCharacter.java fails on some systems that does not accept a non-ASCII char in Path In-Reply-To: <2F4B6425-530C-48FF-9B77-C950C16EE699@oracle.com> References: <2F4B6425-530C-48FF-9B77-C950C16EE699@oracle.com> Message-ID: <0445D039-E0F5-4C69-AD61-9141905F480D@oracle.com> False alarm! Turns out I introduced the bug in another change I am still working on. I'll close this bug. --Max > On Oct 19, 2017, at 2:45 PM, Weijun Wang wrote: > > Please review this patch: > > $ hg qdiff -wb > diff --git a/test/lib/jdk/test/lib/util/JarUtils.java b/test/lib/jdk/test/lib/util/JarUtils.java > --- a/test/lib/jdk/test/lib/util/JarUtils.java > +++ b/test/lib/jdk/test/lib/util/JarUtils.java > @@ -28,6 +28,7 @@ > import java.io.FileOutputStream; > import java.io.IOException; > import java.nio.file.Files; > +import java.nio.file.InvalidPathException; > import java.nio.file.Path; > import java.nio.file.Paths; > import java.util.Enumeration; > @@ -88,12 +89,17 @@ > if (file.equals("-")) { > update = false; > } else if (update) { > + try { > Path p = Paths.get(file); > if (Files.exists(p)) { > changes.put(file, p); > } else { > changes.put(file, file); > } > + } catch (InvalidPathException e) { > + // Fallback if file not a valid Path. > + changes.put(file, file); > + } > } else { > changes.put(file, Boolean.FALSE); > } > > This is a part of the JarUtils::updateJar method. When it adds a new entry into a jar, it will add the actual file if a file with the entry name exists, and fabricate some content if not. Unfortunately just creating a Path object on some systems would fail. > > Noreg-self. > > Thanks > Max > From sean.mullan at oracle.com Fri Oct 20 14:29:18 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 20 Oct 2017 10:29:18 -0400 Subject: RFR 8159535: Mark deprecated javax.security.auth.Policy API with forRemoval=true In-Reply-To: <5AF8833D-352A-40A3-9EEA-02E79BBC8929@oracle.com> References: <5AF8833D-352A-40A3-9EEA-02E79BBC8929@oracle.com> Message-ID: <2ca9e219-b925-aae4-0c27-80ddd53557a5@oracle.com> Looks good to me. Please add a release note subtask to the issue. --Sean On 10/19/17 11:56 PM, Weijun Wang wrote: > Please review this patch. A CSR [1] is also filed. > > diff --git a/src/java.base/share/classes/javax/security/auth/Policy.java b/src/java.base/share/classes/javax/security/auth/Policy.java > --- a/src/java.base/share/classes/javax/security/auth/Policy.java > +++ b/src/java.base/share/classes/javax/security/auth/Policy.java > @@ -152,11 +152,12 @@ > * > * These two APIs provide callers the means to query the > * Policy for Principal-based Permission entries. > + * This class is subject to removal in a future version of Java SE. > * > * @since 1.4 > * @see java.security.Security security properties > */ > - at Deprecated(since="1.4") > + at Deprecated(since="1.4", forRemoval=true) > public abstract class Policy { > > private static Policy policy; > > Thanks > Max > > [1] https://bugs.openjdk.java.net/browse/JDK-8189621 > From weijun.wang at oracle.com Fri Oct 20 16:02:06 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Sat, 21 Oct 2017 00:02:06 +0800 Subject: RFR 8159535: Mark deprecated javax.security.auth.Policy API with forRemoval=true In-Reply-To: <2ca9e219-b925-aae4-0c27-80ddd53557a5@oracle.com> References: <5AF8833D-352A-40A3-9EEA-02E79BBC8929@oracle.com> <2ca9e219-b925-aae4-0c27-80ddd53557a5@oracle.com> Message-ID: https://bugs.openjdk.java.net/browse/JDK-8189746 created. --Max > On Oct 20, 2017, at 10:29 PM, Sean Mullan wrote: > > Looks good to me. Please add a release note subtask to the issue. > > --Sean > > On 10/19/17 11:56 PM, Weijun Wang wrote: >> Please review this patch. A CSR [1] is also filed. >> diff --git a/src/java.base/share/classes/javax/security/auth/Policy.java b/src/java.base/share/classes/javax/security/auth/Policy.java >> --- a/src/java.base/share/classes/javax/security/auth/Policy.java >> +++ b/src/java.base/share/classes/javax/security/auth/Policy.java >> @@ -152,11 +152,12 @@ >> * >> * These two APIs provide callers the means to query the >> * Policy for Principal-based Permission entries. >> + * This class is subject to removal in a future version of Java SE. >> * >> * @since 1.4 >> * @see java.security.Security security properties >> */ >> - at Deprecated(since="1.4") >> + at Deprecated(since="1.4", forRemoval=true) >> public abstract class Policy { >> private static Policy policy; >> Thanks >> Max >> [1] https://bugs.openjdk.java.net/browse/JDK-8189621 From mbalao at redhat.com Mon Oct 23 13:45:08 2017 From: mbalao at redhat.com (Martin Balao) Date: Mon, 23 Oct 2017 10:45:08 -0300 Subject: Code Review Request: JDK-6491070 (Support for RFC 5929-Channel Bindings) In-Reply-To: <1c51ce46-d0e1-1ab8-6e9e-a71e649ff323@oracle.com> References: <97c02f80-88aa-d0a7-f1e0-ba80b3a3b71d@oracle.com> <1c51ce46-d0e1-1ab8-6e9e-a71e649ff323@oracle.com> Message-ID: Hi Xuelei, I would like to propose Webrev 03: * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.03/ (browse online) * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.03/6491070.webrev.03.zip (download) What's new in Webrev 03: * New interface to expose Finished messages verify data handshake material (from both client and server) through a HandshakeListener. It's now up to the API client to figure out which is the tls-unique binding value from that data. TlsChannelBindingTest.java is an example of how to use the proposed API. * The new API allows disabling renegotiations (TLS handshake protocol). * Added HandshakeCompletedListener to SSLEngine as in SSLSocket. It's necessary to be able to get the server certificate used on a handshake for the certificate binding value. * HandshakeListener and HandshakeCompletedListener could be merged into HandshakeCompletedListener, but I think the name "Completed" does not fit -particularly for the callback that gets rid of renegotiations-. Obviously renaming HandshakeCompletedListener to HandshakeListener would be a major and breaking API change. * TlsChannelBindingTest.java updated to the new API. I've done testing running the following test categories and didn't notice any regression: * javax/net/ssl * sun/security/ssl Look forward to your comments. Kind regards, Martin.- On Thu, Aug 31, 2017 at 10:32 PM, Xuelei Fan wrote: > The failure-and-retry mechanism could a nightmare for some applications. > Please think more how could we avoid it. If need more APIs, what the > update may looks like and how complicated it could be? > > If required, Bernd's proposal can be extended a little bit to support > operations other than listening. > > APIs maintain is very complicated, a good design may need more time > currently, but could save much more in the future. > > Thanks, > Xuelei > > On 8/31/2017 11:41 AM, Martin Balao wrote: > >> Hi, >> >> The material is already cached in the handshaker if secure renegotiation >> is enabled. However, I agree with you that we are going to cache the value >> even when secure renegotiation is not enabled, thus, wasting roughly 12 >> bytes (as bytes for an empty array are already consumed). In fact, the >> exact case -adding a few conditionals to webrev.02- is the one in which >> secure renegotiation is disabled and extended master secret is enabled. >> GnuTls and OpenSSL -including its derivatives like Boring SSL and Python >> (through OpenSSL)- always cache this information. >> >> As for the empty / null cases, the API consumer was expected to ask for >> the binding information after the TLS connection is established. It's on >> the API consumer knowledge that asking for the information before (i.e.: >> just after creating a disconnected socket) or while the handshake is taking >> place, makes no sense and no valid value will be obtained (either we define >> this as null or empty). For those providers that do not support this >> feature, the information wouldn't have been available after the handshake. >> However, I agree with you that before the handshake is completed there is >> no means of knowing if the provider does support this API. My first webrev >> (webrev.01) was throwing an UnsupportedOperationException to make this case >> explicit but I had doubts regarding the real value it provides for the API >> consumer. The proposed API was similar to Python, SSLBoring and GnuTLs. >> However, handshake listener callbacks -as Bernd suggests- and the idea of >> just exposing the handshake material (as a lower level API) sounds good to >> me. I can give it a try. I propose then to bring the handshake information >> as part of a HandshakeCompletedEvent instance, even though the callback >> registrant may not consume it. Would that work for you? >> >> In regard to the handshake material update -which I assumed unlikely-, >> the point in which a renegotiation may take place (from the server side) is >> when reading data, not when writing. That cannot be controlled by the >> application because it's JSSE internal and not exposed. Thus, an >> application may read from the socket, get the handshake material and write >> a message using the binding value -which we can make sure that is the valid >> one at that point-. However, as soon as the application reads again from >> the socket, a renegotiation -if requested by the client- may be processed >> and the binding value gets updated. The higher level protocol may fail >> -because the binding value was already sent but not processed on the other >> side- and a re-try needed. This looks independent of whether we use the >> originally proposed API or the handshake listener callback interface (or >> even a sync callback), because the underlying problem is that the >> application cannot really control the renegotiation flow in the lower layer >> (as RFC 5929 suggest). The options I see are adding more complexity to the >> API and let the application control the renegotiation flow or live with >> that and expect the application to retry. >> >> Thanks, >> Martin.- >> >> On Tue, Aug 29, 2017 at 4:34 PM, Xuelei Fan > > wrote: >> >> On 8/26/2017 2:56 PM, Bernd Eckenfels wrote: >> > How about only passing it to an extended handshake listener. The >> > material does not have to be cached (the app can do it if needed) >> and >> > renegotiation works the same way. This can also be helpful for >> things >> > like logging the master secret (for wireshark decryption). It can >> even >> > handle auditing of session resumptions >> Martin, what do you think about Bernd's proposal above and similar >> callback mechanism? >> >> More comment inlines. >> >> On 8/29/2017 11:50 AM, Martin Balao wrote: >> >> Hi Xuelei, >> >> >There are a few protocols that can benefits from exporting >> SSL/TLS handshake materials, including RFC 5929, RFC 5056, token >> binding and TLS 1.3 itself. Can we define a general API so as >> to exposing the handshake materials, so as to mitigate the >> inflating of JSSE APIs? I may suggest make further evaluation >> before move on to following design and code. >> >> Do you prefer an API like "public byte[] >> getTlsHandshakeMaterial(String materialType)" (in SSLSocket and >> SSLEngine) where "materialType" can eventually be >> "clientFinishedMessage"/"finishedMessage" or >> "serverFinishedMessage"/"peerFinishedMessage"? >> >> The problem of the APIs like that is, when applications call the >> method, it is not always return the expected result, and the >> implementation may have to cache the message even if an application >> never use it. See more in the following example. >> >> I cannot think of "serverCertificate" or "masterKey" as this is >> more related to a Session and not neccessarily to a handshake. >> getTlsHandshakeMaterial would be a lower level API and would >> move the burden of knowing which information is required for >> "tls-unique" TLS channel binding to the API consumer. Looks more >> like the OpenSSL approach (instead of the Python, SSLBoring or >> GnuTls approaches). However, OpenSSL have specific methods for >> each piece of information instead of a generic and parametrized >> one. I.e.: SSL_get_finished or SSL_get_peer_finished. What other >> information do you expect the Handshaker to provide? >> >> >The SunJSSE provider happens to cache the finished messages >> in its implementation so you can use it for tls-unique, but it >> may not be true for other provider or other channel bindings. >> Need to define a more reliable approach to get the handshake >> materials. >> >> I focused on SunJSSE provider. I'm not sure about how other >> providers may implement this API and where they can get the >> required information from, without knowing their internals. In >> regard to SunJSSE and "tls-unique" binding type, I leveraged on >> existing data. If data weren't already there, I would have to >> figure out how to get it from the handshake -doing the same that >> was already done would have been an option-. Do you prefer the >> Handshaker to provide a function to get different information >> and not just the finished hash? (as for the public >> SSLSocket/SSLEngine "getTlsHandshakeMaterial" API). Which other >> information may be useful to get from the Handshaker? What do >> you mean by reliable? (given that this is all SunJSSE internal >> and we have no external dependencies). >> >> Let consider the use of the API. >> byte[] getTlsChannelBinding("tls_unique"); >> >> I'm confusing when I try to use it by myself: >> 1. provider does not implement this method >> return null or empty? >> >> It happens because an old provider should still work in new JDK, but >> old provider does not implement new APIs, or a new provider does not >> support this feature. >> >> 2. the method is called before handshaking >> return null or empty? >> >> 3. the method is called during handshaking >> return null, empty or the channel binding value? >> >> 4. the method is called at the same time the handshaking completed? >> return the channel binding value? >> >> 5. the method is called after the handshaking >> return the channel binding value? >> >> 6. the method is called during renegoitation >> return null, empty, the old binding value, or the new binding >> value? >> >> 7. the method is called after handshaking >> return old binding value, or the new binding value? >> >> 8. the method is called after the initial handshaking, but the >> binding value is changed shortly after because of renegotiation. >> how could application use the binding value? >> >> We need a clear define of the behavior of the method. It could be >> complicated if the method is designed as >> getTlsChannelBinding("tls_unique"). >> >> I feel that handshake material should be captured when >> 1. it is requested to capture the handshake material, and >> 2. the handshake material get produced. >> >> For the getTlsChannelBinding("tls_unique") API, it is unknown: >> 1. Is it required to capture the handshake material? >> 2. Is the handshake material produced? >> >> The two points could result in a few unexpected problems, as the >> above 8 items that we may want to consider. >> >> In regard to other channel bindings, it'll depend on the binding >> type the way in which the information is obtained. I.e.: >> "tls-unique" SunJSSE implementation leverages on cached finished >> messages. However, "tls-server-end-point" leverages on stored >> certificates that are obtained from the Session (not from the >> handshaker). Is there any specific channel binding you are >> concerned with? >> >> >If the channel binding is not required, it may be not >> necessary to expose the handshake materials. Need to define a >> solution to indicate the need of the exporting. >> >> Do you mean a lower layer knowing if the upper layer is going to >> require that information and decide to provide it or not based >> on that knowledge? I think I didn't get your point here. >> >> I mean, if an application want to support channel binding, the >> provider can provider the channel binding service; If the an >> application does not want channel binding, the provider should be >> perform the channel binding service. The getTlsChannelBinding() >> make the provider MUST perform channel binding cache or calculation >> no matter application want it or not. >> >> >2. No way to know the update of the underlying handshake >> materials. >> >If renegotiation can takes place, need to define a interface >> to indicate that so that application can response accordingly. >> See section 3 and 7 of RFC 5929. >> >> I intentionally skipped this -at the cost of a spurious >> authentication- to avoid adding complexity to the API. An >> spurious authentication -which does not appear likely to me- can >> easily be retried by the application. The RFC 5929 suggests APIs >> through which the application can *control* the flow (i.e.: hold >> a renegotitation). This would expose JSSE internals. This is >> more than notifying. Notification, in my opinion, adds no value: >> what if the application already used the binding token before >> receiving the notification? The spurious authentication will >> happen anyways and has to be handled -i.e. retried-. It's just a >> timing issue. The real value is controlling the flow as the RFC >> suggests, but at the cost of exposing JSSE internals. >> >> My understanding, the block of the protocol is to make sure >> application is performing the channel binding with the right value, >> or updating the value accordingly if necessary. If you skip this >> and when renegotiation happen, the channel binding could be limited, >> or may not work as expected. >> >> Thanks, >> Xuelei >> >> Kind regards, >> Martin.- >> >> >> On Sat, Aug 26, 2017 at 5:25 PM, Xuelei Fan >> >> >> >> >> wrote: >> >> Hi Marin, >> >> Sorry for the delay. >> >> There are a few protocols that can benefits from exporting >> SSL/TLS >> handshake materials, including RFC 5929, RFC 5056, token >> binding and >> TLS 1.3 itself. Can we define a general API so as to >> exposing the >> handshake materials, so as to mitigate the inflating of >> JSSE APIs? I may suggest make further evaluation before move >> on to following >> design and code. >> >> > >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-64 >> 91070/webrev.02/ >> > 491070/webrev.02/> >> > gehwolf/webrevs/mbalaoal/JDK-6491070/webrev.02/ >> > 491070/webrev.02/>> >> I have two concerns about the design: >> >> 1. Channel binding may be not always required. >> SSLSocket/SSLEngine.getTlsChannelBinding(String >> bindingType); >> >> The SunJSSE provider happens to cache the finished messages >> in its >> implementation so you can use it for tls-unique, but it may >> not be >> true for other provider or other channel bindings. Need to >> define a >> more reliable approach to get the handshake materials. >> >> If the channel binding is not required, it may be not >> necessary to >> expose the handshake materials. Need to define a solution to >> indicate the need of the exporting. >> >> 2. No way to know the update of the underlying handshake >> materials. >> If renegotiation can takes place, need to define a interface >> to >> indicate that so that application can response >> accordingly. See >> section 3 and 7 of RFC 5929. >> >> Thanks, >> Xuelei >> >> On 7/31/2017 8:53 AM, Martin Balao wrote: >> >> Hi, >> >> Here it is an update for the proposed TLS Channel >> Bindings >> support in OpenJDK: >> >> * >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-64 >> 91070/webrev.02/ >> > 491070/webrev.02/> >> > gehwolf/webrevs/mbalaoal/JDK-6491070/webrev.02/ >> > 491070/webrev.02/>> >> (browse online) >> * >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-64 >> 91070/webrev.02/6491070.webrev.02.zip >> > 491070/webrev.02/6491070.webrev.02.zip> >> > gehwolf/webrevs/mbalaoal/JDK-6491070/webrev.02/6491070.webrev.02.zip >> > 491070/webrev.02/6491070.webrev.02.zip>> >> (download) >> >> Changes since v01: >> >> * getTlsChannelBinding API changed to return null by >> default >> (if not implemented), instead of throwing an >> UnsupportedOperationException. >> >> * "tls-server-end-point" TLS channel binding now >> supported. >> >> Kind regards, >> Martin.- >> >> On Wed, Jul 26, 2017 at 4:12 PM, Martin Balao >> >> > >> >> >> >>> >> >> wrote: >> >> Hi, >> >> Here it is my proposal for JDK-6491070 (Support >> for RFC >> 5929-Channel >> Bindings: e.g. public API to obtain TLS finished >> message) [1]: >> >> * >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-64 >> 91070/webrev.01/ >> > 491070/webrev.01/> >> > gehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/ >> > 491070/webrev.01/>> >> > gehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/ >> > 491070/webrev.01/> >> > gehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/ >> > 491070/webrev.01/>>> >> * >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-64 >> 91070/webrev.01/6491070.webrev.01.zip >> > 491070/webrev.01/6491070.webrev.01.zip> >> > gehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/6491070.webrev.01.zip >> > 491070/webrev.01/6491070.webrev.01.zip>> >> > gehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/6491070.webrev.01.zip >> > 491070/webrev.01/6491070.webrev.01.zip> >> > gehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/6491070.webrev.01.zip >> > 491070/webrev.01/6491070.webrev.01.zip>>> >> >> Notes: >> * Implementation based on Channel Bindings for >> TLS (RFC >> 5929) [2] >> >> * Only "tls-unique" currently supported >> >> Look forward to your comments. >> >> Kind regards, >> Martin.- >> >> -- >> [1] - >> https://bugs.openjdk.java.net/browse/JDK-6491070 >> >> > > >> > >> > >> >> [2] - https://tools.ietf.org/html/rfc5929 >> >> > > >> > >> > >> >> >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.mullan at oracle.com Thu Oct 26 19:58:47 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 26 Oct 2017 15:58:47 -0400 Subject: RFR 8180289: jarsigner treats timestamped signed jar invalid after the signer cert expires In-Reply-To: <3D43EBA3-EE5B-4FB3-83A6-6AC7790E56C5@oracle.com> References: <3D43EBA3-EE5B-4FB3-83A6-6AC7790E56C5@oracle.com> Message-ID: <037eeeb0-ec95-b4f9-d3d6-0bcaac6df903@oracle.com> - test/jdk/sun/security/tools/jarsigner/TimestampCheck.java 65 * @bug 6543842 6543440 6939248 8009636 8024302 8163304 8169911 8166222 8180289 should not include 8166222 346 // 8166222: unvalidated TSA cert chain 347 sign("tsnoca") 348 .shouldContain("TSA certificate chain is invalid") 349 .shouldHaveExitValue(64); wrong bugid? Looks fine otherwise. --Sean On 10/19/17 3:11 AM, Weijun Wang wrote: > Please review the fix at > > http://cr.openjdk.java.net/~weijun/8180289/webrev.00/ > > The code change contains: > > - Fix the bug by passing the timestamp to Validator::validate. > > - CertPath validation on timestamp signer cert and related warning messages > > - Output change: "chain not validated" -> "invalid chain". Otherwise it looks jarsigner has not validated them. > > Thanks > Max > From weijun.wang at oracle.com Thu Oct 26 23:43:36 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 27 Oct 2017 07:43:36 +0800 Subject: RFR 8180289: jarsigner treats timestamped signed jar invalid after the signer cert expires In-Reply-To: <037eeeb0-ec95-b4f9-d3d6-0bcaac6df903@oracle.com> References: <3D43EBA3-EE5B-4FB3-83A6-6AC7790E56C5@oracle.com> <037eeeb0-ec95-b4f9-d3d6-0bcaac6df903@oracle.com> Message-ID: <9E9D243E-7F66-4580-B862-384195E90B65@oracle.com> > On Oct 27, 2017, at 3:58 AM, Sean Mullan wrote: > > - test/jdk/sun/security/tools/jarsigner/TimestampCheck.java > > 65 * @bug 6543842 6543440 6939248 8009636 8024302 8163304 8169911 8166222 8180289 > > should not include 8166222 Yes. > > 346 // 8166222: unvalidated TSA cert chain > 347 sign("tsnoca") > 348 .shouldContain("TSA certificate chain is invalid") > 349 .shouldHaveExitValue(64); > > wrong bugid? This part is about the newly added timestamp cert validation. I'll use 8180289. Thanks Max > > Looks fine otherwise. > > --Sean > > On 10/19/17 3:11 AM, Weijun Wang wrote: >> Please review the fix at >> http://cr.openjdk.java.net/~weijun/8180289/webrev.00/ >> The code change contains: >> - Fix the bug by passing the timestamp to Validator::validate. >> - CertPath validation on timestamp signer cert and related warning messages >> - Output change: "chain not validated" -> "invalid chain". Otherwise it looks jarsigner has not validated them. >> Thanks >> Max From weijun.wang at oracle.com Fri Oct 27 06:54:42 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 27 Oct 2017 14:54:42 +0800 Subject: RFR 8159535: Mark deprecated javax.security.auth.Policy API with forRemoval=true In-Reply-To: <5AF8833D-352A-40A3-9EEA-02E79BBC8929@oracle.com> References: <5AF8833D-352A-40A3-9EEA-02E79BBC8929@oracle.com> Message-ID: <1EC0A337-0491-48E0-9A3B-1885C13D1205@oracle.com> Turns out there is a javac warning for "removal". Please review again at http://cr.openjdk.java.net/~weijun/8159535/webrev.00 Thanks Max > On Oct 20, 2017, at 11:56 AM, Weijun Wang wrote: > > Please review this patch. A CSR [1] is also filed. > > diff --git a/src/java.base/share/classes/javax/security/auth/Policy.java b/src/java.base/share/classes/javax/security/auth/Policy.java > --- a/src/java.base/share/classes/javax/security/auth/Policy.java > +++ b/src/java.base/share/classes/javax/security/auth/Policy.java > @@ -152,11 +152,12 @@ > * > * These two APIs provide callers the means to query the > * Policy for Principal-based Permission entries. > + * This class is subject to removal in a future version of Java SE. > * > * @since 1.4 > * @see java.security.Security security properties > */ > - at Deprecated(since="1.4") > + at Deprecated(since="1.4", forRemoval=true) > public abstract class Policy { > > private static Policy policy; > > Thanks > Max > > [1] https://bugs.openjdk.java.net/browse/JDK-8189621 From sean.mullan at oracle.com Fri Oct 27 12:40:24 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Fri, 27 Oct 2017 08:40:24 -0400 Subject: RFR 8159535: Mark deprecated javax.security.auth.Policy API with forRemoval=true In-Reply-To: <1EC0A337-0491-48E0-9A3B-1885C13D1205@oracle.com> References: <5AF8833D-352A-40A3-9EEA-02E79BBC8929@oracle.com> <1EC0A337-0491-48E0-9A3B-1885C13D1205@oracle.com> Message-ID: <28e84dec-3fec-0c86-9f40-21160f6f42f2@oracle.com> Looks good. --Sean On 10/27/17 2:54 AM, Weijun Wang wrote: > Turns out there is a javac warning for "removal". Please review again at > > http://cr.openjdk.java.net/~weijun/8159535/webrev.00 > > Thanks > Max > >> On Oct 20, 2017, at 11:56 AM, Weijun Wang wrote: >> >> Please review this patch. A CSR [1] is also filed. >> >> diff --git a/src/java.base/share/classes/javax/security/auth/Policy.java b/src/java.base/share/classes/javax/security/auth/Policy.java >> --- a/src/java.base/share/classes/javax/security/auth/Policy.java >> +++ b/src/java.base/share/classes/javax/security/auth/Policy.java >> @@ -152,11 +152,12 @@ >> * >> * These two APIs provide callers the means to query the >> * Policy for Principal-based Permission entries. >> + * This class is subject to removal in a future version of Java SE. >> * >> * @since 1.4 >> * @see java.security.Security security properties >> */ >> - at Deprecated(since="1.4") >> + at Deprecated(since="1.4", forRemoval=true) >> public abstract class Policy { >> >> private static Policy policy; >> >> Thanks >> Max >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8189621 > From sha.jiang at oracle.com Mon Oct 30 09:12:12 2017 From: sha.jiang at oracle.com (sha.jiang at oracle.com) Date: Mon, 30 Oct 2017 17:12:12 +0800 Subject: RFR[10] JDK-8190335: Backout changeset for JDK-8176354 due to JDK-8190333 Message-ID: Hi, It has to backout the changeset for JDK-8176354, because it changed a keystore and caused a lot of DTLS test failures. For more details, please see JDK-8190333. Issue: https://bugs.openjdk.java.net/browse/JDK-8190335 Webrev: http://cr.openjdk.java.net/~jjiang/8190335/webrev.00/ Best regards, John Jiang From artem.smotrakov at oracle.com Mon Oct 30 09:44:18 2017 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Mon, 30 Oct 2017 12:44:18 +0300 Subject: RFR[10] JDK-8190335: Backout changeset for JDK-8176354 due to JDK-8190333 In-Reply-To: References: Message-ID: Looks good to me. Please make sure that the tests pass after baking out 8176354. Artem On 10/30/2017 12:12 PM, sha.jiang at oracle.com wrote: > Hi, > It has to backout the changeset for JDK-8176354, because it changed a > keystore and caused a lot of DTLS test failures. For more details, > please see JDK-8190333. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8190335 > Webrev: http://cr.openjdk.java.net/~jjiang/8190335/webrev.00/ > > Best regards, > John Jiang > From mbalao at redhat.com Mon Oct 30 13:57:12 2017 From: mbalao at redhat.com (Martin Balao) Date: Mon, 30 Oct 2017 10:57:12 -0300 Subject: Code Review Request: JDK-8148421 (Extended Master Secret TLS extension) In-Reply-To: <6fbd6a1c-ea69-cca3-27f3-0f4a41bee85b@oracle.com> References: <809c45dd-463d-479b-8194-b78440a22b3a@oracle.com> <6fbd6a1c-ea69-cca3-27f3-0f4a41bee85b@oracle.com> Message-ID: Hi Xuelei, Webrev 04: * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-8148421/webrev.04/ (browse online) * http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-8148421/webrev.04/8148421.webrev.04.zip (download) TlsMasterSecretParameterSpec.java --------------------------------- As far as I know, a mechanism for Extended Master Secret has not been included in PKCS#11 standard (as of v.2.40, which is the current version [1]). That's why I didn't update P11TlsMasterSecretGenerator class. PKCS#11 standard has mechanisms and data structures for the legacy master key derivation. As an example, CK_TLS12_MASTER_KEY_DERIVE_PARAMS data type has a CK_SSL3_RANDOM_DATA member which can hold a "server random" and a "client random" value. That was not intended to hold a "session hash". In regard to implementations, NSS software token uses the "master secret" legacy label as a parameter to the "TLS_PRF" function for the following mechanisms: CKM_TLS12_MASTER_KEY_DERIVE and CKM_TLS12_MASTER_KEY_DERIVE_DH. That is hardcoded: the client cannot request "extended master secret" label to be used. It's not intended for Extended Master Secret. NSS software token includes a custom mechanism for Extended Master Secret (CKM_NSS_TLS_EXTENDED_MASTER_KEY_DERIVE); that is not part of the PKCS#11 standard but a vendor specific mechanism. ClientHandshaker.java: ---------------------- #4 and #5: Good point! Thanks. That's why I prefer a single return point when possible -particularly for long functions-. I remember changing this code block to the end of the function but did not realize the resumption flow. #2: Done. #3: I assume that you mean to always send the extension in abbreviated handshakes when "useExtendedMasterSecretExtension" is set to true. But I have some concerns. If we always send the extension, there may be cases in which a client that previously negotitatied SSLv3 protocol sends the extension -which has no meaning, as the extension applies to TLS 1.0+-. There may be cases in which Extended Master Secret was not used in the previous full-handshake but the abbreviated handshake is still offered because of allowUnsafeServerCertChange (see conditions for #2). I think conditions in #2 are the real/strong check in regard to abbreviated handshakes. SSLSessionImpl.java ------------------- Done. ServerHandshaker.java --------------------- * jsse.useExtendedMasterSecret does not longer disable the extension on the server side. Documentation and Release Notes updated (see below). #B: Done. #C: I'm trying to think of a real-use case for this but sounds weird. Anyways, done (for both client and server). Documentation and Release Notes updated (see below). Testing --------------------- Sucessfully ran the following regression test categories: * javax/net/ssl * sun/security/ssl Feature documentation (for JSSE Reference Guides [1]) ---------------------------------------------------- ## Extended Master Secret Extension The Extended Master Secret Extension is a feature that replaces the algorithm used to derive the *master secret* for a TLS session. The new algorithm provides a security enhancement to mitigate attacks such as the [Triple Handshake Attack](https://www.mitls.org/pages/attacks/3SHAKE). The Extension is defined by [RFC 7627](https://tools.ietf.org/html/rfc7627) (*Transport Layer Security (TLS) Session Hash and Extended Master Secret Extension*) and applies to TLS 1.0+. Clients and servers need to agree on the usage of this extension during the TLS full handshake, or the previous algorithm is used as a fallback. JSSE supports and enables the Extension by default on both client and server sides. However, for compatibility reasons, disabling is possible on the client side by setting *jsse.useExtendedMasterSecret* system property to *false* (i.e. through the *-Djsse.useExtendedMasterSecret="false"* command-line argument). ### How is the new algorithm different than the previous? The original algorithm uses a PRF function to derive the *master secret* from the following inputs: *pre-master secret* (result of a previous key exchange); "master secret" string; and, client and server random values. The new algorithm replaces the client and server random values with a hash of the previously exchanged handshake messages. As a result, the session *hash* contains information from certificates, key exchange parameters and other handshake-specific values; in addition to the client and server random numbers. Through binding *master secret* to the connection, an active man-in-the-middle attacker cannot force the generation of an identical value in a parallel connection. Thus, values that depend on the *master secret* for authentication (such as the TLS "tls-unique" binding value) can be trusted. ### Sessions resumption The *master secret* for a TLS session is established during a full handshake. When a session is resumed, the original *master secret* is used without any further negotiations. However, for checking purposes, Extended Master Secret Extension messages are exchanged during the abreviated handshake to indicate that the Extension was used when the original *master secret* was derived. If the Extension was not used in the original handshake but is present when resuming, the server moves to a full handshake. On the other hand, if the Extension was used in the original handshake but is not present when resuming, both client and server abort the handshake unless "jsse.allowLegacyResumption" system property is set to *true*. Release note: Extended Master Secret Extension support was added to JSSE for both client and server sides --------------------------------------------------------------------------------------------------------- The Extended Master Secret Extension for TLS 1.0+ is now supported on JSSE for both client and server sides. By modifying the algorithm to derive the session *master secret* (during a full handshake) and binding it to connection-specific values, attacks such as the [Triple Handshake Attack]( https://www.mitls.org/pages/attacks/3SHAKE) are mitigated. The extension, enabled by default, can be turned off on the client side by setting *jsse.useExtendedMasterSecret* system property to *false*. See further information about the Extension in [RFC 7627]( https://tools.ietf.org/html/rfc7627) (*Transport Layer Security (TLS) Session Hash and Extended Master Secret Extension*). Kind regards, Martin.- -- [1] - http://docs.oasis-open.org/pkcs11/pkcs11-curr/v2.40/cs01/pkcs11-curr-v2.40-cs01.html On Wed, Oct 18, 2017 at 9:54 AM, Xuelei Fan wrote: > On 10/17/2017 11:45 AM, Martin Balao wrote: > >> >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-81 >> 48421/webrev.03/ >> > > TlsMasterSecretParameterSpec.java > --------------------------------- > This spec update impacts the PKCS11 implementation too. Please update > jdk/src/jdk.crypto.cryptoki/share/classes/sun/security/pkcs1 > 1/P11TlsMasterSecretGenerator.java as well. > > > ClientHandshaker.java: > ---------------------- > Per RFC 7627: > 1. For full handshaking, a client MUST send the "extended_master_secret" > extension. > 2. A client SHOULD NOT offer an abbreviated handshake to resume a session > that that does not use an extended master secret. Instead, it SHOULD offer > a full handshake. > 3. When offering an abbreviated handshake, the client MUST send the > "extended_master_secret" extension in its ClientHello. > 4. For abbreviated handshake, if the original session did not use the > "extended_master_secret" extension but the new ServerHello contains the > extension, the client MUST abort the handshake. > 5. For abbreviated handshake, if the original session used the extension > but the new ServerHello does not contain the extension, the client MUST > abort the handshake. > > If I'm reading correct, the update in ClientHandshaker.java implements #1, > but missing #2, #4 and #5, and a conditional support of #3. > > For the missing of #4 and #5, it might be mainly caused by that the > 800-828 lines are put in a place where full handshaking happens. > Abbreviated handshake return at line 756 and cannot reach line 800. > > For #2, I may suggest combine the extension together with System property > "jdk.tls.allowUnsafeServerCertChange". If not using extended master > secret, and not allowUnsafeServerCertChange, and > useExtendedMasterSecretExtension, do not offer abbreviated handshake. > Using full handshake instead for TLS 1.0+. Besides, if using the > extension, don't use the server certificate change checking any more. See > allowUnsafeServerCertChange comments in ClientHandshaker.java. > > For #3, I may always send the "extended_master_secret" extension, the > server side can handle it property, no matter the original session use the > extension or not. > > > SSLSessionImpl.java > ------------------- > 94 private boolean useExtendedMasterSecret; > 200 void setUseExtendedMasterSecret() { > 211 boolean getUseExtendedMasterSecret() { > > I may suggest use "final" useExtendedMasterSecret (set during > construction), so that the set/get methods do not compete against each > other. Using "final" may need to adjust some source code. Looks like it > is doable. > > > ServerHandshaker.java > --------------------- > For safer, as there is no compatibility impact as if the client request > for the extension, I think we may want to always enable the extension in > server side. It means the system property "jsse.useExtendedMasterSecret" > disables the extension in client side only. And the property cannot be > used to disable server acceptance of the extension. > > Per RFC 7627: > A. For full handshaking, if a server implementing this document receives > the "extended_master_secret" extension, it MUST include the extension in > its ServerHello message. > B. For abbreviated handshake request, If the original session did not use > the "extended_master_secret" extension but the new ClientHello contains the > extension, then the server MUST NOT perform the abbreviated handshake. > Instead, it SHOULD continue with a full handshake. > C. For abbreviated handshake request, if the original session used the > "extended_master_secret" extension but the new ClientHello does not contain > it, the server MUST abort the abbreviated handshake. > D. For abbreviated handshake request, if neither the original session nor > the new ClientHello uses the extension, the server SHOULD abort the > handshake. > E. For abbreviated handshake request, if the new ClientHello contains the > extension and the server chooses to continue the handshake, then the server > MUST include the "extended_master_secret" extension in its ServerHello > message. > > If I'm reading correct, the update in ServerHandshaker.java implements #A, > #C and #E, but missing #B and #C. > > For #B, I think it should be fine to follow the spec: continue with a full > handshake. > > For #C, I was wondering we may need a new system property > (jsse.allowLegacyResumption?) to turn on/off this behavior. If application > want a strict mode, the server abort the abbreviated handshake for case > #C. Otherwise, the server can continue with an abbreviated handshake in > order to support legacy resumption. > > Hope it helps! > > Regards, > Xuelei > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andreas.rosdal at gmail.com Tue Oct 31 20:25:01 2017 From: andreas.rosdal at gmail.com (=?UTF-8?Q?Andreas_R=C3=B8sdal?=) Date: Tue, 31 Oct 2017 21:25:01 +0100 Subject: Tomcat, SPNEGO, Kerberos against two Active Directory services Message-ID: Hello! I would like some help with setting up Tomcat, SPNEGO and Kerberos against two Active Directory services. At the monent I have a Java webapp running on Tomcat, which uses SPNEGO and Kerberos to authenticate users (clients in Internet Explorer) against one (1) Active Directory user database. Currently, there is only one krb5.conf which is configured against one Active Directory. There is some custom Java code (Servlet filters) which extend the integrated Tomcat SPNEGO classes, and authenticate users against the Active Directory. However, I now need to authenticate users against two different Active Directory databases. Some users are found only in one of the Active Directories, while others are found only in the other Active Directory, so I now need to authenticate against both Active Directories. However, the Java configuration only seems to be able to connect to one Active Directory at a time. I can't use forest trust between the two Actice Directories. I would appreciate any information about best-practices of authenticating users in two Active Directory databases. Regards, Andreas R. -------------- next part -------------- An HTML attachment was scrubbed... URL: