From xuelei.fan at oracle.com Fri Sep 1 01:32:30 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Thu, 31 Aug 2017 18:32:30 -0700 Subject: Code Review Request: JDK-6491070 (Support for RFC 5929-Channel Bindings) In-Reply-To: References: <97c02f80-88aa-d0a7-f1e0-ba80b3a3b71d@oracle.com> Message-ID: <1c51ce46-d0e1-1ab8-6e9e-a71e649ff323@oracle.com> The failure-and-retry mechanism could a nightmare for some applications. Please think more how could we avoid it. If need more APIs, what the update may looks like and how complicated it could be? If required, Bernd's proposal can be extended a little bit to support operations other than listening. APIs maintain is very complicated, a good design may need more time currently, but could save much more in the future. Thanks, Xuelei On 8/31/2017 11:41 AM, Martin Balao wrote: > Hi, > > The material is already cached in the handshaker if secure renegotiation > is enabled. However, I agree with you that we are going to cache the > value even when secure renegotiation is not enabled, thus, wasting > roughly 12 bytes (as bytes for an empty array are already consumed). In > fact, the exact case -adding a few conditionals to webrev.02- is the one > in which secure renegotiation is disabled and extended master secret is > enabled. GnuTls and OpenSSL -including its derivatives like Boring SSL > and Python (through OpenSSL)- always cache this information. > > As for the empty / null cases, the API consumer was expected to ask for > the binding information after the TLS connection is established. It's on > the API consumer knowledge that asking for the information before (i.e.: > just after creating a disconnected socket) or while the handshake is > taking place, makes no sense and no valid value will be obtained (either > we define this as null or empty). For those providers that do not > support this feature, the information wouldn't have been available after > the handshake. However, I agree with you that before the handshake is > completed there is no means of knowing if the provider does support this > API. My first webrev (webrev.01) was throwing an > UnsupportedOperationException to make this case explicit but I had > doubts regarding the real value it provides for the API consumer. The > proposed API was similar to Python, SSLBoring and GnuTLs. However, > handshake listener callbacks -as Bernd suggests-?and the idea of just > exposing the handshake material (as a lower level API) sounds good to > me. I can give it a try. I propose then to bring the handshake > information as part of a HandshakeCompletedEvent instance, even though > the callback registrant may not consume it. Would that work for you? > > In regard to the handshake material update -which I assumed unlikely-, > the point in which a renegotiation may take place (from the server side) > is when reading data, not when writing. That cannot be controlled by the > application because it's JSSE internal and not exposed. Thus, an > application may read from the socket, get the handshake material and > write a message using the binding value -which we can make sure that is > the valid one at that point-. However, as soon as the application reads > again from the socket, a renegotiation -if requested by the client- may > be processed and the binding value gets updated. The higher level > protocol may fail -because the binding value was already sent but not > processed on the other side- and a re-try needed. This looks independent > of whether we use the originally proposed API or the handshake listener > callback interface (or even a sync callback), because the underlying > problem is that the application cannot really control the renegotiation > flow in the lower layer (as RFC 5929 suggest). The options I see are > adding more complexity to the API and let the application control the > renegotiation flow or live with that and expect the application to retry. > > Thanks, > Martin.- > > On Tue, Aug 29, 2017 at 4:34 PM, Xuelei Fan > wrote: > > On 8/26/2017 2:56 PM, Bernd Eckenfels wrote: > > How about only passing it to an extended handshake listener. The > > material does not have to be cached (the app can do it if needed) and > > renegotiation works the same way. This can also be helpful for things > > like logging the master secret (for wireshark decryption). It can even > > handle auditing of session resumptions > Martin, what do you think about Bernd's proposal above and similar > callback mechanism? > > More comment inlines. > > On 8/29/2017 11:50 AM, Martin Balao wrote: > > Hi Xuelei, > > ?>There are a few protocols that can benefits from exporting > SSL/TLS handshake materials, including RFC 5929, RFC 5056, token > binding and TLS 1.3 itself.? Can we define a general API so as > to exposing the handshake materials, so as to mitigate the > inflating of JSSE APIs?? I may suggest make further evaluation > before move on to following design and code. > > Do you prefer an API like "public byte[] > getTlsHandshakeMaterial(String materialType)" (in SSLSocket and > SSLEngine) where "materialType" can eventually be > "clientFinishedMessage"/"finishedMessage" or > "serverFinishedMessage"/"peerFinishedMessage"? > > The problem of the APIs like that is, when applications call the > method, it is not always return the expected result, and the > implementation may have to cache the message even if an application > never use it.? See more in the following example. > > I cannot think of "serverCertificate" or "masterKey" as this is > more related to a Session and not neccessarily to a handshake. > getTlsHandshakeMaterial would be a lower level API and would > move the burden of knowing which information is required for > "tls-unique" TLS channel binding to the API consumer. Looks more > like the OpenSSL approach (instead of the Python, SSLBoring or > GnuTls approaches). However, OpenSSL have specific methods for > each piece of information instead of a generic and parametrized > one. I.e.: SSL_get_finished or SSL_get_peer_finished. What other > information do you expect the Handshaker to provide? > > ?>The SunJSSE provider happens to cache the finished messages > in its implementation so you can use it for tls-unique, but it > may not be true for other provider or other channel bindings. > Need to define a more reliable approach to get the handshake > materials. > > I focused on SunJSSE provider. I'm not sure about how other > providers may implement this API and where they can get the > required information from, without knowing their internals. In > regard to SunJSSE and "tls-unique" binding type, I leveraged on > existing data. If data weren't already there, I would have to > figure out how to get it from the handshake -doing the same that > was already done would have been an option-. Do you prefer the > Handshaker to provide a function to get different information > and not just the finished hash? (as for the public > SSLSocket/SSLEngine "getTlsHandshakeMaterial" API). Which other > information may be useful to get from the Handshaker? What do > you mean by reliable? (given that this is all SunJSSE internal > and we have no external dependencies). > > Let consider the use of the API. > ? ?byte[] getTlsChannelBinding("tls_unique"); > > I'm confusing when I try to use it by myself: > 1. provider does not implement this method > ? ?return null or empty? > > It happens because an old provider should still work in new JDK, but > old provider does not implement new APIs, or a new provider does not > support this feature. > > 2. the method is called before handshaking > ? ?return null or empty? > > 3. the method is called during handshaking > ? ?return null, empty or the channel binding value? > > 4. the method is called at the same time the handshaking completed? > ? ?return the channel binding value? > > 5. the method is called after the handshaking > ? ?return the channel binding value? > > 6. the method is called during renegoitation > ? ?return null, empty, the old binding value, or the new binding value? > > 7. the method is called after handshaking > ? ?return old binding value, or the new binding value? > > 8. the method is called after the initial handshaking, but the > binding value is changed shortly after because of renegotiation. > ? ?how could application use the binding value? > > We need a clear define of the behavior of the method.? It could be > complicated if the method is designed as > getTlsChannelBinding("tls_unique"). > > I feel that handshake material should be captured when > 1. it is requested to capture the handshake material, and > 2. the handshake material get produced. > > For the getTlsChannelBinding("tls_unique") API, it is unknown: > 1. Is it required to capture the handshake material? > 2. Is the handshake material produced? > > The two points could result in a few unexpected problems, as the > above 8 items that we may want to consider. > > In regard to other channel bindings, it'll depend on the binding > type the way in which the information is obtained. I.e.: > "tls-unique" SunJSSE implementation leverages on cached finished > messages. However, "tls-server-end-point" leverages on stored > certificates that are obtained from the Session (not from the > handshaker). Is there any specific channel binding you are > concerned with? > > ?>If the channel binding is not required, it may be not > necessary to expose the handshake materials.? Need to define a > solution to indicate the need of the exporting. > > Do you mean a lower layer knowing if the upper layer is going to > require that information and decide to provide it or not based > on that knowledge? I think I didn't get your point here. > > I mean, if an application want to support channel binding, the > provider can provider the channel binding service;? If the an > application does not want channel binding, the provider should be > perform the channel binding service.? The getTlsChannelBinding() > make the provider MUST perform channel binding cache or calculation > no matter application want it or not. > > ?>2. No way to know the update of the underlying handshake > materials. > ?>If renegotiation can takes place, need to define a interface > to indicate that so that application can response accordingly. > See section 3 and 7 of RFC 5929. > > I intentionally skipped this -at the cost of a spurious > authentication- to avoid adding complexity to the API. An > spurious authentication -which does not appear likely to me- can > easily be retried by the application. The RFC 5929 suggests APIs > through which the application can *control* the flow (i.e.: hold > a renegotitation). This would expose JSSE internals. This is > more than notifying. Notification, in my opinion, adds no value: > what if the application already used the binding token before > receiving the notification? The spurious authentication will > happen anyways and has to be handled -i.e. retried-. It's just a > timing issue. The real value is controlling the flow as the RFC > suggests, but at the cost of exposing JSSE internals. > > My understanding, the block of the protocol is to make sure > application is performing the channel binding with the right value, > or updating the value accordingly if necessary.? If you skip this > and when renegotiation happen, the channel binding could be limited, > or may not work as expected. > > Thanks, > Xuelei > > Kind regards, > Martin.- > > > On Sat, Aug 26, 2017 at 5:25 PM, Xuelei Fan > > >> > wrote: > > ? ? Hi Marin, > > ? ? Sorry for the delay. > > ? ? There are a few protocols that can benefits from exporting > SSL/TLS > ? ? handshake materials, including RFC 5929, RFC 5056, token > binding and > ? ? TLS 1.3 itself.? Can we define a general API so as to > exposing the > ? ? handshake materials, so as to mitigate the inflating of > JSSE APIs?? ? ?I may suggest make further evaluation before move > on to following > ? ? design and code. > > ? ? ?> > http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.02/ > > > > > ? ? I have two concerns about the design: > > ? ? 1. Channel binding may be not always required. > ? ? SSLSocket/SSLEngine.getTlsChannelBinding(String bindingType); > > ? ? The SunJSSE provider happens to cache the finished messages > in its > ? ? implementation so you can use it for tls-unique, but it may > not be > ? ? true for other provider or other channel bindings.? Need to > define a > ? ? more reliable approach to get the handshake materials. > > ? ? If the channel binding is not required, it may be not > necessary to > ? ? expose the handshake materials.? Need to define a solution to > ? ? indicate the need of the exporting. > > ? ? 2. No way to know the update of the underlying handshake > materials. > ? ? If renegotiation can takes place, need to define a interface to > ? ? indicate that so that application can response > accordingly.? See > ? ? section 3 and 7 of RFC 5929. > > ? ? Thanks, > ? ? Xuelei > > ? ? On 7/31/2017 8:53 AM, Martin Balao wrote: > > ? ? ? ? Hi, > > ? ? ? ? Here it is an update for the proposed TLS Channel Bindings > ? ? ? ? support in OpenJDK: > > ? ? ? ? ???* > http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.02/ > > > > > ? ? ? ? (browse online) > ? ? ? ? ???* > http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.02/6491070.webrev.02.zip > > > > > ? ? ? ? (download) > > ? ? ? ? Changes since v01: > > ? ? ? ? ???* getTlsChannelBinding API changed to return null by > default > ? ? ? ? (if not implemented), instead of throwing an > ? ? ? ? UnsupportedOperationException. > > ? ? ? ? ???* "tls-server-end-point" TLS channel binding now > supported. > > ? ? ? ? Kind regards, > ? ? ? ? Martin.- > > ? ? ? ? On Wed, Jul 26, 2017 at 4:12 PM, Martin Balao > > ? ? ? ? > > > > ? ? ? ? >>> > wrote: > > ? ? ? ? ?? ? Hi, > > ? ? ? ? ?? ? Here it is my proposal for JDK-6491070 (Support > for RFC > ? ? ? ? 5929-Channel > ? ? ? ? ?? ? Bindings: e.g. public API to obtain TLS finished > message) [1]: > > ? ? ? ? ?? ? ??* > http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/ > > > > > > > > >> > ? ? ? ? ?? ? ??* > http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/6491070.webrev.01.zip > > > > > > > > >> > > ? ? ? ? ?? ? Notes: > ? ? ? ? ?? ? ??* Implementation based on Channel Bindings for > TLS (RFC > ? ? ? ? 5929) [2] > > ? ? ? ? ?? ? ??* Only "tls-unique" currently supported > > ? ? ? ? ?? ? Look forward to your comments. > > ? ? ? ? ?? ? Kind regards, > ? ? ? ? ?? ? Martin.- > > ? ? ? ? ?? ? -- > ? ? ? ? ?? ? [1] - > https://bugs.openjdk.java.net/browse/JDK-6491070 > > ? ? ? ? > > ? ? ? ? ?? ? > ? ? ? ? >> > ? ? ? ? ?? ? [2] - https://tools.ietf.org/html/rfc5929 > > ? ? ? ? > > ? ? ? ? ?? ? > ? ? ? ? >> > > > > From vincent.x.ryan at oracle.com Fri Sep 1 07:58:47 2017 From: vincent.x.ryan at oracle.com (Vincent Ryan) Date: Fri, 1 Sep 2017 08:58:47 +0100 Subject: hello, im a new contributor In-Reply-To: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> Message-ID: <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> Moved to security-dev > On 1 Sep 2017, at 08:28, Philipp Kunz wrote: > > Hello everyone > > I have been developing with Java for around 17 years now and when I encountered some bug I decided to attempt to fix it: https://bugs.openjdk.java.net/browse/JDK-6695402. This also looks like it may not be too big a piece for a first contribution. > > I read through quite some guides and all kinds of documents but could not yet help myself with the following questions: > > May I login to jira to add comments to bugs? If so, how would I request or receive credentials? Or are mailing lists preferred? > > Another question is whether I should apply it to jdk9, but it may be too late now, or to jdk10, and backporting can be considered later. Probably it wouldn't even make much a difference for the patch itself. > > One more question I have is how or where to find the sources from before migration to mercurial. Because some lines of code I intend to change go back farther and in the history I find only 'initial commit'. With such a history I might be able better to understand why it's there and prevent to make the same mistake again. > > I guess the appropriate mailing list for above mentioned bug is security-dev. Is it correct that I can send a patch there and just hope for some sponsor to pick it up? Of course I'd be glad if some sponsor would contact me and maybe provide some assistance or if someone would confirm that sending a patch to the mailing list is the right way to find a sponsor. > > Philipp Kunz From vincent.x.ryan at oracle.com Fri Sep 1 08:00:37 2017 From: vincent.x.ryan at oracle.com (Vincent Ryan) Date: Fri, 1 Sep 2017 09:00:37 +0100 Subject: hello, im a new contributor In-Reply-To: <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> Message-ID: <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> Hello Philipp, I?m happy to sponsor your fix for JDK 10. Have you followed these steps: http://openjdk.java.net/contribute/ ? Thanks. > On 1 Sep 2017, at 08:58, Vincent Ryan wrote: > > Moved to security-dev > > >> On 1 Sep 2017, at 08:28, Philipp Kunz wrote: >> >> Hello everyone >> >> I have been developing with Java for around 17 years now and when I encountered some bug I decided to attempt to fix it: https://bugs.openjdk.java.net/browse/JDK-6695402. This also looks like it may not be too big a piece for a first contribution. >> >> I read through quite some guides and all kinds of documents but could not yet help myself with the following questions: >> >> May I login to jira to add comments to bugs? If so, how would I request or receive credentials? Or are mailing lists preferred? >> >> Another question is whether I should apply it to jdk9, but it may be too late now, or to jdk10, and backporting can be considered later. Probably it wouldn't even make much a difference for the patch itself. >> >> One more question I have is how or where to find the sources from before migration to mercurial. Because some lines of code I intend to change go back farther and in the history I find only 'initial commit'. With such a history I might be able better to understand why it's there and prevent to make the same mistake again. >> >> I guess the appropriate mailing list for above mentioned bug is security-dev. Is it correct that I can send a patch there and just hope for some sponsor to pick it up? Of course I'd be glad if some sponsor would contact me and maybe provide some assistance or if someone would confirm that sending a patch to the mailing list is the right way to find a sponsor. >> >> Philipp Kunz > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vincent.x.ryan at oracle.com Fri Sep 1 13:20:05 2017 From: vincent.x.ryan at oracle.com (Vincent Ryan) Date: Fri, 1 Sep 2017 14:20:05 +0100 Subject: hello, im a new contributor In-Reply-To: <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> Message-ID: <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> That all sounds fine. Let me know when your patch is ready to submit. Thanks. > On 1 Sep 2017, at 13:15, Philipp Kunz wrote: > > Hello Vincent > > Thank you for sponsoring! > So far, I have become a contributor by signing the OCA which has been accepted. Dalibor Topic wrote that he can confirm and it's also here: http://www.oracle.com/technetwork/community/oca-486395.html#p -> Paratix GmbH > Therefore I think I have followed the steps in http://openjdk.java.net/contribute/ at least the ones before actual patch submission. > > Currently, I'm working on getting the existing test cases running locally. Unfortunately, I started with jdk 9 and switched to 10 now. I figured these commands run at least the relevant tests with 9 and hope this also applies to 10: > make run-test-tier1 > make run-test TEST="jdk/test" > they report errors and failures but it hasn't been completed and released which might explain it. If I run the tests I assume relevant for my patch > make run-test TEST="jtreg:jdk/test/sun/security/tools/jarsigner jtreg:jdk/test/java/util/jar" > then it reports zero errors and failures which may be a good starting point. > Do you think that sounds reasonable or do you have another suggestion how to run the tests? > > Next, I will add a test for JDK-6695402 before actually fixing it. As an example, I'll try something in the style of http://hg.openjdk.java.net/jdk9/dev/jdk/file/65464a307408/test/java/util/jar/Manifest/CreateManifest.java . This way, I try to demonstrate the improvement. > > I guess I have identified the following line as the cause: value = new String(vb, 0, 0, vb.length); > http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/51f5d60713b5/src/java.base/share/classes/java/util/jar/Manifest.java#l157 > So I'll try to remove it first including the whole four line if block. > > Philipp > > > On 01.09.2017 10:00, Vincent Ryan wrote: >> Hello Philipp, >> >> I?m happy to sponsor your fix for JDK 10. Have you followed these steps: http://openjdk.java.net/contribute/ ? >> >> Thanks. >> >> >>> On 1 Sep 2017, at 08:58, Vincent Ryan > wrote: >>> >>> Moved to security-dev >>> >>> >>>> On 1 Sep 2017, at 08:28, Philipp Kunz > wrote: >>>> >>>> Hello everyone >>>> >>>> I have been developing with Java for around 17 years now and when I encountered some bug I decided to attempt to fix it: https://bugs.openjdk.java.net/browse/JDK-6695402 . This also looks like it may not be too big a piece for a first contribution. >>>> >>>> I read through quite some guides and all kinds of documents but could not yet help myself with the following questions: >>>> >>>> May I login to jira to add comments to bugs? If so, how would I request or receive credentials? Or are mailing lists preferred? >>>> >>>> Another question is whether I should apply it to jdk9, but it may be too late now, or to jdk10, and backporting can be considered later. Probably it wouldn't even make much a difference for the patch itself. >>>> >>>> One more question I have is how or where to find the sources from before migration to mercurial. Because some lines of code I intend to change go back farther and in the history I find only 'initial commit'. With such a history I might be able better to understand why it's there and prevent to make the same mistake again. >>>> >>>> I guess the appropriate mailing list for above mentioned bug is security-dev. Is it correct that I can send a patch there and just hope for some sponsor to pick it up? Of course I'd be glad if some sponsor would contact me and maybe provide some assistance or if someone would confirm that sending a patch to the mailing list is the right way to find a sponsor. >>>> >>>> Philipp Kunz >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.lloyd at redhat.com Fri Sep 1 14:08:26 2017 From: david.lloyd at redhat.com (David Lloyd) Date: Fri, 1 Sep 2017 09:08:26 -0500 Subject: Code Review Request: JDK-6491070 (Support for RFC 5929-Channel Bindings) In-Reply-To: <1c51ce46-d0e1-1ab8-6e9e-a71e649ff323@oracle.com> References: <97c02f80-88aa-d0a7-f1e0-ba80b3a3b71d@oracle.com> <1c51ce46-d0e1-1ab8-6e9e-a71e649ff323@oracle.com> Message-ID: I can say that from the perspective of SASL that we don't need any special indication about handshake, and it would be much easier to just read the material off of the engine, socket, or session as appropriate. On Thu, Aug 31, 2017 at 8:32 PM, Xuelei Fan wrote: > The failure-and-retry mechanism could a nightmare for some applications. > Please think more how could we avoid it. If need more APIs, what the update > may looks like and how complicated it could be? > > If required, Bernd's proposal can be extended a little bit to support > operations other than listening. > > APIs maintain is very complicated, a good design may need more time > currently, but could save much more in the future. > > Thanks, > Xuelei > > On 8/31/2017 11:41 AM, Martin Balao wrote: >> >> Hi, >> >> The material is already cached in the handshaker if secure renegotiation >> is enabled. However, I agree with you that we are going to cache the value >> even when secure renegotiation is not enabled, thus, wasting roughly 12 >> bytes (as bytes for an empty array are already consumed). In fact, the exact >> case -adding a few conditionals to webrev.02- is the one in which secure >> renegotiation is disabled and extended master secret is enabled. GnuTls and >> OpenSSL -including its derivatives like Boring SSL and Python (through >> OpenSSL)- always cache this information. >> >> As for the empty / null cases, the API consumer was expected to ask for >> the binding information after the TLS connection is established. It's on the >> API consumer knowledge that asking for the information before (i.e.: just >> after creating a disconnected socket) or while the handshake is taking >> place, makes no sense and no valid value will be obtained (either we define >> this as null or empty). For those providers that do not support this >> feature, the information wouldn't have been available after the handshake. >> However, I agree with you that before the handshake is completed there is no >> means of knowing if the provider does support this API. My first webrev >> (webrev.01) was throwing an UnsupportedOperationException to make this case >> explicit but I had doubts regarding the real value it provides for the API >> consumer. The proposed API was similar to Python, SSLBoring and GnuTLs. >> However, handshake listener callbacks -as Bernd suggests- and the idea of >> just exposing the handshake material (as a lower level API) sounds good to >> me. I can give it a try. I propose then to bring the handshake information >> as part of a HandshakeCompletedEvent instance, even though the callback >> registrant may not consume it. Would that work for you? >> >> In regard to the handshake material update -which I assumed unlikely-, the >> point in which a renegotiation may take place (from the server side) is when >> reading data, not when writing. That cannot be controlled by the application >> because it's JSSE internal and not exposed. Thus, an application may read >> from the socket, get the handshake material and write a message using the >> binding value -which we can make sure that is the valid one at that point-. >> However, as soon as the application reads again from the socket, a >> renegotiation -if requested by the client- may be processed and the binding >> value gets updated. The higher level protocol may fail -because the binding >> value was already sent but not processed on the other side- and a re-try >> needed. This looks independent of whether we use the originally proposed API >> or the handshake listener callback interface (or even a sync callback), >> because the underlying problem is that the application cannot really control >> the renegotiation flow in the lower layer (as RFC 5929 suggest). The options >> I see are adding more complexity to the API and let the application control >> the renegotiation flow or live with that and expect the application to >> retry. >> >> Thanks, >> Martin.- >> >> On Tue, Aug 29, 2017 at 4:34 PM, Xuelei Fan > > wrote: >> >> On 8/26/2017 2:56 PM, Bernd Eckenfels wrote: >> > How about only passing it to an extended handshake listener. The >> > material does not have to be cached (the app can do it if needed) >> and >> > renegotiation works the same way. This can also be helpful for >> things >> > like logging the master secret (for wireshark decryption). It can >> even >> > handle auditing of session resumptions >> Martin, what do you think about Bernd's proposal above and similar >> callback mechanism? >> >> More comment inlines. >> >> On 8/29/2017 11:50 AM, Martin Balao wrote: >> >> Hi Xuelei, >> >> >There are a few protocols that can benefits from exporting >> SSL/TLS handshake materials, including RFC 5929, RFC 5056, token >> binding and TLS 1.3 itself. Can we define a general API so as >> to exposing the handshake materials, so as to mitigate the >> inflating of JSSE APIs? I may suggest make further evaluation >> before move on to following design and code. >> >> Do you prefer an API like "public byte[] >> getTlsHandshakeMaterial(String materialType)" (in SSLSocket and >> SSLEngine) where "materialType" can eventually be >> "clientFinishedMessage"/"finishedMessage" or >> "serverFinishedMessage"/"peerFinishedMessage"? >> >> The problem of the APIs like that is, when applications call the >> method, it is not always return the expected result, and the >> implementation may have to cache the message even if an application >> never use it. See more in the following example. >> >> I cannot think of "serverCertificate" or "masterKey" as this is >> more related to a Session and not neccessarily to a handshake. >> getTlsHandshakeMaterial would be a lower level API and would >> move the burden of knowing which information is required for >> "tls-unique" TLS channel binding to the API consumer. Looks more >> like the OpenSSL approach (instead of the Python, SSLBoring or >> GnuTls approaches). However, OpenSSL have specific methods for >> each piece of information instead of a generic and parametrized >> one. I.e.: SSL_get_finished or SSL_get_peer_finished. What other >> information do you expect the Handshaker to provide? >> >> >The SunJSSE provider happens to cache the finished messages >> in its implementation so you can use it for tls-unique, but it >> may not be true for other provider or other channel bindings. >> Need to define a more reliable approach to get the handshake >> materials. >> >> I focused on SunJSSE provider. I'm not sure about how other >> providers may implement this API and where they can get the >> required information from, without knowing their internals. In >> regard to SunJSSE and "tls-unique" binding type, I leveraged on >> existing data. If data weren't already there, I would have to >> figure out how to get it from the handshake -doing the same that >> was already done would have been an option-. Do you prefer the >> Handshaker to provide a function to get different information >> and not just the finished hash? (as for the public >> SSLSocket/SSLEngine "getTlsHandshakeMaterial" API). Which other >> information may be useful to get from the Handshaker? What do >> you mean by reliable? (given that this is all SunJSSE internal >> and we have no external dependencies). >> >> Let consider the use of the API. >> byte[] getTlsChannelBinding("tls_unique"); >> >> I'm confusing when I try to use it by myself: >> 1. provider does not implement this method >> return null or empty? >> >> It happens because an old provider should still work in new JDK, but >> old provider does not implement new APIs, or a new provider does not >> support this feature. >> >> 2. the method is called before handshaking >> return null or empty? >> >> 3. the method is called during handshaking >> return null, empty or the channel binding value? >> >> 4. the method is called at the same time the handshaking completed? >> return the channel binding value? >> >> 5. the method is called after the handshaking >> return the channel binding value? >> >> 6. the method is called during renegoitation >> return null, empty, the old binding value, or the new binding >> value? >> >> 7. the method is called after handshaking >> return old binding value, or the new binding value? >> >> 8. the method is called after the initial handshaking, but the >> binding value is changed shortly after because of renegotiation. >> how could application use the binding value? >> >> We need a clear define of the behavior of the method. It could be >> complicated if the method is designed as >> getTlsChannelBinding("tls_unique"). >> >> I feel that handshake material should be captured when >> 1. it is requested to capture the handshake material, and >> 2. the handshake material get produced. >> >> For the getTlsChannelBinding("tls_unique") API, it is unknown: >> 1. Is it required to capture the handshake material? >> 2. Is the handshake material produced? >> >> The two points could result in a few unexpected problems, as the >> above 8 items that we may want to consider. >> >> In regard to other channel bindings, it'll depend on the binding >> type the way in which the information is obtained. I.e.: >> "tls-unique" SunJSSE implementation leverages on cached finished >> messages. However, "tls-server-end-point" leverages on stored >> certificates that are obtained from the Session (not from the >> handshaker). Is there any specific channel binding you are >> concerned with? >> >> >If the channel binding is not required, it may be not >> necessary to expose the handshake materials. Need to define a >> solution to indicate the need of the exporting. >> >> Do you mean a lower layer knowing if the upper layer is going to >> require that information and decide to provide it or not based >> on that knowledge? I think I didn't get your point here. >> >> I mean, if an application want to support channel binding, the >> provider can provider the channel binding service; If the an >> application does not want channel binding, the provider should be >> perform the channel binding service. The getTlsChannelBinding() >> make the provider MUST perform channel binding cache or calculation >> no matter application want it or not. >> >> >2. No way to know the update of the underlying handshake >> materials. >> >If renegotiation can takes place, need to define a interface >> to indicate that so that application can response accordingly. >> See section 3 and 7 of RFC 5929. >> >> I intentionally skipped this -at the cost of a spurious >> authentication- to avoid adding complexity to the API. An >> spurious authentication -which does not appear likely to me- can >> easily be retried by the application. The RFC 5929 suggests APIs >> through which the application can *control* the flow (i.e.: hold >> a renegotitation). This would expose JSSE internals. This is >> more than notifying. Notification, in my opinion, adds no value: >> what if the application already used the binding token before >> receiving the notification? The spurious authentication will >> happen anyways and has to be handled -i.e. retried-. It's just a >> timing issue. The real value is controlling the flow as the RFC >> suggests, but at the cost of exposing JSSE internals. >> >> My understanding, the block of the protocol is to make sure >> application is performing the channel binding with the right value, >> or updating the value accordingly if necessary. If you skip this >> and when renegotiation happen, the channel binding could be limited, >> or may not work as expected. >> >> Thanks, >> Xuelei >> >> Kind regards, >> Martin.- >> >> >> On Sat, Aug 26, 2017 at 5:25 PM, Xuelei Fan >> >> >> >> >> wrote: >> >> Hi Marin, >> >> Sorry for the delay. >> >> There are a few protocols that can benefits from exporting >> SSL/TLS >> handshake materials, including RFC 5929, RFC 5056, token >> binding and >> TLS 1.3 itself. Can we define a general API so as to >> exposing the >> handshake materials, so as to mitigate the inflating of >> JSSE APIs? I may suggest make further evaluation before move >> on to following >> design and code. >> >> > >> >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.02/ >> >> >> >> > >> > >> I have two concerns about the design: >> >> 1. Channel binding may be not always required. >> SSLSocket/SSLEngine.getTlsChannelBinding(String bindingType); >> >> The SunJSSE provider happens to cache the finished messages >> in its >> implementation so you can use it for tls-unique, but it may >> not be >> true for other provider or other channel bindings. Need to >> define a >> more reliable approach to get the handshake materials. >> >> If the channel binding is not required, it may be not >> necessary to >> expose the handshake materials. Need to define a solution to >> indicate the need of the exporting. >> >> 2. No way to know the update of the underlying handshake >> materials. >> If renegotiation can takes place, need to define a interface >> to >> indicate that so that application can response >> accordingly. See >> section 3 and 7 of RFC 5929. >> >> Thanks, >> Xuelei >> >> On 7/31/2017 8:53 AM, Martin Balao wrote: >> >> Hi, >> >> Here it is an update for the proposed TLS Channel >> Bindings >> support in OpenJDK: >> >> * >> >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.02/ >> >> >> >> > >> > >> (browse online) >> * >> >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.02/6491070.webrev.02.zip >> >> >> >> > >> > >> (download) >> >> Changes since v01: >> >> * getTlsChannelBinding API changed to return null by >> default >> (if not implemented), instead of throwing an >> UnsupportedOperationException. >> >> * "tls-server-end-point" TLS channel binding now >> supported. >> >> Kind regards, >> Martin.- >> >> On Wed, Jul 26, 2017 at 4:12 PM, Martin Balao >> >> > >> >> >> >>> >> >> wrote: >> >> Hi, >> >> Here it is my proposal for JDK-6491070 (Support >> for RFC >> 5929-Channel >> Bindings: e.g. public API to obtain TLS finished >> message) [1]: >> >> * >> >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/ >> >> >> >> > >> > >> >> > >> >> >> > >> >> >> * >> >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK-6491070/webrev.01/6491070.webrev.01.zip >> >> >> >> > >> > >> >> > >> >> >> > >> >> >> >> Notes: >> * Implementation based on Channel Bindings for >> TLS (RFC >> 5929) [2] >> >> * Only "tls-unique" currently supported >> >> Look forward to your comments. >> >> Kind regards, >> Martin.- >> >> -- >> [1] - >> https://bugs.openjdk.java.net/browse/JDK-6491070 >> >> > > >> > >> > >> >> [2] - https://tools.ietf.org/html/rfc5929 >> >> > > >> > >> > >> >> >> >> >> > -- - DML From sean.coffey at oracle.com Fri Sep 1 15:04:24 2017 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 1 Sep 2017 16:04:24 +0100 Subject: RFR: 8170157, 8170245: Enable unlimited cryptographic policy by default in OracleJDK In-Reply-To: <33d4d8cf-90ef-3653-48f0-96c0c327aca8@oracle.com> References: <9ca65218-86b0-1209-8f11-9d0b2fd67c2b@oracle.com> <33d4d8cf-90ef-3653-48f0-96c0c327aca8@oracle.com> Message-ID: <1521acbf-cd03-7be7-6026-d7beb51f07cd@oracle.com> comments inline. On 29/08/17 23:33, Bradford Wetmore wrote: > > Very minor comments/tweaks. > > On 8/18/2017 7:01 AM, Se?n Coffey wrote: >> Looking to backport 8170157 to jdk8u-dev. The 8170245 test bug also >> gets pulled in for this port since some tests need cleaning up to >> deal with unlimited crypto environment. >> >> webrev : >> http://cr.openjdk.java.net/~coffeys/webrev.8170157.8u.01/webrev/index.html > > Update copyright dates. Looks like the original work was done in > December 2016, but this is our actual push. Done. > > JceSecurity.java > ================ > 265: Suggestion since this is JDK/JRE specific: > > jre/lib/security > -> > /lib/security Done. I corrected line 260 also which was using the jre/lib/security syntax. > > Dynamic.java > ============ > Is the removal of the separate ECB case because the parameter will > just be ignored? Why was this a failing test case? I inherited these changes from the JDK 9 edits. It looks like the stronger AES defaults place stricter conditions on the IV length.[1] - The test had some old conditions for some non-GCM mode based ciphers. That seems to be no longer necessary and the call to "ci.init(Cipher.DECRYPT_MODE, key, ci.getParameters());" appears to work for all now. regards, Sean. [1] java.security.InvalidAlgorithmParameterException: Wrong IV length: must be 16 bytes long at com.sun.crypto.provider.CipherCore.init(CipherCore.java:526) at com.sun.crypto.provider.AESCipher.engineInit(AESCipher.java:346) at javax.crypto.Cipher.init(Cipher.java:1394) at javax.crypto.Cipher.init(Cipher.java:1327) at Dynamic.runTest(Dynamic.java:145) at Dynamic.runAllTest(Dynamic.java:89) at Dynamic.run(Dynamic.java:59) at TestAESWithRemoveAddProvider.main(TestAESWithRemoveAddProvider.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) at java.lang.Thread.run(Thread.java:748) > > Thanks. > > Brad > > > From philipp.kunz at paratix.ch Fri Sep 1 12:15:53 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Fri, 1 Sep 2017 14:15:53 +0200 Subject: hello, im a new contributor In-Reply-To: <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> Message-ID: <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> Hello Vincent Thank you for sponsoring! So far, I have become a contributor by signing the OCA which has been accepted. Dalibor Topic wrote that he can confirm and it's also here: http://www.oracle.com/technetwork/community/oca-486395.html#p -> Paratix GmbH Therefore I think I have followed the steps in http://openjdk.java.net/contribute/ at least the ones before actual patch submission. Currently, I'm working on getting the existing test cases running locally. Unfortunately, I started with jdk 9 and switched to 10 now. I figured these commands run at least the relevant tests with 9 and hope this also applies to 10: make run-test-tier1 make run-test TEST="jdk/test" they report errors and failures but it hasn't been completed and released which might explain it. If I run the tests I assume relevant for my patch make run-test TEST="jtreg:jdk/test/sun/security/tools/jarsigner jtreg:jdk/test/java/util/jar" then it reports zero errors and failures which may be a good starting point. Do you think that sounds reasonable or do you have another suggestion how to run the tests? Next, I will add a test for JDK-6695402 before actually fixing it. As an example, I'll try something in the style of http://hg.openjdk.java.net/jdk9/dev/jdk/file/65464a307408/test/java/util/jar/Manifest/CreateManifest.java. This way, I try to demonstrate the improvement. I guess I have identified the following line as the cause: value = new String(vb, 0, 0, vb.length); http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/51f5d60713b5/src/java.base/share/classes/java/util/jar/Manifest.java#l157 So I'll try to remove it first including the whole four line if block. Philipp On 01.09.2017 10:00, Vincent Ryan wrote: > Hello Philipp, > > I?m happy to sponsor your fix for JDK 10. Have you followed these > steps: http://openjdk.java.net/contribute/ ? > > Thanks. > > >> On 1 Sep 2017, at 08:58, Vincent Ryan > > wrote: >> >> Moved to security-dev >> >> >>> On 1 Sep 2017, at 08:28, Philipp Kunz >> > wrote: >>> >>> Hello everyone >>> >>> I have been developing with Java for around 17 years now and when I >>> encountered some bug I decided to attempt to fix it: >>> https://bugs.openjdk.java.net/browse/JDK-6695402. This also looks >>> like it may not be too big a piece for a first contribution. >>> >>> I read through quite some guides and all kinds of documents but >>> could not yet help myself with the following questions: >>> >>> May I login to jira to add comments to bugs? If so, how would I >>> request or receive credentials? Or are mailing lists preferred? >>> >>> Another question is whether I should apply it to jdk9, but it may be >>> too late now, or to jdk10, and backporting can be considered later. >>> Probably it wouldn't even make much a difference for the patch itself. >>> >>> One more question I have is how or where to find the sources from >>> before migration to mercurial. Because some lines of code I intend >>> to change go back farther and in the history I find only 'initial >>> commit'. With such a history I might be able better to understand >>> why it's there and prevent to make the same mistake again. >>> >>> I guess the appropriate mailing list for above mentioned bug is >>> security-dev. Is it correct that I can send a patch there and just >>> hope for some sponsor to pick it up? Of course I'd be glad if some >>> sponsor would contact me and maybe provide some assistance or if >>> someone would confirm that sending a patch to the mailing list is >>> the right way to find a sponsor. >>> >>> Philipp Kunz >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From weijun.wang at oracle.com Tue Sep 5 23:16:03 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 6 Sep 2017 07:16:03 +0800 Subject: RFR 8186931: jdk.security.jarsigner package is missing package summary In-Reply-To: References: <8FC7DB36-00EC-475E-82D1-F2A48FDD03C8@oracle.com> Message-ID: <667126DE-EB9C-4557-BCB5-0BC050FFC828@oracle.com> The package info for com.sun.security.jgss in jdk.security.jgss is also missing. The updated change looks like this (I omit the copyright header): src/jdk.jartool/share/classes/jdk/security/jarsigner/package-info.java: +/** + * This package defines APIs for signing jar files. + */ +package jdk.security.jarsigner; src/jdk.jartool/share/classes/module-info.java: + * This module also defines APIs for signing JAR files. src/jdk.security.jgss/share/classes/com/sun/security/jgss/package-info.java: +/** + * This package defines classes and interfaces for the JDK extensions + * to the GSS-API. + */ package com.sun.security.jgss; src/jdk.security.jgss/share/classes/module-info.java: /** - * Defines Java extensions to the GSS-API and an implementation of the SASL + * Defines JDK extensions to the GSS-API and an implementation of the SASL * GSSAPI mechanism. Thanks Max > On Aug 31, 2017, at 11:10 PM, Sean Mullan wrote: > > On 8/31/17 4:49 AM, Weijun Wang wrote: > >> /** >> * This package contains the {@link jdk.security.jarsigner.JarSigner} API, >> * which backs the signing function of the {@code jarsigner} tool. >> */ > > I think you should say something about that this API can be used to sign JAR files. The fact that it is also used by jarsigner seems less important to mention in the package description. I suggest changing this to: > > "This package contains the {@link jdk.security.jarsigner.JarSigner} API which can be used to sign jar files." > > --Sean From mandy.chung at oracle.com Tue Sep 5 23:17:09 2017 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 5 Sep 2017 17:17:09 -0600 Subject: RFR 8186931: jdk.security.jarsigner package is missing package summary In-Reply-To: <667126DE-EB9C-4557-BCB5-0BC050FFC828@oracle.com> References: <8FC7DB36-00EC-475E-82D1-F2A48FDD03C8@oracle.com> <667126DE-EB9C-4557-BCB5-0BC050FFC828@oracle.com> Message-ID: <9370639e-328f-9c73-87b7-6862aee3a9cf@oracle.com> +1 Mandy On 9/5/17 5:16 PM, Weijun Wang wrote: > The package info for com.sun.security.jgss in jdk.security.jgss is also missing. The updated change looks like this (I omit the copyright header): > > src/jdk.jartool/share/classes/jdk/security/jarsigner/package-info.java: > > +/** > + * This package defines APIs for signing jar files. > + */ > +package jdk.security.jarsigner; > > > src/jdk.jartool/share/classes/module-info.java: > + * This module also defines APIs for signing JAR files. > > > src/jdk.security.jgss/share/classes/com/sun/security/jgss/package-info.java: > > +/** > + * This package defines classes and interfaces for the JDK extensions > + * to the GSS-API. > + */ > package com.sun.security.jgss; > > > src/jdk.security.jgss/share/classes/module-info.java: > > /** > - * Defines Java extensions to the GSS-API and an implementation of the SASL > + * Defines JDK extensions to the GSS-API and an implementation of the SASL > * GSSAPI mechanism. > > Thanks > Max > >> On Aug 31, 2017, at 11:10 PM, Sean Mullan wrote: >> >> On 8/31/17 4:49 AM, Weijun Wang wrote: >> >>> /** >>> * This package contains the {@link jdk.security.jarsigner.JarSigner} API, >>> * which backs the signing function of the {@code jarsigner} tool. >>> */ >> I think you should say something about that this API can be used to sign JAR files. The fact that it is also used by jarsigner seems less important to mention in the package description. I suggest changing this to: >> >> "This package contains the {@link jdk.security.jarsigner.JarSigner} API which can be used to sign jar files." >> >> --Sean From weijun.wang at oracle.com Wed Sep 6 04:17:33 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 6 Sep 2017 12:17:33 +0800 Subject: RFR 8148371: Remove policytool In-Reply-To: References: Message-ID: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> Hi All Please review the change, which spans to root, jdk and langtools repos. http://cr.openjdk.java.net/~weijun/8148371/ I've searched for the "policytool" word in the whole jdk10/jdk10 forests, removed all files having the word inside the path name, and remove almost all occurrences of the word in other places. The exceptions are: 1. Two files with the jdk8 word in file name. I assume I should not touch them. Please advise me. jdk/src/java.base/share/classes/jdk/internal/module/jdk8_packages.dat: 1288 sun.security.tools.jarsigner 1289 sun.security.tools.keytool 1290: sun.security.tools.policytool 1291 sun.security.util 1292 sun.security.validator langtools/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/resources/jdk8_internals.txt: 977 sun.security.tools.jarsigner 978 sun.security.tools.keytool 979: sun.security.tools.policytool 980 sun.security.util 981 sun.security.validator 2. A swing test containing a full JDK 1.4.2 README text. Keep unchanged. jdk/test/javax/swing/JTextArea/4697612/bug4697612.txt: 122 bin/ktab and jre/bin/ktab 123 Kerberos key table manager 124: bin/policytool and jre/bin/policytool 125 Policy File Creation and Management Tool 126 bin/orbd and jre/bin/orbd 3. A manual test on what resource string are used. It is based on the pre-module source layout and needs to be updated anyway. Keep unchanged this time. https://bugs.openjdk.java.net/browse/JDK-8187265 filed. jdk/test/sun/security/util/Resources/NewResourcesNames.java: 62 "sun/security/tools/jarsigner/Resources.java", 63 "sun/security/tools/keytool/Resources.java", 64: "sun/security/tools/policytool/Resources.java", 65 "sun/security/util/Resources.java", 66 "sun/security/util/AuthResources.java", .. 103 // 104 // which is mismatch. There are only two such special cases list above. 105: // For KeyTool, there are 3 calls for showing help. For PolicyTool, 3 106 // for name prefixed with POLICY. They are covered in the two special 107 // cases above. There are some Japanese man pages containing the word. I've filed another bug (https://bugs.openjdk.java.net/browse/JDK-8187262) on it. Thanks Max From erik.joelsson at oracle.com Wed Sep 6 08:53:27 2017 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 6 Sep 2017 10:53:27 +0200 Subject: RFR 8148371: Remove policytool In-Reply-To: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> References: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> Message-ID: From a build point of view this looks good. /Erik On 2017-09-06 06:17, Weijun Wang wrote: > Hi All > > Please review the change, which spans to root, jdk and langtools repos. > > http://cr.openjdk.java.net/~weijun/8148371/ > > I've searched for the "policytool" word in the whole jdk10/jdk10 forests, removed all files having the word inside the path name, and remove almost all occurrences of the word in other places. > > The exceptions are: > > 1. Two files with the jdk8 word in file name. I assume I should not touch them. Please advise me. > > jdk/src/java.base/share/classes/jdk/internal/module/jdk8_packages.dat: > 1288 sun.security.tools.jarsigner > 1289 sun.security.tools.keytool > 1290: sun.security.tools.policytool > 1291 sun.security.util > 1292 sun.security.validator > > langtools/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/resources/jdk8_internals.txt: > 977 sun.security.tools.jarsigner > 978 sun.security.tools.keytool > 979: sun.security.tools.policytool > 980 sun.security.util > 981 sun.security.validator > > 2. A swing test containing a full JDK 1.4.2 README text. Keep unchanged. > > jdk/test/javax/swing/JTextArea/4697612/bug4697612.txt: > 122 bin/ktab and jre/bin/ktab > 123 Kerberos key table manager > 124: bin/policytool and jre/bin/policytool > 125 Policy File Creation and Management Tool > 126 bin/orbd and jre/bin/orbd > > 3. A manual test on what resource string are used. It is based on the pre-module source layout and needs to be updated anyway. Keep unchanged this time. https://bugs.openjdk.java.net/browse/JDK-8187265 filed. > > jdk/test/sun/security/util/Resources/NewResourcesNames.java: > 62 "sun/security/tools/jarsigner/Resources.java", > 63 "sun/security/tools/keytool/Resources.java", > 64: "sun/security/tools/policytool/Resources.java", > 65 "sun/security/util/Resources.java", > 66 "sun/security/util/AuthResources.java", > .. > 103 // > 104 // which is mismatch. There are only two such special cases list above. > 105: // For KeyTool, there are 3 calls for showing help. For PolicyTool, 3 > 106 // for name prefixed with POLICY. They are covered in the two special > 107 // cases above. > > There are some Japanese man pages containing the word. I've filed another bug (https://bugs.openjdk.java.net/browse/JDK-8187262) on it. > > Thanks > Max > From Alan.Bateman at oracle.com Wed Sep 6 09:24:22 2017 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Sep 2017 10:24:22 +0100 Subject: RFR 8148371: Remove policytool In-Reply-To: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> References: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> Message-ID: On 06/09/2017 05:17, Weijun Wang wrote: > Hi All > > Please review the change, which spans to root, jdk and langtools repos. > > http://cr.openjdk.java.net/~weijun/8148371/ > > I've searched for the "policytool" word in the whole jdk10/jdk10 forests, removed all files having the word inside the path name, and remove almost all occurrences of the word in other places. > This looks good, the only change that I'm not sure about is the change to ct.properties as it may be used when compiling to older releases. Someone on compiler-dev should be able to help you on that. -Alan From sean.mullan at oracle.com Wed Sep 6 20:23:47 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Wed, 6 Sep 2017 16:23:47 -0400 Subject: RFR 8148371: Remove policytool In-Reply-To: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> References: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> Message-ID: <65e41154-26ee-6a45-7345-5547a50b9346@oracle.com> The jdk changes look fine to me. --Sean On 9/6/17 12:17 AM, Weijun Wang wrote: > Hi All > > Please review the change, which spans to root, jdk and langtools repos. > > http://cr.openjdk.java.net/~weijun/8148371/ > > I've searched for the "policytool" word in the whole jdk10/jdk10 forests, removed all files having the word inside the path name, and remove almost all occurrences of the word in other places. > > The exceptions are: > > 1. Two files with the jdk8 word in file name. I assume I should not touch them. Please advise me. > > jdk/src/java.base/share/classes/jdk/internal/module/jdk8_packages.dat: > 1288 sun.security.tools.jarsigner > 1289 sun.security.tools.keytool > 1290: sun.security.tools.policytool > 1291 sun.security.util > 1292 sun.security.validator > > langtools/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/resources/jdk8_internals.txt: > 977 sun.security.tools.jarsigner > 978 sun.security.tools.keytool > 979: sun.security.tools.policytool > 980 sun.security.util > 981 sun.security.validator > > 2. A swing test containing a full JDK 1.4.2 README text. Keep unchanged. > > jdk/test/javax/swing/JTextArea/4697612/bug4697612.txt: > 122 bin/ktab and jre/bin/ktab > 123 Kerberos key table manager > 124: bin/policytool and jre/bin/policytool > 125 Policy File Creation and Management Tool > 126 bin/orbd and jre/bin/orbd > > 3. A manual test on what resource string are used. It is based on the pre-module source layout and needs to be updated anyway. Keep unchanged this time. https://bugs.openjdk.java.net/browse/JDK-8187265 filed. > > jdk/test/sun/security/util/Resources/NewResourcesNames.java: > 62 "sun/security/tools/jarsigner/Resources.java", > 63 "sun/security/tools/keytool/Resources.java", > 64: "sun/security/tools/policytool/Resources.java", > 65 "sun/security/util/Resources.java", > 66 "sun/security/util/AuthResources.java", > .. > 103 // > 104 // which is mismatch. There are only two such special cases list above. > 105: // For KeyTool, there are 3 calls for showing help. For PolicyTool, 3 > 106 // for name prefixed with POLICY. They are covered in the two special > 107 // cases above. > > There are some Japanese man pages containing the word. I've filed another bug (https://bugs.openjdk.java.net/browse/JDK-8187262) on it. > > Thanks > Max > From weijun.wang at oracle.com Thu Sep 7 00:07:07 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Thu, 7 Sep 2017 08:07:07 +0800 Subject: RFR 8186884: Test native KDC, Java krb5 lib, and native krb5 lib in one test Message-ID: Please take a review at http://cr.openjdk.java.net/~weijun/8186884/webrev.00/ BasicProc.java is enhanced to use a native JGSS provider, and KDC.java is enhanced to start (not use) a native KDC. For example, you would be able to test interop among Java JGSS, native JGSS (with MIT krb5) and Heimdal KDC with jtreg -Dnative.krb5.lib=/usr/local/krb5/lib/libgssapi_krb5.so \ -Dnative.kdc.path=/usr/local/heimdal \ test/sun/security/krb5/auto/BasicProc.java Without those 2 new system properties, it behaves like before, i.e. Java GSS on the embedded KDC. Another change in Context.java. Instead of using shared states to provide username and password when doing a krb5 login, a callback handler is used. This is considered more common. An extra permission is needed to read the default username (though I think this can coded as optional). Thanks Max From sha.jiang at oracle.com Thu Sep 7 05:52:21 2017 From: sha.jiang at oracle.com (sha.jiang at oracle.com) Date: Thu, 7 Sep 2017 13:52:21 +0800 Subject: RFR[10] 8186057: TLS interoperability testing between different Java versions Message-ID: Hi, Please review this test for checking the interop compatibility on JSSE among different JDK releases (from 6 to 10). It covers the cases, like handshake, data exchange, client authentication and APLN, on all TLS versions (if possible). And the selected TLS cipher suites are: TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA and TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA. For more details, please look though the README. Webrev: http://cr.openjdk.java.net/~jjiang/8186057/webrev.00 Issue: https://bugs.openjdk.java.net/browse/JDK-8186057 Best regards, John Jiang From artem.smotrakov at oracle.com Thu Sep 7 06:32:23 2017 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Thu, 7 Sep 2017 09:32:23 +0300 Subject: RFR[10] 8186057: TLS interoperability testing between different Java versions In-Reply-To: References: Message-ID: <12833c7c-3b34-3d8a-77ee-9299e8cca88d@oracle.com> Hi John, Thanks for starting working on this. I believe this tool is going to be very helpful. Let me skip some coding comments for a while, I have a couple of comments about the design. The main idea is that it should cover as many cases as it can. Even if it might look a bit redundant, I believe it is actually not redundant because we basically never know what's going to break down. 1. At the moment, you define cases in Compatibility.java 59 private static final String[] CASE_TYPES = new String[] { 60 Utils.CASE_HANDSHAKE, 61 Utils.CASE_DATA_EXCHANGE, 62 Utils.CASE_ALPN }; I believe there are much more many case which we can implement later. But this approach doesn't looks flexible. For example, let's consider that we'd like to use a couple of extensions in the same time. Let's say we'd like to use SNI and ALPN. It seems like we have to create a separate case for that, and add it manually to the array. Would it be possible to generate cases automatically? For example, you can define parameters of TLS connection, for example: - handshake type: full, session-resumption - use ALPN: true, false - use SNI: true, false - use client auth: true, false, etc Then you can build a Cartesian product of all parameters. Client and sever should take parameters, and say if they support all of the or not (for example, JDK 6 might not support all features that JDK 9 does). If it does, client and server can configure themselves with those parameters, and start handshaking. 2. With the approach above, it might be possible to avoid those nested loops: 93 for (String caseType : CASE_TYPES) { 94 for (String protocol : PROTOCOLS) { 95 for (String cipherSuite : CIPHER_SUITES) { 96 for (JdkInfo serverJdk : jdkInfos) { 3. I believe we should cover all supported cipher suites. Test run is going to take more time, but it think it's okay. Another option is to introduce two modes: - light: run the test with a couple of cipher suites - heavy: run the test with all supported cipher suites A couple of comments about the code: - What's the reason of using disabled SHA1 and RSA with 1024-bit keys? You might use stronger algorithms and key sizes, and avoid modifying java.security file - Please use try-with-resources if possible (files, sockets, etc) Artem On 09/07/2017 08:52 AM, sha.jiang at oracle.com wrote: > Hi, > Please review this test for checking the interop compatibility on JSSE > among different JDK releases (from 6 to 10). > It covers the cases, like handshake, data exchange, client > authentication and APLN, on all TLS versions (if possible). > And the selected TLS cipher suites are: TLS_RSA_WITH_AES_128_CBC_SHA, > TLS_DHE_DSS_WITH_AES_128_CBC_SHA and > TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA. > For more details, please look though the README. > > Webrev: http://cr.openjdk.java.net/~jjiang/8186057/webrev.00 > Issue: https://bugs.openjdk.java.net/browse/JDK-8186057 > > Best regards, > John Jiang > From artem.smotrakov at oracle.com Thu Sep 7 07:29:12 2017 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Thu, 7 Sep 2017 10:29:12 +0300 Subject: RFR 8186884: Test native KDC, Java krb5 lib, and native krb5 lib in one test In-Reply-To: References: Message-ID: Hi Max, In general, looks fine to me. Below are a couple of comments you might want to address. 1. BasicProc.java, it might be better to use named constants for parameters for once() method. That would make it easier to understand what each particular onse() call does + once(true, true, true); // pure java + if (LIBNAME != null) { + // save a cache for client + Context.fromUserPass(USER, PASS, false) + .ccache("ccache.base"); + + once(false, false, false); // fail fast for all native + once(false, true, true); + once(false, true, false); + once(false, false, true); + + once(true, true, false); + once(true, false, true); + once(true, false, false); Enums may help, and might make it simper as well: + // Just a marker for which test case is finished + String label = (jc?"j":"n") + (js?"j":"n") + (jb?"j":"n"); 2. BasicProc.java, could you please add an exception message? + if (!Arrays.equals(msg, msg2)) { + throw new Exception(); + } + break; 3. BasicProc.java, should the test do some cleanup then? + Files.copy(Paths.get("ccache.base"), Paths.get("ccache." + label)); Artem On 09/07/2017 03:07 AM, Weijun Wang wrote: > Please take a review at > > http://cr.openjdk.java.net/~weijun/8186884/webrev.00/ > > BasicProc.java is enhanced to use a native JGSS provider, and KDC.java is enhanced to start (not use) a native KDC. For example, you would be able to test interop among Java JGSS, native JGSS (with MIT krb5) and Heimdal KDC with > > jtreg -Dnative.krb5.lib=/usr/local/krb5/lib/libgssapi_krb5.so \ > -Dnative.kdc.path=/usr/local/heimdal \ > test/sun/security/krb5/auto/BasicProc.java > > Without those 2 new system properties, it behaves like before, i.e. Java GSS on the embedded KDC. > > Another change in Context.java. Instead of using shared states to provide username and password when doing a krb5 login, a callback handler is used. This is considered more common. An extra permission is needed to read the default username (though I think this can coded as optional). > > Thanks > Max > From sha.jiang at oracle.com Thu Sep 7 07:52:37 2017 From: sha.jiang at oracle.com (sha.jiang at oracle.com) Date: Thu, 7 Sep 2017 15:52:37 +0800 Subject: RFR[10] 8186057: TLS interoperability testing between different Java versions In-Reply-To: <12833c7c-3b34-3d8a-77ee-9299e8cca88d@oracle.com> References: <12833c7c-3b34-3d8a-77ee-9299e8cca88d@oracle.com> Message-ID: Hi Artem, Thanks for your comments! Please see inline. On 07/09/2017 14:32, Artem Smotrakov wrote: > Hi John, > > Thanks for starting working on this. I believe this tool is going to > be very helpful. > > Let me skip some coding comments for a while, I have a couple of > comments about the design. The main idea is that it should cover as > many cases as it can. Even if it might look a bit redundant, I believe > it is actually not redundant because we basically never know what's > going to break down. > > 1. At the moment, you define cases in Compatibility.java > > ? 59???? private static final String[] CASE_TYPES = new String[] { > ? 60???????????? Utils.CASE_HANDSHAKE, > ? 61???????????? Utils.CASE_DATA_EXCHANGE, > ? 62???????????? Utils.CASE_ALPN }; > > I believe there are much more many case which we can implement later. > But this approach doesn't looks flexible. For example, let's consider > that we'd like to use a couple of extensions in the same time. Let's > say we'd like to use SNI and ALPN. It seems like we have to create a > separate case for that, and add it manually to the array. Would it be > possible to generate cases automatically? For example, you can define > parameters of TLS connection, for example: > - handshake type: full, session-resumption > - use ALPN: true, false > - use SNI: true, false > - use client auth: true, false, > etc > > Then you can build a Cartesian product of all parameters. Client and > sever should take parameters, and say if they support all of the or > not (for example, JDK 6 might not support all features that JDK 9 > does). If it does, client and server can configure themselves with > those parameters, and start handshaking. Frankly, what case combinations would be tested is a problem. Currently, I don't expand the combinations too much. For example, I don't make combinations for data exchange and ALPN, and client auth always be enabled. That's why I don't use complex data structures. If the test requirements or case combinations could be confirmed, the data structures would be enriched. > 2. With the approach above, it might be possible to avoid those nested > loops: > > ? 93?????? for (String caseType : CASE_TYPES) { > ? 94???????????? for (String protocol : PROTOCOLS) { > ? 95???????????????? for (String cipherSuite : CIPHER_SUITES) { > ? 96???????????????????? for (JdkInfo serverJdk : jdkInfos) { But, how to build the above Cartesian product automatically (not by manual)? > 3. I believe we should cover all supported cipher suites. Test run is > going to take more time, but it think it's okay. Another option is to > introduce two modes: > - light: run the test with a couple of cipher suites > - heavy: run the test with all supported cipher suites I hesitate to do that. Different JDK builds and TLS versions supports different cipher suites. And many cipher suites use the same algorithms on key exchange, signature, encryption/decryption and authentication, then is it necessary to check each of them? Although we don't know "what's going to break down", I'm not sure a single test could check every point on TLS communication. > A couple of comments about the code: > - What's the reason of using disabled SHA1 and RSA with 1024-bit keys? > You might use stronger algorithms and key sizes, and avoid modifying > java.security file Different JDK builds possibly support different algorithms and have different restrictions on such algorithms and key sizes. Weaker or stronger, that is relative. And this point should not be a focus of the test. So, I just pick the ones that are common in history, and provides an alternative java.security file to avoid possible broken. > - Please use try-with-resources if possible (files, sockets, etc) The test uses only JDK 6-supported language features, but try-with-resources is introduced by JDK 7. Best regards, John Jiang > > Artem > > On 09/07/2017 08:52 AM, sha.jiang at oracle.com wrote: >> Hi, >> Please review this test for checking the interop compatibility on >> JSSE among different JDK releases (from 6 to 10). >> It covers the cases, like handshake, data exchange, client >> authentication and APLN, on all TLS versions (if possible). >> And the selected TLS cipher suites are: TLS_RSA_WITH_AES_128_CBC_SHA, >> TLS_DHE_DSS_WITH_AES_128_CBC_SHA and >> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA. >> For more details, please look though the README. >> >> Webrev: http://cr.openjdk.java.net/~jjiang/8186057/webrev.00 >> Issue: https://bugs.openjdk.java.net/browse/JDK-8186057 >> >> Best regards, >> John Jiang >> > > From artem.smotrakov at oracle.com Thu Sep 7 08:07:01 2017 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Thu, 7 Sep 2017 11:07:01 +0300 Subject: RFR[10] 8186057: TLS interoperability testing between different Java versions In-Reply-To: References: <12833c7c-3b34-3d8a-77ee-9299e8cca88d@oracle.com> Message-ID: <05d9da98-205a-887d-ad88-c88b15832947@oracle.com> Hi John, Please see inline. On 09/07/2017 10:52 AM, sha.jiang at oracle.com wrote: >> >> Then you can build a Cartesian product of all parameters. Client and >> sever should take parameters, and say if they support all of the or >> not (for example, JDK 6 might not support all features that JDK 9 >> does). If it does, client and server can configure themselves with >> those parameters, and start handshaking. > Frankly, what case combinations would be tested is a problem. > Currently, I don't expand the combinations too much. > For example, I don't make combinations for data exchange and ALPN, and > client auth always be enabled. > That's why I don't use complex data structures. > If the test requirements or case combinations could be confirmed, the > data structures would be enriched. That's fine to start with limited amount of parameters, but I believe we can design the test to be able to extend it easily in future. That's my main point here. > >> 2. With the approach above, it might be possible to avoid those >> nested loops: >> >> 93 for (String caseType : CASE_TYPES) { >> 94 for (String protocol : PROTOCOLS) { >> 95 for (String cipherSuite : CIPHER_SUITES) { >> 96 for (JdkInfo serverJdk : jdkInfos) { > But, how to build the above Cartesian product automatically (not by > manual)? "automatically" means writing some code of course, but I believe we don't have to define such an array of cases manually. > >> 3. I believe we should cover all supported cipher suites. Test run is >> going to take more time, but it think it's okay. Another option is to >> introduce two modes: >> - light: run the test with a couple of cipher suites >> - heavy: run the test with all supported cipher suites > I hesitate to do that. > Different JDK builds and TLS versions supports different cipher suites. That's fine. If you test JDK 6 against 9, you can use only cipher suites which are supported by 6. > And many cipher suites use the same algorithms on key exchange, > signature, encryption/decryption and authentication, then is it > necessary to check each of them? I think it would be worth to be able to do that at least. Again, we never know what can go wrong. > Although we don't know "what's going to break down", I'm not sure a > single test could check every point on TLS communication. Right, we can't test everything, but we can to our best. If we know that the test can be easily enhanced in this way, it's worth to do that. >> A couple of comments about the code: >> - What's the reason of using disabled SHA1 and RSA with 1024-bit >> keys? You might use stronger algorithms and key sizes, and avoid >> modifying java.security file > Different JDK builds possibly support different algorithms and have > different restrictions on such algorithms and key sizes. Weaker or > stronger, that is relative. And this point should not be a focus of > the test. > So, I just pick the ones that are common in history, and provides an > alternative java.security file to avoid possible broken. That makes sense. > >> - Please use try-with-resources if possible (files, sockets, etc) > The test uses only JDK 6-supported language features, but > try-with-resources is introduced by JDK 7. Here we come to another issue. I believe that it would be good to use write clients for all JDK versions. If you use the same code for all Java versions, that means that you use only JDK 6 API. Let's consider the following: - we want to test 8 vs 9 - 8 and 9 may have some API (or just functionality) which is not supported by 6 (for example, ALPN, SNI) If you write code which is compatible with 6, then you can use API from newer versions, but we still may want to test it. Artem > > Best regards, > John Jiang >> >> Artem >> >> On 09/07/2017 08:52 AM, sha.jiang at oracle.com wrote: >>> Hi, >>> Please review this test for checking the interop compatibility on >>> JSSE among different JDK releases (from 6 to 10). >>> It covers the cases, like handshake, data exchange, client >>> authentication and APLN, on all TLS versions (if possible). >>> And the selected TLS cipher suites are: >>> TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA and >>> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA. >>> For more details, please look though the README. >>> >>> Webrev: http://cr.openjdk.java.net/~jjiang/8186057/webrev.00 >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8186057 >>> >>> Best regards, >>> John Jiang >>> >> >> > From sha.jiang at oracle.com Thu Sep 7 08:24:10 2017 From: sha.jiang at oracle.com (sha.jiang at oracle.com) Date: Thu, 7 Sep 2017 16:24:10 +0800 Subject: RFR[10] 8186057: TLS interoperability testing between different Java versions In-Reply-To: <05d9da98-205a-887d-ad88-c88b15832947@oracle.com> References: <12833c7c-3b34-3d8a-77ee-9299e8cca88d@oracle.com> <05d9da98-205a-887d-ad88-c88b15832947@oracle.com> Message-ID: <7fcc23e3-dd4e-d127-0cc2-f4990fb5d665@oracle.com> Hi Artem, On 07/09/2017 16:07, Artem Smotrakov wrote: >> >>> - Please use try-with-resources if possible (files, sockets, etc) >> The test uses only JDK 6-supported language features, but >> try-with-resources is introduced by JDK 7. > Here we come to another issue. I believe that it would be good to use > write clients for all JDK versions. If you use the same code for all > Java versions, that means that you use only JDK 6 API. Let's consider > the following: > - we want to test 8 vs 9 > - 8 and 9 may have some API (or just functionality) which is not > supported by 6 (for example, ALPN, SNI) > > If you write code which is compatible with 6, then you can use API > from newer versions, but we still may want to test it. In fact, the current test has already covered ALPN, though only JDK 9 and 10 support the associated methods, like SSLParameters.getApplicationProtocols(). ALPN-associated case combinations are ignored if a JDK doesn't support this feature. Exactly, JdkUtils checks if a specific JDK build contains method SSLParameters.getApplicationProtocols(). I think the same approach could be applied for SNI in the future. Best regards, John Jiang > > Artem >> >> Best regards, >> John Jiang >>> >>> Artem >>> >>> On 09/07/2017 08:52 AM, sha.jiang at oracle.com wrote: >>>> Hi, >>>> Please review this test for checking the interop compatibility on >>>> JSSE among different JDK releases (from 6 to 10). >>>> It covers the cases, like handshake, data exchange, client >>>> authentication and APLN, on all TLS versions (if possible). >>>> And the selected TLS cipher suites are: >>>> TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA and >>>> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA. >>>> For more details, please look though the README. >>>> >>>> Webrev: http://cr.openjdk.java.net/~jjiang/8186057/webrev.00 >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8186057 >>>> >>>> Best regards, >>>> John Jiang >>>> >>> >>> >> > > From artem.smotrakov at oracle.com Thu Sep 7 08:28:48 2017 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Thu, 7 Sep 2017 11:28:48 +0300 Subject: RFR[10] 8186057: TLS interoperability testing between different Java versions In-Reply-To: <7fcc23e3-dd4e-d127-0cc2-f4990fb5d665@oracle.com> References: <12833c7c-3b34-3d8a-77ee-9299e8cca88d@oracle.com> <05d9da98-205a-887d-ad88-c88b15832947@oracle.com> <7fcc23e3-dd4e-d127-0cc2-f4990fb5d665@oracle.com> Message-ID: <894241da-b4d7-46ce-2962-c3e6e9a2227b@oracle.com> In case of SNI, SSLParemeters.setServerNames() method (and others related to SNI) was introduced in 8. I don't think the code which use this method can be compiled with 6 and 7. But if you want to test SNI with 8+, you need to call them. https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLParameters.html#setServerNames-java.util.List- Artem On 09/07/2017 11:24 AM, sha.jiang at oracle.com wrote: > Hi Artem, > > On 07/09/2017 16:07, Artem Smotrakov wrote: >>> >>>> - Please use try-with-resources if possible (files, sockets, etc) >>> The test uses only JDK 6-supported language features, but >>> try-with-resources is introduced by JDK 7. >> Here we come to another issue. I believe that it would be good to use >> write clients for all JDK versions. If you use the same code for all >> Java versions, that means that you use only JDK 6 API. Let's consider >> the following: >> - we want to test 8 vs 9 >> - 8 and 9 may have some API (or just functionality) which is not >> supported by 6 (for example, ALPN, SNI) >> >> If you write code which is compatible with 6, then you can use API >> from newer versions, but we still may want to test it. > In fact, the current test has already covered ALPN, though only JDK 9 > and 10 support the associated methods, like > SSLParameters.getApplicationProtocols(). > ALPN-associated case combinations are ignored if a JDK doesn't support > this feature. Exactly, JdkUtils checks if a specific JDK build > contains method SSLParameters.getApplicationProtocols(). > I think the same approach could be applied for SNI in the future. > > Best regards, > John Jiang >> >> Artem >>> >>> Best regards, >>> John Jiang >>>> >>>> Artem >>>> >>>> On 09/07/2017 08:52 AM, sha.jiang at oracle.com wrote: >>>>> Hi, >>>>> Please review this test for checking the interop compatibility on >>>>> JSSE among different JDK releases (from 6 to 10). >>>>> It covers the cases, like handshake, data exchange, client >>>>> authentication and APLN, on all TLS versions (if possible). >>>>> And the selected TLS cipher suites are: >>>>> TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA and >>>>> TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA. >>>>> For more details, please look though the README. >>>>> >>>>> Webrev: http://cr.openjdk.java.net/~jjiang/8186057/webrev.00 >>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8186057 >>>>> >>>>> Best regards, >>>>> John Jiang >>>>> >>>> >>>> >>> >> >> > From sha.jiang at oracle.com Thu Sep 7 10:47:15 2017 From: sha.jiang at oracle.com (sha.jiang at oracle.com) Date: Thu, 7 Sep 2017 18:47:15 +0800 Subject: RFR[10] 8186057: TLS interoperability testing between different Java versions In-Reply-To: <894241da-b4d7-46ce-2962-c3e6e9a2227b@oracle.com> References: <12833c7c-3b34-3d8a-77ee-9299e8cca88d@oracle.com> <05d9da98-205a-887d-ad88-c88b15832947@oracle.com> <7fcc23e3-dd4e-d127-0cc2-f4990fb5d665@oracle.com> <894241da-b4d7-46ce-2962-c3e6e9a2227b@oracle.com> Message-ID: Hi Artem, On 07/09/2017 16:28, Artem Smotrakov wrote: > In case of SNI, SSLParemeters.setServerNames() method (and others > related to SNI) was introduced in 8. I don't think the code which use > this method can be compiled with 6 and 7. All of Java source are compiled by a JDK 10 build, which is specified by jtreg option "-jdk". Other testing JDKs, including 6 and 7, just use the .class files on runtime. That's why the below line exists in Compatibility.java, 32? * @compile -source 1.6 -target 1.6 Server.java Client.java JdkUtils.java > > But if you want to test SNI with 8+, you need to call them. > > https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLParameters.html#setServerNames-java.util.List- > If a feature, like ALPN and SNI, is not supported, such method would not be called on runtime. Best regards, John Jiang > > Artem > > On 09/07/2017 11:24 AM, sha.jiang at oracle.com wrote: >> Hi Artem, >> >> On 07/09/2017 16:07, Artem Smotrakov wrote: >>>> >>>>> - Please use try-with-resources if possible (files, sockets, etc) >>>> The test uses only JDK 6-supported language features, but >>>> try-with-resources is introduced by JDK 7. >>> Here we come to another issue. I believe that it would be good to >>> use write clients for all JDK versions. If you use the same code for >>> all Java versions, that means that you use only JDK 6 API. Let's >>> consider the following: >>> - we want to test 8 vs 9 >>> - 8 and 9 may have some API (or just functionality) which is not >>> supported by 6 (for example, ALPN, SNI) >>> >>> If you write code which is compatible with 6, then you can use API >>> from newer versions, but we still may want to test it. >> In fact, the current test has already covered ALPN, though only JDK 9 >> and 10 support the associated methods, like >> SSLParameters.getApplicationProtocols(). >> ALPN-associated case combinations are ignored if a JDK doesn't >> support this feature. Exactly, JdkUtils checks if a specific JDK >> build contains method SSLParameters.getApplicationProtocols(). >> I think the same approach could be applied for SNI in the future. >> >> Best regards, >> John Jiang >>> >>> Artem >>>> >>>> Best regards, >>>> John Jiang >>>>> >>>>> Artem >>>>> >>>>> On 09/07/2017 08:52 AM, sha.jiang at oracle.com wrote: >>>>>> Hi, >>>>>> Please review this test for checking the interop compatibility on >>>>>> JSSE among different JDK releases (from 6 to 10). >>>>>> It covers the cases, like handshake, data exchange, client >>>>>> authentication and APLN, on all TLS versions (if possible). >>>>>> And the selected TLS cipher suites are: >>>>>> TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA >>>>>> and TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA. >>>>>> For more details, please look though the README. >>>>>> >>>>>> Webrev: http://cr.openjdk.java.net/~jjiang/8186057/webrev.00 >>>>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8186057 >>>>>> >>>>>> Best regards, >>>>>> John Jiang >>>>>> >>>>> >>>>> >>>> >>> >>> >> > > From weijun.wang at oracle.com Thu Sep 7 12:39:53 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Thu, 7 Sep 2017 20:39:53 +0800 Subject: RFR 8186884: Test native KDC, Java krb5 lib, and native krb5 lib in one test In-Reply-To: References: Message-ID: <9A17CD23-B9D0-484E-8970-21241ACAE123@oracle.com> Updated at http://cr.openjdk.java.net/~weijun/8186884/webrev.01/ Now the libraries can be more freely combined, so you can test interop between one native library and another one: jtreg -Dnative.krb5.libs=j=,n=,m=lib1.so,h=lib2.so BasicProc.java More comments inline below. > On Sep 7, 2017, at 3:29 PM, Artem Smotrakov wrote: > > Hi Max, > > In general, looks fine to me. Below are a couple of comments you might want to address. > > 1. BasicProc.java, it might be better to use named constants for parameters for once() method. That would make it easier to understand what each particular onse() call does I am passing in label and library names now. > > 2. BasicProc.java, could you please add an exception message? > > + if (!Arrays.equals(msg, msg2)) { > + throw new Exception(); > + } > + break; Fixed. > > 3. BasicProc.java, should the test do some cleanup then? > > + Files.copy(Paths.get("ccache.base"), Paths.get("ccache." + label)); Nowadays I prefer to let jtreg do the cleanup/retain. In fact, I am able to find a KDC.java bug by saving the ccache, where the incoming service ticket is invalid and not saved into the ccache. Thanks Max > > Artem > > On 09/07/2017 03:07 AM, Weijun Wang wrote: >> Please take a review at >> >> http://cr.openjdk.java.net/~weijun/8186884/webrev.00/ >> >> BasicProc.java is enhanced to use a native JGSS provider, and KDC.java is enhanced to start (not use) a native KDC. For example, you would be able to test interop among Java JGSS, native JGSS (with MIT krb5) and Heimdal KDC with >> >> jtreg -Dnative.krb5.lib=/usr/local/krb5/lib/libgssapi_krb5.so \ >> -Dnative.kdc.path=/usr/local/heimdal \ >> test/sun/security/krb5/auto/BasicProc.java >> >> Without those 2 new system properties, it behaves like before, i.e. Java GSS on the embedded KDC. >> >> Another change in Context.java. Instead of using shared states to provide username and password when doing a krb5 login, a callback handler is used. This is considered more common. An extra permission is needed to read the default username (though I think this can coded as optional). >> >> Thanks >> Max >> > From sean.coffey at oracle.com Thu Sep 7 17:25:50 2017 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 7 Sep 2017 18:25:50 +0100 Subject: [10] RFR 8186654 : Poor quality of sun.security.util.Cache.EqualByteArray.hashCode() In-Reply-To: References: Message-ID: <0eb59fc1-83a9-622a-5cbb-498d6b1fbeb3@oracle.com> Looks good to me Ivan. regards, Sean. On 23/08/2017 21:54, Ivan Gerasimov wrote: > Hello! > > An auxiliary class EqualByteArray implements hashCode() in a way that > small changes to the content do not change the hash value. > > It is proposed to reuse Arrays.hashCode(byte[]) for the hash code > calculations. > > Also, the private int hash is made non-volatile, as it removes > accessing to a volatile variable in the relatively frequently executed > code for the price of relatively less likely chance of a double hash > code calculation.? (For reference, java.lang.String uses the same > approach.) > > Would you please help review the fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8186654 > WEBREV: http://cr.openjdk.java.net/~igerasim/8186654/00/webrev/ > From ivan.gerasimov at oracle.com Thu Sep 7 17:28:37 2017 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 7 Sep 2017 10:28:37 -0700 Subject: [10] RFR 8186654 : Poor quality of sun.security.util.Cache.EqualByteArray.hashCode() In-Reply-To: <0eb59fc1-83a9-622a-5cbb-498d6b1fbeb3@oracle.com> References: <0eb59fc1-83a9-622a-5cbb-498d6b1fbeb3@oracle.com> Message-ID: <8daa8909-0102-9d6c-376f-c928f4ab9f7f@oracle.com> On 9/7/17 10:25 AM, Se?n Coffey wrote: > Looks good to me Ivan. > Thanks Se?n! With kind regards, Ivan > regards, > Sean. > > On 23/08/2017 21:54, Ivan Gerasimov wrote: >> Hello! >> >> An auxiliary class EqualByteArray implements hashCode() in a way that >> small changes to the content do not change the hash value. >> >> It is proposed to reuse Arrays.hashCode(byte[]) for the hash code >> calculations. >> >> Also, the private int hash is made non-volatile, as it removes >> accessing to a volatile variable in the relatively frequently >> executed code for the price of relatively less likely chance of a >> double hash code calculation. (For reference, java.lang.String uses >> the same approach.) >> >> Would you please help review the fix? >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8186654 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8186654/00/webrev/ >> > > -- With kind regards, Ivan Gerasimov From sean.mullan at oracle.com Thu Sep 7 19:53:51 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 7 Sep 2017 15:53:51 -0400 Subject: RFR 8186931: jdk.security.jarsigner package is missing package summary In-Reply-To: <667126DE-EB9C-4557-BCB5-0BC050FFC828@oracle.com> References: <8FC7DB36-00EC-475E-82D1-F2A48FDD03C8@oracle.com> <667126DE-EB9C-4557-BCB5-0BC050FFC828@oracle.com> Message-ID: <4b0dfb4d-b3c4-a50f-9129-d3c6741941b4@oracle.com> Looks good. --Sean On 9/5/17 7:16 PM, Weijun Wang wrote: > The package info for com.sun.security.jgss in jdk.security.jgss is also missing. The updated change looks like this (I omit the copyright header): > > src/jdk.jartool/share/classes/jdk/security/jarsigner/package-info.java: > > +/** > + * This package defines APIs for signing jar files. > + */ > +package jdk.security.jarsigner; > > > src/jdk.jartool/share/classes/module-info.java: > + * This module also defines APIs for signing JAR files. > > > src/jdk.security.jgss/share/classes/com/sun/security/jgss/package-info.java: > > +/** > + * This package defines classes and interfaces for the JDK extensions > + * to the GSS-API. > + */ > package com.sun.security.jgss; > > > src/jdk.security.jgss/share/classes/module-info.java: > > /** > - * Defines Java extensions to the GSS-API and an implementation of the SASL > + * Defines JDK extensions to the GSS-API and an implementation of the SASL > * GSSAPI mechanism. > > Thanks > Max > >> On Aug 31, 2017, at 11:10 PM, Sean Mullan wrote: >> >> On 8/31/17 4:49 AM, Weijun Wang wrote: >> >>> /** >>> * This package contains the {@link jdk.security.jarsigner.JarSigner} API, >>> * which backs the signing function of the {@code jarsigner} tool. >>> */ >> >> I think you should say something about that this API can be used to sign JAR files. The fact that it is also used by jarsigner seems less important to mention in the package description. I suggest changing this to: >> >> "This package contains the {@link jdk.security.jarsigner.JarSigner} API which can be used to sign jar files." >> >> --Sean > From weijun.wang at oracle.com Fri Sep 8 00:37:13 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 8 Sep 2017 08:37:13 +0800 Subject: RFR 8186831: Kerberos ignores PA-DATA with a non-null s2kparams In-Reply-To: References: Message-ID: Ping again. > On Aug 28, 2017, at 11:12 PM, Weijun Wang wrote: > > Please review the fix at > > http://cr.openjdk.java.net/~weijun/8186831/webrev.00/ > > This is an old bug that is about to bite us soon. > > Thanks > Max > From Xuelei.Fan at Oracle.Com Fri Sep 8 01:30:58 2017 From: Xuelei.Fan at Oracle.Com (Xuelei Fan) Date: Fri, 8 Sep 2017 09:30:58 +0800 Subject: RFR 8186831: Kerberos ignores PA-DATA with a non-null s2kparams In-Reply-To: References: Message-ID: Looks fine to me. A minor comment about the method name "isNew", i'm not very sure what does it means. I would add comments about what the "new" refers to. Xuelei > On Aug 28, 2017, at 11:12 PM, Weijun Wang wrote: > > Please review the fix at > > http://cr.openjdk.java.net/~weijun/8186831/webrev.00/ > > This is an old bug that is about to bite us soon. > > Thanks > Max > From weijun.wang at oracle.com Fri Sep 8 01:36:30 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 8 Sep 2017 09:36:30 +0800 Subject: RFR 8186831: Kerberos ignores PA-DATA with a non-null s2kparams In-Reply-To: References: Message-ID: <56279834-C05A-439D-BB37-091C20CB1808@oracle.com> According to https://tools.ietf.org/html/rfc4120#section-3.1.3: A "newer" enctype is any enctype first officially specified concurrently with or subsequent to the issue of this RFC. The enctypes DES, 3DES, or RC4 and any defined in [RFC1510] are not "newer" enctypes. I'll add a link and rename it to isNewer(). Thanks Max > On Sep 8, 2017, at 9:30 AM, Xuelei Fan wrote: > > Looks fine to me. > > A minor comment about the method name "isNew", i'm not very sure what does it means. I would add comments about what the "new" refers to. > > Xuelei > >> On Aug 28, 2017, at 11:12 PM, Weijun Wang wrote: >> >> Please review the fix at >> >> http://cr.openjdk.java.net/~weijun/8186831/webrev.00/ >> >> This is an old bug that is about to bite us soon. >> >> Thanks >> Max >> > From weijun.wang at oracle.com Fri Sep 8 01:51:07 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 8 Sep 2017 09:51:07 +0800 Subject: RFR 8148371: Remove policytool In-Reply-To: References: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> Message-ID: <7FD840F2-15D3-40F1-B2A1-9F816E4A8BD0@oracle.com> > On Sep 6, 2017, at 5:24 PM, Alan Bateman wrote: > > > > On 06/09/2017 05:17, Weijun Wang wrote: >> Hi All >> >> Please review the change, which spans to root, jdk and langtools repos. >> >> http://cr.openjdk.java.net/~weijun/8148371/ >> >> I've searched for the "policytool" word in the whole jdk10/jdk10 forests, removed all files having the word inside the path name, and remove almost all occurrences of the word in other places. >> > This looks good, the only change that I'm not sure about is the change to ct.properties as it may be used when compiling to older releases. Someone on compiler-dev should be able to help you on that. Jon suggested reverting this change, offline. --Max > > -Alan From weijun.wang at oracle.com Fri Sep 8 03:19:14 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 8 Sep 2017 11:19:14 +0800 Subject: RFR 8186884: Test native KDC, Java krb5 lib, and native krb5 lib in one test In-Reply-To: <9A17CD23-B9D0-484E-8970-21241ACAE123@oracle.com> References: <9A17CD23-B9D0-484E-8970-21241ACAE123@oracle.com> Message-ID: <8C778238-1C3D-4C49-823F-E7C00A211879@oracle.com> Small update on http://cr.openjdk.java.net/~weijun/8186884/webrev.02. All files belong to a single Proc now have the same prefix so they appear together in file list. Thanks Max > On Sep 7, 2017, at 8:39 PM, Weijun Wang wrote: > > Updated at > > http://cr.openjdk.java.net/~weijun/8186884/webrev.01/ > > Now the libraries can be more freely combined, so you can test interop between one native library and another one: > > jtreg -Dnative.krb5.libs=j=,n=,m=lib1.so,h=lib2.so BasicProc.java > > > More comments inline below. > >> On Sep 7, 2017, at 3:29 PM, Artem Smotrakov wrote: >> >> Hi Max, >> >> In general, looks fine to me. Below are a couple of comments you might want to address. >> >> 1. BasicProc.java, it might be better to use named constants for parameters for once() method. That would make it easier to understand what each particular onse() call does > > I am passing in label and library names now. > >> >> 2. BasicProc.java, could you please add an exception message? >> >> + if (!Arrays.equals(msg, msg2)) { >> + throw new Exception(); >> + } >> + break; > > Fixed. > >> >> 3. BasicProc.java, should the test do some cleanup then? >> >> + Files.copy(Paths.get("ccache.base"), Paths.get("ccache." + label)); > > Nowadays I prefer to let jtreg do the cleanup/retain. In fact, I am able to find a KDC.java bug by saving the ccache, where the incoming service ticket is invalid and not saved into the ccache. > > Thanks > Max > >> >> Artem >> >> On 09/07/2017 03:07 AM, Weijun Wang wrote: >>> Please take a review at >>> >>> http://cr.openjdk.java.net/~weijun/8186884/webrev.00/ >>> >>> BasicProc.java is enhanced to use a native JGSS provider, and KDC.java is enhanced to start (not use) a native KDC. For example, you would be able to test interop among Java JGSS, native JGSS (with MIT krb5) and Heimdal KDC with >>> >>> jtreg -Dnative.krb5.lib=/usr/local/krb5/lib/libgssapi_krb5.so \ >>> -Dnative.kdc.path=/usr/local/heimdal \ >>> test/sun/security/krb5/auto/BasicProc.java >>> >>> Without those 2 new system properties, it behaves like before, i.e. Java GSS on the embedded KDC. >>> >>> Another change in Context.java. Instead of using shared states to provide username and password when doing a krb5 login, a callback handler is used. This is considered more common. An extra permission is needed to read the default username (though I think this can coded as optional). >>> >>> Thanks >>> Max >>> >> > From artem.smotrakov at oracle.com Fri Sep 8 12:38:20 2017 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Fri, 8 Sep 2017 15:38:20 +0300 Subject: RFR 8186884: Test native KDC, Java krb5 lib, and native krb5 lib in one test In-Reply-To: <8C778238-1C3D-4C49-823F-E7C00A211879@oracle.com> References: <9A17CD23-B9D0-484E-8970-21241ACAE123@oracle.com> <8C778238-1C3D-4C49-823F-E7C00A211879@oracle.com> Message-ID: Hi Max, Looks good to me. Below are a couple of minor comments you may want to address. No need a new webrev. Thanks! 1. Proc.java, better to use braces http://www.oracle.com/technetwork/java/javase/documentation/codeconventions-142311.html#449 2. If you already updated KDC.java, it may be good to use try-with-resources for streams in a couple of places. Artem On 09/08/2017 06:19 AM, Weijun Wang wrote: > Small update on http://cr.openjdk.java.net/~weijun/8186884/webrev.02. All files belong to a single Proc now have the same prefix so they appear together in file list. > > Thanks > Max > >> On Sep 7, 2017, at 8:39 PM, Weijun Wang wrote: >> >> Updated at >> >> http://cr.openjdk.java.net/~weijun/8186884/webrev.01/ >> >> Now the libraries can be more freely combined, so you can test interop between one native library and another one: >> >> jtreg -Dnative.krb5.libs=j=,n=,m=lib1.so,h=lib2.so BasicProc.java >> >> >> More comments inline below. >> >>> On Sep 7, 2017, at 3:29 PM, Artem Smotrakov wrote: >>> >>> Hi Max, >>> >>> In general, looks fine to me. Below are a couple of comments you might want to address. >>> >>> 1. BasicProc.java, it might be better to use named constants for parameters for once() method. That would make it easier to understand what each particular onse() call does >> I am passing in label and library names now. >> >>> 2. BasicProc.java, could you please add an exception message? >>> >>> + if (!Arrays.equals(msg, msg2)) { >>> + throw new Exception(); >>> + } >>> + break; >> Fixed. >> >>> 3. BasicProc.java, should the test do some cleanup then? >>> >>> + Files.copy(Paths.get("ccache.base"), Paths.get("ccache." + label)); >> Nowadays I prefer to let jtreg do the cleanup/retain. In fact, I am able to find a KDC.java bug by saving the ccache, where the incoming service ticket is invalid and not saved into the ccache. >> >> Thanks >> Max >> >>> Artem >>> >>> On 09/07/2017 03:07 AM, Weijun Wang wrote: >>>> Please take a review at >>>> >>>> http://cr.openjdk.java.net/~weijun/8186884/webrev.00/ >>>> >>>> BasicProc.java is enhanced to use a native JGSS provider, and KDC.java is enhanced to start (not use) a native KDC. For example, you would be able to test interop among Java JGSS, native JGSS (with MIT krb5) and Heimdal KDC with >>>> >>>> jtreg -Dnative.krb5.lib=/usr/local/krb5/lib/libgssapi_krb5.so \ >>>> -Dnative.kdc.path=/usr/local/heimdal \ >>>> test/sun/security/krb5/auto/BasicProc.java >>>> >>>> Without those 2 new system properties, it behaves like before, i.e. Java GSS on the embedded KDC. >>>> >>>> Another change in Context.java. Instead of using shared states to provide username and password when doing a krb5 login, a callback handler is used. This is considered more common. An extra permission is needed to read the default username (though I think this can coded as optional). >>>> >>>> Thanks >>>> Max >>>> From weijun.wang at oracle.com Mon Sep 11 08:07:16 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 11 Sep 2017 16:07:16 +0800 Subject: RFR 8186884: Test native KDC, Java krb5 lib, and native krb5 lib in one test In-Reply-To: References: <9A17CD23-B9D0-484E-8970-21241ACAE123@oracle.com> <8C778238-1C3D-4C49-823F-E7C00A211879@oracle.com> Message-ID: Sorry to update again. Might be because jdk10 is frozen now. http://cr.openjdk.java.net/~weijun/8186884/webrev.03 But changes are real: 1. kdc.supported.enctypes supported by native KDCs correctly. In fact, I am now able to set it to aes256-sha2 (unsupported in Java yet) and test for interop between native libs with a native KDC. 2. A KDC::kinit method is introduced which calls native kinit when KDC is native. This makes the test above possible. Otherwise, Java will not be able to generate a aes256-sha2 ccache file. Thanks Max > On Sep 8, 2017, at 8:38 PM, Artem Smotrakov wrote: > > Hi Max, > > Looks good to me. Below are a couple of minor comments you may want to address. No need a new webrev. > > Thanks! > > 1. Proc.java, better to use braces > > http://www.oracle.com/technetwork/java/javase/documentation/codeconventions-142311.html#449 > > 2. If you already updated KDC.java, it may be good to use try-with-resources for streams in a couple of places. > > Artem > > > On 09/08/2017 06:19 AM, Weijun Wang wrote: >> Small update on http://cr.openjdk.java.net/~weijun/8186884/webrev.02. All files belong to a single Proc now have the same prefix so they appear together in file list. >> >> Thanks >> Max >> >>> On Sep 7, 2017, at 8:39 PM, Weijun Wang wrote: >>> >>> Updated at >>> >>> http://cr.openjdk.java.net/~weijun/8186884/webrev.01/ >>> >>> Now the libraries can be more freely combined, so you can test interop between one native library and another one: >>> >>> jtreg -Dnative.krb5.libs=j=,n=,m=lib1.so,h=lib2.so BasicProc.java >>> >>> >>> More comments inline below. >>> >>>> On Sep 7, 2017, at 3:29 PM, Artem Smotrakov wrote: >>>> >>>> Hi Max, >>>> >>>> In general, looks fine to me. Below are a couple of comments you might want to address. >>>> >>>> 1. BasicProc.java, it might be better to use named constants for parameters for once() method. That would make it easier to understand what each particular onse() call does >>> I am passing in label and library names now. >>> >>>> 2. BasicProc.java, could you please add an exception message? >>>> >>>> + if (!Arrays.equals(msg, msg2)) { >>>> + throw new Exception(); >>>> + } >>>> + break; >>> Fixed. >>> >>>> 3. BasicProc.java, should the test do some cleanup then? >>>> >>>> + Files.copy(Paths.get("ccache.base"), Paths.get("ccache." + label)); >>> Nowadays I prefer to let jtreg do the cleanup/retain. In fact, I am able to find a KDC.java bug by saving the ccache, where the incoming service ticket is invalid and not saved into the ccache. >>> >>> Thanks >>> Max >>> >>>> Artem >>>> >>>> On 09/07/2017 03:07 AM, Weijun Wang wrote: >>>>> Please take a review at >>>>> >>>>> http://cr.openjdk.java.net/~weijun/8186884/webrev.00/ >>>>> >>>>> BasicProc.java is enhanced to use a native JGSS provider, and KDC.java is enhanced to start (not use) a native KDC. For example, you would be able to test interop among Java JGSS, native JGSS (with MIT krb5) and Heimdal KDC with >>>>> >>>>> jtreg -Dnative.krb5.lib=/usr/local/krb5/lib/libgssapi_krb5.so \ >>>>> -Dnative.kdc.path=/usr/local/heimdal \ >>>>> test/sun/security/krb5/auto/BasicProc.java >>>>> >>>>> Without those 2 new system properties, it behaves like before, i.e. Java GSS on the embedded KDC. >>>>> >>>>> Another change in Context.java. Instead of using shared states to provide username and password when doing a krb5 login, a callback handler is used. This is considered more common. An extra permission is needed to read the default username (though I think this can coded as optional). >>>>> >>>>> Thanks >>>>> Max >>>>> > From artem.smotrakov at oracle.com Mon Sep 11 08:49:05 2017 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Mon, 11 Sep 2017 11:49:05 +0300 Subject: RFR 8186884: Test native KDC, Java krb5 lib, and native krb5 lib in one test In-Reply-To: References: <9A17CD23-B9D0-484E-8970-21241ACAE123@oracle.com> <8C778238-1C3D-4C49-823F-E7C00A211879@oracle.com> Message-ID: <107d5e9e-9201-75e7-4f26-d028646a6e5b@oracle.com> Looks good to me. Artem On 09/11/2017 11:07 AM, Weijun Wang wrote: > Sorry to update again. Might be because jdk10 is frozen now. > > http://cr.openjdk.java.net/~weijun/8186884/webrev.03 > > But changes are real: > > 1. kdc.supported.enctypes supported by native KDCs correctly. In fact, I am now able to set it to aes256-sha2 (unsupported in Java yet) and test for interop between native libs with a native KDC. > > 2. A KDC::kinit method is introduced which calls native kinit when KDC is native. This makes the test above possible. Otherwise, Java will not be able to generate a aes256-sha2 ccache file. > > Thanks > Max > > >> On Sep 8, 2017, at 8:38 PM, Artem Smotrakov wrote: >> >> Hi Max, >> >> Looks good to me. Below are a couple of minor comments you may want to address. No need a new webrev. >> >> Thanks! >> >> 1. Proc.java, better to use braces >> >> http://www.oracle.com/technetwork/java/javase/documentation/codeconventions-142311.html#449 >> >> 2. If you already updated KDC.java, it may be good to use try-with-resources for streams in a couple of places. >> >> Artem >> >> >> On 09/08/2017 06:19 AM, Weijun Wang wrote: >>> Small update on http://cr.openjdk.java.net/~weijun/8186884/webrev.02. All files belong to a single Proc now have the same prefix so they appear together in file list. >>> >>> Thanks >>> Max >>> >>>> On Sep 7, 2017, at 8:39 PM, Weijun Wang wrote: >>>> >>>> Updated at >>>> >>>> http://cr.openjdk.java.net/~weijun/8186884/webrev.01/ >>>> >>>> Now the libraries can be more freely combined, so you can test interop between one native library and another one: >>>> >>>> jtreg -Dnative.krb5.libs=j=,n=,m=lib1.so,h=lib2.so BasicProc.java >>>> >>>> >>>> More comments inline below. >>>> >>>>> On Sep 7, 2017, at 3:29 PM, Artem Smotrakov wrote: >>>>> >>>>> Hi Max, >>>>> >>>>> In general, looks fine to me. Below are a couple of comments you might want to address. >>>>> >>>>> 1. BasicProc.java, it might be better to use named constants for parameters for once() method. That would make it easier to understand what each particular onse() call does >>>> I am passing in label and library names now. >>>> >>>>> 2. BasicProc.java, could you please add an exception message? >>>>> >>>>> + if (!Arrays.equals(msg, msg2)) { >>>>> + throw new Exception(); >>>>> + } >>>>> + break; >>>> Fixed. >>>> >>>>> 3. BasicProc.java, should the test do some cleanup then? >>>>> >>>>> + Files.copy(Paths.get("ccache.base"), Paths.get("ccache." + label)); >>>> Nowadays I prefer to let jtreg do the cleanup/retain. In fact, I am able to find a KDC.java bug by saving the ccache, where the incoming service ticket is invalid and not saved into the ccache. >>>> >>>> Thanks >>>> Max >>>> >>>>> Artem >>>>> >>>>> On 09/07/2017 03:07 AM, Weijun Wang wrote: >>>>>> Please take a review at >>>>>> >>>>>> http://cr.openjdk.java.net/~weijun/8186884/webrev.00/ >>>>>> >>>>>> BasicProc.java is enhanced to use a native JGSS provider, and KDC.java is enhanced to start (not use) a native KDC. For example, you would be able to test interop among Java JGSS, native JGSS (with MIT krb5) and Heimdal KDC with >>>>>> >>>>>> jtreg -Dnative.krb5.lib=/usr/local/krb5/lib/libgssapi_krb5.so \ >>>>>> -Dnative.kdc.path=/usr/local/heimdal \ >>>>>> test/sun/security/krb5/auto/BasicProc.java >>>>>> >>>>>> Without those 2 new system properties, it behaves like before, i.e. Java GSS on the embedded KDC. >>>>>> >>>>>> Another change in Context.java. Instead of using shared states to provide username and password when doing a krb5 login, a callback handler is used. This is considered more common. An extra permission is needed to read the default username (though I think this can coded as optional). >>>>>> >>>>>> Thanks >>>>>> Max >>>>>> From rob.mckenna at oracle.com Mon Sep 11 22:29:54 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Mon, 11 Sep 2017 23:29:54 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read Message-ID: <20170911222954.GC3791@vimes> Hi folks, In high latency environments a client SSLSocket with autoClose set to false can hang indefinitely if it does not correctly recieve a close_notify from the server. In order to rectify this situation I would like to suggest that we implement an integer JDK property (jdk.tls.closeRetries) which instructs waitForClose to attempt the close no more times than the value of the property. I would also suggest that 5 is a reasonable default. Note: each attempt times out based on the value of Socket.setSoTimeout(int timeout). Also, the behaviour here is similar to that of waitForClose() when autoClose is set to true, less the retries. http://cr.openjdk.java.net/~robm/8184328/webrev.01/ -Rob From xuelei.fan at oracle.com Wed Sep 13 12:52:30 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 13 Sep 2017 05:52:30 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170911222954.GC3791@vimes> References: <20170911222954.GC3791@vimes> Message-ID: In theory, there are intermittent compatibility problems as this update may not close the SSL connection over the existing socket layer gracefully, even for high speed networking environments, while the underlying socket is alive. The impact could be serious in some environment. For safe, I may suggest turn this countermeasure off by default. And providing options to turn on this countermeasure: 1. Close the SSL connection gracefully by default; or 2. Close the SSL connection after a timeout. It's hardly to say 5 times receiving timeout is better/safer than timeout once in this context. As you have already had a system property to control, you may be able to use options other than the customized socket receiving timeout, so that the closing timeout is not mixed/confused/dependent on/with the receiving timeout. Put all together: 1. define a closing timeout, for example "jdk.tls.waitForClose". 2. the property default value is zero, no behavior changes. 3. applications can set positive milliseconds value for the property. The SSL connection will be closed in the set milliseconds (or about the maximum value between SO_TIMEOUT and closing timeout), the connection is not grant to be gracefully. What do you think? BTW, please file a CSR as this update is introducing an external system property. Thanks, Xuelei On 9/11/2017 3:29 PM, Rob McKenna wrote: > Hi folks, > > In high latency environments a client SSLSocket with autoClose set to false > can hang indefinitely if it does not correctly recieve a close_notify > from the server. > > In order to rectify this situation I would like to suggest that we > implement an integer JDK property (jdk.tls.closeRetries) which instructs > waitForClose to attempt the close no more times than the value of the > property. I would also suggest that 5 is a reasonable default. > > Note: each attempt times out based on the value of > Socket.setSoTimeout(int timeout). > > Also, the behaviour here is similar to that of waitForClose() when > autoClose is set to true, less the retries. > > http://cr.openjdk.java.net/~robm/8184328/webrev.01/ > > -Rob > From xuelei.fan at oracle.com Wed Sep 13 12:54:34 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 13 Sep 2017 05:54:34 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: References: <20170911222954.GC3791@vimes> Message-ID: On 9/13/2017 5:52 AM, Xuelei Fan wrote: > In theory, there are intermittent compatibility problems as this update > may not close the SSL connection over the existing socket layer > gracefully, even for high speed networking environments, while the > underlying socket is alive.? The impact could be serious in some > environment. > > For safe, I may suggest turn this countermeasure off by default.? And > providing options to turn on this countermeasure: > 1. Close the SSL connection gracefully by default; or > 2. Close the SSL connection after a timeout. > > It's hardly to say 5 times receiving timeout is better/safer than > timeout once in this context.? As you have already had a system property > to control, you may be able to use options other than the customized > socket receiving timeout, so that the closing timeout is not > mixed/confused/dependent on/with the receiving timeout. > > Put all together: > 1. define a closing timeout, for example "jdk.tls.waitForClose". > 2. the property default value is zero, no behavior changes. > 3. applications can set positive milliseconds value for the property. > The SSL connection will be closed in the set milliseconds (or about the > maximum value between SO_TIMEOUT and closing timeout), the connection is > not grant to be gracefully. > typo and missed, "not granted to be closed gracefully". > What do you think? > > BTW, please file a CSR as this update is introducing an external system > property. > > Thanks, > Xuelei > > On 9/11/2017 3:29 PM, Rob McKenna wrote: >> Hi folks, >> >> In high latency environments a client SSLSocket with autoClose set to >> false >> can hang indefinitely if it does not correctly recieve a close_notify >> from the server. >> >> In order to rectify this situation I would like to suggest that we >> implement an integer JDK property (jdk.tls.closeRetries) which instructs >> waitForClose to attempt the close no more times than the value of the >> property. I would also suggest that 5 is a reasonable default. >> >> Note: each attempt times out based on the value of >> Socket.setSoTimeout(int timeout). >> >> Also, the behaviour here is similar to that of waitForClose() when >> autoClose is set to true, less the retries. >> >> http://cr.openjdk.java.net/~robm/8184328/webrev.01/ >> >> ???? -Rob >> From rob.mckenna at oracle.com Wed Sep 13 15:52:08 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Wed, 13 Sep 2017 16:52:08 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: References: <20170911222954.GC3791@vimes> Message-ID: <20170913155208.GD3896@vimes> Hi Xuelei, This behaviour is already exposed via the autoclose boolean in: https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- My position would be that allowing 5 retries allows us to say with some confidence that we're not going to get a close_notify from the server. If this is the case I think its reasonable to close the connection. W.r.t. a separate timeout, the underlying mechanics of a close already depend on the readTimeout in this situation. (waiting on a close_notify requires performing a read so the read timeout makes sense in this context) I'm happy to alter that but I think that the combination of a timeout and a retry count is straightforward and lower impact. In my opinion the default behaviour of potentially hanging indefinitely is worse than the alternative here. (bearing in mind that we are closing the underlying socket) I'll file a CSR as soon as we settle on the direction this fix will take. -Rob On 13/09/17 05:52, Xuelei Fan wrote: > In theory, there are intermittent compatibility problems as this update may > not close the SSL connection over the existing socket layer gracefully, even > for high speed networking environments, while the underlying socket is > alive. The impact could be serious in some environment. > > For safe, I may suggest turn this countermeasure off by default. And > providing options to turn on this countermeasure: > 1. Close the SSL connection gracefully by default; or > 2. Close the SSL connection after a timeout. > > It's hardly to say 5 times receiving timeout is better/safer than timeout > once in this context. As you have already had a system property to control, > you may be able to use options other than the customized socket receiving > timeout, so that the closing timeout is not mixed/confused/dependent on/with > the receiving timeout. > > Put all together: > 1. define a closing timeout, for example "jdk.tls.waitForClose". > 2. the property default value is zero, no behavior changes. > 3. applications can set positive milliseconds value for the property. The > SSL connection will be closed in the set milliseconds (or about the maximum > value between SO_TIMEOUT and closing timeout), the connection is not grant > to be gracefully. > > What do you think? > > BTW, please file a CSR as this update is introducing an external system > property. > > Thanks, > Xuelei > > On 9/11/2017 3:29 PM, Rob McKenna wrote: > >Hi folks, > > > >In high latency environments a client SSLSocket with autoClose set to false > >can hang indefinitely if it does not correctly recieve a close_notify > >from the server. > > > >In order to rectify this situation I would like to suggest that we > >implement an integer JDK property (jdk.tls.closeRetries) which instructs > >waitForClose to attempt the close no more times than the value of the > >property. I would also suggest that 5 is a reasonable default. > > > >Note: each attempt times out based on the value of > >Socket.setSoTimeout(int timeout). > > > >Also, the behaviour here is similar to that of waitForClose() when > >autoClose is set to true, less the retries. > > > >http://cr.openjdk.java.net/~robm/8184328/webrev.01/ > > > > -Rob > > From chris.hegarty at oracle.com Wed Sep 13 20:06:55 2017 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 13 Sep 2017 21:06:55 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170913155208.GD3896@vimes> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> Message-ID: <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> Xuelei, Without diving deeper into this issue, Rob?s suggested approach seems reasonable to me, and better than existing out-of-the-box behaviour. I?m not sure what issues you are thinking of, with using the read timeout in combination with a retry mechanism, in this manner? If the network is so slow, surely there will be other issues with connecting and reading, why is closing any different. -Chris. > On 13 Sep 2017, at 16:52, Rob McKenna wrote: > > Hi Xuelei, > > This behaviour is already exposed via the autoclose boolean in: > > https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- > > My position would be that allowing 5 retries allows us to say with some > confidence that we're not going to get a close_notify from the server. > If this is the case I think its reasonable to close the connection. > > W.r.t. a separate timeout, the underlying mechanics of a close already > depend on the readTimeout in this situation. (waiting on a close_notify > requires performing a read so the read timeout makes sense in this > context) I'm happy to alter that but I think that the combination of > a timeout and a retry count is straightforward and lower impact. > > In my opinion the default behaviour of potentially hanging indefinitely > is worse than the alternative here. (bearing in mind that we are closing > the underlying socket) > > I'll file a CSR as soon as we settle on the direction this fix will > take. > > -Rob > > On 13/09/17 05:52, Xuelei Fan wrote: >> In theory, there are intermittent compatibility problems as this update may >> not close the SSL connection over the existing socket layer gracefully, even >> for high speed networking environments, while the underlying socket is >> alive. The impact could be serious in some environment. >> >> For safe, I may suggest turn this countermeasure off by default. And >> providing options to turn on this countermeasure: >> 1. Close the SSL connection gracefully by default; or >> 2. Close the SSL connection after a timeout. >> >> It's hardly to say 5 times receiving timeout is better/safer than timeout >> once in this context. As you have already had a system property to control, >> you may be able to use options other than the customized socket receiving >> timeout, so that the closing timeout is not mixed/confused/dependent on/with >> the receiving timeout. >> >> Put all together: >> 1. define a closing timeout, for example "jdk.tls.waitForClose". >> 2. the property default value is zero, no behavior changes. >> 3. applications can set positive milliseconds value for the property. The >> SSL connection will be closed in the set milliseconds (or about the maximum >> value between SO_TIMEOUT and closing timeout), the connection is not grant >> to be gracefully. >> >> What do you think? >> >> BTW, please file a CSR as this update is introducing an external system >> property. >> >> Thanks, >> Xuelei >> >> On 9/11/2017 3:29 PM, Rob McKenna wrote: >>> Hi folks, >>> >>> In high latency environments a client SSLSocket with autoClose set to false >>> can hang indefinitely if it does not correctly recieve a close_notify >>> from the server. >>> >>> In order to rectify this situation I would like to suggest that we >>> implement an integer JDK property (jdk.tls.closeRetries) which instructs >>> waitForClose to attempt the close no more times than the value of the >>> property. I would also suggest that 5 is a reasonable default. >>> >>> Note: each attempt times out based on the value of >>> Socket.setSoTimeout(int timeout). >>> >>> Also, the behaviour here is similar to that of waitForClose() when >>> autoClose is set to true, less the retries. >>> >>> http://cr.openjdk.java.net/~robm/8184328/webrev.01/ >>> >>> -Rob >>> From xuelei.fan at oracle.com Wed Sep 13 22:52:39 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 13 Sep 2017 15:52:39 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170913155208.GD3896@vimes> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> Message-ID: On 9/13/2017 8:52 AM, Rob McKenna wrote: > Hi Xuelei, > > This behaviour is already exposed via the autoclose boolean in: > > https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- > I did not get the point. What do you mean by this behavior is already exposed? > My position would be that allowing 5 retries allows us to say with some > confidence that we're not going to get a close_notify from the server. You have more chance to get the close_notify, but it does not mean you can always get the close_notify in 5 retries. When you cannot get it, something bad happens. > If this is the case I think its reasonable to close the connection. > > W.r.t. a separate timeout, the underlying mechanics of a close already > depend on the readTimeout in this situation. (waiting on a close_notify > requires performing a read so the read timeout makes sense in this > context) I'm happy to alter that but I think that the combination of > a timeout and a retry count is straightforward and lower impact. > > In my opinion the default behaviour of potentially hanging indefinitely > is worse than the alternative here. (bearing in mind that we are closing > the underlying socket) > I did not get the point, are we really closing the underlying socket (or the layered ssl connection?) for the context of you update? Xuelei > I'll file a CSR as soon as we settle on the direction this fix will > take. > > -Rob > > On 13/09/17 05:52, Xuelei Fan wrote: >> In theory, there are intermittent compatibility problems as this update may >> not close the SSL connection over the existing socket layer gracefully, even >> for high speed networking environments, while the underlying socket is >> alive. The impact could be serious in some environment. >> >> For safe, I may suggest turn this countermeasure off by default. And >> providing options to turn on this countermeasure: >> 1. Close the SSL connection gracefully by default; or >> 2. Close the SSL connection after a timeout. >> >> It's hardly to say 5 times receiving timeout is better/safer than timeout >> once in this context. As you have already had a system property to control, >> you may be able to use options other than the customized socket receiving >> timeout, so that the closing timeout is not mixed/confused/dependent on/with >> the receiving timeout. >> >> Put all together: >> 1. define a closing timeout, for example "jdk.tls.waitForClose". >> 2. the property default value is zero, no behavior changes. >> 3. applications can set positive milliseconds value for the property. The >> SSL connection will be closed in the set milliseconds (or about the maximum >> value between SO_TIMEOUT and closing timeout), the connection is not grant >> to be gracefully. >> >> What do you think? >> >> BTW, please file a CSR as this update is introducing an external system >> property. >> >> Thanks, >> Xuelei >> >> On 9/11/2017 3:29 PM, Rob McKenna wrote: >>> Hi folks, >>> >>> In high latency environments a client SSLSocket with autoClose set to false >>> can hang indefinitely if it does not correctly recieve a close_notify >> >from the server. >>> >>> In order to rectify this situation I would like to suggest that we >>> implement an integer JDK property (jdk.tls.closeRetries) which instructs >>> waitForClose to attempt the close no more times than the value of the >>> property. I would also suggest that 5 is a reasonable default. >>> >>> Note: each attempt times out based on the value of >>> Socket.setSoTimeout(int timeout). >>> >>> Also, the behaviour here is similar to that of waitForClose() when >>> autoClose is set to true, less the retries. >>> >>> http://cr.openjdk.java.net/~robm/8184328/webrev.01/ >>> >>> -Rob >>> From xuelei.fan at oracle.com Wed Sep 13 22:56:30 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 13 Sep 2017 15:56:30 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170913155208.GD3896@vimes> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> Message-ID: <850d1b2a-18af-0f05-a812-4d0f181b9404@oracle.com> On 9/13/2017 8:52 AM, Rob McKenna wrote: > W.r.t. a separate timeout, the underlying mechanics of a close already > depend on the readTimeout in this situation. That's a concerns of mine. In order to work for your countermeasure, applications have to set receiving timeout, and take care of the closing timeout when evaluate what's a right timeout value. The mixing could be misleading and not easy to use. Xuelei From xuelei.fan at oracle.com Wed Sep 13 23:09:08 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 13 Sep 2017 16:09:08 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> Message-ID: It's a little bit complicated for layered SSL connections. Application can build a SSL connection on existing socket (we call it layered SSL connections). The problem scenarios make look like: 1. open a socket for applications. 2. established a SSL connection on the existing socket. 3. close the SSL connection, but leaving data in the socket. 4. establish another SSL connection on the socket, as the existing data in the socket, the connection cannot be established. 5. establish another app connection on the socket, as the existing data in the socket, the connection cannot be established. .... Timeout happens even on very high speed network. If a timeout happens and the SSL connection is not closed gracefully, and then the following applications breaks. IMHO, we need to take care of the case. Xuelei On 9/13/2017 1:06 PM, Chris Hegarty wrote: > Xuelei, > > Without diving deeper into this issue, Rob?s suggested approach seems reasonable to me, and better than existing out-of-the-box behaviour. I?m not sure what issues you are thinking of, with using the read timeout in combination with a retry mechanism, in this manner? If the network is so slow, surely there will be other issues with connecting and reading, why is closing any different. > > -Chris. > >> On 13 Sep 2017, at 16:52, Rob McKenna wrote: >> >> Hi Xuelei, >> >> This behaviour is already exposed via the autoclose boolean in: >> >> https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- >> >> My position would be that allowing 5 retries allows us to say with some >> confidence that we're not going to get a close_notify from the server. >> If this is the case I think its reasonable to close the connection. >> >> W.r.t. a separate timeout, the underlying mechanics of a close already >> depend on the readTimeout in this situation. (waiting on a close_notify >> requires performing a read so the read timeout makes sense in this >> context) I'm happy to alter that but I think that the combination of >> a timeout and a retry count is straightforward and lower impact. >> >> In my opinion the default behaviour of potentially hanging indefinitely >> is worse than the alternative here. (bearing in mind that we are closing >> the underlying socket) >> >> I'll file a CSR as soon as we settle on the direction this fix will >> take. >> >> -Rob >> >> On 13/09/17 05:52, Xuelei Fan wrote: >>> In theory, there are intermittent compatibility problems as this update may >>> not close the SSL connection over the existing socket layer gracefully, even >>> for high speed networking environments, while the underlying socket is >>> alive. The impact could be serious in some environment. >>> >>> For safe, I may suggest turn this countermeasure off by default. And >>> providing options to turn on this countermeasure: >>> 1. Close the SSL connection gracefully by default; or >>> 2. Close the SSL connection after a timeout. >>> >>> It's hardly to say 5 times receiving timeout is better/safer than timeout >>> once in this context. As you have already had a system property to control, >>> you may be able to use options other than the customized socket receiving >>> timeout, so that the closing timeout is not mixed/confused/dependent on/with >>> the receiving timeout. >>> >>> Put all together: >>> 1. define a closing timeout, for example "jdk.tls.waitForClose". >>> 2. the property default value is zero, no behavior changes. >>> 3. applications can set positive milliseconds value for the property. The >>> SSL connection will be closed in the set milliseconds (or about the maximum >>> value between SO_TIMEOUT and closing timeout), the connection is not grant >>> to be gracefully. >>> >>> What do you think? >>> >>> BTW, please file a CSR as this update is introducing an external system >>> property. >>> >>> Thanks, >>> Xuelei >>> >>> On 9/11/2017 3:29 PM, Rob McKenna wrote: >>>> Hi folks, >>>> >>>> In high latency environments a client SSLSocket with autoClose set to false >>>> can hang indefinitely if it does not correctly recieve a close_notify >>>> from the server. >>>> >>>> In order to rectify this situation I would like to suggest that we >>>> implement an integer JDK property (jdk.tls.closeRetries) which instructs >>>> waitForClose to attempt the close no more times than the value of the >>>> property. I would also suggest that 5 is a reasonable default. >>>> >>>> Note: each attempt times out based on the value of >>>> Socket.setSoTimeout(int timeout). >>>> >>>> Also, the behaviour here is similar to that of waitForClose() when >>>> autoClose is set to true, less the retries. >>>> >>>> http://cr.openjdk.java.net/~robm/8184328/webrev.01/ >>>> >>>> -Rob >>>> > From adam.petcher at oracle.com Thu Sep 14 16:59:51 2017 From: adam.petcher at oracle.com (Adam Petcher) Date: Thu, 14 Sep 2017 12:59:51 -0400 Subject: JEP for X25519/X448 key agreement Message-ID: <7c9104dc-ac91-c966-6273-7387410e5a7a@oracle.com> The JEP for X25519/X448 key agreement[1] is now available and ready to review. Please take a look and reply with any feedback you have. The JEP contains a description of the proposed JCA API. We have discussed the API on this mailing list, and I have attempted to incorporate all the feedback I have received. Here is a description of the changes since the last discussion: 1) Multiple people requested more specific types for public/private keys for this algorithm. The latest API design mirrors the "EC" hierarchy and has both interfaces and spec classes for public and private keys. I also added the interface "XDHKey", which serves the same purpose as "ECKey". 2) The representation of public keys was changed from byte[] to a BigInteger which holds the u coordinate of the point. Private keys are still represented using byte[] due to complications related to pruning, and also because BigInteger doesn't provide a branch-free way to get the key into another representation (which is necessary for side-channel-resilient implementations). The proposed API still lacks a standard way to specify arbitrary domain parameters, but I believe the API design could be extended to support this feature. I would prefer to add this API as a separate enhancement in the future, preferably in cooperation with someone who is developing a provider that supports this feature. [1] https://bugs.openjdk.java.net/browse/JDK-8181595 From sean.coffey at oracle.com Thu Sep 14 21:04:21 2017 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 14 Sep 2017 22:04:21 +0100 Subject: RFR: 8170157, 8170245: Enable unlimited cryptographic policy by default in OracleJDK In-Reply-To: <1521acbf-cd03-7be7-6026-d7beb51f07cd@oracle.com> References: <9ca65218-86b0-1209-8f11-9d0b2fd67c2b@oracle.com> <33d4d8cf-90ef-3653-48f0-96c0c327aca8@oracle.com> <1521acbf-cd03-7be7-6026-d7beb51f07cd@oracle.com> Message-ID: <1d1ef856-3fa8-2b0d-e64f-171be66d3e7a@oracle.com> Some modifications to the java.security file(s). Final webrev, I hope : http://cr.openjdk.java.net/~coffeys/webrev.8170157.8u.02/webrev/ regards, Sean. On 01/09/2017 16:04, Se?n Coffey wrote: > comments inline. > > On 29/08/17 23:33, Bradford Wetmore wrote: >> >> Very minor comments/tweaks. >> >> On 8/18/2017 7:01 AM, Se?n Coffey wrote: >>> Looking to backport 8170157 to jdk8u-dev. The 8170245 test bug also >>> gets pulled in for this port since some tests need cleaning up to >>> deal with unlimited crypto environment. >>> >>> webrev : >>> http://cr.openjdk.java.net/~coffeys/webrev.8170157.8u.01/webrev/index.html >> >> Update copyright dates.? Looks like the original work was done in >> December 2016, but this is our actual push. > Done. >> >> JceSecurity.java >> ================ >> 265:? Suggestion since this is JDK/JRE specific: >> >> jre/lib/security >> -> >> /lib/security > Done. I corrected line 260 also which was using the jre/lib/security > syntax. >> >> Dynamic.java >> ============ >> Is the removal of the separate ECB case because the parameter will >> just be ignored?? Why was this a failing test case? > I inherited these changes from the JDK 9 edits. It looks like the > stronger AES defaults place stricter conditions on the IV length.[1] - > The test had some old conditions for some non-GCM mode based ciphers. > That seems to be no longer necessary and the call to > "ci.init(Cipher.DECRYPT_MODE, key, ci.getParameters());" appears to > work for all now. > > regards, > Sean. > > [1] > java.security.InvalidAlgorithmParameterException: Wrong IV length: > must be 16 bytes long > ??????? at com.sun.crypto.provider.CipherCore.init(CipherCore.java:526) > ??????? at > com.sun.crypto.provider.AESCipher.engineInit(AESCipher.java:346) > ??????? at javax.crypto.Cipher.init(Cipher.java:1394) > ??????? at javax.crypto.Cipher.init(Cipher.java:1327) > ??????? at Dynamic.runTest(Dynamic.java:145) > ??????? at Dynamic.runAllTest(Dynamic.java:89) > ??????? at Dynamic.run(Dynamic.java:59) > ??????? at > TestAESWithRemoveAddProvider.main(TestAESWithRemoveAddProvider.java:40) > ??????? at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > ??????? at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ??????? at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ??????? at java.lang.reflect.Method.invoke(Method.java:498) > ??????? at > com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) > ??????? at java.lang.Thread.run(Thread.java:748) > >> >> Thanks. >> >> Brad >> >> >> > From bradford.wetmore at oracle.com Thu Sep 14 21:12:17 2017 From: bradford.wetmore at oracle.com (Bradford Wetmore) Date: Thu, 14 Sep 2017 14:12:17 -0700 Subject: RFR: 8170157, 8170245: Enable unlimited cryptographic policy by default in OracleJDK In-Reply-To: <1d1ef856-3fa8-2b0d-e64f-171be66d3e7a@oracle.com> References: <9ca65218-86b0-1209-8f11-9d0b2fd67c2b@oracle.com> <33d4d8cf-90ef-3653-48f0-96c0c327aca8@oracle.com> <1521acbf-cd03-7be7-6026-d7beb51f07cd@oracle.com> <1d1ef856-3fa8-2b0d-e64f-171be66d3e7a@oracle.com> Message-ID: The java.security files are the only differences? Then looks good to me. Brad On 9/14/2017 2:04 PM, Se?n Coffey wrote: > Some modifications to the java.security file(s). > > Final webrev, I hope : > > http://cr.openjdk.java.net/~coffeys/webrev.8170157.8u.02/webrev/ > > regards, > Sean. > > > On 01/09/2017 16:04, Se?n Coffey wrote: >> comments inline. >> >> On 29/08/17 23:33, Bradford Wetmore wrote: >>> >>> Very minor comments/tweaks. >>> >>> On 8/18/2017 7:01 AM, Se?n Coffey wrote: >>>> Looking to backport 8170157 to jdk8u-dev. The 8170245 test bug also >>>> gets pulled in for this port since some tests need cleaning up to >>>> deal with unlimited crypto environment. >>>> >>>> webrev : >>>> http://cr.openjdk.java.net/~coffeys/webrev.8170157.8u.01/webrev/index.html >>>> >>> >>> Update copyright dates.? Looks like the original work was done in >>> December 2016, but this is our actual push. >> Done. >>> >>> JceSecurity.java >>> ================ >>> 265:? Suggestion since this is JDK/JRE specific: >>> >>> jre/lib/security >>> -> >>> /lib/security >> Done. I corrected line 260 also which was using the jre/lib/security >> syntax. >>> >>> Dynamic.java >>> ============ >>> Is the removal of the separate ECB case because the parameter will >>> just be ignored?? Why was this a failing test case? >> I inherited these changes from the JDK 9 edits. It looks like the >> stronger AES defaults place stricter conditions on the IV length.[1] - >> The test had some old conditions for some non-GCM mode based ciphers. >> That seems to be no longer necessary and the call to >> "ci.init(Cipher.DECRYPT_MODE, key, ci.getParameters());" appears to >> work for all now. >> >> regards, >> Sean. >> >> [1] >> java.security.InvalidAlgorithmParameterException: Wrong IV length: >> must be 16 bytes long >> ??????? at com.sun.crypto.provider.CipherCore.init(CipherCore.java:526) >> ??????? at >> com.sun.crypto.provider.AESCipher.engineInit(AESCipher.java:346) >> ??????? at javax.crypto.Cipher.init(Cipher.java:1394) >> ??????? at javax.crypto.Cipher.init(Cipher.java:1327) >> ??????? at Dynamic.runTest(Dynamic.java:145) >> ??????? at Dynamic.runAllTest(Dynamic.java:89) >> ??????? at Dynamic.run(Dynamic.java:59) >> ??????? at >> TestAESWithRemoveAddProvider.main(TestAESWithRemoveAddProvider.java:40) >> ??????? at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> ??????? at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> >> ??????? at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> ??????? at java.lang.reflect.Method.invoke(Method.java:498) >> ??????? at >> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) >> >> ??????? at java.lang.Thread.run(Thread.java:748) >> >>> >>> Thanks. >>> >>> Brad >>> >>> >>> >> > From sean.coffey at oracle.com Fri Sep 15 07:23:20 2017 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 15 Sep 2017 08:23:20 +0100 Subject: RFR: 8170157, 8170245: Enable unlimited cryptographic policy by default in OracleJDK In-Reply-To: References: <9ca65218-86b0-1209-8f11-9d0b2fd67c2b@oracle.com> <33d4d8cf-90ef-3653-48f0-96c0c327aca8@oracle.com> <1521acbf-cd03-7be7-6026-d7beb51f07cd@oracle.com> <1d1ef856-3fa8-2b0d-e64f-171be66d3e7a@oracle.com> Message-ID: Thanks for the review. Yes - main difference is the java.security files. The minor edits suggested in your first review are also done. regards, Sean. On 14/09/2017 22:12, Bradford Wetmore wrote: > The java.security files are the only differences?? Then looks good to me. > > Brad > > > On 9/14/2017 2:04 PM, Se?n Coffey wrote: >> Some modifications to the java.security file(s). >> >> Final webrev, I hope : >> >> http://cr.openjdk.java.net/~coffeys/webrev.8170157.8u.02/webrev/ >> >> regards, >> Sean. >> >> >> On 01/09/2017 16:04, Se?n Coffey wrote: >>> comments inline. >>> >>> On 29/08/17 23:33, Bradford Wetmore wrote: >>>> >>>> Very minor comments/tweaks. >>>> >>>> On 8/18/2017 7:01 AM, Se?n Coffey wrote: >>>>> Looking to backport 8170157 to jdk8u-dev. The 8170245 test bug >>>>> also gets pulled in for this port since some tests need cleaning >>>>> up to deal with unlimited crypto environment. >>>>> >>>>> webrev : >>>>> http://cr.openjdk.java.net/~coffeys/webrev.8170157.8u.01/webrev/index.html >>>>> >>>> >>>> Update copyright dates.? Looks like the original work was done in >>>> December 2016, but this is our actual push. >>> Done. >>>> >>>> JceSecurity.java >>>> ================ >>>> 265:? Suggestion since this is JDK/JRE specific: >>>> >>>> jre/lib/security >>>> -> >>>> /lib/security >>> Done. I corrected line 260 also which was using the jre/lib/security >>> syntax. >>>> >>>> Dynamic.java >>>> ============ >>>> Is the removal of the separate ECB case because the parameter will >>>> just be ignored?? Why was this a failing test case? >>> I inherited these changes from the JDK 9 edits. It looks like the >>> stronger AES defaults place stricter conditions on the IV length.[1] >>> - The test had some old conditions for some non-GCM mode based >>> ciphers. That seems to be no longer necessary and the call to >>> "ci.init(Cipher.DECRYPT_MODE, key, ci.getParameters());" appears to >>> work for all now. >>> >>> regards, >>> Sean. >>> >>> [1] >>> java.security.InvalidAlgorithmParameterException: Wrong IV length: >>> must be 16 bytes long >>> ??????? at com.sun.crypto.provider.CipherCore.init(CipherCore.java:526) >>> ??????? at >>> com.sun.crypto.provider.AESCipher.engineInit(AESCipher.java:346) >>> ??????? at javax.crypto.Cipher.init(Cipher.java:1394) >>> ??????? at javax.crypto.Cipher.init(Cipher.java:1327) >>> ??????? at Dynamic.runTest(Dynamic.java:145) >>> ??????? at Dynamic.runAllTest(Dynamic.java:89) >>> ??????? at Dynamic.run(Dynamic.java:59) >>> ??????? at >>> TestAESWithRemoveAddProvider.main(TestAESWithRemoveAddProvider.java:40) >>> ??????? at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> ??????? at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> >>> ??????? at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> >>> ??????? at java.lang.reflect.Method.invoke(Method.java:498) >>> ??????? at >>> com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:115) >>> >>> ??????? at java.lang.Thread.run(Thread.java:748) >>> >>>> >>>> Thanks. >>>> >>>> Brad >>>> >>>> >>>> >>> >> From rob.mckenna at oracle.com Fri Sep 15 14:00:25 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 15 Sep 2017 15:00:25 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> Message-ID: <20170915140025.GA4404@vimes> When we call close() on the SSLSocket that calls close() on the underlying java Socket which closes the native socket. -Rob On 13/09/17 04:09, Xuelei Fan wrote: > It's a little bit complicated for layered SSL connections. Application can > build a SSL connection on existing socket (we call it layered SSL > connections). The problem scenarios make look like: > 1. open a socket for applications. > 2. established a SSL connection on the existing socket. > 3. close the SSL connection, but leaving data in the socket. > 4. establish another SSL connection on the socket, as the existing data in > the socket, the connection cannot be established. > 5. establish another app connection on the socket, as the existing data in > the socket, the connection cannot be established. > .... > > Timeout happens even on very high speed network. If a timeout happens and > the SSL connection is not closed gracefully, and then the following > applications breaks. IMHO, we need to take care of the case. > > Xuelei > > On 9/13/2017 1:06 PM, Chris Hegarty wrote: > >Xuelei, > > > >Without diving deeper into this issue, Rob?s suggested approach seems reasonable to me, and better than existing out-of-the-box behaviour. I?m not sure what issues you are thinking of, with using the read timeout in combination with a retry mechanism, in this manner? If the network is so slow, surely there will be other issues with connecting and reading, why is closing any different. > > > >-Chris. > > > >>On 13 Sep 2017, at 16:52, Rob McKenna wrote: > >> > >>Hi Xuelei, > >> > >>This behaviour is already exposed via the autoclose boolean in: > >> > >>https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- > >> > >>My position would be that allowing 5 retries allows us to say with some > >>confidence that we're not going to get a close_notify from the server. > >>If this is the case I think its reasonable to close the connection. > >> > >>W.r.t. a separate timeout, the underlying mechanics of a close already > >>depend on the readTimeout in this situation. (waiting on a close_notify > >>requires performing a read so the read timeout makes sense in this > >>context) I'm happy to alter that but I think that the combination of > >>a timeout and a retry count is straightforward and lower impact. > >> > >>In my opinion the default behaviour of potentially hanging indefinitely > >>is worse than the alternative here. (bearing in mind that we are closing > >>the underlying socket) > >> > >>I'll file a CSR as soon as we settle on the direction this fix will > >>take. > >> > >> -Rob > >> > >>On 13/09/17 05:52, Xuelei Fan wrote: > >>>In theory, there are intermittent compatibility problems as this update may > >>>not close the SSL connection over the existing socket layer gracefully, even > >>>for high speed networking environments, while the underlying socket is > >>>alive. The impact could be serious in some environment. > >>> > >>>For safe, I may suggest turn this countermeasure off by default. And > >>>providing options to turn on this countermeasure: > >>>1. Close the SSL connection gracefully by default; or > >>>2. Close the SSL connection after a timeout. > >>> > >>>It's hardly to say 5 times receiving timeout is better/safer than timeout > >>>once in this context. As you have already had a system property to control, > >>>you may be able to use options other than the customized socket receiving > >>>timeout, so that the closing timeout is not mixed/confused/dependent on/with > >>>the receiving timeout. > >>> > >>>Put all together: > >>>1. define a closing timeout, for example "jdk.tls.waitForClose". > >>>2. the property default value is zero, no behavior changes. > >>>3. applications can set positive milliseconds value for the property. The > >>>SSL connection will be closed in the set milliseconds (or about the maximum > >>>value between SO_TIMEOUT and closing timeout), the connection is not grant > >>>to be gracefully. > >>> > >>>What do you think? > >>> > >>>BTW, please file a CSR as this update is introducing an external system > >>>property. > >>> > >>>Thanks, > >>>Xuelei > >>> > >>>On 9/11/2017 3:29 PM, Rob McKenna wrote: > >>>>Hi folks, > >>>> > >>>>In high latency environments a client SSLSocket with autoClose set to false > >>>>can hang indefinitely if it does not correctly recieve a close_notify > >>>>from the server. > >>>> > >>>>In order to rectify this situation I would like to suggest that we > >>>>implement an integer JDK property (jdk.tls.closeRetries) which instructs > >>>>waitForClose to attempt the close no more times than the value of the > >>>>property. I would also suggest that 5 is a reasonable default. > >>>> > >>>>Note: each attempt times out based on the value of > >>>>Socket.setSoTimeout(int timeout). > >>>> > >>>>Also, the behaviour here is similar to that of waitForClose() when > >>>>autoClose is set to true, less the retries. > >>>> > >>>>http://cr.openjdk.java.net/~robm/8184328/webrev.01/ > >>>> > >>>> -Rob > >>>> > > From xuelei.fan at oracle.com Fri Sep 15 14:07:45 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 15 Sep 2017 07:07:45 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170915140025.GA4404@vimes> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> <20170915140025.GA4404@vimes> Message-ID: <003a2ae4-349c-cc33-fea8-5629b225e86b@oracle.com> On 9/15/2017 7:00 AM, Rob McKenna wrote: > When we call close() on the SSLSocket that calls close() on the > underlying java Socket which closes the native socket. > Sorry, I did not get the point. Please see the close() implementation of SSLSocket (sun.security.ssl.SSLSocketImpl.close()) about the details. Xuelei > -Rob > > On 13/09/17 04:09, Xuelei Fan wrote: >> It's a little bit complicated for layered SSL connections. Application can >> build a SSL connection on existing socket (we call it layered SSL >> connections). The problem scenarios make look like: >> 1. open a socket for applications. >> 2. established a SSL connection on the existing socket. >> 3. close the SSL connection, but leaving data in the socket. >> 4. establish another SSL connection on the socket, as the existing data in >> the socket, the connection cannot be established. >> 5. establish another app connection on the socket, as the existing data in >> the socket, the connection cannot be established. >> .... >> >> Timeout happens even on very high speed network. If a timeout happens and >> the SSL connection is not closed gracefully, and then the following >> applications breaks. IMHO, we need to take care of the case. >> >> Xuelei >> >> On 9/13/2017 1:06 PM, Chris Hegarty wrote: >>> Xuelei, >>> >>> Without diving deeper into this issue, Rob?s suggested approach seems reasonable to me, and better than existing out-of-the-box behaviour. I?m not sure what issues you are thinking of, with using the read timeout in combination with a retry mechanism, in this manner? If the network is so slow, surely there will be other issues with connecting and reading, why is closing any different. >>> >>> -Chris. >>> >>>> On 13 Sep 2017, at 16:52, Rob McKenna wrote: >>>> >>>> Hi Xuelei, >>>> >>>> This behaviour is already exposed via the autoclose boolean in: >>>> >>>> https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- >>>> >>>> My position would be that allowing 5 retries allows us to say with some >>>> confidence that we're not going to get a close_notify from the server. >>>> If this is the case I think its reasonable to close the connection. >>>> >>>> W.r.t. a separate timeout, the underlying mechanics of a close already >>>> depend on the readTimeout in this situation. (waiting on a close_notify >>>> requires performing a read so the read timeout makes sense in this >>>> context) I'm happy to alter that but I think that the combination of >>>> a timeout and a retry count is straightforward and lower impact. >>>> >>>> In my opinion the default behaviour of potentially hanging indefinitely >>>> is worse than the alternative here. (bearing in mind that we are closing >>>> the underlying socket) >>>> >>>> I'll file a CSR as soon as we settle on the direction this fix will >>>> take. >>>> >>>> -Rob >>>> >>>> On 13/09/17 05:52, Xuelei Fan wrote: >>>>> In theory, there are intermittent compatibility problems as this update may >>>>> not close the SSL connection over the existing socket layer gracefully, even >>>>> for high speed networking environments, while the underlying socket is >>>>> alive. The impact could be serious in some environment. >>>>> >>>>> For safe, I may suggest turn this countermeasure off by default. And >>>>> providing options to turn on this countermeasure: >>>>> 1. Close the SSL connection gracefully by default; or >>>>> 2. Close the SSL connection after a timeout. >>>>> >>>>> It's hardly to say 5 times receiving timeout is better/safer than timeout >>>>> once in this context. As you have already had a system property to control, >>>>> you may be able to use options other than the customized socket receiving >>>>> timeout, so that the closing timeout is not mixed/confused/dependent on/with >>>>> the receiving timeout. >>>>> >>>>> Put all together: >>>>> 1. define a closing timeout, for example "jdk.tls.waitForClose". >>>>> 2. the property default value is zero, no behavior changes. >>>>> 3. applications can set positive milliseconds value for the property. The >>>>> SSL connection will be closed in the set milliseconds (or about the maximum >>>>> value between SO_TIMEOUT and closing timeout), the connection is not grant >>>>> to be gracefully. >>>>> >>>>> What do you think? >>>>> >>>>> BTW, please file a CSR as this update is introducing an external system >>>>> property. >>>>> >>>>> Thanks, >>>>> Xuelei >>>>> >>>>> On 9/11/2017 3:29 PM, Rob McKenna wrote: >>>>>> Hi folks, >>>>>> >>>>>> In high latency environments a client SSLSocket with autoClose set to false >>>>>> can hang indefinitely if it does not correctly recieve a close_notify >>>>> >from the server. >>>>>> >>>>>> In order to rectify this situation I would like to suggest that we >>>>>> implement an integer JDK property (jdk.tls.closeRetries) which instructs >>>>>> waitForClose to attempt the close no more times than the value of the >>>>>> property. I would also suggest that 5 is a reasonable default. >>>>>> >>>>>> Note: each attempt times out based on the value of >>>>>> Socket.setSoTimeout(int timeout). >>>>>> >>>>>> Also, the behaviour here is similar to that of waitForClose() when >>>>>> autoClose is set to true, less the retries. >>>>>> >>>>>> http://cr.openjdk.java.net/~robm/8184328/webrev.01/ >>>>>> >>>>>> -Rob >>>>>> >>> From rob.mckenna at oracle.com Fri Sep 15 14:07:59 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 15 Sep 2017 15:07:59 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <850d1b2a-18af-0f05-a812-4d0f181b9404@oracle.com> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <850d1b2a-18af-0f05-a812-4d0f181b9404@oracle.com> Message-ID: <20170915140759.GB4404@vimes> But they are inextricably linked regardless. When we close an SSLSocket it performs a readReply which is subject to the read timeout. So if no read timeout is specified, the call to readReply will hang indefinitely. If a read timeout is specified we would need to maintain two separate timeouts and take each into account when polling. What you are suggesting would effectively necessitate a reimplementation of the close mechanics discarding the read timeout completely. (which would be a significant enough change in terms of compatibility) -Rob On 13/09/17 03:56, Xuelei Fan wrote: > On 9/13/2017 8:52 AM, Rob McKenna wrote: > >W.r.t. a separate timeout, the underlying mechanics of a close already > >depend on the readTimeout in this situation. > That's a concerns of mine. In order to work for your countermeasure, > applications have to set receiving timeout, and take care of the closing > timeout when evaluate what's a right timeout value. The mixing could be > misleading and not easy to use. > > Xuelei From rob.mckenna at oracle.com Fri Sep 15 14:16:50 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 15 Sep 2017 15:16:50 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> Message-ID: <20170915141650.GC4404@vimes> On 13/09/17 03:52, Xuelei Fan wrote: > > > On 9/13/2017 8:52 AM, Rob McKenna wrote: > >Hi Xuelei, > > > >This behaviour is already exposed via the autoclose boolean in: > > > >https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- > > > I did not get the point. What do you mean by this behavior is already > exposed? In SSLSocketImpl.closeSocket() waitForClose is only called if autoclose is true. If not the SSLSocket simply calls super.close(). > > >My position would be that allowing 5 retries allows us to say with some > >confidence that we're not going to get a close_notify from the server. > You have more chance to get the close_notify, but it does not mean you can > always get the close_notify in 5 retries. When you cannot get it, something > bad happens. No, the property would need to be tuned to suit the networking environment in which the application is deployed. Much the same as a timeout would be. > > >If this is the case I think its reasonable to close the connection. > > > >W.r.t. a separate timeout, the underlying mechanics of a close already > >depend on the readTimeout in this situation. (waiting on a close_notify > >requires performing a read so the read timeout makes sense in this > >context) I'm happy to alter that but I think that the combination of > >a timeout and a retry count is straightforward and lower impact. > > > >In my opinion the default behaviour of potentially hanging indefinitely > >is worse than the alternative here. (bearing in mind that we are closing > >the underlying socket) > > > I did not get the point, are we really closing the underlying socket (or the > layered ssl connection?) for the context of you update? We're calling fatal which calls closeSocket which in turn calls super.close(). (this calls Socket.close() via BaseSSLSocketImpl / SSLSocket) As noted in an earlier reply, this will close the underlying native socket. (I'll perform more testing to verify this) -Rob > > Xuelei > > >I'll file a CSR as soon as we settle on the direction this fix will > >take. > > > > -Rob > > > >On 13/09/17 05:52, Xuelei Fan wrote: > >>In theory, there are intermittent compatibility problems as this update may > >>not close the SSL connection over the existing socket layer gracefully, even > >>for high speed networking environments, while the underlying socket is > >>alive. The impact could be serious in some environment. > >> > >>For safe, I may suggest turn this countermeasure off by default. And > >>providing options to turn on this countermeasure: > >>1. Close the SSL connection gracefully by default; or > >>2. Close the SSL connection after a timeout. > >> > >>It's hardly to say 5 times receiving timeout is better/safer than timeout > >>once in this context. As you have already had a system property to control, > >>you may be able to use options other than the customized socket receiving > >>timeout, so that the closing timeout is not mixed/confused/dependent on/with > >>the receiving timeout. > >> > >>Put all together: > >>1. define a closing timeout, for example "jdk.tls.waitForClose". > >>2. the property default value is zero, no behavior changes. > >>3. applications can set positive milliseconds value for the property. The > >>SSL connection will be closed in the set milliseconds (or about the maximum > >>value between SO_TIMEOUT and closing timeout), the connection is not grant > >>to be gracefully. > >> > >>What do you think? > >> > >>BTW, please file a CSR as this update is introducing an external system > >>property. > >> > >>Thanks, > >>Xuelei > >> > >>On 9/11/2017 3:29 PM, Rob McKenna wrote: > >>>Hi folks, > >>> > >>>In high latency environments a client SSLSocket with autoClose set to false > >>>can hang indefinitely if it does not correctly recieve a close_notify > >>>from the server. > >>> > >>>In order to rectify this situation I would like to suggest that we > >>>implement an integer JDK property (jdk.tls.closeRetries) which instructs > >>>waitForClose to attempt the close no more times than the value of the > >>>property. I would also suggest that 5 is a reasonable default. > >>> > >>>Note: each attempt times out based on the value of > >>>Socket.setSoTimeout(int timeout). > >>> > >>>Also, the behaviour here is similar to that of waitForClose() when > >>>autoClose is set to true, less the retries. > >>> > >>>http://cr.openjdk.java.net/~robm/8184328/webrev.01/ > >>> > >>> -Rob > >>> From xuelei.fan at oracle.com Fri Sep 15 14:23:12 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 15 Sep 2017 07:23:12 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170915140759.GB4404@vimes> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <850d1b2a-18af-0f05-a812-4d0f181b9404@oracle.com> <20170915140759.GB4404@vimes> Message-ID: <551f5dca-ea63-e0c6-cb44-3fc5a241bd95@oracle.com> On 9/15/2017 7:07 AM, Rob McKenna wrote: > But they are inextricably linked regardless. > > When we close an SSLSocket it performs a readReply which is subject to > the read timeout. So if no read timeout is specified, the call to > readReply will hang indefinitely. That's one of what I worried about. Applications have to set receiving timeout in your proposal. I don't want closing timeout binding to receiving timeout. It's doable and the impact is minimal. Xuelei > If a read timeout is specified we > would need to maintain two separate timeouts and take each into account > when polling. > > What you are suggesting would effectively necessitate a reimplementation > of the close mechanics discarding the read timeout completely. (which > would be a significant enough change in terms of compatibility) > > -Rob > > On 13/09/17 03:56, Xuelei Fan wrote: >> On 9/13/2017 8:52 AM, Rob McKenna wrote: >>> W.r.t. a separate timeout, the underlying mechanics of a close already >>> depend on the readTimeout in this situation. >> That's a concerns of mine. In order to work for your countermeasure, >> applications have to set receiving timeout, and take care of the closing >> timeout when evaluate what's a right timeout value. The mixing could be >> misleading and not easy to use. >> >> Xuelei From xuelei.fan at oracle.com Fri Sep 15 14:32:21 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 15 Sep 2017 07:32:21 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170915141650.GC4404@vimes> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <20170915141650.GC4404@vimes> Message-ID: On 9/15/2017 7:16 AM, Rob McKenna wrote: > On 13/09/17 03:52, Xuelei Fan wrote: >> >> >> On 9/13/2017 8:52 AM, Rob McKenna wrote: >>> Hi Xuelei, >>> >>> This behaviour is already exposed via the autoclose boolean in: >>> >>> https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- >>> >> I did not get the point. What do you mean by this behavior is already >> exposed? > > In SSLSocketImpl.closeSocket() waitForClose is only called if autoclose > is true. If not the SSLSocket simply calls super.close(). > Did you get something different? I think waitForClose is only called if autoclose is false. No matter the autoclose is true or false, I'm not sure what do you mean by this behavior is already exposed. Can you describe more about the point. >> >>> My position would be that allowing 5 retries allows us to say with some >>> confidence that we're not going to get a close_notify from the server. >> You have more chance to get the close_notify, but it does not mean you can >> always get the close_notify in 5 retries. When you cannot get it, something >> bad happens. > > No, the property would need to be tuned to suit the networking > environment in which the application is deployed. Much the same as a > timeout would be. > >> >>> If this is the case I think its reasonable to close the connection. >>> >>> W.r.t. a separate timeout, the underlying mechanics of a close already >>> depend on the readTimeout in this situation. (waiting on a close_notify >>> requires performing a read so the read timeout makes sense in this >>> context) I'm happy to alter that but I think that the combination of >>> a timeout and a retry count is straightforward and lower impact. >>> >>> In my opinion the default behaviour of potentially hanging indefinitely >>> is worse than the alternative here. (bearing in mind that we are closing >>> the underlying socket) >>> >> I did not get the point, are we really closing the underlying socket (or the >> layered ssl connection?) for the context of you update? > > We're calling fatal which calls closeSocket which in turn calls > super.close(). (this calls Socket.close() via BaseSSLSocketImpl / > SSLSocket) As noted in an earlier reply, this will close the > underlying native socket. (I'll perform more testing to verify this) > When the fatal get called? I may miss something. Could you describe the scenarios in more details? Xuelei > -Rob > >> >> Xuelei >> >>> I'll file a CSR as soon as we settle on the direction this fix will >>> take. >>> >>> -Rob >>> >>> On 13/09/17 05:52, Xuelei Fan wrote: >>>> In theory, there are intermittent compatibility problems as this update may >>>> not close the SSL connection over the existing socket layer gracefully, even >>>> for high speed networking environments, while the underlying socket is >>>> alive. The impact could be serious in some environment. >>>> >>>> For safe, I may suggest turn this countermeasure off by default. And >>>> providing options to turn on this countermeasure: >>>> 1. Close the SSL connection gracefully by default; or >>>> 2. Close the SSL connection after a timeout. >>>> >>>> It's hardly to say 5 times receiving timeout is better/safer than timeout >>>> once in this context. As you have already had a system property to control, >>>> you may be able to use options other than the customized socket receiving >>>> timeout, so that the closing timeout is not mixed/confused/dependent on/with >>>> the receiving timeout. >>>> >>>> Put all together: >>>> 1. define a closing timeout, for example "jdk.tls.waitForClose". >>>> 2. the property default value is zero, no behavior changes. >>>> 3. applications can set positive milliseconds value for the property. The >>>> SSL connection will be closed in the set milliseconds (or about the maximum >>>> value between SO_TIMEOUT and closing timeout), the connection is not grant >>>> to be gracefully. >>>> >>>> What do you think? >>>> >>>> BTW, please file a CSR as this update is introducing an external system >>>> property. >>>> >>>> Thanks, >>>> Xuelei >>>> >>>> On 9/11/2017 3:29 PM, Rob McKenna wrote: >>>>> Hi folks, >>>>> >>>>> In high latency environments a client SSLSocket with autoClose set to false >>>>> can hang indefinitely if it does not correctly recieve a close_notify >>>> >from the server. >>>>> >>>>> In order to rectify this situation I would like to suggest that we >>>>> implement an integer JDK property (jdk.tls.closeRetries) which instructs >>>>> waitForClose to attempt the close no more times than the value of the >>>>> property. I would also suggest that 5 is a reasonable default. >>>>> >>>>> Note: each attempt times out based on the value of >>>>> Socket.setSoTimeout(int timeout). >>>>> >>>>> Also, the behaviour here is similar to that of waitForClose() when >>>>> autoClose is set to true, less the retries. >>>>> >>>>> http://cr.openjdk.java.net/~robm/8184328/webrev.01/ >>>>> >>>>> -Rob >>>>> From rob.mckenna at oracle.com Fri Sep 15 14:41:23 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 15 Sep 2017 15:41:23 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <003a2ae4-349c-cc33-fea8-5629b225e86b@oracle.com> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> <20170915140025.GA4404@vimes> <003a2ae4-349c-cc33-fea8-5629b225e86b@oracle.com> Message-ID: <20170915144123.GE4404@vimes> On 15/09/17 07:07, Xuelei Fan wrote: > On 9/15/2017 7:00 AM, Rob McKenna wrote: > >When we call close() on the SSLSocket that calls close() on the > >underlying java Socket which closes the native socket. > > > Sorry, I did not get the point. Please see the close() implementation of > SSLSocket (sun.security.ssl.SSLSocketImpl.close()) about the details. Running my original test against an instrumented 8u-dev produces the following: java.lang.Exception: Stack trace at java.lang.Thread.dumpStack(Thread.java:1336) at java.net.Socket.close(Socket.java:1491) at sun.security.ssl.BaseSSLSocketImpl.close(BaseSSLSocketImpl.java:624) at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1579) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1980) at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1793) at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1592) at sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1726) at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1615) at ssl.SSLClient.close(SSLClient.java:143) at ssl.SocketTimeoutCloseHang.ReadHang.testSSLServer(ReadHang.java:77) -Rob > > Xuelei > > > -Rob > > > >On 13/09/17 04:09, Xuelei Fan wrote: > >>It's a little bit complicated for layered SSL connections. Application can > >>build a SSL connection on existing socket (we call it layered SSL > >>connections). The problem scenarios make look like: > >>1. open a socket for applications. > >>2. established a SSL connection on the existing socket. > >>3. close the SSL connection, but leaving data in the socket. > >>4. establish another SSL connection on the socket, as the existing data in > >>the socket, the connection cannot be established. > >>5. establish another app connection on the socket, as the existing data in > >>the socket, the connection cannot be established. > >>.... > >> > >>Timeout happens even on very high speed network. If a timeout happens and > >>the SSL connection is not closed gracefully, and then the following > >>applications breaks. IMHO, we need to take care of the case. > >> > >>Xuelei > >> > >>On 9/13/2017 1:06 PM, Chris Hegarty wrote: > >>>Xuelei, > >>> > >>>Without diving deeper into this issue, Rob?s suggested approach seems reasonable to me, and better than existing out-of-the-box behaviour. I?m not sure what issues you are thinking of, with using the read timeout in combination with a retry mechanism, in this manner? If the network is so slow, surely there will be other issues with connecting and reading, why is closing any different. > >>> > >>>-Chris. > >>> > >>>>On 13 Sep 2017, at 16:52, Rob McKenna wrote: > >>>> > >>>>Hi Xuelei, > >>>> > >>>>This behaviour is already exposed via the autoclose boolean in: > >>>> > >>>>https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- > >>>> > >>>>My position would be that allowing 5 retries allows us to say with some > >>>>confidence that we're not going to get a close_notify from the server. > >>>>If this is the case I think its reasonable to close the connection. > >>>> > >>>>W.r.t. a separate timeout, the underlying mechanics of a close already > >>>>depend on the readTimeout in this situation. (waiting on a close_notify > >>>>requires performing a read so the read timeout makes sense in this > >>>>context) I'm happy to alter that but I think that the combination of > >>>>a timeout and a retry count is straightforward and lower impact. > >>>> > >>>>In my opinion the default behaviour of potentially hanging indefinitely > >>>>is worse than the alternative here. (bearing in mind that we are closing > >>>>the underlying socket) > >>>> > >>>>I'll file a CSR as soon as we settle on the direction this fix will > >>>>take. > >>>> > >>>> -Rob > >>>> > >>>>On 13/09/17 05:52, Xuelei Fan wrote: > >>>>>In theory, there are intermittent compatibility problems as this update may > >>>>>not close the SSL connection over the existing socket layer gracefully, even > >>>>>for high speed networking environments, while the underlying socket is > >>>>>alive. The impact could be serious in some environment. > >>>>> > >>>>>For safe, I may suggest turn this countermeasure off by default. And > >>>>>providing options to turn on this countermeasure: > >>>>>1. Close the SSL connection gracefully by default; or > >>>>>2. Close the SSL connection after a timeout. > >>>>> > >>>>>It's hardly to say 5 times receiving timeout is better/safer than timeout > >>>>>once in this context. As you have already had a system property to control, > >>>>>you may be able to use options other than the customized socket receiving > >>>>>timeout, so that the closing timeout is not mixed/confused/dependent on/with > >>>>>the receiving timeout. > >>>>> > >>>>>Put all together: > >>>>>1. define a closing timeout, for example "jdk.tls.waitForClose". > >>>>>2. the property default value is zero, no behavior changes. > >>>>>3. applications can set positive milliseconds value for the property. The > >>>>>SSL connection will be closed in the set milliseconds (or about the maximum > >>>>>value between SO_TIMEOUT and closing timeout), the connection is not grant > >>>>>to be gracefully. > >>>>> > >>>>>What do you think? > >>>>> > >>>>>BTW, please file a CSR as this update is introducing an external system > >>>>>property. > >>>>> > >>>>>Thanks, > >>>>>Xuelei > >>>>> > >>>>>On 9/11/2017 3:29 PM, Rob McKenna wrote: > >>>>>>Hi folks, > >>>>>> > >>>>>>In high latency environments a client SSLSocket with autoClose set to false > >>>>>>can hang indefinitely if it does not correctly recieve a close_notify > >>>>>>from the server. > >>>>>> > >>>>>>In order to rectify this situation I would like to suggest that we > >>>>>>implement an integer JDK property (jdk.tls.closeRetries) which instructs > >>>>>>waitForClose to attempt the close no more times than the value of the > >>>>>>property. I would also suggest that 5 is a reasonable default. > >>>>>> > >>>>>>Note: each attempt times out based on the value of > >>>>>>Socket.setSoTimeout(int timeout). > >>>>>> > >>>>>>Also, the behaviour here is similar to that of waitForClose() when > >>>>>>autoClose is set to true, less the retries. > >>>>>> > >>>>>>http://cr.openjdk.java.net/~robm/8184328/webrev.01/ > >>>>>> > >>>>>> -Rob > >>>>>> > >>> From rob.mckenna at oracle.com Fri Sep 15 14:44:05 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 15 Sep 2017 15:44:05 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <551f5dca-ea63-e0c6-cb44-3fc5a241bd95@oracle.com> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <850d1b2a-18af-0f05-a812-4d0f181b9404@oracle.com> <20170915140759.GB4404@vimes> <551f5dca-ea63-e0c6-cb44-3fc5a241bd95@oracle.com> Message-ID: <20170915144405.GF4404@vimes> Perhaps I'm misunderstanding you here. Can you illustrate this a bit further? Applications already have to set a read timeout, my proposal doesn't alter this fact. (i.e. if the read timeout isn't set applications which call close could potentially get stuck in readReply indefinitely) -Rob On 15/09/17 07:23, Xuelei Fan wrote: > On 9/15/2017 7:07 AM, Rob McKenna wrote: > >But they are inextricably linked regardless. > > > >When we close an SSLSocket it performs a readReply which is subject to > >the read timeout. So if no read timeout is specified, the call to > >readReply will hang indefinitely. > That's one of what I worried about. Applications have to set receiving > timeout in your proposal. I don't want closing timeout binding to receiving > timeout. It's doable and the impact is minimal. > > Xuelei > > >If a read timeout is specified we > >would need to maintain two separate timeouts and take each into account > >when polling. > > > >What you are suggesting would effectively necessitate a reimplementation > >of the close mechanics discarding the read timeout completely. (which > >would be a significant enough change in terms of compatibility) > > > > -Rob > > > >On 13/09/17 03:56, Xuelei Fan wrote: > >>On 9/13/2017 8:52 AM, Rob McKenna wrote: > >>>W.r.t. a separate timeout, the underlying mechanics of a close already > >>>depend on the readTimeout in this situation. > >>That's a concerns of mine. In order to work for your countermeasure, > >>applications have to set receiving timeout, and take care of the closing > >>timeout when evaluate what's a right timeout value. The mixing could be > >>misleading and not easy to use. > >> > >>Xuelei From rob.mckenna at oracle.com Fri Sep 15 14:46:49 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 15 Sep 2017 15:46:49 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <20170915141650.GC4404@vimes> Message-ID: <20170915144649.GG4404@vimes> On 15/09/17 07:32, Xuelei Fan wrote: > On 9/15/2017 7:16 AM, Rob McKenna wrote: > >On 13/09/17 03:52, Xuelei Fan wrote: > >> > >> > >>On 9/13/2017 8:52 AM, Rob McKenna wrote: > >>>Hi Xuelei, > >>> > >>>This behaviour is already exposed via the autoclose boolean in: > >>> > >>>https://docs.oracle.com/javase/8/docs/api/javax/net/ssl/SSLSocketFactory.html#createSocket-java.net.Socket-java.io.InputStream-boolean- > >>> > >>I did not get the point. What do you mean by this behavior is already > >>exposed? > > > >In SSLSocketImpl.closeSocket() waitForClose is only called if autoclose > >is true. If not the SSLSocket simply calls super.close(). > > > Did you get something different? I think waitForClose is only called if > autoclose is false. > > No matter the autoclose is true or false, I'm not sure what do you mean by > this behavior is already exposed. Can you describe more about the point. > Ack, yes, sorry, I got that backwards. If you set autoclose to true SSLSocket.close() will skip waitForClose() and simply call super.close(). When I say this behaviour is already exposed I am referring to the call to super.close(). (which is effectively what this fix does after the specified number of attempts via the call to fatal) -Rob > >> > >>>My position would be that allowing 5 retries allows us to say with some > >>>confidence that we're not going to get a close_notify from the server. > >>You have more chance to get the close_notify, but it does not mean you can > >>always get the close_notify in 5 retries. When you cannot get it, something > >>bad happens. > > > >No, the property would need to be tuned to suit the networking > >environment in which the application is deployed. Much the same as a > >timeout would be. > > > >> > >>>If this is the case I think its reasonable to close the connection. > >>> > >>>W.r.t. a separate timeout, the underlying mechanics of a close already > >>>depend on the readTimeout in this situation. (waiting on a close_notify > >>>requires performing a read so the read timeout makes sense in this > >>>context) I'm happy to alter that but I think that the combination of > >>>a timeout and a retry count is straightforward and lower impact. > >>> > >>>In my opinion the default behaviour of potentially hanging indefinitely > >>>is worse than the alternative here. (bearing in mind that we are closing > >>>the underlying socket) > >>> > >>I did not get the point, are we really closing the underlying socket (or the > >>layered ssl connection?) for the context of you update? > > > >We're calling fatal which calls closeSocket which in turn calls > >super.close(). (this calls Socket.close() via BaseSSLSocketImpl / > >SSLSocket) As noted in an earlier reply, this will close the > >underlying native socket. (I'll perform more testing to verify this) > > > When the fatal get called? I may miss something. Could you describe the > scenarios in more details? > > Xuelei > > > -Rob > > > >> > >>Xuelei > >> > >>>I'll file a CSR as soon as we settle on the direction this fix will > >>>take. > >>> > >>> -Rob > >>> > >>>On 13/09/17 05:52, Xuelei Fan wrote: > >>>>In theory, there are intermittent compatibility problems as this update may > >>>>not close the SSL connection over the existing socket layer gracefully, even > >>>>for high speed networking environments, while the underlying socket is > >>>>alive. The impact could be serious in some environment. > >>>> > >>>>For safe, I may suggest turn this countermeasure off by default. And > >>>>providing options to turn on this countermeasure: > >>>>1. Close the SSL connection gracefully by default; or > >>>>2. Close the SSL connection after a timeout. > >>>> > >>>>It's hardly to say 5 times receiving timeout is better/safer than timeout > >>>>once in this context. As you have already had a system property to control, > >>>>you may be able to use options other than the customized socket receiving > >>>>timeout, so that the closing timeout is not mixed/confused/dependent on/with > >>>>the receiving timeout. > >>>> > >>>>Put all together: > >>>>1. define a closing timeout, for example "jdk.tls.waitForClose". > >>>>2. the property default value is zero, no behavior changes. > >>>>3. applications can set positive milliseconds value for the property. The > >>>>SSL connection will be closed in the set milliseconds (or about the maximum > >>>>value between SO_TIMEOUT and closing timeout), the connection is not grant > >>>>to be gracefully. > >>>> > >>>>What do you think? > >>>> > >>>>BTW, please file a CSR as this update is introducing an external system > >>>>property. > >>>> > >>>>Thanks, > >>>>Xuelei > >>>> > >>>>On 9/11/2017 3:29 PM, Rob McKenna wrote: > >>>>>Hi folks, > >>>>> > >>>>>In high latency environments a client SSLSocket with autoClose set to false > >>>>>can hang indefinitely if it does not correctly recieve a close_notify > >>>>>from the server. > >>>>> > >>>>>In order to rectify this situation I would like to suggest that we > >>>>>implement an integer JDK property (jdk.tls.closeRetries) which instructs > >>>>>waitForClose to attempt the close no more times than the value of the > >>>>>property. I would also suggest that 5 is a reasonable default. > >>>>> > >>>>>Note: each attempt times out based on the value of > >>>>>Socket.setSoTimeout(int timeout). > >>>>> > >>>>>Also, the behaviour here is similar to that of waitForClose() when > >>>>>autoClose is set to true, less the retries. > >>>>> > >>>>>http://cr.openjdk.java.net/~robm/8184328/webrev.01/ > >>>>> > >>>>> -Rob > >>>>> From xuelei.fan at oracle.com Fri Sep 15 14:55:01 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 15 Sep 2017 07:55:01 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170915144123.GE4404@vimes> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> <20170915140025.GA4404@vimes> <003a2ae4-349c-cc33-fea8-5629b225e86b@oracle.com> <20170915144123.GE4404@vimes> Message-ID: On 9/15/2017 7:41 AM, Rob McKenna wrote: > On 15/09/17 07:07, Xuelei Fan wrote: >> On 9/15/2017 7:00 AM, Rob McKenna wrote: >>> When we call close() on the SSLSocket that calls close() on the >>> underlying java Socket which closes the native socket. >>> >> Sorry, I did not get the point. Please see the close() implementation of >> SSLSocket (sun.security.ssl.SSLSocketImpl.close()) about the details. > > Running my original test against an instrumented 8u-dev produces the > following: > > java.lang.Exception: Stack trace > at java.lang.Thread.dumpStack(Thread.java:1336) > at java.net.Socket.close(Socket.java:1491) > at sun.security.ssl.BaseSSLSocketImpl.close(BaseSSLSocketImpl.java:624) > at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1579) > at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1980) > at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1793) > at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1592) > at sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1726) > at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1615) > at ssl.SSLClient.close(SSLClient.java:143) > at ssl.SocketTimeoutCloseHang.ReadHang.testSSLServer(ReadHang.java:77) > It is just one possible stacks of many. There are cases where no fatal() get called. For example, application call close() method directly. Xuelei From xuelei.fan at oracle.com Fri Sep 15 14:57:00 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 15 Sep 2017 07:57:00 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170915144405.GF4404@vimes> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <850d1b2a-18af-0f05-a812-4d0f181b9404@oracle.com> <20170915140759.GB4404@vimes> <551f5dca-ea63-e0c6-cb44-3fc5a241bd95@oracle.com> <20170915144405.GF4404@vimes> Message-ID: On 9/15/2017 7:44 AM, Rob McKenna wrote: > Perhaps I'm misunderstanding you here. Can you illustrate this a bit > further? > The basic point is simple: removing the closing blocking even receiving timeout is not set. > Applications already have to set a read timeout I did not get the point. Applications don't have to set a read timeout. Xuelei >, my proposal doesn't > alter this fact. (i.e. if the read timeout isn't set applications which > call close could potentially get stuck in readReply indefinitely) > > -Rob > > On 15/09/17 07:23, Xuelei Fan wrote: >> On 9/15/2017 7:07 AM, Rob McKenna wrote: >>> But they are inextricably linked regardless. >>> >>> When we close an SSLSocket it performs a readReply which is subject to >>> the read timeout. So if no read timeout is specified, the call to >>> readReply will hang indefinitely. >> That's one of what I worried about. Applications have to set receiving >> timeout in your proposal. I don't want closing timeout binding to receiving >> timeout. It's doable and the impact is minimal. >> >> Xuelei >> >>> If a read timeout is specified we >>> would need to maintain two separate timeouts and take each into account >>> when polling. >>> >>> What you are suggesting would effectively necessitate a reimplementation >>> of the close mechanics discarding the read timeout completely. (which >>> would be a significant enough change in terms of compatibility) >>> >>> -Rob >>> >>> On 13/09/17 03:56, Xuelei Fan wrote: >>>> On 9/13/2017 8:52 AM, Rob McKenna wrote: >>>>> W.r.t. a separate timeout, the underlying mechanics of a close already >>>>> depend on the readTimeout in this situation. >>>> That's a concerns of mine. In order to work for your countermeasure, >>>> applications have to set receiving timeout, and take care of the closing >>>> timeout when evaluate what's a right timeout value. The mixing could be >>>> misleading and not easy to use. >>>> >>>> Xuelei From rob.mckenna at oracle.com Fri Sep 15 15:18:46 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 15 Sep 2017 16:18:46 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <850d1b2a-18af-0f05-a812-4d0f181b9404@oracle.com> <20170915140759.GB4404@vimes> <551f5dca-ea63-e0c6-cb44-3fc5a241bd95@oracle.com> <20170915144405.GF4404@vimes> Message-ID: <20170915151846.GH4404@vimes> Ah, right! This is the part I was missing. So my fix is intended to address this specific circumstance only (where we get caught in the while loop in waitForClose() indefinitely despite having set a read timeout). In this situation it would be reasonable for somebody to set a read timeout in the hope that the close() call would not hang indefinitely. Unfortunately due to the while loop in waitForClose it does regardless. (hence my assertion that applications already have to set a read timeout to attempt to avoid this situation) So you're suggesting that we take the read timeout out of the close mechanics completely and replace it with something more appropriate? Given that closing an SSLSocket requires a read operation in order to receive the close_notify though, I'm not sure how to accomplish that. Can you go into a little bit more detail? -Rob On 15/09/17 07:57, Xuelei Fan wrote: > On 9/15/2017 7:44 AM, Rob McKenna wrote: > >Perhaps I'm misunderstanding you here. Can you illustrate this a bit > >further? > > > The basic point is simple: removing the closing blocking even receiving > timeout is not set. > > >Applications already have to set a read timeout > I did not get the point. Applications don't have to set a read timeout. > > Xuelei > > >, my proposal doesn't > >alter this fact. (i.e. if the read timeout isn't set applications which > >call close could potentially get stuck in readReply indefinitely) > > > > -Rob > > > >On 15/09/17 07:23, Xuelei Fan wrote: > >>On 9/15/2017 7:07 AM, Rob McKenna wrote: > >>>But they are inextricably linked regardless. > >>> > >>>When we close an SSLSocket it performs a readReply which is subject to > >>>the read timeout. So if no read timeout is specified, the call to > >>>readReply will hang indefinitely. > >>That's one of what I worried about. Applications have to set receiving > >>timeout in your proposal. I don't want closing timeout binding to receiving > >>timeout. It's doable and the impact is minimal. > >> > >>Xuelei > >> > >>>If a read timeout is specified we > >>>would need to maintain two separate timeouts and take each into account > >>>when polling. > >>> > >>>What you are suggesting would effectively necessitate a reimplementation > >>>of the close mechanics discarding the read timeout completely. (which > >>>would be a significant enough change in terms of compatibility) > >>> > >>> -Rob > >>> > >>>On 13/09/17 03:56, Xuelei Fan wrote: > >>>>On 9/13/2017 8:52 AM, Rob McKenna wrote: > >>>>>W.r.t. a separate timeout, the underlying mechanics of a close already > >>>>>depend on the readTimeout in this situation. > >>>>That's a concerns of mine. In order to work for your countermeasure, > >>>>applications have to set receiving timeout, and take care of the closing > >>>>timeout when evaluate what's a right timeout value. The mixing could be > >>>>misleading and not easy to use. > >>>> > >>>>Xuelei From rob.mckenna at oracle.com Fri Sep 15 15:22:30 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 15 Sep 2017 16:22:30 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> <20170915140025.GA4404@vimes> <003a2ae4-349c-cc33-fea8-5629b225e86b@oracle.com> <20170915144123.GE4404@vimes> Message-ID: <20170915152230.GI4404@vimes> This test calls close directly. (3rd last line in the stack) I believe this is the only possible stack (with the new parameter) once autoclose is set to false. If autoclose is true we'd skip the call to waitForClose and just go directly to Socket.close() unless I'm mistaken. -Rob On 15/09/17 07:55, Xuelei Fan wrote: > On 9/15/2017 7:41 AM, Rob McKenna wrote: > >On 15/09/17 07:07, Xuelei Fan wrote: > >>On 9/15/2017 7:00 AM, Rob McKenna wrote: > >>>When we call close() on the SSLSocket that calls close() on the > >>>underlying java Socket which closes the native socket. > >>> > >>Sorry, I did not get the point. Please see the close() implementation of > >>SSLSocket (sun.security.ssl.SSLSocketImpl.close()) about the details. > > > >Running my original test against an instrumented 8u-dev produces the > >following: > > > >java.lang.Exception: Stack trace > > at java.lang.Thread.dumpStack(Thread.java:1336) > > at java.net.Socket.close(Socket.java:1491) > > at sun.security.ssl.BaseSSLSocketImpl.close(BaseSSLSocketImpl.java:624) > > at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1579) > > at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1980) > > at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1793) > > at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1592) > > at sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1726) > > at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1615) > > at ssl.SSLClient.close(SSLClient.java:143) > > at ssl.SocketTimeoutCloseHang.ReadHang.testSSLServer(ReadHang.java:77) > > > It is just one possible stacks of many. There are cases where no fatal() > get called. For example, application call close() method directly. > > Xuelei From xuelei.fan at oracle.com Fri Sep 15 23:32:23 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 15 Sep 2017 16:32:23 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170915152230.GI4404@vimes> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> <20170915140025.GA4404@vimes> <003a2ae4-349c-cc33-fea8-5629b225e86b@oracle.com> <20170915144123.GE4404@vimes> <20170915152230.GI4404@vimes> Message-ID: <9d9ab26c-add2-bf2f-ca6d-6592214e03bd@oracle.com> On 9/15/2017 8:22 AM, Rob McKenna wrote: > This test calls close directly. (3rd last line in the stack) > > I believe this is the only possible stack (with the new parameter) once > autoclose is set to false. If autoclose is true we'd skip the call to > waitForClose and just go directly to Socket.close() unless I'm mistaken. > I did not find the call to fatal() in the current implementation. I think you mean you added the call to fatal() in your update so that when timeout, a fatal() will always get called? Thinking about two things: 1. application have to set receiving timeout in order to get receiving timeout. I have a concern about it, as described in other comments. 2. can we close the super socket? It is a surprise to me to close super socket even we don't allocate it. It does not feel right to me, but this is the current behavior. All right, I get your point. Xuelei > -Rob > > On 15/09/17 07:55, Xuelei Fan wrote: >> On 9/15/2017 7:41 AM, Rob McKenna wrote: >>> On 15/09/17 07:07, Xuelei Fan wrote: >>>> On 9/15/2017 7:00 AM, Rob McKenna wrote: >>>>> When we call close() on the SSLSocket that calls close() on the >>>>> underlying java Socket which closes the native socket. >>>>> >>>> Sorry, I did not get the point. Please see the close() implementation of >>>> SSLSocket (sun.security.ssl.SSLSocketImpl.close()) about the details. >>> >>> Running my original test against an instrumented 8u-dev produces the >>> following: >>> >>> java.lang.Exception: Stack trace >>> at java.lang.Thread.dumpStack(Thread.java:1336) >>> at java.net.Socket.close(Socket.java:1491) >>> at sun.security.ssl.BaseSSLSocketImpl.close(BaseSSLSocketImpl.java:624) >>> at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1579) >>> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1980) >>> at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1793) >>> at sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1592) >>> at sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1726) >>> at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1615) >>> at ssl.SSLClient.close(SSLClient.java:143) >>> at ssl.SocketTimeoutCloseHang.ReadHang.testSSLServer(ReadHang.java:77) >>> >> It is just one possible stacks of many. There are cases where no fatal() >> get called. For example, application call close() method directly. >> >> Xuelei From xuelei.fan at oracle.com Fri Sep 15 23:38:47 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Fri, 15 Sep 2017 16:38:47 -0700 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <9d9ab26c-add2-bf2f-ca6d-6592214e03bd@oracle.com> References: <20170911222954.GC3791@vimes> <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> <20170915140025.GA4404@vimes> <003a2ae4-349c-cc33-fea8-5629b225e86b@oracle.com> <20170915144123.GE4404@vimes> <20170915152230.GI4404@vimes> <9d9ab26c-add2-bf2f-ca6d-6592214e03bd@oracle.com> Message-ID: <3044d68a-403f-2af8-099b-184d70efe1ee@oracle.com> I still prefer to not-depends on socket receiving timeout. But I'm fine if you want to move on with it. As we can close the super socket in the current implementation, it implies that application can handle it already. So you may not need the system property and 5 times retries. I think it's fine just call fatal() for the first timeout. Xuelei On 9/15/2017 4:32 PM, Xuelei Fan wrote: > On 9/15/2017 8:22 AM, Rob McKenna wrote: >> This test calls close directly. (3rd last line in the stack) >> >> I believe this is the only possible stack (with the new parameter) once >> autoclose is set to false. If autoclose is true we'd skip the call to >> waitForClose and just go directly to Socket.close() unless I'm mistaken. >> > I did not find the call to fatal() in the current implementation.? I > think you mean you added the call to fatal() in your update so that when > timeout, a fatal() will always get called? > > Thinking about two things: > 1. application have to set receiving timeout in order to? get receiving > timeout. > I have a concern about it, as described in other comments. > > 2. can we close the super socket? > It is a surprise to me to close super socket even we don't allocate it. > It does not feel right to me, but this is the current behavior.? All > right, I get your point. > > Xuelei > >> ???? -Rob >> >> On 15/09/17 07:55, Xuelei Fan wrote: >>> On 9/15/2017 7:41 AM, Rob McKenna wrote: >>>> On 15/09/17 07:07, Xuelei Fan wrote: >>>>> On 9/15/2017 7:00 AM, Rob McKenna wrote: >>>>>> When we call close() on the SSLSocket that calls close() on the >>>>>> underlying java Socket which closes the native socket. >>>>>> >>>>> Sorry, I did not get the point.? Please see the close() >>>>> implementation of >>>>> SSLSocket (sun.security.ssl.SSLSocketImpl.close()) about the details. >>>> >>>> Running my original test against an instrumented 8u-dev produces the >>>> following: >>>> >>>> java.lang.Exception: Stack trace >>>> ????at java.lang.Thread.dumpStack(Thread.java:1336) >>>> ????at java.net.Socket.close(Socket.java:1491) >>>> ????at >>>> sun.security.ssl.BaseSSLSocketImpl.close(BaseSSLSocketImpl.java:624) >>>> ????at >>>> sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1579) >>>> ????at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1980) >>>> ????at >>>> sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1793) >>>> ????at >>>> sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1592) >>>> ????at >>>> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1726) >>>> ????at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1615) >>>> ????at ssl.SSLClient.close(SSLClient.java:143) >>>> ????at >>>> ssl.SocketTimeoutCloseHang.ReadHang.testSSLServer(ReadHang.java:77) >>>> >>> It is just one possible stacks of many.? There are cases where no >>> fatal() >>> get called.? For example, application call close() method directly. >>> >>> Xuelei From philipp.kunz at paratix.ch Sun Sep 17 19:25:44 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Sun, 17 Sep 2017 21:25:44 +0200 Subject: JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> Message-ID: Hello Vincent I narrowed the error down so far and suspect now that it is an effect of sun.security.util.ManifestDigester's constructor, lines 134 and 163-164, in combination with java.util.jar.Manifest.make72Safe. Manifest breaks multi-byte characters in manifest section names across lines and ManifestDigester fails to restore them correctly, which both sounds wrong but ManifestDigester sure is. Just fixing it would be too tempting but then I would not know (I mean not know for sure with evidence from tests) if any existing jar-signature would still be valid and I don't want to break existing signatures. When I had a look at the tests specific for Manifest and ManifestDigester I found very little. There are many more different situations and corner cases and combinations thereof to cover with tests than it looks like at first glance so that writing tests that cover all relevant cases looks to me like quite an effort. I have the impression that test coverage was not a priority when the subjected code was developed or at least the tests might not have been contributed to the open JDK project. Hence, while I'm continuing to complete these tests I miss, if someone has an idea how to simplify that so that I can still convince at least myself that no existing signature will break or where possibly more testcases are that I haven't discovered so far, please let me know. I'm also wondering how much performance should be taken into consideration. There are a few hints such as reusing byte arrays that suggest that it is an issue. Will a patch be rejected if it slows down some tests beyond a certain limit for so little added value or what might be the acceptance criteria here? I don't expect insuperable difficulties with performance but would still appreciate some general idea. My desired or currently preferred approach to fix JDK-6695402 would be to let ManifestDigester any kind of use or extend Manifest in order to let Manifest identify the manifest sections thereby preventing the parsing duplicated that identifies the manifest sections. As a new contributor I could use and would appreciate some guidance particularly about what kind and completeness of test coverage and performance tuning applies or is suggested in the context of the given bug. Otherwise I fear I might contribute too many tests which would have to be reviewed and maintained or quite some work would be for nothing if a patch would not satisfy performance requirements. Regards, Philipp On 01.09.2017 15:20, Vincent Ryan wrote: > That all sounds fine. Let me know when your patch is ready to submit. > Thanks. > > >> On 1 Sep 2017, at 13:15, Philipp Kunz > > wrote: >> >> Hello Vincent >> >> Thank you for sponsoring! >> So far, I have become a contributor by signing the OCA which has been >> accepted. Dalibor Topic wrote that he can confirm and it's also here: >> http://www.oracle.com/technetwork/community/oca-486395.html#p -> >> Paratix GmbH >> Therefore I think I have followed the steps in >> http://openjdk.java.net/contribute/ at least the ones before actual >> patch submission. >> >> Currently, I'm working on getting the existing test cases running >> locally. Unfortunately, I started with jdk 9 and switched to 10 now. >> I figured these commands run at least the relevant tests with 9 and >> hope this also applies to 10: >> make run-test-tier1 >> make run-test TEST="jdk/test" >> they report errors and failures but it hasn't been completed and >> released which might explain it. If I run the tests I assume relevant >> for my patch >> make run-test TEST="jtreg:jdk/test/sun/security/tools/jarsigner >> jtreg:jdk/test/java/util/jar" >> then it reports zero errors and failures which may be a good starting >> point. >> Do you think that sounds reasonable or do you have another suggestion >> how to run the tests? >> >> Next, I will add a test for JDK-6695402 before actually fixing it. As >> an example, I'll try something in the style of >> http://hg.openjdk.java.net/jdk9/dev/jdk/file/65464a307408/test/java/util/jar/Manifest/CreateManifest.java. >> This way, I try to demonstrate the improvement. >> >> I guess I have identified the following line as the cause: value = >> new String(vb, 0, 0, vb.length); >> http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/51f5d60713b5/src/java.base/share/classes/java/util/jar/Manifest.java#l157 >> So I'll try to remove it first including the whole four line if block. >> >> Philipp >> >> >> On 01.09.2017 10:00, Vincent Ryan wrote: >>> Hello Philipp, >>> >>> I?m happy to sponsor your fix for JDK 10. Have you followed these >>> steps: http://openjdk.java.net/contribute/ ? >>> >>> Thanks. >>> >>> >>>> On 1 Sep 2017, at 08:58, Vincent Ryan >>> > wrote: >>>> >>>> Moved to security-dev >>>> >>>> >>>>> On 1 Sep 2017, at 08:28, Philipp Kunz >>>> > wrote: >>>>> >>>>> Hello everyone >>>>> >>>>> I have been developing with Java for around 17 years now and when >>>>> I encountered some bug I decided to attempt to fix it: >>>>> https://bugs.openjdk.java.net/browse/JDK-6695402. This also looks >>>>> like it may not be too big a piece for a first contribution. >>>>> >>>>> I read through quite some guides and all kinds of documents but >>>>> could not yet help myself with the following questions: >>>>> >>>>> May I login to jira to add comments to bugs? If so, how would I >>>>> request or receive credentials? Or are mailing lists preferred? >>>>> >>>>> Another question is whether I should apply it to jdk9, but it may >>>>> be too late now, or to jdk10, and backporting can be considered >>>>> later. Probably it wouldn't even make much a difference for the >>>>> patch itself. >>>>> >>>>> One more question I have is how or where to find the sources from >>>>> before migration to mercurial. Because some lines of code I intend >>>>> to change go back farther and in the history I find only 'initial >>>>> commit'. With such a history I might be able better to understand >>>>> why it's there and prevent to make the same mistake again. >>>>> >>>>> I guess the appropriate mailing list for above mentioned bug is >>>>> security-dev. Is it correct that I can send a patch there and just >>>>> hope for some sponsor to pick it up? Of course I'd be glad if some >>>>> sponsor would contact me and maybe provide some assistance or if >>>>> someone would confirm that sending a patch to the mailing list is >>>>> the right way to find a sponsor. >>>>> >>>>> Philipp Kunz >>>> >>> >> > -- Gruss Philipp ------------------------------------------------------------------------ Paratix GmbH St Peterhofstatt 11 8001 Z?rich +41 (0)76 397 79 35 philipp.kunz at paratix.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 5134 bytes Desc: not available URL: From weijun.wang at oracle.com Mon Sep 18 09:15:24 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 18 Sep 2017 17:15:24 +0800 Subject: RFR 8014628: Support AES Encryption with HMAC-SHA2 for Kerberos 5 Message-ID: <598A4449-E91D-4B13-A4F0-B2313BE36A8B@oracle.com> Hi All Please take a review at http://cr.openjdk.java.net/~weijun/8014628/webrev.00/ Most changes are just duplicating existing classes/methods/fields on AES-SHA1 etypes. One day we might do some refactoring to simplify this. Real changes: - AesSha2DkCrypto.java: 1. A new dr() method, explained in https://tools.ietf.org/html/rfc8009#section-3 2. etype name used in stringToKey(), explained in https://tools.ietf.org/html/rfc8009#section-4 3. A separate deriveKey() method. Not only it reduces duplicated codes, but it is also used in KerberosAesSha2.java the test. - Config.java: Previous AES-SHA1 etypes now have aliases aes128-sha1 and aes256-sha1. - EType.java: The default enctypes set now includes the new aes-sha2 etypes, but aes-sha1 etypes are more preferred. This is also what MIT krb5 is doing. - KerberosAesSha2.java Test vectors in https://tools.ietf.org/html/rfc8009#appendix-A. Thanks Max From sean.mullan at oracle.com Tue Sep 19 19:13:50 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Tue, 19 Sep 2017 15:13:50 -0400 Subject: StackOverflowError - Java 9 Build 181 In-Reply-To: References: Message-ID: <00a010a2-1a93-4f33-9eb7-1c905ca1270e@oracle.com> Cross-posting to security-dev as this is more relevant to that list and bcc-ing core-libs-dev. I think this might be an issue with the JavaWebStart SecurityManager not being granted the proper permissions. It is possible that the deployment policy files are not being loaded or there is some other subtle bootstrapping issue. It should not result in a recursive loop of course, but there may be a workaround. In the meantime, can you send me more information, preferably a test case and a log file with -Djava.security.debug=all enabled? (The latter will help analyze the recursion and see what security checks are failing and for which ProtectionDomains). Also, have you tested this on builds earlier than b181? Thanks, Sean On 9/19/17 2:53 PM, Tom Hood wrote: > I should add that we have not modified or overridden any policy files. > Also, we are not using a custom security manager. > > On Tue, Sep 19, 2017 at 11:52 AM, Tom Hood wrote: > >> Hi, >> >> I hit an infinite recursion loop probably related to PolicyFile that >> exists in Java 9 build 181 for windows 64-bit. It might be related to >> JDK-8077418 >> >> I haven't tracked down what is causing our webstart app to hit this >> problem yet, but I thought I would let you know sooner than later. Also, >> it probably is not a problem for our particular application as I should be >> able to set the security manager to null which I think/hope will bypass >> this issue. I will try today to reproduce it in our app so I can confirm >> if setting security manager to null will work for us. >> >> The stack looks like the following: (with many repeat stacks omitted) >> >> Exception in thread "AWT-EventQueue-2" java.lang.StackOverflowError >> at java.base/java.security.AccessController.doPrivileged(Native Method) >> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >> licyFile.java:1135) >> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >> licyFile.java:1082) >> at java.base/sun.security.provider.PolicyFile.implies(PolicyFil >> e.java:1038) >> at java.base/java.security.provider.ProtectionDomain.implies(Pr >> otectionDomain.java:323) >> at java.base/java.security.provider.ProtectionDomain.impliesWit >> hAltFilePerm(ProtectionDomain.java:355) >> at java.base/java.security.provider.AccessControlContext.checkP >> ermission(AccessControlContext.java:450) >> at java.base/java.security.provider.AccessController.checkPermi >> ssion(AccessController.java:895) >> at java.base/java.lang.SecurityManager.checkPermission(Security >> Manager.java:558) >> at jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec >> kPermission(JavaWebStartSecurity.java:237) >> at java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) >> at java.base/java.io.File.isDirectory(File.java:845) >> at java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >> at java.base/sun.security.provider.PolicyFile.canonicalizeCodeb >> ase(PolicyFile.java:1665) >> at java.base/sun.security.provider.PolicyFile.access$700(Policy >> File.java:263) >> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >> **** and again **** >> at java.base/java.security.AccessController.doPrivileged(Native Method) >> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >> licyFile.java:1135) >> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >> licyFile.java:1082) >> at java.base/sun.security.provider.PolicyFile.implies(PolicyFil >> e.java:1038) >> at java.base/java.security.provider.ProtectionDomain.implies(Pr >> otectionDomain.java:323) >> at java.base/java.security.provider.ProtectionDomain.impliesWit >> hAltFilePerm(ProtectionDomain.java:355) >> at java.base/java.security.provider.AccessControlContext.checkP >> ermission(AccessControlContext.java:450) >> at java.base/java.security.provider.AccessController.checkPermi >> ssion(AccessController.java:895) >> at java.base/java.lang.SecurityManager.checkPermission(Security >> Manager.java:558) >> at jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec >> kPermission(JavaWebStartSecurity.java:237) >> at java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) >> at java.base/java.io.File.isDirectory(File.java:845) >> at java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >> at java.base/sun.security.provider.PolicyFile.canonicalizeCodeb >> ase(PolicyFile.java:1665) >> at java.base/sun.security.provider.PolicyFile.access$700(Policy >> File.java:263) >> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >> **** above lines start the stack that repeats until overflow **** >> at java.base/java.security.AccessController.doPrivileged(Native Method) >> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >> licyFile.java:1135) >> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >> licyFile.java:1082) >> at java.base/sun.security.provider.PolicyFile.implies(PolicyFil >> e.java:1038) >> >> -- Tom >> >> From li.jiang at oracle.com Wed Sep 6 06:15:56 2017 From: li.jiang at oracle.com (Leo Jiang) Date: Wed, 6 Sep 2017 14:15:56 +0800 Subject: RFR 8148371: Remove policytool In-Reply-To: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> References: <76E698F8-551B-4DD8-9D42-3DA2764786EB@oracle.com> Message-ID: Hi All, Kindly reminder: please file a bug to component globalization/translation and assign to Leo Jiang , when you need to add/remove a resource file which should be/was localized. We need to update a tbom file to track all of localizable resource files. Add me to email To: list would allow me being aware of these changes. Thanks, Leo On 09/06/2017 12:17 PM, Weijun Wang wrote: > Hi All > > Please review the change, which spans to root, jdk and langtools repos. > > http://cr.openjdk.java.net/~weijun/8148371/ > > I've searched for the "policytool" word in the whole jdk10/jdk10 forests, removed all files having the word inside the path name, and remove almost all occurrences of the word in other places. > > The exceptions are: > > 1. Two files with the jdk8 word in file name. I assume I should not touch them. Please advise me. > > jdk/src/java.base/share/classes/jdk/internal/module/jdk8_packages.dat: > 1288 sun.security.tools.jarsigner > 1289 sun.security.tools.keytool > 1290: sun.security.tools.policytool > 1291 sun.security.util > 1292 sun.security.validator > > langtools/src/jdk.jdeps/share/classes/com/sun/tools/jdeps/resources/jdk8_internals.txt: > 977 sun.security.tools.jarsigner > 978 sun.security.tools.keytool > 979: sun.security.tools.policytool > 980 sun.security.util > 981 sun.security.validator > > 2. A swing test containing a full JDK 1.4.2 README text. Keep unchanged. > > jdk/test/javax/swing/JTextArea/4697612/bug4697612.txt: > 122 bin/ktab and jre/bin/ktab > 123 Kerberos key table manager > 124: bin/policytool and jre/bin/policytool > 125 Policy File Creation and Management Tool > 126 bin/orbd and jre/bin/orbd > > 3. A manual test on what resource string are used. It is based on the pre-module source layout and needs to be updated anyway. Keep unchanged this time. https://bugs.openjdk.java.net/browse/JDK-8187265 filed. > > jdk/test/sun/security/util/Resources/NewResourcesNames.java: > 62 "sun/security/tools/jarsigner/Resources.java", > 63 "sun/security/tools/keytool/Resources.java", > 64: "sun/security/tools/policytool/Resources.java", > 65 "sun/security/util/Resources.java", > 66 "sun/security/util/AuthResources.java", > .. > 103 // > 104 // which is mismatch. There are only two such special cases list above. > 105: // For KeyTool, there are 3 calls for showing help. For PolicyTool, 3 > 106 // for name prefixed with POLICY. They are covered in the two special > 107 // cases above. > > There are some Japanese man pages containing the word. I've filed another bug (https://bugs.openjdk.java.net/browse/JDK-8187262) on it. > > Thanks > Max > From tom.w.hood at gmail.com Tue Sep 19 21:55:32 2017 From: tom.w.hood at gmail.com (Tom Hood) Date: Tue, 19 Sep 2017 14:55:32 -0700 Subject: StackOverflowError - Java 9 Build 181 In-Reply-To: <00a010a2-1a93-4f33-9eb7-1c905ca1270e@oracle.com> References: <00a010a2-1a93-4f33-9eb7-1c905ca1270e@oracle.com> Message-ID: No luck so far reproducing this problem. The two times it happened to me yesterday have both been with Java 9 build 181 and the application has been idle for awhile. I login to our application, execute various features of the application, go to a meeting, return, and then see the java console repeatedly displaying the stack overflow exception. Maybe meetings are bad for Java 9? :-) I think there are some background threads in our application that are waking up periodically and doing "stuff". I don't know what that "stuff" is yet, but that would be my guess at where I will find the code that triggered the overflow. Assuming I can get our application to the point where I can reproduce the stack overflow, are there particular Java 9 builds that made significant changes to security-relevant code that you'd like me to try? Keep in mind that our app runs on a network not connected to the internet. As it is, I manually typed in the stack trace, so if there's a lot of output I'll have to print it and go through an approval process to show it to you via a scanned pdf. I will continue testing of our app with the security debug turned on so that I'll have the output if it happens again. I also have the logging and tracing enabled in the java control panel. -- Tom On Tue, Sep 19, 2017 at 12:13 PM, Sean Mullan wrote: > Cross-posting to security-dev as this is more relevant to that list and > bcc-ing core-libs-dev. > > I think this might be an issue with the JavaWebStart SecurityManager not > being granted the proper permissions. It is possible that the deployment > policy files are not being loaded or there is some other subtle > bootstrapping issue. It should not result in a recursive loop of course, > but there may be a workaround. > > In the meantime, can you send me more information, preferably a test case > and a log file with -Djava.security.debug=all enabled? (The latter will > help analyze the recursion and see what security checks are failing and for > which ProtectionDomains). Also, have you tested this on builds earlier than > b181? > > Thanks, > Sean > > On 9/19/17 2:53 PM, Tom Hood wrote: > >> I should add that we have not modified or overridden any policy files. >> Also, we are not using a custom security manager. >> >> On Tue, Sep 19, 2017 at 11:52 AM, Tom Hood wrote: >> >> Hi, >>> >>> I hit an infinite recursion loop probably related to PolicyFile that >>> exists in Java 9 build 181 for windows 64-bit. It might be related to >>> JDK-8077418 >>> >>> >>> I haven't tracked down what is causing our webstart app to hit this >>> problem yet, but I thought I would let you know sooner than later. Also, >>> it probably is not a problem for our particular application as I should >>> be >>> able to set the security manager to null which I think/hope will bypass >>> this issue. I will try today to reproduce it in our app so I can confirm >>> if setting security manager to null will work for us. >>> >>> The stack looks like the following: (with many repeat stacks omitted) >>> >>> Exception in thread "AWT-EventQueue-2" java.lang.StackOverflowError >>> at java.base/java.security.AccessController.doPrivileged(Native Method) >>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >>> licyFile.java:1135) >>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >>> licyFile.java:1082) >>> at java.base/sun.security.provider.PolicyFile.implies(PolicyFil >>> e.java:1038) >>> at java.base/java.security.provider.ProtectionDomain.implies(Pr >>> otectionDomain.java:323) >>> at java.base/java.security.provider.ProtectionDomain.impliesWit >>> hAltFilePerm(ProtectionDomain.java:355) >>> at java.base/java.security.provider.AccessControlContext.checkP >>> ermission(AccessControlContext.java:450) >>> at java.base/java.security.provider.AccessController.checkPermi >>> ssion(AccessController.java:895) >>> at java.base/java.lang.SecurityManager.checkPermission(Security >>> Manager.java:558) >>> at jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec >>> kPermission(JavaWebStartSecurity.java:237) >>> at java.base/java.lang.SecurityManager.checkRead(SecurityManage >>> r.java:897) >>> at java.base/java.io.File.isDirectory(File.java:845) >>> at java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >>> at java.base/sun.security.provider.PolicyFile.canonicalizeCodeb >>> ase(PolicyFile.java:1665) >>> at java.base/sun.security.provider.PolicyFile.access$700(Policy >>> File.java:263) >>> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile. >>> java:1139) >>> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile. >>> java:1136) >>> **** and again **** >>> at java.base/java.security.AccessController.doPrivileged(Native Method) >>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >>> licyFile.java:1135) >>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >>> licyFile.java:1082) >>> at java.base/sun.security.provider.PolicyFile.implies(PolicyFil >>> e.java:1038) >>> at java.base/java.security.provider.ProtectionDomain.implies(Pr >>> otectionDomain.java:323) >>> at java.base/java.security.provider.ProtectionDomain.impliesWit >>> hAltFilePerm(ProtectionDomain.java:355) >>> at java.base/java.security.provider.AccessControlContext.checkP >>> ermission(AccessControlContext.java:450) >>> at java.base/java.security.provider.AccessController.checkPermi >>> ssion(AccessController.java:895) >>> at java.base/java.lang.SecurityManager.checkPermission(Security >>> Manager.java:558) >>> at jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec >>> kPermission(JavaWebStartSecurity.java:237) >>> at java.base/java.lang.SecurityManager.checkRead(SecurityManage >>> r.java:897) >>> at java.base/java.io.File.isDirectory(File.java:845) >>> at java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >>> at java.base/sun.security.provider.PolicyFile.canonicalizeCodeb >>> ase(PolicyFile.java:1665) >>> at java.base/sun.security.provider.PolicyFile.access$700(Policy >>> File.java:263) >>> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile. >>> java:1139) >>> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile. >>> java:1136) >>> **** above lines start the stack that repeats until overflow **** >>> at java.base/java.security.AccessController.doPrivileged(Native Method) >>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >>> licyFile.java:1135) >>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po >>> licyFile.java:1082) >>> at java.base/sun.security.provider.PolicyFile.implies(PolicyFil >>> e.java:1038) >>> >>> -- Tom >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.kunz at paratix.ch Tue Sep 19 23:41:03 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Wed, 20 Sep 2017 01:41:03 +0200 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> Message-ID: <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> Hello Vincent Here may be the fix for JDK-6695402. First a test. BEGIN /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java /* * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it * under the terms of the GNU General Public License version 2 only, as * published by the Free Software Foundation. * * This code is distributed in the hope that it will be useful, but WITHOUT * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License * version 2 for more details (a copy is included in the LICENSE file that * accompanied this code). * * You should have received a copy of the GNU General Public License version * 2 along with this work; if not, write to the Free Software Foundation, * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. * * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA * or visit www.oracle.com if you need additional information or have any * questions. */ /* * @test * @bug JDK-6695402 * @summary verify signatures of jars containing classes with names with multi- * byte unicode characters broken across lines * @library /test/lib * @modules jdk.jartool/sun.tools.jar * jdk.jartool/sun.security.tools.jarsigner * @build jdk.test.lib.JDKToolLauncher * jdk.test.lib.process.* * @run main MultibyteUnicodeName */ import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.InputStream; import java.io.UnsupportedEncodingException; import java.nio.file.Files; import java.nio.file.Paths; import java.util.jar.JarFile; import java.util.jar.Manifest; import java.util.stream.Collectors; import java.util.Arrays; import java.util.Map; import java.util.jar.Attributes.Name; import java.util.jar.JarEntry; import static java.nio.charset.StandardCharsets.UTF_8; import sun.security.tools.jarsigner.Resources; import jdk.test.lib.JDKToolLauncher; import jdk.test.lib.process.OutputAnalyzer; import jdk.test.lib.process.ProcessTools; public class MultibyteUnicodeName { public static void main(String[] args) throws Throwable { try { prepare(); testSignJar("test.jar"); testSignJarNoManifest("test-no-manifest.jar"); testSignJarUpdate("test-update.jar"); testSignJarWithIndex("test-index.jar"); testSignJarAddIndex("test-add-index.jar"); } finally { Files.deleteIfExists(Paths.get(keystoreFileName)); Files.deleteIfExists(Paths.get(ManifestFileName)); Files.deleteIfExists(Paths.get(refClassFilename)); Files.deleteIfExists(Paths.get(testClassFilename)); } } static final String alias = "a"; static final String keystoreFileName = "test.jks"; static final String ManifestFileName = "MANIFEST.MF"; static class A1234567890B1234567890C1234567890D12345678exyz { } static class A1234567890B1234567890C1234567890D12345678?xyz { } static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; static void prepare() throws Throwable { tool("keytool", "-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", "-validity", "366") .shouldHaveExitValue(0); Files.write(Paths.get(ManifestFileName), (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); copyClassToFile(refClassFilename); copyClassToFile(testClassFilename); } static void copyClassToFile(String classFilename) throws IOException { try ( InputStream asStream = MultibyteUnicodeName.class.getResourceAsStream(classFilename); ByteArrayOutputStream buf = new ByteArrayOutputStream(); ) { int b; while ((b = asStream.read()) != -1) buf.write(b); Files.write(Paths.get(classFilename), buf.toByteArray()); } } static void testSignJar(String jarFileName) throws Throwable { try { tool("jar", "cvfm", jarFileName, ManifestFileName, refClassFilename, testClassFilename) .shouldHaveExitValue(0); verifyJarSignature(jarFileName); } finally { Files.deleteIfExists(Paths.get(jarFileName)); } } static void testSignJarNoManifest(String jarFileName) throws Throwable { try { tool("jar", "cvf", jarFileName, refClassFilename, testClassFilename) .shouldHaveExitValue(0); verifyJarSignature(jarFileName); } finally { Files.deleteIfExists(Paths.get(jarFileName)); } } static void testSignJarUpdate(String jarFileName) throws Throwable { try { tool("jar", "cvfm", jarFileName, ManifestFileName, refClassFilename) .shouldHaveExitValue(0); tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) .shouldHaveExitValue(0); tool("jar", "uvf", jarFileName, testClassFilename) .shouldHaveExitValue(0); verifyJarSignature(jarFileName); } finally { Files.deleteIfExists(Paths.get(jarFileName)); } } static void testSignJarWithIndex(String jarFileName) throws Throwable { try { tool("jar", "cvfm", jarFileName, ManifestFileName, refClassFilename, testClassFilename) .shouldHaveExitValue(0); tool("jar", "iv", jarFileName).shouldHaveExitValue(0); verifyJarSignature(jarFileName); } finally { Files.deleteIfExists(Paths.get(jarFileName)); } } static void testSignJarAddIndex(String jarFileName) throws Throwable { try { tool("jar", "cvfm", jarFileName, ManifestFileName, refClassFilename, testClassFilename) .shouldHaveExitValue(0); tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) .shouldHaveExitValue(0); tool("jar", "iv", jarFileName).shouldHaveExitValue(0); verifyJarSignature(jarFileName); } finally { Files.deleteIfExists(Paths.get(jarFileName)); } } static Map jarsignerResources = Arrays.stream(new Resources().getContents()). collect(Collectors.toMap(e -> (String) e[0], e -> (String) e[1])); static void verifyJarSignature(String jarFileName) throws Throwable { // actually sign the jar tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) .shouldHaveExitValue(0); verifyClassNameLineBroken(jarFileName); // check for no unsigned entries tool("jarsigner", "-verify", "-keystore", keystoreFileName, "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) .shouldHaveExitValue(0) .shouldNotContain(jarsignerResources.get( "This.jar.contains.unsigned.entries.which.have.not.been.integrity.checked.")); // check that both classes with and without multi-byte unicode characters in their names are signed tool("jarsigner", "-verify", "-verbose", "-keystore", keystoreFileName, "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) .shouldHaveExitValue(0) .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + refClassFilename.replaceAll("\\$", "\\\\\\$") + "$") .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + testClassFilename.replaceAll("\\$", "\\\\\\$") + "$"); // check that both classes with and without multi-byte unicode characters in their names have signing certificates tool("jarsigner", "-verify", "-verbose", "-certs", "-keystore", keystoreFileName, "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) .shouldHaveExitValue(0) .shouldMatch(refClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)") .shouldMatch(testClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)"); } /** * it would be too easy to miss the actual test case by just renaming an identifier so that * the multi-byte encoded character would not any longer be broken across a line break. * * this test verifies that the actual test case is tested based on the manifest * and not based on the signature file because at the moment, the signature file * does not even contain the desired entry at all. * * this relies on the Manifest breaking lines unaware of bytes that belong to the same * multi-ybte utf characters. */ static void verifyClassNameLineBroken(String jarFileName) throws IOException { byte[] eAcute = "?".getBytes(UTF_8); byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; try ( JarFile jar = new JarFile(jarFileName); ) { if (jar.getManifest().getAttributes(testClassFilename) == null) { throw new AssertionError(testClassFilename + " not found in manifest"); } JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); try ( InputStream manifestIs = jar.getInputStream(manifestEntry); ) { int bytesMatched = 0; for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { if ((byte) b == eAcuteBroken[bytesMatched]) { bytesMatched++; if (bytesMatched == eAcuteBroken.length) { break; } } else { bytesMatched = 0; } } if (bytesMatched < eAcuteBroken.length) { throw new AssertionError("self-test failed: " + "multi-byte utf-8 character not broken across lines"); } } } } static OutputAnalyzer tool(String tool, String... args) throws Throwable { JDKToolLauncher l = JDKToolLauncher.createUsingTestJDK(tool); for (String arg : args) { if (arg .startsWith("-J")) { l.addVMArg(arg.substring(2)); } else { l.addToolArg(arg); } } return ProcessTools.executeCommand(l.getCommand()); } } END /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java There are three e with acutes, on lines 84, 89, and 227, just in case the mail suffers bad encoding which might not be absolutely unconceivable regarding the bug's subject. In contrast to the bug description it uses a nested class with a two-byte UTF-8 character rather than one in its own file. I chose to do it that way because then the complete test goes into one file and therefore I assume the overview is kept easier. The test still demonstrates exactly JDK-6695402's problem. It may not have been necessary to also test jar files with indexes but i consider it necessary to have signing unsigned as well as partially signed jars tested, so why not have one more case tested, too? There might also be a more elegant approach to get class files into a jar file as to get their contents through the class loader. I chose to use real class files rather than dummy contents in files named *.class or for instance plain text files in the jar the signing of which to be tested in order to stay as close to the original bug problem as possible even though I don't have any notion that it would make a difference. The amount of code used to copy the class file contents around is comparatively small with respect to the whole test case amount of code. For the demonstration that the multi-byte character actually makes the alleged difference, I concluded that it was necessary to have another class name not previously affected by the bug in order to compare the effects of just different names. Otherwise the signing could fail to any other reason undetectably. In order to express a condition to tell successful from failed test runs the best approach I found was to analyze the jarsigner tool's output which is in my opinion not the most desirable option. I'd have preferred output from a clearer structured api but then I guess the output format will not change too often in the foreseeable future. For instance the check for the s, m, and k flags is more pragmatical approach than obviously perfectly failsafe when operating with regular expressions to match it with the output. Other parts such as 'X.509' and 'CN=' are not even jarsigner tool resources. I put at least a reference to sun.security.tools.jarsigner.Resources. Finally, I afforded kind of a self-test that protects the test from undetected failing because renaming the main class which would move the two-byte unicode character to a position not suitable for the test. It may not be absolutely necessary. BEGIN PATH diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Wed Sep 20 00:56:15 2017 +0200 @@ -28,6 +28,7 @@ import java.security.*; import java.util.HashMap; import java.io.ByteArrayOutputStream; +import static java.nio.charset.StandardCharsets.UTF_8; /** * This class is used to compute digests on sections of the Manifest. @@ -112,7 +113,7 @@ rawBytes = bytes; entries = new HashMap<>(); - ByteArrayOutputStream baos = new ByteArrayOutputStream(); + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); Position pos = new Position(); @@ -131,50 +132,40 @@ if (len > 6) { if (isNameAttr(bytes, start)) { - StringBuilder nameBuf = new StringBuilder(sectionLen); + nameBuf.reset(); + nameBuf.write(bytes, start+6, len-6); - try { - nameBuf.append( - new String(bytes, start+6, len-6, "UTF8")); + int i = start + len; + if ((i-start) < sectionLen) { + if (bytes[i] == '\r') { + i += 2; + } else { + i += 1; + } + } - int i = start + len; - if ((i-start) < sectionLen) { - if (bytes[i] == '\r') { - i += 2; - } else { - i += 1; - } + while ((i-start) < sectionLen) { + if (bytes[i++] == ' ') { + // name is wrapped + int wrapStart = i; + while (((i-start) < sectionLen) + && (bytes[i++] != '\n')); + if (bytes[i-1] != '\n') + return; // XXX: exception? + int wrapLen; + if (bytes[i-2] == '\r') + wrapLen = i-wrapStart-2; + else + wrapLen = i-wrapStart-1; + + nameBuf.write(bytes, wrapStart, wrapLen); + } else { + break; } + } - while ((i-start) < sectionLen) { - if (bytes[i++] == ' ') { - // name is wrapped - int wrapStart = i; - while (((i-start) < sectionLen) - && (bytes[i++] != '\n')); - if (bytes[i-1] != '\n') - return; // XXX: exception? - int wrapLen; - if (bytes[i-2] == '\r') - wrapLen = i-wrapStart-2; - else - wrapLen = i-wrapStart-1; - - nameBuf.append(new String(bytes, wrapStart, - wrapLen, "UTF8")); - } else { - break; - } - } - - entries.put(nameBuf.toString(), - new Entry(start, sectionLen, sectionLenWithBlank, - rawBytes)); - - } catch (java.io.UnsupportedEncodingException uee) { - throw new IllegalStateException( - "UTF8 not available on platform"); - } + entries.put(new String(nameBuf.toByteArray(), UTF_8), + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); } } start = pos.startOfNext; END PATCH The patch is mostly so big because indentation has changed in many lines. There was baos there before without apparent use. The patch renames it and puts it into use. It came in very handy because if it would not have been there already, I would have had to defined one of my own. The main point of the patch is that manifest section names that are broken across lines at possibly arbitrary bytes are no longer converted into strings for each manifest line and then joined. Parts of names broken across lines are now joined first when they are still byte sequences and only when the complete byte sequence is available after processing all manifest lines that contain the same manifest section name decoded into a unicode string. For decoding the bytes into a string I chose a different string constructor than was used before that does not any longer declare UnsupportedEncodingException rendering the try/catch redundant. It couldn't have ever occurred anyway taking into consideration that UTF-8 is mandatory for every Java platform, StandardCharsets says. The difference according to the documentation is that the previously used String constructor returned undefined strings for invalid byte sequences whereas the one used in the patch will replace unparseable portions with valid 'unknown' characters. One question I cannot still answer yet is how the ManifestDigester can be changed at all without complete test coverage or risking to break existing signatures. I already started on that but it's quite a piece of work to almost formally prove that all manifests will continue to produce identical digests, except of course for the ones now fixed. Regards, Philipp On 17.09.2017 21:25, Philipp Kunz wrote: > Hello Vincent > > I narrowed the error down so far and suspect now that it is an effect > of sun.security.util.ManifestDigester's constructor, lines 134 and > 163-164, in combination with java.util.jar.Manifest.make72Safe. > Manifest breaks multi-byte characters in manifest section names across > lines and ManifestDigester fails to restore them correctly, which both > sounds wrong but ManifestDigester sure is. > > Just fixing it would be too tempting but then I would not know (I mean > not know for sure with evidence from tests) if any existing > jar-signature would still be valid and I don't want to break existing > signatures. When I had a look at the tests specific for Manifest and > ManifestDigester I found very little. There are many more different > situations and corner cases and combinations thereof to cover with > tests than it looks like at first glance so that writing tests that > cover all relevant cases looks to me like quite an effort. I have the > impression that test coverage was not a priority when the subjected > code was developed or at least the tests might not have been > contributed to the open JDK project. Hence, while I'm continuing to > complete these tests I miss, if someone has an idea how to simplify > that so that I can still convince at least myself that no existing > signature will break or where possibly more testcases are that I > haven't discovered so far, please let me know. > > I'm also wondering how much performance should be taken into > consideration. There are a few hints such as reusing byte arrays that > suggest that it is an issue. Will a patch be rejected if it slows down > some tests beyond a certain limit for so little added value or what > might be the acceptance criteria here? I don't expect insuperable > difficulties with performance but would still appreciate some general > idea. > > My desired or currently preferred approach to fix JDK-6695402 would be > to let ManifestDigester any kind of use or extend Manifest in order to > let Manifest identify the manifest sections thereby preventing the > parsing duplicated that identifies the manifest sections. > > As a new contributor I could use and would appreciate some guidance > particularly about what kind and completeness of test coverage and > performance tuning applies or is suggested in the context of the given > bug. Otherwise I fear I might contribute too many tests which would > have to be reviewed and maintained or quite some work would be for > nothing if a patch would not satisfy performance requirements. > > Regards, > Philipp > > > > On 01.09.2017 15:20, Vincent Ryan wrote: >> That all sounds fine. Let me know when your patch is ready to submit. >> Thanks. >> >> >>> On 1 Sep 2017, at 13:15, Philipp Kunz >> > wrote: >>> >>> Hello Vincent >>> >>> Thank you for sponsoring! >>> So far, I have become a contributor by signing the OCA which has >>> been accepted. Dalibor Topic wrote that he can confirm and it's also >>> here: http://www.oracle.com/technetwork/community/oca-486395.html#p >>> -> Paratix GmbH >>> Therefore I think I have followed the steps in >>> http://openjdk.java.net/contribute/ at least the ones before actual >>> patch submission. >>> >>> Currently, I'm working on getting the existing test cases running >>> locally. Unfortunately, I started with jdk 9 and switched to 10 now. >>> I figured these commands run at least the relevant tests with 9 and >>> hope this also applies to 10: >>> make run-test-tier1 >>> make run-test TEST="jdk/test" >>> they report errors and failures but it hasn't been completed and >>> released which might explain it. If I run the tests I assume >>> relevant for my patch >>> make run-test TEST="jtreg:jdk/test/sun/security/tools/jarsigner >>> jtreg:jdk/test/java/util/jar" >>> then it reports zero errors and failures which may be a good >>> starting point. >>> Do you think that sounds reasonable or do you have another >>> suggestion how to run the tests? >>> >>> Next, I will add a test for JDK-6695402 before actually fixing it. >>> As an example, I'll try something in the style of >>> http://hg.openjdk.java.net/jdk9/dev/jdk/file/65464a307408/test/java/util/jar/Manifest/CreateManifest.java. >>> This way, I try to demonstrate the improvement. >>> >>> I guess I have identified the following line as the cause: value = >>> new String(vb, 0, 0, vb.length); >>> http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/51f5d60713b5/src/java.base/share/classes/java/util/jar/Manifest.java#l157 >>> So I'll try to remove it first including the whole four line if block. >>> >>> Philipp >>> >>> >>> On 01.09.2017 10:00, Vincent Ryan wrote: >>>> Hello Philipp, >>>> >>>> I?m happy to sponsor your fix for JDK 10. Have you followed these >>>> steps: http://openjdk.java.net/contribute/ ? >>>> >>>> Thanks. >>>> >>>> >>>>> On 1 Sep 2017, at 08:58, Vincent Ryan >>>> > wrote: >>>>> >>>>> Moved to security-dev >>>>> >>>>> >>>>>> On 1 Sep 2017, at 08:28, Philipp Kunz >>>>> > wrote: >>>>>> >>>>>> Hello everyone >>>>>> >>>>>> I have been developing with Java for around 17 years now and when >>>>>> I encountered some bug I decided to attempt to fix it: >>>>>> https://bugs.openjdk.java.net/browse/JDK-6695402. This also looks >>>>>> like it may not be too big a piece for a first contribution. >>>>>> >>>>>> I read through quite some guides and all kinds of documents but >>>>>> could not yet help myself with the following questions: >>>>>> >>>>>> May I login to jira to add comments to bugs? If so, how would I >>>>>> request or receive credentials? Or are mailing lists preferred? >>>>>> >>>>>> Another question is whether I should apply it to jdk9, but it may >>>>>> be too late now, or to jdk10, and backporting can be considered >>>>>> later. Probably it wouldn't even make much a difference for the >>>>>> patch itself. >>>>>> >>>>>> One more question I have is how or where to find the sources from >>>>>> before migration to mercurial. Because some lines of code I >>>>>> intend to change go back farther and in the history I find only >>>>>> 'initial commit'. With such a history I might be able better to >>>>>> understand why it's there and prevent to make the same mistake again. >>>>>> >>>>>> I guess the appropriate mailing list for above mentioned bug is >>>>>> security-dev. Is it correct that I can send a patch there and >>>>>> just hope for some sponsor to pick it up? Of course I'd be glad >>>>>> if some sponsor would contact me and maybe provide some >>>>>> assistance or if someone would confirm that sending a patch to >>>>>> the mailing list is the right way to find a sponsor. >>>>>> >>>>>> Philipp Kunz >>>>> >>>> >>> >> > > > > > ------------------------------------------------------------------------ > > > > Paratix GmbH > St Peterhofstatt 11 > 8001 Z?rich > > +41 (0)76 397 79 35 > philipp.kunz at paratix.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 5134 bytes Desc: not available URL: From weijun.wang at oracle.com Wed Sep 20 01:41:43 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 20 Sep 2017 09:41:43 +0800 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> Message-ID: <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> Hi Philipp The change mostly looks fine. You might want to put everything into a patch file so Vincent can recreate a webrev and post it to cr.openjdk.java.net. One thing I would suggest for the test is that instead of using jarsigner -verify and check the text in output you can open it as a JarFile, consume the content of an entry, and look into its getCertificates(). Also, you might want to reuse test/lib/jdk/test/lib/SecurityTools.java, and there is also JarUtils.java you can use. Maybe Files.copy(InputStream,Path) can be used in copyClassToFile(). Thanks Max > On Sep 20, 2017, at 7:41 AM, Philipp Kunz wrote: > > Hello Vincent > > Here may be the fix for JDK-6695402. First a test. > > BEGIN /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java > /* > * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > * under the terms of the GNU General Public License version 2 only, as > * published by the Free Software Foundation. > * > * This code is distributed in the hope that it will be useful, but WITHOUT > * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License > * version 2 for more details (a copy is included in the LICENSE file that > * accompanied this code). > * > * You should have received a copy of the GNU General Public License version > * 2 along with this work; if not, write to the Free Software Foundation, > * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. > * > * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA > * or visit www.oracle.com if you need additional information or have any > * questions. > */ > > /* > * @test > * @bug JDK-6695402 > * @summary verify signatures of jars containing classes with names with multi- > * byte unicode characters broken across lines > * @library /test/lib > * @modules jdk.jartool/sun.tools.jar > * jdk.jartool/sun.security.tools.jarsigner > * @build jdk.test.lib.JDKToolLauncher > * jdk.test.lib.process.* > * @run main MultibyteUnicodeName > */ > > import java.io.ByteArrayOutputStream; > import java.io.IOException; > import java.io.InputStream; > import java.io.UnsupportedEncodingException; > import java.nio.file.Files; > import java.nio.file.Paths; > import java.util.jar.JarFile; > import java.util.jar.Manifest; > import java.util.stream.Collectors; > import java.util.Arrays; > import java.util.Map; > import java.util.jar.Attributes.Name; > import java.util.jar.JarEntry; > > import static java.nio.charset.StandardCharsets.UTF_8; > > import sun.security.tools.jarsigner.Resources; > > import jdk.test.lib.JDKToolLauncher; > import jdk.test.lib.process.OutputAnalyzer; > import jdk.test.lib.process.ProcessTools; > > public class MultibyteUnicodeName { > > public static void main(String[] args) throws Throwable { > try { > prepare(); > > testSignJar("test.jar"); > testSignJarNoManifest("test-no-manifest.jar"); > testSignJarUpdate("test-update.jar"); > testSignJarWithIndex("test-index.jar"); > testSignJarAddIndex("test-add-index.jar"); > > } finally { > Files.deleteIfExists(Paths.get(keystoreFileName)); > Files.deleteIfExists(Paths.get(ManifestFileName)); > Files.deleteIfExists(Paths.get(refClassFilename)); > Files.deleteIfExists(Paths.get(testClassFilename)); > } > } > > static final String alias = "a"; > static final String keystoreFileName = "test.jks"; > static final String ManifestFileName = "MANIFEST.MF"; > > static class A1234567890B1234567890C1234567890D12345678exyz { } > static class A1234567890B1234567890C1234567890D12345678?xyz { } > > static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + > "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; > static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + > "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; > > static void prepare() throws Throwable { > tool("keytool", "-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", > "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", > "-validity", "366") > .shouldHaveExitValue(0); > > Files.write(Paths.get(ManifestFileName), > (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); > copyClassToFile(refClassFilename); > copyClassToFile(testClassFilename); > } > > static void copyClassToFile(String classFilename) throws IOException { > try ( > InputStream asStream = MultibyteUnicodeName.class.getResourceAsStream(classFilename); > ByteArrayOutputStream buf = new ByteArrayOutputStream(); > ) { > int b; > while ((b = asStream.read()) != -1) buf.write(b); > Files.write(Paths.get(classFilename), buf.toByteArray()); > } > } > > static void testSignJar(String jarFileName) throws Throwable { > try { > tool("jar", "cvfm", jarFileName, > ManifestFileName, refClassFilename, testClassFilename) > .shouldHaveExitValue(0); > verifyJarSignature(jarFileName); > > } finally { > Files.deleteIfExists(Paths.get(jarFileName)); > } > } > > static void testSignJarNoManifest(String jarFileName) throws Throwable { > try { > tool("jar", "cvf", jarFileName, refClassFilename, testClassFilename) > .shouldHaveExitValue(0); > verifyJarSignature(jarFileName); > > } finally { > Files.deleteIfExists(Paths.get(jarFileName)); > } > } > > static void testSignJarUpdate(String jarFileName) throws Throwable { > try { > tool("jar", "cvfm", jarFileName, ManifestFileName, refClassFilename) > .shouldHaveExitValue(0); > tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", > "-storepass", "changeit", "-debug", jarFileName, alias) > .shouldHaveExitValue(0); > tool("jar", "uvf", jarFileName, testClassFilename) > .shouldHaveExitValue(0); > verifyJarSignature(jarFileName); > > } finally { > Files.deleteIfExists(Paths.get(jarFileName)); > } > } > > static void testSignJarWithIndex(String jarFileName) throws Throwable { > try { > tool("jar", "cvfm", jarFileName, > ManifestFileName, refClassFilename, testClassFilename) > .shouldHaveExitValue(0); > tool("jar", "iv", jarFileName).shouldHaveExitValue(0); > verifyJarSignature(jarFileName); > > } finally { > Files.deleteIfExists(Paths.get(jarFileName)); > } > } > > static void testSignJarAddIndex(String jarFileName) throws Throwable { > try { > tool("jar", "cvfm", jarFileName, > ManifestFileName, refClassFilename, testClassFilename) > .shouldHaveExitValue(0); > tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", > "-storepass", "changeit", "-debug", jarFileName, alias) > .shouldHaveExitValue(0); > tool("jar", "iv", jarFileName).shouldHaveExitValue(0); > verifyJarSignature(jarFileName); > > } finally { > Files.deleteIfExists(Paths.get(jarFileName)); > } > } > > static Map jarsignerResources = Arrays.stream(new Resources().getContents()). > collect(Collectors.toMap(e -> (String) e[0], e -> (String) e[1])); > > static void verifyJarSignature(String jarFileName) throws Throwable { > // actually sign the jar > tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", > "-storepass", "changeit", "-debug", jarFileName, alias) > .shouldHaveExitValue(0); > > verifyClassNameLineBroken(jarFileName); > > // check for no unsigned entries > tool("jarsigner", "-verify", "-keystore", keystoreFileName, "-storetype", "JKS", > "-storepass", "changeit", "-debug", jarFileName, alias) > .shouldHaveExitValue(0) > .shouldNotContain(jarsignerResources.get( > "This.jar.contains.unsigned.entries.which.have.not.been.integrity.checked.")); > > // check that both classes with and without multi-byte unicode characters in their names are signed > tool("jarsigner", "-verify", "-verbose", "-keystore", keystoreFileName, "-storetype", "JKS", > "-storepass", "changeit", "-debug", jarFileName, alias) > .shouldHaveExitValue(0) > .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + refClassFilename.replaceAll("\\$", "\\\\\\$") + "$") > .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + testClassFilename.replaceAll("\\$", "\\\\\\$") + "$"); > > // check that both classes with and without multi-byte unicode characters in their names have signing certificates > tool("jarsigner", "-verify", "-verbose", "-certs", "-keystore", keystoreFileName, > "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) > .shouldHaveExitValue(0) > .shouldMatch(refClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)") > .shouldMatch(testClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)"); > } > > /** > * it would be too easy to miss the actual test case by just renaming an identifier so that > * the multi-byte encoded character would not any longer be broken across a line break. > * > * this test verifies that the actual test case is tested based on the manifest > * and not based on the signature file because at the moment, the signature file > * does not even contain the desired entry at all. > * > * this relies on the Manifest breaking lines unaware of bytes that belong to the same > * multi-ybte utf characters. > */ > static void verifyClassNameLineBroken(String jarFileName) throws IOException { > byte[] eAcute = "?".getBytes(UTF_8); > byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; > > try ( > JarFile jar = new JarFile(jarFileName); > ) { > if (jar.getManifest().getAttributes(testClassFilename) == null) { > throw new AssertionError(testClassFilename + " not found in manifest"); > } > > JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); > try ( > InputStream manifestIs = jar.getInputStream(manifestEntry); > ) { > int bytesMatched = 0; > for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { > if ((byte) b == eAcuteBroken[bytesMatched]) { > bytesMatched++; > if (bytesMatched == eAcuteBroken.length) { > break; > } > } else { > bytesMatched = 0; > } > } > if (bytesMatched < eAcuteBroken.length) { > throw new AssertionError("self-test failed: " > + "multi-byte utf-8 character not broken across lines"); > } > } > } > } > > static OutputAnalyzer tool(String tool, String... args) > throws Throwable { > JDKToolLauncher l = JDKToolLauncher.createUsingTestJDK(tool); > for (String arg : args) { > if (arg .startsWith("-J")) { > l.addVMArg(arg.substring(2)); > } else { > l.addToolArg(arg); > } > } > return ProcessTools.executeCommand(l.getCommand()); > } > > } > END /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java > > > There are three e with acutes, on lines 84, 89, and 227, just in case the mail suffers bad encoding which might not be absolutely unconceivable regarding the bug's subject. > > In contrast to the bug description it uses a nested class with a two-byte UTF-8 character rather than one in its own file. I chose to do it that way because then the complete test goes into one file and therefore I assume the overview is kept easier. The test still demonstrates exactly JDK-6695402's problem. > > It may not have been necessary to also test jar files with indexes but i consider it necessary to have signing unsigned as well as partially signed jars tested, so why not have one more case tested, too? > > There might also be a more elegant approach to get class files into a jar file as to get their contents through the class loader. I chose to use real class files rather than dummy contents in files named *.class or for instance plain text files in the jar the signing of which to be tested in order to stay as close to the original bug problem as possible even though I don't have any notion that it would make a difference. The amount of code used to copy the class file contents around is comparatively small with respect to the whole test case amount of code. > > For the demonstration that the multi-byte character actually makes the alleged difference, I concluded that it was necessary to have another class name not previously affected by the bug in order to compare the effects of just different names. Otherwise the signing could fail to any other reason undetectably. > > In order to express a condition to tell successful from failed test runs the best approach I found was to analyze the jarsigner tool's output which is in my opinion not the most desirable option. I'd have preferred output from a clearer structured api but then I guess the output format will not change too often in the foreseeable future. For instance the check for the s, m, and k flags is more pragmatical approach than obviously perfectly failsafe when operating with regular expressions to match it with the output. Other parts such as 'X.509' and 'CN=' are not even jarsigner tool resources. I put at least a reference to sun.security.tools.jarsigner.Resources. > > Finally, I afforded kind of a self-test that protects the test from undetected failing because renaming the main class which would move the two-byte unicode character to a position not suitable for the test. It may not be absolutely necessary. > > > > BEGIN PATH > diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java > --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 > +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Wed Sep 20 00:56:15 2017 +0200 > @@ -28,6 +28,7 @@ > import java.security.*; > import java.util.HashMap; > import java.io.ByteArrayOutputStream; > +import static java.nio.charset.StandardCharsets.UTF_8; > > /** > * This class is used to compute digests on sections of the Manifest. > @@ -112,7 +113,7 @@ > rawBytes = bytes; > entries = new HashMap<>(); > > - ByteArrayOutputStream baos = new ByteArrayOutputStream(); > + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); > > Position pos = new Position(); > > @@ -131,50 +132,40 @@ > > if (len > 6) { > if (isNameAttr(bytes, start)) { > - StringBuilder nameBuf = new StringBuilder(sectionLen); > + nameBuf.reset(); > + nameBuf.write(bytes, start+6, len-6); > > - try { > - nameBuf.append( > - new String(bytes, start+6, len-6, "UTF8")); > + int i = start + len; > + if ((i-start) < sectionLen) { > + if (bytes[i] == '\r') { > + i += 2; > + } else { > + i += 1; > + } > + } > > - int i = start + len; > - if ((i-start) < sectionLen) { > - if (bytes[i] == '\r') { > - i += 2; > - } else { > - i += 1; > - } > + while ((i-start) < sectionLen) { > + if (bytes[i++] == ' ') { > + // name is wrapped > + int wrapStart = i; > + while (((i-start) < sectionLen) > + && (bytes[i++] != '\n')); > + if (bytes[i-1] != '\n') > + return; // XXX: exception? > + int wrapLen; > + if (bytes[i-2] == '\r') > + wrapLen = i-wrapStart-2; > + else > + wrapLen = i-wrapStart-1; > + > + nameBuf.write(bytes, wrapStart, wrapLen); > + } else { > + break; > } > + } > > - while ((i-start) < sectionLen) { > - if (bytes[i++] == ' ') { > - // name is wrapped > - int wrapStart = i; > - while (((i-start) < sectionLen) > - && (bytes[i++] != '\n')); > - if (bytes[i-1] != '\n') > - return; // XXX: exception? > - int wrapLen; > - if (bytes[i-2] == '\r') > - wrapLen = i-wrapStart-2; > - else > - wrapLen = i-wrapStart-1; > - > - nameBuf.append(new String(bytes, wrapStart, > - wrapLen, "UTF8")); > - } else { > - break; > - } > - } > - > - entries.put(nameBuf.toString(), > - new Entry(start, sectionLen, sectionLenWithBlank, > - rawBytes)); > - > - } catch (java.io.UnsupportedEncodingException uee) { > - throw new IllegalStateException( > - "UTF8 not available on platform"); > - } > + entries.put(new String(nameBuf.toByteArray(), UTF_8), > + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); > } > } > start = pos.startOfNext; > END PATCH > > > The patch is mostly so big because indentation has changed in many lines. > > There was baos there before without apparent use. The patch renames it and puts it into use. It came in very handy because if it would not have been there already, I would have had to defined one of my own. > > The main point of the patch is that manifest section names that are broken across lines at possibly arbitrary bytes are no longer converted into strings for each manifest line and then joined. Parts of names broken across lines are now joined first when they are still byte sequences and only when the complete byte sequence is available after processing all manifest lines that contain the same manifest section name decoded into a unicode string. > > For decoding the bytes into a string I chose a different string constructor than was used before that does not any longer declare UnsupportedEncodingException rendering the try/catch redundant. It couldn't have ever occurred anyway taking into consideration that UTF-8 is mandatory for every Java platform, StandardCharsets says. The difference according to the documentation is that the previously used String constructor returned undefined strings for invalid byte sequences whereas the one used in the patch will replace unparseable portions with valid 'unknown' characters. > > One question I cannot still answer yet is how the ManifestDigester can be changed at all without complete test coverage or risking to break existing signatures. I already started on that but it's quite a piece of work to almost formally prove that all manifests will continue to produce identical digests, except of course for the ones now fixed. > > Regards, > Philipp > > > On 17.09.2017 21:25, Philipp Kunz wrote: >> Hello Vincent >> >> I narrowed the error down so far and suspect now that it is an effect of sun.security.util.ManifestDigester's constructor, lines 134 and 163-164, in combination with java.util.jar.Manifest.make72Safe. Manifest breaks multi-byte characters in manifest section names across lines and ManifestDigester fails to restore them correctly, which both sounds wrong but ManifestDigester sure is. >> >> Just fixing it would be too tempting but then I would not know (I mean not know for sure with evidence from tests) if any existing jar-signature would still be valid and I don't want to break existing signatures. When I had a look at the tests specific for Manifest and ManifestDigester I found very little. There are many more different situations and corner cases and combinations thereof to cover with tests than it looks like at first glance so that writing tests that cover all relevant cases looks to me like quite an effort. I have the impression that test coverage was not a priority when the subjected code was developed or at least the tests might not have been contributed to the open JDK project. Hence, while I'm continuing to complete these tests I miss, if someone has an idea how to simplify that so that I can still convince at least myself that no existing signature will break or where possibly more testcases are that I haven't discovered so far, please let me know. >> >> I'm also wondering how much performance should be taken into consideration. There are a few hints such as reusing byte arrays that suggest that it is an issue. Will a patch be rejected if it slows down some tests beyond a certain limit for so little added value or what might be the acceptance criteria here? I don't expect insuperable difficulties with performance but would still appreciate some general idea. >> >> My desired or currently preferred approach to fix JDK-6695402 would be to let ManifestDigester any kind of use or extend Manifest in order to let Manifest identify the manifest sections thereby preventing the parsing duplicated that identifies the manifest sections. >> >> As a new contributor I could use and would appreciate some guidance particularly about what kind and completeness of test coverage and performance tuning applies or is suggested in the context of the given bug. Otherwise I fear I might contribute too many tests which would have to be reviewed and maintained or quite some work would be for nothing if a patch would not satisfy performance requirements. >> >> Regards, >> Philipp >> >> >> >> On 01.09.2017 15:20, Vincent Ryan wrote: >>> That all sounds fine. Let me know when your patch is ready to submit. >>> Thanks. >>> >>> >>>> On 1 Sep 2017, at 13:15, Philipp Kunz wrote: >>>> >>>> Hello Vincent >>>> >>>> Thank you for sponsoring! >>>> So far, I have become a contributor by signing the OCA which has been accepted. Dalibor Topic wrote that he can confirm and it's also here: http://www.oracle.com/technetwork/community/oca-486395.html#p -> Paratix GmbH >>>> Therefore I think I have followed the steps in http://openjdk.java.net/contribute/ at least the ones before actual patch submission. >>>> >>>> Currently, I'm working on getting the existing test cases running locally. Unfortunately, I started with jdk 9 and switched to 10 now. I figured these commands run at least the relevant tests with 9 and hope this also applies to 10: >>>> make run-test-tier1 >>>> make run-test TEST="jdk/test" >>>> they report errors and failures but it hasn't been completed and released which might explain it. If I run the tests I assume relevant for my patch >>>> make run-test TEST="jtreg:jdk/test/sun/security/tools/jarsigner jtreg:jdk/test/java/util/jar" >>>> then it reports zero errors and failures which may be a good starting point. >>>> Do you think that sounds reasonable or do you have another suggestion how to run the tests? >>>> >>>> Next, I will add a test for JDK-6695402 before actually fixing it. As an example, I'll try something in the style of http://hg.openjdk.java.net/jdk9/dev/jdk/file/65464a307408/test/java/util/jar/Manifest/CreateManifest.java. This way, I try to demonstrate the improvement. >>>> >>>> I guess I have identified the following line as the cause: value = new String(vb, 0, 0, vb.length); >>>> http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/51f5d60713b5/src/java.base/share/classes/java/util/jar/Manifest.java#l157 >>>> So I'll try to remove it first including the whole four line if block. >>>> >>>> Philipp >>>> >>>> >>>> On 01.09.2017 10:00, Vincent Ryan wrote: >>>>> Hello Philipp, >>>>> >>>>> I?m happy to sponsor your fix for JDK 10. Have you followed these steps: http://openjdk.java.net/contribute/ ? >>>>> >>>>> Thanks. >>>>> >>>>> >>>>>> On 1 Sep 2017, at 08:58, Vincent Ryan wrote: >>>>>> >>>>>> Moved to security-dev >>>>>> >>>>>> >>>>>>> On 1 Sep 2017, at 08:28, Philipp Kunz wrote: >>>>>>> >>>>>>> Hello everyone >>>>>>> >>>>>>> I have been developing with Java for around 17 years now and when I encountered some bug I decided to attempt to fix it: https://bugs.openjdk.java.net/browse/JDK-6695402. This also looks like it may not be too big a piece for a first contribution. >>>>>>> >>>>>>> I read through quite some guides and all kinds of documents but could not yet help myself with the following questions: >>>>>>> >>>>>>> May I login to jira to add comments to bugs? If so, how would I request or receive credentials? Or are mailing lists preferred? >>>>>>> >>>>>>> Another question is whether I should apply it to jdk9, but it may be too late now, or to jdk10, and backporting can be considered later. Probably it wouldn't even make much a difference for the patch itself. >>>>>>> >>>>>>> One more question I have is how or where to find the sources from before migration to mercurial. Because some lines of code I intend to change go back farther and in the history I find only 'initial commit'. With such a history I might be able better to understand why it's there and prevent to make the same mistake again. >>>>>>> >>>>>>> I guess the appropriate mailing list for above mentioned bug is security-dev. Is it correct that I can send a patch there and just hope for some sponsor to pick it up? Of course I'd be glad if some sponsor would contact me and maybe provide some assistance or if someone would confirm that sending a patch to the mailing list is the right way to find a sponsor. >>>>>>> >>>>>>> Philipp Kunz >>>>>> >>>>> >>>> >>> >> >> >> >> >> >> >> >> Paratix GmbH >> St Peterhofstatt 11 >> 8001 Z?rich >> >> +41 (0)76 397 79 35 >> philipp.kunz at paratix.ch > From sean.mullan at oracle.com Wed Sep 20 19:45:05 2017 From: sean.mullan at oracle.com (Sean Mullan) Date: Wed, 20 Sep 2017 15:45:05 -0400 Subject: StackOverflowError - Java 9 Build 181 In-Reply-To: References: <00a010a2-1a93-4f33-9eb7-1c905ca1270e@oracle.com> Message-ID: Tom, Try adding the following lines to the lib/security/default.policy file in your JDK installation: grant codeBase "jrt:/jdk.javaws" { permission java.security.AllPermission; }; I have a hunch that permissions are not being granted to the jdk.javaws module before it needs them. If that fixes the issue (or you don't see it for a few days), I'll followup and file a bug. Thanks, Sean On 9/19/17 5:55 PM, Tom Hood wrote: > No luck so far reproducing this problem.? The two times it happened to > me yesterday have both been with Java 9 build 181 and the application > has been idle for awhile. I login to our application, execute various > features of the application, go to a meeting, return, and then see the > java console repeatedly displaying the stack overflow exception.? Maybe > meetings are bad for Java 9? :-)? I think there are some background > threads in our application that are waking up periodically and doing > "stuff".? I don't know what that "stuff" is yet, but that would be my > guess at where I will find the code that triggered the overflow. > > Assuming I can get our application to the point where I can reproduce > the stack overflow, are there particular Java 9 builds that made > significant changes to security-relevant code that you'd like me to try? > > Keep in mind that our app runs on a network not connected to the > internet.? As it is, I manually typed in the stack trace, so if there's > a lot of output I'll have to print it and go through an approval process > to show it to you via a scanned pdf.? I will continue testing of our app > with the security debug turned on so that I'll have the output if it > happens again.? I also have the logging and tracing enabled in the java > control panel. > > -- Tom > > > On Tue, Sep 19, 2017 at 12:13 PM, Sean Mullan > wrote: > > Cross-posting to security-dev as this is more relevant to that list > and bcc-ing core-libs-dev. > > I think this might be an issue with the JavaWebStart SecurityManager > not being granted the proper permissions. It is possible that the > deployment policy files are not being loaded or there is some other > subtle bootstrapping issue. It should not result in a recursive loop > of course, but there may be a workaround. > > In the meantime, can you send me more information, preferably a test > case and a log file with -Djava.security.debug=all enabled? (The > latter will help analyze the recursion and see what security checks > are failing and for which ProtectionDomains). Also, have you tested > this on builds earlier than b181? > > Thanks, > Sean > > On 9/19/17 2:53 PM, Tom Hood wrote: > > I should add that we have not modified or overridden any policy > files. > Also, we are not using a custom security manager. > > On Tue, Sep 19, 2017 at 11:52 AM, Tom Hood > wrote: > > Hi, > > I hit an infinite recursion loop probably related to > PolicyFile that > exists in Java 9 build 181 for windows 64-bit.? It might be > related to > JDK-8077418 > > > > > I haven't tracked down what is causing our webstart app to > hit this > problem yet, but I thought I would let you know sooner than > later.? Also, > it probably is not a problem for our particular application > as I should be > able to set the security manager to null which I think/hope > will bypass > this issue.? I will try today to reproduce it in our app so > I can confirm > if setting security manager to null will work for us. > > The stack looks like the following: (with many repeat stacks > omitted) > > Exception in thread "AWT-EventQueue-2" > java.lang.StackOverflowError > at > java.base/java.security.AccessController.doPrivileged(Native > Method) > at java.base/sun.security.provider.PolicyFile.getPermissions(Po > licyFile.java:1135) > at java.base/sun.security.provider.PolicyFile.getPermissions(Po > licyFile.java:1082) > at java.base/sun.security.provider.PolicyFile.implies(PolicyFil > e.java:1038) > at java.base/java.security.provider.ProtectionDomain.implies(Pr > otectionDomain.java:323) > at java.base/java.security.provider.ProtectionDomain.impliesWit > hAltFilePerm(ProtectionDomain.java:355) > at java.base/java.security.provider.AccessControlContext.checkP > ermission(AccessControlContext.java:450) > at java.base/java.security.provider.AccessController.checkPermi > ssion(AccessController.java:895) > at java.base/java.lang.SecurityManager.checkPermission(Security > Manager.java:558) > at jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec > kPermission(JavaWebStartSecurity.java:237) > at > java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) > at java.base/java.io.File.isDirectory(File.java:845) > at > java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) > at java.base/sun.security.provider.PolicyFile.canonicalizeCodeb > ase(PolicyFile.java:1665) > at java.base/sun.security.provider.PolicyFile.access$700(Policy > File.java:263) > at > java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) > at > java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) > **** and again **** > at > java.base/java.security.AccessController.doPrivileged(Native > Method) > at java.base/sun.security.provider.PolicyFile.getPermissions(Po > licyFile.java:1135) > at java.base/sun.security.provider.PolicyFile.getPermissions(Po > licyFile.java:1082) > at java.base/sun.security.provider.PolicyFile.implies(PolicyFil > e.java:1038) > at java.base/java.security.provider.ProtectionDomain.implies(Pr > otectionDomain.java:323) > at java.base/java.security.provider.ProtectionDomain.impliesWit > hAltFilePerm(ProtectionDomain.java:355) > at java.base/java.security.provider.AccessControlContext.checkP > ermission(AccessControlContext.java:450) > at java.base/java.security.provider.AccessController.checkPermi > ssion(AccessController.java:895) > at java.base/java.lang.SecurityManager.checkPermission(Security > Manager.java:558) > at jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec > kPermission(JavaWebStartSecurity.java:237) > at > java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) > at java.base/java.io.File.isDirectory(File.java:845) > at > java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) > at java.base/sun.security.provider.PolicyFile.canonicalizeCodeb > ase(PolicyFile.java:1665) > at java.base/sun.security.provider.PolicyFile.access$700(Policy > File.java:263) > at > java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) > at > java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) > **** above lines start the stack that repeats until overflow > **** > at > java.base/java.security.AccessController.doPrivileged(Native > Method) > at java.base/sun.security.provider.PolicyFile.getPermissions(Po > licyFile.java:1135) > at java.base/sun.security.provider.PolicyFile.getPermissions(Po > licyFile.java:1082) > at java.base/sun.security.provider.PolicyFile.implies(PolicyFil > e.java:1038) > > -- Tom > > > From mandy.chung at oracle.com Wed Sep 20 19:56:50 2017 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 20 Sep 2017 12:56:50 -0700 Subject: StackOverflowError - Java 9 Build 181 In-Reply-To: References: <00a010a2-1a93-4f33-9eb7-1c905ca1270e@oracle.com> Message-ID: <01e4dca0-a289-fb02-3287-8c1beb14b018@oracle.com> FYI.? jdk.javaws is granted with AllPermissions in conf/security/javaws.policy.?? Maybe javaws.policy is not augmented to the security policy at runtime? Mandy On 9/20/17 12:45 PM, Sean Mullan wrote: > Tom, > > Try adding the following lines to the lib/security/default.policy file > in your JDK installation: > > grant codeBase "jrt:/jdk.javaws" { > ??? permission java.security.AllPermission; > }; > > I have a hunch that permissions are not being granted to the > jdk.javaws module before it needs them. If that fixes the issue (or > you don't see it for a few days), I'll followup and file a bug. > > Thanks, > Sean > > On 9/19/17 5:55 PM, Tom Hood wrote: >> No luck so far reproducing this problem. The two times it happened to >> me yesterday have both been with Java 9 build 181 and the application >> has been idle for awhile. I login to our application, execute various >> features of the application, go to a meeting, return, and then see >> the java console repeatedly displaying the stack overflow exception. >> Maybe meetings are bad for Java 9? :-)? I think there are some >> background threads in our application that are waking up periodically >> and doing "stuff".? I don't know what that "stuff" is yet, but that >> would be my guess at where I will find the code that triggered the >> overflow. >> >> Assuming I can get our application to the point where I can reproduce >> the stack overflow, are there particular Java 9 builds that made >> significant changes to security-relevant code that you'd like me to try? >> >> Keep in mind that our app runs on a network not connected to the >> internet.? As it is, I manually typed in the stack trace, so if >> there's a lot of output I'll have to print it and go through an >> approval process to show it to you via a scanned pdf.? I will >> continue testing of our app with the security debug turned on so that >> I'll have the output if it happens again.? I also have the logging >> and tracing enabled in the java control panel. >> >> -- Tom >> >> >> On Tue, Sep 19, 2017 at 12:13 PM, Sean Mullan > > wrote: >> >> ??? Cross-posting to security-dev as this is more relevant to that list >> ??? and bcc-ing core-libs-dev. >> >> ??? I think this might be an issue with the JavaWebStart SecurityManager >> ??? not being granted the proper permissions. It is possible that the >> ??? deployment policy files are not being loaded or there is some other >> ??? subtle bootstrapping issue. It should not result in a recursive loop >> ??? of course, but there may be a workaround. >> >> ??? In the meantime, can you send me more information, preferably a test >> ??? case and a log file with -Djava.security.debug=all enabled? (The >> ??? latter will help analyze the recursion and see what security checks >> ??? are failing and for which ProtectionDomains). Also, have you tested >> ??? this on builds earlier than b181? >> >> ??? Thanks, >> ??? Sean >> >> ??? On 9/19/17 2:53 PM, Tom Hood wrote: >> >> ??????? I should add that we have not modified or overridden any policy >> ??????? files. >> ??????? Also, we are not using a custom security manager. >> >> ??????? On Tue, Sep 19, 2017 at 11:52 AM, Tom Hood > ??????? > wrote: >> >> ??????????? Hi, >> >> ??????????? I hit an infinite recursion loop probably related to >> ??????????? PolicyFile that >> ??????????? exists in Java 9 build 181 for windows 64-bit.? It might be >> ??????????? related to >> ??????????? JDK-8077418 >> ??????????? > > >> >> >> ??????????? I haven't tracked down what is causing our webstart app to >> ??????????? hit this >> ??????????? problem yet, but I thought I would let you know sooner than >> ??????????? later.? Also, >> ??????????? it probably is not a problem for our particular application >> ??????????? as I should be >> ??????????? able to set the security manager to null which I think/hope >> ??????????? will bypass >> ??????????? this issue.? I will try today to reproduce it in our app so >> ??????????? I can confirm >> ??????????? if setting security manager to null will work for us. >> >> ??????????? The stack looks like the following: (with many repeat stacks >> ??????????? omitted) >> >> ??????????? Exception in thread "AWT-EventQueue-2" >> ??????????? java.lang.StackOverflowError >> ??????????? at >> java.base/java.security.AccessController.doPrivileged(Native >> ??????????? Method) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.getPermissions(Po >> ??????????? licyFile.java:1135) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.getPermissions(Po >> ??????????? licyFile.java:1082) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.implies(PolicyFil >> ??????????? e.java:1038) >> ??????????? at >> java.base/java.security.provider.ProtectionDomain.implies(Pr >> ??????????? otectionDomain.java:323) >> ??????????? at >> java.base/java.security.provider.ProtectionDomain.impliesWit >> ??????????? hAltFilePerm(ProtectionDomain.java:355) >> ??????????? at >> java.base/java.security.provider.AccessControlContext.checkP >> ??????????? ermission(AccessControlContext.java:450) >> ??????????? at >> java.base/java.security.provider.AccessController.checkPermi >> ??????????? ssion(AccessController.java:895) >> ??????????? at >> java.base/java.lang.SecurityManager.checkPermission(Security >> ??????????? Manager.java:558) >> ??????????? at >> jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec >> ??????????? kPermission(JavaWebStartSecurity.java:237) >> ??????????? at >> java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) >> ??????????? at java.base/java.io.File.isDirectory(File.java:845) >> ??????????? at >> java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.canonicalizeCodeb >> ??????????? ase(PolicyFile.java:1665) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.access$700(Policy >> ??????????? File.java:263) >> ??????????? at >> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >> ??????????? at >> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >> ??????????? **** and again **** >> ??????????? at >> java.base/java.security.AccessController.doPrivileged(Native >> ??????????? Method) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.getPermissions(Po >> ??????????? licyFile.java:1135) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.getPermissions(Po >> ??????????? licyFile.java:1082) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.implies(PolicyFil >> ??????????? e.java:1038) >> ??????????? at >> java.base/java.security.provider.ProtectionDomain.implies(Pr >> ??????????? otectionDomain.java:323) >> ??????????? at >> java.base/java.security.provider.ProtectionDomain.impliesWit >> ??????????? hAltFilePerm(ProtectionDomain.java:355) >> ??????????? at >> java.base/java.security.provider.AccessControlContext.checkP >> ??????????? ermission(AccessControlContext.java:450) >> ??????????? at >> java.base/java.security.provider.AccessController.checkPermi >> ??????????? ssion(AccessController.java:895) >> ??????????? at >> java.base/java.lang.SecurityManager.checkPermission(Security >> ??????????? Manager.java:558) >> ??????????? at >> jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec >> ??????????? kPermission(JavaWebStartSecurity.java:237) >> ??????????? at >> java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) >> ??????????? at java.base/java.io.File.isDirectory(File.java:845) >> ??????????? at >> java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.canonicalizeCodeb >> ??????????? ase(PolicyFile.java:1665) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.access$700(Policy >> ??????????? File.java:263) >> ??????????? at >> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >> ??????????? at >> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >> ??????????? **** above lines start the stack that repeats until overflow >> ??????????? **** >> ??????????? at >> java.base/java.security.AccessController.doPrivileged(Native >> ??????????? Method) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.getPermissions(Po >> ??????????? licyFile.java:1135) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.getPermissions(Po >> ??????????? licyFile.java:1082) >> ??????????? at >> java.base/sun.security.provider.PolicyFile.implies(PolicyFil >> ??????????? e.java:1038) >> >> ??????????? -- Tom >> >> >> From tom.w.hood at gmail.com Wed Sep 20 21:14:33 2017 From: tom.w.hood at gmail.com (Tom Hood) Date: Wed, 20 Sep 2017 14:14:33 -0700 Subject: StackOverflowError - Java 9 Build 181 In-Reply-To: <01e4dca0-a289-fb02-3287-8c1beb14b018@oracle.com> References: <00a010a2-1a93-4f33-9eb7-1c905ca1270e@oracle.com> <01e4dca0-a289-fb02-3287-8c1beb14b018@oracle.com> Message-ID: Sean, I'll add those lines to the lib/security/default.policy file as you suggested. I left the app running overnight and came in this morning and it was stuck in the infinite recursion loop again. I'll leave it running tonight as well with the default.policy change. How do I set java.security.debug=all within the jnlp? I tried several things, but none worked. Then I tried setting it as a property in the jnlp: I also tried removing the jnlp prefix from the property name: Then I tried setting it at the start of main with Security.setProperty("java.security.debug", "all") as well as System.setProperty("java.security.debug", "all"). No luck. Then I tried wrapping javaws in a windows batch file with the following line and selecting the batch file when firefox asks for the program to open the jnlp with: C:\"Program Files"\Java\jre9b181\bin\javaws -J*-Djava.security.debug=all* %1 which also didn't work. -- Tom On Wed, Sep 20, 2017 at 12:56 PM, mandy chung wrote: > FYI. jdk.javaws is granted with AllPermissions in > conf/security/javaws.policy. Maybe javaws.policy is not augmented to the > security policy at runtime? > > Mandy > > > On 9/20/17 12:45 PM, Sean Mullan wrote: > >> Tom, >> >> Try adding the following lines to the lib/security/default.policy file in >> your JDK installation: >> >> grant codeBase "jrt:/jdk.javaws" { >> permission java.security.AllPermission; >> }; >> >> I have a hunch that permissions are not being granted to the jdk.javaws >> module before it needs them. If that fixes the issue (or you don't see it >> for a few days), I'll followup and file a bug. >> >> Thanks, >> Sean >> >> On 9/19/17 5:55 PM, Tom Hood wrote: >> >>> No luck so far reproducing this problem. The two times it happened to me >>> yesterday have both been with Java 9 build 181 and the application has been >>> idle for awhile. I login to our application, execute various features of >>> the application, go to a meeting, return, and then see the java console >>> repeatedly displaying the stack overflow exception. Maybe meetings are bad >>> for Java 9? :-) I think there are some background threads in our >>> application that are waking up periodically and doing "stuff". I don't >>> know what that "stuff" is yet, but that would be my guess at where I will >>> find the code that triggered the overflow. >>> >>> Assuming I can get our application to the point where I can reproduce >>> the stack overflow, are there particular Java 9 builds that made >>> significant changes to security-relevant code that you'd like me to try? >>> >>> Keep in mind that our app runs on a network not connected to the >>> internet. As it is, I manually typed in the stack trace, so if there's a >>> lot of output I'll have to print it and go through an approval process to >>> show it to you via a scanned pdf. I will continue testing of our app with >>> the security debug turned on so that I'll have the output if it happens >>> again. I also have the logging and tracing enabled in the java control >>> panel. >>> >>> -- Tom >>> >>> >>> On Tue, Sep 19, 2017 at 12:13 PM, Sean Mullan >> > wrote: >>> >>> Cross-posting to security-dev as this is more relevant to that list >>> and bcc-ing core-libs-dev. >>> >>> I think this might be an issue with the JavaWebStart SecurityManager >>> not being granted the proper permissions. It is possible that the >>> deployment policy files are not being loaded or there is some other >>> subtle bootstrapping issue. It should not result in a recursive loop >>> of course, but there may be a workaround. >>> >>> In the meantime, can you send me more information, preferably a test >>> case and a log file with -Djava.security.debug=all enabled? (The >>> latter will help analyze the recursion and see what security checks >>> are failing and for which ProtectionDomains). Also, have you tested >>> this on builds earlier than b181? >>> >>> Thanks, >>> Sean >>> >>> On 9/19/17 2:53 PM, Tom Hood wrote: >>> >>> I should add that we have not modified or overridden any policy >>> files. >>> Also, we are not using a custom security manager. >>> >>> On Tue, Sep 19, 2017 at 11:52 AM, Tom Hood >> > wrote: >>> >>> Hi, >>> >>> I hit an infinite recursion loop probably related to >>> PolicyFile that >>> exists in Java 9 build 181 for windows 64-bit. It might be >>> related to >>> JDK-8077418 >>> >> > >>> >>> >>> I haven't tracked down what is causing our webstart app to >>> hit this >>> problem yet, but I thought I would let you know sooner than >>> later. Also, >>> it probably is not a problem for our particular application >>> as I should be >>> able to set the security manager to null which I think/hope >>> will bypass >>> this issue. I will try today to reproduce it in our app so >>> I can confirm >>> if setting security manager to null will work for us. >>> >>> The stack looks like the following: (with many repeat stacks >>> omitted) >>> >>> Exception in thread "AWT-EventQueue-2" >>> java.lang.StackOverflowError >>> at >>> java.base/java.security.AccessController.doPrivileged(Native >>> Method) >>> at java.base/sun.security.provide >>> r.PolicyFile.getPermissions(Po >>> licyFile.java:1135) >>> at java.base/sun.security.provide >>> r.PolicyFile.getPermissions(Po >>> licyFile.java:1082) >>> at java.base/sun.security.provide >>> r.PolicyFile.implies(PolicyFil >>> e.java:1038) >>> at java.base/java.security.provid >>> er.ProtectionDomain.implies(Pr >>> otectionDomain.java:323) >>> at java.base/java.security.provid >>> er.ProtectionDomain.impliesWit >>> hAltFilePerm(ProtectionDomain.java:355) >>> at java.base/java.security.provid >>> er.AccessControlContext.checkP >>> ermission(AccessControlContext.java:450) >>> at java.base/java.security.provid >>> er.AccessController.checkPermi >>> ssion(AccessController.java:895) >>> at java.base/java.lang.SecurityMa >>> nager.checkPermission(Security >>> Manager.java:558) >>> at jdk.javaws/com.sun.javaws.secu >>> rity.JavaWebStartSecurity.chec >>> kPermission(JavaWebStartSecurity.java:237) >>> at >>> java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) >>> at java.base/java.io.File.isDirectory(File.java:845) >>> at >>> java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >>> at java.base/sun.security.provide >>> r.PolicyFile.canonicalizeCodeb >>> ase(PolicyFile.java:1665) >>> at java.base/sun.security.provide >>> r.PolicyFile.access$700(Policy >>> File.java:263) >>> at >>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >>> at >>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >>> **** and again **** >>> at >>> java.base/java.security.AccessController.doPrivileged(Native >>> Method) >>> at java.base/sun.security.provide >>> r.PolicyFile.getPermissions(Po >>> licyFile.java:1135) >>> at java.base/sun.security.provide >>> r.PolicyFile.getPermissions(Po >>> licyFile.java:1082) >>> at java.base/sun.security.provide >>> r.PolicyFile.implies(PolicyFil >>> e.java:1038) >>> at java.base/java.security.provid >>> er.ProtectionDomain.implies(Pr >>> otectionDomain.java:323) >>> at java.base/java.security.provid >>> er.ProtectionDomain.impliesWit >>> hAltFilePerm(ProtectionDomain.java:355) >>> at java.base/java.security.provid >>> er.AccessControlContext.checkP >>> ermission(AccessControlContext.java:450) >>> at java.base/java.security.provid >>> er.AccessController.checkPermi >>> ssion(AccessController.java:895) >>> at java.base/java.lang.SecurityMa >>> nager.checkPermission(Security >>> Manager.java:558) >>> at jdk.javaws/com.sun.javaws.secu >>> rity.JavaWebStartSecurity.chec >>> kPermission(JavaWebStartSecurity.java:237) >>> at >>> java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) >>> at java.base/java.io.File.isDirectory(File.java:845) >>> at >>> java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >>> at java.base/sun.security.provide >>> r.PolicyFile.canonicalizeCodeb >>> ase(PolicyFile.java:1665) >>> at java.base/sun.security.provide >>> r.PolicyFile.access$700(Policy >>> File.java:263) >>> at >>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >>> at >>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >>> **** above lines start the stack that repeats until overflow >>> **** >>> at >>> java.base/java.security.AccessController.doPrivileged(Native >>> Method) >>> at java.base/sun.security.provide >>> r.PolicyFile.getPermissions(Po >>> licyFile.java:1135) >>> at java.base/sun.security.provide >>> r.PolicyFile.getPermissions(Po >>> licyFile.java:1082) >>> at java.base/sun.security.provide >>> r.PolicyFile.implies(PolicyFil >>> e.java:1038) >>> >>> -- Tom >>> >>> >>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom.w.hood at gmail.com Wed Sep 20 21:17:01 2017 From: tom.w.hood at gmail.com (Tom Hood) Date: Wed, 20 Sep 2017 14:17:01 -0700 Subject: StackOverflowError - Java 9 Build 181 In-Reply-To: References: <00a010a2-1a93-4f33-9eb7-1c905ca1270e@oracle.com> <01e4dca0-a289-fb02-3287-8c1beb14b018@oracle.com> Message-ID: In case useful, our jnlp file also contains this: On Wed, Sep 20, 2017 at 2:14 PM, Tom Hood wrote: > Sean, > > I'll add those lines to the lib/security/default.policy file as you > suggested. I left the app running overnight and came in this morning and > it was stuck in the infinite recursion loop again. I'll leave it running > tonight as well with the default.policy change. > > How do I set java.security.debug=all within the jnlp? I tried several > things, but none worked. > > java-vm-args="*-Djava.security.debug=all* --add-modules=java.corba > --add-exports=java.desktop/com.sun.java.swing.plaf.windows=ALL-UNNAMED > --add-exports=java.desktop/sun.swing=ALL-UNNAMED > --add-exports=java.desktop/sun.awt.shell=ALL-UNNAMED > --add-opens=java.base/java.lang=ALL-UNNAMED --add-exports=java.desktop/sun > .awt.image=ALL-UNNAMED"/> > > Then I tried setting it as a property in the jnlp: > > > > I also tried removing the jnlp prefix from the property name: > > > > Then I tried setting it at the start of main with > Security.setProperty("java.security.debug", "all") as well as > System.setProperty("java.security.debug", "all"). No luck. > > Then I tried wrapping javaws in a windows batch file with the following > line and selecting the batch file when firefox asks for the program to open > the jnlp with: > > C:\"Program Files"\Java\jre9b181\bin\javaws -J*-Djava.security.debug=all* > %1 > > which also didn't work. > > -- Tom > > > On Wed, Sep 20, 2017 at 12:56 PM, mandy chung > wrote: > >> FYI. jdk.javaws is granted with AllPermissions in >> conf/security/javaws.policy. Maybe javaws.policy is not augmented to the >> security policy at runtime? >> >> Mandy >> >> >> On 9/20/17 12:45 PM, Sean Mullan wrote: >> >>> Tom, >>> >>> Try adding the following lines to the lib/security/default.policy file >>> in your JDK installation: >>> >>> grant codeBase "jrt:/jdk.javaws" { >>> permission java.security.AllPermission; >>> }; >>> >>> I have a hunch that permissions are not being granted to the jdk.javaws >>> module before it needs them. If that fixes the issue (or you don't see it >>> for a few days), I'll followup and file a bug. >>> >>> Thanks, >>> Sean >>> >>> On 9/19/17 5:55 PM, Tom Hood wrote: >>> >>>> No luck so far reproducing this problem. The two times it happened to >>>> me yesterday have both been with Java 9 build 181 and the application has >>>> been idle for awhile. I login to our application, execute various features >>>> of the application, go to a meeting, return, and then see the java console >>>> repeatedly displaying the stack overflow exception. Maybe meetings are bad >>>> for Java 9? :-) I think there are some background threads in our >>>> application that are waking up periodically and doing "stuff". I don't >>>> know what that "stuff" is yet, but that would be my guess at where I will >>>> find the code that triggered the overflow. >>>> >>>> Assuming I can get our application to the point where I can reproduce >>>> the stack overflow, are there particular Java 9 builds that made >>>> significant changes to security-relevant code that you'd like me to try? >>>> >>>> Keep in mind that our app runs on a network not connected to the >>>> internet. As it is, I manually typed in the stack trace, so if there's a >>>> lot of output I'll have to print it and go through an approval process to >>>> show it to you via a scanned pdf. I will continue testing of our app with >>>> the security debug turned on so that I'll have the output if it happens >>>> again. I also have the logging and tracing enabled in the java control >>>> panel. >>>> >>>> -- Tom >>>> >>>> >>>> On Tue, Sep 19, 2017 at 12:13 PM, Sean Mullan >>> > wrote: >>>> >>>> Cross-posting to security-dev as this is more relevant to that list >>>> and bcc-ing core-libs-dev. >>>> >>>> I think this might be an issue with the JavaWebStart SecurityManager >>>> not being granted the proper permissions. It is possible that the >>>> deployment policy files are not being loaded or there is some other >>>> subtle bootstrapping issue. It should not result in a recursive loop >>>> of course, but there may be a workaround. >>>> >>>> In the meantime, can you send me more information, preferably a test >>>> case and a log file with -Djava.security.debug=all enabled? (The >>>> latter will help analyze the recursion and see what security checks >>>> are failing and for which ProtectionDomains). Also, have you tested >>>> this on builds earlier than b181? >>>> >>>> Thanks, >>>> Sean >>>> >>>> On 9/19/17 2:53 PM, Tom Hood wrote: >>>> >>>> I should add that we have not modified or overridden any policy >>>> files. >>>> Also, we are not using a custom security manager. >>>> >>>> On Tue, Sep 19, 2017 at 11:52 AM, Tom Hood < >>>> tom.w.hood at gmail.com >>>> > wrote: >>>> >>>> Hi, >>>> >>>> I hit an infinite recursion loop probably related to >>>> PolicyFile that >>>> exists in Java 9 build 181 for windows 64-bit. It might be >>>> related to >>>> JDK-8077418 >>>> >>> > >>>> >>>> >>>> I haven't tracked down what is causing our webstart app to >>>> hit this >>>> problem yet, but I thought I would let you know sooner than >>>> later. Also, >>>> it probably is not a problem for our particular application >>>> as I should be >>>> able to set the security manager to null which I think/hope >>>> will bypass >>>> this issue. I will try today to reproduce it in our app so >>>> I can confirm >>>> if setting security manager to null will work for us. >>>> >>>> The stack looks like the following: (with many repeat stacks >>>> omitted) >>>> >>>> Exception in thread "AWT-EventQueue-2" >>>> java.lang.StackOverflowError >>>> at >>>> java.base/java.security.AccessController.doPrivileged(Native >>>> Method) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.getPermissions(Po >>>> licyFile.java:1135) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.getPermissions(Po >>>> licyFile.java:1082) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.implies(PolicyFil >>>> e.java:1038) >>>> at java.base/java.security.provid >>>> er.ProtectionDomain.implies(Pr >>>> otectionDomain.java:323) >>>> at java.base/java.security.provid >>>> er.ProtectionDomain.impliesWit >>>> hAltFilePerm(ProtectionDomain.java:355) >>>> at java.base/java.security.provid >>>> er.AccessControlContext.checkP >>>> ermission(AccessControlContext.java:450) >>>> at java.base/java.security.provid >>>> er.AccessController.checkPermi >>>> ssion(AccessController.java:895) >>>> at java.base/java.lang.SecurityMa >>>> nager.checkPermission(Security >>>> Manager.java:558) >>>> at jdk.javaws/com.sun.javaws.secu >>>> rity.JavaWebStartSecurity.chec >>>> kPermission(JavaWebStartSecurity.java:237) >>>> at >>>> java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) >>>> at java.base/java.io.File.isDirectory(File.java:845) >>>> at >>>> java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.canonicalizeCodeb >>>> ase(PolicyFile.java:1665) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.access$700(Policy >>>> File.java:263) >>>> at >>>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >>>> at >>>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >>>> **** and again **** >>>> at >>>> java.base/java.security.AccessController.doPrivileged(Native >>>> Method) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.getPermissions(Po >>>> licyFile.java:1135) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.getPermissions(Po >>>> licyFile.java:1082) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.implies(PolicyFil >>>> e.java:1038) >>>> at java.base/java.security.provid >>>> er.ProtectionDomain.implies(Pr >>>> otectionDomain.java:323) >>>> at java.base/java.security.provid >>>> er.ProtectionDomain.impliesWit >>>> hAltFilePerm(ProtectionDomain.java:355) >>>> at java.base/java.security.provid >>>> er.AccessControlContext.checkP >>>> ermission(AccessControlContext.java:450) >>>> at java.base/java.security.provid >>>> er.AccessController.checkPermi >>>> ssion(AccessController.java:895) >>>> at java.base/java.lang.SecurityMa >>>> nager.checkPermission(Security >>>> Manager.java:558) >>>> at jdk.javaws/com.sun.javaws.secu >>>> rity.JavaWebStartSecurity.chec >>>> kPermission(JavaWebStartSecurity.java:237) >>>> at >>>> java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897) >>>> at java.base/java.io.File.isDirectory(File.java:845) >>>> at >>>> java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.canonicalizeCodeb >>>> ase(PolicyFile.java:1665) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.access$700(Policy >>>> File.java:263) >>>> at >>>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >>>> at >>>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >>>> **** above lines start the stack that repeats until overflow >>>> **** >>>> at >>>> java.base/java.security.AccessController.doPrivileged(Native >>>> Method) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.getPermissions(Po >>>> licyFile.java:1135) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.getPermissions(Po >>>> licyFile.java:1082) >>>> at java.base/sun.security.provide >>>> r.PolicyFile.implies(PolicyFil >>>> e.java:1038) >>>> >>>> -- Tom >>>> >>>> >>>> >>>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom.w.hood at gmail.com Wed Sep 20 21:49:29 2017 From: tom.w.hood at gmail.com (Tom Hood) Date: Wed, 20 Sep 2017 14:49:29 -0700 Subject: StackOverflowError - Java 9 Build 181 In-Reply-To: References: <00a010a2-1a93-4f33-9eb7-1c905ca1270e@oracle.com> <01e4dca0-a289-fb02-3287-8c1beb14b018@oracle.com> Message-ID: Sean, Rather than add those lines in my JDK installation, I'm going to add them in my JRE installation, specifically: C:\Program Files\Java\jre9b181\lib\security\default.policy. If this is different than what you intended, please let me know. The infinite recursion is happening when running with the JRE. I haven't tried running with the JDK installation. -- Tom On Wed, Sep 20, 2017 at 2:17 PM, Tom Hood wrote: > In case useful, our jnlp file also contains this: > > > > > > On Wed, Sep 20, 2017 at 2:14 PM, Tom Hood wrote: > >> Sean, >> >> I'll add those lines to the lib/security/default.policy file as you >> suggested. I left the app running overnight and came in this morning and >> it was stuck in the infinite recursion loop again. I'll leave it running >> tonight as well with the default.policy change. >> >> How do I set java.security.debug=all within the jnlp? I tried several >> things, but none worked. >> >> > java-vm-args="*-Djava.security.debug=all* --add-modules=java.corba >> --add-exports=java.desktop/com.sun.java.swing.plaf.windows=ALL-UNNAMED >> --add-exports=java.desktop/sun.swing=ALL-UNNAMED >> --add-exports=java.desktop/sun.awt.shell=ALL-UNNAMED >> --add-opens=java.base/java.lang=ALL-UNNAMED >> --add-exports=java.desktop/sun.awt.image=ALL-UNNAMED"/> >> >> Then I tried setting it as a property in the jnlp: >> >> >> >> I also tried removing the jnlp prefix from the property name: >> >> >> >> Then I tried setting it at the start of main with >> Security.setProperty("java.security.debug", "all") as well as >> System.setProperty("java.security.debug", "all"). No luck. >> >> Then I tried wrapping javaws in a windows batch file with the following >> line and selecting the batch file when firefox asks for the program to open >> the jnlp with: >> >> C:\"Program Files"\Java\jre9b181\bin\javaws -J*-Djava.security.debug=all* >> %1 >> >> which also didn't work. >> >> -- Tom >> >> >> On Wed, Sep 20, 2017 at 12:56 PM, mandy chung >> wrote: >> >>> FYI. jdk.javaws is granted with AllPermissions in >>> conf/security/javaws.policy. Maybe javaws.policy is not augmented to the >>> security policy at runtime? >>> >>> Mandy >>> >>> >>> On 9/20/17 12:45 PM, Sean Mullan wrote: >>> >>>> Tom, >>>> >>>> Try adding the following lines to the lib/security/default.policy file >>>> in your JDK installation: >>>> >>>> grant codeBase "jrt:/jdk.javaws" { >>>> permission java.security.AllPermission; >>>> }; >>>> >>>> I have a hunch that permissions are not being granted to the jdk.javaws >>>> module before it needs them. If that fixes the issue (or you don't see it >>>> for a few days), I'll followup and file a bug. >>>> >>>> Thanks, >>>> Sean >>>> >>>> On 9/19/17 5:55 PM, Tom Hood wrote: >>>> >>>>> No luck so far reproducing this problem. The two times it happened to >>>>> me yesterday have both been with Java 9 build 181 and the application has >>>>> been idle for awhile. I login to our application, execute various features >>>>> of the application, go to a meeting, return, and then see the java console >>>>> repeatedly displaying the stack overflow exception. Maybe meetings are bad >>>>> for Java 9? :-) I think there are some background threads in our >>>>> application that are waking up periodically and doing "stuff". I don't >>>>> know what that "stuff" is yet, but that would be my guess at where I will >>>>> find the code that triggered the overflow. >>>>> >>>>> Assuming I can get our application to the point where I can reproduce >>>>> the stack overflow, are there particular Java 9 builds that made >>>>> significant changes to security-relevant code that you'd like me to try? >>>>> >>>>> Keep in mind that our app runs on a network not connected to the >>>>> internet. As it is, I manually typed in the stack trace, so if there's a >>>>> lot of output I'll have to print it and go through an approval process to >>>>> show it to you via a scanned pdf. I will continue testing of our app with >>>>> the security debug turned on so that I'll have the output if it happens >>>>> again. I also have the logging and tracing enabled in the java control >>>>> panel. >>>>> >>>>> -- Tom >>>>> >>>>> >>>>> On Tue, Sep 19, 2017 at 12:13 PM, Sean Mullan >>>> > wrote: >>>>> >>>>> Cross-posting to security-dev as this is more relevant to that list >>>>> and bcc-ing core-libs-dev. >>>>> >>>>> I think this might be an issue with the JavaWebStart >>>>> SecurityManager >>>>> not being granted the proper permissions. It is possible that the >>>>> deployment policy files are not being loaded or there is some other >>>>> subtle bootstrapping issue. It should not result in a recursive >>>>> loop >>>>> of course, but there may be a workaround. >>>>> >>>>> In the meantime, can you send me more information, preferably a >>>>> test >>>>> case and a log file with -Djava.security.debug=all enabled? (The >>>>> latter will help analyze the recursion and see what security checks >>>>> are failing and for which ProtectionDomains). Also, have you tested >>>>> this on builds earlier than b181? >>>>> >>>>> Thanks, >>>>> Sean >>>>> >>>>> On 9/19/17 2:53 PM, Tom Hood wrote: >>>>> >>>>> I should add that we have not modified or overridden any policy >>>>> files. >>>>> Also, we are not using a custom security manager. >>>>> >>>>> On Tue, Sep 19, 2017 at 11:52 AM, Tom Hood < >>>>> tom.w.hood at gmail.com >>>>> > wrote: >>>>> >>>>> Hi, >>>>> >>>>> I hit an infinite recursion loop probably related to >>>>> PolicyFile that >>>>> exists in Java 9 build 181 for windows 64-bit. It might be >>>>> related to >>>>> JDK-8077418 >>>>> >>>> > >>>>> >>>>> >>>>> I haven't tracked down what is causing our webstart app to >>>>> hit this >>>>> problem yet, but I thought I would let you know sooner than >>>>> later. Also, >>>>> it probably is not a problem for our particular application >>>>> as I should be >>>>> able to set the security manager to null which I think/hope >>>>> will bypass >>>>> this issue. I will try today to reproduce it in our app so >>>>> I can confirm >>>>> if setting security manager to null will work for us. >>>>> >>>>> The stack looks like the following: (with many repeat >>>>> stacks >>>>> omitted) >>>>> >>>>> Exception in thread "AWT-EventQueue-2" >>>>> java.lang.StackOverflowError >>>>> at >>>>> java.base/java.security.AccessController.doPrivileged(Native >>>>> Method) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.getPermissions(Po >>>>> licyFile.java:1135) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.getPermissions(Po >>>>> licyFile.java:1082) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.implies(PolicyFil >>>>> e.java:1038) >>>>> at java.base/java.security.provid >>>>> er.ProtectionDomain.implies(Pr >>>>> otectionDomain.java:323) >>>>> at java.base/java.security.provid >>>>> er.ProtectionDomain.impliesWit >>>>> hAltFilePerm(ProtectionDomain.java:355) >>>>> at java.base/java.security.provid >>>>> er.AccessControlContext.checkP >>>>> ermission(AccessControlContext.java:450) >>>>> at java.base/java.security.provid >>>>> er.AccessController.checkPermi >>>>> ssion(AccessController.java:895) >>>>> at java.base/java.lang.SecurityMa >>>>> nager.checkPermission(Security >>>>> Manager.java:558) >>>>> at jdk.javaws/com.sun.javaws.secu >>>>> rity.JavaWebStartSecurity.chec >>>>> kPermission(JavaWebStartSecurity.java:237) >>>>> at >>>>> java.base/java.lang.SecurityManager.checkRead(SecurityManage >>>>> r.java:897) >>>>> at java.base/java.io.File.isDirectory(File.java:845) >>>>> at >>>>> java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.canonicalizeCodeb >>>>> ase(PolicyFile.java:1665) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.access$700(Policy >>>>> File.java:263) >>>>> at >>>>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >>>>> at >>>>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >>>>> **** and again **** >>>>> at >>>>> java.base/java.security.AccessController.doPrivileged(Native >>>>> Method) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.getPermissions(Po >>>>> licyFile.java:1135) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.getPermissions(Po >>>>> licyFile.java:1082) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.implies(PolicyFil >>>>> e.java:1038) >>>>> at java.base/java.security.provid >>>>> er.ProtectionDomain.implies(Pr >>>>> otectionDomain.java:323) >>>>> at java.base/java.security.provid >>>>> er.ProtectionDomain.impliesWit >>>>> hAltFilePerm(ProtectionDomain.java:355) >>>>> at java.base/java.security.provid >>>>> er.AccessControlContext.checkP >>>>> ermission(AccessControlContext.java:450) >>>>> at java.base/java.security.provid >>>>> er.AccessController.checkPermi >>>>> ssion(AccessController.java:895) >>>>> at java.base/java.lang.SecurityMa >>>>> nager.checkPermission(Security >>>>> Manager.java:558) >>>>> at jdk.javaws/com.sun.javaws.secu >>>>> rity.JavaWebStartSecurity.chec >>>>> kPermission(JavaWebStartSecurity.java:237) >>>>> at >>>>> java.base/java.lang.SecurityManager.checkRead(SecurityManage >>>>> r.java:897) >>>>> at java.base/java.io.File.isDirectory(File.java:845) >>>>> at >>>>> java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.canonicalizeCodeb >>>>> ase(PolicyFile.java:1665) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.access$700(Policy >>>>> File.java:263) >>>>> at >>>>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139) >>>>> at >>>>> java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136) >>>>> **** above lines start the stack that repeats until >>>>> overflow >>>>> **** >>>>> at >>>>> java.base/java.security.AccessController.doPrivileged(Native >>>>> Method) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.getPermissions(Po >>>>> licyFile.java:1135) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.getPermissions(Po >>>>> licyFile.java:1082) >>>>> at java.base/sun.security.provide >>>>> r.PolicyFile.implies(PolicyFil >>>>> e.java:1038) >>>>> >>>>> -- Tom >>>>> >>>>> >>>>> >>>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amanda.jiang at oracle.com Wed Sep 20 23:52:44 2017 From: amanda.jiang at oracle.com (Amanda Jiang) Date: Wed, 20 Sep 2017 16:52:44 -0700 Subject: RFR 8176354: sun/security/ssl/X509KeyManager/PreferredKey.java failed with "Failed to get the preferable key aliases" error Message-ID: Hi All, Please help to review following webrev for fixing a test issue: Bug: https://bugs.openjdk.java.net/browse/JDK-8176354 webrev: http://cr.openjdk.java.net/~amjiang/8176354/webrev.00/ Thanks, Amanda From xuelei.fan at oracle.com Thu Sep 21 01:33:08 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Wed, 20 Sep 2017 18:33:08 -0700 Subject: RFR 8176354: sun/security/ssl/X509KeyManager/PreferredKey.java failed with "Failed to get the preferable key aliases" error In-Reply-To: References: Message-ID: Looks fine to me. Thanks, Xuelei On 9/20/2017 4:52 PM, Amanda Jiang wrote: > Hi All, > > Please help to review following webrev for fixing a test issue: > > Bug: https://bugs.openjdk.java.net/browse/JDK-8176354 > > webrev: http://cr.openjdk.java.net/~amjiang/8176354/webrev.00/ > > Thanks, > Amanda From adam.petcher at oracle.com Thu Sep 21 15:25:50 2017 From: adam.petcher at oracle.com (Adam Petcher) Date: Thu, 21 Sep 2017 11:25:50 -0400 Subject: JEP for X25519/X448 key agreement In-Reply-To: <7c9104dc-ac91-c966-6273-7387410e5a7a@oracle.com> References: <7c9104dc-ac91-c966-6273-7387410e5a7a@oracle.com> Message-ID: <0fbc6db2-e7dc-c72c-508f-37a521f8ca8a@oracle.com> I would like to leave this open for feedback for another week or so. Please reply with your comments by Saturday, September 30, end of day, anywhere on earth. After that time, I plan to move on to the next phase of the process (group lead and architect review prior to submission). On 9/14/2017 12:59 PM, Adam Petcher wrote: > The JEP for X25519/X448 key agreement[1] is now available and ready to > review. Please take a look and reply with any feedback you have. > > The JEP contains a description of the proposed JCA API. We have > discussed the API on this mailing list, and I have attempted to > incorporate all the feedback I have received. Here is a description of > the changes since the last discussion: > > 1) Multiple people requested more specific types for public/private > keys for this algorithm. The latest API design mirrors the "EC" > hierarchy and has both interfaces and spec classes for public and > private keys. I also added the interface "XDHKey", which serves the > same purpose as "ECKey". > 2) The representation of public keys was changed from byte[] to a > BigInteger which holds the u coordinate of the point. Private keys are > still represented using byte[] due to complications related to > pruning, and also because BigInteger doesn't provide a branch-free way > to get the key into another representation (which is necessary for > side-channel-resilient implementations). > > The proposed API still lacks a standard way to specify arbitrary > domain parameters, but I believe the API design could be extended to > support this feature. I would prefer to add this API as a separate > enhancement in the future, preferably in cooperation with someone who > is developing a provider that supports this feature. > > > [1] https://bugs.openjdk.java.net/browse/JDK-8181595 > From amanda.jiang at oracle.com Thu Sep 21 21:26:39 2017 From: amanda.jiang at oracle.com (Amanda Jiang) Date: Thu, 21 Sep 2017 14:26:39 -0700 Subject: RFR 8176354: sun/security/ssl/X509KeyManager/PreferredKey.java failed with "Failed to get the preferable key aliases" error In-Reply-To: References: Message-ID: <30428385-fdf8-d65d-0d77-2241913fdd0b@oracle.com> Thank you for reviewing this. - Amanda On 9/20/17 6:33 PM, Xuelei Fan wrote: > Looks fine to me. > > Thanks, > Xuelei > > On 9/20/2017 4:52 PM, Amanda Jiang wrote: >> Hi All, >> >> Please help to review following webrev for fixing a test issue: >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8176354 >> >> webrev: http://cr.openjdk.java.net/~amjiang/8176354/webrev.00/ >> >> Thanks, >> Amanda From rob.mckenna at oracle.com Thu Sep 21 22:06:21 2017 From: rob.mckenna at oracle.com (Rob McKenna) Date: Thu, 21 Sep 2017 23:06:21 +0100 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <3044d68a-403f-2af8-099b-184d70efe1ee@oracle.com> References: <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> <20170915140025.GA4404@vimes> <003a2ae4-349c-cc33-fea8-5629b225e86b@oracle.com> <20170915144123.GE4404@vimes> <20170915152230.GI4404@vimes> <9d9ab26c-add2-bf2f-ca6d-6592214e03bd@oracle.com> <3044d68a-403f-2af8-099b-184d70efe1ee@oracle.com> Message-ID: <20170921220621.GA2811@vimes> Thanks Xuelei, webrev below: http://cr.openjdk.java.net/~robm/8184328/webrev.02/ Presumably this won't require a CSR? -Rob On 15/09/17 04:38, Xuelei Fan wrote: > I still prefer to not-depends on socket receiving timeout. But I'm fine if > you want to move on with it. > > As we can close the super socket in the current implementation, it implies > that application can handle it already. So you may not need the system > property and 5 times retries. I think it's fine just call fatal() for the > first timeout. > > Xuelei > > On 9/15/2017 4:32 PM, Xuelei Fan wrote: > >On 9/15/2017 8:22 AM, Rob McKenna wrote: > >>This test calls close directly. (3rd last line in the stack) > >> > >>I believe this is the only possible stack (with the new parameter) once > >>autoclose is set to false. If autoclose is true we'd skip the call to > >>waitForClose and just go directly to Socket.close() unless I'm mistaken. > >> > >I did not find the call to fatal() in the current implementation.? I think > >you mean you added the call to fatal() in your update so that when > >timeout, a fatal() will always get called? > > > >Thinking about two things: > >1. application have to set receiving timeout in order to? get receiving > >timeout. > >I have a concern about it, as described in other comments. > > > >2. can we close the super socket? > >It is a surprise to me to close super socket even we don't allocate it. It > >does not feel right to me, but this is the current behavior.? All right, I > >get your point. > > > >Xuelei > > > >>???? -Rob > >> > >>On 15/09/17 07:55, Xuelei Fan wrote: > >>>On 9/15/2017 7:41 AM, Rob McKenna wrote: > >>>>On 15/09/17 07:07, Xuelei Fan wrote: > >>>>>On 9/15/2017 7:00 AM, Rob McKenna wrote: > >>>>>>When we call close() on the SSLSocket that calls close() on the > >>>>>>underlying java Socket which closes the native socket. > >>>>>> > >>>>>Sorry, I did not get the point.? Please see the close() > >>>>>implementation of > >>>>>SSLSocket (sun.security.ssl.SSLSocketImpl.close()) about the details. > >>>> > >>>>Running my original test against an instrumented 8u-dev produces the > >>>>following: > >>>> > >>>>java.lang.Exception: Stack trace > >>>>????at java.lang.Thread.dumpStack(Thread.java:1336) > >>>>????at java.net.Socket.close(Socket.java:1491) > >>>>????at > >>>>sun.security.ssl.BaseSSLSocketImpl.close(BaseSSLSocketImpl.java:624) > >>>>????at > >>>>sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1579) > >>>>????at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1980) > >>>>????at > >>>>sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1793) > >>>>????at > >>>>sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1592) > >>>>????at > >>>>sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1726) > >>>>????at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1615) > >>>>????at ssl.SSLClient.close(SSLClient.java:143) > >>>>????at > >>>>ssl.SocketTimeoutCloseHang.ReadHang.testSSLServer(ReadHang.java:77) > >>>> > >>>It is just one possible stacks of many.? There are cases where no > >>>fatal() > >>>get called.? For example, application call close() method directly. > >>> > >>>Xuelei From Xuelei.Fan at Oracle.Com Thu Sep 21 23:10:05 2017 From: Xuelei.Fan at Oracle.Com (Xuelei Fan) Date: Fri, 22 Sep 2017 07:10:05 +0800 Subject: [RFR] 8184328: JDK8u131 socketRead0 hang at SSL read In-Reply-To: <20170921220621.GA2811@vimes> References: <20170913155208.GD3896@vimes> <257B6642-4142-4080-B2DF-C2E5419E896E@oracle.com> <20170915140025.GA4404@vimes> <003a2ae4-349c-cc33-fea8-5629b225e86b@oracle.com> <20170915144123.GE4404@vimes> <20170915152230.GI4404@vimes> <9d9ab26c-add2-bf2f-ca6d-6592214e03bd@oracle.com> <3044d68a-403f-2af8-099b-184d70efe1ee@oracle.com> <20170921220621.GA2811@vimes> Message-ID: <3F546E1C-1286-4DC6-8F5F-87E7314D3C3F@Oracle.Com> > On Sep 22, 2017, at 6:06 AM, Rob McKenna wrote: > > Thanks Xuelei, webrev below: > > http://cr.openjdk.java.net/~robm/8184328/webrev.02/ > Looks fine to me. > Presumably this won't require a CSR? > Agreed. Xuelei > -Rob > >> On 15/09/17 04:38, Xuelei Fan wrote: >> I still prefer to not-depends on socket receiving timeout. But I'm fine if >> you want to move on with it. >> >> As we can close the super socket in the current implementation, it implies >> that application can handle it already. So you may not need the system >> property and 5 times retries. I think it's fine just call fatal() for the >> first timeout. >> >> Xuelei >> >>> On 9/15/2017 4:32 PM, Xuelei Fan wrote: >>>> On 9/15/2017 8:22 AM, Rob McKenna wrote: >>>> This test calls close directly. (3rd last line in the stack) >>>> >>>> I believe this is the only possible stack (with the new parameter) once >>>> autoclose is set to false. If autoclose is true we'd skip the call to >>>> waitForClose and just go directly to Socket.close() unless I'm mistaken. >>>> >>> I did not find the call to fatal() in the current implementation. I think >>> you mean you added the call to fatal() in your update so that when >>> timeout, a fatal() will always get called? >>> >>> Thinking about two things: >>> 1. application have to set receiving timeout in order to get receiving >>> timeout. >>> I have a concern about it, as described in other comments. >>> >>> 2. can we close the super socket? >>> It is a surprise to me to close super socket even we don't allocate it. It >>> does not feel right to me, but this is the current behavior. All right, I >>> get your point. >>> >>> Xuelei >>> >>>> -Rob >>>> >>>>> On 15/09/17 07:55, Xuelei Fan wrote: >>>>>> On 9/15/2017 7:41 AM, Rob McKenna wrote: >>>>>>> On 15/09/17 07:07, Xuelei Fan wrote: >>>>>>>> On 9/15/2017 7:00 AM, Rob McKenna wrote: >>>>>>>> When we call close() on the SSLSocket that calls close() on the >>>>>>>> underlying java Socket which closes the native socket. >>>>>>>> >>>>>>> Sorry, I did not get the point. Please see the close() >>>>>>> implementation of >>>>>>> SSLSocket (sun.security.ssl.SSLSocketImpl.close()) about the details. >>>>>> >>>>>> Running my original test against an instrumented 8u-dev produces the >>>>>> following: >>>>>> >>>>>> java.lang.Exception: Stack trace >>>>>> at java.lang.Thread.dumpStack(Thread.java:1336) >>>>>> at java.net.Socket.close(Socket.java:1491) >>>>>> at >>>>>> sun.security.ssl.BaseSSLSocketImpl.close(BaseSSLSocketImpl.java:624) >>>>>> at >>>>>> sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1579) >>>>>> at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1980) >>>>>> at >>>>>> sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1793) >>>>>> at >>>>>> sun.security.ssl.SSLSocketImpl.closeSocket(SSLSocketImpl.java:1592) >>>>>> at >>>>>> sun.security.ssl.SSLSocketImpl.closeInternal(SSLSocketImpl.java:1726) >>>>>> at sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:1615) >>>>>> at ssl.SSLClient.close(SSLClient.java:143) >>>>>> at >>>>>> ssl.SocketTimeoutCloseHang.ReadHang.testSSLServer(ReadHang.java:77) >>>>>> >>>>> It is just one possible stacks of many. There are cases where no >>>>> fatal() >>>>> get called. For example, application call close() method directly. >>>>> >>>>> Xuelei From weijun.wang at oracle.com Fri Sep 22 00:25:57 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 22 Sep 2017 08:25:57 +0800 Subject: RFR 8187786: Many javax/net/ssl/DTLS tests failing Message-ID: <6A7148E1-24C4-424E-90BB-3C61F395AC58@oracle.com> Hi All I'm very sorry that my earlier change for JDK-8186884 made a lot of DTLS test failing. A new @modules tag was added to JGSS/krb5 tests, but I didn't realize DTLS tests are also using it. Please take a review at http://cr.openjdk.java.net/~weijun/8187786/webrev.00/ It's quite long, but only contains 2 changes: 1. Add a "@modules java.security.jgss/sun.security.jgss.krb5" tag to every file that already has @modules tag for JGSS internal packages, both .java and TEST.properties. 2. Backout JDK-8187788, which disables SSL tests. I'm running tests for jdk_security3 now. Thanks Max From xuelei.fan at oracle.com Fri Sep 22 00:37:01 2017 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Thu, 21 Sep 2017 17:37:01 -0700 Subject: RFR 8187786: Many javax/net/ssl/DTLS tests failing In-Reply-To: <6A7148E1-24C4-424E-90BB-3C61F395AC58@oracle.com> References: <6A7148E1-24C4-424E-90BB-3C61F395AC58@oracle.com> Message-ID: <6cc454f2-767d-4dce-c211-3d9595bb572b@oracle.com> Looks good. There are some wired Krb5 modules dependency even Krb5 is not used at all in the testing. Anyway, we may want an improvement in the future. Thanks, Xuelei On 9/21/2017 5:25 PM, Weijun Wang wrote: > Hi All > > I'm very sorry that my earlier change for JDK-8186884 made a lot of DTLS test failing. A new @modules tag was added to JGSS/krb5 tests, but I didn't realize DTLS tests are also using it. > > Please take a review at > > http://cr.openjdk.java.net/~weijun/8187786/webrev.00/ > > It's quite long, but only contains 2 changes: > > 1. Add a "@modules java.security.jgss/sun.security.jgss.krb5" tag to every file that already has @modules tag for JGSS internal packages, both .java and TEST.properties. > > 2. Backout JDK-8187788, which disables SSL tests. > > I'm running tests for jdk_security3 now. > > Thanks > Max > From weijun.wang at oracle.com Fri Sep 22 00:41:55 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Fri, 22 Sep 2017 08:41:55 +0800 Subject: RFR 8187786: Many javax/net/ssl/DTLS tests failing In-Reply-To: <6cc454f2-767d-4dce-c211-3d9595bb572b@oracle.com> References: <6A7148E1-24C4-424E-90BB-3C61F395AC58@oracle.com> <6cc454f2-767d-4dce-c211-3d9595bb572b@oracle.com> Message-ID: <91C70E08-7C22-4C27-A55A-37D19D162C35@oracle.com> > On Sep 22, 2017, at 8:37 AM, Xuelei Fan wrote: > > Looks good. Thanks. I'll wait for the test passing and push the change. > > There are some wired Krb5 modules dependency even Krb5 is not used at all in the testing. Anyway, we may want an improvement in the future. My fix this time is a pure "perl -i -pe 's/one krb5/two/krb5/" run. I do see @modules in both TETS.properties and .java files everywhere, and think one would completely shadow the other. Some cleanup is needed. Thanks Max > > Thanks, > Xuelei > > On 9/21/2017 5:25 PM, Weijun Wang wrote: >> Hi All >> I'm very sorry that my earlier change for JDK-8186884 made a lot of DTLS test failing. A new @modules tag was added to JGSS/krb5 tests, but I didn't realize DTLS tests are also using it. >> Please take a review at >> http://cr.openjdk.java.net/~weijun/8187786/webrev.00/ >> It's quite long, but only contains 2 changes: >> 1. Add a "@modules java.security.jgss/sun.security.jgss.krb5" tag to every file that already has @modules tag for JGSS internal packages, both .java and TEST.properties. >> 2. Backout JDK-8187788, which disables SSL tests. >> I'm running tests for jdk_security3 now. >> Thanks >> Max From anders.rundgren.net at gmail.com Fri Sep 22 05:27:47 2017 From: anders.rundgren.net at gmail.com (Anders Rundgren) Date: Fri, 22 Sep 2017 07:27:47 +0200 Subject: JEP for X25519/X448 key agreement In-Reply-To: <0fbc6db2-e7dc-c72c-508f-37a521f8ca8a@oracle.com> References: <7c9104dc-ac91-c966-6273-7387410e5a7a@oracle.com> <0fbc6db2-e7dc-c72c-508f-37a521f8ca8a@oracle.com> Message-ID: <2d775c71-a6fd-8b89-6c75-b01775c16617@gmail.com> On 2017-09-21 17:25, Adam Petcher wrote: > I would like to leave this open for feedback for another week or so. > Please reply with your comments by Saturday, September 30, end of day, > anywhere on earth. After that time, I plan to move on to the next phase > of the process (group lead and architect review prior to submission). This proposal doesn't appear fundamentally different than the one I drafted on https://github.com/cyberphone/java-cfrg-spec sometimes ago. So I'm obviously OK with that but still have a question: How do you envision the corresponding signature algorithms will be supported? IMO, it would be useful knowing that before casting things in concrete. Just for my curiosity, if somebody wanted to implement other variants of XDH curves, would the proper way to do that be through using a new provider name (like "XDH-1") while still using the XDH classes and interfaces? Anders > > > On 9/14/2017 12:59 PM, Adam Petcher wrote: >> The JEP for X25519/X448 key agreement[1] is now available and ready to >> review. Please take a look and reply with any feedback you have. >> >> The JEP contains a description of the proposed JCA API. We have >> discussed the API on this mailing list, and I have attempted to >> incorporate all the feedback I have received. Here is a description of >> the changes since the last discussion: >> >> 1) Multiple people requested more specific types for public/private >> keys for this algorithm. The latest API design mirrors the "EC" >> hierarchy and has both interfaces and spec classes for public and >> private keys. I also added the interface "XDHKey", which serves the >> same purpose as "ECKey". >> 2) The representation of public keys was changed from byte[] to a >> BigInteger which holds the u coordinate of the point. Private keys are >> still represented using byte[] due to complications related to >> pruning, and also because BigInteger doesn't provide a branch-free way >> to get the key into another representation (which is necessary for >> side-channel-resilient implementations). >> >> The proposed API still lacks a standard way to specify arbitrary >> domain parameters, but I believe the API design could be extended to >> support this feature. I would prefer to add this API as a separate >> enhancement in the future, preferably in cooperation with someone who >> is developing a provider that supports this feature. >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8181595 >> > From adam.petcher at oracle.com Fri Sep 22 15:11:57 2017 From: adam.petcher at oracle.com (Adam Petcher) Date: Fri, 22 Sep 2017 11:11:57 -0400 Subject: JEP for X25519/X448 key agreement In-Reply-To: <2d775c71-a6fd-8b89-6c75-b01775c16617@gmail.com> References: <7c9104dc-ac91-c966-6273-7387410e5a7a@oracle.com> <0fbc6db2-e7dc-c72c-508f-37a521f8ca8a@oracle.com> <2d775c71-a6fd-8b89-6c75-b01775c16617@gmail.com> Message-ID: <1cf3c2a9-94f0-d0dc-60b2-ca01c54ef110@oracle.com> On 9/22/2017 1:27 AM, Anders Rundgren wrote: > On 2017-09-21 17:25, Adam Petcher wrote: >> I would like to leave this open for feedback for another week or so. >> Please reply with your comments by Saturday, September 30, end of day, >> anywhere on earth. After that time, I plan to move on to the next phase >> of the process (group lead and architect review prior to submission). > > This proposal doesn't appear fundamentally different than the one I > drafted on > https://github.com/cyberphone/java-cfrg-spec sometimes ago. > > So I'm obviously OK with that but still have a question: How do you > envision > the corresponding signature algorithms will be supported?? IMO, it would > be useful knowing that before casting things in concrete. After the last round of API review, I developed an initial API design (and prototype implementation) for EdDSA to help address questions like this. I'll share a summary of the initial EdDSA API design for the purpose of evaluating the X25519/X448 JEP, but please don't pay too much attention to the details. We will review the EdDSA API separately in the future. The EdDSA API will be similar to the X25519/X448 design. We can use the string "EdDSA" to identify the algorithm across all services. There are some naming issues to sort out for EdDSA, especially related to algorithm variants and selection of hash functions. These issues don't seem to impact X25519/X448, so we can sort them out later. We will develop interfaces and key specs for EdDSA public/private keys, with a structure that is similar to the one in XDH. Private keys will be represented using byte arrays. Public keys are points, and it would probably be best to add a new class for compressed Edwards curve points (which have a BigInteger y coordinate and a boolean for the x coordinate), but there may be other good options here. EdDSA has some additional algorithm parameters, such as the variant (pure, pre-hash, or context) and the context value. So we would need an EdDSAParameterSpec class to hold these parameters along with the curve parameters. The curve parameters will be stored in the EdDSAParameterSpec as an AlgorithmParameterSpec, allowing them to be specified using a NamedParameterSpec or any other method of specifying an Edwards curve. > > Just for my curiosity, if somebody wanted to implement other variants of > XDH curves, would the proper way to do that be through using a new > provider name (like "XDH-1") while still using the XDH classes and > interfaces? If a provider wants to support some other curve, then a simple way to do this would be to use the algorithm name "XDH", and specify the name of the curve in the NamedParameterSpec. All of the XDH key interfaces and spec classes can still be used. This is similar to how providers support different named curves using ECGenParameterSpec today. Example client code (using a hypothetical named curve "X480"): |KeyPairGenerator kpg = KeyPairGenerator.getInstance("XDH"); NamedParameterSpec paramSpec = new NamedParameterSpec("X480"); kpg.initialize(paramSpec); KeyPair kp = kpg.generateKeyPair(); KeyFactory kf = KeyFactory.getInstance("XDH"); BigInteger u = ... XDHPublicKeySpec pubSpec = new XDHPublicKeySpec(paramSpec, u); PublicKey pubKey = kf.generatePublic(pubSpec); KeyAgreement ka = KeyAgreement.getInstance("XDH"); ka.init(kp.getPrivate()); ka.doPhase(pubKey, true); byte[] secret = ka.generateSecret();| Using a different algorithm name may also be a reasonable option, since you could simply define (for example) "X480" as an algorithm. But this is similar to the example above, and the provider should probably define "X480" as "XDH" initialized with the "X480" NamedParameterSpec. > > Anders > > >> >> >> On 9/14/2017 12:59 PM, Adam Petcher wrote: >>> The JEP for X25519/X448 key agreement[1] is now available and ready to >>> review. Please take a look and reply with any feedback you have. >>> >>> The JEP contains a description of the proposed JCA API. We have >>> discussed the API on this mailing list, and I have attempted to >>> incorporate all the feedback I have received. Here is a description of >>> the changes since the last discussion: >>> >>> 1) Multiple people requested more specific types for public/private >>> keys for this algorithm. The latest API design mirrors the "EC" >>> hierarchy and has both interfaces and spec classes for public and >>> private keys. I also added the interface "XDHKey", which serves the >>> same purpose as "ECKey". >>> 2) The representation of public keys was changed from byte[] to a >>> BigInteger which holds the u coordinate of the point. Private keys are >>> still represented using byte[] due to complications related to >>> pruning, and also because BigInteger doesn't provide a branch-free way >>> to get the key into another representation (which is necessary for >>> side-channel-resilient implementations). >>> >>> The proposed API still lacks a standard way to specify arbitrary >>> domain parameters, but I believe the API design could be extended to >>> support this feature. I would prefer to add this API as a separate >>> enhancement in the future, preferably in cooperation with someone who >>> is developing a provider that supports this feature. >>> >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8181595 >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From philipp.kunz at paratix.ch Sun Sep 24 11:51:56 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Sun, 24 Sep 2017 13:51:56 +0200 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> Message-ID: <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> Hi Max and Vincent Thank you for your suggestions. It sure looks better now. I hope this time I got the patch added in the right format. diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Sun Sep 24 10:34:00 2017 +0200 @@ -28,6 +28,7 @@ import java.security.*; import java.util.HashMap; import java.io.ByteArrayOutputStream; +import static java.nio.charset.StandardCharsets.UTF_8; /** * This class is used to compute digests on sections of the Manifest. @@ -112,7 +113,7 @@ rawBytes = bytes; entries = new HashMap<>(); - ByteArrayOutputStream baos = new ByteArrayOutputStream(); + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); Position pos = new Position(); @@ -131,50 +132,40 @@ if (len > 6) { if (isNameAttr(bytes, start)) { - StringBuilder nameBuf = new StringBuilder(sectionLen); + nameBuf.reset(); + nameBuf.write(bytes, start+6, len-6); - try { - nameBuf.append( - new String(bytes, start+6, len-6, "UTF8")); + int i = start + len; + if ((i-start) < sectionLen) { + if (bytes[i] == '\r') { + i += 2; + } else { + i += 1; + } + } - int i = start + len; - if ((i-start) < sectionLen) { - if (bytes[i] == '\r') { - i += 2; - } else { - i += 1; - } + while ((i-start) < sectionLen) { + if (bytes[i++] == ' ') { + // name is wrapped + int wrapStart = i; + while (((i-start) < sectionLen) + && (bytes[i++] != '\n')); + if (bytes[i-1] != '\n') + return; // XXX: exception? + int wrapLen; + if (bytes[i-2] == '\r') + wrapLen = i-wrapStart-2; + else + wrapLen = i-wrapStart-1; + + nameBuf.write(bytes, wrapStart, wrapLen); + } else { + break; } + } - while ((i-start) < sectionLen) { - if (bytes[i++] == ' ') { - // name is wrapped - int wrapStart = i; - while (((i-start) < sectionLen) - && (bytes[i++] != '\n')); - if (bytes[i-1] != '\n') - return; // XXX: exception? - int wrapLen; - if (bytes[i-2] == '\r') - wrapLen = i-wrapStart-2; - else - wrapLen = i-wrapStart-1; - - nameBuf.append(new String(bytes, wrapStart, - wrapLen, "UTF8")); - } else { - break; - } - } - - entries.put(nameBuf.toString(), - new Entry(start, sectionLen, sectionLenWithBlank, - rawBytes)); - - } catch (java.io.UnsupportedEncodingException uee) { - throw new IllegalStateException( - "UTF8 not available on platform"); - } + entries.put(new String(nameBuf.toByteArray(), UTF_8), + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); } } start = pos.startOfNext; diff -r ddc25f646c2e test/sun/security/tools/jarsigner/MultibyteUnicodeName.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java Sun Sep 24 10:34:00 2017 +0200 @@ -0,0 +1,209 @@ +/* + * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +/* + * @test + * @bug JDK-6695402 + * @summary verify signatures of jars containing classes with names with multi- + * byte unicode characters broken across lines + * @library /test/lib + * @modules jdk.jartool/sun.tools.jar + * jdk.jartool/sun.security.tools.jarsigner + * @run main MultibyteUnicodeName + */ + +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.jar.JarFile; +import java.util.jar.Manifest; +import java.util.Arrays; +import java.util.Map; +import java.util.jar.Attributes.Name; +import java.util.jar.JarEntry; + +import static java.nio.charset.StandardCharsets.UTF_8; + +import static jdk.test.lib.SecurityTools.jarsigner; +import static jdk.test.lib.SecurityTools.keytool; +import static jdk.test.lib.util.JarUtils.createJar; +import static jdk.test.lib.util.JarUtils.updateJar; + +public class MultibyteUnicodeName { + + /** + * this class's name will break across lines in the jar manifest at the middle of + * a two-byte utf encoded character due to its e acute letter at its exact position. + * + * @see also eAcute in {@link MultibyteUnicodeName#verifyClassNameLineBroken(String)} + */ + static class A1234567890B1234567890C1234567890D12345678?xyz { } + static class A1234567890B1234567890C1234567890D12345678exyz { } + + static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + + "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; + static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + + "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; + + static final String alias = "a"; + static final String keystoreFileName = "test.jks"; + static final String ManifestFileName = "MANIFEST.MF"; + + public static void main(String[] args) throws Exception { + try { + prepare(); + + testSignJar("test.jar"); + testSignJarNoManifest("test-no-manifest.jar"); + testSignJarUpdate("test-update.jar", "test-updated.jar"); + + } finally { + Files.deleteIfExists(Paths.get(keystoreFileName)); + Files.deleteIfExists(Paths.get(ManifestFileName)); + Files.deleteIfExists(Paths.get(refClassFilename)); + Files.deleteIfExists(Paths.get(testClassFilename)); + } + } + + static void prepare() throws Exception { + keytool("-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", + "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", + "-validity", "366").shouldHaveExitValue(0); + + Files.write(Paths.get(ManifestFileName), + (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); + Files.copy(MultibyteUnicodeName.class.getResourceAsStream(refClassFilename), + Paths.get(refClassFilename)); + Files.copy(MultibyteUnicodeName.class.getResourceAsStream(testClassFilename), + Paths.get(testClassFilename)); + } + + static void testSignJar(String jarFileName) throws Exception { + try { + createJar(jarFileName, ManifestFileName, refClassFilename, testClassFilename); + verifyJarSignature(jarFileName); + + } finally { + Files.deleteIfExists(Paths.get(jarFileName)); + } + } + + static void testSignJarNoManifest(String jarFileName) throws Exception { + try { + createJar(jarFileName, refClassFilename, testClassFilename); + verifyJarSignature(jarFileName); + + } finally { + Files.deleteIfExists(Paths.get(jarFileName)); + } + } + + static void testSignJarUpdate(String initialFileName, String updatedFileName) throws Exception { + try { + createJar(initialFileName, ManifestFileName, refClassFilename); + jarsigner("-keystore", keystoreFileName, "-storetype", "JKS", + "-storepass", "changeit", "-debug", initialFileName, alias) + .shouldHaveExitValue(0); + updateJar(initialFileName, updatedFileName, testClassFilename); + verifyJarSignature(updatedFileName); + + } finally { + Files.deleteIfExists(Paths.get(initialFileName)); + Files.deleteIfExists(Paths.get(updatedFileName)); + } + } + + static void verifyJarSignature(String jarFileName) throws Exception { + // actually sign the jar + jarsigner("-keystore", keystoreFileName, "-storetype", "JKS", + "-storepass", "changeit", "-debug", jarFileName, alias) + .shouldHaveExitValue(0); + + try ( + JarFile jar = new JarFile(jarFileName); + ) { + verifyClassNameLineBroken(jar, testClassFilename); + verifyCodeSigners(jar, jar.getJarEntry(testClassFilename)); + verifyCodeSigners(jar, jar.getJarEntry(refClassFilename)); + } + } + + /** + * it would be too easy to miss the actual test case by just renaming an identifier so that + * the multi-byte encoded character would not any longer be broken across a line break. + * + * this test verifies that the actual test case is tested based on the manifest + * and not based on the signature file because at the moment, the signature file + * does not even contain the desired entry at all. + * + * this relies on the Manifest breaking lines unaware of bytes that belong to the same + * multi-ybte utf characters. + */ + static void verifyClassNameLineBroken(JarFile jar, String className) throws IOException { + byte[] eAcute = "?".getBytes(UTF_8); + byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; + + if (jar.getManifest().getAttributes(className) == null) { + throw new AssertionError(className + " not found in manifest"); + } + + JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); + try ( + InputStream manifestIs = jar.getInputStream(manifestEntry); + ) { + int bytesMatched = 0; + for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { + if ((byte) b == eAcuteBroken[bytesMatched]) { + bytesMatched++; + if (bytesMatched == eAcuteBroken.length) { + break; + } + } else { + bytesMatched = 0; + } + } + if (bytesMatched < eAcuteBroken.length) { + throw new AssertionError("self-test failed: " + + "multi-byte utf-8 character not broken across lines"); + } + } + } + + static void verifyCodeSigners(JarFile jar, JarEntry jarEntry) throws IOException { + // codeSigners is initialized only after the entry has been read + try ( + InputStream inputStream = jar.getInputStream(jarEntry); + ) { + while (inputStream.read() > -1); + + if (jarEntry.getCodeSigners() == null || jarEntry.getCodeSigners().length == 0) { + throw new AssertionError("no signing certificate found for " + jarEntry.getName()); + } + } + + // a check for the presence of code signers is sufficient to check bug JDK-6695402. + // no need to also verify the actual code signers attributes here. + } + +} Regards, Philipp On 20.09.2017 03:41, Weijun Wang wrote: > Hi Philipp > > The change mostly looks fine. You might want to put everything into a patch file so Vincent can recreate a webrev and post it to cr.openjdk.java.net. > > One thing I would suggest for the test is that instead of using jarsigner -verify and check the text in output you can open it as a JarFile, consume the content of an entry, and look into its getCertificates(). > > Also, you might want to reuse test/lib/jdk/test/lib/SecurityTools.java, and there is also JarUtils.java you can use. Maybe Files.copy(InputStream,Path) can be used in copyClassToFile(). > > Thanks > Max > >> On Sep 20, 2017, at 7:41 AM, Philipp Kunz wrote: >> >> Hello Vincent >> >> Here may be the fix for JDK-6695402. First a test. >> >> BEGIN /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >> /* >> * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or modify it >> * under the terms of the GNU General Public License version 2 only, as >> * published by the Free Software Foundation. >> * >> * This code is distributed in the hope that it will be useful, but WITHOUT >> * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or >> * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License >> * version 2 for more details (a copy is included in the LICENSE file that >> * accompanied this code). >> * >> * You should have received a copy of the GNU General Public License version >> * 2 along with this work; if not, write to the Free Software Foundation, >> * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. >> * >> * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA >> * or visit www.oracle.com if you need additional information or have any >> * questions. >> */ >> >> /* >> * @test >> * @bug JDK-6695402 >> * @summary verify signatures of jars containing classes with names with multi- >> * byte unicode characters broken across lines >> * @library /test/lib >> * @modules jdk.jartool/sun.tools.jar >> * jdk.jartool/sun.security.tools.jarsigner >> * @build jdk.test.lib.JDKToolLauncher >> * jdk.test.lib.process.* >> * @run main MultibyteUnicodeName >> */ >> >> import java.io.ByteArrayOutputStream; >> import java.io.IOException; >> import java.io.InputStream; >> import java.io.UnsupportedEncodingException; >> import java.nio.file.Files; >> import java.nio.file.Paths; >> import java.util.jar.JarFile; >> import java.util.jar.Manifest; >> import java.util.stream.Collectors; >> import java.util.Arrays; >> import java.util.Map; >> import java.util.jar.Attributes.Name; >> import java.util.jar.JarEntry; >> >> import static java.nio.charset.StandardCharsets.UTF_8; >> >> import sun.security.tools.jarsigner.Resources; >> >> import jdk.test.lib.JDKToolLauncher; >> import jdk.test.lib.process.OutputAnalyzer; >> import jdk.test.lib.process.ProcessTools; >> >> public class MultibyteUnicodeName { >> >> public static void main(String[] args) throws Throwable { >> try { >> prepare(); >> >> testSignJar("test.jar"); >> testSignJarNoManifest("test-no-manifest.jar"); >> testSignJarUpdate("test-update.jar"); >> testSignJarWithIndex("test-index.jar"); >> testSignJarAddIndex("test-add-index.jar"); >> >> } finally { >> Files.deleteIfExists(Paths.get(keystoreFileName)); >> Files.deleteIfExists(Paths.get(ManifestFileName)); >> Files.deleteIfExists(Paths.get(refClassFilename)); >> Files.deleteIfExists(Paths.get(testClassFilename)); >> } >> } >> >> static final String alias = "a"; >> static final String keystoreFileName = "test.jks"; >> static final String ManifestFileName = "MANIFEST.MF"; >> >> static class A1234567890B1234567890C1234567890D12345678exyz { } >> static class A1234567890B1234567890C1234567890D12345678?xyz { } >> >> static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + >> "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; >> static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + >> "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; >> >> static void prepare() throws Throwable { >> tool("keytool", "-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", >> "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", >> "-validity", "366") >> .shouldHaveExitValue(0); >> >> Files.write(Paths.get(ManifestFileName), >> (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); >> copyClassToFile(refClassFilename); >> copyClassToFile(testClassFilename); >> } >> >> static void copyClassToFile(String classFilename) throws IOException { >> try ( >> InputStream asStream = MultibyteUnicodeName.class.getResourceAsStream(classFilename); >> ByteArrayOutputStream buf = new ByteArrayOutputStream(); >> ) { >> int b; >> while ((b = asStream.read()) != -1) buf.write(b); >> Files.write(Paths.get(classFilename), buf.toByteArray()); >> } >> } >> >> static void testSignJar(String jarFileName) throws Throwable { >> try { >> tool("jar", "cvfm", jarFileName, >> ManifestFileName, refClassFilename, testClassFilename) >> .shouldHaveExitValue(0); >> verifyJarSignature(jarFileName); >> >> } finally { >> Files.deleteIfExists(Paths.get(jarFileName)); >> } >> } >> >> static void testSignJarNoManifest(String jarFileName) throws Throwable { >> try { >> tool("jar", "cvf", jarFileName, refClassFilename, testClassFilename) >> .shouldHaveExitValue(0); >> verifyJarSignature(jarFileName); >> >> } finally { >> Files.deleteIfExists(Paths.get(jarFileName)); >> } >> } >> >> static void testSignJarUpdate(String jarFileName) throws Throwable { >> try { >> tool("jar", "cvfm", jarFileName, ManifestFileName, refClassFilename) >> .shouldHaveExitValue(0); >> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >> "-storepass", "changeit", "-debug", jarFileName, alias) >> .shouldHaveExitValue(0); >> tool("jar", "uvf", jarFileName, testClassFilename) >> .shouldHaveExitValue(0); >> verifyJarSignature(jarFileName); >> >> } finally { >> Files.deleteIfExists(Paths.get(jarFileName)); >> } >> } >> >> static void testSignJarWithIndex(String jarFileName) throws Throwable { >> try { >> tool("jar", "cvfm", jarFileName, >> ManifestFileName, refClassFilename, testClassFilename) >> .shouldHaveExitValue(0); >> tool("jar", "iv", jarFileName).shouldHaveExitValue(0); >> verifyJarSignature(jarFileName); >> >> } finally { >> Files.deleteIfExists(Paths.get(jarFileName)); >> } >> } >> >> static void testSignJarAddIndex(String jarFileName) throws Throwable { >> try { >> tool("jar", "cvfm", jarFileName, >> ManifestFileName, refClassFilename, testClassFilename) >> .shouldHaveExitValue(0); >> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >> "-storepass", "changeit", "-debug", jarFileName, alias) >> .shouldHaveExitValue(0); >> tool("jar", "iv", jarFileName).shouldHaveExitValue(0); >> verifyJarSignature(jarFileName); >> >> } finally { >> Files.deleteIfExists(Paths.get(jarFileName)); >> } >> } >> >> static Map jarsignerResources = Arrays.stream(new Resources().getContents()). >> collect(Collectors.toMap(e -> (String) e[0], e -> (String) e[1])); >> >> static void verifyJarSignature(String jarFileName) throws Throwable { >> // actually sign the jar >> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >> "-storepass", "changeit", "-debug", jarFileName, alias) >> .shouldHaveExitValue(0); >> >> verifyClassNameLineBroken(jarFileName); >> >> // check for no unsigned entries >> tool("jarsigner", "-verify", "-keystore", keystoreFileName, "-storetype", "JKS", >> "-storepass", "changeit", "-debug", jarFileName, alias) >> .shouldHaveExitValue(0) >> .shouldNotContain(jarsignerResources.get( >> "This.jar.contains.unsigned.entries.which.have.not.been.integrity.checked.")); >> >> // check that both classes with and without multi-byte unicode characters in their names are signed >> tool("jarsigner", "-verify", "-verbose", "-keystore", keystoreFileName, "-storetype", "JKS", >> "-storepass", "changeit", "-debug", jarFileName, alias) >> .shouldHaveExitValue(0) >> .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + refClassFilename.replaceAll("\\$", "\\\\\\$") + "$") >> .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + testClassFilename.replaceAll("\\$", "\\\\\\$") + "$"); >> >> // check that both classes with and without multi-byte unicode characters in their names have signing certificates >> tool("jarsigner", "-verify", "-verbose", "-certs", "-keystore", keystoreFileName, >> "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) >> .shouldHaveExitValue(0) >> .shouldMatch(refClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)") >> .shouldMatch(testClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)"); >> } >> >> /** >> * it would be too easy to miss the actual test case by just renaming an identifier so that >> * the multi-byte encoded character would not any longer be broken across a line break. >> * >> * this test verifies that the actual test case is tested based on the manifest >> * and not based on the signature file because at the moment, the signature file >> * does not even contain the desired entry at all. >> * >> * this relies on the Manifest breaking lines unaware of bytes that belong to the same >> * multi-ybte utf characters. >> */ >> static void verifyClassNameLineBroken(String jarFileName) throws IOException { >> byte[] eAcute = "?".getBytes(UTF_8); >> byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; >> >> try ( >> JarFile jar = new JarFile(jarFileName); >> ) { >> if (jar.getManifest().getAttributes(testClassFilename) == null) { >> throw new AssertionError(testClassFilename + " not found in manifest"); >> } >> >> JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); >> try ( >> InputStream manifestIs = jar.getInputStream(manifestEntry); >> ) { >> int bytesMatched = 0; >> for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { >> if ((byte) b == eAcuteBroken[bytesMatched]) { >> bytesMatched++; >> if (bytesMatched == eAcuteBroken.length) { >> break; >> } >> } else { >> bytesMatched = 0; >> } >> } >> if (bytesMatched < eAcuteBroken.length) { >> throw new AssertionError("self-test failed: " >> + "multi-byte utf-8 character not broken across lines"); >> } >> } >> } >> } >> >> static OutputAnalyzer tool(String tool, String... args) >> throws Throwable { >> JDKToolLauncher l = JDKToolLauncher.createUsingTestJDK(tool); >> for (String arg : args) { >> if (arg .startsWith("-J")) { >> l.addVMArg(arg.substring(2)); >> } else { >> l.addToolArg(arg); >> } >> } >> return ProcessTools.executeCommand(l.getCommand()); >> } >> >> } >> END /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >> >> >> There are three e with acutes, on lines 84, 89, and 227, just in case the mail suffers bad encoding which might not be absolutely unconceivable regarding the bug's subject. >> >> In contrast to the bug description it uses a nested class with a two-byte UTF-8 character rather than one in its own file. I chose to do it that way because then the complete test goes into one file and therefore I assume the overview is kept easier. The test still demonstrates exactly JDK-6695402's problem. >> >> It may not have been necessary to also test jar files with indexes but i consider it necessary to have signing unsigned as well as partially signed jars tested, so why not have one more case tested, too? >> >> There might also be a more elegant approach to get class files into a jar file as to get their contents through the class loader. I chose to use real class files rather than dummy contents in files named *.class or for instance plain text files in the jar the signing of which to be tested in order to stay as close to the original bug problem as possible even though I don't have any notion that it would make a difference. The amount of code used to copy the class file contents around is comparatively small with respect to the whole test case amount of code. >> >> For the demonstration that the multi-byte character actually makes the alleged difference, I concluded that it was necessary to have another class name not previously affected by the bug in order to compare the effects of just different names. Otherwise the signing could fail to any other reason undetectably. >> >> In order to express a condition to tell successful from failed test runs the best approach I found was to analyze the jarsigner tool's output which is in my opinion not the most desirable option. I'd have preferred output from a clearer structured api but then I guess the output format will not change too often in the foreseeable future. For instance the check for the s, m, and k flags is more pragmatical approach than obviously perfectly failsafe when operating with regular expressions to match it with the output. Other parts such as 'X.509' and 'CN=' are not even jarsigner tool resources. I put at least a reference to sun.security.tools.jarsigner.Resources. >> >> Finally, I afforded kind of a self-test that protects the test from undetected failing because renaming the main class which would move the two-byte unicode character to a position not suitable for the test. It may not be absolutely necessary. >> >> >> >> BEGIN PATH >> diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java >> --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 >> +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Wed Sep 20 00:56:15 2017 +0200 >> @@ -28,6 +28,7 @@ >> import java.security.*; >> import java.util.HashMap; >> import java.io.ByteArrayOutputStream; >> +import static java.nio.charset.StandardCharsets.UTF_8; >> >> /** >> * This class is used to compute digests on sections of the Manifest. >> @@ -112,7 +113,7 @@ >> rawBytes = bytes; >> entries = new HashMap<>(); >> >> - ByteArrayOutputStream baos = new ByteArrayOutputStream(); >> + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); >> >> Position pos = new Position(); >> >> @@ -131,50 +132,40 @@ >> >> if (len > 6) { >> if (isNameAttr(bytes, start)) { >> - StringBuilder nameBuf = new StringBuilder(sectionLen); >> + nameBuf.reset(); >> + nameBuf.write(bytes, start+6, len-6); >> >> - try { >> - nameBuf.append( >> - new String(bytes, start+6, len-6, "UTF8")); >> + int i = start + len; >> + if ((i-start) < sectionLen) { >> + if (bytes[i] == '\r') { >> + i += 2; >> + } else { >> + i += 1; >> + } >> + } >> >> - int i = start + len; >> - if ((i-start) < sectionLen) { >> - if (bytes[i] == '\r') { >> - i += 2; >> - } else { >> - i += 1; >> - } >> + while ((i-start) < sectionLen) { >> + if (bytes[i++] == ' ') { >> + // name is wrapped >> + int wrapStart = i; >> + while (((i-start) < sectionLen) >> + && (bytes[i++] != '\n')); >> + if (bytes[i-1] != '\n') >> + return; // XXX: exception? >> + int wrapLen; >> + if (bytes[i-2] == '\r') >> + wrapLen = i-wrapStart-2; >> + else >> + wrapLen = i-wrapStart-1; >> + >> + nameBuf.write(bytes, wrapStart, wrapLen); >> + } else { >> + break; >> } >> + } >> >> - while ((i-start) < sectionLen) { >> - if (bytes[i++] == ' ') { >> - // name is wrapped >> - int wrapStart = i; >> - while (((i-start) < sectionLen) >> - && (bytes[i++] != '\n')); >> - if (bytes[i-1] != '\n') >> - return; // XXX: exception? >> - int wrapLen; >> - if (bytes[i-2] == '\r') >> - wrapLen = i-wrapStart-2; >> - else >> - wrapLen = i-wrapStart-1; >> - >> - nameBuf.append(new String(bytes, wrapStart, >> - wrapLen, "UTF8")); >> - } else { >> - break; >> - } >> - } >> - >> - entries.put(nameBuf.toString(), >> - new Entry(start, sectionLen, sectionLenWithBlank, >> - rawBytes)); >> - >> - } catch (java.io.UnsupportedEncodingException uee) { >> - throw new IllegalStateException( >> - "UTF8 not available on platform"); >> - } >> + entries.put(new String(nameBuf.toByteArray(), UTF_8), >> + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); >> } >> } >> start = pos.startOfNext; >> END PATCH >> >> >> The patch is mostly so big because indentation has changed in many lines. >> >> There was baos there before without apparent use. The patch renames it and puts it into use. It came in very handy because if it would not have been there already, I would have had to defined one of my own. >> >> The main point of the patch is that manifest section names that are broken across lines at possibly arbitrary bytes are no longer converted into strings for each manifest line and then joined. Parts of names broken across lines are now joined first when they are still byte sequences and only when the complete byte sequence is available after processing all manifest lines that contain the same manifest section name decoded into a unicode string. >> >> For decoding the bytes into a string I chose a different string constructor than was used before that does not any longer declare UnsupportedEncodingException rendering the try/catch redundant. It couldn't have ever occurred anyway taking into consideration that UTF-8 is mandatory for every Java platform, StandardCharsets says. The difference according to the documentation is that the previously used String constructor returned undefined strings for invalid byte sequences whereas the one used in the patch will replace unparseable portions with valid 'unknown' characters. >> >> One question I cannot still answer yet is how the ManifestDigester can be changed at all without complete test coverage or risking to break existing signatures. I already started on that but it's quite a piece of work to almost formally prove that all manifests will continue to produce identical digests, except of course for the ones now fixed. >> >> Regards, >> Philipp >> >> >> On 17.09.2017 21:25, Philipp Kunz wrote: >>> Hello Vincent >>> >>> I narrowed the error down so far and suspect now that it is an effect of sun.security.util.ManifestDigester's constructor, lines 134 and 163-164, in combination with java.util.jar.Manifest.make72Safe. Manifest breaks multi-byte characters in manifest section names across lines and ManifestDigester fails to restore them correctly, which both sounds wrong but ManifestDigester sure is. >>> >>> Just fixing it would be too tempting but then I would not know (I mean not know for sure with evidence from tests) if any existing jar-signature would still be valid and I don't want to break existing signatures. When I had a look at the tests specific for Manifest and ManifestDigester I found very little. There are many more different situations and corner cases and combinations thereof to cover with tests than it looks like at first glance so that writing tests that cover all relevant cases looks to me like quite an effort. I have the impression that test coverage was not a priority when the subjected code was developed or at least the tests might not have been contributed to the open JDK project. Hence, while I'm continuing to complete these tests I miss, if someone has an idea how to simplify that so that I can still convince at least myself that no existing signature will break or where possibly more testcases are that I haven't discovered so far, please let me know. >>> >>> I'm also wondering how much performance should be taken into consideration. There are a few hints such as reusing byte arrays that suggest that it is an issue. Will a patch be rejected if it slows down some tests beyond a certain limit for so little added value or what might be the acceptance criteria here? I don't expect insuperable difficulties with performance but would still appreciate some general idea. >>> >>> My desired or currently preferred approach to fix JDK-6695402 would be to let ManifestDigester any kind of use or extend Manifest in order to let Manifest identify the manifest sections thereby preventing the parsing duplicated that identifies the manifest sections. >>> >>> As a new contributor I could use and would appreciate some guidance particularly about what kind and completeness of test coverage and performance tuning applies or is suggested in the context of the given bug. Otherwise I fear I might contribute too many tests which would have to be reviewed and maintained or quite some work would be for nothing if a patch would not satisfy performance requirements. >>> >>> Regards, >>> Philipp >>> >>> >>> >>> On 01.09.2017 15:20, Vincent Ryan wrote: >>>> That all sounds fine. Let me know when your patch is ready to submit. >>>> Thanks. >>>> >>>> >>>>> On 1 Sep 2017, at 13:15, Philipp Kunz wrote: >>>>> >>>>> Hello Vincent >>>>> >>>>> Thank you for sponsoring! >>>>> So far, I have become a contributor by signing the OCA which has been accepted. Dalibor Topic wrote that he can confirm and it's also here: http://www.oracle.com/technetwork/community/oca-486395.html#p -> Paratix GmbH >>>>> Therefore I think I have followed the steps in http://openjdk.java.net/contribute/ at least the ones before actual patch submission. >>>>> >>>>> Currently, I'm working on getting the existing test cases running locally. Unfortunately, I started with jdk 9 and switched to 10 now. I figured these commands run at least the relevant tests with 9 and hope this also applies to 10: >>>>> make run-test-tier1 >>>>> make run-test TEST="jdk/test" >>>>> they report errors and failures but it hasn't been completed and released which might explain it. If I run the tests I assume relevant for my patch >>>>> make run-test TEST="jtreg:jdk/test/sun/security/tools/jarsigner jtreg:jdk/test/java/util/jar" >>>>> then it reports zero errors and failures which may be a good starting point. >>>>> Do you think that sounds reasonable or do you have another suggestion how to run the tests? >>>>> >>>>> Next, I will add a test for JDK-6695402 before actually fixing it. As an example, I'll try something in the style of http://hg.openjdk.java.net/jdk9/dev/jdk/file/65464a307408/test/java/util/jar/Manifest/CreateManifest.java. This way, I try to demonstrate the improvement. >>>>> >>>>> I guess I have identified the following line as the cause: value = new String(vb, 0, 0, vb.length); >>>>> http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/51f5d60713b5/src/java.base/share/classes/java/util/jar/Manifest.java#l157 >>>>> So I'll try to remove it first including the whole four line if block. >>>>> >>>>> Philipp >>>>> >>>>> >>>>> On 01.09.2017 10:00, Vincent Ryan wrote: >>>>>> Hello Philipp, >>>>>> >>>>>> I?m happy to sponsor your fix for JDK 10. Have you followed these steps: http://openjdk.java.net/contribute/ ? >>>>>> >>>>>> Thanks. >>>>>> >>>>>> >>>>>>> On 1 Sep 2017, at 08:58, Vincent Ryan wrote: >>>>>>> >>>>>>> Moved to security-dev >>>>>>> >>>>>>> >>>>>>>> On 1 Sep 2017, at 08:28, Philipp Kunz wrote: >>>>>>>> >>>>>>>> Hello everyone >>>>>>>> >>>>>>>> I have been developing with Java for around 17 years now and when I encountered some bug I decided to attempt to fix it: https://bugs.openjdk.java.net/browse/JDK-6695402. This also looks like it may not be too big a piece for a first contribution. >>>>>>>> >>>>>>>> I read through quite some guides and all kinds of documents but could not yet help myself with the following questions: >>>>>>>> >>>>>>>> May I login to jira to add comments to bugs? If so, how would I request or receive credentials? Or are mailing lists preferred? >>>>>>>> >>>>>>>> Another question is whether I should apply it to jdk9, but it may be too late now, or to jdk10, and backporting can be considered later. Probably it wouldn't even make much a difference for the patch itself. >>>>>>>> >>>>>>>> One more question I have is how or where to find the sources from before migration to mercurial. Because some lines of code I intend to change go back farther and in the history I find only 'initial commit'. With such a history I might be able better to understand why it's there and prevent to make the same mistake again. >>>>>>>> >>>>>>>> I guess the appropriate mailing list for above mentioned bug is security-dev. Is it correct that I can send a patch there and just hope for some sponsor to pick it up? Of course I'd be glad if some sponsor would contact me and maybe provide some assistance or if someone would confirm that sending a patch to the mailing list is the right way to find a sponsor. >>>>>>>> >>>>>>>> Philipp Kunz >>> >>> >>> >>> >>> >>> >>> Paratix GmbH >>> St Peterhofstatt 11 >>> 8001 Z?rich >>> >>> +41 (0)76 397 79 35 >>> philipp.kunz at paratix.ch -- Gruss Philipp ------------------------------------------------------------------------ Paratix GmbH St Peterhofstatt 11 8001 Z?rich +41 (0)76 397 79 35 philipp.kunz at paratix.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 5134 bytes Desc: not available URL: From weijun.wang at oracle.com Sun Sep 24 15:12:22 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Sun, 24 Sep 2017 23:12:22 +0800 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> Message-ID: Great. Some suggestions, just my own habit, you are free to decide yourself: 1. In ManifestDigester.java, I'd rather define nameBuf inside the if (isNameAttr(bytes, start)) block. 2. I normally don't use "import static" unless I can save a lot of keystrokes and still do not confuse anyone. Both in the test and in ManifestDigester.java. 3. In the test, there is no need to write "@run main MultibyteUnicodeName" if it is the only action (i.e. no other build/compile) and there is no modifier on main (i.e. no othervm/timeout etc). Several tiny problems: 1. No need for @modules in the test. Maybe you used some internal classes in your previous version? 2. In the new consolidated repo, the test should be inside test/jdk/sun/..., please note the "jdk" after "test". 3. Please update the copyright years. 4. In the test, there should be no "JDK-" in the @test tag. Thanks Max > On Sep 24, 2017, at 7:51 PM, Philipp Kunz wrote: > > Hi Max and Vincent > > Thank you for your suggestions. It sure looks better now. I hope this time I got the patch added in the right format. > > diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java > --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 > +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Sun Sep 24 10:34:00 2017 +0200 > @@ -28,6 +28,7 @@ > import java.security.*; > import java.util.HashMap; > import java.io.ByteArrayOutputStream; > +import static java.nio.charset.StandardCharsets.UTF_8; > > /** > * This class is used to compute digests on sections of the Manifest. > @@ -112,7 +113,7 @@ > rawBytes = bytes; > entries = new HashMap<>(); > > - ByteArrayOutputStream baos = new ByteArrayOutputStream(); > + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); > > Position pos = new Position(); > > @@ -131,50 +132,40 @@ > > if (len > 6) { > if (isNameAttr(bytes, start)) { > - StringBuilder nameBuf = new StringBuilder(sectionLen); > + nameBuf.reset(); > + nameBuf.write(bytes, start+6, len-6); > > - try { > - nameBuf.append( > - new String(bytes, start+6, len-6, "UTF8")); > + int i = start + len; > + if ((i-start) < sectionLen) { > + if (bytes[i] == '\r') { > + i += 2; > + } else { > + i += 1; > + } > + } > > - int i = start + len; > - if ((i-start) < sectionLen) { > - if (bytes[i] == '\r') { > - i += 2; > - } else { > - i += 1; > - } > + while ((i-start) < sectionLen) { > + if (bytes[i++] == ' ') { > + // name is wrapped > + int wrapStart = i; > + while (((i-start) < sectionLen) > + && (bytes[i++] != '\n')); > + if (bytes[i-1] != '\n') > + return; // XXX: exception? > + int wrapLen; > + if (bytes[i-2] == '\r') > + wrapLen = i-wrapStart-2; > + else > + wrapLen = i-wrapStart-1; > + > + nameBuf.write(bytes, wrapStart, wrapLen); > + } else { > + break; > } > + } > > - while ((i-start) < sectionLen) { > - if (bytes[i++] == ' ') { > - // name is wrapped > - int wrapStart = i; > - while (((i-start) < sectionLen) > - && (bytes[i++] != '\n')); > - if (bytes[i-1] != '\n') > - return; // XXX: exception? > - int wrapLen; > - if (bytes[i-2] == '\r') > - wrapLen = i-wrapStart-2; > - else > - wrapLen = i-wrapStart-1; > - > - nameBuf.append(new String(bytes, wrapStart, > - wrapLen, "UTF8")); > - } else { > - break; > - } > - } > - > - entries.put(nameBuf.toString(), > - new Entry(start, sectionLen, sectionLenWithBlank, > - rawBytes)); > - > - } catch (java.io.UnsupportedEncodingException uee) { > - throw new IllegalStateException( > - "UTF8 not available on platform"); > - } > + entries.put(new String(nameBuf.toByteArray(), UTF_8), > + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); > } > } > start = pos.startOfNext; > diff -r ddc25f646c2e test/sun/security/tools/jarsigner/MultibyteUnicodeName.java > --- /dev/null Thu Jan 01 00:00:00 1970 +0000 > +++ b/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java Sun Sep 24 10:34:00 2017 +0200 > @@ -0,0 +1,209 @@ > +/* > + * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. > + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > + * > + * This code is free software; you can redistribute it and/or modify it > + * under the terms of the GNU General Public License version 2 only, as > + * published by the Free Software Foundation. > + * > + * This code is distributed in the hope that it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License > + * version 2 for more details (a copy is included in the LICENSE file that > + * accompanied this code). > + * > + * You should have received a copy of the GNU General Public License version > + * 2 along with this work; if not, write to the Free Software Foundation, > + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. > + * > + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA > + * or visit www.oracle.com if you need additional information or have any > + * questions. > + */ > + > +/* > + * @test > + * @bug JDK-6695402 > + * @summary verify signatures of jars containing classes with names with multi- > + * byte unicode characters broken across lines > + * @library /test/lib > + * @modules jdk.jartool/sun.tools.jar > + * jdk.jartool/sun.security.tools.jarsigner > + * @run main MultibyteUnicodeName > + */ > + > +import java.io.IOException; > +import java.io.InputStream; > +import java.nio.file.Files; > +import java.nio.file.Paths; > +import java.util.jar.JarFile; > +import java.util.jar.Manifest; > +import java.util.Arrays; > +import java.util.Map; > +import java.util.jar.Attributes.Name; > +import java.util.jar.JarEntry; > + > +import static java.nio.charset.StandardCharsets.UTF_8; > + > +import static jdk.test.lib.SecurityTools.jarsigner; > +import static jdk.test.lib.SecurityTools.keytool; > +import static jdk.test.lib.util.JarUtils.createJar; > +import static jdk.test.lib.util.JarUtils.updateJar; > + > +public class MultibyteUnicodeName { > + > + /** > + * this class's name will break across lines in the jar manifest at the middle of > + * a two-byte utf encoded character due to its e acute letter at its exact position. > + * > + * @see also eAcute in {@link MultibyteUnicodeName#verifyClassNameLineBroken(String)} > + */ > + static class A1234567890B1234567890C1234567890D12345678?xyz { } > + static class A1234567890B1234567890C1234567890D12345678exyz { } > + > + static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + > + "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; > + static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + > + "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; > + > + static final String alias = "a"; > + static final String keystoreFileName = "test.jks"; > + static final String ManifestFileName = "MANIFEST.MF"; > + > + public static void main(String[] args) throws Exception { > + try { > + prepare(); > + > + testSignJar("test.jar"); > + testSignJarNoManifest("test-no-manifest.jar"); > + testSignJarUpdate("test-update.jar", "test-updated.jar"); > + > + } finally { > + Files.deleteIfExists(Paths.get(keystoreFileName)); > + Files.deleteIfExists(Paths.get(ManifestFileName)); > + Files.deleteIfExists(Paths.get(refClassFilename)); > + Files.deleteIfExists(Paths.get(testClassFilename)); > + } > + } > + > + static void prepare() throws Exception { > + keytool("-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", > + "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", > + "-validity", "366").shouldHaveExitValue(0); > + > + Files.write(Paths.get(ManifestFileName), > + (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); > + Files.copy(MultibyteUnicodeName.class.getResourceAsStream(refClassFilename), > + Paths.get(refClassFilename)); > + Files.copy(MultibyteUnicodeName.class.getResourceAsStream(testClassFilename), > + Paths.get(testClassFilename)); > + } > + > + static void testSignJar(String jarFileName) throws Exception { > + try { > + createJar(jarFileName, ManifestFileName, refClassFilename, testClassFilename); > + verifyJarSignature(jarFileName); > + > + } finally { > + Files.deleteIfExists(Paths.get(jarFileName)); > + } > + } > + > + static void testSignJarNoManifest(String jarFileName) throws Exception { > + try { > + createJar(jarFileName, refClassFilename, testClassFilename); > + verifyJarSignature(jarFileName); > + > + } finally { > + Files.deleteIfExists(Paths.get(jarFileName)); > + } > + } > + > + static void testSignJarUpdate(String initialFileName, String updatedFileName) throws Exception { > + try { > + createJar(initialFileName, ManifestFileName, refClassFilename); > + jarsigner("-keystore", keystoreFileName, "-storetype", "JKS", > + "-storepass", "changeit", "-debug", initialFileName, alias) > + .shouldHaveExitValue(0); > + updateJar(initialFileName, updatedFileName, testClassFilename); > + verifyJarSignature(updatedFileName); > + > + } finally { > + Files.deleteIfExists(Paths.get(initialFileName)); > + Files.deleteIfExists(Paths.get(updatedFileName)); > + } > + } > + > + static void verifyJarSignature(String jarFileName) throws Exception { > + // actually sign the jar > + jarsigner("-keystore", keystoreFileName, "-storetype", "JKS", > + "-storepass", "changeit", "-debug", jarFileName, alias) > + .shouldHaveExitValue(0); > + > + try ( > + JarFile jar = new JarFile(jarFileName); > + ) { > + verifyClassNameLineBroken(jar, testClassFilename); > + verifyCodeSigners(jar, jar.getJarEntry(testClassFilename)); > + verifyCodeSigners(jar, jar.getJarEntry(refClassFilename)); > + } > + } > + > + /** > + * it would be too easy to miss the actual test case by just renaming an identifier so that > + * the multi-byte encoded character would not any longer be broken across a line break. > + * > + * this test verifies that the actual test case is tested based on the manifest > + * and not based on the signature file because at the moment, the signature file > + * does not even contain the desired entry at all. > + * > + * this relies on the Manifest breaking lines unaware of bytes that belong to the same > + * multi-ybte utf characters. > + */ > + static void verifyClassNameLineBroken(JarFile jar, String className) throws IOException { > + byte[] eAcute = "?".getBytes(UTF_8); > + byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; > + > + if (jar.getManifest().getAttributes(className) == null) { > + throw new AssertionError(className + " not found in manifest"); > + } > + > + JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); > + try ( > + InputStream manifestIs = jar.getInputStream(manifestEntry); > + ) { > + int bytesMatched = 0; > + for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { > + if ((byte) b == eAcuteBroken[bytesMatched]) { > + bytesMatched++; > + if (bytesMatched == eAcuteBroken.length) { > + break; > + } > + } else { > + bytesMatched = 0; > + } > + } > + if (bytesMatched < eAcuteBroken.length) { > + throw new AssertionError("self-test failed: " > + + "multi-byte utf-8 character not broken across lines"); > + } > + } > + } > + > + static void verifyCodeSigners(JarFile jar, JarEntry jarEntry) throws IOException { > + // codeSigners is initialized only after the entry has been read > + try ( > + InputStream inputStream = jar.getInputStream(jarEntry); > + ) { > + while (inputStream.read() > -1); > + > + if (jarEntry.getCodeSigners() == null || jarEntry.getCodeSigners().length == 0) { > + throw new AssertionError("no signing certificate found for " + jarEntry.getName()); > + } > + } > + > + // a check for the presence of code signers is sufficient to check bug JDK-6695402. > + // no need to also verify the actual code signers attributes here. > + } > + > +} > > Regards, > Philipp > > > > On 20.09.2017 03:41, Weijun Wang wrote: >> Hi Philipp >> >> The change mostly looks fine. You might want to put everything into a patch file so Vincent can recreate a webrev and post it to cr.openjdk.java.net. >> >> One thing I would suggest for the test is that instead of using jarsigner -verify and check the text in output you can open it as a JarFile, consume the content of an entry, and look into its getCertificates(). >> >> Also, you might want to reuse test/lib/jdk/test/lib/SecurityTools.java, and there is also JarUtils.java you can use. Maybe Files.copy(InputStream,Path) can be used in copyClassToFile(). >> >> Thanks >> Max >> >> >>> On Sep 20, 2017, at 7:41 AM, Philipp Kunz >>> wrote: >>> >>> Hello Vincent >>> >>> Here may be the fix for JDK-6695402. First a test. >>> >>> BEGIN /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >>> /* >>> * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. >>> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>> * >>> * This code is free software; you can redistribute it and/or modify it >>> * under the terms of the GNU General Public License version 2 only, as >>> * published by the Free Software Foundation. >>> * >>> * This code is distributed in the hope that it will be useful, but WITHOUT >>> * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or >>> * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License >>> * version 2 for more details (a copy is included in the LICENSE file that >>> * accompanied this code). >>> * >>> * You should have received a copy of the GNU General Public License version >>> * 2 along with this work; if not, write to the Free Software Foundation, >>> * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. >>> * >>> * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA >>> * or visit >>> www.oracle.com >>> if you need additional information or have any >>> * questions. >>> */ >>> >>> /* >>> * @test >>> * @bug JDK-6695402 >>> * @summary verify signatures of jars containing classes with names with multi- >>> * byte unicode characters broken across lines >>> * @library /test/lib >>> * @modules jdk.jartool/sun.tools.jar >>> * jdk.jartool/sun.security.tools.jarsigner >>> * @build jdk.test.lib.JDKToolLauncher >>> * jdk.test.lib.process.* >>> * @run main MultibyteUnicodeName >>> */ >>> >>> import java.io.ByteArrayOutputStream; >>> import java.io.IOException; >>> import java.io.InputStream; >>> import java.io.UnsupportedEncodingException; >>> import java.nio.file.Files; >>> import java.nio.file.Paths; >>> import java.util.jar.JarFile; >>> import java.util.jar.Manifest; >>> import java.util.stream.Collectors; >>> import java.util.Arrays; >>> import java.util.Map; >>> import java.util.jar.Attributes.Name; >>> import java.util.jar.JarEntry; >>> >>> import static java.nio.charset.StandardCharsets.UTF_8; >>> >>> import sun.security.tools.jarsigner.Resources; >>> >>> import jdk.test.lib.JDKToolLauncher; >>> import jdk.test.lib.process.OutputAnalyzer; >>> import jdk.test.lib.process.ProcessTools; >>> >>> public class MultibyteUnicodeName { >>> >>> public static void main(String[] args) throws Throwable { >>> try { >>> prepare(); >>> >>> testSignJar("test.jar"); >>> testSignJarNoManifest("test-no-manifest.jar"); >>> testSignJarUpdate("test-update.jar"); >>> testSignJarWithIndex("test-index.jar"); >>> testSignJarAddIndex("test-add-index.jar"); >>> >>> } finally { >>> Files.deleteIfExists(Paths.get(keystoreFileName)); >>> Files.deleteIfExists(Paths.get(ManifestFileName)); >>> Files.deleteIfExists(Paths.get(refClassFilename)); >>> Files.deleteIfExists(Paths.get(testClassFilename)); >>> } >>> } >>> >>> static final String alias = "a"; >>> static final String keystoreFileName = "test.jks"; >>> static final String ManifestFileName = "MANIFEST.MF"; >>> >>> static class A1234567890B1234567890C1234567890D12345678exyz { } >>> static class A1234567890B1234567890C1234567890D12345678?xyz { } >>> >>> static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + >>> "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; >>> static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + >>> "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; >>> >>> static void prepare() throws Throwable { >>> tool("keytool", "-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", >>> "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", >>> "-validity", "366") >>> .shouldHaveExitValue(0); >>> >>> Files.write(Paths.get(ManifestFileName), >>> (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); >>> copyClassToFile(refClassFilename); >>> copyClassToFile(testClassFilename); >>> } >>> >>> static void copyClassToFile(String classFilename) throws IOException { >>> try ( >>> InputStream asStream = MultibyteUnicodeName.class.getResourceAsStream(classFilename); >>> ByteArrayOutputStream buf = new ByteArrayOutputStream(); >>> ) { >>> int b; >>> while ((b = asStream.read()) != -1) buf.write(b); >>> Files.write(Paths.get(classFilename), buf.toByteArray()); >>> } >>> } >>> >>> static void testSignJar(String jarFileName) throws Throwable { >>> try { >>> tool("jar", "cvfm", jarFileName, >>> ManifestFileName, refClassFilename, testClassFilename) >>> .shouldHaveExitValue(0); >>> verifyJarSignature(jarFileName); >>> >>> } finally { >>> Files.deleteIfExists(Paths.get(jarFileName)); >>> } >>> } >>> >>> static void testSignJarNoManifest(String jarFileName) throws Throwable { >>> try { >>> tool("jar", "cvf", jarFileName, refClassFilename, testClassFilename) >>> .shouldHaveExitValue(0); >>> verifyJarSignature(jarFileName); >>> >>> } finally { >>> Files.deleteIfExists(Paths.get(jarFileName)); >>> } >>> } >>> >>> static void testSignJarUpdate(String jarFileName) throws Throwable { >>> try { >>> tool("jar", "cvfm", jarFileName, ManifestFileName, refClassFilename) >>> .shouldHaveExitValue(0); >>> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >>> "-storepass", "changeit", "-debug", jarFileName, alias) >>> .shouldHaveExitValue(0); >>> tool("jar", "uvf", jarFileName, testClassFilename) >>> .shouldHaveExitValue(0); >>> verifyJarSignature(jarFileName); >>> >>> } finally { >>> Files.deleteIfExists(Paths.get(jarFileName)); >>> } >>> } >>> >>> static void testSignJarWithIndex(String jarFileName) throws Throwable { >>> try { >>> tool("jar", "cvfm", jarFileName, >>> ManifestFileName, refClassFilename, testClassFilename) >>> .shouldHaveExitValue(0); >>> tool("jar", "iv", jarFileName).shouldHaveExitValue(0); >>> verifyJarSignature(jarFileName); >>> >>> } finally { >>> Files.deleteIfExists(Paths.get(jarFileName)); >>> } >>> } >>> >>> static void testSignJarAddIndex(String jarFileName) throws Throwable { >>> try { >>> tool("jar", "cvfm", jarFileName, >>> ManifestFileName, refClassFilename, testClassFilename) >>> .shouldHaveExitValue(0); >>> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >>> "-storepass", "changeit", "-debug", jarFileName, alias) >>> .shouldHaveExitValue(0); >>> tool("jar", "iv", jarFileName).shouldHaveExitValue(0); >>> verifyJarSignature(jarFileName); >>> >>> } finally { >>> Files.deleteIfExists(Paths.get(jarFileName)); >>> } >>> } >>> >>> static Map jarsignerResources = Arrays.stream(new Resources().getContents()). >>> collect(Collectors.toMap(e -> (String) e[0], e -> (String) e[1])); >>> >>> static void verifyJarSignature(String jarFileName) throws Throwable { >>> // actually sign the jar >>> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >>> "-storepass", "changeit", "-debug", jarFileName, alias) >>> .shouldHaveExitValue(0); >>> >>> verifyClassNameLineBroken(jarFileName); >>> >>> // check for no unsigned entries >>> tool("jarsigner", "-verify", "-keystore", keystoreFileName, "-storetype", "JKS", >>> "-storepass", "changeit", "-debug", jarFileName, alias) >>> .shouldHaveExitValue(0) >>> .shouldNotContain(jarsignerResources.get( >>> "This.jar.contains.unsigned.entries.which.have.not.been.integrity.checked.")); >>> >>> // check that both classes with and without multi-byte unicode characters in their names are signed >>> tool("jarsigner", "-verify", "-verbose", "-keystore", keystoreFileName, "-storetype", "JKS", >>> "-storepass", "changeit", "-debug", jarFileName, alias) >>> .shouldHaveExitValue(0) >>> .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + refClassFilename.replaceAll("\\$", "\\\\\\$") + "$") >>> .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + testClassFilename.replaceAll("\\$", "\\\\\\$") + "$"); >>> >>> // check that both classes with and without multi-byte unicode characters in their names have signing certificates >>> tool("jarsigner", "-verify", "-verbose", "-certs", "-keystore", keystoreFileName, >>> "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) >>> .shouldHaveExitValue(0) >>> .shouldMatch(refClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)") >>> .shouldMatch(testClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)"); >>> } >>> >>> /** >>> * it would be too easy to miss the actual test case by just renaming an identifier so that >>> * the multi-byte encoded character would not any longer be broken across a line break. >>> * >>> * this test verifies that the actual test case is tested based on the manifest >>> * and not based on the signature file because at the moment, the signature file >>> * does not even contain the desired entry at all. >>> * >>> * this relies on the Manifest breaking lines unaware of bytes that belong to the same >>> * multi-ybte utf characters. >>> */ >>> static void verifyClassNameLineBroken(String jarFileName) throws IOException { >>> byte[] eAcute = "?".getBytes(UTF_8); >>> byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; >>> >>> try ( >>> JarFile jar = new JarFile(jarFileName); >>> ) { >>> if (jar.getManifest().getAttributes(testClassFilename) == null) { >>> throw new AssertionError(testClassFilename + " not found in manifest"); >>> } >>> >>> JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); >>> try ( >>> InputStream manifestIs = jar.getInputStream(manifestEntry); >>> ) { >>> int bytesMatched = 0; >>> for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { >>> if ((byte) b == eAcuteBroken[bytesMatched]) { >>> bytesMatched++; >>> if (bytesMatched == eAcuteBroken.length) { >>> break; >>> } >>> } else { >>> bytesMatched = 0; >>> } >>> } >>> if (bytesMatched < eAcuteBroken.length) { >>> throw new AssertionError("self-test failed: " >>> + "multi-byte utf-8 character not broken across lines"); >>> } >>> } >>> } >>> } >>> >>> static OutputAnalyzer tool(String tool, String... args) >>> throws Throwable { >>> JDKToolLauncher l = JDKToolLauncher.createUsingTestJDK(tool); >>> for (String arg : args) { >>> if (arg .startsWith("-J")) { >>> l.addVMArg(arg.substring(2)); >>> } else { >>> l.addToolArg(arg); >>> } >>> } >>> return ProcessTools.executeCommand(l.getCommand()); >>> } >>> >>> } >>> END /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >>> >>> >>> There are three e with acutes, on lines 84, 89, and 227, just in case the mail suffers bad encoding which might not be absolutely unconceivable regarding the bug's subject. >>> >>> In contrast to the bug description it uses a nested class with a two-byte UTF-8 character rather than one in its own file. I chose to do it that way because then the complete test goes into one file and therefore I assume the overview is kept easier. The test still demonstrates exactly JDK-6695402's problem. >>> >>> It may not have been necessary to also test jar files with indexes but i consider it necessary to have signing unsigned as well as partially signed jars tested, so why not have one more case tested, too? >>> >>> There might also be a more elegant approach to get class files into a jar file as to get their contents through the class loader. I chose to use real class files rather than dummy contents in files named *.class or for instance plain text files in the jar the signing of which to be tested in order to stay as close to the original bug problem as possible even though I don't have any notion that it would make a difference. The amount of code used to copy the class file contents around is comparatively small with respect to the whole test case amount of code. >>> >>> For the demonstration that the multi-byte character actually makes the alleged difference, I concluded that it was necessary to have another class name not previously affected by the bug in order to compare the effects of just different names. Otherwise the signing could fail to any other reason undetectably. >>> >>> In order to express a condition to tell successful from failed test runs the best approach I found was to analyze the jarsigner tool's output which is in my opinion not the most desirable option. I'd have preferred output from a clearer structured api but then I guess the output format will not change too often in the foreseeable future. For instance the check for the s, m, and k flags is more pragmatical approach than obviously perfectly failsafe when operating with regular expressions to match it with the output. Other parts such as 'X.509' and 'CN=' are not even jarsigner tool resources. I put at least a reference to sun.security.tools.jarsigner.Resources. >>> >>> Finally, I afforded kind of a self-test that protects the test from undetected failing because renaming the main class which would move the two-byte unicode character to a position not suitable for the test. It may not be absolutely necessary. >>> >>> >>> >>> BEGIN PATH >>> diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java >>> --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 >>> +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Wed Sep 20 00:56:15 2017 +0200 >>> @@ -28,6 +28,7 @@ >>> import java.security.*; >>> import java.util.HashMap; >>> import java.io.ByteArrayOutputStream; >>> +import static java.nio.charset.StandardCharsets.UTF_8; >>> >>> /** >>> * This class is used to compute digests on sections of the Manifest. >>> @@ -112,7 +113,7 @@ >>> rawBytes = bytes; >>> entries = new HashMap<>(); >>> >>> - ByteArrayOutputStream baos = new ByteArrayOutputStream(); >>> + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); >>> >>> Position pos = new Position(); >>> >>> @@ -131,50 +132,40 @@ >>> >>> if (len > 6) { >>> if (isNameAttr(bytes, start)) { >>> - StringBuilder nameBuf = new StringBuilder(sectionLen); >>> + nameBuf.reset(); >>> + nameBuf.write(bytes, start+6, len-6); >>> >>> - try { >>> - nameBuf.append( >>> - new String(bytes, start+6, len-6, "UTF8")); >>> + int i = start + len; >>> + if ((i-start) < sectionLen) { >>> + if (bytes[i] == '\r') { >>> + i += 2; >>> + } else { >>> + i += 1; >>> + } >>> + } >>> >>> - int i = start + len; >>> - if ((i-start) < sectionLen) { >>> - if (bytes[i] == '\r') { >>> - i += 2; >>> - } else { >>> - i += 1; >>> - } >>> + while ((i-start) < sectionLen) { >>> + if (bytes[i++] == ' ') { >>> + // name is wrapped >>> + int wrapStart = i; >>> + while (((i-start) < sectionLen) >>> + && (bytes[i++] != '\n')); >>> + if (bytes[i-1] != '\n') >>> + return; // XXX: exception? >>> + int wrapLen; >>> + if (bytes[i-2] == '\r') >>> + wrapLen = i-wrapStart-2; >>> + else >>> + wrapLen = i-wrapStart-1; >>> + >>> + nameBuf.write(bytes, wrapStart, wrapLen); >>> + } else { >>> + break; >>> } >>> + } >>> >>> - while ((i-start) < sectionLen) { >>> - if (bytes[i++] == ' ') { >>> - // name is wrapped >>> - int wrapStart = i; >>> - while (((i-start) < sectionLen) >>> - && (bytes[i++] != '\n')); >>> - if (bytes[i-1] != '\n') >>> - return; // XXX: exception? >>> - int wrapLen; >>> - if (bytes[i-2] == '\r') >>> - wrapLen = i-wrapStart-2; >>> - else >>> - wrapLen = i-wrapStart-1; >>> - >>> - nameBuf.append(new String(bytes, wrapStart, >>> - wrapLen, "UTF8")); >>> - } else { >>> - break; >>> - } >>> - } >>> - >>> - entries.put(nameBuf.toString(), >>> - new Entry(start, sectionLen, sectionLenWithBlank, >>> - rawBytes)); >>> - >>> - } catch (java.io.UnsupportedEncodingException uee) { >>> - throw new IllegalStateException( >>> - "UTF8 not available on platform"); >>> - } >>> + entries.put(new String(nameBuf.toByteArray(), UTF_8), >>> + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); >>> } >>> } >>> start = pos.startOfNext; >>> END PATCH >>> >>> >>> The patch is mostly so big because indentation has changed in many lines. >>> >>> There was baos there before without apparent use. The patch renames it and puts it into use. It came in very handy because if it would not have been there already, I would have had to defined one of my own. >>> >>> The main point of the patch is that manifest section names that are broken across lines at possibly arbitrary bytes are no longer converted into strings for each manifest line and then joined. Parts of names broken across lines are now joined first when they are still byte sequences and only when the complete byte sequence is available after processing all manifest lines that contain the same manifest section name decoded into a unicode string. >>> >>> For decoding the bytes into a string I chose a different string constructor than was used before that does not any longer declare UnsupportedEncodingException rendering the try/catch redundant. It couldn't have ever occurred anyway taking into consideration that UTF-8 is mandatory for every Java platform, StandardCharsets says. The difference according to the documentation is that the previously used String constructor returned undefined strings for invalid byte sequences whereas the one used in the patch will replace unparseable portions with valid 'unknown' characters. >>> >>> One question I cannot still answer yet is how the ManifestDigester can be changed at all without complete test coverage or risking to break existing signatures. I already started on that but it's quite a piece of work to almost formally prove that all manifests will continue to produce identical digests, except of course for the ones now fixed. >>> >>> Regards, >>> Philipp >>> >>> >>> On 17.09.2017 21:25, Philipp Kunz wrote: >>> >>>> Hello Vincent >>>> >>>> I narrowed the error down so far and suspect now that it is an effect of sun.security.util.ManifestDigester's constructor, lines 134 and 163-164, in combination with java.util.jar.Manifest.make72Safe. Manifest breaks multi-byte characters in manifest section names across lines and ManifestDigester fails to restore them correctly, which both sounds wrong but ManifestDigester sure is. >>>> >>>> Just fixing it would be too tempting but then I would not know (I mean not know for sure with evidence from tests) if any existing jar-signature would still be valid and I don't want to break existing signatures. When I had a look at the tests specific for Manifest and ManifestDigester I found very little. There are many more different situations and corner cases and combinations thereof to cover with tests than it looks like at first glance so that writing tests that cover all relevant cases looks to me like quite an effort. I have the impression that test coverage was not a priority when the subjected code was developed or at least the tests might not have been contributed to the open JDK project. Hence, while I'm continuing to complete these tests I miss, if someone has an idea how to simplify that so that I can still convince at least myself that no existing signature will break or where possibly more testcases are that I haven't discovered so far, please let me know. >>>> >>>> I'm also wondering how much performance should be taken into consideration. There are a few hints such as reusing byte arrays that suggest that it is an issue. Will a patch be rejected if it slows down some tests beyond a certain limit for so little added value or what might be the acceptance criteria here? I don't expect insuperable difficulties with performance but would still appreciate some general idea. >>>> >>>> My desired or currently preferred approach to fix JDK-6695402 would be to let ManifestDigester any kind of use or extend Manifest in order to let Manifest identify the manifest sections thereby preventing the parsing duplicated that identifies the manifest sections. >>>> >>>> As a new contributor I could use and would appreciate some guidance particularly about what kind and completeness of test coverage and performance tuning applies or is suggested in the context of the given bug. Otherwise I fear I might contribute too many tests which would have to be reviewed and maintained or quite some work would be for nothing if a patch would not satisfy performance requirements. >>>> >>>> Regards, >>>> Philipp >>>> >>>> >>>> >>>> On 01.09.2017 15:20, Vincent Ryan wrote: >>>> >>>>> That all sounds fine. Let me know when your patch is ready to submit. >>>>> Thanks. >>>>> >>>>> >>>>> >>>>>> On 1 Sep 2017, at 13:15, Philipp Kunz >>>>>> wrote: >>>>>> >>>>>> Hello Vincent >>>>>> >>>>>> Thank you for sponsoring! >>>>>> So far, I have become a contributor by signing the OCA which has been accepted. Dalibor Topic wrote that he can confirm and it's also here: >>>>>> http://www.oracle.com/technetwork/community/oca-486395.html#p >>>>>> -> Paratix GmbH >>>>>> Therefore I think I have followed the steps in >>>>>> http://openjdk.java.net/contribute/ >>>>>> at least the ones before actual patch submission. >>>>>> >>>>>> Currently, I'm working on getting the existing test cases running locally. Unfortunately, I started with jdk 9 and switched to 10 now. I figured these commands run at least the relevant tests with 9 and hope this also applies to 10: >>>>>> make run-test-tier1 >>>>>> make run-test TEST="jdk/test" >>>>>> they report errors and failures but it hasn't been completed and released which might explain it. If I run the tests I assume relevant for my patch >>>>>> make run-test TEST="jtreg:jdk/test/sun/security/tools/jarsigner jtreg:jdk/test/java/util/jar" >>>>>> then it reports zero errors and failures which may be a good starting point. >>>>>> Do you think that sounds reasonable or do you have another suggestion how to run the tests? >>>>>> >>>>>> Next, I will add a test for JDK-6695402 before actually fixing it. As an example, I'll try something in the style of >>>>>> http://hg.openjdk.java.net/jdk9/dev/jdk/file/65464a307408/test/java/util/jar/Manifest/CreateManifest.java >>>>>> . This way, I try to demonstrate the improvement. >>>>>> >>>>>> I guess I have identified the following line as the cause: value = new String(vb, 0, 0, vb.length); >>>>>> >>>>>> http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/51f5d60713b5/src/java.base/share/classes/java/util/jar/Manifest.java#l157 >>>>>> >>>>>> So I'll try to remove it first including the whole four line if block. >>>>>> >>>>>> Philipp >>>>>> >>>>>> >>>>>> On 01.09.2017 10:00, Vincent Ryan wrote: >>>>>> >>>>>>> Hello Philipp, >>>>>>> >>>>>>> I?m happy to sponsor your fix for JDK 10. Have you followed these steps: >>>>>>> http://openjdk.java.net/contribute/ >>>>>>> ? >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 1 Sep 2017, at 08:58, Vincent Ryan >>>>>>>> wrote: >>>>>>>> >>>>>>>> Moved to security-dev >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On 1 Sep 2017, at 08:28, Philipp Kunz >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hello everyone >>>>>>>>> >>>>>>>>> I have been developing with Java for around 17 years now and when I encountered some bug I decided to attempt to fix it: >>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-6695402 >>>>>>>>> . This also looks like it may not be too big a piece for a first contribution. >>>>>>>>> >>>>>>>>> I read through quite some guides and all kinds of documents but could not yet help myself with the following questions: >>>>>>>>> >>>>>>>>> May I login to jira to add comments to bugs? If so, how would I request or receive credentials? Or are mailing lists preferred? >>>>>>>>> >>>>>>>>> Another question is whether I should apply it to jdk9, but it may be too late now, or to jdk10, and backporting can be considered later. Probably it wouldn't even make much a difference for the patch itself. >>>>>>>>> >>>>>>>>> One more question I have is how or where to find the sources from before migration to mercurial. Because some lines of code I intend to change go back farther and in the history I find only 'initial commit'. With such a history I might be able better to understand why it's there and prevent to make the same mistake again. >>>>>>>>> >>>>>>>>> I guess the appropriate mailing list for above mentioned bug is security-dev. Is it correct that I can send a patch there and just hope for some sponsor to pick it up? Of course I'd be glad if some sponsor would contact me and maybe provide some assistance or if someone would confirm that sending a patch to the mailing list is the right way to find a sponsor. >>>>>>>>> >>>>>>>>> Philipp Kunz >>>>>>>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Paratix GmbH >>>> St Peterhofstatt 11 >>>> 8001 Z?rich >>>> >>>> +41 (0)76 397 79 35 >>>> >>>> philipp.kunz at paratix.ch > > -- > > > Gruss Philipp > > > > > > > Paratix GmbH > St Peterhofstatt 11 > 8001 Z?rich > > +41 (0)76 397 79 35 > philipp.kunz at paratix.ch From weijun.wang at oracle.com Sun Sep 24 15:41:44 2017 From: weijun.wang at oracle.com (Wang Weijun) Date: Sun, 24 Sep 2017 23:41:44 +0800 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> Message-ID: Some more suggestions on the test: 1. You can use readAllBytes() to ... err ... read all bytes. 2. I normally don?t remove intermediate files. Jtreg will do an automatic cleanup, and it can be configured to retain those files when the test fails. They will be very helpful in debugging. Thanks Max > ? 2017?9?24??23:12?Weijun Wang ??? > > Great. > > Some suggestions, just my own habit, you are free to decide yourself: > > 1. In ManifestDigester.java, I'd rather define nameBuf inside the if (isNameAttr(bytes, start)) block. > > 2. I normally don't use "import static" unless I can save a lot of keystrokes and still do not confuse anyone. Both in the test and in ManifestDigester.java. > > 3. In the test, there is no need to write "@run main MultibyteUnicodeName" if it is the only action (i.e. no other build/compile) and there is no modifier on main (i.e. no othervm/timeout etc). > > Several tiny problems: > > 1. No need for @modules in the test. Maybe you used some internal classes in your previous version? > > 2. In the new consolidated repo, the test should be inside test/jdk/sun/..., please note the "jdk" after "test". > > 3. Please update the copyright years. > > 4. In the test, there should be no "JDK-" in the @test tag. > > Thanks > Max > >> On Sep 24, 2017, at 7:51 PM, Philipp Kunz wrote: >> >> Hi Max and Vincent >> >> Thank you for your suggestions. It sure looks better now. I hope this time I got the patch added in the right format. >> >> diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java >> --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 >> +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Sun Sep 24 10:34:00 2017 +0200 >> @@ -28,6 +28,7 @@ >> import java.security.*; >> import java.util.HashMap; >> import java.io.ByteArrayOutputStream; >> +import static java.nio.charset.StandardCharsets.UTF_8; >> >> /** >> * This class is used to compute digests on sections of the Manifest. >> @@ -112,7 +113,7 @@ >> rawBytes = bytes; >> entries = new HashMap<>(); >> >> - ByteArrayOutputStream baos = new ByteArrayOutputStream(); >> + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); >> >> Position pos = new Position(); >> >> @@ -131,50 +132,40 @@ >> >> if (len > 6) { >> if (isNameAttr(bytes, start)) { >> - StringBuilder nameBuf = new StringBuilder(sectionLen); >> + nameBuf.reset(); >> + nameBuf.write(bytes, start+6, len-6); >> >> - try { >> - nameBuf.append( >> - new String(bytes, start+6, len-6, "UTF8")); >> + int i = start + len; >> + if ((i-start) < sectionLen) { >> + if (bytes[i] == '\r') { >> + i += 2; >> + } else { >> + i += 1; >> + } >> + } >> >> - int i = start + len; >> - if ((i-start) < sectionLen) { >> - if (bytes[i] == '\r') { >> - i += 2; >> - } else { >> - i += 1; >> - } >> + while ((i-start) < sectionLen) { >> + if (bytes[i++] == ' ') { >> + // name is wrapped >> + int wrapStart = i; >> + while (((i-start) < sectionLen) >> + && (bytes[i++] != '\n')); >> + if (bytes[i-1] != '\n') >> + return; // XXX: exception? >> + int wrapLen; >> + if (bytes[i-2] == '\r') >> + wrapLen = i-wrapStart-2; >> + else >> + wrapLen = i-wrapStart-1; >> + >> + nameBuf.write(bytes, wrapStart, wrapLen); >> + } else { >> + break; >> } >> + } >> >> - while ((i-start) < sectionLen) { >> - if (bytes[i++] == ' ') { >> - // name is wrapped >> - int wrapStart = i; >> - while (((i-start) < sectionLen) >> - && (bytes[i++] != '\n')); >> - if (bytes[i-1] != '\n') >> - return; // XXX: exception? >> - int wrapLen; >> - if (bytes[i-2] == '\r') >> - wrapLen = i-wrapStart-2; >> - else >> - wrapLen = i-wrapStart-1; >> - >> - nameBuf.append(new String(bytes, wrapStart, >> - wrapLen, "UTF8")); >> - } else { >> - break; >> - } >> - } >> - >> - entries.put(nameBuf.toString(), >> - new Entry(start, sectionLen, sectionLenWithBlank, >> - rawBytes)); >> - >> - } catch (java.io.UnsupportedEncodingException uee) { >> - throw new IllegalStateException( >> - "UTF8 not available on platform"); >> - } >> + entries.put(new String(nameBuf.toByteArray(), UTF_8), >> + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); >> } >> } >> start = pos.startOfNext; >> diff -r ddc25f646c2e test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >> --- /dev/null Thu Jan 01 00:00:00 1970 +0000 >> +++ b/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java Sun Sep 24 10:34:00 2017 +0200 >> @@ -0,0 +1,209 @@ >> +/* >> + * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. >> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> + * >> + * This code is free software; you can redistribute it and/or modify it >> + * under the terms of the GNU General Public License version 2 only, as >> + * published by the Free Software Foundation. >> + * >> + * This code is distributed in the hope that it will be useful, but WITHOUT >> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or >> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License >> + * version 2 for more details (a copy is included in the LICENSE file that >> + * accompanied this code). >> + * >> + * You should have received a copy of the GNU General Public License version >> + * 2 along with this work; if not, write to the Free Software Foundation, >> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. >> + * >> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA >> + * or visit www.oracle.com if you need additional information or have any >> + * questions. >> + */ >> + >> +/* >> + * @test >> + * @bug JDK-6695402 >> + * @summary verify signatures of jars containing classes with names with multi- >> + * byte unicode characters broken across lines >> + * @library /test/lib >> + * @modules jdk.jartool/sun.tools.jar >> + * jdk.jartool/sun.security.tools.jarsigner >> + * @run main MultibyteUnicodeName >> + */ >> + >> +import java.io.IOException; >> +import java.io.InputStream; >> +import java.nio.file.Files; >> +import java.nio.file.Paths; >> +import java.util.jar.JarFile; >> +import java.util.jar.Manifest; >> +import java.util.Arrays; >> +import java.util.Map; >> +import java.util.jar.Attributes.Name; >> +import java.util.jar.JarEntry; >> + >> +import static java.nio.charset.StandardCharsets.UTF_8; >> + >> +import static jdk.test.lib.SecurityTools.jarsigner; >> +import static jdk.test.lib.SecurityTools.keytool; >> +import static jdk.test.lib.util.JarUtils.createJar; >> +import static jdk.test.lib.util.JarUtils.updateJar; >> + >> +public class MultibyteUnicodeName { >> + >> + /** >> + * this class's name will break across lines in the jar manifest at the middle of >> + * a two-byte utf encoded character due to its e acute letter at its exact position. >> + * >> + * @see also eAcute in {@link MultibyteUnicodeName#verifyClassNameLineBroken(String)} >> + */ >> + static class A1234567890B1234567890C1234567890D12345678?xyz { } >> + static class A1234567890B1234567890C1234567890D12345678exyz { } >> + >> + static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + >> + "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; >> + static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + >> + "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; >> + >> + static final String alias = "a"; >> + static final String keystoreFileName = "test.jks"; >> + static final String ManifestFileName = "MANIFEST.MF"; >> + >> + public static void main(String[] args) throws Exception { >> + try { >> + prepare(); >> + >> + testSignJar("test.jar"); >> + testSignJarNoManifest("test-no-manifest.jar"); >> + testSignJarUpdate("test-update.jar", "test-updated.jar"); >> + >> + } finally { >> + Files.deleteIfExists(Paths.get(keystoreFileName)); >> + Files.deleteIfExists(Paths.get(ManifestFileName)); >> + Files.deleteIfExists(Paths.get(refClassFilename)); >> + Files.deleteIfExists(Paths.get(testClassFilename)); >> + } >> + } >> + >> + static void prepare() throws Exception { >> + keytool("-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", >> + "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", >> + "-validity", "366").shouldHaveExitValue(0); >> + >> + Files.write(Paths.get(ManifestFileName), >> + (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); >> + Files.copy(MultibyteUnicodeName.class.getResourceAsStream(refClassFilename), >> + Paths.get(refClassFilename)); >> + Files.copy(MultibyteUnicodeName.class.getResourceAsStream(testClassFilename), >> + Paths.get(testClassFilename)); >> + } >> + >> + static void testSignJar(String jarFileName) throws Exception { >> + try { >> + createJar(jarFileName, ManifestFileName, refClassFilename, testClassFilename); >> + verifyJarSignature(jarFileName); >> + >> + } finally { >> + Files.deleteIfExists(Paths.get(jarFileName)); >> + } >> + } >> + >> + static void testSignJarNoManifest(String jarFileName) throws Exception { >> + try { >> + createJar(jarFileName, refClassFilename, testClassFilename); >> + verifyJarSignature(jarFileName); >> + >> + } finally { >> + Files.deleteIfExists(Paths.get(jarFileName)); >> + } >> + } >> + >> + static void testSignJarUpdate(String initialFileName, String updatedFileName) throws Exception { >> + try { >> + createJar(initialFileName, ManifestFileName, refClassFilename); >> + jarsigner("-keystore", keystoreFileName, "-storetype", "JKS", >> + "-storepass", "changeit", "-debug", initialFileName, alias) >> + .shouldHaveExitValue(0); >> + updateJar(initialFileName, updatedFileName, testClassFilename); >> + verifyJarSignature(updatedFileName); >> + >> + } finally { >> + Files.deleteIfExists(Paths.get(initialFileName)); >> + Files.deleteIfExists(Paths.get(updatedFileName)); >> + } >> + } >> + >> + static void verifyJarSignature(String jarFileName) throws Exception { >> + // actually sign the jar >> + jarsigner("-keystore", keystoreFileName, "-storetype", "JKS", >> + "-storepass", "changeit", "-debug", jarFileName, alias) >> + .shouldHaveExitValue(0); >> + >> + try ( >> + JarFile jar = new JarFile(jarFileName); >> + ) { >> + verifyClassNameLineBroken(jar, testClassFilename); >> + verifyCodeSigners(jar, jar.getJarEntry(testClassFilename)); >> + verifyCodeSigners(jar, jar.getJarEntry(refClassFilename)); >> + } >> + } >> + >> + /** >> + * it would be too easy to miss the actual test case by just renaming an identifier so that >> + * the multi-byte encoded character would not any longer be broken across a line break. >> + * >> + * this test verifies that the actual test case is tested based on the manifest >> + * and not based on the signature file because at the moment, the signature file >> + * does not even contain the desired entry at all. >> + * >> + * this relies on the Manifest breaking lines unaware of bytes that belong to the same >> + * multi-ybte utf characters. >> + */ >> + static void verifyClassNameLineBroken(JarFile jar, String className) throws IOException { >> + byte[] eAcute = "?".getBytes(UTF_8); >> + byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; >> + >> + if (jar.getManifest().getAttributes(className) == null) { >> + throw new AssertionError(className + " not found in manifest"); >> + } >> + >> + JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); >> + try ( >> + InputStream manifestIs = jar.getInputStream(manifestEntry); >> + ) { >> + int bytesMatched = 0; >> + for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { >> + if ((byte) b == eAcuteBroken[bytesMatched]) { >> + bytesMatched++; >> + if (bytesMatched == eAcuteBroken.length) { >> + break; >> + } >> + } else { >> + bytesMatched = 0; >> + } >> + } >> + if (bytesMatched < eAcuteBroken.length) { >> + throw new AssertionError("self-test failed: " >> + + "multi-byte utf-8 character not broken across lines"); >> + } >> + } >> + } >> + >> + static void verifyCodeSigners(JarFile jar, JarEntry jarEntry) throws IOException { >> + // codeSigners is initialized only after the entry has been read >> + try ( >> + InputStream inputStream = jar.getInputStream(jarEntry); >> + ) { >> + while (inputStream.read() > -1); >> + >> + if (jarEntry.getCodeSigners() == null || jarEntry.getCodeSigners().length == 0) { >> + throw new AssertionError("no signing certificate found for " + jarEntry.getName()); >> + } >> + } >> + >> + // a check for the presence of code signers is sufficient to check bug JDK-6695402. >> + // no need to also verify the actual code signers attributes here. >> + } >> + >> +} >> >> Regards, >> Philipp >> >> >> >>> On 20.09.2017 03:41, Weijun Wang wrote: >>> Hi Philipp >>> >>> The change mostly looks fine. You might want to put everything into a patch file so Vincent can recreate a webrev and post it to cr.openjdk.java.net. >>> >>> One thing I would suggest for the test is that instead of using jarsigner -verify and check the text in output you can open it as a JarFile, consume the content of an entry, and look into its getCertificates(). >>> >>> Also, you might want to reuse test/lib/jdk/test/lib/SecurityTools.java, and there is also JarUtils.java you can use. Maybe Files.copy(InputStream,Path) can be used in copyClassToFile(). >>> >>> Thanks >>> Max >>> >>> >>>> On Sep 20, 2017, at 7:41 AM, Philipp Kunz >>>> wrote: >>>> >>>> Hello Vincent >>>> >>>> Here may be the fix for JDK-6695402. First a test. >>>> >>>> BEGIN /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >>>> /* >>>> * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. >>>> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>> * >>>> * This code is free software; you can redistribute it and/or modify it >>>> * under the terms of the GNU General Public License version 2 only, as >>>> * published by the Free Software Foundation. >>>> * >>>> * This code is distributed in the hope that it will be useful, but WITHOUT >>>> * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or >>>> * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License >>>> * version 2 for more details (a copy is included in the LICENSE file that >>>> * accompanied this code). >>>> * >>>> * You should have received a copy of the GNU General Public License version >>>> * 2 along with this work; if not, write to the Free Software Foundation, >>>> * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. >>>> * >>>> * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA >>>> * or visit >>>> www.oracle.com >>>> if you need additional information or have any >>>> * questions. >>>> */ >>>> >>>> /* >>>> * @test >>>> * @bug JDK-6695402 >>>> * @summary verify signatures of jars containing classes with names with multi- >>>> * byte unicode characters broken across lines >>>> * @library /test/lib >>>> * @modules jdk.jartool/sun.tools.jar >>>> * jdk.jartool/sun.security.tools.jarsigner >>>> * @build jdk.test.lib.JDKToolLauncher >>>> * jdk.test.lib.process.* >>>> * @run main MultibyteUnicodeName >>>> */ >>>> >>>> import java.io.ByteArrayOutputStream; >>>> import java.io.IOException; >>>> import java.io.InputStream; >>>> import java.io.UnsupportedEncodingException; >>>> import java.nio.file.Files; >>>> import java.nio.file.Paths; >>>> import java.util.jar.JarFile; >>>> import java.util.jar.Manifest; >>>> import java.util.stream.Collectors; >>>> import java.util.Arrays; >>>> import java.util.Map; >>>> import java.util.jar.Attributes.Name; >>>> import java.util.jar.JarEntry; >>>> >>>> import static java.nio.charset.StandardCharsets.UTF_8; >>>> >>>> import sun.security.tools.jarsigner.Resources; >>>> >>>> import jdk.test.lib.JDKToolLauncher; >>>> import jdk.test.lib.process.OutputAnalyzer; >>>> import jdk.test.lib.process.ProcessTools; >>>> >>>> public class MultibyteUnicodeName { >>>> >>>> public static void main(String[] args) throws Throwable { >>>> try { >>>> prepare(); >>>> >>>> testSignJar("test.jar"); >>>> testSignJarNoManifest("test-no-manifest.jar"); >>>> testSignJarUpdate("test-update.jar"); >>>> testSignJarWithIndex("test-index.jar"); >>>> testSignJarAddIndex("test-add-index.jar"); >>>> >>>> } finally { >>>> Files.deleteIfExists(Paths.get(keystoreFileName)); >>>> Files.deleteIfExists(Paths.get(ManifestFileName)); >>>> Files.deleteIfExists(Paths.get(refClassFilename)); >>>> Files.deleteIfExists(Paths.get(testClassFilename)); >>>> } >>>> } >>>> >>>> static final String alias = "a"; >>>> static final String keystoreFileName = "test.jks"; >>>> static final String ManifestFileName = "MANIFEST.MF"; >>>> >>>> static class A1234567890B1234567890C1234567890D12345678exyz { } >>>> static class A1234567890B1234567890C1234567890D12345678?xyz { } >>>> >>>> static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + >>>> "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; >>>> static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + >>>> "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; >>>> >>>> static void prepare() throws Throwable { >>>> tool("keytool", "-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", >>>> "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", >>>> "-validity", "366") >>>> .shouldHaveExitValue(0); >>>> >>>> Files.write(Paths.get(ManifestFileName), >>>> (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); >>>> copyClassToFile(refClassFilename); >>>> copyClassToFile(testClassFilename); >>>> } >>>> >>>> static void copyClassToFile(String classFilename) throws IOException { >>>> try ( >>>> InputStream asStream = MultibyteUnicodeName.class.getResourceAsStream(classFilename); >>>> ByteArrayOutputStream buf = new ByteArrayOutputStream(); >>>> ) { >>>> int b; >>>> while ((b = asStream.read()) != -1) buf.write(b); >>>> Files.write(Paths.get(classFilename), buf.toByteArray()); >>>> } >>>> } >>>> >>>> static void testSignJar(String jarFileName) throws Throwable { >>>> try { >>>> tool("jar", "cvfm", jarFileName, >>>> ManifestFileName, refClassFilename, testClassFilename) >>>> .shouldHaveExitValue(0); >>>> verifyJarSignature(jarFileName); >>>> >>>> } finally { >>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>> } >>>> } >>>> >>>> static void testSignJarNoManifest(String jarFileName) throws Throwable { >>>> try { >>>> tool("jar", "cvf", jarFileName, refClassFilename, testClassFilename) >>>> .shouldHaveExitValue(0); >>>> verifyJarSignature(jarFileName); >>>> >>>> } finally { >>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>> } >>>> } >>>> >>>> static void testSignJarUpdate(String jarFileName) throws Throwable { >>>> try { >>>> tool("jar", "cvfm", jarFileName, ManifestFileName, refClassFilename) >>>> .shouldHaveExitValue(0); >>>> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>> .shouldHaveExitValue(0); >>>> tool("jar", "uvf", jarFileName, testClassFilename) >>>> .shouldHaveExitValue(0); >>>> verifyJarSignature(jarFileName); >>>> >>>> } finally { >>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>> } >>>> } >>>> >>>> static void testSignJarWithIndex(String jarFileName) throws Throwable { >>>> try { >>>> tool("jar", "cvfm", jarFileName, >>>> ManifestFileName, refClassFilename, testClassFilename) >>>> .shouldHaveExitValue(0); >>>> tool("jar", "iv", jarFileName).shouldHaveExitValue(0); >>>> verifyJarSignature(jarFileName); >>>> >>>> } finally { >>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>> } >>>> } >>>> >>>> static void testSignJarAddIndex(String jarFileName) throws Throwable { >>>> try { >>>> tool("jar", "cvfm", jarFileName, >>>> ManifestFileName, refClassFilename, testClassFilename) >>>> .shouldHaveExitValue(0); >>>> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>> .shouldHaveExitValue(0); >>>> tool("jar", "iv", jarFileName).shouldHaveExitValue(0); >>>> verifyJarSignature(jarFileName); >>>> >>>> } finally { >>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>> } >>>> } >>>> >>>> static Map jarsignerResources = Arrays.stream(new Resources().getContents()). >>>> collect(Collectors.toMap(e -> (String) e[0], e -> (String) e[1])); >>>> >>>> static void verifyJarSignature(String jarFileName) throws Throwable { >>>> // actually sign the jar >>>> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>> .shouldHaveExitValue(0); >>>> >>>> verifyClassNameLineBroken(jarFileName); >>>> >>>> // check for no unsigned entries >>>> tool("jarsigner", "-verify", "-keystore", keystoreFileName, "-storetype", "JKS", >>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>> .shouldHaveExitValue(0) >>>> .shouldNotContain(jarsignerResources.get( >>>> "This.jar.contains.unsigned.entries.which.have.not.been.integrity.checked.")); >>>> >>>> // check that both classes with and without multi-byte unicode characters in their names are signed >>>> tool("jarsigner", "-verify", "-verbose", "-keystore", keystoreFileName, "-storetype", "JKS", >>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>> .shouldHaveExitValue(0) >>>> .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + refClassFilename.replaceAll("\\$", "\\\\\\$") + "$") >>>> .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + testClassFilename.replaceAll("\\$", "\\\\\\$") + "$"); >>>> >>>> // check that both classes with and without multi-byte unicode characters in their names have signing certificates >>>> tool("jarsigner", "-verify", "-verbose", "-certs", "-keystore", keystoreFileName, >>>> "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) >>>> .shouldHaveExitValue(0) >>>> .shouldMatch(refClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)") >>>> .shouldMatch(testClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)"); >>>> } >>>> >>>> /** >>>> * it would be too easy to miss the actual test case by just renaming an identifier so that >>>> * the multi-byte encoded character would not any longer be broken across a line break. >>>> * >>>> * this test verifies that the actual test case is tested based on the manifest >>>> * and not based on the signature file because at the moment, the signature file >>>> * does not even contain the desired entry at all. >>>> * >>>> * this relies on the Manifest breaking lines unaware of bytes that belong to the same >>>> * multi-ybte utf characters. >>>> */ >>>> static void verifyClassNameLineBroken(String jarFileName) throws IOException { >>>> byte[] eAcute = "?".getBytes(UTF_8); >>>> byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; >>>> >>>> try ( >>>> JarFile jar = new JarFile(jarFileName); >>>> ) { >>>> if (jar.getManifest().getAttributes(testClassFilename) == null) { >>>> throw new AssertionError(testClassFilename + " not found in manifest"); >>>> } >>>> >>>> JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); >>>> try ( >>>> InputStream manifestIs = jar.getInputStream(manifestEntry); >>>> ) { >>>> int bytesMatched = 0; >>>> for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { >>>> if ((byte) b == eAcuteBroken[bytesMatched]) { >>>> bytesMatched++; >>>> if (bytesMatched == eAcuteBroken.length) { >>>> break; >>>> } >>>> } else { >>>> bytesMatched = 0; >>>> } >>>> } >>>> if (bytesMatched < eAcuteBroken.length) { >>>> throw new AssertionError("self-test failed: " >>>> + "multi-byte utf-8 character not broken across lines"); >>>> } >>>> } >>>> } >>>> } >>>> >>>> static OutputAnalyzer tool(String tool, String... args) >>>> throws Throwable { >>>> JDKToolLauncher l = JDKToolLauncher.createUsingTestJDK(tool); >>>> for (String arg : args) { >>>> if (arg .startsWith("-J")) { >>>> l.addVMArg(arg.substring(2)); >>>> } else { >>>> l.addToolArg(arg); >>>> } >>>> } >>>> return ProcessTools.executeCommand(l.getCommand()); >>>> } >>>> >>>> } >>>> END /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >>>> >>>> >>>> There are three e with acutes, on lines 84, 89, and 227, just in case the mail suffers bad encoding which might not be absolutely unconceivable regarding the bug's subject. >>>> >>>> In contrast to the bug description it uses a nested class with a two-byte UTF-8 character rather than one in its own file. I chose to do it that way because then the complete test goes into one file and therefore I assume the overview is kept easier. The test still demonstrates exactly JDK-6695402's problem. >>>> >>>> It may not have been necessary to also test jar files with indexes but i consider it necessary to have signing unsigned as well as partially signed jars tested, so why not have one more case tested, too? >>>> >>>> There might also be a more elegant approach to get class files into a jar file as to get their contents through the class loader. I chose to use real class files rather than dummy contents in files named *.class or for instance plain text files in the jar the signing of which to be tested in order to stay as close to the original bug problem as possible even though I don't have any notion that it would make a difference. The amount of code used to copy the class file contents around is comparatively small with respect to the whole test case amount of code. >>>> >>>> For the demonstration that the multi-byte character actually makes the alleged difference, I concluded that it was necessary to have another class name not previously affected by the bug in order to compare the effects of just different names. Otherwise the signing could fail to any other reason undetectably. >>>> >>>> In order to express a condition to tell successful from failed test runs the best approach I found was to analyze the jarsigner tool's output which is in my opinion not the most desirable option. I'd have preferred output from a clearer structured api but then I guess the output format will not change too often in the foreseeable future. For instance the check for the s, m, and k flags is more pragmatical approach than obviously perfectly failsafe when operating with regular expressions to match it with the output. Other parts such as 'X.509' and 'CN=' are not even jarsigner tool resources. I put at least a reference to sun.security.tools.jarsigner.Resources. >>>> >>>> Finally, I afforded kind of a self-test that protects the test from undetected failing because renaming the main class which would move the two-byte unicode character to a position not suitable for the test. It may not be absolutely necessary. >>>> >>>> >>>> >>>> BEGIN PATH >>>> diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java >>>> --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 >>>> +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Wed Sep 20 00:56:15 2017 +0200 >>>> @@ -28,6 +28,7 @@ >>>> import java.security.*; >>>> import java.util.HashMap; >>>> import java.io.ByteArrayOutputStream; >>>> +import static java.nio.charset.StandardCharsets.UTF_8; >>>> >>>> /** >>>> * This class is used to compute digests on sections of the Manifest. >>>> @@ -112,7 +113,7 @@ >>>> rawBytes = bytes; >>>> entries = new HashMap<>(); >>>> >>>> - ByteArrayOutputStream baos = new ByteArrayOutputStream(); >>>> + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); >>>> >>>> Position pos = new Position(); >>>> >>>> @@ -131,50 +132,40 @@ >>>> >>>> if (len > 6) { >>>> if (isNameAttr(bytes, start)) { >>>> - StringBuilder nameBuf = new StringBuilder(sectionLen); >>>> + nameBuf.reset(); >>>> + nameBuf.write(bytes, start+6, len-6); >>>> >>>> - try { >>>> - nameBuf.append( >>>> - new String(bytes, start+6, len-6, "UTF8")); >>>> + int i = start + len; >>>> + if ((i-start) < sectionLen) { >>>> + if (bytes[i] == '\r') { >>>> + i += 2; >>>> + } else { >>>> + i += 1; >>>> + } >>>> + } >>>> >>>> - int i = start + len; >>>> - if ((i-start) < sectionLen) { >>>> - if (bytes[i] == '\r') { >>>> - i += 2; >>>> - } else { >>>> - i += 1; >>>> - } >>>> + while ((i-start) < sectionLen) { >>>> + if (bytes[i++] == ' ') { >>>> + // name is wrapped >>>> + int wrapStart = i; >>>> + while (((i-start) < sectionLen) >>>> + && (bytes[i++] != '\n')); >>>> + if (bytes[i-1] != '\n') >>>> + return; // XXX: exception? >>>> + int wrapLen; >>>> + if (bytes[i-2] == '\r') >>>> + wrapLen = i-wrapStart-2; >>>> + else >>>> + wrapLen = i-wrapStart-1; >>>> + >>>> + nameBuf.write(bytes, wrapStart, wrapLen); >>>> + } else { >>>> + break; >>>> } >>>> + } >>>> >>>> - while ((i-start) < sectionLen) { >>>> - if (bytes[i++] == ' ') { >>>> - // name is wrapped >>>> - int wrapStart = i; >>>> - while (((i-start) < sectionLen) >>>> - && (bytes[i++] != '\n')); >>>> - if (bytes[i-1] != '\n') >>>> - return; // XXX: exception? >>>> - int wrapLen; >>>> - if (bytes[i-2] == '\r') >>>> - wrapLen = i-wrapStart-2; >>>> - else >>>> - wrapLen = i-wrapStart-1; >>>> - >>>> - nameBuf.append(new String(bytes, wrapStart, >>>> - wrapLen, "UTF8")); >>>> - } else { >>>> - break; >>>> - } >>>> - } >>>> - >>>> - entries.put(nameBuf.toString(), >>>> - new Entry(start, sectionLen, sectionLenWithBlank, >>>> - rawBytes)); >>>> - >>>> - } catch (java.io.UnsupportedEncodingException uee) { >>>> - throw new IllegalStateException( >>>> - "UTF8 not available on platform"); >>>> - } >>>> + entries.put(new String(nameBuf.toByteArray(), UTF_8), >>>> + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); >>>> } >>>> } >>>> start = pos.startOfNext; >>>> END PATCH >>>> >>>> >>>> The patch is mostly so big because indentation has changed in many lines. >>>> >>>> There was baos there before without apparent use. The patch renames it and puts it into use. It came in very handy because if it would not have been there already, I would have had to defined one of my own. >>>> >>>> The main point of the patch is that manifest section names that are broken across lines at possibly arbitrary bytes are no longer converted into strings for each manifest line and then joined. Parts of names broken across lines are now joined first when they are still byte sequences and only when the complete byte sequence is available after processing all manifest lines that contain the same manifest section name decoded into a unicode string. >>>> >>>> For decoding the bytes into a string I chose a different string constructor than was used before that does not any longer declare UnsupportedEncodingException rendering the try/catch redundant. It couldn't have ever occurred anyway taking into consideration that UTF-8 is mandatory for every Java platform, StandardCharsets says. The difference according to the documentation is that the previously used String constructor returned undefined strings for invalid byte sequences whereas the one used in the patch will replace unparseable portions with valid 'unknown' characters. >>>> >>>> One question I cannot still answer yet is how the ManifestDigester can be changed at all without complete test coverage or risking to break existing signatures. I already started on that but it's quite a piece of work to almost formally prove that all manifests will continue to produce identical digests, except of course for the ones now fixed. >>>> >>>> Regards, >>>> Philipp >>>> >>>> >>>>> On 17.09.2017 21:25, Philipp Kunz wrote: >>>>> >>>>> Hello Vincent >>>>> >>>>> I narrowed the error down so far and suspect now that it is an effect of sun.security.util.ManifestDigester's constructor, lines 134 and 163-164, in combination with java.util.jar.Manifest.make72Safe. Manifest breaks multi-byte characters in manifest section names across lines and ManifestDigester fails to restore them correctly, which both sounds wrong but ManifestDigester sure is. >>>>> >>>>> Just fixing it would be too tempting but then I would not know (I mean not know for sure with evidence from tests) if any existing jar-signature would still be valid and I don't want to break existing signatures. When I had a look at the tests specific for Manifest and ManifestDigester I found very little. There are many more different situations and corner cases and combinations thereof to cover with tests than it looks like at first glance so that writing tests that cover all relevant cases looks to me like quite an effort. I have the impression that test coverage was not a priority when the subjected code was developed or at least the tests might not have been contributed to the open JDK project. Hence, while I'm continuing to complete these tests I miss, if someone has an idea how to simplify that so that I can still convince at least myself that no existing signature will break or where possibly more testcases are that I haven't discovered so far, please let me know. >>>>> >>>>> I'm also wondering how much performance should be taken into consideration. There are a few hints such as reusing byte arrays that suggest that it is an issue. Will a patch be rejected if it slows down some tests beyond a certain limit for so little added value or what might be the acceptance criteria here? I don't expect insuperable difficulties with performance but would still appreciate some general idea. >>>>> >>>>> My desired or currently preferred approach to fix JDK-6695402 would be to let ManifestDigester any kind of use or extend Manifest in order to let Manifest identify the manifest sections thereby preventing the parsing duplicated that identifies the manifest sections. >>>>> >>>>> As a new contributor I could use and would appreciate some guidance particularly about what kind and completeness of test coverage and performance tuning applies or is suggested in the context of the given bug. Otherwise I fear I might contribute too many tests which would have to be reviewed and maintained or quite some work would be for nothing if a patch would not satisfy performance requirements. >>>>> >>>>> Regards, >>>>> Philipp >>>>> >>>>> >>>>> >>>>>> On 01.09.2017 15:20, Vincent Ryan wrote: >>>>>> >>>>>> That all sounds fine. Let me know when your patch is ready to submit. >>>>>> Thanks. >>>>>> >>>>>> >>>>>> >>>>>>> On 1 Sep 2017, at 13:15, Philipp Kunz >>>>>>> wrote: >>>>>>> >>>>>>> Hello Vincent >>>>>>> >>>>>>> Thank you for sponsoring! >>>>>>> So far, I have become a contributor by signing the OCA which has been accepted. Dalibor Topic wrote that he can confirm and it's also here: >>>>>>> http://www.oracle.com/technetwork/community/oca-486395.html#p >>>>>>> -> Paratix GmbH >>>>>>> Therefore I think I have followed the steps in >>>>>>> http://openjdk.java.net/contribute/ >>>>>>> at least the ones before actual patch submission. >>>>>>> >>>>>>> Currently, I'm working on getting the existing test cases running locally. Unfortunately, I started with jdk 9 and switched to 10 now. I figured these commands run at least the relevant tests with 9 and hope this also applies to 10: >>>>>>> make run-test-tier1 >>>>>>> make run-test TEST="jdk/test" >>>>>>> they report errors and failures but it hasn't been completed and released which might explain it. If I run the tests I assume relevant for my patch >>>>>>> make run-test TEST="jtreg:jdk/test/sun/security/tools/jarsigner jtreg:jdk/test/java/util/jar" >>>>>>> then it reports zero errors and failures which may be a good starting point. >>>>>>> Do you think that sounds reasonable or do you have another suggestion how to run the tests? >>>>>>> >>>>>>> Next, I will add a test for JDK-6695402 before actually fixing it. As an example, I'll try something in the style of >>>>>>> http://hg.openjdk.java.net/jdk9/dev/jdk/file/65464a307408/test/java/util/jar/Manifest/CreateManifest.java >>>>>>> . This way, I try to demonstrate the improvement. >>>>>>> >>>>>>> I guess I have identified the following line as the cause: value = new String(vb, 0, 0, vb.length); >>>>>>> >>>>>>> http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/51f5d60713b5/src/java.base/share/classes/java/util/jar/Manifest.java#l157 >>>>>>> >>>>>>> So I'll try to remove it first including the whole four line if block. >>>>>>> >>>>>>> Philipp >>>>>>> >>>>>>> >>>>>>>> On 01.09.2017 10:00, Vincent Ryan wrote: >>>>>>>> >>>>>>>> Hello Philipp, >>>>>>>> >>>>>>>> I?m happy to sponsor your fix for JDK 10. Have you followed these steps: >>>>>>>> http://openjdk.java.net/contribute/ >>>>>>>> ? >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On 1 Sep 2017, at 08:58, Vincent Ryan >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Moved to security-dev >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 1 Sep 2017, at 08:28, Philipp Kunz >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Hello everyone >>>>>>>>>> >>>>>>>>>> I have been developing with Java for around 17 years now and when I encountered some bug I decided to attempt to fix it: >>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-6695402 >>>>>>>>>> . This also looks like it may not be too big a piece for a first contribution. >>>>>>>>>> >>>>>>>>>> I read through quite some guides and all kinds of documents but could not yet help myself with the following questions: >>>>>>>>>> >>>>>>>>>> May I login to jira to add comments to bugs? If so, how would I request or receive credentials? Or are mailing lists preferred? >>>>>>>>>> >>>>>>>>>> Another question is whether I should apply it to jdk9, but it may be too late now, or to jdk10, and backporting can be considered later. Probably it wouldn't even make much a difference for the patch itself. >>>>>>>>>> >>>>>>>>>> One more question I have is how or where to find the sources from before migration to mercurial. Because some lines of code I intend to change go back farther and in the history I find only 'initial commit'. With such a history I might be able better to understand why it's there and prevent to make the same mistake again. >>>>>>>>>> >>>>>>>>>> I guess the appropriate mailing list for above mentioned bug is security-dev. Is it correct that I can send a patch there and just hope for some sponsor to pick it up? Of course I'd be glad if some sponsor would contact me and maybe provide some assistance or if someone would confirm that sending a patch to the mailing list is the right way to find a sponsor. >>>>>>>>>> >>>>>>>>>> Philipp Kunz >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Paratix GmbH >>>>> St Peterhofstatt 11 >>>>> 8001 Z?rich >>>>> >>>>> +41 (0)76 397 79 35 >>>>> >>>>> philipp.kunz at paratix.ch >> >> -- >> >> >> Gruss Philipp >> >> >> >> >> >> >> Paratix GmbH >> St Peterhofstatt 11 >> 8001 Z?rich >> >> +41 (0)76 397 79 35 >> philipp.kunz at paratix.ch > From aph at redhat.com Mon Sep 25 14:50:01 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 25 Sep 2017 15:50:01 +0100 Subject: Do we need an unsigned multiplyHigh? Message-ID: We now have a multiplyHigh intrinsic, but it is signed. Unsigned multiplyHigh is in general a more useful primitive for crypto than signed, and I wonder if we need an intrinsic for that as well. I've looked at cooking up an unsigned multiplyHigh in Java, and I think the fastest way is this: private static final long unsignedMultiplyHigh(long a, long b) { long result = Math.multiplyHigh(a, b); if (a < 0) result += b; if (b < 0) result += a; // Can also be written as: // result += (a >> 63) & b; // result += (b >> 63) & a; return result; } It's still about 50% slower than the signed multiplyHigh, though. Thoughts? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From philipp.kunz at paratix.ch Mon Sep 25 17:11:21 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Mon, 25 Sep 2017 19:11:21 +0200 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> Message-ID: <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> Hi Max Thank you for the detailed assistance and I really hope it doesn't annoy you too much with so many beginner's mistakes. Every little point of yours seems to be absolutely justified in my point of view. I did not understand where I could apply the readAllBytes. In case it still applies, would you give me another hint? I'd rather expect some potential for butifying verifyClassNameLineBroken but then I haven't found any really. About where to place the declaration of nameBuf I was a little confused probably/obviously when there was a ByteArrayOutputStream before which, however, actually was not used into assuming that reusing the same buffer for all sections would result in better performance or so but that's certainly not a valid assumption now as I consider it again. About the import static you convinced me but for statically importing java.nio.charset.StandardCharsets.UTF_8 which I prefer this way still. After you wrote I should decide myself, I removed other static imports. If there was kind of an unwritten rule not to use static imports generally that would be a compelling reason but I didn't bother to search for import static through the jdk sources. The @modules was a left-over from previous versions. I'm not sure where you meant I should put the test to exactly. In http://hg.openjdk.java.net/jdk10/jdk10/jdk/file/777356696811/test/jdk I cant find the parent folder sun you mentioned. In case there is some reorganization in progress not having arrived yet at the tip, could the new test be moved together with it? Or did you mean I should create the sun folder but then the existing tests would not be next to the new one? Did I look in the wrong place? When updating the copyright years I was not sure if I should let the existing one, 2011, there in ManifestDigester or replace it with the current year which I did. When looking into other class files I saw none with more than two years but "," looks a little odd to me for indicating a range. I assumed that I remove the JDK- prefix from the @bug tag and not the @test tag unless @bug is considered part of the @test. Is it correct, that a maximum line width of 80 characters is the convention? In case I added some more breaks for that. I just wonder what other style guides I should have respected. Just in case you think it would be more efficient I'd not object that a reviewer or some else would just change these little things to get over with it. On the other hand this way I learn it. If there is a suitable documentation that much detailed, I'd be glad if you could point me at it. That might save a few cycles. Regards, Philipp --------------------------- diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Mon Sep 25 19:09:07 2017 +0200 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1997, 2011, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -28,6 +28,7 @@ import java.security.*; import java.util.HashMap; import java.io.ByteArrayOutputStream; +import static java.nio.charset.StandardCharsets.UTF_8; /** * This class is used to compute digests on sections of the Manifest. @@ -112,8 +113,6 @@ rawBytes = bytes; entries = new HashMap<>(); - ByteArrayOutputStream baos = new ByteArrayOutputStream(); - Position pos = new Position(); if (!findSection(0, pos)) @@ -131,50 +130,42 @@ if (len > 6) { if (isNameAttr(bytes, start)) { - StringBuilder nameBuf = new StringBuilder(sectionLen); + ByteArrayOutputStream nameBuf = + new ByteArrayOutputStream(); + nameBuf.write(bytes, start+6, len-6); - try { - nameBuf.append( - new String(bytes, start+6, len-6, "UTF8")); + int i = start + len; + if ((i-start) < sectionLen) { + if (bytes[i] == '\r') { + i += 2; + } else { + i += 1; + } + } - int i = start + len; - if ((i-start) < sectionLen) { - if (bytes[i] == '\r') { - i += 2; - } else { - i += 1; - } + while ((i-start) < sectionLen) { + if (bytes[i++] == ' ') { + // name is wrapped + int wrapStart = i; + while (((i-start) < sectionLen) + && (bytes[i++] != '\n')); + if (bytes[i-1] != '\n') + return; // XXX: exception? + int wrapLen; + if (bytes[i-2] == '\r') + wrapLen = i-wrapStart-2; + else + wrapLen = i-wrapStart-1; + + nameBuf.write(bytes, wrapStart, wrapLen); + } else { + break; } + } - while ((i-start) < sectionLen) { - if (bytes[i++] == ' ') { - // name is wrapped - int wrapStart = i; - while (((i-start) < sectionLen) - && (bytes[i++] != '\n')); - if (bytes[i-1] != '\n') - return; // XXX: exception? - int wrapLen; - if (bytes[i-2] == '\r') - wrapLen = i-wrapStart-2; - else - wrapLen = i-wrapStart-1; - - nameBuf.append(new String(bytes, wrapStart, - wrapLen, "UTF8")); - } else { - break; - } - } - - entries.put(nameBuf.toString(), - new Entry(start, sectionLen, sectionLenWithBlank, - rawBytes)); - - } catch (java.io.UnsupportedEncodingException uee) { - throw new IllegalStateException( - "UTF8 not available on platform"); - } + entries.put(new String(nameBuf.toByteArray(), UTF_8), + new Entry(start, sectionLen, sectionLenWithBlank, + rawBytes)); } } start = pos.startOfNext; diff -r ddc25f646c2e test/sun/security/tools/jarsigner/MultibyteUnicodeName.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java Mon Sep 25 19:09:07 2017 +0200 @@ -0,0 +1,196 @@ +/* + * Copyright (c) 2017, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +/* + * @test + * @bug 6695402 + * @summary verify signatures of jars containing classes with names + * with multi-byte unicode characters broken across lines + * @library /test/lib + */ + +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.jar.JarFile; +import java.util.jar.Attributes.Name; +import java.util.jar.JarEntry; + +import static java.nio.charset.StandardCharsets.UTF_8; + +import jdk.test.lib.SecurityTools; +import jdk.test.lib.util.JarUtils; + +public class MultibyteUnicodeName { + + /** + * this class's name will break across lines in the jar manifest at the + * middle of a two-byte utf encoded character due to its e acute letter + * at its exact position. + * + * @see also eAcute in + * {@link MultibyteUnicodeName#verifyClassNameLineBroken(String)} + */ + static class A1234567890B1234567890C1234567890D12345678?xyz { } + static class A1234567890B1234567890C1234567890D12345678exyz { } + + static final String refClassFilename = MultibyteUnicodeName.class + .getSimpleName() + "$" + + A1234567890B1234567890C1234567890D12345678exyz.class + .getSimpleName() + ".class"; + static final String testClassFilename = MultibyteUnicodeName.class + .getSimpleName() + "$" + + A1234567890B1234567890C1234567890D12345678?xyz.class + .getSimpleName() + ".class"; + + static final String alias = "a"; + static final String keystoreFileName = "test.jks"; + static final String ManifestFileName = "MANIFEST.MF"; + + public static void main(String[] args) throws Exception { + prepare(); + + testSignJar("test.jar"); + testSignJarNoManifest("test-no-manifest.jar"); + testSignJarUpdate("test-update.jar", "test-updated.jar"); + } + + static void prepare() throws Exception { + SecurityTools.keytool("-keystore", keystoreFileName, "-genkeypair", + "-storepass", "changeit", "-keypass", "changeit", "-storetype", + "JKS", "-alias", alias, "-dname", "CN=X", "-validity", "366") + .shouldHaveExitValue(0); + + Files.write(Paths.get(ManifestFileName), (Name. + MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8)); + Files.copy(MultibyteUnicodeName.class.getResourceAsStream( + refClassFilename), Paths.get(refClassFilename)); + Files.copy(MultibyteUnicodeName.class.getResourceAsStream( + testClassFilename), Paths.get(testClassFilename)); + } + + static void testSignJar(String jarFileName) throws Exception { + JarUtils.createJar(jarFileName, ManifestFileName, refClassFilename, + testClassFilename); + verifyJarSignature(jarFileName); + } + + static void testSignJarNoManifest(String jarFileName) throws Exception { + JarUtils.createJar(jarFileName, refClassFilename, testClassFilename); + verifyJarSignature(jarFileName); + } + + static void testSignJarUpdate( + String initialFileName, String updatedFileName + ) throws Exception { + JarUtils.createJar(initialFileName, ManifestFileName, + refClassFilename); + SecurityTools.jarsigner("-keystore", keystoreFileName, "-storetype", + "JKS", "-storepass", "changeit", "-debug", initialFileName, + alias) + .shouldHaveExitValue(0); + JarUtils.updateJar(initialFileName, updatedFileName, + testClassFilename); + verifyJarSignature(updatedFileName); + } + + static void verifyJarSignature(String jarFileName) throws Exception { + // actually sign the jar + SecurityTools.jarsigner("-keystore", keystoreFileName, "-storetype", + "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) + .shouldHaveExitValue(0); + + try ( + JarFile jar = new JarFile(jarFileName); + ) { + verifyClassNameLineBroken(jar, testClassFilename); + verifyCodeSigners(jar, jar.getJarEntry(testClassFilename)); + verifyCodeSigners(jar, jar.getJarEntry(refClassFilename)); + } + } + + /** + * it would be too easy to miss the actual test case by just renaming an + * identifier so that the multi-byte encoded character would not any longer + * be broken across a line break. + * + * this test verifies that the actual test case is tested based on the + * manifest and not based on the signature file because at the moment, the + * signature file does not even contain the desired entry at all. + * + * this relies on the Manifest breaking lines unaware of bytes that belong + * to the same multi-ybte utf characters. + */ + static void verifyClassNameLineBroken(JarFile jar, String className) + throws IOException { + byte[] eAcute = "?".getBytes(UTF_8); // U+00E9 + byte[] eAcuteBroken = + new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; + + if (jar.getManifest().getAttributes(className) == null) { + throw new AssertionError(className + " not found in manifest"); + } + + JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); + try ( + InputStream manifestIs = jar.getInputStream(manifestEntry); + ) { + int bytesMatched = 0; + for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { + if ((byte) b == eAcuteBroken[bytesMatched]) { + bytesMatched++; + if (bytesMatched == eAcuteBroken.length) { + break; + } + } else { + bytesMatched = 0; + } + } + if (bytesMatched < eAcuteBroken.length) { + throw new AssertionError("self-test failed: multi-byte " + + "utf-8 character not broken across lines"); + } + } + } + + static void verifyCodeSigners(JarFile jar, JarEntry jarEntry) + throws IOException { + // codeSigners is initialized only after the entry has been read + try ( + InputStream inputStream = jar.getInputStream(jarEntry); + ) { + while (inputStream.read() > -1); + } + + if (jarEntry.getCodeSigners() == null + || jarEntry.getCodeSigners().length == 0) { + throw new AssertionError( + "no signing certificate found for " + jarEntry.getName()); + } + // a check for the presence of code signers is sufficient to check + // bug JDK-6695402. no need to also verify the actual code signers + // attributes here. + } + +} --------------------------- On 24.09.2017 17:41, Wang Weijun wrote: > Some more suggestions on the test: > > 1. You can use readAllBytes() to ... err ... read all bytes. > > 2. I normally don?t remove intermediate files. Jtreg will do an automatic cleanup, and it can be configured to retain those files when the test fails. They will be very helpful in debugging. > > Thanks > Max > >> ? 2017?9?24??23:12?Weijun Wang ??? >> >> Great. >> >> Some suggestions, just my own habit, you are free to decide yourself: >> >> 1. In ManifestDigester.java, I'd rather define nameBuf inside the if (isNameAttr(bytes, start)) block. >> >> 2. I normally don't use "import static" unless I can save a lot of keystrokes and still do not confuse anyone. Both in the test and in ManifestDigester.java. >> >> 3. In the test, there is no need to write "@run main MultibyteUnicodeName" if it is the only action (i.e. no other build/compile) and there is no modifier on main (i.e. no othervm/timeout etc). >> >> Several tiny problems: >> >> 1. No need for @modules in the test. Maybe you used some internal classes in your previous version? >> >> 2. In the new consolidated repo, the test should be inside test/jdk/sun/..., please note the "jdk" after "test". >> >> 3. Please update the copyright years. >> >> 4. In the test, there should be no "JDK-" in the @test tag. >> >> Thanks >> Max >> >>> On Sep 24, 2017, at 7:51 PM, Philipp Kunz wrote: >>> >>> Hi Max and Vincent >>> >>> Thank you for your suggestions. It sure looks better now. I hope this time I got the patch added in the right format. >>> >>> diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java >>> --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 >>> +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Sun Sep 24 10:34:00 2017 +0200 >>> @@ -28,6 +28,7 @@ >>> import java.security.*; >>> import java.util.HashMap; >>> import java.io.ByteArrayOutputStream; >>> +import static java.nio.charset.StandardCharsets.UTF_8; >>> >>> /** >>> * This class is used to compute digests on sections of the Manifest. >>> @@ -112,7 +113,7 @@ >>> rawBytes = bytes; >>> entries = new HashMap<>(); >>> >>> - ByteArrayOutputStream baos = new ByteArrayOutputStream(); >>> + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); >>> >>> Position pos = new Position(); >>> >>> @@ -131,50 +132,40 @@ >>> >>> if (len > 6) { >>> if (isNameAttr(bytes, start)) { >>> - StringBuilder nameBuf = new StringBuilder(sectionLen); >>> + nameBuf.reset(); >>> + nameBuf.write(bytes, start+6, len-6); >>> >>> - try { >>> - nameBuf.append( >>> - new String(bytes, start+6, len-6, "UTF8")); >>> + int i = start + len; >>> + if ((i-start) < sectionLen) { >>> + if (bytes[i] == '\r') { >>> + i += 2; >>> + } else { >>> + i += 1; >>> + } >>> + } >>> >>> - int i = start + len; >>> - if ((i-start) < sectionLen) { >>> - if (bytes[i] == '\r') { >>> - i += 2; >>> - } else { >>> - i += 1; >>> - } >>> + while ((i-start) < sectionLen) { >>> + if (bytes[i++] == ' ') { >>> + // name is wrapped >>> + int wrapStart = i; >>> + while (((i-start) < sectionLen) >>> + && (bytes[i++] != '\n')); >>> + if (bytes[i-1] != '\n') >>> + return; // XXX: exception? >>> + int wrapLen; >>> + if (bytes[i-2] == '\r') >>> + wrapLen = i-wrapStart-2; >>> + else >>> + wrapLen = i-wrapStart-1; >>> + >>> + nameBuf.write(bytes, wrapStart, wrapLen); >>> + } else { >>> + break; >>> } >>> + } >>> >>> - while ((i-start) < sectionLen) { >>> - if (bytes[i++] == ' ') { >>> - // name is wrapped >>> - int wrapStart = i; >>> - while (((i-start) < sectionLen) >>> - && (bytes[i++] != '\n')); >>> - if (bytes[i-1] != '\n') >>> - return; // XXX: exception? >>> - int wrapLen; >>> - if (bytes[i-2] == '\r') >>> - wrapLen = i-wrapStart-2; >>> - else >>> - wrapLen = i-wrapStart-1; >>> - >>> - nameBuf.append(new String(bytes, wrapStart, >>> - wrapLen, "UTF8")); >>> - } else { >>> - break; >>> - } >>> - } >>> - >>> - entries.put(nameBuf.toString(), >>> - new Entry(start, sectionLen, sectionLenWithBlank, >>> - rawBytes)); >>> - >>> - } catch (java.io.UnsupportedEncodingException uee) { >>> - throw new IllegalStateException( >>> - "UTF8 not available on platform"); >>> - } >>> + entries.put(new String(nameBuf.toByteArray(), UTF_8), >>> + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); >>> } >>> } >>> start = pos.startOfNext; >>> diff -r ddc25f646c2e test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >>> --- /dev/null Thu Jan 01 00:00:00 1970 +0000 >>> +++ b/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java Sun Sep 24 10:34:00 2017 +0200 >>> @@ -0,0 +1,209 @@ >>> +/* >>> + * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. >>> + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>> + * >>> + * This code is free software; you can redistribute it and/or modify it >>> + * under the terms of the GNU General Public License version 2 only, as >>> + * published by the Free Software Foundation. >>> + * >>> + * This code is distributed in the hope that it will be useful, but WITHOUT >>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or >>> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License >>> + * version 2 for more details (a copy is included in the LICENSE file that >>> + * accompanied this code). >>> + * >>> + * You should have received a copy of the GNU General Public License version >>> + * 2 along with this work; if not, write to the Free Software Foundation, >>> + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. >>> + * >>> + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA >>> + * or visit www.oracle.com if you need additional information or have any >>> + * questions. >>> + */ >>> + >>> +/* >>> + * @test >>> + * @bug JDK-6695402 >>> + * @summary verify signatures of jars containing classes with names with multi- >>> + * byte unicode characters broken across lines >>> + * @library /test/lib >>> + * @modules jdk.jartool/sun.tools.jar >>> + * jdk.jartool/sun.security.tools.jarsigner >>> + * @run main MultibyteUnicodeName >>> + */ >>> + >>> +import java.io.IOException; >>> +import java.io.InputStream; >>> +import java.nio.file.Files; >>> +import java.nio.file.Paths; >>> +import java.util.jar.JarFile; >>> +import java.util.jar.Manifest; >>> +import java.util.Arrays; >>> +import java.util.Map; >>> +import java.util.jar.Attributes.Name; >>> +import java.util.jar.JarEntry; >>> + >>> +import static java.nio.charset.StandardCharsets.UTF_8; >>> + >>> +import static jdk.test.lib.SecurityTools.jarsigner; >>> +import static jdk.test.lib.SecurityTools.keytool; >>> +import static jdk.test.lib.util.JarUtils.createJar; >>> +import static jdk.test.lib.util.JarUtils.updateJar; >>> + >>> +public class MultibyteUnicodeName { >>> + >>> + /** >>> + * this class's name will break across lines in the jar manifest at the middle of >>> + * a two-byte utf encoded character due to its e acute letter at its exact position. >>> + * >>> + * @see also eAcute in {@link MultibyteUnicodeName#verifyClassNameLineBroken(String)} >>> + */ >>> + static class A1234567890B1234567890C1234567890D12345678?xyz { } >>> + static class A1234567890B1234567890C1234567890D12345678exyz { } >>> + >>> + static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + >>> + "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; >>> + static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + >>> + "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; >>> + >>> + static final String alias = "a"; >>> + static final String keystoreFileName = "test.jks"; >>> + static final String ManifestFileName = "MANIFEST.MF"; >>> + >>> + public static void main(String[] args) throws Exception { >>> + try { >>> + prepare(); >>> + >>> + testSignJar("test.jar"); >>> + testSignJarNoManifest("test-no-manifest.jar"); >>> + testSignJarUpdate("test-update.jar", "test-updated.jar"); >>> + >>> + } finally { >>> + Files.deleteIfExists(Paths.get(keystoreFileName)); >>> + Files.deleteIfExists(Paths.get(ManifestFileName)); >>> + Files.deleteIfExists(Paths.get(refClassFilename)); >>> + Files.deleteIfExists(Paths.get(testClassFilename)); >>> + } >>> + } >>> + >>> + static void prepare() throws Exception { >>> + keytool("-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", >>> + "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", >>> + "-validity", "366").shouldHaveExitValue(0); >>> + >>> + Files.write(Paths.get(ManifestFileName), >>> + (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); >>> + Files.copy(MultibyteUnicodeName.class.getResourceAsStream(refClassFilename), >>> + Paths.get(refClassFilename)); >>> + Files.copy(MultibyteUnicodeName.class.getResourceAsStream(testClassFilename), >>> + Paths.get(testClassFilename)); >>> + } >>> + >>> + static void testSignJar(String jarFileName) throws Exception { >>> + try { >>> + createJar(jarFileName, ManifestFileName, refClassFilename, testClassFilename); >>> + verifyJarSignature(jarFileName); >>> + >>> + } finally { >>> + Files.deleteIfExists(Paths.get(jarFileName)); >>> + } >>> + } >>> + >>> + static void testSignJarNoManifest(String jarFileName) throws Exception { >>> + try { >>> + createJar(jarFileName, refClassFilename, testClassFilename); >>> + verifyJarSignature(jarFileName); >>> + >>> + } finally { >>> + Files.deleteIfExists(Paths.get(jarFileName)); >>> + } >>> + } >>> + >>> + static void testSignJarUpdate(String initialFileName, String updatedFileName) throws Exception { >>> + try { >>> + createJar(initialFileName, ManifestFileName, refClassFilename); >>> + jarsigner("-keystore", keystoreFileName, "-storetype", "JKS", >>> + "-storepass", "changeit", "-debug", initialFileName, alias) >>> + .shouldHaveExitValue(0); >>> + updateJar(initialFileName, updatedFileName, testClassFilename); >>> + verifyJarSignature(updatedFileName); >>> + >>> + } finally { >>> + Files.deleteIfExists(Paths.get(initialFileName)); >>> + Files.deleteIfExists(Paths.get(updatedFileName)); >>> + } >>> + } >>> + >>> + static void verifyJarSignature(String jarFileName) throws Exception { >>> + // actually sign the jar >>> + jarsigner("-keystore", keystoreFileName, "-storetype", "JKS", >>> + "-storepass", "changeit", "-debug", jarFileName, alias) >>> + .shouldHaveExitValue(0); >>> + >>> + try ( >>> + JarFile jar = new JarFile(jarFileName); >>> + ) { >>> + verifyClassNameLineBroken(jar, testClassFilename); >>> + verifyCodeSigners(jar, jar.getJarEntry(testClassFilename)); >>> + verifyCodeSigners(jar, jar.getJarEntry(refClassFilename)); >>> + } >>> + } >>> + >>> + /** >>> + * it would be too easy to miss the actual test case by just renaming an identifier so that >>> + * the multi-byte encoded character would not any longer be broken across a line break. >>> + * >>> + * this test verifies that the actual test case is tested based on the manifest >>> + * and not based on the signature file because at the moment, the signature file >>> + * does not even contain the desired entry at all. >>> + * >>> + * this relies on the Manifest breaking lines unaware of bytes that belong to the same >>> + * multi-ybte utf characters. >>> + */ >>> + static void verifyClassNameLineBroken(JarFile jar, String className) throws IOException { >>> + byte[] eAcute = "?".getBytes(UTF_8); >>> + byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; >>> + >>> + if (jar.getManifest().getAttributes(className) == null) { >>> + throw new AssertionError(className + " not found in manifest"); >>> + } >>> + >>> + JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); >>> + try ( >>> + InputStream manifestIs = jar.getInputStream(manifestEntry); >>> + ) { >>> + int bytesMatched = 0; >>> + for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { >>> + if ((byte) b == eAcuteBroken[bytesMatched]) { >>> + bytesMatched++; >>> + if (bytesMatched == eAcuteBroken.length) { >>> + break; >>> + } >>> + } else { >>> + bytesMatched = 0; >>> + } >>> + } >>> + if (bytesMatched < eAcuteBroken.length) { >>> + throw new AssertionError("self-test failed: " >>> + + "multi-byte utf-8 character not broken across lines"); >>> + } >>> + } >>> + } >>> + >>> + static void verifyCodeSigners(JarFile jar, JarEntry jarEntry) throws IOException { >>> + // codeSigners is initialized only after the entry has been read >>> + try ( >>> + InputStream inputStream = jar.getInputStream(jarEntry); >>> + ) { >>> + while (inputStream.read() > -1); >>> + >>> + if (jarEntry.getCodeSigners() == null || jarEntry.getCodeSigners().length == 0) { >>> + throw new AssertionError("no signing certificate found for " + jarEntry.getName()); >>> + } >>> + } >>> + >>> + // a check for the presence of code signers is sufficient to check bug JDK-6695402. >>> + // no need to also verify the actual code signers attributes here. >>> + } >>> + >>> +} >>> >>> Regards, >>> Philipp >>> >>> >>> >>>> On 20.09.2017 03:41, Weijun Wang wrote: >>>> Hi Philipp >>>> >>>> The change mostly looks fine. You might want to put everything into a patch file so Vincent can recreate a webrev and post it to cr.openjdk.java.net. >>>> >>>> One thing I would suggest for the test is that instead of using jarsigner -verify and check the text in output you can open it as a JarFile, consume the content of an entry, and look into its getCertificates(). >>>> >>>> Also, you might want to reuse test/lib/jdk/test/lib/SecurityTools.java, and there is also JarUtils.java you can use. Maybe Files.copy(InputStream,Path) can be used in copyClassToFile(). >>>> >>>> Thanks >>>> Max >>>> >>>> >>>>> On Sep 20, 2017, at 7:41 AM, Philipp Kunz >>>>> wrote: >>>>> >>>>> Hello Vincent >>>>> >>>>> Here may be the fix for JDK-6695402. First a test. >>>>> >>>>> BEGIN /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >>>>> /* >>>>> * Copyright (c) 1999, Oracle and/or its affiliates. All rights reserved. >>>>> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >>>>> * >>>>> * This code is free software; you can redistribute it and/or modify it >>>>> * under the terms of the GNU General Public License version 2 only, as >>>>> * published by the Free Software Foundation. >>>>> * >>>>> * This code is distributed in the hope that it will be useful, but WITHOUT >>>>> * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or >>>>> * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License >>>>> * version 2 for more details (a copy is included in the LICENSE file that >>>>> * accompanied this code). >>>>> * >>>>> * You should have received a copy of the GNU General Public License version >>>>> * 2 along with this work; if not, write to the Free Software Foundation, >>>>> * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. >>>>> * >>>>> * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA >>>>> * or visit >>>>> www.oracle.com >>>>> if you need additional information or have any >>>>> * questions. >>>>> */ >>>>> >>>>> /* >>>>> * @test >>>>> * @bug JDK-6695402 >>>>> * @summary verify signatures of jars containing classes with names with multi- >>>>> * byte unicode characters broken across lines >>>>> * @library /test/lib >>>>> * @modules jdk.jartool/sun.tools.jar >>>>> * jdk.jartool/sun.security.tools.jarsigner >>>>> * @build jdk.test.lib.JDKToolLauncher >>>>> * jdk.test.lib.process.* >>>>> * @run main MultibyteUnicodeName >>>>> */ >>>>> >>>>> import java.io.ByteArrayOutputStream; >>>>> import java.io.IOException; >>>>> import java.io.InputStream; >>>>> import java.io.UnsupportedEncodingException; >>>>> import java.nio.file.Files; >>>>> import java.nio.file.Paths; >>>>> import java.util.jar.JarFile; >>>>> import java.util.jar.Manifest; >>>>> import java.util.stream.Collectors; >>>>> import java.util.Arrays; >>>>> import java.util.Map; >>>>> import java.util.jar.Attributes.Name; >>>>> import java.util.jar.JarEntry; >>>>> >>>>> import static java.nio.charset.StandardCharsets.UTF_8; >>>>> >>>>> import sun.security.tools.jarsigner.Resources; >>>>> >>>>> import jdk.test.lib.JDKToolLauncher; >>>>> import jdk.test.lib.process.OutputAnalyzer; >>>>> import jdk.test.lib.process.ProcessTools; >>>>> >>>>> public class MultibyteUnicodeName { >>>>> >>>>> public static void main(String[] args) throws Throwable { >>>>> try { >>>>> prepare(); >>>>> >>>>> testSignJar("test.jar"); >>>>> testSignJarNoManifest("test-no-manifest.jar"); >>>>> testSignJarUpdate("test-update.jar"); >>>>> testSignJarWithIndex("test-index.jar"); >>>>> testSignJarAddIndex("test-add-index.jar"); >>>>> >>>>> } finally { >>>>> Files.deleteIfExists(Paths.get(keystoreFileName)); >>>>> Files.deleteIfExists(Paths.get(ManifestFileName)); >>>>> Files.deleteIfExists(Paths.get(refClassFilename)); >>>>> Files.deleteIfExists(Paths.get(testClassFilename)); >>>>> } >>>>> } >>>>> >>>>> static final String alias = "a"; >>>>> static final String keystoreFileName = "test.jks"; >>>>> static final String ManifestFileName = "MANIFEST.MF"; >>>>> >>>>> static class A1234567890B1234567890C1234567890D12345678exyz { } >>>>> static class A1234567890B1234567890C1234567890D12345678?xyz { } >>>>> >>>>> static final String refClassFilename = MultibyteUnicodeName.class.getSimpleName() + >>>>> "$" + A1234567890B1234567890C1234567890D12345678exyz.class.getSimpleName() + ".class"; >>>>> static final String testClassFilename = MultibyteUnicodeName.class.getSimpleName() + >>>>> "$" + A1234567890B1234567890C1234567890D12345678?xyz.class.getSimpleName() + ".class"; >>>>> >>>>> static void prepare() throws Throwable { >>>>> tool("keytool", "-keystore", keystoreFileName, "-genkeypair", "-storepass", "changeit", >>>>> "-keypass", "changeit", "-storetype", "JKS", "-alias", alias, "-dname", "CN=X", >>>>> "-validity", "366") >>>>> .shouldHaveExitValue(0); >>>>> >>>>> Files.write(Paths.get(ManifestFileName), >>>>> (Name.MANIFEST_VERSION.toString() + ": 1.0\r\n").getBytes(UTF_8.name())); >>>>> copyClassToFile(refClassFilename); >>>>> copyClassToFile(testClassFilename); >>>>> } >>>>> >>>>> static void copyClassToFile(String classFilename) throws IOException { >>>>> try ( >>>>> InputStream asStream = MultibyteUnicodeName.class.getResourceAsStream(classFilename); >>>>> ByteArrayOutputStream buf = new ByteArrayOutputStream(); >>>>> ) { >>>>> int b; >>>>> while ((b = asStream.read()) != -1) buf.write(b); >>>>> Files.write(Paths.get(classFilename), buf.toByteArray()); >>>>> } >>>>> } >>>>> >>>>> static void testSignJar(String jarFileName) throws Throwable { >>>>> try { >>>>> tool("jar", "cvfm", jarFileName, >>>>> ManifestFileName, refClassFilename, testClassFilename) >>>>> .shouldHaveExitValue(0); >>>>> verifyJarSignature(jarFileName); >>>>> >>>>> } finally { >>>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>>> } >>>>> } >>>>> >>>>> static void testSignJarNoManifest(String jarFileName) throws Throwable { >>>>> try { >>>>> tool("jar", "cvf", jarFileName, refClassFilename, testClassFilename) >>>>> .shouldHaveExitValue(0); >>>>> verifyJarSignature(jarFileName); >>>>> >>>>> } finally { >>>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>>> } >>>>> } >>>>> >>>>> static void testSignJarUpdate(String jarFileName) throws Throwable { >>>>> try { >>>>> tool("jar", "cvfm", jarFileName, ManifestFileName, refClassFilename) >>>>> .shouldHaveExitValue(0); >>>>> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >>>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>>> .shouldHaveExitValue(0); >>>>> tool("jar", "uvf", jarFileName, testClassFilename) >>>>> .shouldHaveExitValue(0); >>>>> verifyJarSignature(jarFileName); >>>>> >>>>> } finally { >>>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>>> } >>>>> } >>>>> >>>>> static void testSignJarWithIndex(String jarFileName) throws Throwable { >>>>> try { >>>>> tool("jar", "cvfm", jarFileName, >>>>> ManifestFileName, refClassFilename, testClassFilename) >>>>> .shouldHaveExitValue(0); >>>>> tool("jar", "iv", jarFileName).shouldHaveExitValue(0); >>>>> verifyJarSignature(jarFileName); >>>>> >>>>> } finally { >>>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>>> } >>>>> } >>>>> >>>>> static void testSignJarAddIndex(String jarFileName) throws Throwable { >>>>> try { >>>>> tool("jar", "cvfm", jarFileName, >>>>> ManifestFileName, refClassFilename, testClassFilename) >>>>> .shouldHaveExitValue(0); >>>>> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >>>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>>> .shouldHaveExitValue(0); >>>>> tool("jar", "iv", jarFileName).shouldHaveExitValue(0); >>>>> verifyJarSignature(jarFileName); >>>>> >>>>> } finally { >>>>> Files.deleteIfExists(Paths.get(jarFileName)); >>>>> } >>>>> } >>>>> >>>>> static Map jarsignerResources = Arrays.stream(new Resources().getContents()). >>>>> collect(Collectors.toMap(e -> (String) e[0], e -> (String) e[1])); >>>>> >>>>> static void verifyJarSignature(String jarFileName) throws Throwable { >>>>> // actually sign the jar >>>>> tool("jarsigner", "-keystore", keystoreFileName, "-storetype", "JKS", >>>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>>> .shouldHaveExitValue(0); >>>>> >>>>> verifyClassNameLineBroken(jarFileName); >>>>> >>>>> // check for no unsigned entries >>>>> tool("jarsigner", "-verify", "-keystore", keystoreFileName, "-storetype", "JKS", >>>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>>> .shouldHaveExitValue(0) >>>>> .shouldNotContain(jarsignerResources.get( >>>>> "This.jar.contains.unsigned.entries.which.have.not.been.integrity.checked.")); >>>>> >>>>> // check that both classes with and without multi-byte unicode characters in their names are signed >>>>> tool("jarsigner", "-verify", "-verbose", "-keystore", keystoreFileName, "-storetype", "JKS", >>>>> "-storepass", "changeit", "-debug", jarFileName, alias) >>>>> .shouldHaveExitValue(0) >>>>> .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + refClassFilename.replaceAll("\\$", "\\\\\\$") + "$") >>>>> .shouldMatch("^" + jarsignerResources.get("s") + jarsignerResources.get("m") + jarsignerResources.get("k") + ".+" + testClassFilename.replaceAll("\\$", "\\\\\\$") + "$"); >>>>> >>>>> // check that both classes with and without multi-byte unicode characters in their names have signing certificates >>>>> tool("jarsigner", "-verify", "-verbose", "-certs", "-keystore", keystoreFileName, >>>>> "-storetype", "JKS", "-storepass", "changeit", "-debug", jarFileName, alias) >>>>> .shouldHaveExitValue(0) >>>>> .shouldMatch(refClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)") >>>>> .shouldMatch(testClassFilename.replaceAll("\\$", "\\\\\\$") + "\\s*\\R*\\s*(\\[" + jarsignerResources.get("entry.was.signed.on") + " .*\\]\\R*)?\\s*X.509, CN=X \\(" + alias + "\\)"); >>>>> } >>>>> >>>>> /** >>>>> * it would be too easy to miss the actual test case by just renaming an identifier so that >>>>> * the multi-byte encoded character would not any longer be broken across a line break. >>>>> * >>>>> * this test verifies that the actual test case is tested based on the manifest >>>>> * and not based on the signature file because at the moment, the signature file >>>>> * does not even contain the desired entry at all. >>>>> * >>>>> * this relies on the Manifest breaking lines unaware of bytes that belong to the same >>>>> * multi-ybte utf characters. >>>>> */ >>>>> static void verifyClassNameLineBroken(String jarFileName) throws IOException { >>>>> byte[] eAcute = "?".getBytes(UTF_8); >>>>> byte[] eAcuteBroken = new byte[] {eAcute[0], '\r', '\n', ' ', eAcute[1]}; >>>>> >>>>> try ( >>>>> JarFile jar = new JarFile(jarFileName); >>>>> ) { >>>>> if (jar.getManifest().getAttributes(testClassFilename) == null) { >>>>> throw new AssertionError(testClassFilename + " not found in manifest"); >>>>> } >>>>> >>>>> JarEntry manifestEntry = jar.getJarEntry(JarFile.MANIFEST_NAME); >>>>> try ( >>>>> InputStream manifestIs = jar.getInputStream(manifestEntry); >>>>> ) { >>>>> int bytesMatched = 0; >>>>> for (int b = manifestIs.read(); b > -1; b = manifestIs.read()) { >>>>> if ((byte) b == eAcuteBroken[bytesMatched]) { >>>>> bytesMatched++; >>>>> if (bytesMatched == eAcuteBroken.length) { >>>>> break; >>>>> } >>>>> } else { >>>>> bytesMatched = 0; >>>>> } >>>>> } >>>>> if (bytesMatched < eAcuteBroken.length) { >>>>> throw new AssertionError("self-test failed: " >>>>> + "multi-byte utf-8 character not broken across lines"); >>>>> } >>>>> } >>>>> } >>>>> } >>>>> >>>>> static OutputAnalyzer tool(String tool, String... args) >>>>> throws Throwable { >>>>> JDKToolLauncher l = JDKToolLauncher.createUsingTestJDK(tool); >>>>> for (String arg : args) { >>>>> if (arg .startsWith("-J")) { >>>>> l.addVMArg(arg.substring(2)); >>>>> } else { >>>>> l.addToolArg(arg); >>>>> } >>>>> } >>>>> return ProcessTools.executeCommand(l.getCommand()); >>>>> } >>>>> >>>>> } >>>>> END /jdk10/jdk/test/sun/security/tools/jarsigner/MultibyteUnicodeName.java >>>>> >>>>> >>>>> There are three e with acutes, on lines 84, 89, and 227, just in case the mail suffers bad encoding which might not be absolutely unconceivable regarding the bug's subject. >>>>> >>>>> In contrast to the bug description it uses a nested class with a two-byte UTF-8 character rather than one in its own file. I chose to do it that way because then the complete test goes into one file and therefore I assume the overview is kept easier. The test still demonstrates exactly JDK-6695402's problem. >>>>> >>>>> It may not have been necessary to also test jar files with indexes but i consider it necessary to have signing unsigned as well as partially signed jars tested, so why not have one more case tested, too? >>>>> >>>>> There might also be a more elegant approach to get class files into a jar file as to get their contents through the class loader. I chose to use real class files rather than dummy contents in files named *.class or for instance plain text files in the jar the signing of which to be tested in order to stay as close to the original bug problem as possible even though I don't have any notion that it would make a difference. The amount of code used to copy the class file contents around is comparatively small with respect to the whole test case amount of code. >>>>> >>>>> For the demonstration that the multi-byte character actually makes the alleged difference, I concluded that it was necessary to have another class name not previously affected by the bug in order to compare the effects of just different names. Otherwise the signing could fail to any other reason undetectably. >>>>> >>>>> In order to express a condition to tell successful from failed test runs the best approach I found was to analyze the jarsigner tool's output which is in my opinion not the most desirable option. I'd have preferred output from a clearer structured api but then I guess the output format will not change too often in the foreseeable future. For instance the check for the s, m, and k flags is more pragmatical approach than obviously perfectly failsafe when operating with regular expressions to match it with the output. Other parts such as 'X.509' and 'CN=' are not even jarsigner tool resources. I put at least a reference to sun.security.tools.jarsigner.Resources. >>>>> >>>>> Finally, I afforded kind of a self-test that protects the test from undetected failing because renaming the main class which would move the two-byte unicode character to a position not suitable for the test. It may not be absolutely necessary. >>>>> >>>>> >>>>> >>>>> BEGIN PATH >>>>> diff -r ddc25f646c2e src/java.base/share/classes/sun/security/util/ManifestDigester.java >>>>> --- a/src/java.base/share/classes/sun/security/util/ManifestDigester.java Thu Aug 31 22:21:20 2017 -0700 >>>>> +++ b/src/java.base/share/classes/sun/security/util/ManifestDigester.java Wed Sep 20 00:56:15 2017 +0200 >>>>> @@ -28,6 +28,7 @@ >>>>> import java.security.*; >>>>> import java.util.HashMap; >>>>> import java.io.ByteArrayOutputStream; >>>>> +import static java.nio.charset.StandardCharsets.UTF_8; >>>>> >>>>> /** >>>>> * This class is used to compute digests on sections of the Manifest. >>>>> @@ -112,7 +113,7 @@ >>>>> rawBytes = bytes; >>>>> entries = new HashMap<>(); >>>>> >>>>> - ByteArrayOutputStream baos = new ByteArrayOutputStream(); >>>>> + ByteArrayOutputStream nameBuf = new ByteArrayOutputStream(); >>>>> >>>>> Position pos = new Position(); >>>>> >>>>> @@ -131,50 +132,40 @@ >>>>> >>>>> if (len > 6) { >>>>> if (isNameAttr(bytes, start)) { >>>>> - StringBuilder nameBuf = new StringBuilder(sectionLen); >>>>> + nameBuf.reset(); >>>>> + nameBuf.write(bytes, start+6, len-6); >>>>> >>>>> - try { >>>>> - nameBuf.append( >>>>> - new String(bytes, start+6, len-6, "UTF8")); >>>>> + int i = start + len; >>>>> + if ((i-start) < sectionLen) { >>>>> + if (bytes[i] == '\r') { >>>>> + i += 2; >>>>> + } else { >>>>> + i += 1; >>>>> + } >>>>> + } >>>>> >>>>> - int i = start + len; >>>>> - if ((i-start) < sectionLen) { >>>>> - if (bytes[i] == '\r') { >>>>> - i += 2; >>>>> - } else { >>>>> - i += 1; >>>>> - } >>>>> + while ((i-start) < sectionLen) { >>>>> + if (bytes[i++] == ' ') { >>>>> + // name is wrapped >>>>> + int wrapStart = i; >>>>> + while (((i-start) < sectionLen) >>>>> + && (bytes[i++] != '\n')); >>>>> + if (bytes[i-1] != '\n') >>>>> + return; // XXX: exception? >>>>> + int wrapLen; >>>>> + if (bytes[i-2] == '\r') >>>>> + wrapLen = i-wrapStart-2; >>>>> + else >>>>> + wrapLen = i-wrapStart-1; >>>>> + >>>>> + nameBuf.write(bytes, wrapStart, wrapLen); >>>>> + } else { >>>>> + break; >>>>> } >>>>> + } >>>>> >>>>> - while ((i-start) < sectionLen) { >>>>> - if (bytes[i++] == ' ') { >>>>> - // name is wrapped >>>>> - int wrapStart = i; >>>>> - while (((i-start) < sectionLen) >>>>> - && (bytes[i++] != '\n')); >>>>> - if (bytes[i-1] != '\n') >>>>> - return; // XXX: exception? >>>>> - int wrapLen; >>>>> - if (bytes[i-2] == '\r') >>>>> - wrapLen = i-wrapStart-2; >>>>> - else >>>>> - wrapLen = i-wrapStart-1; >>>>> - >>>>> - nameBuf.append(new String(bytes, wrapStart, >>>>> - wrapLen, "UTF8")); >>>>> - } else { >>>>> - break; >>>>> - } >>>>> - } >>>>> - >>>>> - entries.put(nameBuf.toString(), >>>>> - new Entry(start, sectionLen, sectionLenWithBlank, >>>>> - rawBytes)); >>>>> - >>>>> - } catch (java.io.UnsupportedEncodingException uee) { >>>>> - throw new IllegalStateException( >>>>> - "UTF8 not available on platform"); >>>>> - } >>>>> + entries.put(new String(nameBuf.toByteArray(), UTF_8), >>>>> + new Entry(start, sectionLen, sectionLenWithBlank, rawBytes)); >>>>> } >>>>> } >>>>> start = pos.startOfNext; >>>>> END PATCH >>>>> >>>>> >>>>> The patch is mostly so big because indentation has changed in many lines. >>>>> >>>>> There was baos there before without apparent use. The patch renames it and puts it into use. It came in very handy because if it would not have been there already, I would have had to defined one of my own. >>>>> >>>>> The main point of the patch is that manifest section names that are broken across lines at possibly arbitrary bytes are no longer converted into strings for each manifest line and then joined. Parts of names broken across lines are now joined first when they are still byte sequences and only when the complete byte sequence is available after processing all manifest lines that contain the same manifest section name decoded into a unicode string. >>>>> >>>>> For decoding the bytes into a string I chose a different string constructor than was used before that does not any longer declare UnsupportedEncodingException rendering the try/catch redundant. It couldn't have ever occurred anyway taking into consideration that UTF-8 is mandatory for every Java platform, StandardCharsets says. The difference according to the documentation is that the previously used String constructor returned undefined strings for invalid byte sequences whereas the one used in the patch will replace unparseable portions with valid 'unknown' characters. >>>>> >>>>> One question I cannot still answer yet is how the ManifestDigester can be changed at all without complete test coverage or risking to break existing signatures. I already started on that but it's quite a piece of work to almost formally prove that all manifests will continue to produce identical digests, except of course for the ones now fixed. >>>>> >>>>> Regards, >>>>> Philipp >>>>> >>>>> >>>>>> On 17.09.2017 21:25, Philipp Kunz wrote: >>>>>> >>>>>> Hello Vincent >>>>>> >>>>>> I narrowed the error down so far and suspect now that it is an effect of sun.security.util.ManifestDigester's constructor, lines 134 and 163-164, in combination with java.util.jar.Manifest.make72Safe. Manifest breaks multi-byte characters in manifest section names across lines and ManifestDigester fails to restore them correctly, which both sounds wrong but ManifestDigester sure is. >>>>>> >>>>>> Just fixing it would be too tempting but then I would not know (I mean not know for sure with evidence from tests) if any existing jar-signature would still be valid and I don't want to break existing signatures. When I had a look at the tests specific for Manifest and ManifestDigester I found very little. There are many more different situations and corner cases and combinations thereof to cover with tests than it looks like at first glance so that writing tests that cover all relevant cases looks to me like quite an effort. I have the impression that test coverage was not a priority when the subjected code was developed or at least the tests might not have been contributed to the open JDK project. Hence, while I'm continuing to complete these tests I miss, if someone has an idea how to simplify that so that I can still convince at least myself that no existing signature will break or where possibly more testcases are that I haven't discovered so far, please let me know. >>>>>> >>>>>> I'm also wondering how much performance should be taken into consideration. There are a few hints such as reusing byte arrays that suggest that it is an issue. Will a patch be rejected if it slows down some tests beyond a certain limit for so little added value or what might be the acceptance criteria here? I don't expect insuperable difficulties with performance but would still appreciate some general idea. >>>>>> >>>>>> My desired or currently preferred approach to fix JDK-6695402 would be to let ManifestDigester any kind of use or extend Manifest in order to let Manifest identify the manifest sections thereby preventing the parsing duplicated that identifies the manifest sections. >>>>>> >>>>>> As a new contributor I could use and would appreciate some guidance particularly about what kind and completeness of test coverage and performance tuning applies or is suggested in the context of the given bug. Otherwise I fear I might contribute too many tests which would have to be reviewed and maintained or quite some work would be for nothing if a patch would not satisfy performance requirements. >>>>>> >>>>>> Regards, >>>>>> Philipp >>>>>> >>>>>> >>>>>> >>>>>>> On 01.09.2017 15:20, Vincent Ryan wrote: >>>>>>> >>>>>>> That all sounds fine. Let me know when your patch is ready to submit. >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 1 Sep 2017, at 13:15, Philipp Kunz >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hello Vincent >>>>>>>> >>>>>>>> Thank you for sponsoring! >>>>>>>> So far, I have become a contributor by signing the OCA which has been accepted. Dalibor Topic wrote that he can confirm and it's also here: >>>>>>>> http://www.oracle.com/technetwork/community/oca-486395.html#p >>>>>>>> -> Paratix GmbH >>>>>>>> Therefore I think I have followed the steps in >>>>>>>> http://openjdk.java.net/contribute/ >>>>>>>> at least the ones before actual patch submission. >>>>>>>> >>>>>>>> Currently, I'm working on getting the existing test cases running locally. Unfortunately, I started with jdk 9 and switched to 10 now. I figured these commands run at least the relevant tests with 9 and hope this also applies to 10: >>>>>>>> make run-test-tier1 >>>>>>>> make run-test TEST="jdk/test" >>>>>>>> they report errors and failures but it hasn't been completed and released which might explain it. If I run the tests I assume relevant for my patch >>>>>>>> make run-test TEST="jtreg:jdk/test/sun/security/tools/jarsigner jtreg:jdk/test/java/util/jar" >>>>>>>> then it reports zero errors and failures which may be a good starting point. >>>>>>>> Do you think that sounds reasonable or do you have another suggestion how to run the tests? >>>>>>>> >>>>>>>> Next, I will add a test for JDK-6695402 before actually fixing it. As an example, I'll try something in the style of >>>>>>>> http://hg.openjdk.java.net/jdk9/dev/jdk/file/65464a307408/test/java/util/jar/Manifest/CreateManifest.java >>>>>>>> . This way, I try to demonstrate the improvement. >>>>>>>> >>>>>>>> I guess I have identified the following line as the cause: value = new String(vb, 0, 0, vb.length); >>>>>>>> >>>>>>>> http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/51f5d60713b5/src/java.base/share/classes/java/util/jar/Manifest.java#l157 >>>>>>>> >>>>>>>> So I'll try to remove it first including the whole four line if block. >>>>>>>> >>>>>>>> Philipp >>>>>>>> >>>>>>>> >>>>>>>>> On 01.09.2017 10:00, Vincent Ryan wrote: >>>>>>>>> >>>>>>>>> Hello Philipp, >>>>>>>>> >>>>>>>>> I?m happy to sponsor your fix for JDK 10. Have you followed these steps: >>>>>>>>> http://openjdk.java.net/contribute/ >>>>>>>>> ? >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 1 Sep 2017, at 08:58, Vincent Ryan >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Moved to security-dev >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 1 Sep 2017, at 08:28, Philipp Kunz >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hello everyone >>>>>>>>>>> >>>>>>>>>>> I have been developing with Java for around 17 years now and when I encountered some bug I decided to attempt to fix it: >>>>>>>>>>> https://bugs.openjdk.java.net/browse/JDK-6695402 >>>>>>>>>>> . This also looks like it may not be too big a piece for a first contribution. >>>>>>>>>>> >>>>>>>>>>> I read through quite some guides and all kinds of documents but could not yet help myself with the following questions: >>>>>>>>>>> >>>>>>>>>>> May I login to jira to add comments to bugs? If so, how would I request or receive credentials? Or are mailing lists preferred? >>>>>>>>>>> >>>>>>>>>>> Another question is whether I should apply it to jdk9, but it may be too late now, or to jdk10, and backporting can be considered later. Probably it wouldn't even make much a difference for the patch itself. >>>>>>>>>>> >>>>>>>>>>> One more question I have is how or where to find the sources from before migration to mercurial. Because some lines of code I intend to change go back farther and in the history I find only 'initial commit'. With such a history I might be able better to understand why it's there and prevent to make the same mistake again. >>>>>>>>>>> >>>>>>>>>>> I guess the appropriate mailing list for above mentioned bug is security-dev. Is it correct that I can send a patch there and just hope for some sponsor to pick it up? Of course I'd be glad if some sponsor would contact me and maybe provide some assistance or if someone would confirm that sending a patch to the mailing list is the right way to find a sponsor. >>>>>>>>>>> >>>>>>>>>>> Philipp Kunz >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Paratix GmbH >>>>>> St Peterhofstatt 11 >>>>>> 8001 Z?rich >>>>>> >>>>>> +41 (0)76 397 79 35 >>>>>> >>>>>> philipp.kunz at paratix.ch >>> -- >>> >>> >>> Gruss Philipp >>> >>> >>> >>> >>> >>> >>> Paratix GmbH >>> St Peterhofstatt 11 >>> 8001 Z?rich >>> >>> +41 (0)76 397 79 35 >>> philipp.kunz at paratix.ch -- Gruss Philipp ------------------------------------------------------------------------ Paratix GmbH St Peterhofstatt 11 8001 Z?rich +41 (0)76 397 79 35 philipp.kunz at paratix.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 5134 bytes Desc: not available URL: From adam.petcher at oracle.com Mon Sep 25 17:21:55 2017 From: adam.petcher at oracle.com (Adam Petcher) Date: Mon, 25 Sep 2017 13:21:55 -0400 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: I agree that an unsigned multiplyHigh would be useful for crypto purposes, and we should consider adding it. Of course, I would much rather have multiply operations that return both 64-bit parts of the result, but that is going to be hard to do well without value types. So it would be nice to have something like this in the meantime. If we are going to add this operation, it should probably be added along with an intrinsic. I think the Java code can simply factor out the else branch from the existing multiplyHigh code. This way, unsignedMultiplyHigh will be at least as fast as multiplyHigh, whether the intrinsic implementation is available or not. If possible, the implementation of this operation should not branch on either operand. This would make it more widely useful for constant-time crypto implementations. Though this property would need to go into the spec in order for constant-time crypto code to use this method, and I don't know how reasonable it is to put something like this in the spec. Side note: at the moment, I am using signed arithmetic in prototypes for Poly1305, X25519, and EdDSA, partially due to lack of support for unsigned operations like this one. I don't think having unsignedMultiplyHigh would, on its own, convince me to use an unsigned representation, but the forces are different for each algorithm/implementation. On 9/25/2017 10:50 AM, Andrew Haley wrote: > We now have a multiplyHigh intrinsic, but it is signed. Unsigned > multiplyHigh is in general a more useful primitive for crypto than > signed, and I wonder if we need an intrinsic for that as well. I've > looked at cooking up an unsigned multiplyHigh in Java, and I think the > fastest way is this: > > private static final long unsignedMultiplyHigh(long a, long b) { > long result = Math.multiplyHigh(a, b); > if (a < 0) result += b; > if (b < 0) result += a; > // Can also be written as: > // result += (a >> 63) & b; > // result += (b >> 63) & a; > return result; > } > > It's still about 50% slower than the signed multiplyHigh, though. > Thoughts? > From aph at redhat.com Mon Sep 25 17:48:37 2017 From: aph at redhat.com (Andrew Haley) Date: Mon, 25 Sep 2017 18:48:37 +0100 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: On 25/09/17 18:21, Adam Petcher wrote: > I agree that an unsigned multiplyHigh would be useful for crypto > purposes, and we should consider adding it. Of course, I would much > rather have multiply operations that return both 64-bit parts of the > result, but that is going to be hard to do well without value types. So > it would be nice to have something like this in the meantime. I take your point, but it won't be excruciatingly difficult for the C2 compiler to turn the multiply operations into a single one, if the CPU can do that. From what I've seen recently, though, on non-x86 it's common for the two halves of the result to be calculated by separate instructions. > If we are going to add this operation, it should probably be added > along with an intrinsic. I think the Java code can simply factor out > the else branch from the existing multiplyHigh code. This way, > unsignedMultiplyHigh will be at least as fast as multiplyHigh, > whether the intrinsic implementation is available or not. Sure. I can do that. > If possible, the implementation of this operation should not branch on > either operand. This would make it more widely useful for constant-time > crypto implementations. Though this property would need to go into the > spec in order for constant-time crypto code to use this method, and I > don't know how reasonable it is to put something like this in the spec. OK. I can do it so that there are no branches in the Java. The Java code for signed multiplyHigh has some data-dependent branches in an attempt to speed it up, though. I don't know how effective they are, and I could have a look at taking them out. > Side note: at the moment, I am using signed arithmetic in prototypes for > Poly1305, X25519, and EdDSA, partially due to lack of support for > unsigned operations like this one. I don't think having > unsignedMultiplyHigh would, on its own, convince me to use an unsigned > representation, but the forces are different for each > algorithm/implementation. Sure. I don't think it really matters from a performance point of view which you use, given intrinsics for both. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From weijun.wang at oracle.com Tue Sep 26 04:19:16 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 26 Sep 2017 12:19:16 +0800 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> Message-ID: > On Sep 26, 2017, at 1:11 AM, Philipp Kunz wrote: > > Hi Max > > Thank you for the detailed assistance and I really hope it doesn't annoy you too much with so many beginner's mistakes. Every little point of yours seems to be absolutely justified in my point of view. I'm flattered. > > I did not understand where I could apply the readAllBytes. In case it still applies, would you give me another hint? I'd rather expect some potential for butifying verifyClassNameLineBroken but then I haven't found any really. I meant you can change "while (inputStream.read() > -1);" to "inputStream.readAllBytes()". Yours is fine. Some people do not like the return value wasted and the auto-increment mechanism in the method is unnecessary here. > > About where to place the declaration of nameBuf I was a little confused probably/obviously when there was a ByteArrayOutputStream before which, however, actually was not used into assuming that reusing the same buffer for all sections would result in better performance or so but that's certainly not a valid assumption now as I consider it again. > > About the import static you convinced me but for statically importing java.nio.charset.StandardCharsets.UTF_8 which I prefer this way still. After you wrote I should decide myself, I removed other static imports. If there was kind of an unwritten rule not to use static imports generally that would be a compelling reason but I didn't bother to search for import static through the jdk sources. I don't know if there is a rule. Yours is OK. I said it was just my habit. > > The @modules was a left-over from previous versions. > > I'm not sure where you meant I should put the test to exactly. In http://hg.openjdk.java.net/jdk10/jdk10/jdk/file/777356696811/test/jdk I cant find the parent folder sun you mentioned. In case there is some reorganization in progress not having arrived yet at the tip, could the new test be moved together with it? Or did you mean I should create the sun folder but then the existing tests would not be next to the new one? Did I look in the wrong place? Oh sorry, we've just recently change it to http://hg.openjdk.java.net/jdk10/master/. You don't really need to rework this time, but if you want to continue hacking on OpenJDK, this is the new repo. > > When updating the copyright years I was not sure if I should let the existing one, 2011, there in ManifestDigester or replace it with the current year which I did. When looking into other class files I saw none with more than two years but "," looks a little odd to me for indicating a range. It is a little uncommon, but that does mean a range. > > I assumed that I remove the JDK- prefix from the @bug tag and not the @test tag unless @bug is considered part of the @test. Correct. @test must be there otherwise jtreg will not treat it as a test. @bug might be used by some reporting tools. > > Is it correct, that a maximum line width of 80 characters is the convention? In case I added some more breaks for that. I just wonder what other style guides I should have respected. Yes that is the convention. Sometimes a line can be longer if wrapping it is very ugly. In general, don't make a Sdiffs page in a webrev non-readable. A Sdiffs page example is here http://cr.openjdk.java.net/~bpb/8186766/webrev.00/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c.sdiff.html > > Just in case you think it would be more efficient I'd not object that a reviewer or some else would just change these little things to get over with it. On the other hand this way I learn it. If there is a suitable documentation that much detailed, I'd be glad if you could point me at it. That might save a few cycles. I don't know if there is a more suitable doc. I have some small suggestions: 1. In fact I don't know exactly if a non-ASCII is allowed in source code now. For safety, please change the ? char to \u1234 style. 2. Maybe it's better to capitalize "b" in the test name? 3. The @see tag has its format, you can just use "See...". The method name in @link has a wrong signature. * @see also eAcute in * {@link MultibyteUnicodeName#verifyClassNameLineBroken(String)} 4. Several "throws Exception" should be indented by 8 chars. 5. There are several trailing spaces in your patch. Maybe because of copy-and-paste. Please send me the new patch as an attachment. That will be more precise and more formal. Please send me what the exact "name " you want to see in the Contributed-by field of the changeset. Thanks Max > > Regards, > Philipp From philipp.kunz at paratix.ch Tue Sep 26 05:37:15 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Tue, 26 Sep 2017 07:37:15 +0200 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> Message-ID: <869f290c-bf85-d080-99b5-df5b9b59460e@paratix.ch> Hi Max This time I got it with readAllBytes. Thank you for the hint. Apparently, UTF characters are allowed in source code, particularly in identifiers here, which also has caused the bug. Even if only for sending patches around I changed it and was surprised to see escaping working not only in strings but also in identifiers. When I had another look at the test I came to the conclusion that it does not need what has been named refClassFileName before. The purpose of the test is only to check a signature of a class with a two byte character in its name and not at the same time to verify that if that test failed it is specifically because of the name. If it fails there is a problem no matter why. In the beginning it was handy to see the difference but I don't think it should be kept and maintained so I removed it. For the update signature case a second file to sign is still required though. I considered multi-byte a one word before but now I also prefer it with a capital b. Anyway, this name might not be the best choice and I changed it to LineBrokenMultiByteCharacter. See attached patch. Regards, Philipp On 26.09.2017 06:19, Weijun Wang wrote: >> On Sep 26, 2017, at 1:11 AM, Philipp Kunz wrote: >> >> Hi Max >> >> Thank you for the detailed assistance and I really hope it doesn't annoy you too much with so many beginner's mistakes. Every little point of yours seems to be absolutely justified in my point of view. > I'm flattered. > >> I did not understand where I could apply the readAllBytes. In case it still applies, would you give me another hint? I'd rather expect some potential for butifying verifyClassNameLineBroken but then I haven't found any really. > I meant you can change "while (inputStream.read() > -1);" to "inputStream.readAllBytes()". Yours is fine. Some people do not like the return value wasted and the auto-increment mechanism in the method is unnecessary here. > >> About where to place the declaration of nameBuf I was a little confused probably/obviously when there was a ByteArrayOutputStream before which, however, actually was not used into assuming that reusing the same buffer for all sections would result in better performance or so but that's certainly not a valid assumption now as I consider it again. >> >> About the import static you convinced me but for statically importing java.nio.charset.StandardCharsets.UTF_8 which I prefer this way still. After you wrote I should decide myself, I removed other static imports. If there was kind of an unwritten rule not to use static imports generally that would be a compelling reason but I didn't bother to search for import static through the jdk sources. > I don't know if there is a rule. Yours is OK. I said it was just my habit. > >> The @modules was a left-over from previous versions. >> >> I'm not sure where you meant I should put the test to exactly. In http://hg.openjdk.java.net/jdk10/jdk10/jdk/file/777356696811/test/jdk I cant find the parent folder sun you mentioned. In case there is some reorganization in progress not having arrived yet at the tip, could the new test be moved together with it? Or did you mean I should create the sun folder but then the existing tests would not be next to the new one? Did I look in the wrong place? > Oh sorry, we've just recently change it to http://hg.openjdk.java.net/jdk10/master/. You don't really need to rework this time, but if you want to continue hacking on OpenJDK, this is the new repo. > >> When updating the copyright years I was not sure if I should let the existing one, 2011, there in ManifestDigester or replace it with the current year which I did. When looking into other class files I saw none with more than two years but "," looks a little odd to me for indicating a range. > It is a little uncommon, but that does mean a range. > >> I assumed that I remove the JDK- prefix from the @bug tag and not the @test tag unless @bug is considered part of the @test. > Correct. @test must be there otherwise jtreg will not treat it as a test. @bug might be used by some reporting tools. > >> Is it correct, that a maximum line width of 80 characters is the convention? In case I added some more breaks for that. I just wonder what other style guides I should have respected. > Yes that is the convention. Sometimes a line can be longer if wrapping it is very ugly. In general, don't make a Sdiffs page in a webrev non-readable. A Sdiffs page example is here > > http://cr.openjdk.java.net/~bpb/8186766/webrev.00/src/java.base/unix/native/libnio/fs/UnixNativeDispatcher.c.sdiff.html > >> Just in case you think it would be more efficient I'd not object that a reviewer or some else would just change these little things to get over with it. On the other hand this way I learn it. If there is a suitable documentation that much detailed, I'd be glad if you could point me at it. That might save a few cycles. > I don't know if there is a more suitable doc. > > I have some small suggestions: > > 1. In fact I don't know exactly if a non-ASCII is allowed in source code now. For safety, please change the ? char to \u1234 style. > > 2. Maybe it's better to capitalize "b" in the test name? > > 3. The @see tag has its format, you can just use "See...". The method name in @link has a wrong signature. > > * @see also eAcute in > * {@link MultibyteUnicodeName#verifyClassNameLineBroken(String)} > > 4. Several "throws Exception" should be indented by 8 chars. > > 5. There are several trailing spaces in your patch. Maybe because of copy-and-paste. > > Please send me the new patch as an attachment. That will be more precise and more formal. Please send me what the exact "name " you want to see in the Contributed-by field of the changeset. > > Thanks > Max > >> Regards, >> Philipp -- Gruss Philipp ------------------------------------------------------------------------ Paratix GmbH St Peterhofstatt 11 8001 Z?rich +41 (0)76 397 79 35 philipp.kunz at paratix.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 5134 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: JDK-6695402.patch Type: text/x-patch Size: 12681 bytes Desc: not available URL: From weijun.wang at oracle.com Tue Sep 26 07:51:27 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 26 Sep 2017 15:51:27 +0800 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: <869f290c-bf85-d080-99b5-df5b9b59460e@paratix.ch> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> <869f290c-bf85-d080-99b5-df5b9b59460e@paratix.ch> Message-ID: > On Sep 26, 2017, at 1:37 PM, Philipp Kunz wrote: > > Hi Max > > This time I got it with readAllBytes. Thank you for the hint. > > Apparently, UTF characters are allowed in source code, particularly in identifiers here, which also has caused the bug. Even if only for sending patches around I changed it and was surprised to see escaping working not only in strings but also in identifiers. See https://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.2 I've submitted your change to our testing server. Once it's OK, I'll push the changeset. I assume "Contributed-by: Philipp Kunz " is good. BTW, there are several TAB chars and trailing spaces in your patch. I've removed them. Thanks for your contribution. --Max > > When I had another look at the test I came to the conclusion that it does not need what has been named refClassFileName before. The purpose of the test is only to check a signature of a class with a two byte character in its name and not at the same time to verify that if that test failed it is specifically because of the name. If it fails there is a problem no matter why. In the beginning it was handy to see the difference but I don't think it should be kept and maintained so I removed it. For the update signature case a second file to sign is still required though. > > I considered multi-byte a one word before but now I also prefer it with a capital b. Anyway, this name might not be the best choice and I changed it to LineBrokenMultiByteCharacter. > > See attached patch. > > Regards, > Philipp > From aph at redhat.com Tue Sep 26 08:31:42 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 26 Sep 2017 09:31:42 +0100 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: On 26/09/17 08:25, Peter Lawrey wrote: > I am looking forward to intrinsic support for 128 bit math using ?Long2? > and XMM (or even YMM, ZMM) instructions. > This is the best way forward, I hope. > > Personally I would like to see a long long type, or even uint128, uint256, > uint512 style notation. > > Another option might be something like long<128> or an annotation like > @uint128 long or even @decimal128 double but who knows. Do you actually need any of that? I think vector types make more sense. Java already has a great many scalar types. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From weijun.wang at oracle.com Tue Sep 26 11:30:41 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 26 Sep 2017 19:30:41 +0800 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> <869f290c-bf85-d080-99b5-df5b9b59460e@paratix.ch> Message-ID: Oops, the new test fails on Linux and Solaris. /scratch/test/jdk/sun/security/tools/jarsigner/LineBrokenMultiByteCharacter.java:54: error: error while writing A1234567890B1234567890C123456789D1?xyz: bad filename RelativeFile[LineBrokenMultiByteCharacter$A1234567890B1234567890C123456789D1?xyz.class] static class A1234567890B1234567890C123456789D1\u00E9xyz { } ^ 1 error I'll ask the compiler team. --Max > On Sep 26, 2017, at 3:51 PM, Weijun Wang wrote: > > >> On Sep 26, 2017, at 1:37 PM, Philipp Kunz wrote: >> >> Hi Max >> >> This time I got it with readAllBytes. Thank you for the hint. >> >> Apparently, UTF characters are allowed in source code, particularly in identifiers here, which also has caused the bug. Even if only for sending patches around I changed it and was surprised to see escaping working not only in strings but also in identifiers. > > See https://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.2 > > I've submitted your change to our testing server. Once it's OK, I'll push the changeset. > > I assume "Contributed-by: Philipp Kunz " is good. > > BTW, there are several TAB chars and trailing spaces in your patch. I've removed them. > > Thanks for your contribution. > > --Max > >> >> When I had another look at the test I came to the conclusion that it does not need what has been named refClassFileName before. The purpose of the test is only to check a signature of a class with a two byte character in its name and not at the same time to verify that if that test failed it is specifically because of the name. If it fails there is a problem no matter why. In the beginning it was handy to see the difference but I don't think it should be kept and maintained so I removed it. For the update signature case a second file to sign is still required though. >> >> I considered multi-byte a one word before but now I also prefer it with a capital b. Anyway, this name might not be the best choice and I changed it to LineBrokenMultiByteCharacter. >> >> See attached patch. >> >> Regards, >> Philipp >> > From aph at redhat.com Tue Sep 26 14:25:02 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 26 Sep 2017 15:25:02 +0100 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: On 26/09/17 11:20, Peter Lawrey wrote: > We have arrays already but we don't have primitive types of more than > 64-bit. If we had uint128 for example we wouldn't need this method. Perhaps not, but why is needing this method a problem? -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From weijun.wang at oracle.com Tue Sep 26 14:54:36 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 26 Sep 2017 22:54:36 +0800 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> <869f290c-bf85-d080-99b5-df5b9b59460e@paratix.ch> Message-ID: <32B873F3-A501-45B5-93E9-4A5B374D27FC@oracle.com> It might be a jtreg issue, but I'll have to get it resolved before pushing your changeset. --Max > On Sep 26, 2017, at 7:30 PM, Weijun Wang wrote: > > Oops, the new test fails on Linux and Solaris. > > /scratch/test/jdk/sun/security/tools/jarsigner/LineBrokenMultiByteCharacter.java:54: error: error while writing A1234567890B1234567890C123456789D1?xyz: bad filename RelativeFile[LineBrokenMultiByteCharacter$A1234567890B1234567890C123456789D1?xyz.class] > static class A1234567890B1234567890C123456789D1\u00E9xyz { } > ^ > 1 error > > I'll ask the compiler team. > > --Max > >> On Sep 26, 2017, at 3:51 PM, Weijun Wang wrote: >> >> >>> On Sep 26, 2017, at 1:37 PM, Philipp Kunz wrote: >>> >>> Hi Max >>> >>> This time I got it with readAllBytes. Thank you for the hint. >>> >>> Apparently, UTF characters are allowed in source code, particularly in identifiers here, which also has caused the bug. Even if only for sending patches around I changed it and was surprised to see escaping working not only in strings but also in identifiers. >> >> See https://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.2 >> >> I've submitted your change to our testing server. Once it's OK, I'll push the changeset. >> >> I assume "Contributed-by: Philipp Kunz " is good. >> >> BTW, there are several TAB chars and trailing spaces in your patch. I've removed them. >> >> Thanks for your contribution. >> >> --Max >> >>> >>> When I had another look at the test I came to the conclusion that it does not need what has been named refClassFileName before. The purpose of the test is only to check a signature of a class with a two byte character in its name and not at the same time to verify that if that test failed it is specifically because of the name. If it fails there is a problem no matter why. In the beginning it was handy to see the difference but I don't think it should be kept and maintained so I removed it. For the update signature case a second file to sign is still required though. >>> >>> I considered multi-byte a one word before but now I also prefer it with a capital b. Anyway, this name might not be the best choice and I changed it to LineBrokenMultiByteCharacter. >>> >>> See attached patch. >>> >>> Regards, >>> Philipp >>> >> > From aph at redhat.com Tue Sep 26 14:57:25 2017 From: aph at redhat.com (Andrew Haley) Date: Tue, 26 Sep 2017 15:57:25 +0100 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: On 26/09/17 15:53, Peter Lawrey wrote: > None, except you end up jumping through hoops to implement 128 bit > arithmetic efficiently or cleanly. At some point language support for such > a basic operation is the simplest and clearest solution. There's nothing inefficient about this approach. I don't quite see how 128-bit types help with cleanliness, because then you'd need a multiplyHigh for 128-bit types, surely? You need that for the type system to be complete. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From weijun.wang at oracle.com Wed Sep 27 01:20:47 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 27 Sep 2017 09:20:47 +0800 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: <32B873F3-A501-45B5-93E9-4A5B374D27FC@oracle.com> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> <869f290c-bf85-d080-99b5-df5b9b59460e@paratix.ch> <32B873F3-A501-45B5-93E9-4A5B374D27FC@oracle.com> Message-ID: <45B1F083-CDB3-4545-B217-24AF40231625@oracle.com> Hi Philipp The problem is that when launching by jtreg javac has difficulties writing the class file with the ? char into the file system on several OSes. I've updated the test a little. Now they are not written to files. Fortunately JarUtils can add non-existing entries. The test now passes on all our testing platforms. http://cr.openjdk.java.net/~weijun/6695402/webrev.01 If you are OK with the new version I'll push them. Thanks Max > On Sep 26, 2017, at 10:54 PM, Weijun Wang wrote: > > It might be a jtreg issue, but I'll have to get it resolved before pushing your changeset. > > --Max > >> On Sep 26, 2017, at 7:30 PM, Weijun Wang wrote: >> >> Oops, the new test fails on Linux and Solaris. >> >> /scratch/test/jdk/sun/security/tools/jarsigner/LineBrokenMultiByteCharacter.java:54: error: error while writing A1234567890B1234567890C123456789D1?xyz: bad filename RelativeFile[LineBrokenMultiByteCharacter$A1234567890B1234567890C123456789D1?xyz.class] >> static class A1234567890B1234567890C123456789D1\u00E9xyz { } >> ^ >> 1 error >> >> I'll ask the compiler team. >> >> --Max >> >>> On Sep 26, 2017, at 3:51 PM, Weijun Wang wrote: >>> >>> >>>> On Sep 26, 2017, at 1:37 PM, Philipp Kunz wrote: >>>> >>>> Hi Max >>>> >>>> This time I got it with readAllBytes. Thank you for the hint. >>>> >>>> Apparently, UTF characters are allowed in source code, particularly in identifiers here, which also has caused the bug. Even if only for sending patches around I changed it and was surprised to see escaping working not only in strings but also in identifiers. >>> >>> See https://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.2 >>> >>> I've submitted your change to our testing server. Once it's OK, I'll push the changeset. >>> >>> I assume "Contributed-by: Philipp Kunz " is good. >>> >>> BTW, there are several TAB chars and trailing spaces in your patch. I've removed them. >>> >>> Thanks for your contribution. >>> >>> --Max >>> >>>> >>>> When I had another look at the test I came to the conclusion that it does not need what has been named refClassFileName before. The purpose of the test is only to check a signature of a class with a two byte character in its name and not at the same time to verify that if that test failed it is specifically because of the name. If it fails there is a problem no matter why. In the beginning it was handy to see the difference but I don't think it should be kept and maintained so I removed it. For the update signature case a second file to sign is still required though. >>>> >>>> I considered multi-byte a one word before but now I also prefer it with a capital b. Anyway, this name might not be the best choice and I changed it to LineBrokenMultiByteCharacter. >>>> >>>> See attached patch. >>>> >>>> Regards, >>>> Philipp >>>> >>> >> > From philipp.kunz at paratix.ch Wed Sep 27 05:49:15 2017 From: philipp.kunz at paratix.ch (Philipp Kunz) Date: Wed, 27 Sep 2017 07:49:15 +0200 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: <45B1F083-CDB3-4545-B217-24AF40231625@oracle.com> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> <869f290c-bf85-d080-99b5-df5b9b59460e@paratix.ch> <32B873F3-A501-45B5-93E9-4A5B374D27FC@oracle.com> <45B1F083-CDB3-4545-B217-24AF40231625@oracle.com> Message-ID: <0ca88ad3-62f5-c7db-c434-ae3e187bc17c@paratix.ch> Hi Max Thank you for your update and its associated effort. I suggest that at least a comment should be added about why and how non-existing files can be added and the test still serves it's purpose. In fact I was quite a bit surprised when I found out that JarUtils.createJar adds the file name as contents if the file cannot be found. Ideally, we would also add some note about why this was relevant, about the test not compiling on certain oses with jtreg together with the acute but it doesn't apply any longer now. Altogether the test has become even simpler now. One more small thing that I just noticed now is that I would prefer ManifestFileName not to start with a capital letter like the other nearby variables. I thought testName was too ambiguous a name and changed it to testClassName. I also reviewed and slightly changed a few comments again. Of course it will never become perfect but now it should do. See attached patch. The contributed by is fine. I'd be glad to share credits with you and please accept more flattering for your collaboration. Regards, Philipp On 27.09.2017 03:20, Weijun Wang wrote: > Hi Philipp > > The problem is that when launching by jtreg javac has difficulties writing the class file with the ? char into the file system on several OSes. > > I've updated the test a little. Now they are not written to files. Fortunately JarUtils can add non-existing entries. The test now passes on all our testing platforms. > > http://cr.openjdk.java.net/~weijun/6695402/webrev.01 > > If you are OK with the new version I'll push them. > > Thanks > Max > >> On Sep 26, 2017, at 10:54 PM, Weijun Wang wrote: >> >> It might be a jtreg issue, but I'll have to get it resolved before pushing your changeset. >> >> --Max >> >>> On Sep 26, 2017, at 7:30 PM, Weijun Wang wrote: >>> >>> Oops, the new test fails on Linux and Solaris. >>> >>> /scratch/test/jdk/sun/security/tools/jarsigner/LineBrokenMultiByteCharacter.java:54: error: error while writing A1234567890B1234567890C123456789D1?xyz: bad filename RelativeFile[LineBrokenMultiByteCharacter$A1234567890B1234567890C123456789D1?xyz.class] >>> static class A1234567890B1234567890C123456789D1\u00E9xyz { } >>> ^ >>> 1 error >>> >>> I'll ask the compiler team. >>> >>> --Max >>> >>>> On Sep 26, 2017, at 3:51 PM, Weijun Wang wrote: >>>> >>>> >>>>> On Sep 26, 2017, at 1:37 PM, Philipp Kunz wrote: >>>>> >>>>> Hi Max >>>>> >>>>> This time I got it with readAllBytes. Thank you for the hint. >>>>> >>>>> Apparently, UTF characters are allowed in source code, particularly in identifiers here, which also has caused the bug. Even if only for sending patches around I changed it and was surprised to see escaping working not only in strings but also in identifiers. >>>> See https://docs.oracle.com/javase/specs/jls/se8/html/jls-3.html#jls-3.2 >>>> >>>> I've submitted your change to our testing server. Once it's OK, I'll push the changeset. >>>> >>>> I assume "Contributed-by: Philipp Kunz " is good. >>>> >>>> BTW, there are several TAB chars and trailing spaces in your patch. I've removed them. >>>> >>>> Thanks for your contribution. >>>> >>>> --Max >>>> >>>>> When I had another look at the test I came to the conclusion that it does not need what has been named refClassFileName before. The purpose of the test is only to check a signature of a class with a two byte character in its name and not at the same time to verify that if that test failed it is specifically because of the name. If it fails there is a problem no matter why. In the beginning it was handy to see the difference but I don't think it should be kept and maintained so I removed it. For the update signature case a second file to sign is still required though. >>>>> >>>>> I considered multi-byte a one word before but now I also prefer it with a capital b. Anyway, this name might not be the best choice and I changed it to LineBrokenMultiByteCharacter. >>>>> >>>>> See attached patch. >>>>> >>>>> Regards, >>>>> Philipp >>>>> -- Gruss Philipp ------------------------------------------------------------------------ Paratix GmbH St Peterhofstatt 11 8001 Z?rich +41 (0)76 397 79 35 philipp.kunz at paratix.ch -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 5134 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: JDK-6695402.patch Type: text/x-patch Size: 12456 bytes Desc: not available URL: From weijun.wang at oracle.com Wed Sep 27 06:12:51 2017 From: weijun.wang at oracle.com (Weijun Wang) Date: Wed, 27 Sep 2017 14:12:51 +0800 Subject: Patch for JDK-6695402: Jarsigner with multi-byte characters in class names In-Reply-To: <0ca88ad3-62f5-c7db-c434-ae3e187bc17c@paratix.ch> References: <5069dcf3-9bf9-444b-8cfe-d2df92cd765e@paratix.ch> <43EAAD13-4174-420C-A034-9261C3E9A647@oracle.com> <2BE6B2AD-1896-4F22-A948-D494BA11F33A@oracle.com> <3dfc3fe9-85e2-9107-3aec-d41806999654@paratix.ch> <843FC180-8128-4638-97B5-DEBA3C9E6FDC@oracle.com> <36dfccef-598a-56d1-8480-f4679726dca0@paratix.ch> <1A96ACFA-8AFA-4E89-AD4E-5328EF22F2C4@oracle.com> <417e192f-d42e-0b42-1f0b-ea4f28eeb5c1@paratix.ch> <4cddd988-3a3f-bc27-402c-9dddca969efe@paratix.ch> <869f290c-bf85-d080-99b5-df5b9b59460e@paratix.ch> <32B873F3-A501-45B5-93E9-4A5B374D27FC@oracle.com> <45B1F083-CDB3-4545-B217-24AF40231625@oracle.com> <0ca88ad3-62f5-c7db-c434-ae3e187bc17c@paratix.ch> Message-ID: <6DE3F1E2-31A2-4E2D-BA48-E543C3D3EC95@oracle.com> Hi Philipp The change is now at http://hg.openjdk.java.net/jdk10/master/rev/f60a42d4b8cd. I only moved the test to its new directory. Everything else unchanged. Thanks again for your contribution, looking forward for more! --Max > On Sep 27, 2017, at 1:49 PM, Philipp Kunz wrote: > > Hi Max > > Thank you for your update and its associated effort. > > I suggest that at least a comment should be added about why and how non-existing files can be added and the test still serves it's purpose. In fact I was quite a bit surprised when I found out that JarUtils.createJar adds the file name as contents if the file cannot be found. Ideally, we would also add some note about why this was relevant, about the test not compiling on certain oses with jtreg together with the acute but it doesn't apply any longer now. Altogether the test has become even simpler now. > > One more small thing that I just noticed now is that I would prefer ManifestFileName not to start with a capital letter like the other nearby variables. I thought testName was too ambiguous a name and changed it to testClassName. I also reviewed and slightly changed a few comments again. Of course it will never become perfect but now it should do. See attached patch. > > The contributed by is fine. I'd be glad to share credits with you and please accept more flattering for your collaboration. > > Regards, > Philipp > From aph at redhat.com Wed Sep 27 13:44:50 2017 From: aph at redhat.com (Andrew Haley) Date: Wed, 27 Sep 2017 14:44:50 +0100 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: <16902fb8-bf7d-d9fd-ee6f-d2f7eaa24abd@redhat.com> On 27/09/17 11:28, Peter Lawrey wrote: > If you need multiplyHigh for 128-bit then you need uint256. At some point > you have to decide whether you need that many bits as a supported > operation. When Java was created a 64-bit long as the widest type made > sense, however CPUs such as x64 now support 128, 256 and 512 bit natively > and having the JVM dong its best to work this out is not as clean as > defining it explicitly. I guess cleanliness is in the eye of the beholder. IMO multiplyHigh is as clean as we need, and I'd rather see more complexity there than in the type system. It'd be nice to be able to return more than one scalar value from a method, for sure. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From peter.lawrey at gmail.com Tue Sep 26 07:25:35 2017 From: peter.lawrey at gmail.com (Peter Lawrey) Date: Tue, 26 Sep 2017 08:25:35 +0100 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: I am looking forward to intrinsic support for 128 bit math using ?Long2? and XMM (or even YMM, ZMM) instructions. This is the best way forward, I hope. Personally I would like to see a long long type, or even uint128, uint256, uint512 style notation. Another option might be something like long<128> or an annotation like @uint128 long or even @decimal128 double but who knows. Regards, Peter. ? On 25 September 2017 at 18:48, Andrew Haley wrote: > On 25/09/17 18:21, Adam Petcher wrote: > > I agree that an unsigned multiplyHigh would be useful for crypto > > purposes, and we should consider adding it. Of course, I would much > > rather have multiply operations that return both 64-bit parts of the > > result, but that is going to be hard to do well without value types. So > > it would be nice to have something like this in the meantime. > > I take your point, but it won't be excruciatingly difficult for the C2 > compiler to turn the multiply operations into a single one, if the CPU > can do that. From what I've seen recently, though, on non-x86 it's > common for the two halves of the result to be calculated by separate > instructions. > > > If we are going to add this operation, it should probably be added > > along with an intrinsic. I think the Java code can simply factor out > > the else branch from the existing multiplyHigh code. This way, > > unsignedMultiplyHigh will be at least as fast as multiplyHigh, > > whether the intrinsic implementation is available or not. > > Sure. I can do that. > > > If possible, the implementation of this operation should not branch on > > either operand. This would make it more widely useful for constant-time > > crypto implementations. Though this property would need to go into the > > spec in order for constant-time crypto code to use this method, and I > > don't know how reasonable it is to put something like this in the spec. > > OK. I can do it so that there are no branches in the Java. The Java > code for signed multiplyHigh has some data-dependent branches in an > attempt to speed it up, though. I don't know how effective they are, > and I could have a look at taking them out. > > > Side note: at the moment, I am using signed arithmetic in prototypes for > > Poly1305, X25519, and EdDSA, partially due to lack of support for > > unsigned operations like this one. I don't think having > > unsignedMultiplyHigh would, on its own, convince me to use an unsigned > > representation, but the forces are different for each > > algorithm/implementation. > > Sure. I don't think it really matters from a performance point of > view which you use, given intrinsics for both. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.lawrey at gmail.com Tue Sep 26 10:20:59 2017 From: peter.lawrey at gmail.com (Peter Lawrey) Date: Tue, 26 Sep 2017 11:20:59 +0100 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: We have arrays already but we don't have primitive types of more than 64-bit. If we had uint128 for example we wouldn't need this method. On 26 Sep. 2017 11:31, "Andrew Haley" wrote: > On 26/09/17 08:25, Peter Lawrey wrote: > > I am looking forward to intrinsic support for 128 bit math using ?Long2? > > and XMM (or even YMM, ZMM) instructions. > > This is the best way forward, I hope. > > > > Personally I would like to see a long long type, or even uint128, > uint256, > > uint512 style notation. > > > > Another option might be something like long<128> or an annotation like > > @uint128 long or even @decimal128 double but who knows. > > Do you actually need any of that? I think vector types make more sense. > Java already has a great many scalar types. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.lawrey at gmail.com Tue Sep 26 14:53:29 2017 From: peter.lawrey at gmail.com (Peter Lawrey) Date: Tue, 26 Sep 2017 15:53:29 +0100 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: None, except you end up jumping through hoops to implement 128 bit arithmetic efficiently or cleanly. At some point language support for such a basic operation is the simplest and clearest solution. On 26 Sep. 2017 17:25, "Andrew Haley" wrote: > On 26/09/17 11:20, Peter Lawrey wrote: > > We have arrays already but we don't have primitive types of more than > > 64-bit. If we had uint128 for example we wouldn't need this method. > > Perhaps not, but why is needing this method a problem? > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.lawrey at gmail.com Wed Sep 27 10:28:19 2017 From: peter.lawrey at gmail.com (Peter Lawrey) Date: Wed, 27 Sep 2017 11:28:19 +0100 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: Message-ID: If you need multiplyHigh for 128-bit then you need uint256. At some point you have to decide whether you need that many bits as a supported operation. When Java was created a 64-bit long as the widest type made sense, however CPUs such as x64 now support 128, 256 and 512 bit natively and having the JVM dong its best to work this out is not as clean as defining it explicitly. ? On 26 September 2017 at 15:57, Andrew Haley wrote: > On 26/09/17 15:53, Peter Lawrey wrote: > > None, except you end up jumping through hoops to implement 128 bit > > arithmetic efficiently or cleanly. At some point language support for > such > > a basic operation is the simplest and clearest solution. > > There's nothing inefficient about this approach. I don't quite see > how 128-bit types help with cleanliness, because then you'd need a > multiplyHigh for 128-bit types, surely? You need that for the type > system to be complete. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.lawrey at gmail.com Wed Sep 27 15:25:32 2017 From: peter.lawrey at gmail.com (Peter Lawrey) Date: Wed, 27 Sep 2017 18:25:32 +0300 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: <16902fb8-bf7d-d9fd-ee6f-d2f7eaa24abd@redhat.com> References: <16902fb8-bf7d-d9fd-ee6f-d2f7eaa24abd@redhat.com> Message-ID: Indeed cleanliness is in the eye of the beholder. ;) I feel for mathematical code like this it should be possible to write something as fast and clear as C++, whether that is desirable or not is another matter. Perhaps project Valhalla will be a way to return multiple values by having a composite value type, or panama with it's support for XMM instructions (or both) ? On 27 September 2017 at 16:44, Andrew Haley wrote: > On 27/09/17 11:28, Peter Lawrey wrote: > > If you need multiplyHigh for 128-bit then you need uint256. At some point > > you have to decide whether you need that many bits as a supported > > operation. When Java was created a 64-bit long as the widest type made > > sense, however CPUs such as x64 now support 128, 256 and 512 bit natively > > and having the JVM dong its best to work this out is not as clean as > > defining it explicitly. > > I guess cleanliness is in the eye of the beholder. IMO multiplyHigh is > as clean as we need, and I'd rather see more complexity there than in > the type system. It'd be nice to be able to return more than one scalar > value from a method, for sure. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Scott-Di.Su at sc.com Tue Sep 26 06:41:02 2017 From: Scott-Di.Su at sc.com (Su, Scott Di) Date: Tue, 26 Sep 2017 06:41:02 +0000 Subject: enable TLS_RSA_WITH_AES_256_CBC_SHA256 in openJDK(build 1.8.0_121-b13) Message-ID: Dear security dev, We are upgrading our app to run it from JDK 5 to JDK 8, and encounter TLS_RSA_WITH_AES_256_CBC_SHA256 not supported error, I have copy JCE download from Oracle website, any change needed to enable it, thanks in advance! Thanks & Regards, Scott This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https://www.sc.com/en/incorporation-details.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.r.rose at oracle.com Wed Sep 27 18:45:24 2017 From: john.r.rose at oracle.com (John Rose) Date: Wed, 27 Sep 2017 11:45:24 -0700 Subject: Do we need an unsigned multiplyHigh? In-Reply-To: References: <16902fb8-bf7d-d9fd-ee6f-d2f7eaa24abd@redhat.com> Message-ID: <97F9D736-E850-4C83-A185-22191658F279@oracle.com> On Sep 27, 2017, at 8:25 AM, Peter Lawrey wrote: > > Perhaps project Valhalla will be a way to return multiple values by having > a composite value type, or panama with it's support for XMM instructions > (or both) That seems likely; we are aiming in both of those directions, to support direct programming with AVX-type registers (not just x86 specific, by the way) and general support for by-value return of structured objects (starting with "minimal value types"). For now, though, every method is limited to at most 64 bits of return value "payload", which means that 128-bit operations need to be split into two method calls, or else buffer their result into a temp object (e.g., array). The JIT knows how to combine two intrinsic calls into a single machine operation, in some very limited circumstances, notably the classic "div/rem" instructions. This technique would probably work for 64-to-128 multiplies. (Also AES-128, by the way.) ? John -------------- next part -------------- An HTML attachment was scrubbed... URL: From ecki at zusammenkunft.net Wed Sep 27 21:34:03 2017 From: ecki at zusammenkunft.net (Bernd Eckenfels) Date: Wed, 27 Sep 2017 21:34:03 +0000 Subject: enable TLS_RSA_WITH_AES_256_CBC_SHA256 in openJDK(build 1.8.0_121-b13) In-Reply-To: References: Message-ID: You need to install the unrestricted Cryptography policy files for JCE to enable AES255 Gruss Bernd -- http://bernd.eckenfels.net ________________________________ From: security-dev on behalf of Su, Scott Di Sent: Tuesday, September 26, 2017 7:41:02 AM To: security-dev at openjdk.java.net Subject: enable TLS_RSA_WITH_AES_256_CBC_SHA256 in openJDK(build 1.8.0_121-b13) Dear security dev, We are upgrading our app to run it from JDK 5 to JDK 8, and encounter TLS_RSA_WITH_AES_256_CBC_SHA256 not supported error, I have copy JCE download from Oracle website, any change needed to enable it, thanks in advance! Thanks & Regards, Scott This email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please delete all copies and notify the sender immediately. You may wish to refer to the incorporation details of Standard Chartered PLC, Standard Chartered Bank and their subsidiaries at https://www.sc.com/en/incorporation-details.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From sean.coffey at oracle.com Thu Sep 28 10:05:59 2017 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 28 Sep 2017 11:05:59 +0100 Subject: enable TLS_RSA_WITH_AES_256_CBC_SHA256 in openJDK(build 1.8.0_121-b13) In-Reply-To: References: Message-ID: <1bf0cbe4-3702-3f5f-c374-d0f7e2b26245@oracle.com> Check out the Oracle 8u152 early access binaries[1] also. There's a new security property to enable unlimited crypto. No more policy file downloads required! You can simply edit the jre/lib/java.security file to set "crypto.policy" to "unlimited" [2] 8u152 GA had a tentative release date for late October 2017. [1] http://jdk.java.net/8/ [2] https://bugs.openjdk.java.net/browse/JDK-8157561 regards, Sean. On 27/09/2017 22:34, Bernd Eckenfels wrote: > You need to install the unrestricted Cryptography policy files for JCE > to enable AES255 > > Gruss > Bernd > -- > http://bernd.eckenfels.net > ------------------------------------------------------------------------ > *From:* security-dev on behalf > of Su, Scott Di > *Sent:* Tuesday, September 26, 2017 7:41:02 AM > *To:* security-dev at openjdk.java.net > *Subject:* enable TLS_RSA_WITH_AES_256_CBC_SHA256 in openJDK(build > 1.8.0_121-b13) > > Dear security dev, > > We are upgrading our app to run it from JDK 5 to JDK 8, and encounter > TLS_RSA_WITH_AES_256_CBC_SHA256 not supported error, I have copy JCE > download from Oracle website, any change needed to enable it, thanks > in advance! > > Thanks & Regards, > > Scott > > > This email and any attachments are confidential and may also be > privileged. If you are not the intended recipient, please delete all > copies and notify the sender immediately. You may wish to refer to the > incorporation details of Standard Chartered PLC, Standard Chartered > Bank and their subsidiaries at > https://www.sc.com/en/incorporation-details.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbalao at redhat.com Thu Sep 28 15:09:51 2017 From: mbalao at redhat.com (Martin Balao) Date: Thu, 28 Sep 2017 11:09:51 -0400 Subject: Code Review Request: JDK-6491070 (Support for RFC 5929-Channel Bindings) In-Reply-To: References: <97c02f80-88aa-d0a7-f1e0-ba80b3a3b71d@oracle.com> <1c51ce46-d0e1-1ab8-6e9e-a71e649ff323@oracle.com> Message-ID: Hi, Renegotiation requirements: * As a client: * Do not proceed upon a received HelloRequest message * If a server sends a HelloRequest message, the client will notice it only when reading from the socket/engine. Because a HelloRequest message is a handshake protocol message (Record.ct_handshake), the client will call initHandshaker (SSLSocketImpl.java / SSLEngineImpl.java). This will trigger a renegotiation, initializating a new Handshaker and proceeding. The proposal would be to check if any "handshake renegotiation event" callback was registered and, if any, call it to let the API client decide how to proceed (instead of just initializating a Handshaker and moving forward). * Do not send ClientHello messages * Once a connection is established (i.e.: connectionState == cs_DATA), the client does not initializate a new Handshaker by its own initiative. * As a server: * Do not send HelloRequest messages * Once a connection is established (i.e.: connectionState == cs_DATA), the server does not initializate a new Handshaker by its own initiative. * Do not handle ClientHello messages * This case is analogous to the case in which a Client receives a HelloRequest from a server. The proposal would be the same. The interface for a SSLSocket/SSLEngine API client to have control over renegotiations would be registering a callback. This callback will be synchronously called when the counterpart wants a renegotiation, and a true/false result is expected to decide whether to continue the renegotiation or dismiss it. In regard to the callback interface, "addHandshakeCompletedListener" cannot be used because it's for async notifications. The API client cannot stop the renegotiation flow unless a synch call is done. I've noticed that "setHandshakeApplicationProtocolSelector" method was added with a similar purpose than ours. So, I see two options: * Add an ad-hoc set/get callback method for handshake renegotiations * Add a generic interface for handshake event callbacks (synchronously called). It's unfortunate that "setHandshakeApplicationProtocolSelector" refactoring would require a hard API change at this point and thus unlikely. But we can prevent future method explosion from now on. The downside of this approach is that we would need to deal with callbacks that have different parameter and return types. Does this work for you? Kind regards, Martin.- PS: sorry for the delay. On Fri, Sep 1, 2017 at 10:08 AM, David Lloyd wrote: > I can say that from the perspective of SASL that we don't need any > special indication about handshake, and it would be much easier to > just read the material off of the engine, socket, or session as > appropriate. > > On Thu, Aug 31, 2017 at 8:32 PM, Xuelei Fan wrote: > > The failure-and-retry mechanism could a nightmare for some applications. > > Please think more how could we avoid it. If need more APIs, what the > update > > may looks like and how complicated it could be? > > > > If required, Bernd's proposal can be extended a little bit to support > > operations other than listening. > > > > APIs maintain is very complicated, a good design may need more time > > currently, but could save much more in the future. > > > > Thanks, > > Xuelei > > > > On 8/31/2017 11:41 AM, Martin Balao wrote: > >> > >> Hi, > >> > >> The material is already cached in the handshaker if secure renegotiation > >> is enabled. However, I agree with you that we are going to cache the > value > >> even when secure renegotiation is not enabled, thus, wasting roughly 12 > >> bytes (as bytes for an empty array are already consumed). In fact, the > exact > >> case -adding a few conditionals to webrev.02- is the one in which secure > >> renegotiation is disabled and extended master secret is enabled. GnuTls > and > >> OpenSSL -including its derivatives like Boring SSL and Python (through > >> OpenSSL)- always cache this information. > >> > >> As for the empty / null cases, the API consumer was expected to ask for > >> the binding information after the TLS connection is established. It's > on the > >> API consumer knowledge that asking for the information before (i.e.: > just > >> after creating a disconnected socket) or while the handshake is taking > >> place, makes no sense and no valid value will be obtained (either we > define > >> this as null or empty). For those providers that do not support this > >> feature, the information wouldn't have been available after the > handshake. > >> However, I agree with you that before the handshake is completed there > is no > >> means of knowing if the provider does support this API. My first webrev > >> (webrev.01) was throwing an UnsupportedOperationException to make this > case > >> explicit but I had doubts regarding the real value it provides for the > API > >> consumer. The proposed API was similar to Python, SSLBoring and GnuTLs. > >> However, handshake listener callbacks -as Bernd suggests- and the idea > of > >> just exposing the handshake material (as a lower level API) sounds good > to > >> me. I can give it a try. I propose then to bring the handshake > information > >> as part of a HandshakeCompletedEvent instance, even though the callback > >> registrant may not consume it. Would that work for you? > >> > >> In regard to the handshake material update -which I assumed unlikely-, > the > >> point in which a renegotiation may take place (from the server side) is > when > >> reading data, not when writing. That cannot be controlled by the > application > >> because it's JSSE internal and not exposed. Thus, an application may > read > >> from the socket, get the handshake material and write a message using > the > >> binding value -which we can make sure that is the valid one at that > point-. > >> However, as soon as the application reads again from the socket, a > >> renegotiation -if requested by the client- may be processed and the > binding > >> value gets updated. The higher level protocol may fail -because the > binding > >> value was already sent but not processed on the other side- and a re-try > >> needed. This looks independent of whether we use the originally > proposed API > >> or the handshake listener callback interface (or even a sync callback), > >> because the underlying problem is that the application cannot really > control > >> the renegotiation flow in the lower layer (as RFC 5929 suggest). The > options > >> I see are adding more complexity to the API and let the application > control > >> the renegotiation flow or live with that and expect the application to > >> retry. > >> > >> Thanks, > >> Martin.- > >> > >> On Tue, Aug 29, 2017 at 4:34 PM, Xuelei Fan >> > wrote: > >> > >> On 8/26/2017 2:56 PM, Bernd Eckenfels wrote: > >> > How about only passing it to an extended handshake listener. The > >> > material does not have to be cached (the app can do it if needed) > >> and > >> > renegotiation works the same way. This can also be helpful for > >> things > >> > like logging the master secret (for wireshark decryption). It can > >> even > >> > handle auditing of session resumptions > >> Martin, what do you think about Bernd's proposal above and similar > >> callback mechanism? > >> > >> More comment inlines. > >> > >> On 8/29/2017 11:50 AM, Martin Balao wrote: > >> > >> Hi Xuelei, > >> > >> >There are a few protocols that can benefits from exporting > >> SSL/TLS handshake materials, including RFC 5929, RFC 5056, token > >> binding and TLS 1.3 itself. Can we define a general API so as > >> to exposing the handshake materials, so as to mitigate the > >> inflating of JSSE APIs? I may suggest make further evaluation > >> before move on to following design and code. > >> > >> Do you prefer an API like "public byte[] > >> getTlsHandshakeMaterial(String materialType)" (in SSLSocket and > >> SSLEngine) where "materialType" can eventually be > >> "clientFinishedMessage"/"finishedMessage" or > >> "serverFinishedMessage"/"peerFinishedMessage"? > >> > >> The problem of the APIs like that is, when applications call the > >> method, it is not always return the expected result, and the > >> implementation may have to cache the message even if an application > >> never use it. See more in the following example. > >> > >> I cannot think of "serverCertificate" or "masterKey" as this is > >> more related to a Session and not neccessarily to a handshake. > >> getTlsHandshakeMaterial would be a lower level API and would > >> move the burden of knowing which information is required for > >> "tls-unique" TLS channel binding to the API consumer. Looks more > >> like the OpenSSL approach (instead of the Python, SSLBoring or > >> GnuTls approaches). However, OpenSSL have specific methods for > >> each piece of information instead of a generic and parametrized > >> one. I.e.: SSL_get_finished or SSL_get_peer_finished. What other > >> information do you expect the Handshaker to provide? > >> > >> >The SunJSSE provider happens to cache the finished messages > >> in its implementation so you can use it for tls-unique, but it > >> may not be true for other provider or other channel bindings. > >> Need to define a more reliable approach to get the handshake > >> materials. > >> > >> I focused on SunJSSE provider. I'm not sure about how other > >> providers may implement this API and where they can get the > >> required information from, without knowing their internals. In > >> regard to SunJSSE and "tls-unique" binding type, I leveraged on > >> existing data. If data weren't already there, I would have to > >> figure out how to get it from the handshake -doing the same that > >> was already done would have been an option-. Do you prefer the > >> Handshaker to provide a function to get different information > >> and not just the finished hash? (as for the public > >> SSLSocket/SSLEngine "getTlsHandshakeMaterial" API). Which other > >> information may be useful to get from the Handshaker? What do > >> you mean by reliable? (given that this is all SunJSSE internal > >> and we have no external dependencies). > >> > >> Let consider the use of the API. > >> byte[] getTlsChannelBinding("tls_unique"); > >> > >> I'm confusing when I try to use it by myself: > >> 1. provider does not implement this method > >> return null or empty? > >> > >> It happens because an old provider should still work in new JDK, but > >> old provider does not implement new APIs, or a new provider does not > >> support this feature. > >> > >> 2. the method is called before handshaking > >> return null or empty? > >> > >> 3. the method is called during handshaking > >> return null, empty or the channel binding value? > >> > >> 4. the method is called at the same time the handshaking completed? > >> return the channel binding value? > >> > >> 5. the method is called after the handshaking > >> return the channel binding value? > >> > >> 6. the method is called during renegoitation > >> return null, empty, the old binding value, or the new binding > >> value? > >> > >> 7. the method is called after handshaking > >> return old binding value, or the new binding value? > >> > >> 8. the method is called after the initial handshaking, but the > >> binding value is changed shortly after because of renegotiation. > >> how could application use the binding value? > >> > >> We need a clear define of the behavior of the method. It could be > >> complicated if the method is designed as > >> getTlsChannelBinding("tls_unique"). > >> > >> I feel that handshake material should be captured when > >> 1. it is requested to capture the handshake material, and > >> 2. the handshake material get produced. > >> > >> For the getTlsChannelBinding("tls_unique") API, it is unknown: > >> 1. Is it required to capture the handshake material? > >> 2. Is the handshake material produced? > >> > >> The two points could result in a few unexpected problems, as the > >> above 8 items that we may want to consider. > >> > >> In regard to other channel bindings, it'll depend on the binding > >> type the way in which the information is obtained. I.e.: > >> "tls-unique" SunJSSE implementation leverages on cached finished > >> messages. However, "tls-server-end-point" leverages on stored > >> certificates that are obtained from the Session (not from the > >> handshaker). Is there any specific channel binding you are > >> concerned with? > >> > >> >If the channel binding is not required, it may be not > >> necessary to expose the handshake materials. Need to define a > >> solution to indicate the need of the exporting. > >> > >> Do you mean a lower layer knowing if the upper layer is going to > >> require that information and decide to provide it or not based > >> on that knowledge? I think I didn't get your point here. > >> > >> I mean, if an application want to support channel binding, the > >> provider can provider the channel binding service; If the an > >> application does not want channel binding, the provider should be > >> perform the channel binding service. The getTlsChannelBinding() > >> make the provider MUST perform channel binding cache or calculation > >> no matter application want it or not. > >> > >> >2. No way to know the update of the underlying handshake > >> materials. > >> >If renegotiation can takes place, need to define a interface > >> to indicate that so that application can response accordingly. > >> See section 3 and 7 of RFC 5929. > >> > >> I intentionally skipped this -at the cost of a spurious > >> authentication- to avoid adding complexity to the API. An > >> spurious authentication -which does not appear likely to me- can > >> easily be retried by the application. The RFC 5929 suggests APIs > >> through which the application can *control* the flow (i.e.: hold > >> a renegotitation). This would expose JSSE internals. This is > >> more than notifying. Notification, in my opinion, adds no value: > >> what if the application already used the binding token before > >> receiving the notification? The spurious authentication will > >> happen anyways and has to be handled -i.e. retried-. It's just a > >> timing issue. The real value is controlling the flow as the RFC > >> suggests, but at the cost of exposing JSSE internals. > >> > >> My understanding, the block of the protocol is to make sure > >> application is performing the channel binding with the right value, > >> or updating the value accordingly if necessary. If you skip this > >> and when renegotiation happen, the channel binding could be limited, > >> or may not work as expected. > >> > >> Thanks, > >> Xuelei > >> > >> Kind regards, > >> Martin.- > >> > >> > >> On Sat, Aug 26, 2017 at 5:25 PM, Xuelei Fan > >> > >> >> > >> > >> wrote: > >> > >> Hi Marin, > >> > >> Sorry for the delay. > >> > >> There are a few protocols that can benefits from exporting > >> SSL/TLS > >> handshake materials, including RFC 5929, RFC 5056, token > >> binding and > >> TLS 1.3 itself. Can we define a general API so as to > >> exposing the > >> handshake materials, so as to mitigate the inflating of > >> JSSE APIs? I may suggest make further evaluation before move > >> on to following > >> design and code. > >> > >> > > >> > >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK- > 6491070/webrev.02/ > >> > >> 6491070/webrev.02/> > >> > >> 6491070/webrev.02/ > >> > >> 6491070/webrev.02/>> > >> I have two concerns about the design: > >> > >> 1. Channel binding may be not always required. > >> SSLSocket/SSLEngine.getTlsChannelBinding(String > bindingType); > >> > >> The SunJSSE provider happens to cache the finished messages > >> in its > >> implementation so you can use it for tls-unique, but it may > >> not be > >> true for other provider or other channel bindings. Need to > >> define a > >> more reliable approach to get the handshake materials. > >> > >> If the channel binding is not required, it may be not > >> necessary to > >> expose the handshake materials. Need to define a solution > to > >> indicate the need of the exporting. > >> > >> 2. No way to know the update of the underlying handshake > >> materials. > >> If renegotiation can takes place, need to define a > interface > >> to > >> indicate that so that application can response > >> accordingly. See > >> section 3 and 7 of RFC 5929. > >> > >> Thanks, > >> Xuelei > >> > >> On 7/31/2017 8:53 AM, Martin Balao wrote: > >> > >> Hi, > >> > >> Here it is an update for the proposed TLS Channel > >> Bindings > >> support in OpenJDK: > >> > >> * > >> > >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK- > 6491070/webrev.02/ > >> > >> 6491070/webrev.02/> > >> > >> 6491070/webrev.02/ > >> > >> 6491070/webrev.02/>> > >> (browse online) > >> * > >> > >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK- > 6491070/webrev.02/6491070.webrev.02.zip > >> > >> 6491070/webrev.02/6491070.webrev.02.zip> > >> > >> 6491070/webrev.02/6491070.webrev.02.zip > >> > >> 6491070/webrev.02/6491070.webrev.02.zip>> > >> (download) > >> > >> Changes since v01: > >> > >> * getTlsChannelBinding API changed to return null by > >> default > >> (if not implemented), instead of throwing an > >> UnsupportedOperationException. > >> > >> * "tls-server-end-point" TLS channel binding now > >> supported. > >> > >> Kind regards, > >> Martin.- > >> > >> On Wed, Jul 26, 2017 at 4:12 PM, Martin Balao > >> > >> > > >> > >> > >> >>>> > >> > >> wrote: > >> > >> Hi, > >> > >> Here it is my proposal for JDK-6491070 (Support > >> for RFC > >> 5929-Channel > >> Bindings: e.g. public API to obtain TLS finished > >> message) [1]: > >> > >> * > >> > >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK- > 6491070/webrev.01/ > >> > >> 6491070/webrev.01/> > >> > >> 6491070/webrev.01/ > >> > >> 6491070/webrev.01/>> > >> > >> 6491070/webrev.01/ > >> > >> 6491070/webrev.01/> > >> > >> 6491070/webrev.01/ > >> > >> 6491070/webrev.01/>>> > >> * > >> > >> http://cr.openjdk.java.net/~sgehwolf/webrevs/mbalaoal/JDK- > 6491070/webrev.01/6491070.webrev.01.zip > >> > >> 6491070/webrev.01/6491070.webrev.01.zip> > >> > >> 6491070/webrev.01/6491070.webrev.01.zip > >> > >> 6491070/webrev.01/6491070.webrev.01.zip>> > >> > >> 6491070/webrev.01/6491070.webrev.01.zip > >> > >> 6491070/webrev.01/6491070.webrev.01.zip> > >> > >> 6491070/webrev.01/6491070.webrev.01.zip > >> > >> 6491070/webrev.01/6491070.webrev.01.zip>>> > >> > >> Notes: > >> * Implementation based on Channel Bindings for > >> TLS (RFC > >> 5929) [2] > >> > >> * Only "tls-unique" currently supported > >> > >> Look forward to your comments. > >> > >> Kind regards, > >> Martin.- > >> > >> -- > >> [1] - > >> https://bugs.openjdk.java.net/browse/JDK-6491070 > >> > >> >> > > >> >> > >> >> >> > >> [2] - https://tools.ietf.org/html/rfc5929 > >> > >> >> > > >> >> > >> >> >> > >> > >> > >> > >> > > > > > > -- > - DML > -------------- next part -------------- An HTML attachment was scrubbed... URL: