Issue with SSL handshake implementation

[chengjingwei (A)]

Hi,



There may be some issue with the SSL handshake implementation.



In sun.security.x509.X500Name.java, there's a static hashmap field named *internedOIDs*, used for caching X.500 attributes.

Each time a new oid is encountered, jdk will cache them into the static hashmap.



With a purposely crafted cert on the client side, it's possible to create some long oids, and letting the server

save them permanently during SSL handshake, which will eat server memory, and cause OOM & DoS in the end.



Is it necessary to make some changes to this?





BTW, The issue was reported by one of our customers, with the following stacktrace.

Although they got this on jdk8u212, I believe the same issue exists in jdk11 too.



```

http-bio-172.24.18.21-443-exec-4

at java.lang.OutOfMemoryError.<init>()V (OutOfMemoryError.java:48)

at java.util.Arrays.copyOf([Ljava/lang/Object;I)[Ljava/lang/Object; (Arrays.java:3181)

at java.util.Vector.grow(I)V (Vector.java:269)

at java.util.Vector.ensureCapacityHelper(I)V (Vector.java:249)

at java.util.Vector.addElement(Ljava/lang/Object;)V (Vector.java:623)

at sun.security.util.DerInputStream.readVector(I)[Lsun/security/util/DerValue; (DerInputStream.java:425)

at sun.security.util.DerInputStream.getSequence(I)[Lsun/security/util/DerValue; (DerInputStream.java:331)

at sun.security.x509.X500Name.parseDER(Lsun/security/util/DerInputStream;)V (X500Name.java:793)

at sun.security.x509.X500Name.<init>(Lsun/security/util/DerInputStream;)V (X500Name.java:306)

at sun.security.x509.X509CertInfo.parse(Lsun/security/util/DerValue;)V (X509CertInfo.java:649)

at sun.security.x509.X509CertInfo.<init>(Lsun/security/util/DerValue;)V (X509CertInfo.java:167)

at sun.security.x509.X509CertImpl.parse(Lsun/security/util/DerValue;)V (X509CertImpl.java:1804)

at sun.security.x509.X509CertImpl.<init>([B)V (X509CertImpl.java:195)

at sun.security.provider.X509Factory.engineGenerateCertificate(Ljava/io/InputStream;)Ljava/security/cert/Certificate; (X509Factory.java:102)

at java.security.cert.CertificateFactory.generateCertificate(Ljava/io/InputStream;)Ljava/security/cert/Certificate; (CertificateFactory.java:339)

at sun.security.ssl.HandshakeMessage$CertificateMsg.<init>(Lsun/security/ssl/HandshakeInStream;)V (HandshakeMessage.java:455)

at sun.security.ssl.ServerHandshaker.processMessage(BI)V (ServerHandshaker.java:230)

at sun.security.ssl.Handshaker.processLoop()V (Handshaker.java:1037)

at sun.security.ssl.Handshaker.process_record(Lsun/security/ssl/InputRecord;Z)V (Handshaker.java:965)

at sun.security.ssl.SSLSocketImpl.readRecord(Lsun/security/ssl/InputRecord;Z)V (SSLSocketImpl.java:1064)

at sun.security.ssl.SSLSocketImpl.performInitialHandshake()V (SSLSocketImpl.java:1367)

at sun.security.ssl.SSLSocketImpl.startHandshake(Z)V (SSLSocketImpl.java:1395)

at sun.security.ssl.SSLSocketImpl.getSession()Ljavax/net/ssl/SSLSession; (SSLSocketImpl.java:2288)

at org.apache.tomcat.util.net.jsse.JSSESocketFactory.handshake(Ljava/net/Socket;)V (JSSESocketFactory.java:293)

at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run()V (JIoEndpoint.java:343)

at java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:624)

at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run()V (TaskThread.java:61)

at java.lang.Thread.run()V (Thread.java:748)

```