[security-dev 00236]: Re: NullPointerException at sun.security.ssl.OutputRecord.writeBuffer

[Kanatoko]

Andrew,

I figured out why sockOutput comes to null.

SSLSocketImpl class is used both from SSLSocket class and
SSLServerSocket class.

The problem occurs when SSLServerSocket instance uses(creates)
SSLSocketImpl
instance only for getting ServerHandshaker instance. The code is below.

-----From share/classes/sun/security/ssl/SSLServerSocketImpl.java

    private void checkEnabledSuites() throws IOException {
        //
        // We want to report an error if no cipher suites were actually
        // enabled, since this is an error users are known to make.  Then
        // they get vastly confused by having clients report an error!
        //
        synchronized (this) {
            if (checkedEnabled) {
                return;
            }
            if (useServerMode == false) {
                return;
            }

            SSLSocketImpl tmp = new SSLSocketImpl(sslContext, useServerMode,            <=================== HERE!
                         enabledCipherSuites, doClientAuth,
                         enableSessionCreation, enabledProtocols);

            ServerHandshaker handshaker = tmp.getServerHandshaker();             

            for (Iterator t = enabledCipherSuites.iterator(); t.hasNext(); ) {
                CipherSuite suite = (CipherSuite)t.next();
                if (handshaker.trySetCipherSuite(suite)) {
                    checkedEnabled = true;
                    return;
                }
            }
-----

This SSLSocketImpl instance 'tmp' does not handle any TCP( or SSL ) connections, so sockOutput is always null.
This method 'checkEnabledSuites()' is called only once for each SSLServerSocket instances, So If you instantiates more SSLServerSockets, more file handle leak occurs.

I think we should patch SSLSocketImpl class. But I don't have a good fix.

-- 
Kanatoko<anvil at jumperz.net>
Open Source WebAppFirewall
http://guardian.jumperz.net/


> Hi Kanatoko,
> 
> I need more time to check why sockOutput comes to null. It is OK to 
> break if the sockOutput is null just as your patch. However, it is never 
> expected to be null, so I wanna take more time to dig it further. Could 
> I ask you help if I need more information?
> 
> Thanks,
> Andrew
> 
> Kanatoko wrote:
> > Hi Andrew,
> > Thank you very much for your reply.
> >
> >   
> >> why the updates is 
> >> necessary?
> >>     
> >
> > This causes file descriptor leak, and denial of service as a result,  on
> > long running servers ( As I wrote in  [security-dev 00204]  ).
> >
> > Now I am developing some kind of SSL proxy server. But because of this
> > issue, it can't keep running for long time. 
> >
> >
> >   
> >> And what's the use case that the OutputStream 's' would be null?
> >>     
> >
> > I'm sorry I don't know.
> > 's' seems to be 'sockOutput' in SSLSocketImpl class. But I don't know
> > why sometimes it would be null.
> >
> > Thanks.
> >
> >   
>