select() returns empty key set

[Robert Larsen]

Hi all

I am having a problem with a set of NIO based servers. They work perfectly fine most of the time, but some of them mysteriously end up using 100% CPU. I have debugged the problem and found that a select(1000) call returns immediately but returns 0.
The JavaDoc says "It returns only after at least one channel is selected, this selector's wakeup method is invoked, the current thread is interrupted, or the given timeout period expires, whichever comes first. "

* No channel is selected
* I do not use wakeup() except from when I shut down the processes
* I do not interrupt threads and deeper debugging seems to confirm that the thread is not interrupted.
* The timeout has not expired

My code looked like this:
    public void runServer() {
        running = true;
        while (running) {
            try {
                selector.select(1000);
                Iterator<SelectionKey> i = selector.selectedKeys().iterator();
                while (i.hasNext()) {
                    SelectionKey key = i.next();
                    i.remove();

                    if (key.isValid() && key.isAcceptable()) {
                        ((ConnectionHandler)key.attachment()).handleAccept(key);
                    }
                    if (key.isValid() && key.isReadable()) {
                        //Existing client either lost connection
                        //or sent data
                        ((ConnectionHandler)key.attachment()).handleRead(key);
                    }
                    if (key.isValid() && key.isWritable()) {
                        ((ConnectionHandler)key.attachment()).handleWrite(key);
                    }
                    if (key.isValid() && key.isConnectable()) {
                        ((ConnectionHandler)key.attachment()).handleConnect(key);
                    }
                }
            } catch (CancelledKeyException cke) {
            } catch (Exception e) {
                error("Exception in runServer(): " + e);
                error(e);
            }
        }
    }


I rewrote it to look like this:
 public void runServer() {
        running = true;
        while (running) {
            try {
                int ready = selector.select();//Sleep forever or until a channel is ready or until we are woken up
                if (ready > 0) {
                    Iterator<SelectionKey> i = selector.selectedKeys().iterator();
                    while (i.hasNext()) {
                        SelectionKey key = i.next();
                        i.remove();

                        if (key.isValid() && key.isAcceptable()) {
                            ((ConnectionHandler)key.attachment()).handleAccept(key);
                        }
                        if (key.isValid() && key.isReadable()) {
                            //Existing client either lost connection
                            //or sent data
                            ((ConnectionHandler)key.attachment()).handleRead(key);
                        }
                        if (key.isValid() && key.isWritable()) {
                            ((ConnectionHandler)key.attachment()).handleWrite(key);
                        }
                        if (key.isValid() && key.isConnectable()) {
                            ((ConnectionHandler)key.attachment()).handleConnect(key);
                        }
                    }
                }
            } catch (CancelledKeyException cke) {
            } catch (Exception e) {
                error("Exception in runServer(): " + e);
                error(e);
            }
        }
    }

and started using wakeup() but this changed nothing. The process is always responsive so clients are served, but the 100% CPU is problematic. Usually the servers do nearly nothing (0 - 5 % CPU usage).

I debugged into EPollArrayWrapper:
    int poll(long timeout) throws IOException {
        updateRegistrations();
        updated = epollWait(pollArrayAddress, NUM_EPOLLEVENTS, timeout, epfd);
        for (int i=0; i<updated; i++) {
            if (getDescriptor(i) == incomingInterruptFD) {
                interruptedIndex = i;
                interrupted = true;
                break;
            }
        }
        return updated;
    }

When the process is in 100% CPU, 'updated' is always one (or greater when IO actually happens) even when the select() call returns 0, and the 'getDescriptor(i)' call never returns 'incommingInterruptFD' so this seems to confirm that the thread is not interrupted.

Can you guys tell me what is going on ?

I sent a mail to the comp.lang.java newsgroup but they haven't been able to help me yet.

By the way, I have tried this on 1.6.0_02 and 1.6.0_12 on both 32 and 64 bit on Linux. Same results.

Cheers,
Robert


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
Url : http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20090305/b3753172/attachment.bin