java.nio and SelectionKey.interestOps(int i)

[Kurt Miller]

Ahh ok, I see now. I see why it is happening on FreeBSD and I suppose
it may be the same reason OS X fails this. On FreeBSD (and OpenBSD too),
upon return from the poll() system call the complete pollfd array is
written to. So what's happening is that after the program prints
"Listener read " your program races around to the selector.select()
call before the fixer thread changes the interestOps to look for
writes. selector.select() is implemented by poll() and the poll
system call. The kernel copies the userland pollfd data to kernel
memory. fixer changes userland's copy of pollfd data to watch out
for writes too.  sel.wakeup() wakes up the poll system call which
overrights userland's pollfd data and erases the POLLOUT flag then
goes back and poll's again for just reads.

This works on Solaris and Linux I assume because the kernel
may write to just the revents field, not the whole struct.

http://www.opengroup.org/onlinepubs/009695399/functions/poll.html

Doesn't mention if events is reset or left untouched but it does
seem like a good idea to leave it alone upon return from the kernel.

Here's a simple test program to demonstrate the problem. You can
try it on OS X to see if it fails there too and file a bug report
against OS X.

#include <err.h>
#include <poll.h>
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>

struct pollfd fds[2];
int pipefds[2];

static void *
thread(void *arg) {
  sleep(1);
  fds[1].events = POLLIN | POLLOUT;
  write(pipefds[0], "", 1);
  return NULL;
}

int
main(int argc, char *argv[]) {
  pthread_t tid;

  if (pipe(pipefds) != 0)
    err(1, "pipe failed");

  fds[0].fd = pipefds[0];
  fds[0].events = POLLIN;

  fds[1].fd = pipefds[1];
  fds[1].events = POLLIN;

  if (pthread_create(&tid, NULL, thread, NULL) != 0)
    err(1, "pthread_create failed");

  poll(fds, 2, -1);

  if (fds[1].events != (POLLIN | POLLOUT)) {
    printf("events overwritten by kernel!\n");
    return 1;
  }

  return 0;
}

Martin Kihlgren wrote:
> Oh, but my program waits for the connecting client to start sending -
> I don't want to spend time selecting for OP_WRITE before I have
> anything to write. So I reset the interestOps when the server has a
> reply, and then expect the selector to return the SelectionKey when
> the endpoint is ready for write.
> 
> Maybe one could consider this (changing the interestOps after having
> registered the key) a weird thing to do, but I don't see anything that
> indicates it should not work, and as you yourself has seen, it works
> in most environments (except, afaik, that BSD and OS X).
> 
> regards,
> //Martin
> 
> On Fri, Jan 23, 2009 at 10:06 PM, Kurt Miller
> <kurt at intricatesoftware.com> wrote:
>> Hi Martin,
>>
>> This looks like bug in your test program:
>> ...
>>    if (selectionKey.isAcceptable()) {
>>        SocketChannel connection = serverSocketChannel.accept();
>>        connection.configureBlocking(false);
>>        connection.register(selector, SelectionKey.OP_READ);
>>
>> The line above only registers the SocketChannel to only be
>> interested in read ops. SelectionKey.OP_READ | SelectionKey.OP_WRITE
>> works as expected.
>>
>> Regards,
>> -Kurt
>>
>> Martin Kihlgren wrote:
>>> Of course!
>>>
>>> I will attach the code, and here is my results when I run it on two
>>> different machines:
>>>
>>> On my linux box:
>>> ------------------------------8<---------------------------
>>> [0:zond at cthulhu ~/tmp]$uname -a
>>> Linux cthulhu 2.6.23 #3 SMP Mon Jan 21 10:44:04 CET 2008 i686 GNU/Linux
>>> [0:zond at cthulhu ~/tmp]$java -version
>>> java version "1.6.0_0"
>>> OpenJDK  Runtime Environment (build 1.6.0_0-b11)
>>> OpenJDK Server VM (build 1.6.0_0-b11, mixed mode)
>>> [0:zond at cthulhu ~/tmp]$md5sum SetInterestOpsTest.java
>>> e538a1c50f35afef6ea1121a842ce8ce  SetInterestOpsTest.java
>>> [0:zond at cthulhu ~/tmp]$javac SetInterestOpsTest.java
>>> [0:zond at cthulhu ~/tmp]$java SetInterestOpsTest
>>> Connecter connected
>>> Connecter sent test1
>>> Listener accepted connection
>>> Listener read test1
>>> Listener wrote test2
>>> Connecter received test2
>>> [0:zond at cthulhu ~/tmp]$
>>> ------------------------------8<---------------------------
>>>
>>> On my OS X Tiger box:
>>> ------------------------------8<---------------------------
>>> [0:zond at sharkattack ~/tmp]$uname -a
>>> Darwin sharkattack.troja.ath.cx 8.11.1 Darwin Kernel Version 8.11.1:
>>> Wed Oct 10 18:23:28 PDT 2007; root:xnu-792.25.20~1/RELEASE_I386 i386
>>> i386
>>> [0:zond at sharkattack ~/tmp]$java -version
>>> java version "1.6.0_03-p3"
>>> Java(TM) SE Runtime Environment (build
>>> 1.6.0_03-p3-landonf_19_aug_2008_14_55-b00)
>>> Java HotSpot(TM) Server VM (build
>>> 1.6.0_03-p3-landonf_19_aug_2008_14_55-b00, mixed mode)
>>> [0:zond at sharkattack ~/tmp]$md5sum SetInterestOpsTest.java
>>> e538a1c50f35afef6ea1121a842ce8ce  SetInterestOpsTest.java
>>> [0:zond at sharkattack ~/tmp]$javac SetInterestOpsTest.java
>>> [0:zond at sharkattack ~/tmp]$java SetInterestOpsTest
>>> Connecter connected
>>> Connecter sent test1
>>> Listener accepted connection
>>> Listener read test1
>>> ^C[130:zond at sharkattack ~/tmp]$
>>> ------------------------------8<---------------------------
>>>
>>> regards,
>>> //Martin Kihlgren
>>>
>>> On Wed, Jan 21, 2009 at 4:51 AM, Kurt Miller <kurt at intricatesoftware.com> wrote:
>>>> Hello Martin,
>>>>
>>>> Can you post a minimal test case that reproduces the issue?
>>>>
>>>> Thanks,
>>>> -Kurt
>>>>
>>>>
>>>> Martin Kihlgren wrote:
>>>>> Hello list!
>>>>>
>>>>> I have this problem that I thought maybe you could help explaining.
>>>>>
>>>>> I have this application, where a server thread loops around a
>>>>> select(), and then does stuff to the different SelectableChannels that
>>>>> pops out from the Selector.
>>>>>
>>>>> If something is read from a channel, for example, I send it to a
>>>>> handler. And if the handler wants to respond in some way, it adds it
>>>>> to a queue and then does interestOps(interestOps() &
>>>>> SelectionKey.OP_WRITE) on its SelectionKey. Then it does wakeup() on
>>>>> its Selector.
>>>>>
>>>>> In Sun's java 6 for linux, this works beautifully. The Selector wakes
>>>>> up, does another select, and finds that suddenly this
>>>>> SelectableChannel wants to be (and can be) written to.
>>>>>
>>>>> In SoyLatte, however, nothing happens! No matter if I wake the
>>>>> selector up one or many times, it doesn't recognize that the
>>>>> SelectableChannel and its SelectionKey is interested in OP_WRITE.
>>>>>
>>>>> Any ideas?
>>>>>
>>>>> regards,
>>>>> //Martin Kihlgren
>>>>>
>>
>