Memory usage of EPollArraySelector

[Vitaly Davidovich]

Patrick,

I'm confused - how many selectors do you have? You mentioned a selector is
created for each accepted connection but then state that selectors are
reused for newly accepted connections.  Do you mean a selector is reused
after its sole connection is closed?

You should bite the bullet and reimplement your application.  If I'm
reading right, you're wasting lots of resources all around, and your
proposal is just a hack (with adverse performance due to additional memory
chasing via HashSet) that will likely catch up to you anyway.

sent from my phone
On Oct 22, 2015 4:51 PM, "Patrick Bergamin" <
patrick.bergamin at wedgenetworks.com> wrote:

> I'm having problems with memory usage of the current implementation
> of EPollArraySelector in 1.8.0_60 for an existing proxy application.
> We've been on java version 1.7.0_05 for a while now because the new
> implementation of EPollArraySelector does not work well with the
> design of this proxy application.
>
> I did find the sun.nio.ch.maxUpdateArraySize property helped to
> reduce memory usage a bit.  But as the proxy application runs
> all the EPollArraySelector objects will slowly accumulate memory.
>
> The current design of the proxy application is to have one Selector
> handle the listen socket.  Once a connection is accepted, management
> of the connection is handed off to another thread on another Selector.
> Basically there is one Selector allocated per socket connection.
> The Selectors are never closed they are reused when a new socket
> connection is accepted.
>
> The application was designed before this latest implementation of
> EPollArraySelector.  Redesigning the application to decouple the
> Selector from the socket connection would be a fair amount work.
>
> We have machines that are capable of running this proxy application
> with around 350,000 open connections.  Since it is a proxy there
> are actually 700,000 open connections.  The file descriptor limit
> is set high (5,000,000) to be able to handle all these open socket
> connections.
>
> Below I've included a patch to show what kinds of things I need
> to do to bring the memory usage of EPollArraySelector down for
> this proxy application.
>
> Is there any interest in including a second epoll implementation
> in openjdk that uses less memory or perhaps have more properties
> to control the memory usage of the existing EPollArraySelector?
>
> thanks,
> Patrick Bergamin
>
> --- EPollArrayWrapper.java    2015-07-30 06:27:02.000000000 -0600
> +++ EPollArrayWrapper2.java    2015-09-28 15:31:41.712607415 -0600
> @@ -29,6 +29,7 @@
>  import java.security.AccessController;
>  import java.util.BitSet;
>  import java.util.HashMap;
> +import java.util.HashSet;
>  import java.util.Map;
>  import sun.security.action.GetIntegerAction;
>
> @@ -122,7 +123,7 @@
>
>      // Used by release and updateRegistrations to track whether a file
>      // descriptor is registered with epoll.
> -    private final BitSet registered = new BitSet();
> +    private final HashSet<Integer> registered = new HashSet<Integer>();
>
>
>      EPollArrayWrapper() throws IOException {
> @@ -187,7 +188,10 @@
>              }
>          } else {
>              Integer key = Integer.valueOf(fd);
> -            if (!isEventsHighKilled(key) || force) {
> +            if (events == KILLED) {
> +                eventsHigh.remove(key);
> +            }
> +            else if (!isEventsHighKilled(key) || force) {
>                  eventsHigh.put(key, Byte.valueOf(events));
>              }
>          }
> @@ -201,6 +205,9 @@
>              return eventsLow[fd];
>          } else {
>              Byte result = eventsHigh.get(Integer.valueOf(fd));
> +            if (result == null) {
> +                return KILLED;
> +            }
>              // result should never be null
>              return result.byteValue();
>          }
> @@ -235,7 +242,7 @@
>          // force the initial update events to 0 as it may be KILLED by a
>          // previous registration.
>          synchronized (updateLock) {
> -            assert !registered.get(fd);
> +            assert !registered.contains(fd);
>              setUpdateEvents(fd, (byte)0, true);
>          }
>      }
> @@ -249,9 +256,9 @@
>              setUpdateEvents(fd, KILLED, false);
>
>              // remove from epoll
> -            if (registered.get(fd)) {
> +            if (registered.contains(fd)) {
>                  epollCtl(epfd, EPOLL_CTL_DEL, fd, 0);
> -                registered.clear(fd);
> +                registered.remove(fd);
>              }
>          }
>      }
> @@ -286,7 +293,7 @@
>              while (j < updateCount) {
>                  int fd = updateDescriptors[j];
>                  short events = getUpdateEvents(fd);
> -                boolean isRegistered = registered.get(fd);
> +                boolean isRegistered = registered.contains(fd);
>                  int opcode = 0;
>
>                  if (events != KILLED) {
> @@ -298,9 +305,9 @@
>                      if (opcode != 0) {
>                          epollCtl(epfd, opcode, fd, events);
>                          if (opcode == EPOLL_CTL_ADD) {
> -                            registered.set(fd);
> +                            registered.add(fd);
>                          } else if (opcode == EPOLL_CTL_DEL) {
> -                            registered.clear(fd);
> +                            registered.remove(fd);
>                          }
>                      }
>                  }
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20151022/3f046935/attachment.html>