Data server design questions

[Tom Lasseter]

Until I actually get into some testing, I can't intelligently answer your
questions.  I appreciate your comments and suggestions and they have
certainly pointed me in the right direction. 

 

I suspect this conversation has gotten off-topic as far as the openjdk-list
group is concerned.  So with your permission Vitaly and Zhong, I'll just
reply to you exclusively in the future.  I was hoping, however, to hear from
someone on the list with comments on the role of the Java 7 Socket API.  I
would like to think that it captures the best of what has been learned from
Netty, ZMQ, and others and that these technologies would in turn embrace the
new API eventually.  Haven't heard that, and am still curious...

 

Tom

 

 

From: Vitaly Davidovich [mailto:vitalyd at gmail.com] 
Sent: Wednesday, September 05, 2012 12:26 PM
To: Tom Lasseter
Cc: nio-dev at openjdk.java.net; Zhong Yu
Subject: RE: Data server design questions

 

Why do you think aio would incur a lot of context switching? If the worker
pool is sized appropriately, then I'd actually imagine that context
switching will be minimized as compared to bio (when under heavy load from
lots of concurrent clients).  Also, if you use non-blocking handoff from i/o
thread to worker and back, you should not be seeing context switching due to
lock contention either.

Just curious - is your server going to have 1 NIC or multiple? What type of
Ethernet is this going to be on (i.e. 1GiB, 10GiB)? What CPU model will be
in the server? Keep in mind that modern CPU/memory controller can move a lot
of memory very quickly (e.g. nehalem has something like 25-30gb per second).
Point being is that 1 non blocking i/o thread can move a ton of data
provided it's supplied with it quickly enough.  The only case I can think of
where more than 1 io thread *may* be useful is if you have a cpu expensive
protocol (e.g. https/SSL).

Sent from my phone

On Sep 5, 2012 2:56 PM, "Tom Lasseter" <t.lasseter at comcast.net> wrote:

An excellent read.thanks for the reference!  It will take me a while to get
thru it and hopefully will shed some light on solving my design problem. 

 

The basic issue is a comparison of the two approaches: Blocking IO (BIO) and
Async IO (AIO).  What makes the comparison difficult is that I want to use a
hardware configuration which provides a large number of processors and
threads and is not really a fixed configuration.  BIO will gobble up lots of
threads and processors while AIO won't be nearly as greedy, but would do a
lot of context switching.  I would think AIO would suffer in this comparison
assuming that both provide the same effective functionality.  So a
comparison might suggest, as Paul's paper does, that BIO is the way to go.
The key though is the "effective functionality".  The zeromq guide suggests
that AIO as implemented there provides much more design flexibility than can
be achieved with standard BIO.  This is a definite plus, as long as it
doesn't come at a significant performance loss.  ZMQ makes the very
attractive claim that simplicity is their major guiding principle which is
incredibly important.    

 

So I keep coming back to the issue of having to make a comparison test, but
am not sure as I've stated above that it would probably be an apples versus
oranges comparison.  Can anyone imagine a fair basis on which to make a
comparison, allowing for a large number of processors and threads?  Would a
comparison of a ZMQ-AIO implementation versus a BIO implementation be a fair
test?      

 

Tom 

 

From: Vitaly Davidovich [mailto:vitalyd at gmail.com] 
Sent: Wednesday, September 05, 2012 5:39 AM
To: Tom Lasseter
Cc: nio-dev at openjdk.java.net; Zhong Yu
Subject: RE: Data server design questions

 

Take a look at zeromq user guide: http://zguide.zeromq.org/page:all

It's a good read in general, but they also recommend one i/o thread and pool
of workers.

In my mind, the question isn't whether nio is faster than bio, but rather
which scales better as number of clients goes up.  There's less
orchestration in bio so it's fair to say it'll be faster than nio for
smallish number of clients, but I don't it scales as well as Paul's
presentation seems to imply.

Also, if you use bio and the actual work that has to be done to service the
request is sufficiently CPU bound, you will most definitely oversubscribe a
commodity server at some not-so-high number of concurrent clients.  At this
point, performance will degrade for every connection - with nio, you can
limit impact to new connections/requests only.

Sent from my phone

On Sep 5, 2012 12:20 AM, "Tom Lasseter" <t.lasseter at comcast.net> wrote:

It would be interesting to see Paul's code to get a better understanding of
the actual data flow and see if your concerns are justified.  I would hope
the test was more realistic and useful than you imply, but it's hard for me
to judge without seeing the code.  Paul seems to be saying that now that we
have hardware with more threads available and more efficient context
switching, it's difficult for asynchronous IO to provide a better software
solution than what can be achieved with hardware.  It would be nice to come
up with a test that you were satisfied is testing the various approaches on
the same hardware configuration.  It wouldn't surprise me as you imply in
your note that the results will depend very much on the data sizes and data
operations that are performed.  It might be that in my case, I will have to
test these two very different approaches to determine which is optimal for
my case, but learning that there is no generalization that can be or should
be made....

Tom



-----Original Message-----
From: Zhong Yu [mailto:zhong.j.yu at gmail.com]
Sent: Tuesday, September 04, 2012 6:03 PM
To: Tom Lasseter
Cc: nio-dev at openjdk.java.net
Subject: Re: Data server design questions

About the claim that traditional socket IO is 30% faster than Selector based
NIO: It's unclear how they did the benchmark. At that time I also did some
benchmark myself, and got the same 30% number. However the test case is too
simplistic. Basically, on the same machine, some clients write() data to the
server; the server does read(). Measure the throughput, the blocking version
is 30% more than the non-blocking version.

There's a critical flaw though. The server calls read(byte[]) or
read(ByteBuffer), but doesn't do anything with the data received.
That's a extremely high throughput but terribly useless server! As soon as
the server does the most basic thing with the data - reading every byte once
- the throughput drops significantly, and there's not much difference
between blocking and non-blocking any more.

If the 30% number they got is also from such extreme and unrealistic setups,
it has litter value for evaluating real server performance.

Zhong Yu



On Tue, Sep 4, 2012 at 3:04 PM, Tom Lasseter <t.lasseter at comcast.net> wrote:
> I am developing a high-performance cloud data server primarily for the
> display of large engineering and scientific datasets.  The client-side
> is a rich Java application though a client browser is possible if the
> GUI were limited.
>
>
>
> The challenge is the diverse libraries and technologies that are
available.
> I have spent weeks analyzing and prototyping.  The decision on what
> path to take is a complex one requiring knowledge of not just the
> detailed software architecture of the various design possibilities,
> but projections/predictions of how effectively they will function in a
> heavy-load environment.
>
>
>
> Despite the fact that most questions on this open-jdk list are
> low-level, I believe this is the right venue for my questions as the
> experts who understand the code well enough to design and improve it
> are the only ones who can expertly answer the questions I'm posing.
>
>
>
> A data server is pretty general, so I will be more explicit about the
> requirements for this one.
>
>
>
> The data consists of ASCII files which are JSON-like key-value files
> each of which describes an object.  Associated with each of these
> object files are a series of binary files which contain the bulk of
> the data.  These data objects fall into two extreme groups: 1) large
> 3D/4D volumes from which the user wishes to extract a subset of the
> data; 2) smaller objects of which the user may wish to completely display
many thousands.
>
>
>
> For the large volume files, the user is often moving a cursor through
> the volume, so the server will need to be constantly delivering new
> segments selected.  It makes sense to have a server thread monitoring
> requests for this volume which may be coming simultaneously from
> multiple users and starting worker threads to deliver the requests.
> Client-server sockets should stay alive for some period of time and
> the associated server work thread might be kept alive as well until the
user ceases activity.
>
>
>
> For the large sets of smaller object files, the options are to make
> individual requests for each file and have the client process and
> display the data as received, or to make a request for all the objects
> and have the server send them sequentially on a single socket.  The
> optimization here is a bit tricky:  the IO can be running on several
> sockets or sequentially on a single socket, and the processing on both
> server and client ends could be run on single or multiple threads in
either case.
>
>
>
> QUESTION 1: Looking at these requirements and scenarios, how would you
> design this client-server system in general terms?
>
>
>
> An interesting analysis and presentation was put together by Paul Tyma:
>
> "Thousands of Threads and Blocking I/O: The old way to write Java
> Servers is New again (and way better)"
>
>
>
> http://www.mailinator.com/tymaPaulMultithreaded.pdf
>
>
>
> where he makes the case that asynchronous IO was developed because of
> performance issues in creating and switching threads, but that
> hardware improvements have since significantly reduced these problems.
> His comparison shows that the multi-thread blocking IO is simpler and
> more efficient than asynchronous IO.  QUESTION 2:  What are your
> comments on Paul's analysis and has it changed since his analysis and
> presentation (2008)?
>
>
>
> Looking at the technologies out there is quite confusing.  Here are
> the main ones I've studied in detail.
>
> 1)      Java 7 with its outstanding NIO.2 API;  Anghel Leonard's book
> describing many of its features is amazing: there are dozens of
> examples which can be rapidly loaded and tested and they all work (!);
> the problem is that no one seems to be adopting the Java 7 socket
> APIs; many such as the Netty group say it's somewhat incompatible and
> will be adopted at a low level under the existing Netty API;  Jetty is
> not supporting asynchronous IO at all (I don't believe); QUESTION 3:
> What are the issues with Java 7 socket APIs that are inhibiting their
adoption?
>
> 2)      Apache MINA; a lightweight server with an FTP server implemented;
> uses blocking IO;
>
> 3)      Netty asynchronous NIO; excellent design for an event-driven
> client-server system; does not have a TCP file-server implementation
> comparable to FTP;
>
> 4)      Kaazing socket gateway server; closed source, not asynchronous;
>
> 5)      Waarp FTP and R66 file servers; heavy-duty file server built on
> Netty by the French government; impressive and complex.
>
> QUESTION 4:  Are there other project and technologies I should be aware
of?
>
>
>
> It seems to me that that the best solution is to use Java 7 and use
> simple blocking IO for the application I've described.  The only issue
> is reliability under heavy-load on a cloud server system.
>
> QUESTION 5: Can I rely on the cloud server itself (such as Amazon EC2)
> to provide me all the bells-and-whistles needed such as
> authentication, load-balancing, etc and do so reliably with the simple
> server I'm envisioning? QUESTION 6:  Do any of the other technologies
> discussed above provide capabilities which cannot be easily built from
scratch?
>
>
>
> Thank you very much for any and all input!
>
>
>
> Tom Lasseter
>
> Email:  tom at gc-rt.com
>
> Website: gc-rt.com
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120905/d499a9cb/attachment-0001.html