Data server design questions

Wed Sep 5 12:26:16 PDT 2012

Why do you think aio would incur a lot of context switching? If the worker
pool is sized appropriately, then I'd actually imagine that context
switching will be minimized as compared to bio (when under heavy load from
lots of concurrent clients).  Also, if you use non-blocking handoff from
i/o thread to worker and back, you should not be seeing context switching
due to lock contention either.

Just curious - is your server going to have 1 NIC or multiple? What type of
Ethernet is this going to be on (i.e. 1GiB, 10GiB)? What CPU model will be
in the server? Keep in mind that modern CPU/memory controller can move a
lot of memory very quickly (e.g. nehalem has something like 25-30gb per
second).  Point being is that 1 non blocking i/o thread can move a ton of
data provided it's supplied with it quickly enough.  The only case I can
think of where more than 1 io thread *may* be useful is if you have a cpu
expensive protocol (e.g. https/SSL).

Sent from my phone
On Sep 5, 2012 2:56 PM, "Tom Lasseter" <t.lasseter at comcast.net> wrote:

> An excellent read…thanks for the reference!  It will take me a while to
> get thru it and hopefully will shed some light on solving my design
> problem. ****
>
> ** **
>
> The basic issue is a comparison of the two approaches: Blocking IO (BIO)
> and Async IO (AIO).  What makes the comparison difficult is that I want to
> use a hardware configuration which provides a large number of processors
> and threads and is not really a fixed configuration.  BIO will gobble up
> lots of threads and processors while AIO won’t be nearly as greedy, but
> would do a lot of context switching.  I would think AIO would suffer in
> this comparison assuming that both provide the same effective
> functionality.  So a comparison might suggest, as Paul’s paper does, that
> BIO is the way to go.  The key though is the “effective functionality”.
> The zeromq guide suggests that AIO as implemented there provides much more
> design flexibility than can be achieved with standard BIO.  This is a
> definite plus, as long as it doesn’t come at a significant performance
> loss.  ZMQ makes the very attractive claim that simplicity is their major
> guiding principle which is incredibly important.    ****
>
> ** **
>
> So I keep coming back to the issue of having to make a comparison test,
> but am not sure as I’ve stated above that it would probably be an apples
> versus oranges comparison.  Can anyone imagine a fair basis on which to
> make a comparison, allowing for a large number of processors and threads?
>  Would a comparison of a ZMQ-AIO implementation versus a BIO implementation
> be a fair test?      ****
>
> ** **
>
> Tom ****
>
> ** **
>
> *From:* Vitaly Davidovich [mailto:vitalyd at gmail.com]
> *Sent:* Wednesday, September 05, 2012 5:39 AM
> *To:* Tom Lasseter
> *Cc:* nio-dev at openjdk.java.net; Zhong Yu
> *Subject:* RE: Data server design questions****
>
> ** **
>
> Take a look at zeromq user guide: http://zguide.zeromq.org/page:all****
>
> It's a good read in general, but they also recommend one i/o thread and
> pool of workers.****
>
> In my mind, the question isn't whether nio is faster than bio, but rather
> which scales better as number of clients goes up.  There's less
> orchestration in bio so it's fair to say it'll be faster than nio for
> smallish number of clients, but I don't it scales as well as Paul's
> presentation seems to imply.****
>
> Also, if you use bio and the actual work that has to be done to service
> the request is sufficiently CPU bound, you will most definitely
> oversubscribe a commodity server at some not-so-high number of concurrent
> clients.  At this point, performance will degrade for every connection -
> with nio, you can limit impact to new connections/requests only.****
>
> Sent from my phone****
>
> On Sep 5, 2012 12:20 AM, "Tom Lasseter" <t.lasseter at comcast.net> wrote:***
> *
>
> It would be interesting to see Paul's code to get a better understanding of
> the actual data flow and see if your concerns are justified.  I would hope
> the test was more realistic and useful than you imply, but it's hard for me
> to judge without seeing the code.  Paul seems to be saying that now that we
> have hardware with more threads available and more efficient context
> switching, it's difficult for asynchronous IO to provide a better software
> solution than what can be achieved with hardware.  It would be nice to come
> up with a test that you were satisfied is testing the various approaches on
> the same hardware configuration.  It wouldn't surprise me as you imply in
> your note that the results will depend very much on the data sizes and data
> operations that are performed.  It might be that in my case, I will have to
> test these two very different approaches to determine which is optimal for
> my case, but learning that there is no generalization that can be or should
> be made....
>
> Tom
>
>
>
> -----Original Message-----
> From: Zhong Yu [mailto:zhong.j.yu at gmail.com]
> Sent: Tuesday, September 04, 2012 6:03 PM
> To: Tom Lasseter
> Cc: nio-dev at openjdk.java.net
> Subject: Re: Data server design questions
>
> About the claim that traditional socket IO is 30% faster than Selector
> based
> NIO: It's unclear how they did the benchmark. At that time I also did some
> benchmark myself, and got the same 30% number. However the test case is too
> simplistic. Basically, on the same machine, some clients write() data to
> the
> server; the server does read(). Measure the throughput, the blocking
> version
> is 30% more than the non-blocking version.
>
> There's a critical flaw though. The server calls read(byte[]) or
> read(ByteBuffer), but doesn't do anything with the data received.
> That's a extremely high throughput but terribly useless server! As soon as
> the server does the most basic thing with the data - reading every byte
> once
> - the throughput drops significantly, and there's not much difference
> between blocking and non-blocking any more.
>
> If the 30% number they got is also from such extreme and unrealistic
> setups,
> it has litter value for evaluating real server performance.
>
> Zhong Yu
>
>
>
> On Tue, Sep 4, 2012 at 3:04 PM, Tom Lasseter <t.lasseter at comcast.net>
> wrote:
> > I am developing a high-performance cloud data server primarily for the
> > display of large engineering and scientific datasets.  The client-side
> > is a rich Java application though a client browser is possible if the
> > GUI were limited.
> >
> >
> >
> > The challenge is the diverse libraries and technologies that are
> available.
> > I have spent weeks analyzing and prototyping.  The decision on what
> > path to take is a complex one requiring knowledge of not just the
> > detailed software architecture of the various design possibilities,
> > but projections/predictions of how effectively they will function in a
> > heavy-load environment.
> >
> >
> >
> > Despite the fact that most questions on this open-jdk list are
> > low-level, I believe this is the right venue for my questions as the
> > experts who understand the code well enough to design and improve it
> > are the only ones who can expertly answer the questions I'm posing.
> >
> >
> >
> > A data server is pretty general, so I will be more explicit about the
> > requirements for this one.
> >
> >
> >
> > The data consists of ASCII files which are JSON-like key-value files
> > each of which describes an object.  Associated with each of these
> > object files are a series of binary files which contain the bulk of
> > the data.  These data objects fall into two extreme groups: 1) large
> > 3D/4D volumes from which the user wishes to extract a subset of the
> > data; 2) smaller objects of which the user may wish to completely display
> many thousands.
> >
> >
> >
> > For the large volume files, the user is often moving a cursor through
> > the volume, so the server will need to be constantly delivering new
> > segments selected.  It makes sense to have a server thread monitoring
> > requests for this volume which may be coming simultaneously from
> > multiple users and starting worker threads to deliver the requests.
> > Client-server sockets should stay alive for some period of time and
> > the associated server work thread might be kept alive as well until the
> user ceases activity.
> >
> >
> >
> > For the large sets of smaller object files, the options are to make
> > individual requests for each file and have the client process and
> > display the data as received, or to make a request for all the objects
> > and have the server send them sequentially on a single socket.  The
> > optimization here is a bit tricky:  the IO can be running on several
> > sockets or sequentially on a single socket, and the processing on both
> > server and client ends could be run on single or multiple threads in
> either case.
> >
> >
> >
> > QUESTION 1: Looking at these requirements and scenarios, how would you
> > design this client-server system in general terms?
> >
> >
> >
> > An interesting analysis and presentation was put together by Paul Tyma:
> >
> > "Thousands of Threads and Blocking I/O: The old way to write Java
> > Servers is New again (and way better)"
> >
> >
> >
> > http://www.mailinator.com/tymaPaulMultithreaded.pdf
> >
> >
> >
> > where he makes the case that asynchronous IO was developed because of
> > performance issues in creating and switching threads, but that
> > hardware improvements have since significantly reduced these problems.
> > His comparison shows that the multi-thread blocking IO is simpler and
> > more efficient than asynchronous IO.  QUESTION 2:  What are your
> > comments on Paul's analysis and has it changed since his analysis and
> > presentation (2008)?
> >
> >
> >
> > Looking at the technologies out there is quite confusing.  Here are
> > the main ones I've studied in detail.
> >
> > 1)      Java 7 with its outstanding NIO.2 API;  Anghel Leonard's book
> > describing many of its features is amazing: there are dozens of
> > examples which can be rapidly loaded and tested and they all work (!);
> > the problem is that no one seems to be adopting the Java 7 socket
> > APIs; many such as the Netty group say it's somewhat incompatible and
> > will be adopted at a low level under the existing Netty API;  Jetty is
> > not supporting asynchronous IO at all (I don't believe); QUESTION 3:
> > What are the issues with Java 7 socket APIs that are inhibiting their
> adoption?
> >
> > 2)      Apache MINA; a lightweight server with an FTP server implemented;
> > uses blocking IO;
> >
> > 3)      Netty asynchronous NIO; excellent design for an event-driven
> > client-server system; does not have a TCP file-server implementation
> > comparable to FTP;
> >
> > 4)      Kaazing socket gateway server; closed source, not asynchronous;
> >
> > 5)      Waarp FTP and R66 file servers; heavy-duty file server built on
> > Netty by the French government; impressive and complex.
> >
> > QUESTION 4:  Are there other project and technologies I should be aware
> of?
> >
> >
> >
> > It seems to me that that the best solution is to use Java 7 and use
> > simple blocking IO for the application I've described.  The only issue
> > is reliability under heavy-load on a cloud server system.
> >
> > QUESTION 5: Can I rely on the cloud server itself (such as Amazon EC2)
> > to provide me all the bells-and-whistles needed such as
> > authentication, load-balancing, etc and do so reliably with the simple
> > server I'm envisioning? QUESTION 6:  Do any of the other technologies
> > discussed above provide capabilities which cannot be easily built from
> scratch?
> >
> >
> >
> > Thank you very much for any and all input!
> >
> >
> >
> > Tom Lasseter
> >
> > Email:  tom at gc-rt.com
> >
> > Website: gc-rt.com
> >
> >****
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20120905/6ec5f5de/attachment.html 