TCP Sockets - Limiting one write at a time is harming performance (at least on windows)

[Alan Bateman]

Leon Finker wrote:
> Yes, short-writes will cause a problem, but on windows with IOCP/overlapped
> model short-writes are not observed in practice (unless resources are
> depleted; in that case connection probably should be closed/cleaned up
> anyway). Not sure about other platforms. 
>
> Have you ever seen (with latest jdk build) that ByteBuffer#hasRemaining is
> true in write's completed() on windows under stress (more likely to happen
> on pre-Vista due to easier non-paged pool resource depletion)? Even though
> Platform SDK IOCP sample does handle this case, I have not seen it being
> handled in other production frameworks. Anyhow, something to ponder on. The
> data throughput is definitely much better.
>   
J1 is this week so a belated reply.

Short-writes are "normal" when there is congestion and insufficient 
space in the socket's send buffer. Even Windows does this with blocking 
and non-blocking I/O. You may be right that Windows doesn't do this with 
asynchronous I/O, at least for when buffering is disabled (which is what 
I think you are doing). Is this specified anywhere? In any case, the 
2x-3x results are kinda surprising. I recall the previous version of 
your tests completely saturated the network so I'm not sure how the 
throughput is increased. What size are the messages and have you 
disabled Nagle by any chance? Also, what is the size of the send buffer 
that you are comparing with?

-Alan.