Possible network issue

[Robert Engels]

Will do. It does seem temporary. For instance with h2load testing, with 100k requests it will fail 10k of them - but this may be overstated by 10x since I have it set to pipeline 10 requests per connection. 

If I let the system sit for a bit - probably cleaning up the TIMEDWAIT connections - it will often run without failure. But rapid invocations of the test lead to the cited exception. 

> On Nov 7, 2023, at 10:33 AM, Alan Bateman <Alan.Bateman at oracle.com> wrote:
> 
> On 07/11/2023 13:28, Robert Engels wrote:
>> I suspect that what is happening is that the JVM gets the epoll notification that the socket is writable but by the time the writer runs the network buffers have been exhausted.
>> 
>> This should be an expected condition and seems like a JDK bug introduced with VT support. I checked earlier JDK releases (jdk15) and the sync -> non blocking code was not present in SocketOutputStream.java
> I don't think this is specific to virtual threads as we're seeing it elsewhere too. If there is no space in the socket write buffer then EAGAIN/EWOULDBLOCK is handled.  The intermittent ENOBUFS, which we're only seen on macos-aarch64 when under load, is hard to explain. We've had also some intermittent ENOMEM when joining multicast groups, also not specific to virtual threads.
> 
> Your mail said you can duplicate this readily. It would be useful if could capture the output of `netstat -nm` at around the time that this happens so see if this is a real resource exhaustion issue or not.
> 
> -Alan
> 
>