Extended socket options and module layers

[Chris Hegarty]

Hi,

Just dropping a note here about a recent issue that I ran into, that
could help others. (I should have known better, but several different
conditions conspired which lead to the bug [1]).

Elasticsearch starts with all code on the module path (the class path
is empty). The org.elasticsearch.server module is the root, and contains
the main entry point. We load extensions to the server in their own
module layer. One of these extensions is a network extension which,
among other things, uses Netty and NIO socket channels. This network
extension sets the TCP_KEEPXXX extended socket options. The code
involved here is a little old, and used reflection so as to compile on
older JDKs, hence the code tolerated the absence of the extended
socket options type (from the jdk.net module). This lead to the bug,
where the TCP_KEEPXXX were not set, but there was no failure or log of
such. But why did this happen?

Our code in the boot layer does not directly use anything from the
jdk.net module, so does not `require jdk.net`. We defined the code in
our network extension to a module layer, l, that is a child of the boot
layer. The network extension code in layer, l, `requires jdk.net`, but
that is too late - the socket channel implementation in java.base is not
in a position to provide support for the extended options, since jdk.net
is not resolved in the boot configuration/layer. The solution is
trivial, `--add-modules jdk.net` [2].

I'm not sure that there is anything that should (or could) be done about
this - an SPI for extended options is not really viable or suitable. Our
use of reflection really obfuscated the bug, but equally code that
deals with the extended socket options is often conditional so as to
tolerate running on different platforms. Anyway, I thought it worth
sharing, since it's both interesting (to me) and informative (in case
someone else encounters similar).

-Chris.

[1] https://github.com/elastic/elasticsearch/issues/88897
[2] https://github.com/elastic/elasticsearch/pull/88935