RFR: 8290349: IP_DONTFRAGMENT doesn't set DF bit in IPv4 header

[Daniel Jeliński]

On Thu, 21 Jul 2022 14:46:58 GMT, Jaikiran Pai <jpai at openjdk.org> wrote:

>> This patch partially fixes the issue where IPv6 sockets were fragmenting outgoing IPv4 datagrams even when IP_DONTFRAGMENT flag was set. Specifically, it fixes the issue on Linux and Windows. As far as I could tell, the issue is unfixable on Mac OS X.
>> 
>> All systems have a separate DONTFRAGMENT flag for IPv4 and IPv6. Each flag only affects packets from its address family; if we want to disable fragmentation of both IPv4 and IPv6 packets sent by an IPv6 socket, we need to set both IPv4 and IPv6 flags. This is similar to other already existing options like IP_TOS or IP_MULTICAST_*.
>> 
>> On Mac OS X it's impossible to set an IPv4 socket option on an IPv6 socket; attempting to do so results in an error. This is a known issue with Mac OS X; on that system we return false from [Net#shouldSetBothIPv4AndIPv6Options](https://github.com/openjdk/jdk/blob/2342684f2cd91a2e5f43dd271e95836aa78e7d0a/src/java.base/unix/native/libnio/ch/Net.c#L159) to avoid setting IPv4 options.
>> 
>> As far as I can tell, non-privileged users have no way to check if the DF flag was set or if the packet was fragmented. I implemented a test that attempted to send a large packet over a physical interface and expected SocketException / EMSGSIZE; the test frequently failed for unrelated reasons, so I decided against including it. Loopback interface has infinite MTU on some systems, so can not be used for this test.
>> 
>> Testing performed (with IP_DONTFRAGMENT flag):
>> Windows 10:
>> With this patch, IPv4 packets sent from IPv6 socket have the DF flag; sending an IPv4 packet larger than the interface MTU results in EMSGSIZE and no packet is sent. Without this patch, the packet is fragmented and sent without DF flag.
>> Sending IPv6 packets larger than the interface MTU usually results in EMSGSIZE. It may succeed if the destination address is non-routable.
>> 
>> For other systems I could not capture packets, so I can only report the observed behavior of sending packets.
>> Windows 2012 and 2016 (IP_MTU_DISCOVER not supported):
>> With this patch, sending any packet exceeding MTU size fails with EMSGSIZE. Without this patch sending a large IPv6 packet succeeds.
>> If a packet is sent to a non-routable address, send succeeds, no error is reported.
>> 
>> Linux:
>> With this patch, sending any packet exceeding MTU size fails with EMSGSIZE. Without this patch, sending large IPv4 packets from IPv6 sockets succeeds.
>> 
>> Mac OS X 12:
>> Sending an IPv6 packet exceeding MTU size fails with EMSGSIZE. Sending large IPv4 packets from IPv4 sockets also fails with EMSGSIZE. Sending large IPv4 packets from IPv6 sockets succeeds. The patch does not change the observed behavior.
>
> src/jdk.net/macosx/native/libextnet/MacOSXSocketOptions.c line 207:
> 
>> 205:         }
>> 206:     }
>> 207:     return JNI_TRUE;
> 
> Hello Daniel, if I'm reading this diff correctly, then we have inverted the check on `fd`. So I suspect this final `return` now needs to be changed to `return JNI_FALSE` to account for `fd` being `-1`?

Hi Jaikiran, the point of this change was to better support systems where either IPv4 or IPv6 was disabled. I'm not sure if that can even happen on Mac. 
The change here is that if we can't create one socket, we don't immediately return false, but instead return the result for the other socket. For example, if creating IPv4 socket fails, we only check if DF is supported on IPv6 socket, and return that as a result. If both `socket` calls fail, we don't know if the DF flag is supported, and return true (before the patch we would return false in that case). I can modify the patch to return false in that case, or roll back this change entirely if you think it makes no sense.

-------------

PR: https://git.openjdk.org/jdk/pull/9575