RFR: JDK-8319382: com/sun/jdi/JdwpAllowTest.java shows failures on AIX if prefixLen of mask is larger than 32 in IPv6 case

[Chris Plummer]

On Wed, 8 Nov 2023 14:37:29 GMT, Matthias Baesken <mbaesken at openjdk.org> wrote:

> In parseAllowedMask (file socketTransport.c) , prefixLen of mask is compared with a maxValue (32 for IPv4, 128 otherwise). This fails on AIX if it is larger than 32, because getaddrinfo seems to often (always ?) detect IPv4 family, even for IPv6 addresses, so we take the wrong maxValue.
> Probably we have to adjust the allowed maxValue on AIX, or adjust the IPv6 check.
> 
> Example:
> images/jdk/bin/java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:0,allow=0:0:0:0:0:0:10:0/106
> Error in allow option: '106'
> ERROR: transport error 103: invalid netmask in allow option
> ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)

src/jdk.jdwp.agent/share/native/libdt_socket/socketTransport.c line 468:

> 466:     maxValue = 128;
> 467: #endif
> 468: 

IIUC, this code would work for any port, but is only required for AIX. And since it is a hack to work around an AIX issue, you didn't want to subject all ports to it. The downside of this change is that it will cause the option parsing code to not properly produce an error if an IPv4 mask prefix between 32 and 128 is given.

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/16561#discussion_r1387008497