RFR: 8277489: Rewrite JAAS UnixLoginModule with FFM [v5]

[Weijun Wang]

On Mon, 5 Jan 2026 18:12:08 GMT, David Lloyd <dmlloyd at openjdk.org> wrote:

>> What you have done is fine. Thanks! However, there is one potential problem left:
>> We are passing `tmpUid` to `getpwuid_r` as an `int`. That results in the following sequence (example from AIX):
>> 
>> [2.537s][trace][foreign,downcall]  ;; { argument shuffle
>> [2.537s][trace][foreign,downcall]   0x0a0001000747d744:   mr      r12,r3
>> [2.537s][trace][foreign,downcall]   0x0a0001000747d748:   extsw   r3,r4
>> [2.537s][trace][foreign,downcall]   0x0a0001000747d74c:   mr      r4,r5
>> [2.537s][trace][foreign,downcall]   0x0a0001000747d750:   mr      r5,r6
>> [2.537s][trace][foreign,downcall]   0x0a0001000747d754:   extsw   r6,r7
>> [2.537s][trace][foreign,downcall]   0x0a0001000747d758:   mr      r7,r8
>> [2.537s][trace][foreign,downcall]  ;; } argument shuffle
>> 
>> The 4 Byte value for `tmpUid` is taken from Register r4, sign extended to 8 Byte long and put into the first argument register r3. The sign extend is wrong because `uid` is an `uint32_t`. That violates the calling convention. We have no way to tell the FFM that we need zero extend.
>> 
>> A possible workaround would be to do the conversion in Java and passing it as long:
>> 
>> diff --git a/src/jdk.security.auth/share/classes/com/sun/security/auth/module/UnixSystem.java b/src/jdk.security.auth/share/classes/com/sun/security/auth/module/UnixSystem.java
>> index ed520529ac8..573513b7bef 100644
>> --- a/src/jdk.security.auth/share/classes/com/sun/security/auth/module/UnixSystem.java
>> +++ b/src/jdk.security.auth/share/classes/com/sun/security/auth/module/UnixSystem.java
>> @@ -25,6 +25,7 @@
>>  
>>  package com.sun.security.auth.module;
>>  
>> +import jdk.internal.util.Architecture;
>>  import jdk.internal.util.OperatingSystem;
>>  
>>  import java.lang.foreign.AddressLayout;
>> @@ -83,6 +84,8 @@ public class UnixSystem {
>>              = (ValueLayout.OfByte) LINKER.canonicalLayouts().get("char");
>>      private static final ValueLayout.OfInt C_INT
>>              = (ValueLayout.OfInt) LINKER.canonicalLayouts().get("int");
>> +    private static final ValueLayout.OfLong C_LONG
>> +            = (ValueLayout.OfLong) LINKER.canonicalLayouts().get("long");
>>      private static final AddressLayout C_POINTER
>>              = ((AddressLayout) LINKER.canonicalLayouts().get("void*"))
>>              .withTargetLayout(MemoryLayout.sequenceLayout(java.lang.Long.MAX_VALUE, C_CHAR));
>> @@ -110,10 +113,14 @@ public class UnixSystem {
>>  
>>      // getpwuid_r does not work on AIX, instead we use another similar function
>>      // exter...
>
> FWIW there is an issue and ML discussion relating to extension of arguments with respect to `ValueLayout`: https://bugs.openjdk.org/browse/JDK-8336664

\What happens if you create a user on AIX with a uid bigger than 2^31 and call `getpwuid_r` on it (by hardcoding `tmpUid` as a negative number)?

-------------

PR Review Comment: https://git.openjdk.org/jdk/pull/28931#discussion_r2662861217