Java 17 bug (?) on Windows with MemorySegment.asSlice and offset =

[Maurizio Cimadamore]

Bug is here:
https://bugs.openjdk.java.net/browse/JDK-8286637

Maurizio

On 12/05/2022 15:01, Uwe Schindler wrote:
> Hi Maurizio,
>
>> mapAddress = MapViewOfFile(
>>           mapping,             /* Handle of file mapping object */
>>           mapAccess,           /* Read and write access */
>>           highOffset,          /* High word of offset */
>>           lowOffset,           /* Low word of offset */
>>           (DWORD)len);         /* Number of bytes to map */ <------
>>
>> ```
>>
>> That cast seems questionable, as DWORD is 32-bit, so casting an 8-byte
>> value bigger than 2^32 into that will surely fail.
>>
>> Of course, this was never an issue with ByteBuffer, as you could never
>> create such a big memory mapped region. But with memory segments this
>> becomes problematic.
>>
>> Fixing the cast to use SIZE_T instead solves the issue you are seeing.
> Do you have a PR / bug report for this already? That's actually a good thing that Erel found that problem! This reminds me about the other issue I found with ignoring the offset on windows (exactly the code you also suspected to be buggy; the code where the rounding of base address was applied at two different places). Luckily those bugs were quickly found by unit tests on Lucene's side (in JDK 16 already).
> Maurizio
> Lucene is testing with MMapDirectory, too - but not on Windows with huge indexes like 8 Gigabytes. Actually for fragmentation reasons we still split larger files into chunks, but those chunk sizes are always multiplies of a power of two - good luck! And during testing we create much smaller chunks to test the chunking, so we have no huge mappings during testing. One note about your workaround: The power of 2 workaround does not work well, if you have an unaligned start offset (like mapping at offset 7 of file for 2 gigabytes), because then the total mapped size is larger (a few bytes added to get the file offset aligned).
>
> I am glad that we did not yet have release Lucene with support for mapped memory segments! �� My problem with those bugs is that you cannot test it easily with our unit/integration tests in our testing infrastructure, because building indexes of that size is not an everyday task.
>
> Uwe
>