Java shared memory

[Radek]

Dear all,

Thank you for your effort, I understand that the proposal was rejected.

I maintain code here https://bitbucket.org/radoslaw_smogura/jshm-jdk9-hotspot/ (please don’t laugh I started with this more as prototype, but remarks are always welcome).

The approach I used is (or actually was) to store the pointer to actuall elements just after a length field (for primitive arrays). This pointer for ordinal arrays would point to heap space just after header, for attached arrays it can point anywhere.

I exposed just attachMemory0 native call (which could be hidden in other ways from unrestricted access) to native library which did all the mappings. Someone could use this to create shared POSIX or other region to store char / bytes of String and then reduplicate Strings among JVMs. As pointer can be arbitrary I’m not limited to page boundaries for C mapping.

With this approach for small cost of loading pointers, and slightly size increase I could maintain arrays as normal and keep assembly code still effective.

The thing except changes to compiler were to properly handle SIGBUS.

And yes it’s faster...

Thanks for your time and have a nice day
Radek

> On 31 May 2016, at 13:31, Vladimir Ivanov <vladimir.x.ivanov at oracle.com> wrote:
> 
> Radek,
> 
> Unfortunately, I don't see any source file changes in the project you linked.
> 
> It seems your project can be split into 2 independent parts:
>  (1) accessing off-heap memory;
>  (2) representing off-heap region as primitive array.
> 
> As I understand, it should cover all use cases you mentioned.
> 
> Off-heap memory access is already supported by NIO (MappedByteBuffer & DirectByteBuffer). In Java 9 VarHandles will provide int[]/long[]/... views over ByteBuffers (see MethodHandles.byteBufferViewVarHandle [1]) with impressive performance characteristics.
> 
> Forward compatibility aspect (represent off-heap region as byte[]/int[]/... instance) fits perfectly into Arrays 2.0 project (see John's presentation at JVMLS 2012 for more details [2]). As Paul already mentioned, David Simms has been working on j.l.Arrayish [2] support and made impressive progress (the idea is the JVM can link *aload/*astore bytecode instructions to methods on Arrayish interface).
> 
> Have you had a chance to look into ByteBuffers/Varhandles & j.l.Arrayish? It would be very interesting to hear about your experience.
> 
> Best regards,
> Vladimir Ivanov
> 
> [1] http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/tip/src/java.base/share/classes/java/lang/invoke/MethodHandles.java#l2405
> 
> [2] http://www.oracle.com/technetwork/java/javase/community/jvmls2012-1840099.html
>    http://cr.openjdk.java.net/~jrose/pres/201207-Arrays-2.pdf
> 
> [3] http://mail.openjdk.java.net/pipermail/valhalla-dev/2016-April/001892.html
> http://hg.openjdk.java.net/valhalla/valhalla/jdk/file/tip/src/java.base/share/classes/java/lang/Arrayish.java
> 
> On 5/30/16 9:48 PM, Radek wrote:
>> Dear all,
>> 
>> Please don’t think I’m so crazy. Recently I was working on JVM changes to add shared memory / memory mapping support.
>> 
>> I’m able to map part of files as primitive java arrays. First performance results are very optimistic (40% boost in searching min-max long in 1GB file) with mapped long[] as a replacement to LongBuffer. The work is not finished. I have adopted G1, c1 and c2, but I have got mainly stable solution (or I think so).
>> 
>> I see three possible benefits of shared memory:
>> 1. Shared memory and interprocess communication.
>> 2. Mapping large files.
>> 3. Aggressive inter-JVM string de-duplication.
>> 
>> I’m looking for sponsorship for this change, with some guidelines how I can move forward with possible adoption into JDK. If solution would be viable, I would like to open JSR for JDK API modification.
>> 
>> I use https://bitbucket.org/radoslaw_smogura/java-shared-memory/wiki/Home as umbrella repo, but I don’t update it quite often.
>> 
>> Thanks in advance,
>> Radek
>> 
>>