more about - System.arraycopy(...) equivalents for ByteBuffer/FileChannel

[Alan Bateman]

On 02/11/2012 01:29, Jeff Hain wrote:
> :
>
>   That's modulo the fact that MappedByteBuffers can't be
> explicitly and portably unmapped (Bug ID: 4724038), so
> you can't always use them.
Yes, a very difficult issue that many people have looked at but didn't 
come up a solution that addresses all the concerns, see:

http://bugs.sun.com/view_bug.do?bug_id=4724038


>
>
>   That's also modulo the fact that, for BB to FC copies,
> if the FileChannel is not readable, you can't use MBB
> (no WRITE_ONLY mapping mode). A FileChannel.isReadable()
> method would allow to use MBB for readable (and writable)
> channels, without risk of an exception being thrown.
It would although if code is running into then it suggests that it might 
not be using the API as intended.


> :
>
>   That's also modulo the fact that when you copy between
> a MappedByteBuffer and a FileChannel, src and dst might
> share memory, but there is no API to figure out if and how,
> thus you don't know if you have to copy forward or backward
> (to avoid erasing bytes to copy with copied bytes).
Right, there isn't anything in this API to know when buffers or mappings 
overlap.


> :
>
>   I found FileChannel.write(ByteBuffer,long) to be very slow
> on WinXP/7, by a factor 15, when the ByteBuffer is "large"
> (like 500Ko), even if it is direct (mapped or not), in which
> case it resolves to sun.nio.ch.IOUtils.writeFromNativeBuffer
> (which I supposed to always be fast)).
>   On Linux there was no slow down.
This is another long standing issue, see:

http://bugs.sun.com/view_bug.do?bug_id=6265734

It comes down to Windows not having the equivalent of a pread/pwrite 
that doesn't change the global position. There are ideas on how to solve 
this but it requires significant refactoring, hopefully some day.

> :
>
>   Intensive benches involving MBBs (namely FC to heap BB
> copies) were hanging from time to time, up to nearly a second,
> and then resumed at usual speed (slightly faster than temporary
> direct ByteBuffer approaches).
> ===>
>   As a result, I completely disabled MBBs usage for my copies.
Was this Windows 32-bit? We've seen a lot of issues with memory 
management on Windows where it takes time to write-out dirty pages and 
unmap the memory. I don't think we can do anything about this, assuming 
this is what you are running into. Windows 64-bit appears to be better 
although it can be a problem too when memory is over committed.

>
>
>
>
> 3) Non-performance remarks:
>
>
>   When copying between direct ByteBuffers, the efficient way
> is to use ByteBuffer.put(ByteBuffer), but its spec says that
> it's about equivalent to
> "while (src.hasRemaining()) dst.put(src.get());",
> i.e. that it is not suited if memory is shared (which no API
> allows to figure out) and srcPos < dstPos (as raw memmory
> positions), since then it could erase bytes to copy with
> copied bytes.
We could potentially clarify the spec here although warning about 
overlapping regions may require changes in several other places too.


> :
>
>   FileChannelImpl.truncate(long):
>   The bug we already talked about (the early return if
> size > size(), which masks the writability check, and the
> fact that size >= size() could be done instead)
> prevents reworking position whether truncation occurs or not,
> as the spec says (even if size > size(), position should be
> set to size if it is superior, but it isn't - or is it a
> spec bug?).
This is a corner case which is probably why it was never reported 
before. We have a bug for it since you brought it up:

http://bugs.sun.com/view_bug.do?bug_id=8000330

>
>
>   FileChannelImpl.map(...):
>   If mode is null, and assertions enabled, "assert (imode >= 0)"
> fails (I have somewhere in my head the idea that in JDK assertions
> should only check private code - well in a sense here it does :).
If mode is passed as null then NullPointerException should be thrown. 
Another 10+ year issue that hasn't been noticed. I'll create a bug for 
this - thanks!


>
>
>   FileChannelImpl.transferFrom(...):
>   This method can grow destination channel, but if the specified
> position (in dst) is > dst.size(), it just returns 0. It looks
> like a bug, as the spec says nothing about this surprising behavior.
This is specified: " If the given position is greater than the file's 
current size then no bytes are transferred"

>
>
>   FileChannel.write(ByteBuffer,long):
>   The Javadoc says that position is not updated, but if the channel
> is in append mode it might be (since then we have position = size,
> and this method can grow the file).
There is a long standing bug tracking updating the spec on this issue, see:

http://bugs.sun.com/view_bug.do?bug_id=6924219


Thanks for bug reports, you've gone in corner cases that our tests for 
this area clearly haven't gone into.

-Alan





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20121102/47ccd343/attachment-0001.html