best practice for processing large binary file
Andrew M
andrew at oc384.net
Tue Sep 27 10:58:23 PDT 2011
On 9/27/2011 8:05 AM, Alan Bateman wrote:
> Are you looking to access these 56 byte records sequentially? If so then
> SeekableByteChannel might not be interested as it's just a ByteChannel
> that maintains a position so is more interesting when the access isn't
> sequential. Also the newBufferedReader method returns a BufferedReader
> which is for processing text files and it sounds like this is binary
> data. If you have lots of memory then by all means try out mapping file
> into memory. A MappedByteBuffer (actually ByteBuffer) defines method for
> accessing binary data which I think you are looking for. You could start
> a thread that invokes the MappedByteBuffer load method to force it to be
> loaded in memory. That said, depending on the usage a simple
> DataInputStream that wraps a buffered stream might be sufficient for
> your needs.
Yes I just need one-time sequential access. Historically I have just
used a DataInputStream like you mention but I'm worried that by still
doing that I may now be passing up some performance gains offered by
newer methods which may implement faster, lower level file access or
better memory mgmt, etc. Also if I could just instantiate some
ByteBuffer and read from it without having to start producer and
consumer threads that would make for prettier code.
-Andrew
More information about the nio-discuss
mailing list