best practice for processing large binary file

Andrew M andrew at oc384.net
Tue Sep 27 10:58:23 PDT 2011


On 9/27/2011 8:05 AM, Alan Bateman wrote:
> Are you looking to access these 56 byte records sequentially? If so then
> SeekableByteChannel might not be interested as it's just a ByteChannel
> that maintains a position so is more interesting when the access isn't
> sequential. Also the newBufferedReader method returns a BufferedReader
> which is for processing text files and it sounds like this is binary
> data. If you have lots of memory then by all means try out mapping file
> into memory. A MappedByteBuffer (actually ByteBuffer) defines method for
> accessing binary data which I think you are looking for. You could start
> a thread that invokes the MappedByteBuffer load method to force it to be
> loaded in memory. That said, depending on the usage a simple
> DataInputStream that wraps a buffered stream might be sufficient for
> your needs.

Yes I just need one-time sequential access.  Historically I have just 
used a DataInputStream like you mention but I'm worried that by still 
doing that I may now be passing up some performance gains offered by 
newer methods which may implement faster, lower level file access or 
better memory mgmt, etc.  Also if I could just instantiate some 
ByteBuffer and read from it without having to start producer and 
consumer threads that would make for prettier code.

-Andrew



More information about the nio-discuss mailing list