madvise(ptr, len, MADV_SEQUENTIAL)

[Uwe Schindler]

But then we also need to map, when writing to files, which is hard to do,
because you do not know how large the mapping area will be (unknown
filesize). As Earwin suggested, we could change MMapDirectory to also mmap
on writing, but maps the buffers in something we call "pages". Filesizes of
written files are then always multiples of this "page size", if this is not
a problem for lucene (e.g. making the random access file larger than
actually needed, having 0-bytes at the unused end). Maybe we need than an
api to tell the directory implementation, how large a segment may become
before writing (when merging, this should be possible, as the filesize is
about the sum of all merged segments).

 

On windows, we then will also rely on the cleaner()-hack to correctly close
files (see current MMap in lucene-trunk, where the cleaner-hack can
optionally be enabled, without guaranties).

 

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe at thetaphi.de

  _____  

From: Jason Rutherglen [mailto:jason.rutherglen at gmail.com] 
Sent: Friday, June 19, 2009 12:33 AM
To: Alan Bateman
Cc: java-dev at lucene.apache.org; nio-discuss at openjdk.java.net
Subject: Re: madvise(ptr, len, MADV_SEQUENTIAL)

 

Hmm... So the list at the bottom of this page looks accurate?
http://www.gnu.org/software/hello/manual/gnulib/posix_005ffadvise.html

If it is, then posix_fadvise works on Linux only?  

Perhaps madvise will be better then (judging by the much smaller unsupported
list)?  It seems to run on most platforms:
http://www.gnu.org/software/hello/manual/gnulib/madvise.html

On Wed, Jun 17, 2009 at 2:19 AM, Alan Bateman <Alan.Bateman at sun.com> wrote:

Jason Rutherglen wrote:

Alan,

Do you think something like FileDescriptor.setAdvise (mirroring
posix_fadvise) makes sense?

-J

Something like a posix_fadvise would be more appropriate for FileChannel or
maybe as a usage hint when opening the file (the new APIs for opening files
are extensible to allow for additional options in the future or even
implementation specific options). I don't think we've had much interest in
doing this, maybe because it would be a no-op on many operating systems.

-Alan.

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.openjdk.java.net/pipermail/nio-discuss/attachments/20090619/293a8774/attachment.html