Changes to RandomAccessFile in jdk9

[Enrico Olivelli]

Il ven 20 apr 2018, 18:32 Ismael Juma <mlists at juma.me.uk> ha scritto:

> Hi all,
>
> I'm a Kafka committer and noticed this discussion. Kafka uses
> RandomAccessFile.setLength in two places.
>
> For the memory mapped index files:
>
> protected var mmap: MappedByteBuffer = {
>     val newlyCreated = file.createNewFile()
>     val raf = if (writable) new RandomAccessFile(file, "rw") else new
> RandomAccessFile(file, "r")
>     try {
>       /* pre-allocate the file if necessary */
>       if(newlyCreated) {
>         if(maxIndexSize < entrySize)
>           throw new IllegalArgumentException("Invalid max index size: " +
> maxIndexSize)
>         raf.setLength(roundDownToExactMultiple(maxIndexSize, entrySize))
>       }
>
>       /* memory-map the file */
>       _length = raf.length()
>       val idx = {
>         if (writable)
>           raf.getChannel.map(FileChannel.MapMode.READ_WRITE, 0, _length)
>         else
>           raf.getChannel.map(FileChannel.MapMode.READ_ONLY, 0, _length)
>       }
>       /* set the position in the index for the next entry */
>       if(newlyCreated)
>         idx.position(0)
>       else
>         // if this is a pre-existing index, assume it is valid and set
> position to last entry
>         idx.position(roundDownToExactMultiple(idx.limit(), entrySize))
>       idx
>     } finally {
>       CoreUtils.swallow(raf.close(), this)
>     }
>   }
>
> For the log segment files (written sequentially via FileChannel) if
> preallocate is true:
>
>     /**
>      * Open a channel for the given file
>      * For windows NTFS and some old LINUX file system, set preallocate to
> true and initFileSize
>      * with one value (for example 512 * 1025 *1024 ) can improve the
> kafka produce performance.
>      * @param file File path
>      * @param mutable mutable
>      * @param fileAlreadyExists File already exists or not
>      * @param initFileSize The size used for pre allocate file, for
> example 512 * 1025 *1024
>      * @param preallocate Pre-allocate file or not, gotten from
> configuration.
>      */
>     private static FileChannel openChannel(File file,
>                                            boolean mutable,
>                                            boolean fileAlreadyExists,
>                                            int initFileSize,
>                                            boolean preallocate) throws
> IOException {
>         if (mutable) {
>             if (fileAlreadyExists) {
>                 return new RandomAccessFile(file, "rw").getChannel();
>             } else {
>                 if (preallocate) {
>                     RandomAccessFile randomAccessFile = new
> RandomAccessFile(file, "rw");
>                     randomAccessFile.setLength(initFileSize);
>                     return randomAccessFile.getChannel();
>                 } else {
>                     return new RandomAccessFile(file, "rw").getChannel();
>                 }
>             }
>         } else {
>             return new FileInputStream(file).getChannel();
>         }
>     }
>
> Enrico, the change in behaviour you have seen is for the index files? I
> suspect so as `preallocate` is only typically enabled by Windows users.
>

I am running brokers on Linux with mostly default values.
The change is about index files


Enrico


> Ismael
>
> On Fri, Apr 20, 2018 at 6:50 AM, Alan Bateman <Alan.Bateman at oracle.com>
> wrote:
>
>> On 20/04/2018 13:09, Enrico Olivelli wrote:
>>
>>>
>>> Alan, Ivan
>>> Thank you for spending time on this.
>>>
>>> Do you think there will be some change (change to previous behavior) or
>>> simply some consolidation on current behavior (docs, tests in the codebase)?
>>>
>>> Should I file some issue on the bugtracker?
>>>
>> I'm confident that Ivan is correct (I have forgotten this was fixed in
>> RandomAccessFile at the same time as FileChannel). So it is a deliberate
>> change and one that should probably have had a release note.
>>
>> -Alan
>>
>
> --


-- Enrico Olivelli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20180420/284f158d/attachment.html>