Memory Mapped file segment / always appending

[Maurizio Cimadamore]

On 27/07/2020 18:12, Johannes Lichtenberger wrote:
> Hi,
>
> as I'm always appending data to the end of a file. I came up with the idea
> to always map a segment in chunks of Integer.MAX, that is the segment
> always get's closed and a new segment gets created with twice the size (for
> reading/writing). When I'm closing the writer I'm simply truncating the
> file(s) to the real length.
>
> However, I just saw that it's horribly slow for a lot of small writes. That
> is I'm always creating a read/write page reader, and closing it again after
> a write. However, truncating a lot of unused space, over and over again is
> a real problem. I've attached a flame graph from YourKit showing that half
> of the time over a large amount of CPU samples is spend with truncating.

Is the time spent in the Panama API, or do you mean that the code is 
slow just natively (e.g. at the mmap level)

We can't see the attachment (the server truncates them :-) )

Maurizio

>
> Any ideas, how to improve performance?
>
> The packe is this and very small:
>
> https://github.com/sirixdb/sirix/tree/master/bundles/sirix-core/src/main/java/org/sirix/io/memorymapped
>
> kind regards
> Johannes