PING: Linux: Support transparent hugepages

[Igor Veresov]

On 4/1/11 11:38 AM, Andrew Haley wrote:
> On 04/01/2011 07:24 PM, Igor Veresov wrote:
>> Why is alignment required? Afaik it's perfectly fine to have mixed page
>> sizes in a single segment.
>
> Alignment is the only way to guarantee that all pages are huge pages.

Right, but my point was that +UseLargePages option would do that 
guaranteeing alignment.

>
>> Also, if use use +UseLargePages, all the heap and generation sizes
>> will be largepage-alinged anyway. Also, some things like the
>> NUMA-allocator will work better is they know kind of pages the
>> underlying segment has.
>
> Yes, but on Linux at least +UseLargePages (using all that SHM stuff)
> needs special permissions, so it tends not to be used as much as it
> could.

Right, but you can choose whether to do via SHM or by just doing madvise 
at runtime. This way your code will nicely fit into the existing 
framework - you'll get automatic page size detection and hence all the 
requests from the runtime for memory will come properly aligned.

>
>> What I would've done is put some logic to distinguish kernels that
>> support transparent large pages and instead of using all the SHM
>> stuff just use a mmap + madvise(MADV_HUGEPAGE).
>
> Actually, I did!  I tried both of these, and simply aligning and
> letting transparent hugepages do the work seemed easier.

Perhaps, but the rest of the runtime has no idea that you have large 
pages. So, for example, the NUMA allocator will case accumulation of 
small pages on the borders of per-node chunks because it would think 
that we're using small pages and will adapt the sizes with small page 
granularity, causing fragmentation.

> If you
> prefer mmap + madvise I'll submit a patch that does that instead.

I think this would make it fit in the existing framework better. Take a 
look at os_solaris.cpp, it has exactly all the same problems. First 
there was the SHM implementation (grep for UseISM) , that is still used 
for ancient Solaris versions, and then there is the MPSS version, which 
is similar to what's now in Linux.

So, basically I would like the code for both OSes look similar and 
follow the same logic.

igor

>
> Both ways solve the problem of needing special permissions.
>
>> On Solaris we do kind of the
>> same thing, but there you have to do mmap + memcntl(MC_HAT_ADVISE).
>> Although Solaris 10 also supports out-of-the-box large pages, so in
>> theory you don't have to do anything, but I guess it's always better to
>> hint the OS that you really want it.
>
> OK.
>
> Andrew.