Скачать или смотреть Large Memory Management issues: Performance, Fragmentation, Movable objects and Huge Page overhead.

Christopher Lameter

http://lca2018.linux.org.au/schedule/...

Memory management with larger memory sizes of around a couple of
terabyte leads to a number of issues that make life hard. This talk is
a look at the issues and at the work in progress on these issues.

First, huge pages are not supported for the page cache which leads to
numerous performance issues. It begins with the long time it takes to
just copy a file of a few gigs in size (minor compared to the size of
the memory in the system of a few terabytes after all...) to the
performance problems when accessing large amounts of 4k pages from a
memory mapped file. Since huge pages are not supported TLB trashing
often becomes an issue. Devices have issues writing large blocks of
data to disk due to large scatter gather lists having to be processed
in memory. Reclaim becomes slow since the system is literally churning
through billions of 4k page to figure out which ones to evict.

Second, huge pages cause a memory overhead that surprises users. The
kernel manages 4k chunks of the huge page in many cases and needs to
have metadata structures for each 4k page. Other metadata also exists.
This results in gigabytes of memory just not being present. The user
bought a system with a lot of memory but the Linux just cannot use it.

Third, fragmentation becomes an issue in particular if huge pages and
4k pages are frequently allocated and freed (like we have to because
the page cache does not support 4k after all). So over time systems
have to be rebooted to allow the allocation of huge pages. Huge pages
are actually only a small problem because devices on large systems
often want to allocate large contiguous buffers for I/O (like
specialized networking devices using RDMA).

This talk was given at Linux.conf.au 2018 (LCA2018) which was held on 22-26 January 2018 in Sydney Australia.

linux.conf.au is a conference about the Linux operating system, and all aspects of the thriving ecosystem of Free and Open Source Software that has grown up around it. Run since 1999, in a different Australian or New Zealand city each year, by a team of local volunteers, LCA invites more than 500 people to learn from the people who shape the future of Open Source. For more information on the conference see https://linux.conf.au/

#linux.conf.au #linux #foss #opensource