Fast Local Page-Tables for Virtualized NUMA Servers with vMitosis

Authors

Ashish Panwar, Reto Achermann, Arkaprava Basu, Abhishek Bhattacharjee, K. Gopinath and Jayneel Gandhi

Venue

Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'21)

Links

[ .doi ] [ .pdf ] [ .bib ]

Abstract

Increasing heterogeneity in the memory system mandates careful data placement to hide the non-uniform memory access (NUMA) effects on applications. However, NUMA optimizations have predominantly focused on application data in the past decades, largely ignoring the placement of kernel data structures due to their small memory footprint; this is evident in typical OS designs that pin kernel objects in memory. In this paper, we show that careful placement of kernel data structures is gaining importance in the context of page-tables: sub-optimal placement of page-tables causes severe slowdown (up to 3.1x) on virtualized NUMA servers.

In response, we present vMitosis -- a system for explicit management of two-level page-tables, i.e., the guest and extended page-tables, on virtualized NUMA servers. vMitosis enables faster address translation by migrating and replicating page-tables. It supports two prevalent virtualization configurations: first, where the hypervisor exposes the NUMA architecture to the guest OS, and second, where such information is hidden from the guest OS. vMitosis is implemented in Linux/KVM, and our evaluation on a recent 1.5TiB 4-socket server shows that it effectively eliminates NUMA effects on 2D page-table walks, resulting in a speedup of 1.8-3.1x for Thin (single-socket) and 1.06-1.6x for Wide (multi-socket) workloads.

Bibtex

@inproceedings{Panwar:2021:FLP,
 author = {Panwar, Ashish and Achermann, Reto and Basu, Arkaprava and Bhattacharjee, Abhishek and Gopinath, K. and Gandhi, Jayneel},
 booktitle = {Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems},
 doi = {10.1145/3445814.3446709},
 id = {Panwar:2021:FLP},
 isbn = {9781450383172},
 location = {Virtual, USA},
 pages = {194–210},
 publisher = {Association for Computing Machinery},
 series = {ASPLOS'21},
 title = {Fast Local Page-Tables for Virtualized NUMA Servers with vMitosis},
 url = {https://doi.org/10.1145/3445814.3446709},
 year = {2021}
}

Contact

The University of British Columbia
Department of Computer Science
2366 Main Mall
ICICS Building, Office 341
Vancouver, BC V6T 1Z4
Canada

achreto [at] cs.ubc.ca
+1 604 827 2446