On the Impact of Memory Allocation on High-Performance Query Processing
arXiv:1905.01135 · doi:10.1145/3329785.3329918
Abstract
Somewhat surprisingly, the behavior of analytical query engines is crucially affected by the dynamic memory allocator used. Memory allocators highly influence performance, scalability, memory efficiency and memory fairness to other processes. In this work, we provide the first comprehensive experimental analysis on the impact of memory allocation for high-performance query engines. We test five state-of-the-art dynamic memory allocators and discuss their strengths and weaknesses within our DBMS. The right allocator can increase the performance of TPC-DS (SF 100) by 2.7x on a 4-socket Intel Xeon server.
Cited by in corpus (5)
- DuckDB-SGX2: The Good, The Bad and The Ugly within Confidential Analytical Query Processing
- MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training
- Quantifying Daily Evolution of Mobile Software Based on Memory Allocator Churn
- GreenMalloc: Allocator Optimisation for Industrial Workloads
- There Ain't No Such Thing as a Free Custom Memory Allocator