Lifetime-Based Memory Management for Distributed Data Processing Systems

Lu Lu, Xuanhua Shi, Yongluan Zhou +5

In-memory caching of intermediate data and eager combining of data in shuffle buffers have been shown to be very effective in minimizing the re-computation and I/O cost in distribu…