METRIC: Tracking Memory Bottlenecks via Binary Rewriting

Abstract

Over recent decades, computing speeds have grown much faster than memory access speeds. This differential rate of improvement between processor speeds and memory speeds has led to an ever-increasing processor-memory gap. Overall computing speeds for most applications are now dominated by the cost of their memory references. Furthermore, memory access costs will grow increasingly dominant as the processor-memory gap widens. In this scenario, characterizing and quantifying application program memory usage to isolate, identify and eliminate memory access bottlenecks will have significant impact on overall application computing performance.

This thesis presents METRIC, an environment for determining memory access inefficiencies by examining access traces. This thesis makes three primary contributions. First, we present methods to extract partial access traces from running applications by observing their memory behavior via dynamic binary rewriting. Second, we present a methodology to represent partial access traces in constant space for regular references through a novel technique for online compression of reference streams. Third, we employ offline cache simulation to derive indications about memory performance bottlenecks from partial access traces. By examining summarized and by-reference metrics as well as cache evictor information, we can pinpoint the sources of performance problems. We perform validation experiments of the framework with respect to accuracy, compression and execution overheads for several benchmarks. Finally, we demonstrate the ability to derive opportunities for optimizations and assess their benefits in several case studies, resulting in up to 40% lower miss ratios.