SYSTEMATIC METHODS FOR MEMORY ERROR DETECTION AND EXPLOITATION

Abstract

Memory errors are persistent threats to computer systems. Memory error exploits have resulted in severe damage in real-world programs. At the same time, exploit mechanisms are rapidly evolving to bypass known protections. To prevent the damage, we need to detect them in advance and predict their evolving trend.

In this thesis, we propose three novel solutions to systematically detect memory errors and explore new exploits. First, we detect memory errors shown in the privilege-based isolation, where the unexpected memory access can cross isolated partitions. Then we look into new exploit mechanisms in the data space. We propose data-flow stitching, a novel method to connect disjoint data flows for severe attacks. Data-flow stitching significantly enlarges the capability of data-oriented attacks. Finally, we explore the expressiveness of data-oriented attacks. We propose data-oriented programming (DOP), which selectively stitches basic data-flow gadgets for a desired purpose. With DOP, we build Turing-complete data-oriented attacks.