Improving the Security of the Heap through Inter-Process Protection and Intra-Process Temporal Protection
No Thumbnail Available
Files
Date
2005-12-07
Authors
Journal Title
Series/Report No.
Journal ISSN
Volume Title
Publisher
Abstract
In most current implementations, memory allocations and deallocations are performed by user-level library code which keeps heap meta-data (heap structure information) and the application's heap data stored in an interleaved fashion in the same address space. Such implementations are inherently unsafe: they allow attackers to use application's vulnerabilities (e.g. lack of heap-based buffer overflow checking) to corrupt its heap meta-data in order to execute malicious code or cause denial-of-service. In this dissertation, we propose an approach where heap meta-data and heap data are protected separately.
Our first solution exploits existing inter-process protection mechanisms through Heap Server, a separate process that maintains heap meta-data on behalf of the application and runs in parallel with it. To perform memory allocations and deallocations, the application sends requests to the Server, which responds to the requests and updates the meta-data. Since the heap meta-data is kept in the Heap Server's address space, attacks on the application can no longer corrupt it. Heap Server is directly implementable in current systems because it does not require new hardware. To optimize Heap Server's performance, we explore non-blocking communication, bulk deallocation, and pre-allocation optimizations. Evaluated on a real system, a fully-optimized Heap Server performs almost identical to a Base heap management library with no protection mechanisms.
As an alternative solution, we propose a new User-level Temporal Intra-Process Protection (UTIPP) mechanism in which a process protects itself from its own vulnerabilities by write-protecting its own heap meta-data, and only removing the protection for legitimate stores in the heap management library. Unlike existing kernel-level page protection which can only be modified in the privileged mode, UTIPP allows a process to modify the new write-protection bit with a single instruction without disrupting normal pipeline flow. Evaluated on a cycle-accurate simulator, UTIPP adds negligible overhead in most benchmarks.
Another contribution of this dissertation is a new heap layout obfuscation technique which relies on randomizing the space between heap chunks and the order of chunks in the heap, making heap data attacks more difficult. This obfuscation is integrated with Heap Server and UTIPP.
Description
Keywords
Obfuscation, Intra-Process Temporal Protection, Heap Management, Computer Security, Heap Security, Heap Server, UTIPP
Citation
Degree
PhD
Discipline
Computer Engineering
