The Operating System (OS) kernel is a key component of modern computing infrastructure, yet it is prone to numerous vulnerabilities, many of which cause memory corruptions that can be exploited by attackers to perform malicious activities. While various techniques have been introduced to secure the Linux kernel, it still constantly gets compromised. CVE-2021-3715 is a kernel bug in the Linux system, which persisted for over six years and was initially fixed without significant attention due to its perceived low severity, leading to it remaining unaddressed in distro kernels for another year until further discoveries revealed its potential for use-after-free memory corruption and exploitation. In 2005, Microsoft introduced Data Execution Prevention (DEP) in Windows XP SP2 to prevent code execution on user memory by marking it as non-executable, but this approach was soon thwarted by Return Oriented Programming (ROP), a technique that allowed attackers to execute code by utilizing code gadgets within the execution region. The above incidents underline the fact that a lack of understanding of kernel exploitability could compromise security. Therefore, I propose to conduct research on understanding kernel exploitability and then reduce exploitability. Kernel exploitation is a process of programming a "weird machine". Analyzing exploitability, which is finding a transition path from the entry point to the exploitation goal in the machine, is naturally challenging as the weird machine remains largely unknown. To address the challenge, I propose analyzing vulnerability capability and exploitation composability. In this dissertation, I develop techniques that combine static and dynamic analysis to explore kernel vulnerability capability, and introduce innovative exploitation techniques capable of bypassing all existing defenses. With this understanding of exploitability, I then propose a mitigation strategy designed to prevent memory corruption, delivering effective protection with minimal overhead. In the future, I aim to continuously investigate the composability of kernel exploitation. Due to its complexity, which precludes deduction through any general formal algorithm, I will progressively analyze this composability, piecing together its intricate portrait. Additionally, the current mitigation strategy in this dissertation operates within userspace. Its adaptation to the kernel space presents a unique set of challenges. My intention is to extend the solution into the kernel space, optimizing overhead through a combination of systematic methodologies and quantitative evaluations.

Creator

• Lin, Zhenpeng

DOI

• https://doi.org/10.21985/n2-xdds-he60

Subject

• Computer science

Language

• en

Alternate Identifier

• etdadmin_upload_1010237
• http://dissertations.umi.com/northwestern:16700

Keyword

• Weird machine
• Kernel vulnerability
• Exploitation
• Exploitability

Date created

• 2023-01-01

Resource type

• Dissertation

Rights statement

• In Copyright