Previous [ 1] [ 2] [ 3] [ 4] [ 5] [ 6] [ 7] [ 8] [ 9] [ 10] [ 11] [ 12] [ 13] [ 14] [ 15] [ 16] [ 17] [ 18] [ 19] [ 20] [ 21] [ 22] [ 23] [ 24]

@

Journal of Information Science and Engineering, Vol. 27 No. 3, pp. 869-890 (May 2011)

A Robust Kernel-Based Solution to Control-Hijacking Buffer Overflow Attacks*

LI-HAN CHEN, FU-HAU HSU, CHENG-HSIEN HUANG, CHIH-WEN OU, CHIA-JUN LIN AND SZU-CHI LIU
Department of Computer Science and Information Engineering
National Central University
Taoyuan, 320 Taiwan

In this paper, we propose a robust kernel-based solution, called AURORA, to a ubiquitous security problem V control-hijacking Buffer Overflow Attacks (BOAs). AURORA utilizes either the addresses of the buffers storing input strings or signatures to detect and block control-hijacking BOA strings in the kernel, including zero-day ones. Although AURORA detects some types of BOAs through signatures, AURORA does not need to create any new signature for new attack instances after its installation because AURORAs signatures are created based on commonality of control-hijacking BOAs. Moreover, even a process is under a BOA, AURORA allows it to continue its execution or to be terminated gracefully without the cost of process idleness or repeated process crashes. Thus, AURORA is robust to control-hijacking BOAs. AURORA does not need to modify the source code of any application programs. Furthermore, AURORA is compatible with existing operating systems and application programs; hence, AURORA could work with other protection mechanisms to provide an extra layer of protection. Our experimental results show that with less than 1% overhead and negligible false positives, AURORA can accurately block various control-hijacking BOAs

Keywords: buffer overflow attack, stack smashing attack, return-into-libc attack, AURORA, control hijacking BOA

Full Text () Retrieve PDF document (201105_05.pdf)

Received October 19, 2009; revised January 26, 2010; accepted March 3, 2010.
Communicated by Chin-Laung Lei.
* This paper was partially supported by National Science Council open source project and Advanced Communication Laboratory in Nation Central University. The number of the project is NSC97-2218-E-008-006.