Buffer overflow protection
A "Stack Smash Attack" is a type of attack based on a "Buffer Overflow" which raises serious computer security vulnerabilities. There have been many attempts to squelch these, one of the most common being StackGuard.
How It Works
StackGuard modifies the organization of data on the stack to include a "canary" value which, when destroyed, shows that a buffer before it has been overflowed. This imposes a negligible performance impact while giving the benefit of preventing an entire class of attacks.
An Example
Normal buffer allocation for x86 architectures and other similar architectures is shown in the Buffer Overflow entry. Here, we will show the modified process as it pertains to StackGuard
Before calling a function, the stack looks like this:
(DATA)(DATA)(...)
When a function is called, the return address and control information is pushed onto the stack, as so:
(CTL_INFO)(RETADDR)(DATA)(DATA)(...)
In our example, a buffer `char a[10]` is allocated on the stack. Before we can do this, we must stick a "canary" on the stack. This canary is one of three types, which we will get to later. The stack looks like this now:
(CANARY)(CTL_INFO)(RETADDR)(DATA)(DATA)(...)
Next, the buffer is allocated, giving the following view:
(a.........)(CANARY)(ADDR)(DATA)(DATA)(...)
When the buffer is deallocated at the function's end, the canary is checked against data somewhere else in ram. If the check passes, the program continues. If not, the program crashes.
Let's consider what happens if the 10 byte buffer 'a' is overflowed with a 14 byte buffer. Here's the general view:
(0123456789)(0123RY)(ADDR)(DATA)(DATA)(...)
The first four bytes of the canary are overwritten by the last 4 bytes of the buffer 'a'. It won't pass the check, and the program will end. Note that without the canary, the program would return to whatever those last four bytes point to, altering the execution flow. If the buffer was longer than 14 bytes, an attacker could inject executable code (shellcode) or return to existing code ((ret2libc).
This technique adds a few instructions of overhead for every dynamic buffer allocation and deallocation. The overhead generated in this technique is not significant. It does work, though, unless the canary remains unchanged. If the attacker can guess the canary, and knows that it's there, he may simply copy over it with itself. This is highly implausable, unless he can sniff the program in realtime and halt its execution. But then, of course, he would have control over it anyway, and wouldn't need a buffer overflow exploit.
Canaries
Canaries are known values that are placed between a buffer and control data on the stack to monitor buffer overflows. When The buffer overflows, it will clobber the canary, making the overflow evident.
There are three types of canaries: Terminator, Random, and Random XOR.
Terminator Canaries
Terminator Canaries are based on the observation that most buffer overflows and stack smash attacks are based on certain string operations which end at terminators. The reaction to this observation is that the canaries are built of NULL terminators, CR, LF, and -1. The undesirable result is that the canary is known. Even with the stack smash protection, an attacker could potentially overwrite the canary and control information, then go back and write a shorter overflow to fix the canary. This is only effective in rare cases where double-overflows are possible.
Random Canaries
Random canaries are randomly generated, usually from an [entropy gathering daemon]], so an attacker doesn't know what it is. It is not usually logically possible or plausable to read the canary for exploiting; the canary is a secure value known only by those who need to know it, the SSP code in this case.
Normally, a random canary is generated at program initialization, and stored in a global variable. This variable is usually padded by unmapped pages, so that attempting to read it using any kinds of tricks that exploit bugs to read off ram cause a segmentation fault, terminating the program. It may still be possible to read the canary, if the attacker knows where it is, or can get the program to read from the stack.