May 2, 2012

An example when /sdl flag doesn't work in Visual Studio 11 Beta

Microsoft introduced a new compiler defense in Visual Studio 11 Beta that is to mitigate simple dangling pointer bugs. When the memory has been freed using the delete operator the pointer is set to 0x8123.

Here is an example when 0x8123 set.
    delete [] ptr;
push edi
call dword ptr ds:[12E2094h]
mov eax,8123h
test edi,edi
cmovne edi,eax

However, consider the following code snippet.
void* arr[1];

arr[0] = new BYTE[MAX];

delete [] arr[0];
delete [] arr[0];
I compiled the code and realized there is no sanitization.
push esi
    void* arr[1];

    arr[0] = new BYTE[MAX];
push 7Bh
call dword ptr ds:[12A2090h]
mov esi,eax


    delete [] arr[0];
push esi
call dword ptr ds:[12A2094h]

    delete [] arr[0];
push esi
call dword ptr ds:[12A2094h]
add esp,0Ch


    return 0;
xor eax,eax
pop esi

I asked Microsoft about this issue, and they confirmed that the /sdl flag doesn't work in this scenario. The reason is that in line delete [] arr[0], the expression arr[0] passed into the delete call is dereference. VS11 provides initial support for pointer sanitization when "the expression passed into the delete call does not involve a dereference." For more info see their blog post.
  This blog is written and maintained by Attila Suszter. Read in Feed Reader.