The blog continues at suszter.com/ReversingOnWindows

August 13, 2012

An approach to detect signedness conversion

Here come the details of the code that is built on the top of the functionality involving detecting uninitialized read access to stack memory.

With the current implementation it's possible to make it visible how certain integers on the stack are being treated: whether signed or unsigned. This means if there is an integer that is accessed from multiple locations in the execution flow and we successfully determined how the integer was treated at each location in the execution flow we are able to tell if the integer is treated both signed and unsigned.

When there is a read access to the stack memory (a local variable read) and the instruction causing the read access exception is CMP we read the next instruction. If the next instruction is one of the followings the comparison is signed because these jumps based on signed comparisons: JG, JGE, JL, JLE, JNG, JNGE, JNL, JNLE. If the next instruction is one of the following the comparison is unsigned: JA, JAE, JB, JBE, JNA, JNAE, JNB, JNBE. Below is an example.

Signedness conversion is not a vulnerability but easily could be. For example, when the developer eliminates a signed/unsigned mismatch compiler warning in an if() condition by using explicit typecast. This could lead to the situation the program works normally but when the variable contains an unexpected value the execution continues on a different code path than it should due to the signedness conversion.

The signedness conversion could happen implicitly, too.

I wrote a test that you can see below to show how the code works.
// TestSignednessConversion.cpp : Example program for signedness conversion.
// Compile with optimization disabled.

#include "stdafx.h"

void set(int* value, int* value2)
{
    if (*value > 2)
    {
        *value = 2;
    }

    if (*value2 < 2)
    {
        *value2 = 2;
    }
}

void set2(unsigned int* value)
{
    if (*value > 2)
    {
        *value = 2;
    }
}

void set3(int* value2)
{
    if (*value2 < 0xffffffffU)
    {
        *value2 = 2;
    }
}

int _tmain(int argc, _TCHAR* argv[])
{
    int          value  = 3; // Treated both as signed and as unsigned (explicit cast)
    int          value2 = 1; // Treated both as signed and as unsigned (implicit)
    unsigned int value3 = 3; // Treated as unsigned only

    // value treated as signed
    // value2 treated as signed
    set(&value, &value2);

    // value treated as unsigned (explicit cast)
    set2((unsigned int*)&value);

    // value2 treated as unsigned (implicit)
    set3(&value2);

    // value3 treated as unsigned
    set2(&value3);

    return 0;
}
And here is the corresponding log of the program that is a Windbg extension. The log is verbose showing both read and write stack memory accesses. I highlighted the parts when the signedness of the comparison determined so you can match the parts to the source code above. You can also see when a variable is treated both as signed and as unsigned. U stands for unsigned comparison, S stands for signed comparison.
0:000> g wmain
eax=002b1a40 ebx=00000000 ecx=5d47471c edx=00000000 esi=00000001 edi=00403374
eip=00401070 esp=0018ff48 ebp=0018ff88 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
TestSignednessConversion!wmain:
00401070 55              push    ebp
0:000> l-t
Source options are 0:
    None
0:000> sxi av
0:000> !load c:\work\ext
0:000> !vprot esp
BaseAddress:       0018f000
AllocationBase:    00090000
AllocationProtect: 00000004  PAGE_READWRITE
RegionSize:        00001000
State:             00001000  MEM_COMMIT
Protect:           00000004  PAGE_READWRITE
Type:              00020000  MEM_PRIVATE
0:000> !region 56 18f000 1000
[Data Access] EIP=00401070 Data=0018ff44 W -
[Data Access] 00401070 55              push    ebp
[Data Access] EIP=00401076 Data=0018ff40 W -
[Data Access] 00401076 c745fc03000000  mov     dword ptr [ebp-4],3
[Data Access] EIP=0040107d Data=0018ff38 W -
[Data Access] 0040107d c745f401000000  mov     dword ptr [ebp-0Ch],1
[Data Access] EIP=00401084 Data=0018ff3c W -
[Data Access] 00401084 c745f803000000  mov     dword ptr [ebp-8],3
[Data Access] EIP=0040108e Data=0018ff34 W -
[Data Access] 0040108e 50              push    eax
[Data Access] EIP=00401092 Data=0018ff30 W -
[Data Access] 00401092 51              push    ecx
[Data Access] EIP=00401093 Data=0018ff2c W -
[Data Access] 00401093 e868ffffff      call    TestSignednessConversion!set (00401000)
[Data Access] EIP=00401000 Data=0018ff28 W -
[Data Access] 00401000 55              push    ebp
[Data Access] Value at 0018ff30 is read.
[Data Access] EIP=00401003 Data=0018ff30 R -
[Data Access] 00401003 8b4508          mov     eax,dword ptr [ebp+8]
[Data Access] Value at 0018ff40 is read.
[Data Access] EIP=00401006 Data=0018ff40 R S
[Data Access] 00401006 833802          cmp     dword ptr [eax],2
[Data Access] Value at 0018ff30 is read.
[Data Access] EIP=0040100b Data=0018ff30 R -
[Data Access] 0040100b 8b4d08          mov     ecx,dword ptr [ebp+8]
[Data Access] EIP=0040100e Data=0018ff40 W -
[Data Access] 0040100e c70102000000    mov     dword ptr [ecx],2
[Data Access] Value at 0018ff34 is read.
[Data Access] EIP=00401014 Data=0018ff34 R -
[Data Access] 00401014 8b550c          mov     edx,dword ptr [ebp+0Ch]
[Data Access] Value at 0018ff38 is read.
[Data Access] EIP=00401017 Data=0018ff38 R S
[Data Access] 00401017 833a02          cmp     dword ptr [edx],2
[Data Access] Value at 0018ff34 is read.
[Data Access] EIP=0040101c Data=0018ff34 R -
[Data Access] 0040101c 8b450c          mov     eax,dword ptr [ebp+0Ch]
[Data Access] EIP=0040101f Data=0018ff38 W -
[Data Access] 0040101f c70002000000    mov     dword ptr [eax],2
[Data Access] Value at 0018ff28 is read.
[Data Access] EIP=00401025 Data=0018ff28 R -
[Data Access] 00401025 5d              pop     ebp
[Data Access] Value at 0018ff2c is read.
[Data Access] EIP=00401026 Data=0018ff2c R -
[Data Access] 00401026 c3              ret
[Data Access] EIP=0040109e Data=0018ff34 W -
[Data Access] 0040109e 52              push    edx
[Data Access] EIP=0040109f Data=0018ff30 W -
[Data Access] 0040109f e88cffffff      call    TestSignednessConversion!set2 (00401030)
[Data Access] EIP=00401030 Data=0018ff2c W -
[Data Access] 00401030 55              push    ebp
[Data Access] Value at 0018ff34 is read.
[Data Access] EIP=00401033 Data=0018ff34 R -
[Data Access] 00401033 8b4508          mov     eax,dword ptr [ebp+8]
[Data Access] Value at 0018ff40 is treated both as signed and as unsigned. 
[Data Access] EIP=00401036 Data=0018ff40 R U
[Data Access] 00401036 833802          cmp     dword ptr [eax],2
[Data Access] Value at 0018ff2c is read.
[Data Access] EIP=00401044 Data=0018ff2c R -
[Data Access] 00401044 5d              pop     ebp
[Data Access] Value at 0018ff30 is read.
[Data Access] EIP=00401045 Data=0018ff30 R -
[Data Access] 00401045 c3              ret
[Data Access] EIP=004010aa Data=0018ff34 W -
[Data Access] 004010aa 50              push    eax
[Data Access] EIP=004010ab Data=0018ff30 W -
[Data Access] 004010ab e8a0ffffff      call    TestSignednessConversion!set3 (00401050)
[Data Access] EIP=00401050 Data=0018ff2c W -
[Data Access] 00401050 55              push    ebp
[Data Access] Value at 0018ff34 is read.
[Data Access] EIP=00401053 Data=0018ff34 R -
[Data Access] 00401053 8b4508          mov     eax,dword ptr [ebp+8]
[Data Access] Value at 0018ff38 is treated both as signed and as unsigned. 
[Data Access] EIP=00401056 Data=0018ff38 R U
[Data Access] 00401056 8338ff          cmp     dword ptr [eax],0FFFFFFFFh
[Data Access] Value at 0018ff34 is read.
[Data Access] EIP=0040105b Data=0018ff34 R -
[Data Access] 0040105b 8b4d08          mov     ecx,dword ptr [ebp+8]
[Data Access] EIP=0040105e Data=0018ff38 W -
[Data Access] 0040105e c70102000000    mov     dword ptr [ecx],2
[Data Access] Value at 0018ff2c is read.
[Data Access] EIP=00401064 Data=0018ff2c R -
[Data Access] 00401064 5d              pop     ebp
[Data Access] Value at 0018ff30 is read.
[Data Access] EIP=00401065 Data=0018ff30 R -
[Data Access] 00401065 c3              ret
[Data Access] EIP=004010b6 Data=0018ff34 W -
[Data Access] 004010b6 51              push    ecx
[Data Access] EIP=004010b7 Data=0018ff30 W -
[Data Access] 004010b7 e874ffffff      call    TestSignednessConversion!set2 (00401030)
[Data Access] EIP=00401030 Data=0018ff2c W -
[Data Access] 00401030 55              push    ebp
[Data Access] Value at 0018ff34 is read.
[Data Access] EIP=00401033 Data=0018ff34 R -
[Data Access] 00401033 8b4508          mov     eax,dword ptr [ebp+8]
[Data Access] Value at 0018ff3c is read.
[Data Access] EIP=00401036 Data=0018ff3c R U
[Data Access] 00401036 833802          cmp     dword ptr [eax],2
[Data Access] Value at 0018ff34 is read.
[Data Access] EIP=0040103b Data=0018ff34 R -
[Data Access] 0040103b 8b4d08          mov     ecx,dword ptr [ebp+8]
[Data Access] EIP=0040103e Data=0018ff3c W -
[Data Access] 0040103e c70102000000    mov     dword ptr [ecx],2
[Data Access] Value at 0018ff2c is read.
[Data Access] EIP=00401044 Data=0018ff2c R -
[Data Access] 00401044 5d              pop     ebp
[Data Access] Value at 0018ff30 is read.
[Data Access] EIP=00401045 Data=0018ff30 R -
[Data Access] 00401045 c3              ret
[Data Access] Value at 0018ff44 is read.
[Data Access] EIP=004010c3 Data=0018ff44 R -
[Data Access] 004010c3 5d              pop     ebp
[Data Access] Value at 0018ff48 is read.
[Data Access] EIP=004010c4 Data=0018ff48 R -
[Data Access] 004010c4 c3              ret
Break reason: 00000010
In the log l-t is set to disable step to next source line and use step to next instruction instead. sxi av is used to let the event callback implementation handle the exception. 18f000 is stack base address 1000 is the size. 56 is to set some flags including verbose logging, etc.
  This blog is written and maintained by Attila Suszter. Read in Feed Reader.