In case you want to see some examples what I mean, earlier last year, I wrote a little about experiences regarding signed/unsigned comparisons.
Some time ago, I started developing a Windbg plugin command that has a tracing functionality, and the ability to break in the debugger when a signed comparison is reached. However, if EIP is not in user defined range e.g. due to an API call, the program executes normally. When EIP is in the user defined range again the program resumes tracing.
I was able to trace some function in a Visual C++ project, but it was needed to run
l-tcommand beforehand to step by assembly instructions rather than source lines. Here is how to use Windbg in VS.
This plugin can be extended to work with other signed instructions than signed comparison ones. In addition, the plugin can be extended to execute the program until comparison is reached rather than to trace, in a similar way to the working of the
One possible area to explore further is to record how the data that is read from the memory is treated in point of signedness. Also, to detect any weak points to attack, or even to detect signedness conversions.