The blog continues at suszter.com/ReversingOnWindows

January 30, 2016

Managed out-of-bound access in DeflateStream (.NET)

Recently I made a test to see the robustness of the Deflate algorithm in .NET Framework. It was written in Visual Studio 2013 using C# and DeflateStream class.

The test came back with one issue worth mentioning in this blog.

Background

IndexOutOfRangeException exception is thrown when the program tries to access out of the buffer boundaries. When this exception is thrown it indicates a missing or insufficient sanity check. When the sanity check works as expected it should prevent the exception from being thrown.

Microsoft explains like this.
Typically, an IndexOutOfRangeException exception is thrown as a result of developer error. Instead of handling the exception, you should diagnose the cause of the error and correct your code.
DeflateStream performs sanity check on the stream during the decompression and when a corrupted stream is detected InvalidDataException is being thrown.

The Issue

The issue the test hit is like this. During the decompression a specially crafted stream can bypass the sanity check and so the code doesn't throw InvalidDataException. Instead it tries to access out of boundaries and IndexOutOfRangeException is thrown.

The isolated testcase to reproduce the issue is like this.

using System;
using System.IO.Compression;
using System.IO;

namespace DeflateTestCase
{
    class Program
    {
        static Byte[] compressedData = {
            0x04, 0xDF, 0x03, 0x20, 0xFC, 0xA1, 0x6F, 0x85, 0xF2, 0x2B, 0xC5, 0xA4, 0xAA, 0x8A, 0xC8, 0xB4,
            0xDE, 0x3A, 0x5C, 0x06, 0xA2, 0x8C, 0xD9, 0x39, 0x41, 0xCB, 0xA6, 0x34, 0xDD, 0xCA, 0xC4, 0x2C,
            0x80, 0x7C, 0xCC
        };

        static void Main(string[] args)
        {
            using (MemoryStream compressedStream = new MemoryStream(compressedData))
            {
                MemoryStream resultStream = new MemoryStream();
                using (DeflateStream decompressionStream = new DeflateStream(compressedStream, CompressionMode.Decompress))
                {
                    try
                    {
                        decompressionStream.CopyTo(resultStream);
                    }
                    catch (System.IO.InvalidDataException e)
                    {
                        System.Console.WriteLine("{0}", e.Message);
                    }
                    catch (System.IndexOutOfRangeException e)
                    {
                        System.Console.WriteLine("{0}", e.Message);
                        System.Console.WriteLine("{0}", e.StackTrace);
                    }
                }
            }
        }
    }
}

Stack trace.

c:\Dev\DeflateTestCase\bin\Release>DeflateTestCase.exe
Index was outside the bounds of the array.
   at System.IO.Compression.HuffmanTree.CreateTable()
   at System.IO.Compression.HuffmanTree..ctor(Byte[] codeLengths)
   at System.IO.Compression.Inflater.DecodeDynamicBlockHeader()
   at System.IO.Compression.Inflater.Decode()
   at System.IO.Compression.Inflater.Inflate(Byte[] bytes, Int32 offset, Int32 length)
   at System.IO.Compression.DeflateStream.Read(Byte[] array, Int32 offset, Int32 count)
   at System.IO.Stream.InternalCopyTo(Stream destination, Int32 bufferSize)
   at System.IO.Stream.CopyTo(Stream destination)
   at DeflateTestCase.Program.Main(String[] args) in c:\Dev\DeflateTestCase\Program.cs:line 24

Consequence

If DeflateStream is used (either implicitly or expicitly) to decompress data and the data can come from untrusted source IndexOutOfRangeException is expected to happen and it should be handled in order to prevent the program from abrupt termination (DoS).

The testcase can be downloaded from here.

UPDATE 31/January/2016 Further test confirm the same issue can be reached via a different code path from GZipStream.

Stack trace.

c:\Dev\GZipTestCase\bin\Release>GZipTestCase.exe
Index was outside the bounds of the array.
   at System.IO.Compression.HuffmanTree.CreateTable()
   at System.IO.Compression.HuffmanTree..ctor(Byte[] codeLengths)
   at System.IO.Compression.Inflater.DecodeDynamicBlockHeader()
   at System.IO.Compression.Inflater.Decode()
   at System.IO.Compression.Inflater.Inflate(Byte[] bytes, Int32 offset, Int32 length)
   at System.IO.Compression.DeflateStream.Read(Byte[] array, Int32 offset, Int32 count)
   at System.IO.Compression.GZipStream.Read(Byte[] array, Int32 offset, Int32 count)
   at System.IO.Stream.InternalCopyTo(Stream destination, Int32 bufferSize)
   at System.IO.Stream.CopyTo(Stream destination)
   at GZipTestCase.Program.Main(String[] args) in c:\Dev\GZipTestCase\Program.cs:line 26

The gzip testcase can be downloaded from here and the gzip file from here.

August 23, 2015

Fully Managed C# Can Corrupt Memory

In this post I'm writing about memory corruption in fully managed C# code. On the internet, many people advocate that managed C# should be immune against memory corruption. Even reputable organizations mention the same but provide no reference so people can't check the source of the information for themselves.

The fact is, Microsoft don't explicitly claim whether or not managed C# should introduce memory corruption in your app.

Input Sanitization

The materials made by Microsoft mentions the importance of sanitizing input values in managed code. This also means that things can go wrong if you just pass untrusted values to framework APIs.

AccessViolationException

.NET Framework has the AccessViolationException exception class for accessing protected memory. The description says:
An access violation occurs in unmanaged or unsafe code when it attempts to read or write to memory that has not been allocated, or to which it does not have access.
This is quite true, however, AccessViolationException can be originated from fully managed code, too.

In the past months, I was doing some C# development, and during the testing, I experienced several times that my app had terminated with the sign of memory corruption:
Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
It looks this error is commonly seen in the search result.

Memory Corruption

I did a bit of research on this topic, and wrote a simplified testcase for one of such corruption. I sent it to the people at MSRC. When they finished their investigation I received a reply from them:
We have successfully reproduced the issue and can confirm that the memory corruption is happening within the application itself and not the system.  As such, this does not appear to be a security issue and I will be closing this case.
The statement above is a bit vague but this is my understanding of what it means.

If there is a C# API that's designed to handle untrusted data (such as a decompression API) and there is a memory corruption bug in the way the API handles the data the bug may be a security issue, and so it has to be fixed by Microsoft.

However, if the bug is triggered due to an erroneous use of the C# API, such as a non-sanitized parameter is supplied, then the bug is considered to be fixed in the application.

Whether the bug is in the system or application you can end up corrupting the memory in the process so it can affect the security of your app. Despite this the bug may not be qualified for a fix in the .NET framework.

.NET framework manages the memory to eliminate the memory problems but this doesn't guarantee that it eliminates all the memory problems. Memory corruption in fully managed C# is possible.

Testcase

The testcase is a Visual Studio 2013 project that reproduces the memory corruption on Windows Phone 8.1 device and Windows Phone 8.1 Emulator.

The testcase creates an array of pixels with the use of IRandomAccessStream, BitmapEncoder and Stream. When checksumming the bytes in the array the checksum changes every time the vulnerability is triggered.

Checksumming is simple like this (the idea is borrowed from one of the public Flash infoleak made by Chris Evans).

            int checksum = 0;
            foreach (byte i in arr)
            {
                checksum += arr[i];
            }

Sometimes the testcase crashes with AccessViolationException but most of the time it's observed that the app continues to run after the memory corruption.

February 24, 2015

Simple Code Coverage Analyzer

coco.cpp is a simple pintool for code coverage analysis. It comes with the Pin Framework.

The reason I write a post about it is because it's really a simple but well-designed tool. The code coverage information for an executable section is stored in a vector of booleans. Each boolean represents a byte in the executable section. If the boolean is set true it means the corresponding byte has been executed. If the boolean is set false the corresponding byte is untouched.

If you want the tool to produce the code coverage hash just add yourself a call that calculates the hash of the vector of booleans. This works well when the pintool is executed on small programs.

However when working with more complex programs it's possible you notice that different code coverage hashes produced for seemingly similar executions. This is not an error. The code coverage can be different between the executions in the finest graduality sense. One example is when the application exits via different path between the executions.

To filter these differences above you can virtually split the vector into many regions and produce the hash of each region. Now you may see that most of the hashes are the same between the executions.

UPDATE 24/February/2015 No official PIN repository to see coco.cpp but Gunther was kind to share it via Twitter for those want to take a look at without the need to download the framework. Thx!

January 18, 2015

Even Calculator Has Bugs

Windows Calculator (calc.exe) has a functional bug that is associated with the clipboard and the integers below.

INT8_MIN  -128
INT16_MIN -32768
INT32_MIN -2147483648
INT64_MIN -9223372036854775808


In Programmer mode (View -> Programmer) when Dec and Byte are selected the minimum value the calculator can accept is -128. It's tricky to enter that value but you can do by the following four key-presses: (1) (+/-) (2) (8). I said it's tricky as you can't type (1) (2) (8) (+/-) because 128 is greater than the maximum value of INT8 that is 127. While it's a bit inconvenience it can be worked around.

To reproduce the bug select Dec and Byte and type -128. Copy it to the clipboard via Edit -> Copy or Ctrl+C. Then paste it via Edit->Paste or Ctrl+V. What happens is -12 is pasted rather than -128, discarding the last digit.
Last digit is discarded when -128 (INT8_MIN) is pasted.

The effect is the same if you copy -128 in Notepad then paste it in Calculator.

While a beeping sound may be heard when pasting, nothing else warns you about the error. Those who muted the speaker may not notice the error. This is especially possible when the value is long in digits (INT64_MIN) and can be easily overlooked.

It's plausible that programmers use INT_MIN values in their calculations as these are commonly used when developing software.

Windows 8.1 is affected by this issue. Other versions are not verified but likely affected.

While this issue is most likely the limitation of the design some may find it embarrassing as a calculator has nothing else to do than to work with numbers without failure.

The issue found when I was recently working on my own calculator for Windows Phone. I realized I had to deal with many corner cases and I checked how these corner cases affect Windows Calculator.

January 11, 2015

Arrangement Of The Variables In Data Structures

Let's assume that an extensive code review had identified the direct security issues and the developers eliminated all. Now the code is free from bugs. How nice to write that. :-)

That would not be the reason to stop code review though. There may be still things to be done to reduce the severity of potential future bugs.

Newly added code can introduce and/or expose issues. The idea is to make changes in the existing code that make the exploitation of a future bug more challenging. This is good to do without introducing new issues like performance degradation, etc.

Given a heap based buffer overflow, one of the common attack scenario is to overwrite a function pointer in any structure. If, however, the function pointer is at the end of the structure, the overflow may:
  • Not be long enough to overwrite the function pointer
  • Overwrite the variables preceding the function pointer and so destroy the internal state of the application. This could lead the application to bail out early before the function gets called.
So putting the function pointer as a last item in the structure can make exploitation more challenging.

A more generic suggestion in security point of view is like this. If possible, arrange the variables to have
  • the sensitive ones at the end of the structure
  • the ones that can bail the execution out early at the beginning of the structure

January 6, 2015

EMET Can Interfere With Pin

Sometimes no matter how simple your pintool is when it's run against the target it crashes. Recently I investigated such crash. One of my pintool was randomly crashed/hung/terminated as well as displayed memory errors.

My target was added to EMET (Enhanced Mitigation Experience Toolkit). Also, EMET was running at the time of experiment. I was thinking... what if I stop EMET? I disabled all the mitigations in EMET and stopped its service. Re-launched my pintool and checked if EMET.dll and EMET64.dll are not loaded in the target's process. This time my pintool was running without any obstacle.

Third party programs can make a pintool to crash. Of course, it's not a surprise but it's something to be aware of when working with a pintool.

If your pintool is still crashing you may want to take look at one of my earlier post.

December 22, 2014

Use of refactor can make Visual C# 2013 to crash

For the change I'm working on a C# project not much to do with security. I was happy to re-discover and use the refactor feature in Visual C# 2013 until the point it crashed. Here are the simplified steps to reproduce the crash.
  • Create an empty Visual C# project, or open any C# project.
  • Add a class like below.
  • Do right-click on hello in hello = true, select Refactor, then select Remove Parameters...

  • Visual C# crashes.
Note, I was able to trigger the crash by an alternate way by simply using the hotkey Ctrl+R, Ctrl+V when the cursor is within hello in hello = true.

The bug reproduces with the up-to-date version of Visual C# 2013 Community Edition.

December 19, 2014

Variable-length permutation with repetition using backtracking

Recently, I needed the implementation of the search algorithm that qualifies for the followings.
  1. It must be able to produce the variable-length permutation with repetition of the given set.
  2. It must be based on backtracking.
  3. It must be a state machine.
  4. It must have basic programming elements only.
  5. It must be clean without irrelevant part.
Even I needed the final implementation in ActionScript, I was searching for the algorithm in C that I would convert later. Thought my odds would be better if I did so. Unluckily I didn't find anything useful, and so I wrote the code myself. Here is the sample implementation in C.

// Features:
// - Variable-length permutation with repetition
// - Backtracking
// - State machine

#include <stdio.h>

#define ITEMS            3                       // number of items
#define LASTITEM_IDX     (ITEMS-1)               // index of the last item
#define INIT             -1                      // initial value
#define EMPTY            0                       // empty

static char set[ITEMS]  = {1,2,3};               // set

// State
static char idx[ITEMS]  = {INIT,INIT,INIT};      // indices into set
static char out[ITEMS]  = {EMPTY,EMPTY,EMPTY};   // output

static int index = INIT;
static bool dir = 1;                             // direction: deep(1) / wide(0)
// State - END

bool backtrack();

// item to try
void add_item() {
    idx[index]++;
    out[index] = set[idx[index]];
}

bool step_wide() {
    if (index == INIT) {
        // reached initial level
        // no more items to take
        return false;
    }
    // way to go wider?
    if (idx[index] < LASTITEM_IDX) {
        add_item();          // take item
        return true;
    }
    else {
        return backtrack();  // backtrack a level higher
    }
}

bool backtrack() {
    // backtrack a level higher
    idx[index] = INIT;
    out[index] = EMPTY;
    index--;

    return step_wide();
}

bool step_deep() {
    // way to go deeper?
    if (index < LASTITEM_IDX) {
        index++;             // step a level deeper
        add_item();          // take item
        return true;
    }
    else {
        return step_wide();  // step wide
    }
}

// returns true if step is successful
// returns false if no more items to take
bool step() {
    if (dir) {
        return step_deep();
    } else {
        return step_wide();
    }
}

int main(int argc, char* argv[])
{
    while (step()) {
        for (int i=0; i<ITEMS && out[i]!=EMPTY; i++) {
            printf("%02x ", out[i]);
        }
        printf("\n");
    }

    return 0;
}

The source can be downloaded from here.

The output of the run looks like this.

01
01 01
01 01 01
01 01 02
01 01 03
01 02
01 02 01
01 02 02
01 02 03
01 03
01 03 01
01 03 02
01 03 03
02
02 01
02 01 01
02 01 02
02 01 03
02 02
02 02 01
02 02 02
02 02 03
02 03
02 03 01
02 03 02
02 03 03
03
03 01
03 01 01
03 01 02
03 01 03
03 02
03 02 01
03 02 02
03 02 03
03 03
03 03 01
03 03 02
03 03 03

August 9, 2014

Instrumenting Flash Player to Inspect JITted Pages for Integer Errors

In this blog post I'm writing about the method I experiment with to discover potential areas, that may or may not be prone to integer errors, in Flash Player.

I have 26k flash files that are used as a corpus to generate the test samples for Flash Player. The test samples have an element of 0x41424241 in the integer pool. I have a total of 344k files generated to test Flash Player with.

During the test, I use a pintool to instrument the JITted code. The pintool is based on this implementation. Since the elements in the integer pool being dereferenced by action script it makes sense to restrict the instrumentation to JITted code.

I use instruction-level instrumentation that allows to check the register values at every single instruction being executed. If any of the general registers have the value of 0x41424241 and the instruction is referencing to that register, the instruction information along with general registers are logged.

The pintool pre-allocates the address of 0x41424241 so Flash Player won't use that memory address and so reducing the irrelevant lines in the log.

There is no need to wait for the test to finish to get partial results. A log file is generated for each test file rendered by Flash Player.

The size of log files are vary. Some are close to size 0 specially if the value above makes the code to fail early. There are many log files with size 4k. When the execution keeps going long the size is about 16k. If the value has to do something with a loop the log can reach 100k but that's rare.

Logs can be grouped and many can be thrown away as they don't contain instructions associated with vulnerabilities.

What I look for is like signed shift, addition, subtraction, or multiplication instructions. If the value is used in displacement with lea instruction that counts suspicious too.

Once an address in the log is chosen for closer inspection, I reproduce the log on isolation with an option to dump the JITted pages so I can manually analyze the surrounding area of that address in disassembler. Knowing the state information it's also possible to debug the code.

If results positive, certain level of automation can be added.

July 30, 2014

Practical Suggestions for Writing a Pintool

This is my list of practical suggestions to people developing a pintool. Since I dealt with these previously I thought to jot them down to help others. By applying this you should be somewhat closer to avoid your pintool from unexpected termination.

Start from scratch. So you use a sample pintool to develop your own. Rather than to modify the sample, start with an empty project and gradually build it up by taking elements from the sample.

Simplicity. Keep the code-base small and easy to understand.

Testing. As a part of development, aim to test if all blocks have been exercised. Refrain from adding unreachable blocks.

Errors. Check for errors as early as possible, specially when returning from a Pin API.

Safe memory dereference. Whenever you have to dereference the target's memory use PIN_SafeCopy. If you want to read an integer you should use this function, too, rather than the dereference "*" operator.

Thread safety. Be aware the target may be running with multiple threads. Possibly, you want your pintool to be thread safe.

Multi-threading. Sometimes you want your variables to be stored in the thread context to have the ability to distinguish the analysis between threads. In that case looking at the sample inscount_tls.cpp is a good start.

Probe mode. Use of probe mode is always preferred as it gives better performance. However, only limited Pin facilities available in probe mode.

Limit instrumentation. Consider restricting the instrumentation to routines or libraries and even can avoid the instrumentation of shared libraries to get better performance.

Standard library. It's good idea to use C++ standard library in a pintool as it provides the most frequently needed data structures.

Visual Studio. Visual C++ project file is available with Pin framework in MyPinTool folder. Alternatively, you can create one for yourself after looking at an earlier post.

Trace vs Ins. Instruction instrumentation is practically the same as trace instrumentation. You can do instruction analysis from the trace by iterating through the instructions.

Output. Having output routines in Fini makes the application to run faster than having them in analysis functions. However if the application terminates unexpectedly and so Fini is not called there will be no results shown. By having output routines in analysis functions makes the application to run slower but if the application terminates unexpectedly partial results may be shown.

July 26, 2014

Inspection of SAR Instructions

SAR stands for Shift Arithmetic Right and the instruction performs arithmetic shift. The instruction preserves the sign of the value to be shifted and so the vacant bits are filled according to the sign-bit.

Compilers generate SAR instruction when right shift operator ">>" is used on a signed integer.

The use of SAR instruction can potentially lead to create a signedness bug if it's assumed the shift is unsigned.

Given the following simplified example.

char retItem(char* arr, int value)
{
    return arr[value>>24];
}

If value is positive the code is working as expected. However if value is negative the program can read out of the bounds of arr.

Other example would be to compare the signed value after the shift to an unsigned value leading to implicit conversion that may lead to trigger bug.

In my experiment, in several cases, it is seen that memory is being dereferenced involving SAR instruction. These places may be worthy to look for bugs, specially if the value to be shifted is a user input or is a controlled one.

If an unsigned jump is followed by a signed shift that could be a potential to look for bugs as well.

Regular expressions or scripts can be used to search for patterns of occurrences of SAR instructions. When it's not feasible to review all occurrences of SAR, a pintool may be used to highlight what SAR instructions have been executed, and only focus on those executed.

July 22, 2014

Examining Native Code by Looking for Patterns

Earlier this year a post was published of examining data format without using the program that reads the format. That post discusses patterns to look for, in order to identify certain constructs. This post focuses on static methods of examining code that can be either the complete code section of the file, memory dump, or just fragment. It also describes selected ideas what patterns to look for when examining a given code.

The reason one may look for patterns in code is to locate certain functionalities or to get high-level understanding of what the code does. Others may look for certain construct that may be the key part of the program in security point of view.

It's true to say one can expect this to be a rapid method compared to other methods such as line-by-line instruction analysis.

But, it's always good to read documentation, if possible at all, to get an overview of the expectations.

There are methods that more effective if performed on small region. Therefore to narrow the scope of the search wherein to look for pattern is something good to do at the beginning of analysis albeit it's not always feasible to do with enough certainty. Anyway, one can always widen the search region if required at a later stage.

Compilers tend to produce executable files with particular layout. Some have the library code at the beginning of the code section, while others have it at the end of the code section.

If there is no information about the compiler or no information about the layout there are other ways to locate the library in the code.

You may look for library function calls that can be visible in disassembler. Library code may have distinct color in disassembler.

Library/runtime code often have many implementations of functions to use the advantage of latest hardware. An example is MSVC. And so SSE instructions/functions may indicate the presence of library/runtime code.

Library code can be spotted by looking for strings can be associated with particular libraries.

Library/runtime code can be spotted by looking for constant values that can be associated with particular libraries such as cryptographic libraries that tend to have many constants.

To guess the compiler that was used to generate the code is possible by analyzing the library/runtime code.

In case the code is just a fragment of user code you may consider examining the instructions how they are encoded. Intel encodings are redundant and one instruction can have multiple encodings. This is something to make guess on what compiler was used.

If multiple encodings of an instruction is found in a binary the code that could be generated with a polymorphic encoder.

Also, code has other characteristics that may differ between compilers such as padding and stack allocation.

Imports and exports as well as strings can tell a lot. You may check where they are referenced in the code.

Debugging symbols can help awfully lot if the disassembler can handle that. Sometimes it's available sometimes it's not.

No matter what code you're looking at it most likely deals with input data. That case it may get the data from file, from network, via standard API calls. These are valuable areas to audit for security problems, and it's possible to follow how the data returned by these APIs. It may require to analyze caller functions as usually these APIs are wrapped around many calls before using the input.

Just like when reading the data the code may write data, or send data via standard API calls. These areas may be security-sensitive.

Programs have centralized, well-established functions. These functions, for example, read dword values, read data into structures and propagate any other internal storage. Discovery of these functions not considered hard, they are normally small, and have instructions of memory read and write. By looking where they referenced from we can find good attack surfaces.

Good to keep in mind that code sections can contain data besides code. But normally data is stored in data section. In the disassembler it's convenient to see how the data is referenced, and may decide if there is an attack surface nearby.

CRC and hash constants may indicate there is some data which is being CRC'd or hashed. You may figure out where is that data from and how can you perform security testing around.

When a library is using a parameter hardcoded it's often encoded as a part of the instruction rather than stored in data section of the executable. Example encoding looks like mov eax, <param> or mov al, <param>.

When a data format is parsed often a magic value is tested. Looking for instructions like cmp reg, <magic> or cmp dword ptr [addr], <magic> or similar instructions can help to locate attack surfaces.

Longer strings may be broken into immediate values and compared with multiple cmp instructions.

Looking for strcmp function calls is good idea to look for if you want to find code that test for data format as often strcmp functions are used for this purpose.

If the code is optimized for speed there are many ways to confirm. Normally the readability of code bad, for example when the code performs division or use the same memory address for multiple variables. If EBP register is used in arithmetic or other than to store stack base address that could indicate the code is optimized.

Perhaps there are circumstances when looking at the frequency of instructions, looking for undocumented instructions, or rare instruction, or instructions that not present can give us valuable clues that help the examination.

Intuitively going through the code and looking for undefined patterns can be good idea if the scientific ways have been exhausted.

July 16, 2014

251 Potential NULL Pointer Dereferences in Flash Player

251 potential NULL pointer dereference issues have been identified in Flash Player 14 by pattern matching approach. The file examined is NPSWF32_14_0_0_145.dll (17,029,808 bytes).

The issues are classified as CWE-690: Unchecked Return Value to NULL Pointer Dereference.

I don't copy&paste all the issues in this blog post but bringing up few examples.

First Example

0:012> uf 5438a1d0
NPSWF32_14_0_0_145!BrokerMainW+0xf6f6b:
5438a1d0 f6410810        test    byte ptr [ecx+8],10h
5438a1d4 8b4104          mov     eax,dword ptr [ecx+4]
5438a1d7 7411            je      NPSWF32_14_0_0_145!BrokerMainW+0xf6f85 (5438a1ea)

NPSWF32_14_0_0_145!BrokerMainW+0xf6f74:
5438a1d9 85c0            test    eax,eax
5438a1db 740b            je      NPSWF32_14_0_0_145!BrokerMainW+0xf6f83 (5438a1e8)

NPSWF32_14_0_0_145!BrokerMainW+0xf6f78:
5438a1dd 8b4c2404        mov     ecx,dword ptr [esp+4]
5438a1e1 8b448808        mov     eax,dword ptr [eax+ecx*4+8]
5438a1e5 c20400          ret     4

NPSWF32_14_0_0_145!BrokerMainW+0xf6f83:
5438a1e8 33c0            xor     eax,eax <--Set return value to NULL

NPSWF32_14_0_0_145!BrokerMainW+0xf6f85:
5438a1ea c20400          ret     4 <--Return with NULL
0:012> u 5438a47b L2
NPSWF32_14_0_0_145!BrokerMainW+0xf7216:
5438a47b e850fdffff      call    NPSWF32_14_0_0_145!BrokerMainW+0xf6f6b (5438a1d0)
5438a480 8a580c          mov     bl,byte ptr [eax+0Ch] <--Dereference NULL 

Second Example

0:012> uf 54362e60
NPSWF32_14_0_0_145!BrokerMainW+0xcfbfb:
54362e60 8b4128          mov     eax,dword ptr [ecx+28h]
54362e63 8b4c2404        mov     ecx,dword ptr [esp+4]
54362e67 3b4804          cmp     ecx,dword ptr [eax+4]
54362e6a 7205            jb      NPSWF32_14_0_0_145!BrokerMainW+0xcfc0c (54362e71)

NPSWF32_14_0_0_145!BrokerMainW+0xcfc07:
54362e6c 33c0            xor     eax,eax <--Set return value to NULL
54362e6e c20400          ret     4 <--Return with NULL

NPSWF32_14_0_0_145!BrokerMainW+0xcfc0c:
54362e71 56              push    esi
54362e72 8b748808        mov     esi,dword ptr [eax+ecx*4+8]
54362e76 56              push    esi
54362e77 e8e4b0faff      call    NPSWF32_14_0_0_145!BrokerMainW+0x7acfb (5430df60)
54362e7c 83c404          add     esp,4
54362e7f 85c0            test    eax,eax
54362e81 7407            je      NPSWF32_14_0_0_145!BrokerMainW+0xcfc25 (54362e8a)

NPSWF32_14_0_0_145!BrokerMainW+0xcfc1e:
54362e83 8b4010          mov     eax,dword ptr [eax+10h]
54362e86 5e              pop     esi
54362e87 c20400          ret     4

NPSWF32_14_0_0_145!BrokerMainW+0xcfc25:
54362e8a 8bc6            mov     eax,esi
54362e8c 83e0f8          and     eax,0FFFFFFF8h
54362e8f 5e              pop     esi
54362e90 c20400          ret     4
0:012> u NPSWF32_14_0_0_145+006b4eb2 L2
NPSWF32_14_0_0_145!BrokerMainW+0xd1c4d:
54364eb2 e8a9dfffff      call    NPSWF32_14_0_0_145!BrokerMainW+0xcfbfb (54362e60)
54364eb7 8b7004          mov     esi,dword ptr [eax+4] <--Dereference NULL

Third Example

0:012> uf 5429979a
NPSWF32_14_0_0_145!BrokerMainW+0x6535:
5429979a 0fb74108        movzx   eax,word ptr [ecx+8]
5429979e 48              dec     eax
5429979f 48              dec     eax
542997a0 740c            je      NPSWF32_14_0_0_145!BrokerMainW+0x6549 (542997ae)

NPSWF32_14_0_0_145!BrokerMainW+0x653d:
542997a2 83e815          sub     eax,15h
542997a5 7403            je      NPSWF32_14_0_0_145!BrokerMainW+0x6545 (542997aa)

NPSWF32_14_0_0_145!BrokerMainW+0x6542:
542997a7 33c0            xor     eax,eax <--Set return value to NULL
542997a9 c3              ret <--Return with NULL

NPSWF32_14_0_0_145!BrokerMainW+0x6545:
542997aa 8d4110          lea     eax,[ecx+10h]
542997ad c3              ret

NPSWF32_14_0_0_145!BrokerMainW+0x6549:
542997ae 8d410c          lea     eax,[ecx+0Ch]
542997b1 c3              ret
0:012> u NPSWF32_14_0_0_145+005f3423 L2
NPSWF32_14_0_0_145!BrokerMainW+0x101be:
542a3423 e87263ffff      call    NPSWF32_14_0_0_145!BrokerMainW+0x6535 (5429979a)
542a3428 8038fe          cmp     byte ptr [eax],0FEh <--Dereference NULL

You can find a list of 251 potential NULL pointer dereferences in Flash Player here.

July 14, 2014

Issues with Flash Player & Firefox in Non-default Configurations

Few months ago I encountered a bug when a fuzzed flash file is being rendered by Flash Player in Firefox. This bug can be reached only in the non-default configuration described below so very unlikely you are affected by this bug.

To trigger the bug the flash player module has to be loaded into Firefox's virtual address space. And this can be achieved if Flash Player protected mode is disabled and Firefox plugin container process is disabled too.

The bug involves to dereference arbitrary memory address via a CALL instruction in the vtable dispatcher. Here you can see the bug in the exception state.

0:048> g
Implementation limit exceeded: attempting to allocate too-large object
error: out of memory
(170fc.16998): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=00000001 ebx=00000000 ecx=0034f670 edx=00000000 esi=1600f2c8 edi=0000001c
eip=5996bd5f esp=0034f638 ebp=0034f668 iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
*** ERROR: Symbol file could not be found.  Defaulted to export symbols for C:\Windows\SysWOW64\Macromed\Flash\NPSWF32_14_0_0_145.dll - 
NPSWF32_14_0_0_145!unuse_netscape_plugin_Plugin+0x5c2:
5996bd5f 8b461c          mov     eax,dword ptr [esi+1Ch] ds:002b:1600f2e4=????????
0:000> u eip L10
NPSWF32_14_0_0_145!unuse_netscape_plugin_Plugin+0x5c2:
5996bd5f 8b461c          mov     eax,dword ptr [esi+1Ch] <--Read unmapped address
5996bd62 a801            test    al,1
5996bd64 7420            je      NPSWF32_14_0_0_145!unuse_netscape_plugin_Plugin+0x5e9 (5996bd86)
5996bd66 33d2            xor     edx,edx
5996bd68 39550c          cmp     dword ptr [ebp+0Ch],edx
5996bd6b 7519            jne     NPSWF32_14_0_0_145!unuse_netscape_plugin_Plugin+0x5e9 (5996bd86)
5996bd6d 8b4e04          mov     ecx,dword ptr [esi+4]
5996bd70 83e0fe          and     eax,0FFFFFFFEh
5996bd73 89461c          mov     dword ptr [esi+1Ch],eax
5996bd76 8b06            mov     eax,dword ptr [esi] <--Read unmapped address
5996bd78 51              push    ecx
5996bd79 8bce            mov     ecx,esi
5996bd7b 895604          mov     dword ptr [esi+4],edx
5996bd7e 895618          mov     dword ptr [esi+18h],edx
5996bd81 ff500c          call    dword ptr [eax+0Ch] <--Dereference arbitrary memory content
5996bd84 eb06            jmp     NPSWF32_14_0_0_145!unuse_netscape_plugin_Plugin+0x5ef (5996bd8c)

I had reported this bug to Adobe and they opened a case PSIRT-2707 on 14/April/2014 but so far Adobe didn't confirm whether or not it was able to reproduce the bug or the exception state reported.

Again, the bug doesn't affect the default configuration, and so very unlikely you're affected by this. However, users using Firefox with plugin-container disabled as well as Flash Player plugin with protected mode disabled are affected by this issue.

The original report is about Flash Player 13_0_0_182 and Firefox 28.0 but the testcase fails with Flash Player 14_0_0_145 and Firefox 30.0 (latest available till today).

These are the steps to reproduce the bug.
  • Edit mms.cfg to have ProtectedMode=0 to disable protected mode in Flash Player
  • Start cmd.exe and type "set MOZ_DISABLE_OOP_PLUGINS=1" to disable plugin-container in Firefox
These settings above required to get Flash Player plugin loaded in firefox.exe's address space.
  • Start Firefox from command prompt opened previously
  • Open fuzzed.swf in Firefox (drag n drop should work)
  • Attach firefox.exe process to Windbg when you notice that Firefox is hanging
  • Exception should occur in few second. If you see the out-of-memory error in the debugger log without exception you may restart the browser and try again.
The fuzzed flash file has the following changes compared to the template file. The value of the first item in the integer pool has been changed to a large value. TagLength of DoAbc tag and FileSize of the main header have been therefore updated to maintain the integrity of the flash file.

Drop me an email if you think you need the testcase.

May 13, 2014

Security Implications of IsBad*Ptr Calls in Binaries

IsBad*Ptr [1] functions are to test whether the memory range specified in the argument list is accessible. Despite the fact they have been banned, they are still being referenced in many binaries shipped with popular applications.

In this post I'm describing the inner working of IsBad*Ptr, the steps the attacker may follow to abuse them, and mention few examples of binaries that have a reference to these banned functions.

Inner Working

When IsBad*Ptr is executed it first registers an exception handler. Then, it attempts to access to the memory specified in the argument list.

For example, IsBadReadPtr has the following instruction to read memory. ECX is the memory address specified in the argument list.

mov al,byte ptr [ecx]

If the instruction raises an exception, the execution is transferred to the exception-handler code. And IsBad*Ptr returns TRUE, meaning, it is a "bad" pointer because the data pointed by is inaccessible.

If the instruction executes without an exception being raised IsBad*Ptr returns FALSE.

Steps of Attacking

The attack against IsBad*Ptr looks like this.
  1. The attacker attempts to supply an invalid pointer parameter to IsBad*Ptr that returns TRUE.
  2. The attacker refines step #1 in a way that the supplied invalid pointer becomes valid due to a forced allocation for the location pointed by the invalid pointer. The attacker refines step #1 in a way that the supplied invalid pointer will point to valid memory location allocated by heap spray. And so, IsBad*Ptr returns FALSE leading to enter in an inconsistent state; that may or may not be an exploitable state.
  3. If the attacker can perform step #2 with IsBadWritePtr, when the call returns, it's expected to reach code that writes the location pointed by the pointer -- and that has attacker controlled data. And so, he reaches a presumably exploitable condition.
Referencing IsBad*Ptr can be easily checked during binary analysis and it is worthy to do.

Examples

This code snippet below can be found in msvbvm60.dll in Windows folder.

.text:72A0FEE5         push    38h                     ; ucb
.text:72A0FEE7         push    edi                     ; attacker supplies pointer was invalid before
.text:72A0FEE8         call    ds:IsBadReadPtr         ; and now it's valid because he's filled memory up
.text:72A0FEEE         test    eax, eax
.text:72A0FEF0         jnz     loc_72A0FF80            ; fall through
.text:72A0FEF6         mov     eax, [edi+4]
.text:72A0FEF9         mov     eax, [eax+4]
.text:72A0FEFC         mov     esi, [eax+8]            ; ESI is attacker controlled
.text:72A0FEFF         and     [ebp+arg_0], 0
.text:72A0FF03         mov     ax, [edi+2]
.text:72A0FF07         test    esi, esi
.text:72A0FF09         jz      short loc_72A0FF42      ; fall through
.text:72A0FF0B         movzx   ebx, ax
.text:72A0FF0E         mov     eax, [esi]              ; EAX is attacker controlled
.text:72A0FF10         push    esi
.text:72A0FF11         call    dword ptr [eax+0Ch]     ; EIP is attacker controlled


This one below can be found in dxtrans.dll in Windows folder.

.text:35C6142C         push    4                       ; ucb
.text:35C6142E         push    esi                     ; attacker supplies pointer was invalid before
.text:35C6142F         call    ds:__imp__IsBadWritePtr@8 ; and now it's valid because he's filled memory up
.text:35C61435         test    eax, eax
.text:35C61437         jz      short loc_35C61440      ; jump is taken
.text:35C61439         mov     eax, 80004003h
.text:35C6143E         jmp     short loc_35C6144A
.text:35C61440 loc_35C61440:                           ; CODE XREF: CDXBaseSurface::GetAppData(ulong *)+14
.text:35C61440         mov     eax, [ebp+this]
.text:35C61443         mov     eax, [eax+24h]
.text:35C61446         mov     [esi], eax              ; ESI is attacker controlled

The next code snippet is taken from v2.0.50727\mscorwks.dll in Windows folder. IsBadReadPtr is used to test the pointer that is passed to MultiByteToWideChar.

.text:7A0D17FB         push    eax                     ; ucb
.text:7A0D17FC         push    ebx                     ; attacker supplies pointer was invalid before
.text:7A0D17FD         call    ds:__imp__IsBadReadPtr@8 ; and now it's valid because he's filled memory up
.text:7A0D1803         test    eax, eax
.text:7A0D1805         jz      short loc_7A0D17A7      ; jump is taken
[...]
.text:7A0D17A7         cmp     edi, esi
.text:7A0D17A9         jle     short loc_7A0D17BF
.text:7A0D17AB         push    esi                     ; cchWideChar
.text:7A0D17AC         push    esi                     ; lpWideCharStr
.text:7A0D17AD         push    edi                     ; cbMultiByte
.text:7A0D17AE         push    ebx                     ; lpMultiByteStr - attacker's data
.text:7A0D17AF         push    1                       ; dwFlags
.text:7A0D17B1         push    esi                     ; CodePage
.text:7A0D17B2         call    ?WszMultiByteToWideChar@@YGHIKPBDHPAGH@Z ; WszMultiByteToWideChar(uint,ulong,char const *,int,ushort *,int)

I was collecting files with IsBad*Ptr in them, and have found plenty others including but not exclusively MSCOMCTL.OCX, EXCEL.EXE, Lenovo's, Corel's, Nokia's, AVerMedia's products...

UPDATE 13/May/2014 To add IsBad*Ptr to the program doesn't automatically mean to create bugs. However if IsBad*Ptr is present we have reason to believe that the function is expecting a pointer that might be invalid in certain circumstances. In that case IsBad*Ptr may be used to attack the program. And that's why it's important to conduct the audit according to this.

UPDATE 16/May/2014 The term "valid pointer" had an ambiguous meaning in the part Steps of Attacking. This is now reworded. Reference.

[1] IsBad*Ptr functions are IsBadReadPtr, IsBadCodePtr, IsBadWritePtr, and IsBadStringPtr. All of them are exports of kernel32.dll.

April 28, 2014

Order of Memory Reads of Intel's String Instructions

Neither the Intel Manual nor Kip R. Irvine's assembly book discusses the behavior I'm describing about x86 string instructions in this post.

Given the following instruction that compares the byte at ESI to the byte at EDI.

cmps    byte ptr [esi],byte ptr es:[edi]

To perform comparison the instruction must read the bytes first. The question is whether byte at ESI or byte at EDI is read first?

Intel Manual says:
Compares the byte, word, doubleword, or quadword specified with the first source operand with the byte, word, doubleword, or quadword specified with the second source operand
Kip R. Irvine's book titled Assembly Language for Intel-Based Computers (5th edition) says:
The CMPSB, CMPSW, and CMPSD instructions each compare a memory operand pointed to by ESI to a memory operand pointed to by EDI
Both of the descriptions explain what the instructions do but none of them says how. So I needed to do some experiments in Windbg to find the answer to the question.

The first experiment was not a good one. Initially, I thought I'd put processor breakpoint (aka memory breakpoint) at ESI and another one at EDI. I also thought to execute CMPS and let the debugger to break-in on either of the processor breakpoints. And here it goes why it was a bad idea. The execution of CMPS has to complete for debugger break-in. And, by the time the CMPS completes it hits both of the breakpoints.

The other experiment I came up with is like this. Set both ESI and EDI to point two distinct memory addresses that are unmapped. The assumption is when CMPS is executed it raises an exception when trying to read memory. By looking at the exception record we can tell the address the instruction tries to read from. Given that, we can tell if that value was assigned to ESI, or to EDI, and so we can tell whether byte at ESI or byte at EDI is read first.

Here is how I did the experiment in Windbg.

I opened an executable test file in Windbg. I assembled CMPS to be placed in the memory at EIP.

0:000> a
011113be cmpsb
cmpsb

I changed ESI to point to invalid memory. And I did the same with EDI.

0:000> resi=51515151
0:000> redi=d1d1d1d1

Here is the disassembly of CMPS. Also, you know from the highlighted text that both ESI and EDI point to unmapped memory addresses.

0:000> r
eax=cccccccc ebx=7efde000 ecx=00000000 edx=00000001 esi=51515151 edi=d1d1d1d1
eip=011113be esp=0022fb70 ebp=0022fc3c iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
test!wmain+0x1e:
011113be a6              cmps    byte ptr [esi],byte ptr es:[edi] ds:002b:51515151=?? es:002b:d1d1d1d1=??

I executed the process leading to an expected access violation triggered by CMPS instruction.

0:000> g
(5468.1eec8): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=cccccccc ebx=7efde000 ecx=00000000 edx=00000001 esi=51515151 edi=d1d1d1d1
eip=011113be esp=0022fb70 ebp=0022fc3c iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
test!wmain+0x1e:
011113be a6              cmps    byte ptr [esi],byte ptr es:[edi] ds:002b:51515151=?? es:002b:d1d1d1d1=?? 

I got the details of the exception like below.

0:000> .exr -1
ExceptionAddress: 011113be (test!wmain+0x0000001e)
   ExceptionCode: c0000005 (Access violation)
  ExceptionFlags: 00000000
NumberParameters: 2
   Parameter[0]: 00000000
   Parameter[1]: d1d1d1d1
Attempt to read from address d1d1d1d1

As you can see the access violation was occurred due to an attempt to read from d1d1d1d1 that is the value of EDI. Therefore, to answer the question at the beginning of the article, the byte at EDI is read first.

To give more to the fun, you may try how emulators handle this - that is the order of memory reads of CMPS instructions.

If you like this you may click to read more debugging posts.

UPDATE 28/April/2014 Of course I don't encourage people to rely on undocumented behavior when developing software. Stephen Canon says Intel may optimize the microcode for operands from time to time, and so we don't have a good reason to believe this behavior to be stable.

UPDATE 29/April/2014 Here is a Windows C++ program to test this behavior on your architecture: cmps-probe.cpp

April 23, 2014

Inspection of Division & Multiplication

Division and multiplication calculations can lead to trigger bugs, and potentially pose as security risks. Here are few things that I believe to be helpful for those who do binary inspection.

Division

Production quality binaries are normally built with optimization enabled which makes the binary to run fast. One of the optimizations technique for the compiler is to emit a series of fast instructions instead of a single slow instruction.

DIV and IDIV instructions are known to be slow. As a part of optimization the compiler emits a series of fast instructions that are functionally equivalent to DIVs. The fast instructions are shift, multiplication, and addition instructions that take magic (constant) values depending on the divisor. Therefore the divisor has to be known at compile-time to apply optimization.

If the optimized binary has any DIVs, that means, the divisor was not known at compile time. Thus it's known at run-time, and so it could be a user-controlled value or a user input taken as it is.

Division can cause exception if the divisor is 0, or if the result is to large to store.

Division by Zero in CLR's Native Code

As an interesting experiment I looked at what happens when an integer is divided by zero in C#.

CLR generates native code with division instruction in it. When the instruction of division by zero is executed, an exception is raised that is handled by CLR's exception handler.

So the generated code with division in it doesn't have a test for the divisor. It's left for the exception handler to handle division by zero situations.

Multiplication

Like division, multiplication can be optimized, too, by using a sequence of fast instructions (sometimes one instruction). Whether or not it's worth optimizing depends on the value of multiplier (or multiplicand).

The multiplication you can see in binary might not be seen on source-code level. And some multiplication cannot be easily spotted in binary code due to optimization. And, multiplications can lead to trigger bugs.

Overflow in Multiplication

Multiplication can lead to integer overflow. Multiplication of two values are more likely to lead to integer overflow than addition of the two values. Multiplication of two word length integers can overflow on 32-bit but addition can't.

Few instances of the IMUL instruction can take immediate value, that is the multiplier. It's easily possible to calculate what multiplicand overflows the multiplication. The challenging part is to determine how the value could be assigned to the multiplicand to trigger overflow.

It's worth searching for MUL and IMUL instructions in the binary using the # (hash mark) Windbg command.

Overflow in Multiplication by Scale Factor

Scale factor is a constant value by which the register in the instruction is multiplied. The scale factor is either 1, 2, 4 or 8.

Use of scale factor could be the result of optimization of multiplication. The example below demonstrates to multiply a value by 4.

lea eax,[edi*4]

Other common use case involves to dereference an element of the array. In the below example the array consists of elements of size 8. On source-level there is no multiplication but on low-level there is.

mov [edi+edx*8],eax

If the value of the register by which the scale factor is multiplied is large enough an integer overflow can occur.

Look at the below instruction. Even the multiplication might not overflow the result can, due to base (esp) and displacement (8000).

mov [esp+ebx*4+8000],eax

Method of Inspection

Generally, it's not feasible to review all the occurrences of certain instructions but on critical areas it might be reasonable to do. Instruction tracing, and tracing like this can be a good start to narrow the area that can be inspected closer.

April 16, 2014

You May Not Need to Debug SSE Instructions

There are binaries that contain implementation of an algorithm in two ways. The first one is optimized to run on all architectures and so it consists of i386 instructions only. The second one is optimized to run fast and therefore it has SSE instructions. When the application runs it checks the architecture to decide which implementation of the algorithm to be executed.

It is common thing that binaries can contain various implementations of the same algorithm. One example is the Microsoft Visual C++ runtime.

You may not need to debug SSE instructions though. What you need to do is to tell your application that SSE support is not available - which is most likely a lie in 2014.

Recently, when I debugged a Windows application I noticed it executes SSE instructions. Here is how I got my application to believe that there is no SSE support available.

I knew about CPUID instruction. It can come back with plenty information about the processor. If CPUID is used with input EAX set to 1 feature information is returned in ECX and EDX.

We only need the SSE-related bits of the feature information. Here are they (source: Intel Developer Manual).

In ECX:
    Bit 0  SSE3 Extensions
    Bit 9  SSSE3 Extensions
    Bit 19 SSE4.1
    Bit 20 SSE4.2

In EDX:
    Bit 25 SSE Extensions
    Bit 26 SSE2 Extensions

The idea is when CPUID is executed with EAX set to 1 we need to clear SSE bits in ECX and EDX. To clear SSE bits we have to mask the registers like below.

ECX<-ECX&FFE7FDFE
EDX<-EDX&F9FFFFFF


I used the following Windbg command to search for CPUID instructions in the code section of the virtual image.

# cpuid <address> L?<size>

I saw CPUID at few places. I checked all of them to find the ones that have EAX set to 1 input. I found few fragments like these.

xor eax,eax
inc eax
cpuid

I put breakpoints just after each of the right CPUID instructions. When the breakpoint hit the SSE flags are cleared and the execution resumes.

bp <address> "reip; recx=ecx&0ffe7fdfe; redx=edx&0f9ffffff; gc"

And it worked as expected in my experiment. The application took the alternate, but slower, code path of i386 instructions.

A final note, this technique may be used to avoid debugging SSE instructions but it can also be useful to increase code coverage during security testing.

April 8, 2014

Examining Unknown Binary Formats

This post is about to discuss the methods for examining unknown binary formats that can be either a file, file fragment, or memory dump.

Before discussing the methods I'm describing few scenarios when examination of an unknown format is appropriate.

Imagine you deal with an application that handles a certain file type that you want to fuzz. You think to carry out some dumb fuzzing with a bit of improvement. Before doing so, you may be examining the format to create an approximate map of the layout. So you'll get an idea what parts of the files are worth fuzzing, and what fuzzing method is reasonable to apply for each part.

In other scenario you might have the binary file but don't have the program that parses it. You want to know as much as possible of the format of the binary file to understand it's layout.

If the application that reads the file format is available you can use debugger to watch how the data is parsed. This scenario is not discussed here.

If the application that writes the format is available you can try the following idea. You may produce output file using the application. This can be done by save, export, convert options available in the application. Next time when producing output you change something minor in the settings that may produce a similar output file. Comparing the two output files you may see what changed.

Entropy analysis is very useful to locate compressed, encrypted, and other way encoded data. Higher entropy can indicate encoding of some kind. Lower entropy is likely anything else including text, code, header, data structures. Redundancy analysis is analogue to entropy analysis; the lower the redundancy the most likely the data is encoded.

Encoded data could be anything, even multimedia data. The compressed streams can have headers and/or magic bytes identify the compression type.

Character distribution of the file can tell us a lot. Creating a byte frequency map is very straightforward by using modern programming languages. That can tell us what are the most and less frequent bytes. We can easily know what are the bytes that are not present at all.

Strings can be discovered even with popular tools like a hex-editor. Most common encodings are ASCII and Unicode. If there is no terminating zero the length of the string is likely stored somewhere in the binary. It's often the preceding byte(s) of the first letter of the string.

Consecutive patterns, byte sequences are seen to be used for padding, for alignment, or to fill slack space.

Random-looking printable characters can indicate some kind of encoding of any data in plain text.

Scattered bytes, scattered zeros, scattered 0FFh bytes can indicate sequence of encoded integers. Integers can be offsets and lengths. Scattered zeros might indicate text in Unicode format.

It could be useful to analyze the density of zeros, printable characters, or of other patterns. This could be applied on the whole file or on a particular region of the file.

Consecutive values, integers might indicate an array of pointers. It might be useful to know if the values increasing, decreasing, or random values.

Also, good to know in what endianness the integers stored.

x86 code can be be detected by running disassembler on the binary. If you see a sequence of meaningful instructions that might be code-area.

There is a simpler way to look for x86 code though. You write a small program in some high level language that searchers for E8 (CALL) / E9 (JMP) patterns and calculates the absolute offset where the instruction jumps. If there is an absolute offset referenced from different places that might be an entry point of a real function. The more functions are identified the better the chance you have found code.

If you know what native code to look for you can search for a sequence of common instructions, like bytes at function entry point.

Meaningful text fragment in high-entropy area might indicate run-length encoding which is also known as RLE compression.

There is data format that looks like this. It consists of a sequence of structures, or chunks. The size of each structure is encoded sometimes as a first value in the structure. It's commonly seen that a sequence of compressed data is stored like that.

If it's known the binary is associated with certain time stamp or version number those constants might worth searching for.

Some methods described here can be combined with xor-search, and with other simple decoding techniques to discover the structure of the file.

April 5, 2014

Thoughts About Finding Race Condition Bugs

Race condition bugs can exist in multi-threaded applications. Improper synchronization can be the root cause of race condition bugs.

Executing stress testing is a good start to find bugs. It might not be an ideal black-box testing method though as it is mostly for developers to test their proprietary software. Injecting delays at various points into the target could help finding bugs but we need to know the right locations to inject the delays. Cuzz is a Microsoft tool for finding concurrency bugs by injecting random delays - it looks promising.

Using DBI (Dynamic Binary Instrumentation) it's possible to tell if an EIP is executed, and if so by what thread(s). Therefore it's possible to tell what code is executed by what thread(s).

Using DBI it is also possible to tell where (value of EIP) the thread context switch happens.

By having the above information we can make educated guesses where to inject the delays.

If a bug is found it might not be reachable from outside. That's always a possibility. However it's good to see if you can provide input that makes the application to run longer near the location of the intended delay. There might be a ReadFile that can take longer to complete if the file is large enough. Or there might be a loop where the iteration count can be controlled by user...
  This blog is written and maintained by Attila Suszter. Read in Feed Reader.