February 11, 2013

Out of Memory Issues in Internet Explorer 9

Recently, I was examining how Internet Explorer 9 performs under low-memory conditions.

The test involves to consume large amount of memory in the virtual address space of the renderer process. In this circumstance the execution flow can proceed on error condition of the memory allocation call, and so we can test how the application behaves when the memory allocation fails.

I used the following Windbg script as a template to exhaust the memory of the renderer process. I run the tests with between 16M and few hundred megabytes of free memory in the virtual address space.
$$ Exhausts the memory in the virtual address space. The memory is considered
$$ to be exhausted when an allocation with 16M fails.
$$
$$ Example Usage: $$>< e:/exhaust.wds
$$
$$ Last Updated: 10/February/2013
r $t0 = 0x40000000;
.while (@$t0 >= 0x100000)
{
   r $t1 = 0;
   .echo
   .echo "Size     Address";
   .echo "-----------------";
   .while (@$t1 == 0) {
     r $t1 = 1;
     .catch {.foreach /pS 5 (Address {.dvalloc /r @$t0}) { .printf "%08x %08x\n", @$t0, Address;}; r $t1 = 0;}
   }
   r $t0 = @$t0 / 2;
}
I opened local HTML/SVG files that were legitimate rather than fuzzed, and visited random websites under low-memory conditions. During my experiment, I observed many access violations because of the failed memory allocations. Even though most of them are harmless ones (NULL pointer crashes) some ends up to read data from invalid memory addresses that are not NULL.

Out of memory problem could be a security risk, and this is an attack surface in Internet Explorer. A mitigation to handle this issue would be to implement the following wrapper around the function that allocates the memory. If the allocation fails the renderer process stops with int 3 exception, so we avoid enter to any potential vulnerable code path.

As a reference, here are some stack traces of NULL pointer crashes.
eax=16acc320 ebx=16acc320 ecx=7765e38c edx=00629d08 esi=00000000 edi=16acc324
eip=678b34c5 esp=04ac6854 ebp=04ac6858 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00210246
MSHTML!TSmartResource<CD2DDCTraits>::Acquire<ID2D1RenderTarget *,enum D2D1_DC_INITIALIZE_MODE,bool>+0x15:
678b34c5 8b06            mov     eax,dword ptr [esi]  ds:002b:00000000=????????
0:005> kb
ChildEBP RetAddr  Args to Child              
04ac6858 678b45bd 00000000 00000000 16dd9af0 MSHTML!TSmartResource<CD2DDCTraits>::Acquire<ID2D1RenderTarget *,enum D2D1_DC_INITIALIZE_MODE,bool>+0x15
04ac686c 678b4498 00000000 04ac689c 00000000 MSHTML!RefCounted<CD2DDCHolder,SingleThreadedRefCount>::Create<IDCHolder,ID2D1RenderTarget *>+0x5a
04ac68e8 678b3f9f 04acd308 04acd404 04ac69e0 MSHTML!CGDIRenderMode::OnBegin+0x37
04ac6914 678b416d 1125d810 04ac69e0 04acd404 MSHTML!CDXRenderTarget::GetDC+0x1e0
04ac6940 678b41ca 04ac6978 04ac69e0 16e55668 MSHTML!TSmartResource<CDispSurfaceDCMode>::Acquire<CDispSurface *,CRect const *>+0x6c
04ac6a68 67ac98b9 04acd268 04ac6c58 04ac6c18 MSHTML!COleLayout::Draw+0xd6f
04ac6a94 67ac9663 04ac6c58 04ac6c18 04acd308 MSHTML!CLayout::DrawClient+0xaa
04ac6d6c 67ac78e9 04acae4c 00000000 00000007 MSHTML!CDispLeafNode::DrawSelf+0x56c
04ac6e84 67ac82d0 169fe050 00000000 00000007 MSHTML!CDispNode::Draw+0x2c8
04ac6ea8 67ad9c5e 00000000 04ac6f74 04acae4c MSHTML!CDispContainer::DrawChildren+0xe4
04ac6f48 67ad9d15 1188efb8 04acae4c 04ac6f74 MSHTML!CDispContainer::DrawContentAdvanced+0x25c
04ac718c 67ac78e9 04acae4c 00000000 00000007 MSHTML!CDispContainer::DrawSelf+0x49b
04ac72a4 67adcb19 1188efb8 00000000 00000007 MSHTML!CDispNode::Draw+0x2c8
04ac72c8 67a82102 0ec815ec 1188efb8 04ac73c8 MSHTML!CDispNode::DrawContainerChild+0xd4
04ac7340 67a80e8b 0ec815a0 00000001 00000003 MSHTML!HtmlLayout::LineBoxBuilder::LsInlineBlockDisplay+0x71
04ac7358 67a801ca 1682e1f0 04ac73a0 0ec815a0 MSHTML!Ptls5::LsUpdateBreakRecordText+0xe4
04ac73d0 67adc0f8 112b8118 04ac7454 00000000 MSHTML!Ptls5::LsDisplayLine+0x19b
04ac747c 67adbf96 11576038 16a20920 1188ef60 MSHTML!HtmlLayout::LineBox::Draw+0x127
04ac75fc 67adbcb6 16a20920 04ac7698 00000000 MSHTML!HtmlLayout::FlowBox::DrawFlowItems+0x417
04ac76d0 67adba8d 04ac77b4 04ac77c4 04acd308 MSHTML!HtmlLayout::FlowBox::DrawClientContainerContent+0x1f7


eax=00000000 ebx=0767dc08 ecx=00000000 edx=67b36c31 esi=00000001 edi=0767dc08
eip=67b50f2c esp=043fd05c ebp=043fd070 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
MSHTML!CElement::AddRefCDocument:
67b50f2c 8b8130010000    mov     eax,dword ptr [ecx+130h] ds:002b:00000130=????????
0:005> kb
ChildEBP RetAddr  Args to Child              
043fd058 67b7ce5d 00000001 0767dc08 0972de00 MSHTML!CElement::AddRefCDocument
043fd070 67b7faa9 0767dc08 0e90b258 0e5f5d00 MSHTML!CElement::PrivateExitTree+0xfa
043fd0dc 67b7d897 0e90b258 00000001 00000001 MSHTML!CMarkup::DestroySplayTree+0x1fd
043fd150 67b87d82 00000000 00000001 0e5f5c20 MSHTML!CMarkup::UnloadContents+0x4bb
043fd170 67b94f28 0e90b258 00000001 00000000 MSHTML!CMarkup::TearDownMarkupHelper+0x4c
043fd198 67b96c2f 00000001 00000000 00000000 MSHTML!CMarkup::TearDownMarkup+0x71
043fd1c8 67b97190 0e5f5c20 00000000 0e5f5c20 MSHTML!CDoc::UnloadContents+0x5ee
043fd1e4 67b2adb8 0af3d9f8 0af3d9f8 043fd21c MSHTML!CDoc::Passivate+0x158
043fd1f4 67b59914 0e5f5c20 6783edfd 00000000 MSHTML!CBase::PrivateRelease+0x33
043fd1fc 6783edfd 00000000 0af3d9f8 00000000 MSHTML!TSmartPointer<IDispBrush>::Release+0x14
043fd20c 67b2adb8 008f8d10 00000000 043fd230 MSHTML!COleSite::Passivate+0x9f
043fd21c 67b2ac22 0af3d9f8 008f8d10 008ff680 MSHTML!CBase::PrivateRelease+0x33
043fd230 67b2acfe 0af3d9f8 043fd2b0 67bc5bdb MSHTML!CElement::PrivateRelease+0x40
043fd23c 67bc5bdb 0af3d9f8 008fecc8 67adfe91 MSHTML!CXDomainRequest::Release+0x10
043fd248 67adfe91 0e5ff8d0 008bc5a8 02000000 MSHTML!CMultimediaLog::Reset+0x28
043fd2b0 67ae017b 0e5ff8d0 00000000 00000001 MSHTML!COmWindowProxy::SwitchMarkup+0xadf
043fd338 679f4971 0e5ff8d0 00000000 0e5ff8d0 MSHTML!CMarkup::SetInteractiveInternal+0x183
043fd36c 67a04c0f 00000001 00000000 0e5ff8d0 MSHTML!CMarkup::RequestReadystateInteractive+0x152
043fd398 679eeaa7 008dd2c0 0aebcf70 0556e892 MSHTML!CMarkup::BlockScriptExecutionHelper+0x184
043fd4a4 679b7cf8 0556e892 008dd2c0 0aebcf70 MSHTML!CHtmPost::Exec+0x4b1


eax=00000000 ebx=00000000 ecx=07f4b8d4 edx=04b695fc esi=08564494 edi=00000000
eip=6789db2b esp=04b695ac ebp=04b695dc iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
MSHTML!CDXImageSoftware::CreateSharedBitmap+0xb4:
6789db2b 8b08            mov     ecx,dword ptr [eax]  ds:002b:00000000=????????
0:004> kb
ChildEBP RetAddr  Args to Child              
04b695dc 6789d969 00000000 081efac4 04b695fc MSHTML!CDXImageSoftware::CreateSharedBitmap+0xb4
04b69610 67ace39f 745b8fc0 081efac4 00000000 MSHTML!CDXImageSoftware::PrepareToRenderImpl+0x82
04b69644 67e7c992 08564480 745b8fc0 00000000 MSHTML!CDXImage::PrepareToRender+0x7c
04b696e0 67c0c4cf 745b8fc0 04b69878 04b6973c MSHTML!CSATBlurRenderer::CreateBlurredTarget+0x33b
04b69774 677faef8 745b8fc0 04b69954 40000000 MSHTML!CDXRenderTarget::Gaussian+0x1bf
04b699b8 677fa4d6 04b69a9c 04b69bd0 00000000 MSHTML!CDispSurface::RenderShadows+0xcc0
04b69cac 67adb82f 6a5a4220 04b69d28 081fbca0 MSHTML!HtmlLayout::ContainerBox::PositionAndDrawBackground+0x1170
04b69d58 67adb6b2 081f1d18 04b6d8d8 081fbca0 MSHTML!HtmlLayout::ContainerBox::DrawBackgrounds+0x210
04b69d88 67aca06f 04b69f6c 04b69e7c 04b6d978 MSHTML!HtmlLayout::ContainerBox::DrawClientBackground+0x6f
04b69e94 67ac9f07 04b6b4bc 00000000 04b6b4bc MSHTML!CDispNode::DrawBackground+0x1c0
04b6a0cc 67ac78e9 04b6b4bc 00000000 00000007 MSHTML!CDispContainer::DrawSelf+0x2b0
04b6a1e4 67ac82d0 081fbca0 00000000 00000007 MSHTML!CDispNode::Draw+0x2c8
04b6a208 67ad9c5e 00000000 04b6a2d4 04b6b4bc MSHTML!CDispContainer::DrawChildren+0xe4
04b6a2a8 67ad9d15 07c69e18 04b6b4bc 04b6a2d4 MSHTML!CDispContainer::DrawContentAdvanced+0x25c
04b6a4ec 67ac78e9 04b6b4bc 00000000 00000007 MSHTML!CDispContainer::DrawSelf+0x49b
04b6a604 678d0958 07c69e18 00000000 00000007 MSHTML!CDispNode::Draw+0x2c8
04b6a64c 67ac78e9 04b6b4bc 00000000 00000007 MSHTML!CDispProxyNode::DrawSelf+0x11d
04b6a764 67ac82d0 0849b398 00000000 00000007 MSHTML!CDispNode::Draw+0x2c8
04b6a788 678d0896 04b6d978 04b6a854 04b6b4bc MSHTML!CDispContainer::DrawChildren+0xe4
04b6a828 67ad9d15 745b8da8 04b6b4bc 04b6a854 MSHTML!CDispContainer::DrawContentAdvanced+0x28d


eax=0684a9a0 ebx=00000000 ecx=727937c0 edx=0000025b esi=73de91e8 edi=00000001
eip=6682bbbb esp=0440d190 ebp=0440d1a0 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
jscript9!Js::JavascriptOperators::OP_GetInstanceScoped+0x2a:
6682bbbb 8b4304          mov     eax,dword ptr [ebx+4] ds:002b:00000004=????????
0:005> kb
ChildEBP RetAddr  Args to Child              
0440d1a0 09ed0071 00000003 0000025b 5ce84740 jscript9!Js::JavascriptOperators::OP_GetInstanceScoped+0x2a
WARNING: Frame IP not in any known module. Following frames may be wrong.
0440d1c8 668085fe 725d5980 00000002 02f5bcf0 0x9ed0071
0440d204 66808523 725d5980 6680cb68 00000002 jscript9!Js::JavascriptFunction::CallFunction+0xc4
0440d268 6680845a 0a244dc8 00000002 0440d360 jscript9!Js::JavascriptFunction::CallRootFunction+0xb6
0440d2a4 668083e6 00000000 0440d2d4 00000002 jscript9!ScriptSite::CallRootFunction+0x4f
0440d2cc 6687e0d7 725d5980 0440d2fc 00000000 jscript9!ScriptSite::Execute+0x63
0440d330 67a824a9 0a23606c 725d5980 00000002 jscript9!ScriptEngine::Execute+0x11a
0440d3b4 67a823d3 725d5980 094bfae8 63beed58 MSHTML!CListenerDispatch::InvokeVar+0x12a
0440d3d4 678ead1f 094bfae8 0440d438 0440d498 MSHTML!CListenerDispatch::Invoke+0x40
0440d5a4 67b6bbfc 094bfae8 728f5120 00000000 MSHTML!CEventMgr::Dispatch+0x537
0440d5cc 68073e2c 728f5120 7ec86aa0 ffffffff MSHTML!CEventMgr::DispatchEvent+0xc9
0440d5e0 68079c98 6a40df58 adc49da3 00000096 MSHTML!CSVGElement::Fire_SVGLoad+0x37
0440d5f8 680760bd 0440d620 67b77389 728f5120 MSHTML!CSVGSVGElement::Fire_SVGLoad+0x53
0440d600 67b77389 728f5120 00000000 00008003 MSHTML!CSVGElement::Fire_SVGLoad_Async_Handler+0x10
0440d620 67b77406 748dbf58 00000001 90fa7997 MSHTML!CAsyncEventQueue::DispatchAllEvents+0x7c
0440d670 74fb62fa 00700e80 00000aae 748dbf58 MSHTML!GlobalWndProc+0x2ed
0440d69c 74fb6d3a 67b145ee 00700e80 00008003 USER32!InternalCallWinProc+0x23
0440d714 74fb77c4 00000000 67b145ee 00700e80 USER32!UserCallWinProcCheckWow+0x109
0440d774 74fb788a 67b145ee 00000000 0440f8a8 USER32!DispatchMessageWorker+0x3bc
0440d784 6b8a205c 0440d7cc 001f6db0 001f6dcc USER32!DispatchMessageW+0xf


eax=04666084 ebx=0bd43ae8 ecx=04666060 edx=0466067f esi=00000000 edi=0bd43b48
eip=6908145b esp=04666078 ebp=046661ec iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
MSHTML!CSVGEmbeddedComponent::CreateDisplayTreeForEmbeddedContent+0x79:
6908145b 8b06            mov     eax,dword ptr [esi]  ds:002b:00000000=????????
0:005> kb
ChildEBP RetAddr  Args to Child              
046661ec 69081acc 0bd43ae8 0bd43ae8 0bd43aec MSHTML!CSVGEmbeddedComponent::CreateDisplayTreeForEmbeddedContent+0x79
04666208 68e5dda5 01d43ae8 0c2c0068 00000001 MSHTML!CSVGEmbeddedComponent::EnsureLayoutForEmbeddedComponent+0xf7
04666260 690b59c3 0b010600 04669bd8 046662b4 MSHTML!CImgHelper::DrawSVGImage+0x29f
046662d8 68ffe222 04669bd8 0466630c 12d406e8 MSHTML!CSVGImageBlock::Draw+0x1a8
0466631c 68ad6f4e 04666354 6901ae94 046663dc MSHTML!HtmlLayout::SvgPrimitiveBox::DrawClient+0x117
04666380 770f26a4 00000000 04669bd8 046677bc MSHTML!CDispDrawContext::GetRedrawRegionBounds+0x7b
04666498 770f256f 0edcc0a8 00000090 04669ce4 ntdll!RtlpReAllocateHeap+0x190
0466650c 68ad6f4e 00570000 00000000 0edcc0a8 ntdll!RtlReAllocateHeap+0x2c5
04666548 68ad8db1 046677bc 04669ce4 00000000 MSHTML!CDispDrawContext::GetRedrawRegionBounds+0x7b
046677bc 00000000 00000000 00000000 04667804 MSHTML!CDispSurface::CClipStack::PushClipRect+0x181


eax=00000024 ebx=00005867 ecx=00000009 edx=00000016 esi=0435d008 edi=00000000
eip=68ffaa2c esp=0435cf98 ebp=0435d18c iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
MSHTML!HtmlLayout::SvgTextBoxBuilder::CalculateGlyphWidths+0xaf:
68ffaa2c f3a5            rep movs dword ptr es:[edi],dword ptr [esi]
0:005> kb
ChildEBP RetAddr  Args to Child              
0435d18c 68ffa7cc 08fab420 0435d1d0 61b9c288 MSHTML!HtmlLayout::SvgTextBoxBuilder::CalculateGlyphWidths+0xaf
0435d1d8 68ff8684 00fab420 6adadd40 0435d478 MSHTML!HtmlLayout::SvgTextBoxBuilder::UpdateVisibleRectangle+0x7f
0435d280 68ff8150 08fab420 0435d478 08fab420 MSHTML!HtmlLayout::SvgTextBoxBuilder::BuildLine+0x4d2
0435d2a8 68a36a8d 0d01ff24 0435d478 0d01ff18 MSHTML!HtmlLayout::SvgTextBoxBuilder::MoveToNextPosition+0xb5
0435d2d4 68a33951 08fab420 0435d478 0d01ff18 MSHTML!HtmlLayout::LayoutBuilder::EnterBlock+0xca
0435d2f0 68a359e4 0435d420 0435d464 00000000 MSHTML!HtmlLayout::LayoutBuilder::Move+0x48
0435d3c0 68a319af 0ad93c90 00000000 0ad93c90 MSHTML!HtmlLayout::LayoutBuilderDriver::StartPartialLayout+0x2d1
0435d4a0 68a2ffcc 0c10ddb8 00000000 00000032 MSHTML!HtmlLayout::CIE9Page::LayoutPage+0x27c
0435d4cc 68a2c3a7 00000000 0c10ddb8 08fa0c90 MSHTML!HtmlLayout::CIE9DocumentLayout::FormatPage+0x65
0435d524 68a272fe 0c8dff24 00000000 08fa0c90 MSHTML!CCssDocumentLayout::FindOrFormatPage+0x272
0435d590 68a2deff 0ad93c90 0c8dff24 00000032 MSHTML!CCssDocumentLayout::GetPage+0x95b
0435d638 68a2dd12 0c8dff20 0435d654 0435d6c8 MSHTML!CMarkupPageLayout::CalcSize+0x28c
0435d6b0 68a2fa31 00100000 0435d6c8 0c8dff34 MSHTML!CMarkupPageLayout::CalcTopLayoutSize+0x101
0435d6d4 68c9b1d1 00100000 00002000 00000000 MSHTML!CMarkupPageLayout::DoLayout+0x56
0435d710 68a16b35 0c602e1c 00100000 00000000 MSHTML!CView::ExecuteLayoutTasks+0x3b
0435d77c 68a3856b 00000000 004d4f80 004d4fb4 MSHTML!CView::EnsureView+0x3bf
0435d7a4 68b49ef9 0c602e1c 00000000 00000000 MSHTML!CView::EnsureViewCallback+0xb8
0435d7e0 68b69768 3286b286 00000000 68b245ee MSHTML!GlobalWndOnMethodCall+0x115
0435d828 764e62fa 000909b6 0000005b 00000000 MSHTML!GlobalWndProc+0x302
0435d854 764e6d3a 68b245ee 000909b6 00008002 USER32!InternalCallWinProc+0x23


eax=00000001 ebx=00000000 ecx=00000000 edx=765d4758 esi=00000000 edi=00000000
eip=7675561c esp=043dcb38 ebp=043dcb40 iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010202
SHLWAPI!StrCmpNICW+0x13:
7675561c 0fb706          movzx   eax,word ptr [esi]       ds:002b:00000000=????
0:005> kb
ChildEBP RetAddr  Args to Child              
043dcb40 765d5704 00000000 765d4758 00000004 SHLWAPI!StrCmpNICW+0x13
043dcb60 765fcf6f 043dd000 72865284 00000000 urlmon!IsKnownProtocol+0x1e
043dcba4 68903047 007e81e8 00000000 00000000 urlmon!COInetSession::CreateBinding+0xf2
043dcbd8 68902f8f 72865284 00000000 00000000 MSHTML!CTridentFilterHost::CreateInetProtBinding+0x45
043dcc04 689c6406 00000000 00000000 043dd000 MSHTML!CTridentFilterHost::BindToInetProt+0x112
043dce98 689c18ba 72865218 00000000 00000004 MSHTML!CDwnBindData::Bind+0x4ba
043dcec0 689c16f6 00000000 68756ef4 00000000 MSHTML!NewDwnBindData+0x19d
043dcf1c 68a0a3bd 68756ee0 043dd000 6db14b88 MSHTML!CDwnLoad::Init+0x25c
043dcf40 68b62217 043dd000 6db14b88 00000001 MSHTML!CImgLoad::Init+0x43
043dcf68 68b6211d 07083df0 00000000 043dd000 MSHTML!CDwnInfo::SetLoad+0x11e
043dcf88 68b6c334 00000001 043dd000 00000000 MSHTML!CDwnCtx::SetLoad+0x86
043dcfb4 689f1547 00000001 043dd000 00000000 MSHTML!CImgCtx::SetLoad+0x4d
043dd058 689cc92d 00000001 0a4a138c 08df93b0 MSHTML!CDoc::NewDwnCtx2+0x337
043dd088 68ab962d 6fcd52a0 00000001 08df93b0 MSHTML!CDoc::NewDwnCtx+0x5b
043dd0d4 68aba4c3 07083df0 679f53e8 00000004 MSHTML!CImgHelper::FetchAndSetImgCtx+0xfb
043dd0f8 68aba3df 043dd1b8 08df93b0 043dd184 MSHTML!CImgHelper::EnterTree+0x132
043dd16c 6908fd6c 06f03f20 043dd1b8 043dd1b8 MSHTML!CImgHelper::Notify+0x2a4
043dd18c 689c2fa1 043dd1b8 00000001 6f48ea00 MSHTML!CSVGImageElement::Notify+0x2c
043dd1ec 689c2e5f 6f48ea00 6db102d8 689c7aa7 MSHTML!CHtmRootParseCtx::FlushNotifications+0x1b6
043dd1f8 689c7aa7 00000000 74702280 74702280 MSHTML!CHtmRootParseCtx::Commit+0xb
  This blog is written and maintained by Attila Suszter. Read in Feed Reader.