« MBS Workshop in der S… | Home | Reading crash reports… »

Analyzing crash dump files for Windows

On Windows if an application like FileMaker Server crashes, you'll find a dmp file for the crash. For example in C:\WINDOWS\MiniDump folder. Once you got the file, you wonder how to read it. It seems like the best was is to use WinDbg application from Microsoft.

First you download the Windows 10 SDK, mount the disk image and run the installer. From all the options in WinSDKSetup, you only need the entry for "Debugging Tools for Windows". All the other entries can be unchecked. So instead of installing several GB of data, only 160 MB are installed.

You can now run WinDbg (X64), the debugger version for 64-bit Windows. Use the menu command "Open Crash Dump..." In the File menu to open the dmp file. You are welcome with a screen like this:

Microsoft (R) Windows Debugger Version 10.0.18362.1 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [C:\Users\Christian\Desktop\Logs\fmserver_1571076915310.DMP]
User Mini Dump File: Only registers, stack and portions of memory are available

Symbol search path is: srv*
Executable search path is:
Windows 8.1 Version 9600 MP (8 procs) Free x64
Product: Server, suite: TerminalServer SingleUserTS
6.3.9600.18217 (winblue_ltsb.160124-0053)
Machine Name:
Debug session time: Mon Oct 14 20:15:15.000 2019 (UTC + 2:00)
System Uptime: not available
Process Uptime: 0 days 0:00:55.000
..................................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(68.1610): Access violation - code c0000005 (first/second chance not available)
For analysis of this file, run !analyze -v
*** WARNING: Unable to verify timestamp for ntdll.dll
ntdll+0x9079a:
00007ffe`5061079a c3 ret

We already see this is a crash report from a Windows Server with an Access violation (highlighted for you in bold). The WinDbg window has a console line, where you can enter commands. Or we just click on the analyze command in the text.

If you see "ntdll.wrong.symbols.dll" in the logs, you don't have symbols loaded.
We run the command ".symfix c:\myCache". Once we got internet connection and symbols fixed, the analzye command takes a bit longer and the names show up.

The full output follows here:

*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************

DEBUG_FLR_EXCEPTION_CODE(c0000374) and the ".exr -1" ExceptionCode(c0000005) don't match

KEY_VALUES_STRING: 1

Key : AV.Fault
Value: Read

Key : Timeline.Process.Start.DeltaSec
Value: 55


PROCESSES_ANALYSIS: 1

SERVICE_ANALYSIS: 1

STACKHASH_ANALYSIS: 1

TIMELINE_ANALYSIS: 1

Timeline: !analyze.Start
Name:
Time: 2019-10-15T06:50:07.79Z
Diff: 45292079 mSec

Timeline: Dump.Current
Name:
Time: 2019-10-14T18:15:15.0Z
Diff: 0 mSec

Timeline: Process.Start
Name:
Time: 2019-10-14T18:14:20.0Z
Diff: 55000 mSec


DUMP_CLASS: 2

DUMP_QUALIFIER: 400

CONTEXT: (.ecxr)
rax=00000000ffffffff rbx=0000000000010027 rcx=0000003d47db3b30
rdx=0000000000800029 rsi=0000003d47db4880 rdi=0000003d47db3b50
rip=00007ffe505b5bb8 rsp=0000003d4911d930 rbp=0000000000000000
r8=0000000000000000 r9=0000000000008000 r10=0000003d47db4890
r11=0000003d4911d67c r12=0000000000000001 r13=0000003d47d918b0
r14=0000000000000029 r15=0000003d47db3b00
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
ntdll!RtlFreeHeap+0x4e8:
00007ffe`505b5bb8 8b50fc mov edx,dword ptr [rax-4] ds:00000000`fffffffb=????????
Resetting default scope

FAULTING_IP:
ntdll!RtlFreeHeap+4e8
00007ffe`505b5bb8 8b50fc mov edx,dword ptr [rax-4]

EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 00007ffe505b5bb8 (ntdll!RtlFreeHeap+0x00000000000004e8)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: 00000000fffffffb
Attempt to read from address 00000000fffffffb

DEFAULT_BUCKET_ID: HEAP_CORRUPTION

PROCESS_NAME: fmserver.exe

FOLLOWUP_IP:
ntdll!RtlFreeHeap+4e8
00007ffe`505b5bb8 8b50fc mov edx,dword ptr [rax-4]

READ_ADDRESS: 00000000fffffffb
ERROR_CODE: (NTSTATUS) 0xc0000005 - Die Anweisung in 0x%p verwies auf Arbeitsspeicher bei 0x%p. Der Vorgang %s konnte im Arbeitsspeicher nicht durchgef hrt werden.
EXCEPTION_CODE: (NTSTATUS) 0xc0000374 - Ein Heap wurde besch digt.
EXCEPTION_CODE_STR: c0000005
EXCEPTION_PARAMETER1: 0000000000000000
EXCEPTION_PARAMETER2: 00000000fffffffb
WATSON_BKT_PROCSTAMP: 4e5dcbf3
WATSON_BKT_PROCVER: 3.0.0.10
PROCESS_VER_PRODUCT: Bonjour
WATSON_BKT_MODULE: ntdll.dll
WATSON_BKT_MODSTAMP: 5d6aa558
WATSON_BKT_MODOFFSET: 35bb8
WATSON_BKT_MODVER: 6.3.9600.19478
BUILD_VERSION_STRING: 6.3.9600.18217 (winblue_ltsb.160124-0053)
MODLIST_WITH_TSCHKSUM_HASH: 93af2701fca4ac53ef261c8a670b18fc371029b1
MODLIST_SHA1_HASH: 90c96f3438de4cda49c9e1ab686ac4455189e52a
DUMP_FLAGS: 8
DUMP_TYPE: 1
ANALYSIS_SESSION_HOST: DESKTOP-CK2F9O6
ANALYSIS_SESSION_TIME: 10-15-2019 08:50:07.0079
ANALYSIS_VERSION: 10.0.18362.1 amd64fre

THREAD_ATTRIBUTES:
ADDITIONAL_DEBUG_TEXT: Enable Pageheap/AutoVerifer ; Followup set based on attribute [Is_ChosenCrashFollowupThread] from Frame:[0] on thread:[PSEUDO_THREAD]
FAULTING_THREAD: 00001610
THREAD_SHA1_HASH_MOD_FUNC: e13263ae6e1446925b0795010b356214878a79e3
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 98972a2fd4f89208ff78e81e8a86bfe496f8ef26
BUGCHECK_STR: APPLICATION_FAULT_INVALID_POINTER_READ_HEAP_CORRUPTION
PRIMARY_PROBLEM_CLASS: APPLICATION_FAULT
PROBLEM_CLASSES:

ID: [0n262]
Type: [HEAP_CORRUPTION]
Class: Primary
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Add
Data: Omit
PID: [0x68]
TID: [0x1610]
Frame: [0] : ntdll!RtlFreeHeap

ID: [0n262]
Type: [HEAP_CORRUPTION]
Class: Primary
Scope: BUCKET_ID
Name: Add
Data: Omit
PID: [0x68]
TID: [0x1610]
Frame: [0] : ntdll!RtlFreeHeap

ID: [0n313]
Type: [@ACCESS_VIOLATION]
Class: Addendum
Scope: BUCKET_ID
Name: Omit
Data: Omit
PID: [Unspecified]
TID: [0x1610]
Frame: [0] : ntdll!RtlFreeHeap

ID: [0n285]
Type: [INVALID_POINTER_READ]
Class: Primary
Scope: BUCKET_ID
Name: Add
Data: Omit
PID: [Unspecified]
TID: [0x1610]
Frame: [0] : ntdll!RtlFreeHeap

LAST_CONTROL_TRANSFER: from 00007ffe323869d8 to 00007ffe505b5bb8

STACK_TEXT:
00000000`00000000 00000000`00000000 heap_corruption!fmserver.exe+0x0

THREAD_SHA1_HASH_MOD: ca4e26064d24ef7512d2e94de5a93c38dbe82fe9
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: heap_corruption!fmserver.exe
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: heap_corruption
IMAGE_NAME: heap_corruption
DEBUG_FLR_IMAGE_TIMESTAMP: 0
STACK_COMMAND: ** Pseudo Context ** ManagedPseudo ** Value: 2c5ec68dae0 ** ; kb
FAILURE_BUCKET_ID: HEAP_CORRUPTION_c0000005_heap_corruption!fmserver.exe
BUCKET_ID: APPLICATION_FAULT_INVALID_POINTER_READ_HEAP_CORRUPTION_heap_corruption!fmserver.exe
FAILURE_EXCEPTION_CODE: c0000005
FAILURE_IMAGE_NAME: heap_corruption
BUCKET_ID_IMAGE_STR: heap_corruption
FAILURE_MODULE_NAME: heap_corruption
BUCKET_ID_MODULE_STR: heap_corruption
FAILURE_FUNCTION_NAME: fmserver.exe
BUCKET_ID_FUNCTION_STR: fmserver.exe
BUCKET_ID_OFFSET: 0
BUCKET_ID_MODTIMEDATESTAMP: 0
BUCKET_ID_MODCHECKSUM: 0
BUCKET_ID_MODVER_STR: 0.0.0.0
BUCKET_ID_PREFIX_STR: APPLICATION_FAULT_INVALID_POINTER_READ_
FAILURE_PROBLEM_CLASS: APPLICATION_FAULT
FAILURE_SYMBOL_NAME: heap_corruption!fmserver.exe
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/fmserver.exe/3.0.0.10/4e5dcbf3/ntdll.dll/6.3.9600.19478/5d6aa558/c0000005/00035bb8.htm?Retriage=1
TARGET_TIME: 2019-10-14T18:15:15.000Z
OSBUILD: 9600
OSSERVICEPACK: 19358
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
SUITE_MASK: 272
PRODUCT_TYPE: 3
OSPLATFORM_TYPE: x64
OSNAME: Windows 8.1
OSEDITION: Windows 8.1 Server TerminalServer SingleUserTS
OS_LOCALE:
USER_LCID: 0
OSBUILD_TIMESTAMP: 2019-05-06 04:41:50
BUILDDATESTAMP_STR: 160124-0053
BUILDLAB_STR: winblue_ltsb
BUILDOSVER_STR: 6.3.9600.18217
ANALYSIS_SESSION_ELAPSED_TIME: 8d8
ANALYSIS_SOURCE: UM
FAILURE_ID_HASH_STRING: um:heap_corruption_c0000005_heap_corruption!fmserver.exe
FAILURE_ID_HASH: {d7a5d9a8-6e0b-efa6-ace0-ce9b8a5e9fb6}

What we read is about a heap corruption detected in runtime library free memory function (RtlFreeHeap).
Sadly for this crash report we can't help as a heap corruption can happen due to overwriting memory by some other unrelated function and a buffer overrun.
15 10 19 - 08:58