Advanced Crash Investigation Using Crash Dumps

This guide is intended to give you an idea on how to use crash dumps to find the root cause of an application crash. I don't really know how to fully utilize crash dumps, but I know enough to get some data out of them.

Following these steps will not lead you to a window that says "here is the problem". Rather, it will give you a bunch of information that you will need to piece together to find the cause of the problem.

Prerequisites

• Watch this video at the linked time: https://youtu.be/qouxznNC2XU?t=4224 (it will make this guide 100x easier to read)
• Download WinDbg on your computer https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools
  • You might need to download the whole ADK, but you only need to install WinDbg
• Enable application dumps on the machine https://msdn.microsoft.com/en-us/library/windows/desktop/bb787181(v=vs.85).aspx
• I enabled Mini dumps in this case, but you might need Full Dumps in some cases
• If you are not getting windows dumps (like in the case of a BSOD) the page file might not be sized properly.

Troubleshooting

We will be troubleshooting this error:

1. On the affected machine, provoke the application into a crash
   1. If you cannot provoke the issue go to the next step and see if there are any files
2. Go to: %LOCALAPPDATA%\CrashDumps. There should be some .dmp files
3. Copy all the files to the machine that has WinDbg installed
4. Open WinDbg and add the symbols server

   1. srv*c:\mss*http://msdl.microsoft.com/download/symbols

5. Open the crash dump
   
6. click on the console window all the way at the bottom and type in

   1. !analyze -v

   
   2. The Debugger will now analyze the Dump
7. The dump spits out a bunch of data that we need to decipher
   1. I immediately see a bunch of references to MtMUifTS.dll 
   
8. A Google search don't reveal any results so let's try something else
9. I run WizTree on the computer and I search for MtMUifTS.dll which returns two DLLs
   
10. I look at the Digital signature on the files and the signer is Black Ice Software
    
11. A google search for the company tells me that they handle document conversion and faxing
    
12. I go to the affected server and I see that there is a Printer called "Multi-Tech FaxFinder", also the port has the name Black Ice
    
    
13. I remove the printer, and the crashing stops

Things to keep in mind

As stated previously, this is a game of assumptions. A lot of times the name of the DLLs will point you in the direction you want to go. For instance, if you see the Audioses.dll pop up in the dump the issue is probably related to Audio and updating/reverting the audio drivers of the machine might fix the issue.