Harvesting Memory Dumps (3)

September 6th, 2009

Most escalation engineers associate their daily work with CSI: collect evidence, piece clues together, and find out the culprit(s). Unfortunately most books put emphasis on the piecing and analysis part, but very few gives clue for evidence collection, which is the reason we spent quite a few articles here to talk about dump collection. It’s not as intuitive as you may think 🙂

When we cannot install any program on the target computer (most likely resulted from corporate IT policy), what shall we do so that we could collect dumps? The trick is that you could install Debugging Tools for Windows on another machine, and just burn the folders into a CD-ROM. You can then run the tools from CD-ROM without installation. This is so-called “xcopy” installation: not really mean you need to copy it over, you could just burn a CD-ROM or plug a USB key to run. BTW, it’s always a good idea to put both 32-bit and 64-bit versions, along with SysInternals suite.

For user-mode dump collection, we’ve recommended using ADPlus as discussed in the first article. However, sometimes the target machine may not be able to run anything based on CScript due to problems like COM registry corruption. In this case we can utilize other tools like userdump or just do steps performed by ADPlus manually. Personally I like the manual steps more since I’ll have much better control over the dump creation.

The flow of ADPlus is

  • validate environment, e.g. determine OS version, collect date/time
  • collect critical info, e.g. process id, breakpoints set by user
  • generate CDB script
  • run the CDB script

The base structure of an ADPlus-generated CDB script is

.logopen <folder>\<log file>
<set aliases>
* set exceptions, there will be multiple sx* generated
sx{e|d|i|n} -c @”<1st chance cmds>-c2 @”<2nd chance cmds>

and the 1st-chance and 2nd-chance exception commands are typically

<bunch of .echo text>
~#kvn250 * dump faulting thread stack, frame depth 250
* create dump: /mdi for 1st chance, /ma for 2nd chance
.dump -u {/mdi|/ma} -c <comments>
{!locks; !runaway} * some additional commands for specific conditions
{!elog_str} * logging for 2nd-chance dumps
gn * go with exception not handled

So, we could definitely attach cdb/ntsd/windbg to the process we intend to debug, and manually do the commands or use pre-cooked script in case we could not run ADPlus on the target machine. (One quick cheat is to use ADPlus on another machine against the same process, copy the script under CDBScripts, edit the script and run the script with either cdb or ntsd.)

Debugging, Technical, Windows | Comments Jump to the top of this page

Comments are closed.

Chef Peon's Melange



Social Links