Memory dump file concepts and processes



[pic]

[pic]

Microsoft( Exchange Server 2003 and Microsoft Exchange 2000 Server

Memory Dump File Concepts and Processes

Microsoft Product Support Services white paper

Written by Steve Justice

Published June 2004

Revised June 2005

Abstract

This white paper describes memory dump files, the tools that are used to collect them, and the techniques that support engineers use to troubleshoot the processes that Exchange relies on.

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.

This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS DOCUMENT.

Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.

Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.

© 2004 Microsoft Corporation. All rights reserved.

Microsoft, Windows, and Windows NT are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

Contents

INTRODUCTION 1

Memory Dump file Concepts 2

Definition of a memory dump file 2

Information that a memory dump file captures 2

Differences between full memory dump files and mini memory dump files 3

Why it takes so long to create a memory dump file 3

Tools that are used to capture memory dump files 4

Memory Dump file Processes 6

Process stops responding 6

Performance problems 8

Crashes 9

Memory leaks 11

Reducing the time to upload memory dump file information 12

CONCLUSION 13

For More Information 14

Appendix A 15

APPENDIX B 17

APPENDIX C 19

APPENDIX D 20

Introduction

This white paper provides general guidelines for troubleshooting certain problematic scenarios. Do not assume that the steps that this white paper provides are the only methods that you have to use to troubleshoot any problem. Some circumstances may require more than one approach to determine the root cause of a problem.

Additionally, memory dump files are not always the best way for a support engineer to determine the root cause of a problem. Sometimes a memory dump file helps direct the troubleshooting, but does not point out the exact root cause of all problems. In those cases, additional troubleshooting steps must be used. Such additional steps may include using other tools or diagnosis techniques.

If a Product Support Services (PSS) support engineer is helping you solve a problem, the steps that you take may differ from the steps in this white paper. Troubleshooting the problems that are listed in this white paper is a complex process. Because of this, the type of information that is necessary to diagnose a problem is largely left to the support engineer’s discretion. The complexity of code-level troubleshooting requires many techniques to gather pertinent information.

The information in this white paper is not intended to supersede the judgment of a PSS support engineer who is diagnosing a problem.

The information in this white paper applies to:

• Exchange 2000 Server

• Microsoft Exchange 2000 Enterprise Server

• Exchange Server 2003

• Microsoft Exchange Server 2003 Enterprise Edition

Memory Dump file Concepts

The concepts in this white paper refer only to Exchange-related processes that run in User mode. If you have to diagnose a Kernel mode process, the steps and methods are different depending on the nature of the problem.

Definition of a memory dump file

A memory dump file is a copy of the contents of the virtual memory for a particular process. A debugger can create a memory dump file. The debugger writes the contents of virtual memory to a file on the hard disk. Later, another debugger can be used to open that memory dump file and treat the file as if it is a "frozen" instance of a live process that is being debugged. A support engineer can use this snapshot of virtual memory to examine things, such as the heap memory and the call stacks for every thread that was running for that process.

Information that a memory dump file captures

Memory dump files capture the contents of the virtual address space at one moment in time for a process that is running. That virtual address space contains almost everything that a process was using at that time to conduct its operations. Some of the most frequently used information in memory dump files is:

• Heap memory. This information includes objects or variables that are created by a process which is using statements in C++ such as "free," or "malloc" in C. Heap memory is generally used to determine what data was loaded at the time that the memory dump file was collected.

• Call stacks for all threads. These call stacks contain all the function calls that were being processed at the time that the memory dump file was collected, together with the variables that were on the stack. Call stacks are used to determine what was happening at the time that the memory dump file was collected.

• Thread environment blocks. These blocks contain information about a running thread. Support engineers can use this information to determine what the state of the thread was and what the thread ID is.

• Assembly code. The assembly code that each process is executing is loaded in virtual memory. A memory dump file can capture that assembly code, and a support engineer can use that code to help debug the problem so that the support engineer can determine how variables or processor registers reached their current state.

• Module information. Most processes use multiple files when they are executing. For example, many DLL files are frequently used to load supporting libraries for a process. A process might contain files that were created by Microsoft or by other companies. A memory dump file captures information about those loaded modules so that their version information and assembly can be examined later.

Differences between full memory dump files and mini memory dump files

A memory dump file can collect a variety of information. Typically, a support engineer must have all the contents of virtual memory to troubleshoot a problem. In other cases, you might want to capture less information to focus on a specific problem. The debugger is flexible. This flexibility lets you limit the information that a memory dump file captures by collecting either full memory dump files or mini memory dump files:

• Full memory dump files. These files contain the contents of virtual memory for a process. These files are the most useful when you are troubleshooting unknown issues. A support engineer can use these files to look anywhere in memory to locate any object, pull up the variable that was loaded on any call stack, and disassemble code to help diagnose the problem. The disadvantage of full memory dump files is that they are large. It also may take additional time to collect these files, and the process that is being recorded must be frozen while the dump file is created.

• Mini memory dump files. A mini dump file is more configurable than a full dump file and can range from only several megabytes (MB) up to the size of a full dump file. The size differs because of the amount of virtual memory that the debugger is writing to disk. Although you can gather mini memory dump files quickly and they are small, they also have a disadvantage. Mini dump files may contain much less information than full dump files. The information that a mini dump file gathers may be virtually useless to a support engineer if the area of memory that the support engineer has to troubleshoot was not captured. For example, if the heap memory is not written to the memory dump file, a support engineer cannot examine the contents of a message that was being processed at the time that the problem occurred. Useful information, such as the subject line and the recipient list, would be lost.

Why it takes so long to create a memory dump file

When a debugger has to copy the contents of virtual memory to disk, all threads for that process must be frozen at the same time. All threads can remain frozen until the memory dump file is finished, and then all threads are unfrozen at the same time. Because of this, a process such as the Store.exe process may cause clients to stop responding (hang) or report that the Exchange computer is unavailable. The time that is necessary to write a memory dump file is directly related to how much information must be written to disk. If a process is using 1.5 gigabytes (GB) of virtual memory, several minutes may pass before the memory dump file can be created. This behavior occurs because approximately 1.5 GB of data is written to the memory dump file. During that time, all threads can remain frozen to prevent unexpected behavior in the process.

Tools that are used to capture memory dump files

Many tools can capture virtual memory. Each of these tools uses some form of a debugger to create memory dump files.

The tools that PSS support engineers use most frequently are:

• Dr. Watson. This tool is a "just in time" debugger. Dr. Watson creates User.dmp files for a process whenever a second chance, unhandled exception is encountered. In most circumstances, a second chance, unhandled exception occurs because of an access violation, a stack overflow, or some condition that an application was not written to handle.

• ADPlus. This is a script that uses the CDB debugger to capture virtual memory. ADPlus can be configured to create memory dump files when hardware fails or a process stops responding. ADPlus has additional flexibility; it can capture multiple processes with one command-line instruction. ADPlus has two modes of operation: Hang mode and Crash mode:

o Hang mode noninvasively attaches to a process and creates a memory dump file that is based on command-line parameters. After the dump file is created, ADPlus detaches the debugger, and the process continues to run. ADPlus gives you a snapshot of what the process was doing at a point in time.

A typical ADPlus command line to capture a Hang mode memory dump file is:

adplus –hang –pn [process name]

o Crash mode invasively attaches to the process and waits for an exception to occur. By default, ADPlus creates a mini memory dump file for every first chance exception that is encountered. Whenever a second chance exception occurs, a full memory dump file is collected and the process is terminated. If ADPlus is in Crash mode, and the user decides to detach ADPlus by pressing CTRL+C, a full memory dump file is created, and then the process is terminated.

A typical ADPlus command line to capture a Crash mode memory dump file is:

adplus –crash –pn [process name]

• Userdump.exe. This is a command-line tool that can capture the virtual memory for one process when a process stops responding or an unhandled exception is encountered. Userdump.exe can create full memory dump files.

• Microsoft Windows® Debugger. If a support engineer is attached to a process with the Windows Debugger, the support engineer can create either full dump files or mini dump files while debugging. This method gives the support engineer the greatest amount of flexibility to create memory dump files.

Note This white paper describes how to use the Adplus utility to configure the debugger to collect memory dump files. You can also use the Windows Debugger, but the syntax that you must use is more complex and beyond the scope of this white paper. Therefore, the equivalent Windows Debugger commands are not included for the examples in this white paper.

Memory Dump file Processes

Different approaches must be taken to use memory dump files to diagnose problems. Support engineers try to minimize the impact that gathering memory dump files has on a customer, but still try to collect all the information that they must have to troubleshoot a problem. To minimize the impact of information gathering, tools have been designed to help customize the way that memory dump files are gathered. This section provides some general guidelines about how to approach certain situations to bring problems to a resolution as quickly as possible. The information in this section is not intended to supersede the judgment of a PSS support engineer who is diagnosing a problem.

Note All the examples in this section assume that the Debugging Tools for Windows are installed on the Exchange server. To install the Debugging Tools for Windows, visit the following Microsoft Web site:



It is recommended that you install the Debugging Tools for Windows to the c:\debuggers folder if you can. Installing to this folder makes executing command line instructions easier.

Process stops responding

When a process stops responding (hangs), one of the many threads that are running has entered a condition that prevents that thread from completing an operation. Other threads may also be affected by the problematic condition that one thread has entered. For example, if a thread is in an infinite loop, that thread consumes 100 percent of the processor's utilization until the process is terminated. If the only data that is collected is one memory dump file of an infinite loop, it is difficult to determine which thread was spinning. You can use System Monitor together with a memory dump file to help isolate the problem, but you may still want multiple dump files. You may want multiple dump files because the thread may have been in several loops at the time. With one dump file, the support engineer can determine which loops were being executed, but with multiple dump files, the support engineer is more likely to be able to isolate the infinite loop. This is because the support engineer can view multiple snapshots of the call stack for a thread to see how the stack changed over the course of several minutes.

When a process stops responding, it is important for the support engineer to be able to see what the threads are doing when the process stops responding. Memory dump files only take a snapshot of all the threads. If you capture multiple snapshots of the same threads, a support engineer can start to identify patterns of behavior to help isolate the problem. In this scenario, there is no good method to do this other than capturing memory dump files. (See Appendix B for an example of a debugging process that has stopped responding.)

There are several methods that can be used to collect data from a process that has stopped responding. The method that is used is determined by the behavior of the server that the process is running on. Factors such as current CPU activity, memory usage, error messages, event logs, and disk performance determine the approach that is taken in each case. However, if a process stops responding, a general approach can be to collect a full memory dump of the process, so that the state of the threads for that process can be inspected. Optionally, a support engineer can use mini memory dump files the same way that those files are used in performance-related problems, to catch thread state changes. The following example uses ADPlus to capture memory dump files when a process is not responding:

1. Open a Command Prompt window.

2. Change to the folder where the Debugging Tools for Windows are installed. (By default, this folder is C:\Program Files\Debugging Tools for Windows.)

3. Type the following command-line instruction:

adplus –hang –MiniOnSecond –pn [process name]

4. Adplus informs you that a memory dump file is being created. Click OK to continue.

5. A memory dump file is created in a subfolder of the location where the Debugging Tools for Windows were installed. The subfolder name is similar to "Hang_Mode__Date_[MM-DD-YYYY]__Time_[HH-MM-SS]PM."

The memory dump file has a name that is similar to "PID-2748__[process name]__full_[YYYY-MM-DD_HH-MM-SS]-416_0ABC.dmp."

6. Wait for approximately three minutes, and then repeat step 3. This creates another mini memory dump file in a new subfolder of the location where the Debugging Tools for Windows were installed. You will have two subfolders, and each subfolder will contain a mini memory dump file.

7. Wait for approximately three minutes longer, and then type the following command-line instruction:

adplus –hang –pn [process name]

This creates another subfolder, but this time a full memory dump file is created for the process.

8. Collect all three of the subfolders that were created, and then use the instructions in Appendix D to send the subfolders to PSS.

Note PSS may also request additional information, such as system monitor logs and event logs, to troubleshoot the problem.

Performance problems

Contention for a resource can cause multiple threads to back up behind each other. If a thread is waiting to write to a file on disk, the thread frequently locks a critical section to do that. For example, if a thread must write to a log file, that thread acquires a lock for the log file, and then writes to the log file. If there are other threads that have to write to the log file, those threads must wait until the file is unlocked, so that they can lock the file and write to the file. If for some reason the disk cannot handle those write requests in a timely manner, the disk becomes a bottleneck. At that point, multiple threads start to back up behind each other while they wait to gain access to the log file. Therefore, the process stops responding or becomes slow to respond.

In these situations, support engineers typically must have multiple dump files, so that they can examine the state of the threads at different points in time. Getting a snapshot of what is occurring over the course of several minutes helps reveal whether a process has stopped responding, or whether the threads are just processing slowly. The difference is significant because it determines the direction that the troubleshooting takes moving forward.

Collecting memory dump files for performance problems can take longer than usual if a lack of system resources causes a process to run more slowly than usual. Also, because collecting a memory dump file itself requires system resources, the server needs time to recover from the memory dump file creation process. Therefore, the performance of the server immediately after you create a memory dump file, especially a full memory dump file, should not be considered typical performance behavior. Typical performance behavior returns after the process catches up from creating the dump file.

The following example uses ADPlus to capture memory dump files when a process is experiencing a performance problem:

1. Open a Command Prompt window.

2. Change to the folder where the Debugging Tools for Windows are installed. (By default, this folder is C:\Program Files\Debugging Tools for Windows.)

3. Type the following command-line instruction:

adplus –hang –MiniOnSecond –pn [process name]

4. Adplus informs you that a memory dump file is being created. Click OK to continue.

5. A memory dump file is created in a subfolder of the location where the Debugging Tools for Windows were installed. The subfolder name is similar to "Hang_Mode__Date_[MM-DD-YYYY]__Time_[HH-MM-SS]PM."

The memory dump file has a name that is similar to "PID-2748__[process name]__full_[YYYY-MM-DD_HH-MM-SS]-416_0ABC.dmp."

6. Wait for approximately three minutes, and then repeat step 3. This creates another mini memory dump file in a new subfolder of the location where the Debugging Tools for Windows were installed. You will have two subfolders, and each of them will contain a mini memory dump file.

7. Wait for approximately three minutes longer, and then type the following command-line instruction:

adplus –hang –pn [process name]

8. This creates another subfolder, but this time a full memory dump file is created for the process.

9. Collect all three of the subfolders that were created, and then use the instructions in Appendix D to send the subfolders to PSS.

Note PSS may also request additional information, such as system monitor logs and event logs, to troubleshoot the problem.

Crashes

A crash occurs when a condition arises that the exception handlers for the process and the operating system cannot handle. Access violations, heap corruption, stack corruption, or a stack overflow may lead to a crash. By design, a crash creates one dump file at the time of the crash, and it is possible that the dump file can be used to isolate the root cause. Sometimes you may have to capture another memory dump file of the crash. For example, a support engineer may require more or different memory dump files from the time of the crash if the memory dump file was corrupted and is unreadable, or if other factors also lead to the crash.

Dr.Watson will capture a crash if Dr.Watson is configured to be the system's "just in time" debugger. To make sure that Dr.Watson is configured to catch process crashes, click Start, click Run, type the following command in the Open box, and then click OK:

drwtsn32

The Dr.Watson interface is displayed. Examine the Dr.Watson interface to make sure that the following settings are configured:

• Number of Instructions: 10

• Number of Errors to Save: 10

• Crash Dump Type: Full

• Dump All Thread Contexts is selected

• Append To Existing Log File is selected

• Create Crash Dump File is selected

To make sure that Dr.Watson is the default "just in time" debugger, click Start, click Run, type the following command in the Open box, and then click OK:

drwtsn32 –i

You receive a notification that Dr.Watson has been registered as the default application debugger. Click OK to clear that notification.

You can also configure the debugger to catch crashes by using ADplus or the Windows Debugger. Those options give you more control of what type of memory dump file is created and what the name of the memory dump file will be.

The advantage of using Dr.Watson to catch crashes is that the debugger does not have to be invasively attached to a process to capture a crash. The disadvantages are that the same file name is always used (User.dmp), and the information in the User.dmp file will be overwritten if another crash occurs. Also, Dr.Watson does not capture first-chance exceptions.

To overcome the limitations of Dr.Watson, you can use ADPlus, the Windows Debugger, or CDB to make collecting memory dump files from processes that have crashed easier. The following example uses ADplus to configure the debugger to catch a crash in a process:

1. Open a Command Prompt window.

2. Change to the folder where the Debugging Tools for Windows are installed. (By default, this folder is C:\Program Files\Debugging Tools for Windows.)

3. Type the following command-line instruction:

adplus –crash –pn [process name]

4. Adplus informs you that the debugger (CDB.exe) is attached and monitoring for crashes for the specified process.

5. A full memory dump file is created if any one of the following situations occur:

• The process encounters an exception that cannot be handled, and the process crashes

• The process is stopped by any means. This includes stopping a service or shutting a process down

• A user presses CTRL+C in the CDB Command Prompt window that ADPlus opened. If a user presses CTRL+C in that window, a memory dump file is created and the process is terminated

6. The memory dump file is created in a subfolder of the location where the Debugging Tools for Windows were installed. The subfolder name is similar to "Crash_Mode__Date_[MM-DD-YYYY]__Time_[HH-MM-SS]PM."

The memory dump file has a name that is similar to "PID-2748__[process name]__full_[YYYY-MM-DD_HH-MM-SS]-416_0ABC.dmp."

Memory leaks

A memory leak is created by code that creates objects in virtual memory but does not remove those objects when the code finishes using them. For example, if a process is opening messages in the store process to scan those messages for viruses, and that process continually opens messages for scanning but never closes them, virtual memory for the store process increases and is never released. A support engineer can use a full memory dump file of a process that is leaking memory to examine the contents of the heap and determine what types of objects are being leaked and which user logon owns them. In this example, one dump file may be sufficient to capture the leak, but if the leak is small or slow to develop, a support engineer may require multiple dump files to examine the contents of virtual memory. Other tools, such as LeakDiag, UMDH, VADump, and System Monitor, are typically employed to diagnose a leak.

Memory leaks can be captured using either an invasive attachment or a noninvasive attachment of the debugger. You can use a noninvasive attachment in this case because the thread state of a process is not as important as the contents of virtual memory when you capture memory leaks. The following example uses the process for collecting a memory dump file of a memory leak:

1. Open a Command Prompt window.

2. Change to the folder where the Debugging Tools for Windows are installed. (By default, this folder is C:\Program Files\Debugging Tools for Windows.)

3. Type the following command-line instruction:

adplus –hang –pn [process name]

4. Adplus informs you that a memory dump file is being created. Click OK to continue.

5. A memory dump file is created in a subfolder of the location where the Debugging Tools for Windows were installed. The subfolder name is similar to "Hang_Mode__Date_[MM-DD-YYYY]__Time_[HH-MM-SS]PM."

The memory dump file has a name that is similar to "PID-2748__[process name]__full_[YYYY-MM-DD_HH-MM-SS]-416_0ABC.dmp."

6. Collect the subfolder that was created, and then use the instructions in Appendix D to send the subfolder to PSS.

Note PSS may also request additional information, such as system monitor logs and event logs, to troubleshoot the problem.

Reducing the time to upload memory dump file information

Full memory dumps files can exceed 1 GB. Therefore, the time that is required to upload those files to PSS can be significant. To speed up the file transfer process, you can extract a subset of information from the full memory dump file, so that you can quickly send the information to PSS. (An example of the size difference would be a full memory dump file that is 1.1 GB; this file can generate a mini memory dump file that is 2.2 MB.) The PSS support engineer can then start analyzing the subset of information while the full memory dump file is being uploaded.

You can use the Windows Debugger to create a new mini memory dump file that is based on the full memory dump file.

To create the new file, follow these steps:

1. Start WinDbg. WinDbg starts the Windows Debugger.

2. On the File menu, click Open Crash Dump.

3. Open the full memory dump file that captured the process at the time of the failure.

4. Ignore any error messages regarding symbols that are displayed. To do this, click OK to close the messages.

5. At the command line in the Windows Debugger, type the following command:

.dump /mi c:\newdump.dmp

This creates a new mini memory dump file that is named C:\Newdump.dmp.

6. Wait until you see the debugger report "Dump successfully written."

7. On the File menu, click Exit. If you are prompted about whether you want to save workspace information, click No.

Make sure that you also upload the full memory dump file to PSS. PSS will probably also need the full memory dump file to complete a detailed investigation of the problem.

CONCLUSION

Memory dump files capture the contents of the virtual address space at one moment in time for a process that is running. That captured virtual address space contains almost everything that a process was using at that time to conduct its operations, such as stacks, heap memory, and thread information.

There are two general types of memory dump files: full memory dump files and mini memory dump files. Full memory dump files contain all the contents of virtual memory for a process. Mini memory dump files may contain much less information than full memory dump files, but mini memory dump files are faster to collect and smaller than full memory dumps.

Different approaches must be taken to use memory dump files to diagnose problems. If you are diagnosing a process that crashed, one full memory dump file may be sufficient to troubleshoot the problem. If a process stops responding (hangs), you typically have to gather multiple memory dump files several minutes apart to diagnose the problem.

In any case, support engineers strive to minimize the impact that gathering memory dump files has on a customer while still trying to collect all the information that they must have to troubleshoot a problem.

For More Information

For the latest information about Exchange, visit the following Microsoft Web sites:

•

•

•

For additional information about how to collect memory dump files and scenarios where memory dump files can be useful on Exchange Server 2003 computers and Exchange 2000 Server computers, click the following article numbers to view the articles in the Microsoft Knowledge Base:

241215 How to use the Userdump.exe tool to create a dump file

823150 How to gather data to troubleshoot Exchange Server 2003 virtual memory

325044 HOW TO: Troubleshoot virtual memory fragmentation in Exchange 2003 and Exchange 2000

286350 Use ADPlus to troubleshoot "hangs" and "crashes"

Appendix A

This appendix includes a table to use for reference when you start to gather data for certain conditions. Note that the information that this table contains is not intended to supersede the judgment of a PSS support engineer who is diagnosing a problem.

The main body of this white paper contains information about collecting mini memory dump files. This information is also included in the following table. Some of those points are restated in this appendix for clarity.

The table includes preferred options and secondary options for both a process that stops responding (hangs) and slow performance. The preferred options involve gathering several full memory dump files during the problematic condition. This option is preferred because support engineers can use all the contents of virtual memory for a running process to investigate the problem. This helps the support engineer examine the objects that were being used at the time that the problem occurred.

The secondary options involve gathering one full memory dump file, and then gathering at least two more mini memory dump files during the problematic condition. The disadvantage of the secondary option is that the mini memory dump files will not contain the objects that were being used during the problematic condition. The support engineer might be able to use the data in the full memory dump file to locate objects that are referenced in the mini memory dump file, but that may not be possible.

Another significant difference between the preferred option and the secondary option is the time that it takes to complete them. Full memory dump files take several minutes longer to create than mini memory dump files. Therefore, the time that it takes to complete the preferred option is longer than the time that it takes to complete the secondary option by at least several minutes. The time difference depends on the amount of virtual memory that a process is using at the time that the memory dump file is collected.

|Exchange Server 2003 and Exchange 2000 Server |

|memory dump file processes |

|Condition |Required data |Optional data |

|Crash |One full memory dump file at the time that the process terminated |x number of mini dump |

|(A process suddenly |You can also use the .dump /mi [filename] command in the Windows Debugger|files created |

|terminates) |to create a mini memory file from the full memory dump file. |leading up to the failure |

| |Application event logs | |

| |System event logs |(Only available in ADPlus |

| | |attached in Crash mode) |

|Hang |One of the following: | |

|(A process stops |Preferred option. Two mini memory dump files at least 2 minutes apart, | |

|responding but still |but no more than 15 minutes apart | |

|appears to be running) |Secondary Option. Two full memory dump files at least 2 minutes apart, | |

| |but no more than 15 minutes apart | |

| | | |

| |One full memory dump file | |

| |System Monitor logs | |

| |Application event logs | |

| |System event logs | |

|Slow Performance |One of the following: | |

|(A process is running |Preferred option. Two mini memory dump files at least 2 minutes apart, | |

|but users are |but no more than 15 minutes apart | |

|experiencing delays) |Secondary option. Two full memory dump files at least 2 minutes apart, | |

| |but no more than 15 minutes apart | |

| | | |

| |One full memory dump file | |

| |System Monitor logs (Specific counters will be determined at the time | |

| |that the problem is occurring) | |

| |Application event logs | |

| |System event logs | |

|Memory Leak |One full memory dump file during the high memory use condition |Possibly LeakDiag or UMDH |

|(Memory is being used |System Monitor logs |tracking |

|and never released) | | |

APPENDIX B

This appendix includes a mock memory dump file to illustrate what a support engineer may have to do to debug and troubleshoot a process that has stopped responding.

From a memory dump file, you can verify that multiple threads are backed up behind one thread. In the following example, thread 102 is blocking the others.

CritSec at 1bfe1544. Owned by thread 113.

Waiting Threads: 57 216 232 240 266 277 288 289 328 332

CritSec at 1bc2e5c4. Owned by thread 102.

Waiting Threads: 113

After you have this information, you must determine what thread 102 doing. To determine what thread 102 is doing, view that call stack.

ChildEBP RetAddr

NTDLL!ZwReadFile+0xb

KERNEL32!ReadFile+0x125

MSExchange!OpIFSRead+0x84

MSExchange!OpRead+0x413

MSExchange!IStreamRTF::Read+0x1b

inetmail!CStreamLockBytes::Read+0x43

inetmail!CInetStream::_HrGetNext+0x33

inetmail!CMessageTree::_HrBindOffset+0x71

inetmail!CMessageTree::Load+0x5e

MSExchange!CConverter::Initialize+0x229

MSExchange!CIMailConvert::HrOpenProperty+0x88

MSExchange!MSG::OpFurnishProp+0x539

MSExchange!MSG::OpGetBasicProp+0x16f

exlib!RPC_INTERFACE::DispatchTo+0x5e

exlib!OSF_SCALL::Dispatch+0xac

exlib!LOADABLE_TRANSPORT::ProcessIO+0x14a

exlib!ProcessIOWrapper+0x9

exlib!BasOpachedThread+0x4f

exlib!ThreadStart+0x18

KERNEL32!ThreadStart+0x52

From this call stack, you can verify that this thread was trying to read a file from disk. This information is recorded at the top of the stack, where the store called IFSRead, which called into ReadFile.

This is useful information, but you cannot be sure that the problem is only a disk performance problem. Next, you must determine what type of data was being read from the file at the time. In this case, this stack is processing a rule. For example, you know that you are doing work in the following folder.

16e34658 "/NON_IPM_SUBTREE/Deferred Action"

16e34698 ""

The only way to determine whether that thread has stopped responding is to obtain another dump file of all the threads, and then see whether thread 102 is still stuck in ReadFile several minutes later, and also several minutes after that. If thread 102 remains in ReadFile for that long, there might be a problem with the disk or SAN.

At this point, the information indicates why the server is performing slowly or has stopped responding, but you do not know for sure. You know that at one moment this thread was in ReadFile. That is not sufficient information to confirm that you have a disk performance problem.

When you obtain the three snapshots of memory, if they show that your threads are no longer backed up but the problem persists, you can examine all the threads to determine what was happening that caused the process to stop responding. You can use System Monitor logs and event logs to help gain a better understanding of why the threads are in this state, and why the process stopped responding.

Typically, PSS asks for multiple dump files because PSS support engineers have to know what the threads are doing at the time of failure during multiple moments in time.

APPENDIX C

This appendix describes how to use the Product Support Services File Transfer Application (PSSFTA) to receive files from Microsoft Support:

1. You will receive an e-mail message from your support engineer at Microsoft. The e-mail message will contain a Web site address, or Uniform Resource Locator (URL).

2. Your support engineer will contact you again, either by e-mail message or telephone, to give you a password:

• You must keep these e-mail messages confidential at all times, to help avoid unauthorized access to your files.

• Files will only remain at the Web site address for 24 hours. If you do not retrieve your file in 24 hours, it will be deleted. If this occurs, please contact your support engineer and ask to the have the files sent again.

3. Start your Web browser, and visit the Web site address that your support engineer provided in the e-mail message.

4. Click Receive Files from Microsoft.

5. Type the password that your support engineer provided, and then click GO. You are given access to the location that contains the files that you want.

6. Right-click the file name, and then click Save Target As.

7. In the Save As dialog box, select the destination path on the computer where you want to save the file, and then click Save. Click Close to close the dialog box. The file is copied to that destination.

8. If you have multiple files in this location, repeat steps 6 and 7 to make sure that all the files are copied.

9. Although Microsoft takes precautions to help make sure that the files we send are clean of any viral infection, always use your antivirus vendor's software to scan the files to help make sure that no infection exists. In the unlikely event that your antivirus software detects an infection, please contact the Microsoft support engineer who you are working with.

10. When you have received your files, exit the Web browser to close your session.

Important If you click any .txt, .htm, .html, or .jpg files, the file may automatically be displayed in the Web browser. For files that have these extensions, right-click the link, and then click Save Target As to save the file. If the file is large, the system may hang, depending on the capacity of the hard disk on your computer.

APPENDIX D

This appendix describes how to use the Product Support Services File Transfer Application (PSSFTA) to send files to Microsoft Support:

1. You will receive an e-mail message from your support engineer at Microsoft. The e-mail message will contain a Web site address, or Uniform Resource Locator (URL).

2. Your support engineer will contact you again, either by e-mail message or telephone, to give you a password.

• Keep these e-mail messages confidential at all times, to help avoid unauthorized access to your files.

• Files will only remain at this location for 24 hours. If for some reason your support engineer does not download your file in 24 hours, it will be deleted. If this happens, you may be asked to send the file again.

3. Start your Web browser, and then visit the Web site address that your support engineer provided in the e-mail message.

4. Click Send Files to Microsoft.

5. Type the password that your support engineer provided.

6. Click Browse, locate the file that you want to send, and then click Upload. If you need to send an additional file, click Send, and then repeat the process.

7. When you finish sending your files, quit your Web browser to close your session.

Important Do not upload a file that is larger than 1.93 GB. If you try to upload a file that is larger than 1.93 GB, the upload will fail. If you have to transfer a file that is larger than 1.93 GB, please contact your support engineer.

Frequently asked questions

Question: What are the file size limitations of the PSSFTA?

The PSSFTA will accept files that are up to 1.93 gigabytes (GB) for sending files to Microsoft and receiving files from Microsoft using PC hardware. Macintosh users are limited to 300 megabytes (MB) for sending files, and 1.93 GB for receiving files from Microsoft. If you have to transfer files that are larger than that, please contact your support engineer. Your support engineer can advise you of the appropriate choices that are available.

Question: What is the hard disk drive space requirement for receiving files?

To receive files from Microsoft, available hard disk drive space must be double the size of the file that is being received.

Question: What type of encryption do you use to help secure my data?

All authentication and transfers with the PSSFTA application use industry standard Secure Sockets Layer (SSL). To learn more about Secure Sockets Layer, visit the following Microsoft Web site and examine the "Understanding SSL" section:



Question: I cannot access my folders any longer and the application says my password is invalid. What happened?

All folders are removed every 15 days unless your support engineer receives an extension for that folder to continue resolving your support issue. The folders that the PSSFTA uses are designed to be used for an individual support issue. If you have to transfer files that are related to other issues to and from your support engineer, your support engineer will create additional folders for you.

Question: Who has access to my file?

Only the support engineer troubleshooting your issue, the support engineer's direct manager, and any additional support personnel who are required to help resolve your support issue will have access to your file. Any person with whom you share the location and password e-mail messages that your support engineer sent you will also have access to this data file. Do not share these e-mail messages at any time with any entity who you do not want to have unrestricted access to the files in those folders.

Question: What should I do if I believe my location and password information have been compromised?

Immediately stop use of the compromised location and contact your support engineer.

Question: What operating systems and Web browsers has the PSSFTA application been tested with?

This application has been tested with the following operating system and Web browser combinations:

o Macintosh with Microsoft Internet Explorer 5.1.x for Macintosh

o Macintosh with Microsoft Internet Explorer 5.2.x for Macintosh

o Microsoft Windows 2000 with Microsoft Internet Explorer 5.0

o Windows 2000 with Microsoft Internet Explorer 5.5

o Windows 2000 with Microsoft Internet Explorer 6

o Microsoft Windows 98 with Microsoft Internet Explorer 5.0

o Windows 98 with Microsoft Internet Explorer 5.5

o Windows 98 with Microsoft Internet Explorer 6

o Windows 98 with Netscape 4

o Microsoft Windows Millennium Edition (Me) with Microsoft Internet Explorer 5.5

o Microsoft Windows NT( 4.0 with Microsoft Internet Explorer 5.0

o Microsoft Windows XP Professional with Microsoft Internet Explorer 6

Question: What do I do when the file upload page is expired?

If this happens, please start again from the beginning; visit the Web page that the Microsoft support professional provided, type the provided password, click Send Files to Microsoft, and then click Go.

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download