Windows Error Reporting (“WER”) (previously “Online Crash Analysis” (“OCA”))

Overview

This program tends to pop up when software crashes. The software might be a user application, such as a web browser, or it might be the entire operating system. If the OS is what terminated uncleanly, Windows Error Reporting may be seen after the system reboots and a uesr logs in. As an example, if Firefox crashes, it is possible that both Mozilla Crash Reporter and also Windows Error Reporting may be run and visible on the screen.

The program is fairly intrusive for three reasons:

  1. The program does have the “Always On Top” behavior enabled, meaning that it ends to be visible a lot, even moreso than a foreground application which does not have that behavior enabled. (Another program that may have that behavior is Task Manager, although Task Manager allows the user to change whether that behavior is enabled or disabled.) In fairness, this behavior probably does help the program to be noticed in some cases where the program would otherwise be hidden by another window and, largely because of that, possibly not be noticed. (So, having this behavior might not be a very poor design choice. Still, that doesn't change the reality that the behavior is a bit intrusive.)
  2. The second reason is that some interactions with the program may cause the program to delete some of the files that contain data which was captured at, or shortly after, the time of the problem. This data can be useful for debugging as well as reproducing information at a later time, and so it can be worthwhile to save a copy of that data before it is deleted. Therefore, saving a copy ends up being a high priority task, because that needs to be done before certain types of interactions with the program that is “Always On Top”.

    [#hwwerdel]: How to cause data loss with Windows Error Reporting:

    Interactions which are believed to possibly lead to data being discarded by “Windows Error Reporting” include selecting an option to close the program, submitting data online, checking online for a solution, or telling “Windows Error Reporting” to restart the program that caused the crash. Be sure not to perform any of those items until making sure that a copy of the data is stored in a location where it will not be removed by “Windows Error Reporting” Essentially, the only safe actions with the window are to view details, copy information from those details, and to move the window around.

  3. Naturally, this tends to occur during a time of an already frustrating experience: when a program interrupts a user's progress by suddenly closing without any prior warning, potentially losing saved work. This isn't really the fault of the “Windows Error Reporting” program, but this fact does mean that the presense of the “Windows Error Reporting” program is a part of the whole experience which is interrupting a task that somebody was trying to perform.
What (not) to do when seeing the “Windows Error Reporting” screen

These are the recommended main goals to accomplish quickly whenever seeing the Windows Error Reporting screen:

  1. Do NOT accidentally cause data loss by interacting with the “Windows Error Reproting” program. (Read the referenced section for details on what to avoid.)
  2. Save a copy of ALL data which the “Windows Error Reporting” program might delete. (This can include files on the hard drive, as well as details that the “Windows Error Reporting” program may temporarily be keeping.) (More details about how to do this are about to be presented.)
  3. Once all that information is saved, then interact with the program. One possible course of action, which may be the best, is to simply close the program. Another possible course of action is to determining how sensitive the data is and then, after doing that, determining whether to submit information online before closing the program. (Submitting information online may help lead to improvements to prevent similar problems to the one that led to “Windows Error Reporting” being run.) After submitting information, “Windows Error Reporting” may run the the “Problem Reports and Solutions” control panel applet, and then close. One way or another, the program will probably be closed (and may remove some data) if one wishes to proceed past the stage of viewing details.

    A main reason to close this program is because the program is a bit intrusive by having the “Always On Top” behavior. A second reason, though, is that there is no reason to keep the program running if it has entirely served its purpose. If the purpose is to submit information, then submitting the information soon allows the information to be submitted before it is accidentally lost, and then allows the program to complete (and then be out of the way).

Of course, desired goals are likely going to be identifying the cause of the error and, if reasonably possible, to prevent such errors from occurring again. However, those can usually be done a few or many seconds or minutes, or even hours or days later. Interacting properly with “Windows Error Reporting” is often a comparitively quick process, and one of priority (in order to avoid loss of the data containing technical details about what happened). So, accomplishing those first three steps quickly is recommended.

The results

In truth, saving these files has definitely provided no quick, direct, visible payoff most of the time. (If software developers have managed to use some of the submitted data, that was behind the scenes and so was not visible.) After years of providing technical support, there was a small number of incidents where using the Minidump information provided some information which might have been useful, although in reality, the resulting decisions that were made were probably often the same decisions that would have been made without that data. So, if the data isn't really useful, why bother? There are reasons.

Saving this data is largely implementing a practice that has reaped rewards. That practice is simply to save information, rather than lose information, if there is a reasonable chance (even a low one) that the data will end up coming in useful at a later time. Following the steps described in this guide has helped by getting the intrusive program out of the way, but also allowing a peace of mind by knowing that the data had been saved instead of being (accidentally) deleted when interacting with the “Windows Error Reporting” software. I could know that the information has been retained, so that the information could be used if a good use for the information could be identified. Having this knowledge could be a source of comfort, which could help a mind to remain focused on a solution rather than being distracted by the despair of feeling like there are no identifiable solutions forthcoming and no visible options on other avenues to pursue.

[#wersvdat]: Save the data!
The initial details

(To be sung to the tune of the famous battle cry, “Save the whales!” Because, after all, the whales were endangered. And, in this case, the data is endangered!)

Some of this information may have been written before it was found that details may still be available. (For instance, are some submitted details available online? As possible examples: Redirector to bucket # 2030376906, Response SGD=bc7d60f9-c605-464e-a99f-b03b57707ef7, Response SGD=8bb9c1ed-35e6-4024-9d93-7df5a7655991) This text might be spreading a bit of caution which, in hindsight, appears may be undue...

First, be aware of what to do and What not to do with Windows Error Reporting before all needed data is saved.

An example of an initial screen from Windows Error Reporting (showing a web browser plugin failing), An example of an initial screen from Windows Error Reporting (after intentionally causing a BSOD)

Naturally, a possible action to take which will be tempting for someone trying to find out what happened is to choose the option that says “View problem details”. (Yes, that sane choice is the one choice which the screenshots show in text which is not bold.) Naturally, that option will end up showing more details.

Some pictures: Picture: The results after tabbing to choose to view more details, An example of more details about a web browser plugin crashing, The bottom of more details about a web browser plugin crashing, and “Windows Error Reporting” (from a web browser plugin crashing) with text scrolled up a bit. “Windows Error Reporting” made active, Results after tabbing once to “Hide problem details”, Results of pressing enter when “View problem details” is the option that is selected, The bottom of more details (visible by tabbing three times), which introduces a blinking text cursor at the bottom of the text area. Then, scrolling up shows:

Picture: “Windows Error Reporting” (with text scrolled up a bit)
.

Overview: Data that gets saved

Once the details are shown, one possible course of action is to manually copy those details (to a Notepad window, or by using something like a pen). Most often, writing/typing down the details that will not be necessary, and therefore is not a recommended use of time. The reason why such manual copying is generally not necessary is that many of the details may alredy be recorded onto the disk. Often, any visible numbers might be directly saved, or be able to be reproduced from the saved data. If the information on the disk is likely to be easily available at a later time (which might not be true in the case of an unreliable disk or where an attacker may be deleting logged information), then manually copying those same hexadecimal digits onto a piece of paper is just not a useful practice.

However, the fact is that “Windows Error Reporting” tends to show up when something incorrect happened, and in some situations where things are going wrong, a technician may not be able to reliably count on usual behaviors. Having given that disclaimer (to reduce the likelihood that somebody will justifiably feel mad about this advice in the rare case when recording the information would have been useful), realize that spending a lot of time storing the information may not be too useful, typically.

If the system is quite unstable (or if the system is being viewed by a remote connection which is not stable), then copying those details soon (before interacting much with the system) may be useful. In the case when a system is being checked remotely, and if no desired data is already in the clipboard, one effective method to copy the details may be to take screenshots and paste the graphic into a program running on the local machine. (Even if the local computer is basically acting like a terminal, the local machine may be the fast/easy way to paste the screenshot. In fact, storing information on a machine which is not exhibiting problems might be an approach that is more prone to avoid problems.)

If one does want to save the data, the fastest way of doing this is going to be to copy the details to the clipboard, and then dumping the first screen of contents into a copy of Microsoft Paint/Paintbrush. This may especially be a useful technique if the system running “Windows Error Reporting”. The graphical data might be easy to capture into the local system's copy of MS Paint. Often the useful information can be stored in just two images, so the second copy can also be pasted into the same MS Paint session. As long as no other changes are made in the Paint session, the two images can be recalled by undoing, and redoing, the second paste command. Just be sure to get that information reliably saved before it is lost.

The main thing that should be done, though, is to determine what data is saved.

Got filenames? Copy the files!
See the saved details to determine if there any filenames mentioned. If so, save a copy of the files (if it is safe to do so, e.g. not if problems are caused because there is no space free on the disk). Especially copy the WER* files (because they seem to be the files most likely to be deleted when “Windows Error Reporting” considers itself to be done with those files).
Compare with the operating system logs

As for the rest of the details, they can be useful for troubleshooting but many of them may already be stored on the hard drive by being stored in the event logs. Check if this is true by looking at the event logs. First, that will require locating related logs in the operating system logs. For each such log, check if it lists any files that haven't yet been copied to a safe place. Also check if the details in the operating system's logs match some of the details shown by “Windows Error Reporting”. If so, then there is little to no need to copy those details from the “Windows Error Reporting” program.

Possible event log entries may include:

If an application crashed
  • Application log, Source = “Windows Error Reporting”, Event ID = 1001, Type/Level = “Information”, Task Category = “None”
  • Application log, Source = “Application Error”, Event ID = 1000, Type/Level = “Error”, Task Category = “(100)”
If the operating system crashed
  • System log, Source = “BugCheck”, Event ID = 1001, Type/Level = “Information”, Task Category = “None”
    • An example of text: “The computer has rebooted from a bugcheck. The bugcheck was: 0x000000d1 (0xfffff88009540210, 0x0000000000000002, 0x0000000000000000, 0xfffffa6009ffa17a). A dump was saved in: C:\Windows\Minidump\Mini092310-01.dmp.”
  • Possibly information can sometimes come from a source called SaveDump (or some similar-named source)?

Note: the dump files have been known to be handled automatically. Sometimes dump files have been erased (or overwritten, erasing the old contents). Perhaps some styles of dump creation may be more prone to having this happen? If free space allows, making a copy of those dump files (by placing them in a different directory/folder) is recommended.

For each event log entry, there are three things to check for initially:

  • When did the error occur? (If the information was recorded before the last reboot, then it may be useful to know that the prior reboot may have had an error.)
  • Are there any files listed in the event log entry which haven't yet been backed up? (If so, back them up.)
  • Is the other text matching what is in “Windows Error Reporting”?

The main reason to compare with the “Windows Error Reporting” is to determine what data, out of the “Windows Error Reporting”, is redundant.

Wrapping up “Windows Error Reproting”

Hopefully one has found that most of the data has been stored in the logs. Some of the data may not be in the user logs, such as the operating system service pack number. However, that data probably also isn't quite as useful.

The next thing to do may be to check if there are any dialog boxes that appeared when the system was started or logged in. Check the System log for an Information Event from the Source “Application Error” and which uses a known, recognizable ID (ID 26?). If so, realize that such a dialog box was probably made visible on the system's main console. If working on the system via remoting into the system, the dialog box may not be visible. However, if someone else (perhaps someone in management or an owner of the organization that owns the computer) sees the message, it may seem less professional to have a bunch of error messages that look like they haven't been dealt with. The goal isn't necessarily to cover one's tracks, but rather to clean up after one's self when assigned the task of dealing with the problem.

After submitting data

The data may turn into a bucket.

Windows Error Reporting: Getting Started.

A “bucket” is a collection of error data on the “Windows Error Reporting” site which is being treated as if it is caused by the same bug as other error data (that has been or will be) submitted to the “Windows Error Reporting” site.

PDF file: “Debugging in the (Very) Large: Ten Years of Implementation and Experience (section 6.3) shows that errors may re-bucket.

Data which is in a bucket may have been moved to C:\ProgramData\Microsoft\Windows\WER. (The files would be placed in a .CAB file, and an event logged. The event would have the source “Windows Error Reproting”, ID 1001, Type/Level: Information. The event lists the files that were attached; those files may be found (renamed) in the .CAB file of the related report.

(The following are some scratch notes that were made, and which are likely related to usage of this program.

Often info may be stored in files or shown in error messages. Back up those files. Compare the info to operating system logs; record info that is not shown in the operating system logs (e.g. under BugCheck source).

Wikipedia's page on Windows Error Reporting says: Solutions are served using Windows Error Reporting Responses info

After wer00015.png, a UAC prompt showed: Problem Reports and Solutions {4BC67F23-D805-4384-BCA3-6F1EDFF50E2C} which is: ERCLuaElevationHelper and this opens a directory such as: C:\Users\Sysop\AppData\Local\Temp\WERDA7.tmp

NirSoft AppCrashView

Does the log show any details about the cause of the issue? Is there bugcheck information logged?

Newer operating systems might have WER's reporting functionality accessible from the Control Panel. (In Windows Vista, perhaps this was called “Windows Problem Reporting”? In Windows 7, perhaps this is in the Action Center control panel applet, by finding the “Check for solutions to unreported problems” and clicking on the hyperlink that says “View problems to report”?)