PST File Maintenance

By: Brien M. Posey

 Reprinted with permission from

If you’ve used Outlook for any length of time at all, you’re no doubt familiar with the concept of personal folders, and the associated PST files. What you may not realize though is that PST files are susceptible to several different types of problems. In this article, I’ll explain how Microsoft recommends using PST files. I’ll then go on to examine what can happen if PST files are used outside of the recommended purposes. Finally, I’ll explain how to recover from these types of problems.

 The Real Reason For PST Files

 Many people tend to use PST files as a large scale data storage solution. However, PST files were never actually intended for this purpose. It’s been my experience over the years that PST files are often used as a primary storage mechanism because of the unreliability of earlier versions of Exchange Server. For example several years ago, Fort Knox had an incident in which one of their Exchange 5.0 servers failed. It was soon determined that both the public and private information stores were corrupt. The obvious natural step was to restore the information stores from backup. Unfortunately, none of the backups were valid. The harsh reality was that for at least a month, Fort Knox had been backing up corrupt data. Because the corruption wasn’t initially severe enough to cause a fatal crash, there had been no way of knowing that the information stores were corrupt. In the end, all of the data on the server was permanently lost.

 The officer in charge mandated that every Exchange user on the base have Outlook configured to automatically download all messages to a network based personal folder. The idea was that if the private information store were to fail again, then a minimal amount of data would be lost because all but the most recent messages already existed in the user’s personal folders.

 In retrospect, this was probably a good solution at the time. However, as you’ll see from reading this article, if the situation were repeated today (in an Exchange 2000 environment), the Fort Knox solution would be one of the worst possible ways of dealing with the problem. To understand why this is the case, you need to know a little bit about the true purpose of personal folders.

 As with offline folders (OST files), Microsoft designed personal folders so that mobile users could take some important messages and other data with them, and access the data while offline. Today, Microsoft recommends using offline folders for this purpose. They recommend that personal folders should be used only by Outlook users who access Internet Mail only (not an Exchange Server). Microsoft’s recommendation is that with the exception of mobile users who need to access data while offline, Exchange users should store all messages within the Exchange information store rather than in a personal folder.

 Network Based PST Files

 There are several reasons why Microsoft discourages the use of personal folder files. One reason is because traditionally, many users tend to store their personal folders on their local hard drive, where the PST files tend not to be backed up. Of course, it’s possible to store a PST file on a network server so that the PST file is backed up regularly, but doing so opens up another can of worms.

One problem with networked PST files is that you can’t backup an open file. This means that if a user happens to leave Outlook open over night, then depending on the backup software that you’re using, the file will either not be backed up at all, or the backed up file will be invalid, and therefore unusable if it ever needed to be restored. Obviously, leaving Outlook open all night is a major security problem, but I’ve seen it happen more times than I can count.

OK, let’s assume that every user closes Outlook and logs out when they go home for the day. There is still another problem with networked PST files. As you’re no doubt aware, the Exchange information stores are databases. As such, any time that someone sends or receives a message, they are actually updating a database table. On the other hand, when ever someone uses a personal folder, the operation is file based rather than database based. This means that read and write operations to personal folders take much longer and place a higher demand on the server than reading and writing directly to the Exchange information stores do. In face, write operations can take up to four times longer than the already slow read operations. As you can see, if you’ve got a lot of users who are continuously updating network based PST files, then the operations can place a tremendous load on the network.

PST File Corruption

Another problem with PST files is that as they get larger, and more heavily used, they can tend to become corrupted. Microsoft does provide a utility called the Inbox Repair Tool that you can use to recover data from a corrupted PST file. However, the Inbox Repair Tool isn’t always able to recover all of the lost data. Essentially, the Inbox Repair Tool works by rewriting the PST file’s header information, and then deleting anything within the file that it isn’t able to understand. Therefore if a PST file has been damaged because the file’s header is damaged, then the Inbox Repair Tool should have no trouble recovering the entire file. If, the actual data has been damaged, then some or all of the file’s contents could be lost.

 If you have a PST file that’s in need of repair, you can find the Inbox Repair Tool in he C:\Program Files\Common Files\System\Mapi\1033\NT directory of any system that’s running Windows 2000 or Windows XP. Other versions of Windows also contain a copy of the Inbox Repair Tool, but the location varies among Windows versions. The utility’s filename is SCANPST.EXE.

 As you can see in Figure A, the Inbox Repair Tool is very simple to use. Simply enter the filename of the damaged file and click Start. The utility does the rest. If the Inbox Repair Tool is able to recover some data, it’s a good idea to create a brand new PST file and copy the data from the damaged PST file to the new PST file, and begin using the new file rather than the previously damaged copy. Doing so insures that absolutely no corruption remains within the file.

 The 2 GB Limit

 Perhaps the strongest reason for not relying on PST files is that a PST file has a 2 GB size limit. At the time when Fort Knox forced everyone to use a PST file, a 2 GB personal folder was absolutely unheard of. However, today with messages often containing large attachments, it isn’t that difficult to accumulate 2 GB of data within a personal folder.

The problem is that once a PST file has exceeded the 2 GB limit, then the next time that Outlook is open, the user will see the following message:

Errors have been detected in the file <file_path> . Quit all mail-enabled applications, and then use the Inbox Repair Tool.

The message is misleading though because the Inbox Repair Tool is incapable of fixing the problem. Instead, you’ll have to use a utility called the Oversized PST and OST Crop Utility. Unfortunately, there are some serious drawbacks to using the utility, so it’s much better to never reach the 2 GB limit than to depend on this utility.

The first limit is that your hard disk must have at least 2 GB of free space, because the utility makes a copy of the oversized file. Perhaps the most compelling reason not to rely on the utility is that the utility functions by truncating the oversized file to a file size below the 2 GB limit. At first this may not sound so bad. After all, if one message pushed the file over the size limit then you won’t lose much data right? Although this sounds good in theory, it isn’t how the file works. Even if a PST file is only slightly above the 2 GB limit, the utility will truncate anywhere from 20 to 25 MB worth of data. Furthermore, there’s no way of controlling what is lost and what is saved.

Running the utility involves selecting the bloated file, specifying a name for the new file, and telling the utility how much data to delete. According to Microsoft, the utility will usually work if you delete 20 to 25 MB of data. Sometimes you may be able to get away with deleting a little less data, but often you may have to delete more. The basic idea is that if the utility doesn’t work, then keep deleting more and more data until it eventually does. Unfortunately, Microsoft makes no guarantees that the utility will ever work at all. Once you’ve used the utility to create a new PST file, you must run the Inbox Repair Tool against the new file before it will be usable.

Still another major deterrent to using the utility is the difficulty in acquiring the utility. The Oversized PST and OST Crop Tool isn’t available for download. You can only get the utility by calling Microsoft’s Product Support Services line. The phone number in the United States is (800) 936-4900. You must use menu option 4 for Exchange and messaging application support. When an operator eventually answers the call, they will ask for a credit card number. To avoid being billed for technical support, explain that you are calling to get a copy of the Oversized PST and OST Crop Tool. According to several sources, you can get the tool for free as long as you explain to the operator that you simply need the tool rather than needing help using the tool.

Obviously, it’s better to avoid having a bloated PST file than to have to resort to using the Oversized PST and OST Crop Tool.  The best way to accomplish this is to not use PST files at all. If this is impractical, then there are a few other techniques that you can use to avoid the problem. One such technique is to obtain and install the Office 2000 SR-1A update or to upgrade to Office XP. Although I have been personally unable to verify it, supposedly Outlook XP and Outlook 2000 SR-1A prevent Outlook from creating PST files over 2 GB. You can read more details by going to http://support.microsoft.com and reading Knowledgebase article Q245025.

Another solution is to put PST files onto a dedicated NTFS partition and use Windows 2000 / Windows XP’s disk quota management feature to generate a warning when the user is getting near the 2 GB limit. If you must keep the PST files on a network drive, you can use the same partition for each user, just make sure that the partition is configured to generate space warnings for every user.

Conclusion

 In this article, I’ve explained that bloated PST files can lead to poor network performance, file corruption, and lost data. I then went on to explain several different ways that you can partially recover from these types of problems.

If you've found this article helpful then please consider making a donation to help with the cost of keeping this site going. To make a donation, please click on the PayPal link below.