Working With IPSec Policies

By: Brien M. Posey, MCSE

If you’ve studied Windows 2000 security much, then you know that one of the biggest security features that’s included in Windows 2000 is the IPSec protocol. IPSec is a protocol that’s designed to protect individual TCP/IP packets traveling across your network by using public key encryption. In a nut shell, the source PC encapsulates the normal IP packet inside of an encrypted IPSec packet. This packet then remains encrypted until it arrives at the destination PC. While this concept sounds simple enough, there’s actually quite a bit that you need to know about IPSec before you can effectively use it on your network.

One of the first things that you should know about IPSec is that it’s slower than a normal IP packet because of the larger packet size and the overhead required for encryption and decryption. The larger packet size also means that IPSec can consume more network bandwidth than traditional IP packets. Needless to say, you probably only want to use IPSec for communications that really need to be secure. Fortunately, using IPSec isn’t an all or nothing situation. There are ways for telling Windows which communications need to be performed through IPSec and which communications can be sent through traditional packets. Such rules can be established through the use of IPSec policies. In this article, I’ll introduce you to the concept of IPSec policies. As I do, I’ll explain how to implement various types of IPSec policies in your organization.

What’s an IPSec Policy?

An IPSec policy is nothing more than a set of rules that govern when and how Windows 2000 uses the IPSec protocol. The IPSec policy interacts directly with the IPSec driver. The policy tells Windows such things as which data to secure and which security method to use.

Elements of an IPSec Policy

Before I jump right in to showing you how to work with IPSec policies, I wanted to take a few moments and explain the basic elements of an IPSec policy. You’ll need to know what each of these elements are and what they do before you’ll be able to effectively use them.

IPSec policies work by determining which IP traffic should be secured and which IP packets should be left alone. This is accomplished through the use of an IP filter list, individual IP filters, and filter actions. The IP filter is a tells Windows that a certain types of IP packets need to have some type of action applied to them. In this case that action (the filter action) may be to secure the packets. The IP filter list is a collection of individual filters that the filter action is applied to.

Once you’ve established the basic IP filtering, you’ll have to provide the IPSec policies with some information about your network. This information may include things like the security method to use, the connection type, and the tunnel settings. The security method simply dictates which security algorithms should be used during the authentication process and which algorithms should be used for key exchanges. The connection type refers to whether the policy should be applied to remote access connections, LAN connections, or all network connections regardless of the type. The tunnel settings are only used if you’re using IPSec over a virtual private network. The tunnel settings define the DNS name or the IP address of the tunnel’s end point.

Each of the elements that I’ve described in this section combined together form a rule. An IPSec policy is a collection of one or more rules..

Editing The Built In IPSec Policies

Now that you know what goes into an IPSec policy, let’s look at a real life policy. To do so, click the Start button and select the Run command from the Start menu. At the Run prompt, enter the MMC command. When the Microsoft Management Console appears, select the Add / Remove Snap-In command from the Console menu. You’ll now see the Add / Remove Snap In dialog box. Click the Add button and you’ll see a list of the available console snap-ins. Select IP Security Policy Management from the list and click the Add button. You’ll now see a dialog box that asks which computer or domain that the policy should manage. For the purposes of this article, I’ll be managing the policies for the local domain. Therefore, select the Manage Domain Policy For This Computer’s Domain radio button and click the Finish button. Now, click the Close button followed by the OK button and you’ll see the IP Security Policy Management snap-in added to the console.

If you select the IP Security Policies On Active Directory object from the Console tree, you’ll see that there are three built in IPSec policies. You can either implement these policies into your network as is, or you can use them as a building block for more complex policies. The first policy on the list is the Client (Respond Only) policy. This policy is designed to be run on client machines that don’t normally need to worry about security. The policy is designed in such a way that the client will never initiate secure communications on its own. However, if a server requests that the client go into secure communications mode, the client will respond appropriately.

The next policy on the list is the Secure Server (Require Security) policy. This policy is only appropriate for servers that require all communications to be secure. Once this policy has been applied, the server will neither send or accept insecure communications. Any client wanting to communicate with the server must use at least the minimum level of security described by the policy.

The final policy on the list is the Server (Request Security) policy. Contrary to the name, this policy can be used on both client and server PCs. This policy will use IPSec security for all outbound security. However, this policy will accept insecure inbound communications. If a client requests a secure session, the policy will allow the client to establish one.

Now that you’re familiar with the individual policies, let’s revue the procedure that you’d use for editing one of them. Because the Secure Server (Require Security) policy is the most complex of the three, let’s look at it more closely. To do so, right click on the policy and select the Properties command from the resulting context menu. When you do, you’ll see the policy’s properties sheet. By default, the Rules tab will be selected. The Rules tab displays a list of all of the rules contained in the policy. Each rule has a check box next to it. If the check box contains a check mark then the rule is active within the policy. You can edit any of the rules by selecting the rule and clicking the Edit button.

When you edit a rule, you’ll see the Edit Rule Properties sheet. This properties sheet contains five different tabs. The default tab is the IP Filter List tab. This tab allows you to add, edit, and remove IP filters. The adjacent tab is the Filter Action tab. The Filter Action tab contains three different radio buttons. These radio buttons allow you to select the type of filter action that you want to use. You can allow insecure IP traffic, or you can set up the filter action to request security or to require security. You can manipulate any of the built in settings by selecting the filter action and clicking either the Add, Edit or Remove button.

On the next row of tabs, the first tab that you’ll come to is the Authentication Methods tab. By default, this tab is set to use Kerberos. However, by using the Add, Edit and Remove buttons you can set the Authentication Method to use a certificate server or to use a pre shared key.

The next tab that you’ll encounter is the Tunnel Settings tab. By default, the rules within this IPSec policy don’t apply to a tunnel. However, you could easily change all that by selecting the The Tunnel Endpoint Is Specified By This IP Address radio button, and entering the corresponding IP address.

The final tab on the Edit Rule Properties sheet is the Connection Type tab. This tab allows you to specify whether the rule should apply to remote access traffic, Local Area Network (LAN) traffic, or all network traffic, by selecting the corresponding radio button. When you’ve finished editing a rule, click the OK button twice to return to the main console screen.

Creating an IPSec Policy

Now that you know how to edit an existing policy, let’s take a look at the procedure for building a new policy from the ground up. To create a new policy, return to the main console screen and right click on the IP Security Policies on Active Directory container and select the Create IP Security Policy command from the resulting context menu. When you do, Windows will launch the IP Security Policy Wizard. Click Next to skip the introduction screen. The next screen that you’ll encounter asks you to specify the name and a description for the policy that you’re creating. Enter this information and click Next to continue.

At this point, you’ll see a screen that explains that in order for there to be any amount of security, the policy must contain a rule that allows it to respond to requests for secure communications. Assuming that you want to leave this rule enabled, make sure that the Activate The Default Response Rule check box is selected and then click Next to move on. The next screen you’ll see is a screen that asks which security method that you want to use for the default rule. By default, Windows is set to use Kerberos version 5. You can select any of the available options, but the Wizard will only let you select one authentication method. You can enable multiple authentication methods, but to do so, you’ll have to go back later on and edit the rule in the same manner as I used earlier. Click Next to continue.

At this point, you’ll see a screen that informs you that you’ve completed the wizard and established a basic IPSec policy. The wizard also gives you the option of editing the policy that you just created. All you have to do is to make sure that the Edit Properties check box is selected and then click the Finish button.

Windows will now open the properties sheet for the policy that you just created. If you need to modify the default rule, you can do so by selecting the rule and clicking the Edit button. The process for editing this rule is identical to the process that I described earlier.

Often times, a single rule simply isn’t enough for a policy. You can add other rules to the policy by making sure that the Use Add Wizard check box is selected and then clicking the Add button. This will launch the Security Rule Wizard. This wizard is a little bit different from the wizard that you used earlier.

Begin by clicking the Next button to bypass the introduction screen. The next screen that you’ll encounter asks if the rule will apply to a tunnel. If the rule applies to a tunnel, select the appropriate radio button and enter the IP address of the tunnel’s end point. Otherwise, select the This Rule Does Not Specify A Tunnel radio button and click Next.

You’ll now see a screen that asks what type of network traffic that the rule should apply to. Select either All Network Connections, Local Area Network (LAN), or Remote Access and click Next.

Next, you’ll see a screen that asks for the authentication method to be used. This screen is identical to the one that you saw earlier. Choose your authentication method and click Next.

The next screen that you’ll see asks if the rule should apply to IP traffic or to ICMP traffic. Make your selection and click the Edit button. You’ll now have the opportunity to configure the filtering options for the protocol that you’ve selected. When you’ve made your selection, click OK followed by next.

You’ll now see a screen similar to the one that you saw earlier. It asks whether you want to use Permit, Request Security, or Require Security as your filter action. Make your selection and click Next.

You’ll now see the last screen of the wizard. This screen gives you the chance to edit the properties of the rule that you just created by selecting the Edit Properties check box. Whether or not you want to edit the rule’s properties, click the Finish button to close the wizard.

As you can see, IPSec policies can go a long way to controlling the way that Windows handles IPSec traffic. If your network needs to secure some, but not all of the traffic flowing across it, you can free up a lot of bandwidth by creating the appropriate IPSec policies.

If you've found this article helpful then please consider making a donation to help with the cost of keeping this site going. To make a donation, please click on the PayPal link below.