Troubleshooting Openswan

Troubleshooting Openswan

Troubleshooting is always important when setting up VPNs, because many things can go wrong. Here are some quick checks you can do to make sure all is working correctly.

Determine the Tunnel Status

The ipsec auto --status command provides a status on Opesnswan running on your VPN device. The output is divided into three sections:

  • IKE Section: Defines the various encrypted key exchange algorithms and their parameters. At least one set of values must match between the left- and right-hand side VPN devices. This is also frequently referred to as the Phase 1 parameters, because the key exchange process is the first thing to occur in establishing a VPN.

  • ESP Section: Defines the various data encryption algorithms and their parameters. At least one set of values must match between the left- and right-hand side VPN devices. This is also frequently referred to as the Phase 2 parameters, because the data encryption process is the second and final thing to occur in establishing a VPN.

  • VPN Section: This is usually prefaced by the name of the VPN tunnel, in this case net-to-net. If there are no entries, then the VPN hasn't been established at all. If there are entries, but no STATE_QUICK_R2 (IPsec SA established) lines then the IPSec parameters are configured, but the tunnel hasn't been established. This can be normal, tunnels become active once the Phase 1 and Phase 2 security associations are created, and this usually only occurs after traffic is flowing. The associations then get torn down after a timeout period. It is always best to pass traffic over the tunnel to activate them. An ICMP ping check is a good way to test this:

    [[email protected] tmp]# ipsec auto --status
    000 algorithm ESP encrypt: id=2, name=ESP_DES, ivlen=8, key-
    sizemin=64, keysizemax=64
    000 algorithm ESP encrypt: id=3, name=ESP_3DES, ivlen=8, key-
    sizemin=192, keysizemax=192
    000 algorithm ESP encrypt: id=7, name=ESP_BLOWFISH, ivlen=8, key-
    sizemin=40, keysizemax=448
    000 algorithm IKE encrypt: id=7, name=OAKLEY_AES_CBC, block-
    size=16, keydeflen=128
    000 algorithm IKE encrypt: id=5, name=OAKLEY_3DES_CBC, block-
    size=8, keydeflen=192
    000 algorithm IKE hash: id=2, name=OAKLEY_SHA, hashsize=20
    000 "net-to-net":   IKE algorithms wanted: 5_000-1-5, 5_000-
    1-2, 5_000-2-5, 5_000-2-2, flags=-strict
    000 "net-to-net": IKE algorithms found: 5_192-1_128-5,
    5_192-1_128-2, 5_192-2_160-5, 5_192-2_160-2,
    000 "net-to-net":    IKE algorithm newest: 3DES_CBC_192-MD5-
    000 "net-to-net":    ESP algorithms wanted: 3_000-1, 3_000-2,
    000 "net-to-net":    ESP algorithms loaded: 3_000-1, 3_000-2,
    000 "net-to-net":    ESP algorithm newest: 3DES_0-HMAC_MD5;
    000 #4: "net-to-net" STATE_QUICK_R2 (IPsec SA established);
    EVENT_SA_REPLACE in 3635s; newest IPSEC; eroute owner
    000 #4: "net-to-net" [email protected]
    [email protected] [email protected] [email protected]
    000 #12: "net-to-net" STATE_MAIN_R3 (sent MR3, ISAKMP SA
    established); EVENT_SA_REPLACE in 1153s; newest ISAKMP
    [[email protected] tmp]#

It is important to know the status of your VPN as it can provide valuable troubleshooting clues when there are problems. This can be especially important when establishing a VPN between your Linux firewall and a non-Linux device. The IKE and ESP timers and encryption, hash, and key exchange algorithms must match at both ends to have success. This command allows you to see the value combinations Openswan is using and helps you configure the VPN device on the other end to have compatible settings.

Testing VPN Connectivity

You can test the VPN connectivity by sending a simple ping from one private network to the other. In this case, the ping goes to the Windows server, which is protected by vpn2, from server, which is protected by vpn1.

If the tunnel is up but ICMP pings don't work, then you need to check that the servers at both ends of the tunnel have routes pointing to the VPN firewalls. Also, check for additional network access controls between the servers and the VPN firewall. There may be additional firewalls in the way or the servers themselves may be running firewall software.


    Pinging with 32 bytes of data:

    Reply from bytes=32 time=20ms TTL=253
    Reply from bytes=32 time<10ms TTL=253
    Reply from bytes=32 time=10ms TTL=253
    Reply from bytes=32 time<10ms TTL=253

    Ping statistics for
        Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
        Minimum = 0ms, Maximum = 20ms, Average = 7ms


Check the Routes

You'll need to check the routes after the VPN tunnel is up as well. As you can see, there is an additional route to the network on firewall vpn2. Incorrect routing on the firewalls can cause problems, check your leftnexthop and rightnexthop values in your /etc/ipsec.conf file:

    [[email protected] tmp]# netstat -nr
    Kernel IP routing table
    Destination Gateway       Genmask           Flags MSS Window irtt Iface     U     40 0          0 eth1   U     40 0          0 eth0     UG    40 0          0 eth0         U     40 0          0 lo           UG    40 0          0 wlan0
    [[email protected] tmp]#

Using tcpdump

If the tunnel doesn't appear to become established, use the tcpdump command as explained in Chapter 4, "Simple Network Troubleshooting," to determine whether the traffic is being seen at both ends of the tunnel. You need to know whether the IPSec packets are even reaching your VPN firewall. Check routing and your Openswan configuration if not.

Protected Interface TCPDump Output from vpn2

Here the TCPdump is done on server2 where you see unencrypted ICMP ping traffic successfully passing back and forth between the two servers:

    [[email protected] tmp]# tcpdump -n -i eth1 icmp
    03:05:53.971308 IP > icmp 64: echo
    request seq 89
    03:05:53.995297 IP > icmp 64: echo reply seq 89
    03:05:54.972759 IP > icmp 64: echo request seq
    03:05:54.972789 IP > icmp 64: echo reply seq 90
    03:05:55.972985 IP > icmp 64: echo request seq
    03:05:55.972999 IP > icmp 64: echo reply seq 91
    [[email protected] tmp]#

Unprotected Interface TCPDump Output from vpn2

Here the encrypted ESP traffic is encapsulating the pings passing back and forth between the two VPN boxes. The true source and destination IP addresses ( and are hidden.

    [[email protected] tmp]# tcpdump -n -i eth1 host
    02:08:23.637149 IP >
    02:08:24.635302 IP >
    02:08:24.637988 IP >
    02:08:25.638015 IP >
    [[email protected] tmp]#

Check syslog Error Messages

As always, check your /var/log/messages file for any messages that may provide clues as to the source of your problems.

Invalid Key Messages

If your left and right public keys were incorrectly applied to your /etc/ipsec.conf file or your regenerated keys were not updated in your /etc/ipsec.secrets file, then you will get messages stating that the keys are invalid and that information is being ignored:

    003 "net-to-net" #1: ignoring informational payload, type
    003 "net-to-net" #1: received and ignored informational message
    003 "net-to-net" #1: discarding duplicate packet; already
    031 "net-to-net" #1: max number of retransmissions (2) reached
    STATE_MAIN_I3. Possible authentication failure: no acceptable
    response to our first encrypted message

     Python   SQL   Java   php   Perl 
     game development   web development   internet   *nix   graphics   hardware 
     telecommunications   C++ 
     Flash   Active Directory   Windows