Cisco VPN Scripts

I have found that the Cisco VPN client occasionally hangs up on its connection. The reason is because the client removes the local network route which works fine until the MAC address cache expires and needs to be refreshed. Linux can't find its local route and basically drops the vpn, at which time the Cisco client conveniently puts the route back making it extra hard to track.

The really nasty part is you can't add the route ahead of time as the client will just remove it no matter how many there are and you can't add it later from the same shell as any attempt to background the client will end badly. My solution is to capture the local route and background a sub-shell which will add the route 60 seconds after the vpn client starts. I thought this would give enough time for the user to establish the connection but not be too long as to expire the MAC address cache. Here it is:

#!/bin/bash
# look for the interface that is up with a gateway assigned (UG) and grab the last field
DEV=`netstat -rn | grep UG | awk '{print $NF}'`
# This should just grab the one local route for the default interface
NETSTAT=`netstat -rn | grep $DEV | grep -v "^127\|^0.0\|169.254"`
NETWORK=`echo $NETSTAT | awk '{print $1}'`
MASK=`echo $NETSTAT | awk '{print $3}'`

# This says after 60 seconds add the local route back in
# you can't do this after vpnclient as it can't be backgrounded without a username / password on the command line
(sleep 60 && sudo /sbin/route add -net $NETWORK netmask $MASK dev $DEV)&

# Finally run the vpnclient
vpnclient connect mypcf_file

In order to make this work, your user account has to be able to execute sudo for /sbin/route without a password. For me I added my group to /etc/sudoers with the following entry:

# visudo
%users  ALL=(ALL) NOPASSWD:  /sbin/route

Cisco VPN Installation

The Cisco VPN module has been a bit of a sore point to get compiled and running. Here are some instructions I used under OpenSuSE 11.3 with kernel 2.6.34-12-desktop but it should work on other distributions too.

You are going to need three pieces of code, the VPN client, a 64 bit patch, and a patch to work with a 2.6.31+ kernel. I have
To start you will need three pieces of code:

• The VPN client itself
• A 64 bit patch
• A 2.6.31+ patch or as a file from here (thanks Ngo)

So lets see what happens with just the base VPN client:

# tar -zxvf vpnclient-linux-x86_64-4.8.02.0030-k9.tar.gz
# cd vpnclient
# ./vpn_install
Making module
make -C /lib/modules/2.6.34-12-desktop/build SUBDIRS=/home/mike/cisco/vpnclient modules
make[1]: Entering directory `/usr/src/linux-2.6.34-12-obj/x86_64/desktop'
make -C ../../../linux-2.6.34-12 O=/usr/src/linux-2.6.34-12-obj/x86_64/desktop/. modules
/usr/src/linux-2.6.34-12/scripts/Makefile.build:49: *** CFLAGS was changed in "/home/mike/cisco/vpnclient/Makefile". Fix it to use EXTRA_CFLAGS.  Stop.
make[3]: *** [_module_/home/mike/cisco/vpnclient] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-2.6.34-12-obj/x86_64/desktop'
make: *** [default] Error 2
Failed to make module "cisco_ipsec.ko".

Not so good, lets install the 64 bit patch and see what happens:

# patch < ../vpnclient-linux-4.8.02-64bit.patch
patching file Makefile
patching file frag.c
patching file interceptor.c
patching file linuxcniapi.c
patching file linuxkernelapi.c
# ./vpn_install
Making module
make -C /lib/modules/2.6.34-12-desktop/build SUBDIRS=/home/mike/cisco/vpnclient modules
make[1]: Entering directory `/usr/src/linux-2.6.34-12-obj/x86_64/desktop'
make -C ../../../linux-2.6.34-12 O=/usr/src/linux-2.6.34-12-obj/x86_64/desktop/. modules
  CC [M]  /home/mike/cisco/vpnclient/linuxcniapi.o
/home/mike/cisco/vpnclient/linuxcniapi.c:14:28: fatal error: linux/autoconf.h: No such file or directory
compilation terminated.
make[4]: *** [/home/mike/cisco/vpnclient/linuxcniapi.o] Error 1
make[3]: *** [_module_/home/mike/cisco/vpnclient] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-2.6.34-12-obj/x86_64/desktop'
make: *** [default] Error 2
Failed to make module "cisco_ipsec.ko".

Now we have a strange error message about a missing autoconf.h file. To fix this we need to know what kernel we are running by using uname. In my case it is 2.6.34-12-desktop. It is the desktop portion that is important as under /usr/src/linux-2.6.34-12-obj/x86_64 there are a few directories, default, desktop, and xen. You need to make sure you are working with the correct one. To get around the error just touch an empty file:

# touch /usr/src/linux-2.6.34-12-obj/x86_64/desktop/include/linux/autoconf.h
# ./vpn_install
Making module
make -C /lib/modules/2.6.34-12-desktop/build SUBDIRS=/home/mike/cisco/vpnclient modules
make[1]: Entering directory `/usr/src/linux-2.6.34-12-obj/x86_64/desktop'
make -C ../../../linux-2.6.34-12 O=/usr/src/linux-2.6.34-12-obj/x86_64/desktop/. modules
  CC [M]  /home/mike/cisco/vpnclient/linuxcniapi.o
  CC [M]  /home/mike/cisco/vpnclient/frag.o
  CC [M]  /home/mike/cisco/vpnclient/IPSecDrvOS_linux.o
  CC [M]  /home/mike/cisco/vpnclient/interceptor.o
/home/mike/cisco/vpnclient/interceptor.c: In function ‘interceptor_init’:
/home/mike/cisco/vpnclient/interceptor.c:132:8: error: ‘struct net_device’ has no member named ‘hard_start_xmit’
/home/mike/cisco/vpnclient/interceptor.c:133:8: error: ‘struct net_device’ has no member named ‘get_stats’
/home/mike/cisco/vpnclient/interceptor.c:134:8: error: ‘struct net_device’ has no member named ‘do_ioctl’
/home/mike/cisco/vpnclient/interceptor.c: In function ‘add_netdev’:
/home/mike/cisco/vpnclient/interceptor.c:271:33: error: ‘struct net_device’ has no member named ‘hard_start_xmit’
/home/mike/cisco/vpnclient/interceptor.c:272:8: error: ‘struct net_device’ has no member named ‘hard_start_xmit’
/home/mike/cisco/vpnclient/interceptor.c: In function ‘remove_netdev’:
/home/mike/cisco/vpnclient/interceptor.c:294:12: error: ‘struct net_device’ has no member named ‘hard_start_xmit’
make[4]: *** [/home/mike/cisco/vpnclient/interceptor.o] Error 1
make[3]: *** [_module_/home/mike/cisco/vpnclient] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-2.6.34-12-obj/x86_64/desktop'
make: *** [default] Error 2
Failed to make module "cisco_ipsec.ko".

Got rid of that autoconf.h message but now we have an interceptor problem. The 2.6.31 patch will take care of that for us.

# patch < ../vpnclient-linux-2.6.31-final.diff
# ./vpn_install
Making module
make -C /lib/modules/2.6.34-12-desktop/build SUBDIRS=/home/mike/cisco/vpnclient modules
make[1]: Entering directory `/usr/src/linux-2.6.34-12-obj/x86_64/desktop'
make -C ../../../linux-2.6.34-12 O=/usr/src/linux-2.6.34-12-obj/x86_64/desktop/. modules
  CC [M]  /home/mike/cisco/vpnclient/interceptor.o
/home/mike/cisco/vpnclient/interceptor.c: In function ‘add_netdev’:
/home/mike/cisco/vpnclient/interceptor.c:284:5: error: assignment of read-only location ‘*dev->netdev_ops’
/home/mike/cisco/vpnclient/interceptor.c: In function ‘remove_netdev’:
/home/mike/cisco/vpnclient/interceptor.c:311:9: error: assignment of read-only location ‘*dev->netdev_ops’
make[4]: *** [/home/mike/cisco/vpnclient/interceptor.o] Error 1
make[3]: *** [_module_/home/mike/cisco/vpnclient] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-2.6.34-12-obj/x86_64/desktop'
make: *** [default] Error 2
Failed to make module "cisco_ipsec.ko".

One more error to fix. This one involved changing netdevice.h in the kernel source tree from const struct net_device_ops *netdev_ops to just struct net_device_ops *net_device_ops. We can do that with one line as shown below

# sed -i 's/const\ struct\ net_device_ops\ \*netdev_ops;/struct\ net_device_ops\ \*netdev_ops;/' `find /usr/src -name netdevice.h`
# ./vpn_install

Success, the module compiles and installs. Now we just need to run it. To do this you will need a pcf file from your VPN administrator. For me, I took the files from a windows client and modified it slightly by removing the value for the ISPPhonebook entry. Place this in /etc/opt/cisco-vpnclient/Profiles and then connect with vpnclient connect PCF_FILE.

Left Over .nfs (dot nfs) Files

I recently had a situation where an NFS file system was constantly filling with these strange .nfs files. If a file is removed while a running process still has it open, that file is renamed as .nfs and a long hex string. The symptoms are fairly easy to reproduce:

# touch testfile
# tail -f testfile
From another session:
# rm testfile
# ls -la
-rw-r--r-- 1 root    root    0 Jul 23 13:31 .nfs000000000033468f00000003

To find the offending process run an lsof .nfsxxxxx and kill it, however, because NFS mounts can be spread across several clients, it may take a bit of searching to find the right one. Once the process is terminated the client should automatically clean the file.

I should also point out that in my testing, both the process and the delete operation have to come from the same client. NFS doesn't enforce any file locking which means if a file is deleted from another machine the system doesn't know to rename it first. It is left up to the application to sort this out. Most do nothing about it which means you will get a stale NFS handle message on the source process.