Scalp: web log file analyzer to detect attacks

September 29, 2008

The tool Scalp written by Romain Gaucher detects attacks onto web applications by analyzing the Apache log files. This python script uses regular expressions from the PHP-IDS-Project to match attacks against PHP web applications. It is able to detect Cross-Site-Scripting (XSS), Cross-Site Request Forgery(CSRF) and SQL-Injection attacks, but as Apache does not save the variables from POST requests it is only possible to detect GET request attacks. Take a look at this example HTML protocol of the script. The program has no problem with some hundred Megabyte big apache log files, but you can also select a specific time period or kind of attacks. To use the analyzer you need to download the python script an the search pattern file.

Need help comparing x with y?

September 28, 2008

Here is something that helps you: WikiVS is the the one stop for up-to-date comparisons. Be it a comparison of MySQL vs PostgreSQL, Lighttpd vs Apache or Qt vs GTK. This website has all to help you base your decision on facts.

What are the benefits of such an site for you? The comparison should be up-to-date and you don’t need to look through long threads (some of them flame wars) discussing that topic. At last you can also contribute to the comparisons.

So it’s the open source / community way of doing something like this and I think thats great!

Which open source PHP image gallery web application?

September 24, 2008

I needed a web image gallery software and as I’ve already some other php stuff on my webserver running I was searching for a PHP based one. There are plenty to try at http://www.opensourcecms.com/ but I think you can only use one of following three:

• Gallery 2
• Zenphoto
• Plogger

I went with Zenphoto as it really easy to install and upgrade and offers not all but the most important features. I’m just missing the feature to rotate images. Gallery 2 has the most features (I think) but it is also much more resource hungry compared to the other two as it is vastly more mature (which in this case pretty much means more plugins/functionality). So just choose one of the three as any of them is a good open source projects to use.

Convert m4b to ogg on Linux

September 23, 2008

I got my hand on some audio books in the m4b format (They are not DRM protected) and I wanted to listen to them on my mp3 player. But the m4b format is a special format for the Ipod and don’t own a Ipod (and I don’t want to own one 😉 )

As I’m a Linux user I decided to write a program which converts them to ogg (which my mp3 player supports). It should be a console program which goes recursively through a directory and its sub directories and converts all mb4 files to ogg. Of course this script does not reinvent the wheel and uses some mighty console programs for the main work. You need to have following stuff installed on your system:

• oggenc (on Ubuntu/Debian you can it install with apt-get install vorbis-tools)
• faad (apt-get install faad)
• python (should be normally already installed, apt-get install python)

Download the m4b2ogg.py script and set the execute permissions. Call m4b2ogg.py -h to get a list of the possible parameters. I hope this script helps also other Linux users – if so drop a line please. Thx.

ps: If you’re searching for converting other media formats into yet an other media formats on linux, take a look at this site.

Update dynDns record periodically

September 20, 2008

I’ve the IP of my dnyDNS record updated via a script in /etc/ppp/ip-up.d/ on my router but sometimes (not often) the ppp connections seems not to be fully up or due to some other problems the update of the IP address does not work. In this case the system is not reachable until the next time I reconnect. I know I can run ddclient as daemon, but I wanted something lighted and easier.

I’ve written a small python script which gets the current IP address of the ppp connection and calls the ddclient to update the IP to the provided one. My script is called by cron every half an hour. This also guards against the problem that I get no disconnect in 3 month (I think it was something in this time frame) and dnyDNS disables my account.

Download the setdyndns.py script. Change the external network interface to your setting in the variable networkInterface.

No Firefox EULA screen on Linux

September 18, 2008

Today the Mozilla Foundation announced that here will be no EULA screen on Linux. Take a look at this post for details. It seams that our protest helped to convince them to remove that idea from the table.

Firefox with EULA-screen in next Ubuntu? (Update)

September 15, 2008

The next Ubuntu release will contain (if it does not change) a Firefox which requires you to accept an EULA. According to Mark Shuttleworth this is a requirement from the Mozilla Corporation keep the name Firefox for the next version of Ubuntu.

I think this is a bad idea and I’m not the only one. I would remove Firefox from Ubuntu and take the re-branded version from Debian. I thought when they went with a new name for Firefox, that they are paranoid, but now it looks like they were right!

I think we as community should step up and that these things must be nipped in the bud. If Firefox starts soon every program gets an EULA-screen. After a fresh install of Ubuntu I need to click > 20 EULA screen then.

Update: 5min try for a banner:

Update:

The Mozilla Cooperation has responded with a blog post. But I still say I don’t want a EULA screen even if it shows a FOSS license. Next time we need to accept an EULA when we do a apt-get install blalba.

nf_conntrack and the conntrack program

September 14, 2008

Today I had a problem with my VoIP connection to my provider. The hardware SIP client did not connect for some hours. I had a look at the packets which went over my router into the internet. At the first glance it looked as everything worked right on my side, but the other side did not answer.


15:04:46.131077 IP xxx.xxx.xxx.xxx.5061 > yyy.yyy.yyy.yyy.5060: SIP, length: 532
15:04:47.147701 IP xxx.xxx.xxx.xxx.5061 > yyy.yyy.yyy.yyy.5060: SIP, length: 335
15:04:50.130068 IP xxx.xxx.xxx.xxx.5061 > yyy.yyy.yyy.yyy.5060: SIP, length: 532
15:04:51.147168 IP xxx.xxx.xxx.xxx.5061 > yyy.yyy.yyy.yyy.5060: SIP, length: 335


But at a closer look i realized that the xxx.xxx.xxx.xxx IP address was not my current IP address, given by the DSL provider, but one from an older ppp session with my provider. It was at once clear that there must be a problem with the connection tracking of IPtables, as the SIP client used an internal IP address and gets masqueraded by the router. If you want to know more about IPtables and connection tracking take a look at this.

Anyway I did at a fast cat /proc/net/nf_conntrack | grep 5060 to get all connection tracking entries for SIP. And I found more than one, here is on example.


ipv4 2 udp 17 172 src=10.xxx.xxx.xxx dst=yyy.yyy.yyy.yyy sport=5061 dport=5060 packets=1535636 bytes=802474523 src=yyy.yyy.yyy.yyy dst=xxx.xxx.xxx.xxx sport=5060 dport=5061 packets=284 bytes=114454 [ASSURED] mark=0 secmark=0 use=1


The timeout for this entry is 180 sec and 172 seconds to go, and the SIP client was all the time sending new probes and therefore the connection was never dropped. What can you do in this instance? You can install conntrack. It is a userspace command line program targeted at system administrators. It enables you to view and manage the in-kernel connection tracking state table. If you want to take a look at the manual without installing it (apt-get install conntrack) you can take a look at this webpage which contains the man page. With this program I did delete the entries with the wrong IP address and everything worked again.

I think this program and the knowledge of the connection tracking is important for many of my readers, so I’ve written this post. The current cause to talk about this topic is only one of many, so take a look at it.

Filter the output of command line programs e.g. run by cron

Ever had to run a program by cron which writes always stuff to stdout and you therefore get every time a mail from cron? You did an > /dev/null to get rid of the messages but errors are not written to stderr by this program but also to stdout?

I have sometimes that problem and therefore I’ve written a small python script which is run by cron and launches itself the real program. It takes any output of that program and filters it by configured regular expression rules. Everything that matches a rules is not reported. As soon as one line does not match, this line is reported and followed by the full output of the program to make the error finding easier.

I often let programs run in verbose mode, as it is filtered anyway by the python script and if an error occurs it is nice to have more information at hand. The python script also forwards any provided parameter to the real program.

There is the python script filterOutput.py. Just download it, set the execute permissions and open it in an file editor. following 3 variables are interesting for you:

• programPath: the path to which the script should change before starting the program
• programCommand: the full path of the program to launch and filter
• regexList: the list of regular expressions rules which are used to filter the output of the called programm

UPC (Austrian ISP) started using Sitefinder Service (Update)

September 3, 2008

The Austrian ISP UPC (Chello, Indo, Telesystem) has activated a system which sends your browser to UPC site if a domain could not be resolved. They say that this helps their less tech-savvy customers but I believe it helps them more. Because they can put some ads on this site. They are not the first to try this. 2003 Versign tried something similar (called Sitefinder) but it was stopped by ICCANN and user protests. But that was not a provider.

The system is an Opt-Out one and not Opt-In. You need to perform 5 clicks, fill out a form and time to wait for a support employee to get it deactivated. You should really Opt-Out as the system can lead to problems if an DNS server is responding too slow and the system tells you you’ve a wrong domain name. The other question is what happens with the data gathered by the search engine on this site, which tries to guess what you meant.

Update:

This site (german) contains all info how you can Opt-Out.