Mini Howto for JMeter, an open source web load testing tool

October 4, 2008

In this post I describe in short how to make your first steps in JMeter. I found JMeter when I wanted to test how stable a new written web application was and searched for a tool that allowed me to record my browsing and rerun it multiple (and parallel) times. JMeter is quite pretty easy to get up and running and it’s quite fun to test and break things on your own server.

As JMeter is a Java application the installation comes basically down to extracting the zip/tar.gz file and start bin/jmeter.sh or bin/jmeter.bat, if you’ve setup your Java correctly.

So now to the actual howto:

Start Jmeter by changing to jakarta-jmeter-2.3.2/bin and executing ./jmeter.sh You’ll see following screen:

First we add a “Thread group” to the “Test Plan” via right click onto “Test Plan” as shown in following screenshot:

After this we add an “HTTP Proxy Server” to the Workbench (as “Non Test Element”) to capture the traffic between your browser and the web site to test. Following screenshot shows how to add the proxy server.

Open the “HTTP Proxy Server” page and change the port if required and set the “Target Controller” to “Test Plan > Thread Group” on the same page.

Now configure your browser to use the Proxy Server (127:0.0.1:8080 in the default settings) and go to bottom of the “HTTP Proxy Server” page and click the “Start” button. Make also sure that you deleted the cache of your browser or even better deactivated it for the test. Otherwise you will not see the full traffic a new visitor would generate.

Now, JMeter will record all the HTTP requests your browser makes, so make sure you have closed all the other tabs you have open, otherwise you will get a mixture of Ad’s and AJAX requests recorded as well. After you did click through the workflow JMeter show test later you click the “Stop” Button and take a first look what JMeter has recorded for you.

Delete any request that you don’t like by right clicking onto the node and selecting “Remove”. Now we’ve recorded everything we need and we wand now to simulate a typical user. For this we want a time delay between the various http requests and the delay should not be fixed. If you want to query the server as fast as possible to you don’t need this step. We add therefore the Gaussian Random Timer as shown in this screenshot:

The last thing we need for a first test run is a Listener which tells us what worked and what not. We use for this “View Results Tree”. This Listener is not good for later use when you want to hammer with multiple threads onto the server.

Now we’re ready for our first run, the default settings are fine (in the Menu: Run > Start).

You should get something like this:

This Listener is good for testing your test setup, as you can look at request and response data. Now it would be the time to add an “Cookie Manager” or “User Parameters”. The First you need if your site requires cookies and the second is handsome when you want different threads to use different user/password combinations to login, as one use can only login once at a time.

After you verified the setup you should disable the “View Results Tree” Listener and choose something like “Aggregate Report”. Change now the settings of “Thread Group” do your likings and hammer your web server.

If you are running a big load test, remember each Listener keeps a copy of the results in memory so you might be better running a Listener > Simple Data Writer instead which writes the results out to a file. You can then read the file in later into any of the reports.

Have fun hammering your web server

ps: Always start with a smaller load, you better off finding and fixing a bug which occurs often under low load, than an obscure bug which occurs only under extreme load.

Scalp: web log file analyzer to detect attacks

September 29, 2008

The tool Scalp written by Romain Gaucher detects attacks onto web applications by analyzing the Apache log files. This python script uses regular expressions from the PHP-IDS-Project to match attacks against PHP web applications. It is able to detect Cross-Site-Scripting (XSS), Cross-Site Request Forgery(CSRF) and SQL-Injection attacks, but as Apache does not save the variables from POST requests it is only possible to detect GET request attacks. Take a look at this example HTML protocol of the script. The program has no problem with some hundred Megabyte big apache log files, but you can also select a specific time period or kind of attacks. To use the analyzer you need to download the python script an the search pattern file.

nf_conntrack and the conntrack program

September 14, 2008

Today I had a problem with my VoIP connection to my provider. The hardware SIP client did not connect for some hours. I had a look at the packets which went over my router into the internet. At the first glance it looked as everything worked right on my side, but the other side did not answer.


15:04:46.131077 IP xxx.xxx.xxx.xxx.5061 > yyy.yyy.yyy.yyy.5060: SIP, length: 532
15:04:47.147701 IP xxx.xxx.xxx.xxx.5061 > yyy.yyy.yyy.yyy.5060: SIP, length: 335
15:04:50.130068 IP xxx.xxx.xxx.xxx.5061 > yyy.yyy.yyy.yyy.5060: SIP, length: 532
15:04:51.147168 IP xxx.xxx.xxx.xxx.5061 > yyy.yyy.yyy.yyy.5060: SIP, length: 335


But at a closer look i realized that the xxx.xxx.xxx.xxx IP address was not my current IP address, given by the DSL provider, but one from an older ppp session with my provider. It was at once clear that there must be a problem with the connection tracking of IPtables, as the SIP client used an internal IP address and gets masqueraded by the router. If you want to know more about IPtables and connection tracking take a look at this.

Anyway I did at a fast cat /proc/net/nf_conntrack | grep 5060 to get all connection tracking entries for SIP. And I found more than one, here is on example.


ipv4 2 udp 17 172 src=10.xxx.xxx.xxx dst=yyy.yyy.yyy.yyy sport=5061 dport=5060 packets=1535636 bytes=802474523 src=yyy.yyy.yyy.yyy dst=xxx.xxx.xxx.xxx sport=5060 dport=5061 packets=284 bytes=114454 [ASSURED] mark=0 secmark=0 use=1


The timeout for this entry is 180 sec and 172 seconds to go, and the SIP client was all the time sending new probes and therefore the connection was never dropped. What can you do in this instance? You can install conntrack. It is a userspace command line program targeted at system administrators. It enables you to view and manage the in-kernel connection tracking state table. If you want to take a look at the manual without installing it (apt-get install conntrack) you can take a look at this webpage which contains the man page. With this program I did delete the entries with the wrong IP address and everything worked again.

I think this program and the knowledge of the connection tracking is important for many of my readers, so I’ve written this post. The current cause to talk about this topic is only one of many, so take a look at it.

Using the browser history to target online customers of selected banks with malware

August 9, 2008

So the first question is: Howto find out what other sides a visitor of your site visted?

You say that’s not possible with the exception of the referrer in the HTTP header and by placing  images/iframes with cookies on some other sites, like google/doubleclick are doing it? Wrong there is an other method which allows you to check the browser history against any list of sites you want to check.

And it is really simple, provide a list with links in a hidden iframe to the browser and a JavaScript. This script checks the style of the links, already visit ones are different than new ones for the browser. For social bookmarking sites you should take a look at following free script, no need to program it by yourself

But maybe you want not only to help your visitors by showing the social bookmarking badge he/she uses, but to get more information on them, e.g. is the visitor a he oder she? You should be able to get that information by the sites the browser has visited, there are ones for likely visited by men and others by women. Check this link out for a test if this site-to-gender formula works for you. (The current version will block your browser for some time).

But now to the more harmful part. You can find out which bank the visitor is using and use this information to do specific attack on the customers of special banks (e.g. the ones for which you’ve a working fake online banking homepage, maybe?). This way an attacker can keep a lower profile as he only tries to attack online banking customers of the banks he wants and not anybody.

Many of such homepages are found by automatic scanning system, but they did not visited the online banking site your want to attack, so you will not show any maleware. This way it is also more unlikely that an attackers site is marked by google as malware infected.

So the question is: Are sites already using this technique to get information about their users?

If you know more about this topic write a comment please!

DFN CERT warns about Linux root kits

August 4, 2008

The CERT of the Germany`s National Research and Education Network (DFN – Deutsches Forschungsnetz) warns about attacks on Linux servers, which hide with a root kit. This root kit hides directories and processes from the administrator. The attack is most likely carried out by stolen SSH keys.

Their experts found the directory /etc/khubd.p2/ on the compromised systems but this directory did not show up with ls -l /etc/. But it was possible to change into that directory. As it is very easy to change the source code of the root kid you should check with following:


$ ls -al /tmp/
total 44
drwxrwxrwt 10 root root 4096 2008-08-04 17:58 .
...


tells you the link count and following counts the entries returned by ls:


ls -al /tmp/ | grep "^d" | wc -l


If the do not match, you should really take a closer look. Of course you should use other directories as well. The second way to find leads on this root kit is to send signals to the hidden processes. If a process id is not in /proc, but responding to signals you should also take a closer look:

#!/bin/bash
for PID in `seq 1 65535`; do
  if kill -0 ${PID} 2>/dev/null
  then
  if ls /proc/*/task/*/cmdline | grep "/${PID}/cmdline" >/dev/null
    then
      true
    else
      CMD=`cat /proc/${PID}/cmdline`
      echo "PID ${PID} versteckt?! cmdline: '${CMD}'"
    fi
  fi
done


On olders systems the task directory is maybe missing, use /proc/*/cmdline in this case. If you find an active root kit, send a mail to cert at dfn-cert.de.

Damn it! Patch your DNS Server!

July 26, 2008

In the last days an exploit has been released, but still many big companies have not patched their recursive DNS server. According to the Austrian CERT which released a report two third of the recursive DNS are still not patched. These are not small companies or system, according to The Register the big US ISPs like AT&T, BT and Time Warner as Bell Canada are not patched. There are even some hints that attacks are already carried out.

So Damn it! Patch your DNS servers, you’re risking the security of your customers!

Even Microsoft released following security warning Increased Threat for DNS Spoofing Vulnerability which tells you to install the patch MS08-037 at once. But what do you think can be even worse?

Apple

They have still not provided a patch for OS-X-Server, that just shows that apple can’t be taken serious in the server world.

And I wondered why Ubuntu did no upgrade of MaraDNS

July 22, 2008

In the last weeks all major DNS vendors and distributions did upgrade their DNS servers, but I was not getting an update for the DNS server which I use: MaraDNS

So I did take a look around and found following blog posting by the MaraDNS guys: MaraDNS is immune to the new cache poisoning attack. Which basically explains that because of DJB (whose DNS server I used before it was removed from Ubuntu Hardy) they use query ID and source port randomization since the first public release. Good that I always search for a secure implantation of a service I need. Why are so many people still using bind that much. It can’t be that every setup needs features which only bind provides. It must be laziness or they don’t care about security.

Lynis – an auditing tool for Linux/Unix

July 19, 2008

The first step to higher security of your system is to assess the current state of the system. Lynis is a small command line tool, licensed under GPL 3, which can help you achieving this. From the authors homepage:

Lynis is an auditing tool for Unix (specialists). It scans the system and available software, to detect security issues. Beside security related information it will also scan for general system information, installed packages and configuration mistakes.

This software aims in assisting automated auditing, software patch management, vulnerability and malware scanning of Unix based systems. It can be run without prior installation, so inclusion on read only storage is no problem (USB stick, cd/dvd).

He also clearly states what Lynis is not:

Not a hardening tool: Lynis does not fix things automatically, it reports only (and makes suggestions).

More to the technicial stuff: The basis of the program are shell scripts which scan the operating system and installed software (e.g. old software) but also stuff like SSL certificates (e.g. expire date). The software checks for accounts without password or wrong file permissions and it takes also a look at your local firewall. It runs under many Linux and Unix versions including Debian and Ubuntu.

“Transparent end-to-end encryption for the Internets” is not the solution

July 13, 2008

Last week the guys behind the popular torrent site The Pirate Bay went public with their project of an opportunistic encryption of all traffic from an computer. The project is called “Transparent end-to-end encryption for the Internets” (IPETEE) but it is not the first time something like this is tried. Due to the pressure by the media industry it has more success chances than the last tries but I think we should target something else.

So what should be our target?

To build an overlay network with allows anonymous, encrypted and censor resident communication for unchanged IP based protocols over the internet. From the user perspective the software creates a virtual network adapter like OpenVPN does and every traffic that goes into that device is send via this overlay network. The overlay network does not send the traffic to the other peer directly (in this case the communication would not be anonymous) but via other peers taking part in that network. The more security you need to more peers it takes, but it will also get slower. This enables the usage of the network by different user groups.

This would be a real solution and not a half one like the idea from The Pirate Bay guys. Ah, before I forget it, following are non-goals.

• This network should not guard against information leak through programs which use this network (e.g. mail client sending real IP address)
• No military security/anonymity is targeted, it should be just enough to have a plausible deniability.
• It is not the goal of this network to provide access through the network to the outside world. What goes into to network should stay there. But this does not mean that any effort is invested to prevent anyone to play proxy. (e.g. http proxy)

What do you think about it?

EU Member States Data Retention Transposition Measures

July 12, 2008

I got my hand on this list which is out of a PDF from the European commission, and I though it maybe also interests others. It shows the status of data retentions laws in the EU member states. So here is the list. (The list is sorted by the native names of the nations and not the English ones I used here, e.g. Deutschland vs Germany)

.