The Search for reduced SPAM load – Part 1

July 27, 2009

This is the first part of a series of articles which show the various ways I try to reduce the memory and CPU usage on one of my mail server due spammer. Why a series? The first and obvious ideas didn’t work …… but lets start from the beginning.

One of the mail server I manage is for a small software company which sells products mainly to Windows power users. As the email addresses are in every product/readme and on the homepage the server get a lot of spam. I use courier as MTA and IMAP backend for it. To fight the spam I’m using DNS RBL as first line of defense, followed by greylisting, spam assassin and some other stuff.

The ratio of spam which gets through is quite low but thats not the problem at this point. The problem are the spam waves that hit the server several times a day. Normally the system has under 30 open SMTP connections to/from others server, but within minutes that value raises drastically. For example look these values which show the amount of open SMTP connections at the given moment.


Mon Jul 27 19:12:17 CEST 2009     4
Mon Jul 27 19:12:22 CEST 2009     5
Mon Jul 27 19:12:27 CEST 2009     5
Mon Jul 27 19:12:32 CEST 2009     7
Mon Jul 27 19:12:37 CEST 2009     13
Mon Jul 27 19:12:42 CEST 2009     9
Mon Jul 27 19:12:47 CEST 2009     8
Mon Jul 27 19:12:52 CEST 2009     50
Mon Jul 27 19:12:57 CEST 2009     88
Mon Jul 27 19:13:02 CEST 2009     101
Mon Jul 27 19:13:07 CEST 2009     120
Mon Jul 27 19:13:12 CEST 2009     129
Mon Jul 27 19:13:17 CEST 2009     135
Mon Jul 27 19:13:23 CEST 2009     145
Mon Jul 27 19:13:28 CEST 2009     150
Mon Jul 27 19:13:33 CEST 2009     163
Mon Jul 27 19:13:38 CEST 2009     167
Mon Jul 27 19:13:43 CEST 2009     175
Mon Jul 27 19:13:48 CEST 2009     180
Mon Jul 27 19:13:53 CEST 2009     183
Mon Jul 27 19:13:58 CEST 2009     190
Mon Jul 27 19:14:03 CEST 2009     202
Mon Jul 27 19:14:08 CEST 2009     204
Mon Jul 27 19:14:13 CEST 2009     204
Mon Jul 27 19:14:18 CEST 2009     203
Mon Jul 27 19:14:23 CEST 2009     206
Mon Jul 27 19:14:28 CEST 2009     207
Mon Jul 27 19:14:33 CEST 2009     211
Mon Jul 27 19:14:38 CEST 2009     216
Mon Jul 27 19:14:43 CEST 2009     218
Mon Jul 27 19:14:48 CEST 2009     220
Mon Jul 27 19:14:53 CEST 2009     222
Mon Jul 27 19:14:58 CEST 2009     227
Mon Jul 27 19:15:03 CEST 2009     229
Mon Jul 27 19:15:08 CEST 2009     228
Mon Jul 27 19:15:13 CEST 2009     229
Mon Jul 27 19:15:18 CEST 2009     232
Mon Jul 27 19:15:23 CEST 2009     234
Mon Jul 27 19:15:28 CEST 2009     234
Mon Jul 27 19:15:34 CEST 2009     237
Mon Jul 27 19:15:39 CEST 2009     240
Mon Jul 27 19:15:44 CEST 2009     241
Mon Jul 27 19:15:49 CEST 2009     243
Mon Jul 27 19:15:54 CEST 2009     244
Mon Jul 27 19:15:59 CEST 2009     244
Mon Jul 27 19:16:04 CEST 2009     246
Mon Jul 27 19:16:09 CEST 2009     245
Mon Jul 27 19:16:14 CEST 2009     246
Mon Jul 27 19:16:19 CEST 2009     247
Mon Jul 27 19:16:24 CEST 2009     246
Mon Jul 27 19:16:29 CEST 2009     244
Mon Jul 27 19:16:34 CEST 2009     250
Mon Jul 27 19:16:39 CEST 2009     253
Mon Jul 27 19:16:44 CEST 2009     252
Mon Jul 27 19:16:49 CEST 2009     253
Mon Jul 27 19:16:54 CEST 2009     256
Mon Jul 27 19:16:59 CEST 2009     258
Mon Jul 27 19:17:04 CEST 2009     260
Mon Jul 27 19:17:09 CEST 2009     264
Mon Jul 27 19:17:14 CEST 2009     262
Mon Jul 27 19:17:19 CEST 2009     273
Mon Jul 27 19:17:24 CEST 2009     301
Mon Jul 27 19:17:30 CEST 2009     301
Mon Jul 27 19:17:35 CEST 2009     301
Mon Jul 27 19:17:40 CEST 2009     301


It is important to notice these are the values after some tuning on my side already and running a special iptables filter, which I describe in a later articel. They where worse before. The current limit are 300 active connections which lead to following entries by courier.


Jul 27 19:17:23 mail courieresmtpd: 300 maximum active connections.
Jul 27 19:18:24 mail courieresmtpd: 300 maximum active connections.
Jul 27 19:21:14 mail courieresmtpd: 300 maximum active connections.


At this point my monitoring reports that the SMTP service is not reachable as courier does not handle new connections. After some minutes the wave got past my server and the active connections go down again and the monitoring reports the the SMTP service is up again.

You may ask why I don’t just raise the limit of 300 active connections? Because with 300 connections I need already 1GB of RAM only for SMTP stuff. This is because courier forks for every connection a “small” process which handles it. Almost all of the connecting servers are in a DNS RBL so it should be quite easy to just block them. But it is not that easy.

First Idea:
courier is waiting a log time before it closes a connection after sending the 5xx verdict. So I looked at a way to reduce the time, by setting following values in seconds:


root@mail:~# cat /etc/courier/esmtptimeout
30
root@mail:~# cat /etc/courier/esmtptimeoutconnect
30
root@mail:~# cat /etc/courier/esmtptimeoutdata
180
root@mail:~# cat /etc/courier/esmtptimeouthelo
30


Which are quit on the lower end and they helped a little bit, specially to get down to normal business after the top of the spam wave has past. But the wave still maxed my system out.

Second Idea:

I thought at first why does courier wait for the spammer to send a mail before sending a 5xx as the IP is on a DNS RBL? A special process could just drop the connections after doing asynchron DNS RBL check and sending a 5xx to the other server.

Sam Varshavchik the lead programmer of courier told me following:

Except that dropping a connection at this point has proven not to work, as soon as you run into a real mail server, but a broken one that interprets an immediately disconnected connection as an invitation to reconnect immediately.

So ok not good and he said following about my idea to drop the connection ….

And watch your bandwidth get eaten up by a broken server that’s trying to dump on you a load of spam, using multiple connections which immediatly try to reconnect as soon as you drop them.

The most reliable way to reject unwanted mail is by RCPT TO. This is why even when it’s a blacklisted sender, it doesn’t get rejected until it actually sends an RCPT TO. Which means that you have to fully implement SMTP up to that point.

This means that you’ll wind up with a model of using a monolithic, huge process for all SMTP connections, or a single lightweight process per connection.

So this is the starting point of my search for a solution to fix my problem with the exception of replacing courier as MTA at this point (if there is nothing really better out there) and specially no hardware upgrades. I hope some readers have some ideas (please write comments!). I had already one which invokes iptables, but it is a more complex setup which I’ll write in the next article of this series.

Firefox Addons I use and recommend

July 25, 2009

I thought I share the Firefox plugins I use and which make me use Firefox in the first place. Without these plugins Firefox would be just a browser of many and the webkit browsers render faster on my Kubuntu ;-).  So these Plugins make the difference for me.

• NoScript: Without I feel “unsafe” in the internet. It allows JavaScript, Java and other executable content to run only from trusted domains of your choice. You can activate JS only for a session also, so you’re save again next time.
• Cookie Monster: This plugin allows me to manage my cookies. I can set from which domains I accept which kind. e.g. I accept cookies only for the session from a domain if it is needed.
• Xmarks: I use this plugin to sync my bookmarks between systems and also to have a backup at all times of them. You can also use it do sync/save your stored passworts securely.  You can also use your own server.
• DownloadHelper: You never know when you would like to download a flash movie or something like this onto your PC. This plugin will enable you to do so.
• Yip: If you’re using something like meebo.com for instant messaging you surely would like to get notifications of a new message also outside the tab in your browser, as it mostlikely happens that you’re working with an other program or in an other tab. If so, take a look at Yip, as its supports  supports Fluid and Prism which cover the large majority (100%?) of currently implemented notifications.

First look at Kubuntu 9.10 (Karmic) and KDE 4.3

Yesterday  I installed the Alpha 3 of Karmic which will become Kubuntu 9.10. It ships with KDE 4.3 (currently the RC3) and KMS, which stands for Kernel Mode Setting. And whats this? Kernel mode-setting (KMS) shifts responsibility for selecting and setting up the graphics mode from X.org to the kernel. When X.org is started, it then detects and uses the mode without any further mode changes. This promises to make booting faster, more graphical, and less flickery.

Buts whats the first impression?

KDE 4.3 really looks cool and polished with the new Air theme. Much better than the old one. Sure as widget style KDE 4.3 still uses Oxygen, but with Air it got rid of the black on black stuff. Maybe this is done by the  Kubuntu guys and not by the KDE ones, but I like it. If you know how a dialog/window looked in KDE 4.2 you see everywhere small things that got improved. Sure they’re small things but it makes it a much improved user experience.

Karmic starts even faster than Jaunty did and it was not slow. I even think that booting is as fast now as shutting down now. Please improve this also. 😉

When you boot into Karmic the first time it asks you for your openDesktop username and password, for the plasma widget on your desktop. I went ahead to the openDesktop homepage and created a user to test it.  At this point I know that it works, as I see other people in my area online.  But I’m not really sure what I should do with it. Should this replace my ICQ/GoogleTalk? Maybe someone can enlighten me.

One thing that did not work out of the box was the sound in flash videos, in KDE and mplayer everything worked. But it was simple to solve it. Just open KMix and raise the PCM volume, which was on my system defaulted to zero.

So far my first impressions – basically I really like it and it runs solid – no crashes with the KDE 4.3 RC3 stuff.

I’ve also updated my additional packages install script for karmic and you can download it here.

ps: Take a look at http://kubuntuguide.org/Karmic for some nice ideas and tips around Karmic.

Mini-howto on How to get the onboard UMTS card (Wireless 5530 HSDPA Minicard) of a Dell E6500 running on (K)Ubuntu 9.04 (Jaunty)

June 18, 2009

I got this week my new company notebook, a Dell E6500 with an internal UMTS card. As you know me I did at once a dual boot install on it. And the Linux in this case is Kubuntu 9.04 (Jaunty). The installation of the 64bit version (as the notebook has 4GB of RAM) went smooth. So I put an UMTS SIM card into it and went on a trip, as I thought it is easy to setup on the fly. Guess? I was wrong. Surely I got an /dev/ttyACM0 device and I was able to set the PIN but as soon as I send the ATZ command I got an “ERROR” returned.

What you will also recognize is that the UMTS LED (right to the bluetooth LED) stays dark even after sending the PIN. So we’re missing here something. The solution is AT+CFUN=1 which turns the RF Circuit Power on (and lets the LED light 😉 ). I got it running before I went home from the trip, to be exactly I’m on the trip and writing this howto.

Following is the full solution for the problem.

1. wvdial install: I use wvdial to get into the internet via UMTS. Install the package wvdial like this apt-get install wvdial
2. wvdial config: Put following into /etc/wvdial.conf and change the APN (“A1.net” in my example) ,PIN (1234), username and password. My example is for the Austrian Provider A1 which I use.
   
[Dialer Defaults]
New PPPD = yes
Stupid Mode = 1

[Dialer pin]
Modem = /dev/ttyACM0
Init1 = AT+CPIN="1234"

[Dialer On]
Modem = /dev/ttyACM0
Init1 = AT+CFUN=1

[Dialer Off]
Modem = /dev/ttyACM0
Init1 = AT+CFUN=4

[Dialer a1]
Init1 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init2 = AT+CGDCONT=1,"IP","A1.net"
Modem = /dev/ttyACM0
Baud = 460800
ISDN = 0
Phone = *99***1#
Password = www
Username = [email protected]

3. PIN: Run wvdial pin to unlock the SIM card with the PIN.
4. Power: Run wvdial on to activate the power and if you’re finished call wvdial off to save battery life.
5. get online: Wait some seconds after you switched the power on so the card has time to find a base station to connect to. Now type wvdial a1 (replace a1 with your provider name (it needs to the same one as in the config file). If everthing worked and you’re online you get following:
   
--> Carrier detected. Starting PPP immediately.
--> Starting pppd at Thu Jun 18 21:03:46 2009
--> Pid of pppd: 3993
--> Using interface ppp0
--> local IP address xxx.xxx.xxx.xxx
--> remote IP address xxx.xxx.xxx.xxx
--> primary DNS address xxx.xxx.xxx.xxx
--> secondary DNS address xxx.xxx.xxx.xxx

   If you want disconnect just press CRTL-C.

Howto backup your “dedicated server” to a foreign FTP server

June 11, 2009

In my last post I’ve written a howto on installing Xen and OpenVZ on a dedicated root server at a locally well know server ISP. This post is now about the method I use to backup this server on the ISP provided FTP space. The backup solution I use provides following:

• Full backup and restore of the whole server or single files
• GnuPG encryption of the data on the FTP server
• full and incremental backup
• open source and free
• simple setup and usage

The base of the backup system is duplicity, but I use ftplicity as front-end which makes the interface easier to handle in this special case. As the hardware node of the server is running under CentOS 5, this howto is centered around it, but the basic idea is the same for any distribution.

1. Repository: We need to add the EPEL5 repository by calling
   
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-3.noarch.rpm

   Info: Replace x86_64 with i386 if you’ve a 32bit system.
2. Packages: I did following to update and install the required packages:
   
yum update
yum install duplicity python-GnuPGInterface

3. Bug fixing: There is a small bug in the duplicity package (duplicity-0.5.06-1.el5) which leads to some unlovely error messages. You can and should correct them like this:
   
wget http://savannah.nongnu.org/bugs/download.php?file_id=17304
patch /usr/bin/duplicity duplicity-sys-exit.patch

   If the patching worked you should get a output like this:
   
patching file /usr/bin/duplicity
Hunk #1 succeeded at 589 (offset 2 lines).

4. ftplicity install: There is currently no package for ftplicity, but as it consists basically only of one bash script that doesn’t really matter. Just use following commands
   
mkdir ftplicity
cd ftplicity/
wget http://surfnet.dl.sourceforge.net/sourceforge/ftplicity/ftplicity_1.4.2.1.tgz
tar xzf ftplicity_1.4.2.1.tgz
cp ftplicity_1.4.2.1/ftplicity /usr/local/sbin/

   to install it.
5. ftplicity configuration:Everything is now installed and we only need to configure it. The shell script creates its own default config if called like this: ftplicity [profilename] create. Choose a profile name, as ftplicity allows you to use more than one profile (e.g. to backup different stuff to different ftp servers). Now change into the directory /root/.ftplicity/[profilename].
   You need now to create a gpg key with gpg --gen-key – use the default options and a complicated pass-phrase. Continue with editing the file conf, by at least setting following variables:
   
GPG_KEY= to the ID of the key just generated
GPG_PW= to the pass-phrase you just entered
TARGET= to your backup server, use a subdirectory and not / as you may want to backup a second profile or server later and create that subdirectory
TARGET_PW= to the ftp password
SOURCE='/' # as we backup everything

   Take also a look at the other parameters but you’re not required to change them. As we’re backing up the whole server it is necessary to exclude some directories. To accomplish this you need to create a file exclude in the profile directory which contains at least following lines.
   
/dev
/proc
/sys
/tmp
/var/cache
/var/tmp
/var/run

   As we’re using the server with OpenVZ we also add following lines:
   
/var/spool
/vz/root
/vz/private/*/proc
/vz/private/*/sys
/vz/private/*/dev
/vz/private/*/var/cache/apt/archives
/vz/private/*/var/lib/courier/allfilters
/vz/private/*/var/lib/dcc
/vz/private/*/var/lib/apache2/fastcgi
/vz/private/*/tmp

6. first backup: Type ftplicity profilename backup to make your first (=full) backup. You should not get any exceptions or error message if you’ve configured every correctly.
7. automatic backup As you see at the first backup, ftplicity is verbosely – this is good if an error occurs but I don’t like an email every day if all worked. Therefore I’ve written the python script ftplicity.py, which parses the output of the shell script and only sends (but than the full verbose output) an email (via cron) to me if something went wrong. Copy the script to /usr/local/sbin and set the execute flag.Now you only need to edit the crontab with following command: crontab -e and place some lines like this there.
5 9 * * * /usr/local/sbin/ftplicity.py profilename backup
6 17 1 * * /usr/local/sbin/ftplicity.py profilename full && /usr/local/sbin/ftplicity.py profilename purge --force && /usr/local/sbin/ftplicity.py hetzner purge-full


That’s it – quite easy? Anyway take a look at ftplicity usage which shows you these usage examples:

• create profile ‘humbug’: ftplicity humbug create (now edit the resulting conf file)
• backup ‘humbug’ now: ftplicity humbug backup
• list available backup sets of profile ‘humbug’: ftplicity humbug status
• list and delete obsolete backup archives of ‘humbug’: ftplicity humbug purge --force
• restore latest backup of ‘humbug’ to /mnt/restore: ftplicity humbug restore /mnt/restore
• restore /etc/passwd of ‘humbug’ from 4 days ago to /root/pw: ftplicity humbug fetch etc/passwd /root/pw 4D (see “man duplicity”, section TIME FORMATS)

Howto Install Xen and OpenVZ on a Hetzner server (EQ4) [Update]

June 9, 2009

I just got my new Hetzner server, an EQ4 with 8GB RAM, Core i7-920 Quad-Core, 2x750GB HD. As my old server is running OpenVZ and I’m quite happy with it I wanted to use also OpenVZ for this one. Additionally a XEN could use the Hardware virtualization this CPU provides, if I at some point need a Linux with a special kernel or a Windows for testing it would come handy. To be able to keep the hardware node running as long as possible without upgrade, I chose CentOS 5.3 as distribution.

This Howto describes how I did the setup and stops at the moment I could start installing/migrating VEs or domU instances. Surely a hardware node needs more setup like backup, firewall, … but this is out of scope for this howto.

1. Base Install: Boot the 64Bit rescue system from the Hetzner Robot. Enter installimage and choose Centos 5.3 64Bit minimal. If you’re using a software raid you can keep the settings with the exception of the hostname. 😉
2. Kernel: After the boot into the new system we need an OpenVZ and XEN kernel. To achieve this you need to enter following first:
   
yum install wget
cd /etc/yum.repos.d
wget http://download.openvz.org/openvz.repo
rpm --import http://download.openvz.org/RPM-GPG-Key-OpenVZ


   Now you can check with yum list | grep ovzkernel which kernel to use. You get something like this:

   
ovzkernel.i686 2.6.18-128.1.1.el5.028stab062.3
ovzkernel.x86_64 2.6.18-128.1.1.el5.028stab062.3
ovzkernel PAE.i686 2.6.18-128.1.1.el5.028stab062.3
ovzkernel-PAE-devel.i686 2.6.18-128.1.1.el5.028stab062.3
ovzkernel-devel.i686 2.6.18-128.1.1.el5.028stab062.3
ovzkernel-devel.x86_64 2.6.18-128.1.1.el5.028stab062.3
ovzkernel-ent.i686 2.6.18-128.1.1.el5.028stab062.3
ovzkernel-ent-devel.i686 2.6.18-128.1.1.el5.028stab062.3
ovzkernel-xen.i686 2.6.18-128.1.1.el5.028stab062.3
ovzkernel-xen.x86_64 2.6.18-128.1.1.el5.028stab062.3
ovzkernel-xen-devel.i686 2.6.18-128.1.1.el5.028stab062.3
ovzkernel-xen-devel.x86_64 2.6.18-128.1.1.el5.028stab062.3


   As I’ve a 64 Bit distribution and want the option for XEN I use following command:
   
yum install ovzkernel-xen.x86_64


3. Grub: Verify and tune the grub configuration. As we will update the Kernel every once a while we want a setup which switches back to a working kernel in case of a kernel panic during boot. Compare your grub configuration with this one. The lines with comments are added/changed by me to provide the wished functionality.
default saved ### changed
timeout 5


title CentOS (2.6.18-128.1.1.el5.028stab062.3xen)
root (hd0,1)
kernel /xen.gz-2.6.18-128.1.1.el5.028stab062.3
module /vmlinuz-2.6.18-128.1.1.el5.028stab062.3xen ro root=/dev/md2 vga=0x317 panic=5 ### last option added
module /initrd-2.6.18-128.1.1.el5.028stab062.3xen.img
savedefault 1 ### added


title CentOS Linux (2.6.18-128.el5)
root (hd0,1)
kernel /boot/vmlinuz-2.6.18-128.el5 ro root=/dev/md2 vga=0x317
initrd /boot/initrd-2.6.18-128.el5.img
savedefault ### added


   This setup will boot the first kernel once, so we need to following to reset the counter after every successful boot.

   
echo "savedefault --default=0 --once" | grub --batch


4. Network: Configure /etc/sysctl.conf corrently for OpenVZ. It should look like this.
   
# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled
net.ipv4.ip_forward = 1
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.conf.default.proxy_arp = 0


# Enables source route verification
net.ipv4.conf.all.rp_filter = 1


# Enables the magic-sysrq key
kernel.sysrq = 1


# We do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0


# normally a good idea
net.ipv4.icmp_echo_ignore_broadcasts=1

5. Some Checks: Take a look at /etc/sysconfig/selinux and make sure that SELINUX=disabled is set. (This should be default with the Hetzner minimal image). Now take a look at cat /proc/mdstat to make sure your raid is rebuilt or at least at the beginning so you don’t loose to much rebuilding work. A rebuild under way looks like this:
Personalities : [raid1] [raid0] [raid6] [raid5] [raid4]
md0 : active raid1 sdb1[1] sda1[0]
4200896 blocks [2/2] [UU]


md1 : active raid1 sdb2[1] sda2[0]
2104448 blocks [2/2] [UU]
resync=DELAYED


md2 : active raid1 sdb3[1] sda3[0]
726266432 blocks [2/2] [UU]
[==>..................] resync = 10.4% (75563776/726266432) finish=132.6min speed=81773K/sec

6. Reboot and hope!
7. Kernel Check: Check with uname -a that you got the correct kernel booted. It should be something like this:
   
Linux yourserver 2.6.18-128.1.1.el5.028stab062.3xen #1 SMP Sun May 10 19:25:45 MSD 2009 x86_64 x86_64 x86_64 GNU/Linux

8. OpenVZ: Now we can install and configure the OpenVZ user space tools. With yum install vzctl.x86_64 vzquota.x86_64 you get the management tools install and chkconfig --add vz is needed so that you don’t need to type the next command after every boot by hand: /etc/init.d/vz start. Now you should be able to call vzlist which reports only Container(s) not found which is correct. The OpenVZ install is done now.
9. XEN: At last we install the XEN user space stuff with following command: yum install xen and add xend to the runlevels with chkconfig --add xend. After we launched it by hand this time with /etc/init.d/xend start you can use xm list to get following:
Name ID Mem(MiB) VCPUs State Time(s)
Domain-0 0 7647 8 r----- 202.4


We’re done. Hope this Howto helps you.

Update:

• You should add following line
  
exclude=kernel

  to the file /etc/yum.conf, which keeps the standard kernel from being updated. I don’t recommend uninstalling the standard kernel package but updating is not necessary as we won’t use it normally.

ATI Radeon HD 3650 under Jaunty

June 2, 2009

This entry is more a note to myself than a post for others. I just installed Kubuntu Jaunty on my Desktop. It was not easy. I think it is the first time in years that a Kubuntu install (Desktop CD version) was not abled to get anything than blank screen instead of the install mananger. I’ve a Radeon HD 3650 in this machine. What I did to solve this was to downloaded the alternative CD version and install it. Surely it booted also in a blank screeen but I was able to switch to the console with ALT-CRTL-F1. I installed than the openssh server which I used to follow this howto to install the ATI own drivers. After this all works again. Maybe this post helps also someone else except me.

On the cutting egde with the Ubuntu Graphic Drivers Repository

May 13, 2009

You want to be on the cutting egde with your system but you still want to be able to work with it? So basically you install or upgrade every 6 month your system to the next Ubuntu release. But you want to have the newest graphic card drivers, still you don’t want to compile them by yourself. If so, take a look at this post. It shows how to add the new Ubuntu Graphic Drivers Respository, which is one step further in supporting graphic cards which where released after the distribution release.

Tips for running Kubuntu 9.04 (Jaunty) on an EeePC 901go [update3]

April 23, 2009

It has been a long time since my last post, but with my house building project time is short. But I found time to upgrade my Netbook from Kubuntu 8.10 to Kubuntu 9.04. The reason I was so eager to get the new version up and running are the graphics problems with 8.10 (poor graphics performance and rendering errors). Please take a look at my Post about Kubuntu 8.10 on my EeePC, as most of the stuff is still valid. I’ll write in this post only what I did differently.

• At the time of writing this (23.4.2009) it is necessary to install a custom kernel to activate UXA for your netbook. I used following howto to install the latest testing kernel 2.6.30-rc2 and updated drm and xserver packages. With these changes I don’t have rendering errors and the performace for the KDE 3D effects is ok. A short look at flash videos also shows that they are also ok, but I’ll take a closer look at videos in the next days and at it to this post.
• I did not install a special EeePC kernel so far as with Kubuntu 8.10. At the time of writing there seems to be also no Update to the Array Kernels and packages for Jaunty anyway. [Update] There is now but I’ll stay with the 2.6.30-rc2 for the time being. [/Update]
• I’v upgraded my script for installing additional packages for Kubuntu. Here is the file.
• I installed eee-applet which comes as Jaunty packages, but it didn’t work. [Update] I searched further and installed the Jaunty package from here. You need to install these packages (apt-get install python-gconf python-notify python-pyorbit) before installing the package like this: dpkg -i eee-control_0.9.0_all~jaunty.deb [/Update]

Sofar everything seems to run nicely. Anyway I’ll update this blog post if I find a problem or come up with an solution.

[Update]

• The WLAN stuff does do seem to have the same performance problems (=hangs) as with the Kubuntu 8.10 kernel. I’ll take a look at the way I solved the problem with the old kernel and try to do the same here. I’ll keep you posted.

[Update2]

• The problem with the Wifi is not as bad as with intrepid, as I don’t have packet loss. Maybe it is also a compatibility problem with my access point.
• I did forget to post the MRRT stuff I changed or better not changed according to the howto. Here is it:
  
lspci -vvnn


00:02.0 VGA compatible controller [0300]: Intel Corporation Mobile 945GME Express Integrated Graphics Controller [8086:27ae] (rev 03)
Subsystem: ASUSTeK Computer Inc. Device [1043:830f]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- SERR-

Latency: 0
Interrupt: pin A routed to IRQ 16
Region 0: Memory at f7f00000 (32-bit, non-prefetchable) [size=512K]
Region 1: I/O ports at dc80 [size=8]
Region 2: Memory at d0000000 (32-bit, prefetchable) [size=256M]
Region 3: Memory at f7ec0000 (32-bit, non-prefetchable) [size=256K]
Capabilities:
Kernel modules: intelfb
cat /proc/mtrr
reg00: base=0x000000000 ( 0MB), size= 2048MB, count=1: write-back
reg01: base=0x07f800000 ( 2040MB), size= 8MB, count=1: uncachable
reg02: base=0x0d0000000 ( 3328MB), size= 256MB, count=1: write-combining


  You see that the prefetchable memory is at d0000000 and with an size of 256 MByte. (As Info: 256MB = 0x10000000). Therefore our MTRR line is
  echo "base=0xd0000000 size=0x10000000 type=write-combining" >| /proc/mtrr

  As you see I’ve this line already, so no need for me to add it.

[Update3]

• As it seems the ath5k driver for the WLAN is still not as good as the madwifi driver. I installed it like this. Go do http://snapshots.madwifi-project.org/madwifi-hal-0.10.5.6/ and download the newest version there. In my case I did following:wget http://snapshots.madwifi-project.org/madwifi-hal-0.10.5.6/madwifi-hal-0.10.5.6-r4016-20090429.tar.gz
Now extract it:
$ tar xzf madwifi-hal-0.10.5.6-r4016-20090429.tar.gz
$ cd madwifi-hal-0.10.5.6-r4016-20090429Now we need some packages installed:
  sudo apt-get install build-essential

  And now the actual compiling and installation
  cd scripts
sudo ./madwifi-unload
sudo ./find-madwifi-modules.sh $(uname -r)
cd ..
sudo make
sudo make install


  At last we need to add blacklist ath5k to /etc/modprobe.d/blacklist-eeepc and append ath_pci to /etc/modules. After a reboot you should have a better working WLAN.

• As of now I’ve an idle power consumption under 11 Watt, most of the time at 10.4 Watt. I’ve changed following in addition to the above stuff. I’m using powertop (apt-get install powertop) to take a look what could be improved.:
  • echo 1500 > /proc/sys/vm/dirty_writeback_centisecs
  • Enable USB Suspend
  • disable HAL polling

Howto re-read a partition table on Linux?

January 29, 2009

Today I needed to partition an additional hard disk and format the partitions. I used cfdisk which is installed by default on Ubuntu to have a nice text GUI and not the plain old fdisk. But after changing the partition table and exiting cfdisk you’ll see that the entries in /dev/ are still missing for the new partitions. Sure you can reboot or reconnect hot plug devices like the onces connected via USB. But thats not elegant nor in the best Unix tradition. 😉

I use partprobe for discovering new partitions, just enter it as user with root privileges and the devices will be created and you can use mkfs.[ext3|vfat|..] to format the partitions. If your system is missing it you can easily install it (on Ubuntu/Debian) like this: apt-get install parted. I’ll guess the package is named similar on other distributions.