meta/LPAR

Thinking inside the Big Blue box.

RFC 5424 syslogging on RHEL 5.x

Here’s how to get non-broken syslog output on RHEL:

  1. yum install rsyslog
  2. Modify the appropriate line of /etc/sysconfig/rsyslog to say SYSLOGD_OPTIONS="-c3"
  3. Insert the following two lines at the top of /etc/rsyslog.conf:
    $template RFC5424fmt,"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg%\n"
    $ActionFileDefaultTemplate RFC5424fmt

It’s probably then a good idea to disable the old syslog. You can also use rsyslog to replace klogd. Here are some other suggested additions to /etc/rsyslog.conf:

$ModLoad immark # provides --MARK-- message capability
$ModLoad imudp # provides UDP syslog reception
$ModLoad imtcp # provides TCP syslog reception
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog # provides kernel logging support (previously done by rklogd)

Posted by meta on 2010-11-30 | Posted in Linux, System administration | Tagged , , | Comment

Useful RPM one-liner

To get a list of all the software installed on your system, in descending order of bloatedness, with descriptions:

rpm -qa --queryformat="%10{size} %{NAME} - %{SUMMARY}\n" | sort -k1,1nr

Posted by meta on 2010-10-22 | Posted in Linux, System administration | Comment

Jay: A program for removing excess kernels

Ubuntu is pretty easy as far as regular software maintenance goes, thanks to APT. However, there’s one task which (as far as I can tell) isn’t automated.

Whenever there’s a minor update to the Linux kernel, apt-get (or GUI equivalent) will diligently download and install it. Unlike with applications, though, APT doesn’t remove the old version– because you’re running it, and bad things can happen if dynamically loadable kernel modules disappear while the system is running.

So as time goes on, your Ubuntu system collects more and more old kernels. My work laptop had over 700MB of them, and a server at work running a Ubuntu LTS release had over 800MB.

There’s a proposal to build a utility to clean out the excess, but as far as I can tell nobody has actually built it. Instead, there are lots of tutorials on how to clean out kernels manually. It’s a pretty ugly task to do properly, because each kernel is split into three or four packages, depending on your hardware–as well as the generic Linux kernel, there can be proprietary hardware drivers, backported modules, and headers for use in software development. You need to make sure that for each kernel you keep, you keep the matching versions of the headers and modules, and throw away the rest.

After doing this manually enough times, I looked to see if someone had written a script to solve the problem. I couldn’t find one, so I wrote a program to do it in Ruby. It works with plain Ruby 1.8 or 1.9, no extra gems or libraries required; it calls the standard APT utilities in Ubuntu 8.x to query the package database and do the actual work.

It also makes use of a neat Ruby feature: the Comparable mixin. I define a class to represent a version number (e.g. 2.6.15), implement the <=> operator, and import Comparable. I can then use my Version objects just like any other number, and compare them and sort them as easily as if they were ordinary integers.

Download it and give it a try, and let me know if you find any bugs or have any suggestions for improvements.

Update 2011-06: This is still needed on some Ubuntu LTS releases that are supported as of 2011. I just removed 1.1GB of old kernels from one machine. Moved the code to GitHub and added a manual page.

Posted by meta on 2010-04-12 | Posted in Linux, Ruby, System administration | Tagged , , , , | 3 Comments

Adjusting ulimit -n

The first time you run IBM Lotus Domino server on a new Red Hat Enterprise Linux (RHEL) box, you get the following message:

WARNING: the maximum number of file handles (ulimit -n) allowed for Domino is 1024.
See Release Notes and set the allowable maximum to 20000.

This message is less than ideal for a couple of reasons. Firstly, it doesn’t tell you how you’re supposed to change ulimit, and secondly it hints at the wrong way of doing it.

The wrong way is to put ulimit -n 20000 into /etc/profile, or root’s .bashrc, or even the /etc/init.d script for Domino.

The right way is to edit /etc/security/limits.conf and set up limits specifically for domino (or whatever user ID you’ve chosen to run the server under). The two values recommended are set by adding two lines to the file:

domino soft nofile 20000
domino hard nofile 49152

Posted by meta on 2010-03-22 | Posted in Domino, Linux, System administration | Tagged , , , , , | Comment

It’s Time and Energy Wasting Time

Once again, it’s time for every clock in America to be messed with. According to the most optimistic estimates from the Department of Energy, this will provide a savings of 0.03% on the country’s annual electricity consumption. According to actual measurements from a study in Australia, it’ll achieve nothing. According to an NBER study of Indiana’s transition to observing DST, DST increases electricity usage by 1%. So, either DST achieves almost nothing, or it wastes energy, we’re not quite sure.

However, energy usage isn’t the whole story. According to studies the DST shift will cause around 17% more car accidents on Monday, with a smaller increase in the fall because it’s easier to sleep in than to get up early.

DST doesn’t benefit farmers either, in spite of what you might have heard. They lobbied against its introduction. Cattle don’t care what the clocks say, so DST just means the farmer has to get up and milk them at a different clock time.

For IT people, of course, DST is more annoying. It means checking every system that keeps time, to verify that it made the change on the correct date, and adjusting systems that don’t adjust themselves. That’s assuming your government doesn’t decide to change the date of the DST switchover, like the Bush administration decided to, causing hundreds of millions of dollars to be spent patching embedded systems, servers and desktops. (Since Java, Linux and IBM Lotus Domino all use their own independent time zone rule databases, I had to apply three separate fixes to some servers.)

Even non-IT people are starting to catch on to the fact that DST is an annoying waste of time and energy. Can we get rid of this stupid practice, please?

Posted by meta on 2010-03-14 | Posted in System administration | Tagged , , , , | Comment