Category Archives: System administration

Sometimes old technology is the best option

A while back, I had an interesting problem to solve. We have a set of servers on their own TCP/IP network, with addresses in the 10.0.0.0/8 private IPv4 space; it’s colloquially referred to as “the 10 network”. It’s cut off from the main corporate network; you generally need to use a VPN client to access it. It’s possible to get SSH connections enabled by special request, but only from the corporate network to the private network, only to unprivileged IDs, only between static IP addresses, and definitely not for any non-SSH protocol.

We needed to be able to send e-mail from the private network, via the corporate e-mail servers. I could set up an SMTP server on the 10 network easily enough, but there was no way it could initiate a connection to any other SMTP server. We couldn’t even initiate an SMTP connection inbound to it.

Colleagues suggested schemes involving POP3 or IMAP to pull down mail and re-send it. There was discussion whether we would be allowed to use a scripted SSH connection from a system running the VPN software, to trigger a temporary port tunnel and then prod the MTA to attempt delivery. However, I had a better idea, because I remembered how e-mail had been delivered in the 1970s through to the early 1990s, when all we had was dial-up connections.

The software in question is called UUCP, the Unix to Unix Copy Protocol. The great thing about UUCP is that all it needs is a way to send and receive bytes via a terminal-type connection. In fact, it doesn’t even need that; back when UUCP was in common use, there were quite a few networks or modems that only supported 7-bit data, so UUCP can escape data to make it work over 7-bit links too. UUCP also doesn’t need the connection to be reliable, or even there all the time—it can be triggered to perform data transfers in batch mode, then shut down.

I vaguely remembered that the free software Taylor UUCP had been given some sort of SSH support. Investigation revealed that protocol t was aimed at TCP/IP connections, where you can assume that the bytes you send will be received accurately at the other end. (Unless, of course, the entire connection fails for some reason, in which case UUCP will detect the problem and retry later.)

Taylor UUCP was still available prepackaged for RHEL, and Postfix still had all the hooks for UUCP support available. I set up the two servers, one on the corporate network and one on the 10 network. The corporate machine had been set up to allow SSH (and only SSH) connections to the 10 network. I set things up so that UUCP ran chrooted under a restricted ID, and I was also able to set the 10 network server’s UUCP user to have uucico as login shell—so even if the ID is compromised, it can’t be used to attack either server.

So now Postfix accepts mail from machines on the isolated network, and passes it to UUCP. UUCP turns the mail into UUCP jobs, and queues them. A few times an hour, a cron job on the the internal network machine starts up uucico, which calls SSH to open a connection to the uucico on the isolated server. They then exchange all the pending UUCP jobs—bidirectionally—and disconnect. Finally, UUCP on both servers processes the batches it just received, and passes the e-mail on to Postfix. And all without even any port forwarding.

It’s worth mentioning that you could go a step further, and not even allow the SSH connection, if you ran a serial cable between the two servers. IBM Notes still supports serial connections too, allowing secure e-mail between machines that can’t route TCP/IP packets.

So if you ever find yourself in a similar bind because of firewall issues or corporate security policies, remember that we used to transfer files and send e-mail before there were even TCP/IP networks to firewall. There are tutorials for how to set up UUCP over SSH, and unlike any port forwarding kludge you’re likely to come up with, UUCP is time-tested software. It may be a little arcane, but it works.

DB2 10.1 db2fmcd general protection fault

The problem:

You want DB2 to start up automatically at boot, so you use the official method and enable the fault monitoring system and then use db2iauto -on instancename to enable autostart.

On rebooting, DB2 isn’t started. You check /var/log/syslog and see:

Jun 13 15:37:05 servername kernel: [ 67.324105] db2fmcd[1357]
general protection ip:7fa9c68306c4 sp:7fff86c43150 error:0 in
libc-2.15.so[7fa9c67c2000+1b5000]

or something similar. Yes, ironically enough, the fault monitor has caused a general protection fault.

The solution:

The DB2 10.1 installer seems to attempt to enable DB2 Administration Server (DAS), which was used for the old DB2 Control Center that is no longer supported in DB2 V10. DAS causes the crash. To disable it:

/opt/ibm/db2/V10.1/bin/dasdrop system

Then I rebooted, and after a minute or so the fault monitor successfully started DB2.

Solution found on the developerWorks forums, summarized and reposted here for added visibility to search.

Seems to affect CentOS, Ubuntu Server and VMware DB2 V10.1 instances, on Intel i7 x86-64. Affects DB2 V10.1 Enterprise Server, DB2 Express-C, and probably other license variants. Does affect clean installs.

Network Manager failing to recognize ethernet

Problem description:

  • Network Manager refuses to start your Ethernet connection using the GUI, with the connection showing up as “Unmanaged”.

  • The command-line interface to Network Manager reports an error “Error: Connection activation failed: Device not managed by NetworkManager or unavailable”.

  • However, ifup/ifdown successfully starts and stops your Ethernet connection.

Solution:

  • Edit /etc/network/interfaces. Comment out any lines referring to eth* devices.

  • Restart Network Manager via /etc/init.d/network-manager restart or upstart.

Apparent cause:

Some VPN software helpfully added the eth0 lines to your list of interfaces handled by the older network scripts. When Network Manager sees that, it refuses to manage eth0 itself.