Living with Linux — Keeping the home fires burning (for baking Pi’s)

It’s been a while since our dour post on the folly of Congress and the personal impact of political maneuvering.  After a quick flurry to catch up projects before the end of October, which was also the end of a contract that we’ve been on continuously, through three changes in contract management, since the summer of 2001, and the last four years as an independent contractor.  We did expect a follow-on contract from a new prime contractor: this is the way many government service contracts go–the management changes, but the workers get picked up by the new company, so there is continuity of service, and a bit of job security, despite lack of “brand loyalty.”  During the 1990s, I seemed to change jobs about every 18 months on average, moving to a new client and a different employer, so keeping the same client is a bonus.

Unfortunately, the chaos resulting from the shutdown meant that the contract transition did not go smoothly.  We departed on a scheduled vacation trip to Kauai with nothing settled, but the first time since 1989 that we’ve gone on vacation without the prospect of work interrupting. Nevertheless, midway through the week, the contract negotiations got underway in earnest, complete with five-hour timezone shift: when we returned, there was a flurry of activity and then, back to the  grind, continuing on the ongoing projects that were interrupted by both the shutdown and the contract turnover.

While we were gone, the usual Pacific Northwest November storms came early, knocking out power to our network, so there was much to do to get things running again.  Several machines needed a rather lengthy disk maintenance check, and the backup system was full, as usual.  So, we took advantage of 1) the possibiliity of future earnings due to contract renewal and 2) delays in getting the paperwork actually signed and work started, to do some system maintenance and planning, starting with acquiring a bigger backup disk.

Secondly, our office Linux workstation had had a bad update, trying to install some experimental software from a questionable repository, to the extent that the video driver crashed and could not be restored.  Upgrading the system to a newer distribution didn’t help, as upgrades depend on a working configuration.  Now, we’ve been using Ubuntu as our primary desktop workstation environment since 2007, through several major upgrades, one of the longest tenures of distros ever (though we did use Solaris for a decade or more, along with various Linux versions).  Ubuntu has one of the best repositories of useful software and easy updates and add-ons of a lot of things we use from day to day.  But, in the last couple of years, the shift has been to a simpler interface and targeting an audience of mostly web-surfers who use computers for entertainment and communication, but little else.  Consequently, the development and productivity support has suffered.  The new interfaces on personal workstations, like Ubuntu’s Unity and Microsoft’s latest fiasco, Windows 8, have turned desktop computers and laptops into giant smart phones, without the phone (unless you have Skype installed).  One of the other items to suffer was the ability to build system install disks from a DVD download.

So, after failing to restore the system with a newer version of Ubuntu, on which it is getting more and more difficult to configure the older, busy Gnome desktop model that we’ve been used to using for the last decade or more, we decided to reinstall from scratch.  Ubuntu also somehow lost the ability to reliably create install disks, as we tried several times to create a bootable CD, DVD, or memory stick, to no avail.  So, since we primarily use Red Hat or its free cousin, CentOS, as the basis for the workhorse science servers at work and to drive our own virtual machine host, I installed CentOS version 6 on the workstation.  All is well, except CentOS (the Community ENTerprise Operating System) is really intended to be a server or engineering workstation, so it has been a slow process of installing the productivity software to do image editing for photos and movies and build up the other programming tools that are not quite so common, including a raft of custom modules.  Since Red Hat and its spin-off evolve a bit more slowly than the six-month update cycle for Ubuntu, there has been some version regression and some things we’re used to using daily aren’t well-supported anymore as we get closer to the next major release from Red Hat.  Since restoring all my data from backup took most of a day and night, and adding software on the fly as needed has been tedious, I’m a bit reluctant to go back. Besides, I need to integrate a physical desktop system with the cluster of virtual machines I’m building on our big server for an upcoming project, so there we are.

The main development/travel machine, a quad-core laptop with a powerful GPU and lots of RAM, is still running Ubuntu 12.04 (with Gnome grafted on as the desktop manager), but has had its own issues with overheating. So, this morning I opened it up for a general checkup.  Everything seemed to be working in the fan department, but I did get a lot of dust out of the radiator on the liquid cooling system, and the machine has been running a lot cooler today.

Because of the power outage, and promises of more to come as the winter progresses, we’ve been looking at a more robust solution to our network services and incoming gateway: up until now, we’ve been using old desktop machines retired from workstation status and revamping them as firewalls and network information servers, which does extend their useful life, but at the expense of being power hungry and a bit unstable.  But, the proliferation of tiny hobby computers has made the prospect of low-power appliances very doable.  So, we are now in the process of configuring a clutch of Raspberry Pi computers, about the size of a deck of playing cards, into the core network services.  These can run for days on the type of battery packs that keep the big servers up for 10 minutes to give them time to shut down, and, if they do lose power, they are “instant on” when the power comes back.  And, they run either Linux or FreeBSD, so the transition is relatively painless.  The new backup disk is running fine, and the old one will soon be re-purposed for archiving data or holding system images for building virtual machines, or extending the backups even further.

So it goes: the system remains a work in progress, but there does finally seem to be progress.  We even have caught up enough to do some actual billable work, following three really lean months of travel and contract lapses.