configure; make install

[Big Job ]

I've installed so many open source software packages in my time, it's frustrating when I occasionally come across one that isn't built and running in five minutes.

./configure; make; make install

What's so hard about that? A monkey could do it (OK, an IT journalist would need some training).

Actually, there should be more to it than that. If you simply accept the defaults and whack this stuff in, very quickly you're going to end up with "the Linux dweeb's home PC". This is a system that contains a thick morass of untraceable, unidentifiable and frequently obsolete files in /usr/local (the default installation directory for all those packages). Don't do it! It gives Windows and Mac users ammunition to complain about the "anarchy" of Unix. Plus, you'll never figure out what's been installed and (potentially) what's causing conflicts with your new OS upgrade.

Lazy people should opt for their platform's native packaging system (RPM, pkg, deb, etc.) and find a trusted source of third party pre-built packages if they need something beyond what the vendor supplies. This way, you have some measure of auditing and control over your software set and you don't have to get your hands (or mouth) dirty compiling from source.

Smart people (like you, right?) create a well-maintained software depot, as described by Limoncelli & Hogan. In a nutshell:

• Pick some standard, sane compilation options and use them for everything. Unless you habitually debug third party software, you probably want to turn on some optimisation (-O) and perhaps strip the binaries (LDFLAGS=-s).
• Check the configuration options (./configure --help), don't just accept the defaults.
• Put each package in a dedicated, self-contained directory, identified by package name and release number (e.g. less-358/, gzip-1.3.3/). That way, you know which files belong to which packages. As a bonus, you can install later versions without overwriting existing ones.
• For ease of use, symlink all those files and subdirectories into a central set of directories that can be added to user PATHs, etc. E.g. /tools/bin/, /tools/lib/, /tools/info/ Use an automated tool for this; I'm a big fan of Graft. (For something fancier and more fascist, try Stow).
• Watch out! Many developers make rash assumptions about how their software will be installed. You may have to add options to locate configuration files and dynamic state information in host-local directories (e.g. /var). Worst case, you may have to frig the package. (Smart developers implement command line options to select different config files.) DON'T store writeable files in your depot tree. I'm assuming you're going to replicate or share your depot across multiple hosts for consistency, in which case local file variations will be overwritten. Also, a package upgrade will need the config carrying over from the previous version unless it's held outside the depot.
• Script the command lines used to configure each package and save the scripts somewhere central. You'll have a record of how each package was built and you'll be able to build later releases identically. I use this generic script as a template for most standard GNU-autoconf packages, inserting additional options as required.
• Use some automated assistance in distributing the depot across multiple hosts. It could be as simple as rsync'ing the entire directory tree to each host, or you could use NFS & automount where appropriate (note: never NFS alone).

Performance graphing on Solaris

[Big Job ]

Problem: you need to get an overall view of the performance of your Solaris servers. Something with pretty graphs. That doesn't require hours of configuration, the installation of complex, insecure client-side agents and a significant overhead in data collection and handling.

Solution: nothing. But you can get close...

My first thought was to use SNMP: it's bundled with Solaris, it's lightweight, it can be locked down (read-only access) and it's designed for this sort of thing (pulling system data from remote nodes). My first discovery was that you should forget about SNMP. Sun's SNMP daemon returns zero for most of the crucial MIB entries. NET-SNMP can return almost any data you wish (with the minor overhead of installing it, but then you use CfEngine, don't you?), and it's portable. However, the available freeware SNMP consoles are unwieldy, complex beasts with fairly obscure configuration methods; most of them are targeted at network administrators rather than system admins. The latest beta versions of Cricket (1.0.4+) contain a SystemPerf utility that builds configurations for hosts running NET-SNMP, but it doesn't scale well without some scripting wrapped around it (NB. I have corroborative statements on this).

About a million people on the SAGE members mailing list said, "Try Orca". Orca graphs arbitrary data from files using RRDtool. The orcallator plugin uses the SEtoolkit for Solaris to record OS performance data which Orca can graph using a canned configuration. The setup advice assumes you mount a directory exported from the central Orca host on each remote Orcallator client via NFS. Those managing ecommerce or other secure sites will probably want to steer well clear of NFS. You could therefore either use a secure distributed filesystem like SFS or pull the data at regular intervals with rsync. For the latter, here are some additional instructions:

1. (Rsync and SSH are givens, right?)
2. Install RRDtool and RRD.pm on your monitoring host/web server. (Note: RRD.pm must be built with the version of Perl you will use to run Orca. If you have a locally-built Perl, ensure it appears first in your PATH and set the PERL environment variable to point to the binary before building, otherwise it will pick up the Solaris version. I had problems building the version of rrdtool included with Orca, so I downloaded and installed the latest release separately.)
3. Install Orca on the monitoring host and create the directory tree for its RRD & HTML files (plus an orcallator/ subdirectory). When configuring the Orca build, set var-dir to somewhere local to each host (e.g. /var/orca).
4. Install the SE Toolkit and orcallator.se on all your hosts. You also need S99orcallator and start_orcallator from your installed Orca kit; it's easier just to install the whole thing on each client (you do have a software distribution mechanism?) Start Orcallator ASAP to begin gathering data immediately, even if Orca itself isn't ready yet.
5. Set up transparent (key-based) SSH access from the monitoring host to each remote host, using a non-root user account. We need this for rsync (you can probably restrict it to rsync-only via the SSH config files if you're truly paranoid).
6. Write a small script to rsync the local Orcallator data directory from each host to the main Orcallator data directory (i.e. the Orca subdirectory) on the monitoring host. Run this script via cron at the same interval as your Orca graphing and Orcallator data recording (default is five minutes). Insert "sleep 15;" in the crontab entry to avoid pulling the files while Orcallator is updating them. Example script (updated 2003-11-21, important fix!).
7. Copy orcallator.cfg somewhere sensible (e.g. your main orca directory) and edit as required. You will need to extend late_interval by about 60 seconds to avoid warnings about out of date files. Note that despite the name, this file configures Orca to graph orcallator data files; it doesn't configure Orcallator itself (do that by editing orcallator.se if required).
8. Perform any required web server configuration for the HTML output files (e.g. a URL alias).
9. Start Orca (orca -d orcallator.cfg). In my experience, with a dozen hosts Orca consumes most of a single CPU on an E450 while updating. The web pages can be slow to render due to the number of graph images present. You may wish to run the Orca process with nice to prevent it undermining the general responsiveness of the system.

[Using this method, I receive warning mails from Orca about the data files being out of date during the rollover at midnight, so the timing may still require some fine-tuning. In fact, eventually I set warn_email to the empty string to disable the warnings.]

As a companion or an alternative, you might also look at SARGE, the SAR Graphing Engine. This is a bit outdated (1998) and suffers from lack of maintenance, but it's handy if you've already got SAR running. I made a few local fixes and improvements (IMO) to it; download the patches here.

There is a commercialised version of SARGE available. I haven't tried it; it seems to run and generate graphs locally on each host which are then accessed via a shared network drive (grouch) by a central CGI program.

Orca vs. SARGE

• SARGE doesn't support RRD and only shows the last 30 days of data. Orca keeps short and long range periods.
• SARGE uses remote shell calls to run SAR and gather SAR files on each host; this is a clunky, distributed overhead that might be better handled by rsync'ing all the SAR files to a single place and processing them centrally (like Orca).
• SARGE makes it easy to compare performance across several hosts (in a SARGE "group"); the volume of information and the lack of grouping makes this more difficult with Orca.
• SARGE uses the bundled performance tools; Orca requires third party software.
• Orca can graph arbitrary data, if you work out how to configure it.

My conclusion is that SARGE would be the better tool (on the grounds of simplicity) if it supported RRD and was updated. However, Orca tells you everything you might ever need to know.

The Ultimate Solution:

• Requires no extra client-side software (unlikely unless vendors standardise and improve their performance utilities).
• Uses RRD.
• Gathers remote data via SNMP or similar - no remote shells.
• Can handle arbitrary data sources.
• Does not assume the use of shared network drives, which are a whole other can of worms.
• Scalable, perhaps via the ability to distribute graphing across several hosts or subnets.

If Cricket's config system was overhauled or automated, it would come very close. ;-)

Other bubbles:

• An alternative approach using Rsync in daemon mode

Converting Psion audio files

[Big Job ]

I'm putting this down here because it's taken some non-trivial web searching to find an answer:

I needed to convert some audio files (alarms) from my Psion Series 3a to my spiffy new Series 5mx. PsiWin allegedly does this for you, except it didn't in my case - probably because PsiWin is The Worst PDA Sync Software In Existence. (I haven't tried all the rest but I find it hard to imagine anything less useful, unless Palm HotSync randomly corrupts files.) If you want to know why Psion is no longer a player in the PDA market, look to PsiWin.

Besides, I use Linux. After much fruitless Googling for Psion sound converters, etc. (typical thread: "use PsiWin!"), I had a wander to the SOX home page in case there was a later version that understood the Series 5/EPOC 32 audio format. I had 12.17.3 installed but lo and behold, 12.17.4 can read and write "Psion Record app" files! It assumes they have .prc extensions, although I haven't seen any official recognition of this as standard. But it definitely works; I converted my three WVE files to PRC in three commands and they play fine on the Series 5.

Mini-review of the Series 5mx for any Psion owners who want to upgrade one last time (cheap refurbs available from Clove Technologies and Pinnock Organiser Services):

• It doesn't seem much faster, probably because the extra horsepower is soaked up by the more complex software.
• Physical size is about the same.
• You need to hammer at the keys with a concerted effort to ensure each press is registered.
• Keyboard layout and shortcuts have changed subtly but significantly from the Series 3. The Psion Key has gone (partly replaced by a Function key that works as a kind of second shift key and is required for many of the symbols). CTRL-E exits an app. Comma is next to the space bar. This and the physical response are enough to be a real pain. In lieu of practice, I still much prefer the old S3 rubber mat.
• Navigating To-do lists in the Agenda is both easier (you can jump to the top or bottom of a list) and harder (pages flip over two lists at a time).
• There are new apps such as a dedicated Contact manager. However, there's nothing to convert a database containing addresses to the format this uses.
• I assume it's possible to navigate without the stylus but I haven't looked up the required shortcuts yet. (Using fingers is one shortcut - to a very greasy screen.)
• Display isn't as bad as everyone makes out, certainly not enough to justify all the moaning. I haven't yet needed the backlight in daytime.
• 16Mb and CompactFlash support is nice. Proper batteries are nicer (anyone want to buy a Revo, as-new?).

It's his show

[Big Noise ]

Peter Gabriel, MEN Arena, 18th May 2003

The MEN Arena is filled with a cacophony of creaks and groans, as a large 30+ aged audience attempts gamely to cope with the worst seating this side of a new Virgin train. Playing the MEN demands a lot of a performer, chiefly the ability to keep peoples' attention despite the crippling onset of arthritis in their knees and lower back. Still, the babysitters of Cheadle and Altrincham must have been doing well on Sunday night, to judge by the attendees at Peter Gabriel's first UK show in ten years. Was that the reason so many left early, or was it to seek emergency osteopathy?

Gabriel remains one of the few musicians to my mind that combines an intense artistry and intellect in his craft with output that you would actually choose to hear. This is something that most other highly regarded (by critics) singer-songwriters would do well to learn or, preferably, take as reason enough to retire if they haven't cracked it yet (goodbye, Nick Cave; farewell, Ryan Adams). Recent album "Up" successfully combines the accessibility of his most commercial period ("So") with the experimentation of his better work, without wearing the World Music tag on its sleeve.

Live, Gabriel favours his instincts for challenging the audience. Appearing on stage at 7:30pm sharp to introduce The Blind Boys Of Alabama is challenging to their timekeeping, expectations and patience, mainly the latter when the Blind Boys turn out to be an unoriginal gospel act.

There are few major favourites or hit singles in the first half dozen songs and it isn't until "Digging In The Dirt" that the hot buttons are collectively pressed (mine anyway - I like that "edge of insanity" vibe that Gabriel projects rarely but so well). And when you think he'll end on a high with "Sledgehammer", he carries on until the final, stark duet with Tony Levin. There were no bad songs here, but it's interesting to note how some album highlights fail to work live; "Darkness" has the most intriguing lyrics but here it degenerates into a formless mush apparently designed to frighten off any newcomers, while "Signal To Noise" perforce relies too much on automation and seems perfunctory (indeed, the band leave the stage during its final moments, with little effect on the music). Conversely, "Sky Blue" and "The Barry Williams Show" gain strength from the setting and extra ornamentation.

If we face facts, Gabriel is suffering the same geriatric backlash as many of his contemporaries and will never again marshal popular acclaim and sales as he did around the time of "Sledgehammer". In live terms, this means that a limited budget was carefully spent on simple, effective visuals rather than a full band (or their hairdressers). Hence, the horns on "Sledgehammer" are sampled while Gabriel careers about the stage in a jacket hung with lights - a straightforward trick but impressive when the main lighting is dropped. His frenetic bouncing in the Zorb during "Growing Up" is almost worth the ticket price alone.

I'm not carping, there's absolutely no reason for it. This was a great show with some fantastic music played by a seasoned yet keen band and led by Genius. It stands apart from ... well, who else is there?

What I recall of the setlist (unordered) was:
Here Comes The Flood
Darkness
Red Rain
Secret World
Downside Up
Mercy Street
Digging In The Dirt
The Barry Williams Show
Growing Up
Animal Nation
More Than This
Solsbury Hill
Sledgehammer
Signal To Noise
That Voice Again/In Your Eyes
Father, Son
(Thankfully, we didn't get "The Drop", the final song on "Up" that was singled out for special praise in every review I read, despite its tentative minimalism. This is rather like eating a sumptuous banquet and then complimenting the staff on the clever way they folded the napkins.)

Swan. Statue. Silhouette.

[Big Picture ]

A contre-jour sort of day in Worcester. (I know all these fancy foreign terms.)

Unlucky for some?

[Big Picture ]

The 13th photo I took.

Nature overtakes Industry

[Big Picture ]

Wouldn't it be great if you could display flowers like this in your own home, instead of using a boring old vase?

Silhouetted tree & clouds at Delamere

[Big Picture ]

Probably my best shot from last year.

Airline Network are privacy-invading f**ks

[Big Deal ]

...So we wanted to go on holiday, so I went to Yahoo! Travel (with an extra flourish on the mouse to match the !) and searched the Airline Network site for cheap deals. And, finding one that was suitable, I filled in the form asking for a personal quote, taking care to unclick all the "please continue to pester me with unwanted calls and emails bearing a tone of desperate pleading for the rest of my life" boxes as usual.

And 24 hours later, I still haven't got a quote (which, as you'll see, isn't my sad loss) but I have received holiday-related spam. At least they've been kind/dumb enough to put their own name to it. And I hope that as they've gone to the trouble of providing an unsubscribe option, it actually works. I could formally complain, but what's the use? Airline Network either don't know or don't care what they do with customer data - and presumably by extension, their customers. Whoever they are.

Update, 2003-05-16: An email from Airline Network:

"Due to the high demand of web emails we have recieved, please could you contact with specific details of your holiday req on either 0870 2340728. (sic)"

Professional. Timely. Efficient. Convenient. And grammatically correct. These are all terms you won't find in the following list of words used to describe Airline Network: Shoddy, lazy, rude, lying, illiterate, patronising dipwads.

A misty day on Coniston Water

[Big Picture ]

I thought this shot across Coniston wouldn't amount to much but in the end, (after some severe lateral correction ;-) it was worth the short walk from the car.

Aberystwyth memorial

[Big Picture ]

Another shot of the war memorial in Aberystwyth, taken at sunset.

I'm actually quite embarrassed by the lack of sharpness (caused by a strong wind against a light tripod) that renders this picture far less successful than it ought to be. If the difference between a pro and an amateur is more selective editing of the output, I'm definitely an amateur. This should probably not have been posted; too late now.