22 August 2005

Relocation aggravation

[Big Picture ]

I've come to the conclusion that I'm definitely almost probably certainly going to allow my Nikonians subscription to lapse early next year. That will also mean the loss of my main online photo gallery. If you're worried that this will mean the disappearance of all those classic images (well, someone might care), you'd be right to worry because I have no intention of laboriously re-uploading them elsewhere. I have enough trouble keeping up with the new images I'm taking, without worrying overmuch about the old ones. If I get time, I might transfer some of the ongoing projects (Anglesey Barracks - which I still hope to revisit one day; Poppy; Abandoned tracks).

I'm still looking for a suitable replacement gallery site. These are the ones I've tried or considered:

Nikonians

Obviously this one's a non-starter. The software (PhotoPost) is great, particularly in the way it displays albums and images and the bulk upload features (either multiple file requesters or accepts ZIP archives). However, the whole raison d'etre insists that only images made with a Nikon camera are allowed, which is increasingly an arbitrary irrelevance when I am equally likely to pick up my Holga, Powershot or Yashicamat and don't particularly care about equipment brands. Also, while it doesn't directly affect my use of the site, there appears to be little place for my kind of photography at Nikonians: it doesn't fit easily into any of the genre-specific forums and a request for like-minded correspondents after I won the October 2004 contest produced zero response. If you like wildlife, sports or sunsets though (and I'm not saying there's anything wrong with that, although none of them appeal to me personally), you'll fit right in. And the B&W contests are now groaning with cute child portraits - as am I.

PhotographyBlog

I got a PB account to host my non-Nikon and casual shots (as previously noted, now a rather arbitrary and pointless distinction). At £5 a year for 50Mb of space, it's a bargain. Unfortunately, it would take you a long time to fill that space because there are no bulk upload facilities (uploading via a URL is currently broken), even though it ostensibly uses the same PhotoPost software as Nikonians. Images must also be limited to 600 pixels maximum on the longest side, which is normally fine but occasionally I might want to put something up at full size (e.g. for print download). It doesn't appear possible to display all your photos together if you organise them into albums, which makes general review quite tedious. However, it would probably meet most of my needs if the bulk upload issue were addressed. (I've even encountered a fellow soul who liked my shots and was posting work in a similar vein himself.)

Flickr

New! Hip! All the cool kids are doing it! It's a whole new paradigm for picture-sharing! Except I've searched a long time to find great images on Flickr, without much success. It has a lot of "joe bloggs" appeal, which is very democratic but means that you'll wade through a metric tonne of camphone shots showing mates mucking about in the pub before you find any real photography (Petteri Sulonen seems to have had more success than me). The ability to tag images with arbitrary categories and form groups is useful if you're searching for particular subjects, and the "RSS feeds everywhere" philosophy puts it streets ahead of most gallery sites (i.e. in the 21st century); for instance, I'm subscribed to feeds for photos of Aber, Manchester and disused railways (unfortunately, there isn't yet a group for "shallow DOF and low light b&w pics").

The other thing that scares me about Flickr is the unlimited storage space (the limits apply to bandwidth usage instead); that's a lot of crap to build up and lose if you ever abandon it.

Fotopic

A homegrown UK cottage industry success, Fotopic allows people to order prints from your pictures (if you let them). Or mugs. Or mousemats (hmm, Poppy on a mousemat - the ultimate slur to her feline dignity). (Of course, for adequate quality you need to upload full size images, which takes forever.) It has a great variety of upload and organisation methods. And even a free account gives you 250Mb of space.

This is very tempting, particularly with the option to upgrade to a pro account and make some money on print sales, if people ever start buying prints online in quantities to make it worthwhile.

Usefilm

Usefilm is my spiritual home. You can search images by particular types, genres and/or locations (for example, black & white still lifes from the UK only - a simple request but many gallery sites make no distinction even between colour and B&W, rendering such a query impossible), and the top-rated ones generally represent the kind of work that I either admire or would like to be creating. In this sense, the content is already targetted at me rather than requiring a lot of fruitless searching. It's possible to build up "karma" points by offering critiques and comments on unrated images, not to mention reciprocated comments on your own work (although the veterans are already moaning about the increasing amount of mutual backslapping that this encourages in lieu of real criticism), and an active editorial team highlights the best posts on the home page regularly. The software behind the site seems a bit old school. No bulk uploads, but then even donors are limited to three a day - the emphasis here is on carefully crafted individual shots, and at least the limit stops people uploading their entire holiday album. The forums seem a bit underutilised. And as yet, I haven't had sufficient time to work on my "karma" and thus gain some useful critiques.

photo.net

The grandaddy of photo communities. Being a sensitive and reserved flower, I've always shied away from the brand of robust, allegedly impersonal criticism frequently offered on pn, not to mention the regular trolls, comment abusers and revenge raters that draw numerous whines in the forums. However, the ability to organise images into folders and yet also combine them arbitrarily into "presentations" is a great idea that sadly almost no one else appears to have copied.

That apart, again the site software is looking somewhat long in the tooth. While I have some images at pn, I've never subscribed and I've stopped maintaining them since they changed the free quota (which I'm now well over - deletion could come at any time).

A photoblog

The DIY option. Hey, everyone else not using Flickr has got one! This has a lot of merit - complete control and ownership of the content, freedom to move it around, automatic RSS feeds, etc. It does leave you isolated from any community (even informal photoblogger groups are essentially collections of unrelated individuals working alone and all to different goals). And, while not an overriding concern, there's no way I'd be posting an image a day. Maybe not even one a month sometimes. I'm still not sure a date-organised stream of images suits my intentions, although a steadily growing archive of work to look back on and review progress has its appeal, if only to me. And there are already too damn many photoblogs on the net, which means that people are less likely to search for or stumble across site dedicated to individuals.

Custom, self-hosted gallery

Maybe for a few images that I really want to highlight, particularly if they form a series (i.e. effectively an online exhibition). Otherwise too much maintenance overhead, and not right for some of the experimental rubbish miscellanea I like to try out.

It appears that no one site meets my needs. At the moment, I'm looking at a combination of PB for general uploads, Fotopic for anything I want to offer as prints and Usefilm for the serious stuff when I've weaned myself off Nikonians.

If anyone is thinking of setting up a new gallery site, or has other ideas for suitable sites, my ideal would be:

  • Bulk uploads. Any one-at-a-time system is taking the piss. Essential when I want to do up to a dozen new images at once, or even import a complete gallery from elsewhere.
  • Ability to create different views of the image catalogue; for example, by album, subject or custom "presentation".
  • RSS feeds with thumbnails for each gallery view, including each member and those like "all new images by everyone"; ideal for sampling current activity.
  • Costs no more than £15 per 50Mb per year.
  • Bulk export functionality; I want to be able to get the images out again without having to search my hard drive for the originals.
  • Ability to bookmark other posters and see their latest images, alá Flickr.
  • A focus on photography as a pursuit rather than people's family snapshots, which just add more crud to trawl through. And preferably, categories that go beyond merely "wildlife", "landscape", "portrait", "macro", etc.

Actually, if we could just mash Flickr, Usefilm and PB together...

Posted by Ade at 03:20 PM | Reply

5 August 2005

Automated software update with Cfengine

[Big Job ]

Here's a Cfengine module written in Perl called module:tools, that creates a Cfengine policy file for updating software packages. If you've been wondering how to do this, it may point you in the right direction. Read on for the gory details and the necessary glue.

This module assumes that you've adopted a sensible system for the installation of third party open source utilities, something like the software depot described by Limoncelli and Hogan. I.e. Each package in its own self-contained, versioned directory, with the components symlinked into common directories for PATHs, etc. So OpenSSH might be in /tools/openssh-4.0p1, which contains bin|lib|man and so forth, and the contents of those directories are symlinked under /tools/bin, /tools/man, etc.

The module takes a colon-separated list of package directories as an argument (to work around argument limits in Cfengine), and generates a Cfengine rules file that can be imported into your existing policy. The rules specify how to update the package files from a master server and create the necessary symlinks. There are also rules to simply check that the listed packages are installed and generate warnings if any are not found. Define the Check and/or Update classes to decide what actions are taken.

Here are the Cfengine rules to incorporate the module and rule file:

control:
    AllowRedefinitionOf = ( tools )
    workdir = ( /var/cfengine )
    moduledirectory = ( $(workdir)/modules )
    pkginstall = ( $(workdir)/inputs/cf.pkgsinstall )
    depot = ( /tools )    # location for all s/w
    mainserver = ( fileserver )    # master software server

classes:
    PolicyServer = ( name of Cfengine master policy host )

    Hr00::
        Check = ( any )    # check packages on all hosts daily
    Sunday.Hr00::
        Update = ( hosts to update weekly )

    !PolicyServer::
        pkgcfexists = ( IsPlain(${pkginstall}) )    # exclude policy server

import:
    (Check|Update).pkgcfexists::
        $(pkginstall)

control:
    # define package lists for each group of hosts
    any::
        tools = ( "openssh-4.0p1:openssl-0.9.7e:grep-2.4.2" )    # common
    internal::
        tools = ( "$(tools):samba-3.4" )
    development::
        tools = ( "$(tools):gcc-3.4:make-3.79" )

    # create toolsinstall rules
    (Check|Update).!PolicyServer::
        actionsequence = ( "module:tools $(tools)" )

Hints

  • The generated rules assume the use of Graft to manage the symbolic links. You can replace this with another system, such as GNU Stow, or even some suitable Cfengine link actions.
  • If Cfengine, Perl and Graft are among the packages managed by these rules, then clearly you will have a bootstrap issue to resolve!
  • Edit the module, enable $DEBUG and run it manually from the command line with a short list of packages to see the resulting rules file (cf.test):
    $ env CFINPUTS=. ./module:tools foo-1.3:bar-2.1.10:snarf-6.4a
    
  • Edit the text of the rules in the module (added to the $output string) to adapt the generated rules for your own needs.
  • Begin by running the module with one or two non-essential packages against a single test host. You could also add action=warn to the generated rules to see what would occur without any changes being made. It is extremely important to test this module carefully before widespread deployment. As usual, if it breaks you get to keep the pieces (but please tell me about it).
  • Note that the pkginstall rules file must not be created on the policy server itself, otherwise it will overwrite the locally generated versions on each Cfengine client during the policy update. This file should be unique to each host.
  • You can create package-specific rule files under a tools/ subdirectory of your Cfengine inputs directory. The module automatically checks for these and adds the appropriate import line to the rules if found. E.g. For restarting daemons.

References

The Practice of System and Network Administration, Thomas A. Limoncelli and Christine Hogan.

This module and technique was based on work by Luke Kanies, which is gratefully acknowledged.

Posted by Ade at 03:58 PM | Reply