Summer's Over

Well, summer's over and the nights are drawing in.  I've enjoyed spending the holidays with the children, but now they're back at school it's time for me to get back to work.

I'm looking for a new opportunity, either within an hour commute of Poole, or remote working.

I'm a full-stack agile developer working in Java and Ruby with plenty of front-end JavaScript experience and a deep understanding of Linux based deployment platforms including AWS. I believe that dev-ops is critical to smooth delivery and have experience setting up an automated containerised deployment pipeline.

Although I'm happiest as a hands on developer, I've been coaching and managing Extreme Programming teams for the last 7 years.

My ideal job would also utilise some of my non-coding skills such as graphic design, video and sound production, music and art, but I'm enthusiastic about any problem I can get my teeth into!

So if you have something that might be of interest, please message me on Linked-in or email me.

Fight for a Secure Encrypted UK

I’m writing today because I’m deeply concerned about reports that the government is covertly planning to undermine the encryption tools that keep us all safe.

Home Secretary Amber Rudd's leaked plan shows that she wants to force tech companies like WhatsApp to "remove electronic protection" and Internet providers to allow "near real time" access to our private online communications.

The Tories appear to see encryption purely as a way for terrorists to hide from scrutiny, but it also plays an absolutely essential part of everyday internet use.  I use encryption whenever possible to reduce my risk of identity theft which could lead to financial ruin.  It protects my business partners' and clients' trade secrets from corporate espionage.  It keeps my connection to my bank secure and my medical records private.  For the most vulnerable members of our society, encryption is the difference between life and death, safety and abuse.

I urge my representative in parliament Robert Syms MP to fight against putting back-doors into encryption technologies and communication services that don't just allow the government to spy on us, they break encryption entirely.  There is no such thing as a back-door that can't be exploited by hackers; if one is added, it's only a matter of time before someone finds a way to exploit it.

Breaking widely used encryption won't expose terrorists, it will just push them further underground into the "dark web" where their activities will be harder to track.  The only people who will suffer are peaceful civilians.

MythTV not running mythfilldatabase on Arch Linux

I run a MythTV PVR server on Arch Linux and have struggled to get mythfilldatabase running automatically as it's supposed to.

In the mythfilldatabase log file the error message was

(Run) - tv_grab_uk_rt  --capabilities failed or we timed out waiting. You may need to upgrade your xmltv graber(Run) - Grabbing XMLTV data using tv_grab_uk_rt is not supported. You may need to upgrade to the latest version of XMLTV.

• tv_grab_uk_rt is a python script that grabs the the UK Radio Times XMLTV listings
• mythfilldatabase runs the above (expecting it to be on the path)
• mythfilldatabase is run from mythbackend
• mythbackend is run as a systemd service (without the python folder on the path)

Solution was to edit /usr/lib/systemd/system/mythbackend.service to add the following line under the [Service] section:

Environment=HOME=/var/lib/mythtv PATH=/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:$PATH

Which adds the Python path to the environment for mythbackend.  Now tv_grab_uk_rt works fine.

Are We Nearly There Yet: The Agile Struggle

My four year old twins are sat in the back of the car, asking "Are we nearly there yet?"

This would seem a reasonable question, but the answer isn't the one they're looking for - we've only just turned out of the end of our street at the start of a three hour journey. So I encourage them to look out of the window and tell me what they can see and soon they're enjoying spotting ponies in the New Forest.

At work, people sometimes ask "How agile are we?"... essentially "Are we nearly there yet?", but agile is a journey not a destination. Agile is a set of ideas that help you with the steps along the way, not a fixed state to be attained.

Even the project you're working on usually won't come to a deliberate fixed end. There may be a short period of bootstrapping to get to an MVP, followed by a much longer period of iterative improvements. The end is most likely a decommissioning of servers that have become irrelevant; a whimper rather than a bang.

A philosophy is not a way of reaching death as efficiently as possible, it's a way of getting the most value out of the life you live.

Don't be disheartened if your agile practice doesn't seem to be going smoothly. Agility requires a constantly self-doubting critical eye that doesn't accept the status quo and questions everything. If, day to day, you don't have disputes about the right way to do things, then you're probably not working in an agile team. When the process becomes static, you may have a simple life, but you really aren't delivering the best value to the business or yourself.

You will never reach agile nirvana, but you may find enlightenment and a sense of tranquillity along the way, because the way you handle the bumps on the winding road makes them seem flatter and the curves smoother.

Blog Ressurrection

Well, my blog has been down for while now. My chosen hosting company went bust and when I tried to move the Rails application to a new server, I realised that I'd let it get criminally out of date and the data migration scripts just couldn't manage. At the time my wife had just given birth to twins, so the amount of time available for personal Web support was just about zero.

I've moved all the old content to Blogger as I think it will require a little less maintenance on my part. Typo (now known as Publify) was a great tool, but I don't really need the customisability or flexibility that it provides.

Now that I have my platform back, hopefully I'll get back on the blogging horse.

EDS Killing Thunderbird

I’ve finally worked out why Thunderbird email has been almost unusable since upgrading my various Ubuntu machines to Precise Pangolin 12.04.

The symptoms were very sluggish Thunderbird interface response, leading to a segmentation fault crash. The crashes only happened sometimes and didn’t seem to follow a pattern of time from start-up.

The EDS Contact Integration Add-on seems to have been responsible. EDS is the Evolution Data Server and although I don’t use Evolution directly, I have registered my various Google and Twitter accounts, which apparently get pulled in by EDS too.

I have quite a few Google contacts (>700), so I can only assume that the add-on is trying to process them all.

Anyway, the immediate problem is solved by disabling the add-on. I use a google contact sync plug-in anyway, which works fine, so as far as I know I’m not missing out on anything.

Statistical Graphing with R

I have been collecting some metrics from the 8 agile teams that I work with and planned to create some nice visual reports to help identify software development practices that could be improved.

The data was initially in an Excel spreadsheet, so I used its built-in graphing capabilities which had been good enough for previous projects. I quickly found problems that made me want to look for a better solution;

• Difficult to create multiple graphs with aligned x-axes.
• Every time I wanted to use the same graph with different data, I’d have to either replace the data or tweak every graph.

So I tried Gnumeric, which has some nice graphing features, but its lack of pivot tables puts it out of the running. Libre Office Calc was much the same as Excel. I tried some Linux graphical plotting applications, the nicest of which was QTIPlot, which solved the common X-axis problem, but its vector output was poor.

Eventually I looked at statistical computing environments and settled on the R Project, a programming language for statistical analysis and graphing. It solved all the problems I was having with GUI based tools and in the process introduced me to a new way of explore my data.

R has a shell, so you can load some data into a variable and then start playing with it. You can easily filter data, apply matrix transformations and feed data through mapping functions.
Once you have the data in the shape you’re interested in, you can run it through one of the built-in functions, which give you lots of standard graph types, or delve into The Comprehensive R Archive Network (CRAN) which is a massive library of user contributed functions for graphing and analysis.

Once you have settled on the transformations and graphs you want to write out, you can write a script that outputs to various file formats, including PDF, SVG and Postscript. At the end of each iteration, I run my script that takes the latest data as a CSV file and outputs all the graphs I need for my report. When I think of new graphs I’d like to include, I add them to my script. It’s easy, reproducible, massively flexible and oh yes… it’s open source too.

Here’s a quick distribution graph (took maybe five minutes)…

This distribution of story estimates (in man-days) shows that;

• Even numbers are more popular than odd numbers
• 9, 11, 17 and 19 are never chosen, presumably due to rounding up or down to 10 and 20.
• There is a general preference for nice small stories.

All this was created with the following R script;

data <- read.table("data.csv", header=TRUE, sep=",")

small_quotes <- data$Original.Estimate[
  data$Original.Estimate <= 576000 & 
  data$Original.Estimate > 0] / 28800

hist(
  small_quotes, col=rainbow(40,0.5),
  main=paste("Histogram of ", length(small_quotes),
    "Quote Sizes <= 20 units"),
  breaks=20, xlab="Estimate", ylab="Frequency"
)

Have fun setting your data free!