Three heresies

I encourage people to think for themselves rather than following cargo cults. You might or might not agree with the three heresies I've written about here, but do at least think about them.

Public fields

In Java code, instead of having a public getter and public setter for a field, why not just make the field public? It's much simpler and less code. If you later need a getter and setter for some reason you can always refactor to that (and many IDEs will give you help doing it). There is a comment by Richard Gomes at the end of this previous article on the subject of public fields for data objects. I think public fields make most sense for NOJOs (data objects) (in the rare case where a NOJO is useful - not very often) but maybe it would sometimes make sense for other sorts of classes too?

Note that having public access to a field is not what I'm trying to encourage. The point I'm making is that if you do have public access to a field then it doesn't matter much whether it is by getter/setter or making the field public, so you might as well use a public field as it is simpler. (But please, tell don't ask instead.)

Magic values instead of constants (in build files)

Instead of always factoring out magic values as properties in your build file, consider just using the magic value where it makes sense. For example, maybe instead of "${src}" just use "src" (and get rid of the property) - this was suggested by Jeffrey Fredrick at CITCON Paris 2009. I think there is a lot of merit in this approach. What are the chances that you'd be able to just modify the value of the "src" property and everything would still work? Probably quite low - you'd probably do a text file search for "src" anyway. What are the chances that you'll want to change it anyway? I think it's worth thinking about whether it's better or worse to factor out constants in some cases.

Make the CI build fail fast rather than run all the tests

Rather than running all the tests in your CI build, how about have the build fail as soon as any test fails? That way, a failing build uses less of your build farm's capacity. If your build farm capacity is limited, then this approach may result in getting a passing build sooner (as when the fix is committed there may be a build agent available for running the build with that commit sooner because it's time isn't being taken running a build which will eventually fail anyway). I think it's often more important to know which commit broke the build than which tests failed in order to know both who should fix the build and what caused the build breakage. This approach might not be so good if you have a flickering build (i.e. randomly failing tests) - however, making the build reliable can be achieved and is worthwhile anyway.

More heresies to follow

I have other heresies to write about. Please suggest your own in the comments.

Copyright © 2009 Ivan Moore

Java Enums with constant-specific methods

One of my colleagues introduced me to this handy language feature of Java 1.5 and I wanted to write an article about it because I hadn't seen this before.

Using google reveals that it is already well documented if you RTFM, but I will repeat it here because talking to other Java developers indicates that it isn't as well known as it deserves to be. Here's a slightly reworded extract from the first hit on google or bing for "java enum":

You can declare abstract methods in an enum and override them with a concrete method in each constant. Such methods are known as constant-specific methods. Here is an example using this technique:

public enum Operation {
  PLUS   { double eval(double x, double y) { return x + y; } },
  MINUS  { double eval(double x, double y) { return x - y; } },
  TIMES  { double eval(double x, double y) { return x * y; } },
  DIVIDE { double eval(double x, double y) { return x / y; } };

  // Do arithmetic op represented by this constant
  abstract double eval(double x, double y);
}

Copyright © 2009 Ivan Moore

New version of Jester released

It's been a while since I last updated Jester (a mutation testing tool).

Today I released a new version of Jester - not much changed - but hopefully a bit easier to get it to work, based on my experiences of trying to use Jester when I haven't tried for a while.

It now doesn't read configuration files from the classpath - instead you specify the relevant files on the command line.

Copyright © 2009 Ivan Moore

XpDay London, 7th & 8th December 2009

It's XpDay soon - book your place now. The Keynotes look particularly good this year.

On Monday there's an experience report that looks very interesting "When Agile Might Not Be The Best Solution" - it's good to see this sort of experience report on the programme because there's probably more to learn from it than one which goes something like "we did XP and it worked".

There are lots of other interesting looking sessions too - plus lots of open space sessions which can be excellent.

Copyright © 2009 Ivan Moore

Pair Programming interviews/auditions

I have done a lot of pair programming interviews for my client. I enjoy doing them and I think they are extremely valuable.

Hiring is probably the most important thing to get right, and I think that pair programming interviews (or "auditions" as some like to call them) are usually the best way to interview developers.

Setting up a pair programming interview

For interviewing Java developers, I have a machine set up with a choice of IntelliJ IDEA and Eclipse with an empty workspace. I think it is important to try to make the candidate as comfortable as possible with the development environment so that it isn't a distraction. If you work with Apple Macs, also provide Windows (and maybe Linux) machines for candidates who aren't familiar with Macs because otherwise the difference in IDE shortcuts gets in the way and the idea of a pair programming interview isn't to see how familiar the candidate is with a particular OS.

I allow one hour for these interviews - I have found that is long enough to get a good idea about the suitability of a candidate.

Choosing a problem

One of the most surprising things is how small the problem has to be in order to be achievable in one hour. Developers (including myself) are often very optimistic about how quickly they can program a solution to a simple problem. I would urge you to try out the problem you want to use for a pair programming interview with a trusted colleague, before using it with a candidate, to find out whether it is realistic. To give you an idea of the size of problem you need, I used to use the problem described in this article.

I know of another team at my client who provide a small code base pre-prepared for the interview which includes a bug which the candidate has to find. I think this is a great idea. Choose a problem which is realistic for the sort of code that the candidate will be writing (or the sort of work they will be doing, e.g. debugging legacy code) in the job you are hiring for. There is no point testing for whether a candidate can write WeakHashMap from scratch if you are hiring for a typical enterprise IT project.

What pair programming interviews demonstrate

Pair programming interviews aren't a silver bullet for recruiting developers - they are good at determining some qualities of a candidate but not all. In particular I think they are more valuable in demonstrating programming style and personal interaction rather than problem solving. However, in typical enterprise IT projects, more problems are caused by over complex solutions to simple problems, or by solving the wrong problem, than having problems that are unique and difficult and require a brand new algorithm.

Sometimes it is good to use a problem which has some potential for ambiguities in the details of what is being asked for. This is a very good way to see whether a candidate asks for clarification or just makes assumptions which aren't justified. In enterprise IT systems, making assumptions about the desired behaviour of a system can cause a lot of rework and bugs so I look for candidates who demonstrate that they are thinking through possible ambiguities and who pro-actively ask for clarifications about the problem.

Being realistic

I make it very clear to candidates that I am looking for production quality, simple code rather than for them to show off all the language features they know. The problems that I choose are simple enough that I'm not interested in just getting a solution. I am looking for code that I would be happy working with. My boss, Pippa Newbold, has a very good rule of thumb. "At the very least, don't hire someone who will make the code base worse". It seems obvious and yet at other companies I have sometimes observed panic hiring just to make up the numbers where this rule would have saved lots of rework and bugs.

Giving something back

I try to teach the candidates something new in a pair programming interview where possible. Often this is something like an IDE shortcut but is occasionally a language feature or a "programming in the small" style discussion. I like candidates to get something back for giving up their time to come in for an interview, and a pair programming interview can be very daunting for candidates not used to pair programming.

Does it work?

As far as I can tell, all candidates that I have "passed" using a pair programming interview have turned out to be worth hiring.

However, that doesn't include those candidates that I've "passed" who didn't take the job and I don't know whether any of those candidates that I've "failed" would have been good either.

My suspicion is that it is a technique which is slightly more prone to "failing" good candidates than hiring poor candidates, but that really is just a gut feeling. If anyone knows any studies on pair programming interviews I'd be very interested to hear more.

Where it probably doesn't work

I guess that interviewing people for research or work which requires solving deep and difficult problems probably requires a different approach. Also, for hiring people with limited programming experience who you want to train up it might not be suitable.

Copyright © 2009 Ivan Moore

Cleaner code

I recommend the book "Clean Code" - I completely agree with its philosophy and have written a few articles of my own about "programming in the small".

Mike Hill and I presented a session on "programming in the small" at the "Software Craftsmanship" conference 2009 in London and at QCON London 2009 - and one of or examples was taken from "Clean Code".

Clean Code

One of the examples in "Clean Code" is some code for parsing command line arguments, written in Java by Bob Martin. It is shown as an example of code which has already been made clean. There is an article (separate from the book) by Bob Martin about the code we use for this example. The source code is available from github:

git clone git://github.com/unclebob/javaargs.git

Mike and I chose this code because we wanted to take code which was already quite clean and show how even good code can sometimes be cleaned up even more - we wanted to push the cleanliness as far as possible in the session.

Just looking at the "Args" class only, this method seemed the one most wanting some further work (sorry, formatting is ugly - reformatted to fit blog page width):

private void parseSchemaElement(String element)
                           throws ArgsException {
   char elementId = element.charAt(0);
   String elementTail = element.substring(1);
   validateSchemaElementId(elementId);
   if (elementTail.length() == 0)
       marshalers.put(elementId,
                      new BooleanArgumentMarshaler());
   else if (elementTail.equals("*"))
       marshalers.put(elementId,
                      new StringArgumentMarshaler());
   else if (elementTail.equals("#"))
       marshalers.put(elementId,
                      new IntegerArgumentMarshaler());
   else if (elementTail.equals("##"))
       marshalers.put(elementId,
                      new DoubleArgumentMarshaler());
   else if (elementTail.equals("[*]"))
       marshalers.put(elementId,
                      new StringArrayArgumentMarshaler());
   else
       throw new ArgsException(INVALID_ARGUMENT_FORMAT,
                               elementId, elementTail);
}

So - what could be better? What struck Mike and I was the duplication of "marshalers.put(elementId, new XXX());"

To remove this duplication, first extract a variable called argumentMarshaler of type ArgumentMarshaler in each of the branches, then move the expression "marshalers.put(elementId, argumentMarshaler);" outside of the if statement, and the declaration of the variable "argumentMarshaler" before the if statement. You end up with:

private void parseSchemaElement(String element)
                           throws ArgsException {
   char elementId = element.charAt(0);
   String elementTail = element.substring(1);
   validateSchemaElementId(elementId);
   ArgumentMarshaler argumentMarshaler;
   if (elementTail.length() == 0) {
       argumentMarshaler =
                  new BooleanArgumentMarshaler();
   } else if (elementTail.equals("*")) {
       argumentMarshaler =
                  new StringArgumentMarshaler();
   } else if (elementTail.equals("#")) {
       argumentMarshaler =
                  new IntegerArgumentMarshaler();
   } else if (elementTail.equals("##")) {
       argumentMarshaler =
                  new DoubleArgumentMarshaler();
   } else if (elementTail.equals("[*]")) {
       argumentMarshaler =
                  new StringArrayArgumentMarshaler();
   } else
       throw new ArgsException(INVALID_ARGUMENT_FORMAT,
                               elementId, elementTail);
   marshalers.put(elementId, argumentMarshaler);
}

Now it's clearer that this method is doing too many things - one of those things being to find the relevant marshaler. We can extract a method for finding the marshaler and, having been extracted can improve it further by removing unnecessary structured programmingness, ending up with:

private void parseSchemaElement(String element)
                         throws ArgsException {
   char elementId = element.charAt(0);
   String elementTail = element.substring(1);
   validateSchemaElementId(elementId);
   marshalers.put(elementId,
        findAppropriateArgumentMarshaler(elementId,
                                         elementTail));
}

private ArgumentMarshaler findAppropriateArgumentMarshaler(
                    char elementId, String elementTail)
                                   throws ArgsException {
   if (elementTail.length() == 0) {
       return new BooleanArgumentMarshaler();
   } else if (elementTail.equals("*")) {
       return new StringArgumentMarshaler();
   } else if (elementTail.equals("#")) {
       return new IntegerArgumentMarshaler();
   } else if (elementTail.equals("##")) {
       return new DoubleArgumentMarshaler();
   } else if (elementTail.equals("[*]")) {
       return new StringArrayArgumentMarshaler();
   } else
       throw new ArgsException(INVALID_ARGUMENT_FORMAT,
                               elementId, elementTail);
}

Notice that "elementId" is only used for the exception to throw if an appropriate ArgumentMarshaler cannot be found. Further investigation shows that actually this parameter isn't used in ArgsException for an "INVALID_ARGUMENT_FORMAT" and we can just delete this argument (and hence the "elementId" parameter of "findAppropriateArgumentMarshaler") with no change to the behaviour of the code!

There is more that can be done with the Args class - but that is beyond the scope of this article.

Conclusion

I hope this has shown that even clean code can be made cleaner - without having to do anything too clever or drastic. It's often easier to see such opportunities in other people's code or code you haven't seen for a while - even the best programmers can miss opportunities for making code simpler.

Copyright © 2009 Ivan Moore

Project management lessons - commitment

Have you ever heard (or said) something like "you've got to work [long hours/weekends] because the customer has been promised [some system/feature] by [some date]"?

Commitments can be made but not assigned

People only feel committed to things they have committed to themselves. Telling someone that they have to work long hours/weekends because of a commitment someone else has made can lead to demotivation and resentment. Productivity is very likely to suffer (quality decreasing, more rework due to mistakes, more stupid things done due to rushing or tiredness). Seeing that the work isn't being completed quickly enough, bad managers think that it just requires even more hours to be put in (they try to impose "more commitment" and concentrate on the inputs rather than the outputs). This is the recipe for a "death march" project.

Estimates are not commitments

As a project manager, you have to understand the difference between an estimate and a quote. An estimate is not a commitment. As a project manager, if you make a commitment that something will be done by a certain date then do not rely on being able to just use estimates directly - you need to apply some project management techniques. One technique that I have found works well, as mentioned in my previous article is "extreme planning".

Commitment can work wonders

When someone really feels committed to something, then they will be motivated and focused. Software developers will often work harder and more effectively on their pet open source project than they do at work due to this motivation. This sort of commitment cannot be forced onto someone. There are lots of ways to ensure it doesn't happen (sadly common) and a few things you can do to provide an environment in which it might happen - and hence see massive improvements in productivity and quality as a result.

One factor is the level of technical autonomy. Many large companies have standards which are intended to reduce costs (for various reasons I won't go into). Unfortunately, I think in many cases these standards tend to reduce teams' technical autonomy, which reduces motivation, having a larger negative effect on productivity (hence cost) than the savings made by the standardization. I'm thinking of things like (for example) being told you have to use Wholly Pointless Server when some free open source app server would be a better choice.

Another factor which I think is very important is how directly developers work with their customers. A lot of developers (and certainly the ones I like working with) like to do things which are useful and appreciated by the users of their software. Sometimes the way teams are set up there is a separation between developers and customers - often a business analyst who understands the business and is more available than the real customer(s). This does have benefits when a developer wants a question answered straight away (but the customer isn't available), because otherwise the developer would be blocked or have to context switch onto another area of work. However, I think the best arrangement (not always possible) is when the real customer is available all the time and the developers talk directly to them. The worst situation is when only the project manager or business analyst talks to the customer and approaches the developers with an attitude of demanding features rather than working collaboratively for the best solution.

There are lots of other factors involved in motivation - I might write more in a future article - if I can be bothered. :-)

Bike ride commitment

As an example of commitment - I said I'd do the London to Aachen bike ride and people sponsored me to do it (many thanks), so I felt a commitment to do it. If I hadn't felt any commitment then I wouldn't have done it because, frankly, it wasn't much fun. Here's a quick summary of how it turned out:

Day 1 - Gatwick to Dover/Dunkerque (85 miles cycling) - it wasn't too bad - a bit of drizzle, nice scenery and the cycling was not too tough. (We then got a ferry to Dunkerque - the staff at Norfolk line ferries were great, putting on a fantastic meal, and a tour of the bridge, specially for us).

Day 2 - Dunkerque to Waterloo (139 miles) - we had an early start (breakfast at 6:30am local time, i.e. 5:30am UK time) - in order to do 70 miles before lunch - the roads were totally flat but it was into a headwind the whole way - it was brutal. Some participants skipped the afternoon cycling and got a minivan to the hotel straight after lunch. Immediately before lunch I felt like going in the van too, but after some food felt much better and thought I'd try to keep going. The afternoon turned out to be much better - it was a bit hilly but much less windy. At around 100 miles for the day, 3 other particpants dropped out and got in the support van. At around 120 miles for the day I was knackered - but just about managed to finish the day's ride - going really slowly for the last 20 miles.

Day 3 - Waterloo to Aachen (106 miles) - I was knackered from the day before - so I took it very easy, along with some of the others who had struggled to complete the previous day's ride. There were some cobbled roads, which on a race bike are very scary. Near the end of the day, going up a long climb (which features in the pro cycling "Amstel Gold" race), we cycled through a thunder storm - we had hail, torrential rain (consequently very bad visibility), thunderbolts and lightning (very very frightening).

Copyright © 2009 Ivan Moore

Signs

Unfortunately, I won't be going to XP 2009 - I'm sure it'll be great - enjoy it if you're going.

While training for the London to Aachen bike ride (sponsor me) I regularly cycle past a couple of signs that somehow make me think of the XP conferences:





Copyright © 2009 Ivan Moore

Project management lessons

I signed up for the National Autistic Society's London to Aachen bike ride. When I signed up it was 3 days, with the longest day being the second at about 100 miles (it was noted that this event hadn't been run before so the itinerary might not be 100% accurate).

A few weeks later, I got an update on the itinerary - I was told that the organizer had done a recce (reconnaissance) of the route and the second day would now be 120 miles (last day longer too but not too bad).

Just over a week ago, I got another update on the itinerary; actually the second day will be 137 miles (still not shown on the web page though!). To put that in perspective - that's only 4km less than the longest stage of the Tour-de-France - ouch.

Therefore, of course, you should sponsor me. Also it gives me a good example of the sort of thing that happens with software projects very frequently; so here are some relevant lessons using this as an example.

How to handle estimation

Most people are rubbish at estimating - not only software developers about code, but lots of people about lots of other things too. The three most obvious reactions to that are (a) get better at estimating (b) take bad estimating into account (c) don't estimate (or at least, not quite like most people do). I have found "extreme planning" (see Planning Extreme Programming and Agile Estimating and Planning) to work very well - however it is only really relevant to an incremental project. Quite literally, YMMV!

For a one off event like the bike ride then there's not much I can do for this event (as consumer of the estimates). However, I will treat future event descriptions as being 40% inaccurate if they haven't been run before (i.e. using "yesterday's weather" in "extreme planning" style).

Admit when you have screwed up and don't know the answers

The first change to the itinerary was not entirely unexpected - I knew the route hadn't been done before - my expectations had been set that this was an estimate.

The second change was a nasty surprise. There are two lessons here. When giving an estimate do not give it a level of confidence that it doesn't merit. If the first change to the mileage had said "and we still aren't quite sure" then the second change would not have been so unexpected. The second lesson is that if you have screwed up an estimate, then be extra generous with the re-estimate. If the first changed estimate had been 130 or 140 then 137 wouldn't have seemed so bad. If the second re-estimate had turned out to be 125 I wouldn't have been so bothered about it dropping from 130 or 140.

Mitigating risk

If as a project manager you find out that something is going wrong (e.g. the first re-estimate of the itinerary) then do something to reduce the risk of that going even more wrong. In my case, I increased the amount of training I've been doing (hence I've been writing fewer blog articles because I'm knackered at the weekends!).

Some things you just can't predict - but maybe everything will work out OK anyway

Unfortunately, my bike has developed a fault with the gears which may make it financially non-viable to repair so I may have to get a new bike - but given the new increased mileage that may be worthwhile even if my current bike can be fixed.

I'm disappointed that the event doesn't appear as well prepared as the London to Paris bike ride that I did in 2007 and that I haven't raised as much money (yet) as in 2007, but you can help with that!

Copyright © 2009 Ivan Moore

Save Bletchley Park - the birthplace of computing!

You've heard of Alan Turing? Turing award? Turing test?

How about the Enigma machines and encryption/decryption?

What about the first computers?

What's the connection between these three?

Bletchley Park. It's a great place and historically very significant - but underfunded and in need of maintenance otherwise it'll be lost forever. Please help save Bletchley Park for future generations. Help support it by visiting - it's a great day out.

If you are an Agile coach - go to the Agile Coaches Gathering which is being held at Bletchley Park.

Copyright © 2009 Ivan Moore

Bootable USB sticks for conference presentations

If you have a programming workshop or tutorial to present at a conference, where people bring their own laptops and you want to give the participants programming things to do, then one of the things that always seems to take more time than you expect (during the session) is setting up environments etc before being able to actually do any programming.

With Stuart Ervine, a colleague of mine who is running a session on GWT at SPA2009 (book now!), we have done some experimenting and now have a setup that I think works quite well - so I wanted to share it here in case it is helpful to others presenting similar sessions.

Having presented "Programming In The Small" (PITS) with Mike Hill at "Software Craftsmanship 2009" I can report that it worked quite well in practice (not perfect though - more on the glitches later).

Providing a complete environment:

Stuart and I wanted an identical environment / setup for all of the participants, so time wouldn't be wasted getting everything working on lots of different platforms. Even if you provide one click installers, participants may be understandably reluctant to install some software just for your session. They may even have things installed which conflict with whatever you want to give them. Therefore, we thought it would be better to provide a complete environment - OS and all.

We considered two approaches, providing a virtual image (e.g. VMWare or VirtualBox) with everything installed and configured, or using a bootable USB stick. I attended Lasse Koskela and Markus Hjort's excellent session "Reading Code Without Psychic Powers" where they provided bootable CDs which included the complete environment for several different languages. It worked very well - if rather slow to initially boot up. Having had that experience as a participant, Stuart and I decided to try bootable USB sticks.

Making a bootable USB stick:

First - download the latest Ubuntu (currently 8.10) - and burn it to a CD (good instructions available on the Ubuntu site). Boot your computer off the CD and be patient while Ubuntu starts (it's slow off a CD). These instructions are provided entirely at your risk.

Create two partitions on the USB stick:

• From the System menu, Administration submenu, select "Partition Editor"
• Take a note of the device(s) available.
• Insert a USB stick that you want to use for this (it will get completely overwritten, and must be at least 1Gb - we used 4Gb sticks).
• Right click on the USB stick that has been recognised and "Unmount Volume".
• In the "Partition Editor" (GParted) select menu item GParted -> Refresh Devices
• In the "Partition Editor" (GParted) select the USB stick from the menu GParted -> Devices
• if you proceed from this point with the wrong device selected then you'll trash your machine, so don't get it wrong. Clues about which device is the USB stick are the size of the device and that it wasn't on the list before you plugged it in.
• For each of the partitions on the USB stick (there will usually only be one) right click and "Delete"
• Click on the (now grey) bar which represents the partitions on the USB stick
• Click "New"
• Drag the right hand side of the bar and create a FAT32 partition large enough for the Ubuntu image (about 710Mb should be enough). See image below.
• Click on the (now grey) bar which represents the unpartitioned space on the USB stick
• Click "New"
• Create another partition - can be FAT32 or ext2 - doesn't really matter much. (I found FAT32 to be fine).
• Click "Apply"
• Close the "Partition Editor" (GParted)

Write Ubuntu onto the USB stick:

• From the System menu, Administration submenu, select "Create a USB startup disk"
• Select the device which is the USB stick (as noted above when creating the partitions). If you don't see your device in the list of "USB Disk to use", you might possibly have to unmount the newly created partitions of the USB stick, remove your USB stick and put it back in.
• Click "Make Startup Disk"
• Get a cup of tea - this'll take a few minutes

Copy programs/files to the other partition:

You can copy whatever programs, files etc the participants need to the other partition (the second one you created). For the PITS session this meant Eclipse, Harmony JDK and an Eclipse workspace with the projects for the session. Running from the USB stick won't be very fast but may be acceptable.

To make your software run faster:

If you want the software to run faster, then set it up for participants to copy to the Ubuntu desktop (where it will be running from memory).

If you want to run memory hogging software from the desktop then participants might not have enough RAM (they might run out of memory and I think the live Ubuntu won't try to use the hard disk for swap space). You can set up Ubuntu to use swap space on the USB stick. To create and mount a 1Gb swap file:

Create an empty file the size of the swap space you want to provide (where /media/disk-1 is where the second partition created earlier is mounted) (this may take a few minutes):

dd if=/dev/zero of=/media/disk-1/swapfile.swp bs=1024 count=1024k

Provide an executable script in the second partition for participants to run if they need swap space, containing:

mkswap swapfile.swp
sudo swapon swapfile.swp

Mass production:

So far so good - however, you don't want to have to do this for every USB stick you need to create. Having created one USB stick which has everything set up just right, you can copy its entire contents (including partition table) to other USB sticks using the following (where /dev/sde is the "master" USB stick and /dev/sdf is the one you want to copy to):

dd if=/dev/sde of=/dev/sdf bs=8M count=250

This copies the first 2Gb from USB stick /dev/sde to USB stick /dev/sdf. In order to make this mass copying faster, for the PITS session we didn't use all the space of the USB sticks. We created the second partition using only part of the remaining space. The numbers for "bs" and "count" were arrived at experimentally to minimize the time it took to make the copies. (Multiply them together to work out the amount copied - i.e. using "bs=4M count=500" copies the same amount, but with my USB sticks was slightly slower). The copying may take a few minutes per stick.

Glitches and gotchas:

• Be really careful about mount points (e.g. "/dev/sde") so as not to trash your hard disk.
• Not all machines will boot off a USB. The big glitch is that Apple Macs don't seem to like booting from a USB stick (and even some regular PCs don't). One potential solution for such participants is to provide a bootable Ubuntu live CD and use the USB stick for the programs to run in the live Ubuntu.
• Not all participants were overjoyed at using Ubuntu.
• Participants already using Ubuntu couldn't just run the software included in the second partition because the paths were set up for the "live CD" user.
• Participants need to have some easy way to go from having booted off the USB stick to running the software for the session - we provided a "run.sh" file but in the rush of people getting their machines to boot off the USB (trying to find which key to press to change the boot device) not everyone heard the instructions (which we should have given before handing out the USB sticks).
• Some participants took a while to find the right key to press to boot off a USB device - a lot of machines print the message saying which key to press for such a short time it is difficult to read it. It may be an idea to suggest that participants figure this out before the session.

Other considerations:

It would have been nice to allow participants to keep the USB sticks but they cost money and we're not working for some large consultancy who might want to do this for promotion.

You need one USB stick per machine (we had participants pairing).

We provided other mechanisms for participants to get the materials for the session - it was fairly straightforward for the PITS session as it was pure Java - so we provided the source (and Eclipse and IntelliJ project files) for participants who already have a Java environment. A lot of participants were very organized and had downloaded these materials from the conference web site before the session. We provided the materials in password protected zips and gave participants the password at the session.

And finally:

If you want to see this in action - go to Stuart's GWT session at SPA2009. Ubuntu rocks. GWT rocks. Stuart rocks. Mike rocks. SPA rocks.

Many thanks to Stuart Ervine for help with working this all out and writing it up. Many thanks to Mike Hill for preparing and co-presenting PITS.

Copyright © 2009 Ivan Moore

Two styles

Even simple programs can be written many different ways. This article shows an example of two different styles for a very simple program - I hope the difference is instructive.

The code is shown in Java - but the two styles are applicable to any object-oriented programming language.

The problem to solve

The problem is a slightly reduced version of the parsing part of this robot blocks programming contest problem. To slightly reduce the problem I just use the first two commands and treat the input as a string rather than a file.

Therefore, the problem is to parse input that looks like this:

10
move 4 over 3
move 3 onto 5
quit

i.e. the first line is the number of blocks (used to set up the robot), the last line is "quit" (that's just how the input is - has no effect on the robot) and the lines in between are commands (each line is one command, e.g. "move over" is a single command with two parameters, the source block and the destination block).

No validation is needed - the input will always be well formed - there are no tricks in the question - it is as it looks.

The input will be used to control a robot. You can come up with whatever api you want - you can treat it as if you will be implementing the robot at some point too.

Now your turn

Please have a go at implementing this in Java or a similar language - (parsing input like that shown) - allow a maximum of one hour. If you have any questions about the problem, take the answer as being whatever makes it simplest - there are no tricks to this.

What did you end up with?

I've seen a lot of solutions to this over the last year. That is because I've been doing a lot of interviewing for my client (using the best interview question) and have used this example many times.

What most people end up with

Here's a solution in the most popular style:

package com.oocode;

import static java.lang.Integer.parseInt;

import java.util.ArrayList;
import java.util.List;

import com.oocode.Command.Type;

public class RobotParser {
  private final int numberOfBlocks;
  private final List commands = new ArrayList();

  public RobotParser(String input) {
    String[] lines = input.split("\n");
    numberOfBlocks = parseInt(lines[0]);
    for (int i = 1; i < lines.length - 1; i++) {
      String line = lines[i];
      String[] parts = line.split(" ");
      int sourceBlock = parseInt(parts[1]);
      int destinationBlock = parseInt(parts[3]);
      Type type = parts[2].equals("over") ? 
            Command.Type.MOVE_OVER :
            Command.Type.MOVE_ONTO;
      commands.add(new Command(sourceBlock, 
                               destinationBlock, 
                               type));
    }
  }

  public int getNumberOfBlocks() {
    return numberOfBlocks;
  }

  public List getCommands() {
    return commands;
  }
}

and:

package com.oocode;

public class Command {
  private final int sourceBlock;
  private final int destinationBlock;
  private final Type type;
  
  public Command(int sourceBlock, 
                 int destinationBlock, 
                 Type type) {
    this.sourceBlock = sourceBlock;
    this.destinationBlock = destinationBlock;
    this.type = type;
  }
  
  public enum Type {
    MOVE_OVER, 
    MOVE_ONTO
  }
  
  public int getSourceBlock(){
    return sourceBlock;
  }
  public int getDestinationBlock(){
    return destinationBlock;
  }
  public Type getType() {
    return type;
  }
}

This is a decent representative of this style. I've chosen a shortish variation of the style to keep down the amount of code you have to wade through (although it could have been slightly shorter still if I'd parsed the command type using the enum). A popular variation is to have a Command interface and different implementations for MoveOverCommand and MoveOntoCommand. One thing that is quite nice about the problem is that there is scope in the solutions for showing different programming style and taste.

As far as the "two styles" that I'm talking about, the important feature is that the RobotParser has getters for the numberOfBlocks and the Commands, which are NoJos.

An alternative solution

Hardly anyone implements this alternative style:

package com.oocode;

import static java.lang.Integer.parseInt;

public class RobotParser {
  private final Robot robot;

  public RobotParser(Robot robot) {
    this.robot = robot;
  }

  public void parse(String input) {
    String[] lines = input.split("\n");
    int numberOfBlocks = parseInt(lines[0]);
    robot.setNumberOfBlocks(numberOfBlocks);
    for (int i = 1; i < lines.length - 1; i++) {
      String line = lines[i];
      String[] parts = line.split(" ");
      int sourceBlock = parseInt(parts[1]);
      int destinationBlock = parseInt(parts[3]);
      if(parts[2].equals("over")) {
        robot.moveOver(sourceBlock, destinationBlock);
      } else {
        robot.moveOnto(sourceBlock, destinationBlock);
      }
    }
  }
}

and:

package com.oocode;

public interface Robot {
  void moveOver(int sourceBlock, int destinationBlock);

  void moveOnto(int sourceBlock, int destinationBlock);

  void setNumberOfBlocks(int numberOfBlocks);
}

A variation of this would be to have a RobotFactory which creates a Robot for a given number of blocks, to prevent the possibility of command methods being called before "setNumberOfBlocks". I would expect the implementation to use a BufferedReader if it were reading from a file rather than a string input. As before, I've gone for a short variation of this style.

In this style, the important feature is that there are no getters, and no Command class or classes. You could possibly describe this solution as being an example of "tell don't ask".

And finally ...

Most of the solutions I have seen to the problem have been during interviews that I have conducted. During these interviews, most solutions are the first style and hardly any of the second style.

I don't know whether the popularity of the first style is skewed by it being an interview situation, by the demographics of the people I am interviewing or for some other reason. Most of the interviewees didn't use TDD but did write automated unit tests after the code. However, the majority of those using TDD still produced the first style.

What do you think? What would you do? Which style is "better"? Is that a valid question? If you come up with a different solution that you think is interesting then please put a comment with a link to your blog - please don't paste code in your comments or they will get too long and probably won't format well anyway.

I'm looking forward to seeing something completely different - regex anyone? I don't know - I'm sure there are other styles but the first one shown is, by far, the one I have seen most often when interviewing for Java developers. (And, of course, different languages could produce completely different styles).

And finally - does anyone have good names for these two styles (maybe "getter style" vs "doer style")?

Copyright © 2009 Ivan Moore

Making fields and methods as private as possible - a postmodern Thatcher.

Fields and methods should be declared as private as possible. As I mentioned in an article on the subject "I thought of writing a tool to do this that I was going to call "Thatcher" - it would privatize as much as possible".

I only realized a few days ago that I've already written it (I really am quite dense), and in fact had written it several years before writing that article.

How do you tell if fields and methods can be made private?

I think the most popular ways for people to tell are:

• look for all references to the field or method and see if any are outside the class it is declared in
• change "public" to "private" and see if it fails to compile (e.g. red Xs in eclipse)
  

Unfortunately, both of these approaches have limitations for most enterprise java applications (particularly web applications). These approaches don't work if the field or method in question is used reflectively, e.g. by a template (e.g. FreeMarker, JSP etc). That's the limitation of a static analysis implementation of "Thatcher".

If you are paranoid, how do you tell if fields and methods can be made private?

The only safe way to tell, for a field or method that you want to see if it can be private, is to make it private and then see if the application still works.

How to automate that?

If you have good test coverage, then "the application still works" is automated as "the build (including tests) passes" (e.g. the build that you use on your CI server).

Then you are left with the problem "how do I tell if "public" can be changed to "private" and the build (including tests) still passes?".

Now that sounds familiar to me - it's rather close to what Jester does.

Jester - a mutation testing tool

I wrote Jester as a mutation testing tool. As it says on the web site: "Jester makes some change to your code, runs your tests, and if the tests pass Jester displays a message saying what it changed." The idea is that if you can change your code and the tests still pass then the tests aren't covering the code (other possibilities are that it's a behaviour preserving change, or the code is redundant).

Jester as a postmodern Thatcher

Jester can be used to implement Thatcher by configuring it to mutate "public" to "private" and "protected" to "private" (package visibility is harder!). I think this could be described as a postmodern programming approach - (re)using some code to do something it wasn't designed for.

I hadn't thought of doing this until it occurred to me a few days ago (and now it seems so obvious) because I had imagined that Thatcher would use static analysis and hence be very different (that's my modernist thinking, it would have to use static analysis to be a fast and "proper" implementation and the fact that it wouldn't always be correct would be a fault of the imperfect world).

It might be that someone has proposed this before (in which case, sorry, I don't remember).

Limitations of Thatcher implemented using Jester

It should be noted that using Jester for implementing Thatcher only works if your build (including tests) has sufficient coverage that you are happy that if it passes then that means the application works.

Also note that using Jester for this could take a really long time. If your build takes an hour then it would take many machine days to privatize even a very small amount of code.

An example run of Thatcher implemented using Jester

Here's an example of using the latest version of Jester ("simple jester" also described here) as Thatcher. For this example, I'm using Checkstyle as the code base for which I want to make fields and methods as private as possible. I chose Checkstyle for this example because the build is very fast!

(To try this out for yourself, you will need Java installed with java, javac and ant on the path. These instructions are for Windows. Replace <wherever> as appropriate).

• download checkstyle
• unzip
• cd to <wherever>checkstyle-src-5.0-beta01
• in order to check that everything is OK so far, execute "ant run.tests"

should get:

BUILD SUCCESSFUL
Total time: 33 seconds

(or whatever time)

• download Jester (the simple-jester variety)
• unzip
• cd to <wherever>simple-jester-1.1
• for the step below to work, execute "setcp.bat"
• in order to check that everything is OK so far, execute "test.bat"

should get:

17 mutations survived out of 19 changes. Score = 11
took 0 minutes

(or whatever time)

In order to check that the checkstyle build can be run from the jester directory (i.e. still in <wherever>simple-jester-1.1) (which makes it simpler for running Jester in this example)

• ant -f <wherever>checkstyle-src-5.0-beta01\build.xml run.tests

should get:

BUILD SUCCESSFUL
Total time: 21 seconds

(or whatever time)

Now edit "mutations.cfg" (in <wherever>simple-jester-1.1) and replace contents with:

%public%private
%protected%private

That configures Jester to try mutating "public" to "private" and "protected" to "private". Unfortunately, Jester cannot currently be configured to not mutate literal numbers - so in order to avoid Jester making mutations to literal numbers you have to be a bit cunning.

Create a text file in folder "<wherever>simple-jester-1.1\\jester" called SimpleIntCodeMangler.java with contents:

package jester;

public class SimpleIntCodeMangler implements CodeMangler {
public SimpleIntCodeMangler(ClassSourceCodeChanger sourceCodeSystem) {
}

public boolean makeChangeToClass() throws SourceChangeException {
return false;
}
}

Then (in <wherever>simple-jester-1.1) execute "javac jester\SimpleIntCodeMangler.java".

If "." is on the classpath before simple-jester.jar then the new version of SimpleIntCodeMangler will replace the version in simple-jester.jar, so Jester won't do the literal number mutations.

Now to run Jester/Thatcher (just on the one class "Checker"), execute:

java jester.TestTester "ant.bat -f <wherever>checkstyle-src-5.0-beta01\build.xml run.tests" <wherever>checkstyle-src-5.0-beta01\src\checkstyle\com\puppycrawl\tools\checkstyle\Checker.java

(note that on windows it has to be "ant.bat" and not just "ant")

Now Jester will run for a while, and eventually you get:

4 mutations survived out of 24 changes. Score = 84
took 5 minutes

If you want a simple way to visualize the results, then run "python makeWebView.py" and have a look at "jester.html".

To get the code changed to make methods and fields private which can be made private, then run "python makeAllChangesFiles.py jesterReport.xml" and you get a version of Checker.java called Checker.jester (in the same folder, i.e. "<wherever>checkstyle-src-5.0-beta01\src\checkstyle\com\puppycrawl\tools\checkstyle\") which has all the privatizations where the tests still pass.

The results for Checker.java show that addFileSetCheck, setModuleFactory, setSeverity and setClassloader can be made private and the "ant run.tests" build still passes. Having had a look at the code - it might be that these methods need to be public for something not covered by the tests but are needed for the application to work (because the only tests are unit tests and not end-to-end tests; this is a limitation mentioned earlier).

And finally

If you like that (or even if you don't) then please donate to my 5 countries (300 miles) in 3 days bike ride to raise money for the National Autistic Society.

Copyright © 2009 Ivan Moore

My favourite conference (SPA) is open for registrations. Book before 31 January to catch the early-bird discount rate.

I'm programme co-chair (with Mike Hill) and very happy with the programme we've been able to assemble.

Book now.

Copyright © 2009 Ivan Moore

Charity Day

I am doing a charity bike ride in June to raise money for the National Autistic Society (NAS). One of my sons is autistic and the NAS have been very helpful. I did a London to Paris bike ride for NAS in 2007.

Please donate!

Time is money

For the 2007 bike ride I paid the costs myself so all money donated went to the NAS. For the 2009 bike ride I'd like to do something equivalent but slightly different. I'm offering a day of my time for a donation that more than covers the costs. (The alternative is that I pay using roughly a day of my usual billing - but I'd like a generous charity minded company to donate more than that).

That day could be:

• Refactoring and TDD Training
• continuous integration - e.g. installing an automated CI system (e.g. TeamCity, build-o-matic or whatever)
• Agile consulting (I've been doing "Agile" a long time - papers at XP2000, XP2001, XP2002, OOPSLA, TOOLS. Presentations at XPDay, SPA, ACCU etc)
• other stuff (dunno - email me and we can discuss)
  

and there is more:

• publicity - I'll write a short blog article - anything from just the name of the company and the donation up to an article about what I did for you (you can review/edit before publication)
• "small print" - I'm based in London, England. Anything outside of London might require you to pay expenses too.
• "more small print" - If I get multiple offers then I'll choose whichever I prefer.
  

To discuss this, or for more details, please email ivan at teamoptimization.com

If you aren't in the market for a day of my time then please make a donation of any amount you'd like!

Copyright © 2009 Ivan Moore