Putting the tea into team

Do what works for you

2011-01-24T14:08:00.000-08:00

I've received literally no emails complaining that my "do the right thing" methodology is too prescriptive, so I've come up with a new methodology called "do what works for you" which I hope will work for those people who find the "do the right thing" methodology too prescriptive.

What you do in this methodology is whatever works for you.

Copyright © 2011 Ivan Moore

SPA 2011

2011-01-16T03:04:00.000-08:00

My favourite conference (SPA) is open for registrations and session proposals.

This year I'm honoured to be conference co-chair with Mike Hill. The programme will be organised by Willem Van Den Ende and Rob Bowley.

Please book your place, or propose a session (do that real soon - you'll have some time to improve your session once it is submitted).

Copyright © 2011 Ivan Moore

The plural of anecdote is not data

2010-09-18T01:35:00.000-07:00

I've just finished reading "Bad Science" and it made me think of how little science has been done about software development.

The only two areas where there has been some research that I can think of (off the top of my head - without doing any research about research) (and that seems very relevant to my day-to-day work) are about pair programming and test driven development.

While I think it is commendable that people have done some research for these topics - it's just not enough (in particular, not by enough different groups or people).

So - what good science is there about software development? Comments welcome.

Copyright © 2010 Ivan Moore

IDE effecting code

2010-09-04T04:57:00.000-07:00

I don't write Java code exactly the same as I used to. Some of the ways my code has changed are due to using an excellent IDE.

One of these I mention in a previous article was about having fields public in some circumstances - I won't repeat the arguments here, I just want to mention that one reason I'd now make a field public when before I would have had a getter/setter is that it is trivial to convert a public field into a getter/setter when you need to (a refactoring called "encapsulate field"** in IDEA). (I still prefer not to have the internals of a class public whether accessed by fields or getters/setters though - tell don't ask.)

Another example is that now I only introduce an interface when it's really needed and not in anticipation of it being needed. Again, it's trivial to introduce the interface when it's needed using the "extract interface" refactoring of the IDE.

I was chatting to Nat Pryce about this and he was agreeing that using an excellent IDE has also changed the way he writes Java. I hope he and others will add comments to mention other ways their programming style has changed as a result of better IDEs.

I know for some people the idea of changing your programming style as a result of what a tool supports is heresy - but I think good development practice means using the tools and language so they work well together.

The examples given above refer to Java development where the team owns all the code rather than when writing an API - I'm not addressing API design here.

** BTW, I think calling the refactoring "encapsulate" is a bit of an exaggeration - really it is just replacing one form of non-encapsulation with another.

Copyright © 2010 Ivan Moore

miniSPA 2010 - Friday September 10th - BCS London

2010-08-18T13:00:00.000-07:00

Mike Hill and I have been volunteered to co-chair the miniSPA conference.

SPA is a fantastic conference - miniSPA is a condensed (and free!) version - it'll be great - all the places will go, so book now.

Here's the announcement and registration link (sorry the registration system is really horrible):

Experience some of the most popular sessions from this year's BCS SPA conference, for free, at miniSPA2010 on Friday September 10th at BCS London (near Covent Garden).

The miniSPA2010 one-day programme features five sessions, in two streams, that give an excellent guide to the variety and quality you'll find at every SPA conference.

We hope that attending miniSPA will encourage you to submit a session proposal for SPA2011, which will be taking place from June 12-15 (also at BCS London).

For more information visit http://bcs-spa.org/minispa-2010.html. Booking is essential. Places are limited so reserve yours now.

See http://bcs-spa.org/ for details of our programme of regular events.

©2010 BCS SPA | 5 Southampton Street | London | WC2E 7HA

Don't just put your documentation in source control

2010-06-19T02:17:00.001-07:00

In the GS04 course at UCL I told the students to put everything into source control; not just source code but also things like documentation and configuration files.

That's all very well, but my current project at work has made me realise that you can do even better. You should not just put your documentation in source control, you should serve it from source control.

On my current project we have all our documentation served directly from our subversion repository, mostly as html.

What about Wikis?

I used to be a big fan of having a project wiki.

A wiki can be great but:

they do have a tendency to get out of date easily
they can accumulate lots of garbage
they can be difficult to version control well (difficult to keep in sync with the project's source code repository - I know there are things like Trac but in a lot of companies you don't get to choose what systems you use)
if the project is abandoned for a long time, the chances are that the wiki might disappear (I've seen it a few times - moving to another wiki system and not migrating old pages) but the source code repository will be around for a long time (and if it isn't then you're screwed anyway).

Wikis still have their place - when collaboration with lots of people is needed, but for developer written documentation I'm happier to have it in the same source control system as the code it refers to.

Taking my own advice

I've updated my semi-abandoned project build-o-matic, so its web site is now served directly from its source code repository. It's now so much easier to update the web site (just edit and commit) that I might even get the web site up-to-date.

But there again, the World Cup is on TV today ...

Copyright © 2010 Ivan Moore

throw null

2010-06-10T11:49:00.000-07:00

My colleagues told me of some code they came across which included the statement: "throw null".

I'd never seen that before - I didn't know what it would do or even if it was valid.

I'll let you think about it. Have a go if you want to check your answer:


public class Surprise {
    public static void main(String[] args) {
        throw null;
    }
}

Tools and Environments - SCM and CI

2010-03-07T10:50:00.000-08:00

When I taught source control and continuous integration in 2007 (for the GS04 course at UCL) I used Subversion for the source control lab and build-o-matic for the continuous integration lab.

In the labs this year, I'll be using Mercurial instead of Subversion, and Hudson instead of build-o-matic.

What would you choose for teaching source control and continuous integration (and for bonus marks, why)?

Copyright © 2009 Ivan Moore

Tools and Environments

2010-02-27T03:39:00.000-08:00

Steve Freeman and I are teaching a course at UCL called "Tools and Environments".
The course we wish we’d had in college, only we didn’t know it at the time.

We cover subjects such as source control systems, automated builds, automated testing and continuous integration.

In preparing for the course, I've been reminded of how few books there are which we can use as a "course text". There are plenty of books for specific tools (e.g. Ant) once you know that you need those tools, but few books which explain the sorts of things that you need for real software development projects, and why you need them.

The book we're using for our "course text" is Practical Development Environments (and we'll also be recommending Continuous Integration as that also covers much of the material of the course).

If you have other recommendations please add a comment!

Copyright © 2009 Ivan Moore

Three heresies

2009-12-05T08:50:00.000-08:00

I encourage people to think for themselves rather than following cargo cults. You might or might not agree with the three heresies I've written about here, but do at least think about them.

Public fields

In Java code, instead of having a public getter and public setter for a field, why not just make the field public? It's much simpler and less code. If you later need a getter and setter for some reason you can always refactor to that (and many IDEs will give you help doing it). There is a comment by Richard Gomes at the end of this previous article on the subject of public fields for data objects. I think public fields make most sense for NOJOs (data objects) (in the rare case where a NOJO is useful - not very often) but maybe it would sometimes make sense for other sorts of classes too?

Note that having public access to a field is not what I'm trying to encourage. The point I'm making is that if you do have public access to a field then it doesn't matter much whether it is by getter/setter or making the field public, so you might as well use a public field as it is simpler. (But please, tell don't ask instead.)

Magic values instead of constants (in build files)

Instead of always factoring out magic values as properties in your build file, consider just using the magic value where it makes sense. For example, maybe instead of "${src}" just use "src" (and get rid of the property) - this was suggested by Jeffrey Fredrick at CITCON Paris 2009. I think there is a lot of merit in this approach. What are the chances that you'd be able to just modify the value of the "src" property and everything would still work? Probably quite low - you'd probably do a text file search for "src" anyway. What are the chances that you'll want to change it anyway? I think it's worth thinking about whether it's better or worse to factor out constants in some cases.

Make the CI build fail fast rather than run all the tests

Rather than running all the tests in your CI build, how about have the build fail as soon as any test fails? That way, a failing build uses less of your build farm's capacity. If your build farm capacity is limited, then this approach may result in getting a passing build sooner (as when the fix is committed there may be a build agent available for running the build with that commit sooner because it's time isn't being taken running a build which will eventually fail anyway). I think it's often more important to know which commit broke the build than which tests failed in order to know both who should fix the build and what caused the build breakage. This approach might not be so good if you have a flickering build (i.e. randomly failing tests) - however, making the build reliable can be achieved and is worthwhile anyway.

More heresies to follow

I have other heresies to write about. Please suggest your own in the comments.

Copyright © 2009 Ivan Moore

Java Enums with constant-specific methods

2009-11-24T23:07:00.000-08:00

One of my colleagues introduced me to this handy language feature of Java 1.5 and I wanted to write an article about it because I hadn't seen this before.

Using google reveals that it is already well documented if you RTFM, but I will repeat it here because talking to other Java developers indicates that it isn't as well known as it deserves to be. Here's a slightly reworded extract from the first hit on google or bing for "java enum":

You can declare abstract methods in an enum and override them with a concrete method in each constant. Such methods are known as constant-specific methods. Here is an example using this technique:


public enum Operation {
  PLUS   { double eval(double x, double y) { return x + y; } },
  MINUS  { double eval(double x, double y) { return x - y; } },
  TIMES  { double eval(double x, double y) { return x * y; } },
  DIVIDE { double eval(double x, double y) { return x / y; } };

  // Do arithmetic op represented by this constant
  abstract double eval(double x, double y);
}

New version of Jester released

2009-11-24T06:54:00.000-08:00

It's been a while since I last updated Jester (a mutation testing tool).

Today I released a new version of Jester - not much changed - but hopefully a bit easier to get it to work, based on my experiences of trying to use Jester when I haven't tried for a while.

It now doesn't read configuration files from the classpath - instead you specify the relevant files on the command line.

Copyright © 2009 Ivan Moore

XpDay London, 7th & 8th December 2009

2009-11-08T05:49:00.000-08:00

It's XpDay soon - book your place now. The Keynotes look particularly good this year.

On Monday there's an experience report that looks very interesting "When Agile Might Not Be The Best Solution" - it's good to see this sort of experience report on the programme because there's probably more to learn from it than one which goes something like "we did XP and it worked".

There are lots of other interesting looking sessions too - plus lots of open space sessions which can be excellent.

Copyright © 2009 Ivan Moore

Pair Programming interviews/auditions

2009-10-31T03:14:00.000-07:00

I have done a lot of pair programming interviews for my client. I enjoy doing them and I think they are extremely valuable.

Hiring is probably the most important thing to get right, and I think that pair programming interviews (or "auditions" as some like to call them) are usually the best way to interview developers.

Setting up a pair programming interview

For interviewing Java developers, I have a machine set up with a choice of IntelliJ IDEA and Eclipse with an empty workspace. I think it is important to try to make the candidate as comfortable as possible with the development environment so that it isn't a distraction. If you work with Apple Macs, also provide Windows (and maybe Linux) machines for candidates who aren't familiar with Macs because otherwise the difference in IDE shortcuts gets in the way and the idea of a pair programming interview isn't to see how familiar the candidate is with a particular OS.

I allow one hour for these interviews - I have found that is long enough to get a good idea about the suitability of a candidate.

Choosing a problem

One of the most surprising things is how small the problem has to be in order to be achievable in one hour. Developers (including myself) are often very optimistic about how quickly they can program a solution to a simple problem. I would urge you to try out the problem you want to use for a pair programming interview with a trusted colleague, before using it with a candidate, to find out whether it is realistic. To give you an idea of the size of problem you need, I used to use the problem described in this article.

I know of another team at my client who provide a small code base pre-prepared for the interview which includes a bug which the candidate has to find. I think this is a great idea. Choose a problem which is realistic for the sort of code that the candidate will be writing (or the sort of work they will be doing, e.g. debugging legacy code) in the job you are hiring for. There is no point testing for whether a candidate can write WeakHashMap from scratch if you are hiring for a typical enterprise IT project.

What pair programming interviews demonstrate

Pair programming interviews aren't a silver bullet for recruiting developers - they are good at determining some qualities of a candidate but not all. In particular I think they are more valuable in demonstrating programming style and personal interaction rather than problem solving. However, in typical enterprise IT projects, more problems are caused by over complex solutions to simple problems, or by solving the wrong problem, than having problems that are unique and difficult and require a brand new algorithm.

Sometimes it is good to use a problem which has some potential for ambiguities in the details of what is being asked for. This is a very good way to see whether a candidate asks for clarification or just makes assumptions which aren't justified. In enterprise IT systems, making assumptions about the desired behaviour of a system can cause a lot of rework and bugs so I look for candidates who demonstrate that they are thinking through possible ambiguities and who pro-actively ask for clarifications about the problem.

Being realistic

I make it very clear to candidates that I am looking for production quality, simple code rather than for them to show off all the language features they know. The problems that I choose are simple enough that I'm not interested in just getting a solution. I am looking for code that I would be happy working with. My boss, Pippa Newbold, has a very good rule of thumb. "At the very least, don't hire someone who will make the code base worse". It seems obvious and yet at other companies I have sometimes observed panic hiring just to make up the numbers where this rule would have saved lots of rework and bugs.

Giving something back

I try to teach the candidates something new in a pair programming interview where possible. Often this is something like an IDE shortcut but is occasionally a language feature or a "programming in the small" style discussion. I like candidates to get something back for giving up their time to come in for an interview, and a pair programming interview can be very daunting for candidates not used to pair programming.

Does it work?

As far as I can tell, all candidates that I have "passed" using a pair programming interview have turned out to be worth hiring.

However, that doesn't include those candidates that I've "passed" who didn't take the job and I don't know whether any of those candidates that I've "failed" would have been good either.

My suspicion is that it is a technique which is slightly more prone to "failing" good candidates than hiring poor candidates, but that really is just a gut feeling. If anyone knows any studies on pair programming interviews I'd be very interested to hear more.

Where it probably doesn't work

I guess that interviewing people for research or work which requires solving deep and difficult problems probably requires a different approach. Also, for hiring people with limited programming experience who you want to train up it might not be suitable.

Copyright © 2009 Ivan Moore

Cleaner code

2009-07-09T13:36:00.000-07:00

I recommend the book "Clean Code" - I completely agree with its philosophy and have written a few articles of my own about "programming in the small".

Mike Hill and I presented a session on "programming in the small" at the "Software Craftsmanship" conference 2009 in London and at QCON London 2009 - and one of or examples was taken from "Clean Code".

Clean Code

One of the examples in "Clean Code" is some code for parsing command line arguments, written in Java by Bob Martin. It is shown as an example of code which has already been made clean. There is an article (separate from the book) by Bob Martin about the code we use for this example. The source code is available from github:

git clone git://github.com/unclebob/javaargs.git

Mike and I chose this code because we wanted to take code which was already quite clean and show how even good code can sometimes be cleaned up even more - we wanted to push the cleanliness as far as possible in the session.

Just looking at the "Args" class only, this method seemed the one most wanting some further work (sorry, formatting is ugly - reformatted to fit blog page width):


private void parseSchemaElement(String element)
                           throws ArgsException {
   char elementId = element.charAt(0);
   String elementTail = element.substring(1);
   validateSchemaElementId(elementId);
   if (elementTail.length() == 0)
       marshalers.put(elementId,
                      new BooleanArgumentMarshaler());
   else if (elementTail.equals("*"))
       marshalers.put(elementId,
                      new StringArgumentMarshaler());
   else if (elementTail.equals("#"))
       marshalers.put(elementId,
                      new IntegerArgumentMarshaler());
   else if (elementTail.equals("##"))
       marshalers.put(elementId,
                      new DoubleArgumentMarshaler());
   else if (elementTail.equals("[*]"))
       marshalers.put(elementId,
                      new StringArrayArgumentMarshaler());
   else
       throw new ArgsException(INVALID_ARGUMENT_FORMAT,
                               elementId, elementTail);
}

So - what could be better? What struck Mike and I was the duplication of "marshalers.put(elementId, new XXX());"

To remove this duplication, first extract a variable called argumentMarshaler of type ArgumentMarshaler in each of the branches, then move the expression "marshalers.put(elementId, argumentMarshaler);" outside of the if statement, and the declaration of the variable "argumentMarshaler" before the if statement. You end up with:


private void parseSchemaElement(String element)
                           throws ArgsException {
   char elementId = element.charAt(0);
   String elementTail = element.substring(1);
   validateSchemaElementId(elementId);
   ArgumentMarshaler argumentMarshaler;
   if (elementTail.length() == 0) {
       argumentMarshaler =
                  new BooleanArgumentMarshaler();
   } else if (elementTail.equals("*")) {
       argumentMarshaler =
                  new StringArgumentMarshaler();
   } else if (elementTail.equals("#")) {
       argumentMarshaler =
                  new IntegerArgumentMarshaler();
   } else if (elementTail.equals("##")) {
       argumentMarshaler =
                  new DoubleArgumentMarshaler();
   } else if (elementTail.equals("[*]")) {
       argumentMarshaler =
                  new StringArrayArgumentMarshaler();
   } else
       throw new ArgsException(INVALID_ARGUMENT_FORMAT,
                               elementId, elementTail);
   marshalers.put(elementId, argumentMarshaler);
}

Now it's clearer that this method is doing too many things - one of those things being to find the relevant marshaler. We can extract a method for finding the marshaler and, having been extracted can improve it further by removing unnecessary structured programmingness, ending up with:


private void parseSchemaElement(String element)
                         throws ArgsException {
   char elementId = element.charAt(0);
   String elementTail = element.substring(1);
   validateSchemaElementId(elementId);
   marshalers.put(elementId,
        findAppropriateArgumentMarshaler(elementId,
                                         elementTail));
}

private ArgumentMarshaler findAppropriateArgumentMarshaler(
                    char elementId, String elementTail)
                                   throws ArgsException {
   if (elementTail.length() == 0) {
       return new BooleanArgumentMarshaler();
   } else if (elementTail.equals("*")) {
       return new StringArgumentMarshaler();
   } else if (elementTail.equals("#")) {
       return new IntegerArgumentMarshaler();
   } else if (elementTail.equals("##")) {
       return new DoubleArgumentMarshaler();
   } else if (elementTail.equals("[*]")) {
       return new StringArrayArgumentMarshaler();
   } else
       throw new ArgsException(INVALID_ARGUMENT_FORMAT,
                               elementId, elementTail);
}

Notice that "elementId" is only used for the exception to throw if an appropriate ArgumentMarshaler cannot be found. Further investigation shows that actually this parameter isn't used in ArgsException for an "INVALID_ARGUMENT_FORMAT" and we can just delete this argument (and hence the "elementId" parameter of "findAppropriateArgumentMarshaler") with no change to the behaviour of the code!

There is more that can be done with the Args class - but that is beyond the scope of this article.

Conclusion

I hope this has shown that even clean code can be made cleaner - without having to do anything too clever or drastic. It's often easier to see such opportunities in other people's code or code you haven't seen for a while - even the best programmers can miss opportunities for making code simpler.

Copyright © 2009 Ivan Moore

Project management lessons - commitment

2009-06-20T07:54:00.000-07:00

Have you ever heard (or said) something like "you've got to work [long hours/weekends] because the customer has been promised [some system/feature] by [some date]"?

Commitments can be made but not assigned

People only feel committed to things they have committed to themselves. Telling someone that they have to work long hours/weekends because of a commitment someone else has made can lead to demotivation and resentment. Productivity is very likely to suffer (quality decreasing, more rework due to mistakes, more stupid things done due to rushing or tiredness). Seeing that the work isn't being completed quickly enough, bad managers think that it just requires even more hours to be put in (they try to impose "more commitment" and concentrate on the inputs rather than the outputs). This is the recipe for a "death march" project.

Estimates are not commitments

As a project manager, you have to understand the difference between an estimate and a quote. An estimate is not a commitment. As a project manager, if you make a commitment that something will be done by a certain date then do not rely on being able to just use estimates directly - you need to apply some project management techniques. One technique that I have found works well, as mentioned in my previous article is "extreme planning".

Commitment can work wonders

When someone really feels committed to something, then they will be motivated and focused. Software developers will often work harder and more effectively on their pet open source project than they do at work due to this motivation. This sort of commitment cannot be forced onto someone. There are lots of ways to ensure it doesn't happen (sadly common) and a few things you can do to provide an environment in which it might happen - and hence see massive improvements in productivity and quality as a result.

One factor is the level of technical autonomy. Many large companies have standards which are intended to reduce costs (for various reasons I won't go into). Unfortunately, I think in many cases these standards tend to reduce teams' technical autonomy, which reduces motivation, having a larger negative effect on productivity (hence cost) than the savings made by the standardization. I'm thinking of things like (for example) being told you have to use Wholly Pointless Server when some free open source app server would be a better choice.

Another factor which I think is very important is how directly developers work with their customers. A lot of developers (and certainly the ones I like working with) like to do things which are useful and appreciated by the users of their software. Sometimes the way teams are set up there is a separation between developers and customers - often a business analyst who understands the business and is more available than the real customer(s). This does have benefits when a developer wants a question answered straight away (but the customer isn't available), because otherwise the developer would be blocked or have to context switch onto another area of work. However, I think the best arrangement (not always possible) is when the real customer is available all the time and the developers talk directly to them. The worst situation is when only the project manager or business analyst talks to the customer and approaches the developers with an attitude of demanding features rather than working collaboratively for the best solution.

There are lots of other factors involved in motivation - I might write more in a future article - if I can be bothered. :-)

Bike ride commitment

As an example of commitment - I said I'd do the London to Aachen bike ride and people sponsored me to do it (many thanks), so I felt a commitment to do it. If I hadn't felt any commitment then I wouldn't have done it because, frankly, it wasn't much fun. Here's a quick summary of how it turned out:

Day 1 - Gatwick to Dover/Dunkerque (85 miles cycling) - it wasn't too bad - a bit of drizzle, nice scenery and the cycling was not too tough. (We then got a ferry to Dunkerque - the staff at Norfolk line ferries were great, putting on a fantastic meal, and a tour of the bridge, specially for us).

Day 2 - Dunkerque to Waterloo (139 miles) - we had an early start (breakfast at 6:30am local time, i.e. 5:30am UK time) - in order to do 70 miles before lunch - the roads were totally flat but it was into a headwind the whole way - it was brutal. Some participants skipped the afternoon cycling and got a minivan to the hotel straight after lunch. Immediately before lunch I felt like going in the van too, but after some food felt much better and thought I'd try to keep going. The afternoon turned out to be much better - it was a bit hilly but much less windy. At around 100 miles for the day, 3 other particpants dropped out and got in the support van. At around 120 miles for the day I was knackered - but just about managed to finish the day's ride - going really slowly for the last 20 miles.

Day 3 - Waterloo to Aachen (106 miles) - I was knackered from the day before - so I took it very easy, along with some of the others who had struggled to complete the previous day's ride. There were some cobbled roads, which on a race bike are very scary. Near the end of the day, going up a long climb (which features in the pro cycling "Amstel Gold" race), we cycled through a thunder storm - we had hail, torrential rain (consequently very bad visibility), thunderbolts and lightning (very very frightening).

Copyright © 2009 Ivan Moore

Signs

2009-05-17T06:29:00.000-07:00

Unfortunately, I won't be going to XP 2009 - I'm sure it'll be great - enjoy it if you're going.

While training for the London to Aachen bike ride (sponsor me) I regularly cycle past a couple of signs that somehow make me think of the XP conferences:

Copyright © 2009 Ivan Moore

Project management lessons

2009-04-26T05:25:00.000-07:00

I signed up for the National Autistic Society's London to Aachen bike ride. When I signed up it was 3 days, with the longest day being the second at about 100 miles (it was noted that this event hadn't been run before so the itinerary might not be 100% accurate).

A few weeks later, I got an update on the itinerary - I was told that the organizer had done a recce (reconnaissance) of the route and the second day would now be 120 miles (last day longer too but not too bad).

Just over a week ago, I got another update on the itinerary; actually the second day will be 137 miles (still not shown on the web page though!). To put that in perspective - that's only 4km less than the longest stage of the Tour-de-France - ouch.

Therefore, of course, you should sponsor me. Also it gives me a good example of the sort of thing that happens with software projects very frequently; so here are some relevant lessons using this as an example.

How to handle estimation

Most people are rubbish at estimating - not only software developers about code, but lots of people about lots of other things too. The three most obvious reactions to that are (a) get better at estimating (b) take bad estimating into account (c) don't estimate (or at least, not quite like most people do). I have found "extreme planning" (see Planning Extreme Programming and Agile Estimating and Planning) to work very well - however it is only really relevant to an incremental project. Quite literally, YMMV!

For a one off event like the bike ride then there's not much I can do for this event (as consumer of the estimates). However, I will treat future event descriptions as being 40% inaccurate if they haven't been run before (i.e. using "yesterday's weather" in "extreme planning" style).

Admit when you have screwed up and don't know the answers

The first change to the itinerary was not entirely unexpected - I knew the route hadn't been done before - my expectations had been set that this was an estimate.

The second change was a nasty surprise. There are two lessons here. When giving an estimate do not give it a level of confidence that it doesn't merit. If the first change to the mileage had said "and we still aren't quite sure" then the second change would not have been so unexpected. The second lesson is that if you have screwed up an estimate, then be extra generous with the re-estimate. If the first changed estimate had been 130 or 140 then 137 wouldn't have seemed so bad. If the second re-estimate had turned out to be 125 I wouldn't have been so bothered about it dropping from 130 or 140.

Mitigating risk

If as a project manager you find out that something is going wrong (e.g. the first re-estimate of the itinerary) then do something to reduce the risk of that going even more wrong. In my case, I increased the amount of training I've been doing (hence I've been writing fewer blog articles because I'm knackered at the weekends!).

Some things you just can't predict - but maybe everything will work out OK anyway

Unfortunately, my bike has developed a fault with the gears which may make it financially non-viable to repair so I may have to get a new bike - but given the new increased mileage that may be worthwhile even if my current bike can be fixed.

I'm disappointed that the event doesn't appear as well prepared as the London to Paris bike ride that I did in 2007 and that I haven't raised as much money (yet) as in 2007, but you can help with that!

Copyright © 2009 Ivan Moore

Save Bletchley Park - the birthplace of computing!

2009-03-08T06:44:00.000-07:00

You've heard of Alan Turing? Turing award? Turing test?

How about the Enigma machines and encryption/decryption?

What about the first computers?

What's the connection between these three?

Bletchley Park. It's a great place and historically very significant - but underfunded and in need of maintenance otherwise it'll be lost forever. Please help save Bletchley Park for future generations. Help support it by visiting - it's a great day out.

If you are an Agile coach - go to the Agile Coaches Gathering which is being held at Bletchley Park.

Copyright © 2009 Ivan Moore

Bootable USB sticks for conference presentations

2009-03-06T14:16:00.000-08:00

If you have a programming workshop or tutorial to present at a conference, where people bring their own laptops and you want to give the participants programming things to do, then one of the things that always seems to take more time than you expect (during the session) is setting up environments etc before being able to actually do any programming.

With Stuart Ervine, a colleague of mine who is running a session on GWT at SPA2009 (book now!), we have done some experimenting and now have a setup that I think works quite well - so I wanted to share it here in case it is helpful to others presenting similar sessions.

Having presented "Programming In The Small" (PITS) with Mike Hill at "Software Craftsmanship 2009" I can report that it worked quite well in practice (not perfect though - more on the glitches later).

Providing a complete environment:

Stuart and I wanted an identical environment / setup for all of the participants, so time wouldn't be wasted getting everything working on lots of different platforms. Even if you provide one click installers, participants may be understandably reluctant to install some software just for your session. They may even have things installed which conflict with whatever you want to give them. Therefore, we thought it would be better to provide a complete environment - OS and all.

We considered two approaches, providing a virtual image (e.g. VMWare or VirtualBox) with everything installed and configured, or using a bootable USB stick. I attended Lasse Koskela and Markus Hjort's excellent session "Reading Code Without Psychic Powers" where they provided bootable CDs which included the complete environment for several different languages. It worked very well - if rather slow to initially boot up. Having had that experience as a participant, Stuart and I decided to try bootable USB sticks.

Making a bootable USB stick:

First - download the latest Ubuntu (currently 8.10) - and burn it to a CD (good instructions available on the Ubuntu site). Boot your computer off the CD and be patient while Ubuntu starts (it's slow off a CD). These instructions are provided entirely at your risk.

Create two partitions on the USB stick:

From the System menu, Administration submenu, select "Partition Editor"
Take a note of the device(s) available.
Insert a USB stick that you want to use for this (it will get completely overwritten, and must be at least 1Gb - we used 4Gb sticks).
Right click on the USB stick that has been recognised and "Unmount Volume".
In the "Partition Editor" (GParted) select menu item GParted -> Refresh Devices
In the "Partition Editor" (GParted) select the USB stick from the menu GParted -> Devices
if you proceed from this point with the wrong device selected then you'll trash your machine, so don't get it wrong. Clues about which device is the USB stick are the size of the device and that it wasn't on the list before you plugged it in.
For each of the partitions on the USB stick (there will usually only be one) right click and "Delete"
Click on the (now grey) bar which represents the partitions on the USB stick
Click "New"
Drag the right hand side of the bar and create a FAT32 partition large enough for the Ubuntu image (about 710Mb should be enough). See image below.
Click on the (now grey) bar which represents the unpartitioned space on the USB stick
Click "New"
Create another partition - can be FAT32 or ext2 - doesn't really matter much. (I found FAT32 to be fine).
Click "Apply"
Close the "Partition Editor" (GParted)

Write Ubuntu onto the USB stick:

From the System menu, Administration submenu, select "Create a USB startup disk"
Select the device which is the USB stick (as noted above when creating the partitions). If you don't see your device in the list of "USB Disk to use", you might possibly have to unmount the newly created partitions of the USB stick, remove your USB stick and put it back in.
Click "Make Startup Disk"
Get a cup of tea - this'll take a few minutes

Copy programs/files to the other partition:

You can copy whatever programs, files etc the participants need to the other partition (the second one you created). For the PITS session this meant Eclipse, Harmony JDK and an Eclipse workspace with the projects for the session. Running from the USB stick won't be very fast but may be acceptable.

To make your software run faster:

If you want the software to run faster, then set it up for participants to copy to the Ubuntu desktop (where it will be running from memory).

If you want to run memory hogging software from the desktop then participants might not have enough RAM (they might run out of memory and I think the live Ubuntu won't try to use the hard disk for swap space). You can set up Ubuntu to use swap space on the USB stick. To create and mount a 1Gb swap file:

Create an empty file the size of the swap space you want to provide (where /media/disk-1 is where the second partition created earlier is mounted) (this may take a few minutes):

dd if=/dev/zero of=/media/disk-1/swapfile.swp bs=1024 count=1024k

Provide an executable script in the second partition for participants to run if they need swap space, containing:

mkswap swapfile.swp
sudo swapon swapfile.swp

Mass production:

So far so good - however, you don't want to have to do this for every USB stick you need to create. Having created one USB stick which has everything set up just right, you can copy its entire contents (including partition table) to other USB sticks using the following (where /dev/sde is the "master" USB stick and /dev/sdf is the one you want to copy to):

dd if=/dev/sde of=/dev/sdf bs=8M count=250

This copies the first 2Gb from USB stick /dev/sde to USB stick /dev/sdf. In order to make this mass copying faster, for the PITS session we didn't use all the space of the USB sticks. We created the second partition using only part of the remaining space. The numbers for "bs" and "count" were arrived at experimentally to minimize the time it took to make the copies. (Multiply them together to work out the amount copied - i.e. using "bs=4M count=500" copies the same amount, but with my USB sticks was slightly slower). The copying may take a few minutes per stick.

Glitches and gotchas:

Be really careful about mount points (e.g. "/dev/sde") so as not to trash your hard disk.
Not all machines will boot off a USB. The big glitch is that Apple Macs don't seem to like booting from a USB stick (and even some regular PCs don't). One potential solution for such participants is to provide a bootable Ubuntu live CD and use the USB stick for the programs to run in the live Ubuntu.
Not all participants were overjoyed at using Ubuntu.
Participants already using Ubuntu couldn't just run the software included in the second partition because the paths were set up for the "live CD" user.
Participants need to have some easy way to go from having booted off the USB stick to running the software for the session - we provided a "run.sh" file but in the rush of people getting their machines to boot off the USB (trying to find which key to press to change the boot device) not everyone heard the instructions (which we should have given before handing out the USB sticks).
Some participants took a while to find the right key to press to boot off a USB device - a lot of machines print the message saying which key to press for such a short time it is difficult to read it. It may be an idea to suggest that participants figure this out before the session.

Other considerations:

It would have been nice to allow participants to keep the USB sticks but they cost money and we're not working for some large consultancy who might want to do this for promotion.

You need one USB stick per machine (we had participants pairing).

We provided other mechanisms for participants to get the materials for the session - it was fairly straightforward for the PITS session as it was pure Java - so we provided the source (and Eclipse and IntelliJ project files) for participants who already have a Java environment. A lot of participants were very organized and had downloaded these materials from the conference web site before the session. We provided the materials in password protected zips and gave participants the password at the session.

And finally:

If you want to see this in action - go to Stuart's GWT session at SPA2009. Ubuntu rocks. GWT rocks. Stuart rocks. Mike rocks. SPA rocks.

Many thanks to Stuart Ervine for help with working this all out and writing it up. Many thanks to Mike Hill for preparing and co-presenting PITS.

Copyright © 2009 Ivan Moore

Two styles

2009-02-14T02:12:00.000-08:00

Even simple programs can be written many different ways. This article shows an example of two different styles for a very simple program - I hope the difference is instructive.

The code is shown in Java - but the two styles are applicable to any object-oriented programming language.

The problem to solve

The problem is a slightly reduced version of the parsing part of this robot blocks programming contest problem. To slightly reduce the problem I just use the first two commands and treat the input as a string rather than a file.

Therefore, the problem is to parse input that looks like this:

10
move 4 over 3
move 3 onto 5
quit

i.e. the first line is the number of blocks (used to set up the robot), the last line is "quit" (that's just how the input is - has no effect on the robot) and the lines in between are commands (each line is one command, e.g. "move over" is a single command with two parameters, the source block and the destination block).

No validation is needed - the input will always be well formed - there are no tricks in the question - it is as it looks.

The input will be used to control a robot. You can come up with whatever api you want - you can treat it as if you will be implementing the robot at some point too.

Now your turn

Please have a go at implementing this in Java or a similar language - (parsing input like that shown) - allow a maximum of one hour. If you have any questions about the problem, take the answer as being whatever makes it simplest - there are no tricks to this.

What did you end up with?

I've seen a lot of solutions to this over the last year. That is because I've been doing a lot of interviewing for my client (using the best interview question) and have used this example many times.

What most people end up with

Here's a solution in the most popular style:

package com.oocode;

import static java.lang.Integer.parseInt;

import java.util.ArrayList;
import java.util.List;

import com.oocode.Command.Type;

public class RobotParser {
  private final int numberOfBlocks;
  private final List commands = new ArrayList();

  public RobotParser(String input) {
    String[] lines = input.split("\n");
    numberOfBlocks = parseInt(lines[0]);
    for (int i = 1; i < lines.length - 1; i++) {
      String line = lines[i];
      String[] parts = line.split(" ");
      int sourceBlock = parseInt(parts[1]);
      int destinationBlock = parseInt(parts[3]);
      Type type = parts[2].equals("over") ? 
            Command.Type.MOVE_OVER :
            Command.Type.MOVE_ONTO;
      commands.add(new Command(sourceBlock, 
                               destinationBlock, 
                               type));
    }
  }

  public int getNumberOfBlocks() {
    return numberOfBlocks;
  }

  public List getCommands() {
    return commands;
  }
}

and:

package com.oocode;

public class Command {
  private final int sourceBlock;
  private final int destinationBlock;
  private final Type type;
  
  public Command(int sourceBlock, 
                 int destinationBlock, 
                 Type type) {
    this.sourceBlock = sourceBlock;
    this.destinationBlock = destinationBlock;
    this.type = type;
  }
  
  public enum Type {
    MOVE_OVER, 
    MOVE_ONTO
  }
  
  public int getSourceBlock(){
    return sourceBlock;
  }
  public int getDestinationBlock(){
    return destinationBlock;
  }
  public Type getType() {
    return type;
  }
}

This is a decent representative of this style. I've chosen a shortish variation of the style to keep down the amount of code you have to wade through (although it could have been slightly shorter still if I'd parsed the command type using the enum). A popular variation is to have a Command interface and different implementations for MoveOverCommand and MoveOntoCommand. One thing that is quite nice about the problem is that there is scope in the solutions for showing different programming style and taste.

As far as the "two styles" that I'm talking about, the important feature is that the RobotParser has getters for the numberOfBlocks and the Commands, which are NoJos.

An alternative solution

Hardly anyone implements this alternative style:

package com.oocode;

import static java.lang.Integer.parseInt;

public class RobotParser {
  private final Robot robot;

  public RobotParser(Robot robot) {
    this.robot = robot;
  }

  public void parse(String input) {
    String[] lines = input.split("\n");
    int numberOfBlocks = parseInt(lines[0]);
    robot.setNumberOfBlocks(numberOfBlocks);
    for (int i = 1; i < lines.length - 1; i++) {
      String line = lines[i];
      String[] parts = line.split(" ");
      int sourceBlock = parseInt(parts[1]);
      int destinationBlock = parseInt(parts[3]);
      if(parts[2].equals("over")) {
        robot.moveOver(sourceBlock, destinationBlock);
      } else {
        robot.moveOnto(sourceBlock, destinationBlock);
      }
    }
  }
}

and:

package com.oocode;

public interface Robot {
  void moveOver(int sourceBlock, int destinationBlock);

  void moveOnto(int sourceBlock, int destinationBlock);

  void setNumberOfBlocks(int numberOfBlocks);
}

A variation of this would be to have a RobotFactory which creates a Robot for a given number of blocks, to prevent the possibility of command methods being called before "setNumberOfBlocks". I would expect the implementation to use a BufferedReader if it were reading from a file rather than a string input. As before, I've gone for a short variation of this style.

In this style, the important feature is that there are no getters, and no Command class or classes. You could possibly describe this solution as being an example of "tell don't ask".

And finally ...

Most of the solutions I have seen to the problem have been during interviews that I have conducted. During these interviews, most solutions are the first style and hardly any of the second style.

I don't know whether the popularity of the first style is skewed by it being an interview situation, by the demographics of the people I am interviewing or for some other reason. Most of the interviewees didn't use TDD but did write automated unit tests after the code. However, the majority of those using TDD still produced the first style.

What do you think? What would you do? Which style is "better"? Is that a valid question? If you come up with a different solution that you think is interesting then please put a comment with a link to your blog - please don't paste code in your comments or they will get too long and probably won't format well anyway.

I'm looking forward to seeing something completely different - regex anyone? I don't know - I'm sure there are other styles but the first one shown is, by far, the one I have seen most often when interviewing for Java developers. (And, of course, different languages could produce completely different styles).

And finally - does anyone have good names for these two styles (maybe "getter style" vs "doer style")?

Copyright © 2009 Ivan Moore

Making fields and methods as private as possible - a postmodern Thatcher.

2009-01-24T04:50:00.000-08:00

Fields and methods should be declared as private as possible. As I mentioned in an article on the subject "I thought of writing a tool to do this that I was going to call "Thatcher" - it would privatize as much as possible".

I only realized a few days ago that I've already written it (I really am quite dense), and in fact had written it several years before writing that article.

How do you tell if fields and methods can be made private?

I think the most popular ways for people to tell are:

look for all references to the field or method and see if any are outside the class it is declared in
change "public" to "private" and see if it fails to compile (e.g. red Xs in eclipse)

Unfortunately, both of these approaches have limitations for most enterprise java applications (particularly web applications). These approaches don't work if the field or method in question is used reflectively, e.g. by a template (e.g. FreeMarker, JSP etc). That's the limitation of a static analysis implementation of "Thatcher".

If you are paranoid, how do you tell if fields and methods can be made private?

The only safe way to tell, for a field or method that you want to see if it can be private, is to make it private and then see if the application still works.

How to automate that?

If you have good test coverage, then "the application still works" is automated as "the build (including tests) passes" (e.g. the build that you use on your CI server).

Then you are left with the problem "how do I tell if "public" can be changed to "private" and the build (including tests) still passes?".

Now that sounds familiar to me - it's rather close to what Jester does.

Jester - a mutation testing tool

I wrote Jester as a mutation testing tool. As it says on the web site: "Jester makes some change to your code, runs your tests, and if the tests pass Jester displays a message saying what it changed." The idea is that if you can change your code and the tests still pass then the tests aren't covering the code (other possibilities are that it's a behaviour preserving change, or the code is redundant).

Jester as a postmodern Thatcher

Jester can be used to implement Thatcher by configuring it to mutate "public" to "private" and "protected" to "private" (package visibility is harder!). I think this could be described as a postmodern programming approach - (re)using some code to do something it wasn't designed for.

I hadn't thought of doing this until it occurred to me a few days ago (and now it seems so obvious) because I had imagined that Thatcher would use static analysis and hence be very different (that's my modernist thinking, it would have to use static analysis to be a fast and "proper" implementation and the fact that it wouldn't always be correct would be a fault of the imperfect world).

It might be that someone has proposed this before (in which case, sorry, I don't remember).

Limitations of Thatcher implemented using Jester

It should be noted that using Jester for implementing Thatcher only works if your build (including tests) has sufficient coverage that you are happy that if it passes then that means the application works.

Also note that using Jester for this could take a really long time. If your build takes an hour then it would take many machine days to privatize even a very small amount of code.

An example run of Thatcher implemented using Jester

Here's an example of using the latest version of Jester ("simple jester" also described here) as Thatcher. For this example, I'm using Checkstyle as the code base for which I want to make fields and methods as private as possible. I chose Checkstyle for this example because the build is very fast!

(To try this out for yourself, you will need Java installed with java, javac and ant on the path. These instructions are for Windows. Replace <wherever> as appropriate).

download checkstyle
unzip
cd to <wherever>checkstyle-src-5.0-beta01
in order to check that everything is OK so far, execute "ant run.tests"

should get:

BUILD SUCCESSFUL
Total time: 33 seconds

(or whatever time)

download Jester (the simple-jester variety)
unzip
cd to <wherever>simple-jester-1.1
for the step below to work, execute "setcp.bat"
in order to check that everything is OK so far, execute "test.bat"

should get:

17 mutations survived out of 19 changes. Score = 11
took 0 minutes

(or whatever time)

In order to check that the checkstyle build can be run from the jester directory (i.e. still in <wherever>simple-jester-1.1) (which makes it simpler for running Jester in this example)

ant -f <wherever>checkstyle-src-5.0-beta01\build.xml run.tests

should get:

BUILD SUCCESSFUL
Total time: 21 seconds

(or whatever time)

Now edit "mutations.cfg" (in <wherever>simple-jester-1.1) and replace contents with:

%public%private
%protected%private

That configures Jester to try mutating "public" to "private" and "protected" to "private". Unfortunately, Jester cannot currently be configured to not mutate literal numbers - so in order to avoid Jester making mutations to literal numbers you have to be a bit cunning.

Create a text file in folder "<wherever>simple-jester-1.1\\jester" called SimpleIntCodeMangler.java with contents:

package jester;

public class SimpleIntCodeMangler implements CodeMangler {
public SimpleIntCodeMangler(ClassSourceCodeChanger sourceCodeSystem) {
}

public boolean makeChangeToClass() throws SourceChangeException {
return false;
}
}

Then (in <wherever>simple-jester-1.1) execute "javac jester\SimpleIntCodeMangler.java".

If "." is on the classpath before simple-jester.jar then the new version of SimpleIntCodeMangler will replace the version in simple-jester.jar, so Jester won't do the literal number mutations.

Now to run Jester/Thatcher (just on the one class "Checker"), execute:

java jester.TestTester "ant.bat -f <wherever>checkstyle-src-5.0-beta01\build.xml run.tests" <wherever>checkstyle-src-5.0-beta01\src\checkstyle\com\puppycrawl\tools\checkstyle\Checker.java

(note that on windows it has to be "ant.bat" and not just "ant")

Now Jester will run for a while, and eventually you get:

4 mutations survived out of 24 changes. Score = 84
took 5 minutes

If you want a simple way to visualize the results, then run "python makeWebView.py" and have a look at "jester.html".

To get the code changed to make methods and fields private which can be made private, then run "python makeAllChangesFiles.py jesterReport.xml" and you get a version of Checker.java called Checker.jester (in the same folder, i.e. "<wherever>checkstyle-src-5.0-beta01\src\checkstyle\com\puppycrawl\tools\checkstyle\") which has all the privatizations where the tests still pass.

The results for Checker.java show that addFileSetCheck, setModuleFactory, setSeverity and setClassloader can be made private and the "ant run.tests" build still passes. Having had a look at the code - it might be that these methods need to be public for something not covered by the tests but are needed for the application to work (because the only tests are unit tests and not end-to-end tests; this is a limitation mentioned earlier).

And finally

If you like that (or even if you don't) then please donate to my 5 countries (300 miles) in 3 days bike ride to raise money for the National Autistic Society.

Copyright © 2009 Ivan Moore

2009-01-17T09:27:00.000-08:00

My favourite conference (SPA) is open for registrations. Book before 31 January to catch the early-bird discount rate.

I'm programme co-chair (with Mike Hill) and very happy with the programme we've been able to assemble.

Book now.

Copyright © 2009 Ivan Moore

Charity Day

2009-01-04T04:08:00.000-08:00

I am doing a charity bike ride in June to raise money for the National Autistic Society (NAS). One of my sons is autistic and the NAS have been very helpful. I did a London to Paris bike ride for NAS in 2007.

Please donate!

Time is money

For the 2007 bike ride I paid the costs myself so all money donated went to the NAS. For the 2009 bike ride I'd like to do something equivalent but slightly different. I'm offering a day of my time for a donation that more than covers the costs. (The alternative is that I pay using roughly a day of my usual billing - but I'd like a generous charity minded company to donate more than that).

That day could be:

Refactoring and TDD Training
continuous integration - e.g. installing an automated CI system (e.g. TeamCity, build-o-matic or whatever)
Agile consulting (I've been doing "Agile" a long time - papers at XP2000, XP2001, XP2002, OOPSLA, TOOLS. Presentations at XPDay, SPA, ACCU etc)
other stuff (dunno - email me and we can discuss)

and there is more:

publicity - I'll write a short blog article - anything from just the name of the company and the donation up to an article about what I did for you (you can review/edit before publication)
"small print" - I'm based in London, England. Anything outside of London might require you to pay expenses too.
"more small print" - If I get multiple offers then I'll choose whichever I prefer.

To discuss this, or for more details, please email ivan at teamoptimization.com

If you aren't in the market for a day of my time then please make a donation of any amount you'd like!

Copyright © 2009 Ivan Moore

Running builds for previous revisions

2008-12-14T02:50:00.000-08:00

Can you run the build for a previous revision of your code? Can you do that fully automatically or do you need manual steps? Should you care?

It's good to have an automated build

As Martin Fowler says "... you should be able to walk up to the project with a virgin machine, do a checkout, and be able to fully build the system."

There are lots of benefits of having a fully automated build. It means all developers are building in the same way, reducing "it works on my machine" problems. It helps with productivity - you don't make mistakes doing the build and then have to do it again to get it right. There are other benefits too - but there are lots of other articles about this topic so I don't want to repeat them here.

Turning the dials up

What Martin Fowler doesn't say is "... do a checkout for any revision ..." (rather than just the most recent), but I think it is what you should aim to do.

Why "any revision"?

One reason you might need to run the build for any revision is in order to work out which commit broke the build in the case where you have multiple commits between builds and the build is broken. There are other reasons too - e.g. finding out what change introduced some bug that has been hiding undetected for a while (i.e. didn't cause the build to break but now you've found it and want to find out how to fix it or when/how it got introduced).

(As an aside - some CI servers can help reduce the occurrence of multiple commits between builds, but even if you are running a CI server which builds multiple revisions in parallel on multiple build agents, if the build farm isn't large enough for the number of commits and the speed of the build, then you will end up with multiple commits between builds. Or if you take the approach of having the CI server run the build before committing then you end up with a bottleneck delaying commits if you don't have enough agents.)

Can you run the build for a previous revision of your code?

If your system includes a database - how do you manage the database schema? In some teams, this is one of the dirty secrets - it's done by running some scripts manually as necessary - hence you can't run the build for a previous revision (or in some cases even the most recent revision) just by checking out the code (for that revision) and running the build.

Many teams use something like dbdeploy so changes to the database are made as delta scripts which make the necessary changes to the database in synch with code changes. Although this allows database migration forwards in time - how many teams implement the rollback scripts to allow the database to be reverted so that it is in synch with previous revisions of the code? Often you have to just rebuild the database from scratch in that case - and the build script might not do that automatically.

What if you use maven and have snapshot dependencies? How do you build the code the same as it was built an hour ago?

Why fully automate?

Any manual steps in something that you don't do frequently makes it likely that you'll find it difficult to do when you need to. Furthermore, manual steps to build previous revisions prevent you from taking advantage of all of the features of some CI servers (TeamCity, Pulse and build-o-matic all allow manually triggered building of a previous revision of the code. build-o-matic will automatically build previous revisions of your code in some circumstances, so if your build doesn't support running the build automatically for a previous revision then it won't always work as expected).

One of the logical conclusions - your development environment should be checked in

Being able to run the build for any revision of your code might require you to have more things in your source control system than you are used to. This can be controversial (even though it's mentioned in Martin Fowler's well read CI article). I like to have absolutely everything that the build depends on in the source control system - even when it isn't source.

For example, on a Java project, I like to have Java itself, Ant etc all checked in. If you change which version of Java you use (which you are more likely to do than you might think) then you want to ensure that everyone is using the same version and that you can build the project as it was in the past (if Java isn't checked in then how are you going to be sure what version it is or was)? This can be controversial (Martin Fowler mentions "Java development environment" as something that you might not check in but I disagree with him on this specific case) - but if you take the ability to check out and build the code at any point in time it to it's logical conclusion then you have to include everything you need to build it - machine set up and all. One of the advantages of taking things to this level is that it is trivial to set up a new development machine - you just check out everything and you are ready. This isn't always possible - particularly on Windows - but is a worthy aim and more achievable than you might think.

Copyright © 2008 Ivan Moore

Software Craftsmanship conference 2009

2008-12-13T06:58:00.000-08:00

The Software Craftsmanship conference 2009 is open for session proposals. Please submit a session proposal. I think the conference is now fully booked for attendees but you can still get in to the conference if you get your proposal accepted.

It's a great idea for a conference. I think a lot of progress has been made in our industry to improve processes. As teams become better in their processes, programming practices and code quality become relatively more important.

The timing of the conference is great - with the release of more books now that concentrate on code quality (like clean code).

Copyright © 2008 Ivan Moore

Refactoring and TDD Training Course

2008-12-11T23:24:00.000-08:00

I'm offering a completely new style of training course on refactoring, TDD and just programming better. It is pair programming with me (or others depending on demand).

It'll be similar cost per person per day to other courses, but instead of a classroom, it's one-to-one instruction at your site. It's also working on your code base, improving it during the course (possibly adding features too). Attendees will retain more learning that is directly applicable compared to more regular classroom courses. They will learn things from this course in addition to refactoring and TDD, like better use of the IDE, frequent commits, retro-fitting unit tests to their actual code base rather than just in the abstract.

See my "programming in the small" and "programming in the large" articles to get an idea of the sort of programming things that the course will include. (My PhD was in refactoring (completed 1996), I started using JUnit in 1998 and mock objects in 1999). Email me to book a course (and discuss course length - anything from one day upwards) - ivan at teamoptimization.com

Copyright © 2008 Ivan Moore

People over Process

2008-12-10T13:07:00.000-08:00

The Agile Manifesto values "Individuals and interactions over processes and tools".

I have heard a lot of people talk about "People over Process" somewhat differently than my interpretation.

In this article, I'll refer to a fictional developer called Chuck - nothing is implied by the name.

Chuck doesn't want to write tests

A frequent interpretation seems to be, if Chuck doesn't want to write tests (for example) then that's OK because people are more important than processes or tools. A lot of the time, people talk as if the question of whether Chuck is worth keeping in the team or not isn't a consideration - you just have to do the best you can with the team you've got; after all, people are more important than processes so that's OK (a non sequitur, but I've heard it said).

This can be a difficult one to argue against. Whether someone is any good at software development or not, they are "more important" than process, tools, software - but that is a different "important".

The worst sort of "Agile"

The worst examples of "Agile" software development are where people take the approach that as long as you do the currently accepted "Agile things" (or at least some easy subset like iterations/sprints, stand up meetings etc etc) and keep your current team (because "people are more important than processes or tools") then you are "Agile". This is the opposite of my interpretation of the agile manifesto - but quite common!

My interpretation of "People over Process"

The single most important thing is to get the right people in the team. Just making your existing team follow a set of processes (like iterations/sprints, stand up meetings etc etc) is much less important than having the right people in the team. I think that is what the Agile Manifesto is trying to say.

As Simon Baker says "Put the right people in the right environment and trust them to get things done."

The consequence of this is that you should hire the best developers you can (this article describes the best way to interview them). The less palitable consequence is that you should remove developers from a team if they are not good for the team. That doesn't necessarily mean removing them from the company - it may be that there is another role or team in which they would contribute more. You should also consider whether a developer can improve - my experience has been that pair programming really helps to develop developers.

So should Chuck be made to write tests (or whatever)?

I'm not sure this is the right question. Really the question should be "is Chuck any good for the team?". If Chuck doesn't want to write tests then I think it's unlikely (but not impossible) that Chuck is (what I consider to be) a good developer. It would probably be counter productive to "make" Chuck write tests, but I'd be happy to pair program with Chuck to show him how it works. If Chuck still doesn't want to write tests then I might prefer not to be on the same team as Chuck (one way or another).

Copyright © 2008 Ivan Moore

The problem with conventional continuous integration servers

2008-11-23T06:32:00.000-08:00

I recently presented a talk on continuous integration at Agile North.

There was one topic in the talk that I think is so important that I decided to write an article about it. Suprisingly, it's a topic that many users of continuous integration (CI) servers (and even the developers of them in some cases!) don't seem to have thought much about.

Continuous integration

This article assumes you already know what continuous integration is.

The problem with conventional continuous integration servers

Imagine there are three developers, Tom, Dick and Harriet, and a continuous integration server. There is some code in a source code repository and these three developers start with the code cleanly checked out.

Tom makes some changes and commits his changes.
The CI server starts running a build.
Harriet makes some changes and commits her changes.
Dick makes some changes and commits his changes.
The CI server reports that the build is OK.
The CI server starts running a build (because there are new changes for it to run the build on).
Tom makes some changes and commits his changes.
The CI server reports that the build is broken.

Here's a diagram representing that:
The question is - who broke the build? (Left as an exercise for the reader). The problem with conventional continuous integration servers is that they can't tell you. A situation like this is inevitable with the vast majority of continuous integration server installations (that actually exist - I know you could install your favourite CI server differently if you had enough money - read on ...).

BTW - Step 7 is a bit of a red herring. It is not necessary for the purpose of the main point of this article, but a common enough situation. Tom thinks he's committed on a green build, but really the build is already broken in terms of the committed code, just not in terms of what the CI server is saying.

Why it is a problem

The problem with not knowing which commit broke the build is that it takes longer to work out who should look at the problem and how they can fix it. If you know which commit broke the build you know who should look at it and they can review their changes to work out why it broke. Furthermore, if you know which commit broke the build (even if you can't work out why), you can revert that change set while the problem is fixed "off line" from the rest of the team.

I am convinced (from years of using CI servers on many teams) that not knowing which commit broke the build is a major contributor to sloppy CI practice - builds staying red for ages, nobody taking responsibility, reduction in commit frequency etc etc. Just knowing which test failed, or "why" the build broke isn't enough. The symptoms don't always tell you the cause. Knowing the cause - i.e. which commit - is what you need to know.

When do you get this problem?

You suffer this problem more as the team gets larger, commits become more frequent, and the length of the build goes up. (Oh, and if developers get sloppier).

You often can't do anything about the size of the team, you'd like to encourage people to commit more frequently and making the build faster can be really hard. (And you might be able to get rid of sloppy developers, but that's quite a different blog post). Ideally the build should be fast, but that is often easier said than done, and even if the build is fast, it doesn't completely eliminate the problem described in this article.

Solutions

Most continuous integration servers don't solve this problem, but some do. These are the solutions that I know about. The CI server can:

a) run the build for the commits (revisions) between the last known good and the first known bad (provided that there is enough capacity in the build farm, e.g. people stop committing while the build is broken).
b) check that the build passes (on a build agent) before committing changes.
c) run multiple (preferably all) commits in parallel on different build agents in a build farm.

build-o-matic does (a) automatically - using a binary search. TeamCity (version 4 EAP) and Pulse (and maybe others) allow running of a previous revision so you can do the equivalent manually (maybe someone will write a plug-in for TeamCity to do what build-o-matic does? There is a precedent).

TeamCity and Pulse do (b) (build-o-matic doesn't - it's a cool feature; I've used it in TeamCity and it works well but you do need a lot of build agents).

build-o-matic does (c). I think TeamCity and possibly some others will too if you have enough build agents. The problem with this is approach is that you really might need a lot of build agents, particularly if the team is large, commit frequently and the build is long.

My preferred solution is to buy enough build agents to do (c) - computers are very cheap. Note however that just because a CI server supports a build farm, even if the build farm is infinitely large it doesn't necessarily mean it'll run the build for all the commits - check your CI server documentation for details. I believe Bamboo has something up it's sleeve on this topic - but I'm not sure if it's public yet (I'll find out and add a comment as appropriate).

There is another solution which is not to use a CI server at all, but instead have a "build token" or an "integration machine" - i.e. serialize all commits, that way you never commit while the build is running. That only works well for small co-located teams with a fast build. But when conditions are suitable, it really works well!

Conclusion

There are lots of CI servers to choose from. I consider working out which commit broke the build as a basic minimum feature of a CI server installation but suprisingly not all CI servers are capable of telling you - and you really need to understand this before you choose a CI server that will leave you with a broken build and not knowing which commit caused it.

Copyright © 2008 Ivan Moore

2008-11-09T10:18:00.000-08:00

In my previous article I mentioned the book "Clean Code" - this article is a brief critique of it (in some cases you'll just have to read the book to see what I'm talking about because I'm not going to rewrite the book here!).

What I liked

The first chapter is excellent - particularly pages 4-6 (buy the book to find out what they contain). I got the same feeling of wanting to get everyone to read these pages as I had from the section about comments in Kent Beck's Smalltalk Best Practice Patterns. I wanted to shout (while shaking people by their lapels) "read this - it's what I've been trying to tell you all this time".

What I didn't like - examples (particularly Chapter 3)

There are some examples that aren't great. The one that stuck out particularly badly was in chapter 3 - HtmlUtil. The problem with HtmlUtil is that it is classic procedural style - which is OK as an example of bad style - but the refactored version does not address that aspect of it's badness.

This example is a method on HtmlUtil with signature:

public static String testableHtml(PageData pageData, boolean includeSuiteSetup) throws Exception

with lots of methods called on pageData. It gets nicely refactored into a short method of a different name but with the same signature.

My main problem with this example is that it looks (to someone who doesn't know the codebase that it is taken from) as if this should be a method on PageData. Maybe there is a good reason it is a static method on HtmlUtil but the author doesn't explain whether there is or not. To me it seems like an obvious question and by not mentioning anything about Utils with static methods that do things to objects that should be quite capable of doing stuff for themselves, it shows an example of arguably bad code, early on, with no explanation or apology for using such an example. I assume that they wanted to focus only on procedural refactoring for the example and deliberately wanted to avoid anything object-oriented - but if so, they should have said that very clearly.

The second red flag that this example waved at me was declaring that the method throws Exception. It isn't clear from reading the code that it needs to. Maybe it does, but again I think if it does then the author should have explained why and apologised for that aspect of the example.

What I didn't like - Structured Programming (Chapter 3)

In the (very short) section on Structured Programming there is a claim for the rules of structured programming; "It is only in larger functions that such rules provide significant benefit." I almost screamed at my fellow passengers on the train into work. My blood pressure is rising as I write this. (Imagine me screaming at the top of my voice - "NO NO NO").

I don't believe there is ANY benefit, let alone significant benefit, of applying the rules of structured programming to Java code. If there is, then the author should justify their comments rather than invoke the "proof by repeated assertion" that the cargo cult followers will no doubt spout in comments on this article.

What I liked - formatting (Chapter 5)

Lots of good stuff - I didn't agree with it all but some of it has definitely made me rethink (or in some cases just think) about what I'm doing with formatting more than before.

What I didn't like - Chapter 11 (Systems)

Any chapter which starts with an analogy for building a software system as building a city, has already got off to a bad start as far as I'm concerned. Whenever I hear of such analogies I wonder what Swiss Toni's analogy would be. "Building a system is very much like making love to a beautiful woman ..."

Apart from the analogy - the author then goes on to talk about Spring somewhat implying that it's a Good Thing. Just don't get me started. That's a whole other blog post.

Nevertheless, despite these things, there is also some good stuff in this chapter.

Overall

Great. Buy it. It's full of good stuff and only very few things that make my blood boil.

Copyright © 2008 Ivan Moore

Programming in the small - Exceptions

2008-11-08T08:56:00.000-08:00

It's a long time since I wrote my previous "programming in the small" article. This article is about Exceptions in Java. I've kept it very short just to cover the absolute minimum.

Handle or throw

What is likely to be wrong with this code?


    public void makeTea() {
        teaPot.prepare();
        try {
            kettle.boil();
        } catch (ElectricityCutOffException e) {
            LOGGER.log("Didn't pay bill; no tea today.");
        }
        kettle.pourContentsInto(teaPot);
    }

Logging that some exception has been thrown is not handling it. In this case, "pourContentsInto" will still be called even if "boil" threw an exception. The exception indicates a problem, and to keep executing will probably mean that the system is in an unknown, inconsistent or bad state.

In this case, I'll end up with cold water in the teaPot, ruining the tea in an unrecoverable way.

In many cases, catching an exception and not handling it causes bugs which are tediously difficult to track down because the system ends up in an inconsistent state and the exception that eventually causes the system to fail, or the error in its functionality, ends up being somewhere that looks completely unrelated to the code that caused the problem by hiding the exception.

If this method is a sensible place to fully handle the exception, then it should do that. For example:


    public void makeTea() {
        teaPot.prepare();
        try {
            kettle.boil();
            kettle.pourContentsInto(teaPot);
        } catch (ElectricityCutOffException e) {
            butler.sendToTeaShop();
        }
    }

If it can't handle the exception (I don't have a butler), then the best thing is to just let the exception percolate up to the calling code to either handle or throw, that is:


    public void makeTea() throws ElectricityCutOffException {
        teaPot.prepare();
        kettle.boil();
        kettle.pourContentsInto(teaPot);
    }

Now calling code has to either handle or throw the exception.

Do NOT declare an exception that a method does not throw

If you declare that a method throws a checked exception that it cannot actually throw, then you are condemning callers to having to handle or throw an exception that cannot happen - percolating unnecessary try/catch code around the system. Don't do it.

For a method which implements or overrides a method of an interface or superclass which declares that it throws an exception, do not make it also declare that it throws that exception unless it actually does.

Unchecked exceptions and more

I have recently read "Clean Code" - I really like the first chapter - worth buying just for that. There is a chapter on exceptions, in which Michael Feathers writes about the use of unchecked exceptions and other things. I'm not going to write more myself - buy "Clean Code" instead.

If you are too cheap to buy "Clean Code" or have read it and want a different opinion, then read my "programming in the small" articles which cover some of the same ground (in some cases better, in some cases worse, and not a complete intersection of topics - in particular there is more stuff covered
in "Clean Code").

Copyright © 2008 Ivan Moore

Why are NOJOs so popular?

2008-11-02T03:48:00.000-08:00

Following on from my previous articles on NOJOs and their frequent complements, Utils classes, I have talked to colleagues about why NOJOs are so popular in enterprise Java development.

Here I will try to write up some of the ideas we discussed (thanks to Mike Hill, Nat Pryce, Pippa Newbold, Rob Dupuis and Tung Mac).

Education by frameworks

A lot of the early examples in the Spring documentation are NOJOs. For example, on the page that introduces The IoC container there are several (here's one):


package examples;

public class ExampleBean {

  // No. of years to the calculate the Ultimate Answer
  private int years;

  // The Answer to Life, the Universe, and Everything
  private String ultimateAnswer;

  public ExampleBean(int years, String ultimateAnswer) {
      this.years = years;
      this.ultimateAnswer = ultimateAnswer;
  }
}

Similarly, have a look at the JavaBeans on wikipedia - PersonBean.java is a NOJO. I think that there are a lot of developers who think that JavaBeans are NOJOs - as indicated by one of the comments on my NOJO article.

Fear of using the framework incorrectly (the framework won't like it)

It is possible that developers think that the early examples are the "correct" way to use such frameworks - and are worried that if they add methods to their NOJO then the framework will do something peculiar. This isn't entirely unreasonable as sometimes such frameworks do unexpected things due to the amount of behind-the-scenes-magic going on.

Fear of using the framework incorrectly (my colleagues won't like it)

Another possible fear is that that the early examples are the "correct" way to use such frameworks - and doing anything else is not how the framework is intended to be used (even though the framework seems to still work).

Separation of concerns

Another fear is that of putting the behaviour in the wrong place - in particular in enterprise Java projects, I think there is a perception that there is no worse "crime" than putting behaviour in the wrong "layer" (or the wrong sort of class). Therefore, rather than risk putting the behaviour on the wrong object in the right layer, enterprise Java developers will choose to put the behaviour on the wrong class in the right layer. Related to this is a desire to stick to some prescribed pattern - e.g. DTOs. I have been in a situation when pair programming with someone where I was told, "you can't add a method to that" followed by some lame reasoning justified by some pattern that they wanted to stick to religiously.

Afraid of "new"

Yet another fear is that you might have to create a new object (shock, horror!). To avoid that, maybe some developers prefer to write static methods on some Utils class so they don't need to create any objects?

Object-oriented programming education and thinking

Perhaps the popularity of NOJOs is just the manifestation of how developers (don't) learn object-oriented programming? Perhaps object-oriented programming simply doesn't suit how all developers think? Certainly, I've come across very good developers who prefer functional programming and don't really "get" object-oriented programming, so it's not meant to be a criticism (or at least, not in all cases). Many developers have learnt their programming from non object-oriented programming languages, like C. Perhaps it's not suprising that to a C programmer an object looks like a struct?

Intra-team APIs

Another source of NOJO programming is that the APIs that people design for communicating between team's subsystems often involve setting up, sending and receiving NOJOs.

Automated Testing

Another possibility is that developers who are not used to TDD find writing tests that use NOJOs easier than, for example, using Mock Objects.

UML

Using UML to "design" a system up front encourages thinking about the fields of objects rather than their behaviour - because that's what is easiest in notions like UML.

IDEs

Java IDEs can do lots of good things. One of the arguably less good things they do is generate getters and setters if you want. It is possible that the easy generation of getters and setters encourages their use, leading to NOJOs rather than objects that do things with their own fields. Using setter injection (probably the most common way people use Spring) also tempts developers into generating the getters too - after all, it's probably an extra click to not generate getters and what harm can some getters do? (Rhetorical).

Should I care that NOJOs are popular? Should I do anything about it?

Left as an exercise for the reader.

Copyright © 2008 Ivan Moore

Utils classes and NOJOs

2008-10-24T14:31:00.000-07:00

A style of class that is a frequent complement to NOJOs (introduced in my previous article) are classes with only static methods. They might be called SomethingUtil or SomethingHelper.

People will often know what you mean if you say "a Utils" class, so I won't propose anything different here. I had wanted to propose a catchier name - SOJOs (Static Only Java Object) but the acronym (with a completely different meaning) is already taken by the SOJO project. In a comment on my NOJO article, John Q Public said he uses the term "function buckets" and I'm sure he's talking about the same thing.

Why do people write Utils classes?

If you have NOJOs then your methods have got to go somewhere. People who prefer the NOJO style often like to create static methods on Utils classes for those methods.

Are Utils classes a Good Thing?

Again, left as an exercise for the reader. An article from my old blog might help to hint about my views (although it is not specifically about this subject) without me having to stay up all night writing more. It seems like people aren't shy about expressing their opinions in comments - some of which I completely agree with, so I'll leave it as an exercise for comment writers as well as the reader (cheeky, I know).

In defence of my previous article

I didn't make any judgements in my previous article - mostly through lack of time but also because I want the term NOJO to be understood without being confused with whether it's any good or not. I am merely making an observation and suggesting a name for a thing I see frequently, that is poorly served by current names. I found it interesting how many people suggested that there was already a term for NOJO but none of their suggestions were entirely accurate.

Copyright © 2008 Ivan Moore

Is that a POJO or a NOJO?

2008-10-23T14:10:00.000-07:00

The term "POJO" is widely used in the Java programming world, but is sometimes used to mean something more specific than what was originally intended.

Furthermore, such code (whether called POJO or not by its authors) is a style which I think would be useful to give a specific name to.

What's a POJO?

The original meaning of POJO was a plain old Java object - that is, an object that isn't tied to a framework (for example, having to implement specific interfaces). A concrete example of a non-POJO would be entity beans (from EJB).

Introducing NOJOs

NOJOs are (instances of) classes which define fields, setters and getters but no other methods. They are popular in enterprise Java code and I think deserve a specific term, to distinguish them from the rather general term POJO.

NOJO stands for "Non Object-oriented Java Object". Although it might sound "negative" it is merely intended to be accurate and have a catchy acronym.

What defines an object?

An "object" (in object-oriented programming) has identity, state and behaviour. A NOJO has identity and state. A function has behaviour but not state. An object has identity, state and behaviour.

Why do you need a term for NOJO?

In many programming languages, there is a language construct for NOJOs (or something very similar) - e.g. "struct" in C, or "record" in Pascal. In Java there isn't an equivalent language construct - so the term NOJO is intended to mean the use of a Java class to implement such a thing, to make it easier to talk about such code.

Are NOJOs a Good Thing?

Left as an exercise for the reader.

Blog moved here

2008-10-12T06:49:00.000-07:00

Joe Walnes, the best developer I've ever worked with, got me started blogging (many thanks Joe) by hosting my blog on his server. Now the time has come for him to retire his server, and so my blog has moved here. I'll be writing more articles soon - I've got a backlog of programming in the small ideas to write up - but have been very busy with work, being (co) programme chair of my favourite conference (SPA), build-o-matic and just trying to keep up with the world.