I have done a lot of pair programming interviews for my client. I enjoy doing them and I think they are extremely valuable.
Hiring is probably the most important thing to get right, and I think that pair programming interviews (or "auditions" as some like to call them) are usually the best way to interview developers.
Setting up a pair programming interview
For interviewing Java developers, I have a machine set up with a choice of IntelliJ IDEA and Eclipse with an empty workspace. I think it is important to try to make the candidate as comfortable as possible with the development environment so that it isn't a distraction. If you work with Apple Macs, also provide Windows (and maybe Linux) machines for candidates who aren't familiar with Macs because otherwise the difference in IDE shortcuts gets in the way and the idea of a pair programming interview isn't to see how familiar the candidate is with a particular OS.
I allow one hour for these interviews - I have found that is long enough to get a good idea about the suitability of a candidate.
Choosing a problem
One of the most surprising things is how small the problem has to be in order to be achievable in one hour. Developers (including myself) are often very optimistic about how quickly they can program a solution to a simple problem. I would urge you to try out the problem you want to use for a pair programming interview with a trusted colleague, before using it with a candidate, to find out whether it is realistic. To give you an idea of the size of problem you need, I used to use the problem described in this article.
I know of another team at my client who provide a small code base pre-prepared for the interview which includes a bug which the candidate has to find. I think this is a great idea. Choose a problem which is realistic for the sort of code that the candidate will be writing (or the sort of work they will be doing, e.g. debugging legacy code) in the job you are hiring for. There is no point testing for whether a candidate can write WeakHashMap from scratch if you are hiring for a typical enterprise IT project.
What pair programming interviews demonstrate
Pair programming interviews aren't a silver bullet for recruiting developers - they are good at determining some qualities of a candidate but not all. In particular I think they are more valuable in demonstrating programming style and personal interaction rather than problem solving. However, in typical enterprise IT projects, more problems are caused by over complex solutions to simple problems, or by solving the wrong problem, than having problems that are unique and difficult and require a brand new algorithm.
Sometimes it is good to use a problem which has some potential for ambiguities in the details of what is being asked for. This is a very good way to see whether a candidate asks for clarification or just makes assumptions which aren't justified. In enterprise IT systems, making assumptions about the desired behaviour of a system can cause a lot of rework and bugs so I look for candidates who demonstrate that they are thinking through possible ambiguities and who pro-actively ask for clarifications about the problem.
I make it very clear to candidates that I am looking for production quality, simple code rather than for them to show off all the language features they know. The problems that I choose are simple enough that I'm not interested in just getting a solution. I am looking for code that I would be happy working with. My boss, Pippa Newbold, has a very good rule of thumb. "At the very least, don't hire someone who will make the code base worse". It seems obvious and yet at other companies I have sometimes observed panic hiring just to make up the numbers where this rule would have saved lots of rework and bugs.
Giving something back
I try to teach the candidates something new in a pair programming interview where possible. Often this is something like an IDE shortcut but is occasionally a language feature or a "programming in the small" style discussion. I like candidates to get something back for giving up their time to come in for an interview, and a pair programming interview can be very daunting for candidates not used to pair programming.
Does it work?
As far as I can tell, all candidates that I have "passed" using a pair programming interview have turned out to be worth hiring.
However, that doesn't include those candidates that I've "passed" who didn't take the job and I don't know whether any of those candidates that I've "failed" would have been good either.
My suspicion is that it is a technique which is slightly more prone to "failing" good candidates than hiring poor candidates, but that really is just a gut feeling. If anyone knows any studies on pair programming interviews I'd be very interested to hear more.
Where it probably doesn't work
I guess that interviewing people for research or work which requires solving deep and difficult problems probably requires a different approach. Also, for hiring people with limited programming experience who you want to train up it might not be suitable.
Copyright © 2009 Ivan Moore