Saturday, October 31, 2009

Pair Programming interviews/auditions

I have done a lot of pair programming interviews for my client. I enjoy doing them and I think they are extremely valuable.

Hiring is probably the most important thing to get right, and I think that pair programming interviews (or "auditions" as some like to call them) are usually the best way to interview developers.

Setting up a pair programming interview

For interviewing Java developers, I have a machine set up with a choice of IntelliJ IDEA and Eclipse with an empty workspace. I think it is important to try to make the candidate as comfortable as possible with the development environment so that it isn't a distraction. If you work with Apple Macs, also provide Windows (and maybe Linux) machines for candidates who aren't familiar with Macs because otherwise the difference in IDE shortcuts gets in the way and the idea of a pair programming interview isn't to see how familiar the candidate is with a particular OS.

I allow one hour for these interviews - I have found that is long enough to get a good idea about the suitability of a candidate.

Choosing a problem

One of the most surprising things is how small the problem has to be in order to be achievable in one hour. Developers (including myself) are often very optimistic about how quickly they can program a solution to a simple problem. I would urge you to try out the problem you want to use for a pair programming interview with a trusted colleague, before using it with a candidate, to find out whether it is realistic. To give you an idea of the size of problem you need, I used to use the problem described in this article.

I know of another team at my client who provide a small code base pre-prepared for the interview which includes a bug which the candidate has to find. I think this is a great idea. Choose a problem which is realistic for the sort of code that the candidate will be writing (or the sort of work they will be doing, e.g. debugging legacy code) in the job you are hiring for. There is no point testing for whether a candidate can write WeakHashMap from scratch if you are hiring for a typical enterprise IT project.

What pair programming interviews demonstrate

Pair programming interviews aren't a silver bullet for recruiting developers - they are good at determining some qualities of a candidate but not all. In particular I think they are more valuable in demonstrating programming style and personal interaction rather than problem solving. However, in typical enterprise IT projects, more problems are caused by over complex solutions to simple problems, or by solving the wrong problem, than having problems that are unique and difficult and require a brand new algorithm.

Sometimes it is good to use a problem which has some potential for ambiguities in the details of what is being asked for. This is a very good way to see whether a candidate asks for clarification or just makes assumptions which aren't justified. In enterprise IT systems, making assumptions about the desired behaviour of a system can cause a lot of rework and bugs so I look for candidates who demonstrate that they are thinking through possible ambiguities and who pro-actively ask for clarifications about the problem.

Being realistic

I make it very clear to candidates that I am looking for production quality, simple code rather than for them to show off all the language features they know. The problems that I choose are simple enough that I'm not interested in just getting a solution. I am looking for code that I would be happy working with. My boss, Pippa Newbold, has a very good rule of thumb. "At the very least, don't hire someone who will make the code base worse". It seems obvious and yet at other companies I have sometimes observed panic hiring just to make up the numbers where this rule would have saved lots of rework and bugs.

Giving something back

I try to teach the candidates something new in a pair programming interview where possible. Often this is something like an IDE shortcut but is occasionally a language feature or a "programming in the small" style discussion. I like candidates to get something back for giving up their time to come in for an interview, and a pair programming interview can be very daunting for candidates not used to pair programming.

Does it work?

As far as I can tell, all candidates that I have "passed" using a pair programming interview have turned out to be worth hiring.

However, that doesn't include those candidates that I've "passed" who didn't take the job and I don't know whether any of those candidates that I've "failed" would have been good either.

My suspicion is that it is a technique which is slightly more prone to "failing" good candidates than hiring poor candidates, but that really is just a gut feeling. If anyone knows any studies on pair programming interviews I'd be very interested to hear more.

Where it probably doesn't work

I guess that interviewing people for research or work which requires solving deep and difficult problems probably requires a different approach. Also, for hiring people with limited programming experience who you want to train up it might not be suitable.

Copyright © 2009 Ivan Moore


Unknown said...

It is a technique thats has worked well for us over the last 6 years in 3 large companies. Invaluable.

Anonymous said...

You discuss the interviewee at some length, but don't discuss the interviewer at all.

If you don't have the right interviewer, you won't get the right (impression) from the interviewee. This would be even more important in your 'paried interview' scenario.

Robin said...

Interesting blog, although I would beware judging the programmer on unimportant parts of their programming style instead of focussing on their ability to actually produce something useful and robust.

Regarding that blog post about programming small - it seems to be encouraging bad practice all over the place:
- First, when there's a fairly short one-line syntax for setting a default if the variable doesn't exist, why would you write and introduce a dependency on a bespoke 3rd party method to do that for you, and then, worst of all, place that method inside a pathological liar of a Utility singleton? Euch.
- Second, IMHO you should never ever encourage people to conserve white space. Are we paying by the character? More white space is almost always better for readability.

The rest of it I largely agree with - always using {}s in conditionals, removing ifs where possible, using closures to help with reducing duplication - all good.

Sidharth said...

I have been interviewing for some time. I have been invited to interviews where there was no mention of any kind of pair programming test. then the code is directly projected on to a large TV and 3 very senior interviewers watching you code. I think this is a very bad way to interview people. A lot of the candidates will freek out when nothing has been especially mentioned and you walk in to an interview and they ask you to code right there... looks like an ambush to trap people or the company is trying to prove to people how great and how twisted they can make the interview process. I have gone on to interviews where a very smart person has taken a decision to hire/no hire with in 20 minutes. I think this makes things very complicated. Give me at least 1 question which cannot be talked about/discussed/ solved on a pencil paper and found out instead of an interview and Ill agree to our point of view.

ropericu said...

this is surely a great way to test candidates. however, it is pretty time-consuming. what if you have 100 potential candidates? how will you filter the best ones?
too much time, IMO :)
for those who value their time too much I'd recommend a recruitment testing software, eg.
they generate the questions (with coding challanges) and send the link to all your candidates. you just wait till they finish and pick those who got the best scores

Michael said...

Really interesting way to hire developers. Only the one problem is that you need to spend a lot of time on live interview the candidates. If you want the hiring process to be easy, just try to use programming tests as initial filter. After you have top 2-4 candidates who pass the test, invite them to interview, and in this case pair programming interview will be great!