Interview Testing

A couple of months ago I was mentally reviewing some of the interviews I had given, and my not so stellar performance in finding real talent. It's difficult to determine if someone will be a good fit for the job just by talking to them. I look over their resume, which may have listed certifications and diplomas, but in some cases that didn't amount to much. I could have asked a set of technical questions, but if they are Java certified, it is likely that they would know the answer. I then thought about how Goolge had a Code Jam recently. In Code Jam the participants are given problems of logic, and then required to write the code to solve the problems. I figured that this was something I wanted to try.

The test I created, and now give to potential hires, consists of implementing two interfaces in Java. The first is a maze object, which has methods to initialize and move around the maze. The methods for movement (moveUp, moveDown, moveLeft, moveRight) return a boolean value, true if the move succeeded, false if the path is blocked. This part of the problem is fairly straight-forward, it only needs to store the maze data and store the current position.

The second interface is the maze solver. This interface has a single method, solveMaze(), which takes a maze object and uses the movement methods to find the end of the maze.

Thus far we have given this as a take home exam, putting no restrictions on how it might be solved. We specifically state that there is no time limit and that the Internet may be used as a resource. We also explicitly state that we will not be running the code, so it is ok if there are syntax errors or flawed logic. Our goal is to get a little insight into their thought process which is done by reading their code, not running it.

When we created the test we started passing it around the office to see how well we coul do. A few of us would have little impromptu meetings to go over someone's solution and discuss it. We found that there were many ways to solve this problem, and most of them were flawed in some manner. One used recursion, but would crash if the maze was too big. One used a complex algorythm, that seemed to stall if the maze was too big. One was very simple, but ended up going around in a circle when presented with a open field. Then there were others that were just plain flawed, and wouldn't solve much of anything.

I found that an important part of the test was the discussion I would have with the participants afterward. If the solution was flawed, I would point it out to them. Some of them verbally offered some suggestions on how they could fix their code. This was something I saw as a positive trait, where someone is being being criticized and offers solutions on how they might fix the problem. The answer I didn't want to hear was their affirmation of the problem, and not propose a fix.

It is also nice to see the outside-the-box thinking. For example, I am impressed when a participant not only provides me with a solution, but also provides me with a test application as well. I have also liked the way the some participants modified the provided Java interface to allow for the code to be cleaner. JavaDocs and an explaination of the solution were also nice to see when provided.

There are also some cases where you look at the code, then look at the resume again, then look at the code again, and scratch your head. When someone says that they are a J2EE programmer, or Sun Certified, you normally expect them to be able to use the language well. This sort problem isn't something that will typically come out in the face-to-face interview, especially if they look over-qualified on paper.

In short, we found that the test was more than just "solving the problem". It allowed us to role play with them, were we played the customer, and they played the vendor. In a small way we replicated the cycle of presenting a problem, presenting a solution, evaluating the solution, and altering the solution as required. This is very close to how the my organization deals with customers, and it allows us to learn a lot more about the interviewee by playing the customer role.


Recently I came across Code Kata, a set of logic problems from Dave Thomas. If you are interested in using testing as part of the interview process, these problems are a good place to start, either to be used as questions, or just to inspire some thought.