My team is always looking for ways to improve the interview process. We want to avoid the perils that come with whiteboard coding but still evaluate the candidate’s coding skills. Usually during the process, we come away with a clear “heck yeah!” or “no”. But occasionally we just can’t tell. After reading The One Method I’ve Used to Eliminate Bad Tech Hires, we decided to add a paid, take home exercise as part of the interview for these candidates.

We added this step last in the interview process. It seemed fairest to answer all the candidate’s questions about the company and the position before asking them to spend four hours (albeit paid) writing some code for us. Rather than trying to reinvent the wheel, we used the exercise from the article:

Create a single page app that lets me enter movies in my home movie collection, store them in offline storage and search through them. I want to search by Genre, Title & Actors.

We also made it clear to the candidate that we didn’t want them to force themselves to use the programming language, frameworks, or libraries that we do. We wanted to see their best work made with the tools they were comfortable with.

And for the first full year after we added this to our interview process, no one was able to complete the exercise and ultimately get hired. In most situations, they simply didn’t implement the functionality as requested. Other folks didn’t follow the technical requirements. One person went so far as to tell us that they couldn’t get their environment set up in four hours to even begin. We were starting to worry that the exercise was more flipism than anything else — we already knew we didn’t want to hire the candidate and were just using the exercise to back that up.

Then someone completed it.

Not only did they meet all the functional requirements, but they completed the “offline storage” technical piece in a way that it would be easy to refactor to use a database later. We hired them and they’ve quickly became a strong contributor to the team. They later asked what was the “trick” in the exercise because it seemed so simple.

There’s no trick. As I’ve previously discussed, some people have degrees or jobs in programming but simply can’t.