How to Freaking Find Great Developers By Having Them Read Code

When hiring developers, there are many things we are looking for, but over the years I have found that raw coding ability is easily the most important quality to look for.   I can quickly train a person to have knowledge in some domain, but I’ve never seen raw coding ability come from anything other than personal commitment to extensive and deep practice.   Because of this, I have found that some methods work better than others to discover talent.

The Old Way

A typical coding interview starts out something like this:  “Write a function that reverses letter order of words in a string.”  Then for the next half hour or more, the candidate scratches out something on the white board (or a shared text document if they are lucky).    This approach is weak for a number of reasons:

  • The same questions get passed around and candidates often practice very hard to memorize answers.   Are you testing their skill or their ability to memorize answers?
  • The problems are often “trick” problems, requiring some kind of deep insight to come up with the O(log(n)) solution.   True insight almost never comes to even the smartest candidates in the timeframe of an interview.
  • It tilts the power balance in favor of the interviewer.  Who likes having to awkwardly produce code in front of a judge who will determine your professional prospects for the next several years?
  • Writing code on a white board or even a text document is unnatural and slow.  Nobody sketches code on a white board or notepad as part of their daily work.   What people really do is sketch code in an IDE with a heavy dose of Google on the side.

A Better Way

So instead of writing code, consider instead having the candidate read existing code and talk about what it does and how it works. This offers some powerful advantages:

  1. Reading probes the most fundamental skills.   Reading code is probably 95% of what a developer does as part of their job.  Whether a developer is writing new code, fixing bugs, or creating documentation, they are constantly reading. What abilities does a coder need to read code well?   There are two important ones: 1)  The ability to remember variables and stack locations and 2) The ability to generalize a piece of code once they understand it.   I can memorize interview coding questions, but I can’t prepare for being plopped into some random code (other than just writing and reading code all the time). It is essentially impossible to fake these skills.
  2. Reading code is way more efficient than writing.  A candidate can tell you a lot about their programming skill in the first five minutes of reading because reading is easily an order of magnitude faster than writing. In a reading interview, I can skip around half a dozen essential topics in the time it would take the same person to write code to reverse characters in a string.
  3. Reading puts candidates at ease compared to writing code.  As an interviewer, stress is your enemy because it raises adrenaline which lowers IQ by several points, causing you to miss good candidates.   Candidates prefer reading partly because they are relieved to not have to write code, but also because the interviewer can easily adjust the reading questions to accommodate for the candidate’s skill.   (This adjustment can include writing code if it seems like the candidate wants to.  )

How I do this in practice

Here’s some practical points on how I actually do this in interviews:  

  • For each fresh interview cycle, I create a set of predict-the-output exercises that start really easy, then get harder.  My current set starts off with a basic function call, then multi-level function calls, then recursion, then side-effects.  These are generally “pretend” functions that are meant to give the candidate some quick success and give me some clues on how the rest of the interview will go.   For more advanced questions, I pull code from stuff I’ve written.   My current “hard” questions explore ability to make abstractions while reading and also following asynchronous operations.   (Some other things that make good reading are unlabeled routines that execute well-known algorithms such as sorts or tree traversals, and finding bugs from error output.)
  • Before I ask my questions to candidates, I calibrate my questions against people I work with so I have a realistic expectation for how to gauge the skill of the candidates.  Calibrating the questions also helps me refine them and weed out confusing parts.
  • At the start of the interview, I explain:
    • I’m NOT testing knowledge of syntax.  Treat me like an AI-enabled google and I will just tell you what some function or operator does.
    • I don’t expect you to finish because nobody does. We will stop after 20 minutes.
    • I don’t expect you to get answers correct.  If the answer is wrong, I would love to see you go back and debug your thinking.  This is just as valuable to me as anything else.
  • The test goes like this: 
    • I show a commented line of code that will call some function and return an output. 
    • The candidate reads the code and predicts the output
    • I uncomment the line and run the program so they can see the answer.   
    • If the answer is different than their prediction, they go back and explain why. 
  • I give the candidate 20 minutes to get as far as they can.  This gives me extra time to ask follow-on questions.    In the interview report, I write how far they got and what strengths and weaknesses they showed.  

Obviously, these special coding skills are not the only thing to probe for in interviews.   Domain knowledge and cultural fit are important, however I find that reading does a great job weeding out candidates who do not fit where it counts the most.

Some of you might want to know how to develop your skills to do well in such an interview.   My answer is simple:  write lots and lots of code because there is no substitute for regular practice. How to get practiced?  The easiest way is to crank up some non-trivial side projects that you care about.   A game, a web site, an app, … whatever.    Target 4-8 hours per week of getting into code you care about and make it something you like to use and are proud to show off.    (Also, it helps to host it somewhere and put the source on github so that future employers can see your activity and get a sense of how you work. ) 

Hope this helps

23 Comments

  1. I like this idea a lot. I wouldn’t do it exclusively, but I think it would be a great initial technical screener, something we currently waste too much time on for weeding out weak candidates.

    I don’t think the title of this article is very helpful. I wanted to share it but I had to explain what it was about since the title doesn’t hint at the concept.

    You also have some typos. “Wether”, “The ability {to} remember”, “to accommodate for {the} candidate’s skill”, “My current “hard” questions explore {their} ability”, “codeing”,

    Like

    Reply

  2. As a highschool senior, this kind of interview seems an order of magnitude more fun than writing a series of functions out of context. I’d [freaking] love to talk about a bunch of code than get written off for one function that required me to memorize a specific Googleable fact.

    Like

    Reply

    1. IMO, interviews should be fun, at least to some degree. The best interviews are where we have a discussion and both learn something from the interaction. This isn’t supposed to be a test in school, real development work involves discussion & collaboration as well.

      Like

      Reply

  3. I wholeheartedly agree. The reality is that all programming jobs require much more reading of code than writing of code. Being able to think clearly and get your head around somebody else’s bullshit is essential. Interviewing someone by having them reverse a string or sort arrays is testing for the wrong capabilities. I don’t care if you have seen this question before and memorized the answer as many algorithmic tests probe for — the only thing that matters is if you can actually think, understand problems, design solutions, and implement them. Being able to read real-world code and say something about its strengths and weaknesses tells me much more about your capabilities than crushing a bubble sort exercise.

    Like

    Reply

  4. There is one additional benefit to reading code for the interview. When you have a 40-60 minute coding challenge, you are really just selecting for the fastest typist.

    Like you mentioned, typing code is the least important part of the job. Reading, understanding, and problem solving are all much more important to select for.

    As someone with a degenerative medical condition, my typing is slow and getting slower. Often I run out of time in code challenges. Not because I don’t have a solution in my head, but because I am just up against the clock.

    The browser based coding environments make the situation worse. On the job I have my editor tuned with auto-complete suggestions, scripts, macros, snippets, etc. In code challenges you are often using something like CoderPad, which has non of the aforementioned tuning — and if from muscle memory you use your typical editor commands, you might close the browser window or something. Eating up more time.

    As an aside, yes I can request accommodation. That is a whole other minefield to navigate while on the job hunt.

    Liked by 1 person

    Reply

  5. This is an awesome insight . I hope the FAANGs and the sort see it . I cannot agree more with you . If someone reading this has really coded in real life , he /she cannot miss the point

    Like

    Reply

  6. You identify a lot of the problems with the common approaches today. And you identify the better approach in general. I’d suggest it’s even better to have them share their own code and go through it like a code review. Or maybe a combination of both yours and that. You get insight into how they structure things in real world (or real world-ish) solutions that way, in addition to on the fly code reading.

    Like

    Reply

    1. I know a company that does this. They ask the recruits to send in their most robust projects and explain certain parts of it.

      Like

      Reply

  7. I have found that reading code and writing code can be separate skills. I worked with a developer once who was very talented at designing and implementing solutions. I saw him encounter some complex undocumented code with a minor bug. He was baffled by it and rather than working to understand it he re-wrote it. His end solution was virtually identical to the original except with the bug fixed. The bugfix would have been a one liner. I’ve been pondering this in my interview process, I think you suggest a good technique to prod both skills.

    Liked by 1 person

    Reply

  8. I agree with the premise that finding people who can read code well is important, but I disagree with a few key elements. By making the coding interview purely about reading code and trying to decipher what the code is doing, and doing so in a very short amount of time, you’ll exclude some very good candidates.

    Interviews are stressful, and not a normal work life situation. So they’re already in not an ideal headspace walking into the situation. In every day working life, when you’re trying to analyze code, there usually isn’t someone staring at you in the same room, listening to you as you’re expected to verbalize as your process, and judging you as you, nor is there a short time limit. Then there are folks who look at code differently than what you’re presenting to them. Some people learn a lot more about the code by following code paths using tools like debuggers/profilers, and some learn more about the code by playing with it more than just looking at the characters on the screen.

    And in general, I personally find soft skills are more critical to a good candidate than their raw coding skills. Coding skills can be easier to teach, grow, and mentor than soft skills. But if someone can’t communicate, or communicates poorly, lacks empathy, emotional intelligence, contextual awareness, etc. then they’ll be a worse candidate than someone who can do all those things well, but isn’t as strong in hard skills.

    Liked by 1 person

    Reply

    1. I agree that the interview should not be 100% code reading. As you say, the soft skills are also important. I think the discussion you give about the human factors of what makes a person feel stressed in an interview is super productive. Any stress induction in the interview process hampers the ability to find good candidates.

      Like

      Reply

    2. I think that’s also the point of starting on easier exercises like OP suggested. That’s also where the soft skills of the interviewer comes into play.

      Like

      Reply

  9. As other people said in the comments, I wouldn’t replace a coding exercise with a reading exercise as you evaluate different skills at the end. Doing both is interesting I think.
    I think you could even go further with the reading exercise by evaluating code reviewing skills. How the candidate would formulate some code to improve? What if the developer don’t want to change it ? For a given piece of code with various things to improve, what will the candidate notice first ? What did he miss and why ?…

    Regarding the arguments you highlights in the “old way” of doing coding interviews, they can be fixed easily while still writing code. I’ve written an article about that here : https://medium.com/p/17038af3378d

    Like

    Reply

  10. Thanks for the inspirational read!
    For my technical engineering interviews I ask candidates to review a short fictional pull request, leaving comments and suggesting changes (they get 20 minutes to do so). The code changes in the pull request are quite simple, but contain a variety of flaws (structural, inconsistencies, performance issues, …). I don’t expect anybody to point them all out, but it tells me a lot regarding framework/language knowledge, attention to detail etc.
    As a bonus you get an idea of the candidate’s written communication style…
    The creation of the pull request is mostly automated via a git .diff-file and it just takes me a minute to set it up before each interview, I candidates use their on GH account and I grant/revoke access to the repository directly via the interview.

    Liked by 2 people

    Reply

  11. Interesting post! I really think most software engineering interviews are broken (among other reasons, because, as you pointed out, they tend to fail to get the signals they are supposedly looking for), but I have never thought of this approach before.

    However, similarly, I remember having interviewed once for a company not having to write a single line of code. Instead, for one of the assignments I was given a piece of code and I had to perform a (simulated) PR review. Even if I failed, I had a very positive experience. I think you can get many good patterns from a candidate with these approaches (e.g. ability to understand the code, thinking/debugging processes, communication).

    Thank you for sharing!

    Like

    Reply

Leave a Reply to Long Long Man Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s