Technical Interviewing: Be Better

Technical interviewing is hard, but that’s no excuse for it to be so terrible. Since candidates generally don’t have much leverage to negotiate the interview process, it’s incumbent on employers and hiring managers to improve the fairness, effectiveness, and efficiency of the interview process.

If you’re involved in your organization’s technical interview process, here are five things you can do today to make it better:

• Establish clear job requirements. The interview process is supposed to determine whether a candidate meets the job requirements. Hence, clear job requirements are a prerequisite for an effective interview process. Articulating a handful of clear job requirements (e.g., fluency in Python, ability to use embeddings for text applications, etc.) requires time and effort, but it’s worth the return on that investment. You shouldn’t even evaluate a resume until you’ve established clear job requirements. And you should enumerate those requirements in the job description, so that candidates know upfront what is expected of them.
• Formalize your evaluation rubrics. As long as humans do interviewing, we’ll bring our own biases to the process. But one of the best things we can do to mitigate those biases is to formalize the rubrics we use for evaluating candidates. For example, we might evaluate Python fluency based on a combination of code correctness, code readability, and the use of standard Pythonic conventions. This breakdown might feel like replacing one subjective evaluation with another. But narrowing the scope of evaluation can help to reduce that subjectivity.
• Create a bank of calibrated interview questions. Now that you have clear job requirements and evaluation rubrics, you can design — or borrow — suitable interview questions. For each question, enumerate examples of good, acceptable, and unacceptable answers. For example, if you’re evaluating the ability to solve an algorithmic problem (like this one) efficiently, you might evaluate an exponential time solution as unacceptable, a polynomial time solution as acceptable, and a provably optimal solution as good. Make sure to test these problems on your own coworkers as part of calibrating them! If your interview problems are too hard for your coworkers, they’re surely too hard for candidates who have to solve them under more stressful conditions.
• Train your interviewers! Great software engineers can be terrible interviewers. After all, they’re hired for their engineering skills, not for their interviewing skills. You need to train interviewers, which generally includes having new interviewers shadow experienced ones and vice versa. Training is also critical to make sure that your interviewers know what they can and can’t legally or ethically ask candidates — something your HR and legal departments will appreciate.
• Respect the candidate’s time. You may not enjoy interviewing, but at least you’re getting paid for it. For candidates, the interview process, on top of all of its stress and unpleasantness, is unpaid labor. So don’t waste a candidate’s time with an inefficient process. Streamline the number of interviews to the minimum needed to evaluate all of your rubrics. Short-circuit the process if it’s clear the candidate isn’t going to make it through. But be human about it — don’t schedule a full day of interviews and send the candidate home early. And don’t ask candidates to invest hours in a take-home assignment — at least not until demonstrably making a similar investment as an employer.

Finally, be careful about using automated interview tools. For all of the exciting developments in AI, we don’t have tools that can replace human interviewers. Automated at-home coding tests may be suitable for testing basic fluency, but they impose a lot of overhead and return binary evaluation; whereas an interviewer would be able to make a more granular evaluation as well as interacting with the candidate by answering questions, offering hints, etc.

Technical interviewing is hard. But it can and should be better. Please be part of the solution.

Full disclosure: I am an advisor to Karat, a company devoted to improving the effectiveness, efficiency, and fairness of technical interviewing.