Interviewing from the other side of the table
Published on 2012-04-21
In the last few months, I've had the opportunity to interview a lot of new candidates for our compiler teams. Being the interviewer rather than the interviewee is still pretty new to me. I've come to the conclusion that interviewing is a technical skill that must be learned and refined like any other. Hopefully some of the advice here will be useful to other new interviewers.
1. Goals of the interview
Always remember you are trying to accomplish a couple key objectives during an interview. First, you are trying to determine if the candidate is a good fit for your team. There are several aspects to this. Most importantly you are looking for a candidate who is, as Joel Spolsky says, smart and gets things done. You're also looking for someone who communicates their ideas well and can get along with your team. If your team works in a specific technical area, you might want someone familiar with that domain. For instance, my team works on compilers and virtual machines, and we would be leery about hiring someone who had never written a line of assembly.
Second, you are trying to give the candidate a positive impression of your company and your team so they will actually accept your offer if you decide to hire them. This probably isn't as much of an issue for companies with great reputations like Google or Apple, but most other companies are going to have to work at this.
2. Phone screen first
Before you invite a candidate out for a day of on-site interviewing, you can save everyone some time by doing one or two technical phone interviews first.
Typically, the candidate will first speak to someone in H.R. or preferably the hiring manager. It's important to explain exactly what work will be involved in the position the candidate is applying for. This isn't usually clear from job listings, and it doesn't hurt to clarify. You might also want to explain exactly what your company does and what your business model is and answer any questions the candidate has. Again, this isn't always clear from the outside.
There should always be some technical questions following this. Asking questions over the phone is hard because neither you nor the candidate can write anything down. I've heard horror stories about companies that ask people to dictate syntactically correct C programs over the phone, including every brace and semicolon. Do not do this. You could use Google Docs or some other web-based collaboration tool for writing code, but these still don't let you draw diagrams. Stick to conceptual questions like "how does out of order execution work?" or "what is tradeoff between longer and shorter pipelines?" or "how would you debug a program exhibiting this behavior?".
A great way to get know a candidate, especially someone who is fresh out of school, is to ask about previous projects, research, or work experience. This will allow the candidate to demonstrate their communication skills as well. Most people will list projects or publications on their rÃ©sumÃ©, so you can think of some technical questions ahead of time.
3. Use time efficiently
Once the candidate is on site, you will have a very short time to get to know them. Many companies will schedule a full day of one-on-one technical interviews with 6-8 engineers, so each interview will be scheduled for at most 60 minutes. Since there will be delays and breaks between interviews, realistically, you'll only have 40-50 minutes to make up your mind about the candidate, so don't waste time asking questions that won't help you decide whether to hire them.
Before the interview starts, coordinate with other interviewers to make sure people don't ask the same questions or neglect to ask questions in an important area. You don't want to bring in a candidate for a second day of interviews because no one asked any C++ questions. It's a good idea to have each interviewer focus on a particular area, e.g., programming, algorithms/problem solving, architecture/hardware, etc.
If possible, coordinate with other interviewers after each session as well. If another interviewer identifies some particular strengths or weaknesses, other interviewers can ask more questions in those areas. Also, if more than one interviewer forms a negative impression of the candidate, feel free to dismiss the candidate early. You shouldn't hire the candidate in this case, and you would be wasting everyone's time by continuing the interview.
4. Ask precise questions
The most important advice I can give is to be prepared for the interview. Know what school the candidate went to and what companies they've worked for. Ask an icebreaker question about their professional experience, and maybe ask a technical follow-up question about something they worked on. See if they are excited about it.
Plan to ask 3-5 technical questions which require the candidate to write code or draw diagrams on a whiteboard. You might develop a library of questions you like, but choose a small number to ask in a single session, based on the candidate and the position. This seems like a small number, but believe me, interviews are really short. A long multi-part question is much better than several short questions because you can go into more depth, and you don't have to spend time explaining the context for each new part. For instance, you might ask the candidate to write a ring buffer class in C++, and as a follow-up, you could ask them to make it thread safe.
Define the boundaries of each problem as much as possible. If you expect the candidate to call certain "given" functions to solve a problem, write out the function names and types on the board. Well-defined problems can be solved much more quickly. Encourage the candidate to ask clarifying questions (and update your question if you use it in the future).
Remember that writing code without an editor or compiler is hard, so don't get hung up on small details. If the candidate forgets a semicolon or misspells a variable, don't even bother mentioning it. Unless you're testing programming ability in a specific language, ask candidates to write pseudocode. Algorithms are much more important than syntax. If you see a logic error, don't point it out directly to the candidate, but at the same time, don't ask the candidate to find some unspecified bug. Ask leading questions, like "what happens when the input is the empty string?" or "how would this perform on a data set of a million elements?"
Feel free to ask broad questions in areas the candidate is not familiar with. Smart candidates should be able to pick up new ideas quickly and come up with creative solutions. I like to ask questions about the design and implementation of garbage collectors. Don't expect a specific answer. Have a conversation about potential solutions and trade-offs between them.
Avoid asking overly specific "quiz show" style questions (what are all the different meanings of the "static" keyword in C++) or brainteasers (how does the farmer get the wolf, the sheep, and the cabbage across the river). These questions won't help you determine whether the candidate is smart. Either they know the answer or they don't. Even if they know the answer, they might have just read it in an interview preparation book instead of learning it from real world experience. I would even avoid asking Fermi questions (how many piano tuners are there in New York) since it's so easy to come up with better questions that will give you more insight into the candidate's ability.
5. Keep the candidate fresh
An all-day interview is an exhausting experience for anyone, especially if it's preceded by a long flight across several time zones to a strange city. You need to do what you can to keep the candidate's mind fresh and ready to solve new problems.
Give the candidate at least five minutes between each interview. Make sure they have a chance to stretch their legs or use the restroom. Give them a chance to grab coffee, tea, water, or snacks from the break room.
If you can, try to schedule all of the interviews in one or two rooms. Being shuffled around to various offices and conferences rooms all day is distracting. It's easier to relax and answer questions more clearly in a familiar setting.
In the middle of the day, take the candidate out to lunch. Lunch is an opportunity to tell the candidate how cool your company is and how great it is to live in your city. Bring two or three engineers and talk about interesting projects you've been working on. You might discuss the candidate's projects, too, but don't grill them on technical details. Lunch should still be considered a break.
If you have time, give the candidate a short tour of your company's campus. If you have a fitness center, a swimming pool, a statue garden, or a nap room, show it off! I've heard Google's office in Zurich has a slide! If I were interviewing there, I would definitely want to see if that's true.
Technical interviewing is a really important skill for all engineers. Identifying good candidates and encouraging them to join your team is necessary for the success of any company.