Technical Interviews Don't Have to Suck by Kaitlin Jones-Muth
Technical interviews for engineering jobs are notoriously dreadful-- but they don't have to be. In this talk, you'll learn practical strategies for conducting technical interviews that are more welcoming, more useful and more fun for everyone involved.
Hi, my name is Kaitlyn, and I use she her pronouns. I'm based out of Minneapolis, Minnesota, and I'm a senior engineer at CrowdStrike. My team at CrowdStrike is a really supportive culture that I think starts from our interview process. We've been able to scale up our team pretty quickly, so it has worked well for us. When I started three years ago, we had ten front end engineers, and since then we've hired 60. So, among other things, that means I've spent a lot of time in tactical interviews, and by no means am I an expert on this. By no means is CrowdStrike process perfect, but I think that it's important for us to talk about what makes technical interviews good, because there aren't a lot of resources out there. Even though it's a pretty ubiquitous part of hiring in an engineering role, it's really hard to get better at something if you don't know what that looks like. So I'm going to spend the next 30 minutes telling you about things that I've learned from my own experience about how to make interviews more welcoming, more inclusive, and more useful. And the main point of this talk is be kind to people. It's really that easy, and it's also really that difficult. So what does that look like? Well, the first thing that we can do to make our interviews better is to plan for them. And this is going to look a little bit different everywhere based on who is involved in the hiring process, how much information you have beforehand, what roles you're hiring for, all of that. But let me tell you about what it looks like for me at CrowdStrike. So I work on a team that's distributed across many time zones in many countries. So that means that I don't always have a lot of time to prepare with the other person I'm going to be conducting an interview with, and we're hiring at a pretty dramatic pace. I think we have nine engineering managers who are hiring front end candidates right now. So that also means that I don't always know what role or what team a candidate might be placed in. So it's not ideal, but there are still things I can do to plan ahead from that. One thing Crowdstick has done is we have a set list of interview questions that we work from, and that's really helpful to keep interviews more consistent from interviewer to interviewer. We also have a rubric for those questions, which makes it easier to train up new folks on how to conduct interviews. And so what I will do is I'll work on that list of questions. I'll pick some that help me to establish a pretty broad baseline understanding of somebody's skills as an engineer. And then once I have a sense of where their passions lie or what they're really strong at, I'll dig more in those areas to try and figure out a little bit about who they are and how they think. And for you, this might look completely different. Maybe for you, what this looks like is sitting down with the hiring manager and with your team and figuring out, okay, what are we really hoping to get out of hiring for this position? And you might be talking about what skills you're hoping to bring into the team. You might be talking about where you think the team is already really strong and has the capacity to do some mentoring. You might be looking at notes from phone screenings or looking at even internal resources about different engineering levels, just trying to get all that information together so you know what you want. Once you know what you want, then we have to think about how do we evaluate that. And it's a little harder than it sounds. One thing that it's easy to pick on in technical interviews is whiteboarding, which I know everyone hates. Well, not everyone. I'm sure there are people who have figured out how to do it well. I have not. And if you try to ask me a whiteboarding question during an interview, what you will learn is that I'm really bad at whiteboarding. You won't learn much else. So to put this another way, if you try to make ramen in the coffee pot, you shouldn't blame the coffee pot when it doesn't go well. One thing that has been helpful for me in trying to reframe my thinking about this has been to imagine that the candidate is already on my team, and I'm trying to decide whether or not to work with them on a particular project. And that just helps me be focused a little bit and get a little bit more creative about how I start to look for some of these things to be demonstrated in my interviews. And I want to be really clear here that I'm not saying that you should plan out the entire interview and then just do that and not make any kind of allowance for what the candidate is like. You absolutely should adjust your interview to the candidate, but if you have a plan ahead of time, that gives you something to work with. And also you should know if your plan is based on any sort of assumptions about the candidates level of skill. People aren't always great at assessing themselves against your company's internal set of engineering levels, so if you have any assumptions about their skill level, it's good to validate those right away before you dive into some super complicated questions that might just put a complete pause on the interview with someone who's not well prepared. All right, so we plan for our interview. We've got a candidate in front of us. Now. What I think the next thing we want to focus on is making sure that the candidate feels welcome, feels like they can relax, and that they leave the interview feeling good. There are a couple of different reasons for this. Candidates are almost always nervous in interviews and those nerves can become a huge blocker to figuring out what they're actually like in real life. So anything we can do to break through nerves and find out what the candidate's real self is like, that's time well spent. And in terms of how the candidate feels at the end of the interview. I've been to technical interviews before where I just feel crushed when I leave. And that's bad for a couple of reasons. There's the very human, like, I don't like making people feel bad level. But there's also a business aspect to this because people remember the way we make them feel about themselves and they remember that when they're deciding later on which company they want to work for. So if you can start to demonstrate your team's culture and your team's values from the interview process, you're giving that candidate better information to work with and hopefully you're making it more likely that they will want to accept your company's offer and come work with you. All right? So all that said, what are some things we can do to make people feel welcome? The first thing is just make it a conversation as much as possible. I think just making some small talk, being really human with the candidate, helps them to know that you are thinking about them as a human too. One side note on this nervous candidates aren't always great at small talk. So if that falls flat, don't blame them for that, but at least give them a chance. And then the other thing is just to be really mindful of what you are doing to create a welcoming atmosphere. And what I mean by that is paying attention to things like your tone of voice, your facial expressions, your body language. Maybe think back to some interviews you've been in in the past where you haven't felt super comfortable and ask yourself if there was anything that the interviewer did or didn't do that changed how you felt. And it's OK to not have a super warm personal style. It's totally okay to be very direct about this and just say to the candidate, interviews can be nerve wracking and I want you to feel as comfortable as possible. And I'm also not trying to say here that you have to be Chris Traeger, who is literally highfiving candidates and telling them that they are beautiful, charismatic, intelligent superheroes. What I am saying is that you should try not to be Ron Swanson who's sitting there with his eyes narrowed and his arms crossed and literally pointing at the door. Try to find a middle ground between the two. The other thing that's a really great tool for us in helping candidates to relax is just telling them what to expect. As human beings, we love knowing what to expect and we don't always love it while the explanation is happening. But later on, when something happens and we expected it. Oh, it feels so good. So one thing that I do here is I try and tell the candidates right away in this interview, it is okay to say I don't know or to say I'm not sure, but here's my best guess. I think that helps set the tone for the interview. And I also really genuinely want them to do that, but I know I can't expect them to do that unless I tell them explicitly because not every interview is like that. And the flip side of this is you should also tell candidates if there's anything they should expect not to happen. So to pick on whiteboarding again for a second, we don't do any whiteboarding or live coding as a part of our technical interview at CrowdStrike. And I try and make sure everybody knows that before we launch into the interview part of the interview. And you would be surprised how many people just visibly relax when you tell them, hey, you're not going to have to do this. You can stop worrying about it. And that's exactly what we're trying to do, is to get them to be more like their normal selves. All right? So once we've got our candidates to relax a little bit, the next thing that's a really useful tool for us is how we engage with those candidates during the interview process itself. And what we really want to do here is get beyond those nerves and figure out what a candidate knows and perhaps more importantly, how they think, what they're going to be like to actually work with. So the way that I like to do this is with some interactive coaching. And I've got an example question for us here so we can talk about some different strategies that we might use for this depending on which candidate we're interviewing and how they respond to our questions. So the example that we're going to be using here is that we have an interactive table that we're adding to our page and we want to know how we can make that table accessible. So for a candidate who is less experienced or just a candidate who's really nervous and kind of getting stuck, one thing that we can do to follow up on this question is just to make the problem area smaller and more specific. So in this case, I might say, hey, what about a user who isn't using a screen reader, but they also still have some visual impairments. What can we do to make our page better for that user? And what I'm doing here is also trying to make the question itself a little bit less difficult because a candidate who has no accessibility experience can still find some things to engage with there. And a candidate who has more experience can show me that still. But it lets us keep having a conversation so that I can keep learning about what the candidate is like rather than us just sitting there and them feeling uncomfortable because they don't know and me not learning anything about the candidate. Another strategy that can be really helpful here is just to pivot to another skill. If we're talking about accessibility, and I'm talking to a candidate who's never done it before, we're not going to get very far. So in this case I might say, hey, let's say that we have to make some design changes and now we need to go back to talk to the designer or to a project stakeholder and explain why that needs to happen and get them on board with us. How would you do that? What would you think about? And this is great because it gives the candidate, again, an opportunity to show me something, even if it's not something I was originally going forward with the question. And this can also be really useful if you have a candidate who is much more experienced with something than you are, because I have definitely run into the situation where a candidate knows way more than I do about something to the point where I'm not able to accurately assess them anymore. And this helps us to continue to have a conversation where they can continue to show me new things because I've already written them down as off the charts on that particular thing. All right, another thing that can be helpful for those more experienced candidates is to make the problem more difficult. So again, we want to stay pretty specific. So in this case, one thing I might do is say, all right, let's add columns that the user can adjust the width of. How do we keep that accessible? And that gives them another challenge to work off of. It gives them, again, a way to demonstrate to me some more deep skills. But we're still on the same page and we're still having a good conversation about accessibility and how to apply it. All right? And then the other thing that we can run into is a candidate who tells us part of what we were hoping to hear but doesn't get to everything. And we're kind of left wondering, do they know all of it or do they just know that part or did they forget the other part because they're nervous? So, for instance, if I'm talking about accessibility and a candidate just mentioned Aria, one of the things I can do is just be really direct in response to that and say, hey, when do we want to use Aria and when do we want to use semantic HTML? How do we make that decision? And it's not like by saying that I'm telling them all about semantic HTML or giving them the answer, I'm just helping to guide the conversation so that it can be more productive and so that at the end I'm not guessing about how much they know and what they can do. The other thing that can be helpful here is to ask about a candidate's personal experience. Everybody can talk about the projects that we've worked on with more detail than people can talk about some abstract example. And this can also give you really helpful material for the rest of the interview because you can kind of keep referring back to some of these same experiences. And you're also going to get information about what people have actually done in real life, which is probably a better predictor of their future behavior than what they imagined they might do. So as you can see, lots of different strategies. These are all appropriate ways to adjust the conversation based on the candidate who's in front of you. So another thing that I want to emphasize here is that using concrete examples is really helpful. Unfortunately, I've had a lot of experiences where I'm talking to the candidate and I ask a question that I think is really clear and there's some jargon in there that the candidate just isn't familiar with for whatever reason. And often this can happen with people who are really experienced, who just haven't brushed up on exactly which words go with which skills right now. And then we spend a lot of time just trying to get on the same page about what I'm even asking them and that time isn't useful for them and it's not useful for me. So concrete examples allow us to sidestep that entirely. And I think they also allow people to show up again more in the way that they would normally show up to their jobs because it's putting them into a more familiar situation. The other thing I should say about this is I did not come up with all of those examples on the previous slides, just on the fly. I know some really skilled engineers who can do that and it's so incredible to watch some interview. But I can't do that. I have to prepare ahead of time and I have to really think about what the candidate might say, what I might say, and find questions I like and kind of keep coming back to those. And I'm telling you that just so you know, if you are like that too, that is okay. Put in the work, it's worth it. And one thing I want to emphasize here too is that it is okay to do a lot of coaching. I think sometimes we get this feeling like the interview is somehow supposed to be like a quiz and the candidate is getting graded on their answers, but it really is more about how they get there and what the final answer is. And you are there, you're watching the interview, you're having the conversation with them. So you'll know at the end when you're providing your feedback, you'll know how much you helped them and you'll know what they came up with unprompted and you'll know what they were like as they were getting there. So just make sure that that's reflected in your feedback and you'll be fine. And that brings us to feedback, which is a really important part of the interview process. In fact, unless you're the hiring manager for the role, the feedback that you give about a technical interview is the main point of doing that interview. So it's worth taking the time to do it right. A couple of things about what's helped me to be good at this. The first is I have to capture feedback when it's really fresh in my head. I in the past, have had a call with a colleague immediately after we did an interview together and already our memories from the interview are starting to kind of drift apart and we have to remind each other exactly what happened. So if I spend too much time thinking about something else, I really can't provide useful feedback. And calendars can be really tough sometimes. So if you have to jump into another meeting right away, just try and take some notes for yourself, maybe about anything that really stood out to you or any specific things you want to make sure you mention. And then when you have time, you can come back and reflect on those notes and get back into that headspace. And I do think it's important to make that space to really think about the candidate and think about the interview. And it takes me between 15 and 30 minutes to provide feedback and our technical interviews are an hour. So I'm telling you that just to say however long it takes you is how long it takes you. But this is a really important part of the hiring process and it's worth it to invest the time in it now, because getting it wrong takes a lot more time, whether that means incorrectly rejecting a candidate or recommending a candidate to move forward when that turns out to be a waste of their time and the company's time. So the other thing that is really helpful here is we probably want to go beyond just yes or no in our feedback. I try and frame this for the hiring manager as these are the things that I think will make the candidate successful. And these are the areas where I think they might need a little bit more support or coaching or maybe just some more time to come up to speed. And those details are really useful to a hiring manager. For instance, if there are multiple strong candidates for a role that they can only hire one, that helps them compare candidates better. It also helps if there is one candidate and multiple roles that they could hire into, it helps them to decide what's going to be the best fit for this particular person and that makes everybody happier and makes all of us able to succeed more. So it's worth it. The other thing that, again, I think is really helpful here, just like it is in the interview is to provide concrete examples and kind of show your work for the person reviewing your feedback. One thing that could happen is I don't have all the context for a person being hired and what role we're hiring them into. So maybe I ask them like a pretty basic CSS question, and the candidate says, you know, I'm not really familiar with that. I would have to get back to you. And my first reaction to that as an interviewer is like, oh great, it's nice that they were honest with me. But if a hiring manager knows something I don't like, let's say the hiring manager knows we're trying to hire that candidate to do all this fit and finish work on our brand new design library, and we really need them to know CSS backwards and forwards. It's important for the hiring manager to know how I came to that decision that the candidate was honest, because that would give some other information about whether or not the rule that we're looking at is the right fit. And then the last thing I want to emphasize here is we want to provide feedback on the whole candidate. It can be easy to feel like because this is called a technical interview, that we're really only looking at people's mastery of specific skills. But the whole time we're interviewing somebody, we're evaluating their ability to communicate clearly about technical concepts. And there are just a lot of skills go into being a good engineer, whether that's having a skill for writing clear documentation or having empathy for your users or your teammates, or having a really good business sense and knowing when to bring other people in and how to make good decisions with the information you have in front of you. These are all really important things. So we should include things that are relevant to all of these skills in our feedback. And that actually brings me to my secret second main point of this talk. I think that technical interviews are really actually behavioral interviews. I think that how we get to the answer is more important than what the answer is. And I think that because in my experience, it's much easier to teach somebody a particular language teacher or to coach them out of a particular bad habit than it is to change something that's closer to their personality. So I would rather say to somebody, hey, we found that turnaround statements make our code base a little bit better to maintain. So we try and be careful about how we use them. That takes like 3 minutes. It takes way longer to teach somebody how to be empathetic to users. I had an experience a few years ago at a different company that I'm still mad about where a candidate was rejected because the file structure that they used in their homework example didn't match the internal preferences that the company had for how files should be set up for a vanilla JavaScript application. It was just such a waste. And I know it was a waste because my team went back to that candidate. We interviewed them, we hired them, and they were incredibly successful and a huge asset to our team. So make sure that we're paying attention to what the candidate will be like to work with and everything that they'll bring and not getting blindsided by smaller things that it's easier to teach people out of. All right, I want to leave you with a challenge here, which is to think about how power shows up in our interviews. Whether or not we are the final hiring manager or the final decision maker in a role, we still have power to influence someone's future when we're conducting a technical interview. And I think it's important to be mindful about how we use that power. I also think that we get a lot of messages from the culture around us about what interviews are supposed to be like. And I think a lot of those messages tell us that the imbalance of power between the interviewer and the person being interviewed is important and that it's the job of the person conducting the interview to really lean into that imbalance of power and maybe even abuse it. And that makes the interviews suck. It's not fun to do that to someone. It's not fun when that's being done to you. And beyond that, that also means that the people who are most likely to get hurt are the people who are already starting from a less powerful place. That could be somebody who it comes from a background that's less represented in tech. It could be somebody who's new to the industry or just somebody who really needs a job right now. But regardless of who it is, we have to remember that interviews are gates and who we let in and who we keep out influence what our industry looks like. And the good side of this is it means that if we can change what interviews are like, if we can change how interviews feel, we can change what our industry looks like, and we can change who's able to succeed in it. So please be kind to people out there. Conducting your interviews in a way that is kinder and more intentional is going to give you better information about your candidates, more candidates to choose from, and maybe even some really great candidates that other companies are passing on. Thank you.