Vibe hiring

The coining of the phrase vibe coding was a big revelation for me. It gave me the word “vibe” to use again but in the context of work. Vibe coding is what it is and boy golly are people in an uproar about it. It’s not “engineering” and it’s not “building software,” it’s more hope than plan. Which, yeah, I get it and I do agree, vibe coded things making it into production are typically a nightmare. So let’s talk about your interview process.

We’ve been vibe hiring for decades and it’s not often that this gets addressed. Shoot, we’ve been vibe people managing for a while now too. Whose job is it to research and engineer performance management and hiring processes for software engineering? Can it be mine? It sounds like a fun job and I feel like not a lot of people are doing it.

The typical interview process that I see today is something like: Phone screen → Take home screen → Live coding (read LeetCode) → System Design → Culture fit. There’s some wiggle in there but really not a whole lot. I’d say the most wiggle is in what the take home and live coding are. I’ve seen a double up on LeetCode problems, I’ve seen build a function + build a service around it, and I’ve seen build me a whole product including the terraform to deploy it so that we can make sure you know how to configure a VPC/subnets LIKE THAT’S SOMETHING PEOPLE DO EVERY DAY. What signals are you really collecting here?

Alright, the phone screen. It’s a vibe check, you need a bit of a vibe check, I won’t dog you too hard here. I like the idea of making sure someone can hold a conversation with someone who isn’t an engineer. I’ve worked with a number of recruiters in my life and the best ones hit on a vibe check better than anyone I’ve ever met. Keep it up.

The take home is where it starts to really get dumb. This is a measure of free time more than ability. A senior engineer with a full time job and a kid noped out the moment they saw “4-6 hours” and will not be coming back. The ones who do submit let Claude write most of it which is pretty normal at this point but what are we looking for here? Reasonable LLM guidance?

Then there’s the LeetCode take home group. I promise you most of your answers are LLM’d or Googled, at the very least to verify their answers. Put that aside and assume that they aren’t. What’s the signal here? Foundations? Algo knowledge that applies to… what part of your stack exactly? I’m sure there are places where this is relevant still, I don’t doubt it, but let’s not cherry pick here, we’re talking about the rest of the industry. I just…

Live coding interview briefs will start with, “We want to see how you solve problems,” right before they ask you straight up to implement depth first search with a follow up of “can you implement this using a stack.” The answer is yes but not right now, we’re going to magically discover how to do that in the last 8 minutes of the interview ✨ together.

The system design interview is the other component that I think is really solid. It’s sometimes a contrived space (DataDog asked me to design Mint.com) and sometimes a design-our-system type of deal. Either way, you get to probe about security, scalability, observability, cost management, etc… the things that vibes won’t (can’t? yet?) cover. But my goodness you must probe for more information. I had an engineer describe to me once how he centralizes span attribution naming with dataclasses in Python to ensure that naming doesn’t break across PRs. That was a great idea and something new I learned but we had to dig deep into observability to get to that insight.

The last piece of this puzzle is the cultural fit interview. You can sneak right on by if you know the STAR-method of storytelling and haven’t thrown hands at standup. If you’ve rehearsed a “tell me about a conflict” answer in the past, you’re clear. The point is to get a read on the type of person that the candidate is, so you ask the hard questions and get the, hopefully, good answers. No one wants to work with someone who doesn’t handle conflict well, I get that. But you just threw a junior engineer at this because they don’t have the chops for the technical stuff yet as if reading humans is easier than writing code.

Early in my career, I was the untrained engineer who got tossed into the interview loop with no direction. I wanted in on hiring, but I didn’t know enough to run the technical interviews with senior+ engineers, so a number of times, I got stuck in the behavioral loop. I had no idea what to look for, and even if I had, I didn't have enough experience with life or people to actually find it.

Alright, enough doom and gloom, let’s find some ways we can improve on this loop without blowing the entire thing out of the water. We can’t collect signal toward a target that we haven’t defined, so first and foremost, we need to start with some goals. When I'm building an interview pipeline, I'm after signal that predicts the job, that the person in the room can actually read, and that I could get without costing a good candidate their weekend.

LeetCode is an anti-pattern to me at this point, it’s not an indicator of day to day work, it’s a hobby, let it be a hobby. Readable in the room comes up because of the behavioral interview. Lots of questions get asked, lots of stories get told, but no one is being trained on what signal to look for and how to coax it. The last bit: this process can’t be a massive time sink for us or the candidate. It can be great for signal but it’s not exactly pragmatic or respectful of anyone’s time.

You can’t chase any of this signal until you have defined what good actually looks like for this role and team. I don’t want to call it homework because it’s not extra, it’s part of the job. It’s just that this part of the job doesn’t usually make it into the sprint. I’ve been a part of a number of interview processes that were inefficient at best and exactly what I’ve been griping about at worst.

In a past life, I was reviewing a 5-question multiple-choice quiz that we’d send to candidates as an initial screen to determine if they could get into the office for a power day. Most of the best resumes we got went radio silent when we sent them the link, others would at least do us the courtesy of telling us off. We didn’t hire any of the folks who came in - hell, most of them never even made it past lunch. The interview was a challenging but vapid gauntlet of algorithm problems with absolutely no bearing on the job because we failed to do the hard stuff. We had never taken the time to define what was important to us from a technical and cultural perspective. This post has officially gone on long enough, so stay tuned for the follow-up where I talk about the things I’ve been doing to fix it, code and all.