The Simple Numbers That Could Change How You Hire Technical Talent (Snyder Cut)
Back in 2014, I gave a presentation at First Round Capital’s CTO Summit on engineering recruiting, which the talented First Round content team then turned into this article: The Simple Numbers That Could Change How You Hire. Seven years later, I still reference it during conversations with founders because that age-old entrepreneurial question of how to hire engineers is still a frequent topic of interest. As long as startups need technical talent, they’ll also have questions on best practices for how to hire them.
While that article is still useful many years later, I found myself wanting to update it to both capture the benefit of hindsight, and also make it more relevant for today’s startup. So to that end, I present below the Snyder Cut of that First Round article (although one that no one but me is asking for…). A longer, more complete, and hopefully not too boring deep dive into how to analyze and optimize your hiring process like you would any other critical business funnel.
You can’t manage what you don’t measure
A helpful approach to frame and contextualize product and engineering recruiting is to think of the overall process as yet another business funnel. Just like a sales funnel or a customer acquisition funnel or a purchase funnel, engineering recruiting involves a series of steps that leads towards a predefined goal. And at each step, you can analyze the effectiveness of your funnel (or conversion) in generating outcomes.
But to quote statistician Wiliam Edwards Demming (often considered the world’s first data scientist): “You can’t manage what you don’t measure”. To be great at managing a recruiting funnel, you also need to be as data driven about it as you would other parts of your core product. Do you take candidate close rates as seriously as your notification click through rates? Do you know what percentage of your new hires came through referral channels as you would know what percentage of new users came through invites? Can your hiring managers not just tell you that hiring is going well, but use numbers to explain why?
The Recruiting Funnel
Let’s use the following simple, 4 step technical recruiting funnel to encapsulates the hiring process:
You start with candidates at the top of the funnel, and hopefully end with a new hire at the bottom as a percentage of the candidates pass through each step (i.e. your conversion rate). But what conversion rate at each step do you need for an effective recruiting funnel? I offer this rule of thumb: do math by four. To come through the funnel with a hire, you’ll need to source approximately 64 candidates, screen 16 of them, interview 4, and hire 1. Or another way to do the math is that you’ll need at least a 25% conversion rate at each step to generate a successful outcome.
The 25% conversion rate is only an approximate, but is still a useful benchmark. It can help set expectations for the scope of work involved in hiring (ex. you’ll need to source dozens of candidates in order to have a shot at hiring one) as well as help identify inefficiencies in your hiring process (ex. if you’re not making offers to 25% of the candidates you interview, the way you screen or interview may be flawed). Companies need to understand and measure each step in their recruiting funnel in order to effectively manage their hiring process.
And at each step in the funnel, there are some useful strategies every startup can implement to maximize the likelihood of overall hiring success.
SOURCING: From 64 to 16
Before you can optimize your engineering recruiting funnel, you need qualified candidates to be in the funnel in the first place. And the best way to do that is to not just have a single sourcing strategy, but instead to look far and wide across diverse channels for qualified candidates.
There are three primary channels for sourcing: referrals, outbound, and inbound. A healthy and diverse sourcing strategy takes advantage of all 3. When one source dramatically outpaces the others, treat that as a negative that you will endeavor to change as you want meaningful exposure to all three channels.
Referrals
Referrals (candidates who are acquaintances of your employees) is the most common sourcing channel, particularly for startups. It’s not uncommon for 100% of your first dozen or so PMs, designers, and engineers to all be referrals. Over time, you’ll build a more balanced pipeline of candidates through outbound and inbound channels. But along the way, there are ways to invest more in referrals, beyond just creating a referral bonus program as it takes more than just cash incentives to execute a successful referral strategy.
1. Evangelize referrals
To catalyze a referral program, start by constantly making sure team members are aware of the open positions you are recruiting for. Highlight these positions regularly at weekly huddles, all-hands meetings, and internal communications, and make sure people know how to refer their contacts for those positions. Also, celebrate referral wins publicly to bring even more awareness to them. At my last CTO role at Flipboard, we would introduce new hires during all-hands. But in addition to welcoming the new team member, we’d also recognize who referred that person and present them their bonus on the spot.
Remember that your own team is the audience you need to be marketing your referral program to.
2. Create good referral habits
Like with any behavior, you need triggers and routines to allow habits around referrals to form at your company. At Flipboard, we made referrals part of our new hire onboarding process and had all new employees meet with the recruiting team. We’d then talk through the referral program, review open positions, and do a quick pass of their network in case any connections could be candidates.
We’d also hold regular referral events where we’d invite team members to grab their laptops and sit down together over pizzas. Hiring managers would come in and share their open positions and then everyone would comb through LinkedIn, Facebook, Twitter, GitHub — anywhere they might know people — and collectively build a referral list together. When we found a connection that could be a fit, we’d also help the team member draft and personalize the outreach to increase the likelihood of getting a response. Every little edge helps.
In the end, great referral programs are those that are hard to forget, and easy to participate in.
Outbound
When people think of outbound sourcing (reaching out to product and engineering talent they’d like to recruit), they think of LinkedIn and then usually stop there. LinkedIn is a great starting point, but only that. The key to a strong outbound sourcing program is to boldly cast a wide net.
1. Look far and wide
Beyond LinkedIn, social media services are obvious places you should be sourcing for technical talent. Over 90% of professional recruiters source via social media so follow their lead. In addition, there are other popular places engineers hang out online like GitHub, Stack Overflow, HackerRank, and AngelList. And talent firms like Riviera Partners have phenomenal networks so while they are expensive to work with, the quality of candidates can justify the cost particularly for senior hires.
Finally, I’m a huge fan of University talent, not only for their technical capabilities, but also for the diverse networks they can bring. Startups often don’t often invest in University recruiting early enough. In my first CTO role at Hulu, we started an internship program and actively recruited both part time and full time engineers at over a dozen universities in our very first year of existence. College recruiting networks like Handshake make that process super efficient so there’s no excuse to not be looking at early career talent.
2. Be bold
Like with many things in life, fortune favors the bold when it comes to outbound sourcing. According to LinkedIn, while only 1/3 of professionals are actively looking for new job opportunities, 90% are open to talking to a recruiter. The odds are in your favor that the amazing data scientist or mobile engineer or interaction designer you’ve come across is at least going to read your outreach.
Being bold doesn’t mean to be pushy or aggressive or arrogant. It means to be proud of your company and excited to tell everyone who will listen about the career opportunities you have to offer.
Inbound
It doesn’t matter if you’re a tiny startup or Google, you should make it a goal to have qualified candidates reaching out inbound to you and applying for your jobs proactively. And to do that takes 2 things: good storytelling, and good candidate service.
1. Tell a compelling story
Companies that excel at recruiting go beyond traditional job descriptions that just list requirements, and instead tell a story about their company and why the work they do matters. For example at Hulu, we sent out a What Defines Hulu document to every recruiting candidate to give them a sense of our culture, what life is like at our company, and what they could anticipate if they joined. Or take Zumper (a company I had the privilege of serving on the board of) who prominently calls out their community values on their careers site because of how important they are to every employee at the company.
Once you’ve crafted a compelling story that highlights your job opportunity, make sure to get that story in front of as many people as possible, which means posting to a wide range of job boards beyond LinkedIn such as ZipRecruiter, Indeed, Glassdoor, CareerBuilder, Dice, and more. And of course, leverage social media to get the word out, like Shopify CEO Tobi Lutke expertly did:
Finally, keep your job posts fresh. After a few months, it’s natural to see a drop-off in inbound interest. If you’ve passed that point, just pull the post down, freshen it up, wait a week, and repost.
You know that your startup is an exciting, dynamic place to work. Let others know that too.
2. Provide great candidate service
Once you have candidates reaching out to you inbound, commit to engaging quickly in order to keep them interested in you. To start, make sure to have a 100% response rate. When somebody takes the time to reach out to your company, it’s only fair for your company to then take the time to respond back. Think of inbound candidates as customers. If product customers were to contact a company to report a bug, the support team would respond consistently. If sales customers were to contact a company to ask for a demo, the sales team would respond consistently. Similarly, treat recruiting candidates with that same level of service. And in a world with Glassdoor, social networks, and plenty of other places to leave company reviews, your level of candidate service — whether it’s good or bad — is something that everyone will hear about.
Add it all together, the Sourcing step in your funnel might look like this:
SCREENING: From 16 to 4
At this point, your sourcing efforts have yielded a strong crop of prospective engineers, product managers, and other technical employees that you are engaging with (about 25% of all the candidates you sourced through referrals, outbound, and inbound channels). The next step in the recruiting funnel is to screen those top 16 engaged candidates and select the right 4 to interview. This vital step between sourcing and interviewing is where I’ve found companies’ hiring processes have the most room for improvement. And it all starts with data.
For starters, if you’re not using an applicant tracking system (ATS), you should be. In a recent survey of recruiting professionals, 80% of respondents said that an ATS helped them recruit higher quality candidates and do so faster. The system that you use isn’t the important factor here — there are plenty of good solutions to choose from. But once you have an ATS in place, you’ll be able to track the data that can give you visibility into how your recruiting efforts are performing.
Here are some key buckets of data to track:
- Open positions. What are all the open positions, how long have they been open, where are the job listings posted, etc.
- Candidate profiles. How many candidates are in the pipeline for your open positions and who are they, what are their backgrounds, how did they enter the pipeline, etc.
- Engagement history. What are your communications with candidates, how frequently are you communicating with candidates and what’s your response times, etc.
- Evaluation feedback. What are all the questions being asked during screening and interviewing and what are all the candidate responses, who is asking those questions, etc.
If you start to record those buckets of data, you’ll have the raw ingredients to be able to analyze and understand your recruiting funnel. Which brings us back to screening. Screening is the first moment you start to gather custom data about candidates — information and feedback that’s specific to the job you’re hiring for. And not only is it the first moment, screening is the most efficient moment in the entire recruiting funnel to gather this data. Screening typically takes less than an hour, is done remotely, and even asynchronously. By comparison, interviewing is a much larger commitment for both the company and the candidate (often involving multiple people and in person meetings) so better screening can save everyone time and effort.
The screening process affords lots of opportunities for creative assignments, from design portfolio reviews to product case studies to market analysis presentations. But engineering roles offer one of the most interesting types of projects to riff on with candidates: coding challenges.
For engineering hiring, coding challenges are an efficient and thorough way to learn about a candidate during the screen, anchor your live conversation with them, and have fun in the process. At Flipboard, we built the site challenge.flipboard.com to host our coding challenges. A large percentage of our candidates would be asked to complete a challenge, and the feedback from them was overwhelmingly positive around the experience. Turns out, engineers generally like to code.
A few tips for implementing coding challenges into your screening process:
- Build your own coding challenge. You can use third-party services (and there are a bunch of them) but it’s actually fun to build your own, and even more fun to take your own. At Flipboard, our engineers would come up with challenge questions and then try and solve each other’s both to calibrate difficulty but more importantly for bragging rights. It became a source of team bonding.
- Keep the challenge really open-ended. The purpose of the coding challenge is to learn about the candidate’s broad technical capabilities and thought process, not to learn about a specific skill. Create a challenge that can be solved with any language in a variety of ways, so it’s accessible and applicable to as many people as possible (both candidates and employees reviewing the solutions). The more general the challenge is, the more efficient and reusable it will be too.
- Don’t get fancy. You don’t need multiple unique challenges for every type of position that you can then A/B test. That’s not only low ROI, but saps the fun out of the entire exercise. Don’t create more work than you have to.
Remember, the coding challenge isn’t a replacement for an actual conversation with the engineering candidate; live interaction with the candidate is still critical. But coding challenges can help accelerate those conversations. Armed with insight from the candidate’s progress on the coding challenge, you can jump right into the heart of the screen and discuss language and design choices the candidate made, see their overall problem solving approach, and also get a sense for their communication skills and style, all of which are valuable learnings to track in your ATS.
INTERVIEWING: From 4 to 1
By the time you get to the interview step in your recruiting funnel, you’re dealing with the very top of your technical candidate pool having gone from 64 sourced candidates to 16 screened candidates to 4 left to interview. Soon, you’ll be welcoming one of these individuals as the newest member of your company, which also means the other 75% of this talented, vetted group won’t be working with you.
The interview stage is actually the most controversial in the whole recruiting lifecycle because of this high failure rate. Given interviewing is the most expensive stage of your funnel — you’re committing a full day of combined people hours and possibly paying for flights, hotels, dinners, and more — that failure rate might sound like an unacceptably steep cost. But the truth is it’s entirely worthwhile.
In the interview step, there are 3 main goals you’re trying to accomplish:
- Mutually assess fit. You’re asking the candidate questions, and answering the candidate’s questions to probe on their skillset, qualifications, experiences, interest level, and personality to determine if they would meet the needs of the role and be a strong contributor to the company. You’re also getting them excited about joining in preparation of convincing them to accept your potential job offer. This is what most people think of when you bring up the topic of interviewing.
- Build internal alignment. Recruiting is a team effort and the team needs to be well aligned on what everyone is looking for in the candidate to make a confident hiring decision. Getting to meet with candidates and discuss feedback afterwards together is the best way to get properly calibrated.
- Evangelize your brand. When you interview a candidate, you’re also giving them a behind the scenes look at your company. Candidates are going to tell people about that look (three quarters of candidates will share their interview experience with others) whether it’s good or bad. So after days or even weeks and months engaging with a candidate throughout your recruiting funnel, you will inevitably be creating either a new fan or a new critic of your company, and that can directly impact your business.
What’s interesting is that you can benefit from the last two goals (Build internal alignment + Evangelize your brand) regardless of whether you hire the candidate or not. Debriefing about a technical candidate you don’t hire gives everyone more clarity around project responsibilities and team chemistry that can aid the recruiting process and also give people a better understanding of the needs of the org. And evangelizing your culture and mission to a person you don’t hire can lead to a new customer, a new supporter, and even a new evangelist that can spread the message of your brand and values far and wide.
Even a no-hire outcome can still be positive outcome.
HIRING: Roll up your sleeves because it’s worth it
Putting it all together, once you’ve taken the time to understand and measure your recruiting lifecycle — broken down into stages, complete with conversion targets at each stage — you’re now in position to effectively manage it. If you ended up interviewing too many people, your screening process may not be selective enough. Or if you only interviewed 1 person, you may have not sourced enough candidates to begin with. The answers are in the data, which can help guide you towards changing, improving, experimenting, and ultimately succeeding at recruiting.
Finally, it’s important to point out that none of the above tips and best practices are shortcuts to the main ingredient to recruiting: hard work. Recruiting is flat out hard. It requires long hours and tremendous effort and dedication. It takes commitment from an entire organization, from hiring managers to their recruiting team members and everyone in between, all helping in sourcing, screening, and interviewing, to finally result in a hire.
But think about when you do get to that outcome, when you’ve built a world class team one great hire at a time ideally suited for your company’s mission. Think about what you’ll be able to accomplish and the fun that you’ll have along the way. It’ll make all the work worth it.
