Skip to main content
All CollectionsProduct Guide for Searchlight Users
How to Interpret Searchlight Candidate Data
How to Interpret Searchlight Candidate Data
Andrew Olsen avatar
Written by Andrew Olsen
Updated over a year ago

Hi! 👋 You’re about to make a big decision, and we’re here to help.
Each Searchlight candidate profile is a gold mine of information, let's take a quick tour!

Section 1 - Welcome

Section 2 - Our References

Section 3 - Searchlight Score

Section 4 - Interpreting the Scorecard

Section 5 - Get to know the Soft Skills

Section 6 - Frequently Discussed Topics

SECTION 1

Welcome

Searchlight works with dozens of partner companies 🏢, we’ve analyzed thousands of candidates and references 📊, and we’ve developed our science in collaboration with psychologists 🥼.

Searchlight offers:

  1. Deep visibility into skills, patterns, and themes about each candidate

  2. The insight to focus on what behaviors are most important to your team

  3. Validation and guidance to make the best hiring decision

SECTION 2

Our References

To ensure the security and legitimacy of our data, Searchlight verifies each reference by gathering data like

🔢 IP addresses
☎️ phone numbers
💻 LinkedIn URLs

What to pay attention to 🔍

  • Are there at least 2 references submitted?
    One single datapoint on a candidate is not enough to make an informed judgement. The power of references is when they converge on patterns and themes. Searchlight makes that easy.

  • Who are the references?
    We recommend all candidates submit at least 1 direct manager reference, as these are the highest signal. We often hear that our customers weigh past manager references more heavily over other types of references. Past managers probably had to assess a candidate during a past performance review and track their professional performance closely, and have less reason to be biased.

  • How recently did the reference and candidate work together?
    References who ended their work relationship with the candidate more than 5 years ago will not have as accurate a picture of who the candidate is today.

SECTION 3

The Searchlight Score

At a glance, the Searchlight score answers the question: How respected is this candidate by their peer group and how excited would they be to work with this candidate again?

Our intelligence is informed by thousands of references, and our proprietary benchmarks map multiple heuristics from the references into a single score.


Some inputs we use:

  • Peer ranking percentiles

  • Competency scores

  • Excitement percentiles

  • Referencer relevance

  • Has the referencer managed before?

  • Referencer years of experience

  • Length of work relationship

  • Recency of shared work experience

  • Strength to weaknesses ratio

  • Overall strength of the recommendation

Our score tiers:

Neutral

This candidate falls into the bottom 60th percentile of all reference feedback. A followup call with a reference may be necessary

Good (60 - 75th Percentile)

Good, but there are some cautionary signals. Read reference feedback carefully

Great (75 - 85th Percentile)

References speak highly of the candidate, but there may be some specific areas for growth.

Excellent (85 - 95th Percentile)

References speak highly of the candidate.

Stellar (95 - 100th Percentile)

References are exceptional but to be taken with a grain of salt. Pay extra attention to qualitative feedback as that may provide helpful feedback on areas of growth.

Pro-tip ✨: Sometimes a candidate will get a Searchlight score, and hiring managers will wonder if this is a sign to not hire. The truth is, every team is different and no candidate is perfect! If you feel the Searchlight Score is low, then proceed deliberately to better understand how the candidate might fit on your unique team.

Interpreting the Scorecards

For each assessment, the hiring manager or recruiting team customizes the competencies important to the role. Searchlight asks each reference to rate the candidate across these competencies, and our Scorecards visualize the results.


Each column represents an individual reference’s feedback. The last column displays the candidate’s percentile score, which is calculated by taking a weighted average of individual reference ratings and then benchmarking the scores against our database.

What to pay attention to 🔍

When 2 or more references agree on the strength and/or area of development 👀

Notice in the Scorecard above that 2 references gave the candidate a great rating on Leadership, and 2 references scored the candidate low on Pragmatism. This is very high signal data, especially because the references individually corroborated this assessment without knowing what other references shared.

When a reference gives the “The best I’ve seen” rating on every question 🚨

Notice that 2 references gave perfect scores for all competencies. In this situation, Searchlight recommends:

  1. Carefully evaluate the written responses for these references to understand if the candidate truly is a rockstar.

  2. If these references didn't seem to put in any thought in their other responses, then take their opinions with a grain of salt and/or schedule a follow-up call.

SECTION 5:

Get to know the Soft Skills

Each reference was individually asked to select the candidate’s top strengths and weaknesses from a randomized list, without knowing what others selected. It's very high signal for people to converge on the same attributes!

What to pay attention to 🔍

Focus on when 2+ references agree 🌟

References choose attributes from a randomized list. It is a very high signal when references converge (especially when one is a manager or supervisor). This is true of strengths and gaps.

Protip ✨: Hover over the attribute tags to read the definition.

Protip ✨: We've heard from multiple hiring managers that these attributes are highly correlated with how a new hire shows up during their first week on the job.


Some examples of Searchlight attributes

SECTION 6:

Frequently Discussed Topics

Here are some quick pointers that hiring managers have found useful. 🎉

Open Text Responses

This is where references share as much as they like about the candidate on a particular topic. If you have the time to read through everything they say, it’s like accessing the candidate's last performance review!

Peer Ranking

This is a favorite question among hiring managers, because the agreement between references gives an accurate picture of a candidate’s true ranking.

  • It's a positive indicator if all references converge on the top 2 answers. It's a very high recommendation to be rated "Best I've Ever Seen." Only a small percentage of candidates receive this score.

  • Usually we find that anything less than Outstanding (Top 5%) warrants a deeper look at the referencers’ open-ended responses.

  • Lastly, a red flag would be when a reference skips this answer. This is relatively rare, but when it happens, it’s worth a closer look.

Excitement

From our experience, any candidate profile with references that answer lower than 5/5 (Ecstatic) merits a deeper look. But, as always, it’s important to take all of the data in context, rather than a single answer in and of itself.

Having one person who isn’t a 5/5 is a yellow flag, but this becomes a bigger concern if the manager is the one who isn’t excited. Multiple people who aren’t 5/5 excited is a red flag 🚨

Advice for Hiring Managers

This is always a must-read section for onboarding-focused managers (which should be everyone)! If you’re invested in your new hire’s professional growth, these answers are prescriptive and offer valuable suggestions to help you set your new hire up to succeed and stay engaged from day one.

Protip ✨: We've heard that many hiring managers use these insights to create personalized coaching plans for their new hires.

The 150 summary

This is the icing on the cake! References provide their own short summary for the candidate. While this may not provide as nuanced information as other sections of their response, sometimes it’s a kicker.

🚨If the summary seems to be a “surprise” based on what showed up in the rest of the reference, we should take the summary with a grain of salt.

If you have any deeper questions please feel free to reach out to [email protected] or your Customer Success Manager

Did this answer your question?