The Searchlight Predicted Performance™ is a proprietary assessment measuring a candidate's likelihood of being a high performer for a job based on key quantitative and qualitative factors. Using the Searchlight Predicted Performance™, Hiring Managers and Recruiters have an ethical and predictive AI copilot to ensure hiring decisions are high quality and unbiased.
How the Searchlight Predicted Performance™ is generated
The Searchlight Predicted Performance™ is powered by Searchlight's proprietary AI engine that evaluates candidates on these factors:
Relationship Context: Positive references and recommendations from previous managers or colleagues would increase the score, as they vouch for the candidate's abilities and work ethic. For each reference, we examine the relationship between the candidate and the peer or manager. We take into consideration how many years worked together, the frequency of interactions within a given week, and the qualifications and experiences of each reference provider. Similarly, for our Quality of Hire scores, which measures post-hire outcomes, we deliberately collect information from both the manager and the employee at the 90-day and 180-day mark.
Peer Ranking & Past Performance: We also take into consideration how the candidate was ranked relative to their peers based on past performance, which provides unique insight into the candidate’s past performance and professional reputation.
Soft Skills: 89% of mishires are due to missing soft skills. Searchlight has a proprietary skills ontology that includes soft skills and working styles that are notoriously difficult to measure, and are empirically linked to post-hire performance. Some example skills include adaptability, collaboration, problem-solving, and influencing others.
Cultural Alignment: An assessment of the candidate's values, work styles, and attitudes. The cultural alignment factors that we use are backed by organizational psychology research.
Competencies & Job Requirements: How closely the candidate's profile matches the specific job requirements and responsibilities influences the score. For each job family, we assess job-specific competencies that further adds context to our model. Our scorecard capabilities allow us to evaluate job-specific competencies, so in the case of a sales role, we can incorporate sales-specific competencies like “sales effectiveness”, “value selling”, etc.
Post-Hire Performance Labels as Ground Truth: Unlike other algorithms that use the “hired” event as the success case, our algorithm trains on performance on-the-job. This surfaces our commitment to outcomes-based hiring, because we know that there can be large discrepancy between interview ratings and actual employee success.
Future Data Sources
In the future, we intend to incorporate more data, including:
Candidate Qualifications & Specialized Skills: The score takes into account the candidate's educational background, relevant work experience, certifications, and any specialized skills required for the job. Traditionally, these components have been captured in a resume and primarily used for screening. We believe this incorporating this into our AI will further improve our ability to identify the right candidates.
Role-specific Outcome Metrics: In the future, role-specific outcome metrics will further enhance the reliability of the Searchlight Predicted Performance™ by training the model to detect role-specific nuance that can better translate specific skills to specific business outcomes. For example, for Sales-specific roles, we would be able to train our AI on a Sales metric like “Quota Attainment”. Alternatively, for Customer Success-roles, we could train our AI to be trained on a Customer Success metric like “Net Retention”.
Experience Relevance: The relevance of the candidate's past roles to the current job opening plays a role in the score calculation. For example, one of the most common challenges hiring teams face is translating experiences from a large company to a start-up and being able to discern which experiences translate and which do not. By doing so, we can better measure and infer transferable skills between organizations.
How Hiring Managers and Recruiters use the Searchlight Predicted Performance™
The Searchlight Predicted Performance™ can be used by Hiring Managers and Recruiters as a tool to objectively evaluate candidates and make more informed decisions during the selection process.
Every candidate that runs through Searchlight will receive a Searchlight Predicted Performance™. These scores are expressed into 4 broad buckets:
Top Candidate: the top 20% of candidates across Searchlight will receive this score. Based on our historical data, candidates with this rating have a >70% chance of being a top performer on your team.
Above Average: the next 20% of candidates across Searchlight will receive this score. We project that this candidate will most likely fall in the 60-80th percentile in terms of on-the-job performance.
Average: the next 40% of candidates across Searchlight will receive this score. We project that this candidate will most likely fall in the 40th-50th percentile in terms of on-the-job performance.
Risky: the bottom 20% of candidates across Searchlight will receive this score. Based on our historical data, candidates with this rating have a >70% probability of being a mishire -- which means not only are they not a great hire, but also likely to be a low performer.
Using the Searchlight Predicted Performance™, Hiring Managers and Recruiters have an ethical and predictive AI copilot to ensure hiring decisions are high quality and unbiased.
Why we made this change
We made this change for two reasons: 1) improved predictive power to be greater than Structured Interviews and 2) ensuring no adverse impact.
Predictive Power Greater than Structured Interviews
Previously, our Searchlight score gave a numeric percentile value for every candidate; however, as our AI has advanced, we have been able to add greater precision in evaluating candidates across many more factors.
The result of this is that we reached a major breakthrough in which the Searchlight Predicted Performance™ can outperform Structured Interviews. In a random test of >1500 sample candidates, we found, the Searchlight Predicted Performance™ outperformed or was on par with Structured Interviews for every major job group.
Compared to structured interviews, Searchlight Reference Checks are 1.3x more predictive in accurately identifying high performers for client-facing roles and on par with structured interviews for all other roles.
Compared to traditional reference checks, Searchlight Reference Checks are 2x to 3x more predictive in accurately identifying high performers.
Removal of Bias through Ethical AI
Core to our mission was to not only improve the predictive power of our AI, but to also ensure all improvements to our AI were made ethically to remove any possible bias.
At Searchlight, we believe building a diverse team and inclusive culture is win-win for candidates and organizations. The studies back this up — diverse teams financially outperform their peers by 36 percent (McKinsey reports on inclusion and diversity) and inclusive teams outperform their peers by 80 percent (Deloitte). But diversity isn’t a box to check, it’s a commitment to thinking about talent and non-traditional candidates in a new way. Using Ethical AI, we tie diversity initiatives to business outcomes at every part of the hiring process.
Searchlight’s AI systems are assessed continually through internal data reporting and periodically through an independent 3rd party audit from legal, policy and technical experts in AI. With Searchlight, you can be confident that your hiring process will be less biased, more effective, and compliant with current and upcoming laws and regulation.
Adverse Impact: Searchlight is proud to be the leading company in Ethical AI. We are audited by experts in AI against NYC Law 144’s Bias Audit standards and proven ourselves to meet EEOC guidelines, even though we are not an AEDT.
Getting Started
The new Searchlight Predicted Performance™ has been rolled out in August 2023. If you have any questions or feedback, please reach out to [email protected] and/or connect with Andrew, our Head of Customer Success.