SmartAssistant's MatchScore is a discrete score on a 0 to 100 scale, calculated using the information in a candidate’s application and in the job advert, that provides an interpretation of the candidate’s fit for a particular job.
How is the MatchScore calculated?
SmartAssistant will process the information from the job description by looking at key things like job title, required years of experience and skills, and education and produces a set of criteria for the job from this information. It will complete a similar process for the candidate, again considering the experience, education, skills, and previous company from their application. The MatchScore is then calculated by comparing these two criteria sets to determine how much of a fit the candidate is based on the job description.
Note: Only data in the Job Description, Qualifications and Additional Information fields on a Job are considered when calculating the MatchScore. The Company Description field is not processed because it isn't relevant to the specific job. If the Company Description field is the only one populated then no MatchScore will be calculated for candidates associated with this job.
The score is calculated when the candidate’s application is added to SmartRecruiters (by the candidate or a recruiter), and doesn’t change once calculated. As the details of the job are a critical input to the calculation, candidates will receive an independent MatchScore for each job. Candidate who apply to multiple jobs will have different MatchScore calculated for each application. Candidates will never see their MatchScore and this information should never be shared with them.
SmartRecruiters currently calculates the MatchScore for candidates who apply with applications in English, Chinese, Croatian, Danish, Dutch, Finnish, French, German, Italian, Norwegian, Portuguese, Slovak, Spanish and Swedish to jobs written in any of those languages.
Where can I find the MatchScore?
Assuming the logged in user has been assigned the correct access to SmartAssistant, the MatchScore can be found in these two locations:
1. On the Job Page
2. On the Candidate Profile
How can I use the MatchScore when reviewing applications for a job?
You can filter candidates in the Applicant list of a job by their MatchScore using the slider selection to chose the range of MatchScores to display. When you use this filter candidates without a MatchScore will be moved to the bottom of the list. Those without a MatchScore are included by default but if you wish to exclude them you can do so by unchecking the Include applicants without MatchScore option.
What are the MatchScore highlights?
On the Candidate Profile, you’ll be able to review a summary of the attributes which contributed to the calculation of the MatchScore for that application. This summary is broken down into different categories, such as Work Experience, Skills, and Education.
- Work experience attributes show job titles that a candidate has held and companies where the candidate has worked.
- Skills attributes lists the relevant skills found on the candidate's profile
- Education attributes shows the name of the schools that a candidate has attended and the name of the majors/degrees and names of certifications found on candidate's profile.
Can recruiters provide feedback on a MatchScore calculation?
With any algorithm like MatchScore, feedback is important to make sure that accuracy and precision are maintained and areas of improvement are highlighted. In the MatchScore Highlights screen there’s a section for providing feedback where you can let us know whether the MatchScore was useful. This feedback is reviewed by our Data Science team who use it to enhance the calculations behind the scene which ultimately helps to improve the quality of the scores.
Keep in mind that selecting Yes if the MatchScore was useful is a quick and simple step to provide feedback which gives validation that in this situation the score reflected the quality of the candidate well.
How can I provide useful feedback if I disagree with the MatchScore?
If you select the No option to the Is this information useful? question, you will be presented with an input box to provide additional details. As these are viewed manually by the team, we encourage that you provide as much details in the General Comments box as to why you selected one of these options.
✖ Example of Bad Feedback: "Score was bad"
✓ Example of Good Feedback: "Score should be higher because the candidate had a Professional Scrum with User Experience (PSU I) qualification which is very sought after for this job type"
How is the feedback processed and when will we see the outcome from them?
The data science team will review the part of the algorithm which is associated with the feedback provided to identify ways that improvements can be implemented. Any manual review of feedback is not going to be as efficient as it would be if feedback was used to update the models unsupervised, which is our ultimate goal, but unsupervised learning can lead to issues with biases so we want to be certain we have the right level of controls in place before we move to that type of feedback model.
The product is built to work for our entire user base so when changes are made to the underlying machine learning models and deployed they will benefit all companies. Improvements will be communicated via the Delights process.
Why might a MatchScore not be calculated for a candidate?
If you don’t see a MatchScore for a particular candidate, it’s most likely because SmartRecruiters doesn’t have sufficient data to calculate one. If the candidate didn't provide enough information on their CV or there were issues with the CV information being parsed then no MatchScore can be calculated. When reviewing a candidates profile, if the Education and Employment fields are empty or have very limited information in them, no MatchScore can be calculated.
Will MatchScore recalculate if new information is added to a Candidate Profile?
In the situation where a candidate applies with limited/no CV information (for example via LinkedIn so containing Job Titles/Dates only) but later provides a detailed CV, once that CV is uploaded to the Candidate's record, by the recruiter, the associated MatchScore(s) will be recalculated to take into account the new information available.
What does it mean when I only have low MatchScores for a job?
If you are seeing consistently low MatchScores for every applicant for a job the best place to start is to review the Job Description again. As this is a key input into the calculation, if the description is vague, short or very general then it will be hard for SmartAssistant to differentiate between the candidates because there will be less information it can use to separate and score them. If you review and update a job description, it will trigger a recalculation of the MatchScores which should result in a better spread of scores.
If, after reviewing the job description, the scores are still low this could be an indication that the quality of candidates are not at the required level. External factors could play a large part in this if the market is slow or the skills being sought are scarce.
What should I check first if I think the MatchScores for a job seem incorrect?
Often the Experience Level field doesn't accurately reflect the level of the job. For example, a very senior position could be set to have a very low experience level when the job was originally setup. SmartAssistant uses this information as an input into the MatchScore calculation so any mismatches could result in MatchScores which don't reflect the experience a candidate has to the precision that it should and could.
How is University/Institution data used when calculating MatchScore?
The impact that a candidate's university/institution has on the overall MatchScore is limited because it is not used in a generalised way (ie, candidate went to x university so they get a higher score). Instead, we hold reference data related to which institutions are recognised as being leaders for a given subject and we must identify, with high confidence, both that education is an important factor for the Job and the mapping of the candidates institution/degree subject to our reference data. If we don't have this high confidence then no further action is taken. If we have the required level of confidence, we would only consider this reference data to provide an uplift on this element of our scoring calculation and the size of any potential uplift is small but could potentially be enough to separate identical candidates who all have the same degree subject but from different institutions when the final MatchScore is calculated.
Can I get an extract of MatchScores?
The MatchScore isn't available in Report Builder and we have no plans at this time for it to be added. The rationale behind the decision is that although a MatchScore could/should influence who you chose to take forward in the process, there are a large number of external factors that impact the final hiring decision which in turn affects any correlation between them. We would suggest tracking a metric like the number of days applications remains in New status, which should drop, to determine the effectiveness of the SmartAssistant product.
Does MatchScore take into account requirements for spoken languages for a Job?
At the moment this isn't reflected in the MatchScore calculation. The challenges we face with this is how to handle the cases where the spoken language isn't explicitly called out on the CV because the absence of it doesn't always indicate they don't have that skill. If this were to be added in future, its impact on the score wouldn't be too strong because these is no guarantee that a candidate would include it on a CV and therefore ranked lower than someone who did. If language is a must have, a screening question would be a better way to highlight they have this skill and it could be set up as a knock out question if you wouldn't want to consider that person at all without it.