Create Interview Scorecards - Overview
Interview scorecards are a great way to help your hiring team make consistent hiring decisions and through structured feedback. The article below will go through the three different ways that this can be done:
Scorecard Criteria Library
Under Settings/Admin > Configuration, there is now a new page called “Scorecard Criteria Library” which allows administrators to perform the following functions:
Add new criteria
See list of all criteria that have been defined
Search through the criteria based on Name and Description
Filter list of criteria based on the org fields that have been set up (find more details under “Creating a new Scorecard Criteria” section below)
Sort the list of criteria based on Name (alphabetically), Created Date, Last Update Date
Under each criteria, administrators will be able to see the criteria name and description by default. In addition to that, they will also be able to perform the following functions:
Expand the criteria to see questions that have been added
Edit and Delete the criteria
Hover over the information icon to see additional information (last edited date users and number of active jobs this criteria is applied to)
Note: All edits and deletions will not affect existing jobs and reviews. Changes will only be applied on a going-forward basis (e.g. new jobs that are created after changes are made).
Creating a new Scorecard Criteria
After selecting the + add new button on the top right, administrators will be brought to the criteria details page where they can define the following:
Interview questions (optional)
Org field filters
Hiring Process and Hiring Step Assignment
Editable at job level (or not)
Typing in the Criteria Name will allow you to search through the existing default criteria library (refer to default scorecard criteria page for full list) and existing custom criteria. This will ensure administrators will not be creating duplicate criteria by accident. To copy from an existing criteria, they will only need to select the desired criteria in the dropdown and the relevant information will automatically populate for them to make the necessary edits.
You can also format scorecard descriptions to support the level of detail you want to include for the scorecards, as well as leverage hyperlinking to reference the right content onto the scorecards.
You can build detailed descriptions for scorecards
You can format your descriptions by bolding , italicizing, underlining, aligning your content as well as changing font color and size.
You can add hyperlinks to web pages or content directly into the scorecard descriptions. All hyperlinks open content onto a new browser tab / page.
The formatting is retained when the scorecards are displayed on the jobs on the ‘Hiring Process’ tab as well when you see the scorecards for evaluating candidates on the ‘Reviews’ tab
Org field filters will determine which jobs will automatically apply the scorecard criteria at the point of job creation. For example, if the scorecard criteria “Programming Skills” has “Department = Engineering” selected, then all jobs created where Department = Engineering will have “Programming Skills” added to its scorecard when such jobs are created. This will also apply to jobs that are created through public API.
Assignment to Hiring Process / Hiring Steps will determine what specific hiring process and hiring steps this scorecard criteria will be assigned to. Therefore, only jobs meeting the org field filters criteria AND assigned hiring process criteria will see the scorecard show under Reviews depending on which step of the process the scorecard was assigned to.
Editable at job level toggle will control whether Hiring Teams can edit the scorecard criteria in the job regardless of their scorecard permissions in Admin/Settings > Hiring Team Roles. If this toggle is “on” Hiring Teams with edit scorecard permissions will be able to edit and/or delete the criteria after it has been added to the job. If this toggle is set to “off”, then the criteria cannot be edited or deleted from the job.
Any changes made to the criteria at the job level will not feed back to the library and will also not be visible in other jobs.
Send changes to active jobs toggle if selected will apply the changes made to the criteria to all active jobs only. This will be helpful if there are minor edits such as spelling mistakes, clarifying information etc to criteria. By default, this toggle will always be set as "off" each time you edit a criteria (meaning changes will only apply to newly created jobs). This is to avoid having inconsistencies between the criteria that candidates are reviewed by within the same job.
NOTE: This option is exclusively for updating the criteria that is already assigned to jobs. This option will not add criteria to a job where it does not exist already. Send changes to active Jobs will not apply the criteria to a Job it has not already been added to.
The following are the Scorecard criterion field character limits:
- Criterion name: max 200 characters
- Description: max 2000 characters
- Questions: max 200 characters
Adding Scorecard Criteria to a Job
- SmartRecruit and SmartStart
As briefly mentioned at the beginning, there are 3 ways at which criteria can be added to the scorecard of a job:
- Have criteria added to the job from the Scorecard Criteria Library
- Add criteria directly to a job directly.
- Add a scorecard to a job that is saved as a template
How each of them works:
If a scorecard criteria has been defined in the library and the org fields configured matches the org fields of the job created, then the scorecard criteria will automatically be populated into the job and be visible in the job’s “Hiring Process” tab upon creation. This will also apply to jobs which are created through public API.
If the job template or the job that is being copied already has criteria included, all criteria will also be added to the job automatically (as per current behaviour). Meaning if users are creating a job from a job template and the org fields also match the scorecard criteria configured through the library, the job will include both sets of criteria (everything that was in the job template, plus the criteria that were added based on the scorecard criteria library configuration).
If a hiring team member with edit access to the scorecard would like to add criteria directly to the job, they can continue to do so in the job’s “Hiring Process” tab. The criteria search available in the “criteria name” field will no longer return criteria across all jobs. Instead, it will only return default system criteria and criteria from the scorecard criteria library. This will allow administrators to drive consistency and standardization across jobs, and also avoid hiring teams from being distracted by position-specific criteria created by other users.
Administrators and hiring teams will not be able to create criteria with duplicate names (in both the scorecard criteria library and within the job).
The only scenario where this may happen would be if:
“Criteria A” was added to “Job A” from the scorecard criteria library, then edits were made in the description to better meet the job’s needs (let’s call this “Criteria A (edited)”).
When creating “Job B”, the user decided to clone “Job A” (which contains “Criteria A (edited)”). And because the org fields also matched the scorecard criteria library setup, “Criteria A” is also added.
As a result, “Job B” will then see 2 versions of the Criteria A - “Criteria A” from the library, and “Criteria A (edited)” cloned from “Job A”. In this case, users will need to choose which criteria they would like to remove from the job to avoid duplicates which may confuse interviewers.
Scorecards when Building Jobs using Job Templates
When you build a job and save it as a template, all the scorecard criteria tied to the job will be saved in the job template (regardless of if the criteria were part of the job from the scorecard library or if they were manually added to the job).
When you create a new job and leverage a job template, all the scorecard tied to the template will be added to the new job you are creating using the template.
Editing or Deleting Scorecards brought over from Job Templates
When a job is created using a job template, scorecards from that job template carry over into the newly created job.
If the scorecard criteria is configured as editable in the library, any user with 'Write' permission to scorecard should be able to remove or edit the scorecard on the job. If scorecard is not configured as editable, the edit and delete buttons will be disabled.
Assigning Scorecards to Hiring Process(es) and Hiring Step(s) from the Library
As an admin, you can set up the system to automatically assign scorecards to specific steps within hiring processes, so that only the associated scorecards will display under the Reviews for candidate evaluation tab when a candidate is on a given step in the hiring process. For admins, this will reduce their dependence on hiring managers or recruiters to set up scorecards for each individual job as well as reduce the amount of time it takes to do the assignment.
By default, scorecards will be set to All for the hiring process, meaning they will be assigned to all hiring processes and all steps under those hiring processes. You can see this under the Hiring Process section when adding or editing a scorecard in the library.
To customize the configuration for step assignment, admins can select the Only Some option under the Hiring Process section and choose the specific hiring processes that should be associated with the scorecard. Under each selected hiring process, admins can select the specific steps that should be associated with the scorecard. By default, when a hiring process is assigned to a scorecard, all steps under the hiring process are assigned to that scorecard (unless the admin changes the configuration).
If a job matches the org field criteria AND the associated hiring process, the scorecard will be added to the appropriate matching jobs based on the org field association and hiring process/step assignment that is configured. If a scorecard is assigned to hiring processes and steps but not tied to any org fields, it will not show up on any jobs. To honor the assignment and carry the scorecard into the jobs, the scorecard must satisfy BOTH the org field configuration and the hiring step assignment configuration.
Here are some potential scenarios.
If a scorecard is matched to a job by org fields and assigned to all hiring processes, it will be added to a job with assignment for all steps for the associated hiring process tied to that job.
If a scorecard is matched to a job by org fields and assigned to only some hiring processes and if a job was matched with a hiring process that the scorecard is not configured for, the scorecard will not be added to the job.
If a scorecard is matched to a job by org fields and assigned to only some hiring processes and if a job was matched with a hiring process that the scorecard is configured for, the scorecard will be added to the job based on the assignment that was configured in the library.
If a scorecard is not matched to a job by org fields and assigned to all or some of the hiring processes, it will not be added to the job.
Assignment can be further modified for any job at the job level. Updating assignment of a scorecard does not disconnect the scorecard from the library.
If you choose to add a scorecard criteria to a job at the job level, then the assignment of it must also be configured at the job level, it will not carry over from the library based on the admin’s configuration. However, if the user does not specifically assign any steps when adding a scorecard to a job, then by default the scorecard is assigned to all the steps in the hiring process.
NOTE: Configurations only take place for any new jobs created after the configuration was applied. It will not modify how the scorecards take effect on any existing jobs.
What happens if I use a job template?
If you still use a job template, your scorecards will carry over from the template into the new job. However, assigning the scorecard to hiring steps is only depicted based on the configuration set in the library for applicable scorecards.
If there is a scorecard in the job template, that comes from the scorecard library inclusive of the assignment configuration. But if this scorecard is not matched to the new job, then the assignment for that scorecard will not be added/carried over to the new job.
How will the ‘Editable at job level’ option in the scorecard library function with respect to assignment?
The Editable at job level option in the scorecard library will NOT be inclusive of an assignment. It will only allow users to edit a scorecard definition. Even if a user cannot edit a scorecard at the job level, they can still assign/reassign it as long as they have the Write permissions. If a scorecard is manually added by the user to a job, the user can also delete the scorecard.
What happens to the assignment configuration in the library if the hiring processes or steps are changed/added/removed?
If any hiring processes or steps are removed, scorecard assignment will no longer be honored for those steps or processes. If hiring processes or steps are added or changed, the scorecard assignment must be updated by the admin appropriately.
If a scorecard is assigned to ALL hiring processes, any newly added hiring processes will automatically also be assigned to that scorecard. If a scorecard is assigned to only some hiring processes (this includes if all hiring processes are selected individually), any newly added hiring processes will NOT be automatically assigned to that scorecard.
If a scorecard is assigned to ONLY SOME hiring processes then some scenarios could be as follows if an admin has selected the ONLY SOME option for hiring process assignment:
A new hiring process is added: In this case, the scorecard will not be assigned to this process
An existing hiring process gets removed: If a scorecard was assigned to any other process then it would continue to be associated with those hiring processes. However, if the existing hiring process was the only process assigned and is removed, then it will not be possible to take actions on the scorecard without selecting a new process assignment. In the meantime, the scorecard will not be added to any job.
New steps are added to a hiring process that is already assigned: If a new step is introduced to a hiring process that is already assigned to a scorecard, the scorecard will automatically be assigned to the new step. However, if the scorecard was only assigned to some of the steps within a hiring process, and a new step is introduced to that hiring process, the scorecard will not be automatically assigned to the new step within that hiring process.
Steps are removed from an assigned hiring process: If no steps match, a scorecard will not be added to the job.
How does sorting on the scorecards criteria work in relation to the assignment?
If the Library feature is enabled, and criteria are brought over from job templates ONLY, the ordering will match the order on the template. If the criteria are coming both from templates and library automation, then the criteria from the template will follow the same ordering, showing at the bottom, and the criteria from the library will show on top with no sorting specified. When the scorecard Library feature is NOT enabled, criteria can only show on jobs from job templates, and ordering is the same as it is on the job template.
How does this work for delegated admins?
if a delegated admin has access to Scorecard Library, he/she will be also able to configure the assignment in the library
Adding Custom Criteria to a Job
Users can attach up to 80 criterias to a specific job - each can contain high-level descriptions of core behavioral competencies that are critical for professional success. When you add criteria to the scorecard, you can also add custom questions to help interviewers rate and review the candidate against the criterion. Each criterion can have up to 10 questions.
Scorecard character limits (applicable when adding scorecards directly to a job via the 'Hiring Process' tab):
- Criterion name: 250 characters
- Description: 1000 characters
- Questions: 1000 characters
To create a scorecard:
- Go to the Hiring Process tab on the job's profile and scroll down to the 'Interview Scorecard' section.
- Click Add and start typing in the Add interview criteria field. SmartRecruiters will match the query against the list of default criteria or criteria created in the scorecard criteria library.
- To choose a criteria from this list, just select the criteria name. The name won't be editable, but depending on whether the criteria is configured to be editable by your administrator, you may be able to change the description and/or add questions. You can also format scorecard descriptions to support the level of detail you want to include for the scorecards, as well as leverage hyperlinking to reference the right content onto the scorecards. The formatting is retained when the scorecards are displayed on the jobs on the ‘Hiring Process’ tab as well when you see the scorecards for evaluating candidates on the ‘Reviews’ tab
- When finished, click Add to save the criterion and its questions. The scorecard is immediately usable by the job's hiring team on the Review tab.
All criteria added to the job through this method will not affect the Scorecard Criteria Library and will also not be searchable in other jobs.
If custom criteria needs to be reusable on other jobs, please work with your administrator to have this added to the Scorecard Criteria Library or add the criteria to a job template so it can easily be reused.
Assigning Scorecards to Hiring Steps on Jobs
Once scorecards are associated with jobs, interviewers face a challenge to clearly understand which scorecards they are responsible for evaluating the candidate on during their interview. To address this, you can assign scorecards to various steps of a hiring process for any job. This enables the interviewers to only see the assigned scorecards for the hiring step the candidate is in, narrowing down the specific criteria they need to focus on for evaluating the candidate.
Once a job is created, as a user you will get a notification and task in the dashboard to assign scorecards on a job and ensure they are tied to specific hiring steps for the job’s hiring process.
Assigning scorecards to hiring steps for a job
1. Within a job, you can navigate to the “Hiring Process” tab and move down to the ‘Interview Scorecards’ section. Here you will find a list of all the associated scorecards for that job.
2. For each scorecard listed, you can click the ellipses (triple dot) icon and click the ‘Assign’ link which allows you to choose the specific hiring steps this scorecard should be assigned to (or evaluated on) and save the assignment.
NOTE: By default, all steps are assigned to all scorecards for a given job, unless configured differently.
3. Assigned hiring steps will be listed as tags for the scorecard.
4. If you are creating a new scorecard for the job, you can also assign the scorecard to hiring steps during the scorecard creation process. Simply choose the hiring steps from the ‘Assign to Step’ section before saving the scorecard to the job.
NOTE: As an administrator, it is important to note that any changes to the hiring processes associated with existing jobs should be changed carefully, as this may affect the assignment of scorecards to hiring process steps.
Unassigning scorecards from hiring steps for a job
1. Within a job, you can navigate to the “Hiring Process” tab and move down to the ‘Interview Scorecards’ section. Here you will find a list of all the associated scorecards for that job.
2. To unassign the scorecard from hiring process steps, choose the scorecard, and click the ‘Assign’ link again under the ellipses (triple dot) icon, and uncheck the steps that the scorecard should not be associated with and save the changes.
3. You can also simply choose to click the ‘X’ on the steps listed as tags for that scorecard. This allows you to unassign the scorecard from that step as well.
Scorecards and Job Templates
When you add a scorecard to a job that is saved as a template, the scorecard becomes part of the template, meaning when users creates a new job with the template, the scorecard of the job template will be copied over as well. Any changes made to the job template will only be applied to newly created jobs (and will not impact existing jobs).
When interviewers go to provide feedback on the Reviews tab of the candidate profile, they'll be able to:
- View the criterion's definition, and any questions related to it.
- Rate the candidate against each criterion.
- Provide additional comments, up to 20,000 characters!
Q: Will assigning a scorecard on a job, disassociate the scorecard from the library?
A: No, assigning a scorecard on a job is not the same as editing a scorecard on a job. Assigning scorecard to hiring steps does not dissociate the scorecard from the scorecard library.
Q: Can we control who has permissions to assign a scorecard on a job?
A: The permissions that control Write and Read for Scorecards are also tied to assignment of scorecard. You can only assign scorecards if you have Write permissions for scorecards as part of your role. This is managed under Settings / Admin > Permissions > Hiring Team Roles. You also need to ensure that your permissions are not set to “None” for ‘Internal job details’. This will prevent you from seeing any scorecards, hiring process or hiring team since you do not have access to internal job details.
Q: What happens to the scorecard assignment if the hiring process changes?
If the hiring process changes completely, because the job was matched with another hiring process, the scorecards' assignment is reset to the default state where all scorecards are assigned to all steps. If the hiring process stays the same but the steps within this hiring process change, it will be reflected on the scorecards' assignment. If the step is added, all scorecards with assignment to all steps will be automatically assigned to this new step. If a step is deleted, it will no longer show up in the assignment.
Q: Would I be notified if any changes impact the scorecard assignments?
There is no direct notification, but there is a note added under the settings, which reminds the user of the implications to scorecard assignment, if the hiring process is modified.
Learn more about filling out scorecards when reviewing a candidate.