Review Groups brings flexibility to the review process, allowing the sharing of scores across scholarships in the same group. The article below includes a full overview of this feature. If you'd prefer to learn about a specific step, feel free to click the links below:
Review Groups are configured in the Review tab (in the left navigation menu).
Creating a Review Group
To create a new Review Group, navigate to the Review tab and click the ‘Add New’ button in the upper right hand corner of the screen.
The Details tab will appear, allowing you to begin building your review group. Begin by entering a name for the group. Next, feel free to add any instructions for your reviewers in the 'Instructions for Reviewers' box. This is where you can provide additional information that your reviewers should refer to during the scoring process.
For example, you might want to provide a detailed scoring rubric for them to reference during review to help them score the students appropriately. Since the instruction box is optional, you can proceed without adding anything in this box. You can always come back at a later time to edit instructions.
Next, select which scholarships will be included in the group. Click on the 'Add Scholarships' dropdown to view your list of scholarships and start checking the checkboxes for those you want added.
Note: If you are utilizing the Departments feature you can first select the department that you are creating the review group for and that will populate a list of scholarships within that department only.
As scholarships are added to a review group, they will be removed from the dropdown (since they cannot be added to more than one group). Click on the ‘Save’ button to create the review group.
After the Review Group is created, you'll notice additional tabs where you can configure options for the review group.
Note: You'll notice that the deadline for review will be visible next the review group name. This can be edited/customized under Settings> Award Cycles> Award Cycle End Date.
In case you created the review group by mistake and need to delete it, on the Details tab of the review group you'll find the 'Delete' button under the list of scholarships that you've assigned to the review group.
Options for Blinding Applicant Data
The Question Visibility tab allows you to hide applicant names and specific application questions from Reviewers within the selected review group. To access this tab head to the Review tab, and select a review group of your choice.
You'll first notice that the Applicant name feature is disabled, which means that the applicant's full name is by default hidden from your Reviewers. In the case that you want it to be visible, you can check the checkbox, and your changes will be automatically saved.
Note: This will not blind the applicant's name from any uploaded documents, any essays the applicant has provided or in any recommendation letters submitted on behalf of the applicant.
Next, in the Question Visibility for Reviewers section, you'll be able to configure which questions will be visible to Reviewers when scoring applicants. You'll notice that this area is categorized by General Application questions (further categorized by your application sections) and Before Applying questions.
By checking the checkbox next to a question or application section, you will be making them visible to Reviewers. Any question left unchecked will remain hidden. Again, no need to click save, as your changes will save automatically.
Note: You'll notice a total number of questions indicator populate in the right corner for General Application questions, and separately for Before Applying questions, specifying how many questions the Reviewers will see in total.
Configuring Score Cards
Score cards are used by Reviewers to evaluate and score the applications they are reviewing. Select a review group (via the Review tab) and click on the Score Cards tab to get started with creating scoring criteria.
To create a score card, click on 'Manual Review' under the Create New Card section.
Enter your Score Card Name (e.g. Financial Need) and a Total Possible Score (e.g. 5). Click the 'Save' button to save your new score card in the review group.
You can add as few or as many score cards as you’d like, which will all populate under the Score Cards section on the page.
Note: In order for applications to be visible to a Reviewer, you will need to create at least one score card in the review group.
To remove a score card from a review group, expand it and click the 'Delete' button.
Configuring Reviewer Assignments
Under the Reviewer Assignment tab you can add Reviewers to score applicants in the selected group.
To get started, select a review group (via the Review tab) and click on the 'Reviewer Assignment' tab.
Next, click into the search box under 'Reviewers & Assignments'. A list of reviewer names will be displayed. Either scroll through the list or start typing a name to find the reviewer(s) you'd like to add. Click on the reviewer's name to add them to the group.
Note: If you cannot locate a reviewer in the dropdown list, this means that they do not yet have an account. To learn how to create an account for them, click here.
After adding the desired Reviewers, click the dropdown menu that says 'Select One' under How Reviewers are Assigned. This enables you to choose how applicants are distributed to your Reviewers.
There are 3 assignment methods you can choose from:
- All: assigns every applicant to every reviewer (e.g. if you have five applicants in the group and two reviewers, both reviewers will be assigned all five applications to review). After selecting this method and clicking 'Save', the assignment will automatically be updated to assign all applicants to all reviewers.
- Randomly: assigns applicants to reviewers randomly and equally (e.g. if you have 20 applicants in the group and two reviewers, each reviewer will be assigned 10 applicants). This method allows you to set a number of reviewers per applicant by entering a number in the 'Number of Reviewers per Applicant' box. After doing so and clicking 'Save', the assignment will automatically update to randomly distribute the applicants.
- Manually: allows you to assign applicants manually. With this method you can also set a number of reviewers per applicant by entering a number in the 'Number of Reviewers per Applicant' box. After doing so and clicking 'Save', you'll be able to manually assign applicants to reviewers.
In the lower right corner under 'Unassigned Applicants', you will see a list of applicants who have not yet been assigned to any reviewers.
For example, to manually assign an applicant to a reviewer, check the checkbox by his/her name and click Assign to Reviewer.
Click on a reviewer name to select them as the reviewer for this application, then click 'Done' to complete the assignment.
The number of Applicants for the corresponding reviewer will increase.
To unassign an application from a reviewer, click the 'Edit' button by that reviewer’s name.
A full list of students assigned to this reviewer will be displayed. Click the checkbox by the applicant’s name and click 'Remove' to unassign them from this reviewer's assignments.
Applicants
This tab will list all applicants in the review group that need to be scored or are currently being scored. An applicant is displayed here once, regardless of them being applied to one or more scholarships in the review group. The applicant's reviewer score is visible here once scoring has begun.
Reviewer View with Review Groups
Reviewers only have access to the applications that are assigned to them for review.
Under the Question Visibility tab in a review group, you'll find the option to display the applicant's full name to Reviewers. This means that by default the applicant's full name is hidden from Reviewers. In the case you want to make it visible you can check the checkbox.
Note: Blinding applicant names does not hide names in all areas where names may have been used (e.g. document uploads, letters of recommendation, etc).
List of applicants with blinded names:
List of applicants with their full names displayed:
In order for Reviewers to score applicants, upon logging in they will need to select a review group; there, they will find a list of applicants they were assigned to.
Once they select an applicant, they'll have access to all questions from the application that were made visible to them and the score board. Their score board includes a section for notes which is optional for them to complete. Once they finish scoring an applicant they can click on Next Applicant or the Next button.
On every applicant's page they will also have access to the Instructions tab where they can read instructions for scoring that you've provided.
If you are needing or wanting to Impersonate a Reviewer user, click here to learn how.