Assignments & Progress

Purpose

Assignments & Progress holds the assignment list. It is where Auto Assign results land and where you assign them, where you fix any single submission or reviewer, and where you follow the reviews and email the reviewers who owe one. Approving a suggestion is what gives the reviewer the review task and the notification.

When you use it

• After Auto Assign, to assign its suggestions (the link Review and approve in Assignments & Progress opens the list on the submissions with suggestions).
• To find submissions that still need reviewers, and reviewers with no submissions or too many.
• Whenever a reviewer must be added, replaced or removed.
• While reviewing runs, to see which reviews are in, missing or late, and to remind those reviewers.

Walkthrough

1. 1

   Open Review → Reviewer Assignment → Assignments & Progress.

2. 2

   Read the summary: Submissions, Covered, Need reviewers, Suggested, Assigned, Reviewers (with how many have no submissions) and Conflicts. Click a number to jump to the matching rows; hover it, or click the ? after the numbers, for what it counts. Covered and Need reviewers count papers: a paper is covered when its assigned and suggested reviewers together reach its target. Suggested and Assigned count reviewer and paper pairs: suggested ones come from Auto Assign and wait for your approval; assigned ones are approved (assigned, waiting for the reviewer, or reviewed). So a paper can be covered by suggestions alone.

3. 3

   Choose View by Submission or View by Reviewer. Next to it, pick a submission type (Paper, Abstract or Proposal, with how many of each; a type with none is not shown) and a submission category in Category (or All). The summary and the list follow both.

4. 4

   Pick a filter in Show (All, Needs reviewers, Covered, Has suggestions, Has assigned, Has conflicts; View by Reviewer offers All, Has suggestions, Has assigned, Has conflicts, Over load and No papers; a reviewer's load is always their full total in this program, every type, against their own Max Papers or the program's Max Papers Per Reviewer), or search by submission number, title, category, reviewer ID or name. Sort by any column header that shows arrows.

5. 5

   View by Submission, each row shows the Submission, its Submission Type and Submission Category, and its reviewers as chips: the reviewer ID with its mark on top, the reviewer's name underneath. Hover a chip for the email. View by Reviewer, Load shows the submissions held against the most per reviewer, for example 6 of 8, in red when over. Click the eye icon beside a paper ID, in either view, to open its Submission Details in a window you can drag by its title bar; Prev and Next step through the submissions listed.

6. 6

   Click Assign All Suggested, at the right end of the sub-tab bar, to assign every suggested reviewer at once, or the check button on a row to assign that row's suggestions. A window first shows how many pairs and reviewers it covers (suggestions hidden by the type or category filter are included). Keep Email each reviewer their new papers on to send each reviewer one message, by email and in their Webmail, listing their new papers and when their reviews are due. Edit the subject and message there: {{name}} becomes the reviewer's name, {{papers}} their list of papers and {{reviewDeadline}} the review deadline from Important Dates (the paper's category's, else the program's). Click Preview email and pick a reviewer to see exactly what they will get, then click Assign and email (or Assign with the email off). Every new assignment also gives the reviewer a review task.

7. 7

   Click the edit button on a row to open Edit Submission Reviewer Assignment (or Edit Reviewer Assignment in the reviewer view). Add a reviewer (or submission) by search, assign or decline a suggestion, remove an assigned reviewer, or Replace one: click the swap button, then pick the new reviewer in the search. Each pick asks first: a small window shows the reviewer's number, name, organization, email and affinity, and the pair changes only when you click Add (or Replace). What you add is marked Just added at the top of the list. The search covers every active reviewer of the program, also those Prepare matching data has not paired with the paper yet; they show no affinity, and adding one gives them a review task like any assignment.

8. 8

   Removing or replacing an assigned reviewer asks first, with an email to that reviewer saying the paper is no longer theirs (the Papers taken off template). Check Preview email, edit the message if you like, and click Remove and email (or Replace and email). The change is saved first and the email sent right after, with a copy in their Webmail; a note at the top of the window says how both went. Clear the checkbox to remove without an email. Declining a suggestion sends nothing, since the reviewer was never told about it.

9. 9

   A reviewer who has already written (a draft or a submitted review, for example a review written only by AI) can be removed or replaced too. The window warns that their review of this paper goes with them: it no longer counts in scores or decisions, and the authors, the other reviewers and the reviewer no longer see it. It is kept only for the chairs' record, with an optional Reason you type (for example "written by AI"). This works also after the decision is released. Edit the email if the default "You do not need to review them" does not fit.

10. 10

    Follow the reviews. The Status column lists, one line per status and each in its own colour, how many reviews are late (red: not submitted after the review deadline, in the program's time zone), draft saved, not started, waiting for reviewer, submitted, suggested or in conflict. Each reviewer box takes the colour of its status too; hover it for the deadline and any reminder sent. The summary adds Reviews submitted (reviews in from assigned reviewers), Reviews missing (reviews still owed), Late and Conflicts (pairs with a conflict of interest, shown as red boxes); click one to list its rows. Show has the same filters. The columns Type and Category are the submission's. Under the reviewer boxes, 2 of 3 is how many reviewers the submission has (assigned and suggested) of its target, the number for its category in Auto Assign (or the program's Target); 6 of 2 means more reviewers than the target. Click the ? beside Reviewers for a guide to the column.

11. 11

    Email reviewers. Select rows (the header box selects the page; Select all N matching selects every row of the filter), then click Email selected. In View by Reviewer it emails the selected reviewers about the reviews they still owe; in View by Submission it emails the reviewers of the selected submissions whose review is not in (the window says who is left out). To email those left out as well, check Also email the reviewers with no review owed (off by default); their email lists the papers they already reviewed, so pick Custom if the Reminder text does not fit them. Pick Reminder, Late review or Custom, edit the subject and message ({{name}}, {{papers}} and {{reviewDeadline}} are filled in for each reviewer) under Edit message, click Preview email and pick a reviewer to see exactly what they get, then Send. Each reviewer gets one email from the program's committee (replies go to the program's contact email) and a copy in their Webmail. The Reminder and Late review texts are the program's templates in Settings, Review Config. For more than 5 reviewers a progress bar shows how many are sent. You can close the window: the emails keep going.

12. 12

    Check what went out. Recent emails, above the list, shows the last sends (Email selected, Assign All Suggested, Clean all assignments): when, which email, who sent it and how many were sent. Click one to see every reviewer's outcome: Sent, Not sent with the reason, or Unknown. In View by Reviewer the Email column shows the last email each reviewer got (green sent, red not sent, amber unknown); hover it for the last few, automatic reminders included.

13. 13

    Not happy with the assignments? Click Clean all assignments, next to Assign All Suggested, to start again. The window says how many suggestions and approved assignments it removes, and lists the ones that stay: a pair whose reviewer saved a draft or submitted a review is never removed. Removed pairs lose their review task. Check Email each reviewer the papers taken off (off by default) to send each reviewer who loses approved papers one message listing them; edit it and preview it as with Assign All Suggested. Click Remove and email (or Remove). Then change the rules in Auto Assign and click Start Auto Assign.

Tips