A Practical Guide for Chairs – Review Process by Stages
This manual is designed to accompany a Chair through the entire review process in Confy+, from setting up the initial page all the way through to Camera Ready stage.
In Confy+ the review process is divided into stages to split the actions across time. An efficient review process rests on an active TPC committee, which consists of 4 different roles: General/Program Chair, Track Chair, TPC Member, Reviewer.
The responsibilities of each role are listed in the individual stages and each has a dedicated section in this guide, it can be found below.
This section of the guide is intended for the following roles: General Chairs, Program Chairs, Track Chairs (they manage their track almost entirely the same way as a conference is managed, apart from Initial Setup).
Before the conference can be open for submissions, these aspects need to be configured in the setup:
Note: For EAI conferences, the Initial Setup stage is usually completed by the Conference Manager to allow the maximal possible time for accepting submissions. Being aware of the settings can still prove helpful during the review process.
In order to begin receiving submissions, at least one Standard Track needs to be created. Tracks are used to gather submissions and are distinguished into two categories: Standard tracks and Custom tracks. Tracks are created and managed in the ‘Create Tracks’ tab in the Global settings
Standard tracks move with the conference. There can be one (e.g. Main track), or multiple Standard tracks (e.g. by topics). Standard tracks inherit the conference deadlines and they all move through the stages simultaneously with the conference. Standard tracks must be created before submission opens.
Custom tracks have more flexibility and autonomy. Their deadlines can be changed individually and each can move through the review stages at their own pace (quicker or slower than Standard tracks). This allows you to open a Late track while the Standard tracks begin review, or to keep a Workshop open longer. Custom tracks can be created while submission is open or during the review process. This means that you can open submission even if you are still waiting to confirm Workshops, Special Sessions etc.
Review Model and Paper Length need to be set prior to the Paper Submission stage to ensure authors are instructed correctly during their submission. Both can be found in their respective tabs in the Global settings.
Set Review Model – the default mode is Double blind. If you wish, you can change this to change to Single blind. the Double blind model is recommended, as anonymity of submissions decreases potential review bias.
Manage Paper Length – the default is Regular Paper (10-20 pages). If you wish to create additional paper types (e.g. Short paper), this is managed here. You can also change the length of abstracts; the default is 200-1500 characters.
Next Stage: Once you have created at least one track, you can move to the next stage – Paper Submission. The conference (track) will stay in this stage until it is moved to the Validation stage after the submission deadline. Chairs should use this time to gather an international technical program committee. Read more about this stage below.
During this stage authors submit papers and the committee prepares for review. Making sure you have distributed the roles to the right people and that the TPC committee is sufficiently large to handle the number of incoming submissions allows for a more efficient review process after submission period closes.
The following actions need to be performed before moving to the next stage:
A distribution of roles between reliable members of the committee is a key element of a successful review. In order to prepare for the review process, invite Track Chairs and TPC members before the submission closes. Gathering a strong TPC committee is also an important element that aids the promotion of the conference, so it is beneficial to invite international and qualified members to join the TPC committee early.
To manage and invite TPC members and Track Chairs, click on Invite PC in Global settings:
The following roles are needed:
- Track Chair – each track needs to have at least one Track Chair. If you wish to take on this role, you can invite yourself as Chair for one, or all tracks. Track Chair manages the review of their track(s).
- Program Committee Members (TPC members) – make sure you have enough people invited to form a quality international TPC committee. A TPC member is responsible for their assigned papers during the review process. This means that they invite Reviewers, oversee that a sufficient number of reviews is delivered, and submit recommendation to Chairs.
- IMPORTANT NOTE: TPC member and Reviewer are different roles. Chairs can only invite TPC members at this stage. Then, assigned TPC members invite individual Reviewers during the review process. This is to ensure compliance with the Double blind review model.
- TIP: When inviting, you may want to select the role called ‘All Tracks – Program Committee Member’. This means that if you later create additional tracks, these people will be available to handle papers in these tracks too.
- TIP: It might come in useful to also invite yourself (and other Chairs that might want to be active in the review process) as the ‘All Tracks – Program Committee Member’ so you can later also handle some of the submissions.
- Publications Chair – this person helps with submission validation and prepares the proceedings after Camera Ready deadline. Usually, the Publications Chair is appointed in the SAF document and the Conference Manager adds them when creating the conference in Confy+. You can invite additional people to the role.
The Invite PC page has two sections – Invite (form to add new members) and Overview (to monitor who is involved with the conference, their roles and tracks).
Some deadlines are initially set by the Conference Manager (Submission, Notification and Camera Ready). The Chairs need to set the review deadlines in the ‘View Deadlines’ tab.
NOTE: For Standard tracks, the Conference deadlines apply (this means that if multiple Standard tracks exist, they all share the same deadlines and stages). The Standard track deadlines can only be changed on the homepage of the conference. For any Custom tracks, the deadlines and stage can be changed in each track separately. This allows for you to run a Late track, Invited track or keep a Workshop track open for submission after the Standard track(s) have closed their submission.
TIP: Moving stages within the review process triggers notification emails to various members of the committee, usually TPC members and Chairs. Therefore, it is useful to keep the deadlines updated and double-check them right before moving stages.
To see how many submissions are in the tracks, click on All Papers Overview in the Submissions settings.
Here, you will see the total number of submissions and a breakdown between tracks. You can click on each track and also on each individual submission to view and manage it.
TIP: above the track breakdown, there are various files available for download. These can come in useful if you will be working together with other Chairs outside of Confy+.
TIP: This page is useful all through the review process and in Camera Ready stage as well, it gathers statistics regarding the number of submitted, accepted or camera ready papers.
Next Stage: You should only move to the next stage after the Submission deadline, so make sure your program committee is aware of their responsibilities and prepared to carry out the review process efficiently when this occurs. The following stage (Submission Validation) marks the beginning of the review process, so make sure you complete all required actions before the Submission deadline.
In this stage, you need to go through all papers to ensure only qualified submissions continue to the review process and that all papers are placed in the correct tracks. This can be done by General/Program Chair, Track Chair (for their track only) or Publications Chair.
To begin, click on the button Validate (in red). The following actions need to be performed before moving to the next stage:
You can run the check on all submissions simultaneously, clicking Check All (in red), or you can do this for each paper individually (after clicking on each submission you will see the button to check plagiarism). The results are color-coordinated to indicate the level of plagiarism (over 30% is considered high and is marked red).
To validate a paper means to check if it is suitable to continue to the review process. This includes technical parameters, such as length, plagiarism (guide above), compliance with the double-blind review model etc., but also quality to establish a threshold.
If the paper is suitable, you can click Make Available for Review and it continues to the next stage.
A paper might not be suitable for several reasons, see the common ones with tips on how to proceed below:
- Format is incorrect (e.g. authors not anonymous, wrong template) – for these papers, the best solution is to create a Corrections track by clicking on Create Tracks on the conference homepage (link to tracks guide). You need to inform authors about the necessary corrections, so they can re-submit the paper into the Corrections track in the right format. Make sure to announce a deadline for corrections to avoid delays.
- TIP: After you have notified the authors, it is recommended to reject the original submission to avoid confusion with having multiple version of the same paper in the review process.
- Quality might not be adequate (e.g. paper is too short/long, a work-in-progress/study, outside of track scope) – for these papers, it might be suitable to Move Paper to a different Track (such as one that suits the scope or length better), like a Workshop, Poster, or Works-in-Progress track. In case you do not have alternative tracks, you can create them by clicking on Create Tracks on the conference homepage (link to tracks guide)
- If none of the above apply, you can Reject the paper here and it will not be considered for review.
- Note: Papers rejected during validation will count towards the rejected papers when you calculate your acceptance rate. In All Papers Overview, they will appear strikethrough.
Next Stage: When the number of Unprocessed papers is 0, you have completed this stage and you can return to the conference Homepage and move to Paper Assignment.
In the following stage (Paper Assignment), all invited TPC members will receive a notification that they can request papers for handling. Make sure you have invited the TPC committee in the Invite PC section –> Overview or do so now, before moving to the next stage (link to Invite PC).
Check the upcoming deadlines to make sure that they are set correctly. The TPC members will see these deadlines in the notifying email and it aids them to handle the review of their assigned papers efficiently. Having the reviews done on time is essential in order to notify authors at the Notification deadline.
As Chair, in this stage your task is to assign one TPC member to each paper. The assigned TPC member is responsible for handling the paper review in the subsequent stages – they invite Reviewers and based on the delivered reviews, the TPC member will submit a recommendation that you can use to decide whether to accept or reject each paper. For full description of TPC member responsibilities, click here.
When you entered Paper Assignment, all TPC members received a notification instructing them to express their interest in the individual papers. You can use their indications of interest to distribute the assignments. To keep the process efficient, it is best to divide the work among multiple TPC members, so each one only has a few papers to handle.
NOTE: If you want to directly invite Reviewers, you need to assign yourself as TPC member to all submissions. In case you do not see yourself in the list of TPC members, this means you have not been assigned this role. In the global settings, click on Invite PC, to invite yourself to the role ‘All Tracks – Program Committee Member’. Full instructions for inviting TPC members and other roles are available here.
Then, you can return to the page for Assignments and assign yourself to any (or all) submissions. This means that you will be responsible for review delivery for the assigned papers.
To view bids and assign TPC members, click on the ‘Assign’ icon in Next Steps (in red below). If your conference has several tracks, you have to do this for each track.
In the Assignment window, you see TPC members’ preferences and these can guide your assignment, see screenshot below.
TIP: If you want to make changes to a paper where a TPC member is already assigned, go into All Papers Overview –> Requests. See instructions for accessing All Papers Overview here.
Next Stage: You can move to Review Management once all papers have an assigned TPC member.
In Review Management, your role as Chair is more supervisory, compared to the previous stages. If all submissions have their TPC members assigned, you can monitor the progress of the review process. Click on ‘Make a Preliminary Decision’ (in red below) to see Review Information for all papers.
In this overview(screenshot below), papers are ordered from highest to lowest, based on their average scores. On the right hand-side, you can observe the number of completed reviews and invited Reviewers for each submission. Clicking on each paper opens its details and all activity that happened throughout the review.
TIP: You can also see the papers that have already received the TPC member Recommendation – they will be in green (accept) or red (reject). Once a Recommendation has been delivered, you can make the preliminary decision for those submissions already in this stage.
If you see that some papers are behind, you can do the following:
- Contact the TPC member to remind them of the approaching deadline
- Click on ‘Remind‘ to notify the invited Reviewers
- Invite additional Reviewers – the form is just below the invited Reviewers
- Deliver a Review – in order to do so, invite yourself as a Reviewer for the particular paper. You will then receive an email with the necessary information to deliver your review.
NOTE: If you as Chair have also assigned yourself as the TPC member or Reviewer, then you assume their responsibilities. Click below to view the role descriptions in Review Management:
The main objective of this stage is to ensure each paper has at least three reviews. As the TPC member, you are responsible for inviting these Reviewers, therefore your activity is crucial in order to move forward. You receive an email with the details (such as the deadline and next steps) when you have been assigned submissions. To handle the paper, log into Confy+ and click on My Assignments. Here, you can see all your pending assignments and you need to go through all of them to handle each one.
The following actions are needed in each of your assigned submissions:
- 3 reviews need to be delivered – this means you must invite Reviewers (it is recommended to invite at least 4-5 reviewers, as not all may deliver their reviews on time). There are two ways of inviting Reviewers in the Manage Reviewers section (highlighted in red on the above screenshot)
- Invite Reviewer – using the form where you fill in the Reviewer’s details (name, email, affiliation)
- Community Bids – Community Reviewers have already expressed interest in this particular paper, so they are motivated to deliver a quality review promptly. You can see Community Bids under the form.
- Submit recommendation – once the reviews have been delivered, you have to evaluate the Reviewers and submit a recommendation to the Chairs based on the reviews and your evaluation. This is an important indicator when making decisions, so it is advisable to try to complete the recommendation promptly.
Submitting the recommendation to Chairs is the final step of the TPC member responsibilities.
You have been invited to deliver a review via email. In order to complete this task, log into Confy+, click on My Assignments and here you can see a list titled My Pending Review Assignments. By clicking on each one, you can see the abstract and manuscript of the paper. In order to accept or decline the invitation, click ‘Submit a Review’ If you accept, clicking the button will take you to the review form. You can also decline this invitation.
Next Stage:You do not need to wait until all papers have 3 reviews delivered to move to the next stage, Reviewers can deliver reviews and new ones can still be invited once the conference is in Decision & Notification stage. You can begin deciding on the papers where you are certain to avoid a heavier workload at the end.
The final stage of the review process is deciding which papers to accept and reject. Bear in mind the required acceptance rate to retain quality of the main track and consider potential avenues for papers in alternative tracks (Workshops, Short papers, Posters) – for all EAI conferences, this can be consulted with the Conference Manager.
To complete the decisions, click on ‘Make’ a Preliminary Decision or ‘Make‘ a Final Decision to see Review Information for all papers (screenshot below, in red).
Here, the papers will be ordered from highest to lowest, based on their average scores. You can also see TPC member Recommendations – they will be in green for accept, red for reject, or grey if the recommendation has not yet been delivered.
Decisions have 3 stages in Confy+ before author notifications:
Recommendation should be delivered by the TPC member who is handling the paper, based on the reviews and their own expertise. However, if the TPC member has not delivered the recommendation, you as Chair can do this instead. The Recommendation requires evaluating Reviewers, filling in two text boxes (one for the author and one for the Chairs) and selecting Accept/Reject for the paper.
In the Decision window, you can recognize papers without recommendations, since they are marked grey.
See a screenshot of the form above. In order to complete the paper recommendation and Reviewer evaluation, you can see the paper pdf in ‘View Paper’ and the Reviewer ratings and comments in ‘Manage Reviewers’ (both are highlighted in red). In Manage Reviewers, you can scroll down to view the Review Details in order to read the full reviews. This information enables you to complete the form before moving on to the decisions.
Preliminary decision should be made by the Track Chair. If the General or Program Chair is handling this role, they have to make the preliminary decision, as well as the final decision.
See a screenshot of the form above. The paper will be highlighted (red for reject and green for accept) based on the recommendation made by the handling TPC member. You, as Chair, have to make the preliminary decision, based on the following information:
- Recommendation message by the TPC member
- Reviewer ratings and comments in ‘Manage Reviewers’ (you can scroll down to view the Review Details in order to read the full reviews if needed)
- The submission pdf in ‘View Paper’
This information enables you to complete the form and save your preliminary decision on each paper.. You can overrule the Recommendation choice.
Final decisions can only by made by a General Chair or Program Chair.
See a screenshot of the form above. The paper will be highlighted (red for reject and green for accept) based on the preliminary decision made by the Track Chair. You have to make the final decision, based on the following information:
- Recommendation message by the TPC member
- Preliminary decision message by Track Chair
- Reviewer ratings and comments in ‘Manage Reviewers’ (you can scroll down to view the Review Details in order to read the full reviews if needed)
- The submission pdf in ‘View Paper’
This information enables you to complete the form and save your final decision on each paper.. You can overrule the previous choice.
Notifications can only be sent once all papers in the track have their final decisions made. On the homepage of the conference, click on ‘Send‘ (see in red on screenshot below). You can make changes to the acceptance and rejection messages before hitting Send.
NOTE: If any papers are missing the final decision, you will not be able to send notifications. You can see the papers that are blocking your progress on the bottom of the page (see screenshot below).
Next Stage: Sending Notifications marks the final step of the review process and now the conference is in Camera Ready stage. During this time, authors register and submit the Camera Ready papers – the progress can be viewed in All Papers Overview.
In Camera Ready, authors register and submit the Camera Ready papers – the progress can be viewed in All Papers Overview.
