Thank you for agreeing to be a reviewer for contributions to RSECon23! You are an essential part of making this conference a success, and therefore the Programme Team has put together some guidance to help you with what can be a tricky task. We have tried to balance detail with conciseness to respect your time, so if anything is unclear, please reach out to the Programme Team by email at [email protected].
Abstention
If there are any reviews you’ve been assigned but wish to decline due to, for example, a conflict of interest, or not feeling qualified to assess the topic, please send a list to the programme team. We have attempted to avoid any obvious conflicts at assignment but some non-obvious cases are likely to have gotten through.
Bias and inclusivity
The Society of RSEs is committed to creating a welcoming and diverse environment in our conference.
We ask that you be mindful of common undesirable biases when making your decisions. The first line of defence is awareness [Nature 526, 163 (2015)] so for your part, we ask that you simply review the definitions of relevant common forms of bias so that they are in the forefront of your mind as you read submissions.
Common forms of bias to watch out for
- Affinity bias
- Similarity bias
- Confirmation bias
- Affect heuristics
Bear in mind that submissions may come from people for whom English is not their native language, or text is not their preferred method of communication and ultimately the sessions will be presented live, so text will not be the final format. Please disregard any errors of spelling and grammar from your considerations, though constructive feedback is welcomed as speakers will have an opportunity to amend their abstracts prior to the conference.
Your decision
The recommendation for each submission can be: accept or reject. And each submission will have two reviewers. The combination of your decision with that of your (anonymous) co-reviewer will inform the final acceptance of the contribution. However, the final decision rests with the talks, workshops, and programme chairs.
🟢 Accept: This is judged to be a high-quality contribution which will add value either via seeding discussion, or presenting a complete and high-quality piece of work. It is well-suited to the audience of the conference.
🔴 Reject: This submission does not fit well with the audience of the conference and/or it will not add significantly to existing or previous discussions on the same topic. Note that this does not necessarily mean that you judge the contribution to be low quality. It’s expected that you would reject even high-quality contributions about a topic that is already well covered elsewhere or does not relate to research software engineering.
We also ask that you write a sentence or two of constructive feedback to the submitter, and a sentence of justification to the committee (which will not be shared with the submitter).
You will also be given the opportunity to flag up contributions that you think might be mis-categorised. For example, a submission for a talk which might be better suited to a panel discussion.
Submissions should not be explicitly scored for fit with the conference themes, but you may wish to highlight the fit into a theme in your comments to the committee. The themes will be used when scheduling submissions to create a consistent programme from the submissions recommended for acceptance. For reference, this year our themes are:
- Working with and as Researchers
- Open Science and Open Source
- Working with Industry and Commercialisation
- Tooling for Research Software Engineering
We have deliberately left these relatively vague and open to interpretation to encourage a wide range of contribution submissions.
There will be many good submissions, so it is likely that the programme committee will have to select from among the accepted abstracts and some abstracts which receive positive feedback will ultimately be rejected. Please bear this in mind when preparing your feedback, and consider suggesting ideas that would further strengthen a submission you would recommend to accept.
Selection criteria
To help you give your recommendation, we’ve drafted a set of criteria. You are welcome to copy and use the table below as a ranking system if this would be helpful, but you are afforded leniency in how and if you use our table. We suggest a score for each criteria out of 5, and a total score >15 would be accepted.
| Description | Your score / 5 |
|---|---|
| Facilitation of discussion This submission is likely to generate positive discussion. If it’s a workshop or hackathon, you’d expect it to be well-participated. If it’s a talk or panel there are likely to be good questions. | |
| Relevance to attendees This submission is relevant to, useful for, and will find an audience within the RSE community. Even for submissions within a specific domain, we would expect there to be a focus on aspects of interest to Research Software Engineers. | |
| Novelty and excitement around topic for attendees Does the work or topic to be presented bring something new to the table? Is what it brings something that will generate buzz? | |
| Quality of the work (if applicable). | |
| Format Will the submission be well structured, and is it realistic in terms of time and resource constraints? | |
| Abstract quality Ignoring the quality of the written language, how well does the abstract communicate the target audience, excitement of the topic, and what content will be included? | |
| Total score |