Guidelines for Reviewers

AMEE MedEdPublish – Guidelines for Peer Reviewers
Version 01: July 2018
 
 
Contents
 
1.            General
2.            Layout of the Review     
3.            Further Requirements
3.1          Have the required expertise
3.2          Declare competing interests
3.3          Set personal influences aside
3.4          Offensive or inflammatory language
3.5          Correctly identify oneself
3.6          Guard against recommending your own papers
4.            How to submit a review 
5.            Star-Ratings
6.            Other Reviewers              
7.            Participation Points and levels     
8.            Further Acknowledgement           
9.            Further Documentation
 
 
AMEE MedEdPublish – Guidelines for Peer Reviewers
Version 01: July 2018
 
Please note that the Review Guidelines of AMEE MedEdPublish are intended to conform to the COPE Ethical Guidelines for Peer Reviewers available at: https://publicationethics.org/files/Ethical_Guidelines_For_Peer_Reviewers_2.pdf
and the ICMJE Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals, Updated December 2017, available at: http://www.icmje.org/icmje-recommendations.pdf
 
1. General

How it Works:  
To view a diagram of the AMEE MedEdPublish process from submission to final revision, click here

AMEE MedEdPublish is post-publication peer-reviewed.  Your reviews are essential for maintaining the standards of the journal.  Post-publication peer review should be conducted as a constructive discussion, the aim of which is to enhance understanding of the subject and refine future work.
 
Some reviewers will have been contacted by AMEE MedEdPublish specifically to write a review.  These reviewers are requested to place their reviews as promptly as possible, although ensuring that the quality of the review is not compromised.
 
Critical but constructive feedback is encouraged.  Reviewer’s names are published with their reviews, and will appear immediately.  To stimulate discussion and ensure an open and transparent process, authors are encouraged to reply to their reviews.  Comments deemed unintelligible, inappropriate or offensive will be removed.
 
2. Layout of the Review
Reviews should:
  • Be written in good English;
  • Average around 300 words in length;
  • Outline both the strengths and the weaknesses of the paper;
  • Give a flavour of the interested targeted reader.
 
Although AMEE MedEdPublish does not prescribe headings, some useful guideline questions for a reviewer to ask, where appropriate, are:
  • Is the topic appropriate to medical education?
  • Is the Abstract an accurate and concise description of the information in the paper?
  • Does the Introduction supply enough background to the paper, including reference to the relevant literature, and the specific goals of the paper?
  • Are the Methods adequately described (to the point that, under ideal conditions, they could be re-produced)? 
  • Are the Results displayed in a way that is logical and understandable, and is there some degree of analysis or commentary?
  • Does the Discussion explore deeper implications of the results (usually referring back to the information contained in the Introduction)?
  • Are the Conclusions supported by the data in the Results and the Discussion?
  • Where appropriate, have alternative interpretations and / or weaknesses been identified?
  • Are there structural, logical flaws, errors of misinterpretation of referenced literature, presentation of data?

     
Although it is useful to reviewers to draw on their own expertise on the subject, they should remember that the review is aimed at reviewing the paper, and the focus of the review should always be on that.
 
3. Further Requirements
(Adapted from the COPE Guidelines at: http://publicationethics.org/files/Peer%20review%20guidelines.pdf)
 
3.1 Have the required expertise
Reviewers should agree to review only articles for which they have the required expertise.  If there are significant areas in which the reviewers do not have expertise, they should identify this clearly in their review, or consider not reviewing the paper.
 
3.2 Declare competing interests
Reviewers are asked to declare any competing interests.  Competing interests may be financial e.g. holding a patent or receiving fees from a company that may lose or gain financially from the publication of the paper, personal, religious, political or other non-financial interests.
 
In addition, if the reviewer is from the same institution as the authors of the paper, it would be preferred if they do review that paper, especially if the reviewer is on the Review Panel and/or is a senior member of staff at that institution.  If such a reviewer does feel compelled to write a review, then they must state explicitly that they are from the same institution as the authors.
 
3.3 Set personal influences aside
Reviewers should not be influenced by issues of a personal nature, such as the paper’s country of origin, religion, culture, gender, etc., unless these issues have a direct bearing on the content of the paper, in which case, this should be stated explicitly.
 
3.4 Offensive or inflammatory language
Reviews should be written in a formal academic manner, avoiding language that is offensive, inflammatory, libellous, unfair, or derogatory.
 
3.5 Correctly identify oneself
As reviews are published in one’s name, it is essential that reviewers correctly identify themselves by name.  While common name variations are permitted (e.g. “Bob” for “Robert”), if the name varies substantially from the person’s name by which they are normally known, they may be requested by AMEE MedEdPublish to alter the name in their profile.
 
3.6 Guard against recommending your own papers
Reviewers who see shortcomings in the literature cited in a paper will frequently wish to recommend that the authors read and refer to particular sources.  As reviewers will frequently write reviews on papers in areas of their own expertise, it is likely that they will have written their own papers on those topics covered by the papers that they are reviewing.  As a result, they may wish to recommend their own paper as source.
 
In this situation, they should take care not to recommend their own papers merely for the sake of boosting citations.  If they really think that their own paper will be of benefit to the author, they should declare their identity as the author of that paper, in order to clearly indicate a possible conflict of interests.
 
4. How to submit a review

In order to submit a review you will need to log in with your AMEE or AMEE MedEdPublish username or password.  (We strongly recommend that you use only one account, so that your reviews and contribution points are easily identified as yours).  If you are not an AMEE member or have not previously registered on the AMEE MedEdPublish website please consult our User Guide  to follow the registration process.
 
Once you have logged into AMEE MedEdPublish, go to the paper you wish to review.  Once you have opened the paper:
  1. Click on the ‘Reviews’ button to the right of the paper.  This will take you down to the bottom of the paper where you will be able to see any existing reviews for the paper.
  2. Click on the ‘Submit Review’ button. This will open up a text box into which you can type or copy and paste your review of the paper. 
  3. Once you have entered your review, check the box to declare that you have no conflicts of interests. Only reviewers with no conflicting interests will be allowed to submit a review.
  4. Rate the paper by clicking on the appropriate star-rating.  The number of stars displayed in blue will indicate your decision.  Guidance on the rating system can be found by clicking the link beneath the stars. Each review must indicate a rating for the paper.
  5. Click on the ‘Submit’ button next to the star ratings to submit your review.
 
Your review should appear immediately, and you will receive a confirmation email to your registered address.  You may also be informed if any replies to your review are received. To decide what activity you are alerted to, go to ‘My Account’, click on the ‘Notification Preferences’ tab,  and scroll down to check the boxes of those options you wish to receive notification of.
 
5. Star-Ratings
Once an article has been reviewed, the reviewer is asked to consider the extent to which they recommend the article by selecting one of the following:
★★★★★   I strongly recommend this article as an important contribution to the field that should be read by everyone with an interest in the area.
 
★★★★   I recommend this article as a useful contribution to the field that should be read by those with an interest in the area.
 
★★★   I recommend this article to be read by those working in the field despite having some reservations as noted in my review.
 
★★   I have reservations about this article as noted in my review but it may be of interest to those working in the field.
 
  I have serious reservations about this article but aspects may be of limited interest to those working in the particular area.

Your review should justify your decision to recommend or not recommend an article and should offer feedback to the authors.
 
An article will be considered Recommended if it receives the following ratings from Review Panel Members.
 
  • At least two 4-star (or above) ratings, OR
  • At least three 3-star ratings AND one 4-star (or above) rating.
Articles that have reached Recommended status will be deemed to have met suitable academic standards for indexing, and will be submitted to indexing systems for indexing.


In commenting and allocating a star-rating, reviewers of a paper should take account that AMEE MedEdPublish publishes reports, in addition to original research studies that replicate previous work, case studies, opinion pieces and studies with negative findings.  The need for replication studies has been recognised in education, and the fact that a paper confirms results previously reported should not detract from its value and the award of a high star-rating.  Similarly, in an opinion piece, if the thoughts are valuable and clearly argued, the paper could also attract a high rating.  A particular element of a paper, such as a useful table or figure or list of references may also merit a positive recommendation to the reader.

By recommending an article you also confirm that you have read the article and feel you have an appropriate level of expertise to review the article.
 
6. Other Reviewers
Although all other reviewers’ reviews are available to you, AMEE MedEdPublish recommends reviewers do not read the reviews before reading the manuscript and drafting your review.  This will enable you to read the manuscript with an open mind, and not be prejudiced in any way by other reviews.
 
Once you have drafted your review, you may wish to consult the other reviews to comment what they have said, but your opinion of the manuscript should not be unduly influenced by these.
 
If you feel unsure, it would be better to post your review without reading the other reviews at all.  After you have posted your review, you may use the comment feature to comment on other reviews.
 
7. Participation Points and levels
As an indication of participation level, reviewers will be awarded participation points for their reviews and comments, and their level of on-going participation (Gold, Silver or Bronze) will be indicated next to their name.
 
  1. Points to be calculated in same manner as for the current contribution points system with regard to reviews: 30 for reviews and 10 for comments.
  2. Bronze Award will be awarded once 100 points have been achieved
  3. Silver Award will be awarded once 200 points have been achieved
  4. Gold Award will be awarded once 300 points have been achieved.
  5. An Award icon will appear against the reviewer’s name when they post a review or comment, and will also appear against the panel member information and on the individual’s My Reviewing Activity area of the website.  The awards medals will be awarded to everyone, not only panel members.
  6. Each person will progress through the award levels as they accumulate points, until the end of each year.
  7. The start date for each user will be the date that the user places their first review or comment after the new points system has been implemented.
  8. All award level medals will last for 1 year (unless they are improved within the year), and whichever Award level the user has achieved at the end of the current year will be the starting point for the following year.
  9. At the end of the year, the points are returned to zero, and the points’ accumulation starts again from zero.  Users will have another year to progress through the award levels. (e.g. If a user starts the year on bronze and they achieve enough points within the year to progress to silver, the award medal will change to silver at the time that 200 points have been achieved and then to gold if they progress even further.  If a user starts the year on gold, they will stay on gold for the whole year, however if they don’t accumulate enough points (300) to retain the gold by the end of the year, they will start the new year with whichever award medal they have achieved.)
  10. There will be a two tier points system with tier one ensuring that users can progress through the points system every year and tier two to track the cumulative points from the start of each year.
  11. The My Reviewing Activity area will show the individual their current medal rating, the date that will expire (unless improved) and a display of their progress for the next year i.e. with a sliding scale showing where their current award sits on a progress bar.
 
8. Further Acknowledgement
At the completion of your review, you will be sent a confirmation email thanking you for your review.  This email can be submitted to Publons (https://publons.com) and added to your Publons profile.  (If you do not have a Publons account, we recommend that you create one).
 
Reviewers will also be acknowledged at the annual AMEE conference.
 
9. Further Documentation
If reviewers wish to familiarise themselves with further peer-review guidelines, they should consult the COPE Guidelines at: http://publicationethics.org/files/Peer%20review%20guidelines.pdf