Guidelines for Reviewers

MedEdPublish – Guidelines for Peer Reviewers
Version 02: June 2019
 

Contents

1. General 2. Layout of the review     
3. Further requirements 4. How to submit a review
5. Star-ratings
6. Other reviewers          
7. Participation points and levels    
8. Further acknowledgement       
9. Further documentation
 

MedEdPublish – Guidelines for Peer Reviewers

Please note that these guidelines are intended to conform to the COPE Ethical Guidelines for Peer Reviewers available at: https://publicationethics.org/files/Ethical_Guidelines_For_Peer_Reviewers_2.pdf
and the ICMJE Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals, Updated December 2017, available at: http://www.icmje.org/icmje-recommendations.pdf
 

1. General

For an overview of the MedEdPublish process from submission to final revision, click here.

Reviews are essential for maintaining the standards of the journal and will determine whether an article is considered suitable for indexing. 

MedEdPublish uses an open identities post-publication peer-review model. This means that all feedback is submitted publicly under the reviewer's name and reviewers are also aware of authors' identities. Post-publication peer review should be conducted as a constructive discussion, the aim of which is to enhance understanding of the subject and refine future work: critical but constructive feedback is therefore encouraged. To stimulate discussion and ensure an open and transparent process, authors are encouraged to reply to reviews of their article. Comments deemed unintelligible, inappropriate or offensive will be removed.

1.1 Author/Editor nominated reviewers

Anyone can place a review at any time in MedEdPublish, but sometimes reviewers will be invited by the editor to review a specific article and will have the option to accept or decline. If the invitation is accepted, reviews should be placed as promptly as possible, although ensuring that the quality of the review is not compromised. Occasionally non-Panel members will be contacted by an Editor to review an article felt to be within their specific area of expertise.  In this situation the reviewer will be granted Panel member status for that article only and their star-rating may contribute to that article achieving a 'recommended' status. 
 

2. Layout of the Review

Reviews should:
  • Be written in good English;
  • Average around 300 words in length (there is a 500 word limit);
  • Outline both the strengths and the weaknesses of the article;
  • Give a flavour of the type of reader who might find the article valuable.
 Although MedEdPublish does not prescribe headings, some useful guideline questions for a reviewer to ask, where appropriate, are:
  • Is the topic appropriate to medical education?
  • Is the abstract an accurate and concise description of the information in the article?
  • Does the introduction supply enough background to the article, including reference to the relevant literature, and the specific goals of the article?
  • Are there structural or logical flaws, errors of misinterpretation, lack of referenced literature, or poor presentation of data?
  • Who are the target audience for this article and what could they expect to learn from it?
When reviewing a research article, additional questions may include:
  • Are the methods adequately described (to the point that, under ideal conditions, they could be re-produced)? 
  • Are the results displayed in a way that is logical and understandable, and is there some degree of analysis or commentary?
  • Does the discussion explore deeper implications of the results (usually referring back to the information contained in the introduction)?
  • Are the conclusions supported by the data in the results and the discussion?
  • Where appropriate, have alternative interpretations and / or weaknesses been identified?
Although it is useful for reviewers to draw on their own expertise on the subject, they should remember that the review is aimed at evaluating the article, and the focus of the review should always be on that.
 

3. Further Requirements

(Adapted from the COPE Guidelines at: http://publicationethics.org/files/Peer%20review%20guidelines.pdf)
 

3.1  Required expertise

Reviewers should agree to review only articles for which they have the required expertise.  If there are significant areas in which the reviewers do not have expertise, they should identify this clearly in their review, or consider not reviewing the article.
 

3.2  Competing interests

Reviewers are asked to declare any competing interests. Competing interests may be financial e.g. holding a patent or receiving fees from a company that may lose or gain financially from the publication of the article, personal, religious, political or other non-financial interests.
 
In addition, if the reviewer is from the same institution as the authors of the article, it would be preferred if they do not review that article, especially if the reviewer is on the Review Panel and/or is a senior member of staff at that institution. If such a reviewer does feel compelled to write a review, then they must state explicitly that they are from the same institution as the authors.
 

3.3 Personal influences

Reviewers should not be influenced by issues of a personal nature, such as the article’s country of origin, religion, culture, gender, etc., unless these issues have a direct bearing on the content of the article, in which case, this should be stated explicitly.
 

3.4 Offensive or inflammatory language

Reviews should be written in a formal academic manner, avoiding language that is offensive, inflammatory, libellous, unfair, or derogatory.
 

3.5 Open identities

As reviews are published openly, it is essential that reviewers correctly identify themselves by name.  While common name variations are permitted (e.g. “Bob” for “Robert”), if a name varies substantially from the name by which someone is normally known, they may be requested by MedEdPublish to alter the name in their profile.
 

3.6 Self-promotion

Reviewers who see shortcomings in the literature cited in an article will frequently wish to recommend that the authors read and refer to particular sources.  As reviewers will frequently write reviews on articles in areas of their own expertise, it is likely that they will have published articles on those topics covered by the articles that they are reviewing.  As a result, they may wish to recommend their own article as a source.
 
In this situation, they should take care not to recommend their own articles merely for the sake of boosting citations. If they really think that their own article will be of benefit to the author, they should declare their identity as the author of that article, in order to clearly indicate a possible conflict of interests.
 

4. How to submit a review

In order to submit a review you will need to log in with your AMEE or MedEdPublish username or password.  (Please ensure you use only one account so that your reviews and contribution points are easily identified as yours). If you are not an AMEE member or have not previously registered on the MedEdPublish website please consult our User Guide to follow the registration process.
 
Once you have logged into MedEdPublish, go to the article you wish to review.  Once you have opened the article:
  1. Click on the ‘Reviews’ button to the right of the article. This will take you down to the bottom of the article where you will be able to see any existing reviews for the article.
  2. Click on the ‘Submit Review’ button. This will open up a text box into which you can type or copy and paste your review of the article. 
  3. Once you have entered your review, check the box to declare that you have no Conflicts of Interest or enter details of the conflict of interest to be disclosed.
  4. Rate the article by clicking on the appropriate star-rating. The number of stars displayed in blue will indicate your decision.  Guidance on the rating system can be found by clicking the link beneath the stars. Each review must indicate a rating for the article.
  5. Click on the ‘Submit’ button next to the star ratings to submit your review.
 
Your review should appear immediately, and you will receive a confirmation email to your registered address. You may also be informed if any replies to your review are received. To decide what activity you are alerted to, go to ‘My Account’, click on the ‘Notification Preferences’ tab, and scroll down to check the boxes of those options you wish to receive notification of.
 

5. Star-Ratings

Once an article has been reviewed, the reviewer is asked to consider the extent to which they recommend the article by selecting one of the following:
★★★★★   I strongly recommend this article as an important contribution to the field that should be read by everyone with an interest in the area.
 
★★★★   I recommend this article as a useful contribution to the field that should be read by those with an interest in the area.
 
★★★   I recommend this article to be read by those working in the field despite having some reservations as noted in my review.
 
★★   I have reservations about this article as noted in my review but it may be of interest to those working in the field.
 
  I have serious reservations about this article but aspects may be of limited interest to those working in the particular area.

Your review should justify your decision to recommend or not recommend an article and should offer feedback to the authors.
 
An article will be considered Recommended if it receives the following ratings from Review Panel Members.
 
  • At least two 4-star (or above) ratings, OR
  • At least two 3-star ratings AND one 4-star (or above) rating.
Articles that have reached Recommended status will be deemed to have met suitable academic standards for indexing, and will be submitted to any indexing systems which recognise MedEdPublish.


In commenting and allocating a star-rating, reviewers of an article should take account that MedEdPublish publishes reports, in addition to original research studies that replicate previous work, case studies, opinion pieces and studies with negative findings. The need for replication studies has been recognised in education, and the fact that an article confirms results previously reported should not detract from its value and the award of a high star-rating.  Similarly, in an opinion piece, if the thoughts are valuable and clearly argued, the article could also attract a high rating. A particular element of an article, such as a useful table or figure or list of references may also merit a positive recommendation to the reader.

By recommending an article you also confirm that you have read the article and feel you have an appropriate level of expertise to review the article.
 

6. Other Reviewers

Although all other reviewers’ reviews are available to you, MedEdPublish recommends reviewers do not read the reviews before reading the manuscript and drafting your review. This will enable you to read the manuscript with an open mind, and not be prejudiced in any way by other reviews.
 
Once you have drafted your review, you may wish to consult the other reviews to comment what they have said, but your opinion of the manuscript should not be unduly influenced by these.
 
If you feel unsure, it would be better to post your review without reading the other reviews at all.  After you have posted your review, you may use the comment feature to comment on other reviews.
 

7. Participation points and levels

As an indication of how much an individual is contributing to the journal, reviewers will be awarded participation points for their reviews and comments, and their level of on-going participation (Gold, Silver or Bronze) will be indicated next to their name.
  • Contribution points and award medals are awarded to everyone, not only panel members.
  • 30 points are awarded for reviews and 10 for comments.
  • Medals are awarded as follows:
    • Bronze Award: 100 points
    • Silver Award: 200 points
    • Gold Award: 300 points
  • An Award icon will appear against the reviewer’s name when they post a review or comment, and will also appear against the panel member information and on the individual’s My Reviewing Activity area of the website. 
  • Each person will progress through the award levels as they accumulate points, until the end of each year.
  • The start date for each user will be the date that the user places their first review or comment.
  • All medals are awarded for 1 year (unless they are improved within the year), and whichever Award level the user has achieved at the end of the current year will be the starting point for the following year.
  • At the end of the year, points accumulation starts again from zero. If a user starts the year on gold, they will stay on gold for the whole year, however if they don’t accumulate enough points (300) to retain the gold by the end of the year, they will start the new year with whichever award medal they have achieved.
  • The My Reviewing Activity area will show the individual their current medal rating and its expiry date, and a sliding scale showing where their current award sits on a progress bar.
 

8. Further Acknowledgement

At the completion of your review, you will be sent a confirmation email thanking you for your review. This email can be submitted to Publons (https://publons.com) and added to your Publons profile. (If you do not have a Publons account, we recommend that you create one).

Participation points can be used as evidence of your contribution to AMEE's mission should you wish to apply for Fellowship or Associate Fellowship of AMEE.
 
Reviewers will also be acknowledged at the annual AMEE conference.
 

9. Further Documentation

If reviewers wish to familiarise themselves with further peer-review guidelines, they should consult the COPE Guidelines at: http://publicationethics.org/files/Peer%20review%20guidelines.pdf