Comment where possible on each section and take notes.
Title. Does the title encapsulate the message? Some readers read further only if the title fits their interests, so these few words are important. For research and evaluation manuscripts, titles should define the study design. Better titles help readers choose which abstracts to read.
Abstract. Does the abstract summarise adequately the manuscript? Many readers will not read complete articles, so it is important that the purpose, methods and outcomes are clear. Structured abstracts make this easier for research papers.
Introduction/background. Does the introduction/background section provide sufficient context to ‘set the scene’? This section should explain where the author began the journey, why this study is necessary and why this manuscript should be read.
Methods. This section is likely to be present only in reports of research or evaluation papers, where it is essential. Are the methods clear, grounded in theory and appropriate for the question? Quantitative research includes many different methods. The two most common are quasi-experimental designs (where attempts are made to reduce the number of dependent variables) and non-parametric approaches (because interventions are often applied to small populations that do not reflect a ‘normal’ curve). If in any doubt, seek or recommend opinion from a statistician. Qualitative research also includes many methods, but is very useful for seeking understanding of complex issues. There should be sufficient separation between researchers and the information gathering or analysis to minimize the potential for perceived bias. If in any doubt, seek or recommend an opinion from a qualitative researcher. The term ‘mixed methods’ is currently popular, but this is a rather lazy term that needs to be justified by explanation of why each method was chosen to address each part of the usually complex question. Manuscripts are often mostly descriptive, without complex analyses, but are still of value in medical education as similar initiatives are often applied in very different contexts.
Results. Are the results clearly presented, at least initially free of interpretation? The facts should speak for themselves. Ideally, tables and figures should summarise and augment the text, but the text should clearly describe results without the tables and figures. There should not be more than three tables or figures, which should summarise the key findings. If more space is required to provide a more complete view of results, these can be added as an appendix.
Interpretation and Discussion. These sections are usually separate for quantitative studies and often combined for qualitative studies. Are the findings supported by the results? Over-interpretation is not uncommon, particularly for novice researchers. Not all quantitative designs can clearly attribute causation. Evaluations of educational activities are often low level (satisfaction) and short term (before and just after), where impacts can be measured more easily, but may not be worth reporting. Demonstrating longer term impact on practice is harder, but more valuable. Ideally, evaluation of impacts on health care practices and outcomes are attempted, but this can be very difficult in education research. While quantitative studies may produce results that may be applicable in other contexts, qualitative research is best at identifying influencing factors and enhancing understanding, but cannot attribute causation or association or be extrapolated easily to other contexts. Are there implications for other education programs? What research questions have been addressed and what further questions have been identified? Are recommendations for change in practice supported by the analysis? Descriptive manuscripts are just descriptions without much analysis or interpretation, but can still inform or inspire others. Manuscripts reporting ‘negative’ findings may be very important, as others can learn from knowing what did not work.
Conclusion/summary. All manuscripts should end with a ‘so what’ statement. For research and evaluation reports, this could be a single paragraph that summarises why the study was done, the findings and what they might mean. This can be similar to the abstract, only shorter and less focused on methods. This is not the right place for discussion or justification of results and their implications.
References. Are the references current, comprehensive and accurate? It is worthwhile checking briefly some references at random by pasting them into a search engine. While ‘seminal’ references may be older, most references should be from the last 5 years or so and authors should not self-cite too often. Citations of original research papers are better than other papers that cite (and perhaps misinterpret) original research. Researchers sometimes observe that their research is cited to support something different from what the research found! Are any key references missing? Literature searches do not always identify every relevant paper; much depends on the searcher and the search engine.
Limitations. Are limitations adequately presented and discussed? Small studies may not have sufficient scale, sampling may have been restricted or unrepresentative, or specific contextual factors may affect validity; these should be mentioned.