Research article
Open Access

Medical Education in the Mountains - The Educational Environment in the Diploma of Mountain Medicine

Lawrence Rory Kidd[1]

Institution: 1. North Bristol NHS Trust
Corresponding Author: Dr Lawrence Rory Kidd ([email protected])
Categories: Educational Strategies, Learning Outcomes/Competency, Postgraduate (including Speciality Training)
Published Date: 16/04/2018

Abstract

Aims: The Diploma of Mountain Medicine is a postgraduate blended learning course, combining online and distance learning with fully immersive face-to-face components. Utilising the Dundee Ready Educational Environment Measure (DREEM) to evaluate educational environment, alongside overall global course evaluation using the Postgraduate Taught Experience Survey (PTES), we aimed to further an understanding of the educational environment in this dynamic and increasingly popular method of course design.

Methods & Results: All participants completing the Diploma between 2012-2015 were invited complete the online survey. Answers were recorded using a modified Likert scale. Response rate was 57% (n=68). Overall course satisfaction was positive using the PTES, with an overall mean score of 3.77 (out of 5). Only 9.4% of responses were negative. Educational environment using the DREEM was scored as “Excellent” (mean total score of 150.68; range 100-199). When the two tools used were compared, overall scores showed close agreement (r=0.775). The DREEM domain, Perception of Learning was most closely correlated with PTES overall score (r=0.714).

Female participants rated the course lower than their male counterparts using both DREEM and PTES tools. Those who had <1 year since their most recent education, and those for whom highest study was at undergraduate level had lower DREEM scores than those with previous postgraduate experience. This may reflect differences in expectations and learning objectives.

Conclusions: These multifaceted tools demonstrate a close alignment between educational environment and overall course satisfaction. Managing learner expectations and gender differences are potential challenges to overcome. 

Keywords: blended learning; postgraduate; learning environment; DREEM

Introduction

Blended learning provides a mixture of learning in terms of mode, distance, time and technology. It is growing in popularity and availability, and has developed to encompass a wide range of courses. As the uptake of these courses increases and extends into those with a greater inter-professional approach, the need to ensure quality remains tantamount and therefore the evaluation and subsequent improvement of the students’ learning experiences should be performed.

There are a number of advantages of blended learning – broadly divided into personal, institutional and pedagogical (1). The busy professional lives of postgraduate learners, necessitates a flexible approach to learning (2) - one reason for increasing availability of blended courses. Delivery is not without its challenges, including the potential for perceived isolation (3). This is one component of the learning environment. In healthcare the learning environment has been shown to be the strongest predictor of preparedness for independent clinical practice (4). It involves all aspects of the learning surroundings, which potentially might be challenged or augmented by a blended approach. As such, measuring the learning environment is important and the DREEM questionnaire (Dundee Ready Educational Environment Measure) questionnaire (5) is one of the most widely used tools available.

As well as assessing the Learning Environment, increasingly a more global evaluation of students’ experiences is performed. The Postgraduate Taught Experience Survey (PTES) is a nationally used postgraduate course evaluation tool (6) that continues to undergo refinement ensuring robustness and quality (7).

This case-study using the postgraduate medical education course, the Diploma of Mountain Medicine, is therefore designed to help those involved with course design and provision to understand and therefore overcome some of the challenges involved in blended learning. 

Methods

In accordance with institutional guidelines, ethical approval was given by the Board of Studies as part of the research approval process (Minute 15/M07). An online survey link (www.esurv.org) was circulated to all participants completing the final residential module between 2012-2015.

In addition to demographic information, the survey utilised the PTES tool; 39 questions divided into six domains and one overall evaluation question. One domain (dissertation) was omitted from the latest iteration (7) this was not applicable. The final section was the DREEM questionnaire (5).

All data was anonymous, and underwent processing using Microsoft Excel 2010. Statistical procedures undertaken included correlation coefficients for measuring the association between pairs of variables, with standard critical values assigned (r=0.11, p<0.05; r=0.15 p<0.01). The level of alpha (significance) was set at p=0.05. 

Setting

The Diploma of Mountain Medicine is a multi-disciplinary postgraduate medical education course combining electronic and distance-based pre-course essays and learning, with fully immersive and residential components. Originally established in 1997, and validated by the Medical Commissions of the UIAA/IKAR/ISMM. It is taught in 12 countries worldwide with over 3500 Diplomas issued globally (8). Participants include nurses, paramedics and doctors from a range of specialities and grades. 

Results

A total of 81 surveys were completed (response rate 68%) of which 13 were excluded from analysis due to incomplete data. Cronbach’s alpha was 0.92 for the PTES and 0.93 for the DREEM.

52 (76.4%) participants were male. Ages and years since academic study are shown in Table 1. 67.6% had an undergraduate degree as their highest previous qualification.

Overall scores demonstrated good results with a PTES mean score of 3.77 out of a maximum of 5 (Table 2), and the learning environment classified as “Excellent” (mean 150.68, range 100-199, sd 18.47, see Table 3).

In both the PTES and the DREEM, the course was scored less positively by female students (p=0.00094 and p=0.00012 respectively). Other trends of note relate to delays since previous study. Those who had a delay in studying of ten years or more had significantly higher scores in the PTES domain of engagement scores (p=0.0017, 0.0019, 0.046) than those in all other groups. Likewise, DREEM outcomes were lower in those with longer delays scored higher than other groups (p=0.0025, 0.0026). Lower scores were also found in those participants whose highest previous study was at undergraduate degree (p=0.024).

When the two overall scores were compared (see Table 4), they showed close agreement (r=0.775). When compared with both overall PTES score and the final question, the Perception of Learning was consistently the most closely correlated domain (r=0.71 & 0.720). By contrast the DREEM domain Social Self Perception showed the least agreement (r=0.548 & 0.281).

The overall DREEM score showed good correlation with the PTES final question (r=0.695). Of the PTES domains, Organisation and Management showed the closest correlation with the DREEM score (r=0.631). The least closely associated domain was Learning Resources and Support Services (r=0.355). 

Discussion

This study has evaluated the learning environment using the DREEM, and the overall course satisfaction using the PTES in participants undertaking a postgraduate blended learning course with a broad intake of different healthcare professions, medical specialities, experience and backgrounds. In doing so, we have demonstrated high levels of satisfaction the learning environment, and the course as a whole.

There participants demonstrated heterogeneity in terms of ages, previous higher education and delays between prior formal education. Some differences emerged between the groups, including female participants scoring lower in both the PTES and DREEM scores compared to male counterparts in overall scores, and in several domains. This is not consistently found elsewhere (7), and this may be influenced by relatively lower proportions of female participants. However, conversely it may have been that the low numbers of women on the course may have meant those who were involved had a poorer experience. This would be an interesting avenue for future research, as the tools used in this study are too crude to elucidate this.

Another area where interesting patterns emerged was in delay between most recent academic study. Those who had studied more recently (<1 year) had a tendency to score worse compared to those who had longer delays. This may represent differences in expectations, and highlight the improvement in the quality of higher education provision (7,9). On a similar note, those whose highest previous study was at undergraduate had lower DREEM scores than those with previous postgraduate experience. This may reflect an appreciation of the difference in teaching styles, supervision, feedback and assessment i.e. the many but subtle differences that compose the learning environment. In this sense, the DREEM is the perfect tool for evaluating this difference.

One of the potential challenges of blended learning in the literature is potential isolation. However, we can see that this was not the case e.g. Social Self Perception scored second highest. This may be due to recall bias as the most recent component was a residential week, or the extent of the immersion in the face-to-face component something that may be of relevance to those involved in course design.

Some subtle challenges of a blended approach are evident - questions relating to sufficient contact time [Q5] and sufficient opportunities to discuss work with other students [Q9] both scored relatively badly. Maintaining a community of learning in online and blended teaching can be challenging (10) and these results reflect this. However, despite these potential challenges the learning environment was classified as “Excellent”, and the overall PTES scores were overwhelmingly positive.

When overall scores between the two tools were evaluated, they were closely correlated. The impact of the learning environment on numerous outcomes including course satisfaction, is well established. However, an evaluation using these tools in a blended course structure has not been performed previously. This study adds weight not only to the importance of the learning environment, but also the extent to which the perception of the learning environment is interwoven with course satisfaction and evaluation.

The learning environment domain most consistently correlated with the PTES scores was the Perception of Learning. This suggests that it is perhaps the learning experience that candidates value above all else, although clearly the other aspects should not be overlooked. Although some PTES domains such as Organisation & Management, showed close correlation DREEM scores, it is vital to highlight that not one single domain showed a greater correlation with the overall score than that between the overall DREEM and the overall PTES scores. Overall course evaluation and the learning environment are both complex concepts, that will have elaborate interactions between their different domains. These closely allied scores demonstrate that in elucidating the relationships within these concepts it is ultimately the whole is greater than the sum of the parts.

In this case study of a multidisciplinary postgraduate medical blended learning course, the learning environment showed close association with overall course satisfaction. As with any postgraduate course, numerous challenges exist. Ones highlighted for potential development and exploration include managing the community of learning and avoiding any gender and discrepancies resulting from delays in prior education. As blended learning grows in popularity and availability, educators should continue to consider the importance of the learning environment on course satisfaction, and be aware of the challenges that may manifest within this course design. 

Take Home Messages

  • Both tools showed overwhelmingly positive results, and both tools showed close correlation. 
  • Perception of Learning was the DREEM environmental domain most closely linked to the overall course evaluation score.
  • Female participants and those with the least delay since last formal education scored worse than other participants. Therefore managing learner expectations is vital.
  • This multidisciplinary postgraduate course featuring diverse participants shows that the blended approach was successful in terms of learning environment and overall course evaluation. 

Notes On Contributors

Lawrence Kidd is an Anaesthetic Registrar in the Bristol, United Kingdom. Interests include mountain and expedition medicine, course design and cognitive barriers in human factors.

Acknowledgements

Many thanks to Professor Karen Mattick for providing input and advice throughout this process, and Dr Peter Barry for his supervision during my MSc studies.

Bibliography/References

1. Partridge H, Ponting D, McCay M. Good practice report: Blended learning (2011). http://eprints.qut.edu.au/47566/1/47566.pdf Accessed 19th November 2015.

2. Benfield G, Roberts G, Francis R. (2006). The undergraduate experience of blended e-learning: a review of UK literature and practice. York: Higher Education Academy. https://www.heacademy.ac.uk/system/files/sharpe_benfield_roberts_francis_0.pdf  Accessed 19th November 2015.   

3. Smyth S, Houghton C, Cooney A, Casey D. Students' experiences of blended learning across a range of postgraduate programmes. Nurse Education Today. 2012; 32(4), 464-468.

https://doi.org/10.1016/j.nedt.2011.05.014

4. Dijkstra IS, Pols J, Remmelts P, Rietzschel EF, Cohen-Schotanus J, Brand PL. How educational innovations and attention to competencies in postgraduate medical education relate to preparedness for practice: the key role of the learning environment. Perspectives on medical education. 2015, 1-8.

https://doi.org/10.1007/s40037-015-0219-3   

5. Roff S, McAleer S, Harden RM, Al-Qahtani M, Ahmed AU, Deza H, Groenen G, Primparyon, P. Development and validation of the Dundee ready education environment measure (DREEM). Medical Teacher. 1997; 19(4), 295-299.

https://doi.org/10.3109/01421599709034208   

6. Smith A, Bradshaw T, Burnett K, Docherty D, Purcell W, Worthington S. One step beyond: Making the most of postgraduate education (2012). Report for UK Department for Business, Innovation and Skills.  http://www.bis.gov.uk/assets/BISCore/corporate/docs/p/10-704-one-step-beyond-postgraduate-education.pdf  Accessed 21st November 2015.   

7. Soilemetzidis I, Bennett P, Leman J (2013). PTES 2014: Findings from the postgraduate taught experience survey. Higher Education Academy (HEA). York. https://www.heacademy.ac.uk/sites/default/files/resources/ptes_2014_report.pdf Accessed 21st November 2015.   

8. Diploma. UIAA. http://www.theuiaa.org/mountain-medicine-diploma.html Accessed 6th July 2014.   

9. House G (2010). Postgraduate Education in the United Kingdom. London: Higher Education Policy Institute and The British Library. https://www2.le.ac.uk/departments/gradschool/about/external/publications/pg-education.pdf Accessed 18th November 2015.   

10. Cobb SC. Social presence, satisfaction, and perceived learning of RN-to-BSN students in web-based nursing courses. Nursing Education Perspectives. 2011; 32(2), 115-119.

https://doi.org/10.5480/1536-5026-32.2.115

Appendices

Age

Number of participants

Years since academic study

Number of participants

≤30

15

<1

10

31-35

29

1-3

40

36-40

11

4-9

13

41-50

7

≥10

5

≥51

6

   

Table 1. Demographics of Diploma participants (n=68).

 

Domain

 

Agreement

Neutral

Disagreement

Mean score

Teaching & Learning

Number

309

70

29

3.98

 

%

75.74

17.16

7.11

 

Engagement

Number

268

57

15

4.04

 

%

78.82

16.76

4.41

 

Assessment & Feedback

Number

148

68

55

3.36

 

%

54.61

25.09

20.3

 

Organisation & Management

Number

261

56

21

3.91

 

%

77.22

16.57

6.21

 

Learning resources &

Number

170

59

23

3.68

support services

%

67.46

23.41

9.13

 

Development of skills

Number

198

153

47

3.48

 

%

49.75

38.44

11.81

 

Final Question

Number

61

3

4

4.25

 

%

89.71

4.41

5.88

 

Overall

Number

1415

466

194

3.77

 

%

68.19

22.46

9.35

 

Table 2. Overall scores in PTES (Postgraduate Taught Experience Survey) tool.

 

Overall response rating

Learning (%)

Course Organisers (%)

Academic Self (%)

Atmosphere (%)

Social Self (%)

Overall (%)

Negative (0/1)

6.8

6.84

12.86

5.37

3.22

7.08

Equivocal (2)

14.64

9.66

21.35

58.93

9.59

24.54

Positive (3/4)

78.57

83.5

65.79

35.7

87.18

68.52

Table 3. Overall score summary in DREEM (Dundee Ready Educational Environment Measure) tool.

 

Measure A

Measure B

Correlation

PTES Overall

DREEM (total)

0.78

PTES Overall

Perception of Learning

0.71

PTES Overall

Perception of Course organisers

0.69

PTES Overall

Academic Self-Perception

0.66

PTES Overall

Perceptions of Atmosphere

0.63

PTES Overall

Social Self Perceptions

0.55

PTES Final Question (Q44)

DREEM (total)

0.7

PTES Final Question (Q44)

Perception of Learning

0.72

PTES Final Question (Q44)

Perception of Course organisers

0.66

PTES Final Question (Q44)

Academic Self-Perception

0.51

PTES Final Question (Q44)

Perceptions of Atmosphere

0.61

PTES Final Question (Q44)

Social Self Perceptions

0.28

Teaching & Learning

DREEM (total)

0.54

Teaching & Learning

Perception of Learning

0.53

Teaching & Learning

Perception of Course organisers

0.49

Teaching & Learning

Academic Self Perception

0.48

Teaching & Learning

Perceptions of Atmosphere

0.46

Teaching & Learning

Social Self Perceptions

0.26

Engagement

DREEM (total)

0.58

Engagement

Perception of Learning

0.55

Engagement

Perception of Course organisers

0.56

Engagement

Academic Self Perception

0.39

Engagement

Perceptions of Atmosphere

0.39

Engagement

Social Self Perceptions

0.39

Assessment & Feedback

DREEM (total)

0.47

Assessment & Feedback

Perception of Learning

0.37

Assessment & Feedback

Perception of Course organisers

0.49

Assessment & Feedback

Academic Self Perception

0.26

Assessment & Feedback

Perceptions of Atmosphere

0.43

Assessment & Feedback

Social Self Perceptions

0.43

Organisation & Management

DREEM (total)

0.63

Organisation & Management

Perception of Learning

0.55

Organisation & Management

Perception of Course organisers

0.55

Organisation & Management

Academic Self Perception

0.53

Organisation & Management

Perceptions of Atmosphere

0.49

Organisation & Management

Social Self Perceptions

0.58

Learning Resources & Support Services

DREEM (total)

0.36

Learning Resources & Support Services

Perception of Learning

0.28

Learning Resources & Support Services

Perception of Course organisers

0.34

Learning Resources & Support Services

Academic Self Perception

0.39

Learning Resources & Support Services

Perceptions of Atmosphere

0.21

Learning Resources & Support Services

Social Self Perceptions

0.33

Development of Skills

DREEM (total)

0.53

Development of Skills

Perception of Learning

0.53

Development of Skills

Perception of Course organisers

0.37

Development of Skills

Academic Self Perception

0.59

Development of Skills

Perceptions of Atmosphere

0.41

Development of Skills

Social Self Perceptions

0.3

Table 4: Comparison between PTES (Postgraduate Taught Experience Survey) and DREEM (Dundee Ready Educational Environment Measure) outcomes. Correlations calculated using Microsoft Excel for Mac v.15.22.

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Reviews

Please Login or Register an Account before submitting a Review

Trevor Gibbs - (22/06/2018) Panel Member Icon
/
I read and reviewed this paper thinking that I was going to read about a very specific course that probably very few of my colleagues would ever be involved in. However soon I realised that this paper was about evaluating a blended learning course.
Before reading about the evaluation I felt that I was already at a little bit of a loss having read only a very short and not very descriptive outline of what blended learning was and how it was manifested with this course. I felt that would have been useful, even if it had been in some sort of table format.
I would agree with my co-reviewer that there are some discrepancies in the research methodology; I will not repeat these. However I would suggest that the paper could have been better by expanding on the findings and how one might improve in those areas that need improvement.
Ken Masters - (22/06/2018) Panel Member Icon
/
An interesting paper, but requiring quite a bit of work.

Although the student response rate was a little low, this is becoming relatively standard in course evaluations, and the author did also exclude 13 responses.

Some things that might improve the paper:

The Setting should be moved to before the Methods (Probably as a sub-section of the Introduction).

The Setting could also mention the number of students on the course; this has to be calculated by the reader, based on the fact that 81 was a response rate of 68%, therefore, I calculated that the total number of students on the course was 119. (This is supported by the fact that that 13 exclusions led to 68, which gave a response rate of 57%, mentioned in the Abstract, but not the body of the paper.)

For an international audience, it might have been useful if the author had spent a little more time discussing “Mountain Medicine”, modules taught, the delivery of the course, and its specific characteristics that set it apart from other types of Medicine, especially those that might be seen as closely-related. This would also give an indication of why this type of diploma has value. Also, some information about the background of the students would be useful.

In the Results, I would recommend that p values should be given to only 3 decimal places. If they are 0.000, then should be written as p <0.001.

One part of the paper says the statistics were calculated using Excel 2010, another says Excel for Mac v.15.22. If these are the same, then the author should use only one name only; if different, then the author should explain why different versions were used.

Because there was a statistically significant difference in results between males and females, it might have been useful to reflect these figures in the Tables.

In the Discussion, when you refer to particular items, it is also good to use the same wording that you do in the tables, or at least number the items in the tables, and then let the Discussion refer to Table and item number. For example, “questions relating to sufficient contact time [Q5] and sufficient opportunities to discuss work with other students [Q9]” leave a reader confused, as none of the Tables use “contact time” and I can’t find Q5 or Q9. If these are not in the table, then it would have been useful if they had been as an appendix.

So, while the paper does look useful, the author is expecting the readers to do far too much work, and it would have been better if some more detail had been supplied.