Research article
Open Access

APPRENTICE STYLE TO COMPETENCY STYLE PSYCHIATRIC TRAINING: 5 YEAR STUDY OF SINGAPORE PSYCHIATRIC TRAINING

Phern-Chern Tor[1], Peak-Chiang Chiam[1]

Institution: 1. Institute of Mental Health (Singapore)
Corresponding Author: Dr Phern-Chern Tor ([email protected])
Categories: Comparative Medical Education, Educational Strategies, Postgraduate (including Speciality Training)
Published Date: 02/06/2016

Abstract

Introduction

Singapore had an apprentice style medical specialization training program until 2010 when it adopted an ACGME certified residency training program. A recent survey showed Singapore psychiatry residents rating their educational environment worse than non-residency psychiatry trainees. To confirm these findings we repeated a 2008 survey of perceived importance and adequacy of 11 aspects of psychiatric training among psychiatry trainees in Singapore.

Methods

We sent out an electronic survey to all psychiatry trainees in Singapore (residents and non-residents) to rate their perceived importance and adequacy on 11 aspects of psychiatric training on a 5 point Likert scale. Results were analyzed with paired or independent t-test analysis as appropriate with p-values at 0.05.

Results

The survey was conducted from Apr to Aug 2013 with a response rate of 57.9% (N=44). The perceived adequacy of training was significantly better in 2013 than 2008 with 6 of 11 aspects of training rated higher in 2013. Residents had higher ratings in 16 of 19 significant between group differences of perceived adequacy of training.

Conclusions

A competency based style residency program has resulted in improved perceived adequacy of psychiatric training compared to an apprentice style program. Further research to assess objective adequacy of psychiatric training under the residency program is recommended.

Keywords: Curriculum; Asian; Singapore; Learning style

Introduction

Singapore is a small city state (710 km2) with a population of 5.3 million people consisting of three main ethnic groups (Chinese, Malay and Indian). Its official language of business and education is English and the literacy rate is 96%. Per capital GDP is USD $50,523. (Department of Statistics Singapore, 2012). Almost all hospitals in Singapore are Joint Commission International certified and it is the first country outside the United States to have an Accreditation Council for Graduate Medical Education (ACGME) certified residency training program since 2010.

Before 2008 the training program was modeled after British medical postgraduate training in the 1970s-1980s. This was an apprentice style rigid system of time-based (3 years of basic and advanced specialty training each) and phasic progression (determined by the passing of a Master of Medicine examination before progression to the advanced phase). After 2008 the system evolved to a competency-based and seamless progression of training (requiring only 5 years).  The rigor of the training increased with the implementation of standardized assessments, new modules on basic sciences and research and a new examination format with objective structured clinical examinations (OSCE) in addition to MCQs and essay questions (Ministry of Health Singapore, 2013). We conducted a survey of Singapore psychiatry trainees in 2008 to assess their perceived importance and adequacy of various aspects of psychiatric training (Phern-Chern, Tze-Pin, & Ee-Heok, 2010). The study demonstrated there were perceived deficiencies in all aspects of training that supported the need to move to a more structured training system.

After the improvements of the seamless training program, Singapore adopted the American style competency based residency training for psychiatry in 2010 which promised significant advances to the postgraduate training program with an increased emphasis on structured and formative training (Minisitry of Health SIngapore, 2013). The improvements included institutional structures providing adequate resources for training; a clearly defined and well planned curricula permitting trainees to undertake progressively graded responsibilities; regular formative assessments on core competencies; and protected time for core faculty and residents to implement the program. (Accreditation Council for Graduate Medical Education International LLC, 2013).

Despite the training improvements, a recent study (Mahendran, Broekman, Wong, Lai, & Kua, 2013) found that psychiatry residents in Singapore were dissatisfied with their training. Areas of dissatisfaction included purported strengths of the residency program like protected educational time and accessibility to their clinical teachers, even compared to the non-residency trainees. If confirmed these findings have major ramifications for psychiatric training in Singapore and possibly for the increasingly dominant competency based medical education movement (Iobst et al., 2010) which has come under attack for demotivating trainees, decreasing educational content (Leung, 2002) with a focus on a minimum acceptable level of skill (Talbot, 2004).

The Mahendran et al paper had two major limitations: Firstly, the generic nature of the PHEEM meant it was not designed to assess postgraduate psychiatric training programs. The second limitation was the cross sectional nature of the study which would not be able to discern if the results of the survey were due solely to the training program or the interaction of the training programs with time / progressive cohorts or the effect of having multiple training programs interacting alongside one other.

We set out to repeat our 2008 survey of the perceived importance and adequacy of various aspects of psychiatric training in Singapore (Tor, Ng, & Kua, 2010) which has a scale specifically developed to assess psychiatric training. Repeating the study would also allow us to compare the scores of the trainees in 2008 with the residents in 2013 and help determine the effect of the training programs on trainee’s perception while accounting for a time / cohort effect. We hypothesize that the residents will have higher perceived adequacy of training due to their more structured and competency based training program, and equivalent perceived importance scores for psychiatric training compared to other trainees.

Methods

A list of all trainees and their emails was obtained from the specialist training committee and an electronic survey form was emailed out to all trainees inviting them to participate in the survey. Multiple emails were sent out to increase response rate. The survey form asked respondents to rate 11 aspects of psychiatric training on a 5 point Likert scale (where 1 was not important or not adequate and 5 was very important or very adequate) that were based on the psychiatric syllabus. Respondents rated the perceived importance and adequacy of the 11 aspects of psychiatric separately.

The survey was conducted from April to Aug 2013. All analysis were done with SPSS 14 (Chicago, 2005). Comparisons of perceived importance and adequacy within groups were analyzed with paired t-tests and comparisons between groups were done with independent t-tests. Two-tailed tests of significance were used and the level of significance was set at p<0.05. The work was carried out in accordance with the declaration of Helsinki and ethical approval was granted by the institutional review board (NHG DSRB 2013/00199).

Results

A total of 44 responses (57.9%) were obtained, comparable with the response rate of trainees in 2008 (n=27, 58.7% response). 73.5% (n=25) of residents responded and 45.2% (n=19) of non-resident trainees responded. The perceived adequacy of training was significantly lower than the perceived importance for all 11 aspects of training in both 2008 and 2013. The perceived importance of only 2 of 11 aspects of training (Disorder and Diagnosis knowledge and Basic Neuroscience) were statistically different between years, with the former being higher and the latter lower in the 2013 cohort. However for perceived adequacy, 6 of 11 aspects of training were rated significantly different and all 6 were rated higher by the 2013 cohort compared to the 2008 cohort. (See Table 1)

The scores for perceived adequacy between groups showed a clear pattern for residents rating their perceived adequacy significantly higher than other groups in 16 of 19 statistically significant scores (details available from author on request).

Discussion

Our survey showed that trainees consistently rated perceived adequacy lower than perceived importance of training in all three training programs in both 2008 and 2013. However the perceived adequacy in 2013 was demonstrably better than 2008 with more than half (6 of 11) of the aspects rated significantly higher in 2013. Most of this effect came from residents rating their perceived adequacy of training higher than other trainees. The perceived importance of different aspects of psychiatric training were very similar between 2008 and 2013 with 9 of 11 aspects not being statistically different and the 2 statistically significant differences being in different directions. Taken together these findings suggest that the changes in training programs did not affect the perceived importance of the various aspects of psychiatric training but did improve the perceived adequacy of training of the residents.

These results are consistent with the hypothesis that residents would rate their perceived adequacy of training higher than other trainees and may be due to their more structured and formative training program. The residency program would have provided a training environment more consistent with the requirements of this learning model compared with the other trainees who had a training environment that had service requirements rather than training as a primary consideration.

Why were our results ostensibly opposite from Mahendran et al who found that residents rated their training worse than other trainees in areas where residency was supposed to be stronger? One possible reason could be that the 2 surveys assessed different aspects of training. The PHEEM survey was designed to assess the overall training environment while our survey form specifically looked at the desired outcomes of psychiatric training. Looking at the specific areas where residents rated their training poorly (protected educational time, opportunities to participate in training activities and accessibility of their clinical teachers), the resident’s ratings may be been influenced by their baseline expectations (Oliver, 1977). Residents were given high expectations for protected educational time, structured training and close, personal and individual training, especially in comparison to previous trainees where the concept of protected training time was somewhat academic and trainees often had to take turns to attend national training programs in order to ensure continuity of clinical service provision. Residents could thus have given lower ratings than other trainees as their baseline expectations were higher, even if they received objectively more protected training time, opportunities for training and individual supervision than other trainees.

This bias of baseline expectations was unlikely to affect our survey for perceived adequacy of training as there was no expectation that residents would be better trained than their non-resident trainee counterparts. If anything there was an expectation that residents may not be as adequately trained as their non-resident trainee counterparts as their training was up to 2 years shorter than non-resident trainees.

The other possible reason for seemingly different results with Mahendran et al could be that a statistically significant difference in scores may not have a clinically significant difference. The difference between residents and non-resident trainees on the teaching subscale of the PHEEM (range 0 to 60) was 0.6 in favour of the non-resident trainees in the Mahendran et al study.

Lastly our survey was conducted a year after the Mahendran et al survey and our survey population had a higher proportion of residents. The trends in the resident training group compared with other trainees could have become clearer during our study.

Limitations of our study include the relatively low response rate and lack of statistical correction for multiple comparisons. However response rates of 58% are generally considered acceptable for such surveys and are similar to the response rate in 2008. No correction for multiple statistical comparisons was made which may have resulted in spurious statistically significant results. However the overall trend seen in the residents group compared to the other trainees were seen in multiple aspects of training and consistent with the expected changes of the training program. We feel confident that our survey results measured a real difference in perceived adequacy between the residents and other trainees.

Take Home Messages

Our survey shows that transitioning from a serviced based psychiatric training program to a more structured training program with a greater emphasis on formative training results in a higher perceived adequacy of training. These results support the move towards residency training. More research to assess the objective training adequacy of trainees and its correlation with the trainee’s self-perception of adequacy is recommended.

Notes On Contributors

Dr Tor is Head of Neurostimulation Service and a consultant with the Mood Disorders Unit, Department of General Psychiatry, Institute of Mental Health (Singapore). He is also on a National Examiner with the Singapore Ministry of Health and a core faculty member of the Singapore Psychiatry Residency Program.

Associate Professor Chiam is a senior consultant with the Department of Geriatric Psychiatry, Institute of Mental Health (Singapore) and Vice Chairman Medical Board (Education) and the immediate past Director of Education Director’s Office at the Institute of Mental Health (Singapore) and immediate past Program Director of the National Psychiatry Residency Program (Singapore).

Acknowledgements

Bibliography/References

REFERENCES

Accreditation Council for Graduate Medical Education International LLC. (2013). Advanced Specialty Program Requirements for Psychiatry. Retrieved from http://www.acgme-i.org/web/requirements/psychiatry.html

Department of Statistics Singapore. (2012). Key Annual Indicators. Retrieved from http://www.singstat.gov.sg/stats/keyind.html#popnarea

Iobst, W. F., Sherbino, J., Cate, O. T., Richardson, D. L., Dath, D., Swing, S. R., Frank, J. R. (2010). Competency-based medical education in postgraduate medical education. Medical Teacher, 32(8), 651-656.

http://dx.doi.org/10.3109/0142159X.2010.500709

Leung, W.-C. (2002). Competency based medical training: review. BMJ: British Medical Journal, 325(7366), 693.

http://dx.doi.org/10.1136/bmj.325.7366.693

Mahendran, R., Broekman, B., Wong, J. C., Lai, Y. M., & Kua, E. H. (2013). The educational environment: Comparisons of the British and American postgraduate psychiatry training programmes in an Asian setting. Medical teacher, 35(11), 959-961.

http://dx.doi.org/10.3109/0142159X.2013.815707

Ministry of Health Singapore. (2013). About Residency. Retrieved from http://www.physician.mohh.com.sg/residency/about_residency.html#about-residency

Ministry of Health Singapore. (2013). Seamless Psychiatry Training Program. Retrieved from http://www.healthprofessionals.gov.sg/content/hprof/sab/en/topnav/specialist_training/general_overview.html

Oliver, R. L. (1977). Effect of expectation and disconfirmation on postexposure product evaluations: An alternative interpretation. Journal of Applied Psychology, 62(4), 480.

http://dx.doi.org/10.1037/0021-9010.62.4.480

Phern-Chern, T., Tze-Pin, N., & Ee-Heok, K. (2010). Adequacy of psychiatric training: a Singaporean perspective. Acad Psychiatry, 34(1), 50-53.

http://dx.doi.org/10.1176/appi.ap.34.1.50

Talbot, M. (2004). Monkey see, monkey do: a critique of the competency model in graduate medical education. Medical education, 38(6), 587-592.

http://dx.doi.org/10.1046/j.1365-2923.2004.01794.x

Tor, P.-C., Ng, T.-P., & Kua, E.-H. (2010). Adequacy of Psychiatric Training: A Singaporean Perspective. Academic Psychiatry, 34(1), 50-53.

http://dx.doi.org/10.1176/appi.ap.34.1.50

Appendices

Table 1. Comparison of Perceived Importance and Adequacy of Psychiatric Training by trainees in Singapore between 2008 and 2013 (Mean / SD)

 

2008 trainees (n=27)

2013 trainees (n=44)

(Importance)†

(Adequacy)†

(Importance)†

(Adequacy)†

Disorder and diagnosis knowledge

4.6 (0.6)*

3.9 (0.7)*

4.9 (0.4)*

4.4 (0.7)*

Pharmacological treatment knowledge

4.7 (0.6)

3.6 (0.6)*

4.8 (0.7)

4.0 (0.7)*

Psychotherapy treatment knowledge

4.1 (0.6)

2.1 (0.8)*

4.2 (0.7)

2.9 (1.0)*

Psychosocial treatment knowledge

4.1 (0.7)

2.5 (0.4)*

4.2 (0.7)

3.1 (0.9)*

Clinical interview skills

4.8 (0.4)

3.0 (0.6)*

4.9 (0.4)

3.8 (1.0)*

Clinical treatment skills

4.7 (0.4)

3.5 (0.8)*

4.8 (0.5)

4.0 (0.8)*

Team leadership and collegial skills

3.8 (0.8)

2.4 (0.9)

3.8 (0.9)

2.8 (1.1)

Research skills

3.7 (0.8)

2.3 (0.8)

3.3 (0.9)

2.6 (0.8)

Ethics

4.1 (0.8)

2.7 (0.9)

3.3 (0.8)

3.0 (1.1)

Basic Neuroscience

4.0 (0.7)*

2.4 (0.9)

3.6 (0.9)*

2.8 (1.0)

Cultural aspects

3.9 (0.7)

2.6 (0.9)

3.7 (0.9)

2.8 (1.1)

* - Difference between respective aspects of training in 2008 and 2013 for Perceived Importance / Adequacy statistically significant at p<0.05

† - Difference between respective aspects of training for Perceived Importance and Adequacy within groups (2008 or 2013) all statistically significant at p<0.05

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Reviews

Please Login or Register an Account before submitting a Review

Trevor Gibbs - (08/06/2016) Panel Member Icon
/
The topic of evaluation of our curriculum activities is very important, and in the case of this paper, the evaluation of a training programme based upon an American model imported into Singapore, very important. The potential for other curriculum reformers throughout the world to learn from such papers should be high on the outcomes agenda of such papers. Unfortunately, and in relation to this paper, that potential has not been realised and the paper remains rather firmly a locally relevant one.
That said, I was left unsure of what the authors were trying to compare over a time period where much change could have / should have happened, and by using such a low-level evaluation method. The main message appears to be that some improvement has occurred but there is still a lot of work to do
Richard Hays - (08/06/2016) Panel Member Icon
/
This paper describes an evaluation of the recently-introduced Singapore residency training program in psychiatry, using the USA model. While this is a worthy aim, there are some flaws in the conceptualisation and methods that limit the analysis and interpretation. First, the transition was probably not ‘from’ an ‘apprenticeship’ model ‘to’ a ‘competency-based’ model. Both UK and USA models have elements of apprenticeship and competency assessment, but they may be described and operationalised differently. It may be that both models produce sound graduates. Secondly, most of the respondents may have had little experience of the older model, so comparative comments a may be less valid, based on hearsay from those either against or in favour of the change. Third, it is not possible to say that one model is better than another without more data, including perspectives of those who completed training in both models, and ideally some impact/outcome data. There is some value in hearing that there are still some weaknesses in training and these should be addressed.
Style: Well written and easy to read
Target audience: Of most interest to the local Singapore medical educators.