Research article
Open Access

Identifying the perceived challenges that affect the use of portfolio as an assessment tool for family medicine residents in Jeddah program

Badr Binhimd[1], Hatim Mohammed Alawi Al-Jifrfee[2], Muhammad Anwar Khan[3]

Institution: 1. King Saud bin Abdulaziz University for Health Sciences - Jeddah, 2. King Abdulaziz Medical City for National Guard - Jeddah, 3. King Saud bin Abdulaziz University for Health Sciences - Jeddah,
Corresponding Author: Dr Badr Binhimd [email protected]
Categories: Assessment, Medical Education (General)

Abstract

Background: Portfolio can be defined as  "Purposeful collection of student work that exhibits to the student (and others) the student’s efforts, progress, or achievement in a given area(s).” It encourages the student for self-learning and reflection, and it has a significant educational impact in supervision and training. The purpose of the study is to increase the awareness about the factors that enhance or inhibit the utilization of portfolio as an assessment and educational tool from participants perspective to the scientific committee of family medicine program.

Objective: To explore the challenges and obstacles in using the portfolio as perceived by residents of family medicine program in Jeddah.

Methodology: Interviews were conducted using purposeful sampling technique till saturation was reached. All National Guard family medicine residents in Jeddah from R2-R4 (R2=9, R3=7, R4=5) were included. The total number of participants were 21. Interview time, date and location were arranged with each one of them through direct contact after he/she confirmed their participation. Data collection was done through face-to-face, in-depth, semi-structured interview (in English) for each resident separately. All recorded interviews were transcribed. Predetermined themes were used to analyze the data.

Results: Eleven interviews were conducted from November to December 2016. Analysis revealed eight themes: six major and  two minor. The major themes are those representing the adult learning principles. The minor themes are additional important factors that affect portfolio utilization.

Conclusion: The main problem was in the implementation and understanding. Most of the residents agreed that the limited time for them and their supervisors was the main challenge. The benefits and impact of the portfolio on learning and practice were good when they discussed the topic/case with their supervisors and got feedback. 

Keywords: Portfolio, Postgraduate, Family Medicine, Saudi Arabia,

Introduction / Background

The portfolio is one of the effective assessment methods in medical education. It is a great addition to residents’ assessment because it provides learner and teacher with the needed information about the learning process and student progression. Also, it encourages the student for self-learning, and it has a significant educational impact in supervision and training.1

Portfolio can be define as “a collection of material, made by a professional, that records, and reflects on, key events and processes in that professional’s career”.2 "Purposeful collection of student work that exhibits to the student (and/or others) the student’s efforts, progress, or achievement in a  given area(s)”.3 So, it is a tool to assess the residents continuously and represent his/her achievements, and also to assess their defects, needs, and progress. Self-directed learning and reflection are the main characteristics of this method of assessment. Accordingly, the use of portfolio is based on the adult learning theory. In 1980, Knowles made four assumptions about the characteristics of adult learners (andragogy) that are different from the assumptions about child learners (pedagogy). In 1984, Knowles added the fifth assumption. These five assumptions include: self-concept, adult learner experience, readiness to learn, orientation to learning and motivation to learn.4

The attitudes toward portfolio were assessed in a study which was done at the University of Dundee Medical School over four years (1999, 2000, 2002 and 2003).5 They found that the attitude and acceptability of students improved over the years of the study but concerns about the load of fulfilling all the requirement in short time. But on the other hand, the portfolio was helpful in gaining knowledge and skills and supporting their lifelong learning and reflective clinical practice.5 Another study in South Africa done to explore registrars’ and supervisors’ experience regarding the portfolio’s educational impact, acceptability, and perceived usefulness for assessment of competence showed that portfolio had a substantial educational influence on the training and assessment.6 There are some limitations with the use of portfolios that have been reported including time-consuming, distracting from other forms of learning. In addition to concerns about uniformity, acceptance, utilization and educational benefits of the portfolio.7 Until now the utilization of portfolio in Saudi medical colleges is limited and under expectations from both faculty and students.8

The family medicine module’s portfolio has four main components: The learning contract, evidence of learning, reflections as per Gibbs and logbook. So, the perception, acceptability, and experience of the portfolio by both residents and their trainers are important factors in the implementation and better utilization of this method of assessment. Accordingly, involving all stakeholders in selecting the topic and competencies that can be assessed using this tool is very important to achieve the maximum benefits. So, the aim of the study is to identify the challenges which affect the use of portfolio by family medicine residents.

Rationale

  1. Increase the awareness about the factors that enhance or inhibit the utilization of portfolio as an assessment and educational tool from participants perspective to the scientific committee of family medicine program.
  2. Up to my knowledge there were no similar qualitative studies done for family medicine residents.

Aim

To identify the challenges which affect the use of portfolio by family medicine residents.

Objectives

To explore the challenges in using the portfolio as perceived and experienced by residents of family medicine program in Jeddah.

Methodology

This qualitative study was conducted in the family medicine program. Interviews were conducted using purposeful sampling technique till saturation was reached. Data collection was done through face-to-face, in-depth, semi-structured interview (in English) for each resident separately. All recorded interviews were transcribed. Predetermined themes were used to analyze the data.

Ethical Considerations:

Permission from the program director of Family Medicine department was taken to start the study after approval from KAIMRC. All participants signed an informed consent form before each interview. All the information was confidential, and no name was written or asked during the interview. Confidentiality ensured by keeping the records of interviews in safe digital password protected vault. All participants’ identity was kept confidential and was identified by a coded serial numbers. The researcher, supervisor, and co-supervisors are the ones who can access the data. No incentives were given to participate in the study.

Study Area / Setting

This study was conducted in the family medicine program center in Jeddah which was established in 1994. The program is four years, consisting of many rotations to cover most of the specialties. Residents’ assessment depends on both formative and summative methods. Since two years National Guard Family Medicine program was separated and recently became independent. The total number of National Guard residents is 39 supervised by ten family medicine consultants in addition to other trainers participating in different centers.

Sample Size

All National Guard family medicine residents in Jeddah from R2-R4 (R2=9, R3=7, R4=5) were included. The total number of participants was 21. R1 residents excluded because they have just started their residency and they don’t have any experience with the portfolio.

Study Design

The research question was answered through a phenomenological study which explored the perspective and experience of study subjects.9 Phenomenological study has been chosen to allow the exploration of residents’ perceptions and experiences of the factors that enhance or inhibit the use of portfolio as an assessment tool. So, this design examined residents lived experiences through the descriptions provided by them.

Sampling Technique

Silverman recommends using purposive sampling in research based on interviews.10 In this sampling technique, investigator depends on his judgment when selecting members of the population to join in the study for the interview.

Data Collection Method

An invitation to all residents (R2-R4) was sent (with all the details) through email to participate in the study. Also, the idea of the research was explained to all of them through their chief resident. Each resident required to confirm his/her willingness and availability for participation through email. Interview time, date and location were arranged with each one of them through direct contact after he/she confirmed their participation.

One day before the interview the location, time, date, day and duration (30 min ) were confirmed by direct contact to all residents. Before starting the interview the place was prepared (privacy, away from noise and distraction, comfortable seats, suitable room temperature), the resident was welcomed and provided water, coffee/tea to be comfortable and ready for the interview.The interview instructions were explained to all of them and written consent was taken.

At the beginning of recording the date, day, time, the title of the research, participant code were mentioned, and verbal consent was taken for each resident. At the end of the interview, each resident received a certificate of thanks and appreciation.

Data collection started using a list of ten open-ended questions and probes which were pre-determined: Six of them based on the adult learning theory4 ( Q.3-Q.8), three of them were added to explore more ideas about factors which can affect the utilization of portfolio (Q.1,2,9 ) and finally, the last question (Q.10) was about the recommendations from residents’ perspectives.

A pilot study was conducted before the intended study on two newly graduate assistant consultant family medicine to test the research process and questions. Both assistant consultants were not included in the main study.

Data Management and Analysis Plan

All recorded interviews were transcribed immediately after the interviews were done.Adult learning theory themes were used to analyze the data 4 in addition to two more important themes. Manual data analysis was done. Transcripts were reviewed two times and coded based on the predetermined themes (Template Analysis).11

All transcripts were compared to the recorded material and reviewed to have a general understanding and comprehension of each participant (first review). Then categorizing (coding /labeling) relevant information according to the predetermined themes  (each question was representing a theme) was done (second review). Finally, results were written by describing the categories and finding.

Validity

The results of the study were discussed again with a focus group of the participant to compare the finding to their answers. All of them agreed with the results and said they represent their answers.

Reliability

Silverman contends that reliability in qualitative research involves selection of what is recorded, the technical quality of recordings and the adequacy of transcripts.10 Every effort was made to accurately record and transcribe the interviews using the best equipment as well as backup recorders.

Results

Eleven interviews were conducted. Analysis revealed eight themes: six major and two minor. Major themes are those representing the adult learning principles, and the minor themes are additional important factors that affect portfolio utilization.

Table 1. Themes Description

 

Theme

Overview

Major

Motivation

 

-As a person matures, he or she need the motivation to learn

-The residents receive their motivation from internal (mainly) and external factors

Practicality

-Residents want to learn what they can apply and implement in real life

Self-directed learning

-Is an instructional strategy where the students, with guidance from the teacher, decide what and how they will learn

-It can be done individually or with group learning, but the overall concept is that students take ownership and responsibility for their learning

Life experience (self-reflection)

 

-Educators help students in drawing out relevant past knowledge and experiences and relate the sum of learners’ experiences to the current learning experiences

-Adults like to be given the opportunity to use their existing  knowledge and experiences gained throughout their life and apply it to their new learning experiences

Goal and relevancy oriented

 

-An adult needs clear goals and objectives to learn. They should have learning goals to be ready to learn

-Adult learners want to know the relevance of what they are learning to what they want to achieve

Respect and collaboration

 

-When learners are considered by their instructors as colleagues, they become more productive

-Respect can be demonstrated to your student by:

   -Taking an interest in the residents’ work

   -Guide, support, help the resident in his / her training

   - Appreciate and respect  his /her  effort and experiences

   - Follow the resident progress

Minor

Orientation and understanding

 

It a very important step at the beginning of the residency program for both residents, supervisors, and mentors for deep and comprehensive understanding of portfolio

Challenges

Factors which impede the utilization of portfolio

 

Table 2. Study results

Theme

Results

Quotations

Motivation

 

-Some (6/11) residents said that it shows them their limitations and what they need to improve and it motivate their learning

 -Other residents (5/11)  said that portfolio is not a motivational tool because they can enhance their learning as adult and know what they need without wasting their time with portfolio

“It motivates your reading and discussion with your supervisor.”

“I don’t see it as a motivational tool at all, I see it as requirement which needs to be done to get the grades ”

 

Practicality

 

-Some residents  (7/11) felt that using portfolio helped them in the clinic with their patients when they discuss the case with their supervisors and get feedback from them

-Others (4/11) said that the benefits and impact of portfolio on their learning and  practice was limited

 

“ It is beneficial and help me with patients because I'm writing the cases in my logbook and  discussing them with my supervisor as CBD or Mini -CEX  and this will help  me to improve myself with the next patient specially if I face the same case ”

“Not significant impact. For me it is only an assessment tool, with it or without it does not make a big difference in dealing with my patients and patient outcome.”  

“As I said It’s not about the portfolio, I do it for myself; I face a case and read about it.  I face a situation I ask my seniors.  Even if there are no documented papers, the consultant feedback is still positive.”

Self-directed learning

-Most residents agreed (7/11 ) that portfolio may help in self-directed learning

-Few residents (4/11) mentioned that portfolio did not enhance their self-directed learning

“Even if there is no portfolio or supervisor, I can put my goals and follow my progress and achievements, and I can decide what I need ”

Life experience (self-reflection)

-Some residents (6/11) believed that portfolio is not helpful because they are doing that daily without the portfolio

 -Others (5/11) said that it helped them to certain extent

“self-reflection is part of adult learning process, and as a resident, in my level of maturity, I’m doing self-reflection on a daily basis. So portfolio not that much helps me at this point. ”

Goal and relevancy oriented

 

-Most of the residents (8/11) said writing objectives and discussing them with the supervisor will help in the learning process.

-Few (3/11) felt it’s a waste of time and not helpful and if they need to write some goals, they will write them in their way in another place other than portfolio

“It enhances me to achieve my goals.”

“It’s in my mind, why I should write it again.”

Respect and collaboration

 

-All the residents said they were respected and appreciated from their supervisors.

-Some residents (6/11 ) said it enhances the collaborative work and effort between them and their supervisors.Other said (5/11) said they didn’t feel the difference while using portfolio.

“I was lucky to have a good supervisor last year which was supporting and guiding me for the whole year. She was showing me my strength and area of improvement. This year they changed my supervisor with another one which concentrates only on assessment and knowledge which has little impact on my practice.”

Orientation and understanding

 

-Many residents (7/11 ) said that the orientation about portfolio wasn’t enough and they didn’t understand it very well.

-Also, most residents (9/11) mentioned that some supervisors and mentors still not fully oriented about how to utilize portfolio in teaching and assessment.

“Unfortunately, I did not have the full orientation about the portfolio.  Still, there are things that I didn’t understand about it.” 

“The orientation at the beginning was vague; they give us the guidelines,  curriculum,  nobody explained  the portfolio clearly at first,  there were no clear examples how should we utilize and complete the content of the portfolio ”

Challenges

-Most residents (10 /11) agreed that there were some challenges which affected the implementation and utilization of portfolio:

-Restricted time was the primary factor for both residents and supervisors.

-Understanding of the utility and implementation of the portfolio by some of supervisors /mentors was not clear.

-Limited content of the portfolio which is too specific in types and number.

-The portfolio became the main method of assessment:

     - This started from this year 2016-2017

     - Led to more stress and bias in

       assessment

“It affects our time of studying just to finish writing and filling the forms.”

“I think we miss some point for the role of the supervisor.  It was like just monitoring.”

“We are searching for someone to see the case and evaluate it with us. Maybe if all the medical staff were familiar with portfolio, it would be much easier.”

Discussion

The portfolio was a good motivational tool for some residents mainly when they discussed it with their supervisors who helped in improving their learning and practice. Also, it enhances their self-directed learning because it allowed them to write their goals and plans which guided them in their training and act as a reference for them. Others felt that they are mature enough to motivate themselves and to write their goals if needed. They believed that discussing with their supervisors without portfolio is more beneficial because they were not restricted to forms or marks.

Some residents were satisfied with their supervisors because of their help, support, and guidance. They felt the portfolio enhanced the relationship between them and their supervisors who were oriented about portfolio and enthusiastic to teach. This collaborative work was reflected positively on their learning and practice. There were some factors which affected the collaborative work between both of them. The main one was the restricted time which was one of the primary challenges for both residents (time to write and fulfill the requirements and time for discussing with their supervisors /mentors) and supervisors who were busy in their work and with other residents (4 or more). Another factor was the understanding of portfolio and how to utilize it efficiently in assessment and education which was not clear for some supervisors and mentors who still not fully oriented about their role. The orientation wasn’t enough and concentrated on the portfolio as an assessment tool, not as a life-long educational tool. This result was similar to the one mentioned in the Saudi Medical Journal. They found that the main problem in utilizing portfolio was the implementation and understanding.8 The content and methodology were explained to all residents, but inspite of that, it wasn’t adequate for most of them who perceived it as extra paperwork which consumed their time.

The residents started to understand and accept it more during their rotations year by year through practice and from the further explanation from their mentors. So, they became more familiar with it and knew how to utilize it in the minimum time without affecting their learning to fulfill all the requirements and get the maximum benefit from it. This finding was similar to the result of the study, which was done at the University of Dundee Medical School over four years (1999, 2000, 2002 and 2003). They found that the attitude and acceptability of students improved over the years of the study but concerns about the load of fulfilling all the requirement in short time. 5  Also , the results were similar with the same study regarding portfolio as a helping tool in gaining knowledge and skills to a certain extent.

Regarding self-reflection, some residents didn’t feel that portfolio has that much impact on their reflection because they are an adult learner and they were doing that daily to improve their practice regardless the portfolio which is mainly for assessment. This underutilization of portfolio in self-reflection was because of their limited understanding about the main idea behind portfolio which is an educational tool which encourages self-directed learning and reflection.

Most residents agreed that portfolio was a subjective method of assessment because it was based on specific forms (which didn’t include other achievements and work) which was assessed and signed by a supervisor who even didn’t see the case with them sometimes. Also, using portfolio didn’t reflect the real competencies of the residents and led to more bias (from the side of resident, supervisor, and mentor) and stress. Some residents did more tasks and chose the best one of them to be given to their supervisor for evaluation. In addition to the final evaluation at the end of each month which was filled by their mentors who didn’t supervise them during their training and rely on the portfolio documents, some discussion with residents and evaluation from a supervisor in the department where the training was done.

So, most of them found portfolio a helpful tool for learning which guided them, organizing their learning and help them to follow their progress and improve themselves specially when they discuss and got feedback from their supervisors /mentors. On the other hand, most of them found it a weak tool for assessment because of its subjectivity and limitations.Another objective method of assessment should be added like MCQs. So, instead of thinking how to overcome those challenges which affect the utilization and implementation of the portfolio as an assessment tool, we should think about these questions: what is the efficacy of portfolio as an assessment tool? Which competencies can portfolio cover as an assessment tool? Can portfolio be the only assessment tool at the end of each rotation? Do we need to use the portfolio as an assessment tool and what it will add to the assessment?

Conclusion

Most of the residents agreed that the limited time for them and their supervisors was the main challenge. Orientation and understanding about the utilization and implementation of the portfolio were the second important challenge. The benefits and impact of the portfolio on learning and practice were good when they discuss topic/case with their supervisors and get feedback (still need improvement, more time for discussion and feedback, better understanding). The portfolio was not convincing as an assessment tool because it’s subjectivity and limitations. All the residents emphasized on the important role of their supervisors /mentors to get the maximum benefits from the portfolio. The main problem was in the implementation and understanding.

Recommendations

More orientation sessions for supervisors, mentors and residents.

Add other content in the portfolio which reflect the residents’ work. 

Give more time for both residents and supervisors for discussion and feedback.

Reconsider using the portfolio as an assessment tool or the only tool during each rotation.

Consider adding other methods for assessment like MCQ to decrease the subjectivity.

Do further research about the efficacy of portfolio as an assessment tool.

Include supervisors and mentors in further research to know their experience and perspectives about the portfolio.

Take Home Messages

A- Portfolio is a good tool which encourages the residents for self-learning and reflection.

B- Portfolio has a significant educational impact in supervision and training.

C- Overcoming the main obstacles will allow both residents and supervisors to utilize it effectively.

D- Feedback and reflection are the cornerstones of portfolio and they need more time and orientation.

E- The main problem is in the implementation and understanding mainly by the supervisors and mentors.

F- Portfolio can not reflect the real competencies of the residents and lead to more bias and stress. So, it is not a convincing tool for assessment because it’s subjectivity and limitations which necessitate considering other methods for assessment and confine the role of portfolio for education and following the resident's progress.

Notes On Contributors

Dr. Badr Binhimd: Consultant Family Medicine, Trainer in the Postgraduate Family Medicine Program in Jeddah-KSA at National Guard -WR Saudi and Arab Board of Family Medicine from the Joint Program of Family Medicine in Jeddah, Master of Medical Education from KSAU.

Dr. Hatim Mohammed Alawi Al-Jifrfee, M.B, Ch.B.(Hon), FRCSC, MMedEd, CHSE. Gynecological Oncology Consultant, Assistant Professor, Department of Oncology, KAMC KAMC-JEDDAH NGHA, KSAU-HS College of Medicine - Jeddah.

Dr. Muhammad Anwar Khan, MBBS, DCPS (HCSM), MBA, PGD, Lecturer, Department of Medical Education, KSAU-HS College of Medicine - Jeddah.

 

 

Acknowledgements

  • My supervisor: Dr. Hatim Al-Jefree
  • My co-supervisor: Dr. Muhammad Anwar Khan
  • MME department staff in Jeddah and Riyadh for their support and follow up

Bibliography/References

1. Challis M. AMEE Medical Education Guide No. 11 (revised): Portfolio-based learning and assessment in medical education. Med Teach. 1999;21(4):370-86.

https://doi.org/10.1080/01421599979310   

2. Hall D. Professional development portfolios for teachers and lecturers. British Journal of In-Service Education. 1992;18(2):81-6.

https://doi.org/10.1080/0305763920180203   

3. Reckase MD. Portfolio assessment: A theoretical esthnate of score reliability. Educational measurement: issues and practice. 1995;14(1):12-4.

https://doi.org/10.1111/j.1745-3992.1995.tb00846.x   

4. Knowles M. Adult learning processes: pedagogy and andragogy. Religious Education. 1977;72(2):202-11.

https://doi.org/10.1080/0034408770720210   

5. Davis MH, Ponnamperuma GG, Ker JS. Student perceptions of a portfolio assessment process. Med Educ. 2009;43(1):89-98.

https://doi.org/10.1111/j.1365-2923.2008.03250.x   

6. Jenkins L, Mash B, Derese A. The national portfolio of learning for postgraduate family medicine training in South Africa: experiences of registrars and supervisors in clinical practice. BMC Med Educ. 2013;13(1):149.

https://doi.org/10.1186/1472-6920-13-149

7. Joshi MK, Gupta P, Singh T. Portfolio-based learning and assessment. Indian pediatr. 2015;52(3):231-5.

https://doi.org/10.1007/s13312-015-0613-2   

8. Fida NM, Shamim MS. Portfolios in Saudi medical colleges: Why and how? Saudi Med J. 2016;37(3):245-8.

https://doi.org/10.15537/smj.2016.3.12937   

9. Marshall MN. Sampling for qualitative research. Family practice. 1996;13(6):522-6.

https://doi.org/10.1093/fampra/13.6.522   

10. Silverman D, editor. Qualitative research: theory, method and practice. London: Sage; 2006. Brooks J, McCluskey S, Turley E, King N. The utility of template analysis in qualitative psychology research. Qual Res Psychol. 2015 Apr 3;12(2):202-222. Epub 2014 Sep 2.

11. Brooks J, McCluskey S, Turley E, King N. The utility of template analysis in qualitative psychology research. Qual Res Psychol. 2015 Apr 3;12(2):202-222. Epub 2014 Sep 2.

https://doi.org/10.1080/14780887.2014.955224

Appendices

Questions of portfolio study (Qualitative) 

  1. What is your understanding of portfolio (content, needs, assessment tool, efficiency)?
  2. How was the orientation about using the portfolio?
  3. What do you think about portfolio as a motivational tool for learning?
  4. What is the educational benefits or impact of the portfolio on your learning and real-life situations (practicality regarding learning and dealing with patients)?
  5. Explain how portfolio enhances the collaborative work and effort (to guide and support your learning) between you and your supervisor?
  6. Explain how portfolio help you to take the ownership and responsibility of your learning?
  7.  Discuss the effect of the portfolio on self-reflection?
  8. To which extent portfolio help you in formulating and achieving your learning goals?
  9. What are the limitations, obstacles or challenges which affect the implementation and utilization of portfolio?
  10. In your opinion, how you can overcome these challenges?
There are no conflicts of interest.

Please Login or Register an Account before submitting a Review

Reviews

Ken Masters - (09/04/2017) Panel Member Icon
/
A somewhat useful paper describing the impressions of portfolios. Although the researchers have gone to some length to record and document their methodology, there are problems with it.

The authors appear to have selected ideas from different methodological approaches, and these different approaches do not always gel: The authors have used the saturation method for their sampling limits, which is fine, and is common in much qualitative work. But, when it is combined with predetermined themes, rather than emergent themes, that leads to a weakness of data exclusion or artificially inserting data into pre-conceived ideas. I wonder if the authors could expand on why they chose to do this, rather than to have the themes emerge from that saturation process. Also, what happened to information that did not conform to the themes? It is important for the authors to address this potential weakness.

Compounding this problem, the authors state that “The research question was answered through a phenomenological study which explored the perspective and experience of study subjects.” They give a paper by MN Marshall as their reference. There are three problems here:
• Nowhere else in their paper do the authors elaborate on any activity they perform that follows phenomenological principles. Given that this is the method used for answering the research question, the authors need to explain more carefully, with direct reference to phenomenological approaches, how they have used phenomenological approaches to answer the research question.
• Phenomenological approaches focus on deep probing, often requiring repeat interviews with the subjects. From the descriptions given in the paper, the authors’ interviews appear to follow standard qualitative interviews with a set of questions, and have not described anything that looks like deep probing. This may be a flaw in the methodology or the description, but, either way, needs to be addressed. There were also no repeat interviews (These are not always necessary in phenomenological studies, but they would lend weight to the contention that the authors were using a phenomenological study to answer the research question).
• The reference given to support the use of a phenomenological approach is “Marshall MN. Sampling for qualitative research. Family Practice. 1996;13(6):522-6.” That paper, however, does not mention or allude to phenomenological design, so is an inappropriate paper to use for this reference. (In addition, the paper’s pages are 522-525, not 526, as given by the researchers – this is a minor error, but merely adds to the impression of sloppy referencing). The authors need to take greater care when citing research – if the researchers have incorrectly cited a paper that provides the rationale for the approach in answering their research question, then a reader will be sceptical of other citations also.

So, it would be useful if the authors could clarify, in some detail, exactly how they have used phenomenological principles and approaches, and also the source of their theoretical perspective. Otherwise, the impression might be given that the authors have used a sort of modified Grounded Theory approach, with passing reference to phenomenological principles.
Trevor Gibbs - (07/04/2017) Panel Member Icon
/
Portfolios are a very well research educational tool for teaching,learning, and when used appropriately, assessment. This paper looked at the utilisation of portfolios through a very structured and very directed interview technique, that although the authors spoke of inquiry until saturation, appeared to be very time-limited and limited in its depth. Recognition should also be given that it was conducted in only a very small nuber of students from one programme. I think the authors are very right in suggesting that much more work, both from faculty and student, is required to investigate the use of the portfolios in these students. May I also suggest that the authors research the subject a little more, especially regarding the use of portfolios as an assessment tool, which may steer them away from their last Take Home message