Achieving Competency in Surgical Training: An analysis of Procedural Learning Curves and the RACS Trainee Experience
Authors List Andrew, D., Department of ORL, Christchurch Hospital, Canterbury District Health Board, Christchurch, New Zealand
MacArthur, F., Department of ORL, Auckland City Hospital, Auckland District Health Board, Auckland, New Zealand
Joice, P., Department of ORL, Dunedin Hospital, Southern District Health Board, Dunedin, New Zealand
Introduction: Otorhinolaryngology (ORL) training within the Royal Australasian College of Surgeons (RACS) has recently moved to a competency-based system. However, there is still debate surrounding how competency can be objectively measured through the technical aspects of training, and a reliable model is yet to be established. Learning Curve Cumulative Summation (LC-CUSUM) modelling has been used in literature to assess when technicians reach a predetermined level of competency, enabling immediate feedback on individual progress, and allowing early identification of those struggling1. So far this has not been applied to ORL procedures.
Aim: To determine the validity of LC-CUSUM modelling in measuring the competency of trainees performing myringoplasties throughout training.
Method: Using the mandatory myringoplasty audit data collected by New Zealand ORL trainees, an LC-CUSUM model was applied to assess competency. In keeping with international guidelines, a success rate of 80% was selected and the LC-CUSUM algorithm accordingly assigned a score of 0.29 for a successful operation and -0.69 for failure, with the competency threshold calculated at 2.24. As trainees followed up their patients, these values were then plotted sequentially representing the trainees learning curve for myringoplasties throughout training.
Results: Ten trainees’ data have been analysed, with 439 surgeries collectively and an average success rate of 84.9% +/- 2.6% (95% CI) per trainee. The mean number of surgeries to achieve competency, based on the LC-CUSUM model, was 22 +/- 8 (95% CI), a point by which seven of the ten trainees were deemed competent. All trainees reached competency by the end of SET 3.
Conclusion: The LC-CUSUM is a valid competency-based model for assessing surgical training and providing feedback to trainees. This may have a role in assessing further technical aspects throughout training
1Ward ST, et al. An analysis of the learning curve to achieve competency at colonoscopy. Gut 2014;63:1746–1754
MacArthur, F., Department of ORL, Auckland City Hospital, Auckland District Health Board, Auckland, New Zealand
Joice, P., Department of ORL, Dunedin Hospital, Southern District Health Board, Dunedin, New Zealand
Introduction: Otorhinolaryngology (ORL) training within the Royal Australasian College of Surgeons (RACS) has recently moved to a competency-based system. However, there is still debate surrounding how competency can be objectively measured through the technical aspects of training, and a reliable model is yet to be established. Learning Curve Cumulative Summation (LC-CUSUM) modelling has been used in literature to assess when technicians reach a predetermined level of competency, enabling immediate feedback on individual progress, and allowing early identification of those struggling1. So far this has not been applied to ORL procedures.
Aim: To determine the validity of LC-CUSUM modelling in measuring the competency of trainees performing myringoplasties throughout training.
Method: Using the mandatory myringoplasty audit data collected by New Zealand ORL trainees, an LC-CUSUM model was applied to assess competency. In keeping with international guidelines, a success rate of 80% was selected and the LC-CUSUM algorithm accordingly assigned a score of 0.29 for a successful operation and -0.69 for failure, with the competency threshold calculated at 2.24. As trainees followed up their patients, these values were then plotted sequentially representing the trainees learning curve for myringoplasties throughout training.
Results: Ten trainees’ data have been analysed, with 439 surgeries collectively and an average success rate of 84.9% +/- 2.6% (95% CI) per trainee. The mean number of surgeries to achieve competency, based on the LC-CUSUM model, was 22 +/- 8 (95% CI), a point by which seven of the ten trainees were deemed competent. All trainees reached competency by the end of SET 3.
Conclusion: The LC-CUSUM is a valid competency-based model for assessing surgical training and providing feedback to trainees. This may have a role in assessing further technical aspects throughout training
1Ward ST, et al. An analysis of the learning curve to achieve competency at colonoscopy. Gut 2014;63:1746–1754
