Motion Tracking System in Surgical Training

THE CHALLENGES IN HEALTHCARE AND SURGICAL TRAINING

The nature of surgical training is consistently evolving in the past decade along with continuous changes in the healthcare system worldwide. The modern healthcare system has been pressurized by the current law that involves a restricted number of working hours. The legal working hours per week can be as low as 48 hours in the European countries and 80 hours in North America. These working mandates are deemed necessary to guard against human errors that may be related to stress and fatigue in a high‐pressured working environment. In addition, there is also an increasing popularity in reporting medico‐legal cases in the current media. High profile medical reports such as the Kennedy Report (UK) and the Institute of Medicine (US) report ‘To Err is Human’ have highlighted surgical errors that turned the spotlight immediately on the adequacy of surgical training and, by extension, the quality of surgical trainees. These current changes in healthcare system could be continued to cause negative impact upon the surgical training of many aspiring surgeons.

Historically, surgical training has been based on the apprenticeship model throughout many years. The trainee surgeons are taught on how to perform procedures by senior surgeons with on‐the‐job training. Therefore, the training is opportunistic and the trainees were expected to demonstrate their skills in the operating theatre under supervision of their consultants. This was coined by William Halstead who exemplified the training approach as ‘see one, do one and teach one’. The traditional teaching method is largely relying upon variable cases that the trainees encounter during their daily work routine. Typically, junior doctors learn from their seniors and more experienced colleagues and their consultants. The skill level of consultants is perceived as the proficiency level and therefore, the desired precision in surgical training. The trainees are expected to reach the proficiency level that would allow them to perform surgical procedures in the real operating theatre. However, it is a challenge to assess surgical skills and obtain an objective proficiency level.

This training model is less favourable in the current climate of healthcare system. Due to the restriction, the trainees have limited opportunities to gain competencies, and therefore the training period is prolonged. As a direct consequence of these challenges, interest in laboratories with formal curricula, specifically designed to teach surgical skills, has increased dramatically. The attention has been shifted towards training in a simulation lab using inanimate bench models, animals (cadaveric or live), hybrid or virtual reality (VR) simulators. In United Kingdom, the use of live animals is not permitted under the current law, unlike in Europe, United States and other countries.

Therefore, simulation technology has gained its popularity among surgical training institution worldwide. With the advancement of laparoscopic and minimally invasive surgery (MIS) and steep learning curve in this specialty, a burst of simulators became available in the market for over a decade ago. Some examples of validated virtual reality (VR) simulators available in laparoscopy are MIST VR, LapSim, LapMentor and Xitact LS500. The trainees are able to practice their skills in hand‐eye co‐ordination, intracorporeal suturing and procedures such as laparoscopic cholecystectomy and appendicectomy by using these simulators. In general, the laparoscopic instruments used are fitted with sensors that allow the cameras to track their movement. The simulator then displays a two‐dimensional graphic of an operative field such as the internal organs on a computer screen. From this, the simulators are able to track and quantify the movement that would be converted into meaningful metrics such as path length, smoothness and economy of movement. These metrics provide an objective automated measurement of technical skill proficiency instantaneously.

The advancement of simulation technology has allowed training bodies such as The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) to develop training models for both laparoscopic and endoscopic skills. In the recent years, surgical training institutions have been incorporating simulators in surgical skills assessment and trainees’ selection process. We found that only 56% of the studies in the literature employed simulator‐generated objective metrics in the laparoscopic skills assessment either exclusively or combined with other assessment tools, unlike other industries, such as the aviation industry, which are more advanced in the simulation technology.

The MIS and laparoscopic surgery is only a branch of surgical specialties. In general, the progression of simulator development has tended to target minimally invasive surgery (MIS). However, open surgery remains the foundation of all surgical specialties. The surgical trainees are expected to master technical skills in open surgery before they are allowed to progress to more complex surgical procedures such as MIS and microsurgery. Examples of technical skills in open surgery include hand knot tying, suturing skill, repair of nerve or tendon and open hernia repair. The surgeon’s ability to tie knots securely is of paramount importance as loosening of surgical knots, during or after tying, can compromise the outcome of a surgical procedure.

Despite its importance, the training of open surgical techniques is largely depending on inanimate bench models. The trainees would practice surgical skills on bench models such as skin pads and saphenofemoral junction model from Limbs and Things™ (Bristol, United Kingdom) and laparotomy model from Simulab Corporation (Seattle, WA). This is in contrast with MIS or laparoscopic surgery simulators. Typically, in order to assess their competency in this skill, a trainee will perform a specific procedure such as excision of sebaceous cyst using an inanimate model and an observer who has extensive experience in the field such as a consultant or a senior registrar will watch the trainees and assess their skills using observer‐dependent assessment tools. This can be done either by face‐to‐face or video recording.

The classic observational assessment tool for open surgical skills is the objective structured assessment of technical skills (OSATS) (Figures 1 and 2). It was coined by Professor Reznick and his research team in Canada. It is based on observation and completing two sets of checklists. The first checklist consists of important steps in a specific procedure and trainees are assessed whether they have taken all these steps or not. The second checklist is the global rating scale (GRS) which examines the global performance of the trainees by using five‐point Likert scale. It assesses the fluidity and efficiency of movement during completion of a surgical task.

FIGURE 1.

A sample of task‐specific checklist from OSATS.

FIGURE 2.

The global rating scale (GRS) of operative performance.

The observational assessment tool requires the recruitment of expert surgeons to assess trainees. This proves to be labour‐intensive and time‐consuming. One would argue that there could be human bias or favouritism when scoring trainees using this type of assessment. Data in several studies suggested that unblinded raters give higher scores than blinded raters (as would be expected if knowledge of a learner subconsciously influences a rater’s behaviour). Therefore, surgical training is moving away from the observer‐dependant assessment tools but towards more objective and quantifiable analysis of the technical skills. This would allow the assessment of the trainees’ skill level and measure their reached precision according to their corresponding training years.

OPEN SURGICAL SKILLS

Open surgical skills are fundamental in surgery. The skills involve hand dexterity using surgical instruments. Thomas Morstede stated more than 500 years ago that surgeons should ‘be dextrous, have steady untrembling hands, and clear sight’. A good surgeon is perceived as having a greater economy and precision of hand and instrument movement.

Open surgical skills vary from simple technique, such as hand knot tying and suturing, to more complex procedures, such as tendon or nerve repair, laparotomy and vessel anastomosis. All of the surgical trainees are required to master the open basic surgical skills, particularly in suturing and hand knot tying skill. A good technique would ensure that the wound edges are approximated neatly without causing any gaping if the sutures are loose or skin necrosis if the sutures are too tight. The trainees would hone their skills by practising on bench models as shown in Figures 3 and 4.

FIGURE 3.

A standard surgical knot tying task performed using the knot tying training jig from Limbs and Things™ (Bristol, UK).

FIGURE 4.

A trainee performing a simple interrupted suturing task using skin pads with simulated wound edges.

The movement of the hands and fingers has to be precise and economical to ensure that the procedure runs smoothly with minimum complication. However, the assessment of dexterity, smoothness and economy of hand movement using surgical instruments has been subjective and several attempts have been made to quantify dexterity, but many of these are unsatisfactory. Many have associated dexterity with the time taken to complete a surgical task. It is a crude assessment and it is a poor measurement of technical skills. Although operative speed is a desirable surgical quality to lower the time spent under anaesthesia, it fails to assess the quality of surgical performance.

The objective assessment of open surgical skills is slow to evolve, unlike MIS and laparoscopic surgical training. We require an assessment tool that could quantify the hand motion and provide an objective scale on the performance when completing a surgical task. Therefore, we explored the potential use of motion analysis in assessing open surgical skills. It would be a non‐observational assessment tool that is automated and objective.

Materials and methods

PARTICIPANTS DEMOGRAPHICS

All medical students in pre‐clinical years (Years 1–3) from the Royal College of Surgeons in Ireland (RCSI), basic surgical trainees (Years 1 and 2) and consultant surgeons were invited to participate in our study. This allowed us to divide the participants into three different subject groups: novice, trainees and experts. It was made clear that the participation is voluntary. Ethical approval was granted by Research Ethics Committee of RCSI.

Figure 5 showed the demographics of participants in this study.

FIGURE 5.

Participants demographics.

BASIC SURGICAL SKILLS ASSESSMENT

Inanimate bench models are used in this study. The bench models were from Limbs and Things™ (Bristol, UK) which include the knot tying trainer jig, skin pads, skin pad jig and cyst pads. All participants were required to perform two fundamental tasks. Below are the tasks involved and their description.

Task 1: One‐handed knot tying skill

The participants were required to perform surgical knots using the one‐handed technique. They were given a standardised length of 2/0 Mersilk (Ethicon) suture tie. The surgical knots were performed on the knot tying trainer jig.

Task 2: Suturing simple interrupted technique

The participants were required to perform simple interrupted sutures on a simulated wound. This task was performed on skin pads (Limbs and Things, Bristol, UK). A 3/0 Mersilk suture (Ethicon) and surgical instruments were provided.

The participants’ performances were assessed using observational tool (GRS) and non‐observational tool (motion tracking device). The data allowed us to analyse the validity of motion tracking device as an assessment tool in comparison with the well‐established GRS scoring system. During the experiment, videos were recorded in anonymous fashion. Each video was labelled by a random code generator so that the assessor could not identify the level of experience of each participant. The participants also had a sensor attached to their right index finger to track hand motion and this will be discussed in detail in the next section.

As for observational assessment, two assessors were selected to assess each video using the GRS. The assessors were expert surgeons with greater than 10 years of consultant experience and are involved in teaching and educating surgical trainees in Ireland. The experiment process is outlined in Figure 6.

FIGURE 6.

Flow diagram of the experiment process.

Motion analysis in surgical skill assessment

THE ROLE OF MOTION ANALYSIS

Surgical specialties have initiated a trend towards a more objective and quantifiable measure of technical skill proficiency. In minimally invasive surgery (laparoscopy and endoscopy), simulators have been developed with the ability to quantify the associated skills with specific metrics including total path length, movement efficiency and smoothness. Motion smoothness in handling surgical tools is an essential skill that surgical residents must acquire before independently operating on patients.

The use of motion analysis has been pioneered in gait analysis. It is used in tracking the movement of body parts. These methods usually make use of markers located on body articulations to garner movement information from a particular limb. Its application is evident in various areas including sports such as golf, training an apprentice in spray painting and also in diagnostic simulators such as ultrasound simulation. Undoubtedly, one of the most promising technological tools in medical training are the simulators for the acquisition of clinical skills using motion sensors. The surgical arena has used this technology to try and quantify surgical performance. Motion analysis allows assessment of surgical dexterity using parameters that are extracted from movement of the hands or laparoscopic instruments. The motion analysis provides parameters that measure the precision of hand motion when performing surgical skills. Hence, surgical competencies, particularly in surgical trainees, can be ascertained by using these parameters.

Lord Ara Darzi and his researchers pioneered the use of an electromagnetic motion tracking device in surgery, called the Imperial College Surgical Assessment Device (ICSAD). This is the combination of a commercially available electromagnetic tracking system (Isotrak, Polhemus Inc, Colchester, VT) and a bespoke computer software program. This motion analysis device uses an alternating current electromagnetic system with passive receiver attached to the dorsum of the hand over the mid‐shaft of the third metacarpal. It measures the time taken, the number of movements and the path length. All of these metrics have been shown to change with experience in laparoscopic surgery and in open surgery (bowel anastomosis and vein patch insertion).

We used a commercially available motion tracking device called The Patriot™ from Polhemus Inc., Colchester, VT. This device utilises electromagnetic technology and tracks 6 degrees of freedom (6DOF) measurements of the sensor’s movement. In our study, we attached the sensor on to the participants’ right index finger. Figure 7 showed the airplane image that indicates the sensor. It will move to the position and orientation of the right index finger. The retrieved position and orientation are displayed as numbers in six columns (upper part of screenshot), from left to right, positions in X‐, Y‐ and Z‐axis and orientation in yaw, pitch and roll. The Patriot™ collects these raw data which in turn convert to a set of meaningful metrics using our bespoke software.

FIGURE 7.

A screenshot of the PiMgr software.

CONSTRUCT VALIDITY OF MOTION ANALYSIS IN SURGICAL SKILLS ASSESSMENT

Every evaluative tool needs to provide invaluable information on what it measures or examines and that the conclusions drawn from the tool are dependable. A validated assessment device should be able to differentiate level of surgical skills according to the level of competency and this is classified as construct validity. One inference of construct validity is the extent to which a test discriminates between various levels of expertise. Mason et al. have reviewed the published evidence as it relates to motion analysis and the assessment of surgical performance. This systematic review reported construct validity of ICSAD and other forms of motion analysis devices such as ProMIS augmented reality simulator and Hiroshima University Endoscopic Surgical Assessment Device (HUESAD) in assessing laparoscopic skills.

Our research further assessed the use of a novel electromagnetic tracking system in basic surgical skill tasks by using our own in‐house computer software with a finger sensor. Figures 8 and 9 showed the standard set up for knot tying task and suturing task, respectively, with the Patriot™ motion tracking device. Our in‐house software was designed to generate the classic metrics that are time and total path length (TPL). In addition, new metrics were developed: average deviation distance from X‐, Y‐ and Z‐axis and average distance from centre of the bench model. The centre of the bench model is labelled as a point of interest (POI), as we believe that hand motion is most efficient when the hands are at certain distance away from the centre of the workstation. Subjectively, when performing a certain task in open surgery, such as tying surgical knots or suturing, a novice would have unnecessary movement of their hands which include moving hands further away from the field of surgery. This is thought to be inefficient in view of the economy of the hand movement.

FIGURE 8.

Knot tying model with the Patriot™ motion tracking device.

FIGURE 9.

Simple interrupted suturing model and instruments with the Patriot™ motion tracking device.

Our results demonstrated construct validity for both fundamental skills which were one‐handed knot tying task (Figure 10) and the simple interrupted suturing skill (Figure 11) for the metrics of time, total path length, point of interest and deviation from the Z‐axis.

FIGURE 10.

Distribution of the time taken (a) and total path length (b), average distance from the POI (c) and Z‐axis (d) between the three subject groups in completing one‐handed knot tying skill.

FIGURE 11.

Distribution of the time taken (a), total path length (b), average distance from the POI (c) and Z‐axis (d) between the three subject groups in completing simple suturing task.

The box and whiskers plot shows a significant difference between experts, trainees and novices (p < 0.001). This was analysed using Kruskal Wallis statistical test. The horizontal lines within boxes are the median. The boxes and whiskers represent interquartile range and range, respectively. The dot represents outlier.

The novel parameters were able to differentiate subjects according to level of experience along with the validated metrics as reported in literature. This implies that a surgical novice moved his or her hand further away from the virtual Z‐axis and mid‐point of the workstation than experts or surgical trainees, as seen subjectively in the video recordings. Therefore, it is postulated that this pattern of movement is less efficient. The lack of significant change in X‐ and Y‐axis may reflect the standard suture tie length used in this experiment. This limits the movement of the hand in these axes.

CONCURRENT VALIDITY OF MOTION ANALYSIS IN SURGICAL SKILLS ASSESSMENT

A further validation of the motion analysis device was required to prove that it is a robust assessment tool. It is important that the metrics from motion analysis have a good correlation with the gold standard assessment tool, which is the global rating scale (GRS), as mentioned previously. This would prove the concurrent validity of this novel device.

Datta et al. revealed that there was a strong correlation between number of hand movements analysed using the ICSAD and the GRS in suturing vein patch on an inanimate model (Spearman coefficient of −0.587, p < 0.01). In another study by Ezra et al., concurrent validity was demonstrated between these two assessment tools in microsurgery suturing task. The metrics used in this study were path length, hand movements and time.

In our chapter, for the one‐handed knot tying skill, our results demonstrated a significant correlation between all the metrics generated by the Patriot™ motion tracking device and the items of the GRS scoring tool. The only parameter that failed to demonstrate a significant relationship was deviation from the x‐axis and ‘respect for tissue’. For the simple interrupted suturing skill, we found a significant correlation between time, total path length and deviation from the z‐axis and the total GRS score.

However, the metrics from Patriot™ motion tracking system failed to show a more convincing correlation with the scale assessing tissue handling. This may be explained by the fact that the ‘respect for tissue’ component on the GRS is a very subjective parameter. This is reflected in the poor inter‐rater reliability of the GRS scoring system. Apart from this, the metrics correlated well with the GRS items especially items involving motion and flow of operation. We could safely suggest that the Patriot provides more objective score than the observer‐dependant scale.

APPLICATION OF MOTION ANALYSIS IN SURGICAL TRAINING

The use of motion tracking and analysis in assessing surgical skills has been described mainly in laparoscopic skills. There is a lack of literature that describes the integration of such technology in surgical training curricula across the globe, despite a myriad of validation studies.

The surgical trainees learn fundamental basic skills at an early stage. Open basic skills remains to be the principal skills across all surgical specialties. Therefore, any aspiring surgeons are expected to be proficient in these skills before they can proceed to perform simple procedures such as excision of skin or subcutaneous lesion or more complex procedures such as repair of tendon or nerve. The trainees would require direct guidance and abundance of practice in order to be proficient in these skills, as the saying goes ‘practice makes perfect’. By having an expert or supervisor to observe them consistently during practice session is not feasible when clinical work takes priority. Therefore, motion analysis system would be necessary to provide an automated system that allows the trainees to practice and record their performance in their own time.

Proficiency‐based training has been described as learning environments in which the trainee progresses from less to more technically demanding skills and tasks only after achieving predefined criteria. One widely available simulation‐based assessment and certification program is the fundamentals of laparoscopic surgery (FLS) developed by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and now administered by SAGES and the American College of Surgeons. The FLS program incorporates tasks from the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) program, including laparoscopic suturing, and uses well‐described, low‐fidelity inanimate models [39]. The proficiency scores were determined by a group of experts in the skills and the trainees or users are required to reach these predetermined scores before they could proceed to the next level or task. These proficiency scores act as an aim for the trainees to achieve and subsequently motivate them to keep practising until a high standard of surgical skills is accomplished. In order to do this, an automated objective measurement is much desirable, as it does not require any expert surgeons or observers to monitor and assess the performance.

We applied the concept of proficiency‐based training by using the validated metrics from the Patriot™ motion tracking system. We determined the proficiency goals or desired precision for each of these metrics in knot tying and suturing skills. This was achieved by gathering the experts’ scores from the motion analysis and calculating the proficiency target as follows:

The performance of surgical trainees in Years 1 and 2 of the surgical training programme was assessed using the Patriot™ device. Their scores were then analysed against these predetermined proficiency goals. Our intention was to have

objective automated tool that can be integrated into the national training curricula as part of the training module. This will help the trainees to practise and eventually achieve the desired precision or performance in the most fundamental skills in surgery.

Figure 12 showed a sample of trainees’ performance in suturing which was mapped out against the proficiency target. The dashed line represents the proficiency level of 143.6 mm. The diamond shape points below the dashed line are the trainees who have shorter path length and considered as proficient in their skill (n = 13, 52%). The round shape points above the dashed line are the trainees with longer path length and did not reach proficiency level (n = 22.48%).The performance graph is very useful when there is a group of surgical trainees assessing their suturing technique and they are able to compare their reached precision with the desired precision.

FIGURE 12.

The total path length (TPL) of all the trainees in the study who performed simple interrupted suturing skill.

The main advantage of motion analysis system in surgical training is that it is capable of producing automated objective scoring system and does not require a group of observers to assess the performance in any particular surgical skills. In our study, the Patriot™ motion analysis system has shown a promising potential in a learner‐oriented proficiency curriculum. By providing an objective and numerical rating, trainees could benchmark and aim to improve their score through enhancement of surgical skill. As surgical educators, this assessment tool is useful in identifying any surgical trainees who are underperform according to the proficiency standard at an early stage of their training years. A remedial session can be offered to these surgical trainees and their training module can be customized for them in order to be able to reach proficiency as required. The motion analysis system can be used continuously by the trainees during practice session and also in any departmental assessment settings.

The learning curve in surgical skills is steep. The trainees are required to improve their skills or reached precision over time and progress in their surgical training. They are expected to practice the skills, preferably in the simulation lab until they achieve the desired precision. The training programmes are designed to teach the trainees skills that are appropriate to their levels. The early part of the learning curve is associated with a higher complication rate. The improvement in their reached precision in the simulation lab will allow them to perform procedures on patients in real operating room with confidence. They will also progress to a more complex procedure such as tendon repair, vessel anastomosis and bowel resection.

Motion analysis provides an objective measurement of the skills that can be used to map out the learning curve. In order to reach proficiency in the learning curve, using time only as a metric is not reliable. It measures how fast someone completes a task. This does not include how efficient it was performed. Therefore, it is regarded as an adjuvant tool to assess surgical technical skills due to its unique properties including non‐observer dependent, automated and feasibility.

LIMITATION OF MOTION ANALYSIS AND FUTURE RESEARCH

The main limitation of motion analysis is that its inability to detect surgical errors. Hand‐tracking data appear to confirm that skilled individuals demonstrate a shorter path length, make fewer movements and take less time to perform an operation, but with the caveat that this improved performance is not accompanied by an increase in error. In minimally invasive surgical training such as laparoscopic skills, the technology in VR simulators such as LapSim and LapMentor is more advanced than open surgical skills training. These simulators are programmed to identify any surgical errors as well as analysing the movement of the instruments.

Therefore, this vital limitation of motion analysis may be overcome by incorporating an assessment of the end‐product following a completion of surgical task. For instance, the quality of the surgical knots can be assessed by a force gauge device in order to ensure that the knots do not slip under certain tension. It is important that the surgical knots are secure as knot slippage in a real operating setting can cause catastrophic bleeding which leads to morbidity towards patients.

This shortcoming highlights that the surgical competency is multimodal and there is no single solution for surgical assessment. We propose that surgical educators should incorporate motion analysis and assessment of the end‐product quality when assessing surgical techniques. Further research should be focused on creating an all‐in‐one package in assessing surgical competency that would be objective, automated and most importantly independent from any observers.

Another limitation of motion analysis is that its use in the real operating setting. All the studies in the literature showed the use of motion analysis system in a simulation lab. The fundamental assumption of simulation‐based training is that the skills acquired in simulated settings are directly transferable to the operative setting. The current motion tracking devices that are readily available use electromagnetic field to track sensors on the hands. These devices are sensitive to surrounding metal objects such as electronic machines, metal bars or large electrical cables in the walls that can cause erratic reading. These metal objects are certainly present in all real operating theatres in the hospitals. In addition, the sensors on the devices are attached via cables, which potentially could interfere with the sterility of the operating field. Due to these limitations, it is not feasible to utilize these devices in assessing surgical skills in a real operating theatre. Therefore, a new invention of a system that is wireless and not susceptible to the surrounding metal objects is much desired.

Summary

Open surgical skill training requires an assessment tool that is independent, automated and objective. The validity of motion analysis in assessing fundamental surgical skills has been proven and showed positive results. It has demonstrated its potential use in a proficiency‐based training as a step away from the traditional method of surgical training. The future of simulation‐based surgical training in open surgical skills appears promising and it will finally shape the pathway towards creating top quality surgeons in the current climate of healthcare system.