Targeted Muscle Reinnervation for Real-time Myoelectric Control of Multifunction Artificial Arms FREE

The loss of one or both arms is a major disability that profoundly limits the everyday capabilities and interactions of individuals with upper-limb amputation. Currently available prostheses do not adequately restore the function of an individual's arm and hand. The most commonly used prostheses are body-powered. These devices capture remaining shoulder motion with a harness and transfer this movement through a cable to operate the hand, wrist, or elbow. With this control method, only 1 joint can be operated at a time. Myoelectric prostheses use the electromyogram (EMG) signals (the electrical signals generated during muscle contraction) from residual limb muscles to control motorized arm joints.

Current control strategies use the amplitudes of the EMG signals from 1 or 2 remaining muscles to sequentially operate each function in the prosthesis.¹ For example, the biceps and triceps muscles are used by an individual with transhumeral amputation to control the elbow, wrist, and hand. The user must trigger a “mode switch” to sequentially select which of these devices is to be actuated. This type of operation is not intuitive, because the residual muscles control physiologically unrelated movements. The use of currently available arm prostheses is cumbersome and slow for individuals with transhumeral or shoulder-disarticulation amputations, whose disability is most severe.^2- 8

We have developed a new surgical technique, called targeted muscle reinnervation (TMR), to improve control of myoelectric prostheses.^9- 14 With TMR, remaining arm nerves are transferred to residual chest or upper-arm muscles that are no longer biomechanically functional due to loss of the limb (Figure 1). Once reinnervated, these muscles serve as biological amplifiers of motor commands from the transferred arm nerves and provide physiologically appropriate EMG signals for control of the elbow, wrist, and hand. The TMR technique has been successfully performed in patients with transhumeral and shoulder-disarticulation amputations and has markedly improved their functional use of prostheses.^10,12- 13 Using a simple control method based only on the amplitude of EMG signals from reinnervated muscles, patients who have undergone TMR surgery can intuitively and simultaneously control opening and closing of the hand as well as extension and flexion of the elbow.

Further investigation has shown that TMR provides a rich source of motor control information. Electrode arrays were used to record EMG signals as patients who had undergone TMR surgery attempted 16 different motions involving the elbow, wrist, thumb, and fingers. The patterns produced by the combined EMG signals during the performance of different movements were used by a computer to create a “classifier.” The classifier was then used to decipher which motion was being performed based on the current pattern of EMG signals. This strategy is called “pattern recognition.” Analysis of the data revealed that the intended motions could be classified with a mean classification accuracy of 95%.^15- 16 However, it is unknown whether reinnervated muscles can stably and accurately provide myoelectric signals for real-time control of multifunction prostheses. Real-time performance metrics are required to examine the clinical robustness and accuracy of myoelectric prosthetic control with TMR.

This study assessed the real-time control of multifunction prostheses based on TMR combined with a pattern-recognition algorithm. Performance metrics (motion selection time, motion completion time, and motion completion rate) were quantified by training and testing with a virtual multifunction prosthesis. This study also provided a qualitative assessment of ability to operate advanced experimental upper-limb prostheses.

This study was conducted with 5 patients who had undergone TMR surgery 11 to 70 months prior to testing. For comparison, 5 control participants without amputation were also included. This study was approved by the Northwestern University institutional review board and conducted between January 2007 and January 2008 at the Rehabilitation Institute of Chicago. All participants provided written informed consent and provided permissions for publication of videos and photographs for scientific and educational purposes.

Five of the 6 individuals with shoulder-disarticulation or transhumeral amputation who had undergone TMR surgery in collaboration with the Rehabilitation Institute of Chicago agreed to participate in this study. Three of these participants had shoulder-disarticulation amputations. Patient S1 was a 54-year-old man who underwent bilateral shoulder-disarticulation amputations in May 2001 following high-voltage electrical injuries to both arms. During TMR surgery performed in February 2002, his residual musculocutaneous, median, radial, and ulnar nerves were transferred to the pectoralis major and pectoralis minor muscles (Figure 1B).^10- 11,17 Patient S2 was a 24-year-old woman who underwent left shoulder-disarticulation (very short residual humerus) amputation in May 2005 following a motor vehicle collision. During TMR surgery performed in August 2005, the musculocutaneous, median, radial, and ulnar nerves were transferred to portions of the pectoralis major and serratus anterior muscles.¹³ Patient S3 was a 37-year-old man who underwent right shoulder-disarticulation amputation in February 2005 following electrical burns. During TMR surgery, performed in July 2006, the musculocutaneous, median, radial, and ulnar nerves were transferred to the pectoralis major, pectoralis minor, and latissimus muscles.

Two patients with transhumeral amputations also participated in the study. Patient T4 was a 50-year-old man who underwent right transhumeral amputation in April 2004 following a motor vehicle collision. During TMR surgery performed in January 2005, the median nerve was transferred to the medial biceps, and the distal radial nerve was transferred to the brachialis muscle.¹⁴ Patient T5 was a 38-year-old woman who underwent left transhumeral amputation in April 2006, also following a motor vehicle collision. During TMR surgery performed in October 2006, the median nerve was transferred to the medial biceps, and the distal radial nerve was transferred to the lateral triceps (Figure 1B).

For comparison, 5 healthy control participants without amputation (3 men and 2 women, aged 20 to 45 years) also participated in the study. These control participants were chosen to represent both sexes and had an age range similar to that of the TMR patients.

For each patient who underwent TMR surgery, 12 self-adhesive bipolar EMG electrodes were placed on the skin over the reinnervated muscles. Four electrodes were placed at sites chosen previously through clinical evaluation to control the patients' prostheses.^10- 12 The 8 additional sites were determined by an electrode-placement optimization algorithm,¹⁶ which sought to maximize the classification accuracy for different movements. For control participants, 12 electrodes were used to record EMG signals from physiologically appropriate muscles in the arm and hand. One electrode was placed over the biceps muscle and a second over the triceps muscle; 6 were placed around the proximal forearm; 1 was placed on the dorsal side of the wrist; and 3 were placed on the hand (medial and lateral thenar eminence and hypothenar eminence). The EMG signals were amplified and band-pass filtered from 5 to 400 Hz. Data were sampled at 1 kHz by an analog-to-digital converter (USB-1616FS; Measurement Computing Corp, Norton, Massachusetts) and processed in real time on a desktop computer using Matlab version 7.3.0.267 (The Mathworks, Natick, Massachusetts).

The 11 motion classes were elbow flexion, elbow extension, wrist flexion, wrist extension, wrist pronation, wrist supination, hand opening, 3 types of hand grasps, and no movement. Patients who had undergone TMR surgery were allowed to try 5 different hand-grasp patterns: 3-jaw chuck, fine pinch, key grip, power grip, and tool grip (Figure 2). Each patient chose 3 of these grips based on relative ease and reliability of control. For control participants, the 3 grasps were 3-jaw chuck, fine pinch, and tool grip; these were the 3 grasps most commonly chosen by the TMR patients.

Demonstrations of each movement were displayed in random order on a computer screen (Figure 2). For each movement, all participants were instructed to follow the demonstration of the movement and to perform the movement with a comfortable and consistent level of effort. The prompt was displayed with a countdown during the rest time between trials to give patients time to prepare.

Electromyogram data were collected in 8 consecutive trials. In each trial, each motion was repeated twice and held for 4 seconds, producing 8 seconds of EMG recordings per motion. There was a 3-second time interval between motions in the 4 even-numbered trials. A variable rest time of 0 to 3 seconds was used in the 4 odd-numbered trials in an attempt to keep the participants engaged and enhance the classifier's robustness. Electromyogram data from the 8 trials were split into 2 groups: the 4 odd-numbered trials were combined and used to create the classifier; the 4 even-numbered trials were combined and used to test the classifier.

The pattern-recognition algorithm used in this study was implemented as follows: EMG recordings were segmented into a series of 150-ms analysis windows with 50 ms of overlap, resulting in a new classification every 100 ms. Four time-domain features^3,15 were extracted from EMG signals in each analysis window. The combined features from the even-numbered trials were used to create a linear discriminant analysis classifier.^3,15- 16,18 This classifier was then used to classify the combined features from the testing set. The classification accuracy for each movement was the percentage of total analysis windows for that class that were correctly classified. The overall classification accuracy was the average of these values for all 11 movements. The linear discriminant analysis classifier was then used in real time to classify features extracted from real-time EMG signals, produce a new prediction of the motion class every 100 ms, and control a virtual-reality arm or a physical prosthesis, as described below. Computational time for each analysis window was less than 3 ms.

Experiments with a virtual prosthesis were performed immediately after classifier training. All participants were instructed to follow visual prompts for each movement, and a virtual arm that responded to the classifier output was displayed on the screen (Figure 2). Once the participants correctly selected the desired movement, they were asked to maintain it until the virtual arm completed the movement. The time of movement onset was identified as the time of the last “no movement” classification (Figure 3). Each of the 10 motions was randomly presented 3 times in a trial, and the trials were repeated 6 times for a total of 180 movements (72 hand-grasp motions and 108 elbow and wrist motions). These data were used to evaluate the speed and consistency of control using real-time pattern recognition.

The performance metrics used to assess virtual prosthesis control were motion selection time, motion completion time, and motion completion (“success”) rate. The motion selection time was the time taken to correctly select a target motion and was defined as the time from movement onset to the first correct classification (Figure 3). This quantity measures how quickly motor commands can be translated into correct motion predictions.

The motion completion time was defined as the time from movement onset to the 10th correct classification (which represented the full range of motion for any movement) (Figure 3). The fastest possible speed to complete any motion was 1 second, corresponding to 10 consecutive correct classifications, with new classifications occurring every 100 ms. If the correct class was not selected within a 5-second time limit, the movement was considered a failure.

The motion completion rate was the percentage of successfully completed motions out of the total attempted motions (72 attempted motions for the hand, 108 attempted motions for the elbow and wrist) within the time limit. Because the motion selection and motion completion data for each participant were highly skewed, the median value for all 6 arm movements (elbow and wrist) and all 4 hand movements (hand open and 3 hand grasps) were calculated for each participant, and these values were averaged across the 5 patients who had undergone TMR and the 5 control participants.

Preliminary research demonstrated that hand-grasp patterns were more difficult to perform than elbow and wrist movements. Therefore, the control scheme for hand grasps was modified. A hand grasp could only be selected when the hand was fully open. Once a grasp was selected, any hand-grasp pattern would close the hand in the initially selected grasp. However, if the initial hand-grasp pattern selected was incorrect, the patient would have to fully open the hand and try again.

Three of the patients who had undergone TMR surgery were able to test advanced upper arm prosthesis prototypes developed under the Defense Advanced Research Project Agency's Revolutionizing Prosthetics program. Video of this initial testing is presented in the Multimedia feature.

The Johns Hopkins University Applied Physics Laboratory and collaborators developed a prosthetic arm with 7 degrees of freedom that was tested with patient S1 in January 2007. Patient S1 controlled flexion and extension of the motorized shoulder by using residual shoulder motion to operate a mechanical rocker switch. A motorized humeral rotator was controlled with EMG signals from the residual deltoid and latissimus dorsi muscles. Powered elbow flexion/extension, wrist pronation/supination, wrist flexion/extension, and a hand that allowed 3-jaw chuck and lateral pinch grip were controlled with EMG signals from reinnervated muscles and the pattern recognition algorithm.

DEKA Integrated Solutions Corporation and collaborators developed a prosthetic arm system with 10 degrees of freedom that was tested with patients S1, S2, and T5 in May, June, and July 2007, respectively. A shoulder controller operated with residual shoulder movement allowed patients with shoulder-disarticulation amputation to simultaneously operate shoulder flexion/extension and abduction/adduction. Humeral rotation was controlled with EMG signals from the latissimus dorsi and deltoid muscles. The powered elbow, wrist, and hand were controlled with pattern recognition of EMG signals recorded over reinnervated muscles. For patient T5, the humeral rotator was controlled with a switch, while the elbow, wrist, and hand were controlled with pattern recognition of EMG signals recorded over reinnervated muscles. The DEKA hand had multiple motors and was able to form a variety of hand-grasp patterns, including those shown in Figure 2.

Surface electrodes were either self-adhesive or built into the patients' prosthetic sockets. The arm systems were trained at the beginning of each session with a short pattern-recognition protocol similar to the one described above. Training and testing with the prostheses occurred over a 2-week period for each patient. Sessions generally lasted 2 to 3 hours, with one session in the morning and one in the afternoon.

The mean classification accuracy was 88% (SD, 7%) for patients who had undergone TMR surgery and 97% (SD, 2%) for control participants.

The majority of movements were selected quickly, with motion selection times less than 0.3 seconds (Figure 4). The mean motion selection times for elbow and wrist movements (elbow flexion/extension, wrist rotation, and wrist flexion/extension) were 0.22 seconds (SD, 0.06) for TMR patients and 0.16 seconds (SD, 0.03) for control participants (Table). The mean motion completion rate for elbow and wrist movements was high (96.3% [SD, 3.8%] for TMR patients and 100% [SD, 0%] for control participants). For TMR patients, the selection of the appropriate hand-grasp patterns took longer and had, on average, a 9.4% lower success rate than wrist and elbow movements (Table). For control participants, the mean motion selection time for hand grasps was similar to that of elbow and wrist movements (Table). The motion completion rate for hand grasps was slightly (3.3%) lower (Table).

The movements performed by TMR patients as well as control participants were also completed quickly, consistent with the high classification rates (Figure 4). The fastest possible motion completion time was 1 second, representing perfect classification of the intended movement. The mean motion completion times for elbow and wrist movements were 1.29 seconds (SD, 0.15) for TMR patients and 1.08 seconds (SD, 0.05) for control participants. For both groups, hand grasps took longer to complete than arm movements; the mean motion completion times for hand grasps were 1.54 seconds (SD, 0.27) for TMR patients and 1.26 seconds (SD, 0.18) for control participants.

The mean motion completion rates within a 3-second time limit were 94.1% (of 108 elbow and wrist movements) and 80.3% (of 72 hand grasps) for TMR patients and 99.6% (of 108 elbow and wrist movements) and 93.6% (of 72 hand grasps) for control participants (Figure 5). The motion completion rate for hand grasps was lower for 2 of the 5 TMR patients. The motion completion rate increased as the time limit was increased up to approximately 2 seconds and then began to plateau; the maximum motion completion rate was generally reached by 6 seconds.

Building on the virtual-arm training experience, 3 patients who had undergone TMR surgery were able to demonstrate control of physical arm systems (Multimedia feature). All 3 patients were able to perform basic operations using pattern-recognition control on the first day of testing. Over a 2-week trial period, their proficiency improved with practice, systems debugging, and minor systems improvements.

The patients were able to operate all functions of the prosthetic arm prototypes. Control of arm and hand movements using pattern-recognition of EMG signals from reinnervated muscles provided the ability for intuitive, sequential control of the elbow, wrist, and hand. The patients with shoulder-disarticulation amputation were able to simultaneously operate the shoulder and arm. Patients generally performed 1 motion at a time and would occasionally operate 2 joints simultaneously for reaching tasks.

The joints on these prostheses were capable of relatively high speeds. The speed range was customized to each patient, because the patients preferred to operate the arms at slower speeds to allow more accurate control. This control is demonstrated by patient S1 catching checkers rolling across a table, patient S2 stirring a spoon in a cup, and patient T5 moving small blocks (Multimedia feature).

The powered-shoulder systems markedly increased the work space of the prostheses. Motorized shoulder flexion allowed the patients with shoulder-disarticulation amputation to reach above their heads. Motorized shoulder flexion also allowed these patients to have a deeper work space, as demonstrated by patient S2 performing reaching motions over a table (Figure 6). Humeral rotation and shoulder abduction widened the work space for all patients and facilitated reaching to the midline for bimanual tasks. The powered wrist served mostly to preposition a functional wrist angle and facilitate better hand operation. The increased number of powered joints also allowed more precise orientation of the hand in space. For example, a ring could be moved across a geometric wire (Figure 6B).

Many different hand grasps were possible with the DEKA arm system; patients were able to attempt 3-jaw chuck, lateral pinch, fine pinch, power grasp, and tool grip. Patients varied in their abilities to control different grasps: one patient could reliably control 4 hand-grasp patterns, another could control up to 3, and the third could reliably control 2. Training with a smaller number of hand grasps improved performance. Choosing different hand-grasp patterns was also more difficult than operating the wrist or elbow motions, consistent with the virtual data presented above. The different grasps facilitated different functional activities. For example, the power grip allowed a firm grasp of a hammer, and the fine pinch enabled picking up small objects (Figure 6A).

This study presents experiments on real-time control of highly articulated artificial arms in patients with targeted muscle reinnervation—a novel neural-machine interface. In this study, we demonstrated that a pattern-recognition algorithm can be used to decode surface EMG data from reinnervated muscles and provide intuitive control of powered elbows, wrists, and hands.

The quantification of outcomes is always a challenge with respect to arm function. Studies of EMG pattern recognition generally report classification accuracies found with able-bodied participants. The accuracies are generally in the range of 90% to 100% regardless of the classification algorithm chosen, demonstrating a ceiling effect. However, classification accuracy is a limited metric of control function. For instance, Lock et al were unable to demonstrate a high correlation between pattern classification accuracy and simple performance testing.¹⁹ In our study, classification accuracy for patients who had undergone TMR surgery was lower than that for control participants. The values measured here were similar to previously reported data from our laboratory and from other investigations using pattern recognition of EMG signals to classify intended movements.^16,20- 21 However, classification accuracy is not a dynamic estimation of performance but rather only the averaged accuracy of being able to hold a motion for several seconds.

In this study, we developed a protocol to assess the control of a virtual arm. We believe that this protocol is more challenging and allows measurement of more insightful performance parameters. The data from control participants performing the same tests are presented as a reference and represent the performance possible with a more complete EMG data set. Our protocol allows the quantification of several key performance metrics. The motion selection time is the speed at which the user can access a function in the prosthesis. Faster is clearly better, but what is good enough? Farrell et al found that participants did not appreciate a time delay of less than 100 ms,²² and others have advocated that a delay of up to 300 to 400 ms is acceptable.^20- 21,23 Thus the motion selection times of TMR patients for arm function are quite good (≤220 ms), and the motion selection time for hand grasps is perhaps marginal at 380 ms. It should be noted that this delay is not due to computational processing, which takes only a few milliseconds. The motion selection time delay is intrinsic to EMG control, because it represents the need for sufficient data to accumulate before an accurate decision can be made.

The motion completion time is a measure of speed of use. We set a normalized gain so that a task could be completed in 1 second, which correlates to a reasonable speed for prosthesis function of 90°/s to 120°/s. Here we see that the TMR patients did quite well compared with the control participants. Perhaps the most important metric was the motion completion rate, which represents a measure of robustness. A very high success rate is needed to allow adequate function and prevent user frustration. One patient had excellent completion rates of 99% for the elbow and wrist and 96% for the hand grasps; this was comparable to the rates for the control participants. Two other patients had difficulty performing multiple hand-grasping functions. In general, the success rates for elbow and wrist functions were high, but the success rates for controlling multiple hand grasps were much lower, demonstrating that this is more challenging for some individuals.

Quantifying operation of a virtual arm allowed measurement of some useful metrics in the laboratory. However, the ultimate goal is for amputees to operate more dexterous prosthetic arms. Controlling a real prosthesis introduces many practical challenges, such as stability of EMG signal recording, interference from muscles controlling remaining joints, and the effects of tissue loading and arm dynamics.²⁴

As part of this program of the Defense Advanced Research Projects Agency, we had the opportunity to conduct initial testing of advanced prosthetic arms with 3 patients in our study. The arms were operated using practical control schemes, including pattern recognition control using EMG signals from reinnervated muscles, conventional myoelectric technology, and custom powered-shoulder controllers. Each patient was able to gain some mastery of the systems in the first day of testing. Within 2 weeks they were all able to demonstrate encouraging control of these complex devices.

Our training and testing periods were brief. Improvement in control and function would be expected with more practice. Simultaneous operation of all shoulder motions and 1 arm movement was demonstrated. However, patients would use simultaneous operation of only 2 joints for reaching tasks and usually operated only 1 motion at a time. This is likely owing to the cognitive burden of operating such a complex device.

The lack of sensory feedback is an obstacle for controlling such complex devices. Patients currently must rely on visual feedback. Improved sensory feedback, especially proprioception, will be critical to the long-term goal of neural integration and more natural control of complex prosthetic arms.

These early trials demonstrate the feasibility of using TMR to control complex multifunction prostheses. Additional research and development need to be conducted before field trials can be performed. Improving EMG signal recording repeatability and stability are required to minimize or eliminate daily classifier training. Work is ongoing to develop more robust surface EMG recording systems and prosthetic interfaces. Implantable EMG systems eliminate some of the surface EMG recording problems and may provide more stable systems.²⁵ Adaptive pattern-recognition algorithms also may improve the stability of control with extended use. Various existing hierarchical control schemes may be more robust for some patients; customization of control hierarchy is an accepted practice in modern prosthetics.

The prosthetic arms tested in this study performed very well as early prototypes. Further improvements are needed and have been planned, including reducing the size and weight and increasing the robustness of these advanced prostheses.