Share This Article:

Application of Multiple Sensor Data Fusion for the Analysis of Human Dynamic Behavior in Space: Assessment and Evaluation of Mobility-Related Functional Impairments

Abstract Full-Text HTML XML Download Download as PDF (Size:3363KB) PP. 182-203
DOI: 10.4236/jbise.2017.104015    627 Downloads   952 Views  


The authors have applied a systems analysis approach to describe the musculoskeletal system as consisting of a stack of superimposed kinematic hier-archical segments in which each lower segment tends to transfer its motion to the other superimposed segments. This segmental chain enables the derivation of both conscious perception and sensory control of action in space. This applied systems analysis approach involves the measurements of the complex motor behavior in order to elucidate the fusion of multiple sensor data for the reliable and efficient acquisition of the kinetic, kinematics and electromyographic data of the human spatial behavior. The acquired kinematic and related kinetic signals represent attributive features of the internal recon-struction of the physical links between the superimposed body segments. In-deed, this reconstruction of the physical links was established as a result of the fusion of the multiple sensor data. Furthermore, this acquired kinematics, kinetics and electromyographic data provided detailed means to record, annotate, process, transmit, and display pertinent information derived from the musculoskeletal system to quantify and differentiate between subjects with mobility-related disabilities and able-bodied subjects, and enabled an inference into the active neural processes underlying balance reactions. To gain insight into the basis for this long-term dependence, the authors have applied the fusion of multiple sensor data to investigate the effects of Cerebral Palsy, Multiple Sclerosis and Diabetic Neuropathy conditions, on biomechanical/neurophysiological changes that may alter the ability of the human loco-motor system to generate ambulation, balance and posture.

1. Introduction

Human dynamic behavior in space is very complex because it involves many physical, perceptual and motor aspects [1] . The active maintenance of the human body configuration and orientation in space is dependent on visual and proprioceptive cues. This is a vital motor function since the maintenance of posture is a non-volitional activity based on pre-determined inborn neural mechanisms. Therefore, the efficient control of the basic posture is equally important for standing, walking and for providing support during voluntary movements of the limbs, trunk, and head [1] [2] [3] . The role of vestibular input and its interaction with visual and somatosensory cues for human postural control is still not well understood [2] [3] [4] . But we know that other systems intervene to compensate for sensory loss of another system. An example of this case is a diabetic patient with a peripheral neuropathy and diminished sensory proprioceptive feedback; this patient relies more on the visual clues to help with gait and balance. Learning the interplay between these cues may provide the key to understanding the complex function.

There have been many attempts to elucidate the mechanisms underlying the intelligent adaptive behavior in the complex human locomotor system. These attempts have involved the direct capture of the activities of the neuronal system in human locomotion [5] [6] [7] . They do not, however, clarify how the nervous system adaptively functions as a dynamic system and how it effectively coordinates adaptive interactions with the musculo-skeletal system during locomotion [6] [7] . Other studies have tried to artificially emulate locomotion by using mathematical models and robots based on control theory. Here again, the successful locomotor control of a simulation model or robot does not lead to an understanding of biological locomotor mechanisms, as the control laws are artificially constructed solely based on an engineering perspective independent of actual biological mechanisms [6] [7] [8] [9] . Innovations in the quantitative analysis of gait, physiology and biomechanics are essential to truly understand the principles of adaptive behavior in human locomotion and locomotion rehabilitation. Quantitative gait analysis has been used to elucidate characteristic features of neurological gait disturbances. Although a number of studies compared single patient groups with controls, there are only a few studies comparing gait parameters between patients with different neurological disorders affecting gait [9] [10] [11] [12] [13] .

Human motor control fundamentally involves a series of transformations of information among different levels and components of the neuromuscular and skeletal systems [14] . The proprioceptive and exteroceptive sensory information data are transduced by sensory structures that in turn transfer a subset of their information to the central nervous system which, following yet another transformation, issues a set of motor commands [4] [6] [8] . The mechanical coupling between musculoskeletal elements and the muscles controlling them is yet another transformation of information in the system [14] [15] [16] [17] . The control problem revolves around the specific transfer functions that describe each transformation [6] [14] [15] . The transfer functions depend on the rules of organization and operation that determine the dynamic behavior of each subsystem.

The ability to control balance is dependent on sensory inputs from somatosensory, visual and vestibular systems [2] [3] . Information concerning the position and movement of body segments with reference to each other and the support surface and the distension of the respective muscles is provided through the somatosensory system, the proprioceptors and the mechanical sensitivity of cutaneous and subcutaneous tissue [1] [2] [3] [4] .

To gain insight into the basis for this long-term dependence, the authors have applied the fusion of multiple sensor data to investigate the effects of Cerebral Palsy, Multiple Sclerosis and Diabetic Neuropathy conditions, on biomechanical/neurophysiological changes that may alter the ability of the human locomotor system to generate ambulation, balance and posture.

The acquired kinematic and related kinetic signals represent attributive features of the internal reconstruction of the physical links between the superimposed body segments. The signals also depict the global variables necessary for sensorimotor adaptation.

This study does demonstrate that the application of multiple sensor data fusion for the analysis of human dynamic behaviour in space exhibits efficiency, validity, reliability, responsiveness and practicability in the assessment and evaluation of mobility-related functional impairment. It will enable physicians and therapists to gain adequate knowledge of the patient and disease characteristics that determine functional outcome.

2. Method

A Systems Analysis Approach Was Applied to Enable the Understanding of Human Dynamic Behavior in Space

I. Theoretical Systems Analysis Approach:

The active control of posture and balance is crucial for tasks of daily living. The neural control of postural orientation and equilibrium involves virtually all body segments and several sensory systems [1] [2] . Postural equilibrium refers to balance, that is, resisting external forces acting on the body to maintain a desired body position. On earth, gravity is the dominant external force affecting equilibrium. Postural orientation refers to body geometry, or the position of the body segments with respect to each other and with respect to various reference frames in the environment. Balance is achieved during stance when the downward projection of the center of mass remains within the base of support. The base of support is defined by parts of the body in contact with the environment. Upright stance requires at least two functions, antigravity support, or maintaining the center of mass at the required height, and balance, or controlling the horizontal plane trajectory of the center of mass. Other functions may also be required.

In order to study the postural control system, this experimental neuroscience approach has applied psychophysics to quantitatively investigate the relationships between physical stimuli (subject’s subjective selection of treadmill speed as input) and the sensations and perceptions they affect (balance, posture) and measured the human dynamic behavior in terms of the ground reaction forces under the feet, the positions of the body segments, and electromyographic (EMG) activity. The combination of all three measurements allows us to infer the active neural processes underlying balance reactions.

Figure 1. Illustration of the human body as a stack of hierarchically structured and superimposed segments.

head, etc., during standing. The internal notion of space is required for both perception and postural control, and this especially in situations where we have no haptic or visual information of space. The same applies to our feet or trunks, when they are stationary.

Figure 1 illustrates the internal kinematic reference system that depicts a reconstruction of physics, with a gravitational anchoring of head on trunk on legs on feet on foot support in space. The perceptual notion of space is derived mainly through vestibular sensors. The continuous internal reconstruction of the system is shaped by the available sensors, their transfer characteristics, their locations within the body, etc.

Main objective of this study is to perform an experimental systems analysis to acquire and process the biomechanical and physiological attributive features of human subjects to enable the evaluation and assessment of human dynamic behaviour in space.

Clinical relevance: It is possible that in disease states as a sensory system degrades, another system compensates for the deficiencies. If however, the compensatory capacity is limited, functional limitations will manifest themselves.

II. Applied Systems Analysis Approach

Figure 2 depicts the practical acquisition of the information granules relating to the kinematics, kinetics and the elctromyographic data of the human dynamic

Figure 2. Experimental design for the data acquisition.

Figure 3. Block diagram of the experimental data acquisition of physiologic/ biomechanics data (according to Figure 2).


In Figure 2, a cluster of human movement data is acquired through the application of three main hardware devices, the instrumented treadmill (depicted in Figure 2 as Force Plates) for the Ground Reaction Forces (GRFs), the surface electromyography (sEMG) device for measurement of the electrical activities of the muscles, and wearable inertial sensors (goniometers, gyroscopes, accelerometers) for the kinematics.

Figure 3 illustrates a block diagram representation of the experimental procedure introduced in Figure 2.

3. Data Acquisition

The Institutional Review Board (IRB) of the University of Texas at El Paso (UTEP) approved this study, and all subjects signed an informed consent form prior to participation.Subjects were identified primarily from the affiliated institutions of the Texas Tech Medical Center at El Paso. Flyers announcing the opportunity to participate in this research was posted at these sites and circulated among receptive community physicians and therapists identified as likely to be familiar with potential subjects. Once identified, the potential subjects were assessed and screened for eligibility by Dr. Miguel Pirela-Cruz, Director of the Orthopedic and Rehabilitation Surgery of the Paul L. Foster School of Medicine, Texas Tech University Medical Center. Potential subjects were encouraged to view the experimental set-up prior to signing the consent. All subjects were able to read and understand the Informed Consent. For patients, a family member was encouraged to participate in hearing the explanation of the study and to help the potential subject evaluate participation in the study. After reading the consent, subjects were asked to restate their understanding of their commitments over the time line of the study and to restate their understanding that they could stop their participation at any time without concern about their present or future care.

Each subject walked bare-footed on the dual-belt instrumented treadmill at a self-selected natural speed continuously for three minutes. The instrumented treadmill measures the ground reaction forces (GRF) in three planes (Fx―ante- rior posterior; Fy―medial lateral; Fz―vertical) during the walking tasks.

The Delsys Trigno wireless EMG system measures the dynamic activities of the muscles on both sides of the lower extremity. It consists of 16 wireless EMG sensors, and raw data were acquired at a sampling frequency of 1000 samples/s, and also tri-axial accelerometers. Eight muscles for each side were selected as illustrated in Figure 2; soleus (Sol), tibialis anterior (TA), gastrocnemius lateralis (LG), vastus lateralis (VL), rectus femoris (RF), biceps femoris (BF), gluteus medius (Gmed), and erector spinae (ES). The placement of the electrodes on the subjects was performed based on [18] and the signals were tested before the data acquisition. The wearable inertial sensor array consists of accelerometers, gyroscopes and goniometers that measure the dynamic motion in three dimensional space, as well as analog to digital (A/D) converter.

4. Data Analysis and Processing

The data processing and filtering methods were performed based on the type of data acquired. Three types of raw data cluster were recorded for each subject; the GRFs, EMG, and kinematics data, respectively.

The GRFs data were acquired at a frequency of 100 Hz and filtered using 20 Hz low pass filter. The GRFs amplitudes were normalized based on body mass [19] [20] [21] [22] . The gait cycle/stride time and phases are determined based on the vertical component of GRFs [23] . Time-normalization of strides is accomplished by re-sampling and expressing each stride in percentage rather than time [24] . In addition, the amplitudes of the GRFs were normalized based on the subject’s body weight.

Figure 4 illustrates the processing steps of the EMG data. Due to the concentration of the EMG data within the band between 20 Hz and 200 Hz [24] , a band pass filter (20 Hz - 200 Hz) was applied to the raw EMG data to reduce the noise effects such as the motion artefact noise. The filtered data were then full-wave rectified to generate the absolute value of the EMG. Linear envelope is a common way to manipulate EMG signal [25] . It can be called also as moving average. The linear envelope is produced by applying a second order Butterworth low pass filter with a cut-off frequency of 7 Hz to the full-wave rectified EMG signal. EMG Amplitude normalization is performed due to the variability between individuals such as the inherent physiological variability and the variability associated with electrode placement [26] . The normalization is carried out for each subject based on the peak or the mean of the EMG signal [27] . Time-nor- malization is also performed for the strides time similar to the GRFs. The output of inertial sensors was acquired at 100 Hz sampling rate. A low pass filter with cut-off frequency of 6 Hz is typically applied to lower the noise and improve the resolution of the accelerometers [24] . Time-normalization is applied for the

Figure 4. EMG data processing. (a) Row EMG data, (b) Band pass filtered EMG Data, (c) Full-wave rectified EMG data, and (d) Linear envelope of EMG data, X-axis is the time in seconds and Y-axis is amplitude in volts.

strides time as well.

The authors have taken advantage of the performance of the data acquisition scheme for the human dynamic behavior described and applied in [28] [29] [30] .

From the experimental process as depicted in Figure 2 and Figure 3, a cluster of data relating to the kinematics, kinetic and muscle activities were acquired using an array of inertial sensors, instrumented treadmill and electromyography devices during normal walking tasks. This measured data cluster was partitioned into so-called homogenous clusters to indicate that all points in the same group are close to each other and not close to points in other groups. This clustering process was then used to build pattern classes or to reduce the size of the set of data while retaining relevant information.

The data were treated as aggregate information granules that enabled the efficient partition of input space and more rapid analysis. This means that we deal with the relationships of the kinematic, kinetic and muscle activity functions within the gait cycle. The relationships depict the attributive features of the human movement and are expressed in an implication table giving rise to a fuzzy relational matrix, established between the dynamic activities during the walking tasks [31] [32] [33] [34] . In [28] [29] [35] , a fuzzy-based semi-heuristic method for gait assessment using fuzzy relational matrices, rule-base, and fuzzy similarity algorithm was proposed.

The described relational matrix depicts a rule-based system representing the strength of association or interaction amongst the elements of Fuzzy sets.

Fuzzy relational matrix (rule-base) can be used to represent the strength of association or interaction amongst the elements of gait functions (data cluster), and it depicts a rule-base that can be used to provide a model of feature matrix. A fuzzy relational matrix of size n × m may be developed as an equation of the form in:


The separation of clusters is a fuzzy notion, and the representation of clusters by fuzzy sets may seem more appropriate insituations involving human spatial dynamic behavior. The captured data exhibits some imprecision owing to subjective and systematic occurrences during the acquisition process.

When determining the presence or absence of association, interaction or interconnection between the elements of the kinetic, kinematic and electromyographic data within the gait phases, the mapping is usually nonfuzzy. The mapping also depicts transfer functions involving the joint angles, velocities, accelerations and muscle activities within the gait cycle whereby the input to the system is the evoked potential (stimuli) given by the speed of the treadmill.

However, when the severity level of an element of the gait function must be determined, then fuzzy sets are often involved in the mapping. The application of cluster analysis is entered on the formulation of relationships for the variables under consideration. Fuzzy clustering provides a richer description of the geometric structure of the data set in many cases, and has a lesser tendency to get stuck in local minima. Memberships can be interpreted as degrees of typicality of degrees of sharing.

This experimental procedure defined information granules in a finite universe of discourse for the representation of the multidimensional experimental data sets:

1) The establishment of the mapping R(x, y) describes the relationship between

・ The 3-dimensional ground reaction forces (GRF) : whose attributes include:

; and

・ The functional phases in a gait cycle: whose attributes include:


This relation may be viewed as a transfer function (a rule or rule-base) that may be used to provide a model. This relationship, R(x, y), depicts a feature space that describes the association, interaction or interconnection between the elements of the ground reaction forces data within the gait phases. This table is then expressed in a relational form R(x, y) as:


2) The establishment of the mapping S(y, z) describes the relationship between

・ The muscle activity, proprioception, to provide information about muscle length and tension in the limbs, whose attributes include:

; and

・ The functional phases in a gait cycle: whose attributes include:


This relationship may be viewed as transfer function (a rule or rule-base) that may provide a model. This relationship, S(y, z), depicts a feature space that describes the association, interaction or interconnection between the elements of the muscle activity and the gait phases. The table is then expressed in a relational form S(y, z) as:


3) We establish the mapping Q(w, y) as the relationship between

・ The joint angles, and the segmental accelerations for proprioception and kinesthesia sensation: whose attributes include: and

・ The functional phases in a gait cycle: whose attributes include:

as this matrix that depicts a transfer function (a feature space or rule base)that describes the association, interaction or interconnection between the elements of the segmental accelerations and the gait phases. The table is then expressed in a relational form Q(w, y) as:


The measured feature pattern (input pattern represents the impaired subject) is compared with the reference feature pattern (healthy subject) within the fuzzy rule-base. This procedure offers an aggregation between the input and reference feature patterns, and it is expressed as the fuzzy similarity algorithm. This algorithm provides an evaluation methodology for determining the behavior of three main gait features, individual muscles, accelerations and forces within various gait phases. The application of the fuzzy similarity measure, therefore, enables the comparison between the reference pattern features and the measured impaired subject feature patterns. Equation (5) illustrates the example of the fuzzy similarity measure between the healthy subject, , and the impaired subject, ,


where depicts the attributive features of the healthy subject (reference subject) and are the attributive feature of the impaired subject.

Data Processing

The application of gait phases for the analysis of human walking pattern offers the attributive features that are significant towards the synergistic motion between the stack of the superimposed segments and joints. This paper takes advantage of the definition of the gait cycle into seven phases [29] [30] [36] in which each gait phase has a functional objective to accomplish its required dynamic behavior. The sequence of the gait phase combinations enables the limb to perform the three significant tasks, weight acceptance, single-limb support, and limb advancement. Therefore, the rules of the organization and operation that determine the dynamic behavior of each segment (foot, shank, thigh, and hip) are expressed through transfer functions. Motor control involves a series of transformations of information and revolves around the specific transfer functions that describe each transformation.

The experimentally established mappings that are expressed as relational matrices are stored in a Knowledge Base system as in Figure 5. It may be borne in mind that the stored transfer functions of Able-bodied subjects are entered in the knowledge base system as the reference features (Reference Feature Matrix). Acquired feature matrices of test subjects (impaired subjects) serve as input into

Figure 5. Principle of the data processing.

the data processing system depicted in Figure 5. A fuzzy similarity analysis, Equation (5) is performed in order to determine the level and/or severity of an impairment [28] [29] [30] [35] [36] [37] [38] .

5. Results and Analysis

Fluctuations in the duration of the gait cycle appear to be intrinsic to the healthy locomotor system. The stride interval depicts fractal dynamics and long-range correlations in healthy human subject. Hence, the correlations associated with the stride-interval may be altered by changes in mobility-related functional impairments associated with certain skeletal and/or neuromuscular disorders and/or injuries.

The mechanisms responsible for these stride-interval correlations are largely unknown [39] . They may be a consequence of peripheral input or lower motorneuron control, or they may be related to higher nervous system centers that control walking rhythm. Although the breakdown of long range correlations during metronomic walking suggests that supraspinal influences (e.g., a metronome) can override the normally present long-range correlations, their origin and function remain to be determined [39] .

Current application of single sensors in the analysis of the complex human locomotor system may often offer only marginal results [40] - [45] . These single-sensor single-algorithm systems work well in situations where the environment is structured and the objects are well known, but are severely limited in their ability to resolve ambiguities in complex systems like the human locomotor system involving multi-body dynamics, biomechanics, physiology and other factors that obscure the interactions amongst the vestibular, somatosensory and visual inputs for postural control [6] [9] [46] [47] . To gain insight into the basis for this long-term dependence, the authors have applied the fusion of multiple sensor data to investigate the effects of Cerebral Palsy, Multiple Sclerosis and Diabetic Neuropathy conditions, onbiomechanical/neurophysiological changes that may alter the ability of the human locomotor system to generate ambulation, balance and posture.

Sensor data fusion involves the process of combining observations from a number of different sensors to provide a robust and complete description of an environment or process of interest.

In Figure 3, the authors introduce a sensor data fusion scheme to map the acquired kinematic, kinetics and electromyogyaphic data acquired from subjects within the seven gait phases. This mapping (crisp relationships) depict a feature space that describes the association, interaction or interconnection between the elements of the kinematics, kinetics and electromyographic data within the gait phases. The combination of different information from multiple sensors and sensor types may enhance the efficiency and reliability; reduce uncertainty and ambiguity inherent in making decisions based on a single information source, thus, increasing accuracy and resolving ambiguities in the knowledge about the complexity of gait dynamics. The application of the fuzzy relational matrix depicts the concept of the fusion of multiple sensor data.

Therapists and clinicians would require these global variables as objective measures of the variances from normal patterns that are necessary to be fed as compensatory strategies into the local proprioceptive feedback loops of joints together with voluntary commands. Neurophysiological studies on spinocerebellar neurons suggest that sensory feedback signals from proprioceptors in muscles and joints are integrated in the spinal circuitry to encode global parameters of the limb movement, i.e., the orientation and length of the axis connecting most proximal joint and distal position of a limb (limb axis) [48] [49] [50] . This finding implies that the global parameters describing limb kinematics, along with individual local proprioceptive inputs, are actually utilized as important sensory inputs for locomotion. In addition, muscle activation pattern is also suggested to be generated primarily based on global kinematic parameters. Grasso et al. showed that kinematic pattern of human forward and backward locomotion was basically the same while muscle activation is quite different [51] .

5.1. Analysis of Biomechanical/Physiological Parameters for Gait Pattern

Able-bodied Subjects

During the initial contact and in the loading response phases, the tibialis anterior, the pretibial muscles and the biceps femoris contract concentrically, whilst the vastus lateralis exhibits eccentrical contraction.

During the midstance phase, the soleus and the calf muscles generally are eccentrically contracting and concentrically contracting in terminal stance and preswing. This process is responsible for the control of the ankle plantarflexion. The activity of the lateral head of the gastrocnemius muscle is similar to the soleus but with more of an eversion moment and some action at the knee as well as the ankle as it crosses two joints. It controls plantar flexion and to a lesser extent knee flexion.

During the preswing and initial swing phases, the biceps femoris exhibit eccentric contraction. This also controls knee flexion and to a small extent hip extension and stabilization.

During the initial, mid and terminal swing phases, the vastus lateralis exhibit eccentric contraction. This behavior also controls knee extension. The tibialis anterior and the pretibial muscles are concentrically contracted in initial swing, mid swing and terminal swing. The tibialis anterior controls dorsiflexion. The soleus exhibits concentrically contraction at the terminal stance and preswing phase. The rectus femoris exhibits similar behaviour like the vastus lateralis but without the valgus force at the knee. It controls knee extension.

Throughout the 60% stance phase of the gait cycle, the gluteus medius is eccentrically contracted. It prevents too much pelvic tilt or what is called a Trendelenberg gait. It controls hip abduction but is used in an open chain fashion in the stance phase of gait. Both the gluteus medius and erector Spinae muscle groups are involved in stabilizing gait and minimizing excursions of the center of gravity.

5.2. Comparative Analysis of the Acquired Neuromorphic Data from Patients

5.2.1. Case Study: The Patient with Cerebral Palsy

Figure 6 illustrates the measurement and analysis of the 3D ground reaction forces normalized by the bodyweight, muscle activities, and 3D acceleration of the lower limbs for the quantified evaluation of a subject/patient with cerebral palsy (CP). The graphical representation of the able-bodied subjects depict a clear vertical force curve with the two-peak “M” shape that illustrates weight transfer from the heel to the mid-foot and the mid-foot to the ball of the foot for push-off. Usually, subjects/patients with cerebral palsy often experience significant problems in supporting their bodyweight (BW) and decelerating the downward velocity in late stance [52] . This is evidenced through a decreased second peak in the vertical ground reaction force (GRFz) in this subject/patient.

Figure 6. Multiple Sensory Data Fusion for Diagnosis of a Patient with Diplegia Cerebral Palsy (3D accelerations in foot, shank and thigh color indications: blue = saggital; green = frontal; red = transverse).

The reduced second peak of GRFz may be a compensatory mechanism for preventing the collapse of the affected limb. In addition, the CP patient showed a smaller slope at the end of the stance phase and pre-swing phase which does not indicate a clear toe-off (foot clearance problem). This CP patient also showed much lower anterior-posterior ground reaction force (GRFy) negative peak. There was no significant variation of the mediolateral ground reaction force (GRFx) compared with the healthy group.

Approximately 80 per cent of patients with cerebral palsy have varying degrees of spasticity (increased muscle tone or tension), which can lead to an equinus gait pattern. This CP patient’s soleus and gastrocnemius showed spasticity with two peaks in the stance phase. The tibialis anterior was not activated in the initial contact and loading responses, but was hyperactive during the rest of the stance phase. This may be as a result of compensation mechanisms and exhibition of spasticity in the two calf muscles during the stance phase. The patient also showed an increased activation in the stance phase and inactivation in the swing phase on the Biceps Femoris. This data also emphasizes the lack of coordination between the gravity and antigravity systems in cerebral palsy. The degree of spasticity that is also out of sync during the appropriate phase of gait is also highlighted.

The profiles of the acceleration of the CP patient’s differ from those of the able-bodied ones, i) there is longer or delayed stance phase for the CP patient. This is because the patient walks very slowly in order to prepare for the next step during the stance phase; ii) CP patient showed delayed and lower foot acceleration in the anterior-posterior plane during the initial contact phase, which depicts an abnormality in the heel strike; iii) There is increased anterior-posterior acceleration, lower and delayed mediolateral acceleration prior to the initial swing phase on the CP patient’s shank; iv) There was the occurrence of unstable curves in the thigh-acceleration of the CP patient.

5.2.2. Case Study: The Patient with Multiple Sclerosis

Figure 7 shows the output signals from the array of the multiple wearable sensor system for the quantitative evaluation of a patient suffering from Multiple Sclerosis (MS). The curve corresponding to the vertical ground reaction forces, the GRFz, indicates that the MS patient exhibits a flat foot contact as compared to that of the healthy subjects that illustrates clear first and second peaks that form the biomechanical/physiological M-shaped force pattern in the vertical direction. The vertical ground reaction force also demonstrated longer ground contact duration in this MS patient.

The MS patient exhibits early activation of the soleus and gastrocnemius muscles in the stance phase. The tibialis anterior and vastus laterlias muscles exhibited delayed and increased activations during the initial contact period of the stance phase. The increased EMG activity that was apparent in the ankle and knee flexor muscles in the MS patient is thought to be a mechanism to counteract balance deficits and may have implications of both fatigue and spasticity [53] . The EMG curves of quadriceps and harmstring group muscles (Rectus

Figure 7. Multiple Sensory Data Fusion for Diagnosis of a Patient with Multiple Sclerosis (3D accelerations in foot, shank and thigh color indications: blue = saggital; green = frontal; red = transverse).

Femoris and Biceps Femoris respectively) showed some levels of weakness during the swing phase. The EMG activity of the Erector Spinae (ES) was found to be sustained and greater in the MS patient, which is likely to be a factor in the instability during single support, as well as serving to reduce the risk of falling.

There is a sharp rise in acceleration in the initial contact and sharp deceleration in terminal swing phase in this MS patient’s foot, which could be the result of foot-drop and a mechanism to counteract balance deficits. Inconsistent acceleration pattern occurs in the lower extremity (in particular the thigh in both anterior-posterior and mediolateral directions) indicates some levels of tremor in this subjects.

5.2.3. Case Study: The Patient with Diabetic Neuropathy (Diabetic Foot)

Figure 8 illustrates the acquired multiple sensor data including the 3D ground reaction forces that are normalized by the bodyweight, the 3D joint angles of the hip, the knee, and the ankle, and the muscle activities, respectively, for the quantitative analysis of a patient with Diabetic Neuropathy. This patient showed flat foot contact without two peaks in the vertical GRF, even higher force during the mid-stance phase. In the anterior-posterior plane, there were much lower magnitudes of GRF compared with the able-bodied subjects.

The results demonstrated that the subject with diabetic neuropathy had less ankle mobility, i.e. lower peak ankle dorsiflexion and ankle plantar flexion. The patient also showed much less flexion and extension of the knee and the hip compared with able-bodied subjects. There was a greater rotation of all three an-

Figure 8. Multiple Sensory Data Fusion for Diagnosis of a Patient with Diabetic Neuropathy. (3D joint angles in foot, shank and thigh color indications: blue = frontal; green = saggital; red = transverse).

gles which may indicate some levels of instability of the gait.

The patient showed a specific pattern of muscle spasticity, notably 1) the initial contact with respect to the heel strike for the Soleus and the Bicep Femoris (hamstring muscle group) expressed themselves significantly earlier and prolonged during the gait cycle in subjects with diabetes than in the control subjects. The cessation times of the Soleus, the Vastus Laterlias, the Bicep Femoris, and the gluteus medius muscles were significantly prolonged in this patient. There was a delayed muscle activity in the Vastus Laterlias muscles.

In all the case studies, the combination of all three attributive measurements (kinetics, kinematic and electromyographic data) may infer into the active neural processes underlying balance reactions.

6. Conclusions

The main objective of therapeutic intervention and (neurological-) rehabilitation is to enable individual patients to achieve their full potential and to maximize the benefits from training, in order to attain the highest possible degrees of phy- sical and psychological performance. Therefore, clinicians charged with facilitating optimal therapeutic intervention/rehabilitation and appropriate discharge planning and implementation are challenged to reliably measure, evaluate and assess the effectiveness of the treatments and therapeutic procedures. Still, a gap remains between prognostic research and rehabilitation practice. Therapists and physicians need to formulate their functional goals as precisely as possible and this process requires adequate knowledge of the patient and disease characteristics that determine functional outcome [54] [55] .

The main objective of this study was to apply neuroscience tools and techniques to measure and evaluate the functional status of the musculoskeletal/neu- romuscular system as it reacts to treadmill-speeds as base-of-support. In such challenged state, the kinematic and related kinetic signals that represent the glo- bal variables necessary for sensorimotor adaptation were efficiently measured using a combination of unobtrusive wireless-based multiple wearable sensor system. The fusion of the multiple sensor data enables improved accuracies and more specific inferences than that could be achieved by the use of a single sensor alone [28] [29] [35] [56] [57] .


The authors wish to thank the Stern Foundation for providing the funds for this research work, and the Ampoma Restorative NeuroTechnologies, LLC for In-Kind assistance to this project. The authors would also like to thank all the subjects who participated in this study.

Competing Interests

The authors declare that they have no competing interests.

Authors’ Contributions

H. Y. carried out the experimental data acquisition of the human data, participated in the analysis and interpretation of the data. M. B. participated in the data analysis and processing. N. V. H. carried out the recruitment of volunteers through screening and interviews, and participated in the data analysis. M. P. C. carried out the recruitment of the impaired subjects, and offered the neurophy- siological analysis/interpretation of the data.

T. S. G. carried out the design of the experimental set-up, participated in the recruitment of both able-bodied and impaired subjects, participated in the data acquisition and analysis.

Conflicts of Interest

The authors declare no conflicts of interest.

Cite this paper

Sarkodie-Gyan, T. , Yu, H. , Bogale, M. , Hernandez, N. and Pirela-Cruz, M. (2017) Application of Multiple Sensor Data Fusion for the Analysis of Human Dynamic Behavior in Space: Assessment and Evaluation of Mobility-Related Functional Impairments. Journal of Biomedical Science and Engineering, 10, 182-203. doi: 10.4236/jbise.2017.104015.


[1] Mergner, T. (2002) The Matryoshka Dolls Principle in Human Dynamic Behaviour in Space: A Theory of Linked References for Multisensory Perception and Control of Action. Cahiers de Psychologie Cognitive, 21, 129-212.
[2] Maurer, C., Mergner, T., Bolha, B. and Hlavacka, F. (2000) Vestibular, Visual, and Somatosensory Contributions to Human Control of Upright Stance. Neuroscience Letters, 281, 99-102.
[3] Mergner, T. (2004) Meta Level Concept versus Classic Reflex Concept for the Control of Posture and Movement. Archives Italiemes de Biologie, 142, 175-198.
[4] Mergner, T. and Becker, W. (2003) A Modeling Approach to the Human Spatial Orientation System. Annals of the New York Academy of Sciences, 1004, 303-315.
[5] Berbyuk, V., Demydyuk, M. and Lytwyn, B. (2005) Mathematical Modeling and Optimization of Walking of Human Being with Prosthesis of Crus. Journal of Automation and Information Sciences, 37, 46-60. Automat Inf Scien.v37.i6.60
[6] Berbyuk, V., Krasyuk, G. and Nischenko, N. (1997) Mathematical Modeling of Human Gait Dynamics in Sagittal Plane. Journal of Mathematical Methods and Physicomechanical Fields, 40, 127-138.
[7] Beryuk, V.E. and Nishenko, N.I. (1997) Energetically Optimal Control over Human Motion in the Phase of Support by an Artificial Leg. Probl. Upravl. Inf., No. 2, 75-86.
[8] Hausdorff, J.M. (2007) Gait Dynamics, Fractals and Falls: Finding Meaning in the Stride-to-Stride Fluctuations of Human Walking. Human Movement Science, 26, 554-589.
[9] Winter, D.A. (2005) Biomechanics and Motor Control of Human Movement. 3rd Edition, John Wiley & Sons, Inc., Hoboken.
[10] Jensen, K.J. (2005) Diagnostics of Human Locomotion. M.S. Thesis, New Mexico Institute of Mining and Technology, Department of Mechanical Engineering, Mechatronics Program.
[11] Jensen, K. and Sarkodie-Gyan, T. (2004) The Paradigm of a Smart Gait Emulator. Proceedings of the 2004 Japan-USA Symposium on Flexible Automation, Denver, 19-21.
[12] Frenkel-Toledo, S., Giladi, N., Peretz, C., Herman, T., Gruendlinger, L. and Hausdorff, J.M. (2005) Treadmill Walking as an External Pacemaker to Improve Gait Rhythm and Stability in Parkinson’s Disease. Movement Disorders, 20, 1109-1114.
[13] Hausdorff, J.M., Peng, C.K., Ladin, Z., Wei, J.Y. and Goldberger, A.L. (1995) Is Walking a Random Walk? Evidence for Long-Range Correlations in Stride Interval of Human Gait. Journal of Applied Physiology, 78, 349-358.
[14] Nishikawa, K., Biewener, A.A., Aerts, P., et al. (2007) Neuromechanics: An Integrative Approach for Understanding Motor Control. Integrative and Comparative Biology, 47, 16-54.
[15] Hausdorff, J.M. (2005) Gait Variability: Methods, Modeling and Meaning. Journal NeuroEngineering and Rehabiliation, 2, 19.
[16] Platz, T., van Kaick, S., Moller, L., Freund, S., Winter, T. and Kim, I.H. (2005) Impairment-Oriented Training and Adaptive Motor Cortex Reorganization after Stroke: A fTMS Study. Journal of Neurology, 252, 1363-1371.
[17] Hausdorff, J.M., Peng, C.K., Ladin, Z., Wei, J.Y. and Goldberger, A.L. (1995) Is Walking a Random Walk? Evidence for Long-Rang Correlations in Stride Interval of Human Gait. Journal Applied Physiology, 78, 349-358.
[18] Cram, J.R., Kasman, G.S. and Holtz, J. (1998) Introduction to Surface Electromyography. Aspen Publishers, Maryland.
[19] Lai, D.T.H., Levinger, P., Begg, R.K., Gilleard, W.L. and Palaniswami, M. (2009) Automatic Recognition of Gait Patterns Exhibiting Patellofemoral Pain Syndrome Using a Support Vector Machine Approach. IEEE Transactions on Information Technology in Biomedicine, 13, 810-817.
[20] Lafuente, R., Belda, J.M., Sanches-Lacuesta, J., Soler, C. and Prat, J. (1997) Design and Test of Neural Networks and Statistical Classifiers in Computer Aided Movement Analysis: A Case Study on Gait Analysis. Clinical Biomechincs, 13, 216-229.
[21] Begg, R.K., Palaniswami, M. and Owen, B. (2005) Support Vector Machines for Automated Gait Classification. IEEE Transactions on Biomedical Engineering, 52, 828-883.
[22] Begg, R. and Kamruzzaman, J. (2005) A Machine Learning Approach for Automated Recognition of Movement Patterns Using Basic, Kinetic and Kinematic Gait Data. Journal of Biomechanics, 38. 401-408.
[23] Pasparkis, D. and Darras, N. (2009) Normal Walking: Principles, Basic Concepts, Terminology, 3-Dimensional Clinical Gait Analysis. Hellenic Orthopaedic Association, 60, 183-194.
[24] Winter, D.A. (2009) Biomechanics and Motor Control of Human Movement. 4th Edition, John Wiley & Sons, Hoboken, New Jersey.
[25] De Luca, C.J. (2006) Electromyography Encyclopedia of Medical Devices and Instrumentation. John Wiley Publisher, Hoboken, 98-109.
[26] Allison, G.T., Marshal, R.N. and Singer, K.P. (1993) EMG Signal Amplitude Normalization Technique in Stretch-Shortening Cycle Movements. Journal of Electromyography and Kinesiology, 3, 236-244.
[27] Ricamato, A.L. and Hidler, J.M. (2005) Quantification of the Dynamic Properties of EMG Patterns during Gait. Journal of Electromyography and Kinesiology, 15, 384-392.
[28] Murad, A., Yu, H.Y., Brower, R., Abdelgawad, A. and Sarkodie-Gyan, T. (2011) Application of Wearable Sensors for Human Gait Analysis Using Fuzzy Computational Algorithm. Engineering Applications of Artificial Intelligence, 24, 1018-1025.
[29] Yu, H.Y., Alaqtash, M., Spier, E. and Sarkodie-Gyan, T. (2010) Analysis of Muscle Activity during Gait Cycle Using Fuzzy Rule-Based Reasoning. Measurement, 9, 1106-1114.
[30] Alaqtash, M. (2012) The Application of Fuzzy Granular Computing for the Analysis of Human Dynamic Behavior in 3D Space. PhD Dissertation, University of Texas, El Paso.
[31] Pedrycz, W. and Vukovich, G. (2002) Feature Analysis through Information Granulation and Fuzzy Sets. Pattern Recognition, 35, 825-834.
[32] Yu, F.S. and Pedrycz, W. (2009) The Design of Fuzzy Information Granules: Tradeoffs between Specificity and Experimental Evidence. Applied Soft Computing, 9, 264-273.
[33] Mencar, C. and Fanelli, A.M. (2008) Interpretability Constraints for Fuzzy Information Granulation. Information Sciences, 178, 4585-4618.
[34] Pedrycz, W. and Gacek, A. (2002) Temporal Granulation and Its Application to Signal Analysis. Information Sciences, 143, 47-71.
[35] Sarkodie-Gyan, T., Yu, H.Y., Murad, A., Abdelgawad, A., Spier, E. and Brower, R. (2011) Measurement of Functional Impairments in Human Locomotion Using Pattern Analysis. Measurement, 44, 181-191.
[36] Sarkodie-Gyan, T., Yu, H.Y., Alaqtash, M., Spier, E. and Brower, R. (2009) Recognition and Deci-sion-Making Algorithm in Human Locomotion Based on the Principles of Fuzzy Reasoning. Proceedings of the IEEE International Conference on Robotics and Biomimetics, Guilin, 19-23 December, 2009, 529-534.
[37] Yu, H., Nava, P., Brower, R., Ceberio, M. and Sarkodie-Gyan, T. (2007) Identification of Human Gait Using Fuzzy Inferential Reasoning. Proceedings of the International Conference on Rehabilitation Robotics, Netherlands, 13-15 June 2007, 656-660.
[38] Yu, H.Y., Riskowski, J., Brower, R. and Sarkodie-Gyan, T. (2009) Gait Variability While Walking with Three Different Speeds. Proceedings of the International Conference on Rehabilitation on Robotics, Kyoto, 23-26 June 2009, 823-827.
[39] Hausdorff, J.M., Mitchell, S.L., Firton, R.E., Peng, C.K., Cudkowicz, M.E., Wei, J.Y. and Goldberger, A.L. (1997) Altered Fractal Dynamics of Gait: Reduced Stride-Interval Correlations with Aging and Huntington’s Disease. Journal of Applied Physiology, 82, 262-269.
[40] Liu, T., Inoue, Y. and Shibata, K. (2009) Development of a Wearable Sensor System for Quantitative Gait Analysis. Measurement, 42, 978-988.
[41] Giansanti, D., Macellari, V. and Maccioni, G. (2005) The Development and Test of a Device for the Reconstruction of 3-D Position and Orientation by Means of a Kinematic Sensor Assembly with Rate Gyroscopes and Accelerometers. IEEE Transactions on Biomedical Engineering, 52, 1271-1277.
[42] Ruth, E., Mayagoitaia, A., Nene, V. and Veltink, P.H. (2002) Accelerometer and Rate Gyroscope Measurement of Kinematics: An Inexpensive Alternative to Optical Motion Analysis Systems. Journal of Biomechanics, 35, 537-542.
[43] Nolan, L., Wit, A., Dudzinski, K., Lees, A., Lake, M. and Wychowanski, M. (2003) Adjustments in Gait Symmetry with Walking Speed in Trans-Femoral and Trans-Tibial Amputees. Gait and Posture, 17, 142-151.
[44] Aminian, K., Trevisan, C., Najafi, B., Dejnabadi, H., Frigo, C., Pavan, E., Telonio, A. and Robert, P. (2004) Evaluaiton of an Ambulatory System for Gait Analysis in Hip Osteoarthritis and after Total Hip Replacement. Gait and Posture, 20, 102-107.
[45] Mathie, M.J., Coster, A.C.F., Lowell, N.H. and Cellar, B.G. (2004) Accelerometry: Providing an Integrated Practical Method for Long-Term, Ambulatory Monitoring of Human Movement. Physiological Measurement, 25, 1-20.
[46] Berbyuk, V., Demydyuk, M. and Lytwyn, B. (2005) Mathematical Modeling and Optimization of Human Walking on a Below-Knee Prosthesis. Int. J. Problems of Control and Informatics, No. 3, 128-144.
[47] Berbyuk, V. (1996) Multibody System Modeling and Optimization Problems of Lower Limb Prostheses. In: Bestle, D. and Schiehlen, W., Eds., Proceedings of the IUTAM Symposium on Optimization of Mech. Systems, Kluwer Academic Publishers, Dordrecht, 25-32.
[48] Bosco, G. and Poppele, R.E. (2001) Proprioception from a Spinocerebellar Perspective. Physiological Reviews, 81, 539-568.
[49] Poppele, R.E., Bosco, G. and Rankin, A.M. (2002) Independent Representations of Limb Axis Length and Orientation in Spinocerebellar Response Components. Journal of Neurophysiology, 87, 409-422.
[50] Poppele, R. and Bosco, G. (2003) Sophisticated Spinal Contributions to Motor Control. Trends in Neurosciences, 26, 269-276.
[51] Grasso, R., Bianchi, L. and Lacquaniti, F. (1998) Motor Patterns for Human Gait: Backward versus forward Locomotion. Journal of Neurophys, 80, 1868-1885.
[52] Williams, S.E., Gibbs, S., Meadows, C.B., et al. (2011) Classification of the Reduced Vertical Component of the Ground Reaction Force in Late Stance in Cerebral Palsy Gait. Gait and Posture, 34, 370-373.
[53] Kelleher, K.J., Spence, W.D., Solomonidis, S.E. and Apatsidis, D. (2010) The Characterization of Gait Patterns with Multiple Sclerosis. Disability and Rehabilitation, 32, 1242-1250.
[54] Restoring Neurological Function (2004) Putting the Neurosciences to Work in Neurorehabilitation. Academy of Medical Sciences.
[55] Neurorehabilitation Disorders (2006) Public Health Chal-lenges. World Health Organization.
[56] Ohtaki, Y., Sagawa, K. and Inooka, H. (2009) A Method for Gait Analysis in a Daily Living Environment by Body-Mounted Instruments. Japan Society of Mechanical Engineers (JSME) International Journal, 44, 1125-1132.
[57] Garza-Ulloa, J., Yu, H.Y. and Sarkodie-Gyan, T. (2012) A Mathematical Model for the Validation of the Ground Reaction Force Sensor in Human Gait Analysis. Elsevier Measurement, 45, 755-762.

comments powered by Disqus

Copyright © 2018 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.