In an explorative laboratory study, 8 healthy young adults (21–29-years-old) were recruited from the university population. Participants were signed on an informed consent and approval by the Helsinki committee of Barzilai University medical center, Ashkelon, Israel (ClinicalTrials.gov Registration number #NCT01439451).
Experimental protocol
In the first stage of the experiment participants were instructed to stand upright in a narrow base standing (heels and toes touching). In the second stage of the experiment, the participants were exposed to two unexpected right and left perturbations with maximal acceleration of 9.8 m/s2 and a top velocity of 0.7 m/s and 10 cm horizontal translation movement. Participants were instructed to respond in a "natural" manner (no instructional constraints). The participants had no knowledge regarding the direction and the timing of perturbation. In the third stage of the study we recorded the participants during comfortable treadmill walking. Finally, the participants were exposed to unexpected right and left platform perturbations (i.e., similar to the standing trails), while the participants walked on the treadmill [24]. Thus each subject went through 4 perturbations: two while standing, and two while walking. To prevent injury during loss of balance, the participants wore loose safety harnesses that arrested the fall, but allowed them to execute step recovery reactions.
Kinematic data during the experiment were collected with 3D Ariel Performance Analysis System (APAS) sampled at a frequency of 60 Hz and stored on a hard disk for later processing.
At the same time, data from the Microsoft Kinect system, sampled at a frequency of 30 Hz, was also stored on a hard disk for later processing. The Ariel Performance Analysis System is a computer-assisted video motion analysis evaluating human kinematics both in research and clinical applications. Klein and DeHaven [25] determined the upper limits of accuracy and consistency of linear and angular measures obtained using the Ariel Performance Analysis System. Reference standards included a meter stick and a universal 360° goniometer. Average mean error observed for reconstruction of absolute point estimates was found to be less than 3.5 mm. Mean error estimate for 3D reconstruction of a linear standard was found to be 1.4 mm (SD 0.30). Average mean angular error observed for 3D reconstruction of goniometer settings of 10° to 170° was found to be 0.26° (mean SD 0.21). In this experiment we compared the different measurements that were recorded by the Ariel Performance Analysis System and Microsoft Kinect system. Recently, Clark et al. [26] found that the Microsoft Kinect can validly assess kinematic strategies of postural control (i.e., forward reach, lateral reach, and single-leg eyes-closed standing balance). The Accuracy of Kinect landmark movements against Vicon marker locations was found to be very high for upper extremity and trunk movements and lower for ankle and foot motion depended on movement dimension, landmark location and performed task [27]. In general vertical movements had the lowest correlations between both systems.
Data analysis
First, we used the following data transformations:
-
1)
Each Kinect joint was matched with the appropriate APAS marker. The Kinect X-axis is switched with the APAS Z-axis (i.e., mediolateral direction) thus Kinect measures were multiplied by −1.
-
2)
The APAS system measures distance in centimeters, thus the Kinect measurements were translated to centimeters.
-
3)
The two systems have a different absolute zero. To normalize both systems we used the average position of the first 10 recorded seconds of the experiment, when the participants were standing still, as the absolute 0 position for each system.
-
4)
We smoothed the Kinect readings using a simple first order filter with k = 5. That is, the smoothed value of a Kinect data point at time t is the average of the raw data points from t-2 to t + 2.
-
5)
The two systems, APAS and Kinect, did not start recording at the exact same time. Hence, we had to compute a time offset to match the readings from the two systems on the time axis. To do so, we used only the extreme data points of the left ankle X-axis (i.e., mediolateral direction). The APAS system able to "identify" accurately the platform horizontal translation movement since marker was placed on the perturbation system. However the Kinect system is not designed to detect perturbation system position. Note that this is not a true limitation, as the perturbation system is controlled by a computer that was connected to the Kinect sensor. Thus, it was simple to record the Kinect readings on the same time scale as the perturbation system control signals.
Following the data transformations we computed the root of the mean square error (RMSE) of the Kinect measurements compared with the APAS measurements. For each joint j, the error for a time point t for the Kinect measurement is computed using
$$ Erro{r}_t= Kinec{t}_t- A P A{S}_{t\hbox{'}} $$
where APAS
t’
is the APAS reading at the time point closest to t, which is always within 1 millisecond of t. Then, the RMSE of the Kinect system with respect to the APAS measurements is computed using
$$ RMSE=\sqrt{\frac{\sum_t Erro{r}_t^2}{n}} $$
Two parameters were measured: 1) the length of the compensatory stepping response (i.e., step length); 2) The step reaction time and step execution duration. The following events were extracted from the motion analysis APAS system and the Microsoft Kinect system: (1) The unexpected platform perturbation was detected as the first medio-lateral deviation of the perturbation system and from the computer that was connected to the Kinect sensor respectively; (2) Foot-off was defined at the sudden elevation of the foot off the ground using the ankles’ vertical values; (3) Foot-contact was defined as the foot contacted the ground using the ankles’ vertical values; (4) Compensatory step initiation time (in milliseconds) was calculated as the time from perturbation to foot-off the ground; (5) Compensatory step duration (in milliseconds) was calculated as the time from platform perturbation to foot contact the ground; (6) Compensatory step length (cm) was calculated as the displacement of the ankle marker from the beginning of the step to the end of the step in mediolateral direction.
Statistical analysis
Independent t-test for independent measures were used to evaluate the differences between APAS and Microsoft Kinect systems to measure step parameters during the standing and walking trials (p < 0.05). Once the timing of each temporal event was determined, the average values of the two perturbation trials standing as well as walking for each outcome measure for the Microsoft Kinect and 3D APAS methods were compared using Pearson’s r correlation, ordinary least products (OLP) regression [28]. The following guidelines were used when interpreting Pearson’s r magnitudes: absent to little (r = 0.00–0.25), low (0.26–0.49), moderate (0.50–0.69), high (0.70–0.89), or very high (0.90–1.00) [29]. All statistics were analyzed using SPSS (version 16, Chicago, IL USA).
Additionally, to assess systematic changes of the mean, Bland and Altman analyses were used [30], including the following calculations:
-
1)
d = mean difference between the two measurement methods.
-
2)
SDdif = SD of the difference between the two measurement methods.
-
3)
95% confidence interval of d (95% CI).
-
4)
Limits of Agreement (LOA) = d ± 1.96 SDdif.
Graphics were used to improve analyses of Kinect and APAS systems from the four different trials and to guide interpretation of discordance patterns. This included plotting a difference in mean from "Bland and Altman plots" [30]. The Bland and Altman graphs depict the measurement error as the differences (y-axis/vertical direction) between Kinect system and APAS plotted against the mean (x-axis/mediolateral direction) of the two trails for each subject. The advantage of Bland and Altman plots is in examining the differences for each parameter measured by the two systems. The plots also provide LOA, when most differences (95%) lie in LOA, normal distribution of differences can be assumed. Bland and Altman plots were generated using MedCalc (version 14.10.2.0, MedCalc Software bvba, Ostend, Belgium).
