Evaluation of the performance of accelerometer-based gait event detection algorithms in different real-world scenarios using the MAREA gait database
Introduction
The development of gait event detection (GED) algorithms using various sensing modalities has been an active area of research for many years [1]. In the past decade, several GED algorithms have been developed using motion capture systems, present in gait labs, and recent studies have compared and evaluated their performance [2], [3], [4], [5]. Alternatively, inertial sensors are being used as they allow the possibility of long-term monitoring in everyday life and provide spatio-temporal information that can be fused to obtain the entire trajectory of the limb segment [6], [7]. While many GED algorithms have been developed using gyroscopes, others have used accelerometers as they are miniature, inexpensive and low-powered devices [1], [8]. However, as accelerometers suffer heavily from noise due to mechanical vibrations, they require robust algorithms for accurate event detection. Almost all existing accelerometer-based GED algorithms have been developed and assessed using data collected in controlled indoor experiments, that usually involves instructing the subjects to walk in a straight line or a given path at self-selected pace [9], [10] or predefined walking speeds [11], [12], [13]. On the contrary, in the real-world, human gait is quite dynamic in different environments, often involving varying gait speeds, changing walking surfaces and varying surface inclinations, among others. Therefore, it needs to be assessed whether such dynamic and uncontrolled real-world scenarios have an impact on the performance of existing GED methods which have been developed from controlled protocols in laboratory settings. However, as almost all publicly available accelerometer-based gait databases [14], [15] and recent comparative studies [16] also consist of only controlled indoor experiments, there is a lack of gait datasets or any studies that evaluate the performance of existing GED methods in various real-world settings; especially when portable wearable systems can be readily used to conduct experiments directly in humans’ natural environment.
Consequently, a new gait database called MAREA: Movement Analysis in Real-world Environments using Accelerometers, was collected that comprises of various gait activities in different environments, both indoors and outdoors. The objective of this study is two-fold: (1) to introduce the MAREA database which is made publicly available for all readers, and (2) to assess the impact of different real-world scenarios on the performance of state-of-the-art GED algorithms, using the MAREA database. The database is made publicly available at http://islab.hh.se/mediawiki/Gait_database (Table 1).
Section snippets
MAREA gait database
20 healthy adults (12 males and 8 females, average age: 33.4 ± 7 years, average mass: 73.2 ± 10.9 kg, average height: 172.6 ± 9.5 cm) participated in the study that was approved by the Ethical Review Board of Lund, Sweden. Each subject had a 3-axes Shimmer3 (Shimmer Research, Dublin, Ireland) accelerometer (±8 g) attached to their waist, left wrist and left and right ankles using elastic bands and velcro straps. Fig. 1 shows the position and orientation of the accelerometers at the beginning of each
Results
Fig. 2 shows the F1 scores for detecting HSs and TOs in five different scenarios, defined earlier in Section 2.2. Each colored boxplot consists of the F1 scores of applying a particular GED method on the data from all subjects collected for a given scenario. The first two subfigures in Fig. 3 show the MAE in detecting HSs and TOs by various GED methods in different scenarios along with the number of true positives detected by the respective methods, for each scenario. The remaining subfigures
Discussion
The statistical results of applying GED methods in various real-world scenarios reveal that the performance of most GED methods for detecting HS and TO is not consistent across different environments and activities. As shown in Table 2, the experimental protocol of GED methods usually involves walking on a flat surface such as a corridor which is represented by the Indoor Walk scenario. In this setting, the best performance is exhibited by ASK and AJR with high median F1 scores of 0.99 for all
Conclusion
To conclude, a new gait database called MAREA is presented that consists of various gait activities in different environmental settings, both indoors and outdoors. The performance of existing GED methods is evaluated in various scenarios defined using the MAREA database. It is observed that while all GED methods exhibit good performance for the scenario of steady walking in a controlled indoor environment, they demonstrate decreased performance in other environments and more dynamic scenarios
Conflict of interest
There are no conflicts of interest.
Acknowledgements
This study was supported in part by the Knowledge Foundation, Sweden.
References (25)
- et al.
Methods for gait event detection and analysis in ambulatory systems
Med. Eng. Phys.
(2010) - et al.
Identifying gait events without a force plate during running: a comparison of methods
Gait Posture
(2011) - et al.
Automated event detection algorithms in pathological gait
Gait Posture
(2014) - et al.
A comparison of kinematic algorithms to estimate gait events during overground running
Gait Posture
(2015) - et al.
Accelerometer and rate gyroscope measurement of kinematics: an inexpensive alternative to optical motion analysis systems
J. Biomech.
(2002) - et al.
Measurement of foot placement and its variability with inertial sensors
Gait Posture
(2013) - et al.
Accelerometry: a technique for quantifying movement patterns during walking
Gait Posture
(2008) - et al.
Gait event detection for use in FES rehabilitation by radial and tangential foot accelerations
Med. Eng. Phys.
(2014) - et al.
The largest inertial sensor-based gait database and performance evaluation of gait-based personal authentication
Pattern Recognit.
(2014) - et al.
Accuracy, sensitivity and robustness of five different methods for the estimation of gait temporal parameters using a single inertial sensor mounted on the lower trunk
Gait Posture
(2014)
Real-time gait event detection using wearable sensors
Gait Posture
An enhanced estimate of initial contact and final contact instants of time using lower trunk inertial sensor data
Gait Posture
Cited by (100)
Human Locomotion Databases: A Systematic Review
2024, IEEE Journal of Biomedical and Health InformaticsGAIT MEASUREMENT METHODS: SYSTEMATIC REVIEW AND COMPARATIVE STUDIES
2024, Journal of Mechanics in Medicine and BiologySmartphone-Based Pedestrian Inertial Tracking: Dataset, Model, and Deployment
2024, IEEE Transactions on Instrumentation and MeasurementA Low-Cost Instrumented Shoe System for Gait Phase Detection Based on Foot Plantar Pressure Data
2024, IEEE Journal of Translational Engineering in Health and Medicine