Elsevier

Gait & Posture

Volume 63, June 2018, Pages 290-295
Gait & Posture

ESMAC BEST PAPER 2017: Using machine learning to overcome challenges in GMFCS level assignment

https://doi.org/10.1016/j.gaitpost.2018.04.017Get rights and content

Abstract

We used the random forest classifier to predict Gross Motor Function Classification System (GMFCS) levels I–IV from patient reported abilities recorded on the Gillette Functional Assessment Questionnaire (FAQ). The classifier exhibited outstanding accuracy across GMFCS levels I–IV, with 83%–91% true positive rate (TPR), area under the receiver operation characteristic (ROC) curve greater than 0.96 for all levels, and misclassification by more than one level only occurring 1.2% of the time. This new approach to GMFCS level assignment overcomes several difficulties with the current method: (i) it is based on a broad spectrum of functional abilities, (ii) it resolves functional ability profiles that conflict with existing GMFCS level definitions, (iii) it is based entirely on self-reported abilities, and (iv) it removes complex age dependence. Further work is needed to examine inter-center differences in classifier performance—which would most likely reflect interpretive differences in GMFCS level definitions between centers.

Introduction

In the past, functional mobility of children diagnosed with cerebral palsy (CP) was often classified using terms like “independent”, “crutch-aided”, “walker-aided”, or “wheelchair-aided”. To many, these four descriptors seemed sufficient to adequately paint a general picture of a child’s mobility and level of function. However, in 2000, the gross motor classification system (GMFCS) burst on the scene with the revelation that five classes, rather than four, were needed to describe mobility [1]. The primary additions of the GMFCS were to split crutch-aided gait into two classes, based largely on stair climbing ability, and to add age stratification. Cartoon archetypes depicting the various GMFCS levels were also created. These were necessary since the Greek numerals used to denote the GMFCS classes had lost the descriptive nature of the old-fashioned, “call it like you see it”, terminology.

While the GMFCS was an instant sensation, it was not the first attempt to classify function among children with mobility impairments. For example, nearly 25 years earlier, Hoffer proposed a scheme for classifying function in children diagnosed with myelomeningocele [2]. The Hoffer system consists of four categories that form a superset of those listed in the GMFCS: 1) community ambulators, who “may need crutches or braces, or both”, 2) household ambulators, who “walk only indoors and with apparatus”, 3) non-functional ambulators, for whom walking is restricted to “therapy session”, and 4) non-ambulators, who “are wheel-chair bound but usually can transfer from chair to bed”. The newer Functional Mobility Scale has similarities to both the GMFCS and the Hoffer system, but further clarifies function by three distinct walking distances (5, 50, and 500 m) [3].

Another commonly used classification scheme for children with CP is the Gillette Functional Assessment Questionnaire (FAQ) [4]. The FAQ is a validated, survey based measure of function. The survey contains a single-valued 10-point walking ability score (FAQ-10), along with reported abilities on 22 higher-level skills (FAQ-22). The FAQ-10 spans a wide range of performance, with 10 defined as “Walks, runs, climbs without difficulty or assistance” and 1 defined as “Cannot take any steps”. The skills comprising the FAQ-22 cover a variety of activities such as running, jumping, and climbing stairs, using a Likert-like rating system consisting of “easy”, “somewhat difficult”, “very difficult”, “unable to do”, or “too young for activity”. An important feature of the FAQ is that it has been a patient or parent reported measure since its inception. This allows for the patient or parent, who presumably has the extensive knowledge of the patient’s functional ability, to assess the skills. In 2011 Gorton et al. evaluated the FAQ-22 using Rasch analysis [5]. Gorton was able to create a ranking of item difficulties, and, more importantly, was able to show that “the FAQ 22-item skill set is a hierarchical set of interval scaled items suitable for measuring locomotor skill ability in children”.

In 2000, Palisano constructed gross motor function curves for the five GMFCS classes, using the Gross Motor Function Measure (GMFM) as the dependent variable [6]. The concurrent validity of the GMFCS with respect to the GMFM was a compelling reason for the new measure to be so readily embraced. While the GMFM is largely viewed as a gold standard for assessing function, it is time consuming, and requires administration by a trained expert. The implication of the validation study was that these motor function curves allowed clinicians to accurately foretell a child’s functional future; something that would be invaluable to patients, their caregivers, and the clinicians who treat them.

Closer inspection of the data, along with a modest dose of common sense, suggested that function in children with CP is not neatly categorized into five classes, but rather falls along a continuum. In fact, Palisano describes the GMFCS vs. GMFM results as follows (emphasis added for clarity):

Level I represents the continuum between children with neuromuscular and musculoskeletal impairments whose functional limitations are not pronounced and children who have traditionally been diagnosed as having CP of minimal or mild severity. … At the other end of the continuum, children classified at level V have multiple impairments that restrict voluntary control of movement and the ability to maintain antigravity head and trunk postures [6].

Since function clearly lies on a continuum, it is therefore inevitable that many children will not be easily categorized by the GMFCS scheme, but rather will reside somewhere on a murky netherworld between adjacent levels. Recently, in the halls of the clinic, the offices of the gait lab, and the auditoriums of the research conferences, it is common to hear people talking about children who are not easily placed into one of the GMFCS’s five bins. One researcher may describe a child functioning like “…a mild level III”, while another therapist may have treated a child who resembles “level II and a two-thirds”.1

The continuous nature of function and the ever-changing and unpredictable nature of growing children create a technical problem with the GMFCS. Unlike traditional classification schemes, the GMFCS is impermanent. A Setosa never becomes a Versicolor [7]. However, we often observe that a child can shift by one (or rarely, two) GMFCS level over time, and this shift may occur with or without treatment. A retrospective review of 562 patients seen in our center two or more times revealed that 23% of them changed GMFCS levels at least once.

In the original development of the system, it was noted that raters had “concerns about classifying a child’s motor function at a particular level if … the therapist believed the child’s function would eventually be classified at a higher level” [1]. As noted above, this is most likely a reflection of the fact that rather than distinct classes, children diagnosed with CP exist on a changeable continuum reflecting a dizzyingly complex combination of functional abilities and characteristics, with a vexingly unpredictable temporal trajectory not clearly reflected by the motor function curves.

Additional issues make GMFCS use challenging in practice. The GMFCS has a “…particular emphasis on sitting, walking, and wheeled mobility” [8]. Gait analysts would generally concur that walking and transfers are important. However, these two categories may not capture all that we want to understand about function and how it is impacted by treatment.

Another difficulty arises from the fact that the GMFCS was historically not self-reported. Even with the advent and validation of self-reporting via the GMFCS Family Report Questionnaire, the GMFCS level is frequently assigned by a clinician who may have only known the child for a few minutes or a few hours [9].

There is a relatively complex age-dependent set of definitions for GMFCS levels. While seasoned raters may have these definitions committed to memory, it seems likely that many assessors have a single, age-independent definition of GMFCS levels in mind.2 Even if the age dependency is memorized and perfectly implemented, distinct cut-points in categorical variables invariably lead to troublesome discontinuities. Consider the 3 year and 364 day old child who can floor sit but may have difficulty with balance when both hands are free to manipulate objects. Suppose this child has clinic visit, then comes back a few days later to pick up the pair of ankle-foot orthoses that were prescribed for her. Does her functional ability really change by an entire level (from II to III) in less than a week? Was it the orthoses that were responsible for this change?

Finally, there is the problem of children whose combination of abilities conflict with the GMFCS definitions. The GMFCS level definitions are neither exclusive nor exhaustive. It is common to observe that a child can walk on his own without using walking aids, but cannot climb stairs without assistance. What is his GMFCS level? This is just one example, but plenty of these children exist, as will be demonstrated later. Raters are instructed to choose the level that “best represents [the] child’s movement abilities”. But what is meant by a best representation? Are these instructions interpreted similarly by all clinicians, or by clinicians and parents? These conflicts lead to the distinct possibility of a mismatch between the description of a GMFCS level and the abilities of the child who is sitting on the examination table. That is precisely the problem that will be solved using the techniques described in this study.

The purpose of this study is to derive a scheme for assigning GMFCS level based on the comprehensive data contained in the patient reported FAQ survey. This approach to assigning functional level addresses several shortcomings of the current GMFCS assignment scheme. The new approach:

  • Leverages the patient’s knowledge of themselves and their abilities.

  • Resolves inherent conflicts in the description of the GMFCS levels.

  • Simplifies the role of age in determining GMFCS level.

  • Provides a uniform and objective basis for assigning GMFCS level.

Section snippets

Data

This was a retrospective study. We queried the database at the Gillette Children's Specialty Healthcare Center for Gait and Motion Analysis for individuals meeting the following criteria

  • Primary diagnosis of CP

  • Age 2–18 years of age

  • GMFCS level I–IV. Note that while level IV indicates primary movement by a means other than walking, a small fraction children classified as level IV are able to walk for short distances with assistance (walker and/or support from a person), and can therefore complete

Description of sample

The retrospective query produced a balanced set of observations reflecting the routine clinical practice of our laboratory (Table 1). There were 2747 exams (1179 female, 1568 male). Overall GMFCS level distribution was 29% level I, 34% level II, 29% level III, and 8% level IV. The average age was 10.0 years. A random selection of 80% of these observations was used for training the classifier, and the remaining 20% for testing. There were no noteworthy differences between the overall, training,

GMFCS levels can be accurately predicted

This study demonstrated that GMFCS levels can be accurately predicted from the patient-reported FAQ survey using a random forest classifier. This approach allows for a uniform, objective, and systematic assignment of GMFCS levels based solely on an array of self-reported functional abilities. The random forest model used to make the prediction had an overall accuracy of 87%, and misclassification by more than one level was exceedingly rare (1.2%). The area under the ROC curves was above 0.96

References (13)

There are more references available in the full text version of this article.

Cited by (0)

View full text