Accuracy of Depth-Sensing Recordings in Classifying Expanded Disability Status Scale Subscores of Motor Dysfunction in Patients with Multiple Sclerosis (I8.004)

Objectives: To test the accuracy of 3D-depth-sensing camera recordings analyzed by machine learning in classifying Expanded Disability Status Scale (EDSS) subscores for standardized movements.

Background: Clinical assessment of disability in Multiple Sclerosis (MS) remains the most important outcome in therapeutic trials and is commonly assessed with the EDSS. The EDSS however exhibits high rater variability. The ASSESS MS system is being developed as a non-invasive, consistent and potentially finer grained tool to measure motor dysfunction in MS. ASSESS MS is using advanced machine learning algorithms (MLAs) to analyze 3D-depth-sensor recordings of standardized tests of motor dysfunction.

Methods: Standardized finger-to-nose test (FNT) and truncal-ataxia test (TAT) were recorded in 264 patients with a 3D-depth-sensor camera. Two neurologists scored -severity of dysfunctions using Neurostatus-EDSS subscore definitions (grade 0=normal, 1=signs only, 2=mild, 3=moderate and 4=severe). Resulting scores were used to train a MLA to automatically classify the subscores from 3D-depth-sensor recordings. The agreement between the MLA-classifications and the neurologists’ scorings was analyzed and contrasted with the neurologists’ long-term intra-rater agreement.

Results: The overall agreement between MLA predictions and neurologists’ scorings was 77.3[percnt] for FNT (78.5[percnt] for grade 0, 57.8[percnt] for grade 1, 75.6[percnt] for grade 2 and 97.4[percnt] for grade 3) and 84.1[percnt] for TAT. The overall one-month intra-rater agreement of the neurologists was 76.9[percnt] for FNT (68.1[percnt] for grade 0, 83.2[percnt] for grade 1, 74.3[percnt] for grade 2, 80[percnt] for grade 3 and 100[percnt] for grade 4) and 67.6[percnt] for TAT, while retest agreement of the MLA is always 100[percnt].

Conclusions: Automated classification of motor dysfunction by MLAs reproduced neurologists’ scoring with accuracy similar to the neurologists’ own long-term intra-rater agreement, and allows an acceptable assessment of motor dysfunction that is consistent through time. ASSESS MS may thus improve the evaluation of disability progression in clinical studies and clinical trials.

Disclosure: Dr. D’Souza has received research support from Bayer AG, Teva, and Genzyme as well as research support from the University of Basel. Dr. Burggraaff has received personal compensation for activities with Novartis Pharma AG (Basel, CH) for one research project. Dr. Steinheimer has received personal compensation for activities with Bayer AG. Dr. Kontschieder has received personal compensation for activities with Microsoft Research as an employee. Dr. Morrison has received personal compensation for activities with Microsoft Research as an employee. Dr. Dorn has received personal compensation for activities with Novartis Pharma AG as an employee. Dr. Bulo has received personal compensation for activities with Microsoft Research. Dr. Tewarie has received personal compensation for activities with Novartis Pharma. AG. Dr. Miciunaite has nothing to disclose. Dr. Sellen has received personal compensation for activities with Microsoft Research as an employee. Dr. Criminisi has received personal compensation for activities with Microsoft as an employee. Dr. Dahlke has received personal compensation for activities with Novartis Pharma AG as an employee. CP Kamm has received honoraria for lectures and consulting from Biogen-idec, Novartis, Bayer Schweiz AG, Teva, Merck-Serono, Genzyme and Pfizer. Dr. Uitdehaag has received personal compensation for activities with Biogen Idec, Novartis, EMD Serono, Teva Pharmaceuticals, Genzyme, and Roche. Dr. Kappos9s institution (University Hospital Basel) has received royalty payments from Neurostatus Systems GmbH.