Assessment of Disability in Multiple Sclerosis Using the Kinect-Camera System: A Proof-of-Concept Study

OBJECTIVE: To demonstrate that the Microsoft Kinect depth sensor technique and new image analysis algorithms may be used to discriminate between healthy volunteers (HV) and patients with multiple sclerosis (MS) and categorize patients based on the level of selected motor dysfunctions.

BACKGROUND: Preventing neurological deterioration is the primary goal of long-term treatment in MS. Clinical assessment of impairment and disability is the most important outcome in patient care and therapeutic trials. The validity and reproducibility of clinical assessment can be insufficient, especially across different raters. Therefore, we have developed automatic image analysis algorithms built upon the Kinect depth sensor, which enable motion analysis in an automatic and reproducible fashion using advanced machine learning and computer vision techniques.

DESIGN/METHODS: Kinect was used to capture defined neurologic movements, which represent upper and lower extremity functions, and truncal stability in 86 HV and 72 MS patients. All patients also received a standardized EDSS-assessment. Kinect allows recording of depth video at 30 frames per second. Videos were anonymized and pre-processed and our machine learning technique was applied to automatically extract measurements, relevant for the differentiation between HV and MS patients.

RESULTS: We were able to capture defined neurological movements in HV and MS patients. Data analysis revealed clusters for subgroups of subjects that show similar movement patterns (e.g. jitter, speed of movements). HV and non-affected MS patients could be distinguished from affected MS patients based on these patterns.

CONCLUSIONS: Kinect can be used in conjunction with tailored image analysis algorithms to discriminate HV and non-affected MS patients from those MS patients with motor dysfunction and probably enable categorizing the levels of motor dysfunctions. This technique has the potential to contribute to a more objective quantification of motor abnormalities in MS. Although results are promising, further evaluation and additional movements are needed.

Disclosure: Dr. Souza has received research support from Novartis. Dr. Kamm has received personal compensation for activities with Biogen Idec, Novartis, Bayer Pharmaceuticals Corp., Merck Serono, Genzyme Corp., and Pfizer Inc. Dr. Burggraaff has nothing to disclose. Dr. Tewarie has nothing to disclose. Dr. Glocker has received personal compensation for activities with Microsoft as an employee. Dr. Dorn has received personal compensation for activities with Novartis as an employee. Dr. Vogel has received personal compensation for activities with Novartis as an employee. Dr. Morrison has received personal compensation for activities with Microsoft as an employee. Dr. Sellen has received personal compensation for activities with Microsoft as an employee. Dr. Machacek has received personal compensation for activities with Novartis as an employee. Dr. Chin has received personal compensation for activities with Novartis as an employee. Dr. Chin holds stock and/or stock options in Novartis, which sponsored research in which Dr. Chin was involved as an investigator. Dr. Uitdehaag has received personal compensation for activities with Novartis and Merck Serono as a consultant. Dr. Uitdehaag has received personal compensation in an editorial capacity for Ariez Medical Publishing. Dr. Uitdehaag has received research support from Biogen Idec, Bayer Schering Pharma, GlaxoSmithKline, Inc., Novartis, Merck Serono and Teva Neuroscience. Dr. Criminisi has received personal compensation for activities with Microsoft as an employee. Dr. Dahlke has received personal compensation for activities with Novartis as an employee. Dr. Polman has received personal compensation for activities with Biogen Idec, Schering AG, Teva Neuroscience, EMD Serono, Novartis, GlaxoSmithKline, Inc., UCB Pharma, AstraZeneca Pharmaceuticals, Roche Diagnostics Inc., Actelion, and Antisense Therapeutics as a consultant. Dr. Polman has received research support from Biogen Idec, Schering AG, GlaxoSmithKline, Inc., Novartis, EMD Serono and Teva Neuroscience. Dr. Kappos has received personal compensation for activities with the University Hospital Basel. Dr. Kappos has received research support from the Swiss MS Society, Swiss National Research Foundation, the European Union, Gianni Rubatto Foundation, Novartis, and Roche Diagnostics Corp.