Psychiatric symptom recognition without labeled data using distributional representations of phrases and on-line knowledge
Yaoyun Zhang, Olivia Zhang, Yonghui Wu, Hee-Jin Lee, Jun Xu, Hua Xu, and Kirk Roberts. 2017. “Psychiatric symptom recognition without labeled data using distributional representations of phrases and on-line knowledge.” J Biomed Inform, 75S, Pp. S129-S137.Abstract
OBJECTIVE: Mental health is becoming an increasingly important topic in healthcare. Psychiatric symptoms, which consist of subjective descriptions of the patient's experience, as well as the nature and severity of mental disorders, are critical to support the phenotypic classification for personalized prevention, diagnosis, and intervention of mental disorders. However, few automated approaches have been proposed to extract psychiatric symptoms from clinical text, mainly due to (a) the lack of annotated corpora, which are time-consuming and costly to build, and (b) the inherent linguistic difficulties that symptoms present as they are not well-defined clinical concepts like diseases. The goal of this study is to investigate techniques for recognizing psychiatric symptoms in clinical text without labeled data. Instead, external knowledge in the form of publicly available "seed" lists of symptoms is leveraged using unsupervised distributional representations. MATERIALS AND METHODS: First, psychiatric symptoms are collected from three online repositories of healthcare knowledge for consumers-MedlinePlus, Mayo Clinic, and the American Psychiatric Association-for use as seed terms. Candidate symptoms in psychiatric notes are automatically extracted using phrasal syntax patterns. In particular, the 2016 CEGS N-GRID challenge data serves as the psychiatric note corpus. Second, three corpora-psychiatric notes, psychiatric forum data, and MIMIC II-are adopted to generate distributional representations with paragraph2vec. Finally, semantic similarity between the distributional representations of the seed symptoms and candidate symptoms is calculated to assess the relevance of a phrase. Experiments were performed on a set of psychiatric notes from the CEGS N-GRID 2016 Challenge. RESULTS & CONCLUSION: Our method demonstrates good performance at extracting symptoms from an unseen corpus, including symptoms with no word overlap with the provided seed terms. Semantic similarity based on the distributional representation outperformed baseline methods. Our experiment yielded two interesting results. First, distributional representations built from social media data outperformed those built from clinical data. And second, the distributional representation model built from sentences resulted in better representations of phrases than the model built from phrase alone.
Symptom severity classification with gradient tree boosting
Yang Liu, Yu Gu, John Chu Nguyen, Haodan Li, Jiawei Zhang, Yuan Gao, and Yang Huang. 2017. “Symptom severity classification with gradient tree boosting.” J Biomed Inform, 75S, Pp. S105-S111.Abstract
In this paper, we present our system as submitted in the CEGS N-GRID 2016 task 2 RDoC classification competition. The task was to determine symptom severity (0-3) in a domain for a patient based on the text provided in his/her initial psychiatric evaluation. We first preprocessed the psychiatry notes into a semi-structured questionnaire and transformed the short answers into either numerical, binary, or categorical features. We further trained weak Support Vector Regressors (SVR) for each verbose answer and combined regressors' output with other features to feed into the final gradient tree boosting classifier with resampling of individual notes. Our best submission achieved a macro-averaged Mean Absolute Error of 0.439, which translates to a normalized score of 81.75%.
Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 CEGS N-GRID shared tasks Track 2
Michele Filannino, Amber Stubbs, and Özlem Uzuner. 2017. “Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 CEGS N-GRID shared tasks Track 2.” J Biomed Inform, 75S, Pp. S62-S70.Abstract
The second track of the CEGS N-GRID 2016 natural language processing shared tasks focused on predicting symptom severity from neuropsychiatric clinical records. For the first time, initial psychiatric evaluation records have been collected, de-identified, annotated and shared with the scientific community. One-hundred-ten researchers organized in twenty-four teams participated in this track and submitted sixty-five system runs for evaluation. The top ten teams each achieved an inverse normalized macro-averaged mean absolute error score over 0.80. The top performing system employed an ensemble of six different machine learning-based classifiers to achieve a score 0.86. The task resulted to be generally easy with the exception of two specific classes of records: records with very few but crucial positive valence signals, and records describing patients predominantly affected by negative rather than positive valence. Those cases proved to be very challenging for most of the systems. Further research is required to consider the task solved. Overall, the results of this track demonstrate the effectiveness of data-driven approaches to the task of symptom severity classification.
The UAB Informatics Institute and 2016 CEGS N-GRID de-identification shared task challenge
Duy Duc An Bui, Mathew Wyatt, and James J Cimino. 2017. “The UAB Informatics Institute and 2016 CEGS N-GRID de-identification shared task challenge.” J Biomed Inform, 75S, Pp. S54-S61.Abstract
Clinical narratives (the text notes found in patients' medical records) are important information sources for secondary use in research. However, in order to protect patient privacy, they must be de-identified prior to use. Manual de-identification is considered to be the gold standard approach but is tedious, expensive, slow, and impractical for use with large-scale clinical data. Automated or semi-automated de-identification using computer algorithms is a potentially promising alternative. The Informatics Institute of the University of Alabama at Birmingham is applying de-identification to clinical data drawn from the UAB hospital's electronic medical records system before releasing them for research. We participated in a shared task challenge by the Centers of Excellence in Genomic Science (CEGS) Neuropsychiatric Genome-Scale and RDoC Individualized Domains (N-GRID) at the de-identification regular track to gain experience developing our own automatic de-identification tool. We focused on the popular and successful methods from previous challenges: rule-based, dictionary-matching, and machine-learning approaches. We also explored new techniques such as disambiguation rules, term ambiguity measurement, and used multi-pass sieve framework at a micro level. For the challenge's primary measure (strict entity), our submissions achieved competitive results (f-measures: 87.3%, 87.1%, and 86.7%). For our preferred measure (binary token HIPAA), our submissions achieved superior results (f-measures: 93.7%, 93.6%, and 93%). With those encouraging results, we gain the confidence to improve and use the tool for the real de-identification task at the UAB Informatics Institute.
Validation of a risk stratification tool for fall-related injury in a state-wide cohort
Thomas H McCoy, Victor M Castro, Andrew Cagan, Ashlee M Roberson, and Roy H Perlis. 2017. “Validation of a risk stratification tool for fall-related injury in a state-wide cohort.” BMJ Open, 7, 2, Pp. e012189.Abstract
OBJECTIVE: A major preventable contributor to healthcare costs among older individuals is fall-related injury. We sought to validate a tool to stratify such risk based on readily available clinical data, including projected medication adverse effects, using state-wide medical claims data. DESIGN: Sociodemographic and clinical features were drawn from health claims paid in the state of Massachusetts for individuals aged 35-65 with a hospital admission for a period spanning January-December 2012. Previously developed logistic regression models of hospital readmission for fall-related injury were refit in a testing set including a randomly selected 70% of individuals, and examined in a training set comprised of the remaining 30%. Medications at admission were summarised based on reported adverse effect frequencies in published medication labelling. SETTING: The Massachusetts health system. PARTICIPANTS: A total of 68 764 hospitalised individuals aged 35-65 years. PRIMARY MEASURES: Hospital readmission for fall-related injury defined by claims code. RESULTS: A total of 2052 individuals (3.0%) were hospitalised for fall-related injury within 90 days of discharge, and 3391 (4.9%) within 180 days. After recalibrating the model in a training data set comprised of 48 136 individuals (70%), model discrimination in the remaining 30% test set yielded an area under the receiver operating characteristic curve (AUC) of 0.74 (95% CI 0.72 to 0.76). AUCs were similar across age decades (0.71 to 0.78) and sex (0.72 male, 0.76 female), and across most common diagnostic categories other than psychiatry. For individuals in the highest risk quartile, 11.4% experienced fall within 180 days versus 1.2% in the lowest risk quartile; 57.6% of falls occurred in the highest risk quartile. CONCLUSIONS: This analysis of state-wide claims data demonstrates the feasibility of predicting fall-related injury requiring hospitalisation using readily available sociodemographic and clinical details. This translatable approach to stratification allows for identification of high-risk individuals in whom interventions are likely to be cost-effective.