Please use this identifier to cite or link to this item: https://doi.org/10.2196/medinform.8204
Title: Validation of a natural language processing algorithm for detecting infectious disease symptoms in primary care electronic medical records in Singapore
Authors: Hardjojo A. 
Gunachandran A. 
Pang L. 
Abdullah M.R.B.
Wah W. 
Chong J.W.C. 
Goh E.H. 
Teo S.H.
Lim G.
Lee M.L. 
Hsu W. 
Lee V. 
Chen M.I.-C. 
Wong F.
Phang J.S.K.
Issue Date: 2018
Citation: Hardjojo A., Gunachandran A., Pang L., Abdullah M.R.B., Wah W., Chong J.W.C., Goh E.H., Teo S.H., Lim G., Lee M.L., Hsu W., Lee V., Chen M.I.-C., Wong F., Phang J.S.K. (2018). Validation of a natural language processing algorithm for detecting infectious disease symptoms in primary care electronic medical records in Singapore. Journal of Medical Internet Research 20 (6) : e36. ScholarBank@NUS Repository. https://doi.org/10.2196/medinform.8204
Abstract: Background: Free-text clinical records provide a source of information that complements traditional disease surveillance. To electronically harness these records, they need to be transformed into codified fields by natural language processing algorithms. Objective: The aim of this study was to develop, train, and validate Clinical History Extractor for Syndromic Surveillance (CHESS), an natural language processing algorithm to extract clinical information from free-text primary care records. Methods: CHESS is a keyword-based natural language processing algorithm to extract 48 signs and symptoms suggesting respiratory infections, gastrointestinal infections, constitutional, as well as other signs and symptoms potentially associated with infectious diseases. The algorithm also captured the assertion status (affirmed, negated, or suspected) and symptom duration. Electronic medical records from the National Healthcare Group Polyclinics, a major public sector primary care provider in Singapore, were randomly extracted and manually reviewed by 2 human reviewers, with a third reviewer as the adjudicator. The algorithm was evaluated based on 1680 notes against the human-coded result as the reference standard, with half of the data used for training and the other half for validation. Results: The symptoms most commonly present within the 1680 clinical records at the episode level were those typically present in respiratory infections such as cough (744/7703, 9.66%), sore throat (591/7703, 7.67%), rhinorrhea (552/7703, 7.17%), and fever (928/7703, 12.04%). At the episode level, CHESS had an overall performance of 96.7% precision and 97.6% recall on the training dataset and 96.0% precision and 93.1% recall on the validation dataset. Symptoms suggesting respiratory and gastrointestinal infections were all detected with more than 90% precision and recall. CHESS correctly assigned the assertion status in 97.3%, 97.9%, and 89.8% of affirmed, negated, and suspected signs and symptoms, respectively (97.6% overall accuracy). Symptom episode duration was correctly identified in 81.2% of records with known duration status. Conclusions: We have developed an natural language processing algorithm dubbed CHESS that achieves good performance in extracting signs and symptoms from primary care free-text clinical records. In addition to the presence of symptoms, our algorithm can also accurately distinguish affirmed, negated, and suspected assertion statuses and extract symptom durations. © Antony Hardjojo, Arunan Gunachandran, Long Pang, Mohammed Ridzwan Bin Abdullah, Win Wah, Joash Wen Chen Chong, Ee Hui Goh, Sok Huang Teo, Gilbert Lim, Mong Li Lee, Wynne Hsu, Vernon Lee, Mark I-Cheng Chen, Franco Wong, Jonathan Siung King Phang.
Source Title: Journal of Medical Internet Research
URI: https://scholarbank.nus.edu.sg/handle/10635/174601
ISSN: 1438-8871
DOI: 10.2196/medinform.8204
Appears in Collections:Elements
Staff Publications

Show full item record
Files in This Item:
File Description SizeFormatAccess SettingsVersion 
10_2196_medinform_8204.pdf1.23 MBAdobe PDF

OPEN

PublishedView/Download

SCOPUSTM   
Citations

2
checked on Nov 25, 2020

Page view(s)

28
checked on Nov 20, 2020

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.