A preliminary study of clinical abbreviation disambiguation in real time

Y. Wu, J. C. Denny, S. T. Rosenbloom, R. A. Miller, D. A. Giuse, Min Song, Hua Xu

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Objective: To save time, healthcare providers frequently use abbreviations while authoring clinical documents. Nevertheless, abbreviations that authors deem unambiguous often confuse other readers, including clinicians, patients, and natural language processing (NLP) systems. Most current clinical NLP systems “post-process” notes long after clinicians enter them into electronic health record systems (EHRs). Such post-processing cannot guarantee 100% accuracy in abbreviation identification and disambiguation, since multiple alternative interpretations exist. Methods: Authors describe a prototype system for real-time Clinical Abbreviation Recognition and Disambiguation (rCARD) – i.e., a system that interacts with authors during note generation to verify correct abbreviation senses. The rCARD system design anticipates future integration with webbased clinical documentation systems to improve quality of healthcare records. When clinicians enter documents, rCARD will automatically recognize each abbreviation. For abbreviations with multiple possible senses, rCARD will show a ranked list of possible meanings with the best predicted sense at the top. The prototype application embodies three word sense disambiguation (WSD) methods to predict the correct senses of abbreviations. We then conducted three experments to evaluate rCARD, including 1) a performance evaluation of different WSD methods; 2) a time evaluation of real-time WSD methods; and 3) a user study of typing clinical sentences with abbreviations using rCARD. Results: Using 4,721 sentences containing 25 commonly observed, highly ambiguous clinical abbreviations, our evaluation showed that the best profile-based method implemented in rCARD achieved a reasonable WSD accuracy of 88.8% (comparable to SVM – 89.5%) and the cost of time for the different WSD methods are also acceptable (ranging from 0.630 to 1.649 milliseconds within the same network). The preliminary user study also showed that the extra time costs by rCARD were about 5% of total document entry time and users did not feel a significant delay when using rCARD for clinical document entry. Conclusion: The study indicates that it is feasible to integrate a real-time, NLP-enabled abbreviation recognition and disambiguation module with clinical documentation systems.

Original languageEnglish
Pages (from-to)364-374
Number of pages11
JournalApplied Clinical Informatics
Volume6
Issue number2
DOIs
Publication statusPublished - 2015 Jan 1

Fingerprint

Natural language processing systems
System program documentation
Processing
Costs
Systems analysis
Health
Natural Language Processing
Clinical Studies
Documentation
Costs and Cost Analysis

All Science Journal Classification (ASJC) codes

  • Health Informatics
  • Computer Science Applications
  • Health Information Management

Cite this

Wu, Y., Denny, J. C., Rosenbloom, S. T., Miller, R. A., Giuse, D. A., Song, M., & Xu, H. (2015). A preliminary study of clinical abbreviation disambiguation in real time. Applied Clinical Informatics, 6(2), 364-374. https://doi.org/10.4338/ACI-2014-10-RA-0088
Wu, Y. ; Denny, J. C. ; Rosenbloom, S. T. ; Miller, R. A. ; Giuse, D. A. ; Song, Min ; Xu, Hua. / A preliminary study of clinical abbreviation disambiguation in real time. In: Applied Clinical Informatics. 2015 ; Vol. 6, No. 2. pp. 364-374.
@article{9ce1684d4e894bc69f05ab43119ff41f,
title = "A preliminary study of clinical abbreviation disambiguation in real time",
abstract = "Objective: To save time, healthcare providers frequently use abbreviations while authoring clinical documents. Nevertheless, abbreviations that authors deem unambiguous often confuse other readers, including clinicians, patients, and natural language processing (NLP) systems. Most current clinical NLP systems “post-process” notes long after clinicians enter them into electronic health record systems (EHRs). Such post-processing cannot guarantee 100{\%} accuracy in abbreviation identification and disambiguation, since multiple alternative interpretations exist. Methods: Authors describe a prototype system for real-time Clinical Abbreviation Recognition and Disambiguation (rCARD) – i.e., a system that interacts with authors during note generation to verify correct abbreviation senses. The rCARD system design anticipates future integration with webbased clinical documentation systems to improve quality of healthcare records. When clinicians enter documents, rCARD will automatically recognize each abbreviation. For abbreviations with multiple possible senses, rCARD will show a ranked list of possible meanings with the best predicted sense at the top. The prototype application embodies three word sense disambiguation (WSD) methods to predict the correct senses of abbreviations. We then conducted three experments to evaluate rCARD, including 1) a performance evaluation of different WSD methods; 2) a time evaluation of real-time WSD methods; and 3) a user study of typing clinical sentences with abbreviations using rCARD. Results: Using 4,721 sentences containing 25 commonly observed, highly ambiguous clinical abbreviations, our evaluation showed that the best profile-based method implemented in rCARD achieved a reasonable WSD accuracy of 88.8{\%} (comparable to SVM – 89.5{\%}) and the cost of time for the different WSD methods are also acceptable (ranging from 0.630 to 1.649 milliseconds within the same network). The preliminary user study also showed that the extra time costs by rCARD were about 5{\%} of total document entry time and users did not feel a significant delay when using rCARD for clinical document entry. Conclusion: The study indicates that it is feasible to integrate a real-time, NLP-enabled abbreviation recognition and disambiguation module with clinical documentation systems.",
author = "Y. Wu and Denny, {J. C.} and Rosenbloom, {S. T.} and Miller, {R. A.} and Giuse, {D. A.} and Min Song and Hua Xu",
year = "2015",
month = "1",
day = "1",
doi = "10.4338/ACI-2014-10-RA-0088",
language = "English",
volume = "6",
pages = "364--374",
journal = "Applied Clinical Informatics",
issn = "1869-0327",
publisher = "Schattauer GmbH",
number = "2",

}

Wu, Y, Denny, JC, Rosenbloom, ST, Miller, RA, Giuse, DA, Song, M & Xu, H 2015, 'A preliminary study of clinical abbreviation disambiguation in real time', Applied Clinical Informatics, vol. 6, no. 2, pp. 364-374. https://doi.org/10.4338/ACI-2014-10-RA-0088

A preliminary study of clinical abbreviation disambiguation in real time. / Wu, Y.; Denny, J. C.; Rosenbloom, S. T.; Miller, R. A.; Giuse, D. A.; Song, Min; Xu, Hua.

In: Applied Clinical Informatics, Vol. 6, No. 2, 01.01.2015, p. 364-374.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A preliminary study of clinical abbreviation disambiguation in real time

AU - Wu, Y.

AU - Denny, J. C.

AU - Rosenbloom, S. T.

AU - Miller, R. A.

AU - Giuse, D. A.

AU - Song, Min

AU - Xu, Hua

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Objective: To save time, healthcare providers frequently use abbreviations while authoring clinical documents. Nevertheless, abbreviations that authors deem unambiguous often confuse other readers, including clinicians, patients, and natural language processing (NLP) systems. Most current clinical NLP systems “post-process” notes long after clinicians enter them into electronic health record systems (EHRs). Such post-processing cannot guarantee 100% accuracy in abbreviation identification and disambiguation, since multiple alternative interpretations exist. Methods: Authors describe a prototype system for real-time Clinical Abbreviation Recognition and Disambiguation (rCARD) – i.e., a system that interacts with authors during note generation to verify correct abbreviation senses. The rCARD system design anticipates future integration with webbased clinical documentation systems to improve quality of healthcare records. When clinicians enter documents, rCARD will automatically recognize each abbreviation. For abbreviations with multiple possible senses, rCARD will show a ranked list of possible meanings with the best predicted sense at the top. The prototype application embodies three word sense disambiguation (WSD) methods to predict the correct senses of abbreviations. We then conducted three experments to evaluate rCARD, including 1) a performance evaluation of different WSD methods; 2) a time evaluation of real-time WSD methods; and 3) a user study of typing clinical sentences with abbreviations using rCARD. Results: Using 4,721 sentences containing 25 commonly observed, highly ambiguous clinical abbreviations, our evaluation showed that the best profile-based method implemented in rCARD achieved a reasonable WSD accuracy of 88.8% (comparable to SVM – 89.5%) and the cost of time for the different WSD methods are also acceptable (ranging from 0.630 to 1.649 milliseconds within the same network). The preliminary user study also showed that the extra time costs by rCARD were about 5% of total document entry time and users did not feel a significant delay when using rCARD for clinical document entry. Conclusion: The study indicates that it is feasible to integrate a real-time, NLP-enabled abbreviation recognition and disambiguation module with clinical documentation systems.

AB - Objective: To save time, healthcare providers frequently use abbreviations while authoring clinical documents. Nevertheless, abbreviations that authors deem unambiguous often confuse other readers, including clinicians, patients, and natural language processing (NLP) systems. Most current clinical NLP systems “post-process” notes long after clinicians enter them into electronic health record systems (EHRs). Such post-processing cannot guarantee 100% accuracy in abbreviation identification and disambiguation, since multiple alternative interpretations exist. Methods: Authors describe a prototype system for real-time Clinical Abbreviation Recognition and Disambiguation (rCARD) – i.e., a system that interacts with authors during note generation to verify correct abbreviation senses. The rCARD system design anticipates future integration with webbased clinical documentation systems to improve quality of healthcare records. When clinicians enter documents, rCARD will automatically recognize each abbreviation. For abbreviations with multiple possible senses, rCARD will show a ranked list of possible meanings with the best predicted sense at the top. The prototype application embodies three word sense disambiguation (WSD) methods to predict the correct senses of abbreviations. We then conducted three experments to evaluate rCARD, including 1) a performance evaluation of different WSD methods; 2) a time evaluation of real-time WSD methods; and 3) a user study of typing clinical sentences with abbreviations using rCARD. Results: Using 4,721 sentences containing 25 commonly observed, highly ambiguous clinical abbreviations, our evaluation showed that the best profile-based method implemented in rCARD achieved a reasonable WSD accuracy of 88.8% (comparable to SVM – 89.5%) and the cost of time for the different WSD methods are also acceptable (ranging from 0.630 to 1.649 milliseconds within the same network). The preliminary user study also showed that the extra time costs by rCARD were about 5% of total document entry time and users did not feel a significant delay when using rCARD for clinical document entry. Conclusion: The study indicates that it is feasible to integrate a real-time, NLP-enabled abbreviation recognition and disambiguation module with clinical documentation systems.

UR - http://www.scopus.com/inward/record.url?scp=84930695018&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84930695018&partnerID=8YFLogxK

U2 - 10.4338/ACI-2014-10-RA-0088

DO - 10.4338/ACI-2014-10-RA-0088

M3 - Article

VL - 6

SP - 364

EP - 374

JO - Applied Clinical Informatics

JF - Applied Clinical Informatics

SN - 1869-0327

IS - 2

ER -