The 9-Item Physician Documentation Quality Instrument (PDQI-9) score is not useful in evaluating EMR (scribe) note quality in Emergency Medicine

Katherine J. Walker; Andrew Wang; William Dunlop; Hamish Rodda; Michael Ben-Meir; Margaret Staples

doi:10.4338/ACI2017050080

Applied Clinical Informatics, Table of Contents

Appl Clin Inform 2017; 08(03): 981-993
DOI: 10.4338/ACI2017050080

Research Article

Schattauer GmbH

The 9-Item Physician Documentation Quality Instrument (PDQI-9) score is not useful in evaluating EMR (scribe) note quality in Emergency Medicine

Katherine J. Walker

¹Emergency Department, Cabrini Hospital, Melbourne, Australia

⁴Monash University, Department of Epidemiology and Preventative Medicine, Melbourne, Australia

,

Andrew Wang

²Cabrini Institute, Melbourne, Australia

³Monash University Faculty of Medicine, Nursing and Health Sciences, Melbourne, Australia

,

William Dunlop

¹Emergency Department, Cabrini Hospital, Melbourne, Australia

,

Hamish Rodda

¹Emergency Department, Cabrini Hospital, Melbourne, Australia

⁵Box Hill Hospital, Melbourne, Australia

,

Michael Ben-Meir

¹Emergency Department, Cabrini Hospital, Melbourne, Australia

,

Margaret Staples

²Cabrini Institute, Melbourne, Australia

⁴Monash University, Department of Epidemiology and Preventative Medicine, Melbourne, Australia

› Author Affiliations

Abstract

Summary

Background: Scribes are assisting Emergency Physicians by writing their electronic clinical notes at the bedside during consultations. They increase physician productivity and improve their working conditions. The quality of Emergency scribe notes is unevaluated and important to determine.

Objective: The primary objective of the study was to determine if the quality of Emergency Department scribe notes was equivalent to physician only notes, using the Physician Documentation Quality Instrument, Nine-item tool (PDQI-9).

Methods: This was a retrospective, observational study comparing 110 scribed to 110 non-scribed Emergency Physician notes written at Cabrini Emergency Department, Australia. Consultations during a randomised controlled trial of scribe/doctor productivity in 2016 were used. Emergency physicians and nurses rated randomly selected, blinded and de-identified notes, 2 raters per note. Comparisons were made between paired scribed and unscribed notes and between raters of each note. Characteristics of individual raters were examined. The ability of the tool to discriminate between good and poor notes was tested.

Results: The PDQI-9 tool has significant issues. Individual items had good internal consistency (Cronbach’s alpha=0.93), but there was very poor agreement between raters (Pearson’s r=0.07, p=0.270). There were substantial differences in PDQI-9 scores allocated by each rater, with some giving typically lower scores than others, F(25,206)=1.93, p=0.007. The tool was unable to distinguish good from poor notes, F(3,34)=1.15, p=0.342. There was no difference in PDQI-9 score between scribed and non-scribed notes.

Conclusions: The PDQI-9 documentation quality tool did not demonstrate reliability or validity in evaluating Emergency Medicine consultation notes. We found no evidence that scribed notes were of poorer quality than non-scribed notes, however Emergency scribe note quality has not yet been determined.

Citation: Walker KJ, Wang A, Dunlop W, Rodda H, Ben-Meir M, Staples M. The 9-Item Physician Documentation Quality Instrument (PDQI-9) score is not useful in evaluating EMR (scribe) note quality in Emergency Medicine. Appl Clin Inform 2017; 8: 981–993 https://doi.org/10.4338/ACI2017052017050080

Keywords

Clinical Documentation and Communications - Scribe - Emergency Medicine - Quality - Safety Culture

Full Text

References

References
1 Tischendorf JJ, Crede S, Herrmann P, Bach N, Bomeke C, Manns MP, Schaefer O, Trautwein C. Encoding of diagnosis by medical documentation assistant or ward physician. Dtsch med Wochenschr 2004; 129 (33) 1731-5.
2 Walker K, Johnson M, Dunlop W, Staples M, Rodda H, Turner I, Ben-Meir M. Feasibility evaluation of a pilot scribe-training program in an Australian emergency department. Aust Health Rev. 2017 doi: 10.1071/AH16188. PubMed PMID: 28355527.
3 Arya R, Salovich DM, Ohman-Strickland P, Merlin MA. Impact of scribes on performance indicators in the emergency department. Academic emergency medicine 2010; 17 (05) 490-4.
4 Bastani A, Shaqiri B, Palomba K, Bananno D, Anderson W. An ED scribe program is able to improve throughput time and patient satisfaction. The American journal of emergency medicine 2014; 32 (05) 399-402.
5 Heaton HA, Nestler DM, Jones DD, Lohse CM, Goyal DG, Kallis JS, Sadosty AT. Impact of scribes on patient throughput in adult and pediatric academic EDs. The American journal of emergency medicine 2016; 34 (10) 1982-5.
6 Heaton HA, Nestler DM, Jones DD, Varghese RS, Lohse CM, Williamson ES, Sadosty AT. Impact of Scribes on Billed Relative Value Units in an Academic Emergency Department. J Emerg Med. 2016 doi: 10.1016/j.jemermed.2016.11.017. PubMed PMID: 27988262.
7 Heaton HA, Nestler DM, Lohse CM, Sadosty AT. Impact of scribes on emergency department patient throughput one year after implementation. The American journal of emergency medicine. 2016 doi: 10.1016/j.ajem.2016.11.017. PubMed PMID: 27856140.
8 Hess JJ, Wallenstein J, Ackerman JD, Akhter M, Ander D, Keadey MT, Capes JP. Scribe Impacts on Provider Experience, Operations, and Teaching in an Academic Emergency Medicine Practice. West J Emerg Med 2015; 16 (05) 602-10.
9 Walker K, Ben-Meir M, O’Mullane P, Phillips D, Staples M. Scribes in an Australian private emergency department: A description of physician productivity. Emergency medicine Australasia 2014; 26 (06) 543-8.
10 Walker KJ, Ben-Meir M, Phillips D, Staples M. Medical scribes in emergency medicine produce financially significant productivity gains for some, but not all emergency physicians. Emergency medicine Australasia 2016; 28 (03) 262-7.
11 Shuaib W, Hilmi J, Caballero J, Rashid I, Stanazai H, Ajanovic A, Moshtaghi A, Amari A, Tawfeek K, Khurana A, Hasabo H, Baqais A, Mattar AA, Gaeta TJ. Impact of a scribe program on patient throughput, physician productivity, and patient satisfaction in a community-based emergency department. Health Informatics Journal. 2017. Epub March 1,. 2017 doi: 10.1177/1460458217692930.
12 Bank AJ, Obetz C, Konrardy A, Khan A, Pillai KM, McKinley BJ, Gage RM, Turnbull MA, Kenney WO. Impact of scribes on patient interaction, productivity, and revenue in a cardiology clinic: a prospective study. ClinicoEconomics and outcomes research 2013; 5 (01) 399-406.
13 Bank AJ, Gage RM. Annual impact of scribes on physician productivity and revenue in a cardiology clinic. ClinicoEconomics and outcomes research 2015; 7: 489-95.
14 Koshy S, Feustel PJ, Hong M, Kogan BA. Scribes in an ambulatory urology practice: patient and physician satisfaction. The Journal of urology 2010; 184 (01) 258-62.
15 Carnes KM, de Riese CS, Werner TW. A Cost-Benefit Analysis of Medical Scribes and Electronic Medical Record System in an Academic Urology Clinic. Urology Practice. 2015 2(3).
16 Misra-Hebert AD, Amah L, Rabovsky A, Morrison S, Cantave M, Hu B, Sinsky CA, Rothberg MB. Medical scribes: How do their notes stack up?. J Fam Pract 2016; 65 (03) 155-9.
17 Yan C, Rose S, Rothberg MB, Mercer MB, Goodman K, Misra-Hebert AD. Physician, Scribe, and Patient Perspectives on Clinical Scribes in Primary Care. J Gen Intern Med 2016; 31 (09) 990-5.
18 Dunlop W, Hegarty L, Staples M, Levinson M, Ben-Meir M, Walker K. Medical scribes have no impact on the patient experience of an emergency department. Emergency Medicine Australasia. 2017 doi: 10.1111/1742–6723.12818.
19 Hess JJ, Akhter M, Ackerman J, Wallenstein J, Ander D, Keadey M, Capes JP. Prospective evaluation of a scribe program’s impacts on provider experience, patient flow, productivity, and teaching in an academic emergency medicine practice. Annals of emergency medicine 2013; 1: S1.
20 Heaton HA, Samuel R, Farrell KJ, Colletti JE. Emergency department scribes: A two-step training program. Annals of emergency medicine 2015; 1: S159.
21 Walker KJ, Dunlop W, Liew D, Staples MP, Johnson M, Ben-Meir M, Rodda HG, Turner I, Phillips D. An economic evaluation of the costs of training a medical scribe to work in Emergency Medicine. Emerg Med J 2016; 33 (12) 865-9.
22 Stetson PD, Morrison FP, Bakken S, Johnson SB, eNote Research T. Preliminary development of the physician documentation quality instrument. J Am Med Inform Assoc 2008; 15 (04) 534-41.
23 Stetson PDB. et al. Assessing Electronic Note Quality using the Physician Documentation Quality Instrument (PDQI-9). Applied Clinical Informatics 2012; 3: 164-74.
24 Combs T. Recognizing the Characteristics of Quality Documentation. J AHIMA 2016; 87 (05) 32-3.
25 Kargul GJ, Wright SM, Knight AM, McNichol MT, Riggio JM. The hybrid progress note: semiautomating daily progress notes to achieve high-quality documentation and improve provider efficiency. Am J Med Qual 2013; 28 (01) 25-32.
26 Burke HB, Hoang A, Becher D, Fontelo P, Liu F, Stephens M, Pangaro LN, Sessums LL, O’Malley P, Baxi NS, Bunt CW, Capaldi VF, Chen JM, Cooper BA, Djuric DA, Hodge JA, Kane S, Magee C, Makary ZR, Mallory RM, Miller T, Saperstein A, Servey J, Gimbel RW. QNOTE: an instrument for measuring the quality of EHR clinical notes. J Am Med Inform Assoc 2014; 21 (05) 910-6.
27 Jamieson T, Ailon J, Chien V, Mourad O. An electronic documentation system improves the quality of admission notes: a randomized trial. J Am Med Inform Assoc 2017; 24 (01) 123-9.
28 Bierman JA, Hufmeyer KK, Liss DT, Weaver AC, Heiman HL. Promoting Responsible Electronic Documentation: Validity Evidence for a Checklist to Assess Progress Notes in the Electronic Health Record. Teach Learn Med 2017; 12: 1-13.