Iterative Development and Evaluation of a Pharmacogenomic-Guided Clinical Decision Support System for Warfarin Dosing

Brittany L. Melton; Alan J. Zillich; Jason J. Saleem; Alissa L. Russ; James E. Tisdale; Brian R. Overholser

doi:10.4338/ACI-2016-05-RA-0081

Applied Clinical Informatics, Table of Contents

Appl Clin Inform 2016; 07(04): 1088-1106
DOI: 10.4338/ACI-2016-05-RA-0081

Research Article

Schattauer GmbH

Iterative Development and Evaluation of a Pharmacogenomic-Guided Clinical Decision Support System for Warfarin Dosing

Brittany L. Melton

¹School of Pharmacy, University of Kansas, Lawrence, KS

,

Alan J. Zillich

²College of Pharmacy, Purdue University, West Lafayette, IN

³Center for Health Information and Communication, Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development

⁴Regenstrief Institute, Inc

⁵Indiana University Center for Health Services and Outcomes Research

,

Jason J. Saleem

⁶Department of Industrial Engineering, University of Louisville, Louisville, KY

,

Alissa L. Russ

²College of Pharmacy, Purdue University, West Lafayette, IN

³Center for Health Information and Communication, Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development

⁴Regenstrief Institute, Inc

⁵Indiana University Center for Health Services and Outcomes Research

,

James E. Tisdale

²College of Pharmacy, Purdue University, West Lafayette, IN

⁷Indiana University School of Medicine, Indianapolis, IN

,

Brian R. Overholser

²College of Pharmacy, Purdue University, West Lafayette, IN

⁷Indiana University School of Medicine, Indianapolis, IN

› Author Affiliations

Abstract

Summary

Objective Pharmacogenomic-guided dosing has the potential to improve patient outcomes but its implementation has been met with clinical challenges. Our objective was to develop and evaluate a clinical decision support system (CDSS) for pharmacogenomic-guided warfarin dosing designed for physicians and pharmacists.

Methods Twelve physicians and pharmacists completed 6 prescribing tasks using simulated patient scenarios in two iterations (development and validation phases) of a newly developed pharmacogenomic-driven CDSS prototype. For each scenario, usability was measured via efficiency, recorded as time to task completion, and participants’ perceived satisfaction which were compared using Kruskal-Wallis and Mann Whitney U tests, respectively. Debrief interviews were conducted and qualitatively analyzed. Usability findings from the first (i.e. development) iteration were incorporated into the CDSS design for the second (i.e. validation) iteration.

Results During the CDSS validation iteration, participants took more time to complete tasks with a median (IQR) of 183 (124–247) seconds versus 101 (73.5–197) seconds in the development iteration (p=0.01). This increase in time on task was due to the increase in time spent in the CDSS corresponding to several design changes. Efficiency differences that were observed between pharmacists and physicians in the development iteration were eliminated in the validation iteration. The increased use of the CDSS corresponded to a greater acceptance of CDSS recommended doses in the validation iteration (4% in the first iteration vs. 37.5% in the second iteration, p<0.001). Overall satisfaction did not change statistically between the iterations but the qualitative analysis revealed greater trust in the second prototype.

Conclusions A pharmacogenomic-guided CDSS has been developed using warfarin as the test drug. The final CDSS prototype was trusted by prescribers and significantly increased the time using the tool and acceptance of the recommended doses. This study is an important step toward incorporating pharmacogenomics into CDSS design for clinical testing.

Citation: Melton BL, Zillich AJ, Saleem JJ, Russ AL, Tisdale JE, Overholser BR. Iterative development and evaluation of a pharmacogenomic-guided clinical decision support system for warfarin dosing.

Keywords

Clinical decision support systems - pharmacogenetics - CPOE - computer-assisted drug therapy - warfarin

Full Text

References

References
1 Food and Drug Administration: Table of pharmacogenomics biomarkers in drug labeling. [cited 2014 Dec 1]. Available from: http://www.fda.gov/drugs/scienceresearch/researchareas/pharmacogenetics/ucm083378.htm
2 Arwood MJ, Chumnumwat S, Cavallari LH, Nutescu EA, Duarte JD. Implementing Pharmacogenomics at Your Institution: Establishment and Overcoming Implementation Challenges. Clin Transl Sci. 2016. doi: 10.1111/cts.12404. [Epub ahead of print].
3 Weitzel KW, Elsey AR, Langaee TY, Burkley B, Nessl DR, Obeng AO, Staley BJ, Dong HJ, Allan RW, Liu JF, Cooper-Dehoff RM, Anderson RD, Conlon M, Clare-Salzler MJ, Nelson DR, Johnson JA. Clinical pharmacogenetics implementation: approaches, successes, and challenges. Am J Med Genet C Semin Med Genet 2014; 166C (01) 56-67.
4 Levy KD, Decker BS, Carpenter JS, Flockhart DA, Dexter PR, Desta Z, Skaar TC. Prerequisites to implementing a pharmacogenomics program in a large health-care system. Clin Pharmacol Ther 2014; 96: 307-309.
5 Gottesman O, Scott SA, Ellis SB, Overby CL, Ludtke A, Hulot JS, Hall J, Chatani K, Myers K, Kannry JL, Bottinger EP. The CLIPMERGE PGx Program: clinical implementation of personalized medicine through electronic health records and genomics-pharmacogenomics. Clin Pharmacol Ther 2013; 94: 214-217.
6 Johnson JA, Elsey AR, Clare-Salzler MJ, Nessl D, Conlon M, Nelson DR. Institutional profile: University of Florida and Shands Hospital Personalized Medicine Program: clinical implementation of pharmacogenetics. Pharmacogenomics 2013; 14: 723-726.
7 O’Donnell PH, Bush A, Spitz J, Danahey K, Saner D, Das S, Cox NJ, Ratain MJ. The 1200 patients project: creating a new medical model system for clinical implementation of pharmacogenomics. Clin Pharmacol Ther 2012; 92: 446-449.
8 Pulley JM, Denny JC, Peterson JF, Bernard GR, Vnencak-Jones CL, Ramirez AH, Delaney JT, Bowton E, Brothers K, Johnson K, Crawford DC, Schildcrout J, Masys DR, Dilks HH, Wilke RA, Clayton EW, Shultz E, Laposata M, McPherson J, Jirjis JN, Roden DM. Operational implementation of prospective genotyping for personalized medicine: the design of the Vanderbilt PREDICT project. Clin Pharmacol Ther 2012; 92: 87-95.
9 Shuldiner AR, Palmer K, Pakyz RE, Alestock TD, Maloney KA, O’Neill C, Bhatty S, Schub J, Overby CL, Horenstein RB, Pollin TI, Kelemen MD, Beitelshees AL, Robinson SW, Blitzer MG, McArdle PF, Brown L, Jeng LJ, Zhao RY, Ambulos N, Vesely MR. Implementation of pharmacogenetics: the University of Maryland Personalized Anti-platelet Pharmacogenetics Program. Am J Med Genet C Semin Med Genet 2014; 166C (01) 76-84.
10 Teich JM, Osheroff JA, Pifer EA, Jenders RA. CDS Expert Review Panel. Clinical decision support in electronic prescribing: recommendations and an action plan: report of the joint clinical decision support workgroup. J Am Med Inform Assoc 2005; 12: 365-376.
11 Kaplan B. Evaluating informatics applications- clinical decision support systems literature review. I Int J Med Inform 2001; 64: 15-37.
12 Devine EB, Lee CJ, Overby CL, Abernethy N, McCune J, Smith JW, Tarczy-Hornoch P. Usability evaluation of pharmacogenomics clinical decision support aids and clinical knowledge resources in a computerized provider order entry system: a mixed methods approach. Int J Med Inform 2014; 83 (07) 473-483.
13 Nishimura AA, Shirts BH, Salama J, Smith JW, Devine B, Tarczy-Hornoch P. Physician perspectives of CYP2C19 and clopidogrel drug-gene interaction active clinical decision support alerts. Int J Med Inform 2016; 86: 117-125.
14 Budnitz DS, Pollock DA, Weidenbach KN, Mendelsohn AB, Schroeder TJ, Annest JL. National surveillance of emergency department visits for outpatient adverse drug events. JAMA 2006; 296: 1858-1866.
15 Food and Drug Administration. FDA clears genetic lab test for warfarin sensitivity. Sep 17, 2007. [cited 2011 Feb 9]. Available from: http://www.fda.gov/newsevents/newsroom/pressannouncements/2007/ucm108984.htm
16 Takeuchi F, McGinnis R, Bourgeois S, Barnes C, Eriksson N, Soranzo N, Whittaker P, Ranganath V, Kumanduri V, McLaren W, Holm L, Lindh J, Rane A, Wadelius M, Deloukas P. A Genome-Wide Association Study Confirms VKORC1, CYP2C9, and CYP4F2 as Principal Genetic Determinants of Warfarin Dose. PLoS Genetics 2009; 05 (03) 1-9.
17 Swen J, Wilting I, de Goede AD, Grandia L, Mulder H, Touw DJ, de Boer A, Conemans JM, Egberts TC, Klungel OH, Koopmans R, van der Weide J, Wilffert B, Guchelaar HJ, Deneer VH. Pharmacogenetics: From Bench to Byte. Clin Pharmacol Ther 2008; 83: 781-787.
18 Food and Drug Administration: FDA approves updated warfarin (Coumadin) prescribing information: new genetic information may help providers improve initial dosing estimates of the anticoagulant for individual patients. [cited 2011 Feb 9]. Available from: http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/2007/ucm108967.htm
19 Baudhuin LM, Langman LJ, O’Kane DJ. Translation of pharmacogenetics into clinically relevant testing modalities. Clin Pharmacol Ther 2007; 82: 373-376.
20 Nutescu EA, Drozda K, Bress AP, Galanter WL, Stevenson J, Stamos TD, Desai AA, Duarte JD, Gordeuk V, Peace D, Kadkol SS, Dodge C, Saraf S, Garofalo J, Krishnan JA, Garcia JG, Cavallari LH. Feasibility of implementing a comprehensive warfarin pharmacogenetics service. Pharmacotherapy 2013; 33: 1156-1164.
21 Pirmohamed M, Burnside G, Eriksson N, Jorgensen AL, Toh CH, Nicholson T, Kesteven P, Christersson C, Wahlström B, Stafberg C, Zhang JE, Leathart JB, Kohnke H, Maitland-van der Zee AH, Williamson PR, Daly AK, Avery P, Kamali F, Wadelius M. EU-PACT Group. A randomized trial of genotype-guided dosing of warfarin. N Engl J Med 2013; 369 (24) 2294-2303.
22 Kimmel SE, French B, Kasner SE, Johnson JA, Anderson JL, Gage BF, Rosenberg YD, Eby CS, Madigan RA, McBane RB, Abdel-Rahman SZ, Stevens SM, Yale S, Mohler 3rd ER, Fang MC, Shah V, Horenstein RB, Limdi NA, Muldowney 3rd JA, Gujral J, Delafontaine P, Desnick RJ, Ortel TL, Billett HH, Pendleton RC, Geller NL, Halperin JL, Goldhaber SZ, Caldwell MD, Califf RM, Ellenberg JH. COAG Investigators. A pharmacogenetic versus a clinical algorithm for warfarin dosing. N Engl J Med 2013; 369 (24) 2283-2293.
23 Ahmadian L, van Engen-Verheul M, Bakhshi-Raiez F, Peek N, Cornet R, de Keizer NF. The role of standardized data and terminological systems in computerized clinical decision support systems: Literature review and survey. Int J Med Inform 2011; 80: 81-93.
24 Anderson JL, Horne BD, Stevens SM, Grove AS, Barton S, Nicholas ZP, Kahn SF, May HT, Samuelson KM, Muhlestein JB, Carlquist JF. Couma-Gen Investigators. Randomized trial of genotype-guided versus standard warfarin dosing in patients initiating oral anticoagulation. Circulation 2007; 116: 2563-2570.
25 Manzi SF, Fusaro VA, Chadwick L, Brownstein C, Clinton C, Mandl KD, Wolf WA, Hawkins JB. Creating a scalable clinical pharmacogenomics service with automated interpretation and medical record result integration- experience from a pediatric tertiary care facility. J Am Med Inform Assoc. 2016 Jun 14.
26 Hicks JK, Stowe D, Willner MA, Wai M, Daly T, Gordon SM, Lashner BA, Parikh S, White R, Teng K, Moss T, Erwin A, Chalmers J, Eng C, Knoer S. Implementation of clinical pharmacogenomics within a large health system: from electronic health record decision support to consultation services. Pharmacotherapy. 2016 Jun 17.
27 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76.
28 Russ AL, Weiner M, Russell SA, Baker DA, Fahner WJ, Saleem JJ. Design and implementation of a hospital-based usability laboratory: insights from a Department of Veterans Affairs laboratory for health information technology. Jt Comm J Qual Patient Saf 2012; 38: 531-540.
29 Talmon J, Ammenwerth E, Brender J, de Keizer N, Nykänen P, Rigby M. STARE-HI--Statement on reporting of evaluation studies in Health Informatics. Int J Med Inform 2009; 78: 1-9.
30 Lenzini P, Wadelius M, Kimmel S, Anderson JL, Jorgensen AL, Pirmohamed M, Caldwell MD, Limdi N, Burmester JK, Dowd MB, Angchaisuksiri P, Bass AR, Chen J, Eriksson N, Rane A, Lindh JD, Carlquist JF, Horne BD, Grice G, Milligan PE, Eby C, Shin J, Kim H, Kurnik D, Stein CM, McMillin G, Pendleton RC, Berg RL, Deloukas P, Gage BF. Integration of genetic, clinical, and INR data to refine warfarin dosing. Clin Pharmacol Ther 2010; 87: 572-578.
31 International Warfarin Pharmacogenetics Consortium. Klein TE, Altman RB, Eriksson N, Gage BF, Kimmel SE, Lee MT, Limdi NA, Page D, Roden DM, Wagner MJ, Caldwell MD, Johnson JA. Estimation of the warfarin dose with clinical and pharmacogenetic data. N Engl J Med 2009; 360: 753-64.
32 Nielsen J, Landauer T. A mathematical model of the finding of usability problems. Proceedings of the INTERACT ’93 and CHI ’93 conference on Human factors in computing systems. Amsterdam, The Netherlands: 1993: 206-13.
33 Nielsen J. Ch 2: What is Usability? In: Usability Engineering. San Francisco, CA: Morgan Kaufmann Publishers; 1993: 23-48.
34 Nielsen J. Usability 101: Introduction to Usability. Jan 4, 2012 [cited 2013 Feb 5]; Available from: http://www.nngroup.com/articles/usability-101-introduction-to-usability/
35 Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 1995; 07: 57-78.
36 ECRI Institute. Warfarin therapy management. May 12, 2011 [cited 2013 Aug 9]; Available from: http://www.guideline.gov/content.aspx?id=24513
37 Crews KR, Cross SJ, McCormick JN, Baker DK, Molinelli AR, Mullins R, Relling MV, Hoffman JM. Development and implementation of a pharmacist-managed clinical pharmacogenetics service. Am J Health Syst Pharm 2011; 68: 143-150.
38 Weitzel KW, Alexander M, Bernhardt BA, Calman N, Carey DJ, Cavallari LH, Field JR, Hauser D, Junkins HA, Levin PA, Levy K, Madden EB, Manolio TA, Odgis J, Orlando LA, Pyeritz R, Wu RR, Shuldiner AR, Bottinger EP, Denny JC, Dexter PR, Flockhart DA, Horowitz CR, Johnson JA, Kimmel SE, Levy MA, Pollin TI, Ginsburg GS. IGNITE Network. The IGNITE network: a model for genomic medicine implementation and research. BMC Med Genomics 2016; 09: 1.
39 Kushniruk AW, Santos SL, Pourakis G, Nebeker JR, Boockvar KS. Cognitive analysis of a medication reconciliation tool: applying laboratory and naturalistic approaches to system evaluation. Stud Health Technol Inform 2011; 164: 203-207.
40 Kravitz RL, Neufeld JED, Hogarth MA, Paterniti DA, Dager W, White RH. From insight to implementation: lessons from a multi-site trial of a PDA-based warfarin dose calculator. In: Henriksen K, Battles JB, Marks ES, Lewin DI. editors. Advances in Patient Safety: From Research to Implementation. Rockville, MD: Agency for Healthcare Research and Quality; 2005
41 Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform 2012; 45: 1202-1216.
42 Shaw PB, Donovan JL, Tran MT, Lemon SC, Burgwinkle P, Gore J. Accuracy assessment of pharmacogenetically predictive warfarin dosing algorithms in patients of an academic medical center anticoagulation clinic. J Thromb Thrombolysis 2010; 30: 220-225.
43 Nutescu EA, Drozda K, Bress AP. Feasibility of Implementing a Comprehensive Warfarin Pharmacogenetics Service. Pharmacotherapy 2013; 33 (11) 1156-1164.
44 Nielsen J. How many test users in a usability study?. June 4, 2012 [cited 2013 Aug 6]; Available from: http://www.nngroup.com/articles/how-many-test-users/
45 Barnum C. The ‘magic number 5’ is it enough for web testing. Information Design Journal and Document Design 2003; 11 (2/3): 160-170.
46 Smith-Jackson TL, Wogalter MS. Methods and Procedures in Warnings Research. Handbook of Warnings. In: Wogalter MS. editor. Mahwah, NJ: Lawrence Erlbaum Associates; 2006