Rebuilding the Standing Prescription Renewal Orders

Keywords

clinical protocols - prescription drugs - drug classes - instance-based mapping - clinical decision support - knowledge management

CME/MOC-II*

*To earn credit, visit AMIA for details.

Background and Significance

Managing prescription renewal requests in the ambulatory care setting is challenging. Prescription renewals occur when a patient's current prescription either expires or runs out of authorized refills. In the United States, prescriptions for noncontrolled substances are valid for up to 1 year; therefore, patients taking chronic medications for more than 1 year need to renew their prescriptions to continue therapy. Patients can request a new prescription from their provider during a visit or have the pharmacy submit a prescription renewal request on their behalf. Responding to these prescription renewal requests requires a significant amount of clinician and clinic staff time.[1] [2] [3] [4] Excluding requests during patient visits, primary care providers handle approximately 12 prescription renewal requests daily, which commonly include multiple medications.[1] Renewal requests require chart review to assess the patient's current and past medications and dosages and to determine that appropriate monitoring for side effects occurred.[1]

In recognition of the important role nurses play in ambulatory care, the Vanderbilt University Medical Center (VUMC) collaborated with the local board of pharmacy to develop and approve standing prescription renewal orders for VUMC's e-prescribing system in 2009. The standing renewal orders allowed nurses to authorize prescription renewal requests when specific conditions were met. These standing renewal orders reduced resources needed to process prescription renewal requests,[5] helped clinicians determine if a renewal request could be approved, and improved patient medication adherence by reducing time to appropriate prescription renewals. Subsequently, many VUMC clinics developed their own clinic-specific standing prescription renewal orders.

Each legacy standing prescription renewal order contained a header with metadata and general rules for the standing order. Rules included the requirements that (1) the medication had to be listed on the patient's active home medication list, (2) nurses could not renew controlled substances, (3) the patient had to have visited the clinic in the past 12 months, (4) the licensed nurse could only renew (not alter the prescription), and (5) medication, dose, route, frequency, quantity, and authorizing prescriber had to remain unchanged. After the general rules, each clinic's standing order then grouped medications authorized for renewal with specific renewal criteria for each group ([Fig. 1]). For example, Clinic A's medication group “proton-pump inhibitors (PPIs),” which included omeprazole, lansoprazole, esomeprazole, etc., had renewal criteria of (1) patient must have visited in the past 12 months, (2) the patient was not taking clopidogrel, and (3) the patient did not have a diagnosis of osteoporosis. Clinic A would also have medication group “asthma medications,” which had a different list of medications and renewal criteria. The clinic's medical director approved each legacy standing renewal order, which had been reviewed and approved by the VUMC's Pharmacy and Therapeutics (P&T) committee.

Fig. 1 Legacy standing prescription renewal order structure.

Maintaining the standing renewal orders quickly became a challenging task. Clinics had dedicated substantial time and resources to develop the standing orders and to obtain time-consuming approval. After the development of legacy standing renewal orders, few resources were allocated for their maintenance and periodic review. As new medications entered the market and old medications were discontinued, the volume of standing orders and medications, along with the lack of resources caused many of the standing orders to become outdated. As clinics were tasked to create and maintain their own standing renewal orders, many inconsistencies across clinics arose, such as the medication group name, listed medications, and the specific renewal criteria. To demonstrate the divergence of standing orders, [Fig. 2] shows that both Clinic A and Clinic B had a medication group “PPIs”. However, the medication group for Clinic B included ranitidine (not a PPI), and Clinic A had more detailed renewal criteria than Clinic B.

Fig. 2 Variation across clinic standing renewal orders.

Additional challenges arose due to the creation of the standing renewal orders as free-text documents that were printed, signed, and scanned into a database as image documents that were not searchable. The lack of coded or structured medications made automated enforcement of the standing orders in the e-prescribing system impossible.

After 5 years of using the clinic-specific standing renewal orders, the P&T committee—overwhelmed by change requests—requested assistance in maintaining standing renewal orders and converting them from a homegrown electronic health record (EHR) to its commercial EHR replacement. Ambulatory clinics considered transforming the standing orders to a new EHR platform a high priority because of the efficiency gains for providers. The transition also offered an opportunity to review and standardize the standing renewal orders, develop a knowledge management process with approval and periodic review, document metadata (e.g., clinical owners, subject-matter experts, review dates), and improve reporting and compliance. The objective of this project was to review and restructure the standing prescription renewal orders into a modular and scalable format for use in the new EHR.

Methods

Legacy Local Standing Orders to a Centralized Standing Order

The P&T committee recommended moving away from clinic-specific standing orders and creating a main enterprise-wide standing renewal order usable for all clinics. The novel central prescription standing renewal order consisted of general rules and several subgroups, each with a medication group, renewal criteria, and clinics for which the subgroups applied ([Fig. 3]). As a result, a single standing prescription renewal order at the VUMC was created to combine all the subgroups (medication groups, criteria, and applicable clinics). Initially, the P&T committee focused on the most common medication renewal groups, which could be renewed in any clinic. As a result, the initial standing order did not include location restrictions.

Fig. 3 Revised prescription standing renewal order structure. ARBs, angiotensin II receptor blockers.

For legacy standing order conversion, pharmacy students manually extracted the metadata (e.g., title, approval date, date document loaded into e-prescribing system, associated clinics), medication group names, medication names (normalized to generic ingredients), and the medication group's renewal criteria from all approved standing prescription renewal orders to a spreadsheet. Initially, data were extracted by the students together with the authors, and then each student was assigned a set of documents with random spot-checking of the extracted data. Medication names were mapped to First Databank (FDB) medication name codes and RxNorm ingredient codes.

Medication Group Comparison

We evaluated the medication groups, their medications, and their renewal criteria across all clinic standing orders to identify reoccurring groups or themes. Comparing the groups lexically was difficult due to the lack of a naming convention (e.g., one clinic named a medication group “PPIs,” while another clinic named the same group of medications lexically differently “gastrointestinal [GI] acid reducers”). Therefore, we did not compare medication groups lexically, but rather compared the medication groups based on the included medication items, which provided a better comparison of intent or semantic meaning than the group name alone.[6] [7] This comparison technique of “instance-based matching” has been used for medication class matching[6] [8] [9] [10] and compares medication groups in a pairwise manner by calculating a similarity or equivalence score (ES) between the two groups using a modified Jaccard coefficient if the groups share at least one common medication.[6] ES values range from 0 to 1, with scores approaching 0 indicating little to no similarity between the medication groups, and scores approaching 1 representing high similarity. Using the example above, “PPI” and “GI acid reducers” when compared had one medication unique to each group (ranitidine, dexlansoprazole) and five shared medications resulting in an ES of 0.65. For our purposes, we selected an ES greater than 0.3 as indicating significant similarity based on prior research.[10] In addition to matching by medication groups, we also analyzed the complete clinic standing orders. All instance-based matching analysis was done with STATA 13 (StataCorp. 2013, College Station, Texas, United States).

Design of New Standing Renewal Order

After identifying common medication groups, we mapped and normalized the medication groups to FDB Enhanced Therapeutic Classifications (ETC). Several classes were joined together to create a medication group and the listed medications were filtered by intended route (e.g., oral, subcutaneous, and inhalation). New medication groups were named using a precise standard naming convention. For example, the medication group of hydroxyzine HCl, hydroxyzine pamoate, and buspirone was called “anxiolytics” in the past. This name neglected that many other medications are used as anxiolytics. Thus, we expanded the name to “Anxiolytic—noncontrolled substance.” Since these medications were restricted to the oral route only, we would modify the name to “Anxiolytic—noncontrolled substance (oral).” For safety reasons, we excluded some medications from medication groups, per the P&T committee request, by composing the group of ETC classes that did not include the medication in question. For example, isotretinoin was not included in the group “Acne therapy (oral),” by not including the retinoids and derivatives ETC class.

When similar classes were unavailable, we created new medication groups using generic medication names (similar to RxNorm ingredients) for select antifungals, antibiotics, and hypoglycemia treatments. Medication groups were instantiated by deriving all dispensable medication information (medication name + dose form + strength, RxNorm semantic clinical drugs, and semantic branded drugs) belonging to the medication subgroup and route. Renewal criteria for each medication group were reviewed and proposed to the P&T committee for approval ([Fig. 4]).

Fig. 4 ARB standing renewal order. ARBs, angiotensin II receptor blockers.

Standing Renewal Order Integrated in Commercial EHR

A subcommittee of P&T met weekly to review and approve the proposed medication groups and renewal criteria. The finalized list was implemented by the HealthIT ambulatory team using clinical decision support tools in the new EHR, which would trigger when a nurse or provider opened a medication renewal request. The tool would display the requested medication, its prescription instructions (or directions for use), and the medication's renewal criteria with a green checkmark for each condition if it was met or a red “x” if the condition was not met ([Fig. 5]). We provided clinicians with a link to a searchable document, which was similar in appearance to the legacy standing orders; however, it was autogenerated each time from querying the standing renewal order content from the EHR data warehouse ([Fig. 4]).

Fig. 5 Example of clinical decision support for standing prescription renewal orders.

Maintenance and Review Process

We established processes for clinics to request new subgroups to be reviewed and approved by the P&T committee. We also developed a process for the P&T subcommittee to perform regular review and maintenance of all standing prescription renewal orders using metadata attached to the subgroup records built in the EHR. Metadata included subgroup subject matter experts, when the subgroup was last reviewed and by whom, next review date, the status of the subgroup (i.e., under review, approved, retired), and any additional notes.

Results

Legacy Standing Orders

Seventy-eight legacy standing prescription renewal orders covered 135 clinics (some standing orders applied to multiple clinics). The oldest standing order was 7 years old and had never been reviewed since approved, and the newest was created in the same month as the analysis was conducted. On average, the time from the last review or creation was 3 years (median of 4 years).

There were 870 distinct medication groups with 971 distinct medications, of which 656 were listed in multiple legacy standing orders. Seventy-five medications could not be mapped to FDB or RxNorm codes, of which 47 had been discontinued and 28 were compounded medications (such as Mary's Magic Mouthwash), ambiguous due to multiple salt forms or dose forms (e.g., “betamethasone,” “multivitamin,” “calcium,” or “oral contraceptives”), or were medical supplies (e.g., oxygen or other durable medical equipment). The standing orders listed seven controlled substances—several since discontinued. Some standing orders included outdated information on unavailable generic alternatives (e.g., stating, “there is no generic for Singulair”) as generics had become available after the standing order was developed. Additionally, there were 12 misspelled medication names.

The 870 medication group class names mainly represented the therapeutic intent of the medications (e.g., “analgesics” or “antiemetics”). Some drug class names described the mechanism of action (e.g., “antihistamines” or “anticoagulants”). Some group names also included the intended route of administration (e.g., “topical antiacne medications” or “inhaled corticosteroids”). Two main groups represented chemical structure (e.g., “steroids” and “vitamins), and there were two names reflecting an anatomical site (e.g., “urinary medications” and “GI medications”). Each medication group listed an average of 12 medications (range, 1–37, median, 11). Some medication groups had nonspecific names (e.g., “miscellaneous,” “supplies,” “unspecified”).

Instance-Based Matching Analysis

Legacy Clinic Standing Orders

When comparing the clinic standing orders, 46 clinic standing orders had (by our definition) significant overlap (ES > 0.3). Three clinic standing orders had perfect overlap with 31 shared medications (“Trauma Clinic,” “General Surgery,” and “Emergency General Surgery”). Two other standing orders overlapped perfectly with 67 shared medications (“Pediatric Allergy and Immunology” and “Allergy Medication”), and two others had near perfect overlap with 112 shared medications and 5 unique (“Internal Medicine” and “HMG-Internal Medicine”). Several other clinic standing orders had close matches, such as “Mohs” with “Franklin Dermatology” (with 83 shared medications and 16 unique) and “Green-Hills Internal Medicine” with “Internal Medicine” and “HMG-Internal Medicine” (with 99 shared medications and 20 unique).

Medication Groups

From 870 distinct medication groups, 839 (96.4%) shared at least one medication with another group. The remaining 31 medication groups, which included 55 medications, were excluded from this analysis; however, they were manually reviewed and seven of the medication groups were included in the proposed final medication group list. There were 696 legacy medication groups with significant overlap (ES > 0.3), which were reviewed and narrowed down to 54 themes and then subdivided based on the proposed renewal criteria into 132 medication groups. The remaining 143 legacy medication groups had poor overlap with others; however, we were able to reduce them to 26 medication groups not previously included.

Design of a New Standing Renewal Order

The 165 proposed medication groups were mapped to FDB ETC classes (Pharmaceutical Subclasses in Epic). Six medication groups with poor matching were created manually. The proposed renewal criteria had 379 distinct legacy refill criteria, which were narrowed by analyzing for common themes to 21 criteria rules. The resulting medication groups and renewal criteria underwent iterative review and approval first by the P&T subcommittee and then the full P&T committee. Of the 165 proposed subgroups, 13 (7.9%) were not approved by the subcommittee and removed (e.g., “decongestants,” “monoamine oxidase inhibitors,” certain antibiotics, “dopamine receptor agonists”). Additionally, 12 subgroups were added after gaps in the proposed list were identified by clinics (e.g., melatonin, supplements including calcium and magnesium, medications to treat allergic anaphylaxis). In total, 164 prescription renewal subgroups were approved for build ([Fig. 6]). A sample list of medication groups is provided in [Appendix A].

Fig. 6 Flow diagram for medication groups. P&T, Pharmacy and Therapeutics Committee; EHR, electronic health record.

Standing Renewal Order Build in Commercial EHR

The build estimate for the standing order decision support implementation in the new EHR was over 480 hours; however, structured and coded format of the subgroups allowed us to import most of our design directly into the commercial EHR after some minor transformations to comply with the import specifications. All 164 subgroups were built prior to the impending go-live of the new EHR in approximately 10 hours. Building the criteria and testing the decision support were also completed in a fraction of the estimated time. Next, we created a Structured Query Language (SQL)-based Crystal report containing the medications used for building the prescription renewal subgroups and composed a searchable document of the new enterprise-wide standing prescription renewal order, which we hosted on our local intranet and available within the EHR with monthly updates.

Maintenance and Review Process

Similar to a new medication formulary request, any new medications, criteria, or subgroup requests are submitted to the P&T committee using a request form that includes the required information to review and add new requests. To assure maintenance, subgroups are continually reviewed. Any time a change is made to a subgroup, the P&T subcommittee reviews the medications and criteria included in the subgroup. If new safety concerns are identified, the P&T committee will address them without waiting for a scheduled review. To track review and approval of the subgroups, we created searchable metadata records associated with each subgroup. Adherence to the standing order is monitored by the P&T committee using reporting tools from the EHR. Using the standing order decision support, nursing staff send approximately 4,300 prescriptions per week in our system (615/day average). Additionally, providers use the standing order decision support for approximately 3,000 prescriptions per week (an average of 428/day).

Discussion

Summary

In this article, we describe a methodology for standardizing and scaling standing prescription renewal orders at an enterprise level. Prescription renewals are a common, time-consuming burden for ambulatory care clinics. The management of standing prescription renewal orders (electronic or on paper) presents many challenges to the enterprise. We redesigned our legacy standing orders to allow nurses to authorize prescription renewals from within the EHR workflow. Using instance-based matching techniques, we were able to identify, standardize, consolidate, and subsequently build 164 medication groups with standardized renewal criteria.

Variations across clinics and practice sites for prescription renewals are common. A study by Guirguis-Blake et al surveyed 11 family medicine residency practices within the University of Washington Family Medicine Residency Network.[11] All sites had a process for handling prescription renewals; however, no two sites had the same procedure, and only four of the sites could identify a written protocol. A written protocol was associated (nonsignificantly) with more frequent chart review prior to authorization and lower likelihood that the renewal request would be forwarded to another provider.

A study by Ferrell et al surveyed five clinics and also found significant variations.[3] Three of the five sites had medical assistants or nurses involved in the decision-making process for authorizing prescription renewals; however, only two sites had a written protocol. The two clinics with protocols were more likely to send requests to the providers for review, possibly due to strict protocols, varying nurse experience levels, or random chance due to small sample size. They found that medical assistants and nurses were capable of making accurate decisions for the majority of renewal decisions and determined that the renewal process needed to be standardized across all clinics and experience levels via protocol.

In contrast, some organizations manage prescription renewals centrally, for example, by pharmacy.[4] [12] [13] Rim et al described a centralized renewal authorization program within an academic health system with two pharmacists and two pharmacy technicians.[4] The program consisted of a collaborative practice agreement that authorized the pharmacists to approve the prescription renewal requests. On average, the program processed 12,000 renewal requests monthly for 10 clinics. Given the number of unique clinics at VUMC exceeds 800 (with more than 30,000 renewal requests monthly), a distributed model of nurses in the individual clinics reviewing and approving the renewal requests represented a more scalable approach. However, our approach required the standing renewal order to be standardized across the many sites to reduce the maintenance burden and the risk of errors.[14] Nevertheless, the methodology we describe for standardizing and scaling standing prescription renewal orders should be applicable to these varied organizational management strategies (i.e., centralized pharmacy-managed vs. distributed clinic-managed).

Benefits of New Standing Renewal Order Design

The centralization and modularization of the VUMC's standing prescription renewal order governance and build has provided standardization and consistency across the enterprise. Our approach decreased the management burden on local clinic staff, improved the ease of standing renewal order maintenance, and has helped to ensure timely review and update of standing order content. The centralized and modular format allows the standing prescription orders to be scalable because they do not need to be built by each of our clinics and can be implemented across the entire enterprise. The modular format is also extensible because the medication groups were built using medication classes maintained by our medication data vendor (FDB), and medication classes are less likely to be added or removed compared with individual medications. Additionally, using a SQL query to generate the list of medications allowed for renewal by standing order reduces the task of maintaining that list of medications and ensures the EHR decision support correlates with the standing order. Furthermore, our implementation facilitated build integration of the standing renewal order directly in a vendor EHR, which supports existing vendor clinical decision support tools. The implementation displays clinical data relevant to the triggered renewal subgroup which we anticipate to decrease nurse chart review time (not measured). Monitoring and reporting of nurse compliance with standing renewal order criteria are vastly improved since all data are now discrete. Furthermore, an automated drug database update-driven process is now in place that indicates when a subgroup must be revised due to medications being added to or removed from the market, or becoming available in generic form. Finally, our approach is valuable in that it is scalable and sharable among health care organizations.

Challenges of New Standing Renewal Order Design

Standardization of standing renewal orders, by definition, means the loss of individualization across the enterprise. A nontechnical challenge we encountered in this process was defining and gaining consensus for acceptable enterprise-level renewal criteria, which took several months for analysis, review, and approval. For some clinics, standardization meant the loss of approved specialty-specific use cases or medications (such as pulmonary antihypertensives). However, we plan to extend the functionality of our implementation in the future to handle these situations and we currently have a process in place for new subgroup requests.

Reconciling varied medication groupings as we moved to a standardized model also presented unique challenges. Our approach was to avoid unnecessary custom medication groupings by using existing higher-level concepts, such as medication classes, provided by our drug database vendor. We used custom groupings sparingly due to their inherent increased maintenance costs.

As with any implementation in a vendor environment, our approach is limited by the expected workflow and specified functionality of the EHR. For example, the standing renewal order rules are triggered when medication renewal requests come from pharmacies (via Surescripts requests), a specific mechanism from the patient portal, or if a specific message-type is created by a clinician within the EHR. Requests originating from other means, such as from phone calls or other mechanisms, could bypass the standing renewal order decision support. Ultimately, we can standardize the standing prescription renewal orders, clinical decision support, and reporting functionality but we cannot change the prescription renewal workflows, ways that clinics receive renewal requests (such as via phone, which would bypass the decision support), or the implementation of certain decision support elements within the EHR.

Next Steps…

Planned future enhancements of our implementation include adding clinic or location specificity and expanded scope for requested medications not currently covered by our existing standing order. We also plan to refine the monitoring and reporting functionality to enhance operational support of standing renewal order compliance. Finally, we hope to work with our EHR vendor to expand the number of workflows in which renewal content can be displayed to the end user and to improve the overall quality of clinical decision support delivered in the prescription renewal workflows.

Conclusion

Managing prescription renewal requests in the ambulatory care setting is a challenge. We describe a methodology for standardizing standing prescription renewal orders and scaling it at the enterprise level by utilizing knowledge management.

Clinical Relevance Statement

By centralizing the clinic-specific standing prescription renewal orders into a single standing order, nurses are provided with consistent and sustainable clinical decision support based on an enterprise-wide standing prescription renewal order. This approach helps reduce the burden of prescription renewals for ambulatory providers.

Multiple Choice Questions

1. Which of the following is a benefit of using a modular and enterprise-wide approach to a prescription standing renewal order?

   • Increased personalization by clinics.

   • Decreased maintenance over time.

   • Supported by all EHR workflows.

   • Easier to gain consensus from clinics.

   Correct Answer: The correct answer is option b. Having the standing order document generated based on EHR content deceases the burden of maintaining the document and keeps the document and EHR build in sync. It is also easier to review and maintain 164 subgroups versus 870.

2. Which of the following methods efficiently compares medication groups based on their membership (representative medications)?

   • Lexical matching.

   • Natural language processing.

   • Instance-based matching.

   • Manual review.

   Correct Answer: The correct answer is option c. Instance-based matching compares medication groups in an automated and pairwise manner calculating a similarity or equivalence score (ES) between the two groups if the groups share at least one medication in common.

3. Which RxNorm term types represent dispensable medications used in clinical decision support?

   • Ingredient.

   • Semantic clinical drug.

   • Clinical drug component.

   • All of the above.

   Correct Answer: The correct answer is option b. Dispensable medications require a medication name, dose form, and strength, similar to the RxNorm semantic clinical drugs, and semantic branded drugs term types. In this project we instantiated our medication groups to a list of dispensable medications for the clinical decision support in the EHR.

4. In this study, how were medications grouped or classified?

   • By frequency.

   • By therapeutic intent.

   • Alphabetically.

   • By shelf life.

   Correct Answer: The correct answer is option b. There are many different ways to group medication, each depending on the use case and area of interest of the terminology or ontology. Some are grouped by therapeutic intent (e.g., “analgesics” or “anti-emetics”), mechanism of action (e.g., “antihistamines” or “anti-coagulants”), chemical structure (e.g., “steroids” and “vitamins”), anatomical site (e.g., “urinary medications” and “GI medications”), or even combinations of the above (e.g., “tricyclic antidepressants”).

• References

• 1 Baron RJ. What's keeping us so busy in primary care? A snapshot from one practice. N Engl J Med 2010; 362 (17) 1632-1636
• 2 Edwards ST, Rubenstein LV, Meredith LS. , et al. Who is responsible for what tasks within primary care: perceived task allocation among primary care providers and interdisciplinary team members. Healthc (Amst) 2015; 3 (03) 142-149
• 3 Ferrell CW, Aspy CB, Mold JW. Management of prescription refills in primary care: an Oklahoma Physicians Resource/Research Network (OKPRN) study. J Am Board Fam Med 2006; 19 (01) 31-38
• 4 Rim MH, Thomas KC, Hatch B, Kelly M, Tyler LS. Development and implementation of a centralized comprehensive refill authorization program in an academic health system. Am J Health Syst Pharm 2018; 75 (03) 132-138
• 5 Shipman SA, Sinsky CA. Expanding primary care capacity by reducing waste and improving the efficiency of care. Health Aff (Millwood) 2013; 32 (11) 1990-1997
• 6 Winnenburg R, Bodenreider O. A framework for assessing the consistency of drug classes across sources. J Biomed Semantics 2014; 5: 30
• 7 Merabti T, Abdoune H, Letord C, Sakji S, Joubert M, Darmoni SJ. Mapping the ATC classification to the UMLS metathesaurus: some pragmatic applications. Stud Health Technol Inform 2011; 166: 206-213
• 8 Mortensen J, Bodenreider O. , eds. Comparing Pharmacologic Classes in NDF-RT and SNOMED CT. Proceedings of the Fourth International Symposium on Semantic Mining in Biomedicine; 2010
• 9 Bahr NJ, Nelson SD, Winnenburg R, Bodenreider O. Eliciting the intension of drug value sets - principles and quality assurance applications. Stud Health Technol Inform 2017; 245: 843-847
• 10 Nelson SD, Parker J, Lario R. , et al. Interoperability of Medication Classification Systems: lessons learned mapping Established Pharmacologic Classes (EPCs) to SNOMED CT. Stud Health Technol Inform 2017; 245: 920-924
• 11 Guirguis-Blake J, Keppel GA, Force RW, Cauffield J, Monger RM, Baldwin LM. Variation in refill protocols and procedures in a family medicine residency network. Fam Med 2012; 44 (08) 564-568
• 12 Billups SJ, Delate T, Newlon C, Schwiesow S, Jahnke R, Nadrash A. Outcomes of a pharmacist-managed medication refill program. J Am Pharm Assoc (2003) 2013; 53 (05) 505-512
• 13 Riege VJ. A patient safety program & research evaluation of U.S. Navy Pharmacy Refill Clinics. In: Henriksen K, Battles JB, Marks ES, Lewin DI. , eds. Advances in Patient Safety: From Research to Implementation (Vol. 1: Research Findings). Advances in Patient Safety. Rockville, MD: Agency for Healthcare Research and Quality (AHRQ); 2005
• 14 Lynch J, Rosen J, Selinger HA, Hickner J. Medication management transactions and errors in family medicine offices: a pilot study. In: Henriksen K, Battles JB, Keyes MA, Grady ML. , eds. Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 4: Technology and Medication Safety). Advances in Patient Safety. Rockville: MD: Agency for Healthcare Research and Quality (AHRQ); 2008