Insulin Bolus Calculator in a Pediatric Hospital

Summary

Background: Insulin dosing in hospitalized pediatric patients is challenging and requires dosing to be matched with the specific clinical and nutritional circumstances. We implemented a customized subcutaneous insulin bolus dose calculator tool integrated with the electronic health record to improve patient care. Here we describe this tool, its utilization and safety, and assess user satisfaction and perceptions of the tool.

Methods: Blood glucose results for all patients who received insulin with and without the calculator tool were compared to assess safety. To assess user perceptions and satisfaction, a survey was sent to all identified users who interacted with the tool during the period from May 2015 to the end of November 2015. Survey responses were summarized, mean user satisfaction calculated, and correlation of Likert scale items with overall satisfaction assessed.

Results: Hypoglycemia rates (2.2% and 2.9%, p = 0.17) and severe hypoglycemia rates (0.04% and 0.1%, p = 0.21) were similar for the groups that received insulin with and without the calculator tool. Overall satisfaction for all survey respondents was high (4.05, SD = 0.83). Physicians indicated a slightly higher satisfaction than nurses (4.33 versus 3.94, p = 0.04). User agreement with improvement of quality of care showed the highest correlation with overall satisfaction (r = 0.80, 95% CI 0.7 –0.87).

Conclusion: Implementation of an insulin calculator tool streamlined ordering and administration of insulin in a pediatric academic institution while maintaining patient safety. Users indicated high overall satisfaction with the tool.

Citation: Ateya MB, Aiyagari R, Moran C, Singer K. .:Insulin bolus calculator in a pediatric hospital: Safety and user perceptions. Appl Clin Inform 2017; 8: 529–540 https://doi.org/10.4338/ACI-2016-11-RA-0187

Keywords

Diabetes mellitus - electronic health records and systems - inpatient CPOE - medication administration records - orderset - clinical decision support - satisfaction

1. Background and Significance

Hyperglycemia in hospitalized patients is associated with poor clinical outcomes [[1]]. Intravenous insulin is the main treatment option for critically-ill patients while scheduled subcutaneous insulin is the preferred option for patients outside the intensive care unit (ICU) [[1]]. Inpatient subcutaneous insulin regimens need to be matched with the specific clinical and nutritional circumstances. Subcutaneous insulin regimens typically consist of basal, nutritional, and correction components to dose calculation [[1]]. Nutritional carbohydrate dosing and hyperglycemia correction insulin doses are both described as bolus dose insulin. Calculating, ordering, and administering nutritional and correction subcutaneous insulin doses is challenging in the inpatient setting, due to unpredictable eating patterns and the variable medical status of patients [[1]]. Additionally, hypoglycemia is the most common acute complication in children with type 1 diabetes and careful insulin dose calculation is warranted to prevent it [[2]]. The practice of sliding scale insulin was previously widely used however; this requires consistent carbohydrate intake at mealtimes [[1]]. With the absence of a technological solution to accommodate the complex dose calculation requirements, one possible solution is to order every bolus dose as a one-time order. However, this process is inefficient and increases risk of error. A well-designed computerized order entry system for insulin has the potential to improve accuracy and efficiency of insulin ordering. In our own institution the standard insulin ordering was previously done every time a pediatric patient would request a meal or require correction for hyperglycemia or ketones. These individual orders were not efficient as the patient would have to contact the nurse, the nurse would contact the resident physician, and the resident physician then would confirm the dose and write an order, at which time the nurses would receive this and then administer insulin. Given the long delays from patient request to administration we trialed standing mealtime doses of insulin for a carbohydrate dose and range of blood sugar similar to sliding scale. When satisfaction increased with this we implemented an integrated subcutaneous insulin bolus dose calculator tool that was more similar to a home dosing regimen.

2. Objectives

This integrated and customized subcutaneous insulin bolus dose calculator tool (“insulin calculator tool”) has been put into practice at our pediatric hospital. The value of computerized dose calculators in preventing dose errors and improving pediatric inpatient workflows such as total parenteral nutrition and intravenous fluids has been previously demonstrated [[3], [4]]. Other studies have shown the safety and benefits of insulin bolus dose calculators integrated with insulin pumps [[5]], or integrated with blood glucose meters [[10]]. There is paucity of literature describing inpatient insulin bolus dose calculators that are fully integrated within the electronic health record (EHR). We aim to describe the insulin calculator tool, workflow, and satisfaction of clinical users and their perception of its impact on work efficiency, quality of patient care, and measure its impact on incidence of hypoglycemia to assess safety of its utilization. We believe this evaluation will provide valuable information to clinicians looking to utilize similar workflows and to informatics researchers concerned with designing EHRs.

3. Methods

3.1 Tool and Workflow Description

Within the integrated EHR (Epic, Verona, WI), ordering providers enter orders for insulin lispro detailing dosing parameters including: blood glucose targets for daytime and nighttime, hyperglycemia correction factors for daytime and nighttime, carbohydrate coverage ratios for breakfast, lunch, dinner, bedtime, AM snacks, PM snacks, and ketone correction percentages for large and moderate urine ketones. Nighttime hours are defined as 9 PM to 6 AM. The urine ketone correction percentage is a percentage increase applied to the calculated hyperglycemia correction if ketones are present. The ordering provider may select different percentage corrections based on the amount of ketone present (small, medium, or large). A dosing range for when this order set is acceptable to use is entered by the provider, and nursing is requested to contact physician if calculation, glucose or ketone values are outside of this range. Two separate orders can be placed, one for scheduled dose of mealtime insulin with a default frequency of three times daily before meals (►[Figure 1]), and one on an as needed basis for snacks (►[Figure 2]). To prevent frequent overlapping correction insulin doses, the snack order only provides carbohydrate coverage with no hyperglycemia correction component. This insulin calculator also allows for flexibility of ordering hyperglycemia correction alone or carbohydrate coverage alone. Orders for the insulin calculator tool are part of the institution’s pediatric diabetes order set which contains groupings of orders for medications, labs, and other types of orders frequently needed for admitted pediatric patients with diabetes. Providers can find this order set by going to the ordering activity within the EHR and searching order sets using any part of the order set name or its synonyms. It is important to note that the use of the insulin order with the dose calculator is highly encouraged but is not mandated by the EHR, and providers can choose to follow traditional methods of ordering such as placing orders for a single dose at a time. Patients admitted with insulin pumps were allowed to continue using their home pumps and were not converted to the insulin calculator tool unless medically necessary. In addition, it is recommended within the orderset that providers consult the Pediatric Endocrinology service with any questions on use of the orderset.

Fig. 1 Order composer for scheduled insulin lispro calculator

Fig. 2 Order composer for “as needed for snacks” insulin lispro calculator

The insulin dose calculator is part of the electronic Medication Administration Record (eMAR). In the eMAR form, the nurse enters the most recent point of care (POC) blood glucose result; the most recent urine ketone result (if available), the amount of carbohydrate to be consumed by the patient, and chooses the appropriate meal or time of the day (breakfast, lunch, dinner, or bedtime) (►[Figure 3]). A dose is automatically calculated based on the time of the day and the selected meal. The dose is rounded to the nearest half unit. The nurse administers the calculated dose and accepts the eMAR form and states the location of the injection. If the calculated dose ends up greater than the maximum limit entered by the provider, a warning message is displayed to the nurse necessitating communication with the provider in case the dosing parameter or dose range of the order requires modification. Providers should make sure there is only one active scheduled insulin calculator order and only one active snacks insulin calculator order to avoid having duplicate parameters. A duplicate medication warning is displayed to providers if they try to place a second duplicate order, and the calculator on the eMAR displays a message to the nurse to contact provider to reconcile the duplicate parameters.

Fig. 3 Insulin calculator on the eMAR

A flowsheet-style report in the patient summary activity is used by clinicians to view administered doses of all insulin types, blood glucose results, urine ketone results, insulin dosing parameters associated with an administered dose, IV fluids with dextrose, and carbohydrate intake (►[Figure 4]).

Fig. 4 Flowsheet-style hyperglycemia and insulin monitoring report

3.2 Tool Development

A multidisciplinary team designed the insulin calculator tool and workflow including: two pediatric endocrinologists, pediatric medication safety officer, clinical informatics pharmacist, and clinical nurse educator. The team followed an agile development philosophy by going through an iterative process of making small changes, testing, getting feedback, and then applying further changes. Training was provided to nurses and physicians through information sessions, tip sheets and interactive e-learning videos. Hyperlinks to the training materials were added within the order set where providers placed the orders. The tool went live initially June 2014. In the first version of the tool, entry of insulin dosing parameters was separate from the insulin order itself, and the parameters were not clearly visible on the eMAR. These issues were viewed by users as barriers and prompted the same group to reconvene and make changes to improve the tool’s usability. The changes addressed the need for usability improvement but did not affect the background calculations. The revised version of the tool went live by the end of April 2015.

3.3 Safety Assessment

One concern for creation of a standing orderset was concern for safety as residents would have less frequent contact by nursing staff. Hence, blood glucose results were retrospectively compared for patients who received insulin lispro orders through the insulin calculator tool (“insulin calculator group”) versus those for patients who received their insulin lispro orders without using the tool (“insulin traditional ordering group”). Blood glucose results between the initial implementation of the tool on 06/07/2014 until performing this analysis on 07/31/2016 were included for both groups. The assessment included only patients under the age of 21 years who required insulin during an inpatient hospital encounter regardless of their diagnosis. Encounters in the emergency department, operating room, and procedural areas were excluded. Blood glucose results obtained before the administration of the first dose of insulin or 6 hours after the last administration of insulin were excluded from the analysis. Patient baseline characteristics including age, gender, and length of stay were compared. Rates of hypoglycemia (blood glucose < 70 mg/dl) and severe hypoglycemia (blood glucose < 40 mg/dl) were compared. To determine statistical differences, Student t-test was used for age, Wilcoxon rank sum test for length of stay and number of insulin bolus doses administered, X² test for gender, and Fischer exact test for rates of hypoglycemia and severe hypoglycemia. All statistical analyses were conducted with R stats package [[8]]. We also conducted a search of the hospital’s patient safety self-reporting system for any adverse events or near-miss events related to the insulin calculator tool during the study period.

3.4 Survey Instrument

A survey was developed based on a validated survey used by a previous study that evaluated user satisfaction with implementation of physician order entry [[7]]. We customized and modified the survey based on input from various users at our institution. Our survey was designed to obtain information about respondents, assess user satisfaction, and obtain suggestions for future improvements of the insulin calculator tool (supplementary online ►Appendix A). The survey collected information about users’ profession, type of interaction with the tool, level of training (for resident physicians), primary work location (for nurses), and level of comfort with patient education about insulin dosing (for nurses). Users were asked to rate their level of satisfaction with the calculator’s ease of use, adequacy of training, reliability, efficiency, reduction of errors, quality of patient care, reduction in the need to page ordering providers, and overall satisfaction using a five-point Likert scale where 1 indicates strong disagreement, and 5 indicates strong agreement with the survey item. The last part of the survey asked for both negative and positive aspects of the tool, as well as suggestions for future improvements. The survey was created using Qualtrics software (Qualtrics, Provo, UT).

All insulin calculator orders from 05/1/2015 to 11/30/2015 were extracted from the EHR. Names of ordering, verifying, and administering users were obtained from the data extraction. The survey was distributed via email to those identified users. The distribution email contained a unique survey link to each user. With the unique survey link, users could only complete the survey once. Email reminders were sent to users who did not complete the survey. The survey was closed on 12/31/2015. Points were summed for all items and an average total score calculated. Difference in user satisfaction based on profession was assessed using a two-tailed Student t-test. Correlation of the different items of the user satisfaction question with the overall satisfaction item was assessed using Pearson’s product-moment correlation. All analyses were conducted using R software stats and Likert packages [[8],[9]].Responses to open-ended questions were tagged with topics to summarize them into fewer categories and identify positive and negative aspects with the highest frequency.

4. Results

4.1 Utilization and Safety Assessment

During the study period, patients in the insulin calculator group received 4292 bolus doses during 349 hospital admission encounters (mean = 12.3, median = 7), while patients in the traditional insulin ordering group received 1196 doses during 117 hospital admission encounters (mean = 10.2, median = 6). The difference in number of doses administered per encounter was statistically significant (p = 0.01). 231 of the 4292 bolus doses administered in the insulin calculator group were orders placed by providers outside the insulin calculator tool. The percentage of encounters with at least one episode of hypoglycemia (< 70 mg/dl) was (19% and 11%, p = 0.29) and with at least one episode of severe hypoglycemia (< 40 mg/dl) was (0.9% and 1.7%, p = 0.3). Overall hypoglycemia rates were (2.2% and 2.9%, p = 0.17) and severe hypoglycemia rates were (0.04% and 0.1%, p = 0.21) for the insulin calculator group and traditional ordering group respectively. There was no significant difference in age, gender, or length of stay between the two groups (►[Table 1]). Four events were reported in the hospital patient safety self-reporting system. Two reports were related to mismatch between the patient’s home insulin dosing parameters prior to admission and the parameters entered by the provider into the calculator, one report indicated that the provider entered the parameters incorrectly due to lack of familiarity with the tool, and one related to a nurse who initially did not administer the full dose estimated by the calculator and had to administer the rest of the dose shortly after. None of these reports were associated with any patient harm and prompted education and workflow clarification.

Table 1

Baseline characteristics and hypoglycemia results

	Insulin ordering with the calculator	Insulin ordering without the calculator	p-value [*]
Count of encounters, n	349	117
Age (years), mean ± SD[a]	12.94 ± 4.3	12.78 ± 4.9	0.76
Length of Stay (Days), median (IQR[†])	2.2 (1.8 – 3.8)	2.7 (1.7 – 4.5)	0.65
Female, n (%)	182 (52%)	66 (56%)	0.42
Total number of bolus doses administered	4292	1196
Number of bolus doses administered per encounter, median (IQR[†])	7 (4 – 11)	6 (3 – 9)	0.01
Hypoglycemia ( < 70 mg/dl), %	2.2	2.9	0.17
Encounters with at least one episode of hypoglycemia, n (%)	66 (19%)	13 (11%)	0.29
Severe Hypoglycemia ( < 40 mg/dl), %	0.04	0.1	0.21
Encounters with at least one episode of severe hypoglycemia, n (%)	3 (0.9%)	2 (1.7%)	0.3

* Student t-test for age, Wilcoxon rank sum test for length of stay and number of doses administered per encounter, X 2 test for gender, and Fischer Exact test for hypoglycemia and severe hypoglycemia. a SD: Standard deviation † IQR: Intra-quartile range.

4.2 Survey Respondents Information

The electronic survey was sent to the 272 users described above (179 nurse, 61 resident/fellow physicians, 29 pharmacists, 2 attending physicians, and 1 physician assistant). A total of 83 respondents completed the survey corresponding to a total response rate of 30.5%. Of the 83 completed surveys, 52 were from nurses, 24 from physician residents, 3 from pharmacists, 2 from physician fellows, and 2 from attending physicians. 23 users indicated that they used the insulin calculator tool to enter new orders, 20 users used it to modify existing orders, 55 users used it to calculate and document insulin dose on the MAR, 13 users used it to verify insulin orders, and 7 users indicated that they have never used the tool. Users who indicated that they have never used the tool were automatically taken to the end of the survey and were not presented with the rest of the questions.

4.3 User Satisfaction

76 respondents completed the survey section for assessing user satisfaction. Mean overall satisfaction score for all respondents was 4.05 (SD = 0.83). ►[Figure 5] lists mean user agreement with the ease of use, feeling adequately trained, reliability, reduction of dose calculation errors, reduction of timing to administration, improvement of quality of patient care, and reduced need for paging items. The physicians indicated slightly more overall satisfaction than the nurses (mean for physicians = 4.33, mean for nurses = 3.94, p=0.04). Satisfaction with reduction of pages was higher in the physician group compared to nursing group (mean for physicians = 4.54, mean for nurses = 4, p = 0.01) (►[Figure 6]).

Fig. 5 Summary of satisfaction ratings for all respondents

Fig. 6 Nurses and physicians mean satisfaction ratings (* = p-value < 0.05)

The item measuring overall satisfaction was positively correlated with the other seven items to varying degrees. Improvement of quality of patient care showed the highest correlation, followed by ease of use, reliability, reduction of dose calculation errors, reduction of time needed to administer insulin, and reduced need for paging. Feeling adequately trained was only fairly positively correlated with overall satisfaction (►[Table 2]).

Table 2

Correlates of overall satisfaction with the insulin calculator tool

Survey item	Correlation with Overall Satisfaction (p-value)
Improvement of quality of patient care	0.80 (<0.0001)
Ease of use	0.73 (<0.0001)
Reliability	0.67 (<0.0001)
Reduction of dose calculation errors	0.65 (<0.0001)
Reduction of time needed to administer dose	0.61 (<0.0001)
Reduction of need to page providers	0.58 (<0.0001)
Adequate training	0.33 (< 0.01)

►[Table 3] summarizes the top answers to question 4, 5, and 6. The most frequent responses to the open-ended question about most negative aspects(s) of the tool (question 4) were “concern for error in order entry” and “lack of training”. The most frequent responses to the open-ended question about most positive aspects(s) of the tool (question 5) were “faster administration” and “reduction of paging”. The most frequent responses to the open-ended question about one thing users would like to change to improve the tool (question 6) were “better education” and “make the order easier to find”.

Table 3

Most common responses to open-ended survey questions

Most common responses to open-ended questions
Negative Aspects
Concern for error in entering dose parameters
Lack of training
Difficult to find
Must scan med to use the calculator on the MAR
Less communication with providers
Hard to use for complex regimens
Confusing order prompts
Positive Aspects
Faster administration
Reduce paging
Easy to use
Streamlined
Nurse independence
Visible dosing parameters
Reduce error
User friendly
Accurate
Suggestions for improvement
Better education
Make it easier to find
More clear wording
Display algorithm in the order

5. Discussion

The insulin calculator tool implemented at our children’s hospital streamlined the ordering and administration of insulin without increasing rates of hypoglycemia or severe hypoglycemia. The majority of patients received their insulin through the tool although some providers elected to order insulin as one-time order every time the patient required a dose since using the tool was optional. The details of the tool configuration were made available to other hospitals on the vendor’s customer portal. At the time of submission of this article, eleven hospitals had implemented a version of the calculator.

Overall satisfaction of survey respondents with the tool was high. Users ranked the tool high on ease of use, reliability, patient care quality improvement, reduction of the need to page ordering providers, reduction of dose calculation errors, and reduction of time needed to administer insulin when a dose is due. Users were less satisfied with the training they received compared with the other items mentioned above. This lower satisfaction with training occurred even with the several methods of communication and training that were used. This could be a manifestation of a general fatigue with trainings given the simultaneous new EHR platform rollout. Mean overall satisfaction was higher for physicians compared with nurses. This may be attributed to that physicians and nurses use different parts of the tool, physicians use the tool to place an order with parameters, then nurses use the tool on the eMAR to calculate and document insulin doses. Several studies of user satisfaction with computerized-order entry systems (CPOE) found differences in satisfaction and perception of nurses compared with physicians [[7], [10]]. This emphasizes that different types of users have different perspectives and should be involved in the design of new tools and workflows. In our study, the perception of improvement of quality of patient care had the strongest correlation with overall satisfaction. This later finding is in contrast to the study by Lee et al. which found stronger correlation with overall satisfaction for items related to system performance compared to items related to perceived quality of patient care [[7]].

We attribute the high user satisfaction to the iterative process of applying improvements to the tool and the involvement of end users including physicians, nurses, and pharmacists in the design of the tool and in the improvement efforts. With each design group meeting, small changes were proposed and configured in the EHR’s proof-of-concept environment, then demonstrated to the group for feedback before applying further changes. This agile style of development of the tool facilitated getting feedback from the appropriate subject matter experts, and increased understanding of problems and the needed design decisions that need to be made to address them.

The open-ended survey questions highlighted the main aspects that users value in the tool and provided valuable insights into possible future improvements. Users listed many positive aspects including: faster administration, reduction of paging, ease of use, automation and streamlining of the insulin workflow, increased nurse independence, transparency and visibility of dosing parameters allowing nurses to compare them with the patient’s home dosing values, accuracy and reduction of errors. Based on survey responses, it is clear that this streamlined insulin calculator tool reduced the time needed to administer an insulin dose to pediatric patients. One of the respondent summarized that saying: “Saves so much time, I hate waiting for the MD to put in an order while the kid stares at their pizza which is getting cold by the minute.”. In addition to considerations related to providing more education and training, users indicated concern about lack of safety checks on the insulin dosing parameters. The insulin calculator tool requires the ordering provider to enter a dose range which runs through a dose-check rule and displays a warning if the range is too high, but there are no rules to check if the entered glucose correction values or the carbohydrate coverage ratios are appropriate. Other negative aspects of the insulin calculator tool mentioned by users included: difficulty to find the order set, that nurses must scan the actual barcode on the insulin medication to access the calculator, that the streamlined workflow decreased direct communication between nurses and providers, and that some of the order prompts are not universally understood by providers, nurses, and parent of patients. Many of these concerns can be subjects for another future round of continuous quality improvement of this tool.

The insulin calculator tool provided a way to customize the insulin regimens for pediatric patients, and allowed easier patient education at discharge, and more transparent comparison between insulin home and hospital regimens. The use of the insulin calculator tool allowed storing insulin dosing parameters as discrete data elements for hospitalized patients. This in turn enables better ability to use such data for patient care decisions, trending, and analysis of treatment outcomes. Although this study focused on using the calculator in the inpatient context, a similar calculator and workflow using the same data elements can be implemented within a mobile application for use by patients and families. There are currently several commercially-available mobile applications and blood glucose monitors that incorporate insulin bolus calculators [[11]]. Integration of those mobile

applications and devices with the EHR and the personal health record (PHR) has the potential to improve insulin management, communication with patients, and transitions of care.

One of the limitations of our study is that the overall response rate was only 30.5% which may bias toward positive results since those who felt it was worth the time to fill out the survey may represent a more enthusiastic group. One possible explanation for the lower than expected response rate is that some of the resident physicians who were surveyed had left the institution due to completion of their training. Another limitation is the that it was difficult to assess the impact of using the tool on hyperglycemia given that patients were admitted for a variety of reasons (new-onset diabetes, resolving DKA, cystic fibrosis exacerbations, illness etc…) which could not be assessed in this retrospective review of the safety of the tool. We did, however, assess the safety of the tool by measuring its impact on incidence of hypoglycemia which is important in the pediatric population [[2]]. We cannot though directly assess if a hypoglycemia event lead to a change in doses in the insulin calculator.

6. Conclusion

We implemented an integrated inpatient subcutaneous insulin bolus dose calculator tool at an academic pediatric institution. The tool was safe to use and achieved high overall user satisfaction due to its perceived reliability, positive impact on ease of use, timeliness of insulin administration, reduction of errors, improved quality of care, and reduction of paging. We found that improved user education and additional safety checks can further increase user satisfaction. Physicians and nurses showed varying degrees of satisfaction and agreement emphasizing that different groups of users have different perspective and should continue to be involved in the design phase of the EHR tools.

Clinical Relevance Statement

An inpatient insulin subcutaneous bolus dose calculator tool can be implemented for hospitalized pediatric patients to streamline insulin ordering. The tool has the potential to increase user and patient satisfaction while maintaining patient safety.

Assessment Questions

1. What parameter should be considered in creating an insulin bolus dose calculator?

• Carbohydrate intake

• HbA1c

• Age

• Sex

Correct answer: (A) Amount of insulin bolus a patient needs varies based on the amount of their carbohydrate intake. HbA1c, age, and sex are important factors but do not directly affect insulin bolus dose calculation.

Conflicts of Interest

The authors declare that they have no competing financial interests.

Acknowledgements

We would like to acknowledge Ming Chen, Cathy Lewis, and Deborah Pasko for their help with the design of the insulin calculator tool. Permission was obtained from Epic for use of the copyrighted screenshots in figures 1, 2, 3, and 4.

Protection of Human and Animal Subjects

The Institutional Review Board determined that this study is exempt from review.

• References

• 1 Moghissi ES, Korytkowski MT, DiNardo M, Einhorn D, Hellman R, Hirsch IB, Inzucchi S, Ismail-Beigi F, Kirkman M, Umpierrez G. American Association of Clinical Endocrinologists and American Diabetes Association consensus statement on inpatient glycemic control. Diabetes Care 2009; 32 (06) 1119-1131.
• 2 Ly TT, Maahs DM, Rewers A, Dunger D, Oduwole A, Jones TW. Assessment and management of hypoglycemia in children and adolescents with diabetes. Pediatr Diabetes 2014; 15 S20 180-192.
• 3 Gross TM, Kayne D, King A, Rother C, Juth S. A Bolus Calculator Is an Effective Means of Controlling Postprandial Glycemia in Patients on Insulin Pump Therapy. Diabetes Technol Ther 2003; 5 (03) 365-369.
• 4 Schmidt S, Meldgaard M, Serifovski N, Storm C, Christensen TM, Gade-Rasmussen B, Nørgaard K. Use of an Automated Bolus Calculator in MDI-Treated Type 1 Diabetes. Diabetes Care 2012; 35 (05) 984-990.
• 5 Lehmann CU, Conner KG, Cox JM. Preventing Provider Errors: Online Total Parenteral Nutrition Calculator. Pediatrics 2004; 113: 748-753.
• 6 Lehmann CU, Kim GR, Gujral R, Veltri MA, Clark JS, Miller MR. Decreasing errors in pediatric continuous intravenous infusions. Pediatr Crit Care Med 2006; 7 (03) 225-230.
• 7 Lee F, Teich JM, Spurr CD, Bates DW. Implementation of Physician Order Entry: User Satisfaction and Self-Reported Usage Patterns. J Am Med Informatics Assoc. The Oxford University Press 1996; 3 (01) 42-55.
• 8 R Core Team (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org
• 9 Jason Bryer and Kimberly Speerschneider (2015). Likert: Functions to Analyze and Visualize Likert Type Items. http://jason.bryer.org/likert . http://github.com/jbryer/likert
• 10 Weiner M, Gress T, Thiemann DR, Jenckes M, Reel SL, Mandell SF, Bass EB. Contrasting Views of Physicians and Nurses about an Inpatient Computer-based Provider Order-entry System. J Am Med Informatics Assoc 1999; 6 (03) 234-244.
• 11 Klonof DC. The Current Status of Bolus Calculator Decision-Support Software. J Diabetes Sci Technol 2012; 6 (05) 990-994.