Improving the Observed-to-Expected Mortality Ratio with the Combination of Standardized Documentation and a Multidisciplinary Mortality Review Committee

Abstract

Background

Many academic medical centers (AMCs) rely on systems like the Vizient Quality and Accountability Scorecard to track quality metrics such as the observed-to-expected (O/E) mortality ratio. The O/E mortality ratio calculation relies on clinical documentation. Missed documentation of diagnoses and risk factors for mortality leads to an underestimated expected mortality, which negatively affects the O/E metric.

Objectives

We aimed to reduce our O/E mortality ratio from a median of 1.08 (± 0.10) to a median well below 0.90 within 12 months by improving the accuracy of clinical documentation.

Methods

We used a continuous quality improvement process that began with creating a rule-based tool within a standardized documentation template. The tool was designed to pull pertinent discrete electronic health record data into clinician documentation. The tool only pulled in data that were present on admission, and it especially prioritized inclusion of frequently missed risk factors according to prior coding query data. We then formed a multidisciplinary mortality review committee where providers reviewed mortality cases, made suggestions for documentation clarification, and found potential diagnoses and risk factors that the patient had which were missing from the documentation. We then leveraged the committee's expertise and feedback to improve the rule-based clinical tool.

Results

Over the 21-month period following implementation, the median O/E mortality ratio decreased by 30%, from 1.08 (± 0.10) to 0.72 (± 0.13) and consistently remained below the prior levels. Importantly, the intervention also led to a reduction in the total number of coding queries sent to clinicians, indicating a lower administrative burden for clinicians and coders.

Conclusion

Our interventions showed a clear improvement in the O/E mortality ratio at our AMC and in the expected mortality percentage compared with other similar institutions without significantly increasing burden on clinicians or coding specialists.

Keywords

clinical documentation - clinical documentation integrity - clinical notes - inpatient care - quality improvement - mortality review - mortality risk factors - observed to expected mortality ratio - clinical coding queries

Background and Significance

Health care outcome data are increasingly visible and scrutinized to differentiate health systems from one another. The Vizient Quality and Accountability scorecard[1] is one rating system that provides hospitals with key metrics such as observed mortality, expected mortality, and the observed-to-expected (O/E) mortality ratio over time. This scorecard allows hospitals to compare themselves to institutions with similar patient populations (e.g., academic medical centers [AMCs]). The observed mortality rate is defined as the number of in-hospital deaths divided by the number of hospital discharges. The expected mortality rate is determined by a predictive model that incorporates data about patient comorbidities and acuity. Higher acuity and presence of certain comorbidities will increase the expected mortality ratio. An O/E mortality ratio of greater than 1.0 suggests that in-hospital deaths are beyond what would be expected.

The expected mortality rate for a given hospital relies on information about the patient comorbidities and acuity by utilizing what is present in clinician documentation (e.g. admission notes, progress notes, or discharge summaries). To improve the accuracy of documentation, many have invested in the creation of clinical documentation integrity (CDI) programs.[2] The CDI specialists typically have a clinical background and perform chart reviews in real time while the patient is still in the hospital. They can query providers to clarify the clinical picture and documentation can be adjusted to include missing information using terminology that will be recognized by the future billing and coding teams. Some successful quality improvement projects went a step further to partner CDI specialists with physician leads to retrospectively review charts, identify missed documentation examples, and then disseminate their findings to other physicians in the form of educational lectures, tip sheets, and/or emails.[3] [4] [5] [6] [7]

Clinician education can be a wonderful tool to improve clinical documentation but typically has a tradeoff in the form of increased cognitive load, which can affect clinician performance.[8] Other initiatives have attempted to go beyond CDI expertise plus clinician education by adding standardized note templates with automated incorporation of patient data.[9] Inspired by these initiatives, in 2021, we set out to improve the O/E mortality ratio at our AMC, utilizing documentation standardization (with an embedded rule-based tool) along with the formation of a mortality review committee (MRC) to review mortality cases.

Objectives

In 12 months, we aimed to move our O/E mortality ratio at our AMC from a median of 1.08 (± 0.10) to well below 0.90. If achieved, this improvement would place our AMC in the top quartile of AMCs nationally. Our focus was on ensuring that clinical documentation accurately reflected the acuity and complexity of our patients while avoiding excess administrative burden on clinicians and coding specialists.

Methods

The first step for this quality improvement project was the creation of a set of standardized note templates for inpatient providers. These note templates were the default templates available when a new note was created, making them accessible and user-friendly. Hospitalist use of the template was repeatedly encouraged (e.g., at meetings or via email). Next, our CDI team compiled a list of frequently omitted diagnoses ([Table 1]) based on their experiences with prior provider queries. With these diagnoses in mind, the medical informaticists added a rule-based section to the note templates that would be populate with text if certain conditions were met. For example, if the patient had low potassium level on their admission laboratory tests, “hypokalemia” would be added to the note. The tool evaluated relevant discrete electronic health record data that were present-on-admission to populate appropriate diagnoses into the note.

Table 1 The clinical documentation integrity specialists noted these diagnoses were frequently present for the patient but not documented in the clinical documentation (history and physical, progress notes, or discharge summary)

Frequently missed diagnoses in clinician documentation

• Fluid and electrolyte disorders • Malnutrition • Acute blood loss anemia • Congestive heart failure (acute and/or chronic) • Encephalopathy

The next step for this project was to form a multidisciplinary MRC with backgrounds in billing and coding, CDI, clinical practice, and quality improvement. These members either conducted independent reviews that they sent to their colleagues or would join a weekly virtual meeting to discuss their findings. The members with backgrounds in coding, CDI, and quality improvement remained the same and the clinical practitioners rotated depending on their specialty area (e.g. Neurology, Cardiology, General Medicine, or Oncology). Starting in November 2021, all mortality cases with diagnostic-related groups (DRGs) that mapped to the Hospital Medicine or Oncology service lines were reviewed weekly by the billing and coding team and brought to the committee meeting.

For those cases brought to the committee, the certified coding specialists would do an initial review, identifying all variables that were found in the clinical documentation. They would also include other potential risk variables in that particular DRG on a spreadsheet, which may impact mortality. The next layer of review was done by clinicians and as of November 2021, the main reviewers were hospitalists. The clinicians would review the chart, looking for diagnoses or risk variables that were present-on-admission for the patient but were not adequately reflected in the documentation. The committee would also discuss whether or not the case had the correct DRG depending on the clinical context. If diagnoses were indeed present but not in the documentation, the coding specialist would query the treating clinician for any remaining clarifications and gaps in documentation would be closed if deemed appropriate by the treating clinician.

These mortality reviews revealed additional missed diagnoses ([Table 2]), and this information was conveyed to the clinical informatics team. This feedback created a database that helped to update the rule-based documentation template tool to further improve documentation.

Table 2 These components are diagnoses that have a larger impact on the observed mortality rates for patients and factor in to their expected mortality risk

Diagnoses that impact observed and expected mortality

• Malnutrition, cachexia, or underweight • Coagulation defect/coagulopathy • Thrombocytopenia/platelet defect • Acute renal failure/chronic renal failure • Coma (no response to painful stimuli) • Severe brain conditions (i.e., cerebral edema or brain compression) • Respiratory failure (requires specified acuity and hypoxic/hypercapnic) • Fluid and electrolyte disorders (hypovolemia, hypokalemia, hypo/hypernatremia, etc.) • Comfort care/palliative care and DNR/DNI documented • Cardiac arrhythmias (PVCs, tachycardia, atrial fibrillation) • Shock • Liver failure/diseases (cirrhosis or hepatitis) • Encephalopathy

Abbreviation: PVC, premature ventricular contraction.

Results

The O/E mortality ratio baseline prior to the MRC reviews (October 2020–October 2021) ranged from 0.94 to 1.24 with median of 1.08 (± 0.10). When compared with the median O/E ratio prior to the MRC reviews, the median O/E ratio after the MRC reviews (November 2021–July 2023) decreased by 30% and fell well below the goal of 0.90 ([Fig. 1]). From the time the mortality reviews began in November 2021 and for approximately 21 months afterward (through July 2023), the O/E ratio ranged from 0.56 to 0.99 with a median of 0.72 (± 0.13). The O/E mortality ratio for our comparison group (made up of ∼118 other US AMCs) remained fairly static over the same time frame. The median O/E ratio for the combined AMCs was 0.97 (± 0.05) from October 2020 to October 2021and the median O/E ratio from November 2021 to July 2023 was 0.89 (± 0.06; [Fig. 2])

Fig. 1 The observed-to-expected (O/E) mortality ratio at our institution over time (dotted line) with the coronavirus disease 2019 delta and omicron peaks depicted by arrows during the pre- and postintervention periods, respectively, superimposed on the median O/E ratio (solid line) and the goal O/E ratio (dashed line).

Fig. 2 The observed-to-expected (O/E) mortality ratio for the entire academic medical center (AMC) comparison group (made up of ∼118 other AMCs across the United States) with actual ratio (dotted line) superimposed on median ratio (solid line) for the pre- and postintervention periods.

The mortality reviews had a larger impact on expected mortality percentage given that increased capture of risk variables from documentation sometimes (but not universally) increased the expected mortality. Prior to the MRC reviews (October 2020–October 2021), the expected mortality percentage at our AMC from ranged from 2.03 to 2.87% with a median of 2.26% (± 0.24%; [Fig. 3]), which was lower than the expected mortality percentage at other AMCs (which ranged 2.58 to 3.69% with a median of 2.85% [ ± 0.35%]; [Fig. 4]). After the MRC reviews (November 2021–July 2023), the expected mortality percentage at our AMC ranged from 2.59 to 3.72% with a median of 2.94 (± 0.26%) compared with the expected mortality percentage at comparable AMCs (which ranged 2.63–3.80% with a median of 2.81% [ ± 0.27%]). This trend suggests our intervention had an impact on expected mortality percentage compared with national health trends.

Fig. 3 The expected mortality rate at our academic medical center during the intervention period, with the coronavirus disease 2019 delta and omicron peaks depicted within the pre- and postintervention periods, respectively.

Fig. 4 The expected mortality rate during the intervention period for the entire academic medical center comparison group, superimposed on median ratio for the pre- and postintervention periods with the coronavirus disease 2019 delta and omicron peaks depicted within the pre- and postintervention periods, respectively.

Given the scrutiny of the clinical documentation, there was concern about excess billing and coding queries. As a result, a balancing measure of this project was to monitor the number of queries sent to clinicians during the project. Fortunately, there was actually a decrease in total queries sent to clinicians in 2022 and 2023. The decrease in queries sent was evident within a few months after the documentation template launch and continued to drop as use of the template (especially by hospitalists) approached 90% compliance. Further, the total queries related to the mortality reviews was only 5% of all queries. In fact, the documentation improved to such an extent that in mid-2022, mortality reviews could be expanded to other service lines, and additional cases from other hospital sites in the organization could be reviewed by the MRC. Eventually, however, the mortality reviews were discontinued as there were fewer clinical documentation opportunities identified. By April 2023, the reviews did not need to be conducted weekly and were instead conducted monthly and then quarterly.

Discussion

At first glance, the utilization of a tool to capture additional diagnoses in clinical documentation may be perceived solely as an attempt to increase reimbursement. The focus of the project was on accuracy of the patient condition (not reimbursement). Further, the inpatient coding team noted that incorporating additional diagnoses in the documentation did not always lead to a change in anticipated reimbursement. Additionally, during the development of the electronic health record-integrated tool, the medical informaticists and inpatient coding specialists worked together to set appropriate triggers for automatic populating of diagnoses in the notes. These triggers were developed with clinical reasoning, which meant only clinically significant diagnoses or diagnoses requiring monitoring or treatment would be incorporated. The inpatient coding specialists utilized the ICD-10 (International Classification of Diseases, 10th Revision) coding guidelines to ensure that the diagnoses met criteria to be captured by coders. Further, if a diagnosis was incorporated into a clinician note and there was lack of clarity regarding the clinical significance, the CDI specialists would query the provider who wrote the note regarding whether the diagnosis was appropriate or not. Clinicians also had the ability to delete any information that was pulled into the note if it was not applicable.

Many AMCs are charged with training large numbers of medical students and residents. Given that there is very limited undergraduate and graduate medical education on inpatient coding and clinical documentation,[10] our automated tool was an asset to increase accuracy of documentation for trainees as well. Anecdotally, some practitioners appreciated the tool as it increased visibility for any potential missing diagnoses that might affect that patient's care in real time. Future areas of research could focus on whether or not the incorporation of missing diagnoses into the note leads to a meaningful clinician changing management (e.g., a medication was added or subtracted).

One limitation of this project was that it did not affect observed mortality, which remains a meaningful goal for medical centers and improving patient outcomes. However, our medical center was concurrently focused on other initiatives targeting observed mortality and our observed mortality did drop during this time frame as well ([Fig. 5]). Hospitals across the country also observed improvements their observed mortality ([Fig. 6]). Although this clinical documentation initiative did not directly impact observed mortality, improving clinical documentation in a standardized fashion can establish more accurate benchmarks for the organization to utilize for future quality improvement initiatives. Future mortality reviews can also focus on provision of medical care and/or utilize the more accurate documentation to consistently identify risk factors that portend a worse outcome. Another limitation of the mortality reviews was the investment of time required for chart reviews and meetings. For those involved, the time investment was felt to be manageable but larger hospital groups considering adopting an MRC could consider a time-limited term of service on the MRC, which would allow for broader and more rapid disbursement of knowledge sharing within the workforce.

Fig. 5 The observed mortality rate at our academic medical center during the intervention period with the coronavirus disease 2019 delta and omicron peaks depicted within the pre- and postintervention periods, respectively.

Fig. 6 The observed mortality rate for the academic medical center comparison group with the coronavirus disease 2019 delta and omicron peaks depicted within the pre- and postintervention periods, respectively.

Conclusion

After standardizing documentation, creating a rule-based automated documentation tool, and forming a multidisciplinary committee to implement a standardized mortality review process, we achieved a clear and sustained improvement in our O/E mortality ratio. We more accurately captured the expected mortality without significantly increasing the work burden of clinicians or coding specialists. The mortality reviews by the committee frequently identified diagnoses and risk factors for mortality that were not documented by the clinician and occasionally discovered inaccurate DRGs initiated by the coding team. These reviews then led to improvements in the rule-based, automated documentation template tool, which subsequently led to fewer coding queries that benefited both clinicians and coding specialists. After more accurately capturing the acuity and complexity of the patient population with this project, we look forward to continued use of clinical informatics tools and multidisciplinary collaboration for future projects to further improve patient care.

Clinical Relevance Statement

Institutions seeking to improve the accuracy of their clinical documentation should consider rule-based automated documentation tools in conjunction with a multidisciplinary standardized mortality review. When utilized appropriately, these low-cost, minimally burdensome, interventions can lead to sustained improvements in clinical documentation, resulting fewer coding queries and improved hospital metrics.

Multiple-Choice Questions

1. Which of the following interventions was used to improve the accuracy of clinical documentation?

   • Mandatory educational modules for all providers

   • Incorporation of rule-based automated documentation tool into a note template already in use

   • A mortality review committee comprised of physicians with expertise in billing and coding

   • Implementing a weekly mandatory quota of coding queries for all inpatient coding specialists

   Correct Answer: The correct answer is option b. In addition to the incorporation of a rule-based automated documentation tool into a note template already in use, the formation of a multidisciplinary team (including billing and coding specialist, hospital medicine physicians with no additional training in billing) and a standardized mortality review process all helped to improve the accuracy of clinical documentation that led to a more accurate estimation of expected mortality and a decrease in the institutions observed-to-expected mortality ratio. Mandatory educational modules and mandatory quotas for coding specialists were not used as a part of this intervention and the long-term benefits of these types of inventions is limited.

2. Which of the following was identified as a frequently missed diagnosis in clinician documentation?

   • Sepsis

   • Atrial fibrillation

   • Cirrhosis

   • Malnutrition

   Correct Answer: The correct answer is option d. As described in [Table 1], fluid and electrolyte disorders, malnutrition, acute blood loss anemia, congestive heart failure (acute or chronic), and encephalopathy were all identified as frequently missed diagnoses by inpatient providers.

3. Which of the following diagnoses will commonly have an impact on the observed-to-expected mortality ratio?

   • Osteoarthritis

   • Dementia (Alzheimer's, Lewy body, vascular)

   • Cardiac arrhythmias (premature ventricular contractions, tachycardia, atrial fibrillation)

   • Type 2 diabetes mellitus

   Correct Answer: The correct answer is option c. As described in [Table 2], cardiac arrhythmias (premature ventricular contractions, tachycardia, atrial fibrillation), malnutrition, cachexia, a coagulation defect/coagulopathy, thrombocytopenia or platelet defect, acute renal failure/chronic renal failure, liver failure/diseases (cirrhosis or hepatitis), encephalopathy, shock, coma (defined by no response to painful stimuli), severe brain conditions (i.e., cerebral edema or brain compression), respiratory failure, fluid, and electrolyte disorders (hypovolemia, hypokalemia, hypo/hypernatremia, etc.) all were frequently found to impact expected mortality, which then impacts the observed-to-expected mortality ratio.

4. True or False: After implementing a rule-based clinical documentation tool and standardized multidisciplinary mortality review committee, institutions should expect a decrease in their observed mortality.

   • True

   • False

   Correct Answer: The correct answer is option b. After implementing a rule-based clinical documentation tool and multidisciplinary mortality review committee, institutions should expect improved accuracy of their clinical documentation, which may lead to increasing their expected mortality. Having more accurate clinical documentation, however, could subsequently lead to more accurate data to utilize for future quality improvement projects focused on observed mortality.

Conflict of Interest

None declared.

Acknowledgement

The authors wish to thank Leslie Selvy, RHIT, CCS (Inpatient Coding Supervisor), Nissa Perry (Quality Improvement Consultant), and Jenna Nytes (Senior Health Information Data Analyst) for all their contributions to this work.

Protection of Human and Animal Subjects

This project was reviewed by our local institutional review board who determined it did not meet the definition for human research.

Note

Data from the Vizient Clinical Data Base used by permission of Vizient, Inc.

Ethical Approval

Institutional review board reviewed the work (approval no.: STUDY00024292) and deemed it not human research.

^* These authors contributed equally to this work

• References

• 1 Vizient Clinical Data Base. Irving, TX: Vizient, Inc.; 2024 . Accessed December 1, 2024 at: https://www.vizientinc.com
• 2 Pine KH, Landon LA, Bossen C, VanGelder ME. Innovations in clinical documentation integrity practice: continual adaptation in a data-intensive healthcare organisation. HIM J 2023; 52 (02) 119-124
• 3 Reardon KE, Foley CM, Melvin P, Agus MSD, Sanderson AL. Impact of a clinical documentation integrity program on severity of illness of expired patients. Hosp Pediatr 2021; 11 (03) 298-302
• 4 Aiello FA, Judelson DR, Durgin JM. et al. A physician-led initiative to improve clinical documentation results in improved health care documentation, case mix index, and increased contribution margin. J Vasc Surg 2018; 68 (05) 1524-1532
• 5 Horwood CR, Latimer T, Powers CJ, Moffatt-Bruce SD, Rushing GD, Eiferman DS. Improving the mortality index by capturing patient acuity through interprofessional real-time documentation improvement in a single hospital system. Surgery 2018; 164 (04) 687-693
• 6 Kessler BA, Catalino MP, Jordan JD. Reducing the reported mortality index within a neurocritical care unit through documentation and coding accuracy. World Neurosurg 2020; 133: e819-e827
• 7 DeCicco D, Krupica TM, Pellegrino R, Dimachkie ZO. Hospital-wide intervention in billing and coding to capture complexity of care at an academic referral center. J Healthc Manag 2022; 67 (06) 416-424
• 8 Asgari E, Kaur J, Nuredini G. et al. Impact of electronic health record use on cognitive load and burnout among clinicians: narrative review. JMIR Med Inform 2024; 12: e55499
• 9 Seligson MT, Lyden SP, Caputo FJ, Kirksey L, Rowse JW, Smolock CJ. Improving clinical documentation of evaluation and management care and patient acuity improves reimbursement as well as quality metrics. J Vasc Surg 2021; 74 (06) 2055-2062
• 10 Spellberg B, Harrington D, Black S, Sue D, Stringer W, Witt M. Capturing the diagnosis: an internal medicine education program to improve documentation. Am J Med 2013; 126 (08) 739-743.e1

Publication History

Received: 25 June 2025

Accepted: 06 December 2025

Accepted Manuscript online:
12 December 2025

Article published online:
24 December 2025