Virtual Consultations—“Should We Be Using Them?”: A Two-Cycle Clinical Audit

Abstract

Objective In response to the coronavirus disease 2019 pandemic, the use of virtual appointments increased substantially. An audit was designed to evaluate the efficacy of virtual appointments at Alder Hey Children's Hospital. Patients and clinicians satisfaction of virtual appointments were also recorded.

Materials and Methods Patients having virtual appointments were identified, and data were recorded, including demographics and outcome of the appointment. A pilot study was conducted to identify any necessary modifications to the data collection process. Data were collected over two cycles, and the results were analyzed.

Results The first cycle was performed retrospectively and there were 158 patients identified who had a virtual appointment. An outcome could be drawn from the virtual appointment in over 70% of cases. Recommendations following the first cycle were implemented. A second cycle was then completed prospectively. A total of 145 patients were identified. An outcome could be drawn in 51% of cases. Patient and clinician feedback on virtual appointments was generally positive.

Conclusion This study demonstrated that a successful outcome could be drawn in the majority of virtual appointments, and clinicians and patients had positive experiences regarding virtual clinics. This highlights the usefulness of this service, which is particularly relevant due to the increased reliance on virtual clinics during the pandemic.

Keywords

audit - virtual - coronavirus

Introduction

Due to the emergence of the coronavirus disease 2019 (COVID-19) pandemic, various measures were implemented to cope with changes in the delivery of healthcare. Virtual appointments by video or telephone have been used to reduce the number of patients attending secondary or tertiary care. Use of virtual appointments aims to reduce footfall through hospitals, while still providing essential and routine care for patients. Teledentistry and telemedicine have been discussed since the 1990s, and the use of technology to improve healthcare access for patients has been well documented.[1] [2] Within the field of dentistry, general dentists were observed with greater understanding and perception of teledentistry and its known benefits, that is, early detection of caries, surveyance, and providing preventative advice to the masses.[3]

Previous studies in healthcare have explored the usage and benefits of virtual appointments for patients with the hopes of providing recommendations for future practice, and to identify areas for development within respective departments.[4] [5] [6] Research has found that virtual appointments may provide a promising alternative for remote assessment, screening, advice, monitoring, referrals, and/or prescriptions to patients for dental problems.[7] [8] [9] [10] [11] A recent study performed in a dental hospital in East Surrey explored the patient satisfaction and acceptance of teledentistry services.[12]

This led the authors to undertake this quality improvement project at Alder Hey Children's Hospital, to identify how virtual appointments are being used at a secondary/tertiary care center, and the usefulness of virtual appointments to patient and provider. Alder Hey Children's Hospital is a children's hospital that provides secondary/tertiary dental care. Among the services provided include cleft, craniofacial, orthodontics, oral and maxillofacial surgery (OMFS), oral surgery, and pediatric dentistry. This quality improvement project was designed to identify and gather information regarding the provision of virtual appointments across all dental specialties at Alder Hey Children's Hospital.

The aims of the audit were to determine if a successful outcome could be drawn at the end of the virtual appointment, and patients and clinicians perspectives of the virtual appointments. Further aims were to identify the Did Not Attend (DNA) rate of virtual appointments, to determine the reasons for DNA, and to determine the call quality of virtual appointments.

Materials and Methods

Pilot Study

A pilot study was conducted for the duration of 1 week in September 2020 to identify the various clinics to be included in the study and to collect sample data. The study did not require ethical approval and was validated by the Ethics and Quality Improvement Committee at Alder Hey Children's Hospital. Data collected was analyzed and changes were made to the data collection process. Following the completion of the pilot study, the first cycle was commenced.

Standards

The following standards were set:

1. A successful outcome recorded from 75% of virtual appointments. This was set based on opinion from consultants in all involved specialties.

2. The DNA rate for virtual appointments should be comparable to the DNA rate in a similar time period in 2019 (8.5%).

3. The call quality to be recorded in 100% of cases.

Data Collection

All video and telephone consultations across all dental specialties at Alder Hey Children's Hospital were identified and analyzed. Specialties assessed were cleft, craniofacial, orthodontics, OMFS/oral surgery, and pediatric dentistry. Data collected from virtual appointments included patient age, gender, date of consultation, clinician grade and specialty, attended/DNA, type of appointment (video/telephone), outcome of appointment, reason for outcome, and call quality. In the second cycle, patient and operator satisfaction was also recorded. Data was recorded on Microsoft Excel for analysis and tabulation.

First Cycle

The first cycle of the audit ran for 5 weeks from September 28, 2020 to November 6, 2020 when services commenced after the lockdown in June 2020. Data from virtual consultations was recorded retrospectively. Virtual appointments were categorized according to being “successful” or “unsuccessful” following the criteria specified by the audit team. Criteria is defined as shown in [Table 1]. Results from the first cycle were then presented at the departmental clinical governance meeting in December 2020, to discuss and disseminate results with recommendations to be implemented within each department. The following recommendations were made to be implemented prior to the commencement of the second cycle:

Table 1 Criteria for successful/unsuccessful outcomes following virtual appointment

Successful outcome	Unsuccessful outcome
Initial advice given virtually, planned face-to-face follow-up	Did not attend/failed to attend
Discharge—declined treatment, does not require treatment, suitable for primary care	Virtual appointment was deemed inadequate to make a clinical decision
Awaiting further tooth development or growth	Technical issues causing inability to continue appointment
Completion of care
Patient to consider options given
Patient booked for records collection

1. Call quality to be recorded for video appointments.

2. A clinician-led decision at the end of the appointment should be made, to determine if the outcome was successful or not.

3. Data capture forms were generated to be filled in by clinicians in the second cycle.

4. Patient and operator satisfaction to be recorded.

Second Cycle

Data was collected prospectively for patients who had a virtual consultation from January 15, 2021 to February 19, 2021. Data capture forms were used and filled in by clinicians at the time of the virtual consultation. Outcomes of virtual appointment were standardized according to results from the first cycle for ease of data tabulation and analysis. The results were transferred to Microsoft Excel for analysis and dissemination.

Results

First Cycle

A total of 158 patients were identified in the first cycle. The age of patients ranged from 2 months to 36 years old, and 82.9% (n = 131) of patients were 16 years old and under. Nearly 57% (n = 89) were male.

Of the virtual appointments, 85.4% (n = 135) were conducted by video, and the remaining 14.6% (n = 23) via telephone consultation. When assessed by specialty, 34.2% of consultations were cleft, 22.8% orthodontics, 21.5% oral surgery/OMFS, 17.1% pediatric dentistry, and 4.4% craniofacial. Review appointments accounted for 46.2% of virtual clinics, followed by new patient consultations (28.5%) and multidisciplinary team meetings (MDTs) (25.3%).

The DNA rate was 9.5% (n = 15). The majority of cases did not provide a reason for DNA. Among some of the reasons obtained were that the patient was unwell, the parent/guardian forgot the appointment, the appointment letter was not received, and a possible safeguarding concern. Three patients were also attempted to be contacted via telephone after failing to attend their video consult; however, clinicians were still unable to contact the patient/guardian.

The call quality was not mentioned in 64% of cases. In cases where the call quality was mentioned, 19.3% had good call quality, 11.1% with issues to video/sound; however, consultation could still be performed, and 2.2% had poor video/sound leading to the consultation being postponed or changed (see [Fig. 1]).

Fig. 1 Recording of call quality of virtual appointment in first cycle.

A successful outcome could be drawn in 70.9% of virtual consultations (see [Fig. 2]). When assessing new patient consultations via virtual appointments, 51.1% had a successful outcome. In virtual MDT clinics, 72.5% had a successful outcome. In review patients, 82.2% had a successful outcome.

Fig. 2 Outcomes of virtual appointment in first cycle.

Second Cycle

A total of 145 patients were identified in the second cycle. The age of patients ranged from 1 month old to 43 years old, with 90% (n = 130) of patients aged 16 years and under. Eighty-nine (61.4%) patients identified were male.

Of the virtual appointments, 87% (n = 101) were by video, with the remaining via telephone. When assessing the specialties, 40.7% were cleft, 11% oral surgery/OMFS, 15.2% orthodontics, and 33.1% pediatric dentistry. Review appointments accounted for 25.5% of appointments, while MDT and new patient consultations were 40.7 and 33.8%, respectively.

The DNA rate was 21% (n = 30). In patients that DNA, 36.7% were subsequently discharged due to multiple previous DNAs, 56.7% were rebooked for a virtual appointment, 1 patient was sent a letter to contact the department, and 1 patient was rebooked for a face-to-face (FTF) appointment.

The call quality was recorded in 99.1% (n = 114) of cases. In appointments where the call quality was recorded, it was acceptable in 34.8% of cases, 42.6% were good, 15.7% were intermittently poor, and 6.1% were poor leading to the appointment being postponed or changed. The call quality was not recorded in one appointment ([Fig. 3]).

Fig. 3 Recording of call quality of virtual appointment in second cycle.

In patients who attended a virtual appointment, 11.4% were subsequently discharged, 52.6% had a subsequent FTF appointment, 12.3% required a virtual follow-up, and 23.7% were placed under “watchful waiting” (see [Fig. 4]). The overall successful outcome of virtual appointments was 55.9%. In new patient consultations, 43.8% had a successful outcome. In virtual MDT appointments, 67.8% had a successful outcome. In review patients, 54.1% had a successful outcome.

Fig. 4 Outcomes of virtual appointment in second cycle.

In patients who required a subsequent FTF appointment, 35.7% required clinical examination/assessment, while 32.1% returned for special investigations and 32.1% required FTF appointments for treatment.

The majority of patients (65.2%) were satisfied with the virtual appointment, while 13.9% preferred a FTF appointment (see [Fig. 5]). When assessing the operator satisfaction of the virtual appointment, 36.7% felt “neutral,” with 18.3% being extremely satisfied and 10.1% not satisfied at all (see [Fig. 6]).

Fig. 5 Patient satisfaction with virtual appointment.

Fig. 6 Operator satisfaction with virtual appointment.

Discussion

Multiple studies have reported benefits of virtual consultations including improving patient access to healthcare and increasing engagement in patients with complex medical needs.[13] [14] [15] This study aimed to explore the outcomes of virtual appointments from a secondary/tertiary care provider and to assess patient follow-up after the appointment. By gaining an insight into the patient pathway and journey, this not only provides the operator with greater clarity into how virtual appointments are currently being used but also allows the healthcare provider to ensure patient-centered care.

Table 2 Comparison of first and second cycle results of audit

	First cycle (Sept–Nov 2020)	Second cycle (Jan–Feb 2021)
Number of patients	158	148
Patients under the age of 16 years	82%	90%
Attend anywhere (video)appt	85%	87%
DNA rate	9.5%	21%
Call quality recorded	64%	99%
Successful outcome	70%	55.9%
Patient/parent satisfaction	Not recorded	65% happy with virtual appt
Clinician satisfaction	Not recorded	53% satisfied

Abbreviation: DNA, Did Not Attend.

Values in bold indicate standards were not met.

Results from our clinical audit demonstrate the majority of patients had a successful outcome from the virtual appointment (see [Table 2]). “Successful” outcomes of virtual appointment were defined in cases where the virtual appointments were used as a substitute instead of a supplement to FTF appointments (see [Table 1]). It is important to note that the measure of success is subjective and varies according to the provider/specialty and aims of the virtual appointment. In our study, there was a reduction in “successful” outcomes in the second cycle as majority of patients who were categorized as “unsuccessful” required a further appointment for special investigations, for example, impression taking, radiographs, or photographs. This, in our opinion, does not showcase a reduction in standard or quality of care; however, it highlights the importance of systematically triaging referrals for suitability as virtual appointments. However, the outcome from cleft patients undertaking virtual appointments was successful in 83.2% of cases that met the set standard. This indicates that some specialties in dentistry may be able to utilize it more successfully than others. There were no changes in the success of outcomes when assessed by the ages of the patient. Patients who were placed under “watchful wait” or “follow-up virtual consultation” highlighted the success of the virtual consultation in providing routine dental care for patients.

The DNA rate of virtual appointments in the first cycle was 9.5, and 21% in the second cycle. The DNA rate of the first cycle was consistent with the DNA rate from a similar time period in 2019 (8.5%); however, there was an unexpected increase in DNA rate during the second cycle. The majority of patients did not provide a reason or explanation for failing to attend the virtual appointment and perhaps this is an area that requires further development and improvement. Further insight will be helpful to identify the various reasons leading to reduced patient engagement, although reasons are often multifactorial.[16] [17] [18] [19] However, one would assume with the lockdown measures in place currently in the United Kingdom at the time this article was written, with majority of the workforce working from home (unless unable to do so); this would allow for greater flexibility and compliance in attending virtual consultations.

When assessing the patient's experience of the virtual appointments at Alder Hey Children's Hospital, 65% of patients were satisfied with the consultation and would attend a virtual consultation again. This was reinforced with the clinicians' experiences, as the majority felt satisfied with the outcome and usage of the virtual consultation (50.5%). This was consistent with previous studies, with research recommending triaging by clinicians to ensure patients suitable for a remote consultation are offered one to ensure the efficient allocation of resources.[20] [21] Interestingly, a large proportion of clinicians (36.7%) was neither satisfied nor dissatisfied with the virtual appointment, with 10.1% not satisfied with the virtual appointment. It would be worthwhile to study in greater depth the clinician/operator's perspective of the virtual appointment to identify and assess reasons for deviations in satisfaction of usage of virtual clinics when compared with the patient's outcomes and their reported satisfaction.

As dentistry involves examination and assessment of the oral cavity, limitations to this study include basic limitations with virtual appointments.[22] [23] Patients with complex dental conditions often required a further FTF appointment to allow in-depth clinical examination, and this would have limited the usefulness of the virtual appointment. Further limitations of this study were basic questions in regard to the patient and clinicians' experience of the virtual consultation. Another limitation to this study was the parameters deemed a successful and unsuccessful outcome by the audit team may have not been agreed upon from the point of view of the operator.

This study sought to explore how virtual appointments are being used across different dental specialties in a secondary/tertiary care provider, and what the outcomes were from these appointments. From our study, we identified that virtual appointments are being used as an adjunct to FTF appointments in most dental specialties at Alder Hey Children's Hospital. Virtual appointments can be utilized when a thorough clinical examination is not indicated, for example, providing oral hygiene advice and toothbrushing advice, follow-up of patients postoperatively, updating medical and social history, discussing likely treatment options and benefits and risks of treatment, and building rapport with anxious patients.[24] The benefits of teledentistry for new patient consultations are uncertain, which suggests further insight and development into triaging of new patients referred to the various specialties to determine the suitability for virtual consultation. Virtual appointments offer numerous possibilities beyond the current COVID-19 pandemic, where integration of these systems will allow us to develop a more standardized approach to healthcare. There were some obvious benefits of virtual appointments:

1. Outreach clinics continued virtually that otherwise would have been discontinued.

2. Clinicians who were shielding could participate in clinics.

3. Patients who were shielding could attend virtual appointments.

4. Virtual clinics did not add to the waiting room footfall providing the flexibility to see more patients.

Conclusion

The rise of virtual appointments in response to the COVID-19 pandemic has presented certain challenges to the routine delivery of dental care. Results from our study show a successful outcome could be drawn from the majority of virtual appointments. Clinicians and patients were generally satisfied with using the “Attend Anywhere” virtual platform. It is the decision of the individual units and clinicians to consider advantages and disadvantages of virtual clinics in different specialties in dentistry. It makes it challenging to ascertain if the outcome was successful or unsuccessful in certain consultations. It is interesting to note that subsequent to this audit and with further experience with virtual clinics, clinicians are continuing to use the virtual clinics in 2021.

No conflict of interest has been declared by the author(s).

Authors' Contribution

YWJL lead the study. YWJL, AP, and SA undertook the data collection. YWJL, AP, and SA drafted a paper, and MS contributed to the final draft. All authors gave their final approval and agreed to be accountable for all aspects of the work.

• References

• 1 McLaren P, Ball CJ. Telemedicine: lessons remain unheeded. BMJ 1995; 310 (6991): 1390-1391
• 2 Wootton R. Telemedicine in the National Health Service. J R Soc Med 1998; 91 (12) 614-621
• 3 Abbas B, Wajahat M, Saleem Z, Imran E, Sajjad M, Khurshid Z. Role of teledentistry in COVID-19 pandemic: a nationwide comparative analysis among dental professionals. Eur J Dent 2020; 14 (S 01): S116-S122
• 4 Capampangan DJ, Wellik KE, Bobrow BJ. et al. Telemedicine versus telephone for remote emergency stroke consultations: a critically appraised topic. Neurologist 2009; 15 (03) 163-166
• 5 Handschu R, Scibor M, Willaczek B. et al; STENO Project. Telemedicine in acute stroke: remote video-examination compared to simple telephone consultation. J Neurol 2008; 255 (11) 1792-1797
• 6 Thiyagarajan A, Grant C, Griffiths F, Atherton H. Exploring patients' and clinicians' experiences of video consultations in primary care: a systematic scoping review. BJGP Open 2020;4(01):bjgpopen20X101020
• 7 Avula H. Tele-periodontics - oral health care at a grass root level. J Indian Soc Periodontol 2015; 19 (05) 589-592
• 8 Smales FC, Samaranyake LP. Maintaining dental education and specialist dental care during an outbreak of a new coronavirus infection. Part 2: Control of the disease, then elimination. Br Dent J 2003; 195 (12) 679-681
• 9 Smales FC, Samaranyake LP. Maintaining dental education and specialist dental care during an outbreak of a new coronavirus infection. Part 1: a deadly viral epidemic begins. Br Dent J 2003; 195 (10) 557-561
• 10 Daniel SJ, Wu L, Kumar S. Teledentistry: a systematic review of clinical outcomes, utilization and costs. J Dent Hyg 2013; 87 (06) 345-352
• 11 Dar-Odeh N, Babkair H, Alnazzawi A, Abu-Hammad S, Abu-Hammad A, Abu-Hammad O. Utilization of teledentistry in antimicrobial prescribing and diagnosis of infectious diseases during COVID-19 lockdown. Eur J Dent 2020; 14 (S 01): S20-S26
• 12 Rahman N, Nathwani S, Kandiah T. Teledentistry from a patient perspective during the coronavirus pandemic. Br Dent J 2020
• 13 Greenhalgh T, Vijayaraghavan S, Wherton J. et al. Virtual online consultations: advantages and limitations (VOCAL) study. BMJ Open 2016; 6 (01) e009388
• 14 Greenhalgh T, Shaw S, Wherton J. et al. Real-world implementation of video outpatient consultations at macro, meso, and micro levels: mixed-method study. J Med Internet Res 2018; 20 (04) e150
• 15 Vijayaraghavan S, O'Shea T, Campbell-Richards D. et al. DAWN: Diabetes appointments via webcam in Newham. Br J Diabetes 2015; 15 (03) 123-126
• 16 Ullah S, Rajan S, Liu T. et al. Why do patients miss their appointments at primary care clinics?. J Fam Med Dis Prev 2018;4(03 Available from https://www.clinmedjournals.org/articles/jfmdp/journal-of-family-medicine-and-disease-prevention-jfmdp-4-090.php?jid=jfmdp cited 2021 Jan 17 [Internet]
• 17 Rutherford E, Noray R. Ó hEarráin C, et al. Potential benefits and drawbacks of virtual clinics in general surgery: pilot cross-sectional questionnaire study. JMIR Perioper Med 2020; 3 (01) e12491
• 18 Morris NP. Virtual visits and the future of no-shows. J Gen Intern Med 2020; 35 (08) 2449-2450
• 19 Bhatia R, C Vora E, Panda A. Pediatric dental appointments no-show: rates and reasons. Int J Clin Pediatr Dent 2018; 11 (03) 171-176
• 20 Byrne E, Watkinson S. Patient and clinician satisfaction with video consultations during the COVID-19 pandemic: an opportunity for a new way of working. J Orthod 2021; 48 (01) 64-73 DOI: 10.1177/1465312520973677.
• 21 Wosik J, Fudim M, Cameron B. et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc 2020; 27 (06) 957-962
• 22 Randhawa RS, Chandan JS, Thomas T, Singh S. An exploration of the attitudes and views of general practitioners on the use of video consultations in a primary healthcare setting: a qualitative pilot study. Prim Health Care Res Dev 2019; 20: e5
• 23 Patel T, Wong J. The role of real-time interactive video consultations in dental practice during the recovery and restoration phase of the COVID-19 outbreak. Br Dent J 2020; 229 (03) 196-200
• 24 Newton T, Asimakopoulou K, Daly B, Scambler S, Scott S. The management of dental anxiety: time for a sense of proportion?. Br Dent J 2012; 213 (06) 271-274

Publication History

Article published online:
07 March 2022