Development and Usability Evaluation of GreyMatters: A Memory Clinic Information System

Abstract

Objective This paper presents the development process of GreyMatters, a memory clinic system, outlining the conceptual, practical, technical, and ethical aspects, and focuses on the usability evaluation of the system. There was a need for a system to be developed for the memory clinics of Berkshire Healthcare National Health Service (NHS) Foundation Trust (BHFT) to aid the clinical and administrative processes of assessing, diagnosing, managing, and treating patients with cognitive disorders and mental health problems.

Methods The methodology for development of the information system involved phases of requirements gathering, modeling, and prototype creation, and “bench testing” the prototype with experts. The standard Institute of Electrical and Electronics Engineers (IEEE) recommended approach for the specifications of software requirements was adopted. An electronic health record (EHR) standard (EN13606) was used, and clinical modeling was done through archetypes and the project complied with data protection and privacy legislation. Usability evaluation of GreyMatters was done using the IBM questionnaires.

Results Though the initial development was complex, the requirements, methodology, and standards adopted made the construction, deployment, adoption, and population of a memory clinic and research database feasible. The electronic patient data including the assessment scales and scores provide a rich source of objective data for audits and research. In the usability evaluation of GreyMatters, overall responses to the Computer System Usability Questionnaire and After-Scenario Questionnaire demonstrated mild-to-moderate satisfaction with the overall system and with individual tasks. The results support that the system is an acceptable tool for clinical, administrative, business, and research use and forms a useful part of the wider information architecture. The implementation and sustainability issues and the lessons learnt were noted.

Discussion The development of a system needs to take into account the existing data collection methods and other information systems that will be used alongside. Use of graphical development tools to communicate requirements, build interfaces, and prototype may improve the quality and efficiency of system development. Standardized data collection assists in the provision of reports for clinical, audit, and service development use to meet the requirements of commissioners and to allow the easier identification of potential research participants. It is possible that in the usability evaluation, the satisfaction scores are overall lower due to the extra complication of using this system in addition to the Trust's main EHR. The small number of users is a limitation.

Conclusion The establishment of requirements and methodology, addressing issues of data security and confidentiality, future data compatibility, and interoperability and medicolegal aspects, such as access controls and audit trails, led to a robust and useful system. The system was modeled around health record standards that are based on long established research on EHR standards and archetypes which differentiates GreyMatters from simple web-based capture forms that were built in house by the Trust. Its strength is that it provides flexibility to record clinical information that the existing Trust systems can't. The evaluation supports that the system is an acceptable tool for clinical, administrative, and research use. Some aspects of the system like prescribing module do need further work.

Keywords

memory clinic system - requirements - methodology - standards - usability evaluation - sustainability - EHR system

Introduction

Good memory clinics are multidisciplinary and holistic, integrating health, and social care, as well as the voluntary sector, to meet the needs of mental health patients and their relatives and carers.[1] These services may be provided in an outpatient setting, mental health or physical health hospital, general practitioner (GP) surgery, patient’s home, or any other community setting. Standards for memory clinics are specified in the United Kingdom by the Memory Services National Accreditation Program.[2] At its conception, much time was spent in capturing clinical information, prescribing dementia drugs, and monitoring the treatment. It was also recognized that valuable clinical data could be used for service development, research recruitment, and primary research purposes. The importance of research has been a key component of the G8 Dementia Summit pledge to find a cure or disease modifying treatment by 2025.[3]

The trust had been using a system sponsored by a drug company aimed at capturing assessment scale data, as well as recording what medication a patient is on. The system had useful features, such as the ability to add new assessment scales with some validation criteria, and it will also display data graphically and produce a simple letter. It had significant limitations, though as it did not permit electronic prescribing and most importantly it was based on an Access database which, although it can be stored on a network drive, will not permit multiple accesses without corrupting data. This is not suitable for use in a busy memory clinic with multiple clinicians and administrators requiring simultaneous access. Implementation of systems, like RiO, was not in the scope when the development of GreyMatters was planned. Thus, there was a need for a system to be developed for the memory clinics of BHFT to aid the clinical and administrative processes of assessing, diagnosing, managing, and treating patients with cognitive disorders and mental health problems and to facilitate recruitment of patients in clinical trials. This memory clinic system was named “GreyMatters” and its usability evaluation was performed.

Methods

The main steps in the process were documenting formal set of requirements, clinical modeling using archetypes, knowledge driven application, iterative design, deployment, and end user satisfaction testing.

Development of Requirements Standards for Electronic Health Records (EHRs) and EHR Systems and Clinical Knowledge Modeling through Archetypes

Requirements were gathered from meetings with clinicians, pharmacists, and administrative staff from BHFT. Requirements were put down depending on the daily workflow needs of the staff in the memory clinic and related faculties, systems used, paper records, and paper forms used and the aspirations of the staff for better workflow of the clinic. Google forms builder was used for specifying requirements and a web-based Wiki named JIRA was used to be able to post any issues, concerns, or suggestions, so that they could be shared between BHFT and University College London (UCL). Wiki was also used for feedback from all the members till the issues were tackled and dealt with.

The software requirements specifications followed the IEEE standard,[4] and an EHR standard EN13606[5] was used. The international standards for EHRs, like ISO EN13606 and HL7 (accredited by the American National Standards Institute), are extensive and it was not within the scope of the memory clinic system specification to replicate the entire requirements for such systems. Instead they have been used where possible as a background reference and to validate certain of the specific requirements as they arose against a wider context. Clinical archetypes[6] help to fix the hierarchy and representation of the clinical data reducing the variations in the data representation of a particular clinical domain. The clinical, prescribing and research workflow of BHFT was well studied by our clinical team and then modeled to give the design and framework for the Memory Clinic Information system to be built. The clinical data along with their specifications was formally represented by building relevant Clinical Archetypes[7] using the Archetype editor tool, “Object Dictionary Client” (ODC)[8] developed by UCL.

Ethical Approval

Ethical approval was not sought as the proposed development work did not directly involve patients and the implementation was a means to support existing care. Furthermore, the exposure to patient identifiable information was no more than the clinical team was required to access as part of their daily work as a clinician. No patient identifiable information has been included in this paper.

Clinical Application Development

The application takes advantage of a framework built at UCL based on the ISO EN13606 standard for electronic healthcare record exchange. Clinical model designs created in the ODC are embedded in Java classes using relevant class file metadata.[9]

An example of this is given below for a “Score:”

• Package uk.co.chimeventures.record.clinical.

• @Version(1).

• @ArchetypeName(“Score”).

• @uk.co.chimeventures.archetypes.annotations.Double.

• @Entity.

• @DateOfIncorporation(1240996220000L).

• @LibraryName(“uk.co.chimeventures.record.clinical”).

• @PublicationStatus(PublicationStatusType.PRIVATE).

• @ArchetypeIdentifier(“Score”).

• @DateLastVerified(1240996220000L).

• Public class Score extends uk.co.chimeventures.record.Element.

• Implements uk.co.chimeventures.record.Double {.

Some of these fields such as “Version” and “Publication Status” are simply archival information related to the status of the clinical model itself. They are transcribed from the Object Dictionary Client. Others, such as “Archetype Name” and “Archetype Identifier,” give rise to the Java file itself. The latter is entirely boilerplate code that simply serves to put a compilable wrapper on the metadata expression. In the example given above, the Java package is derived from the @LibraryName and the class name is derived from the @ArchetypeIdentifier. When the application is presented on screen, it will by default have a label given by the @ArchetypeName.

In use, the application runs in an application container JBoss[10] which is installed with an Object-Relational-Mapping (ORM) tool called Hibernate.[11] This provides a rapid means of creating standard-compatible storage for health care data. The server could stand alone and accept access requests from any client that can authenticate using Enterprise JavaBeans.[12] However, we have created a screen generation framework that behaves rather as an ORM tool does for a database, examining class file metadata for aggregation and type information to provide screens based on a clinical model expression automatically. This additional facility provides a complete turnkey application development paradigm entirely driven by the original clinical model expression.[13]

System implementation, deployment, adoption, and usability evaluation and sustainability issues[14] are described as part of the Results section.

Results

Memory Clinic Archetypes Built for the GreyMatters System

The following is a list of the main archetypes developed: demographics, GP details, alerts/allergies, consent, diagnosis, clinical registers, cognitive symptoms, assessment scales, mental health liaison, referral data, medical summary, medication, prescribing and dispensing, research application screen ([Figs. 1] and [2]).

Fig. 1 Example of an archetype model (built with “Object Dictionary Client” tool underpinned by EN13606 standard) for “Assessment Scales” and the corresponding screenshot of the application.

Fig. 2 Screenshot of the application for ‘Liaison Team Referral.’

System Implementation

The application consists of three broad feature sets. The first, for a system administrator, permits “accounts” to be created, “roles,” “users,” and “patients.” An account is a grouping within which users are associated with patients and can see their own care lists. Roles are the named purposes for which users access the care record of a patient. The second feature set is one of care delivery. A set of screens is provided that enable data to be captured in the clinic. Finally, a major component of the system was to offer prompts for repeat prescribing, transfer of prescriptions, and dispensing. This area of development proved to be more complex than anticipated and requirements changed as the Trust moved to shared care prescribing. Although built this part of the system was not deployed.

Deployment and Adoption

The system was deployed within a National Health Service (NHS)-managed server environment as a web application with a supporting database. The web application framework, EHR records, and integration with the database are all constructed using the Java programming language; the user interface has been built using the Java Server Faces (JSF) toolkit, part of the Enterprise Java Framework[12]; the EHR records have been built using Java itself. The Java based framework called Hibernate maps the Java EHR objects to a relational database schema, managed by the PostgreSQL database system, where EHR data items are stored in a relational structure. It was necessary to install an alternative browser on user's machines as the Trust's version of Internet Explorer wasn't standards compliant.

To date, there have been over 14,000 patients registered on the system. It is currently used by two of the six locality memory clinics running within the Trust, as well as the Research Department, and the mental health liaison team for older people in the acute trust. Plans are for it to be rolled out to a further locality shortly. It is primarily used by clinic administrators and secretaries (∼six users) who have a key role in recording clinical data, two research staff, and up to six clinicians. To integrate the system as far as possible with the Trust's existing systems and to minimize the burden on administrators, an XML extract was created on a nightly basis of the demographic data of all newly registered patients on the main information system. In an automated process, patients are imported into the correct GreyMatters accounts within 24 hours of referral to the service. The system can track mental health assessments and repeatedly measured clinical parameters for a patient as this data are captured over a period of time and can be tabulated. The clinical registry included these data along with the demographics to be integrated. This is important for the longitudinal management of patients, population management, and also for research purposes. The same system was used for service referrals and the mode of the referral (e.g., letter and e-mail) was noted in the system, and the referral process was tracked and its completion was monitored.

The system can capture multiple clinic visits and the data for every patient and the date stamp for each entry is done by the system. The data of previous visits can be accessed and edited (if required) and the log of any such changes made would be captured by the system (date and person logged in making the change).

Usability Evaluation

Usability Testing

An evaluation was undertaken to assess user satisfaction and system usability of GreyMatters using the IBM computer usability satisfaction questionnaires.[15] The After-Scenario Questionnaire (ASQ) and Computer System Usability Questionnaire (CSUQ)[8] [15] were chosen as they have good internal consistency and allow assessment at overall system level, as well as for individual system functions. The CSUQ provides three factors of analysis (system usefulness, information quality, and interface quality), as well as an overall score. The ASQ asks for any given task three questions relating to satisfaction of ease of use, time to complete task, and support information. Both the CSUQ and ASQ use a 7-point scale to record responses ranging from strongly agree (1) to strongly disagree (7) with positively expressed questions and a score of 4 representing the mid or neutral view.

A list of 16 past and present users of the system were e-mailed a copy of the CSUQ, as well as a list of seven scenarios that reflected real world use of the system. These ranged from searching for and registering a patient to entering clinic or research data (see [Table 1] below for an example). They were asked to choose a minimum of three scenarios that reflected their normal use of the system and asked to complete the ASQ for each of the relevant scenarios.

Table 1 Example scenario

Scenario 1	Recording consent to research
	Please log in to GreyMatters and find the required patient. You may need to complete scenario 1 if the patient doesn't already exist. Please open up the research consent page and record the full research consent details for that patient including the caregiver research and contact details when necessary.

There were a total of 10 responders (5 administrative staff, 3 clinicians, and 2 research staff) who completed the CSUQ and a total of 29 ASQs covering six scenario areas. Results for the CSUQ and ASQ are presented in [Tables 2] and [3], respectively.

Table 2 Computer System Usability Questionnaire results

	All	Admin	Clinical	Research
Overall (1–19)	3.35	3.75	2.32	3.34
Sysuse (1–8)	3.35	3.74	2.25	3.42
Infoqual (9–15)	3.37	4.06	2.14	2.98
Interqual (16–18)	3.33	3.23	3.00	3.78

Table 3 After-Scenario Questionnaire (ASQ) results

All users (n = 10) Average (range)	Overall ASQ	Admin users (n = 5) Average (range)	Admin ASQ	Research users (n = 2) Average (range)	Research ASQ	Clinician users (n = 3) Average (range)	Clinician ASQ
	2.94		3.17		2.50		Register a patient (n = 5)
3 (1–6)		3.25 (1–6)		2.5 (2–3)		Overall satisfied
2.67 (1–5)		2.75 (1–5)		2.5 (2–3)		Satisfied with time
3.17 (1–7)		3.5 (1–7)		2.5 (2–3)		Satisfied with support info
	2.79		3.78		1.83		2.44	Record research consent (n = 8)
2.63 (1–5)		3.33 (2–5)		2 (2–2)		2.33 (1–5)		Overall satisfied
2.38 (1–4)		3.33 (3–4)		1.5 (1–2)		2 (1–3)		Satisfied with time
3.38 (1–7)		4.67 (3–7)		2 (2–2)		3 (1–6)		Satisfied with support info
	2.25		3.33		1.33		1.00	Record assessment scale data (n = 8)
2.25 (1–7)		3.25 (1–7)		1.5 (1–2)		1 (1–1)		Overall satisfied
2.25 (1–7)		3.5 (1–7)		1 (1–1)		1 (1–1)		Satisfied with time
2.25 (1–7)		3.25 (1–7)		1.5 (1–2)		1 (1–1)		Satisfied with support info
	5.50		4.67		6.33	Record new drug (n = 2)
5 (4–6)		4 (4–4)		6 (6–6)		Overall satisfied
5.5 (4–7)		4 (4–4)		7 (7–7)		Satisfied with time
6 (6–6)		6 (6–6)		6 (6–6)		Satisfied with support info
	Record diagnosis (n = 0)
–
	7.00		7.00		Complete liaison referral (n = 1)
7 (7–7)		7 (7–7)		Overall satisfied
7 (7–7)		7 (7–7)		Satisfied with time
7 (7–7)		7 (7–7)		Satisfied with support info
	2.60		2.50		2.67	Record medication plan (n = 5)
2.6 (1–6)		2.5 (1–4)		2.67 (1–6)		Overall satisfied
2.6 (1–6)		2.5 (1–4)		2.67 (1–6)		Satisfied with time
2.6 (1–6)		2.5 (1–4)		2.67 (1–6)		Satisfied with support info

Averaged responses on the CSUQ (see [Table 2]) for all user types show consistency across the three factors with overall score of 3.35. Breakdown of the staff groups suggest greater satisfaction from clinicians (overall score, 2.32) than administrative staff (3.75). Analysis of individual user scores suggest polarized views in the clinician and administrative staff groups with responses ranging from 1 to 7 in both groups. Quality of supporting information was rated worst by the administrative group (4.06).

Responses on the ASQ ([Table 1]) show variation according to the task performed. Greater satisfaction was expressed for recording assessment scale data (2.25), recording a medication plan (2.60), recording research consent (2.79), and registering a patient (2.94). Dissatisfaction was expressed with recording a new drug (5.5) and entering liaison referral data (7.0). Clinicians and research staff were generally more satisfied than administrators. Analysis of individual responses again shows considerable polarity with responses ranging from 1 to 7.

Overall responses to the CSUQ and ASQ demonstrate mild-to-moderate satisfaction with the overall system and with individual tasks.[16] Most notably recording of new drug and recording of liaison referral data were considered unsatisfactory, although the low number of responders on these tasks (2 and 1, respectively) may reduce the validity of these results. It is apparent that certain individuals across the staff groups are strongly satisfied and others strongly dissatisfied with the same tasks and with the system overall. This may reflect the relative burden on individuals, especially administrative staff to enter greater volumes of data, the particular level of ease with which different individuals adopted to new systems or the amount of involvement individuals have had to engage in the development process. This could also reflect the lack of interoperability and that most of the work of double entry was shifted to the administrators adding to their work load.

It is possible that satisfaction scores are overall lower due to the extra complication of using this system in addition to the Trust's main EHR and also the need to maintain at least three other spread sheets to capture and manipulate required data. Further, limited resources have meant that written supporting information is scant and as yet updates to address known issues have not been possible. The results, however, support that the system is an acceptable tool for clinical, administrative, business, and research use and forms a useful part of the wider information architecture.

Research Application

Patient and caregiver agreement to be contacted about research is recorded in GreyMatters. A previous research database is also being migrated into the GreyMatters database.

From the clinicaltrials.gov research register, it is apparent that the breadth of data items required to identify potential participants for the trials is unlikely to be met by a single system. However, GreyMatters dataset captures important data items such as age, sex, diagnosis, assessment scales (e.g., Mini-Mental State Examination, Neuropsychiatric Inventory, and Bristol Activities of Daily Living Scale), dementia drug use, antipsychotic use, driving status, and accommodation type. The patient health record including the assessment scales and scores provides a rich source of objective data for audits and research. It helps to establish study feasibility as demographics and “inclusion and exclusion criteria” can be used in a database search to derive numbers of potential participants. A browser-based data-mining tool was built on the SQL Server Reporting Platform to pull together data from GreyMatters and other Trust information systems to allow the research department to perform such searches. There are plans to extend the scope to include data from primary care and other secondary care providers.

The latest version of the ODC used to develop the formal information model underpinning GreyMatters is a web-based application now known as Aruchi, which is recently published.[17] Specific models relating to dementia have been published and are open source for use.[18] GreyMatters itself may be licensed in future.

Factors Affecting Deployment and Usability

Clinical testing and use of the GreyMatters identified further issues. The system ensures that changes in a form design are captured in the database as part of the archetype versioning. Some necessary changes in the archetypes early on meant that certain screens would have needed to be rewritten to show both old and new versions of data. To keep costs down the decision was taken to make some direct changes to the clinical data in the database to overcome these issues. These were undertaken successfully with SQL queries after a period of testing.

GreyMatters user accounts compartmentalize data, so that different system users can view or interact with a patient's data for a particular reason, for example, researchers on different research studies. This has advantages for data security and confidentiality. It did, however, cause difficulties for users who work across the whole service needing to constantly log in and out of accounts. In the end, necessary to default all patients to a global account for use by such users. The accounts also meant that it was not possible (by design) for a clinical administrator to find a patient on the system when the patient is not already a member of their account. This led to duplicate registrations of the same patients. The ability to search across the entire database for a patient is restricted to a system administrator role. This restriction enhances security and is workable when there are resources to have a system administrator who can search and register patients at the request of the different locality teams. With the limited resources available, it was necessary to allow individuals from each service to have system administrator rights to search for patients across the whole database. However, the system administrator is unable to enter clinical data which meant that they would have to log out of the administrator role back into a clinical role. When dealing with potentially tens of patients in a session, this became very frustrating and led to dissatisfaction. This was eventually resolved by automatic system registered patients.

Factors Affecting Sustainability in the Future

CSE Servelec's web-based electronic care record system, RiO, is designed for health and social care organizations that need a single source of information about their clients.[11] It operates across mental health, child health, and community care settings and interoperates easily with other systems. RiO manages both administrative and clinical processes and can be tailored to your organization's specific needs. It was first deployed to trusts as part of the National Program for IT in the NHS in 2006.

The concept of developing GreyMatters had taken place when RiO was not considered as an option to be adopted within the trust. GreyMatters was designed to have a comprehensive record of mental health patients for the clinicians, as well as researchers. The RiO electronic records system takes this further, with an additional benefit that patient records are available at multiple sites in real time to those who need to consult them. This has resulted in more secure, efficient communication between different teams, and improved risk assessment, resulting in better patient care. Critical decisions are taken more quickly because key staff from different teams can review notes, discuss clinical problems, and record their conclusions. With GreyMatters, this process would take at least a few hours, to integrate feedback from different sites and not possible in real time. The integration of GreyMatters with RiO and other existing systems used is being considered. Recently, Healthcare NHS Foundation Trust has begun to provide patients with a mobile application to support their mental health.[19] How the data from patient faced mobile applications would be integrated with the clinic systems or whether they would only run in parallel is not yet established. These are various factors that can influence the sustainability of GreyMatters as the main system to be used within the trust. Studying the variability between these systems in detail will be part of future work.

Discussion

A challenge in modern health care is the constantly shifting requirements for information recording to meet stakeholder needs. GreyMatters has found a niche within the ecosystem of IT systems in the Trust. Its strength is that it provides flexibility to record clinical information that the other Trust systems can't. It can therefore be considered a supplementary or complementary health record that sits alongside the main Trust system. The use of multiple systems is not always ideal and presents a real challenge to time pressured staff. This and the lack of documentation and on screen help are reflected in the variability of user scores in the evaluation. Overall, the assessment of the system suggests that it is an acceptable solution.

In this approach, the system was developed step-by-step with the active participation of end users who work directly with software engineers. The advantage of this approach, albeit it is more time consuming and cost intensive, is that it can provide highly contextual, system-specific information about usability elements, not only general impressions which tend to be less actionable than direct feedback in a codesign environment. The development of a system like GreyMatters needs to take into account the existing data collection methods and other information systems that will be used alongside. It was challenging to come to a shared view of the design and requirements of the system given the multidisciplinary nature of the stakeholders, which include Trust members, developers, and end users. Significant gaps or errors may not be picked up until systems have been built and are ready for testing or use. The use of the Google forms builder for specifying requirements improved the efficiency and quality of interactions with end users. The benefits of the system have been further enhanced by developing data flows between the different systems. For instance, new patient registrations within the main Trust system are automatically imported into GreyMatters. The browser-based recruitment tool for clinical trials added to the benefits of the system. There are plans to roll out GreyMatters further to other locality memory clinics within the Trust.

Limitations of the Development and of the Evaluation of the system

There are a few limitations of this study, as follows:

1. System usability evaluation: the small number of staff doing the evaluation is a limitation of the system evaluation.

2. The architectural limitation includes “patient paneling” issues in the EHR and restriction on access to patient records. With the idea of supporting team-based comprehensive care, ideally staff should be able to access patient data in their entire clinic’s population.

3. There is inability to show through the interface successive versions of forms.

4. Due to lack of resources, pharmacy module was not built.

Conclusion

The establishment of requirements and methodology, the importance of the underlying system to address issues of data security and confidentiality, future data compatibility, and interoperability and medicolegal aspects, such as access controls and audit trails, led to a robust and useful system. It was beneficial to use a system modeled around standards like IEEE and EN13606 that are based on long established research. Standardized data collection assists in the provision of reports for clinical audit and service development use, to meet the requirements of commissioners and to allow the easier identification of potential research participants. The use of EHR standards helps ensure that the medico legal challenges of EHR are met. Firstly, the details of standard EHR information model, for example ISO 13606 Part 1, include properties of the data that would be used to ensure that you know who created the data, when, where, and if it has ever been revised. These cover parts of the medical legal requirements of trustworthy clinical data. Secondly, if EHR data are standardized it becomes possible to apply access rules in a standardized way. This differentiates GreyMatters from simple web-based capture forms and this provides the confidence that the system can meet the medico legal challenges of an EHR. The next consideration was to have a flexible approach to capturing clinical data, so that it could be reworked to adapt to changing requirements over time. In part, this was met by GreyMatters but lack of resources meant that development did not allow the application to exploit certain features to their full potential, for instance the inability to show through the interface successive versions of forms. Its strength is that it provides flexibility to record clinical information, controls its access, provides an audit trail and a browser-based recruitment tool for clinical trials. The evaluation supports that the system has been deemed acceptable for clinical, administrative and research purposes by most users, although there is dissatisfaction with some aspects and needs further work along with the development of pharmacy module. Ability to integrate the system with other systems like RiO within the trust would form an interesting part of future work.

Conflict of Interest

None declared.

• References

• 1 Improving services and support for people with dementia. Report by the Comptroller and Auditor General. HC 604 Session 2006-2007, 4 July 2007; National Audit Office. Available at: https://www.nao.org.uk/wp-content/uploads/2007/07/0607604.pdf. Accessed October 8, 2020
• 2 Memory Services National Accreditation Programme Standards for Memory Services, sixth Edition April 2018, Pub. No. CCQI273 Royal College of Psychiatrists. Available at: https://www.rcpsych.ac.uk/docs/default-source/improving-care/ccqi/quality-networks/memory-clinics-msnap/msnap-standards-6th-edition-2018.pdf?sfvrsn=d3a9bc94_2 Accessed October 17, 2020
• 3 G8 dementia summit agreements. Available at: https://www.gov.uk/government/publications/g8-dementia-summit-agreements. Accessed October 18, 2020
• 4 IEEE Standards. 830–1998 - IEEE Recommended Practice for Software Requirements Specifications. Available at: https://standards.ieee.org/standard/830-1998.html. Accessed October 8, 2020
• 5 Kalra D. ISO 13606-1:2019. Health informatics—Electronic health record communication—Part 1: Reference Model. Available at: https://www.iso.org/standard/67868.html. Accessed October 8, 2020
• 6 Open EHR. Available at: http://www.openehr.org Accessed October 20, 2020
• 7 Tapuria A, Kalra D, Kobayashi S. Contribution of clinical archetypes, and the challenges, towards achieving semantic interoperability for EHRs. Healthc Inform Res 2013; 19 (04) 286-292
• 8 Austin T. The development and comparative evaluation of middleware and database architectures for the implementation of an electronic healthcare record, PhD Thesis. University of London; 2004
• 9 Oracle Java. Available at: http://www.oracle.com/technetwork/java/index.html. Accessed October 20, 2020
• 10 Red hat. Available at: www.jboss.com. Accessed October 20, 2020
• 11 Hibernate. Available at: www.hibernate.org. Accessed October 20, 2020
• 12 Oracle Enterprise JavaBeans Technology. Available at: http://www.oracle.com/technetwork/java/javaee/ejb/index.html. Accessed October 20, 2020
• 13 Tapuria A, Evans M, Curcin V, Austin T, Lea N, Kalra D. Establishment of requirements and methodology for the development and implementation of GreyMatters, a memory clinic information system. Stud Health Technol Inform 2017; 235: 18-22
• 14 Servelec RiO EPR System. Available at: https://www.servelec.co.uk/product-range/rio/. Accessed October 20, 2020
• 15 Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 1995 2007; 7 (01) 57-78
• 16 Tapuria A, Evans M, Austin T, Lea N, Kalra D. Development and evaluation of a memory clinic information system. Stud Health Technol Inform 2014; 205: 106-110
• 17 Austin T, Sun S, Lea N, Tapuria A, Kalra D. Patterns: a simple but expressive data modelling formalism. Int. J. Knowledge Engineering and Data Mining 2016; 4 (01) 5-15
• 18 Aruchi Helicon. Available at: https://aruchi-helicon.rhcloud.com/pattern/describe?id=134. Accessed May 8, 2020
• 19 Berkshire NHS Trust deploys mental health app. Available at: https://www.ukauthority.com/articles/berkshire-nhs-trust-deploys-mental-health-app/. Accessed October 12, 2020

Publikationsverlauf

Eingereicht: 14. April 2020

Angenommen: 18. September 2020

Artikel online veröffentlicht:
09. Dezember 2020