Considerations on the Improved Integration of Medical Guidelines into Routine Clinical Practice – a Review and Concept Proposal

Abstract

Medical guidelines have become established as the standard for the comprehensive synopsis of all available information (scientific trials, expert opinion) on diagnosis and treatment recommendations. The transfer of guidelines to clinical practice and subsequent monitoring has however proven difficult. In particular the potential interaction between guideline developers and guideline users has not been fully utilised. This review article analyses the status quo and existing methodological and technical information solutions supporting the guideline life cycle. It is shown that there are numerous innovative developments that in isolation do not provide comprehensive support. The vision of the “Living Guidelines 2.0” is therefore presented. This outlines the merging of guideline development and implementation on the basis of clinical pathways and guideline-based quality control, and building on this, the generation of information for guideline development and research.

Zusammenfassung

Medizinische Leitlinien haben sich als Standard zur Aggregation der gesamten Informationslage (wissenschaftliche Studien, Expertenmeinungen) zu Diagnose- und Therapieempfehlungen etabliert. Die Überführung und Überwachung der Empfehlungen in die Praxis erweisen sich jedoch als schwierig. Vor allem wird das Interaktionspotenzial zwischen Leitlinienentwicklern und -anwendern bisher nicht ausgeschöpft. Der Beitrag analysiert die aktuelle Situation sowie bestehende methodische und informationstechnische Lösungen zur Unterstützung des Leitlinienlebenszyklus. Das Ergebnis zeigt, dass es eine Vielzahl innovativer Entwicklungen gibt, diese jedoch als Insellösungen keine ganzheitliche Unterstützung ermöglichen. In der vorliegenden Übersichtsarbeit wird daher die Vision der „Living Guidelines 2.0“ vorgestellt. Sie umreißt das Zusammenwachsen von Leitlinienentwicklung und der Implementierung dieser auf Basis klinischer Behandlungspfade sowie leitlinienbasierter Qualitätssicherung und darauf aufbauender Informationsgenerierung für Leitlinienentwicklung und Forschung.

Key words

medical guidelines - evidence based medicine - clinical pathways - quality management

Schlüsselwörter

medizinische Leitlinien - evidenzbasierte Medizin - klinische Behandlungspfade - Qualitätsmanagement

Introduction

The changes in the structure of the German health system increasingly require stronger interdisciplinary and specialised care. Doctors are presented with the increasingly difficult task of finding a balance between optimising economics and meeting care and quality standards. The inclusion of evidence and consensus-based recommendations in routine clinical practice supports doctors in keeping up with these developments.

Medical guidelines have become established as the standard for synopsis of current literature and arriving at consensus for clinical recommendations [1]. As scientifically based and systematically developed management tools for doctors and patients they serve to support appropriate care for specific health problems [2]. Various studies have shown the positive effects of guideline-conform treatments. As an example Wöckel et al. (2010) showed a direct association between adherence to guidelines and both survival rates and recurrence-free interval in breast cancer patients [3].

Clinical pathways support the immediate implementation of medical guidelines and the recommendations they contain in clinical practice. They provide fixed documentation of the optimal management for typical, defined disease patterns [4], [5]. They are interdisciplinary descriptions of treatment sequences for specific patient types, adapted to local circumstances [6].

Despite the value of both tools being widely recognised [7], [8], German specialist associations state that guidelines are not yet used in routine medical practice to the extent that those who compile them would hope [9], implying that there is optimisation potential. Barriers to their use include lack of practice relevance, contradiction to other guidelines, difficulties with implementation and poor availability [9], [10]. Likewise goal number 6 of the National Cancer Plan points out that it is not sufficient to simply make medical guidelines available in the internet or medical journals. Rather, in order to further develop guidelines in a timely manner, the development of a system to continuously maintain and update the various guideline versions is necessary [11]. Although the methodological substance of guideline development, especially S3 guidelines, is at a very high level, the information technological support and rigorous quality assurance throughout the guideline life cycle[ ¹ ] has proven difficult.

In addition it is apparent that the practical implementation of guidelines in general and their adaption to specific clinical pathways have not been explicitly addressed during guideline creation. Instead, the two instruments are often viewed as separate entities [12], their synergistic potential remaining only partially exploited.

Doctors presumably still have reservations about the implementation and use of clinical pathways and these have counterproductive effects. It can be concluded that doctors are aware of medical guidelines [1], however it is not yet recognised that they have the potential to define medical process standards. In the attempt to provide both guideline-conform and economically sustainable medical care operative (executive) processes and medical guidelines will have to become more strongly interlinked, especially in the hospital sector. The current state of the practice of guideline compilation, distribution, implementation and revision is summarised in [Table 1].

Table 1 Baseline situation – state of current practice in the guideline life cycle (derived from [8]).

Planning, organisation, development and editing: Delay and high labour intensity in guideline development and revision due to inadequate information technological support

Time and labour intensive processes (e.g. achieving consensus, coordination and communication between role-players in the guideline commission, the systematic literature search and evidence evaluation) [13], [14] Use of Microsoft Office Solutions (MS Word, MS Excel) Manual administration and verification of correlation between guideline components (e.g. fundamental information, supporting texts, recommendations, statements, published evidence) and with other guidelines

Dissemination and implementation: construction of guideline-based clinical pathways impeded by lack of guidance for adaption within the medical guidelines themselves

Passive dissemination of guidelines through their publication via specialist associationʼs publication media and databases [15] Supply in document form (prose, tables, algorithms) Supply in various versions for different interest groups/purposes: full version, abridged version, patient guideline, pocket or dia (software) versions [16], [17] No process template or implementation tips for guideline users

Evaluation and update planning: No opportunity to monitor guideline implementation in practice or for documentation and evaluation of divergence from guidelines (“guideline compliance”) due to a lack of user-oriented quality control

Optional assistance for development and use of guideline quality indicators (QI) [18] No generic method for comprehensive (all speciality groups) measurement and evaluation of the use of guideline recommendations in practice, i.e. no practical evaluation or knowledge/theory generation Pioneering role of oncology guidelines: The standardised process of identifying and implementing (result oriented) QI in medical guidelines by the Oncology Guidelines Program [19], and the uniform certification processes including standards for data communication and outcome measurement [20] Long updating cycles

Living Guidelines 2.0 – a Concept Proposal

The rigorous approach to active guidelines implementation affects almost all phases of the guideline life cycle ([Table 1]). The concept of “Living Guidelines 2.0” is formulated here as a framework for a desired scenario. The aim is to provide guidance for agreeing on and performing individual activities in the context of more stringent guideline use.

Origins of the term

The term “living guidelines” is used in the literature to describe keeping medical guidelines as up to date as possible with the best possible level of current medical knowledge and recommendations[ ² ]. This requires that guidelines be permanently updated, giving doctors timely access to the latest evidence. Current research is focused increasingly on formalising the if-then logic used in guidelines, to improve dealing with changes and to support computer-assisted interpretation (e.g. decision support systems) [21], [22].

The term “Living Guidelines 2.0” goes further than simply staying up to date. It represents a paradigm of care that has guideline-driven, quality-assured practice at its core. This aim brings with it the challenge of a comprehensive interlinkage of guideline compilation, practice implementation and maintenance (updating). The effect of guideline recommendations on routine clinical practice, and the reflection of actual practice in the guidelines would amount to direct integration of guidelines, thus strengthening the relationship between guideline recommendations and clinical practice[ ³ ].

Consequence – Vision Living Guidelines 2.0

In the “Living Guidelines 2.0” concept, guidelines are no longer static, inert documentations of current consensus opinion on treatment recommendations, but are applied rather as dynamic, practice-rooted and therefore living tools. The concept expands the previous focus of “updating guidelines” through integrating guideline recommendations into the care process, implementing quality management adapted to the guideline, and feeding back of practice-generated treatment outcomes into the further development of the guideline and thus into the entire guideline life cycle ([Fig. 1]). In this way a more comprehensive, evidence-based standard of care should result, achieving closer interlinkage of research/guideline compilation and clinical practice. A description of the key fields of action follows.

Fig. 1 The “Living Guidelines 2.0” concept in the guideline life cycle (derived from [9]) – integration points and action recommendations.

In order to tackle deficits in current practice through the development and updating of guidelines a guideline management system must be added to the existing MS Office landscape. Such systems allow guideline fragments such as recommendations, proven evidence, explanatory statements (background texts) and their shared properties to be saved and maintained, semantically correct, in the guideline context. A clear modularisation of guideline content is necessary to facilitate searching for individual guideline elements, exchangeability and maintenance. The necessary improvement in content alignment between different guidelines is also made possible [23]. Systematic guideline compilation is further supported by required acceptance and testing procedures (e.g. consensus process) feeding directly into a workflow controlled and documented by a guideline management system. Linking in to literature data bases and the specification of continuous search requests would enable monitoring of the latest, relevant publications and would thus further facilitate the updating process.

Guideline dissemination and implementation

A modularised system of guideline management on the basis of an electronic guideline management system allows not only the circulation of guideline content in various versions, and thus better adapted to the userʼs needs, but also supplies individual modules relevant to specific user groups. Guideline content can be adapted flexibly into diverse formats for various devices, and at the press of a button be transferred to Apps for tablets or smartphones. Guideline users could also be allowed various levels of right to access (e.g. reading, editing, administration). This would make searching and finding content relevant to individual doctors and patients easier in general and for specific treatment situations. On the basis of treatment relevant information it would be possible to provide doctors with guideline-based recommendations for specific patients during the treatment process. Doctors could simultaneously provide feedback on recommendations in the decision-making phase, rating or commenting on them. This would indeed make the guideline a living tool that brings together clinical practice and research.

The current unsupported, laborious adaption of guideline recommendations to clinical practice negatively affects the rapid application to routine clinical practice. For these reasons a user-friendly, practical method of creating guideline-based clinical pathways that can be integrated into existing structures needs to be developed and evaluated. Existing methods can be used for this purpose (e.g. [24], [25]). The aim should be to provide guideline users with instructions (e.g. in handbook form) to support the systematic development of quality assured clinical pathways based on guideline recommendations. A possibility would be to provide the pathway developer with a “pathway skeleton” or framework within the guideline that would need to be adapted to hospital-specific circumstances. For this an expansion of the guideline structure including an additional document would be necessary. Current medical guidelines incorporate both explicit process descriptions in the form of clinical algorithms, and implicit descriptions in background texts. The clinical algorithm, as the accepted standard, should be utilised for describing and expanding the pathway framework. Implicit and explicit process descriptions should be combined in such a way, with respect to treatment scenarios, that the guideline user arrives at the fundamental medical course of action intended by the guideline [24]. Such process descriptions should be compiled and published within the process of guideline development. A first step could be to include the most important guideline processes as an appendix, with descriptions in clinical flow diagrams that are already incorporated.

Integrated quality control

If the transfer of guideline recommendations into clinical practice were methodically and technologically better supported (with the aid of a guideline management system), it is conceivable that data from guideline implementation in routine clinical practice could be used for evaluation and for the generation of knowledge or theories for research purposes. For this a standardised method of integration of guideline-based quality indicators into clinical pathways is required (e.g. building on [26]). The expansion of guideline quality control systems into internal hospital process standards improves procedural aspects of current outcome-based orientation. From the perspective of processes, this would make a substantial contribution towards enabling audit of guideline compliance, for which there is currently limited opportunity. The interaction with clinical data intelligence methods may also have great potential. These methods are suited to analysing general clinical practice data, organised in various data bases, with a general framework overview and e.g. with respect to guideline adherence. Available data can thus be used in a more targeted way for the evaluation of medical and procedural issues (SmartData). It is clear that the prevailing IT infrastructure as well as data protection organisation face great challenges.

The feedback of pathway-related QIs can provide important insights into the provision of care, in turn allowing evaluation of guideline recommendations. The chain of information, from first compilation through to final appraisal goes through the following steps: compilation, annotation/commentary, conversion and appraisal. In this way e.g. repeated departure from recommendations should be identified. Such application models are able to highlight the possible need for revision of guideline recommendations. Research hypotheses would continue to arise dependant on the analysis of specific care processes. These functions have direct benefits for the compilation and updating processes since they meet the requirements for problem-oriented prioritisation of guideline topics, for tools demonstrating the effects of guideline implementation, and for the inclusion of guideline users in guideline development [23].

Status Quo – Tools and Problem Solving Approaches

The potential of support for guideline development and updating has been recognised and addressed in various initiatives that have supplied specific IT products or methods. Current approaches only function selectively and do not facilitate integrated, comprehensive support as provided by “Living Guidelines 2.0” ([Fig. 2]). A number of typical solutions are discussed below. So far the guideline development phase has experienced the most methodological and information technological support. The AWMF Regelwerk (Engl.: handbook) [16], the German guideline appraisal tool DELBI (Deutsche Leitlinien-Bewertungsinstrument) and the GRADE method (Grading of Recommendations Assessment, Development and Evaluation) for evaluating the quality of research results and medical evidence, and for the development and compilation of reliable, trustworthy recommendations, are all established tools and examples of methodological support for guideline development. Other guideline rating instruments were analysed and compared by Semlitsch et al. (2015) [27]. They found that tools have increasingly been developed that provide an overview of guideline quality using the systems of evidence analysis, the evidence supporting recommendations, the clear formulation of aims and the naming of author teams as evaluation criteria. The User Group – Med. Leitlinien e. V. (Engl.: Med. Guidelines) supplies the guideline development portal to support collaboration between role-players in guideline compilation (e.g. preparation of timescales, discussion forums, central data administration, tools for reaching consensus and literature administration) [28]. The portal solution does not incorporate structural, semantic support for maintenance and identification of guideline content. In contrast, the MagicApp developed within the EU-funded MAGIC project does cover this aspect. As a development and publication platform it facilitates the compilation and revision processes as well as the application of recommendations in practice. The XML export function creates the basis for integration of guidelines into patient records, web portals or apps for mobile devices. The toolʼs applicability to compilation ends, however, when content deviates from suggested structure and in terms of predefined elements. The software tool BridgeWiz [29], which was developed with the GLIDES (Guidelines into Decision Support) project, also supports the compilation process of individual recommendations using typical wh-questions (who, when, what, for whom, how and why). The tool serves to formalise medical guidelines in order to translate them into computer interpretable languages with the aim of supporting clinical decision-making. It does not however permit the management of guidelines generated.

Fig. 2 Categorisation of existing solutions within the “Living Guidelines 2.0” concept.

The dissemination of guideline documents currently occurs passively via databases such as provided by the AWMF. Distribution strategies have been compiled for various interest groups (doctors, managers, patients, nursing staff, general public) as part of the EU-funded DECIDE [ ⁴ ] (Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence) project and a corresponding toolkit has been developed. The Cancer Council Australia wiki platform [30] presents guideline content in the form of a wiki, enabling quick revision and linking up of elements within and between guidelines. Though this fulfils the “Living Guidelines 2.0” requirement for dynamism, interlinkage with guideline implementation in clinical practice is lacking. There are various other methods for this purpose that support and guide the compilation of clinical pathways derived from medical guidelines [24], [25], [31]. These use completed guideline documents as their basis and have not yet been implemented in the phase of guideline development.

Oncology Care in Obstetrics and Gynaecology as a National Benchmark

The fundamental problem of interlinking of clinical practice and medical guidelines has been recognised in the context of routine practice as well as by the previously described approaches that mostly originate from scientific observations, and initial solutions have been developed. The system of quality assurance among certified oncology centres, which was essentially conceptualised by the Oncology Guideline Program (OL program) and the German Cancer Association, is a good example. This system describes the direct integration of QIs into medical guidelines and has also defined standardised methods of documentation and recycling of QIs from clinical practice.

The process of identifying and implementing QIs in medical guidelines was developed and standardised within the Oncology Guideline Program [19]. As an example, the current version of the interdisciplinary S3 guideline “Diagnosis, Treatment and Follow-up of Breast Cancer” followed/applied the standard. The cornerstones of care coordination are outlined in chapter 7 of the guideline emphasising clearly that all measures must be transparent and quality controlled. The quality of the structure, process and outcome quality must also be continuously appraised and the clinical cancer register notified of relevant treatment data [32].

Furthermore, the agreed version of the guideline already contains a fixed, minimum number of QIs derived on the basis of strong guideline recommendations and a corresponding quality objective ([Table 2]).

Table 2 Integrated quality indicators for axillary lymphadenectomy in DCIS (guideline illustration) [32].

QI no. 3 – Axillary lymphadenectomy in DCIS
Quality indicator	Content of recommendation (wording, LOE, EG); quality aim	Statements from the breast cancer guideline with respect to: a. Impact on the health system b. Source of statement from the 2012 guideline c. Evidence base
Z: Patients with axillary lymph node resection N: Patients with histology showing “DCIS” and completed operative treatment for primary disease and breast conserving therapy	LOE 1b, recommendation level A Quality target: As few patients with primary axillary dissection or sentinel node biopsy (SLNE) in DCIS with breast conserving therapy as possible Target: < 5 %	a. Impact on the health system Axillary dissection should not be performed in DCIS. Sentinel node biopsy should only be performed when it is not possible to perform a secondary sentinel node biopsy for technical reasons (e.g. after mastectomy) b. Source of statement from the 2012 guideline Statement Präinv-3b c. Evidence base Christiaens M et al. 2007; NZGG 2009

The QIs in turn form the basis of data to be incorporated into the tumour documentation systems for analysis of structure, process and outcome quality. The standardised measurement of results is assured via the “XML-OncoBox” whose central element is the uniform data interface for submission of result data from tumour documentation systems and certified cancer centre registers [20]. There is however still room for optimisation with respect to compatibility of gynaecological and gynaecological oncology tumour registers to avoid double documentation.

For certification and re-certification centres must show their documentation and compliance with key figures specified by the German Cancer Association (DKG). For certification a key figure assessment form must be submitted, which is generated via the XML-OncoBox and includes a result quality matrix.

The advantage of the XML-OncoBox is the support provided for calculation of the necessary key figures required for certification. A single, uniform data interface has been defined for this purpose that enables data transfer from the cancer centreʼs tumour documentation system. Since the XML-OncoBox is directly linked to the tumour documentation system data do not need to be exported externally but remain within the internal system environment, which is a significant factor for data protection. For compatibility with the multitude of tumour documentation systems the XML-OncoBox defines a unified data schema. Data are standardised and the necessary key figures calculated using standardised algorithms. In this way results from different centres can be compared. When a documentation system is connected to it, the OncoBox can calculate all the key figures and statistics required by the DKG, analyse trends, detect discrepancies or derive hypotheses for further study.

The specification of QIs during guideline compilation and the requirement for documentation and quality management systems based on these are fundamental premises for achieving a closer correspondence between medical guidelines and routine clinical care.

This should be supported simultaneously by pragmatic measures such as standardisation of graphic depictions of current guidelines. Even such simple measures have the potential to boost guideline dissemination and implementation. As an example, a uniform guideline template analogous to that of the OL program with additional features and functions was developed within the DGGGʼs guideline program. This forms the basis of the recently published guideline “Uterine Sarcoma”.

To conclude, [Fig. 3] illustrates how an integrated, quality system promotes the concept of living guidelines (Living Guidelines 2.0). Further work is needed especially in the integration of clinical pathway design recommendation and procedural orientation of hospital information systems.

Fig. 3 Guideline integration in practice – best practice taken from guideline-based cancer treatment in Germany.

Conclusion

The provision of individual therapeutic measures always requires the consideration of current scientific knowledge, the doctorʼs own expertise and the patientʼs consent. Nevertheless medical guidelines offer the chance to further standardise medical treatments and quality management and make them more comparable. The mode of action of guidelines should be organized and determined systematically and not depend on the individual doctorʼs desire to implement them. The realms of the medical guideline and routine clinical practice should match up far better. Herein lies the chance of reliable tools towards a quality-oriented, value-based remuneration system.

The existing quality management concept within certified cancer centres already shows that the German health system is in a position to design systematic innovations for large scale implementation although it is usually accused of being reluctant to change. It is however clear that non-comprehensive measures do not take ground. An overall strategy is required which allows current developments to be incorporated.

This article has provided an overview of the basic structure and existing framework for guideline-conform treatment in Germany. The status of research in methodological and information technological support of medical guideline development, dissemination and implementation is analysed. The “Living Guidelines 2.0” concept forms the central focus of discussion and individual aspects are illustrated. The interactions between guideline developers and users is analysed in terms of the typical guideline life cycle, and it is shown how clinical pathways and a coordinated quality system can form the link between these two groups.

The analysis of the current situation shows that guideline compilation and (passive) dissemination already enjoy good methodological and information technological support. There is however very little support for implementation, monitoring at the hospital level together with user and process-oriented guideline quality management, and recycling of generated information back to the guideline group for updating. Currently existing solutions with great potential for innovation are not yet comprehensively integrated, functioning in isolation or fitting poorly together.

The major deficits at the interface between developers and users can only be overcome if a large number of compatible methods and tools are developed and promoted in a unified manner on the basis of a common vision for guideline production, implementation and development. The “Living Guidelines 2.0” concept provides such a vision, which now needs to be converted to a common plan of action. To this end measures have already been undertaken by Prof. Beckmann (DGGG), the DKG e. V. and the TU Dresden. The increasing number of activities towards integrated quality assurance shows that development is in progress among both developers and users. The vision of “Living Guidelines 2.0” will give this process a name.

Conflict of Interest

None.

¹ The guideline life cycle consists of 5 phases: (1) Planning and organisation, (2) guideline development/compilation, (3) editing and dissemination, (4) implementation and (5) evaluation and update planning [9].

² According to the American College of Chest Physicians, living guidelines are defined as recommendations for the diagnosis and treatment of disease that are systematically developed, evidence based and continuously updated (URL: http://www.g-i-n.net/events/document-store/g-i-n-conferences/seoul-2011/presentations-seoul-2011/short-orals-seoul-2011/o06_zelman-lewis_living-guidelines_290811.pdf).

³ These fields are reflected in the aims of the Guidelines International Network (G-I-N): to lead, strengthen and support collaboration and work within the guideline development, adaptation and implementation community (G-I-N 2013).

⁴ http://www.decide-collaboration.eu/

• References

• 1 Kopp IB. Perspektiven der Leitlinienentwicklung und -implementation aus der Sicht der AWMF. Z Rheumatol 2010; 69: 298-304
• 2 AWMF, ÄZQ. Leitlinien-Glossar. Das Leitlinienglossar, 20-Juni-2007. Online: http://www.awmf.org/fileadmin/user_upload/Leitlinien/Werkzeuge/ll-glossar.pdf last access: 03.01.2012
• 3 Wöckel A, Kurzeder C, Geyer V et al. Effects of guideline adherence in primary breast cancer—a 5-year multi-center cohort study of 3976 patients. Breast 2010; 19: 120-127
• 4 Kinsman L, Rotter T, James E et al. What is a clinical pathway? Development of a definition to inform the debate. BMC Med 2010; 8: 31-33
• 5 Panella M, Vanhaecht K. Is there still need for confusion about pathways?. Intl J Care Pathw 2010; 14: 1-3
• 6 Küttner T, Roeder N. Definition Klinischer Behandlungspfade. In: Roeder N, Küttner T, Hrsg. Klinische Behandlungspfade. Mit Standards erfolgreicher arbeiten. Köln: Deutscher Ärzte-Verlag; 2007: 19-27
• 7 Grimshaw JM, Thomas RE, MacLennan G et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004; 8: iii-iv 1–72
• 8 Woolf SH, Grol R, Hutchinson A et al. Potential benefits, limitations, and harms of clinical guidelines. BMJ 1999; 318: 527-530
• 9 Muche-Borowski DC, Kopp I. Wie eine Leitlinie entsteht. Z Herz-Thorax-Gefäßchir 2011; 25: 217-223
• 10 Perleth M, Jakubowski E, Busse R. ‚Best Practice‘ im Gesundheitswesen – oder warum wir evidenzbasierte Medizin, Leitlinien und Health Technology Assessment brauchen. ZaeFQ 2000; 94: 741-744
• 11 BMG. Nationaler Krebsplan. 2012 Online: https://www.bundesgesundheitsministerium.de/fileadmin/dateien/Publikationen/Praevention/Broschueren/Broschuere_Nationaler_Krebsplan_-_Handlungsfelder__Ziele_und_Umsetzungsempfehlungen.pdf last access: 24.09.2013
• 12 Wollersheim H, Burgers J, Grol R. Clinical guidelines to improve patient care. Neth J Med 2005; 63: 188-192
• 13 Troschke J, Selbmann HK, Encke A. Versorgungsforschung und Leitlinien – die Perspektive der Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften (AWMF). Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen 2006; 100: 597-602
• 14 Lenfant C. Clinical research to clinical practice – lost in translation?. N Engl J Med 2003; 349: 868-874
• 15 Medves J, Godfrey C, Turner C et al. Systematic review of practice guideline dissemination and implementation strategies for healthcare teams and team-based practice. Int J Evid Based Healthc 2010; 8: 79-89
• 16 Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften (AWMF) – Ständige Kommission Leitlinien. AWMF-Regelwerk ‚Leitlinien‘. 2012 Online: http://www.awmf.org/leitlinien/awmf-regelwerk.html last access: 13.01.2014
• 17 DKG; DKH, AWMF. Leitlinienprogramm Onkologie. 2015 Online: http://www.leitlinienprogramm-onkologie.de http://leitlinienprogramm-onkologie.de/uploads/tx_sbdownloader/2015_IEQV_NKP_OL.pdf; last access: 11.04.2016
• 18 Ärztliches Zentrum für Qualität in der Medizin (ÄZQ). Programm für Nationale VersorgungsLeitlinien von BÄK, KBV und AWMF. Qualitätsindikatoren – Manual für Autoren. Neukirchen: Make a Book; 2009
• 19 Follmann M, Kopp I, Pottkämper K et al. Updating Guideline based Quality Indicators. The Methodology of the German Breast Cancer Guideline Development Group. Gehalten auf der Guidelines International Network. Berlin: G-I-N Conference; 2012: DocO16
• 20 Wesselmann S, Dieng S. Das Zertifizierungssystem der Deutschen Krebsgesellschaft. mdi 2013; 15: 38-43
• 21 Miller K. S3-Leitlinie Prostata-Ca aktualisiert mit therapeutischen Spielräumen. Medical Tribune 2011; 6: 14
• 22 Kaiser K, Miksch S. Versioning computer-interpretable guidelines: semi-automatic modeling of ʼLiving Guidelinesʼ using an information extraction method. Artif Intell Med 2009; 46: 55-66
• 23 Ollenschläger G, Kirchner H, Fiene M. Leitlinien in der Medizin – scheitern sie an der praktischen Umsetzung?. Internist 2001; 42: 473-483
• 24 Schlieter H. Ableitung von Klinischen Pfaden aus Medizinischen Leitlinien – Ein Modellbasierter Ansatz. Dresden: kummulativ; 2012
• 25 Jacobs B. Ableitung von Klinischen Pfaden aus evidenzbasierten Leitlinien am Beispiel der Behandlung des Mammakarzinoms der Frau [Dissertation]. Universität Duisburg-Essen; 2006
• 26 Follmann M, Wesselmann S, Nothacker M et al. Entwicklung von Leitlinien basierten Qualitätsindikatoren – Methodenpapier für das Leitlinienprogramm Onkologie. Leitlinienprogramm Onkologie (Deutsche Krebsgesellschaft (DKG), Deutsche Krebshilfe (DKH), Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften (AWMF)). 2013 Online: http://leitlinienprogramm-onkologie.de/uploads/tx_sbdownloader/QIEP_OL_2013.pdf last access: 11.04.2016
• 27 Semlitsch T, Blank WA, Kopp IB et al. Bewertung von Leitlinien: Ein Überblick über die wichtigsten Qualitätsaspekte. Dtsch Ärztebl 2015; 112: 471-478
• 28 User Group Clinical Guideline Services. Leitlinienportal. 2015 Online: https://www.guideline-service.de/ last access: 16.09.2015
• 29 Shiffman RN, Michel G, Rosenfeld RM et al. Building better guidelines with BRIDGE-Wiz: development and evaluation of a software assistant to promote clarity, transparency, and implementability. J Am Med Inform Assoc 2012; 19: 94-101
• 30 Cancer Council Australia. Cancer Guidelines Wiki. 2015 Online: http://wiki.cancer.org.au last access: 16.09.2015
• 31 Böckmann B, Heiden K. Extracting and transforming clinical guidelines into pathway models for different hospital information systems. Health Inf Sci Syst 2013; 1: 1-8
• 32 Leitlinienprogramm Onkologie der AWMF, Deutschen Krebsgesellschaft e. V. und Deutschen Krebshilfe e. V. Hrsg. Interdisziplinäre S3-Leitlinie für die Diagnostik, Therapie und Nachsorge des Mammakarzinoms (Langversion 3.0). 2012. Leitlinienregister: AWMF-Register-Nummer: 032–045OL. Online: http://www.awmf.org/leitlinien/detail/ll/032-045OL.html last access: 11.04.2016