|
|
Methods and tools for the development of computer-interpretable guidelines |
|
| GEM |
|
|
XML-based guideline document model that can store and organize the
heterogeneous information contained in practice guideline documents.
GEM is an international ASTM standard for the representation of
practice guidelines in XML format.
|
|
| keywords |
Clinical practice guideline document model, XML, computer interpretable guidelines, markup, standards |
| developed by |
Yale Center for Medical Informatics at Yale University School of Medicine |
| introduced |
2000 |
| status |
International ASTM standard. In use / under continued development. |
| support |
US National Library of Medicine, the Robert Wood Johnson Foundation, and the National Institute
of Standards and Technology |
| in use |
|
| tools |
|
|
| description |
GEM is intended to facilitate the translation of natural language
guideline documents into a standard computer interpretable format. A strength of the format is that it encodes considerable
information about guideline recommendations in addition to the recommendations themselves, including the reason for each recommendation,
the quality of evidence that supports it, and the recommendation strength assigned by the developers.
GEM-encoding of guideline knowledge is pursued through a markup process that does not require programming knowledge.
The GEM Cutter application is an XML editor that facilitates guideline markup. The software (plus user manual) can be downloaded for evaluation purposes from the
GEM Web site at Yale University, subject to a license agreement.

View of GEM Cutter showing XML code for a guideline generated by the application.
GEM is intended to be used throughout the entire guideline lifecycle to model information pertaining to guideline development,
dissemination, implementation, and maintenance. Information at both high and low levels of abstraction can be accommodated.
GEM preserves the intent of guideline developers by marking up the actual guideline language.
GEM is constructed as a hierarchy with more than 100 discrete tags and more than 9 major branches (including Identity, Developer, Purpose, Intended Audience,
Target Population, Method of Development, Testing, Review Plan, and Knowledge Components), as shown in the screen below.
View of GEM Cutter illustrating part of the structure of the GEM model
Other GEM-related tools include GEM-COGs, a web-based application that generates an assessment report on whether a guideline
(encoded in GEM) meets the COGS checklist of required components of a practice guideline.
GEM-Q Online performs a similar function - generated reports are here based on the GQAQ (Guidelines Quality Assessment Questionnaire) rating instrument (published in Shaneyfelt et al, 1999).
In 2002, GEM was successfully balloted as an international ASTM standard (no. E2210-02) for the representation of
practice guidelines in XML format.
Members of the GEM team teaching at the Yale Center for Medical Informatics also participate in guideline modeling efforts through HL7.
|
|
| plans |
The GEM
team is currently investigating reusable methods (and creating tools) to
facilitate guideline development and implementation using GEM.
GEM II is also under development: the goal is to improve the comprehensiveness and usability of the
model whilst maintaining backward compatibility.
|
|
| references |
|
E2210-02 Standard Specification for Guideline Elements Model (GEM)-Document Model for Clinical Practice Guidelines
Developed by Subcommittee: E31.28
Book of Standards Volume: 14.01. ASTM International, 2003.
[ASTM]
[]
|
"
This specification covers a document type definition (DTD) that specifies a standard representation for storing and organizing the heterogeneous information contained in clinical practice guidelines. This specification is intended to facilitate translation of natural-language guideline documents into a format that can be processed by computers. It can be used to represent document content throughout the entire guideline life cycle. Information at both high and low levels of abstraction can be accommodated. This specification is based on the guideline elements model (GEM) created at the Yale Center for Medical Informatics and designed to serve as a comprehensive XML-based guideline document representation.
" |
Shiffman RN, Michel G, Essaihi A, Thornquist E.
Bridging the guideline implementation gap: a systematic, document-centered approach to guideline implementation.
J Am Med Inform Assoc. 2004 Sep-Oct;11(5):418-26.
[PubMed]
[]
|
"
OBJECTIVE: A gap exists between the information contained in published clinical practice guidelines and the knowledge and information that are necessary to implement them. This work describes a process to systematize and make explicit the translation of document-based knowledge into workflow-integrated clinical decision support systems. DESIGN: This approach uses the Guideline Elements Model (GEM) to represent the guideline knowledge. Implementation requires a number of steps to translate the knowledge contained in guideline text into a computable format and to integrate the information into clinical workflow. The steps include: (1) selection of a guideline and specific recommendations for implementation, (2) markup of the guideline text, (3) atomization, (4) deabstraction and (5) disambiguation of recommendation concepts, (6) verification of rule set completeness, (7) addition of explanations, (8) building executable statements, (9) specification of origins of decision variables and insertions of recommended actions, (10) definition of action types and selection of associated beneficial services, (11) choice of interface components, and (12) creation of requirement specification. RESULTS: The authors illustrate these component processes using examples drawn from recent experience translating recommendations from the National Heart, Lung, and Blood Institute's guideline on management of chronic asthma into a workflow-integrated decision support system that operates within the Logician electronic health record system. CONCLUSION: Using the guideline document as a knowledge source promotes authentic translation of domain knowledge and reduces the overall complexity of the implementation task. From this framework, we believe that a better understanding of activities involved in guideline implementation will emerge.n
" |
|
Shiffman RN, Michel G, Essaihi A. Markup once, reuse often: reusing
knowledge about appropriate practice with GEM-encoded clinical
guidelines. Proc Health Informatics New Zealand 2003.
[]
[]
|
"
" |
Georg G, Seroussi B, Bouaud J.
Extending the GEM model to support knowledge extraction from textual guidelines.
Int J Med Inform. 2005 Mar;74(2-4):79-87.
[PubMed]
[]
|
"
Clinical Practice Guidelines (CPGs) are being developed as a tool to promote best practice in medicine. However, the diffusion of paper guidelines has been shown to only have a limited impact. This is why computerization of CPGs has recently been suggested as a means to improve their dissemination as well as physicians' compliance. The Guideline Elements Model (GEM) has been proposed to facilitate the encoding of CPGs and support the automatic processing of marked-up documents. In this paper, we explore the automatic generation of a rule base from a textual guideline using GEM. In this study, we propose an extension of the GEM model that introduces additional levels of structuring centered on decision variables. This allows a more efficient representation of the decision processes, which supports the automatic generation of decision rules from textual guidelines. The 1999 Canadian recommendations for the management of hypertension have been marked-up as a GEM-encoded instance of our extended DTD. We derived a rule base using an XML parser to extract the relevant elements to instantiate the IF and THEN clauses of decision rules. The rule base automatically generated compares favourably with the manual generation of decision rules in the ASTI project. This approach is an interesting case study in the computerization of CPGs, as it illustrates processing steps that are relevant to the various aspects of CPGs life-cycle, from production to consultation and use.
"
|
|
Georg G, Seroussi B, Bouaud J.
Does GEM-Encoding Clinical Practice Guidelines Improve the Quality of
Knowledge Bases? A Study with the Rule-Based Formalism.
Proc AMIA Symp. 2003;:254-8.
[PubMed]
[]
|
"
The aim of this work was to determine whether the GEM-encoding step could improve the representation of clinical practice guidelines as formalized knowledge bases. We used the 1999 Canadian recommendations for the management of hypertension, chosen as the knowledge source in the ASTI project. We first clarified semantic ambiguities of therapeutic sequences recommended in the guideline by proposing an interpretative framework of therapeutic strategies. Then, after a formalization step to standardize the terms used to characterize clinical situations, we created the GEM-encoded instance of the guideline. We developed a module for the automatic derivation of a rule base, BR-GEM, from the instance. BR-GEM was then compared to the rule base, BR-ASTI, embedded within the critic mode of ASTI, and manually built by two physicians from the same Canadian guideline. As compared to BR-ASTI, BR-GEM is more specific and covers more clinical situations. When evaluated on 10 patient cases, the GEM-based approach led to promising results.
" |
|
Shiffman RN, Agrawal A, Deshpande AM, Gershkovich P.
An approach to guideline implementation with GEM.
Patel V, ed., Proceedings Medinfo 2001;10(Pt 1):271-5.
[PubMed]
[]
|
"
Implementation of practice guidelines refers to the creation of strategies and systems to operationalize the knowledge and recommendations set forth by guideline developers. We describe an approach to guideline implementation that makes direct use of the guideline document as a knowledge base. The Guideline Elements Model (GEM) provides an XML-based guideline document model that facilitates implementation of guidelines. Knowledge extraction using GEM requires document markup rather than programming and can promote authenticity and consistent knowledge encoding. Knowledge customization for the local enterprise requires addition of meta-information to pertinent components of the GEM hierarchy in a design database. GEM provides an audit trail to track local adaptation. Knowledge integration with patient data can be promoted using information management services. A design goal is to devise a system that can be applied by local clinical domain experts, quality assurance experts, and information systems programmers without requiring trained informaticians and knowledge engineers to serve as intermediaries
" |
|
Agrawal A, Shiffman RN.
Evaluation of guideline quality using GEM-Q.
Patel V, ed., Proceedings MEDINFO 2001;1097-1101.
[PubMed]
[]
|
"
A variety of rating instruments that evaluate the quality of practice guidelines have been published. Application of these instruments can be difficult and time-consuming. In a literature review, we identified two evaluation instruments that are comprehensive, have clearly defined constructs, and have undergone validation/testing-the Guidelines Quality Assessment Questionnaire (GQAQ) and the Appraisal Instrument for Clinical Guidelines (AICG). Overall, the AICG is more comprehensive. The AICG addresses the implementability of a guideline, which is not evaluated by the GQAQ. However, the GQAQ is more amenable to computerization. GEM-Q is a Guideline Elements Model (GEM)-derived application intended to facilitate automated evaluation of guideline quality using one of the published instruments. To develop GEM-Q, various items in the GQAQ were mapped to corresponding elements in the GEM hierarchy and a customized XSL stylesheet was designed based on this mapping. GEM-Q selectively extracts text components of the guideline relevant to quality evaluation and displays the results in HTML format. GEM-Q was applied to a set of six guidelines to test its reliability. It ranked two guidelines as of "good" quality, two as "intermediate", and two as "poor". In all six instances, GEM-Q ranked guidelines in the same order of quality as the experts who validated the GQAQ. This work demonstrates the feasibility of developing an application to facilitate automated guideline quality evaluation.
" |
Shiffman RN, Michel G.
Toward improved guideline quality: using the COGS statement with GEM.
Medinfo. 2004;11(Pt 1):159-63.
[PubMed]
[]
|
"
The Conference on Guideline Standardization (COGS) was convened to create a standardized documentation checklist for clinical practice guidelines in an effort to promote guideline quality and facilitate implementation. The statement was created by a multidisciplinary panel using a rigorous consensus development methodology. The Guideline Elements Model (GEM) provides a standardized approach to representing guideline documents using XML. In this work, we demonstrate the sufficiency of GEM for describing COGS components. Using the mapping between COGS and GEM elements we built an XSLT application to examine a guideline's adherence (or non-adherence) to the COGS checklist. Once a guideline has been marked up according to the GEM hierarchy, its knowledge content can be reused in multiple ways.
"
|
|
Agrawal A, Shiffman RN.
Using GEM-encoded guidelines to generate medical logic modules.
Bakken S, ed. Proc AMIA Symp 2001;7-11.
[PubMed]
[]
|
"
Among the most effective strategies for changing the process and outcomes of clinical care are those that make use of computer-mediated decision support. A variety of representation models that facilitate computer-based implementation of medical knowledge have been published, including the Guideline Elements Model (GEM) and the Arden Syntax for Medical Logic Modules (MLMs). We describe an XML-based application that facilitates automated generation of partially populated MLMs from GEM-encoded guidelines. These MLMs can be further edited and shared among Arden-compliant information systems to provide decision support. Our work required three steps: (a) Knowledge extraction from published guideline documents using GEM, (b) Mapping GEM elements to the MLM slots, and (c) XSL transformation of the GEM-encoded guideline. Processing of a sample guideline generated 15 MLMs, each corresponding to a conditional or imperative element in the GEM structure. Mechanisms for linking various MLMs are necessary to represent the complexity of logic typical of a guideline.
" |
|
Gershkovich P, Shiffman RN.
An implementation framework for GEM encoded guidelines.
Bakken S, ed. Proc AMIA Symp 2001; 204-8.
[PubMed]
[]
|
"
Access to timely decision support information is critical for delivery of high-quality medical care. Transformation of clinical knowledge that is originally expressed in the form of a guideline to a computable format is one of the main obstacles to the integration of knowledge sharing functionality into computerized clinical systems. The Guideline Element Model (GEM) provides a methodology for such a transformation. Although the model has been used to store heterogeneous guideline knowledge, it is important to demonstrate that GEM markup facilitates guideline implementation. This report demonstrates the feasibility of implementation of GEM-encoded guideline recommendations using Apache Group's Cocoon Web Publishing Framework. We further demonstrate how XML-based programming allows for maintaining the separation of guideline content from processing logic and from presentation format. Finally, we analyze whether the guideline authors original intent has been sufficiently captured and conveyed to the end user.
" |
|
Shiffman RN, Karras BT, Agrawal A, Chen R, Marenco L, Nath S.
GEM: a proposal for a more comprehensive guideline document model using XML.
J Am Med Inform Assoc. 2000 Sep-Oct;7(5):488-98.
[PubMed]
[PubMedCentral]
|
"
OBJECTIVE: To develop a guideline document model that includes a sufficiently broad set of concepts to be useful throughout the guideline life cycle. DESIGN: Current guideline document models are limited in that they reflect the specific orientation of the stakeholder who created them; thus, developers and disseminators often provide few constructs for conceptualizing recommendations, while implementers de-emphasize concepts related to establishing guideline validity. The authors developed the Guideline Elements Model (GEM) using XML to better represent the heterogeneous knowledge contained in practice guidelines. Core constructs were derived from the Institute of Medicine's Guideline Appraisal Instrument, the National Guideline Clearinghouse, and the augmented decision table guideline representation. These were supplemented by additional concepts from a literature review. RESULTS: The GEM hierarchy includes more than 100 elements. Major concepts relate to a guideline's identity, developer, purpose, intended audience, method of development, target population, knowledge components, testing, and review plan. Knowledge components in guideline documents include recommendations (which in turn comprise conditionals and imperatives), definitions, and algorithms. CONCLUSION: GEM is more comprehensive than existing models and is expressively adequate to represent the heterogeneous information contained in guidelines. Use of XML contributes to a flexible, comprehensible, shareable, and reusable knowledge representation that is both readable by human beings and processible by computers.
" |
|
B.T. Karras, S.D. Nath, R.N. Shiffman.
A preliminary evaluation of guideline content markup using GEM - an XML guideline elements model.
In: Proc. AMIA Annual Symposium, 2000.
[PubMed]
[AMIA]
|
"
OBJECTIVE: To describe application of GEM to analysis and categorization of guideline content. METHOD: We examined the application of GEM constructs to the AAP guideline on neurodiagnostic evaluation of febrile seizures. Subjects at 4 sites marked-up the guideline content using a hierarchical template that includes branches for identity, developer, purpose, intended audience, method of development, knowledge components, testing, and review. The types of elements used were tabulated. Subjects were surveyed regarding the usability of the model. RESULTS: Eight subjects analyzed the guideline, using between 46 and 149 elements to model its content. There was considerable variation in the application of elements. The number of elements used correlated with time to complete the task. Subjects found application of GEM to be straightforward in 6 of 8 categories and sufficiently comprehensive to model the guideline's information content. CONCLUSIONS: Subjects found GEM constructs were able to model the content of the guideline. Improved editing tools will facilitate translation.
" |
Shaneyfelt TM, Mayo-Smith MF, Rothwangl J.
Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature.
JAMA. 1999 May 26;281(20):1900-5.
[PubMed]
[]
|
"
CONTEXT: Practice guidelines play an important role in medicine. Methodological principles have been formulated to guide their development. OBJECTIVE: To determine whether practice guidelines in peer-reviewed medical literature adhered to established methodological standards for practice guidelines. DESIGN: Structured review of guidelines published from 1985 through June 1997 identified by a MEDLINE search. MAIN OUTCOME MEASURES: Mean number of standards met based on a 25-item instrument and frequency of adherence. RESULTS: We evaluated 279 guidelines, published from 1985 through June 1997, produced by 69 different developers. Mean overall adherence to standards by each guideline was 43.1% (10.77/25). Mean (SD) adherence to methodological standards on guideline development and format was 51.1% (25.3%); on identification and summary of evidence, 33.6% (29.9%); and on the formulation of recommendations, 46% (45%). Mean adherence to standards by each guideline improved from 36.9% (9.2/25) in 1985 to 50.4% (12.6/25) in 1997 (P<.001). However, there was little improvement over time in adherence to standards on identification and summary of evidence from 34.6% prior to 1990 to 36.1 % after 1995 (P = .11). There was no difference in the mean number of standards satisfied by guidelines produced by subspecialty medical societies, general medical societies, or government agencies (P = .55). Guideline length was positively correlated with adherence to methodological standards (P = .001). CONCLUSION: Guidelines published in the peer-reviewed medical literature during the past decade do not adhere well to established methodological standards. While all areas of guideline development need improvement, greatest improvement is needed in the identification, evaluation, and synthesis of the scientific evidence.
"
|
|
|
| contact |
Prof. Richard Shiffman
Yale Center for Medical Informatics
PO Box 208009
New Haven CT 06520-8009, USA
W: Prof. Richard Shiffman
|
| links |
|
|
| acknowledgements |
| Rick Shiffman and the GEM team at Yale University. |
| page history |
Entry on OpenClinical: January 2002
Last main update: 13 February 2004.
Design - template v0.3: 25 June 2005. |
|
|
|
|