OpenClinical logo

Clinical demonstrators

Methods and tools for the development of computer-interpretable guidelines

the Guideline Interchange Format
GLIF is a computer-interpretable language for modeling and executing clinical practice guidelines. GLIF supports sharing of computer-interpretable clinical guidelines across different medical institutions and system platforms. GLIF has a formal representation. It defines an ontology for representing guidelines, as well as a medical ontology for representing medical data and concepts. Tools are under development to support guideline authoring and execution.
keywords Clinical guidelines, computer-interpretable clinical guidelines, guideline model, decision-support systems, sharing, InterMed Collaboratory, GLEE, GESDOR, GELLO
developed by InterMed Collaboratory (= "collaboration" + "laboratory")(Stanford Medical Informatics, Harvard University, McGill University and Columbia University).
introduced GLIF2 (the first published version of GLIF): 1998.
GLIF3: 2000
status In use / under continued development
support US National Library of Medicine, the Agency for Healthcare Research and Quality and the US Army.
in use Members of the InterMed Collaboratory (Dongwen Wang at Columbia University and Mor Peleg at the University of Haifa, Israel) are undertaking research into the local adaptation, versioning [Peleg & Kantor, 2003] and EMR integration of GLIF3-encoded guidelines. A diabetes foot guideline has been locally adapted to the setting of Israeli primary care physicians in outpatient clinics, linked with a web-based EMR, and enacted using the GLEE execution engine [Wang & Shortliffe, 2002].

GLIF/GLEE have been used at Columbia University for post-CABG (Coronary Artery Bypass Grafting) patient care planning. The integration of GLIF/GLEE with the clinical information system at the Columbia-Presbyterian Hospital as an infrastructure for clinical decision support is being explored.
tools  bullet  Authoring and validation tool created using the Protégé knowledge-modeling environment. The tool runs by executing Protégé and loading the GLIF ontology.  bullet  GLIF ontology for authoring and validation stored as a Protágé project
GLIF2 [Ohno-Machado et al, 1998] was designed to support guideline modelling as a flowchart of structured steps which represented clinical actions and decisions. However, the attributes of these constructs were defined as text strings that could not be parsed, preventing the resulting guidelines from being able to make inferences during computerised execution.

[Click for larger image.]

Part of the GLIF encoding of guideline for the management of stable angina. completed using the Protégé knowledge modeling tool. The "Education and risk factor modification" action step on the top left is nested into a subguideline, shown on the right. The subguideline's algorithm is a network of guideline steps. Action steps are represented by green squares, and decisions are represented as diamonds. Blue diamonds represent deterministic decisions that are automated (case step), whereas purple diamonds represents a choice that should be decided by a clinician (choice step). Double clicking on any of the objects (nodes or arcs) in the algorithm shows further details of the object's attribute. The details of the case step "High Cholesterol?" are shown in the insert on the bottom left. The criterion "LDL_Cholesterol> 160 mg/DL" is compared to two options: yes and no using the operator "equals". Control flow is directed from the case step to other guideline steps depending on the comparison result. The "didactics" slot holds the URL of the Stable Angina narrative guideline.

GLIF3 augments the GLIF2 specification with several new constructs, and requires a more formal definition of decision criteria, action specifications and patient data. The most significant extensions of GLIF2 implemented in GLIF3 are:
  • Inclusion of a formal expression language for specifying decision criteria and patient states. In the period of time before May 2001, GLIF had been using an expression language called GEL [Peleg et al, 2001], based on the Arden Syntax's logic grammar. Later, GELLO [Sordo et al, 2004; Ogunyemi et al, 2001], an object-oriented expression language, was developed. GELLO is better suited for GLIF's object-oriented data model, is extensible and allows implementation of expressions that are not supported by the Arden Syntax.
  • A layered patient data model to enable GLIF3 steps to refer to patient data items defined by a controlled terminology that includes standard medical vocabularies (such as UMLS), as well as standard data models for medical data (such as HL7's Reference Information Model, or the "virtual medical record" (vMR)that is being developed by HL7 to provide a standard data model as an intermediary to heterogeneous medical record systems).
An action specification hierarchy has also been implemented.

GLIF3 has been designed to support computer-based guideline execution: it has a computable level of specification which formally defines logical criteria, definitions of patient data items, clinical actions and the flow of the guidelines. The computable level of the specification may be regarded as coming between the abstract flowchart level (supported by GLIF2) and the implementation level (currently only partly supported by GLIF3). The abstract flowchart level helps authors and users view and understand a guideline. The implementation level includes non-shareable, institution-specific details which enable guidelines to be incorporated into operational clinical information systems. Shareable components of a guideline are therefore explicitly separated from institution-specific or vendor platform-specific (non-shareable) components.

GLIF's object-oriented query and expression language for clinical decision support, GELLO, was successfully balloted by HL7 in 2005 for incorporation as a standard.

Standardization of guideline control-flow by HL7 also draws upon the GLIF model of linked guideline steps. InterMed members are active participants of the HL7 Clinical Guidelines Special Interest Group and the Clinical Decision Support Technical Committee, thus contributing to the process of standardization of a shareable guideline modeling language, which draws upon experiences from the GLIF project.
current work and plans
The Guideline Execution Engine (GLEE) has been developed to interpret guidelines encoded in the GLIF3 format and to integrate with clinical information systems for guideline implementation.

At Columbia University, GLEE is being integrated with the Clinical Event Monitor and the computerized physician order entry (CPOE) system to provide clinical decision support. GLEE is also being tested for quality assurance purposes - to examine the potential deviation of the care of specific patients from the standard.

Haifa University is developing:

  • a process for local adaptation of GLIF-encoded guidelines
  • a mapping ontology from GLIF-encoded guidelines to EMRs.


Boxwala AA, Peleg M, Tu S et al. GLIF3: a representation format for sharable computer-interpretable clinical practice guidelines. J Biomed Inform. 2004 Jun;37(3):147-61.

[PubMed]   [ScienceDirect]

" The Guideline Interchange Format (GLIF) is a model for representation of sharable computer-interpretable guidelines. The current version of GLIF (GLIF3) is a substantial update and enhancement of the model since the previous version (GLIF2). GLIF3 enables encoding of a guideline at three levels: a conceptual flowchart, a computable specification that can be verified for logical consistency and completeness, and an implementable specification that is intended to be incorporated into particular institutional information systems. The representation has been tested on a wide variety of guidelines that are typical of the range of guidelines in clinical use. It builds upon GLIF2 by adding several constructs that enable interpretation of encoded guidelines in computer-based decision-support systems. GLIF3 leverages standards being developed in Health Level 7 in order to allow integration of guidelines with clinical information systems. The GLIF3 specification consists of an extensible object-oriented model and a structured syntax based on the resource description framework (RDF). Empirical validation of the ability to generate appropriate recommendations using GLIF3 has been tested by executing encoded guidelines against actual patient data. GLIF3 is accordingly ready for broader experimentation and prototype use by organizations that wish to evaluate its ability to capture the logic of clinical guidelines, to implement them in clinical systems, and thereby to provide integrated decision support to assist clinicians. "
M. Peleg, A. A. Boxwala, O. Ogunyemi et al. GLIF3: The Evolution of a Guideline Representation Format. In: Proc. AMIA Annual Symposium, 2000.



"GLIF3 is a new version of GLIF designed to support computer-based execution. GLIF3 builds upon the framework set by GLIF2 but augments it by introducing several new constructs and extending GLIF2 constructs to allow a more formal definition of decision criteria, action specifications and patient data. GLIF3 enables guideline encoding at three levels: a conceptual flowchart, a computable specification that can be verified for logical consistency and completeness, and an implementable specification that can be incorporated into particular institutional information systems."
Ohno-Machado L, Gennari JH, Murphy SN, et al. The guideline interchange format: a model for representing guidelines. Journal of the American Medical Informatics Association 5(4):357-372, 1998.

[PubMed] [PubMed Central]

This paper provides: a discussion of the development of a common representation for guidelines to support the move from paper-based guidelines to computer-based guidelines; a description of currently recognized guideline types and formats; a description of the GLIF development process, including an analysis of four precursor guideline systems that contributed to that development; a discussion of a pilot study in which four clinical practice guidelines were encoded "to assess its expressivity and to study the variability that occurs when two people from different sites encode the same guideline".
Peleg M, Boxwala AA, Tu S et al. The InterMed approach to sharable computer-interpretable guidelines: a review. J Am Med Inform Assoc. 2004 Jan-Feb;11(1):1-10.

[PubMed]   [JAMIA]

" InterMed is a collaboration among research groups from Stanford, Harvard, and Columbia Universities. The primary goal of InterMed has been to develop a sharable language that could serve as a standard for modeling computer-interpretable guidelines (CIGs). This language, called GuideLine Interchange Format (GLIF), has been developed in a collaborative manner and in an open process that has welcomed input from the larger community. The goals and experiences of the InterMed project and lessons that the authors have learned may contribute to the work of other researchers who are developing medical knowledge-based tools. The lessons described include (1) a work process for multi-institutional research and development that considers different viewpoints, (2) an evolutionary lifecycle process for developing medical knowledge representation formats, (3) the role of cognitive methodology to evaluate and assist in the evolutionary development process, (4) development of an architecture and (5) design principles for sharable medical knowledge representation formats, and (6) a process for standardization of a CIG modeling language. "

Wang D, Peleg M, Tu SW et al. Design and implementation of the GLIF3 guideline execution engine. J Biomed Inform. 2004 Oct;37(5):305-18.

[PubMed]   []

" We have developed the GLIF3 Guideline Execution Engine (GLEE) as a tool for executing guidelines encoded in the GLIF3 format. In addition to serving as an interface to the GLIF3 guideline representation model to support the specified functions, GLEE provides defined interfaces to electronic medical records (EMRs) and other clinical applications to facilitate its integration with the clinical information system at a local institution. The execution model of GLEE takes the "system suggests, user controls" approach. A tracing system is used to record an individual patient's state when a guideline is applied to that patient. GLEE can also support an event-driven execution model once it is linked to the clinical event monitor in a local environment. Evaluation has shown that GLEE can be used effectively for proper execution of guidelines encoded in the GLIF3 format. When using it to execute each guideline in the evaluation, GLEE's performance duplicated that of the reference systems implementing the same guideline but taking different approaches. The execution flexibility and generality provided by GLEE, and its integration with a local environment, need to be further evaluated in clinical settings. Integration of GLEE with a specific event-monitoring and order-entry environment is the next step of our work to demonstrate its use for clinical decision support. Potential uses of GLEE also include quality assurance, guideline development, and medical education. "

Choi J, Currie LM, Wang D, Bakken S Encoding a clinical practice guideline using guideline interchange format: A case study of a depression screening and management guideline. [JOURNAL ARTICLE] Int J Med Inform 2007 Jun 26.

[PubMed]   []

" PURPOSE: Clinical practice guidelines (CPGs) are common tools for clinicians in daily practice. In order to use CPGs effectively at the point of care, representing CPGs into computer-interpretable format is essential. Since computer-interpretable guidelines (CIGs) have been reported to increase clinicians' usage of guidelines and improve patient's outcomes, it is critical to assess health care knowledge translated from CPGs into CIGs. The overall goal of this study was to illustrate the steps involved in encoding a guideline in guideline interchange format-3 (GLIF3) through a case study of a depression screening and management CPG for a nursing decision support system (DSS). METHODS: This study consisted of five steps: (1) Selection of a CPG; (2) extraction and categorization in GLIF3 of concepts related to depression screening and management tasks; (3) converting GLIF3 steps to a flowchart; (4) creation of scenarios representing possible execution paths; (5) guideline execution engine (GLEE) execution of scenarios. DISCUSSION: This study contributes to the body of knowledge regarding creation of CIGs and the use of GLEE as an evaluation tool for the encoded CIG in GLIF format for a depression CPG. CPG representation using GLIF3 and its evaluation by GLEE are useful methods to prepare nursing CPGs for implementation in a DSS. "

Wang D, Peleg M, Bu D et al. GESDOR – a generic execution model for sharing of computer-interpretable clinical practice guidelines. Proc AMIA Symp. 2003;:694-8.

[PubMed]   []

" We developed the Guideline Execution by Semantic Decomposition of Representation (GESDOR) model to share guidelines encoded in different formats at the execution level. For this purpose, we extracted a set of generalized guideline execution tasks from the existing guideline representation models. We then created the mappings between specific guideline representation models and the set of the common guideline execution tasks. Finally, we developed a generic task-scheduling model to harmonize the existing approaches to guideline task scheduling. The evaluation has shown that the GESDOR model can be used for the effective execution of guidelines encoded in different formats, and thus realizes guideline sharing at the execution level. "

Wang D, Shortliffe EH. GLEE - a model-driven execution system for computer-based implementation of clinical practice guidelines. Proc AMIA Symp. 2002;:855-9.

[PubMed]   []

" We have developed the GLEE system for execution of guidelines encoded in the GLIF3 format. This system can be integrated with a local clinical information system through standard interfaces to EMRs and clinical applications. The execution model of GLEE takes the "system suggests, user controls" approach. A tracing system is used to record the state of guideline steps and their transitions. GLEE provides an internal event-driven execution model that can be hooked up with the clinical event monitor in a local environment. We discuss the execution flexibility provided by GLEE and issues related to its integration in a local environment. Potential use of GLEE includes clinical decision support, quality assurance, guideline development and medical education. "
Sordo M, Ogunyemi O, Boxwala AA, Greenes RA, Tu S. GELLO: An Object-Oriented Query and Expression Language for Clinical Decision Support. Summary Report prepared for OpenClinical, 17 March 2004.

" GELLO is designed to be a standard query and expression language for decision support. Its specification has been developed in coordination with the HL7 Clinical Decision Support TC (CDSTC). The effort, begun in 2001, has been carried out with input from other TCs and SIGs as well, in order to take account of common needs for constraint specification and query formulation, and the following groups have been consulted in developing the specification: Control/Query, Modeling and Methodology, and Templates. ... "
Ogunyemi O, Zeng Q, Boxwala AA. BNF and built-in classes for object-oriented guideline expression language (GELLO). Technical Report: Brigham and Women's Hospital; 2001. Report No.: DSG-TR-2001-018.


GELLO is an object-oriented expression language that is used to specify decision and eligibility criteria in GLIF. It was developed by InterMed in collaboration with the HL7 Clinical Decision Support Technical Committee, and is now being considered as an HL7 standard. GELLO functions are implemented as methods of classes. The classes that GELLO supports include data types, collections, medical record entities, concepts and their relationships, and utility classes that perform operations on the other GELLO classes (e.g., arithmetic, logical, comparison, presence of patient data items, and temporal operations). GELLO also includes query language constructs similar to those of the Object Query Language (OQL). This paper presents the syntax of GELLO in BNF format.
Peleg M, Kantor R. Approaches for Guideline Versioning Using GLIF. Proc AMIA Symp. 2003;:509-13.

[PubMed]   [paper - U. Haifa]

" Computer-interpretable clinical guidelines (CIGs) aim to eliminate clinician errors, reduce practice variation, and promote best medical practices by delivering patient-specific advice during patient encounters. Clinical guidelines are being regularly updated and revised to handle expanding clinical knowledge. When revising CIGs, much effort can be saved by specifying changes among versions instead of encoding revised guidelines from scratch. A representation of differences between versions could focus the process of re-implementing CIGs in a clinical environment and help users understand and embrace changes. Guideline versioning has not been adequately dealt with by existing CIG formalisms. We present three approaches for CIG versioning. Focusing on one approach, we developed a versioning tool based on version 3 of the GuideLine Interchange Format (GLIF3), and used it to represent two guideline versions for management of community-acquired pneumonia (CAP) and the changes between them. "
Peleg M, Ogunyemi O, Tu S et al. Using features of Arden Syntax with object-oriented medical data models for guideline modeling. Proc AMIA Symp 2001;:523-7.

[PubMed] []

"In developing version 3 of the GuideLine Interchange Format (GLIF3), we used existing standards as the medical data model and expression language. We investigated the object-oriented HL7 Reference Information Model (RIM) as a default data model. We developed an expression language, called GEL, based on Arden Syntax's logic grammar. Together with other GLIF constructs, GEL reconciles incompatibilities between the data models of Arden Syntax and the HL7 RIM. These incompatibilities include Arden's lack of support for complex data types and time intervals, and the mismatch between Arden's single primary time and multiple time attributes of the HL7 RIM."

Kolesa P, Spidlen J, Zvarova J. Obstacles to Implementing an Execution Engine for Clinical Guidelines Formalized in GLIF. Stud Health Technol Inform. 2005;116:563-8.

[PubMed]   []

" This article is on obstacles we faced when developing an executable representation of guidelines formalized the Guideline Interchange Format (GLIF). The GLIF does not fully specify the representation of guidelines at the implementation level as it is focused mainly on the description of guideline's logical structure. Our effort was to develop an executable representation of guidelines formalized in GLIF and to implement a pilot engine, which will be able to process such guidelines. The engine has been designed as a component of the MUltimedia Distributed Record system version 2 (MUDR(2)). When developing executable representation of guidelines we paid special attention to utilisation of existing technologies to achieve the highest reusability.Main implementation areas, which are not fully covered by GLIF, are a data model and an execution language. Concerning the data model we have decided to use MUDR(2)'s native data model for this moment and to keep watching the standardisation of a virtual medical record to implement it in execution engine in the near future. When developing the execution language, first of all we have specified necessities, which the execution language ought to meet. Then we have considered some of the most suitable candidates: Guideline Execution Language (GEL), GELLO, Java and Python. Finally we have chosen GELLO although it does not completely cover all required areas. The main GELLO's advantage is that it is a proposed HL7 standard. In this paper we show some of the most important disadvantages of GELLO as an executable language and how we have solved them. "

references: GLIF in use

Mor Peleg, Dongwen Wang, Adriana Fodor et al. Adaptation of Practice Guidelines for Clinical Decision Support: A Case Study of Diabetic Foot Care. Proceeding of the the biennial European Conference on Artificial Intelligence (ECAI) 2006 Workshop: AI techniques in healthcare: computerized guidelines and protocols, Riva del Garda, Italy.

[]   [U Haifa]

" Clinical guidelines usually need to be adapted to fit local practice before they can be actually used by clinicians. Reasons for adaptation include variations of institution setting such as type of practice and location, availability of resources, difference of patient populations, local policies, and practice patterns. When a guideline is implemented for clinical decision support and integrated with an institution's clinical information system, the data model of the local electronic medical record (EMR) and the data actually collected and stored in it also influence the guideline's adaptation. The purpose of this work is: (1) to characterize a tool-supported process for guideline encoding that addresses local adaptation and EMR integration, and (2) to identify the types of changes in guideline encoding during the local adaptation process. "

Choi J, Sapp J, Bakken S. Encoding a depression screening guideline using GLIF. Stud Health Technol Inform. 2006;122:905-6.

[PubMed]   []

" Computer-interpretable guidelines (CIGs) have been reported to increase clinicians' usage of guidelines and to improve patient outcomes. The Guideline Interchange Format (GLIF) represents a guideline as a flowchart of clinical decision and action steps. We encoded a depression screening guideline using GLIF to transform it into a CIG. After identifying all guideline steps from the text format guideline, a flowchart was drawn using the encoded steps in GLIF. The flowchart was validated by an experienced psychiatric nurse clinician. "

Choi J, Bakken S. Creation of a gold standard for validation of a computer-interpretable depression screening guideline. Stud Health Technol Inform. 2006;122:95-9.

[PubMed]   []

" Clinical practice guidelines (CPGs) are common tools for clinicians in daily practice. In order to use CPGs effectively at the point of care, representing CPGs into computer-interpretable format is essential. Since computer-interpretable guidelines (CIGs) have been reported to increase clinicians' usage of guidelines and improve patient's outcomes, it is critical to validate health care knowledge translated from CPGs into CIGs. A comprehensive method of developing and evaluating a gold standard for a depression screening CIG was performed and analyzed in this study. GLIF encoding was conducted for a depression screening CPG and 21 clinical scenarios were created based on the encoded depression screening CIG. Two nurse practitioners were recruited to generate initial management for each scenario. Proficiency and efficiency scores were calculated for each scenario. In 13 of 21 scenarios, both experts had proficiency scores of 100%. Proficiency scores were lowest for scenarios that included bipolar disease. Implications of our findings for development of a gold standard are discussed. "

Choi J, Bakken S. Comparison of primary care expert and computer-interpretable depression screening guideline recommendations. AMIA Annu Symp Proc. 2006;:887.

[PubMed]   [PubMed Central]

" Computer-interpretable guidelines (CIGs) have been reported to increase clinicians inverted exclamation mark usage of guidelines and to improve patient outcomes. We encoded a depression screening guideline using Guideline Interchange Format (GLIF) to transform it into a CIG. We compared primary care (PC) expert and CIG performance on 21 scenarios.completeness of translation were assessed by comparing expert and CIG performance on 21 scenarios. "

contact Mor Peleg
Department of Management Information Systems
Rabin Bldg., room 7047
Faculty of Social Sciences
University of Haifa
Israel, 31905



Dongwen Wang
Department of Biomedical Informatics
Columbia University
622 West 168th Street
New York
NY 10032

links  bullet  GLIF  bullet  The InterMed Collaboratory  bullet  An introduction to GLIF - presentation by Mor Peleg and Aziz Boxwala  bullet  GLIF: A language for sharing and executing clinical guidelines (presentation from OpenClinical one-day workshop, Methods for the Representation of Computer-Interpretable Clinical Guidelines, London, September 6 2001)  bullet  The GLIF3 Guideline Execution Engine (GLEE)  bullet  The GESDOR Model (Guideline Execution by Semantic Decomposition Of Representation)  bullet  GELLO object-oriented expression language [OC]  bullet  HL7  bullet  HL7 Reference Information Model  bullet  Arden Syntax [OC]
Mor Peleg, Haifa University, Israel; Dongwen Wang, Columbia University, New York, USA.
page history
Entry on OpenClinical: 2001
Last main updates: 31 July 2002; 17 March 2004; 06 March 2006.


Search this site
Privacy policy User agreement Copyright Feedback

Last modified:
© Copyright OpenClinical 2002-2011