OpenClinical logo

open@openclinical.org Please report any difficulties that may occur with the demonstration noting the problem, the computer and browser used.

DISCLAIMER

The applications accessible from this site are for demonstration purposes only. They have not been validated for clinical use and must not be used for real patient encounters.


Demonstrations of clinical applications
HOMEY     EU
Interactive spoken dialogue management system
Speech and natural language interfaces for point of care clinical services

developed by clinical domains keywords
Cancer Research UK (as part of the EU 5th Framework IST HOMEY project - Home Monitoring through an Intelligent Dialogue System). Ontology component supplied by Language and Computing NV, Belgium The current demonstration is designed to provide referrals advice in breast cancer care. A further system is under development in genetic risk assessment for breast cancer. Speech, Spoken dialogue, Dialogue Manager (DM), Intelligent Dialogue System (IDS), Interactive Voice Response (IVR), VoiceXML, Tele-medicine, clinical guidelines, Process modelling, PROforma technology, Ontology, Semantic network.
status access demonstrator
Experimental demonstrator Before runnning the demonstration, we recommend that you read through the introduction below.

The demonstration consists of a Macromedia Flash presentation in two parts. Part one provides an overview of the dialogue system architecture and the system's main components (see also figures 1-3 below). In part 2, the user is taken through a breast cancer referral scenario, illustrating the system in use.

To view the demonstration, you must have the latest Flash plugin installed on your computer, and a screen resolution of at least 1024x768.


 bullet   Homey Breast Cancer Referrals Demonstrator: Quicktime movie (17MB)
Introduction
This demonstrator has been built under the HOMEY project, funded by the EU IST Framework 5 programme. The main goal of the project is to develop practical spoken interfaces for clinical information systems. The clinical partners in HOMEY are the University of Pavia who have developed a system for capturing cardiology data provided by patients using the telephone, and Cancer Resarch UK who are researching principles of natural, flexible, robust mixed-initiative dialogue, demonstrating practical applications of such technology in clinical decision support services.

HOMEY technology forming the basis of the demonstration consists of:
  • A special-purpose dialogue manager which handles all the interactions with the user
  • A medical ontology and ontology server (provided by Language and Computing NV) which provides the dialogue manager with the required knowledge of cancer
  • A task manager built using PROforma technology which defines the clinical context for the dialogue
  • A VoiceXML interpreter and speech recogntion and synthesis software. In the demonstration, an "off the shelf" component (IBM Websphere VoiceServer) was used. (Within the Homey project, multilingual call-centre software is being developed by Reitek SpA and ITC-Irst in Italy.)

The demonstration you can access here illustrates the use of PROforma technology in a "remote consultation". In the demonstration scenario, a clinician is seeking advice on whether a woman with suspected breast cancer should be urgently referred to a specialist oncologist or not. Essentially the system takes a clinical history, then analyses the information about the patient and then recommends the appropriate action. The system is accessed by telephone, microphone or multi-modal browser. The demonstration illustrates a number of natural conversational features (e.g. accepting information in response to questions or volunteered, interpreting "non-standard" terms, giving explanations on request and so on).

The demonstration is in two parts: an explanatory overview of the system precedes an interactive dialogue demonstration. After the introduction, the system goes through the dialogue. The "clinician" and system take turns in a natural conversational pattern. You can control the dialogue yourself, selecting or repeating a step, or requesting further explanatory detail for each step as you wish. The dialogue is accompanied by explanatory voiceovers for the main exchanges, and technical animations for explanatory detail.

The demonstrator can be used in different ways:
  1. The first time through the demonstration, we recommend that you run the introduction then step through the dialogue using the "continue" button which appears after each step (this will take about 5 minutes).
  2. Once you have a sense of the complete dialogue you can play or replay segments as you wish, and click the "explain" button when you want more detail. The explanations consist of animations showing ontology search and PROforma task enactment.
  3. If you want to repeat a segment just click on the relevant piece of text.
  4. You may click the "?" button for instructions or the pause button at any time
Overview of the dialogue system architecture and its main components
The starting point for the development of the demonstration is the implementation of the guideline. In this case, a fragment from the UK Department of Health Referral guidelines for suspected cancer [London: DoH, 2000], is used.

A domain plan (figure 1) is developed to specify the high-level tasks to be carried out (take the patient history then make the decision). In the current implementation, the domain plan is encoded in the PROforma process specification language, developed by Cancer Research UK. This provides the basis for deriving the intentional structure of the dialogue.

PROforma representation
Figure 1: High level view of the dialogue system's domain plan.


spacer

A fragment of the domain ontology used is shown in figure 2. This ontology allows task-specific knowledge to be augmented with a conceptual model that describes general medical knowledge (links, relations etc. such as breast cancer is-a cancer, discharge is-symptom-of breast cancer.

Ontology
Figure 2: A part of the domain ontology of the dialogue system


spacer

The PROforma plan and the domain ontology are integrated in the dialogue system. The system's architecture (figure 3) allows both sources of knowledge to be used in determining the dialogue state at any point. First, the task and conceptual knowledge contained in the underlying medical technologies is mapped into an abstract task specification which defines all the domain knowledge required at any point for the dialogue. The task and conceptual structures of the abstract task specification are then mapped into a high-level dialogue specification. Currently this consists of a set of dialogue games which need to be played in order to complete tasks.

Architecture
Figure 3: System architecture

spacer

 

references
Beveridge M, Milward D. Combining task descriptions and ontological knowledge for adaptive dialogue. In: V. Matoušek and P. Mautner (Eds.) Proceedings of the 6th International Conference on Text, Speech and Dialogue (TSD’03), 8th –11th September, Ceské Budejovice, Czech Republic, Lecture Notes in Artificial Intelligence (LNAI 2807), Springer Verlag, Berlin, pp. 341 – 348.

[Springer]   [Paper - Cancer Research UK]

"This paper investigates the use of abstract task specifications for dialogue management in the medical domain. In most current dialogue systems, possible interactions with the system are hand-coded in the design. This is an expensive process, especially for complex dialogues. This paper motivates the use of a task description language for building flexible and adaptive dialogue systems in ontologically rich domains such as medicine. It describes the components of a task specification, and proposes an architecture for dialogue systems which allows integration of domain reasoning and dialogue. A high-level dialogue specification is used to support multimodal input and output, including generation of HTML pages, and generation of fragments of VoiceXML for spoken interaction. "
Beveridge M, Fox J. & Milward D. Speech Interfaces for Point-of-Care Guideline Systems. Proc. 9th Conference on Artificial Intelligence in Medicine in Europe (AIME-03), 18–22 October, Cyprus. Lecture Notes in Artificial Intelligence (LNAI 2780), Springer Verlag, Berlin, pp. 76–80.

[Springer]   [Paper - Cancer Research UK]

"A major limiting factor in the acceptability of interactive guideline and decision support systems is the ease of use of the system in the clinic. A way to reduce demands upon users and increase flexibility of the interface is to use natural language dialogues and speech based interfaces. This paper describes a voicebased data capture and decision support system in which knowledge of underlying task structure (a medical guideline) and domain knowledge (disease ontologies and semantic dictionaries) are integrated with dialogue models based on conversational game theory resulting in a flexible and configurable interface. "
Department of Health. Referral guidelines for suspected cancer. London: DoH, 2000.

[DoH]

"The Government White Paper entitled ‘The new NHS – Modern, Dependable’ guaranteed that everyone with suspected cancer will be able to see a specialist within two weeks of their GP deciding that they need to be seen urgently and requesting an appointment. ...
The aim of these guidelines is to facilitate appropriate referral between primary and secondary care for patients whom a GP suspects may have cancer. The guidelines should help GPs to identify those patients who are most likely to have cancer and who therefore require urgent assessment by a specialist. Equally it is hoped that the guidelines will help GPs to identify patients who are unlikely to have cancer and who may appropriately be observed in a primary care setting or who may require non-urgent referral to a hospital... "
contact links
John Fox
Advanced Computation Laboratory
Cancer Research UK
PO Box 123
Lincoln's Inn Fields
London WC2A 3PX UK
 bullet  Homey Breast Cancer Referrals Demonstrator: Macromedia Flash format  bullet  Homey Breast Cancer Referrals Demonstrator: Quicktime movie (17MB)  bullet  HOMEY project [OC]  bullet  PROforma specification language and technology [OC]  bullet  HOMEY project (co-ordinated by Engineering Ingegneria Informatica SpA, Rome)  bullet  Advanced Computation Laboratory, Cancer Research UK  bullet  Speech Technology Group, Laboratory for Medical Informatics, University of Pavia  bullet  Reitek SpA, Milan  bullet  ITC-Irst (Centro per la ricerca scientifica e tecnologica), Trento
acknowledgements
Martin Beveridge & John Fox, Cancer Research UK; David Milward, Linguamatics Ltd, Cambridge

Macromedia Flash presentation designed and implemented by Mirko Parmigiani, motion graphics designer, monkefilms.com. Contact: mirko@monke.co.uk

Entry on OpenClinical: 01 May 2004
Last main update: 24 May 2004
Search this site
 
Privacy policy User agreement Copyright Feedback

Last modified:
© Copyright OpenClinical 2002-2011