| Demonstrations of clinical applications
|
HOMEY 
Interactive spoken dialogue management system
|
Speech and natural language interfaces for point of care clinical services
| developed by |
clinical domains |
keywords |
| Cancer Research UK (as part of the EU 5th Framework IST HOMEY project - Home Monitoring through an Intelligent Dialogue System). Ontology component supplied by Language and Computing NV, Belgium |
The current demonstration is designed to provide referrals advice in breast cancer care.
A further system is under development in genetic risk assessment for breast cancer. |
Speech, Spoken dialogue, Dialogue Manager (DM), Intelligent Dialogue System (IDS), Interactive Voice Response (IVR), VoiceXML, Tele-medicine, clinical guidelines, Process modelling, PROforma technology, Ontology, Semantic network. |
| status |
access demonstrator |
| Experimental demonstrator
|
Before runnning the demonstration, we recommend that you read through the introduction below.
The demonstration consists of a Macromedia Flash presentation in two parts. Part one
provides an overview of the dialogue system architecture and the system's main components
(see also figures 1-3 below). In part 2, the user
is taken through a breast cancer referral scenario, illustrating the system in use.
To view the demonstration, you must have the latest Flash plugin installed on your computer, and a screen resolution of at least 1024x768.
|
| Introduction |
This demonstrator has been built under the HOMEY project, funded by the EU IST Framework 5 programme.
The main goal of the project is to develop practical spoken interfaces for clinical
information systems.
The clinical partners in HOMEY are the
University of Pavia who have developed a system for capturing
cardiology data provided by patients using the telephone, and Cancer Resarch
UK who are researching principles of natural, flexible, robust
mixed-initiative dialogue, demonstrating practical applications of such
technology in clinical decision support services.
HOMEY technology forming the basis of the demonstration consists of:
- A special-purpose dialogue manager which handles all the interactions
with the user
- A medical ontology and ontology server (provided by Language and
Computing NV) which provides the dialogue manager with the required knowledge of
cancer
- A task manager built using PROforma technology which defines the clinical context for the
dialogue
- A VoiceXML interpreter and speech
recogntion and synthesis software. In the demonstration,
an "off the shelf" component (IBM Websphere
VoiceServer) was used. (Within the Homey project,
multilingual call-centre software is being developed
by Reitek SpA and ITC-Irst in Italy.)
The demonstration you can access here illustrates the use of PROforma technology
in a "remote consultation". In the demonstration scenario, a clinician is
seeking advice on whether a woman with suspected breast cancer should be
urgently referred to a specialist oncologist or not. Essentially the system
takes a clinical history, then analyses the information about the patient
and then recommends the appropriate action. The system is accessed by
telephone, microphone or multi-modal browser. The demonstration illustrates
a number of natural conversational features (e.g. accepting information in
response to questions or volunteered, interpreting "non-standard" terms,
giving explanations on request and so on).
The demonstration is in two parts: an explanatory overview of the
system precedes an interactive dialogue demonstration. After the
introduction, the system goes through the dialogue. The "clinician" and system take turns in a natural
conversational pattern. You can control the dialogue yourself, selecting or
repeating a step, or requesting further explanatory detail for each step as
you wish. The dialogue is accompanied by explanatory voiceovers
for the main exchanges, and technical animations for explanatory detail.
The demonstrator can be used in different ways:
-
The first time through the demonstration, we recommend that you run the
introduction then step through the dialogue using the "continue" button
which appears after each step (this will take about 5 minutes).
-
Once you have a sense of the complete dialogue you can play or replay
segments as you wish, and click the "explain" button when you want more
detail. The explanations consist of animations showing ontology search and
PROforma task enactment.
-
If you want to repeat a segment just click on the relevant piece of text.
You may click the "?" button for instructions or the pause button
at any time
|
| Overview of the dialogue system architecture and its main components |
|
The starting point for the development of the demonstration is the implementation of the guideline.
In this case, a fragment from the UK Department of Health Referral guidelines for suspected
cancer [London: DoH, 2000], is used.
A domain plan (figure 1) is developed to specify the high-level tasks to be carried out
(take the patient history then make the decision).
In the current implementation, the domain plan is encoded in the PROforma process specification language, developed by Cancer Research UK.
This provides the basis for deriving the intentional structure of the dialogue.
Figure 1: High level view of the dialogue system's domain plan.
A fragment of the domain ontology used is shown in figure 2.
This ontology allows task-specific knowledge to be augmented with a conceptual model that
describes general medical knowledge (links, relations etc. such as breast cancer is-a cancer, discharge is-symptom-of breast cancer.
Figure 2: A part of the domain ontology of the dialogue system
The PROforma plan and the domain ontology are integrated in the dialogue system.
The system's architecture (figure 3) allows both sources of knowledge to be used in determining the
dialogue state at any point. First, the task and conceptual knowledge contained in the
underlying medical technologies is mapped into an abstract task specification which
defines all the domain knowledge required at any point for the dialogue.
The task and conceptual structures of the abstract task specification are then mapped
into a high-level dialogue specification.
Currently this consists of a set of dialogue games which need to be played in order
to complete tasks.
Figure 3: System architecture
|
| references |
|
Beveridge M, Milward D. Combining task descriptions and ontological knowledge for adaptive dialogue.
In: V. Matoušek and P. Mautner (Eds.) Proceedings of the 6th International Conference on Text, Speech and Dialogue (TSD’03), 8th –11th September, Ceské Budejovice, Czech Republic, Lecture Notes in Artificial Intelligence (LNAI 2807), Springer Verlag, Berlin, pp. 341 – 348.
[Springer]
[Paper - Cancer Research UK]
|
"This paper investigates the use of abstract task specifications for dialogue management
in the medical domain. In most current dialogue systems, possible interactions
with the system are hand-coded in the design. This is an expensive
process, especially for complex dialogues. This paper motivates the use of a
task description language for building flexible and adaptive dialogue systems in
ontologically rich domains such as medicine. It describes the components of a
task specification, and proposes an architecture for dialogue systems which allows
integration of domain reasoning and dialogue. A high-level dialogue
specification is used to support multimodal input and output, including generation
of HTML pages, and generation of fragments of VoiceXML for spoken interaction. " |
|
Beveridge M, Fox J. & Milward D.
Speech Interfaces for Point-of-Care Guideline Systems.
Proc. 9th Conference on Artificial Intelligence in Medicine in Europe (AIME-03), 18–22 October, Cyprus.
Lecture Notes in Artificial Intelligence (LNAI 2780), Springer Verlag, Berlin, pp. 76–80.
[Springer]
[Paper - Cancer Research UK]
|
"A major limiting factor in the acceptability of interactive guideline and decision
support systems is the ease of use of the system in the clinic. A way to reduce
demands upon users and increase flexibility of the interface is to use natural
language dialogues and speech based interfaces. This paper describes a voicebased
data capture and decision support system in which knowledge of underlying
task structure (a medical guideline) and domain knowledge (disease ontologies
and semantic dictionaries) are integrated with dialogue models based on
conversational game theory resulting in a flexible and configurable interface. " |
|
Department of Health. Referral guidelines for suspected
cancer. London: DoH, 2000.
[DoH]
|
"The Government White Paper entitled ‘The new NHS – Modern, Dependable’
guaranteed that everyone with suspected cancer will be able to see a specialist
within two weeks of their GP deciding that they need to be seen urgently and
requesting an appointment.
...
The aim of these guidelines is to facilitate appropriate referral between primary and
secondary care for patients whom a GP suspects may have cancer. The guidelines
should help GPs to identify those patients who are most likely to have cancer and
who therefore require urgent assessment by a specialist. Equally it is hoped that
the guidelines will help GPs to identify patients who are unlikely to have cancer
and who may appropriately be observed in a primary care setting or who may
require non-urgent referral to a hospital...
" |
|
| contact |
links |
John Fox
Advanced Computation Laboratory
Cancer Research UK
PO Box 123
Lincoln's Inn Fields
London WC2A 3PX UK
|
|
| acknowledgements |
| Martin Beveridge & John Fox, Cancer Research UK; David Milward, Linguamatics Ltd, Cambridge
Macromedia Flash presentation designed and implemented by Mirko Parmigiani,
motion graphics designer, monkefilms.com.
Contact: mirko@monke.co.uk
|
Entry on OpenClinical: 01 May 2004
Last main update: 24 May 2004
|
|
|
|
Homey Breast Cancer Referrals Demonstrator: Quicktime movie (17MB)
