OpenClinical logo

DSS benefits, drawbacks, success factors

Potential benefits and drawbacks of the use of CDSSs; Factors which may help determine the successful use of CDSSs in clinical practice

contents
 bullet  Preamble  bullet  Potential Benefits of CDSSs  bullet  Potential drawbacks related to the use of of CDSSs  bullet  Factors which may help determine the acceptance of DSS in clinical practice  bullet  References

 bullet  DSS: Introduction  bullet  DSS: Benefits, Drawbacks, Success factors  bullet  DSS: Evaluation  bullet  DSS: Evaluation studies  bullet  DSS: Evaluation reviews

Preamble
The first computerised clinical decision support systems (CDSSs) used in clinical practice were developed in the 1970s, but even though there have been a number of individual successes since then, their impact on routine clinical practice has not been as strong as expected. There has certainly been no large-scale roll out of CDSSs, and barriers to their implementation appear largely to remain in place. Arguments supporting and questioning the value of CDSSs have been well-rehearsed over the years. On this page we list a number of frequently stated potential benefits and drawbacks of CDSSs before considering factors which may help determine the implementation and use of CDSSs in clinical practice
Potential Benefits of CDSSs
The potential benefits of using electronic decision support systems in clinical practice fall into three broad categories (Coiera quoting Sintchenko et al., 2002):
  1. Improved patient safety e.g. through reduced medication errors and adverse events and improved medication and test ordering;
  2. Improved quality of care e.g. by increasing clinicians’ available time for direct patient care, increased application of clinical pathways and guidelines, facilitating the use of up-to-date clinical evidence, improved clinical documentation and patient satisfaction;
  3. Improved efficiency in health care delivery e.g. by reducing costs through faster order processing, reductions in test duplication, decreased adverse events, and changed patterns of drug prescribing favouring cheaper but equally effective generic brands.

Informal list of potential benefits

  • Automatic provision of relevant, personalised expert advice, expertise and recommendations sourced from up-to-date, best practice knowledge
  • Reduce variation in the quality of care
  • Can support medical education and training
  • Can help overcome problems of inefficient coding of data
  • Can be cost-effective after initial capital costs and update and maintenance costs
  • Can provide immediate feedback to patients
  • If integrated with an EMR, can help streamline workflow (history taking, diagnosis, treatment) and encourage more efficient data gathering
  • Can provide an audit trail and support research
  • Can maintain and improve consistency of care
  • Can supply clinical information anytime, anywhere it's needed.
Potential drawbacks related to the use of of CDSSs
  • Potential 'deskilling' effect
  • Can be perceived as a threat to clinical judgment
  • Can be considered too inflexible (can appear prescriptive, can appear to direct proceedings; can be difficult to depart from ordered, pre-prepared paths)
  • Promote over-reliance on software; limit clinicians' freedom to think ?
  • Difficult to evaluate - lack of accepted evaluation standards
  • Can be time-consuming to use, possibly lead to longer clinical encounters and create extra work
  • Uncertain and untested ethical and legal status
  • Costs: maintenance, support and training required after initial outlay
  • A clinician's experience and imagination cannot be duplicated in a computer application.
Factors which may help determine the acceptance and use of CDSSs in clinical practice

  • Cost
  • Attitude of targeted users: breadth and depth of commitment
  • Degree of user acceptance prior to and after installation
  • Ease of use - time needed to learn to use and to use
  • Type, timing, length of training to be provided
  • Availablility of support and maintenance
  • Interoperability: ease/extent of integration with legacy systems (hardware, other devices) and existing software programs (integration with patient record and/or any relevant clinical terminologies would avoid need to re-enter patient data)
  • Ease of integration within organisational context and routine workflow - degee to which it entails aredesign of clinical processes
  • Legal and ethical issues
  • User interface: design, structure, number of forms
  • Style, manner of presentation of advice/ recommendations/ results to user
  • Patients' attitudes to use
  • Provision of evidence justifying advice and/or recommendations
  • Involvement of local users during development phase
  • The quality and reliability of a system and its knowledge base which should be populated with trusted, up-to-date and maintainable knowledge

In the last resort, widespread use of clinical decision support systems in clinical practice will not occur without electronic patient record systems using terminology and data standards that will allow them to be accessed effortlessly during routine patient care.

references: CDSS success factors

Mollon B, Chong J Jr, Holbrook AM, Sung M, Thabane L, Foster G. Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials. BMC Med Inform Decis Mak. 2009 Feb 11;9:11.

[PubMed]   []

" BACKGROUND: Computerized decision support systems (CDSS) are believed to have the potential to improve the quality of health care delivery, although results from high quality studies have been mixed. We conducted a systematic review to evaluate whether certain features of prescribing decision support systems (RxCDSS) predict successful implementation, change in provider behaviour, and change in patient outcomes. METHODS: A literature search of Medline, EMBASE, CINAHL and INSPEC databases (earliest entry to June 2008) was conducted to identify randomized controlled trials involving RxCDSS. Each citation was independently assessed by two reviewers for outcomes and 28 predefined system features. Statistical analysis of associations between system features and success of outcomes was planned. RESULTS: Of 4534 citations returned by the search, 41 met the inclusion criteria. Of these, 37 reported successful system implementations, 25 reported success at changing health care provider behaviour, and 5 noted improvements in patient outcomes. A mean of 17 features per study were mentioned. The statistical analysis could not be completed due primarily to the small number of studies and lack of diversity of outcomes. Descriptive analysis did not confirm any feature to be more prevalent in successful trials relative to unsuccessful ones for implementation, provider behaviour or patient outcomes. CONCLUSION: While RxCDSSs have the potential to change health care provider behaviour, very few high quality studies show improvement in patient outcomes. Furthermore, the features of the RxCDSS associated with success (or failure) are poorly described, thus making it difficult for system design and implementation to improve. "

Holbrook A, Xu S, Banting J. What factors determine the success of clinical decision support systems? AMIA Annu Symp Proc. 2003;:862.

[PubMed]   []

" Computerized decision support systems (CDSS) which improve the quality of patient care are strong and necessary incentives for clinicians to use electronic medical records. We have noted previously that the logical path of CDSS design, which would be to determine the factors that predict success before the system is designed, appears rarely to have been followed. In this overview update of the literature on predictors of successful CDSS, we conclude that the predictors have not been adequately identified and the success of CDSS may improve when they are. "

E. Coiera. The Guide to Health Informatics (2nd Edition). Arnold, London, October 2003.

Free sample chapters include: [Chapter 25 - Clinical decision support systems]

Coiera, 2nd Edition, 2003 Chapter summary: " 1. Artificial intelligence (AI) systems are intended to support healthcare workers with tasks that rely on the manipulation of data and knowledge. 2. Expert systems are the commonest type of CDSS in routine clinical use. They contain medical knowledge about a very specifically defined task. Their uses include: · alerts and reminders, · diagnostic assistance, · therapy critiquing and planning, · prescribing decision support · information retrieval, · image recognition and interpretation. 3. Reasons for the failure of many expert systems to be used clinically include dependence on an electronic medical record system to supply their data, poor human interface design, failure to fit naturally into the routine process of care, and reluctance or computer illiteracy of some healthcare workers. 4. Many expert systems are now in routine use in acute care settings, clinical laboratories, educational institutions, and incorporated into electronic medical record systems. 5. Some CDSS systems have the capacity to learn, leading to the discovery of new phenomena and the creation of medical knowledge. These machine learning systems can be used to: · develop the knowledge bases used by expert systems, · assist in the design of new drugs, · advance research in the development of pathophysiological models from experimental data. 6. Benefits from CDSS include improved patient safety, improved quality of care, and improved efficiency in health care delivery. "

Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005 Apr 2;330(7494):765. Epub 2005 Mar 14. Review.

[PubMed]   [PubMed Central]   [BMJ]

" OBJECTIVE: To identify features of clinical decision support systems critical for improving clinical practice. DESIGN: Systematic review of randomised controlled trials. DATA SOURCES: Literature searches via Medline, CINAHL, and the Cochrane Controlled Trials Register up to 2003; and searches of reference lists of included studies and relevant reviews. STUDY SELECTION: Studies had to evaluate the ability of decision support systems to improve clinical practice. DATA EXTRACTION: Studies were assessed for statistically and clinically significant improvement in clinical practice and for the presence of 15 decision support system features whose importance had been repeatedly suggested in the literature. RESULTS: Seventy studies were included. Decision support systems significantly improved clinical practice in 68% of trials. Univariate analyses revealed that, for five of the system features, interventions possessing the feature were significantly more likely to improve clinical practice than interventions lacking the feature. Multiple logistic regression analysis identified four features as independent predictors of improved clinical practice: automatic provision of decision support as part of clinician workflow (P < 0.00001), provision of recommendations rather than just assessments (P = 0.0187), provision of decision support at the time and location of decision making (P = 0.0263), and computer based decision support (P = 0.0294). Of 32 systems possessing all four features, 30 (94%) significantly improved clinical practice. Furthermore, direct experimental justification was found for providing periodic performance feedback, sharing recommendations with patients, and requesting documentation of reasons for not following recommendations. CONCLUSIONS: Several features were closely correlated with decision support systems' ability to improve patient care significantly. Clinicians and other stakeholders should implement clinical decision support systems that incorporate these features whenever feasible and appropriate."

Zheng K, Padman R, Johnson MP, Diamond HS. Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. Int J Med Inform. 2005 Aug;74(7-8):535-43.

[PubMed]   []

" BACKGROUND: Evaluation studies of clinical decision support systems (CDSS) have tended to focus on assessments of system quality and clinical performance in a laboratory setting. Relatively few studies have used field trials to determine if CDSS are likely to be used in routine clinical settings and whether reminders generated are likely to be acted upon by end-users. Moreover, such studies when performed tend not to identify distinct user groups, nor to classify user feedback. AIM: To assess medical residents' acceptance and adoption of a clinical reminder system for chronic disease and preventive care management and to use expressed preferences for system attributes and functionality as a basis for system re-engineering. DESIGN OF STUDY: Longitudinal, correlational study using a novel developmental trajectory analysis (DTA) statistical method, followed by a qualitative analysis based on user satisfaction surveys and field interviews. SETTING: An ambulatory primary care clinic of an urban teaching hospital offering comprehensive healthcare services. 41 medical residents used a CDSS over 10 months in their daily practice. Use of this system was strongly recommended but not mandatory. METHODS: A group-based, semi-parametric statistical modeling method to identify distinct groups, with distinct usage trajectories, followed by qualitative instruments of usability and satisfaction surveys and structured interviews to validate insights derived from usage trajectories. RESULTS: Quantitative analysis delineates three types of user adoption behavior: "light", "moderate" and "heavy" usage. Qualitative analysis reveals that clinicians of distinct types tend to exhibit views of the system consistent with their demonstrated adoption behavior. Drawbacks in the design of the CDSS identified by users of all types (in different ways) motivate a redesign based on current physician workflows. CONCLUSION: We conclude that this mixed methodology has considerable promise to provide new insights into system usability and adoption issues that may benefit clinical decision support systems as well as information systems more generally. "

references: Health information system success factors

Brender J, Ammenwerth E, Nykanen P, Talmon J. Factors influencing success and failure of health informatics systems-a pilot Delphi study. Methods Inf Med. 2006;45(1):125-36.

[PubMed]   []

" OBJECTIVES: The aim is to gain information on factors influencing success and failure for Health Informatics applications from a group of medical informaticians. METHODS: Based on the presentations at a special topic conference on success and failure in Health ICT and analysis of the proceedings, we conducted a Delphi study on success and failure aspects. RESULTS: A total of 110 success factors and 27 failure criteria were identified, distributed on categories like functional, organizational, behavioral, technical, managerial, political, cultural, legal, strategy, economy, education and user acceptance. These factors and criteria were rated for six different system types. Unanimously it was agreed that "collaboration and co-operation" and "setting goals and courses" are "essential for the success" of clinical systems, and "user acceptance" for educational systems. Similarly, the score "essential in order to avoid a failure" were given unanimously on clinical systems for "response rate and other performance measures" and on administrative systems for "not understanding the organizational context" with "not understanding or foreseeing the extent to which the new IT-system affects the organization, its structure and/or work procedures" as the highest scoring sub-item. CONCLUSIONS: All success factors and failure criteria were considered relevant by the Delphi expert panel. There is no small set of relevant factors or indicators, but success or failure of a Health ICT depends on a large set of issues. Further, clinical systems and decision support systems depend on more factors than other systems. "

Varonen H, Kortteisto T, Kaila M; EBMeDS Study Group. What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians. Fam Pract. 2008 Jun;25(3):162-7. Epub 2008 May 25.

[PubMed]   []

" OBJECTIVES: To identify potential barriers and facilitators to implementing computerized decision support systems (CDSSs) in health care as perceived by clinicians. METHODS: We carried out a qualitative focus group study with primary and secondary health care settings in six areas of Finland. A total of 39 interviewed physicians, of whom 22 practised in primary care and 17 in secondary care. The main outcome measures physicians' expectations, preconceived barriers and facilitators were explicitly identified by the participants during the interviews. RESULTS: Identified barriers were earlier experience of dysfunctional computer systems in health care, potential harm to doctor-patient relationship, obscured responsibilities, threats to clinician's autonomy and potential extra workload due to excessive reminders. Identified facilitators were self-control of frequency and contents of CDSS and noticeable help of CDSS in clinical practice. It was easy for the physicians to think of applications and clinical topics for CDSS that could help them to avoid mistakes and improve work processes. CONCLUSIONS: Physicians had relatively positive attitudes towards the idea of CDSS. They expected flexibility, individuality and reliability of the CDSS. The rather high level of computerized practices and wide use of electronic guidelines probably have paved the way for the CDSS in Finland. "

acknowledgements
 
page history
Entry on OpenClinical: 30 May 2005
Last main updates: 27 June 2005; 11 September 2005




Search this site
 

Privacy policy User agreement Copyright Feedback

Last modified:
© Copyright OpenClinical 2002-2011