ISABEL: The Use of a Web-Based clinical support system

J.S. Briggs, Tineke Fitch
University of Portsmout, Portsmouth, UK

Introduction: ISABEL [1] is a web-based paediatric clinical decision support system for use by healthcare professionals. The website is owned and has been developed by the ISABEL Medical Charity, a UK-registered charity. According to the website, the project started as a result of a little girl called Isabel falling dangerously ill with complications resulting from chicken pox. She had to spend 4 weeks in intensive care to save her life because her serious condition was not recognised in time. As a result of this bad experience, Isabel's parents led an effort to develop a resource that could be used by healthcare professionals to assist in the diagnosis of children.
Using proprietary pattern recognition software (Autonomy[2]) to search standard paediatric textbooks, a differential diagnostic tool produces a list of up to 15 diagnoses to consider for any given set of clinical features. Further decision support is provided by text, annotated images, and practice guidelines specific to each diagnosis. A section entitled "experience" attempts to capture and highlight common clinical lessons learnt at various steps within the guidelines and at relevant points in the diagnostic process [3].
The UK Department of Health (DoH) is examining the potential effectiveness of the ISABEL website in the wider clinical context and exploring options for promoting its wider use in the UK National Health Service (NHS). This paper reports on work commissioned by the DoH to review the existing use of ISABEL.
Objectives and study design:
The objectives of the study were to determine the extent of ISABEL's use, to find out user's attitudes to ISABEL and finally to identify impediments to its development. The study was done in two parts.
1. Web server log data was analysed to determine how many people actually use ISABEL and how often they do so. We examined two log files provided to us by the company that hosts the ISABEL website. The first was the conventional web server log. Analysis of this was performed using a number of web log analysis tools, including Sawmill [4]. The second was a user log recording validated logins to the site, plus new user registrations. This was analysed using custom analysis tools developed by the authors. Both covered the period from when ISABEL was first set up (18th July 2001) to when our survey ended (19th December 2002).
2. A 24-item questionnaire was sent by email to 4436 of the 7179 registered users of the system. This was designed to determine user attitudes to the system. Since the review concerned ISABEL's use with respect to its impact in the UK, the questionnaire was sent only to those whose registration details showed them to be based in the UK. A letter accompanied it from the ISABEL team encouraging recipients to complete the form. Returns could be sent by email, post or fax.
Results of the web survey: The web survey showed that of the total number of users (7179), 3330 (46%) had only used ISABEL once, 5186 (72%) used it no more than twice, and 6493 (90%) used it no more than 5 times. However there was a core of approximately 50 users who had used the system more than 25 times, with an average frequency of one week between visits.
Usage was highest during the weekday working day, with reduced usage out of normal hours and at weekends.
Most usage (where identifiable) was from the UK, with at least a quarter of accesses coming from somewhere within the NHS, but there was also a significant amount of access via Internet service providers. A detailed analysis of precisely where in the NHS users were located proved to be impossible because of anonymisation implemented by NHS firewalls.
Results of the user questionnaire
523 responses were received (11.8%) of which 518 (11.7%) were completed sufficiently for analysis. Of particular interest to the study for the purposes of analysis was the distinction between responses from those who were paediatric specialists (58% of respondents) and those who were not (42%).
As expected, the paediatric specialists used ISABEL slightly more frequently than non-specialists. Three-quarters say they would use it more but for time constraints and lack of access to information technology. For those who would not use ISABEL more often, the main reason is that they use ISABEL as much as they require. However, one-third of paediatric consultants and one-fifth of General Practitioners say ISABEL is unsatisfactory in its present form and improvements are needed.
The aspects of ISABEL that are particularly liked are the differential diagnostic tool and the guidelines, and approximately two-thirds are of the opinion that ISABEL assists in clinical management.
Discussion: Findings suggest that there is a small and very supportive community of devotees, including those who are involved in the development of the site, but that the website is either not sufficiently useful or not sufficiently convenient for the vast majority of users who have tried it out. The reasons for the latter are largely systemic to the NHS and the way it delivers its services, and would require considerable resources to make more Internet-connected computers available, and to give staff more time and opportunity to use them.
The website itself exhibits some good properties: there is little evidence of broken links or changing structure. Most output is dynamically generated using Java Server Pages but one criticism of the site is its use of frames, which are deprecated by most web page experts. Their adoption on this site has made some analysis of ISABEL's usage (such as how users proceed through the site) impossible.

References:
[1] ISABEL, website available at http://www.isabel.org.uk (Accessed July 2003)
[2] Autonomy, website available at http://autonomy.com (Accessed July 2003)
[3] Ramnarayan P, Britto J. Paediatric clinical decision support systems. Archives of Disease in Childhood, November 2002.
[4] Sawmill website available at http://www.flowerfire.com/sawmill/ (Accessed July 2003)