The Initial Development of the WebMedQual Scale: Domain Assessment of the Construct of Quality of Health Web Sites

Melanie Provost -
Department of Clinical and Administrative Pharmacy, College of Pharmacy, -- USA

Educobjective: To present an overview of the domain representing the construct of quality of health Web sites. To gain an understanding of the complexity and multidimensional aspect of quality. The audience will be also gain an understanding of the psychometric approach of scale development used in this research project.

Abstract Concerned with the variation in quality of the health information on the Internet available to millions of users daily, researchers and associations have developed various sets of criteria to assess the quality of health related web sites. Silberg et al.s criteria are often cited, but researchers have defined the criterion in different manners based on their interpretation leading to various scoring systems. Most guidelines proposed are also limited in their interpretability and ease of use for objective and feasible measurement. There is a need for reliable and valid assessment of quality and standard definitions of the measured constructs.
objectives To develop a comprehensive instrument assessing quality of health related web sites inclusive of several existing guidelines and instruments.

Methods : Phase I: A literature review was performed (Medline 1990-2001) to identify constructs thought to indicate web site quality. Guidelines and rating tools mentioning criteria were included, but awards were excluded as a source of potential items. Item generation and domain coverage of the constructs was done from existing rating tools, guidelines, and articles published in the medical literature or on the web. A conceptual evaluation of the sub-constructs defining the construct of quality was developed in order to define sub-categories when assessing quality of health web sites. During content analysis, duplicate items were eliminated and items that were not clear, meaningful, or measurable were reworded or removed. Some items were generated by the authors for a more complete assessment of some constructs. All items were structured in question format (yes/no). Phase II: Six reviewers in the field of health care (pharmacists, nurse) and management information systems were individually convened to assess each item for its relevance and importance to the construct and to assess item clarity and feasibility. Based on the scores obtained from the expert panel, a cut-off criterion was set at 1 standard deviation below the average score on the evaluation of clarity, relevance and importance in order to remove items. Internal agreement between reviewers was also assessed.

Results: A total of 384 items were generated from 26 sources. The first content analysis reduced the scale to 101 items. High concordance in expert agreement on the relevance, feasibility and clarity of each item was observed: 3 out of 4, or all raters agreed on 76% to 85% of items for relevance, feasibility and clarity rating. Averages of all item scores were 1.80, 1.78, and 1.81 for relevance, importance and feasibility, respectively (maximum score 2.0). Based on the cut-off criterion and on expert comments, 9 items were removed, 3 items added, and 10 items revised. The domain coverage represented in this first version of the instrument, named WebMedQual consists of 8 categories, 8 sub-categories, 95 items and 3 supplement items to assess web site quality. The constructs are: Content (19 items), Authority of Source (18 items), Design (19 items), Accessibility and Availability (6 items), Links (4 items), User Support (9 items), Confidentiality and Privacy (17 items), e-commerce (6 items).

Conclusions Based on existing guidelines and rating instruments, the "WebMedQual" represents a first step toward comprehensive and standard quality assessment of health websites. This scale will allow relatively easy assessment of quality with numeric scoring which may help researchers in further studies looking at the relationship of the quality of a site with health behaviors, impact on health (positive or negative). The next phase will be to assess the scales reliability and validity.