Systematic review of studies of patient satisfaction with telemedicine

Search strategy

To identify telemedicine satisfaction studies the following electronic databases were searched: Medline 1966to 1998, Embase 1988-98, Science Citation Index 1981-98, Social Sciences Citation Index 1981-98, Arts and Humanities Citation Index 1981-98, and the TIE (Telemedicine Information Exchange) database. Searches were restricted to English language papers, and the keywords used were: “patient satisfaction,” “consumer satisfaction,” “telecommunications,” and “telemedicine.”The reference lists of papers identified were hand searched for other relevant references.

We included only clinical trials that explored patient satisfaction with teleconsultation, specifically those clinical consultations between healthcare providers from any discipline and patients that involve the use of real time interactive video. We excluded review or discussion papers, studies in which the use of telecommunications technologies was primarily for educational or administrative purposes and not linked to direct patient care, and studies in which the patient was not physically present at either point of care. In addition, if any single study resulted in multiple publications, we reviewed only the principal paper focusing on patient satisfaction. The studies we reviewed had evaluation of patient satisfaction either as the main outcome measure or at least as a prominent feature of their overall assessment of the project. We did not include telemedicine projects that did not directly measure patient satisfaction but reported “unsolicited feedback” that suggests a reasonable degree of satisfaction with telemedicine services.2

Selection criteria

It is acknowledged that well designed and executed trials, particularly randomised controlled trials, provide the most reliable evidence for inclusion in any systematic review.3 However, in view of the limited number of patient satisfaction studies that met the search criteria outlined above, we analysed data from all clinical trials identified irrespective of sample size or methodologies used. Titles and abstracts of the studies identified by the outlined search strategy were read to determine their potential eligibility for the review. The full articles were then assessed for relevance.

Outcome measures and data extraction

The outcome measures we examined included patients' satisfaction (principally overall satisfaction with the telemedicine service but also including levels of satisfaction with communication via this medium, telemedicine consultations compared with traditional face to face consultations, and technical performance) and patients' willingness to use telemedicine in the future.

We recorded the studies' bibliographic details; descriptions of study setting and study population; subject selection criteria; details of form and delivery of the intervention; and outcome measures. We also noted patient numbers, response rates, study methodologies, and other factors affecting the validity of results, including effect modifiers.

Qualitative analysis

In view of the heterogeneous nature of the studies identified, the dearth of randomised controlled trials, and the preponderance of demonstration and feasibility studies, the data available did not permit the use of formal statistical techniques such as meta-analysis. Instead, we conducted a broad qualitative overview of the data, including a critical review of the strength of the findings. We judged the reliability and validity of data by the methodologies used in each study and judged their generalisability from the study context. We did not use a formal scoring method as no well validated instrument for qualitative review yet exists.3 4 However, as the basis for our critical appraisal of the studies, we used a checklist designed for assessing the methodological quality of both randomised and non-randomised studies of healthcare interventions.4

Measures of patient satisfaction

The studies mainly used simple survey instruments to ascertain patient satisfaction. Firm conclusions are limited by methodological difficulties, but it would seem that the patients found teleconsultations acceptable;noted definite advantages, particularly increased accessibility of specialist expertise, less travel required, and reduced waiting times; but also had some disquiet about this mode of healthcare delivery, particularly relating to communication between provider and client via this medium.

Shortcomings of studies

We identified several problems with the studies that affect their reliability and validity. Many studies had small sample sizes, almost a third having 20 or fewer participants, and low response rates, as low as 50%.22 Patient selection criteria were often not clearly specified, or there were no formal selection criteria. Most of the studies (28) used volunteers or physician referrals and provided no information about refusal rates at point of initial referral. Thus, it is not possible to discount selection bias in favour of those likely to be positive about teleconsultation.

Methodologies used for assessing satisfaction were not clearly specified in many studies, making interpretation and comparison of results problematic. Most studies sought to measure whether patients would use the systems again or were “satisfied” with the service. However, few studies defined what satisfaction meant. Therefore, we are unable to discern whether the participants said they were satisfied because telemedicine didn't kill them, or that it was “OK,” or that it was a wonderful experience. The available evidence does not help us to understand the reasons underlying satisfaction or dissatisfaction. In addition, most of the studies presented only initial impressions and failed to explore what happened to patient satisfaction over time, thereby making it possible that the novelty value of the technology resulted in a positive bias.

The cost of teleconsultations compared with routine consultations was not addressed. This is particularly pertinent to the US studies, which account for over 45% of the studies found. The US system of healthcare delivery is a fee for service system, yet the US studies do not mention whether patients attending for teleconsultation paid for the service in the usual way or whether they received this service free of charge. As many US telemedicine projects are primarily grant funded, it is possible that in some studies participants received free teleconsultations, which could affect their satisfaction with the service provided.

Because of the survey nature of most of the studies, there are often inconsistencies in responses that remain unexplained. One possible explanation lies in the survey design. Many surveys have questions with multiple constructs (such as: “I felt the physician was easy to talk to and understood everything I said”). When a single question contains two constructs it is not possible to know which actual construct the participant is responding to, making the data difficult to interpret.

The effects, if any, of telemedicine on communicative behaviours and the interaction between provider and patient during the consultation remained virtually unexplored. There was a lack of data examining patients' perceptions.

Generalisability of results

The generalisability of much of the published research is limited because of effect modifiers such as study setting. One of the largest studies examined teleconsultation in a prison in the United States.5 Clearly, there are several reasons why satisfaction in prisoners may be different from that in the general population. Thus, the peculiarities of the setting mean that this study's results cannot be applied reliably to the general population of that country or more widely.

Furthermore, the delivery of health care was somewhat artificial in many studies. Participants often received a teleconsultation in addition to a routine consultation, and so were really being asked to make a hypothetical judgment as to its value. In many studies participants also received “special” treatment, with every effort being made to minimise inconvenience. Satisfaction in these somewhat artificial contexts may not be readily translatable to satisfaction with telemedicine when it is being used in routine practice.