Abstract
Background Aortic valve replacement is one of the most common cardiac operations currently performed. Patients increasingly use the internet for information about their diagnosis and it would therefore be important to know how reliable this is.
Aim To determine the reliability of internet information on aortic valve replacement surgery.
Design & setting This was a web-based project scoring sites that might be accessed by a patient.
Method The first 50 websites found on each of the four most popular search engines in the UK were viewed, as well as the first 50 videos found on the most popular video-host website. Eligible websites were assessed according to seven positive criteria and three negative criteria, giving a possible range of scores from −6 to 14.
Results There were 79 sites and the median score was 5 (range −1 to 14). There were statistically significant differences between organisation/educational sites with score 7 (2 to 14), hospital sites with score 2 (−1 to 10), commercial sites with score 2.5 (0 to 9) and videos with score 5 (2 to 11). The highest scores went to three NHS sites (score 13 or 14), .gov sites (median score 8.5) and Health On the Net Foundation (HON) accredited sites (median score 7).
Conclusion Information on the internet about aortic valve replacement is variable but NHS sites provide the most reliable information.
How this fits in
Aortic valve replacement is an increasingly common procedure and results are improved if patients are involved in decision making if they wish to be. Patient empowerment requires information for which the internet is a major source. However, the reliability of this information is variable and clinicians need to direct patients to the best sites since these may not appear on a brief internet search. Supplementary information will also need to be given since most websites are limited in scope.
Introduction
Aortic valve replacement surgery is increasingly common as our population ages. In 2008, there were 7280 aortic valve replacement operations in the UK and Ireland compared with 22 808 isolated coronary bypass procedures.1 Not all patients wish to be involved in discussion of the timing of surgery, or the choice of valve type.2 However, if they do, an informed discussion may improve quality of life after surgery.2,3 An estimated 68% of British patients will search for medical information online4 may then request a discussion with their GP or practice nurse. The quality and reliability of general information available on the internet is known to vary greatly4–7 but there are few critical appraisals of medical sites. The research team assessed the information that might be accessed on the internet by a patient needing an aortic valve replacement.
Method
The authors used the four most popular search engines in the UK (Google, Yahoo, Bing, and Ask), which cover >98% of search engine usage in the UK.8 In addition, a video search on YouTube was performed. The phrase 'aortic valve replacement' was searched. To prevent bias in the search algorithm, an incognito window was used to prevent cookies and search history from influencing search results.
The first 50 links from each search engine and YouTube were followed. Of the 250 results, there was significant overlap (especially between Google, Yahoo, and Bing searches), leaving 165 unique websites. Of these, 86 sites were irrelevant, such as advertisements, forums, other recommended searches, and sites detailing other procedures (for example, aortic root replacement). There were 79 relevant websites remaining. These were categorised as:
organisational/educational (for example, published by a large charitable organisation, university or other information provider);
hospital-based;
commercial (for example, published by a prosthetic valve manufacturer); and
video.
The sites were also categorised according to top level domains (TLD): .com, .co.uk, .org, .edu, .gov and variations on these, on the assumption that patients c use these as a guide to quality.
There were no previously published scoring systems available so the researchers designed one (Appendix 1) based on:
the types of information known to be important and likely to be of interest to a patient;
a previously published test of reliability;9 and
adaptations of HON10 guidelines, relevant to the topic studied.
Each website was assessed according to 10 criteria and given scores from 0 to 2 for each positive attribute and 0 to −2 for each negative attribute assessing accuracy, reliability, and appropriateness of information (Appendix 1). The minimum achievable score was −6 and the highest was 14; the higher the score the better the site. Bias was defined as recommending a procedure or type of prosthetic valve outside established guidelines.11,12 There were three patient forums associated with these websites. These frequently contained inaccurate information or unhelpful subjective comments, for example 'a metal valve can start to fail after 25 years' and 'I didn’t want a pig to die so I could have its valve, only to have it taken out some years on; it seemed wasteful and wrong.' These were not assessed within this study as these forums do not vet their responders, and the study was confined to sites that a patient might assume are authoritative and comprehensive.
Websites were checked for their accreditation by HON by keeping the 'HON toolbar' open.12 HON is a non-profit, non-governmental organisation, accredited by the Economic and Social Council of the United Nations created in 1995 which promotes and guides the deployment of useful and reliable online health information, and its appropriate and efficient use. Although HON does not assess medical accuracy, validity, or appropriateness, it is a good guide to the quality and accountability of websites according to eight standards: information must be authoritative; the purpose of the website must be known; confidentiality must be respected; information must be documented (that is, referenced and dated); claims must be justified (for example, with scientific evidence); contact details must be made available; funding sources must be disclosed; and, there must be an advertising and editorial policy available for viewers to see.
Statistical analysis
The scores were not normally distributed using the Kolmogorov-Smirnov normality test (P<0.05) so medians and ranges were reported. Comparisons between scores in the categories were made using the Mann-Whitney U-test and Kruskal-Wallis one-way analysis of variance.13
Results
Of the 79 sites assessed (Appendix 2), 33 (43%) were organisational or educational, 24 (30%) hospital-based, 8 (10%) commercial, and 14 (17%) videos. The median score was 5 (range −1 to 14) as seen in Figure 1. Different search engines produced different results with ask.com providing 67% of the hospital websites and 78% of the sites whose main focus was transcatheter procedures.
A sample of websites and their scores (five best, five worst and five in between) is given in Table 1.
There were seven (9%) HON accredited sites and these had higher scores (median 7) than those without HON accreditation (median 4; P<0.05). There were also statistically significant differences in the scores between organisational/education and hospital websites (P<0.05), organisational/educational and commercial websites (P<0.05), and organisational/educational and video websites (P<0.05) are provided in Table 2. No website contained false information, but a number favoured a procedure or types of valve beyond established guidelines or had gaps in information typically about the postoperative recovery process. Categorisation of sites by TLD-types showed there were 52 .com/co.uk/.uk sites and these scored 5 (0 to 14), 18 .org sites which scored 4 (−1 to 11), four .gov sites which scored 8.5 (6 to 11) and five .edu sites which scored 4 (2 to 7). Statistical tests of these scores were underpowered due to small numbers.
Discussion
Summary
Web-based information on aortic valve replacement is highly variable with scores between −1 and 14. The educational sites had the highest median score (7) but scores ranged from 2 to 14. Hospital sites had scores (median 2) similar to the commercial sites (median 2.5).
Strengths and limitations
The ideal study may have been to ask a number of patients, say 100, to search freely and for the research team to then score the sites they read. The methodology used was far simpler, and cannot have mimicked a potentially idiosyncratic individual search. The authors expect that they have accessed all sites that a patient would likely find by using the four most popular search engines and looking at the first 50 results from each.
Comparison with the literature
The quality of websites is known to be variable4–7 although there is surprisingly little work on medical sites despite their obvious potential relevance to patients. A web-based study9 has shown that information about ‘murmur’ is often inaccurate tending to stress the worst possible outcome and rarely informing that a benign flow murmur is the most common cause.
Implications for practice
Patient empowerment requires information for which the internet is a major resource. However, there are a number of problems faced by patients seeking information. The first is determining the reliability of a site. HON accreditation requires high governance standards and HON-accredited sites had higher median scores than the others. Only 7(9%) of sites were HON-accredited and many of the best sites were not HON-accredited. HON criteria focus on accessibility, accountability, author qualifications, and overall good governance and do not assess the reliability of specialist information provided. The authors suggest that HON should collaborate with the specialist professional societies relevant to the website being evaluated to add a further criterion concerned with clinical reliability. A patient may expect hospital sites to be reliable but these tended to be biased towards the specialist procedures offered by the hospital rather than offering more comprehensive information.
Another problem is finding a site. The best and most reliable did not appear on the top page, for example the best Yahoo site was ranked 63rd. During the preparation of this report, a new site was launched by BUPA and the Society for Cardiothoracic Surgery in Great Britain and Ireland (SCTS) (http://scts.org/patients/having-heart-surgery/). It scored 12 so was among the best sites but still does not appear on search engines.
Most sites contained limitations, most frequently about possible complications of surgery and the recovery process. The reasons for choosing one type of prosthesis over another were often not discussed including the possible limitations of warfarin and concerns around pregnancy. None of the sites discussed the psychological effects of surgery and there was little reference to support groups. Clinicians need to be aware of these gaps and provide additional complementary information at the same time as directing patients to the highest scoring sites which were the BUPA/SCTS site and the three NHS sites. The NHS sites contained similar information but on different domains: nhsdirect.wales.nhs.uk; nhs.uk; and nhsinform.co.uk. It would seem economic to combine these or at least develop cross-links to reduce maintenance costs.
Appendix
| Positive |
|
| Negative |
Scoring system designed based on: |
Notes
Funding
This study was not funded.
Ethical approval
No ethics approval was necessary.
Provenance
Freely submitted; externally peer reviewed.
Competing interests
The authors declare that no competing interests exist.
- Received August 21, 2016.
- Accepted September 19, 2016.
- Copyright © The Authors 2017
This article is Open Access: CC BY license (https://creativecommons.org/licenses/by/4.0/)
In this issue
Jump to section
