Aortic valve surgery: how reliable are health information websites?

Background Aortic valve replacement is one of the most common cardiac operations currently performed. Patients increasingly use the internet for information about their diagnosis and it would therefore be important to know how reliable this is. Aim To determine the reliability of internet information on aortic valve replacement surgery. Design & setting This was a web-based project scoring sites that might be accessed by a patient. Method The first 50 websites found on each of the four most popular search engines in the UK were viewed, as well as the first 50 videos found on the most popular video-host website. Eligible websites were assessed according to seven positive criteria and three negative criteria, giving a possible range of scores from −6 to 14. Results There were 79 sites and the median score was 5 (range −1 to 14). There were statistically significant differences between organisation/educational sites with score 7 (2 to 14), hospital sites with score 2 (−1 to 10), commercial sites with score 2.5 (0 to 9) and videos with score 5 (2 to 11). The highest scores went to three NHS sites (score 13 or 14), .gov sites (median score 8.5) and Health On the Net Foundation (HON) accredited sites (median score 7). Conclusion Information on the internet about aortic valve replacement is variable but NHS sites provide the most reliable information.


Introduction
Aortic valve replacement surgery is increasingly common as our population ages.In 2008, there were 7280 aortic valve replacement operations in the UK and Ireland compared with 22 808 isolated coronary bypass procedures. 1 Not all patients wish to be involved in discussion of the timing of surgery, or the choice of valve type. 2 However, if they do, an informed discussion may improve quality of life after surgery. 2,3An estimated 68% of British patients will search for medical information online 4 may then request a discussion with their GP or practice nurse.The quality and reliability of general information available on the internet is known to vary greatly [4][5][6][7] but there are few critical The sites were also categorised according to top level domains (TLD): .com,.co.uk, .org,.edu, .gov and variations on these, on the assumption that patients c use these as a guide to quality.
There were no previously published scoring systems available so the researchers designed one (Appendix 1) based on: . the types of information known to be important and likely to be of interest to a patient; . a previously published test of reliability; 9 and .adaptations of HON 10 guidelines, relevant to the topic studied.
Each website was assessed according to 10 criteria and given scores from 0 to 2 for each positive attribute and 0 to À2 for each negative attribute assessing accuracy, reliability, and appropriateness of information (Appendix 1).The minimum achievable score was À6 and the highest was 14; the higher the score the better the site.Bias was defined as recommending a procedure or type of prosthetic valve outside established guidelines. 11,12There were three patient forums associated with these websites.These frequently contained inaccurate information or unhelpful subjective comments, for example 'a metal valve can start to fail after 25 years' and 'I didn't want a pig to die so I could have its valve, only to have it taken out some years on; it seemed wasteful and wrong.'These were not assessed within this study as these forums do not vet their responders, and the study was confined to sites that a patient might assume are authoritative and comprehensive.
Websites were checked for their accreditation by HON by keeping the 'HON toolbar' open. 12HON is a non-profit, non-governmental organisation, accredited by the Economic and Social Council of the United Nations created in 1995 which promotes and guides the deployment of useful and reliable online health information, and its appropriate and efficient use.Although HON does not assess medical accuracy, validity, or appropriateness, it is a good guide to the quality and accountability of websites according to eight standards: information must be authoritative; the purpose of the website must be known; confidentiality must be respected; information must be documented (that is, referenced and dated); claims must be justified (for example, with scientific evidence); contact details must be made available; funding sources must be disclosed; and, there must be an advertising and editorial policy available for viewers to see.

Statistical analysis
The scores were not normally distributed using the Kolmogorov-Smirnov normality test (P<0.05)so medians and ranges were reported.Comparisons between scores in the categories were made using the Mann-Whitney U-test and Kruskal-Wallis one-way analysis of variance. 13

Results
Of the 79 sites assessed (Appendix 2), 33 (43%) were organisational or educational, 24 (30%) hospital-based, 8 (10%) commercial, and 14 (17%) videos.The median score was 5 (range À1 to 14) as seen in Figure 1.Different search engines produced different results with ask.com providing 67% of the hospital websites and 78% of the sites whose main focus was transcatheter procedures.
A sample of websites and their scores (five best, five worst and five in between) is given in Table 1.
There were seven (9%) HON accredited sites and these had higher scores (median 7) than those without HON accreditation (median 4; P<0.05).There were also statistically significant differences in the scores between organisational/education and hospital websites (P<0.05),organisational/  educational and commercial websites (P<0.05), and organisational/educational and video websites (P<0.05) are provided in Table 2.No website contained false information, but a number favoured a procedure or types of valve beyond established guidelines or had gaps in information typically about the postoperative recovery process.Categorisation of sites by TLD-types showed there were 52 .com/co.uk/.uksites and these scored 5 (0 to 14), 18 .orgsites which scored 4 (À1 to 11), four .govsites which scored 8.5 (6 to 11) and five .edusites which scored 4 (2 to 7).Statistical tests of these scores were underpowered due to small numbers.
reliability of a site.HON accreditation requires high governance standards and HON-accredited sites had higher median scores than the others.Only 7(9%) of sites were HON-accredited and many of the best sites were not HON-accredited.HON criteria focus on accessibility, accountability, author qualifications, and overall good governance and do not assess the reliability of specialist information provided.The authors suggest that HON should collaborate with the specialist professional societies relevant to the website being evaluated to add a further criterion concerned with clinical reliability.
A patient may expect hospital sites to be reliable but these tended to be biased towards the specialist procedures offered by the hospital rather than offering more comprehensive information.
Another problem is finding a site.The best and most reliable did not appear on the top page, for example the best Yahoo site was ranked 63 rd .During the preparation of this report, a new site was launched by BUPA and the Society for Cardiothoracic Surgery in Great Britain and Ireland (SCTS) (http://scts.org/patients/having-heart-surgery/).It scored 12 so was among the best sites but still does not appear on search engines.
Most sites contained limitations, most frequently about possible complications of surgery and the recovery process.The reasons for choosing one type of prosthesis over another were often not discussed including the possible limitations of warfarin and concerns around pregnancy.None of the sites discussed the psychological effects of surgery and there was little reference to support groups.Clinicians need to be aware of these gaps and provide additional complementary information at the same time as directing patients to the highest scoring sites which were the BUPA/SCTS site and the three NHS sites.The NHS sites contained similar information but on different domains: nhsdirect.wales.nhs.uk;nhs.uk; and nhsinform.co.uk.It would seem economic to combine these or at least develop cross-links to reduce maintenance costs.