Hum. Reprod. Advance Access originally published online on March 27, 2008
Human Reproduction 2008 23(7):1520-1525; doi:10.1093/humrep/den072
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Infertility information on the World Wide Web: a cross-sectional survey of quality of infertility information on the internet in the UK
Assisted Conception Unit, Guy's and St. Thomas' Hospital NHS Foundation Trust, Thomas Guy House, Guys Hospital, 4th Floor, London SE1 9RT, UK
1 Correspondence address. Tel: +44-207-188-0496; Fax: +44-207-188-0490; E-mail: arricoomar{at}blueyonder.co.uk, arri.coomarasamy{at}gstt.nhs.uk
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
BACKGROUND: The internet is a frequently used source of information for infertile couples. Previous studies suggested that the quality of health information on the internet is poor. The aim of this study was to assess the quality of websites providing information on infertility and its management in the UK. Differences between website types and affiliations were assessed.
METHODS: A Google search for the keyword ‘infertility’ was performed and 107 relevant websites were identified and categorized by type. Websites were assessed for credibility, accuracy and ease of navigation using predefined criteria.
RESULTS: The total scores for all types of websites were low, particularly in the accuracy category. Websites affiliated to the UK National Health Service (NHS) scored higher than those affiliated to private fertility clinics and other clinics providing non-conventional fertility treatment. Specifically, NHS websites were more likely to report success rates (92.9% versus 60% and 0%, P 
0.05) and display information about their sources of funding (85.7% versus 15% and 14.8%, P 0.0001).
CONCLUSIONS: Internet resources available to infertile patients are variable. Differences in the quality of infertility information exist between the different types of websites.
Key words: infertility/internet information quality/NHS/ART clinics/private clinics
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The internet is becoming an increasingly popular tool for patients seeking medical information. According to Internet World Statistics (2007)
, approximately two-thirds of the population in the UK are current internet users. Of those people who browse the World Wide Web, 60–80% have used it to obtain health information (The Harris Poll Group, 2007), and their treatment choices could be influenced by what they read.
The infertile couple is no different from other ‘e-health users’. A recent survey of 200 infertile couples found that nearly half were using the internet for fertility-related information and two-thirds of female users were influenced by online information when seeking therapy (Huang et al., 2003).
In order to find fertility-related sites, 91% of infertile patients use internet search engines (Huang et al., 2003). Different search engines, such as Google or AltaVista, use different criteria when supplying individuals with a list of relevant sites. Any internet user who accesses information via search engines will almost certainly visit the first few sites listed on the search page before any of the later listed sites. However, the order in which sites appear does not necessarily correlate with the quality of the site (Theodosiou and Green, 2003).
A number of tools have been developed for assessment of website quality (Silberg et al., 1997; Kim et al., 1999). The most frequently used assessment criteria include content (reliability and accuracy), design and aesthetics (layout and interactivity), currency of information and disclosure of authors and sponsors. Okamura et al. (2002) evaluated 197 fertility websites that an infertile patient in the USA might reasonably visit and reported that fewer than half of the websites satisfied one or more of the four accountability standards (authorship, attribution, disclosure and currency), and only 2% satisfied all four standards. Likewise, Abusief et al. (2007) evaluated website general characteristics as well as adherence to guidelines produced by the American Society for Reproductive Medicine (ASRM, 2004) and the Society for Assisted Reproductive Technology (SART) of 286 SART-registered clinics advertising their services. The authors found that the majority of websites did not follow the ASRM/SART mandatory guidelines for advertising. Furthermore, the publication of success rates did not adhere to the guidelines set by ASRM/SART, potentially misleading patients about chances of success.
The Human Fertilisation and Embryology Authority (HFEA, 2007), the fertility regulatory body in the UK, have recently published a new code of practice stating that all UK clinics offering fertility treatment must provide core information to patients. Such information should include clear outcomes and limitations of procedures, costs of treatments and possible side effects and risks of any treatment.
To our knowledge, there have been no studies evaluating the quality of infertility information available to patients on the internet in the UK. Our aim was to assess UK infertility websites and make direct comparisons between NHS and privately owned treatment providing websites.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Search strategy
In order to replicate the search that might be performed by an infertile couple, the search term ‘infertility’ was used on the search engine website Google (2007)
to generate a list of sites. The first 100 of these (that were functional and relevant) were accessed and classified into one of the following categories: treatment-providing sites [either National Health Service (NHS), assisted reproductive technology (ART) clinics, private ART clinics or non-ART treatment providing clinics] or information-providing sites (either commercial, non-profit-making or patient forum). Sites were discarded if they were duplicates of already selected sites, irrelevant, inaccessible or inactive. Search engine sites were discarded, as were journal websites, news websites and guideline websites, bookshop websites (such as Amazon), research centre sites and all non-UK websites. In addition to the Google search on the keyword ‘infertility’, a more targeted search was performed for NHS ART clinic websites and seven additional websites were found.
An independent investigator performed a similar search on 6th November 2007 to ensure that the websites found in the original search were representative of those that patients might encounter. The independent search confirmed that the websites used in this study were still functioning and that they were placed in the correct categories for the purpose of their evaluation.
- NHS ART clinic: clinic managed and funded by the NHS. A website was considered to be NHS sponsored if the affiliation was clearly stated in the web pages (Fig. 1).
- Private ART clinic: clinic managed and funded by a private organization.
- Non-ART treatment-providing: organization or clinic selling fertility treatments or tests (e.g. reflexology, acupuncture, ovulation-testing kits) other than ART.
- Commercial information-providing: information-providing site, owned and managed by a profit-making company.
- Non-profit-making information-providing: information-providing site, owned and managed by a registered charity or other non-profit-making organization.
- Patient forum: forum/chat room designed for patients to discuss their infertility experiences.
|
Website quality evaluation
The quality of each website was assessed against three key features: credibility, accuracy and navigability (Silberg et al., 1997
; Eysenbach and Diepgen, 1998; Ambre et al., 1999; Kim et al., 1999; Price and Hersh, 1999; Kunst et al., 2002).
Credibility
Credibility was assessed using six criteria, namely authorship [the name(s) of author(s) are clearly displayed on the webpage either under the title of a piece of writing or as a signature at the bottom of the page], currency (either date last amended, copyright date or date that individual information is written), presence of a disclaimer (legal statement whereby the authors of the website state they do not take responsibility for what is written on the site and what readers choose to do with the information), presence of a review process (information provided about a review board for the material on the website), presence of funding or sponsorship information (information about whom the organization is funded by) and absence of advertising of financial incentives (such as three cycles for the price of 2).
Accuracy
Accuracy was assessed using three criteria: claims supported by valid evidence or professional body guidelines, provision of clearly defined success rates with a numerator and denominator (such as live birth rate per cycle started) and display of accurate references.
Navigability
Navigability was assessed using six criteria; the presence of functional internal and external links, viewer is not redirected to unintended sites, the presence of an active feedback mechanism with e-mail-based contact details clearly provided, a frequently asked questions (FAQ) page featured, the presence of a site map (as a separate webpage or a section of the main home page) and the presence of an internal search engine facility.
Scoring system
For each of the 15 criteria defined above (i.e. six criteria from credibility, three from accuracy and six from navigability), a score of one point was allocated to the website under evaluation. If a website failed to meet the criteria, a score of zero was allocated. We calculated the score for each key feature (credibility, accuracy and navigability) as well as the total score for each website. As we assessed 107 websites, the maximum score available for credibility was 642 (6x107), for accuracy 321 (3x107) and for navigability it was again 642 (6x107).
Statistical analysis
Results were processed using StatsDirect (StatsDirect Ltd, Cheshire, UK). Because the data were non-parametric, the median scores for each type of websites were compared using the Mann–Whitney U-test. NHS ART clinic websites were compared with private ART clinic websites and non-ART treatment-providing sites using median scores. Differences were considered statistically significant if P < 0.05.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Of the 107 websites analysed, 27 were non-ART treatment-providing websites, 23 were commercial information-providing websites, 20 belonged to private ART clinics, 16 were non-profit-making information providing websites, 14 were NHS ART clinics and 7 were patient forum websites. The position of a website in each category is represented in Fig. 2, along with the spread of websites within each category.
|
According to the Google ranking, the types of websites that patients are most likely to come across during a search for the keyword ‘infertility’ are commercial information-providing websites. These types of websites feature consistently throughout the Google results (the position of commercial information-providing websites ranging from 1st to 94th) and they are ranked higher than all the other types of websites (the first commercial information-providing website was ranked in first place, whereas the first NHS ART clinic website did not appear until the 45th position).
Website scores
Overall, the total scores for all types of websites in each category were low. Websites were particularly low-scoring in the ‘accuracy’ category, where only 50 points out of a possible 321 (16%) were achieved. The category in which websites scored the most was navigability in which 60% (387/642) of available points were scored by all websites. For credibility, the websites scored 43% (275/642) of the available points.
Within the credibility category, only 7% of websites exhibited information about a review process, 25% included authors' names and 35% detailed who financed them. Just over half of websites (56%) included a legal disclaimer. Websites scored highly in the ‘currency’ and ‘no financial incentives’ criteria (66% and 68% of websites, respectively).
Within the accuracy category, only 10% of websites supported the claims they made with evidence from scientific research or published guidelines, 23% reported clear success rates and 14% of websites displayed the sources of the information they provided.
The number of websites scoring in the navigability category was generally high. Nearly 90% of websites had fully functional internal and external links and almost all of them (96%) did not redirect the viewer to unintentional sites. However, only 63% of websites supplied a means for the viewer to provide feedback, 44% had an internal search engine, 39% had a site map and 31% of websites featured a FAQ page.
Comparison of NHS ART clinic websites with private ART clinic websites
The collective median score of NHS ART clinic websites was significantly higher than that of private clinic websites in all three categories and in the overall total score (Table I). Significantly more NHS ART clinic websites than their private counterparts displayed currency of the site (P = 0.02), information about funding and sponsorship (P < 0.0001) and clearly-reported success rates (P = 0.04). Forty percentage of private ART clinic websites either failed to report success rates or reported them without clear indication of numerator and denominator. None of the NHS clinic websites used financial incentives, whereas 65% of the private clinics did (P = 0.01). Both groups had very few, if any, websites which commented on a peer review process, gave support for claims made or displayed their sources (Table II).
|
|
Comparison of NHS ART clinic websites with non-ART treatment-providing websites
The median score of NHS ART clinic websites was significantly higher than that of non-ART treatment-providing websites in all three categories and in the overall total score (Table I). Furthermore, NHS ART clinic websites were significantly more likely to provide information about their funding and sponsorship (P < 0.0001), report success rates correctly (P < 0.0001) and to contain a FAQ page and a site map (P = 0.02 and 0.01, respectively) compared with non-ART treatment-providing websites (Table II). Non-ART treatment-providing sites were more likely to use financial incentives (P = 0.001), even though none of them reported success rates correctly if at all.
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
This is the first study to evaluate the quality of infertility information available to patients on the internet in the UK. The results of this study confirm that the quality of infertility information on the internet is, at best, variable. Scores for all types of websites in all three key quality features, namely credibility, accuracy and navigability, were low.
It has been suggested that the most pertinent issue regarding infertility information is the accuracy of the content of the website (Jain and Barbieri, 2005). We found that all websites, regardless of type, were particularly poor in the accuracy category, with the majority of them failing to meet any of the accuracy criteria. Moreover, very few websites showed evidence of a review process, a procedure which is paramount in the publication of information in medical journals. This highlights the lack of regulatory mechanisms provided for the posting of information on the World Wide Web and erodes the level of credibility necessary to enable the online infertile patient to make decisions about the accuracy of the content offered.
The category in which websites scored most highly was the navigability category. This is perhaps unsurprising as it is relatively easy for a website to improve its navigability, but ensuring that its claims are supported by valid evidence and referencing all of their sources is a harder and more time-consuming task (Epstein and Rosenberg, 2005).
Our study also showed that NHS ART clinic websites scored higher than private ART clinic and non-ART treatment-providing websites in all three quality categories. In particular, NHS ART clinics were significantly more likely to display information about the currency of the website and source of funding, thus giving the viewer an indication of how well-maintained the site is as well as its credibility. Of course, the relative lack of currency information or source of funding on private ART clinics and non-ART treatment-providing clinics does not necessarily mean that these sites were not up to date or not credible; it simply means that patients accessing the sites are less able to assess their overall quality or make a judgement on the information they provide (Huang et al., 2005; Selman et al., 2006).
In May 2004, the ASRM produced guidelines for advertising by fertility clinics. The Society placed particular importance on the publishing of clear and understandable IVF success rates as a means of attracting patients to a clinic or centre. In the current study, websites were assessed for the presence of success rates with the clear identification of terms comprising the numerator and denominator of the success rate (e.g. live birth per cycle started). NHS ART clinic websites were significantly better at displaying clear, easily interpretable success rate figures than the private ART clinic or other non-ART treatment-providing websites. Nearly all of the NHS ART clinic websites displayed success rates correctly compared with just over half of the private ART clinics and none of the non-ART treatment-providing websites. This finding represents a clear indication of the disparity in these websites' overall quality (Huang et al., 2005; Jain and Barbieri, 2005).
Furthermore, our study also highlights a fundamental flaw in accessing medical information on the internet. Patients searching for the term ‘infertility’ in a search engine are most likely to access consumer-related sites and sites owned by non-government organizations. In this study, only 7% of the websites found by Google were NHS ART clinics. Furthermore, NHS ART clinic websites were so elusive that after the initial Google search returned only 7 NHS websites out of 459 websites, a new Google search had to be performed using the phrase ‘NHS fertility clinic’ in order to retrieve more websites for the NHS ART clinic group. Even with a more targeted Google search, the NHS ART clinic websites were difficult to find. This suggests that NHS ART clinic websites, as well as being less numerous than most other types of websites, are also failing to employ successful e-commerce marketing strategies (Ellsworth and Ellsworth, 1996). With the increase in internet use by all health consumers (Theodosiou and Green, 2003; Selman et al., 2006), ignoring the potential of e-marketing and e-commerce could prove detrimental for the NHS.
When considering the implications of this study, it is important to note two limitations. The first is the intrinsic difficulty in assessing quality of information using scoring criteria. Although the assessment tool used in this study was deemed to be comprehensive, it is nevertheless an indirect measure of quality (Meric et al., 2002).
Second, the scoring system was designed such that each scoring criterion was equally weighted, as were the three key quality features. In future studies, thought might be given to developing a tool where certain criteria are given more weighting depending on their importance to the issue of quality. For example, more credit might in future be given to those websites which scored points in the accuracy category in an attempt to improve the assessment process.
The regulation of information published on the World Wide Web is a near impossible task to achieve. One suggestion for ensuring that fertility patients access websites of high quality is for general practitioners and fertility specialists to actively recommend sites that they themselves have reviewed using similar criteria to the one developed for this study. Authorities such as the HFEA might consider a similar review of websites that are likely to be accessed by infertile couples in order that they too can formally recommend websites of higher quality. The recent Code of Practice published by the HFEA states that publicity material published by any fertility centre must include the centre's own live birth rate per treatment cycle, as well as the national live birth rate per treatment cycle. The HFEA does not, however, stipulate that this guideline applies to information published on the internet. We suggest that the Code of Practice be amended in order that both private and NHS clinic websites are explicitly required to conform to this existing recommendation. This item can also become part of the formal clinic inspection process by the HFEA.
Patients are likely to be better informed if the internet is used as an adjunct to, rather than a substitute for, the information provided by the clinician in the consulting room, both in primary and secondary care. We hope our study will encourage researchers in other European countries to assess the quality of infertility information available to infertile couples across Europe. Such studies would need to account for differences in language, internet search engines and the structure of local infertility services.
In conclusion, the quality of internet resources available to infertile couples is variable, but generally poor. Differences in the quality of information provided exist between the different types of websites. Clinicians have a responsibility to guide their patients towards reliable and accurate information on the internet.
Conflict of interest: The authors declare they have no conflict of interest.
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Abusief M, Hornstein M, Jain T. Assessment of United States fertility clinic websites according to the American Society for Reproductive Medicine (ASRM)/Society for Assisted Reproductive Technology (SART) guidelines. Fertil Steril (2007) 87:88–92.[CrossRef][Web of Science][Medline]
Ambre J, Guard R, Perveiler FM, Renner J, Rippen H. Criteria for Assessing the Quality of Health Information on the Internet: Health Information Technology Institute, Agency for Health Care Policy and Research (1999) http://hitiweb.mitretek.org/iq/onlycriteria.html.
American Society for Reproductive Medicine. Guidelines for advertising by ART programs. ARSM Practice Committee Report. (2004) Birmingham, AL: American Society for Reproductive Medicine. May.
Ellsworth J, Ellsworth M. Marketing on the Internet—Multimedia Strategies for the WWW (1996) New York: John Wiley & Sons.
Epstein Y, Rosenberg H. Assessing infertility information on the internet: challenges and possible solutions. Fertil Steril (2005) 83:553–555.[CrossRef][Web of Science][Medline]
Eysenbach G, Diepgen TL. Towards quality management of medical information on the internet: evaluation, labeling, and filtering of information. BMJ (1998) 317:1496–1500.
Google. http://www.google.co.uk/ (13 March 2007, date last accessed).
Huang J, Al-Fozan H, Tan S, Tulandi T. Internet use by patients seeking infertility treatment. Int J Gynecol Obsetr (2003) 83:75–76.[CrossRef]
Huang J, Discepola F, Al-Fozan H, Tulandi T. Quality of fertility clinic websites. Fertil Steril (2005) 83:538–544.[CrossRef][Web of Science][Medline]
Human Fertilisation Embryology Authority (HFEA). Code of Practice. In: 7th revision (2007) London: Human Fertilisation and Embryology Authority.
Internet World Stats. http://www.internetworldstats.com (11 November 2007, date last accessed).
Jain T, Barbieri RL. Website quality assessment: Mistaking apples for oranges. Fertil Steril (2005) 83:545–547.[CrossRef][Web of Science][Medline]
Kim P, Eng T, Deering MJ, Maxfield A. Published criteria for evaluating health related web sites: review. BMJ (1999) 318:647–649.
Kunst H, Groot D, Latthe P, Latthe M, Khan KS. Accuracy of information on apparently credible websites: survey of five common health topics. BMJ (2002) 324:581–582.
Meric F, Bernstam E, Mirza N, Hunt K, Ames F, Ross M, Kuerer H, Pollock R, Musen M, Singletary S. Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites. BMJ (2002) 324:577–581.
Okamura K, Bernstein J, Fidler A. Assessing the quality of infertility resources on the world wide web: tools to guide clients through the maze of fact and fiction. J Midwifery Womens Health (2002) 47:264–268.[CrossRef][Web of Science][Medline]
Price S, Hersh W. Filtering Web pages for quality indicators: an empirical approach to finding high quality consumer health information on the World Wide Web. (1999) Proceedings of AMIA Symposium. 911–915.
Selman T, Prakesh T, Khan K. Quality of health information for cervical cancer treatment on the internet. BMC Women's Health (2006) 6:9–13.[CrossRef][Medline]
Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling and assuring the quality of medical information on the internet. JAMA (1997) 277:1244–1245.
The Harris Poll®. http://www.harrisinteractive.com/harris_poll/index.asp (24 February 2007, date last accessed).
Theodosiou L, Green J. Emerging challenges in using health information from the internet. Adv Psychiatr Treat (2003) 9:387–396.
Submitted on November 27, 2007; resubmitted on January 23, 2008; accepted on February 4, 2008.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  I. W.H. van Empel, W. L.D.M. Nelen, E. T. Tepe, E. A.P. van Laarhoven, C. M. Verhaak, and J. A.M. Kremer Weaknesses, strengths and needs in fertility care according to patients Hum. Reprod., October 27, 2009; (2009) dep362v1. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||