Behaviour Change & Technology

Before determining if an Internet-delivered intervention has been disseminated successfully, one should first determine when an intervention is successfully disseminated. This is not as straightforward as it may seem. Absolute figures may cause unrealistic optimism, since thousands of visitors are not uncommon for websites in general. Relative figures, on the other hand, may cause unrealistic pessimism, since they may be very low if one relates number of visitors to Internet penetration rates. To estimate the reach of a recruitment approach and the generalisability of results, it is important to report the target group, the number exposed to recruitment, the number who respond, the number eligible, and the number who actually participate (Graham, Bock, Cobb, Niaura, & Abrams, 2006). Furthermore, one should determine a final target in advance. There is, however, no “default” cut off point for such a target to determine successfulness of dissemination, since this may differ per intervention (e.g., an intervention aimed at reducing alcohol use among high school students versus party drug use among school drop-outs). Issues such as the general accessibility of the target group and the prevalence of behaviours among the target group should be taken into account when determining a final target for successfulness of dissemination.

The determination if an Internet-delivered intervention has been disseminated successfully does not only depend on your final target, but also on the denominator you choose to estimate reach. In a study by Graham et al. (2006), for example, Internet users were recruited based on use of the terms quit(ting) smoking or stop(ping) smoking in a major search engine. When a user clicked on a link to the intervention in the results of a search engine query, an intercept page appeared inviting them to participate in the study of Graham and colleagues. If they accepted, three questions were asked to determine preliminary eligibility. Reach estimates vary depending on the denominator selected: 2.7% of all Internet users seeking cessation information; 6.9% of those who demonstrated preliminary interest in the study; 13.7% of those who were eligible to participate; 21.1% of those eligible and recruited; and 51.3% of those consented.

This also raises an interesting statistical issue; since it is difficult to identify the population to which samples refer when there is no clear sampling method (i.e., everybody can access your intervention). Questions need to be asked to determine eligibility to participate. It remains unclear, however, whether missing data regarding eligibility is due to visitors’ perception of non-eligibility (e.g., parents visiting the intervention because they are interested in the subject or non-smokers accidentally hitting on a website aimed at smoking cessation) or actual drop-out of potential participants. Appropriate methods of analysis are needed to deal with the vast amount of missing data (Christensen, Griffiths, & Korten, 2002) and to determine the recruited sample size from your target group.

The difficulties regarding measurement of online reach are not limited to the field of health promotion. The Interactive Advertising Bureau Europe (IAB Europe) and the European Interactive Advertising Association (EIAA) have announced to develop a measurement standard for website reach. In this project, denominated as Measurement of Interactive Audience Project (MIA Project), several online measurement methods will be evaluated (Marketing Online, 2007).

Implications for practice
It is recommended to report the target group of your intervention, the number exposed to recruitment, the number who responded, the number eligible, and the number who actually participated to determine whether dissemination has been successful. These numbers should be compared with a final target which should be determined in advance and which depends on the intervention (e.g., its subject and target group). Despite this dependence on the intervention itself, one may look at other Internet-delivered interventions to get an idea about the order of magnitude of a suitable final target.

Implications for future research
Analyses regarding data from Internet-delivered interventions are not as straightforward as they may seem, since data can be missing for several reasons (e.g., visitors which do not perceive the intervention as being relevant to them once they logged on or visitors whose needs are fulfilled halfway the intervention). Although it is probably too conservative to consider all missing data as drop-outs, this is recommended as long as appropriate methods of analysis are lacking. Furthermore, due to the openness of Internet as a medium, not all visitors of your intervention are necessary members of your target group (e.g., parents, people who are generally interested in the topic of your intervention). These “participants” should be excluded from data analysis.

References:
Christensen, H., Griffiths, K. M., & Korten, A. (2002). Web-based cognitive behavior therapy: analysis of site usage and changes in depression and anxiety scores. Journal of Medical Internet Research, 4, e3.

Graham, A. L., Bock, B. C., Cobb, N. K., Niaura, R., & Abrams, D. B. (2006). Characteristics of smokers reached and recruited to an internet smoking cessation trial: a case of denominators. Nicotine & Tobacco Research, 8, S43-48.

Marketing Online. (2007). Europese standaard voor meten online bereik in de maak [European standard for measuring online reach in preparation]. Retrieved October 26, 2009, from http://www.marketingonline.nl/nieuws/ModuleItem52230.html.

Send article as PDF to

Evidence from efficacy trials indicates that exposure rates to Internet-delivered interventions are low (De Nooijer, Oenema, Kloek, Brug, De Vries, & De Vries, 2005), and they may be even lower when these interventions are implemented in real life rather than in a research setting (Evers, Cummins, Prochaska, & Prochaska, 2005). Exposure of individuals to the intervention content, through use of the intervention, is necessary since attention is a prerequisite to establish desired behaviour change (McGuire, 1985). Therefore, it remains important to assess exposure to Internet-delivered interventions.

There are several measures to assess exposure to Internet-delivered interventions, such as frequency and duration of visits, but there is no gold standard. Each exposure measure relates to a different aspect of exposure (Danaher, Boles, Akers, Gordon, & Severson, 2006). One can visit an intervention very frequent, for example, but only for a short period of time. Duration of visits, on the other hand, does not necessarily give a clear picture of participants’ online activity, since one does not know to how much information participants are possibly exposed. Therefore, number of visited web pages would be more appropriate to assess online activity. All such exposure measures can be tracked objectively and are, in contrast to self-reported exposure measures, independent of participants’ memory, interpretation, or social desirability.

Linking exposure measures to variables at the individual level

To fruitfully utilise exposure measures it has to be possible to link them to variables at the individual level, to be able to compare subgroups that differ on socio-demographic, psycho-social, or behavioural measures. Moreover, exposure measures on the individual level can also be linked to outcome measures of an intervention. By doing so, it is possible to study potentially mediating effects of objectively tracked exposure on interventions’ outcome measures. The latter, however, is no common practice to date (Crutzen, De Nooijer, Brouwer, Oenema, Brug, & De Vries, submitted). For example, recent studies (An et al., 2006; Chen & Yeh, 2006; Escoffery, McCormick, & Bateman, 2004) reported a limited number of exposure measures and did not relate them to the intervention’s outcome measures, making it impossible to study potentially mediating effects of exposure.

Practical implications

There are no known technical barriers to track exposure measures of Internet-delivered interventions. It is important, however, to realize this from the start of an intervention development process and to involve technical staff during this initial phase (Crutzen, De Nooijer, Brouwer, Oenema, Brug, & De Vries, e-pub ahead of print). Furthermore, dependent on the laws in certain countries or states, it may be necessary to take additional steps. In a recent Dutch project, for example, it was necessary to register the project at the Dutch Data Protection Authority, which supervises the fair and lawful use and security of personal data (Crutzen, De Nooijer, Candel, & De Vries, 2008). If these legal issues are covered, we recommend to track as many exposure measures as possible since there is no gold standard. Furthermore, having exposure measures available is also useful during process evaluation of Internet-delivered interventions, as has been shown in other studies (Barak & Fisher, 2003; Lou, Zhao, Gao, & Shah, 2006; Patten et al., 2007; Roberto, Zimmerman, Carlyle, & Abner, 2007). These exposure measures provide detailed insight into where participants either leave the intervention website or have come to a standstill. This information can be used to adapt interventions to users’ needs and therewith increase exposure rates and probability of behaviour change.

References:

An, L. C., Perry, C. L., Lein, E. B., Klatt, C., Farley, D. M., Bliss, R. L., et al. (2006). Strategies for increasing adherence to an online smoking cessation intervention for college students. Nicotine & Tobacco Research, 8 (S1), S7-S12.

Barak, A., & Fisher, W. A. (2003). Experience with an Internet-based, theoretically grounded educational resource for the promotion of sexual and reproductive health. Sexual and Relationship Therapy, 18, 293-308.

Chen, H.-H., & Yeh, M.-L. (2006). Developing and evaluating a smoking cessation program combined with an Internet-assisted instruction program for adolescents with smoking. Patient Education and Counseling, 61, 411-418.

Crutzen, R., De Nooijer, J., Brouwer, W., Oenema, A., Brug, J., & De Vries, N. K. (e-pub ahead of print). A conceptual framework for understanding and improving adolescents’ exposure to Internet-delivered interventions. Health Promotion International.

Crutzen, R., De Nooijer, J., Brouwer, W., Oenema, A., Brug, J., & De Vries, N. K. (submitted). How to facilitate exposure to Internet-delivered health behavior change interventions aimed at adolescents or young adults? A systematic review.

Crutzen, R., De Nooijer, J., Candel, M. J. J. M., & De Vries, N. K. (2008). Adolescents who intend to change multiple health behaviours choose greater exposure to an Internet-delivered intervention. Journal of Health Psychology, 13, 906-911.

Danaher, B. G., Boles, S. M., Akers, L., Gordon, J. S., & Severson, H. H. (2006). Defining participant exposure measures in web-based health behavior change programs. Journal of Medical Internet Research, 8, e15.

De Nooijer, J., Oenema, A., Kloek, G., Brug, J., De Vries, H., & De Vries, N. K. (2005). Bevordering van Gezond Gedrag via Internet: Nu en in de Toekomst [Promotion of Healthy Behavior through the Internet: Now and in the Future]. Maastricht: Maastricht University.

Escoffery, C., McCormick, L., & Bateman, K. (2004). Development and process evaluation of a web-based smoking cessation program for college smokers: innovative tool for education Patient Education and Counseling, 53, 217-225.

Evers, K. E., Cummins, C. O., Prochaska, J. O., & Prochaska, J. M. (2005). Online health behavior and disease management programs: are we ready for them? Are they ready for us? Journal of Medical Internet Research, 7, e27.

Lou, C. H., Zhao, Q., Gao, E. S., & Shah, I. H. (2006). Can the Internet be used effectively to provide sex education to young people in China? Journal of Adolescent Health, 39, 720-728.

McGuire, W. J. (1985). Attitudes and attitude change. In M. Lindsay & E. Aronson (Eds.), The Handbook of Social Psychology (pp. 233-346). New York: Random House.

Patten, C. A., Rock, E., Meis, T. M., Decker, P. A., Colligan, R. C., Pingree, S., et al. (2007). Frequency and type of use of a home-based, Internet intervention for adolescent smoking cessation. Journal of Adolescent Health, 41, 437-443.

Roberto, A. J., Zimmerman, R. S., Carlyle, K. E., & Abner, E. L. (2007). A computer-based approach to preventing pregnancy, STD, and HIV in rural adolescents. Journal of Health Communication, 12, 53-76.

Send article as PDF to