Behaviour Change & Technology

It is recommended (Crutzen, De Nooijer, Brouwer, Oenema, Brug, & De Vries, submitted) to conduct experimental research in more controlled settings to increase evidence-based insight into effectiveness of strategies regarding dissemination of and exposure to Internet-delivered interventions, before applying these strategies in practice. Advantage of such a controlled setting is the minimisation of possible confounding effects. Disadvantage of these experimental settings, however, is the isolated way in which strategies are tested on effectiveness, which is less comparable with real-life implementation.

Even if strategies are immediately applied in practice, there are appropriate designs, such as a time series design (Chen et al., 2005; Murry, Stam, & Lastovicka, 1993), to test effectiveness of these strategies. In such a design, strategies are first applied separately. Subsequently, combinations of several strategies are applied. Intervention use is monitored during the whole period to determine which strategy or combination between them is most effective. Although this design is more sensitive to confounding effects (e.g. changing environment), strategies are tested in a less isolated way and this could give more insight into effectiveness regarding dissemination and exposure in real life.

If all methods described above are inapplicable due to limited resources, one could test the effectiveness of dissemination strategies by simply asking visitors where they came from and how they heard about the intervention (Gordon, Akers, Severson, Danaher, & Boles, 2006). Although this method is based on self-report and therefore less objective, it is easily applicable and less time consuming. Yet, this method remains a last resort (when all else fails).

Dissemination and exposure do not only depend on the intervention itself, however, but also on its users. There is evidence that the acquisition of skills to use a website may influence its adoption (Paswan & Ganesh, 2003). It has also been shown, however, that Internet self-efficacy is not a significant predictor of exposure (Steele, Mummery, & Dwyer, 2007). If familiarity with a website increases, then perceived usability influences loyalty to the website (Casaló, Flavián, & Guinalíu, 2008). This is in line with the principle of cognitive lock-in, accounting for users’ preference of better-known websites (Murray & Häubl, 2002). In terms of Internet-delivered interventions, this could be conducive to revisits of the intervention.

Implications for practice
Caution regarding the use of strategies to improve dissemination and exposure is recommended as long as evidence-based insight into effectiveness is scarce. Evidence-based insight into strategies which aim to improve dissemination and exposure, however, could also be gained by applying strategies in practice and investigating, for example, server registrations (objective) or visitors’ self-reporting (subjective). Applying strategies in practice can be seen as natural experiments to test effectiveness of these strategies. Although this research method is more common in fields where laboratory experiments are more difficult, such as sociology and economics (Angrist & Evans, 1998), there are also examples in the field of health promotion – e.g. with regard to the effect of a smoking ban (Sargent, Shepard, & Glantz, 2004).

Implications for future research
To increase evidence-based insight into effectiveness of strategies regarding dissemination and exposure, future research should not be limited to experimental research in controlled settings, but also use alternatives such as a time series design. Such a time series design can also be used to test effectiveness of dissemination strategies for existing Internet-delivered interventions.

References:
Angrist, J., & Evans, W. (1998). Children and their parents’ labor supply: evidence from exogenous variation in family size. American Economic Review, 88, 450-477.

Casaló, L., Flavián, C., & Guinalíu, M. (2008). The role of perceived usability, reputation, satisfaction and consumer familiarity on the website loyalty formation process. Computers in Human Behavior, 24, 325-345.

Chen, J., Smith, B. J., Loveday, S., Bauman, A., Costello, M., Mackie, B., et al. (2005). Impact of a mass media campaign upon calls to the New South Wales Hep C helpline. Health Promotion Journal of Australia, 16, 11-14.

Crutzen, R., De Nooijer, J., Brouwer, W., Oenema, A., Brug, J., & De Vries, N. K. (submitted). Strategies to facilitate exposure to Internet-delivered health behaviour change interventions aimed at adolescents or young adults: a systematic review.

Gordon, J. S., Akers, L., Severson, H. H., Danaher, B. G., & Boles, S. M. (2006). Successful participant recruitment strategies for an online smokeless tobacco cessation program. Nicotine & Tobacco Research, 8, S35-S41.

Murray, K. B., & Häubl, G. (2002). The fiction of no friction: A user skills approach to cognitive lock-in. Advances in Consumer Research, 29, 8-10.

Murry, J. P., Stam, A., & Lastovicka, J. L. (1993). Evaluating an anti-drinking and driving advertising campaign with a sample survey and time series intervention analysis. Journal of the American Statistical Association, 88, 50-56.

Paswan, A. K., & Ganesh, G. (2003). Familiarity and interest: in a learning center service context. Journal of Services Marketing, 17, 393-419.

Sargent, R. P., Shepard, R. M., & Glantz, S. A. (2004). Reduced incidence of admissions for myocardial infarction associated with public smoking ban: before and after study. British Medical Journal, 328, 977-980.

Steele, R. M., Mummery, W. K., & Dwyer, T. (2007). Examination of program exposure across intervention delivery modes: face-to-face versus internet. International Journal of Behavioral Nutrition and Physical Activity, 4, 7.

Send article as PDF to

Before determining if an Internet-delivered intervention has been disseminated successfully, one should first determine when an intervention is successfully disseminated. This is not as straightforward as it may seem. Absolute figures may cause unrealistic optimism, since thousands of visitors are not uncommon for websites in general. Relative figures, on the other hand, may cause unrealistic pessimism, since they may be very low if one relates number of visitors to Internet penetration rates. To estimate the reach of a recruitment approach and the generalisability of results, it is important to report the target group, the number exposed to recruitment, the number who respond, the number eligible, and the number who actually participate (Graham, Bock, Cobb, Niaura, & Abrams, 2006). Furthermore, one should determine a final target in advance. There is, however, no “default” cut off point for such a target to determine successfulness of dissemination, since this may differ per intervention (e.g., an intervention aimed at reducing alcohol use among high school students versus party drug use among school drop-outs). Issues such as the general accessibility of the target group and the prevalence of behaviours among the target group should be taken into account when determining a final target for successfulness of dissemination.

The determination if an Internet-delivered intervention has been disseminated successfully does not only depend on your final target, but also on the denominator you choose to estimate reach. In a study by Graham et al. (2006), for example, Internet users were recruited based on use of the terms quit(ting) smoking or stop(ping) smoking in a major search engine. When a user clicked on a link to the intervention in the results of a search engine query, an intercept page appeared inviting them to participate in the study of Graham and colleagues. If they accepted, three questions were asked to determine preliminary eligibility. Reach estimates vary depending on the denominator selected: 2.7% of all Internet users seeking cessation information; 6.9% of those who demonstrated preliminary interest in the study; 13.7% of those who were eligible to participate; 21.1% of those eligible and recruited; and 51.3% of those consented.

This also raises an interesting statistical issue; since it is difficult to identify the population to which samples refer when there is no clear sampling method (i.e., everybody can access your intervention). Questions need to be asked to determine eligibility to participate. It remains unclear, however, whether missing data regarding eligibility is due to visitors’ perception of non-eligibility (e.g., parents visiting the intervention because they are interested in the subject or non-smokers accidentally hitting on a website aimed at smoking cessation) or actual drop-out of potential participants. Appropriate methods of analysis are needed to deal with the vast amount of missing data (Christensen, Griffiths, & Korten, 2002) and to determine the recruited sample size from your target group.

The difficulties regarding measurement of online reach are not limited to the field of health promotion. The Interactive Advertising Bureau Europe (IAB Europe) and the European Interactive Advertising Association (EIAA) have announced to develop a measurement standard for website reach. In this project, denominated as Measurement of Interactive Audience Project (MIA Project), several online measurement methods will be evaluated (Marketing Online, 2007).

Implications for practice
It is recommended to report the target group of your intervention, the number exposed to recruitment, the number who responded, the number eligible, and the number who actually participated to determine whether dissemination has been successful. These numbers should be compared with a final target which should be determined in advance and which depends on the intervention (e.g., its subject and target group). Despite this dependence on the intervention itself, one may look at other Internet-delivered interventions to get an idea about the order of magnitude of a suitable final target.

Implications for future research
Analyses regarding data from Internet-delivered interventions are not as straightforward as they may seem, since data can be missing for several reasons (e.g., visitors which do not perceive the intervention as being relevant to them once they logged on or visitors whose needs are fulfilled halfway the intervention). Although it is probably too conservative to consider all missing data as drop-outs, this is recommended as long as appropriate methods of analysis are lacking. Furthermore, due to the openness of Internet as a medium, not all visitors of your intervention are necessary members of your target group (e.g., parents, people who are generally interested in the topic of your intervention). These “participants” should be excluded from data analysis.

References:
Christensen, H., Griffiths, K. M., & Korten, A. (2002). Web-based cognitive behavior therapy: analysis of site usage and changes in depression and anxiety scores. Journal of Medical Internet Research, 4, e3.

Graham, A. L., Bock, B. C., Cobb, N. K., Niaura, R., & Abrams, D. B. (2006). Characteristics of smokers reached and recruited to an internet smoking cessation trial: a case of denominators. Nicotine & Tobacco Research, 8, S43-48.

Marketing Online. (2007). Europese standaard voor meten online bereik in de maak [European standard for measuring online reach in preparation]. Retrieved October 26, 2009, from http://www.marketingonline.nl/nieuws/ModuleItem52230.html.

Send article as PDF to

Evidence from efficacy trials indicates that exposure rates to Internet-delivered interventions are low (De Nooijer, Oenema, Kloek, Brug, De Vries, & De Vries, 2005), and they may be even lower when these interventions are implemented in real life rather than in a research setting (Evers, Cummins, Prochaska, & Prochaska, 2005). Exposure of individuals to the intervention content, through use of the intervention, is necessary since attention is a prerequisite to establish desired behaviour change (McGuire, 1985). Therefore, it remains important to assess exposure to Internet-delivered interventions.

There are several measures to assess exposure to Internet-delivered interventions, such as frequency and duration of visits, but there is no gold standard. Each exposure measure relates to a different aspect of exposure (Danaher, Boles, Akers, Gordon, & Severson, 2006). One can visit an intervention very frequent, for example, but only for a short period of time. Duration of visits, on the other hand, does not necessarily give a clear picture of participants’ online activity, since one does not know to how much information participants are possibly exposed. Therefore, number of visited web pages would be more appropriate to assess online activity. All such exposure measures can be tracked objectively and are, in contrast to self-reported exposure measures, independent of participants’ memory, interpretation, or social desirability.

Linking exposure measures to variables at the individual level

To fruitfully utilise exposure measures it has to be possible to link them to variables at the individual level, to be able to compare subgroups that differ on socio-demographic, psycho-social, or behavioural measures. Moreover, exposure measures on the individual level can also be linked to outcome measures of an intervention. By doing so, it is possible to study potentially mediating effects of objectively tracked exposure on interventions’ outcome measures. The latter, however, is no common practice to date (Crutzen, De Nooijer, Brouwer, Oenema, Brug, & De Vries, submitted). For example, recent studies (An et al., 2006; Chen & Yeh, 2006; Escoffery, McCormick, & Bateman, 2004) reported a limited number of exposure measures and did not relate them to the intervention’s outcome measures, making it impossible to study potentially mediating effects of exposure.

Practical implications

There are no known technical barriers to track exposure measures of Internet-delivered interventions. It is important, however, to realize this from the start of an intervention development process and to involve technical staff during this initial phase (Crutzen, De Nooijer, Brouwer, Oenema, Brug, & De Vries, e-pub ahead of print). Furthermore, dependent on the laws in certain countries or states, it may be necessary to take additional steps. In a recent Dutch project, for example, it was necessary to register the project at the Dutch Data Protection Authority, which supervises the fair and lawful use and security of personal data (Crutzen, De Nooijer, Candel, & De Vries, 2008). If these legal issues are covered, we recommend to track as many exposure measures as possible since there is no gold standard. Furthermore, having exposure measures available is also useful during process evaluation of Internet-delivered interventions, as has been shown in other studies (Barak & Fisher, 2003; Lou, Zhao, Gao, & Shah, 2006; Patten et al., 2007; Roberto, Zimmerman, Carlyle, & Abner, 2007). These exposure measures provide detailed insight into where participants either leave the intervention website or have come to a standstill. This information can be used to adapt interventions to users’ needs and therewith increase exposure rates and probability of behaviour change.

References:

An, L. C., Perry, C. L., Lein, E. B., Klatt, C., Farley, D. M., Bliss, R. L., et al. (2006). Strategies for increasing adherence to an online smoking cessation intervention for college students. Nicotine & Tobacco Research, 8 (S1), S7-S12.

Barak, A., & Fisher, W. A. (2003). Experience with an Internet-based, theoretically grounded educational resource for the promotion of sexual and reproductive health. Sexual and Relationship Therapy, 18, 293-308.

Chen, H.-H., & Yeh, M.-L. (2006). Developing and evaluating a smoking cessation program combined with an Internet-assisted instruction program for adolescents with smoking. Patient Education and Counseling, 61, 411-418.

Crutzen, R., De Nooijer, J., Brouwer, W., Oenema, A., Brug, J., & De Vries, N. K. (e-pub ahead of print). A conceptual framework for understanding and improving adolescents’ exposure to Internet-delivered interventions. Health Promotion International.

Crutzen, R., De Nooijer, J., Brouwer, W., Oenema, A., Brug, J., & De Vries, N. K. (submitted). How to facilitate exposure to Internet-delivered health behavior change interventions aimed at adolescents or young adults? A systematic review.

Crutzen, R., De Nooijer, J., Candel, M. J. J. M., & De Vries, N. K. (2008). Adolescents who intend to change multiple health behaviours choose greater exposure to an Internet-delivered intervention. Journal of Health Psychology, 13, 906-911.

Danaher, B. G., Boles, S. M., Akers, L., Gordon, J. S., & Severson, H. H. (2006). Defining participant exposure measures in web-based health behavior change programs. Journal of Medical Internet Research, 8, e15.

De Nooijer, J., Oenema, A., Kloek, G., Brug, J., De Vries, H., & De Vries, N. K. (2005). Bevordering van Gezond Gedrag via Internet: Nu en in de Toekomst [Promotion of Healthy Behavior through the Internet: Now and in the Future]. Maastricht: Maastricht University.

Escoffery, C., McCormick, L., & Bateman, K. (2004). Development and process evaluation of a web-based smoking cessation program for college smokers: innovative tool for education Patient Education and Counseling, 53, 217-225.

Evers, K. E., Cummins, C. O., Prochaska, J. O., & Prochaska, J. M. (2005). Online health behavior and disease management programs: are we ready for them? Are they ready for us? Journal of Medical Internet Research, 7, e27.

Lou, C. H., Zhao, Q., Gao, E. S., & Shah, I. H. (2006). Can the Internet be used effectively to provide sex education to young people in China? Journal of Adolescent Health, 39, 720-728.

McGuire, W. J. (1985). Attitudes and attitude change. In M. Lindsay & E. Aronson (Eds.), The Handbook of Social Psychology (pp. 233-346). New York: Random House.

Patten, C. A., Rock, E., Meis, T. M., Decker, P. A., Colligan, R. C., Pingree, S., et al. (2007). Frequency and type of use of a home-based, Internet intervention for adolescent smoking cessation. Journal of Adolescent Health, 41, 437-443.

Roberto, A. J., Zimmerman, R. S., Carlyle, K. E., & Abner, E. L. (2007). A computer-based approach to preventing pregnancy, STD, and HIV in rural adolescents. Journal of Health Communication, 12, 53-76.

Send article as PDF to