Empowering young people to make Positive Choices: Evidence-based resources for the prevention of alcohol and other drug use in Australian schools

There is increasing evidence that drug education and prevention programs implemented in schools are effective in reducing alcohol and drug use and associated harm. Despite this, evidence-based programs are not widely implemented in schools. We describe the development and evaluation of Positive Choices, an online portal to improve access to, and implementation of, evidence-based drug education in Australian schools. The portal was developed in consultation with drug and alcohol experts, as well as target users (teachers, parents and students). Research literature and drug education websites were systematically reviewed to identify resources meeting pre-specified inclusion criteria for relevance and quality. An evaluation survey was conducted with 82 teachers to clarify drug education practices and attitudes, and examine use and impact of Positive Choices in the first eight months post-launch of the site. Teacher survey responses reflected a strong preference for evidence-based teaching approaches, yet evidence-based programs were implemented by less than one in four of them. 153 Learning Communities | Special Issue: 2017 30th AChPER International Conference | Number 21 – November 2017 Scoping identified 114 evidence-informed resources, including a range of games, videos, and curriculum packages. Beta-testing feedback was overwhelming positive, indicating the Positive Choices portal was easy to navigate, relevant and useful. Teachers who used Positive Choices were more likely to consider supporting evidence when selecting a resource, reported their students were more engaged and felt more comfortable discussing drug and alcohol since using the portal. This study indicates the Positive Choices portal is a valuable, free and easily accessible online database for students, parents and teachers seeking up-to-date information and evidencebased drug education resources.


Introduction
Drug and alcohol use is common in Australia and globally, and can be associated with considerable harm at the individual and societal level. The negative impact of alcohol and other drug use is most significant among young Australians, corresponding with the typical period of onset of alcohol and other drug use. In the past year, one in three Australian teenagers aged 12-17 years drank a full serve of alcohol, one in ten put themselves at risk of short-term alcohol-related harm at least monthly, one in five were victims of an alcohol-related incident, and one in six tried cannabis (Australian Institute of Health and Welfare, 2014). The earlier a young person initiates alcohol and drug use, the greater the risk of negative outcomes including comorbid mental health problems, juvenile offending, and poorer educational outcomes, all of which negatively impact on current functioning and future life options (Grant, Stinson, & Harford, 2001;Tucker, Ellickson, Orlando, Martino, & Klein, 2005). To interrupt this trajectory, reduce harms and alleviate costs, effective prevention is critical. Secondary schools are an ideal location to equip young people with knowledge and skills that empower them to make informed and safe choices about alcohol and other drug use. Parents, and school staff are the primary sources of contact for young people seeking advice or help for drug use issues (Hampshire & Di Nicola, 2011;Sawyer et al., 2001), thus it is crucial they are equipped with accurate information and evidence-based response strategies. Given the considerable harms associated with alcohol and drug use, substantial societal benefit can be gained from even modest reductions in drug and alcohol use (Nherera & Jacklin, 2009).
A number of curriculum-based packages aimed at preventing harms relating to alcohol and other drugs have been developed for implementation in schools both in Australia and internationally. These programs have demonstrated varying degrees of effectiveness with outcomes of some programs compromised by obstacles to program implementation and dissemination, and inclusion of an abstinence-based as opposed to a harm-minimisation approach to prevention (Champion, Newton, Barrett, & Teesson, 2013;Lee, Cameron, Battams, & Roche, 2016;Teesson, Newton, & Barrett, 2012). A recent review conducted by the National Centre for Education and Training on Addiction systematically examined the effectiveness of school-based alcohol prevention approaches (Lee et al., 2016). Several important findings emerged from this review. First, the review identified three prevention programs (Climate Schools, Project Alert and All Stars) with good evidence of a positive effect according to the National Health and Medical Research Council (NHMRC) guidelines for assessing weight of research evidence. One of these programs, Climate Schools, was developed in Australia and has to date been evaluated in six randomised controlled trials across 155 Australian schools (Champion et al., 2016;Newton, Conrod, Rodriguez, & Teesson, 2014;Newton, Teesson, Vogl, & Andrews, 2010;Newton, Vogl, Teesson, & Andrews, 2009;Vogl, Newton, Champion, & Teesson, 2014). Trial results indicate that compared to health education as usual, implementation of Climate Schools is associated with increased student knowledge of drug effects and risks, and reduced frequency of alcohol, cannabis and ecstasy use for up to two years following program delivery. A further four programs were identified as having some evidence of a positive effect, including another program developed in Australia, the School Health and Alcohol Harm Reduction Project (SHAHRP) (McBride, Farringdon, Midford, Meuleners, & Phillips, 2004). The systematic review also provided guidance about the common characteristics of programs that were effective. These included: i) being based on accurate information, theoretical understandings of adolescent behaviour, and supported by empirical research; ii) going beyond providing factual information, and focusing on harm minimisation and skill development; iii) providing feedback about social acceptability, and highlighting that alcohol and other drug use is not as widespread as young people might think; iv) use of interactive styles and methods, and maximising students' interest using up-to-date materials and information; and v) promoting student resilience, social connectedness, encouraging strong relationships and communication between students, parents and staff (see Lee et al., 2016 for a detailed discussion of effective components).
Despite the availability of school-based programs that are effective at increasing student's knowledge and reducing alcohol and other drug use, evidence-based programs are not widely implemented in schools (Cuijpers, 2003). For best outcomes, it is critical that teachers, school staff and parents have access to evidence-based information and strategies that equip them to respond most effectively (Kyrrestad Strøm, Adolfsen, Fossum, Kaiser, & Martinussen, 2014). This paper describes the development of the Positive Choices online portal (www.positive choices.org.au), which was developed to improve access to and implementation of evidencebased drug and alcohol information and prevention resources. The Positive Choices portal was developed to align with the principles of effective prevention that have been identified by systematic review of the literature (Champion et al., 2013;Foxcroft & Tsertsvadze, 2011;Lee et al., 2016;Teesson et al., 2012). Positive Choices seeks to address the gap in implementation of evidence-based drug education in Australian schools, and responds to a call from principals for resources to support schools in building capacity to respond to drug and alcohol issues (Australian National Council on Drugs, 2013). This paper provides an overview of the development of the Positive Choices resource database and portal, which was conducted in consultation with principals, teachers, and parents of young people. It also reports findings of an evaluation study among 82 school teachers and staff, which aimed to: i) clarify current sources of information, and barriers to implementation of evidence-based resources; and ii) examine the use and impact of Positive Choices in the first 8 months post-launch of the site. This paper describes the development of the Positive Choices online portal (www.positive choices.org.au), which was developed to improve access to and implementation of evidencebased drug and alcohol information and prevention resources. The Positive Choices portal was developed to align with the principles of effective prevention that have been identified by systematic review of the literature (Champion et al., 2013;Foxcroft & Tsertsvadze, 2011;Lee et al., 2016;Teesson et al., 2012). Positive Choices seeks to address the gap in implementation of evidence-based drug education in Australian schools, and responds to a call from principals for resources to support schools in building capacity to respond to drug and alcohol issues (Australian National Council on Drugs, 2013). This paper provides an overview of the development of the Positive Choices resource database and portal, which was conducted in consultation with principals, teachers, and parents of young people. It also reports findings of an evaluation study among 82 school teachers and staff, which aimed to: i) clarify current sources of information, and barriers to implementation of evidence-based resources; and ii) examine the use and impact of Positive Choices in the first 8 months post-launch of the site.

Stage 1: Initial consultation and development
The Positive Choices drug prevention portal (www.positivechoices.org.au) was developed by researchers at the National Drug Research Centre (www.ndarc.med.unsw.edu.au) and the National Drug Research Institute (www.ndri.curtin.edu.au), with funding support from the Australian Government Department of Health. This section describes the development process, which followed a co-development model (Schuler & Namioka, 1993), with target users consulted at multiple points through the development process.

Method
Formative consultation informed development of a beta-version of the site, which was subsequently reviewed by topic area experts and the target audience. Development was overseen by an Expert Advisory Group (EAG) made up of experts in the drug and alcohol and/ or education field. Approval for Stage 1 and Stage 2 of the study was obtained from the UNSW Human Research Ethics Committee (HREC12548).

Formative consultation
In late 2013, focus groups were conducted with students, parents and teachers to inform content development for the online portal. Participants reviewed material providing facts about various drugs and their effects, prevention and harm minimisation strategies, and provided feedback on the language, relevance, appropriateness and usefulness of the materials (see Stapinski et al., Under Review). In January 2014, interviews were conducted with 14 teachers to inform the initial design and development of the portal, including content needs, website interface, navigation and features.

Development
Development of the Positive Choices portal was structured with the aim of compiling evidence-based alcohol and other drug information and resources for three key user groups: teachers, parents and students (See Figure 1). Information fact pages and webinars relating to different drug types and prevention related topics were developed by topic-area experts with reference to the relevant research literature. Additional drug education and prevention resources were identified from recent systematic reviews (Champion et al., 2013;Teesson et al., 2012), and keyword searches on academic databases and general search engines. Target resources for students included videos, games, and apps, and for teachers, lesson plans and full curriculum packages spanning multiple lessons. Identified resources were reviewed against specific criteria identified in consultation with the EAG to ensure only resources of high relevance and quality were included in the resource database. General resources (i.e. videos, games, apps) were eligible for inclusion provided they: i) had an alcohol or other drug prevention focus, ii) were relevant to Australian context, and iii) were informed by evidence. For curriculum programs, an additional inclusion criterion was that the program had been tested in schools, with benefits demonstrated in at least one study published in a peer review journal 1 . For each resource listed in the database, an access link is provided, along with a description of the resource, information about the developers, implementation considerations including cost, benefits, and strength of evidence supporting the resource, including links to relevant supporting publications where applicable. In the case of lesson plans and curriculum programs, information and links are provided to guide alignment with the Australian National curriculum (version 7.5): Health and Physical Education (Australian Curriculum Assessment and Reporting Authority, 2015). Identification and review of resources was completed in September 2014. Thereafter, resource scoping, review and updating has been conducted annually to ensure the content remains up-to-date. Web development, design and optimisation were conducted in collaboration with Netfront (www.netfront.com.au).

Beta-testing and review
A beta-version of the portal was reviewed by the EAG, who provided feedback and suggestions for improvement. Beta-testing of the site was conducted with 20 teachers and 10 parents who completed a series of tasks, followed by a comprehensive online survey and phone interview. Two Sydney-based teachers also completed in-house beta-testing, which involved observation of structured and unstructured browsing. Responses to quantitative feedback items were examined by calculating summary statistics, and a thematic analysis was applied to qualitative data.

Formative consultation
Consultation with 14 teachers based in New South Wales (NSW; 64%), Victoria (VIC; 14%), Tasmania (TAS; 7%), the Australian Capital Territory (ACT; 7%), or Western Australia (WA; 7%), indicated that a central access point for drug information and resources would be valuable. In particular, they noted the value of providing up-to-date information and facts, and access to a range of educational resources that could be searched and filtered according to drug type, Year level, resource type, and Australian national curriculum requirements (Australian Curriculum Assessment and Reporting Authority, 2015). To guide subsequent development, teachers provided feedback about their information needs, common questions, and how the portal could be best organised to facilitate evidence-based lesson planning.

Beta-testing and review
Review of the beta-version of the Positive Choices portal by the EAG generated 72 modification suggestions, each of which was addressed in the final version. Suggested modifications included improvements to language and navigability, the provision of additional "help" information for users, and video tutorials to demonstrate use of the portal functions. The site was also reviewed and endorsed by Principals Australia Institute (www.pai.edu.au), the leading national representative body for school leaders.
The beta-version of the portal was also reviewed by 20 teachers (80% female) and 10 parents (90% female) from NSW (60%), VIC (23%), WA (10%), and the ACT (7%). While most teachers were secondary school teachers (85%), teachers from Foundation Year through to Year 12 were represented. Parents had children in secondary school (10%), primary school (30%), or both (60%). Overall, the feedback from participants was overwhelmingly positive. The majority of respondents reported they found the portal useful (93%) and would recommend it to a friend (93%). Specific feedback indicated most respondents liked the layout (87%) and graphics (77%), found the site easy to navigate (90%), and found the information well-written and easy to understand (95%). When asked to complete a series of tasks involving refining resources by several criteria (i.e. resource type, Year level), the majority of participants indicated they were able to do so with ease (82%). Participants also provided qualitative feedback (see Table 1 for common themes). Several points of revision were made in line with participant or expert feedback, these revisions are detailed elsewhere (Stapinski et al., 2015).

Feedback on content
"I do kind of crave a 'feature content' section or something on the front page. A window that could update or change occasionally that might provide stimulus to deeper investigation. Providing accessible quickly understood facts or questions, a short video clip from the content, a hook that draws the site visitor in" -Teacher #20

Method
The Positive Choices portal was launched in December 2015, and was announced by Prime Minister Malcolm Turnbull and Senator Fiona Nash, as part of the Australian Government's drug education and prevention strategy. A post-launch evaluation was conducted, guided by the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) model, which provides a framework for evaluation based on these five domains indicating the impact and outcomes of health promotion initiatives (Glasgow, Vogt, & Boles, 1999). Two complementary strategies (described below) were used to examine outcomes in line with this framework.

Site use analytics
Google analytics is a free web analytics service that provides information to site administrators about how people are using a particular website (Crutzen, Roosjen, & Poelman, 2012). Information such as site traffic, acquisitions, demographics, and behaviour provides an objective measure of user activity that can complement other assessment strategies.

Evaluation survey
A survey was conducted between August and October 2016 to clarify current teaching practices and attitudes, and examine use and impact of Positive Choices in the first eight months post-launch of the site. The RE-AIM framework (Glasgow et al., 1999)/ guided the design of the survey questions. Participants involved in drug education were recruited via the Positive Choices mailing list, and paid advertising on social media. All participants were asked about their teaching practices, attitudes and barriers to drug education. They also completed questions adapted from the Nursing Educators' Evidence-Based Practice Questionnaire (Hussein & Hussein, 2013) to assess attitudes towards evidence-based teaching. Respondents who reported they had used the Positive Choices portal (41%) were asked additional questions about their use of the portal. Respondents who currently taught drug education, were asked specific questions about how they selected resources for the classroom, before and after they encountered the Positive Choices portal (where relevant). All quantitative data was analysed using the SPSS software package. Summary statistics were obtained to examine quantitative feedback data, and chi-square tests were applied to examine differences in practices between teachers who had, versus had not used Positive Choices. A thematic analysis was performed to explore qualitative data.

Site use analytics
As at 21 st November 2016, Positive Choices had attracted 51,340 website visits and 327,041 page views. Of the 37,707 unique visitors, 26.5% were returning visitors. The average duration of these visits was four minutes, 33 seconds. Site visitors were predominantly located in Australia (70.2% of total visitors), but the site was also popular in the United States (13.5%), the United Kingdom (3.3%), and Russia (3.1%).

Evaluation survey
The evaluation survey was completed by a total of 82 participants who were involved in drug and alcohol prevention and/or education in Australia. There were two participant sub-groups, distinguished by whether they had (n = 34) versus had not (n = 48) used the Positive Choices website on at one least occasion. Thirteen Positive Choices users and five teachers from the general sample were not currently teaching drug education at their school and thus answered only a subset of the questions. The age range of the sample was 18 to 65, with the majority aged between 31 and 45. They were 67% female, and were based in NSW (36%), QLD (30%), the ACT (6%), the Northern Territory (NT; 1%), South Australia (SA; 8%) Tasmania (TAS; 1%) VIC (9%) and WA (8%). The majority of respondents were based at schools in metro areas (51.1%), with some based in regional (31.8%), rural (13.6%), and remote (3.4%) areas. Respondents taught a range of subjects, most commonly Personal Development Health and Physical Education (38.6%), Social Science (27.3%), and English (23.9%).
Teaching practices, attitudes and needs. Among all respondents (n = 82), responses reflected a strong preference for evidence-based teaching approaches (See Figure 2). However, nearly one-third reported that their workload is too high to keep up with new evidence. It was common for teachers to go online to find information and resources for the classroom, with the majority reporting they did so weekly (22%) or more than weekly (65%).The most common sources for drug and alcohol information were government websites (69.9%) and google search (66%). When asked about drug education approach, less than one in four respondents had implemented a tested drug prevention approach with evidence of effectiveness. The most common approaches were inviting a guest speaker from the drug and alcohol field, or implementing programs that do not have a strong evidence base (see Figure 3). When implementing programs, only a minority of teachers followed implementation guidelines exactly (4.4%), with 26.8% reporting they did some reworking, 20.2% reporting lots of reworking, and 36.6% adapted the guidelines freely to suit their needs. Qualitative information was provided about the challenges in providing drug and alcohol education (see Figure 4). Typical barriers for teachers were lack of confidence, support, and time.  Use and impact of Positive Choices. Among participants who reported they had used the Positive Choices portal (n=34), the majority used the site at least once per term to find information or resources (Once per term: 34.8%; Monthly 30.4%; Weekly: 8.7%). Feedback from site users indicated the vast majority found the site useful, and felt more confident discussing alcohol and other drug use with young people since using the site. The majority reported their students were more engaged since using the site, considered it likely they would continue using the site, and would recommended Positive Choices to others (see Figure 5). The most accessed resources on Positive Choices were the factsheets (access by 87.0% of teachers), games (56.6%), videos (47.8%), and brief class activities (47.8%).

Figure 5. Feedback among teachers who has used the Positive Choices portal.
To examine the association between Positive Choices users and evidence-based practice, Positive Choices users were compared to the general teacher sample on two items assessing whether, when selecting resources, they considered the evidence-base and whether the resources had been tested in schools. Results are presented in Figure 6, and indicated Positive Choices users were more likely to consider whether the resource had proven benefits, although this difference was not statistical significant (t(31) = 1.20, p = 0.24).

Discussion
The Positive Choices online portal provides access to reliable, up-to-date alcohol and other drug information and prevention resources. The site was developed using a co-development model, with formative consultation conducted initially with teachers, parents and students to guide content and site development. Following from this consultation, information fact pages and webinars relating to different drug types (including but not limited to alcohol, tobacco, cannabis, methamphetamine, ecstasy, hallucinogens, inhalants, emerging psychoactive drugs) and prevention related topics (e.g. how to respond in an emergency, effective communication) were developed for the website by topic-area experts with reference to the recent research literature. Additional drug education and prevention resources (including 13 videos, 17 games, four apps, and 19 curriculum programs) were identified through review of the research literature and drug prevention sites. Only resources meeting specific criteria for relevance, quality, and evidence-basis were listed in the resource database. A key objective of Positive Choices is to facilitate implementation of evidence-based drug education in Australia, by providing access to a range of curriculum packages that have been tested in schools and demonstrated to have Note. 16.4% of respondents reported "unsure".
Note. 12% of respondents reported "unsure". a positive impact. For each resource, an access link is provided, along with information about the developers, implementation considerations, benefits, and strength of evidence supporting the resource, including links to relevant supporting publications where applicable. Userspecific links guide teachers, students and their parents to appropriate and relevant content, and search filters assist users to refine information according to their needs. A beta-version of the site was reviewed by teachers, parents, and drug and alcohol and/or education experts who provided feedback to guide improvements to the site. Feedback from beta-testing was overwhelming positive, indicating the site was useful, easy to navigate, and the information included in the portal was well-written and easy to understand.
Positive Choices was launched in December 2015, and an evaluation between August-October 2016 provided insights into teaching practices, barriers to implementation of evidence-based resources, and the use and impact of Positive Choices in the first eight months post-launch.
Several key findings emerged. Among teachers, using the Internet to access drug information and resources was typical, and responses reflected a strong preference for evidence-based teaching approaches. Despite this, evidence-based programs were implemented by less than one in four teachers, fidelity to implementation guidelines was low, and the most common drug education approach was to invite a guest speaker on the topic of alcohol and other drugs. The most common barriers teachers faced when implementing drug education lessons were lack of confidence, support, and time. Among teachers who had used the Positive Choices online portal, the majority accessed the site at least once per term, reported they would continue using the site, would recommend the site to others, felt more comfortable discussing alcohol and other drug use, and felt their students were more engaged with drug education since using the site. Findings also suggested that Positive Choices use was associated with evidencebased practice. Teachers who had used Positive Choices were more likely when selecting an educational resource to consider whether the resource had demonstrated benefits compared to the general sample.
Previous research has highlighted that implementation of evidence-based drug and alcohol prevention is low (Cuijpers, 2003;Dusenbury & Falco, 1995;Ennett et al., 2003), and this is in line with our findings, where the majority of teachers implemented approaches with limited evidence, and reported a tendency to adapt resources freely to suit their needs. This research highlights the challenges when translating evidence-based approaches to practice, whereby rigid fidelity to the intended program implementation can result in a lack of adherence, yet incomplete implementation may not yield the benefits that have been demonstrated in studies (Cook & Odom, 2013). It is important to acknowledge that the feasibility of implementing evidence-based approaches is moderated by factors such as classroom context and characteristics. Therefore, allowing professionals additional flexibility in resource selection and the opportunity for teachers to plan lessons tailored for particular classroom needs with a range of appropriate evidence-based resources is key. Our study highlights that lack of support and time are primary barriers to effective drug education, a finding that is consistent with a recent survey of secondary school principals across Australia (Australian National Council on Drugs, 2013). Positive Choices responds to the call articulated in this report for additional resources to support schools in building capacity to respond to drug and alcohol issues.
The current findings should be considered in the context of a number of limitations. First, betatesting and the evaluation survey were conducted within a small sample, who were recruited via convenience sampling and therefore may not be representative of Australian teachers and parents more generally. Secondly, in order to evaluate the impact of Positive Choices, the extent to which Positive Choices users considered evidence-base was compared to a general sample of teachers. Positive Choices users were more likely to consider evidence-base, although this finding was not statistically significant, and we note the study is likely underpowered due to small sample size. Furthermore, our research design does not allow us to infer causation as it may be that evidence-aware teachers are more likely to seek out a website like Positive Choices. To assess the impact of Positive Choices we also asked teachers to reflect on their teaching practices before and after using Positive Choices. Again, this methodology is not ideal as it relies on retrospection, which can introduce biases. These limitations are common to studies of research translation, reflecting the challenges in evaluating large scale efforts to roll out evidence-based approaches (Sax Institute, 2016). These limitations notwithstanding, this study contributes significantly to the education and drug prevention field. It provides a free and easily accessible resource database that allows teachers, parents and students to access alcohol and other drug information and prevention resources that have been reviewed to ensure quality and relevance. The Positive Choices development and evaluation process demonstrates a number of strengths, including the use of a co-development model, whereby end users were consulted during the formative design phase, development phase, beta-testing phase, and post-launch to determine the impact of the site on key outcomes. Development of Positive Choices was guided by the effective principles and components identified through recent systematic review (Lee et al., 2016). Our evaluation followed an established health promotion evaluation framework (the RE-AIM model), and drew on multiple sources of information including user self-report and objective use data from google analytics.

Conclusion and implications
Positive Choices is a drug and alcohol prevention portal (positivechoices.org.au) that can be freely accessed by teachers, parents and students via the Internet. The portal provides access to over 114 drug prevention resources, all of which have been developed or reviewed by drug and/or education experts, and meet the Positive Choices inclusion criteria for relevance, quality and evidence-based. Although developed with the goal of increasing implementation of evidence-based drug education in Australian schools, the portal appeals to a broader audience and has already been accessed by over 11,000 users internationally, with a total reach of over 37,000 users to date. Beta-testing and post-launch evaluation with teachers and parents indicate that the site is highly useful, and has positively impacted on evidence-based practice in schools.