Best Practices in School-Based Mentoring Programs for Adolescents

Abstract

Mentoring is a promising approach to promote healthy development and prevent poor youth outcomes; however, bad mentoring experiences have yielded negative results. Thus, it is critical that youth service providers take heed of the current research and resources for establishing an effective youth mentoring program. Given the increased interest in school-based mentoring programs, presented herein are best practices in developing, implementing, and evaluating a school-based mentoring program for adolescents. Program development strategies are provided around laying the groundwork, establishing partnerships, and coordinating the program. The elements of effective practice for mentoring (MENTOR, 2009) are summarized and additional implementation considerations are presented. Lastly, the realities of program evaluation are discussed in juxtaposition to program evaluation standards.

KEYWORDS:

• adolescents
• intervention
• prevention
• program development
• program evaluation
• program implementation
• school-based
• youth mentoring

According to the National Mentoring Partnership, mentoring is defined as a “structured and trusting relationship that brings young people together with caring individuals who offer guidance, support, and encouragement aimed at developing the competence and character of the mentee” (MENTOR, 2010, April 16). Traditionally, mentoring involves matching an adult mentor with an individual youth or mentee, but today there are various types of mentoring (e.g., group mentoring, e-mentoring, and peer mentoring) and kinds of programs (e.g., community-based, school-based, etc.; Weinberger, 2005). Additionally, multiple theoretical models related to youth mentoring have evolved over time, ranging from those suggesting a strong and meaningful relationship is the mechanism for yielding positive youth outcomes (Rhodes, 2005) to those models suggesting a need for greater structure and focus on instrumental goals in order to find positive results (Larose & Tarabulsy, 2005). To date, there is little evidence to suggest which approach is better or whether a combination of the two might benefit youth (Darling, 2005). The focus of this article is on developing, implementing, and evaluating a mentoring program where one-to-one mentoring is provided by an adult mentor to an adolescent in a school setting.

THE PROMISES OF MENTORING

A positive and enduring mentoring relationship can impact the self-worth, self-esteem, and social competence of youth according to Dubois, Holloway, Valentine, and Cooper (2002) and Rhodes, Spencer, Keller, and Liang (2006). Additionally, quality mentoring can increase school engagement and connectedness, motivation, attendance, and academic competencies (Klem & Connell, 2004; Larose & Tarabulsy, 2005; Rhodes et al., 2006). Students with greater school connectedness are less likely to engage in problem behaviors at school (e.g., fighting, bullying, and truancy) and are less likely to drop out of school, which means they are more likely to actualize their academic potential (Portwood & Ayers, 2005). Also, if mentoring is provided after school, it can offer another layer of protection. For instance, mentees are engaging in a supervised, productive, and positive activity during a typically unstructured and unsupervised period of free time after school that often presents a prime opportunity for adolescents to engage in risky behaviors (e.g., substance abuse, sexual experimentation, delinquency; Rhodes & Spencer, 2005).

Beyond giving adolescents a positive social outlet, there are certain program-related factors that predict positive outcomes for mentored youth. For example, a strong mentoring relationship, adequate intensity of mentoring (frequency/duration), provision of structured activities for mentors and mentees, similarity of interests between mentor and mentee, screening for appropriateness of participation, pre-match orientation and training, post-match training and ongoing support, monitoring of program implementation, and parental support and involvement are all practices or program components that predict positive results (Portwood & Ayers, 2005; Rhodes & Spencer, 2005).

SCHOOL-BASED MENTORING

While mentoring has the potential to promote student success and healthy development, school-based programs have advantages and disadvantages to be considered prior to moving forward with implementing a program. Advantages include minimal demands on mentors relative to traditional community-based mentoring, heightened supervision and thus fewer safety concerns, easy access for students, and potentially heightened cost-effectiveness (Portwood & Ayers, 2005). Furthermore, mentors in school-based programs typically make less of a time commitment, frequently are provided with greater structure and supervision, and often benefit from mentoring alongside other mentors who can share ideas and strategies. As for being potentially more cost-effective, the overhead for school-based programs may be fiscally supported, in part, by the school, such as use of equipment, space, and supplies (Portwood & Ayers, 2005). Additionally, school personnel are generally willing to offer their time and support for school-based interventions, whether by assisting with training, orientation, case management, or monitoring of student progress.

The primary disadvantage of school-based mentoring programs is the lower intensity of mentoring provided relative to what is typically provided in community-based programs (Portwood & Ayers, 2005). Moreover, due to time constraints at the school (e.g., school calendar), mentor–mentee matches typically meet much less frequently and for a shorter duration as compared to community-based matches, which translates to less opportunity to form a necessary bond. One example of a strategy used to facilitate relationship building is the use of icebreakers and dialogue journals, where mentors share their responses to written prompts with their mentees to facilitate conversation (King, Vidourek, Davis, & McClellan, 2002). According to Liang and Rhodes (2007), not only adequate intensity but a strong mentoring relationship is vital to achieve positive outcomes from mentoring. When it comes to intensity, research suggests a minimum of six months of regular mentoring to find some positive effects (Grossman & Rhodes, 2002; Karcher, 2005). However, twelve months or more of mentoring is recommended, since matches with established strong mentoring relationships that meet more frequently over a longer period of time find greater benefits (Grossman & Rhodes, 2002). Program developers are advised to carefully consider a number of other factors related to the program and its participants, beyond how long the program will last and how frequently matches will meet. Additional considerations with regard to program development, implementation, and evaluation are presented herein.

PROGRAM DEVELOPMENT

Designing an effective and worthwhile mentoring program is quite a complex process. Youth mentoring programs require extensive planning in order to develop a program that will make a positive impact (see Table 1 for suggested timeline). Furthermore, systematic and thoughtful planning leads to heightened intervention fidelity, such that the intervention is implemented consistently and as intended, which yields better outcomes and ensures that negative consequences are avoided (Weinberger, 2005). Moreover, Weinberger argues that good intentions are insufficient given that research points to negative consequences resulting from poorly run programs. Thus, the idea that something is better than nothing does not apply if programs are to avoid doing harm.

TABLE 1 Program Development Timeline

Year 1:	Research and planning
	• Explore all existing resources/tools• Investigate potential funding sources; write grant*• Identify and engage all key players and partners
Year 2:	Develop program
	• Involve all key players• Assign roles/responsibilities• Develop program outline and budget*• Create policies and procedures• Determine target populations, type of mentoring, mission/goals• Design program evaluation• Hire coordinator(s)
Year 3:	Implement program
	• Recruit, screen, orient, and train participants• Collect data (minimally process data)• Document procedures and participant characteristics/attendance• Support matches; monitor progress• Refine program as needed

So how do program implementers get it right? The good news is that a wealth of information and readily available resources exist to guide mentoring program development. For instance, the National Mentoring Partnership (www.mentoring.org) has published a toolkit that encapsulates best practices or elements of effective practice in youth mentoring (MENTOR, 2005). Additionally, the most recent edition of Elements of Effective Practice for Mentoring (MENTOR, 2009), also available online, clearly stipulates standards and benchmarks for program implementation, including a useful checklist to guide program development.

Laying the Groundwork

Certain prerequisites provide the basis for developing an effective mentoring program: using data to inform decisions, ensuring program–environment fit, and establishing a clear mission and goals that align with measurable outcomes (Godber, 2008; Portwood & Ayers, 2005; Weinberger, 2005). Once a strong foundation based on these key elements is in place, designing the actual program (e.g., structure, logistics, personnel, policies, and data tracking procedures) can commence. An initial task that informs planning is to gather data by conducting a needs assessment. Needs assessments help with identifying potential gaps in existing supports, which youth to target for mentoring, and priorities for the program and/or how to best meet the needs identified (Weinberger, 2005). It is critical to collect data from and consider all stakeholders' voices (e.g., students, parents, faculty/staff, and administration) at the planning stage (Portwood & Ayers, 2005). Conducting focus groups or interviews and administering surveys are effective means for gathering data (Weinberger, 2005). Input and buy-in from each stakeholder will ensure that the program operates smoothly and is adequately supported. For instance, students can provide insight into their specific needs and preferences, while parents can help identify ways to garner parental involvement. School counselors have a clear sense of scheduling constraints and unmet needs and have ready access to students. Lastly, administrators can propose which campus or district resources are available to the program, as well as which facilities, equipment, and/or personnel can support the program.

Beyond an initial needs assessment, forming an advisory council with school staff, parents, students, and community members is also recommended (Weinberger, 2005). An advisory council enables the formation of in-house community liaisons who can help with various program development activities (e.g., obtaining funds, increasing visibility on campus and in the greater community, strengthening family involvement, and increasing potential mentee participation). Inviting input from key stakeholders is crucial to fully understand the context so as to make certain that the selected site is appropriate for the program. Ensuring that there is a good program–environment fit involves preparing the school for the program, which will aid in successfully implementing and sustaining the program (Godber, 2008). Furthermore, the success of the program is contingent on the school's support, as is the success of a program evaluation (Portwood & Ayers, 2005).

The needs assessment and advisory board can contribute to establishing a focus for the mentoring program that will inform the mission of and goals for the program, as well as the type and structure of the program (Weinberger, 2005). The program is ultimately built around the mission and goals, which help to formulate predetermined measurable outcomes—the changes/gains you want to see as a result of the intervention (Weinberger, 2005). The National Mentoring Partnership provides useful tools to aid the development of a mission statement, goals, and objectives. Keep in mind that designing a program and designing a program evaluation are reciprocal processes, each informing the other. Decisions about how to measure effectiveness or impact need to occur simultaneous to program development. For example, if the program aims to improve student achievement levels, the program will need to decide which sources of data (i.e., grades, test scores, self-report, teacher-report, etc.) will best measure the intended outcomes and is most feasible. Once the focus and aim of the program are well established, the groundwork has been laid for designing the actual program.

Before even beginning to develop a program, it is crucial to invest time in researching and planning for development and, as with any new program, it is necessary to start small. Again, there are seemingly endless resources and tools that exist and are readily available to program developers. ¹ One of the first steps is to devise a budget in order to explore avenues for funding and to allow adequate time for grant writing. While program funding is beyond the scope of this article, program developers are encouraged to consult the National Mentoring Center's guide for Sustainability Planning and Resource Development (Bowman et al., 2008). Next, key players and partners will need to be identified and engaged prior to development. Communicating explicit roles and responsibilities early on will help with determining which key players are able to make the necessary commitment. Aside from getting school-related personnel and stakeholders committed and involved, there are numerous community agencies and businesses which can provide support in terms of personnel, mentors, fundraising, grant writing, in-kind donations, and marketing. Some businesses are willing to develop workplace mentoring programs whereby employees serve as mentors; however, typically this type of program requires the mentees to be mentored on the site of the business, which can be prohibitive due to transportation issues (i.e., cost and time) and means that the program is no longer a school-based mentoring program.

University Partnerships

The Mentoring Resource Center (USDOE, 2007) suggests partnering with universities, which are “resource-rich environments” (p. 2) that can provide a wide-range of support/resources to school-based mentoring programs, depending on the needs of the program. It is a matter of program developers having clearly identified needs and then making connections with the right people on a university campus—individuals, programs, or organizations—who will find the partnership mutually beneficial, which is key to a successful partnership (U.S. Department of Education, 2007).

Partnering universities can provide access to students (or faculty/staff) who can serve as volunteer mentors. Whether students are looking to give back to their community or seeking opportunities for professional growth (e.g., students studying education, social work, or psychology), there are motivated individuals who are eager to make a difference in the life of a child, but may not be aware of existing programs in the greater community (local schools) and simply need to be presented with the prospect of mentoring. Additionally, undergraduate or graduate students may be expected to complete community service hours as part of their program of study or coursework or they may be involved in student organizations that expect members to be involved in community service. Note, undergraduate students in particular tend to be a transient group of volunteers (Darling, 2005), but unmarried volunteers ages 26–30 are least likely to prematurely terminate their commitment to mentoring (Grossman & Rhodes, 2002).

Beyond academic programs and student organizations, other student affairs departments on campus that may be good entry points are career development centers, alumni associations, and campus ministries. Also, faculty within certain departments on campus may serve as an advisory board member or consultant to lend their expertise related to developing, implementing, and evaluating prevention programs. Faculty also might be willing to provide professional development or create community-based learning experiences for their students. For example, this author, who oversees a school-based mentoring program at a local high school, teaches within a school counseling program for which candidates are required to complete 20 pre-practicum hours in a school setting as part of their training. These candidates are not only motivated to gain meaningful professional experience but, also, they are provided with an additional incentive to complete a program requirement by volunteering as a mentor. This is a good example of mutually beneficial collaboration. Beyond committed mentors, there are a number of key players who will play a role in program development, with a central role being that of program coordinator.

Program Coordination

Once all key players are on board, roles and responsibilities can be delineated, an outline of the program can be created, and a program coordinator can be hired. Funding the program coordinator's position is generally the greatest cost for school-based mentoring programs, but a necessary cost in order to develop and implement an effective program. The Elements of Effective Practice Toolkit (MENTOR, 2005) recommends a full-time coordinator for a program that has approximately 25 matches; a part-time coordinator is only sufficient for a significantly lighter caseload (far less than 25 matches).

Coordinating a mentoring program which serves adolescents is recognized as presenting more challenges than a program for younger children who tend to be more receptive (or less resistant) to mentoring for a variety of reasons (Darling, 2005; Portwood & Ayers, 2005). One reason may be that adolescents tend to be more vulnerable due to inconsistent or difficult prior relationships, which may contribute to a tendency to approach mentoring tentatively and be perceived as indifferent or resistant (Grossman & Rhodes, 2002; Rhodes & Spencer, 2005). This perceived resistance may lead to premature termination with the potential for negative outcomes (e.g., lower self-worth; Grossman & Rhodes, 2002). Research shows that adolescents are 65% more likely to terminate a mentoring relationship compared to 10- to 12-year-olds (Grossman & Rhodes, 2002; Weinberger, 2005). Therefore, mentoring programs serving adolescents require greater resources to support increased case management, involvement, and oversight by the program coordinator in order to retain satisfied and engaged participants for the program's duration (Grossman & Rhodes, 2002; Weinberger, 2005).

An effective program coordinator will build and sustain relationships in order to retain participants (mentors and mentees) for the duration of the program and facilitate positive mentoring experiences. It is critical for the program coordinator to provide a consistent presence so that all stakeholders feel supported. The program coordinator also is responsible for managing the overall operations of the program, including event/activity planning; recruiting, screening, and matching participants; training and orienting all mentors and mentees; resolving any problems/concerns; and maintaining all records (see Toolkit for a position description). An initial task for program coordinators is to devise an infrastructure, policies, procedures, and materials to support program operations. Devising data-tracking systems in advance will help to monitor participation, including contact information for matches, applications, attendance, screening, training, and all correspondence and documentation related to case management. At the most basic level, sign-in sheets and/or mentor logs can help with monitoring attendance. A spreadsheet for tracking expenses will enable the program to stay within budget, and monthly newsletters also may help to document correspondence and events. Additionally, training curriculum, agenda, and materials will need to be created, as will marketing material to recruit potential mentors and mentees. Again, there is no need to reinvent the wheel; but rather, consult the various existing resources and tools which can be readily adapted to meet the needs of specific programs.

After expending the time and effort necessary to plan for and develop a school-based mentoring program, stakeholders often are eager to roll out the program, and because school-based programs are typically constrained by the school year, the timing of all components can present challenges. One cautionary note: be certain to balance enthusiasm for beginning the program with recognition of any constraints and/or capacity to move forward with implementation. All resources (i.e., budget, personnel, space, materials, etc.) need to be in place before starting the program which means, in some instances, launching the program may need to be postponed to the following school year. Recall that programs should avoid operating (mentoring youth) for less than six months, and school calendars will pose limitations (e.g., holidays/breaks, early dismissals, and testing schedules). Successful programs work closely with key players, such as school counselors and administrators, who can ensure the program logistics are practical and fit within the framework of the school. Once program developers and coordinator(s) are certain that all essential elements are in place, the program is ready for implementation.

PROGRAM IMPLEMENTATION

Programs which are deemed most effective are those that are thoughtfully planned, implement best practices in youth mentoring, and are grounded in theory (Darling, 2005). As with any intervention, attention needs to be paid to implementing the program with integrity, which means monitoring and documenting implementation procedures and processes along the way in order to ensure the program is being implemented as intended. To assist with ensuring fidelity, the Elements of Effective Practice for Mentoring (MENTOR, 2009) offers evidence-based standards and benchmarks, which if drive program implementation, will ensure that most matches are sustained for the duration of the program and actually receive the intended benefits. The six standards postulated as necessary to implement high quality mentoring programs that result in successful mentoring and positive outcomes are summarized below.

The first standard is related to recruitment. While it is important to communicate the benefits derived from mentoring for prospective participants, it is necessary that a realistic description of the program aims, expected outcomes, and potential challenges is provided to potential mentors and mentees, so as to avoid unfulfilled expectations or disillusionment on the part of participants (MENTOR, 2009). A lack of clarity and direction at the outset of the program or relationship might lead to premature termination (MENTOR, 2009) and consequently harmful effects (Grossman & Rhodes, 2002). Programs will want to utilize a variety of outreach approaches or multimedia to successfully recruit both mentors and mentees (e.g., letters, flyers, phone calls, emails, news releases, and Web sites). Strategies may vary from program to program depending on whom the program is targeting for participation and the resources available to the program. Due to the inevitability of participant attrition at various stages in the process, it is recommended that programs develop a waitlist for both mentors and mentees in order to maintain a pool of participants and sustain matches. Mandatory information sessions for prospective mentors and mentees is one strategy that can serve a dual purpose of clarifying expectations while screening out individuals whose expectations or other commitments conflict with the program's framework and policies (Shepard, 2009).

The second standard has to do with screening, a costly necessity that will limit the pool of prospective mentors and mentees (Stukas & Tanti, 2005). Screening prospective mentors and mentees helps to ascertain appropriateness for the program and level of commitment, as well as provide safety checks (e.g., parental consent, applications, interviews, reference/background checks, and fingerprinting; MENTOR, 2009). Simply being motivated to volunteer does not qualify as sufficient criteria for potential mentors; it is important to select suitable mentors “who are likely to establish and maintain effective relationships with youth” (Stukas & Tanti, 2005, p. 245). Stukas and Tanti (2005) indicate that suitability is determined based on program goals and mentee characteristics. In this author's experience, adolescents typically do not self-select mentoring; their parents or teacher(s) want them to participate. The source of and reason for referral will need to be addressed during screening in order to determine whether the student demonstrates a willingness to actively partake in the program. Additionally, programs will want to determine whether the potential participants' availability for the duration of the program is in line with program expectations and will not change during the period of commitment in order to minimize attrition and maximize impact. For instance, class schedules, extracurricular activities, and/or employment will present scheduling challenges for both mentors and adolescent mentees and will need to be explored prior to participation to determine program-participant fit. Lastly, given that research suggests that mentoring works best for youth who are merely at-risk as opposed to those manifesting problems (Darling, 2005; Rhodes & Lowe, 2008; Rhodes & Spencer, 2005), students who are exhibiting significant social or emotional problems should be screened out and referred for more appropriate services.

The third standard pertains to training with implications for the sustainability of a quality mentoring relationship, which is necessary to positively impact those involved. It is recommended that minimally two hours of pre-match training is provided to mentors to not only clarify expectations but, also, to foster skills for building effective mentoring relationships (MENTOR, 2009). According to Rhodes and Spencer (2005), mentors who received less than two hours of training reported the lowest satisfaction and poorest mentoring relationships. Suggested training topics include: adolescent development, communication techniques, cross-cultural mentoring, boundaries and role clarification, anticipated challenges and realistic expectations, suspected abuse/neglect, confidentiality, mentoring strategies, policies and procedures, and mission and goals for the program (MENTOR, 2009). This author's program found it challenging to fit pre-match training content into the four hours designated. In addition to the suggested topics, training calls for focusing on the needs of the particular population served (e.g., foster youth, recent immigrants, or children of incarcerated parents, etc.; MENTOR, 2009). Additionally, this standard encapsulates orienting mentees and their parents or legal guardians regarding guidelines, roles, and responsibilities (MENTOR, 2009). Post-match training also is recommended and is discussed under the fifth standard pertaining to monitoring and support (as follows).

The fourth standard is matching. Mentoring relationships are more likely to endure when mentors and mentees are matched along certain dimensions (e.g., mutual interests, relationship styles, and personality characteristics; MENTOR, 2009). However, there is no research to support matching mentors and mentees based on demographic variables, such as gender and race (Darling, 2005; MENTOR, 2009). Thorough screening procedures (i.e., applications and interviews) will aid in the complex, but important, matching process. It is essential to recognize that both mentees and mentors have their own set of motives and needs, which contributes to the complexity of matching (Stukas & Tanti, 2005).

Programs are advised to encourage mentors and mentees to give relationship building adequate time to develop and not jump to the conclusion that a poor match has been made. Experiencing challenges early on is not predictive of the long-term quality of the mentoring relationship (Darling, 2005). However, any obstacles that a match faces need to be addressed as soon as possible so as to avoid disappointment with mentoring, which may result in premature termination and potential harm to the mentee. Furthermore, research has demonstrated significant declines in the self-worth and academic competence of mentees with prematurely terminated relationships as compared to nonmentored youth (Rhodes & Spencer, 2005). Thus, providing case management is a critical component of an effective program, wherein matches are closely monitored and provided with ongoing support.

The fifth standard speaks to the necessary monitoring and support that contributes to more satisfying and successful mentoring experiences, which translate into positive youth outcomes (MENTOR, 2009). Furthermore, it is recommended that ongoing guidance, support, and training be provided for the duration of the match. Inevitably, mentors develop concerns about the match or their mentee and want feedback on how to effectively reach or respond to their mentee. Mentors often encounter situations where they wish to consult with a staff member regarding how to proceed with problem solving and decision making, or they simply may be looking for ideas on how to spend time with their mentee. Scheduled sessions where mentors can debrief about their mentoring experiences help the program respond to any concerns and provide opportunities to validate the mentors' contributions. Campus Kids is one example of a mentoring program that conducts “monthly reflection meetings” for its mentors (Shepard, 2009, p. 41).

Programs are advised to conduct periodic assessments of each match's relationship and experiences with the mentoring program in order to ensure that all participants' needs are being met (MENTOR, 2009). If a mentor's or mentee's needs are not met or concerns not responded to in a timely manner, chances are that the mentor may begin to feel unappreciated or overwhelmed and the mentee may feel unsupported or disappointed, which, in either case, may lead to premature termination and negative consequences for the adolescent mentee (Grossman & Rhodes, 2002). Mentees, particularly at-risk adolescents, will not necessarily seek out support on their own but will require close monitoring in order to ascertain whether their needs are being met by the program; determine how best to accommodate their unique circumstances and goals and combat any existing resistance or perceived rejection. Building in time at regular intervals to ask mentees to reflect on their mentoring experiences is one strategy programs can utilize to monitor and support adolescent mentees.

While feeling supported by and satisfied with the program may contribute to participant retention (Stukas & Tanti, 2005), getting volunteer mentors to commit beyond one year is uncommon. Thus, closure, the sixth and final standard, is an inevitable and important part of the mentoring process. Depending on the circumstances, terminated mentoring relationships may result in the need to re-match mentors or mentees in the current or subsequent year. Effective programs establish policies and procedures for handling terminations in a consistent and sensitive manner. Training specific to closure will prepare all participants for both predictable and unpredictable terminations (MENTOR, 2009). Programs will want to build in closing activities and time for reflection in the weeks leading up to the end of the program and perhaps include some type of culminating project or experience in which mentors and mentees can participate to mark the end of the program and their relationship. Also, it is recommended that programs conduct exit interviews, affirm all stakeholders' contributions (i.e., recognize volunteers/contributors), and assess the experiences of all participants (MENTOR, 2009). An end of the year party to honor program participants provides an excellent opportunity for reflection, recognition, and evaluation.

Additional considerations for program implementers include family involvement and program visibility, both of which fall under the purview of the advisory council. As with any school-based prevention or intervention, family involvement is recommended for mentoring programs (King et al., 2002; Weinberger, 2005). In fact, Weinberger proposed the Families and Mentors Involved in Learning with Youth (FAMILY) model, which suggests holding three events per year in the evening or on weekends where refreshments, transportation, and childcare are provided. Other programs have utilized monthly newsletters to keep parents informed about the program and opportunities for parental involvement (Hanlon, Simon, O'Grady, Carswell, & Callaman, 2009). Another critical component of successful mentoring programs involves marketing or public/media relations, which may be an area in which it is necessary to partner with local agencies in order to get the word out about the program. The more visible the program, the easier it is to recruit potential mentors and partners and solicit needed funding. Clearly, there is an array of program components to be addressed by program implementers, and youth mentoring is a “complex and multifaceted phenomenon” (DuBois & Silverthorn, 2005, p. 44) requiring careful planning, implementation, and documentation. While the program is being implemented, measures for evaluating the program can be administered, whether the program has decided to conduct a formalized program evaluation or simply wishes to utilize process data to inform further program development and to ensure effectiveness.

PROGRAM EVALUATION

It has been well established that poor mentoring experiences or relationships lasting less than three months may lead to negative outcomes, perhaps due to vulnerability associated with experiences of rejection or loss (Grossman & Rhodes, 2002; Karcher, 2005). Thus, it is crucial for programs to demonstrate that they have “merit and worth” or that the mentoring program has achieved what it set out to accomplish (Godber, 2008). Having data that demonstrate the prevention/intervention implemented makes a positive impact on the youth served will inform programmatic decisions, foster the sustainability of the program, and aid in obtaining funding to support the program (Darling, 2005). In addition to looking closely at outcomes or impact, measuring progress and processes is needed for program refinement and development (Grossman, 2005). Practitioners will want to ask themselves early on if they are equipped to conduct a program evaluation. If not, the question becomes, who might serve as potential partners or who might the program consult to design a solid evaluation of the program?

It is important to recognize that applied research in and of itself is complicated (DuBois & Silverthorn, 2005). While program evaluation is difficult, there are good examples of recent program evaluations (see Table 2). Examining the effectiveness of mentoring programs is not something that can be taken lightly, nor should it be an afterthought. Furthermore, as mentioned previously, decisions about what will be measured and how to assess facets of the program and the impact on its participants will need to be made simultaneous to program development, since each process informs the other. Thus, it is advantageous for program coordinators to collaborate with program evaluators on an ongoing basis to establish the procedural consistency and integrity necessary to achieve the intended outcomes (King et al., 2002). A sound research design should inform program development and implementation practices and will contribute to upholding ethical responsibilities to do good and avoid harm (DuBois & Silverthorn, 2005). The ideal program evaluation is a longitudinal experiment with multiple informants and sources of data where data are collected from both the intervention group and a comparison group prior to and following the intervention (DuBois & Silverthorn, 2005).

TABLE 2 Program Evaluation Examples

Author(s)	Date
DuBois, Neville, Parra, & Pugh-Lilly	2002
Hanlon, Simon, O'Grady, Carswell, & Callaman	2009
Herrera et al.	2007
Karcher	2005
Karcher	2008
King, Vidourek, Davis, & McClellan	2002
Portwood, Ayers, Kinnison, Waris, & Wise	2005

Achieving this ideal study, one that is valid, reliable, and unbiased is no easy task and presents numerous challenges. Some hurdles related to conducting a program evaluation, aside from inadequate resources (e.g., time and money), involve achieving adequate sample sizes, obtaining parental consent, and administering surveys, as well as the mere complexity of unearthing outcomes resulting from an intervention. In order to achieve adequate statistical power, which is necessary to conduct a quantitative analysis of the measures implemented, sample sizes generally need to be above 64 participants, which breaks down to 32 individuals receiving the intervention and 32 like individuals who did not receive the intervention (a comparison/control group). Note that a program this size will require one to two full-time program coordinators. The inevitability of attrition also means that there is a need to overshoot the targeted numbers in order to ensure a sufficient sample size. Securing a comparison group typically requires either forming a waitlist (Jent & Niec, 2009) or offering an alternative intervention (Karcher, 2008), which both require advanced planning.

Obtaining parental consent can be difficult as well, given the time constraints of collecting data prior to the start of the program. One solution for the intervention group is to have consent for the research study embedded within the consent form for the program or to have parents provide their written consent for participation in the program evaluation at the program's orientation. However, there are always those families who are not in attendance and do not respond in a timely manner to solicitations for participation when sent home. Survey administration has upsides and downsides that need to be carefully considered prior to choosing this measurement strategy/tool. For instance, surveys with strong psychometric properties are considered acceptable behavioral measures which on face value appear easy to administer. On the contrary, administering surveys presents a number of obstacles such as limited administration time, misunderstood questions, incomplete answers, and/or multiple answers selected. Ideally, surveys are administered in person in order to maximize return rate and minimize some of these practicalities; though, this tactic is still not failsafe due to absences, schedule conflicts, and human error.

Data collection tribulations are one thing, interpreting results is yet another. Even if program evaluators have achieved a sizeable sample large enough to reach adequate statistical power to measure impact and achieve external validity, more sophisticated statistical analyses (e.g., multivariate approaches) are necessary to detect the subtle but important changes that mentoring may produce (DuBois & Silverthorn, 2005; Portwood & Ayers, 2005). Additionally, it is important to recognize that a lack of statistical significance may not mean that there was no effect; rather, some outcomes may simply be indirect or latent (not evident immediately) (DuBois & Silverthorn, 2005). Outcomes need to be examined in relation to various program-related factors such as intensity and fidelity of mentoring as well as other variables specific to the individual and match that might be contributing to the observed outcomes. For example, the strength of the mentoring relationship and specific mentoring behaviors (i.e., how the match spends their time together) are associated with the mentee's satisfaction with mentoring, which impacts related outcomes (Barrera & Bonds, 2005).

Parsing out which factors contribute to the findings of a program evaluation or outcome study is challenging, and program evaluation is complicated and requires advanced research designs and methodologies. Thus the very rigorous research studies touted as necessary are rather costly in terms of the time, effort, and resources needed to conduct such a study, which is why it is suggested that collaborative investigations be employed, rather than individual attempts to examine single programs at one time (DuBois & Silverthorn, 2005; Grossman, 2005). Should a program evaluation, where outcomes are examined, be deemed impractical; it is recommended that programs minimally collect data that will enable progress/process monitoring and program refinement (Grossman, 2005). The use of mentor activity logs is a good way to track both frequency of mentoring and how time is spent by matches during mentoring meetings (Karcher, 2008). Grossman (2005) further recommends tracking data related to program participant characteristics, intensity of mentoring, and mentoring relationship quality. Without this information, programs are unable to ascertain what is working or what needs to change in order to optimize anticipated gains for the youth served.

By utilizing data and available resources to inform your decisions, youth service providers are well positioned to implement an effective school-based mentoring program and realize the positive changes that mentoring promises. The research shows that mentees will be the beneficiaries of this careful planning, attention to implementation, and assessment of progress and outcomes. Youth service providers who are interested in mentoring seek to foster healthy youth development and adjustment. To achieve this aim, practitioners are advised to examine the existing literature and resources that are readily available. Having a thorough understanding of best practices or what are considered the elements of effective practice is a necessary prerequisite to designing an effective school-based mentoring program for adolescents.

Notes

Note. *A program budget will need to be developed in order to write a grant.

The Mentor Consulting Group, www.mentorconsultinggroup.com; MENTOR/National Mentoring Partnership, www.mentoring.org; and the National Mentoring Center, http://educationnorthwest.org/nmc.