Lessons from an Evaluation of a School-Based Teen Pregnancy Prevention Program in Texas

Evaluation of Re:MIX

Child Trends used a clustered randomized controlled design to evaluate the impact of Re:MIX on measures of knowledge, attitudes, self-efficacy, and intentions about sexual activity, contraceptive use, and healthy relationships. These measures were targeted by the Re:MIX curriculum.

For more information about the implementation of the Re:MIX peer education model, see the research brief.

Recruitment and enrollment

The evaluation study enrolled students in grades 8–10 in three Austin, TX charter schools across five semesters from 2016 to 2018. To participate in the study, students had to return parent consent and student assent forms. The project team worked with schools, teachers, parents, and students to fill out the required consent forms, with an 89 percent return rate. Overall:

• Sixty-six (66) percent of students enrolled in the study.
• Twenty-three (23) percent agreed to receive Re:MIX lessons but did not participate in the evaluation.
• The remaining 11 percent of students (or their parents) opted out of receiving the Re:MIX sexual education program and participating in the evaluation.

Fifty-seven (57) classes and 626 students were enrolled in the study, as shown in the figure below. Across all schools and semesters, 30 classes were randomly assigned to receive Re:MIX (347 students) and 27 classes were assigned to the control condition (279 students). The control classrooms could receive either a general health education curriculum or continue with their regularly scheduled classes (not related to sexual health). The evaluation had strong response rates: 95 percent of participants who took the baseline survey completed the post-intervention survey. The final analytic sample following post-test consisted of 327 treatment students and 267 control students.

Participant demographics

Participant characteristics at the time of enrollment in the study were similar for the intervention and comparison groups.

• Participants were in grades 8–10 and were, on average, 14 years old.
• Seventy-one students (71) percent were Latino.
• Forty-one (41) percent of students primarily spoke Spanish at home.
• Thirty-six (36) percent of participants had mothers with less than a high school education.
• Twenty-nine (29) percent of participants were children of a teen mother.

Design and methods

• Each semester, the evaluation team randomized classes within schools and grade levels. Classes were assigned to either the Re:MIX (treatment) intervention or to a control group.
• Evaluators confirmed that students in the treatment and control groups were equivalent in terms of demographics, sexual experience, and outcomes of interest at baseline.
• The evaluation team ran multivariate linear regression models to assess the impact of Re:MIX. For each outcome at post-test, models included the baseline value of the outcome and controlled for sexual experience at baseline, age, grade, gender, race/ethnicity, and school. The team incorporated cluster-robust standard errors to account for the fact that students were grouped in classrooms.

Participant attendance and engagement

• Participants in Re:MIX attended the majority of sessions. Ninety (90) percent of treatment students participated in at least 8 of the 10 Re:MIX sessions.
• Participants responded positively to the Re:MIX program. Most said that they learned something from Re:MIX (98 percent), that they would probably or definitely recommend Re:MIX to a friend (78 percent), and that they liked the peer educators and the health educators (90 percent and 89 percent, respectively).

Evaluation Successes

• The evaluation team enrolled a large sample of students and classes into the study. Of the 626 students across 57 classes, 71 percent were Latino and many spoke Spanish at home.
• Most participants completed the follow-up surveys. Ninety-five (95) percent of participants who completed the baseline survey completed the post-intervention survey. Eighty-five (85) percent of participants from the first four semesters completed the one-year follow-up.
• The intervention and control groups were equivalent on important characteristics at the beginning of the study. Both groups were comparable on age, gender, race/Hispanic ethnicity, grade, parental educational attainment, and sexual behaviors.
• At the conclusion of the study, more Re:MIX group participants than control group participants remembered receiving information about reproductive health from any source. At the end of the intervention, 93 percent of Re:MIX participants said they had received information about sexual and reproductive health, compared to 71 percent of control group participants. This included information about healthy relationships (89 vs. 63 percent, respectively), abstinence from sex (87 vs. 49 percent), methods of birth control (90 vs. 54 percent), and sexually transmitted infections (88 vs. 54 percent).

Lessons Learned

Intensive outreach increases consent rates. The study team obtained completed consent forms for 89 percent of students by:

• Meeting with teachers (to urge them to encourage students to return consent forms), students (to describe the program and to encourage them to bring in forms), and parents (to describe the program and to encourage them to fill out the consent forms)
• Calling and texting participants and their parents to remind them to return forms
• Hosting pizza parties if at least 90 percent of students returned consent forms in their class; the study team did not pressure students or parents to say “yes” to the study

Reaching out to parents helps limit the number of program opt-outs. The state of Texas mandates that all students and their parents can opt out of sexual health education delivered in schools. Re:MIX experienced an opt-out rate of 16 percent during the first semester, which was higher than expected. High opt-out rates reduce the number of students who participate in the study and could make findings less generalizable. The study team achieved an overall opt-out rate of 11 percent across all five semesters by:

• Developing informational flyers to accompany the program opt-out form that provided background on the Re:MIX program and highlighted its key components
• Hosting information sessions with parents at each school
• Working with school liaisons to promote and further explain the program, and to confirm opt-out status with parents and students

The evaluation team must comply with mandatory reporting laws. These laws vary by state; Texas requires that researchers and educators must report any child who has had sex before the legal age of consent (age 14). The Re:MIX study enrolled eighth graders (many of whom were age 13) and included a question about sexual experience on all surveys. To comply with mandatory reporting, the evaluation team:

• Included information about mandatory reporting in the parent consent and student assent forms, and on the evaluation surveys to inform students and parents
• Reported any student under age 14 who indicated (on the survey) having had sex to the online reporting system of the Texas Department of Family Protective Services (DFPS); Texas DFPS replied that each report did not appear to involve a substantial risk of abuse or neglect (in which they have the authority to investigate)

On-the-ground local data collectors and online mobile-friendly surveys help maintain high response rates. The project team reached a 95 percent response rate on the post-intervention surveys (and an 85 percent response rate on the one-year follow-up) by:

• Partnering with University of Texas-Austin staff, who collected surveys in school; and with EngenderHealth staff, who scheduled survey dates for all classes (along with back-up survey dates for students who were absent from class on the main survey date)
• Developing online surveys for easy data collection in school computer labs; the evaluation team also developed mobile-friendly surveys so that absent students could easily complete surveys on a phone or tablet
• Providing $10 gift card incentives for the one-year follow-up survey

---

This brief was updated on June 10, 2019. You can download the original text here.

This publication was made possible by Grant Number TP2AH000033 from the Office of Adolescent Health (OAH), U.S. Department of Health and Human Services (HHS). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the OAH or HHS.

Photographs courtesy of Will Gallagher from Will Gallagher Studios

References

¹Office of Adolescent Health. (2018). Trends in teen pregnancy and childbearing. Washington, DC: U.S. Department of Health and Human Services.