Survey response over 15 years of follow-up in the Millennium Cohort Study | BMC Medical Research Methodology

Participants

Across the five panels, 260,228 participants enrolled in the Study between 2001 and 2021 (enrolled in 2001–2003, 2004–2006, 2007–2008, 2011–2013, and 2020–2021). While Panel 1 was invited from a random sample of service members from active duty and Reserve/National Guard rosters in October 2000, subsequent panels randomly sampled more recently accessed service members (i.e., with 1–5 years of military service) and oversampled certain subgroups (e.g., women, Marine Corps personnel) to enable sufficient sample size for between-group comparisons. At the start of each follow-up wave, participants were excluded if they withdrew from the study or were deceased. Panel 1 participants were eligible for five follow-up waves, but each subsequent panel was eligible for fewer follow-up waves due to the staggered multi-panel and multi-wave study design (Fig. 1).

Methods to increase follow-up survey response

Historically, the Study employed multiple strategies to increase follow-up survey response such as targeting outreach (e.g., contacts intended for specific service branches or veterans), updating contact information from multiple sources, and using incentives. Some techniques, such as postcard and email reminders to complete the survey, were consistently utilized over time, but the frequency and timing of those techniques varied by survey cycle (Table 1). Reminder emails notified participants to start or complete the web survey and postcards were sent to nonresponders. During the 2007–2008 and 2011–2013 survey cycles, the Study used reminder voice messages and during the 2014–2016 survey cycle, one primer email was sent before the survey cycle began. The Study also sent out annual commemorative emails and postcards for Memorial Day and Veterans Day that served as reminders for participants to complete the survey if the holiday fell within a survey cycle. Endorsement letters from Department of Defense leadership were sent to participants during the 2011–2013 and 2014–2016 surveys. Type of incentives and timing of distribution (either before or after survey completion) changed over time; an experiment conducted during the 2014–2016 cycle found that monetary pre-incentives produced a modest increase in follow-up survey response [23]. Finally, during the 2014–2016 survey cycle a short (2-page) paper survey was created and mailed to nonresponders 18 and 20 months into the cycle to increase response.

Measures

Military and demographic covariates

Demographic and military service factors such as birth year, sex, race and ethnicity, pay grade, service branch, military occupation, length of service (LOS), and service component were obtained from administrative records maintained by the Defense Manpower Data Center (DMDC). Military factors were measured concurrent with the follow-up response since they could be obtained from administrative records and did not rely on participant response. All other self-reported factors listed in the measures section were measured as ever occurring before each follow-up wave. Deployment dates were obtained from the Contingency Tracking System (CTS) and were used to calculate the total years deployed before each follow-up survey. Educational attainment and marital status were self-reported on the survey and backfilled with DMDC data if missing.

Stressful life events

Combat deployment experience was identified using a combination of administrative CTS records with a 5-item combat experience scale on the survey with responses categorized into one of four categories (i.e., deployed with combat, deployed without combat, deployed with unknown combat, not deployed) consistent with prior research [24]. Life stressors were assessed from five modified items (i.e., divorce, financial issues, sexual assault, sexual harassment, and physical assault) from the Holmes and Rahe Social Readjustment Rating Scale, and participants were categorized as reporting 0, 1, or 2 + life stressors [25].

Mental and physical health and unhealthy behaviors

The Study survey includes a variety of validated instruments to screen for mental health conditions. The 17-item Posttraumatic Stress Disorder (PTSD) Checklist − Civilian Version was used to measure the severity of PTSD symptoms [26, 27]. The Patient Health Questionnaire was used to identify depression, panic, other anxiety, and binge eating disorder, respectively [27,28,29]. Positive screens on these five validated mental health screeners were summed and categorized as 0, 1, or 2 + mental health conditions.

Physical health indicators were identified using a combination of self-reported provider diagnoses, height, and weight. A body mass index greater than 30 kg/m² and four specific diagnoses (hypertension, high cholesterol, migraine, and sleep apnea) were chosen from a list of 38 diagnoses because they were the most prevalent physical health conditions reported on the 2014–2016 survey and were always included on the survey. The sum of the five physical health indicators was categorized as 0, 1, or 2 + .

Unhealthy behaviors included current cigarette smoking, heavy weekly alcohol drinking, and unhealthy sleep duration. Current cigarette smoking was indicated if participants endorsed smoking 100 or more cigarettes in their lifetime and not successfully quitting smoking. Heavy weekly alcohol consumption was determined using the daily number of alcoholic beverages consumed in the past week with a threshold of 7 or 14 drinks per week for females and males, respectively [30]. Unhealthy sleep duration was assessed from a single item coded such that 6 or fewer and 10 or more average hours in a 24-h period indicated unhealthy sleep duration based on National Sleep Foundation recommendations [31]. The sum of the three unhealthy behaviors was categorized as 0, 1, or 2 + .

Statistical analyses

For each survey wave, we calculated the frequency and percentage of follow-up survey response (yes vs. no) and mode of response (web vs. paper) stratified by panel. Frequencies of military and demographic characteristics, stressful life events, mental and physical health, and unhealthy behaviors were also calculated for eligible participants. In addition, to examine the bivariate relationships between participant characteristics with follow-up survey response, Wave 2 response rate and consistent follow-up response (i.e., responded to all follow-up surveys) were reported for each category of participant-level characteristics. For survey mode, the percentages of web and paper surveys completed were reported among Wave 2 responders and those who responded by one mode more frequently across all follow-up surveys. Multicollinearity was tested among characteristics with a variance inflation factor threshold ≥ 4. Generalized estimating equations (GEEs) were used to estimate the associations between variables of interest and response to follow-up surveys over time. GEEs are a subclass of semiparametric models, which, unlike the parametric generalized linear mixed-effects models, impose no mathematical model for the distributions of the multivariate response due to repeated assessments and thereby provide valid inference for virtually all data distributions [32]. GEEs are not only robust, but also most efficient, or powerful, in the sense that they have the largest power among all such semiparametric models. For both outcomes, survey response and response mode, effect estimates were generated from fully-adjusted models that included all variables of interest [33] (birth year, sex, race and ethnicity, pay grade, service branch, military occupation, LOS, service component, deployment experience, life stressors, mental and physical health, and unhealthy behaviors), panel, and wave were reported. All statistical analyses were conducted using SAS software version 9.4 (SAS Institute, Inc., Cary, North Carolina, USA). The study was approved by the Naval Health Research Center Institution Review Board (protocol number NHRC.2000.0007).