Food insecurity is under-reported in surveys that ask about the past year

Abstract

Introduction

Food insecurity is defined as not having access to enough food to maintain an active and healthy lifestyle for all individuals in a household, and is characterized by reductions in food intake because of limited money or resources.¹ Risk factors for food insecurity include financial hardship and high costs of living,^2,3 and job losses and missed work during the COVID-19 pandemic put many people at risk of food insecurity.^4,5 Local reports of spiking food insecurity during the early months of the pandemic came from Los Angeles (LA) County, California, and other U.S. regions,^6–8 and the Household Pulse Survey⁹ showed national and state-level increases in food insecurity.¹⁰ Follow-up surveys suggested that food insecurity returned to pre-pandemic levels by late 2021.^11,12

In contrast, national data from the Current Population Survey (CPS), reported by the U.S. Department of Agriculture (USDA),¹³ showed that annual food insecurity rates remained stable across 2019 (10.5%), 2020 (10.5%), and 2021 (10.2%).¹ The USDA also reported stable or decreased state-level food insecurity rates, including a decline in California from 11.2% in 2015-2017 to 9.8% in 2018-2020.¹ However, these estimates seem implausible, given unprecedented long lines at food pantries nationwide, particularly in the early months of the pandemic in 2020.^14,15

One explanation for discrepancies in food insecurity rates in 2020 may be related to the period that participants are asked to recall in food insecurity questions. Each December, the CPS Food Security Supplement asks individuals to recall their experiences of food insecurity over the “past year”.¹⁶ In contrast, studies conducted throughout 2020 asked about food insecurity over shorter time periods (e.g., “past week”).^6,17 Under-reporting is more likely when surveys ask participants to remember events over longer recall periods,¹⁸ such as the past month or their lifetime rather than the past day or week, on topics such as health problems, community violence, and alcohol use.^19–22

The recall bias literature suggests that under-reporting of past-year food insecurity may especially be seen in survey respondents who experienced food insecurity that was less frequent, did not occur close to the annual survey (less recent), and/or was less severe. First, self-reports tend to be less accurate for events that are less frequent; e.g., healthcare research shows that annual visitors were more likely to under-report the healthcare usage documented in their records compared to monthly visitors.^23,24 Second, research demonstrates that under-reporting is more likely for events that were less recent.^25,26 For example, patients’ memory of the painfulness of a medical procedure reflects the last three minutes of the procedure rather than the earlier moments of the procedure.²⁷ Third, less severe events may be under-reported; e.g., drivers who watched film clips of traffic situations and were asked two weeks later about what they saw were less likely to report mild near-accidents than severe near-accidents.²⁸

This study investigates food insecurity under-reporting using data from the Understanding America Study (UAS). Participants completed multiple survey waves, and were asked to report past-week food insecurity multiple times throughout 2021 and past-year food insecurity in December 2021. These past-week and past-year questions were selected, respectively, to understand the rapidly changing circumstances of the COVID-19 pandemic, and to represent the most common recall period used to assess food insecurity. The first aim was to examine whether past-year food insecurity was under-reported by participants, compared to these same participants’ reports of past-week food insecurity, provided multiple times throughout the year. The second aim was to investigate if under-reporting was explained by recall bias: whether, among participants who reported past-week food insecurity at any time during 2021, less frequent, less recent, and less severe past-week food insecurity were associated with a greater likelihood of under-reporting of past-year food insecurity (relative to participants’ past-week reports).

Methods

Measures

Past-year food insecurity was measured in December 2021 with the U.S. Household Food Security Survey Module short form.¹⁶ Referencing the last 12 months, participants indicated (i) “The food that I bought just didn’t last, and I didn’t have money to get more,” and (ii) “I couldn’t afford to eat balanced meals” (often=1, sometimes=1, never=0); (iii) “Did you ever cut the size of your meals or skip meals because there wasn’t enough money for food?” (almost every month=2, some months but not every month=2, only 1 or 2 months=1, no=0); (iv) “Did you ever eat less than you felt you should because there wasn’t enough money for food?” (yes=1, no=0); and (v) “Were you ever hungry but didn’t eat because there wasn’t enough money for food?” (yes=1, no=0). Consistent with scoring guidelines,¹⁶ participants who had scores of 2 or more across the five questions were considered to have experienced past-year food insecurity in 2021 (1=past-year food insecurity in 2021, 0=no past-year food insecurity in 2021).

Past-week food insecurity was measured during 11 survey waves fielded between January and December 2021 (see Figure 1). The Food Insecurity Experiences Scale (FIES) was utilized because of its brevity and validity in assessing past-week food insecurity.³⁰ Two FIES questions that correspond to items in the USDA past-year measure (see Appendix Table 1) were included, referencing the past seven days: (i) “Did you eat less than you thought you should because of a lack of money or other resources?” and (ii) “Did you go without eating for a whole day because of a lack of money or other resources?” (yes=1, no=0). These questions assessed behavioral markers of moderate and severe levels of food insecurity, respectively. Participants who responded “yes” to at least one question in a given survey wave were considered to have experienced past-week food insecurity in that wave, while participants who responded “no” to both questions or didn’t respond were considered to have not experienced food insecurity in that wave.

Table 1Sample characteristics

Note. CI=confidence interval; FPL=federal poverty line.

Participants who experienced past-week food security in at least one wave were considered to have experienced any past-week food insecurity in 2021 (1=any past-week food insecurity in 2021; 0=no reported past-week food insecurity in 2021).

To compute frequency of past-week food insecurity during 2021, participants were grouped into five categories based on reported past-week food insecurity: no waves (0), one wave (1), two or three waves (2), four or five waves (3), six or more waves of past-week food insecurity (4).

To compute recency of past-week food insecurity, participants who responded “yes” to either or both past-week food insecurity questions in the December 2021 survey were considered to have experienced recent past-week food insecurity (1=reported past-week food insecurity in December 2021; 0=no past-week food insecurity in December 2021).

To compute severity of past-week food insecurity during 2021, participants who responded “yes” to the question about severe past-week food insecurity in any survey wave were considered to have experienced severe food insecurity (1=reported severe past-week food insecurity; 0=no severe past-week food insecurity).

Finally, inconsistent reporting of past-year food insecurity was defined by comparing participants’ reports of past-week and past-year food insecurity. Participants who reported any past-week food insecurity in 2021 but did not report past-year food insecurity for 2021 were considered inconsistent or under-reporters (see Figure 1, Panel A), whereas those who reported both past-week food insecurity and past-year food insecurity were considered consistent reporters (see Figure 1, Panel B).

Regarding demographics, household poverty level was computed using participants’ December 2021 reports of past-year household income and household size:³¹ <100% of federal poverty level (FPL); 100-200% of FPL; 200-300% of FPL; and >300% of FPL. In December 2021, participants also reported their sex (with response options male or female); race (with response options Black, white, Asian, American Indian/Alaska Native, and Native Hawaiian/other Pacific Islander); ethnicity (with response options Hispanic or non-Hispanic); age in years; educational attainment (with 16 response options representing years of school completed); employment status (with response options currently working, unemployed, retired, disabled, or other); and household structure (i.e., number of children in household; number of children under 5 years in household). Race and ethnicity were re-coded as one variable (non-Hispanic Black, non-Hispanic white, non-Hispanic Asian, non-Hispanic other race); age was re-coded as a categorical variable (18-30, 31-40, 41-50, 51-64, 65+); and educational attainment was re-coded into a three-category variable (high school diploma or less, some college, Bachelor’s degree or more).

Results

Data from 1135 LA County residents who responded to the December 2021 UAS survey wave (fielded December 9^th, 2021 to January 9^th, 2022) were analyzed for this study. All participants who completed the food insecurity measures in the December 2021 wave were included in the study sample, even if they missed other survey waves in 2021, because past-year food insecurity on the December 2021 survey was used to assess consistency in reporting. Participation rates for UAS survey waves included in this study are shown in Appendix Table 3.

Among the 206 participants who reported any past-week food insecurity in 2021, under-reporters of past-year food insecurity were more likely than consistent reporters to report past-week food insecurity that was: (i) less frequent (55.9% vs. 23.9% at one survey wave; 8.8% vs. 39.9% at six or more waves); (ii) less recent (10.3% vs. 63.0% reported past-week food insecurity in December 2021); and (iii) less severe (51.5% vs. 61.6% reported any severe past-week food insecurity). Considering demographic differences, a larger proportion of under-reporters had incomes above 300% of FPL (39.7% vs. 16.7% of consistent reporters), whereas a larger proportion of consistent reporters had incomes below 100% of FPL (39.9% vs. 19.1% of under-reporters).

Additionally, household poverty level significantly predicted under-reporting of food insecurity. Compared to individuals living in poverty (<100% of FPL), those with incomes 100-200% of FPL had 3.36 times the odds of under-reporting past-year food insecurity (95% CI 1.04-10.85), and those with incomes >300% of FPL had 7.20 times the odds of under-reporting past-year food insecurity (95% CI 1.95-26.55). Other covariates were not significantly associated with under-reporting.

Discussion

This study examined the consistency with which participants responded to survey questions about food insecurity that asked about different recall periods: the “past week” and the “past year.” Findings suggest substantial under-reporting of food insecurity when people are asked about the past year: only two-thirds of participants who reported past-week food insecurity one or more times in 2021 also reported, in December 2021, experiencing food insecurity in the past year. This under-reporting was explained by recall biases: among individuals who reported any past-week food insecurity during 2021, under-reporting of past-year food insecurity in December 2021 was more common among those who reported less frequent and less recent past-week food insecurity, and those with higher incomes. These results may help to explain why surveys assessing past-year food insecurity, typically conducted in December, yielded lower food insecurity rates in 2020¹ compared to surveys assessing past-week food insecurity more frequently throughout that first pandemic year.^6,10,17

The finding that less frequent and less recent food insecurity predicted under-reporting of past-year food insecurity is in line with research documenting recall bias for less frequent and less recent events.^19–23 Yet measuring peoples’ experiences of past-year food insecurity is a widely used method in public health,^32,33 as it is practical regarding implementation and cost. Measuring experiences of past-week food insecurity likely yields more accurate results, but requires more frequent assessments. One solution may involve including past-week food insecurity measures multiple times throughout the year on the CPS and other surveys, although this may be impractical due to cost. Another solution may involve assessing past-year food insecurity several times throughout the year, thus sampling peoples’ experiences at different times. These solutions could help ensure that variations in food insecurity over time, such as the food insecurity spikes in 2020, are captured. Additionally, future research should assess the ideal recall period to avoid under-reporting on food insecurity surveys.

Poverty is a known risk factor of food insecurity.^3,34 Thus, surveillance and interventions are often focused, justly, on low-income populations; indeed, numerous food assistance programs were launched and expanded during the COVID-19 pandemic to help lower-income individuals. But during the pandemic, many people experienced food insecurity who likely had never experienced it before,³⁵ including some with higher incomes.^6,8,36 The extent of this issue may be underestimated given the finding that households living above the federal poverty line are more likely to under-report past-year food insecurity. These findings defy stereotypes of who is vulnerable to food insecurity and should challenge policymakers to consider whether surveillance and food assistance efforts meet the needs of all food insecure households, particularly during a crisis.

Future research should investigate why individuals with higher incomes were more likely to under-report past-year food insecurity. Experiencing food insecurity has been related to perceived social stigma.³⁷ It is possible that higher-income individuals in our sample accurately reported their past-week food insecurity, but after having time to reflect, inaccurately reported past-year food insecurity due to social desirability bias (i.e., providing an answer they think makes them look good, rather than the answer representing their experience³⁸). Moving forward, researchers should consider that survey data about food insecurity are likely influenced by both recall bias and social stigma.

Substantial research demonstrates the negative health impacts of food insecurity, including increased risk of chronic disease (e.g., diabetes, hypertension)³⁹ and poor mental health.⁴⁰ However, under-reporting of food insecurity may lead to underestimates of its impacts on population health. Future work should explore this. Additionally, race and ethnicity typically do not predict food insecurity (or, in this study, food insecurity under-reporting). However, research has demonstrated that food insecurity is more common among some communities of color,^6,32 which have greater exposures to barriers that increase the risk of food insecurity. Accurately measuring food insecurity is critical for documenting and addressing inequities in food access and health.

Limitations

This study has limitations. First, two different measures were used to assess past-week and past-year food insecurity, with different response options and scoring methods. This could explain some mismatch between participants’ responses. However, this is unlikely to explain the results for several reasons: (i) participants’ responses to the past-week and past-year food insecurity measures on the December 2021 survey were moderately correlated (r=0.61, p<0.001); (ii) the measure of past-year food insecurity encompassed a wider range of food insecurity experiences, yet substantial under-reporting was still seen compared to the past-week measure; (iii) evidence of under-reporting held when limiting analyses to the most similar question on the past-week and past-year measures; and (iv) under-reporting was explained by systematic recall biases, and thus unlikely to be random response error. A second limitation is that, due to cost and concerns for respondent burden, it was not feasible to ask questions with additional recall periods (e.g., “past month”). Future research should examine if recall bias is reduced when asking about the “past month” rather than the “past year.” Third, past-week food insecurity data were collected for only eight months of 2021; thus, it was not possible to examine over-reporting of past-year food insecurity. Finally, it is unclear how the results from this sample of individuals in LA County who reported past-week food insecurity may generalize to other populations.

Conclusions

Substantial under-reporting of food insecurity was found on a measure that assesses experiences over the past year, compared to repeated measures of past-week food insecurity in the same study sample. Recall biases may have caused rates of food insecurity to be underestimated in many studies, particularly involving crises that increase the risk for food insecurity early in the calendar year (e.g., those caused by the COVID-19 pandemic early in 2020, when past-year food insecurity was assessed in December), or involving acute events that lead to spikes in food insecurity (e.g., natural disasters). As measuring food insecurity in the past year is more affordable, conducting such surveys at multiple timepoints during the year may improve the quality of food insecurity estimates.

References

CRediT authorship contribution statement

Michelle S. Livings: Conceptualization, Methodology, Formal analysis, Writing – original draft, Writing – review & editing, Visualization. Wändi Bruine de Bruin: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Supervision. John P. Wilson: Conceptualization, Writing – review & editing. Bruce Y. Lee: Writing – review & editing, Supervision. Mengya Xu: Conceptualization, Writing – review & editing. Alison Frazzini: Conceptualization, Writing – review & editing. Swati Chandra: Conceptualization, Writing – review & editing. Kate Weber: Conceptualization, Writing – review & editing. Marianna Babboni: Conceptualization, Writing – review & editing. Kayla de la Haye: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Supervision.

Acknowledgements

We would like to acknowledge the University of Southern California Center for Economic and Social Research, which administered the Understanding America Study and provided support for the data analysis. The research presented in this paper is that of the authors and does not reflect the official policy of the NSF.

Funding sources

This work was supported by the University of Southern California Dornsife Emergency Fund and by a research grant from the University of Southern California Keck School of Medicine COVID-19 Research Fund, through a generous gift from the W.M. Keck Foundation (PI: de la Haye). It was also supported by a grant from the National Science Foundation (S&CC-IRG Award 2125616, PI: de la Haye). The funders had no role in study design; in the collection, analysis, and interpretation of data; in the writing of the manuscript; or in the decision to submit the manuscript for publication.

Appendix. SUPPLEMENTAL MATERIAL

Related Articles