Between March 2020 and October 2021, GiveDirectly distributed $1,000 one-time relief payments to nearly 200,000 low-income households as part Project100, the largest privately funded cash transfer program in U.S. history. Researchers at University of Michigan ran two randomized controlled trials to evaluate this program, focusing on the impact on recipients’ material and mental well-being. The studies, released last month, found few conclusive quantitative differences in subjective measures of well-being between households receiving the $1,000 payments and those who did not.
We did not expect these one-time payments to transform lives, but to help households weather the shocks associated with the pandemic: lost jobs, finding care for school-age children at home, increased food costs, etc. Our take is that the program likely worked to this end, as suggested by recipients’ own voices, but that this was not enough to move broader, subjective measures of well-being given the magnitude of recipients’ need and the deluge of government assistance. However, we take these results seriously, and there are still lessons to learn, which we unpack here.
Researchers found little measurable effect on measures of subjective well-being from our U.S. COVID-19 transfers
In May 2020, researchers launched the first of two randomized controlled trials evaluating GiveDirectly’s U.S. COVID response. The first study surveyed recipients twice, about 1 month and 3 months following payments. It found modest effects: reductions in material and mental hardship – but only among households earning less than $500 per month. In September 2020 the researchers ran a second, larger study to learn more about these results, but it failed to find any significant results at all.
These results could be attributed to a number of causes
We expected to see modest but positive results on these aggregate measures, but the studies detected few effects overall. The quantitative results show only a small improvement in material hardship from the cash payments when compared to the control group, and only in the poorest sub-sample of the first study.
Qualitatively, families told researchers they received their transfers and used the cash to keep food on the table, medicine in their cabinets, and roofs over their heads. Self-reported surveys reveal most spent their money paying bills or rent (Appendix 2). So why did these benefits not translate into significant impacts on subjective measures of well-being?
Two hypotheses could explain it:
(1) A $1,000 transfer was not enough to make an impact on broad, subjective measures of well-being during the pandemic
Unprecedented federal aid: When Project100 launched in March 2020, the federal government had not yet responded with monetary aid to the public. The following month, the CAREs Act expanded unemployment insurance and sent $1,200 stimulus checks to low and middle-income households. During both studies, unemployed recipients could be receiving as much as $5,000 per month in combined federal aid. The effects of our $1,000 cash transfer could have been diluted, essentially creating a comparison of an average of $5,000 in the control group to $6,000 in the treatment group.
Unprecedented need: It’s also plausible a relatively modest, one-time payment could not move the needle during a pandemic. Families were already financially burdened before 2020: the average debt of households earning <$35,000/year is $3,380 (nearly 10% of their earnings). Lockdowns further limited opportunity – widespread unemployment and closed schools – making it harder for recipients to spend transfers in ways that meaningly improved their livelihood. A separate research study of $500 and $2000 pandemic payments found no difference in outcomes between the two amounts, suggesting that even doubling transfer sizes would not have made the difference.
(2) Characteristics of the research may have reduced the ability to detect effects
Survey timing: The survey asked study participants about their subjective material and mental hardship. Recipients self-administered the surveys at least one month after they received the transfer. It could be that the cash aid immediately yielded positive results, which faded by the time of the surveys. Given the unique challenges caused by the pandemic, it’s conceivable that a family used GiveDirectly’s transfer to solve immediate needs, but had new, equally taxing needs a month later.
Measures used: While standard practice, the hardship measures used by researchers may not have been the most relevant for capturing changes in recipients’ experiences. Most social interventions are evaluated on specific goals (e.g. does an education program improve test scores? Does rental assistance improve housing security?). Direct cash enables recipients to spend on a wide range of goals, making the full impact difficult to capture.
Low survey response: Running studies in crisis contexts is difficult and both surveys struggled with low response rates (60% and 30% respectively). While the research team went to considerable lengths to increase the survey response rates, many among the treatment and control groups still did not respond. While non-response was balanced across treatment and control groups, it necessarily means that we can see impacts only on the kinds of households that did respond, and these may have been different from those who did not. However, it’s not immediately obvious in what direction this would skew results, if at all.
We need to keep studying what size & cadence of cash works best
A majority of U.S. evidence still weighs in favor of cash-based disaster relief, and we plan to continue delivering cash to Americans in need. However, these results suggest we need to learn more about how to design programs to maximize impact, and which outcomes are and are not movable with a given design.
It’s clear that one-time, $1,000 payments will have a weaker impact in the U.S. than they do in emerging markets. It may also be the case that $1,000 is not enough in the context of 2020’s pandemic economy – though it could still be an effective amount to yield other results, such as accelerating recovery from a natural disaster. Starting this year, we’re running guaranteed monthly income experiments in Georgia and Chicago where the total sum will be much higher ($20K & $6K, respectively). These projects should tell us more about what threshold is needed to have similar impacts as we see in our Africa-based programs.
Appendix 1
Appendix 2