bond implementation and evaluation: sixth-year snapshot of ......sep 11, 2019 · daniel gubits ....

BOND Implementation and Evaluation

Sixth-Year Snapshot of Earnings and Benefit Impacts for Stage 2 Deliverable 24f.2(2)

September 11, 2019 Submitted To: Social Security Administration Attn: Ms. Joyanne Cobb Office of Research, Demonstration, and Employment Support 500 E Street, SW 9th Floor, RM # 905 Washington, DC 20254

Contract No. SS00-10-60011

Prepared by Judy Geyer Daniel Gubits Denise Hoffman David Stapleton Stephen Bell

Report Context

As part of the Ticket to Work and Work Incentives Improvement Act of 1999, Congress asked the Social Security Administration (SSA) to test alternative Social Security Disability Insurance (SSDI) work rules designed to increase the incentive for SSDI beneficiaries to work and reduce their reliance on benefits. In response, SSA has undertaken the Benefit Offset National Demonstration (BOND), a random assignment test of variants of SSDI program rules governing work and other supports. SSA, in conjunction with several contractors led by Abt Associates, developed the infrastructure and supports required to implement BOND.

The BOND project includes two stages. Stage 1 examines how a national benefit offset would affect earnings and program outcomes for the entire SSDI population. Stage 2 tests the impact of the offset for those expected to be most likely to use the offset—recruited and informed volunteers. Stage 2 also analyzes the extent to which enhanced counseling (EWIC) affects impacts.

This report documents Stage 2 impacts on earnings and benefit outcomes during the sixth calendar year of implementation (2016). A parallel report is being produced for Stage 1. These reports expand the follow-up period one year beyond that covered in the Final Evaluation Report.

Summary of Key Findings The findings in this report concern the SSDI beneficiaries thought most likely to respond to the offset work incentives: volunteers who presumably wished to have the offset rules applied to them and who did not initially receive Supplementary Security Income (SSI). The impact estimates show that:

• There is confirmatory evidence that the offset rules combined with standard work incentives counseling (WIC) increased SSDI benefits paid compared to current law earnings rules and counseling services. For this policy comparison, there is also exploratory evidence (from non-confirmatory tests) of an increase in the proportion

of beneficiaries with earnings above the BOND Yearly Amount (BYA), as well as an increase in the number of months with SSDI payments.

• There is some confirmatory evidence that the offset rules combined with enhanced work incentives counseling (EWIC) increased SSDI benefits paid when compared to current law. For this policy comparison, there is also exploratory evidence of increases in the proportions of beneficiaries with any employment and with earnings above BYA, as well as an increase in the number of months with SSDI payments.

• When added to the offset, EWIC did not have any detectable incremental effects on 2016 earnings and benefit outcomes relative to WIC.

• BOND is a time-limited demonstration. For roughly 10 percent of Stage 2 treatment subjects, rules governing benefits reverted from the BOND benefit offset rules to current law in 2016. Despite this reversion, the full sample 2016 impacts are not statistically different from the 2015 impacts.

The BOND Evaluation Team Abt Associates, in partnership with 25 other organizations, is implementing and evaluating the BOND under contract to the U.S. Social Security Administration. To ensure the objectivity of the evaluation, separate teams conduct the implementation and evaluation components of the project. The current report reflects exclusively the views of the evaluation team, led by Evaluation Co-Directors Daniel Gubits of Abt Associates and David Stapleton of Mathematica Policy Research. These individuals have no role in implementing or overseeing the BOND intervention they are studying, nor do any members of their evaluation team. Separation of implementation and evaluation does not extend throughout the project, however. The Abt Project Director (Michelle Wood) has responsibility for coordinating the implementation and evaluation efforts, including, respectively, managing the day-to-day operations of the project and overseeing the effective and efficient implementation of the BOND design. Within this structure, full authority over and responsibility for the content of all evaluation reports rests with the Evaluation Co-Directors.

BOND Implementation and Evaluation Contract No. SS00-10-60011

Abt Associates BOND Stage 2 Sixth-Year Snapshot Report i

Table of Contents

Acronyms Used in This Report ................................................................................................................. iii

1. Introduction ....................................................................................................................................... 1 1.1. Current SSDI Rules and the BOND Innovation ........................................................................ 1 1.2. BOND Stage 2 Implementation and Random Assignment ....................................................... 2 1.3. Purpose ...................................................................................................................................... 4 1.4. Organization of the Report ........................................................................................................ 5

2. Methodology and Context ................................................................................................................ 6 2.1. Outcome Definitions and Theoretical Impacts .......................................................................... 7 2.2. Administrative Features of the Offset That Could Influence Impacts ....................................... 8 2.3. Impact Estimation Methodology ............................................................................................. 10 2.4. Reversion to Current Law ....................................................................................................... 12 2.5. Final Analysis Sample Sizes ................................................................................................... 12

3. Impact Findings, Full Sample ........................................................................................................ 14 3.1. Confirmatory Impacts .............................................................................................................. 15 3.2. Exploratory Impacts ................................................................................................................ 18

3.2.1. Exploratory Impacts on Earnings-Related Outcomes ................................................ 18 3.2.2. Exploratory Impacts on Benefit-Related Outcomes ................................................... 19

3.3. Subgroups Defined By Duration of SSDI Participation .......................................................... 20 3.4. Summary ................................................................................................................................. 20

4. Influence of Some Treatment Subjects’ Reversion to Current Law for At Least One Month in 2016 .................................................................................................................................. 22 4.1. 2016 Size and Scale of Some Treatment Subjects’ Reversion to Current Law ....................... 22 4.2. The Effect of Some Treatment Subjects’ Reversion to Current Law on 2016 Impacts .......... 24

5. Discussion ......................................................................................................................................... 30

References .................................................................................................................................................. 31

Appendix A. Benefits Due Impact Estimates ...................................................................................... 33

Appendix B. Subgroup Analysis of pre-BOND TWP Completers ................................................... 36 B.1 Pre-BOND TWP Completion Subgroups ................................................................................ 36 B.2 Composition of the Pre-BOND TWP Completion Subgroups ................................................ 37

Appendix C. Stage 2 Impact Estimates for Subgroups Defined by Duration of SSDI Receipt ...... 39

Appendix D. Comparison of 2015 and 2016 Impacts on Benefit Outcomes for the Full Sample ... 43


Abt Associates BOND Stage 2 Sixth-Year Snapshot Report ii

List of Exhibits

Exhibit 2-1. Definitions of Confirmatory and Exploratory Outcomes and Predicted Signs of Impacts ................................................................................................................................ 9

Exhibit 2-2. Stage 2 Analysis Sample Composition ............................................................................. 13 Exhibit 3-1. Estimated Impacts on 2016 Total Earnings and Total SSDI Benefits Paid of Stage 2

Volunteers: All Policy Comparisons ................................................................................ 16 Exhibit 3-2. Estimated Confirmatory Impacts on Total Earnings and Total SSDI Benefits Paid

of Stage 2 Volunteers: All Policy Comparisons, All Evaluation Years ............................ 17 Exhibit 3-3. Estimated Impacts on 2016 Earnings and Benefits of Stage 2 Volunteers:

Exploratory Results, All Policy Comparisons .................................................................. 19 Exhibit 4-1. Number of Treatment Group Subjects who Reverted to Current Law in Each

Month, April through December of 2016, by Treatment Group ....................................... 23 Exhibit 4-2. Percent of Treatment Subjects Whose Benefits Fell or Did Not Fall under Current

Law for Any Month in 2016, by Offset Use (as of Jan. 2018) ......................................... 24 Exhibit 4-3. Estimated Impacts on the Difference Between 2016 and 2015 Outcomes of the

Offset Compared to Current Law (T22 + T21 vs C2) for Subgroups Defined by Random Assignment before December 2011, Among Subjects Who Completed a TWP Prior to Random Assignment .................................................................................. 26

Exhibit 4-4. Estimated Impacts on 2015 and 2016 Earnings and Employment of Stage 2 Volunteers: All Policy Comparisons ................................................................................ 28

Exhibit A-1. Estimated Impacts on 2016 Benefits Due ......................................................................... 33 Exhibit A-2. 2016 Impacts on Benefits Due Compared to Benefits Paid: T21 Versus C2 ................... 34 Exhibit A-3. 2016 Impacts on Benefits Due Compared to Benefits Paid: T22 Versus C2 ................... 35 Exhibit A-4. 2016 Impacts on Benefits Due Compared to Benefits Paid: T22 Versus T21 ................. 35 Exhibit B-1. Stage 2 Decomposition of Treatment Group by Pre-BOND TWP Completion,

Random Assignment Date, and Reversion to Current Law .............................................. 37 Exhibit C-1. Estimated Impacts on 2016 Outcomes of the Offset Compared to Current Law (T21

Vs. C2) for Subgroups Defined by Duration of SSDI Receipt ......................................... 39 Exhibit C-2. Estimated Impacts on 2016 Outcomes of the Offset Compared to Current Law (T22

Vs. C2) for Subgroups Defined by Duration of SSDI Receipt ......................................... 40 Exhibit C-3. Estimated Impacts on 2016 Outcomes of the Offset Compared to Current Law (T22

Vs. T21) for Subgroups Defined by Duration of SSDI Receipt ....................................... 41 Exhibit C-4. Estimated Impacts on 2016 Outcomes of the Offset Compared to Current Law (T22

+ T21 Vs. C2) for Subgroups Defined by Duration of SSDI Receipt .............................. 42 Exhibit D-1. Estimated Impacts on 2015 and 2016 SSDI Benefits of Stage 2 Volunteers: All

Policy Comparisons .......................................................................................................... 44


Abt Associates BOND Stage 2 Sixth-Year Snapshot Report iii

Acronyms Used in This Report

AEE Annual Earnings Estimate

BODS BOND Operations Data System

BOND Benefit Offset National Demonstration

BPP BOND Participation Period

BYA BOND Yearly Amount (equal to 12 × the monthly SGA level)

DAC Disabled Adult Child

DAF Disability Analysis File

DWB Disabled Widow/Widowers Benefits

EWIC Enhanced Work Incentives Counseling

eWork Electronic Work System

GP Grace Period

IRS Internal Revenue Service

MEF Master Earnings File

PHUS Payment History Update system

SGA Substantial Gainful Activity

SSA Social Security Administration

SSDI Social Security Disability Insurance

SSI Supplemental Security Income

SSR Supplemental Security Record

TWP Trial Work Period

WIC Work Incentives Counseling


Abt Associates BOND Stage 2 Sixth-Year Snapshot Report 1

1. Introduction

The Benefit Offset National Demonstration (BOND) is a random assignment demonstration that tests a variant of Social Security Disability Insurance (SSDI) program rules governing work and other supports. This Snapshot Report concerns Stage 2 of BOND, which was designed to learn about the impacts of the benefit offset for those most likely to use it, and to determine the marginal effects of the delivery of more intensive counseling services than those offered under current law. This report is the final in a series of Stage 2 Snapshot and Interim Reports about these innovations’ impacts on earnings and benefits paid. This report focuses on administrative earnings and benefits data for 2016, the sixth calendar year of implementation. This introductory chapter describes the benefit offset and Stage 2 of the demonstration, explains the purpose of this report, and ends with an outline of the remainder of the report.

1.1. Current SSDI Rules and the BOND Innovation Under current program rules, SSDI beneficiaries lose all SSDI benefits after a sustained period of substantial earnings and risk potential loss of other (non-SSDI) benefits.1 Specifically, benefits are lost if after completing a nine-month Trial Work Period (TWP) and a three-month Grace Period (GP), an SSDI beneficiary’s countable monthly earnings exceed the monthly Substantial Gainful Activity (SGA) amount. In 2016, the SGA amount was $1,130 per month for non-blind beneficiaries and $1,820 per month for blind beneficiaries. The complete loss of benefits for earnings in excess of the SGA amount is sometimes called the “cash cliff.” The cash cliff likely discourages some beneficiaries from working at all and encourages some who could work with earnings above the SGA level to keep their earnings below the SGA level.

BOND replaces the cash cliff with a “ramp” (benefit offset), with the policy objective of encouraging beneficiaries to increase their earnings and reduce their reliance on benefits.2 The benefit offset is expected to increase the earnings of those who might otherwise not work at all and those who already work but might not attempt to earn more than the SGA amount. Those who engage in SGA under current law eventually lose their benefits entirely, whereas under the benefit offset, many (perhaps most) will be eligible for a reduced SSDI benefit. While still on the ramp—that is, when earning above the SGA amount but less than the zero-benefit amount at the end of the ramp—beneficiaries can increase their benefits by working at less than their full earnings potential.

In addition to the benefit offset, there are also two key differences between the way SSA administers the benefit offset and current law. First, the benefit offset replaces the monthly SGA calculation with an annualized measure of SGA, referred to as the BOND Yearly Amount (BYA). BYA is equal to 12 times the monthly SGA amount (in 2016, $13,560 for non-blind and $21,840 for blind treatment subjects). The benefit offset reduces benefits by $1 for every $2 in countable annual earnings in excess of the BYA following completion of the TWP and GP. Use of the annualized measure requires changes in SSA’s

1 Other benefits include Medicare for those receiving SSDI for at least 24 months. Medicare benefits continue for

a lengthy period following suspension of SSDI benefits, but not indefinitely. Some beneficiaries also receive Supplemental Security Income, Medicaid, or a variety of other public or private benefits that are contingent on earnings.

2 See Exhibit 1-1 of the Stage 2 Early Assessment Report (Gubits et al. 2013) for a detailed comparison of current SSDI program rules with BOND offset rules related to work.



administrative procedures for adjusting benefits. For treatment subjects who have completed the TWP and GP, SSA pays benefits monthly under offset rules and the monthly payment amount is based on expected annual earnings. In the following calendar year, SSA reconciles payments to actual countable earnings, based on information provided by the Internal Revenue Service (IRS) and documentation provided by the beneficiary.

Second, BOND treatment subjects can use the benefit offset during a 60-month participation period. For beneficiaries who completed the TWP before random assignment, the BOND participation period starts the month after random assignment. For other beneficiaries, the BOND participation period begins the month after the TWP is completed. In both cases, the offset applies once all GP months are used up. SSA does not terminate SSDI entitlement because of work during this period, even if benefits fall to zero because of earnings that are well above the BYA.3

1.2. BOND Stage 2 Implementation and Random Assignment To support rigorous estimation of the impacts of offering the benefit offset to the SSDI beneficiary population, BOND has two components, referred to as “Stage 1” and “Stage 2.” Stage 1 was designed to examine how a national benefit offset and accompanying administrative changes would affect earnings and program outcomes for the entire SSDI population nationally. Stage 2 was designed to learn more about the impacts of the benefit offset for those most likely to use it (recruited and informed volunteers from those SSDI beneficiaries not also receiving SSI) and to determine the extent to which significant enhancements to the basic BOND-focused work incentives counseling affect offset utilization and impacts.

Stage 2 also compares enhanced work incentives counseling (EWIC) to work incentives counseling (WIC) that is tailored to the benefit offset but are otherwise intended to be comparable to counseling services available to all beneficiaries under current law. The primary difference between EWIC and WIC is that EWIC staff take a proactive approach to contacting beneficiaries on an on-going basis to inform them about demonstration services. The more intensive components of EWIC services include counselor outreach to routinely contact the beneficiary, the development of a detailed employment support plan based on assessments of vocational skills and interests, and assistance in helping beneficiaries obtain the resources and support they need to find or maintain employment.

BOND takes place in 10 large sites, each corresponding to the service area of one of 53 SSA Area Offices. The 10 sites are a random sample of the 53 candidate areas to ensure that the evaluation’s findings are nationally representative. Eligible beneficiaries in those sites were first assigned at random to a Stage 1 offset-only treatment group, a Stage 1 control group, or a pool to be solicited as volunteers for Stage 2.4 Of those beneficiaries who were solicited to participate in the Stage 2 study, about 5 percent

3 SSA will apply current rules at the end of the 60-month participation period and will terminate the benefits of

those engaged in SGA after all remaining GP months have been used. 4 The Stage 1 impact analysis compares outcomes of the Stage 1 treatment group with outcomes of the Stage 1

control group.



volunteered for the study.5 Those who volunteered were then randomly assigned to one of the three groups:

• T21 subjects (Stage 2 Offset + WIC subjects): a group that receives the $1 for $2 benefit offset with Work Incentives Counseling (WIC) ;

• T22 subjects (Stage 2 Offset + EWIC subjects): a group that receives the $1 for $2 benefit offset with Enhanced Work Incentives Counseling (EWIC); or

• C2 subjects (Stage 2 control subjects): a control group that is not offered the offset or EWIC and is subject to current law.

Random assignment for Stage 2 occurred between March 1, 2011 and September 28, 2012, with 40 percent of volunteers enrolling in the study in 2011 and 60 percent of volunteers enrolling in 2012. In total, 12,954 beneficiaries were randomly assigned to the three groups. The random assignment ratio for the three assignment groups was 8:5:8; ultimately, 4,935 volunteers were assigned to the T21 group, 3,089 volunteers were assigned to the T22 group, and 4,930 volunteers were assigned to the C2 group.

The impact analysis for Stage 2 addresses three research questions via three pairwise comparisons:

Research Question Addressed by

Comparison of

A. What is the impact of the benefit offset on outcomes for SSDI-only beneficiaries who volunteer for BOND, compared to current law?

T21 to C2

B. What is the impact of the benefit offset plus enhanced work incentives counseling on outcomes for SSDI-only beneficiaries who volunteer for BOND, compared to current law?

T22 to C2

C. What is the incremental effect of enhanced work incentives counseling on outcomes when added to the benefit offset, for SSDI-only beneficiaries who volunteer for BOND?

T22 to T21

In addition to the benefit offset, WIC and EWIC, and the change to an annual accounting period, some differences in how SSDI is administered under the BOND offset policy (relative to conventional SSA procedures) might influence impact estimates for the first two research questions. The administrative procedures established to provide T21 and T22 subjects with information and to implement benefit adjustments under the offset determine the speed with which retroactive payment adjustments are made and improper past payments are recovered. Because of how they are measured, these adjustments are especially important for the estimated impacts on benefits paid.

5 The Stage 2 outreach is described in detail in the Stage 2 Early Assessment Report (Gubits et al. 2013). Initial

recruitment waves yielded 7 percent who volunteered. Later recruitment efforts were truncated due to sufficient sample sizes, and thus had lower volunteer rates.



1.3. Purpose This Snapshot Report presents estimates for the three Stage 2 pairwise impact comparisons in the sixth year of the demonstration. The report refers to differences in T21 vs. C2 outcomes as benefit offset impacts, to T22 vs. C2 differences as benefit offset plus EWIC impacts, and to T22 vs. T21 differences as EWIC vs. WIC impacts.

This report is the final in a series of BOND Stage 2 evaluation reports. In the initial reports, we described the BOND design, the framework for estimating the impacts, and early assessment activities on the infrastructure to support Stage 2 service delivery (Stapleton et al. 2010; Bell et al. 2011; Gubits et al. 2013, respectively). Annual earnings and benefit impacts for Stage 2 subjects from 2011 through 2015 are reported in a series of Snapshot Reports and Interim Reports (Gubits et al 2014; Gubits et al 2017; Geyer et al. 2018a; Geyer et al. 2018b). A series of parallel reports documents results for Stage 1 of the demonstration. The BOND Final Evaluation Report describes a comprehensive set of outcomes for both stages of BOND from 2011 through 2015 (Gubits et al. 2018a, 2018b).

The purpose of this report is to examine the magnitude of impacts for the Stage 2 sample in 2016, the sixth year of implementation. We did not include 2016 in the main follow-up period covered by the Final Evaluation Report (Gubits et al. 2018a, 2018b) because some treatment subjects ended their BOND participation period and reverted to current law during 2016. Nevertheless, most treatment subjects remained under the treatment condition in 2016, which makes impact analysis of data from this year potentially informative for policy.

This third Stage 2 Snapshot Report uses the identical evaluation framework used in the Final Evaluation Report (Gubits et al. 2018a, 2018b).6 Within that framework, the two most important evaluation outcomes—referred to as confirmatory outcomes—are total earnings and total SSDI benefits paid. In keeping with those designations, impacts on mean earnings in 2016 and mean benefits paid in 2016 serve as the confirmatory findings in this report. Hence, statistically significant findings for the confirmatory outcomes in this report should be interpreted as confirming that the benefit offset had an impact on at least one of two outcomes: 2016 earnings and/or SSDI benefits paid in 2016.

The report also presents exploratory impact findings for other beneficiary outcomes related to 2016 earnings and benefits paid in 2016. Significant findings for these outcomes cannot confirm that the benefit offset or EWIC had impacts; they can only suggest where such effects might have occurred. These estimates provide more information on the potential impacts of the benefit offset and EWIC, but receive less weight than the confirmatory findings in assessing the overall success of the tested treatments.

The impact estimates in this Snapshot Report focus on benefits paid in 2016 rather than benefits due in 2016. All of the Snapshot Reports and Interim Reports for Stage 1 and Stage 2 have presented impact estimates for the same benefit measure: benefits paid. Benefits paid in a year is the most readily available benefit measure. Benefits paid is the sum of the monthly benefit checks a beneficiary received in a year. The measure is limited by the fact that it includes reconciliation of improper payments in earlier years and does not include retroactive adjustments to benefits. The Final Evaluation Report (Gubits et al. 2018) at least partially addressed these limitations by presenting impact estimates through 2015 for benefits due, which is the sum of monthly SSDI benefits that SSA should have paid to the beneficiary based on

6 Appendix B of the Final Evaluation Report (Gubits et al. 2018b) provides full details of the methodology.



eligibility and earnings in the corresponding month.7 The measure excludes reconciliation of earlier years’ improper payments and incorporates retroactive adjustments through May 2017 (17 months after the end of the analysis period). In this report, we focus on benefits paid in 2016 but provide an appendix with an analysis of benefits due in 2016 (Appendix A). The benefits due estimates must be considered preliminary because they reflect retroactive adjustments through January 2018 only.

In addition to full sample results, this Snapshot Report presents Stage 2 2016 impacts for subgroups defined by SSDI benefit duration prior to BOND, which have been reported in all previous Stage 2 evaluation reports. We introduce an additional subgroup analysis to explore an issue that first emerged in 2016: reversion to current law earnings rules for some Stage 2 treatment subjects. To understand the impact of reversion to current law, we sought to identify—based on pre-randomization characteristics—a group that likely reverted to current law if assigned to T21 or T22. Of those whose benefits revert to current law in 2016, 86 percent completed a TWP prior to volunteering and were randomly assigned in 2011. Thus, among those who completed a TWP prior to volunteering, we define the subgroups as those who: (1) were randomly assigned prior to December 1, 2011 and (2) all others—i.e. those who were randomly assigned after December 1, 2011.

1.4. Organization of the Report The remainder of this report consists of three chapters. Chapter 2 provides background information on the impact estimation methodology and descriptive findings that provide context for the impact estimates. Chapter 3 presents the impact findings for the confirmatory and exploratory outcomes for 2016. Chapter 4 presents findings from an investigation of how some subjects’ reversion to current law may affect the full sample results reported in Chapter 3. Chapter 5 includes a brief discussion of the results and their implications.

7 For more information about SSDI benefit measures and concepts, see Appendix A in the Final Evaluation

Report, Volume 2 (Gubits et al. 2018b).



2. Methodology and Context

The goals for the Stage 2 evaluation are to learn about the impacts of the benefit offset for those most likely to use it (recruited and informed volunteers) and to determine the extent to which significant enhancements to the basic BOND-focused work incentives counseling affect offset utilization and impacts. For practical reasons, the design restricted the beneficiaries in Stage 2 to those most likely to use the offset. Specifically, attainment of the Stage 2 objectives requires more intensive data collection and more complex service delivery than is required for Stage 1. Restricting Stage 2 eligibility to those most likely to use the benefit offset reduces the sample sizes required for Stage 2 groups from tens of thousands to thousands.

Two aspects of this strategy for selecting the sample ensured that Stage 2 subjects would be likely to use the offset. First, concurrent beneficiaries—i.e., those receiving both SSDI and SSI—were excluded from Stage 2. The interaction between SSI and SSDI substantially diminishes the value of the SSDI offset to concurrent beneficiaries, so it was expected that relatively few would use the SSDI offset. Second, in contrast to the Stage 1 sample which is randomly selected from all eligible SSDI beneficiaries, the Stage 2 sample is composed of self-selected volunteers from randomly selected eligible SSDI-only beneficiaries. These self-selected volunteers were then randomly assigned to one of the Stage 2 treatment or control groups. It is presumed that interest in using the offset led to the decision to volunteer for the study, and that this interest means that Stage 2 subjects will be more likely to use the offset than the average Stage 1 subject.8

For this report, administrative data for calculating earnings and benefit impacts were available through the end of calendar year 2016. Earnings are measured from the SSA Master Earnings File (MEF), which contains longitudinal information on wages (from employer W-2 forms) and self-employment income reported to the Internal Revenue Service (IRS). The MEF records were almost 100 percent complete for calendar year 2016 when SSA extracted them for this report.9 Benefit outcomes are measured from SSA’s Payment History Update System (PHUS) for SSDI and the Supplemental Security Record (SSR), for SSI. Benefit due outcomes are measured from SSA’s Master Beneficiary Record (MBR) for SSDI and the SSI-Longitudinal File for SSI.10

The remainder of this chapter describes our methodology for estimating benefit offset impacts. We initially specified the methodology and outcomes for the impact analysis in Bell et al. (2011). This methodology was later refined for the First-Year Stage 1 Snapshot Report (Stapleton et al. 2013), and again in the Stage 2 Interim Process, Participation, and Impact Report (Gubits et al. 2017). The impacts we report are generalizable to the national population of SSDI beneficiaries not currently receiving SSI

8 A comparison of 2011 employment rates between the Stage 1 and Stage 2 samples shows that about 16 percent

of Stage 1 subjects had at least some earnings in 2011, compared to about 37 percent of Stage 2 subjects. 9 Because the data are collected by the IRS and are therefore subject to IRS access rules, SSA staff have direct

access to MEF data, but contractors do not. Consequently, qualified SSA staff accessed the data, submitted programs developed by the BOND Evaluation Team to estimate impacts, reviewed output to ensure that it complied with privacy requirements, and then transmitted the output to the evaluation team. The MEF earnings data are updated annually. The 2016 earnings data for this report were extracted in February 2018.

10 We extracted the PHUS, MBR, SSR, and SSI-Longitudinal File data in January 2018.



who would volunteer for this study if given the opportunity. We review the outcome definitions, anticipated impacts, estimation methodology, and analysis sample below.

2.1. Outcome Definitions and Theoretical Impacts The nine outcomes for which this document reports estimates of impact include two confirmatory outcomes (total earnings in 2016 and total SSDI benefits paid in 2016) and seven exploratory outcomes (related to 2016 employment and benefits). The exploratory earnings outcomes include indicators for earnings in excess of each of three annual earnings thresholds defined by multiples of BYA (one, two, and three times BYA) and an indicator for any employment during 2016 (defined as any earnings in 2016). The exploratory benefit outcomes include number of months with SSDI payments, total SSI benefits paid, and number of months with SSI payments11—each in 2016.

The earnings measure is subject to two limitations. First, about 6 percent of people in the U.S. work force hold jobs not covered by Social Security taxes. Second, taxable earnings for Social Security are capped at a maximum amount (e.g., $118,500 for 2016). We do not expect the cap on taxable earnings to more than trivially bias the results of this analysis because very few study subjects have earnings at or above the maximum taxable amount. In 2016, 0.03 percent (three one-hundredths of a percent) of all Stage 2 subjects had earnings at or above $118,500. Beneficiaries who have earnings at or above that amount are unlikely to have a behavioral response to the offset.

In the discussion that follows, we consider the expected direction of benefit offset impacts on these outcomes, abstracting from administrative factors that could themselves influence the impacts. Exhibit 2-1 summarizes the expected direction of benefit offset impacts on each of the outcomes. We then turn to a discussion of administrative factors and their potential influence on impacts.

Although BOND was designed to test whether replacing the SGA cash cliff with the $1 for $2 offset ramp would increase return to work and earnings, and reduce beneficiary’s reliance on SSDI benefits, the theoretical direction of impacts of the benefit offset on mean earnings and benefits is ambiguous (third column of Exhibit 2-1). As described in detail in Bell et al. (2011), this ambiguity arises because the incentives created by the benefit offset vary with what the beneficiary’s earnings would be under current law. T21 and T22 subjects who would have had no earnings or earnings below BYA under current law are expected, on average, to have higher earnings and lower SSDI benefits under the benefit offset. Conversely, some T21 and T22 subjects who would have had earnings well above BYA under current law are expected to have lower mean earnings and higher mean SSDI benefits under the benefit offset.12 Positive impacts on the mean earnings for all beneficiaries require that positive impacts for those whose earnings would be less than BYA under current law are sufficiently large to offset possible negative impacts for those who would earn more than BYA under current law.

11 Although eligibility criteria for Stage 2 required that beneficiaries not be receiving SSI benefits at the time

eligibility was determined (in the first six months of 2011), Stage 2 subjects could potentially become SSI recipients (for example, after spending down their assets enough to meet the resource test). Therefore, SSI benefits are included as an outcome variable.

12 Empirically, there is evidence that some high-earning beneficiaries will reduce their earnings, but not reduce employment. Weathers and Hemmeter (2011) found evidence of a reduction in earnings by beneficiaries earning above SGA before random assignment in the Benefit Offset Pilot Demonstration.



Similarly, the predicted impact on benefits depends on what the earnings of the beneficiary would have been under current law. For those with no earnings or earnings below BYA, the predicted impact on benefits is negative; if these individuals earn more than BYA under the offset, their benefits will fall. Conversely, for those who would have had earnings above BYA under current law, benefits for many under the offset are expected to be higher because they will be eligible for a partial benefit rather than no benefit at all, as under current law. Hence, to generate a reduction in mean benefits paid, the reduction in benefits paid to those whose earnings would be less than BYA under current law must exceed the increase in benefits paid to those who would earn more than BYA under current law.

While ambiguous regarding the confirmatory outcomes, theory does predict the signs of the impacts for five of the seven exploratory outcomes (see Exhibit 2-1). Theory predicts positive impacts on employment, on the percentage of beneficiaries with earnings above BYA, and on months with SSDI payments. These predictions can be verified by separately considering the impacts for those whose earnings would be below or above BYA under current law. As indicated earlier, for those who would have earnings below BYA under current law, theory predicts that the offset will increase both the percentage employed and the percentage of beneficiaries with earnings above BYA. Those who would have earnings above BYA under current law will have a stronger incentive to remain employed and to keep their earnings above BYA under the offset than they do under current law—even though some might work and earn less under the offset. It is not possible to predict the direction of impacts on the percentage with earnings well above BYA (for example, two and three times BYA); however, it is expected that some T21 and T22 subjects whose earnings would be well above BYA under current law will reduce their earnings in response to the benefit offset.

Theory also predicts that the impact on SSI benefits paid will be negative. The offset might have an impact on SSI payments to T21 and T22 subjects who are SSDI-only beneficiaries at the outset of the demonstration and whose SSDI benefits are below the maximum federal SSI benefit amount. Under current law, some such subjects are likely to enter SSI after they spend down their assets to the point at which they satisfy the SSI resource test. Higher earnings under the offset might reduce or slow the entry of such SSDI-only subjects into SSI.13

2.2. Administrative Features of the Offset That Could Influence Impacts The previous discussion abstracts from the administrative features of the benefit offset that were designed and implemented to facilitate use of the offset by T21 and T22 beneficiaries. As described in Bell et al. (2011), because these processes are necessarily different from current law processes, they are part of the T21 and T22 interventions being tested under BOND.

The administrative adjustment of benefits for treatment subjects—the special process implemented for T21 and T22 subjects in contrast to the current law process that applies to C2 subjects—has the potential to affect the measurement of impacts on SSDI benefits, but not on annual earnings. The benefit measure used in the body of this report, benefits paid in 2016, includes reconciliation of improper pre-2016 payments undertaken in 2016 and excludes retroactive adjustments for 2016. There will be some difference between the mean benefit outcomes reported in the body of the report (benefits paid in 2016) and mean benefits due for 2016 after all adjustments are made. Appendix A reports impacts on benefits due, but these should be considered preliminary because they are based on adjustments made through 13 See Riley and Rupp (2012).



January 2018 only. We expect impacts based on both benefits due and benefits paid measures to be similar. The magnitude of the differences in estimated impacts will depend on how adjustments for the T21 and T22 subjects compare with the adjustments for C2 subjects.14

The change from monthly to annual accounting is also likely to impact benefit outcomes in another way. Under monthly accounting, earnings above SGA in any month reduce benefits for that month, but under annual accounting, the benefit reduction for those same earnings might be smaller or zero because of earnings below the SGA amount in other months of the same year. Holding earnings constant, this administrative change is expected to increase the benefits paid to some beneficiaries; any increase (decrease) in earnings due to this factor will reduce (increase) benefits. The theoretical sign of the impact of this administrative change on earnings is ambiguous.

Exhibit 2-1. Definitions of Confirmatory and Exploratory Outcomes and Predicted Signs of Impacts

Definition Predicted Sign Confirmatory Outcomes

Total earnings in 2016 2016 Social Security earnings ?

Total SSDI benefits paid in 2016

Sum of SSDI benefit payments from January through December 2016; for SSDI workers, this includes benefits for dependent spouses and minor children, but not for DACa; for DAC and DWB, it includes only benefits payable to the DAC or DWB

?

Exploratory Outcomes Earnings Outcomes (January–December 2016)b Employment in 2016 Indicator for any 2016 Social Security earnings +

Earnings above BYA Indicator for 2016 Social Security earnings greater than or equal to $13,560 (non-blind subjects) or $21,840 (blind subjects) +

Earnings above 2 × BYA Indicator for 2016 Social Security earnings greater than or equal to $27,120 (non-blind subjects) or $43,680 (blind subjects) ?

Earnings above 3 × BYA Indicator for 2016 Social Security earnings greater than or equal to $40,680 (non-blind subjects) or $65,520 (blind subjects) ?

Benefit Outcomes (January–December 2016) Number of months with SSDI payments Number of months in 2016 with SSDI benefits paid above zero +

Total SSI benefits paid Sum of SSI benefit payment amounts from January through December 2016 –

Number of months with SSI payments Number of months in 2016 with SSI benefits paid above zero –

Notes: Bell et al. (2011) provide detailed discussion on the hypothesized impacts of benefit offset. a For a description of family benefits, see https://www.ssa.gov/pubs/EN-05-10024.pdf; accessed May 27, 2014. b Earnings relative to BYA is based on earnings reported in the MEF.

14 See Stapleton et al. (2014) for descriptions of the differences in the adjustment processes for treatment and

control subjects.



The year 2016 coincides with an additional important administrative factor. The benefit offset is a time-limited opportunity. Treatment subjects eligible for the offset may use it during a 60-month period known as the BOND Participation Period (BPP).15 The first Stage 2 treatment subjects to revert to current law at the end of their five-year BPP did so between May and December 2016.16 Reversion to current law may influence 2016 impacts if beneficiaries respond to the different incentives under current law by changing their earnings. Some beneficiaries who were earning above BYA under the offset policy may reduce their earnings to below SGA under current law to maintain benefit eligibility. Other beneficiaries who were earning above BYA under the offset policy may increase their earnings under current law to compensate for the loss of partial benefits (an income effect) and in response to the fact that increases in earnings are no longer partially offset by a benefit reduction (a substitution effect).

2.3. Impact Estimation Methodology SSA included Stage 2 in the demonstration in order to provide information about the impact of the benefit offset on beneficiaries who volunteer for the study and about the impact of EWIC vs. WIC. Given the self-selected nature of the Stage 2 sample, the impacts from Stage 2 do not generalize to the national SSDI caseload or to any easily identifiable subpopulation. Conceptually, the Stage 2 impacts generalize to the national pool of SSDI-only beneficiaries who would have volunteered for the Stage 2 benefit offset “offer” had they been solicited.

To estimate impacts, we compare mean outcomes for the T21, T22, and C2 groups to each other. The mean outcomes are weighted for differences in site-selection probabilities and in sampling rates into the solicitation pool across sampling strata. The means are adjusted for the effects of small random differences in baseline characteristics.17 The adjustments for differences in baseline characteristics also serve to reduce the standard errors of the impact estimates. For each specific outcome, we test the null hypothesis of no impact. For each hypothesis test, we indicate statistical significance at the 10-, 5-, and 1-percent significance levels. For example, a 10 percent significance level means that if the null hypothesis is true, there is only a 10 percent chance that the test will mistakenly reject it.

The impact estimates are “intent to treat” estimates. For example, the benefit offset impacts capture the mean impact of the applicability of the benefit offset rules to the earnings of all T21 subjects, whether or not those subjects work and use the offset. Likewise, the benefit offset plus EWIC impacts capture the impact on all T22 subjects, whether or not they work and use the offset. Hence, the impact estimates

15 For those who completed the TWP before BOND random assignment (in April 2011), the BPP begins in the

month following random assignment and continues through April 2016. In May 2016, those beneficiaries revert to current law and SSA will terminate benefits of those engaged in SGA in any month thereafter (excluding any unused GP months). Beneficiaries who completed the TWP after BOND random assignment and before September 30, 2017 began the 60-month eligibility period in the month after TWP completion. Those who do not complete the TWP by September 30, 2017 lose the opportunity to use the offset.

16 These beneficiaries are a subset of those who completed their TWP in or before November 2011, only excluding those whose SSDI benefits were terminated for death, medical recovery, or other reasons before reaching their would-be-reversion to current law in 2016.

17 See the Stage 2 Interim Process, Participation, and Impact Report (Gubits et al. 2017) for a full description of the estimation model and the construction of analysis weights.



reflect “no impacts” for those treatment subjects who would not have any earnings under current law or the offset.

The Stage 2 impact analysis has a total of six confirmatory hypothesis tests: tests of impacts on the two confirmatory outcomes in each of the three pairwise comparisons. We group the four tests in the T21 vs. C2 and T22 vs. C2 comparisons together because they both involve impacts of the benefit offset. We perform a multiple comparison procedure on these four tests together to adjust the p-values of the tests. We perform a separate multiple comparison procedure to adjust the p-values of the two confirmatory tests in the T22 vs. T21 comparison. These adjustments are necessary because we are performing multiple hypothesis tests, making the probability of at least one Type I error (rejecting a true null hypothesis) larger than the significance level for the individual tests. To compensate for this effect, we adjust the test statistics for the confirmatory tests so that the probability of rejecting the null hypothesis of no impact within the “family” of tests (i.e., either within the four tests of T21 vs. C2 and T22 vs. C2 or within the two tests of T22 vs. T21) is equal to the specified significance level if the null hypothesis of no impact on any outcome in the tested group is true.18

We make no multiple comparison adjustment to the tests for exploratory outcomes. Readers are advised to give less evidentiary weight to any individually significant result from an exploratory test than they would to an equally significant result from a confirmatory test. We estimate impacts for the full Stage 2 assignment groups and for subgroups defined by duration of SSDI benefit receipt at the point of solicitation into the demonstration (see Section 2.5 for information on sample size).19 The duration subgroups are of interest because prior research and program rules suggest that subjects who have been on the rolls for a short duration (defined here as three years or less) may respond to the benefit offset differently from those who have been on the rolls for a long duration (more than three years).20 More specifically, we expect more short-duration subjects to work in comparison to long-duration subjects. However, we expect it will take longer for short-duration subjects to actually have their benefits adjusted, because they will have completed fewer TWP and GP months at the outset of the demonstration in comparison to long-duration subjects.

18 Our approach adjusts the p-values for the confirmatory outcomes using the Westfall-Young stepdown method.

Details of the p-value adjustments for tests of impacts on the confirmatory outcomes appear in Appendix B of the Final Evaluation Report. See Schochet (2009) for further discussion of the multiple comparisons problem.

19 We measure the duration of SSDI receipt from the outreach release date rather than from the date of random assignment in order to prevent endogenous selection into the duration subgroups. Some beneficiaries may have responded faster to outreach than others and the speed of their response may be correlated with their earnings and benefit outcomes. A short-duration beneficiary who took a long time to respond to outreach before enrolling in the study may have crossed the threshold into the long-duration definition (37 months or more of SSDI receipt) if duration is measured from random assignment. In order to rule out the possibility of subjects determining their subgroup membership after exposure to the study (which occurred when subjects were first solicited to enroll), we measure duration from outreach release date.

20 It was expected that beneficiaries who had received SSDI for a short duration (defined as three years or less) would be more likely to work—and so be more responsive to the work incentives in BOND—than beneficiaries who had received SSDI for a longer time. Therefore, SSA especially sought information in Stage 2 for beneficiaries who had received SSDI for a short duration. To get that information, SSA set a goal of having at least 50 percent of volunteers be short-duration recipients. Because only 32 percent of SSDI-only beneficiaries overall fall into this subpopulation, this goal was accomplished by oversampling short-duration beneficiaries.



2.4. Reversion to Current Law In addition to the subgroup analyses examined in the previous Snapshot Reports, this report contains subgroup impact estimates that explore the potential effects of reversion to current law at the end of the BOND Participation Period. Treatment subjects in BOND are subject to offset rules during their BOND Participation Period (BPP), a 60-month period that begins the month after the end of a beneficiary’s TWP. A small proportion of Stage 2 treatment subjects, 9.1 percent of T21 subjects and 10.6 percent of T22 subjects, reached the end of their BPP before December 1, 2016, and thus their benefits reverted to current law for at least one month in 2016. Of those whose benefits reverted to current law in 2016 and were randomly assigned in 2011, 95 percent completed a TWP prior to volunteering.21

To understand the impact of reversion to current law, we sought to identify—based on pre-randomization characteristics—a group that likely reverted to current law if assigned to T21 or T22. By defining both the subsample and the subgroups exclusively based on baseline characteristics, we can be confident that significant differences in outcomes across treatment and control members of each subgroup are internally valid impact estimates for the subpopulations represented. Among those who completed a TWP prior to volunteering (“pre-BOND TWP completers”), we define the subgroups as those who: (1) were randomly assigned prior to December 1, 2011 and (2) all others- i.e. those who were randomly assigned after December 1, 2011. Appendix B explains the construction of these subgroups and shows sample sizes in each subgroup.

We treat all subgroup analyses, including the tests of earnings and SSDI benefits paid, as exploratory.

2.5. Final Analysis Sample Sizes Exhibit 2-2 presents the sizes for the overall sample and the subgroups. The final Stage 2 analysis sample contains a total of 12,744 subjects, spread across T21 (4,854), T22 (3,041), and C2 (4,849).

The baseline characteristics (not shown) for the T21, T22, and C2 samples are statistically equivalent to each other (Gubits et al. 2018b, Exhibit B-7). These descriptive findings give us a high level of confidence in the internal validity of the impact estimates. In other words, baseline equivalence bolsters the case that any study findings of statistically significant outcome differences between these samples represent real impacts of the interventions, rather than systematic preexisting differences between the three groups or their environments. The impact estimates are generalizable to the national population of SSDI beneficiaries who would have volunteered for Stage 2 had they been offered the opportunity to enroll in the study.

21 Some beneficiaries had their benefits rules revert to current law in 2016 even though they did not complete their

TWP prior to random assignment. This reversion is possible if they completed their TWP before December 1, 2011 and were randomly assigned prior to December 1, 2011.



Exhibit 2-2. Stage 2 Analysis Sample Composition

Random Assignment

Group

Duration Completed TWP Pre-BOND

Full Sample Short

Duration Long

Duration

Randomly Assigned Prior to

December 1, 2011

Randomly Assigned

After December 1,

2011 Stage 2 Sample Unweighted Counts

T21 4,854 3,125 1,729 319 638 T22 3,041 1,914 1,127 229 366 C2 4,849 3,102 1,747 336 640

Stage 2 Sample Weighted Percentages T21 100% 43.2% 56.8% 9.7% 15.4% T22 100% 40.8% 59.2% 11.1% 14.0% C2 100% 42.1% 57.9% 10.4% 15.3%

Source: BOND Operations Data System (BODS). Notes: The total sample size (T21 + T22 + C2) is 12,744. The Stage 2 analysis sample excludes 210 beneficiaries who are related to other BOND subjects (e.g., a primary and a DAC or two DACs with the same primary) to avoid contamination effects that might arise from the fact that almost all such beneficiaries (204 of the 210) were assigned to different BOND groups. Because only six of these beneficiaries would have been able to be retained, it was not feasible to replicate the approach used for the Stage 1 analysis (where we were able to include pairs in which both members were assigned to the same group and revise the weights so that impact estimates reflect impacts for all beneficiary pairs with at least one member in Stage 1 (Stapleton et al. 2013)). Weights are used to account for differing probabilities of selection into the Solicitation Pool by site and duration of SSDI receipt. The weighted Stage 2 sample size is 278,585 (the estimated number of Stage 2-eligible beneficiaries in the nation who would have volunteered had all Stage 2-eligible beneficiaries been offered the opportunity to enroll in the study). 2,578 subjects (25.3 percent, weighted) completed a TWP prior to random assignment. This subsample is used in Chapter 4. The subsample is split into subgroups defined by random assignment prior to, or after, December 1, 2011.



3. Impact Findings, Full Sample

This chapter presents impact findings for Stage 2 of BOND in 2016, when the average subject had been enrolled slightly less than 60 months. Those randomly assigned to one of the treatment groups became subject to the offset work incentives starting in 2011 (40 percent of T21 and T22 subjects) or during the first nine months of 2012 (60 percent).

There are three policy comparisons:

• The impact of the benefit offset with standard work incentives counseling (WIC) compared to current law (T21 vs. C2);

• The impact of the benefit offset and enhanced work incentives counseling (EWIC) compared to current law (T22 vs. C2); and,

• The incremental impact of adding EWIC to the benefit offset (T22 vs. T21).

For each policy comparison, we report estimates of impact on two confirmatory outcomes and seven exploratory outcomes. Exhibit 3-1 in Section 3.1 displays the impacts on the confirmatory outcomes and Exhibit 3-3 in Section 3.2 displays the impacts on the exploratory outcomes. The last section of the chapter breaks out findings into separate results for different subpopulations of the SSDI beneficiaries in the Stage 2 sample.

For each outcome, the exhibits present regression-adjusted average outcomes for the three random assignment groups22 and impact estimates (i.e., regression-adjusted differences between the mean outcomes). Thus for total earnings (first row of Exhibit 3-1), the estimated effect of the offset (plus WIC) compared to current law—shown in the fourth column as $375—equals the difference between the average T21 outcome of $5,274 and the average C2 outcome of $4,899. Other impact columns and other rows of the exhibit follow this same structure.

As explained in Chapter 2, the significance levels for full-sample estimates of impacts on the confirmatory outcomes (total earnings and total SSDI benefits) are adjusted to address the multiple comparisons problem. The statistical significance of the confirmatory impact estimates at the 10-, 5-, and 1-percent significance levels are indicated with “#” symbols in the last three columns of the exhibit. For the other exploratory outcomes, and for all subgroup analyses, the impact estimates are considered exploratory and their significance levels are not adjusted for multiple comparisons. The significance levels of the exploratory estimates are indicated by asterisks. For the confirmatory outcomes, we describe estimates that are statistically significant at the 10-percent level as “some confirmatory evidence” of demonstration impact, while those significant at the 5-percent level are described as “confirmatory evidence” of impact and those significant at the 1-percent level are characterized as “strong confirmatory evidence.” We term as “not statistically significant” any confirmatory impact estimate not significant at even the 10-percent level. Findings concerning exploratory outcomes are dubbed “suggestive” when found statistically significant at any of the three significance levels, since they are not adjusted to contain

22 The regression-adjusted average outcomes are calculated as the average predicted outcomes in the three groups

using the common set of coefficients estimated in the regression model. See Appendix B in the Final Evaluation Report (Gubits et al. 2018b) for a description of the regression model.



the heightened risk of false positive findings when multiple tests of significance are run. The last section of the chapter reports findings for different subpopulations of the SSDI beneficiaries in the Stage 2 sample.

3.1. Confirmatory Impacts Among the many outcomes analyzed in the BOND evaluation, two outcomes are of paramount interest. These we examine for confirmatory evidence that one or both of the Stage 2 interventions compared to current law are having an impact on beneficiaries23:

1. Total earnings in the most recent available year (2016 in this report)

2. Total SSDI benefits paid in the most recent available year (2016 in this report)

The Stage 2 impact analysis has a total of six confirmatory hypothesis tests: tests of impacts on these two confirmatory outcomes in each of the three pairwise comparisons. We group the four tests in the T21 vs. C2 and T22 vs. C2 comparisons together because they both involve impacts of the benefit offset. We perform a multiple comparison procedure on these four tests together to adjust the p-values of the tests. We perform a separate multiple comparison procedure to adjust the p-values of the two confirmatory tests in the T22 vs. T21 comparison that solely concern a difference in counseling approaches (EWIC vs. WIC).

Relative to current law, we do not find a statistically significant effect of the offset policy on 2016 earnings for either treatment group. (Exhibit 3-1). Estimated impacts on mean earnings (first row of the exhibit) are $375 for the offset plus WIC compared to current law, and $406 for the offset plus EWIC compared to current law.

Similar to the previous findings for each year of the demonstration thus far (2011 to 2015), we find no evidence of an incremental effect on earnings of EWIC compared to WIC. The point estimate of impact on earnings for this comparison is small ($31), and not statistically significant.

23 These two outcomes were identified in the BOND Evaluation Analysis Plan (Bell et al. 2011) for confirmatory

analysis, prior to the research team having access to outcome data for study subjects. Pre-specifying outcomes for confirmatory analysis prior to having access to outcome data is standard evaluation practice. It makes transparent that researchers have selected the study’s confirmatory outcomes based on hypotheses developed prior to looking at the data, rather than based on the estimates of impact for many different outcomes. See the discussion of confirmatory outcomes in Chapter 6, Section 6.1, of Bell et al. 2011.



Exhibit 3-1. Estimated Impacts on 2016 Total Earnings and Total SSDI Benefits Paid of Stage 2 Volunteers: All Policy Comparisons

Outcome

Average Outcome

with Offset and WIC

(T21) (1)

Average Outcome

with Offset and EWIC

(T22) (2)

Average Outcome

under Current Law

(C2) (3)

Estimated Impact of Offset + WIC vs

Current Law (T21 vs. C2)

(4)

Estimated Impact of Offset + EWIC vs


(5)

Estimated Impact of

EWIC instead of WIC Given

Offset (T22 vs. T21)

(6) Total earnings (January – December 2016)

$5,274 $5,305 $4,899 $375 a ($241)

$406 a ($267)

$31 ($274)

Total SSDI benefits paid (January – December 2016)

$12,347 $12,232 $11,739 $608b## ($149)

$493b# ($163)

$-115 ($165)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. Unweighted sample sizes: T21 = 4,854, T22 = 3,041, C2 = 4,849 #/##/### Impact estimate is significantly different from zero at the .10/.05/.01 level, respectively, using a confirmatory standard of evidence (p-value adjusted by the multiple comparisons procedure) and a two-tailed t-test with 9 degrees of freedom. a The impact estimates for total earnings for T21 vs. C2 and for T22 vs. C2 both had p-values after multiple comparison adjustment of 0.302 and hence do not provide confirmatory evidence of an impact. b The impact estimate for total SSDI benefits paid for T21 vs. C2 and for T22 vs. C2 had p-values after multiple comparison adjustments of 0.016 and 0.054, respectively. Hence, the data provide confirmatory evidence of an impact.

There is confirmatory evidence that the offset increased total SSDI benefits paid in 2016. Estimated impacts on benefits are $608 annually for the comparison of the offset plus WIC to current law and $493 for the comparison of the offset plus EWIC to current law. The size of these impacts are 5.2 percent and 4.2 percent of the C2 mean, respectively.

Similar to all previous findings, we find no evidence of an incremental effect on SSDI benefits of EWIC compared to WIC. The point estimate of impact on SSDI benefits for this comparison is small (-$115) and not statistically significant.

These findings are consistent with those reported for other years, as shown in Exhibit 3-2. Across separate annual analyses, there was no evidence of an effect on earnings for the offset combined with either WIC or EWIC (Gubits et al. 2018a).



Exhibit 3-2. Estimated Confirmatory Impacts on Total Earnings and Total SSDI Benefits Paid of Stage 2 Volunteers: All Policy Comparisons, All Evaluation Years

Outcome

Average Outcome

with Offset and WIC

(T21)

Average Outcome


(T22)

Average Outcome

under Current Law

(C2)

Estimated Impact of

Offset + WIC vs Current

Law (T21 vs. C2)

T21 vs C2 SE



T22 vs C2 SE

Estimated Impact of EWIC instead of WIC Given Offset (T22 vs. T21)

T22 vs T21 SE

Earning ($ in year) 2011 $3,251 $3,387 $3,053 $198 ($105) $334 ($192) $136 ($129) 2012 $4,040 $4,098 $3,683 $356 ($165) $415 ($277) $58 ($195) 2013 $4,396 $4,381 $3,993 $403 ($186) $388 ($292) $-15 ($208) 2014 $4,649 $4,619 $4,219 $431 ($210) $400 ($306) $-30 ($225) 2015 $4,924 $5,017 $4,616 $309 ($243) $402 ($287) $93 ($253) 2016 $5,274 $5,305 $4,899 $375 ($241) $406 ($267) $31 ($274)

SSDI Benefits Paid ($ for Year) 2011 $13,933 $13,960 $13,779 $155 ($134) $181 ($143) $26 ($192) 2012 $13,209 $13,235 $12,986 $223 ($133) $249 ($126) $26 ($157) 2013 $13,046 $13,063 $12,681 $365 ($134) $381 ($138) $17 ($173) 2014 $12,637 $12,688 $12,265 $372# ($128) $423# ($142) $50 ($197) 2015 $12,639 $12,668 $12,119 $520## ($138) $549## ($153) $29 ($193) 2016 $12,347 $12,232 $11,739 $608## ($149) $493# ($163) $-115 ($165)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are inflation-adjusted to 2016 dollars using the Consumer Price Index for Urban Wage Earners and Clerical Workers (CPI-W). Unweighted sample sizes: T21 = 4,854, T22 = 3,041, C2 = 4,849 #/##/### Impact estimate is significantly different from zero at the .10/.05/.01 level, respectively, using a confirmatory standard of evidence (p-value adjusted by the multiple comparisons procedure) and a two-tailed t-test with 9 degrees of freedom.



3.2. Exploratory Impacts The previous section reported results for confirmatory outcomes, finding some evidence of an impact on SSDI benefits paid but no evidence of an impact on earnings. This section considers potential impacts on other earnings- and benefit-related outcomes—outcomes tested for effects on an exploratory rather than confirmatory basis.

Seven other outcomes related to earnings and benefit amounts are available in administrative data: any employment during the year and in various dollar ranges relative to BYA, number of months of SSDI receipt over a year, and total dollars and number of months of payments from the Supplemental Security Income (SSI) program. We report impacts on these measures for 2016 in this section. Consistent with the BOND Evaluation Analysis Plan (Bell et al. 2011), we consider these analyses to be exploratory and therefore do not make any correction for multiple comparisons. As a result, any statistically significant findings are suggestive of where further effects of the benefit offset plus EWIC or WIC may have taken place. Even if the intervention had no impact on any of the measures examined here, we would expect some of the impact estimates to be statistically significant by chance alone due to the fact that we conduct many hypothesis tests in this section.

3.2.1. Exploratory Impacts on Earnings-Related Outcomes

As stated in Chapter 2, the offset is predicted to have two countervailing effects on earnings: a positive effect on average earnings for those who would not engage in SGA under current law (i.e. without the offset) and a negative effect on average earnings for those who would earn above the SGA level under current law. We find evidence of the former, and no evidence of the latter. Exhibit 3-3 shows that in the current law control group, 36 percent of beneficiaries had some employment in 2016 and 11 percent had earnings above BYA. The offset plus EWIC increased the proportion employed by 2.7 percentage points (a 7 percent increase, after rounding) and the proportion with earnings above the BYA by 1.8 percentage points (a 16 percent increase, after rounding). While the offset plus WIC did not yield a statistically significant increase in the total employment rate compared to the current law control group, it did increase the proportion with earnings above BYA by 2.4 percentage points (a 22 percent increase, after rounding). These findings conform to an unambiguous prediction of theory that by removing the benefit cliff, the offset will increase employment and the proportion of beneficiaries with earnings above BYA.

That these employment effects took place without confirmatory evidence of impact on average earnings could be due to multiple factors. One possible explanation is that average earnings in the treatment groups may have increased, but not enough to be statistically significant. In particular, the estimated impact on the proportion with earnings above BYA is small (2.36 percentage points for the offset plus WIC, and 1.84 percentage points for the offset plus EWIC), and modest differences in mean earnings within this small proportion of the study sample are hard to detect because they are averaged with earnings for the rest of study sample. It is also possible that even as a greater proportion of subjects chose to earn above BYA, average earnings within one or more of the earnings ranges far above BYA may have declined (for illustration, a person who would earn 2.9 times BYA without the offset might choose to earn 2.0 times BYA if offered the offset). This possibility is consistent with theory, which predicts that subjects who under current law would choose to earn between BYA and the amount where benefits would be reduced to $0 under the offset (i.e., the end of the offset “ramp,” about three times BYA) will decrease their earnings if the offset is available (thereby obtaining more leisure time at the same or greater total income).



Similar to all previous years’ findings, we find no evidence of an incremental effect of EWIC compared to WIC on employment or earnings above one, two, or three times BYA.

Exhibit 3-3. Estimated Impacts on 2016 Earnings and Benefits of Stage 2 Volunteers: Exploratory Results, All Policy Comparisons

Outcome

Average Outcome

with Offset and WIC

(T21) (1)

Average Outcome


(T22) (2)

Average Outcome

under Current

Law (C2) (3)

Estimated Impact of Offset + WIC vs Current

Law (T21 vs. C2)

(4)

Estimated Impact of Offset + EWIC vs Current

Law (T22 vs. C2)

(5)

Estimated Impact of

EWIC instead of WIC Given

Offset (T22 vs. T21)

(6) Earnings Outcomes (January–December 2016)

Employment during year (%) 38.05 38.82 36.17 1.88 (1.04)

2.65* (1.40)

0.77 (1.47)

Earnings above BYA (%) 13.03 12.51 10.67 2.36** (0.79)

1.84** (0.80)

-0.52 (0.84)

Earnings above 2x BYA (%) 5.19 4.90 4.87 0.32 (0.49)

0.03 (0.53)

-0.29 (0.56)

Earnings above 3x BYA (%) 2.28 2.38 2.20 0.07 (0.43)

0.17 (0.40)

0.10 (0.39)

Benefit Outcomes (January–December 2016) Number of months with

SSDI payments 10.49 10.46 10.01 0.47***

(0.09) 0.45*** (0.10)

-0.02 (0.10)

At least one month with an SSDI benefit paid (%)

90.01 89.44 85.71 4.30*** (0.74)

3.73*** (0.83)

-0.57 (0.79)

Total SSI benefits paid $37 $36 $42 $-5 ($10)

$-5 ($13)

$-0 ($17)

Number of months with SSI payments

0.18 0.18 0.19 -0.00 (0.03)

-0.01 (0.04)

-0.00 (0.04)

At least one month with an SSI benefit paid (%)

1.98 1.76 2.04 -0.06 (0.33)

-0.28 (0.39)

-0.22 (0.46)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. Unweighted sample sizes: T21 = 4,854, T22 = 3,041, and C2 = 4,849 */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (and with no multiple comparisons adjustment).

3.2.2. Exploratory Impacts on Benefit-Related Outcomes

This section considers exploratory impacts on benefit-related outcomes. There is strong exploratory evidence that the offset—both with WIC and with EWIC—increased the mean number of months of SSDI receipt, by 0.47 and 0.45 months, respectively. These estimates represent a 4 to 5 percent increase over the average number of months of SSDI receipt for C2 subjects. Similarly, the offset—both with WIC and with EWIC—increased the proportion with at least one month with an SSDI benefit by 4 percentage points over the proportion for C2 subjects (86 percent). These findings are consistent with the confirmatory finding that the offset plus WIC and the offset with EWIC both increased benefits paid in



2016. These findings are also expected because treatment subjects whose earnings would have been above BYA under current law receive partial benefits under the offset but would have had their benefits suspended under current law.

There is no evidence of any impact of the offset plus WIC or the offset plus EWIC on SSI benefits received or number of months of SSI receipt in 2016. There is also no evidence that EWIC does more than WIC to affect the number of months of SSDI receipt, number of months of SSI receipt, or amount of SSI benefits.

3.3. Subgroups Defined By Duration of SSDI Participation This section presents findings for the comparison of impacts between subgroups defined by duration of SSDI participation. All subgroup analyses are exploratory. The significance tests are not adjusted for multiple comparisons. Therefore, at best, these subgroup results provide only suggestive evidence of impacts for subpopulations. We compare impact estimates across duration groups for each of the follow impacts:

• The offset plus WIC compared to current law (T21 versus C2);

• The offset plus EWIC compared to current law (T22 versus C2);

• The offset with either type of work incentives counseling compared to current law (T22 combined with T21 versus C2); and

• The offset plus EWIC compared to the offset plus WIC (T22 versus T21).

We explored whether the Stage 2 treatments affected earnings and benefits differently for beneficiaries who had been receiving SSDI for a relatively long duration prior to random assignment, compared to those who had received SSDI for a shorter duration prior to random assignment. It was expected that beneficiaries who had received SSDI for a short duration would be more responsive to the work incentives in BOND than beneficiaries who had received SSDI for a longer time. We define short-duration beneficiaries as those who had received SSDI for up to three years (36 months) at the time they were solicited to volunteer for the study. All other sample members are considered long-duration beneficiaries.

The findings appear in Exhibits C-1, C-2, C-3, and C-4 (Appendix C). The evidence of the offset impacts seen in the full sample, given either type of counseling, is generally similar for both short-duration (36 months or less) and long-duration SSDI beneficiaries. Out of 28 hypothesis tests, no difference between estimated impacts for the two subgroups is statistically significant (shown in column 7 of Exhibits C-1, C-2, C-3, and C-4).

3.4. Summary Similar to all prior years of the demonstration there is no confirmatory evidence that the offset plus WIC or the offset plus EWIC had an impact on earnings in 2016. There is exploratory evidence that the offset plus WIC and the offset plus EWIC increased the proportion of beneficiaries with earnings above BYA in 2016, but the increases were not large enough to lead to detectable positive impacts on earnings for the confirmatory analysis. There is no evidence that EWIC had a different impact on earnings than WIC.



Similar to the prior two years of the demonstration, there is confirmatory evidence that the offset plus WIC increased benefits paid in 2016. There is also some confirmatory evidence that the offset plus EWIC increased benefits paid in 2016. There is no evidence that EWIC had a different impact on benefits paid than WIC.

The next chapter describes our analysis of how the 2016 findings may be influenced by some treatment subjects’ reversion to current law for at least one month of 2016.



4. Influence of Some Treatment Subjects’ Reversion to Current Law for At Least One Month in 2016

Treatment subjects in BOND are subject to offset rules during their BOND Participation Period (BPP), a five-year period that begins the month after the end of a beneficiary’s TWP. A small proportion of Stage 2 treatment subjects ended the BPP in 2016 and thus their benefit rules reverted to current law. As a result of this reversion, some of their employment decisions in 2016 may have been made on the basis of current law rather than the offset rules. Section 4.1 of this chapter presents numbers and weighted percentages explaining the size of the issue. Section 4.2 examines the extent to which reversion to current law affected the behavior of subjects most likely to have their benefit rules revert to current law. These analyses were not pre-specified in the Evaluation Analysis Plan because the plan did not include analysis of sixth year impacts.

4.1. 2016 Size and Scale of Some Treatment Subjects’ Reversion to Current Law A small proportion of Stage 2 treatment subjects, 9.1 percent of T21 subjects and 10.6 percent of T22 subjects, reached the end of their BPP prior to December 1, 2016, and thus had benefits subject to current law for at least one month in 2016.24 This timing complicates the interpretation of the 2016 impact estimates presented in Chapter 3 because some treatment group subjects’ employment decisions in 2016 may have been made on the basis of current law rather than the offset rules.

Reversions to current law began in April 2016, so a small proportion of treatment subjects were not subject to benefit offset rules for most of the calendar year. Exhibit 4-1 shows the counts of treatment subjects who reverted to current law in each month between April and December of 2016 (no subjects reverted to current law prior to April 2016). These subjects—if still receiving SSDI benefits—began receiving benefits under current law rules in the month after completing their BPP. Based on the counts in Exhibit 4-1, we calculate that 3.6 percent of the treatment groups’ weighted person-months in 2016 were after reversion to current law. This small fraction of person-months suggests that any consequences of this reversion for the 2016 impact estimates for Stage 2 would be fairly small.

24 Reversion to current law occurs in the month following the end of the BPP. Therefore, those treatment subjects

who reverted to current law for at least one month in 2016 had their final month of the BPP in November 2016 or before.



Exhibit 4-1. Number of Treatment Group Subjects who Reverted to Current Law in Each Month, April through December of 2016, by Treatment Group

Month in 2016 Number of Subjects Who Reverted to Current Law in Month

T21 T22 April 13 7 May 14 13 June 20 22 July 29 30

August 44 28 September 50 19

October 49 42 November 47 27 December 42 32

Total 308 220 Total as percent of assignment group 9.1% 10.6%

Source: SSA administrative records. Notes: The percentages in the last row are weighted. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment.

Although only a small number of person-months in 2016 occurred after reversion to current law, this group may influence the estimated impacts because they represent a disproportionately large share of offset users. We expect that any observed impacts of the offset policy are largely generated by use of the benefit offset. In T21, 17.6 percent of subjects who had ever used the offset reverted to current law in 2016. In T22, 24.8 percent of offset users reverted to current law in 2016. Thus, even though only 3.6 percent of person-months in 2016 occurred after reversion to current law, we hypothesized that impact estimates may be dampened by the noticeable proportion of offset users reverting to current law in 2016. Exhibit 4-2 shows the percentage of treatment subjects in each treatment group, by offset use and reversion to current law for at least one month in 2016.



Exhibit 4-2. Percent of Treatment Subjects Whose Benefits Fell or Did Not Fall under Current Law for Any Month in 2016, by Offset Use (as of Jan. 2018)

T21 T22

Number

Percent of All

Subjects

Percent of Known

Offset Users Number

Percent of All

Subjects

Percent of Known

Offset Users Known Offset Users

Reverted to current law by December 2016a

117 3.1% 17.6% 92 4.4% 24.8%

Reverted to current law after December 2016

739 14.5% 82.4% 451 13.5% 75.2%

Total Offset Users 856 17.6% 100% 543 17.9% 100% Non-Users of the Offset

Reverted to current law by December 2016

191 6.0% 0% b 128 6.2% 0% b

Reverted to current law after December 2016

3,807 76.4% 0% b 2,370 75.9% 0% b

Total Non-Users of the Offset

3,998 82.4% 0% b 2,498 82.1% 0% b

All Subjects 4,854 100% 100% 3,041 100% 100% Source: SSA administrative records. Notes: The percentages are weighted. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. a A small number of the subjects listed in the “Reverted to current law by December 2016” rows were no longer SSDI beneficiaries due to death or medical recovery as of December 31, 2016. See Appendix B for more details. b There are zero beneficiaries in this category, by construction.

The 2016 impacts may be influenced by the behaviors of the treatment subjects who reverted to current law. If treatment subjects were going to change their earnings behavior because of reversion to current law, it is not clear when they would do so. They might wait until the first month of benefit payments under current law. Or, they might change their behavior in advance of reversion to current law. Beginning in January 2016 SSA sent letters to treatment subjects three months before their benefits reverted to current law. It is possible that some subjects changed their earnings behavior at the time of letter receipt, either because they thought it wise to anticipate and move toward their desired post-BOND earnings levels or because they mistook the letter as announcing an immediate end of the offset rules.

In the next section, we examine impacts on subjects most likely to revert to current law in 2016.

4.2. The Effect of Some Treatment Subjects’ Reversion to Current Law on 2016 Impacts

We hypothesized that reversion to current law would have the opposite effect of the benefit offset rules. In particular, reversion to current law may have caused some treatment group subjects to reduce their 2016 earnings to below BYA so that they would continue to receive SSDI benefits. We also hypothesized that reversion to current law may have caused others to increase earnings to above two or three times BYA because they wished to remain employed and no longer had the option to substitute hours worked with partial benefits. To explore this further, we estimated impacts for two new subgroups, both of which are comprised of subjects who completed the TWP prior to random assignment.



Stage 2 random assignment took place from March 2011 to September 2012 rather than at a single point in time. The majority of Stage 2 subjects (66 percent) enrolled on or after December 1st, 2011; none of these individuals randomized to the treatment groups reverted to current law in 2016. Some Stage 2 treatment subjects randomly assigned prior to December 1st, 2011, reverted to current law in 2016. Among those who completed a TWP prior to volunteering (“pre-BOND TWP completers”), we define the subgroups as those who: (1) were randomly assigned prior to December 1, 2011 and (2) all others- i.e. those who were randomly assigned on or after December 1, 2011. Pre-BOND TWP completers who were randomly assigned prior to December 1, 2011 comprise 92 percent of treatment group members whose benefit rules reverted to current law for at least one month of 2016 (92 percent for T21 and also 92 percent for T22).25 No pre-BOND TWP completer randomly assigned on or after December 1, 2011, had benefit rules revert to current law during 2016.

The comparison of these two subgroups is a nonexperimental analysis. Although subjects in both subgroups had completed TWP before random assignment, these subgroups are not statistically equivalent because the timing of random assignment was partly self-determined. The date of solicitation was randomly determined but the timing between solicitation and volunteering was determined by the beneficiaries themselves.26 In the Stage 2 solicitation process, 75.1 percent of volunteers were initially solicited in 2011, but only 40.0 percent responded quickly enough to volunteer in 2011. Hence, we would not expect impacts for the two groups to be the same in any year. However, we would expect changes in impacts across years to be roughly similar across both groups with one exception. Between 2015 and 2016, treatment subjects in just one group reverted to current law and thus a comparison of the two groups might produce evidence of impacts of reversion to current law.

For this subgroup analysis, we pooled the two Stage 2 treatment groups to increase statistical power. Prior analyses have found few differences in impacts for the two treatment groups, and it seems likely that any effects of reversion to current law would be quite similar for the two groups. Exhibit 4-3 displays the earnings and employment results for 2016 along with the corresponding results for 2015 and changes from 2015 to 2016 for the two subgroups.

Consistent with theory, there is some evidence that reversion to current law reduces the impact of the offset on earnings. Earnings impacts were $626 lower in 2016 than in 2015 for those reverting to current law in 2016. Although this decrease is not statistically significant by itself, the decrease is statistically significantly different from the change between 2015 and 2016 for the group that did not revert to current law (Exhibit 4-3, Column 7). There is also some evidence that reversion to current law motivates a small number of treatment subjects working above BYA to reduce their earnings below the BYA threshold. As the theory would predict, the point estimate for the change in the percentage with earnings above BYA for those reverting to current law in 2016 is negative (-2.4 percentage points) (Exhibit 4-3, Column 3). Similar to the earnings measure, this point estimate is not statistically significantly different from zero. However, the decrease is statistically significantly different from the change between 2015 and 2016 for the group that did not revert to current law (Exhibit 4-3, Column 7). This evidence is consistent with the hypothesis that reversion to current law dampened the full sample impact. However, it is possible that the difference between the subgroups is at least partially due to the possibility that these subgroups have

25 The other treatment subjects who reverted to current law in 2016 completed TWP after random assignment. 26 Some outreach cohorts did not cover all geographic areas in the sites so timing of volunteering was also partly

influenced by geography.



different underlying characteristics that pre-date BOND, rather than the result of reversion to current law in one of the two subgroups.

Exhibit 4-3. Estimated Impacts on the Difference Between 2016 and 2015 Outcomes of the Offset Compared to Current Law (T22 + T21 vs C2) for Subgroups Defined by Random Assignment before December 2011, Among Subjects Who Completed a TWP Prior to Random Assignment

Outcome

Random Assignment Before Dec 2011

Random Assignment During or After December 2011

Estimated Difference in Impact

(7)

Average Outcome

with Offset and WIC

and EWIC (T22 + T21)

(1)

Average Outcome

under Current

Law (C2) (2)

Impact Estimate

(3)

Average Outcome

with Offset and WIC


(4)

Average Outcome

under Current

Law (C2) (5)

Impact Estimate

(6) Earnings

2015 $9,665 $6,770 $2,895** ($932)

$9,079 $9,299 $-220 ($755)

$3,115†† ($1,128)

2016 $9,686 $7,417 $2,269** ($1,002)

$8,997 $8,891 $106 ($719)

$2,164†† ($822)

Change from 2015 to 2016

$21 $647 $-626 ($527)

$-83 $-408 $325 ($406)

$-951† ($509)

Employment 2015 58.54 54.48 4.06

(3.53) 62.81 63.04 -0.23

(2.57) 4.29

(4.56) 2016 55.60 53.22 2.37

(3.62) 59.73 58.03 1.70

(2.74) 0.67

(4.82) Change from 2015 to

2016 -2.94 -1.25 -1.69

(2.55) -3.07 -5.00 1.93

(2.21) -3.62 (2.72)

Earnings above BYA 2015 24.56 15.30 9.26***

(2.79) 23.98 20.39 3.58

(2.25) 5.67†† (2.13)

2016 23.53 16.71 6.82** (2.92)

23.12 18.54 4.58* (2.20)

2.24 (3.36)


-1.03 1.42 -2.44 (2.27)

-0.86 -1.85 0.99 (1.79)

-3.43† (1.70)

Earnings above 2xBYA 2015 12.00 6.61 5.39**

(2.07) 8.44 8.18 0.25

(1.40) 5.14†† (1.76)

2016 10.88 6.63 4.25* (2.05)

7.66 9.34 -1.68 (1.47)

5.93††† (1.42)


-1.12 0.02 -1.14 (1.62)

-0.78 1.15 -1.93 (1.14)

0.79 (1.35)



Outcome

Random Assignment Before Dec 2011

Random Assignment During or After December 2011


(7)

Average Outcome

with Offset and WIC


(1)

Average Outcome

under Current

Law (C2) (2)

Impact Estimate

(3)

Average Outcome

with Offset and WIC


(4)

Average Outcome

under Current

Law (C2) (5)

Impact Estimate

(6) Earnings above 3xBYA

2015 5.57 1.77 3.80* (1.75)

3.17 4.81 -1.64 (1.00)

5.43†† (1.75)

2016 6.14 2.55 3.60** (1.48)

3.17 4.11 -0.94 (1.02)

4.54†† (1.62)


0.57 0.77 -0.20 (1.15)

-0.00 -0.70 0.70 (0.75)

-0.90 (1.22)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are inflation-adjusted to 2016 dollars using the Consumer Price Index for Urban Wage Earners and Clerical Workers (CPI-W). Unweighted sample sizes: Random Assignment Before Dec 2011 T21+T22 = 548, C2 = 336. Random Assignment During or After December 2011 T21+T22 = 1,004, C2 = 640 */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (and with no multiple comparisons adjustment). †/††/††† Difference in impact estimates is significantly different from zero at the .10/.05/.01 levels, respectively, using an F-test.

From our examination of earnings behavior changes in the small subsample of pre-BOND TWP completers, we conclude that reversion to current law had a negligible effect on the Stage 2 full sample impact estimates for 2016. Although there is some evidence that reversion has a dampening effect on the impacts on earnings and proportion with earnings above BYA, the size of the group affected by reversion is small and the estimated effect of reversion is not large enough to generate detectable effects on full sample impact estimates. To illustrate that the effect is small, Exhibit 4-4 displays the earnings and employment results for 2016 along with the corresponding results for 2015 and changes from 2015 to 2016 for the full sample. In the full sample, there is no exploratory evidence that earnings and employment impacts were different in the sixth year of the demonstration (2016) than in the fifth year (2015), as evidenced by the lack of statistical significance in any impact estimate displayed in rows showing the change from 2015 to 2016 (Exhibit 4-4). We conclude that the full sample impacts are informative of the fifth or sixth year (depending on year of enrollment) of exposure to the benefit offset rules.27 Appendix D presents changes in 2015 and 2016 impacts on benefits outcomes and concludes that the full sample impact estimates of 2016 benefits outcomes are also informative of the fifth or sixth year of exposure to the benefit offset rules.

Though unrelated to the discussion of reversion to current law, it is worth noting the impact of the offset on the 2015 earnings of the pre-BOND TWP completers assigned to a treatment group before December 2011 (Exhibit 4-3, top row, column three). The estimated impact is statistically significant and quite 27 There are 548 beneficiaries in the T21+T22 subgroup defined by pre-BOND TWP completion and random

assignment prior to December 1, 2011. The estimated change in the impact on proportion above BYA from 2015 to 2016 is -3.32 percentage points in this subgroup. This implies that roughly 18 subjects decreased their earnings in the calendar year that they reverted to current law (which is 0.23 percent of the treatment subjects). This small group will not noticeably affect the full sample estimates.



substantial: $2,895, or 43 percent of the control mean. It is also much larger than the estimated impact for the pre-BOND TWP completers assigned to a treatment group in December 2011 or later—an estimate that is actually negative (-$220), but not statistically different from zero. Over the course of the evaluation, we have very few instances of statistically significant impacts on earnings for all treatment subjects or for any subgroup of subjects (Gubits et al. 2018a, 2018b). One plausible explanation for this large impact on earnings is that, compared to other treatment subjects, many pre-BOND TWP completers who enrolled before December 2011 were exceptionally eager to take advantage of the offset opportunity to earn more and still keep some of their benefits. For this same subgroup, there is no evidence of a statistically significant impact of the offset on employment, so the large increase in earnings is driven by employed beneficiaries who chose to increase their earnings above BYA.

Exhibit 4-4. Estimated Impacts on 2015 and 2016 Earnings and Employment of Stage 2 Volunteers: All Policy Comparisons

Outcome

Average Outcome

with Offset and WIC

(T21) (1)

Average Outcome


(T22) (2)

Average Outcome

under Current

Law (C2) (3)



(4)



(5)

Estimated Impact of

EWIC instead of WIC Given Offset(T22 vs.

T21) (6)

Earningsa 2015 $5,056 $5,150 $4,758 $298

($238) $392

($255) $93

($254) 2016 $5,274 $5,305 $4,899 $375

($241) $406

($267) $31

($274) Change from 2015 to 2016 $218 $155 $140 $77

($135) $15

($182) $-62

($168) Employment

2015 37.92 39.17 36.81 1.11 (1.23)

2.36* (1.15)

1.25 (1.16)

2016 38.05 38.82 36.17 1.88 (1.04)

2.65* (1.40)

0.77 (1.47)

Change from 2015 to 2016 0.14 -0.35 -0.63 0.77 (1.03)

0.29 (1.36)

-0.48 (1.09)

Earnings above BYA (%) 2015 12.82 13.14 10.16 2.66***

(0.76) 2.98*** (0.79)

0.32 (0.82)

2016 13.03 12.51 10.67 2.36** (0.79)

1.84** (0.80)

-0.52 (0.84)

Change from 2015 to 2016 0.21 -0.63 0.51 -0.30 (0.58)

-1.14 (0.67)

-0.84 (0.72)

Earnings above 2xBYA (%) 2015 5.16 4.87 4.70 0.46

(0.52) 0.17

(0.53) -0.28 (0.55)

2016 5.19 4.90 4.87 0.32 (0.49)

0.03 (0.53)

-0.29 (0.56)

Change from 2015 to 2016 0.04 0.03 0.17 -0.14 (0.36)

-0.14 (0.43)

-0.01 (0.45)



Outcome

Average Outcome

with Offset and WIC

(T21) (1)

Average Outcome


(T22) (2)

Average Outcome

under Current

Law (C2) (3)



(4)



(5)

Estimated Impact of


T21) (6)

Earnings above 3xBYA (%) 2015 2.26 2.23 2.37 -0.12

(0.40) -0.14 (0.42)

-0.02 (0.37)

2016 2.28 2.38 2.20 0.07 (0.43)

0.17 (0.40)

0.10 (0.39)

Change from 2015 to 2016 0.02 0.14 -0.17 0.19 (0.25)

0.31 (0.27)

0.12 (0.32)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are inflation-adjusted to 2016 dollars using the Consumer Price Index for Urban Wage Earners and Clerical Workers (CPI-W). a 2015 earnings estimates in this exhibit differ from those in Exhibit 3-2. Exhibit 3-2 presents earnings impacts from the Final Evaluation Report. This exhibit uses data extracted at a later time, allowing for the possibility that some beneficiaries’ 2015 earnings records were updated due to receipt of new or revised tax records. Unweighted sample sizes: T21 = 4,854, T22 = 3,041, and C2 = 4,849 */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (and with no multiple comparisons adjustment).



5. Discussion

The findings in this report apply to the Stage 2 sample in calendar year 2016. The Stage 2 sample is composed of the SSDI beneficiaries thought most likely to respond to the offset work incentives. Specifically, the Stage 2 sample is made up of volunteers who wished to have the offset rules applied to them and who did not initially receive SSI.

We found confirmatory evidence that the offset rules combined with standard work incentives counseling (WIC) or enhanced work incentives counseling (EWIC) increased mean SSDI benefits paid compared to what they would have been under current law earnings rules and counseling services. There is no confirmatory evidence that the offset paired with either WIC or EWIC had an impact on total earnings in 2016. Similarly, we found no confirmatory evidence that the offset plus EWIC had impacts different from the offset plus WIC.

Some exploratory findings showed impacts:

• There is exploratory evidence that the offset plus WIC and the offset plus EWIC increased the proportions of beneficiaries with earnings above BYA.

• There is some exploratory evidence that the offset plus EWIC increased the proportion of beneficiaries with any employment in 2016.

• Consistent with the confirmatory findings, there is strong exploratory evidence that both the offset plus WIC and the offset plus EWIC led to an increase in the number of months with SSDI benefits payments.

All three of these exploratory results are consistent with theory and with findings in previous years. The evidence of the offset impacts, given either type of counseling, arises for both short-duration (36 months or less) and long-duration SSDI beneficiaries, and is not appreciably stronger for either group.

Roughly four percent of the person-months in the treatment group fell under current law instead of the offset rules, due to some treatment subjects’ reversion to current law for at least one month in 2016. For treatment subjects’ most likely to use the offset (due to having completed a TWP prior to random assignment), there is some exploratory evidence that reversion to current law resulted in lower impacts on the proportion with earnings above BYA. This suggests that reversion to current law may have dampened the Stage 2 full sample impact estimates for 2016. Any dampening effect is small, however. In the full sample, there is no exploratory evidence that earnings and employment impacts were different in the sixth year of the demonstration (2016) than in the fifth year (2015) in the full sample of Stage 2 subjects.



References

Bakker, B. F. 2009. Trek alle register open! [Open up the registers!]. Amsterdam, Netherlands: VU University.

Bell, Stephen H., Daniel Gubits, David Stapleton, David Wittenburg, Michelle Derr, Arkadipta Ghosh, Sara Ansell, and David Greenberg. 2011. “BOND Implementation and Evaluation: Evaluation Analysis Plan.” Submitted to the Social Security Administration (contract deliverable 16.1 under Contract SS00-10-60011), Office of Program Development & Research. Cambridge, MA: Abt Associates, and Washington, DC: Mathematica Policy Research.

Geyer, Judy, Daniel Gubits, Stephen Bell, David Judkins, and Utsav Kattel. 2018a. “BOND Implementation and Evaluation: Fourth-Year Snapshot of Earnings and Benefit Impacts for Stage 2.” Submitted to the Social Security Administration (contract deliverable 24c.6 under Contract SS00-10-60011), Office of Research, Demonstration, and Employment Support. Cambridge, MA: Abt Associates.

Geyer, Judy, Daniel Gubits, Stephen Bell, Tyler Morrill, Denise Hoffman, Sarah Croake, Katie Morrison, David Judkins, and David Stapleton. 2018b. “BOND Implementation and Evaluation: 2017 Stage 2 Interim Process, Participation, and Impact Report.” Submitted to the Social Security Administration (contract deliverable 24c2.4 under Contract SS00-10-60011), Office of Research, Demonstration, and Employment Support. Cambridge, MA: Abt Associates.

Groen, Jeffrey A. 2012. “Sources of Error in Survey and Administrative Data: The Importance of Reporting Procedures.” Journal of Official Statistics 28(2): 173-198.

Gubits, Daniel, Michelle Derr, Jillian Berk, Ann Person, David Stapleton, Denise Hoffman, Stephen Bell, Rachel Cook, and David Wittenburg. 2013. “BOND Implementation and Evaluation: Stage 2 Early Assessment Report.” Submitted to the Social Security Administration (contract deliverable 24a.2 under Contract SS00-10-60011), Office of Program Development & Research. Cambridge, MA: Abt Associates, and Washington, DC: Mathematica Policy Research.

Gubits, Daniel, Winston Lin, Stephen Bell, and David Judkins. 2014. “BOND Implementation and Evaluation: First- and Second-Year Snapshot of Earnings and Benefit Impacts for Stage 2.” Submitted to the Social Security Administration (contract deliverable 24c.5 under Contract SS00-10-60011), Office of Research, Demonstration, and Employment Support. Cambridge, MA: Abt Associates.

Gubits, Daniel, Judy Geyer, Denise Hoffman, Sarah Croake, Utsav Kattel, David Judkins, Stephen Bell, and David Stapleton. 2018. “BOND Implementation and Evaluation: 2015 Stage 2 Interim Process, Participation, and Impact Report.” Submitted to the Social Security Administration (contract deliverable 24c.5 under Contract SS00-10-60011), Office of Research, Demonstration, and Employment Support. Cambridge, MA: Abt Associates, and Washington, DC: Mathematica Policy Research.

Gubits, Daniel, David Stapleton, Stephen Bell, Michelle Wood, Denise Hoffman, Sarah Croake, David R. Mann, Judy Geyer, David Greenberg, Austin Nichols, Andrew McGuirk, Meg Carroll, Utsav



Kattell, and David Judkins. 2018a. “BOND Implementation and Evaluation: Final Evaluation Report, Volume 1.” Submitted to the Social Security Administration (contract deliverable 24e.2 under Contract SS00-10-60011), Office of Research, Demonstration, and Employment Support. Cambridge, MA: Abt Associates, and Washington, DC: Mathematica Policy Research.

Gubits, Daniel, David Stapleton, Stephen Bell, Michelle Wood, Denise Hoffman, Sarah Croake, David R. Mann, Judy Geyer, David Greenberg, Austin Nichols, Andrew McGuirk, Meg Carroll, Utsav Kattell, and David Judkins. 2018b. “BOND Implementation and Evaluation: Final Evaluation Report, Volume 2 – Technical Appendices.” Submitted to the Social Security Administration (contract deliverable 24e.2 under Contract SS00-10-60011), Office of Research, Demonstration, and Employment Support. Cambridge, MA: Abt Associates, and Washington, DC: Mathematica Policy Research.

Riley, Gerald R., and Kalman Rupp. 2012. “Expenditure Patterns Under the Four Major Public Cash Benefit and Health Insurance Programs for Working-Age Adults with Disabilities.” Journal of Disability Policy Studies 25(2): 71-80.

Schochet, Peter Z. 2008. “Technical Methods Report: Guidelines for Multiple Testing in Impact Evaluations.” Submitted to the U.S. Department of Education, Institute of Education Science, NCEE 2008-4018. Princeton, NJ: Mathematica Policy Research.

Stapleton, David C., Stephen H. Bell, David C. Wittenburg, Brian Sokol, and Debi McInnis. 2010. “BOND Implementation and Evaluation: BOND Final Design Report.” Submitted to the Social Security Administration (contract deliverable 2.2 under Contract SS00-10-60011), Office of Program Development & Research. Cambridge, MA: Abt Associates, and Washington, DC: Mathematica Policy Research.

Stapleton, David, David C. Wittenburg, Daniel Gubits, David Judkins, David Mann, and Andrew McGuirk. 2013. “BOND Implementation and Evaluation: First-Year Snapshot of Earnings and Benefit Impacts for Stage 1.” Submitted to the Social Security Administration (contract deliverable 24c.1 under Contract SS00-10-60011), Office of Program Development & Research. Cambridge, MA: Abt Associates, and Washington, DC: Mathematica Policy Research.

Stapleton, David, David Wittenburg, David R. Mann, Denise Hoffman, and Andrew McGuirk. 2014. “BOND Implementation and Evaluation: Second-Year Snapshot of Earnings and Benefit Impacts for Stage 1.” Submitted to the Social Security Administration (contract deliverable 24c.2 under Contract SS00-10-60011), Office of Research, Demonstration, and Employment Support. Cambridge, MA: Abt Associates, and Washington, DC: Mathematica Policy Research.

Weathers III, Robert R. and Jeffrey Hemmeter. 2011. “The impact of changing financial work incentives on the earnings of Social Security Disability Insurance (SSDI) beneficiaries.” Journal of Policy Analysis and Management 30(4): 708-728.



Appendix A. Benefits Due Impact Estimates

In this appendix, we present benefits impact estimates using the benefits due measure and then compare them to the main impacts in this report, which are measured using benefit paid in 2016. The benefits due estimates must be considered preliminary because they reflect retroactive adjustments through January 2018 only. For total SSDI benefits, the impact estimates based on benefits due show strong evidence of positive impacts of the offset plus WIC compared to current law and the offset plus EWIC compared to current law (Exhibit A-1, $659 and $623 respectively). In addition, the findings show that treatment subjects typically receive SSDI benefits for a longer period (0.79 months and 0.71 months over a 12 month period for T21 and T22, respectively) than C2 subjects. Corroborating those findings, treatment subjects are more likely to have at least one month with an SSDI benefit due (6.19 and 5.34 percentage point increases for T21 and T22, respectively, compared to C2). There are no statistically significant impacts on either SSI outcome.

Exhibit A-1. Estimated Impacts on 2016 Benefits Due

Outcome

Average Outcome

with Offset and WIC

(T21) (1)

Average Outcome


(T22) (2)

Average Outcome

under Current

Law (C2) (3)



(4)



(5)

Estimated Impact of


T21) (6)

Total SSDI benefits due $12,291 $12,255 $11,632 $659*** ($149)

$623*** ($163)

$-36 ($172)

Number of months with SSDI benefit due

10.61 10.53 9.82 0.79*** (0.09)

0.71*** (0.10)

-0.08 (0.10)

At least one month with an SSDI benefit due (%)

90.25 89.40 84.07 6.19*** (0.76)

5.34*** (0.85)

-0.85 (0.80)

Total SSI benefits due $34 $36 $41 $-7 ($10)

$-5 ($12)

$2 ($15)

Number of months with SSI benefit due

0.18 0.18 0.19 -0.01 (0.03)

-0.00 (0.04)

0.00 (0.04)

At least one month with an SSI benefit due (%)

1.87 1.84 1.91 -0.04 (0.32)

-0.07 (0.39)

-0.03 (0.42)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. Unweighted sample sizes: T21 = 4,854, T22 =3,041, and C2 = 4,849 */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (and with no multiple comparisons adjustment).

The differences between SSDI benefits due and SSDI benefits paid are presented in Exhibits A-2, A-3, and A-4. The group means and estimated impacts for SSDI benefits, SSI benefits, and months with SSI benefits are substantively similar (Exhibits A-2, A-3, and A-4). For both T21 versus C2 and T22 versus C2, the estimated impact on months with SSDI benefits due is roughly 0.3 months (roughly 9 days) higher than the estimated impacts on months with SSDI benefits paid. Similarly, the estimated impact on the proportion with at least one month of an SSDI benefit due is 1.9 or 1.6 percentage points higher than



the impact on the proportion with at least one month of an SSDI benefit paid (T1 versus C2 and T22 versus C2, respectively). However, the impact on total benefits due in 2016 is not statistically significantly different from the impact on total benefits paid in 2016 in any of the comparisons.

Estimated impacts for SSDI benefits due versus estimated impacts for SSDI benefits paid in previous years are presented in Gubits et al. (2018, Exhibits F-11, F-12, F-13). Estimated impacts for benefits due are larger in 2014 and 2015 than the corresponding estimates for benefits paid by a statistically significant amount. Hence, the fact that the impact on benefits due in 2016 is not statistically different from the impact on benefits paid in 2016 is a departure from findings in recent years. There are several possible reasons for this departure. First, the underlying data used to create benefits due was extracted 17 months after the end of the analysis period used in Gubits et al. (2018), compared to 13 months for the 2016 estimate. We would expect to observe additional changes to benefits due with a later extraction date and these changes may be uneven across treatment and control groups. Second, fewer retroactive adjustments may have been required for the 2016 treatment subjects because by the end of 2016 the backlog of their pending work CDRs was much smaller than at the end of 2015. Third, 2016 was the first year in which some beneficiaries reverted to current law. SSA took efforts to ensure that the benefits of significant number of offset users in this group were adjusted timely, presumably reducing the likelihood of retroactive adjustments. Further, retroactive adjustments made to the benefits of those who reverted to current law rules may have in some instances reduced their benefits due relative to their benefits paid. Finally, it is important to reiterate that the differences in all years are not substantively meaningful. They all point to the same conclusion: the benefit offset had a positive impact on SSDI benefits.

Exhibit A-2. 2016 Impacts on Benefits Due Compared to Benefits Paid: T21 Versus C2

Outcome

Benefits Due Benefits Paid Difference in Impact T21 Mean C2 Mean

Impact Estimate T21 Mean C2 Mean

Impact Estimate

SSDI benefits $12,291.06 $11,631.61 $659.44*** ($149.35)

$12,347.15 $11,738.69 $608.46*** ($149.40)

$50.98 ($107.47)

Number of months with SSDI benefits

10.61 9.82 0.79*** (0.09)

10.49 10.01 0.47*** (0.09)

0.32††† (0.04)

At least one month with an SSDI benefit (%)

90.25 84.07 6.19*** (0.76)

90.01 85.71 4.30*** (0.74)

1.89††† (0.34)

SSI benefits $33.73 $40.58 $-6.85 ($9.91)

$36.70 $41.73 $-5.04 ($10.03)

$-1.82 ($2.29)


0.18 0.19 -0.01 (0.03)

0.18 0.19 -0.00 (0.03)

-0.00 (0.01)

At least one month with an SSI benefit (%)

1.87 1.91 -0.04 (0.32)

1.98 2.04 -0.06 (0.33)

0.02 (0.13)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (resulting from a research design involving 10 study sites) and with no multiple- comparisons adjustment. †/††/††† Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test.



Exhibit A-3. 2016 Impacts on Benefits Due Compared to Benefits Paid: T22 Versus C2

Outcome

Benefits Due Benefits Paid Difference in Impact T22 Mean C2 Mean

Impact Estimate T22 Mean C2 Mean

Impact Estimate

SSDI benefits $12,254.85 $11,631.61 $623.23*** ($163.05)

$12,231.99 $11,738.69 $493.30** ($163.48)

$129.93 ($86.81)


10.53 9.82 0.71*** (0.10)

10.46 10.01 0.45*** (0.10)

0.26††† (0.05)


89.40 84.07 5.34*** (0.85)

89.44 85.71 3.73*** (0.83)

1.61††† (0.39)

SSI benefits $36.00 $40.58 $-4.58 ($12.32)

$36.43 $41.73 $-5.30 ($13.22)

$0.72 ($2.58)


0.18 0.19 -0.00 (0.04)

0.18 0.19 -0.01 (0.04)

0.00 (0.01)


1.84 1.91 -0.07 (0.39)

1.76 2.04 -0.28 (0.39)

0.21 (0.12)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. Unweighted sample sizes: T22 = 3,041, C2 = 4,849. */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (resulting from a research design involving 10 study sites) and with no multiple- comparisons adjustment. †/††/††† Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

Exhibit A-4. 2016 Impacts on Benefits Due Compared to Benefits Paid: T22 Versus T21

Outcome

Benefits Due Benefits Paid Difference in Impact T22 Mean T21 Mean

Impact Estimate T22 Mean T21 Mean

Impact Estimate

SSDI benefits $12,254.85 $12,291.06 $-36.21 ($171.91)

$12,231.99 $12,347.15 $-115.16 ($164.76)

$78.95 ($64.78)


10.53 10.61 -0.08 (0.10)

10.46 10.49 -0.02 (0.10)

-0.06 (0.04)


89.40 90.25 -0.85 (0.80)

89.44 90.01 -0.57 (0.79)

-0.29 (0.29)

SSI benefits $36.00 $33.73 $2.27 ($14.56)

$36.43 $36.70 $-0.27 ($16.97)

$2.54 ($3.19)


0.18 0.18 0.00 (0.04)

0.18 0.18 -0.00 (0.04)

0.01 (0.01)


1.84 1.87 -0.03 (0.42)

1.76 1.98 -0.22 (0.46)

0.20 (0.13)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. Unweighted sample sizes: T22 = 3,041, T21 = 4,854. */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (resulting from a research design involving 10 study sites) and with no multiple- comparisons adjustment. †/††/††† Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test.



Appendix B. Subgroup Analysis of pre-BOND TWP Completers

This appendix provides details about how we constructed the pre-BOND TWP completer subgroup. Section B.1 describes the definition and data sources used to classify treatment and control subjects in an identical manner. Section B.2 presents statistics on the pre-BOND TWP completion status of treatment subjects, to verify that their status lined up with expectations about reversion to current law. Section B-2 ends by considering the reasons for minor but unavoidable discrepancies.

B.1 Pre-BOND TWP Completion Subgroups To understand the impact of reversion to current law, we sought to identify—based on pre-randomization characteristics—a group that would likely revert to current law if assigned to T21 or T22. The subgroup classification relies on two baseline characteristics: (1) whether a beneficiary had completed a TWP prior to random assignment while in the then-current entitlement period, and (2) whether a beneficiary was randomly assigned prior to December 2011. Exhibit B-1 shows that the classification is very strongly associated with reversion to current law in 2016 or not.

• Among T21 and T22 subjects assigned to the subgroup “completed TWP pre-BOND and randomly assigned prior to December 1, 2011,” 84 percent actually reverted to current law in 2016.

• Among T21 and T22 subjects assigned to the subgroup “completed TWP pre-BOND and not randomly assigned prior to December 1, 2011,” 90 percent did not revert to current law in 2016 while still remaining SSDI beneficiaries and almost all others were no longer SSDI beneficiaries by the end of 2016 due to death or medical recovery.

We identify the subgroups based on SSA administrative data. First, we identify beneficiaries with TWP completion dates in or before their random assignment date, as recorded in either a March 2018 extract from the Disability Control File (DCF) or a January 2018 extract from SSA’s electronic work reporting (eWork) system. If a beneficiary has multiple TWP completion dates in or before random assignment, we sort the dates chronologically and select the date that is closest to, but before, random assignment. Second, because some of the TWP completion dates may correspond to previous entitlement periods, we only consider TWP completion dates that are after the start of the entitlement period during which the beneficiary was randomly assigned to BOND. To identify the start of the relevant entitlement period, we use administrative data from the MBR provided by SSA in October 2018. We identify the most recent entitlement date that is before random assignment and only consider TWP dates that occurred after that entitlement date. If the beneficiary was missing information on entitlement date in the MBR, we followed the same process if an entitlement date was available in the 2016 Disability Analysis File (DAF). We assign beneficiaries with TWP completion dates after the SSDI start date associated with BOND and before BOND random assignment to the first subgroup. If the beneficiary was missing information on entitlement date in both the MBR and DAF, we do not consider entitlement date and only compare TWP completion date to the BOND random assignment date.



B.2 Composition of the Pre-BOND TWP Completion Subgroups We present statistics on reversion to current law, as determined by the Implementation Team in 2016 (rows two through four in Exhibit B-1). As can be seen, the two TWP completion subgroups approximately correspond to beneficiaries who reverted to current law in 2016 and those who did not, but there are some differences. The bulk of the pre-BOND TWP completers in T21 and T22 randomly assigned prior to December 1, 2011, reverted to current law in 2016 (84 percent weighted); a small minority did not (8 percent). No pre-BOND TWP completers in T21 and T22 randomly assigned after December 1, 2011, reverted to current law in 2016.

SSA had terminated the benefits of some treatment subjects prior to December 2016 due to death or medical recovery, so there was no reason for their benefits to revert to current law in 2016 or later. We retain this group in our subgroup analysis, however, because our research classification is based on TWP completion before random assignment regardless of outcomes after BOND random assignment, and it is possible, although perhaps unlikely, that random assignment could have resulted in benefit termination due to death or medical recovery. About 9 percent of pre-BOND TWP completers and 10 percent of all others were no longer an SSDI beneficiary as of December 31, 2016 due to death or medical improvement.

Exhibit B-1. Stage 2 Decomposition of Treatment Group by Pre-BOND TWP Completion, Random Assignment Date, and Reversion to Current Law

T21+T22 Subjects Subgroup: Completed TWP Pre-BOND and Randomly Assigned

Prior to December 1, 2011

Subgroup: Completed TWP Pre-BOND and Randomly Assigned

After December 1, 2011

Subgroup: Did Not Complete

TWP Pre-BOND No longer an SSDI beneficiary due to death or medical recovery, as of December 31st, 2016

47 (8%)

90 (9%)

630 (10%)

Still an SSDI beneficiary as of December 31st, 2016 Reverted to current law in 2016, and in 61st month after random assignment

464 (84%)

0 (0%)

1a (0%)

Reverted to current law in 2016, but after 61st month after random assignment

0 (0%)

0 (0%)

23 (1%)

Did not revert to current law in 2016 37 b (8%)

914 (91%)

5,689 (90%)

Total 548 (100.0%)

1,004 (100.0%)

6,343 (100.0%)

Source: BOND Operations Data System (BODS). Notes: The total sample size (T21 + T22) is 7,895. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Reversion to current law can happen prior to the end of BPP if SSDI benefits are terminated. a According to available data, this beneficiary received a letter informing them of their reversion to current law in 2016. b Some beneficiaries completed a TWP prior to random assignment but not did revert to current law in 2016. Seven of these cases were found to have been terminated from the SSDI roles prior to random assignment. Two cases withdrew from the BOND study. Another two should have been mailed a letter notifying them of reversion to current law in 2016 but were not. Current SSA information confirms that the remaining 26 cases did not revert to current law in 2016.



There are several reasons why the subgroup definition based on characteristics and random assignment may not align with the Implementation Team information on reversion to current law:

• Timing differences. SSA determined reversion to current law in 2016 based on information available at that time and the research classifications are based on administrative data extracted two years later. If SSA made any retroactive updates in the intervening period, this could generate misalignment between the two classifications.

• Matching error. The BOND analysis sample includes beneficiaries entitled on multiple records. In such cases, implementation staff could identify the correct record via a case review of treatment subject data. Our algorithm, however, may select a record different than the one used to determine reversion to current law. This is because our algorithm applies hard-code rules to electronic records in an identical manner for all BOND subjects, not just treatment subjects.

• Errors in SSA administrative data files.28 In some cases, there may be errors in one or multiple SSA administrative data sources. All data sources may contain errors, and administrative data is no exception. Researchers have documented a variety of potential errors in administrative sources including processing errors and editing errors, among others (Groen 2012, Bakker 2009).

We defined the analysis groups based on TWP completion dates and not on Implementation Team information on reversion to current law because the latter source is not available for control subjects. That is, control beneficiaries are always subject to current law and hence do not have BPP end dates for reversion to current law. While we could have used the Implementation Team data to classify treatment subjects, that would have violated the principle of identifying treatment and control subgroups in an identical manner to ensure that differences in outcomes for the two groups provide unbiased estimates of impacts and are not confounded by differences in subgroup selection methodology.

28 SSA staff explained that they correct data errors as soon as they are identified. Indeed, SSA staff conducted case

reviews to better understand the source of misalignment between our research classification and the Implementation Team’s administrative classifications. SSA staff identified two cases with incorrect administrative data records, which they immediately updated.



Appendix C. Stage 2 Impact Estimates for Subgroups Defined by Duration of SSDI Receipt

Exhibit C-1. Estimated Impacts on 2016 Outcomes of the Offset Compared to Current Law (T21 Vs. C2) for Subgroups Defined by Duration of SSDI Receipt

Short Duration Long Duration

Outcome

Average Outcome

with Offset

and WIC (T21)

(1)

Average Outcome

under Current

Law (C2) (2)

Impact Estimate

(3)

Average Outcome

with Offset

and WIC (T21)

(4)

Average Outcome

under Current

Law (C2) (5)

Impact Estimate

(6)



Total earnings $5,367 $5,353 $14 ($429)

$5,215 $4,569 $647 ($444)

$-633 ($751)

Employment during year (%) 36.42 35.25 1.17 (1.29)

39.28 36.85 2.43 (1.56)

-1.26 (2.27)

Earnings above BYA (%) 12.72 11.35 1.37 (1.32)

13.29 10.18 3.10** (1.21)

-1.73 (1.97)

Earnings above 2x BYA (%) 5.63 6.03 -0.41 (0.83)

4.90 4.03 0.86 (0.74)

-1.27 (1.27)


1.94 1.64 0.30 (0.55)

-0.54 (0.66)

Benefit Outcomes (January–December 2016) Total SSDI benefits paid $12,726 $12,110 $616***

($189) $12,072 $11,469 $604**

($219) $12

($288) Number of months with SSDI

payments 10.39 9.91 0.48**

(0.15) 10.56 10.09 0.47***

(0.14) 0.01

(0.22) Total SSI benefits paid $39 $47 $-8

($15) $35 $38 $-3

($16) $-5

($24) Number of months with SSI

payments 0.23 0.23 -0.00

(0.04) 0.15 0.16 -0.01

(0.05) 0.00

(0.06) Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. Unweighted sample sizes: Short Duration T21 = 3,125, Short Duration C2 = 3,102, Long Duration T21 = 1,729, Long Duration C2 = 1,747. */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (and with no multiple comparisons adjustment). †/††/††† Difference in impact estimates is significantly different from zero at the .10/.05/.01 levels, respectively, using an F-test.



Exhibit C-2. Estimated Impacts on 2016 Outcomes of the Offset Compared to Current Law (T22 Vs. C2) for Subgroups Defined by Duration of SSDI Receipt


Outcome

Average Outcome

with Offset

and EWIC (T22)

(1)

Average Outcome

under Current

Law (C2) (2)

Impact Estimate

(3)

Average Outcome

with Offset

and EWIC (T22)

(4)

Average Outcome

under Current

Law (C2) (5)

Impact Estimate

(6)



Total earnings $5,533 $5,353 $180 ($352)

$5,139 $4,569 $570 ($386)

$-390 ($518)

Employment during year (%) 39.68 35.25 4.43** (1.75)

38.25 36.85 1.40 (1.83)

3.03 (2.34)


12.35 10.18 2.16 (1.19)

-0.77 (1.54)


4.62 4.03 0.59 (0.78)

-1.34 (1.03)


2.05 1.64 0.42 (0.68)

-0.59 (0.87)

Benefit Outcomes (January–December 2016) Total SSDI benefits paid $12,660 $12,110 $550**

($200) $11,922 $11,469 $453*

($241) $96


payments 10.39 9.91 0.48***

(0.12) 10.52 10.09 0.44**

(0.15) 0.04

(0.22) Total SSI benefits paid $26 $47 $-21*

($11) $44 $38 $5

($22) $-26 ($25)


0.18 0.23 -0.05 (0.05)

0.18 0.16 0.02 (0.06)

-0.07 (0.08)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. Unweighted sample sizes: Short Duration T22 = 1,914, Short Duration C2 = 3,102, Long Duration T22 = 1,127, Long Duration C2 = 1,747. */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (and with no multiple comparisons adjustment). †/††/††† Difference in impact estimates is significantly different from zero at the .10/.05/.01 levels, respectively, using an F-test.



Exhibit C-3. Estimated Impacts on 2016 Outcomes of the Offset Compared to Current Law (T22 Vs. T21) for Subgroups Defined by Duration of SSDI Receipt


Outcome

Average Outcome

with Offset

and EWIC (T22)

(1)

Average Outcome

with Offset

and WIC (T21)

(2)

Impact Estimate

(3)

Average Outcome

with Offset

and EWIC (T22)

(4)

Average Outcome

with Offset

and WIC (T21)

(5)

Impact Estimate

(6)



Total earnings $5,533 $5,367 $167 ($554)

$5,139 $5,215 $-76 ($357)

$243 ($802)

Employment during year (%) 39.68 36.42 3.26 (1.95)

38.25 39.28 -1.03 (1.95)

4.29 (2.67)


12.35 13.29 -0.94 (0.88)

0.96 (1.80)


4.62 4.90 -0.27 (0.82)

-0.07 (1.41)

Earnings above 3x BYA (%) 2.81 2.75 0.07 (0.54)

2.05 1.94 0.11 (0.40)

-0.04 (0.88)

Benefit Outcomes (January–December 2016) Total SSDI benefits paid $12,660 $12,726 $-66

($174) $11,922 $12,072 $-150

($196) $84


payments 10.39 10.39 -0.00

(0.09) 10.52 10.56 -0.03

(0.08) 0.03


($15) $44 $35 $8

($26) $-21 ($28)


0.18 0.23 -0.05 (0.07)

0.18 0.15 0.03 (0.05)

-0.07 (0.08)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. Unweighted sample sizes: Short Duration T22 = 1,914, Short Duration T21 = 3,125, Long Duration T22 = 1,127, Long Duration T21 = 1,729. */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (and with no multiple comparisons adjustment). †/††/††† Difference in impact estimates is significantly different from zero at the .10/.05/.01 levels, respectively, using an F-test.



Exhibit C-4. Estimated Impacts on 2016 Outcomes of the Offset Compared to Current Law (T22 + T21 Vs. C2) for Subgroups Defined by Duration of SSDI Receipt


Outcome

Average Outcome

with Offset

and WIC and EWIC

(T22 + T21) (1)

Average Outcome

under Current

Law (C2) (2)

Impact Estimate

(3)

Average Outcome

with Offset

and WIC and EWIC

(T22 + T21) (4)

Average Outcome

under Current

Law (C2) (5)

Impact Estimate

(6)



Total earnings $5,429 $5,353 $76 ($300)

$5,185 $4,569 $616 ($367)

$-541 ($543)

Employment during year (%) 37.63 35.25 2.39* (1.14)

38.87 36.85 2.02 (1.38)

0.36 (1.91)


12.91 10.18 2.73** (0.96)

-1.35 (1.47)


4.79 4.03 0.75 (0.62)

-1.29 (0.93)


1.98 1.64 0.35 (0.57)

-0.56 (0.54)

Benefit Outcomes (January–December 2016) Total SSDI benefits paid $12,701 $12,110 $591***

($165) $12,013 $11,469 $544**

($195) $47


payments 10.39 9.91 0.48***

(0.13) 10.54 10.09 0.46***

(0.12) 0.02


($10) $39 $38 $0

($15) $-13 ($20)


0.21 0.23 -0.02 (0.04)

0.16 0.16 0.01 (0.04)

-0.03 (0.04)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are in nominal 2016 dollars. Unweighted sample sizes: Short Duration T21+T22 = 5,039, Short Duration C2 = 3,102, Long Duration T21+T22 = 2,856, Long Duration C2 = 1,747. */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (and with no multiple comparisons adjustment). †/††/††† Difference in impact estimates is significantly different from zero at the .10/.05/.01 levels, respectively, using an F-test.



Appendix D. Comparison of 2015 and 2016 Impacts on Benefit Outcomes for the Full Sample

Section 4.2 of this report focuses on how reversion to current law affects earnings behavior. That section concludes that although there is some evidence that reversion to current law reduces the impact of the offset on earnings, the impact of reversion and the number of subjects affected by reversion are both small. Therefore, the full sample earnings impacts remain informative of the fifth or sixth year (depending on year of enrollment) of exposure to the benefit offset rules.

If the impacts on earnings-related outcomes did not change between 2015 and 2016, we would expect no changes in impacts on benefits-related outcomes between 2015 and 2016. Exhibit D-1 provides estimated impacts on benefits-related outcomes in the 2015 and 2016 calendar years. There is no statistically significant evidence of differences between 2015 and 2016 impacts on total benefits paid (SSDI or SSI), the proportion with at least one month of benefits payments (SSDI or SSI), and the number of months with any payments (SSDI). Out of 18 tests of changes in impact on benefit outcomes (three policy contrasts for six different outcomes), only a single test is statistically significant. That statistically significant test provides some evidence that reversion to current law reduced the impact on the number of months with SSI payments (p = 0.096) for the T21 versus C2 policy comparison in 2016.

Overall, we conclude that the 2016 benefit impacts estimated using the full sample are informative of the fifth or six year (depending on year of enrollment) of exposure to the benefit offset rules. Despite the single statistically significant result, the evidence that reversion to current law affected full sample impacts on benefits is very weak. First, the result is the lone statistically significant change in impact detected among the 18 changes examined and the probability of one of them being different from zero due to chance alone is high. Second, theory suggests that the offset should have a negative impact on the number of months of SSI payments. Therefore, a dampening of this effect due to reversion to current law would not be expected to push the impact in a more negative direction (i.e., the change has the “wrong sign”), as observed here. Finally, no impact estimate for number of months with SSI payments is statistically significant in any year, including 2016, in either Stage 1 or Stage 2 of BOND. 29

29 See Gubits et al. 2018b for impact estimates for the years 2011 to 2015.



Exhibit D-1. Estimated Impacts on 2015 and 2016 SSDI Benefits of Stage 2 Volunteers: All Policy Comparisons

Outcome

Average Outcome

with Offset and WIC

(T21) (1)

Average Outcome


(T22) (2)

Average Outcome

under Current

Law (C2) (3)



(4)



(5)

Estimated Impact of


T21) (6)

Total SSDI Benefits Paid 2015 $12,639 $12,668 $12,119 $520***

($138) $549*** ($153)

$29 ($193)

2016 $12,347 $12,232 $11,739 $608*** ($149)

$493** ($163)

$-115 ($165)

Change from 2015 to 2016 $-292 $-436 $-380 $88 ($84)

$-55 ($81)

$-144 ($85)

Number of Months with SSDI Payments (%) 2015 10.75 10.79 10.31 0.44***

(0.08) 0.48*** (0.09)

0.04 (0.09)

2016 10.49 10.46 10.01 0.47*** (0.09)

0.45*** (0.10)

-0.02 (0.10)

Change from 2015 to 2016 -0.26 -0.33 -0.29 0.03 (0.05)

-0.03 (0.05)

-0.06 (0.05)

Any Months with SSDI Payments 2015 92.00 92.34 88.49 3.51***

(0.67) 3.85*** (0.72)

0.34 (0.70)

2016 90.01 89.44 85.71 4.30*** (0.74)

3.73*** (0.83)

-0.57 (0.79)

Change from 2015 to 2016 -1.99 -2.90 -2.78 0.79 (0.45)

-0.12 (0.52)

-0.91 (0.58)

Total SSI Benefits Paid 2015 $38 $36 $38 $-0

($12) $-2

($12) $-2

($15) 2016 $37 $36 $42 $-5

($10) $-5

($13) $-0

($17) Change from 2015 to 2016 $-1 $1 $4 $-5

($8) $-3 ($9)

$2 ($6)

Number Months with SSI Payments (%) 2015 0.20 0.20 0.17 0.02

(0.04) 0.02

(0.04) -0.00 (0.05)

2016 0.18 0.18 0.19 -0.00 (0.03)

-0.01 (0.04)

-0.00 (0.04)

Change from 2015 to 2016 -0.02 -0.02 0.01 -0.03* (0.02)

-0.03 (0.02)

-0.00 (0.02)



Outcome

Average Outcome

with Offset and WIC

(T21) (1)

Average Outcome


(T22) (2)

Average Outcome

under Current

Law (C2) (3)



(4)



(5)

Estimated Impact of


T21) (6)

Any Months with SSDI Payments 2015 1.94 1.93 1.89 0.05

(0.38) 0.05

(0.39) -0.01 (0.39)

2016 1.98 1.76 2.04 -0.06 (0.33)

-0.28 (0.39)

-0.22 (0.46)

Change from 2015 to 2016 0.04 -0.18 0.15 -0.11 (0.26)

-0.33 (0.24)

-0.21 (0.17)

Source: SSA administrative records and data from the Stage 2 Baseline Survey. Notes: See Chapter 2 for variable definitions. Weights are used to ensure that the BOND subjects who met analysis criteria are representative of the national population of SSDI-only beneficiaries who would volunteer for study enrollment. Standard errors are in parentheses. Means and impact estimates are regression-adjusted for baseline characteristics. All dollar amounts are inflation-adjusted to 2016 dollars using the Consumer Price Index for Urban Wage Earners and Clerical Workers (CPI-W). Unweighted sample sizes: T21 = 4,854, T22 = 3,041, and C2 = 4,849 */**/*** Impact estimate is significantly different from zero at the .10/.05/.01 levels, respectively, using a two-tailed t-test with 9 degrees of freedom (and with no multiple comparisons adjustment).

bond implementation and evaluation: sixth-year snapshot of ......sep 11, 2019 · daniel gubits ....

Documents