Evaluating OSHA inspections for intended and unintended outcomes
Mike ToffelAssociate Professor
Harvard Business School
Liberty Mutual Research Institute for Safety
August 2012
1
OSHA, a much-criticized agency
Too lenient! (?)• Inspections too seldom• Penalties too small• Lengthy process to adopt new regulations
compromises worker safety
Too costly! (?)• Stifling job creation / job killer• Increasing labor costs• Eroding America's competitiveness
3
Challenges evaluating impact of OSHA
inspections:
Causality
Most OSHA inspections are not random: After accidents and deaths When employees complain
If accidents/deaths are rare events, outcomes will feature mean reversion: Problems likely decline after inspections…
but even without inspections
Our approach Examine random inspections and compare
to a control group.5
Several studies have relied on company logs But inspections can lead companies to
improve logs’ comprehensiveness, increasing reported injuries
This cloaks changes in actual injury rates
Our approach Rely on workers’ compensation claims Annual number of claims Annual cost of all claims
Challenges evaluating impact of OSHA
inspections:
Measuring outcomes
6
3 data sources
WCIRBNumber of claims
Cost of claimsOccupation
classesPayroll
OSHA IMISInspections
Dun & Bradstreet
IndustrySingle-plantEmployment
Sales
Data restrictions California Cal/OSHA Some high hazard industries (randomization
targets) Single-plant firms7
Data issuesWCIRB data• MOU between California Department of
Insurance (CDI) and Commission on Health and Safety and Worker’s Compensation listing us as subcontractors
• Required to keep data anonymous
OSHA data• Complex data structure• Limited documentation• Many conversations with Cal/OSHA to
understand implementation of randomization process8
Developing a matched sampleTreatments Single-plant establishments randomly selected for
a programmed inspection High-hazard industries Cal/OSHA targeted for
random inspection each year
Matched controls Find population of single-plant establishments at
risk of random inspection, but not selected Exclude if < 10 employees or recently
inspected For each treatment, select one control:
Same industry and region Closest sizeResult: 409 matched pairs
9
Matching led to a balanced sample of very similar treatments and controls
In the two years before the match year, the 409 matched pairs had very similar: Sales Employment Payroll Credit scores (PAYDEX, Comprehensive
Credit Appraisal) Annual number of WC claim Annual total cost of WC claims
10
Evaluation model
Did treatments experience a greater decline in annual injuries (or injury-related costs) after inspections than the controls, examined over the same time period? Fixed effects regression Control for establishment characteristics Difference-in-differences approach Compares two groups over time
14
Unanticipated consequences of inspections?
Inspections (and consequences) cause interruptions, but are they substantial? Sales impact? Credit worthiness? Employment, payroll? Firm survival?
21
Unanticipated consequences of inspections?
No difference in employment, payroll, sales (Table 2) Tight confidence intervals enable us to rule out
that inspections caused big declines of employment or payroll
No difference in credit ratings (Table S8) Late bills, etc. more sensitive to financial burden
than firm death Two D&B metrics of financial distress: PAYDEX &
CCA
No difference in firm survival (Table S7) Approx. 5% of treatments and of controls died
Difference not statistically significantResult robust to survival regressions
23
Summary of results
Evidence of intended results Annual injuries reduced by 9.4% Annual injury costs reduced by 26%
No evidence of unintended consequences Sales impact Credit worthiness Employment, payroll Firm survival
24
Conclusions
Cal/OSHA is not entirely toothless• Injuries and injury costs decline a lot• Social value of very approximately $350,000 per inspection• Includes medical costs and lost wages, but not
productivity loss, pain and suffering, etc.
• With many assumptions (similar effects across all states and inspection type, etc.) translates to $22 B per year nationwide
OSHA is not detectably a job killer• But we cannot rule out some costs
25
Levine, David I., Michael W. Toffel, and Matthew S. Johnson. "Randomized Government Safety Inspections Reduce Worker Injuries with No Detectable Job Loss." Science 336, no. 6083 (May 18, 2012)
Levine, David I., Michael W. Toffel, and Matthew S. Johnson. "Randomized Government Safety Inspections Reduce Worker Injuries with No Detectable Job Loss." Science 336, no. 6083 (May 18, 2012)
Future research in this domain
1. When do random inspections bolster regulatory compliance?
High vs. low hazardous industries
Strong vs. weak compliance history
VPP participants versus others
Prior OSHA consultation versus not
2. Do random inspections have spillover effects? Organizational: bolster compliance at corporate siblings?
Geographic: bolster compliance at neighboring facilities?
Regulated domains: bolster compliance with EPA regulations?
3. Your ideas? Loss prevention vs. OSHA inspections?
26
Other research…
Environmental reporting & carbon disclosure• Responding to public and private politics: Corporate
disclosure of climate change strategies, Strategic Management Journal (2009)
• Engaging supply chains in climate change. Working Paper (2012)
• When do firms greenwash? Corporate visibility, civil society scrutiny, and environmental disclosure, Working Paper (2012)
Environmental ratings• How well do social ratings actually measure corporate social
responsibility? Journal of Economics & Management Strategy (2009)
• How firms respond to being rated, Strategic Management Journal (2010)
27
Environmental policy evaluationVoluntary violations disclosure (EPA Audit Policy)• Coerced confessions: Self-policing in the shadow of the
regulator, Journal of Law, Economics and Organization (2008)• Coming clean and cleaning up: Does voluntary self-reporting
indicate effective self-policing, Journal of Law and Economics (2011)• Making self-regulation more than merely symbolic: The
critical role of the legal environment, Administrative Science Quarterly (2010)
Mandatory performance disclosure• How firms respond to mandatory information disclosure,
Strategic Management Journal (forthcoming)
Tailpipe emissions fraud• Competition and illicit quality, Working Paper (2012)• The role of organizational scope and governance in
strengthening private monitoring, Working Paper (2012)28
Quality management: ISO 9001 evaluation• Quality management and job quality: How the ISO 9001
standard for quality management systems affects employees and employers, Management Science (2010).
Occupational health and safety • Randomized government safety inspections reduce worker
injuries with no detectable job loss, Science (2012).
29
Thank you!
Prof. Michael ToffelHarvard Business School
Morgan Hall 497Boston, MA [email protected]
www.people.hbs.edu/mtoffel/
30