louisville, ky

Industry Briefing, April 2011

FMC-CSA-10-002

FMC-CSA-10-002

Louisville, KY

The Life Cycle of a Roadside Inspection

Violation(Part 2)

April 2013

Inspection Violation Lifecycle | April 20132

• Compliance, Safety, Accountability (CSA) Overview– Safety Measurement System (SMS), Enhancements and

Effectiveness• Safety Data on Drivers• Data Qs: the process for requesting review and correcting

inspection data• Interventions: designed to change carrier behavior• CSA Website Demonstration• 2013 Plans• Roundtable Q&A Session

Life Cycle Seminar (Part 2): Agenda


What is CSA?

• CSA is a program of the Federal Motor Carrier Safety Administration (FMCSA)

• FMCSA’s mission is to prevent crashes, injuries, and fatalities involving large trucks and buses

• CSA has not resulted in any new regulations

• CSA was designed, field-tested, and refined over five years prior to national launch in December 2010


CSA’s Three Core Components

1. A Safety Measurement System (SMS) that: – Identifies and categorizes unsafe carrier and driver behaviors that lead to

crashes– Uses all safety-based roadside inspection violations to assess compliance

with existing regulations– Includes investigation findings

2. A Safety Interventions process that:– Includes an array of interventions– Focuses on specific unsafe behaviors– Identifies causes of safety problems– Defines and requires corrective actions

3. A Safety Fitness Determination (SFD) process that:– Requires rulemaking, – Would be incorporate on-the-road safety performance in addition to

investigation findings


Safety Measurement System (SMS)

SMS is FMCSA’s workload prioritization tool that:

• Uses State-reported crash records and roadside inspection violations to identify carriers for FMCSA intervention– Previous system used only out-of-service and certain

moving violations• Uses 24 months of data; recent events are

weighted more heavily than older ones• Weights violations based on relationship to

crash risk


SMS (cont.)

• Calculates safety performance based on seven Behavior Analysis and Safety Improvement Categories (BASICs)

• Triggers the Safety Interventions process (e.g., warning letters, investigations)

• Will feed the new SFD process once rulemaking is completed

• Is designed to be continually improved as more information is available through data and analysis


SMS Methodology Overview

• Gather 24 months of on-road safety event data (i.e., inspections, crashes) to create a safety event history

• Place each carrier violation and/or crash into a BASIC

• Convert BASIC data into a quantifiable measure/rate– Time, severity, OOS weight

• Develop a percentile rank for each BASIC based on each carrier BASIC measure– Safety Event Groups

• Safety Event Data

• BASIC Data

• BASIC Measur

es• Percen

tile


BASICs focus on behaviors linked to crash riskPublic1. Unsafe Driving (Parts 392 & 397)2. Hours-of-Service (HOS) Compliance (Parts 392 & 395)3. Driver Fitness (Parts 383 & 391)4. Controlled Substances/Alcohol (Parts 382 & 392)5. Vehicle Maintenance (Parts 392, 393 & 396)

Not Public6. Hazardous Materials (HM) Compliance

(Federal Motor Carrier Safety Regulation Part 397 & HM Regulations (HMRs) Parts 171, 172, 173, 177, 178, 179, and 180)

7. Crash Indicator

SMS BASICs


• Carriers with BASICs above the threshold ( ) are prioritized for intervention

• Percentile ranks are determined by comparing carriers’ measures other measures of carriers with similar number of safety events

• Clean Roadside Inspections are used in SMS and can improve a carrier’s BASIC percentile ranks

• Motor carriers do NOT inherit any of a newly-hired driver’s past violations

• Only those inspections that a driver receives while driving under a carrier’s authority can be applied to that carrier’s SMS profile– Driver violations remain on the carrier’s SMS for two years,

even if the driver was fired

SMS (cont.)


SMS Screenshots

The following slides provide

examples of SMS results and data


Carrier Measurement: SMS Results


Violation Details Provided in the SMS


Further Drilldown in the SMS


Recent SMS Improvements


March 2012 • Motor carriers and enforcement staff previewed the

changes before implementation• Preview participants were able to compare/ contrast

current vs. proposed ― View impact on measures and percentile ranks

March 27, 2012 through July 30,2012 • Federal Register Docket open for comments about the

SMS Preview

December 2012• SMS public website updated with the SMS changes

discussed– Both logged-in users and the general public can see percentile ranks– Crash Indicator and the HM Compliance BASIC percentiles are available

to only logged-in users

Recent SMS Changes


1. Strengthened the Vehicle Maintenance BASIC by incorporating cargo/load securement violations from the Cargo-Related BASIC – Allows for appropriate workload prioritization while reducing a

bias in the Cargo-Related BASIC whereby flatbed operators are disproportionately identified for intervention

2. Changed the Cargo-Related BASIC to the HM Compliance BASIC to better identify safety problems related to HM– Removed load securement violations so that only HM violations

remain; consequences of crashes and cargo spills can be greatly exacerbated when HM are involved

SMS Improvements


3. Better aligned the SMS with IEP regulations– SMS now properly attributes each Intermodal Equipment Provider

(IEP) trailer violation to either the IEP or the motor carrier based on the ability of the driver to find the violation as part of a pre-trip inspection

– IEP violations assigned to a motor carrier are now used towards calculating the carrier’s Vehicle Maintenance BASIC

4. Aligned violations included in the SMS with CVSA inspection levels by eliminating vehicle violations derived from driver-only inspections and driver violations from vehicle-only inspections– Prompted by enforcement and industry concerns that some

vehicle violations fall outside the scope of a driver-only inspection and could bias the BASIC results

SMS Improvements (cont.)


5. More accurately identify carriers involved in transporting HM or passengers

These carriers are subject to more stringent thresholds in the SMS


HM CarriersNew definition enables FMCSA to focus resources on carriers involved in the majority of placardable HM transport• At least 2 HM placardable

vehicle inspections; 1 within past 12 months

• At least 5% total inspections indicated as HM placardable vehicle inspections

Passenger CarriersNew definition removes many low-capacity vehicles (e.g., vans and taxis) that are generally outside of FMCSA’s authority• Adds all for-hire carriers with

9-15 passenger capacity vehicles and private carriers with 16+ capacity

• Removes all carriers with only 1-8 capacity and private carriers with 1-15 passenger capacity


6. Modified the SMS Display to:– Replace the terms Insufficient Data and Inconclusive to fact-

based definitions– Break out crashes with injuries and crashes with fatalities

7. Removed 1 to 5 mph speeding violations– FMCSA has aligned speeding violations to be consistent with

current speedometer regulations (49 CFR 393.82) that require speedometers to be accurate within 5 mph

– Applies to the prior 24 months of data, and all future SMS data

8. Lowered the severity weight for speeding violations that do not designate a mph range above the speed limit to 1.



9. Aligned the severity weight of paper and electronic logbook violations (they are now weighted equally)

10.Changed the name of the Fatigued Driving (HOS) BASIC to the HOS Compliance BASIC– This BASIC continues to have a strong

association with future crash risk. – Change was made to reflect that the

BASIC includes HOS recordkeeping requirements that, by themselves, do not necessarily indicate fatigued driving or driving in excess of allowable hours.



Operating CMV while disqualified / CDL suspended (391.15A and 383.51A)• safety- or non-safety-based suspension • whether or not the carrier had the capacity to know

about the suspension

Suspended License Changes (ASPEN Change)


Effectiveness Tests

and Results


CSA Effectiveness

• FMCSA is continually monitoring the effectiveness of the CSA program

• FMCSA plans to have a complete CSA effectiveness report peer reviewed by an independent third party


Overall CSA Program Effectiveness

• The assessment of the CSA program effectiveness will build off and include each of the four elements 1. SMS Effectiveness2. Intervention Effectiveness3. Refresh of independent evaluation

effectiveness measures4. New effectiveness measures

• Results will be refreshed at regular intervals to assess the effectiveness over time


1. SMS Effectiveness Report

• Objective– Quantify how effectively the SMS identifies high-risk

motor carriers • Methodology

– Use historical data to examine the future crash rate of motor carriers

• Results will be published following peer review


Effectiveness Results

Carriers meeting the above definition of “high risk” have substantially higher crash rates: more than twice the crash rate

of the general carrier population

Group # Carriers

Post Crashes per 100 Post PU

Carriers under FMCSA's Jurisdiction 276,855 3.58Carriers with Sufficient Data for 1+ BASICs (SMS 3.0) 161,555 3.82Carriers with Above Threshold in 1+ BASICs (SMS 3.0) 41,789 5.05High-Risk Carriers 6,731 8.15

Crash OR HOS Compliance OR Unsafe ≥ 85

+1 other BASIC at or above

the “all other” motor carrier threshold

ORAny 4 or more BASICs at or above the “all other” motor

carrier threshold


Effectiveness Results (cont.)

• Unsafe Driving, HOS Compliance, Vehicle Maintenance BASICs, and the Crash Indicator are the best predictors of future crash risk

• Other BASICs show a weaker relationship to crash risk• FMCSA optimizes resources through more stringent

Intervention Thresholds for BASICs with strongest associations to crash risk

Crash rates of Carriers above and below BASIC thresholds

BASIC

Above Threshold :Crashes per

100 PU

Below Threshold:

Crashes per 100 PU

Increase in Crash Rate

Unsafe Driving 7.10 3.90 82%HOS Compliance 6.97 4.00 74%Driver Fitness 2.85 4.43 -36%Controlled Substances / Alcohol 2.81 5.25

-47%

Vehicle Maintenance 5.79 3.87 50%HM Compliance 5.27 4.04 31%Crash 6.59 3.58 84%1+ BASIC (any BASIC) 5.05 3.05 66%


Effectiveness – HOS Compliance BASIC

• Strong relationship between HOS Compliance BASIC and future crash risk

• Recent American Transportation Research Institute (ATRI) paper shows similar findings


Effectiveness – Driver Fitness BASIC

• Negative relationship between Driver Fitness BASIC and future crash risk• Recent ATRI paper shows similar findings• Three quarters of carriers above Driver Fitness, exceed threshold in another BASIC• Recent Aspen improvements provide for higher severity weights for safety-based

“operating while suspended” violations


SMS Effectiveness - ATRI Report Summary

• Findings:– Positive relationship between Unsafe, HOS, and

Vehicle BASICs and crash risk.– Negative relationship between Driver Fitness and

Controlled Substance/Alcohol BASICs and crash risk.– The more BASIC percentiles at ‘alert’, the higher the

crash risk.• Recommendations:

– Rank carriers by number of BASICs at ‘alert’.– Present profiles on the CSA website based on number

of ‘alerts’.


2. CSA Intervention Effectiveness

• CSA Intervention Effectiveness Model is currently under development (replaces CREM)

• Calculate pre vs. post intervention crash rates and compares to control groups

• Measures crashes avoided, injuries prevented, and lives saved, as a result of interventions in a given year


3. Refresh of Effectiveness Metrics

• The independent evaluation by UMTRI of the CSA Operational Model Test will be updated based on national implementation of the CSA program– Not all measures will be updated ; dependent on the

availability of data– Will apply lessons learned and improve methodology

• Example measures to be updated based on national implementation:– Crash rates over time for the motor carrier population– Crash rates by motor carrier’s BASIC performance– Intervention cycle analysis – what interventions have

motor carrier’s received since implementation– Estimating benefits of various interventions


4. New Effectiveness Measures

• New effectiveness measures will be developed to evaluate the CSA program

• The following may be potential new measures:– Further break out existing measures by carrier operation

type (e.g. Hazardous Material or Passenger Carriers)– Violation rates over time

• By BASIC performance• Before and after interventions

– BASIC measure trends over time– Measures looking at the volume of traffic viewing SMS

websites


CSA Stats• Since CSA rollout (Dec 2010), most dramatic decrease in

violation rates in a decade– violations per roadside inspection are down nearly 14

percent and – driver violations per inspection are down 17 percent

• SMS identifies high-risk carriers have– twice national crash rate – 56% more crashes than carriers identified using SafeStat

• SMS Online hosted 48 million user sessions in 2012– Ten fold increase over SafeStat and 60% increase since last year

• The SMS has enough performance data to evaluate nearly 40% of active carriers (these carriers are involved in >92% of reported crashes)

• As of Dec. 2012, FMCSA sent more than 70,000 warning letters


What about safety data on drivers?


FMCSA uses violations reported to Agency on an inspection report.

During the inspection, inspector may also choose to issue a State warning/citation/ticket for the infraction.

Driver Safety Data


1. State Motor Vehicle Record (MVR)

2. Federal Pre-Employment Screening Program (PSP)

Driver Safety Data: Two Distinct Records


1. State Motor Vehicle Record (MVR)– Same system as before, not impacted by CSA– Includes State conviction data– States (not Federal Government) may

suspend CDL based on conviction data accumulated on MVR

– Employers required by Federal regulations to check annually (49 CFR 391.25)

– To acquire this report contact the State Department of Motor Vehicles that issued your CDL

Driver Safety Data - State


2. Federal Pre-Employment Screening Program (PSP) Report

– Congress mandated program independent of CSA – Voluntary screening tool that allows motor carriers and

drivers to purchase Federal inspection and crash history• 5 years reportable crashes• 3 years inspections – includes violations recorded on

inspections and also clean inspections– Carrier access to report requires driver consent– Driver can acquire his/her report anytime for a $10 fee

at www.psp.fmcsa.dot.gov or for free via FOIA request– PSP does not directly impact a driver’s CDL– No scores, points, ratings or assessments under PSP

Driver Safety Data - Federal

http://www.psp.fmcsa.dot.gov/


Carriers Evaluating Drivers– Carriers and independent third parties

aggregate PSP and other driver data and refer to it as a “CSA score” or “CSA driver scorecard”

– Important to understand that this evaluation is coming from the carrier, NOT the Federal/State Government.

– There is no universal evaluation methodDrivers should ask their carriers

how they are being evaluated

Driver Safety Data - Non-Government


• State Data (citation/tickets) State Court– Challenge citation data in State court, otherwise data goes on driver’s MVR,

possibly impacting your CDL

• Federal Data (recorded violations) DataQs– Drivers can dispute Federal data on their PSP via DataQs– Visit DataQs Website: https://dataqs.fmcsa.dot.gov/ (Register for free)– Follow the instructions on the site to file a request for data review (RDR) on an

inspection and/or reportable crash– Must provide supporting documentation when filing an RDR– RDRs are handled by the State where the inspection/crash occurred

• If a driver receives a Federal violation and a State citation/ticket for the same infraction:– The driver must dispute them separately using the appropriate method above– Removal of a violation/citation under one system DOES NOT guarantee

automatic removal from the other

Disputing Driver Data

• 46

https://dataqs.fmcsa.dot.gov/

https://dataqs.fmcsa.dot.gov/


Driver Safety Measurement System (DSMS)

• FMCSA’s internal tool to address the safety of CMV drivers:– Provides enhanced information on individual drivers

to FMCSA investigators to identify drivers with safety problems

– Allows for prioritizing driver sampling during carrier investigation

– Supports investigator follow-up on significant violations (e.g., Red Flag Violations)

– DSMS data are not available to the public or to employing motor carriers; they are used only by FMCSA as an investigative tool

• Red Flag Violations:– Reviewed during a carrier investigation

• Under CSA, individual drivers are not assigned safety ratings or safety fitness determinations


DataQs

Mike Wilson (ISP)


Safety Interventions


The Safety Interventions process addresses the:

• WHAT Discovering violations anddefining the problem

• WHY Identifying the cause or where the processes broke down

• HOW Determining how to fix it/prevent it by using the Safety Management Cycle (SMC)

Safety Interventions Process


CSA Intervention Tools

FOLLOW-ONCORRECTIVE

ACTIONSCSPNOV

NOC/OOS

WARNING LETTER INVESTIGATION

INTERVENTIONSOffsite

Focused OnsiteComprehensive

OnsiteSafety Mgmt Cycle

TARGETED ROADSIDE

INSPECTION

Safety Measurement BASIC Scores

Increasing Severity

• .

• .• .

• .

• .


CSA Warning Letter

Warning Letter


•Any BASIC exceeding the threshold (percentile)

OR•Any BASIC with a Serious Violation on the most recent investigation

OR•BASICs associated with a complaint

BASICs Requiring Investigation


CSA Carrier Investigations

• Shifted from 1 Compliance Review (CR) to 3 Investigation Options– Offsite– Focused Onsite– Comprehensive Onsite

• Focused on BASICs requiring investigation

INVESTIGATION INTERVENTIONS

Offsite Focused OnsiteComprehensive



Shifting from What to What, Why, and How

• Perform investigation to identify violations and corresponding process breakdowns in the Safety Management Cycle

• Identify and discuss recommended remedies to improve the carrier’s Safety Management Processes

CSA Investigation Process

INVESTIGATION INTERVENTIONS

Offsite Focused OnsiteComprehensive



Safety Management Cycle

What Safety Management Processes are breaking down?

Why are the Safety Management Processes breaking down?


Safety Management Cycle Assessment

• What is the

violation?

• What Safety

Management process is broken/miss

ing?

• Why is the process breaking down?

• How can the carrier

best address these

breakdowns?

Part B. Violations

Part B. Safety Management Breakdowns

Part B. Description of

Process Breakdown

Part B. Remedies

• Policies and Procedures• Roles and Responsibilities• Qualification and Hiring• Training and

Communication• Monitoring and Tracking• Meaningful Action

• Implement Safety Improvement Practices• Seek out Resources: Local

Resources/ Associations• Understand Why

Compliance Saves Time and Money• Apply Adequate Resources• Document and Follow

through on Action Plans


• An onsite in-depth examination of a carrier’s recent reportable crash events

• Concentrates on drivers and vehicles involved in crashes

• CBI Crash Analysis Toolkit

Crash BASIC Investigation (CBI)


CSA Follow-On Corrective Actions

• Notice of Claim (NOC)/ Operations out-of-service order (OOS)

• Notice of Violation (NOV)

• Cooperative Safety Plan (CSP)

FOLLOW-ON CORRECTIVE

ACTIONS

CSPNOV

NOC/OOS


Rating Process

Existing Safety Rating Process:• Defined in regulations: Part 385• Based on the results of an onsite investigation• Critical / Acute violations impact factors

• Comprehensive Onsite Investigations– May result in Satisfactory, Conditional, Unsatisfactory

• Focused Onsite Investigations– May result in Conditional or Unsatisfactory


Proposed Safety Fitness Determination (SFD):• Would require rulemaking• Would continue to include major safety violations

found as part of investigations• Would also incorporate on-road safety performance• Considerations:

– Absolute (measure) vs. Relative performance (percentile)– Use of crash data – Safety event groups– Due process

SFD

Draft rulemaking is currently under development


Website Demonstration http://csa.fmcsa.dot.gov/resources

http://csa.fmcsa.dot.gov/resources


• Enhancements to public display of SMS data

• Analyzing the impact of proposed changes

• Crash Weighting Plan• Safety Fitness Determination (SFD) –

Notice of Proposed Rulemaking

2013 Plans


• Enhancements under development• Objectives: – Reinforce SMS as prioritization – Recognize distinctions between BASICs and

crash risk– Focus on setting foundation for SFD– Reinforce safety mission – One stop shop– Address concerns with “insufficient data” carriers – Retain detailed data

• Proposed enhancements would be previewed

SMS Public Display Enhancements


Potential future Improvements to SMS

What kind of changes to SMS are under examination?

• Comprehensive modifications to roadside violation severity weights• Recalibration of the Utilization Factor used to

incorporate VMT for the Crash and Unsafe Driving BASICs• Adjustment to safety event groupings in all

BASICs


Recap of Improvements to SMS and broader CSA

What kind of changes to SMS are being considered for the next round of proposed improvements?

• Comprehensive modifications to roadside violation severity weights per MCSAC• Recalibration of the Utilization Factor used to

incorporate VMT for the Crash and Unsafe Driving BASICs• Adjustment to safety event groupings in all

BASICs• Other?


Crash Weighting Study

• SMS currently uses all crashes which has shown to be one of the better predictors of future crash risk

• FMCSA is conducing research study to better understand the safety benefits of adjusting crash weights in SMS based on the motor carrier’s role in the crash (i.e. preventability)

• Analysis complete summer 2013


Crash Weighting – Three Research Questions

1. Do police accident reports (PARs) across the nation provide sufficient, consistent, and reliable information to support crash weighting determinations? 2. Will a crash weighting determination process offer an even stronger predictor of crash risk than crash involvement and how will crash weighting be implemented into SMS? 3. How would the Agency manage the process for making crash weight determinations including public input to the process?


Crash Weighting – Benefit Analysis Approach

• Conduct literature review to examine similar studies

• Create several carrier based and driver based models to determine if weighted crashes are a better predictor of future crash risk than crash involvement based on available data

• Optimize the SMS Crash Indicator by using the SMS effectiveness test


Crash Weighting

• Complete plan is available at:https://csa.fmcsa.dot.gov/Documents/CrashWeightingResearchPlan_7-2012.pdf

https://csa.fmcsa.dot.gov/Documents/CrashWeightingResearchPlan_7-2012.pdf




• CSA Information Line: 877-254-5365• Ask a CSA-related question or give feedback

electronically, https://csa.fmcsa.dot.gov/CSA_Feedback.aspx

• Find the phone number for your local FMCSA office, http://www.fmcsa.dot.gov/about/contact/offices/displayfieldroster.aspx

Questions?

https://csa.fmcsa.dot.gov/CSA_Feedback.aspx

https://csa.fmcsa.dot.gov/CSA_Feedback.aspx

http://www.fmcsa.dot.gov/about/contact/offices/displayfieldroster.aspx

http://www.fmcsa.dot.gov/about/contact/offices/displayfieldroster.aspx


Thank you!


Roundtable Discussion

louisville, ky

Documents

safety problemsdefines

safety interventions

roadside inspection

safety event history

road safety performance

road safety event data

carrier violation andor

basic convert basic