VALIDATION OF THE INDIANA RISK ASSESSMENT SYSTEM

FINAL REPORT

Edward Latessa Ph.D.

Principle Investigator

Brian Lovins, Ph.D.

Project Director

Matthew Makarios, Ph.D.

Research Associate

University of Cincinnati

School of Criminal Justice

Center for Criminal Justice Research

PO BOX 210389

Cincinnati, Ohio 45221

April 2013

ii

EXECUTIVE SUMMARY

This report outlines the validation of the Indiana Risk Assessment System.

Indiana has taken several steps to ensure that the implementation of evidence based interventions

is translated to practice, starting with the expectation that local jurisdictions use a validated risk

and need assessment to help guide decisions. Overtime, each segment of the criminal justice

system has developed standards regarding the use of validated risk and needs assessments.

Because these standards varied among agencies, local jurisdictions have implemented different

assessment tools. Many jurisdictions have adopted a validated risk assessment like the Level of

Service/Case Management Inventory, while others have chosen to use other assessments or

relied on staff’s professional discretion.

In order to improve coordination among the criminal justice agencies and reduce

duplication of efforts, the Indiana Risk Assessment Task Force was formed to explore options

for a statewide risk assessment. The Task Force ultimately recommenced that the Judicial

Conference of Indiana and the Department of Correction adopt the Ohio Risk Assessment

System (ORAS). Benefits that led Indiana to adopt the ORAS included:

Developed on a Midwest population

Expands as the offender moves deeper into the system

Builds upon prior assessment

Ability to norm and validate on local population

Public domain

Prospective data collection

User friendly, easily implemented

Indiana contracted with the University of Cincinnati, Center for Criminal Justice Research to

implement a risk assessment system that provided assessments at multiple points in the criminal

justice system and that was validated on an Indiana population. A major goal of the project was

to develop assessments that abided by the principles of effective classification by constructing

iii

assessments that 1) separated Indiana offenders into risk groups based on their likelihood to

recidivate, 2) identified dynamic risk factors that can be used to prioritize programmatic needs,

and 3) identify potential barriers to treatment.

The Indiana Risk Assessment System was validated using a prospective design that

involved conducting in-depth structured interviews of almost 1,000 offenders that were on

community supervision or were preparing to be released from prison. After interviews were

conducted, offenders were tracked for nearly 2 years to gather follow-up information on

recidivism. Three assessment instruments were validated in this process: the Community

Supervision Tool, the Community Supervision Screening Tool and the Prison Reentry Tool.

Validation involved examining the predictive power of the assessment instruments. The

results reveal that all assessment instruments are able to significantly distinguish between risk

levels. Further, the strength of the correlation is acceptable for revalidation purposes, although

the instruments had more difficulty distinguishing between moderate and high risk females.

Given the information, the authors provide some suggestions regarding revisions to cut points on

the risk assessment instruments.

iv

TABLE OF CONTENTS

ACKNOWLEDGEMENTS………………………………………………………………….. ii

EXECUTIVE SUMMARY....................................................................................................... iii

INTRODUCTION.................................................................................................................... 3

The Principles of Effective Classification………………………………………………. 3

The Advantages of a Risk Assessment System…………………………………………. 5

METHODS ………………………………………………………………………………….. 6

VALIDATION RESULTS.………………………………………………………………….. 10

The Community Supervision Tool ……………………………………………………... 10

The Prison Reentry Tool ……………………………........................................................ 19

SUMMARY AND CONCLUSIONS………………………………………………………... 25

REFERENCES………………………………………………………………………………. 26

5

INTRODUCTION

In 2008, the Indiana Judicial Center, as staff agency for the Indiana Risk Assessment

Task Force, contracted with the University of Cincinnati, Center for Criminal Justice Research to

implement a risk and needs assessment system that improved consistency and facilitated

communication across criminal justice agencies. The goal was to implement risk/needs

assessment tools that were predictive of recidivism at multiple points in the criminal justice

system for offenders in Indiana. Specifically, assessment instruments were to be implemented

for community supervision and at the time of institutional reentry. The University of Cincinnati

worked with Indiana to implement two risk assessment instruments that were initially

constructed and validated on a sample of Ohio offenders: the Community Supervision Tool and

the Prison Reentry Tool. To ensure that these instruments were predictive of recidivism for

offenders in Indiana, the current research seeks to revalidate these instruments on two samples of

Indiana offenders. Indiana also adopted two other instruments developed by the University of

Cincinnati as a part of this assessment system: the Pre-trial Tool and the Prison Intake Tool.

A major goal of the assessment system is to conform to the principles of effective

classification. In doing so, Indiana seeks to encourage practitioners to efficiently allocate

supervision resources and structure decision-making in a manner that reduces the likelihood of

recidivism. As a result, the Indiana Risk Assessment System was implemented to classify the

risk level of offenders in the system while also identifying both criminogenic needs and barriers

to programming.

The Principles of Effective Classification

Several recent studies of correctional programming in Ohio suggest that the effectiveness

of both residential and community based programs are mitigated by the risk level of the clientele

6

that they serve. For example, in 2002, Lowenkamp and Latessa evaluated the effects of

Halfway Houses and Community Based Correctional Facilities and found consistently higher

effect sizes for offenders who were moderate to high risk. Similar results were found for

Community Corrections Act funded programs. Lowenkamp and Latessa (2005) found that

correctional interventions in the community that targeted higher risk offenders produced

significantly lower rates of recidivism than programs that did not.

The Ohio studies suggest that it is important to utilize risk assessment instruments in

order to efficiently allocate resources in a manner that reduces recidivism. The principles of

effective classification have been developed to guide criminal justice agencies in the use of risk

assessment systems. In short, the principles of effective classification suggest that programs

should use actuarial assessment tools and identify dynamic risk factors, especially in high risk

offenders. The four major principles of effective classification are: the risk principle, the needs

principle, the responsivity principle, and the professional discretion principle (Andrews, Bonta,

& Hoge, 1990).

The risk principle suggests that correctional agencies should assess the likelihood of

recidivism for the offenders that they manage and use this information to allocate resources.

Correctional interventions are most effective when the intensity of the intervention is matched to

the risk level of the clientele (Andrews, Bonta, & Hoge, 1990; Lowenkamp, Latessa, &

Holsinger, 2006; Van Voorhis, 2009). The most intensive programs should be allocated to

moderate and high risk cases, while low risk cases be allocated little if any programming.

Practically, the risk principle suggests that the majority of supervision and treatment resources

should be reserved for the highest risk cases. In fact, some research indicates that when low risk

cases are targeted with intensive programs they actually perform worse than those who were left

7

alone. This is because programming can expose offenders to higher risk cases and disrupt pro-

social networks (see Lowenkamp & Latessa, 2004).

A consistent finding in correctional programming is that the most effective programs

target dynamic risk factors (Andrews et al., 1990; Lowenkamp, Latessa, & Smith, 2005;

Lowenkamp & Latessa, 2004). Dynamic risk factors are criminogenic needs that when changed

have been shown to result in a reduction in recidivism. Dynamic risk factors can include

substance abuse, personality characteristics, antisocial associates, and antisocial attitudes (for a

review see Gendreau, Little, & Goggin, 1996). The needs principle suggests that effective

classification systems should identify dynamic risk factors to prioritize programmatic needs.

The responsivity principle focuses on identifying barriers to treatment (Andrews, Bonta,

& Hoge, 1990; Van Voorhis, 2009). Although dynamic risk factors are directly related to

recidivism, there are other issues that are likely to keep individuals from engaging in treatment.

Some examples of responsivity factors include intelligence, reading ability, language barriers,

and cultural barriers. If left unaddressed, it is likely that these influences can interfere with the

completion of treatment and as a result indirectly prevent a reduction in recidivism from

occurring. Thus, effective classification systems should also gather information on potential

barriers to the successful completion of correctional interventions.

Although risk assessment instruments remove a degree of professional discretion from

criminal justice actors, it is important to emphasize that the judgment of practitioners should not

be overlooked (Andrews, Bonta, & Hoge, 1990). The principle of professional discretion

recognizes that case managers and counselors are responsible for processing the risk, need, and

responsivity information and making decisions based on the information provided (Andrews,

Bonta, & Hoge, 1990). Further, actuarial tools are designed to treat offenders in the aggregate

8

and cannot be structured to anticipate every possible case or scenario. As a result, it is important

to allow criminal justice personnel the ability to override the assessment instruments in specific

circumstances. It is also important that overrides be used on a limited percentage of cases and

that measures are taken to oversee the override process.

The Advantages of a Validated Risk Assessment System

For over a decade, many criminal justice agencies have been implementing standardized

risk classification instruments in order to efficiently and effectively manage their target

populations. Because assessment instruments are expensive to construct and validate, resource

constraints often limit the development of risk assessment instruments for specific jurisdictions

and populations (Jones, 1996). As a result, many criminal justice agencies often use empirically

derived tools that that have been developed on samples from a different population. Although

this is less cost restrictive, it assumes that the instrument is a valid predictor of recidivism for

each agency’s specific population (Wright, Clear, & Dickerson, 1984; Jones, 1996; Gottfredson

& Moriarty, 2006). Also, it is likely that there are different populations of offenders within

jurisdictions. For example, the population of defendants on community supervision is likely

different than the population of individuals who are in prison. Given that it is unlikely for a

single instrument to have universal applicability across various offending populations, there is a

clear necessity to validate risk assessment instruments to each specific target population (Wright,

Clear, & Dickerson, 1984). The Indiana Risk Assessment System (IRAS) was thus designed to

predict the recidivism of Indiana offenders who were either under community supervision or

incarcerated in a state correctional institution. Two separate instruments were implemented that

were previously constructed and validated on Ohio offenders: the Community Supervision Tool

(CST), and the Prison Reentry Tool (RT) (see Latessa et al., 2009). The current project

9

examines the predictive validity of these tools on populations of offenders involved in the

criminal justice system in Indiana. The pretrial tool and the prison intake tool were not included

in this validation for two reasons. First, the pretrial population was difficult to access and given

their non-adjudicated status questions pertaining to their current offense was not available. As for

the prison intake tool, the follow-up time frame was planned for a much shorter period of time

which would not have given an adequate amount of time for the offenders at prison intake to be

released into the community.

Implementing two separate instruments at different stages in the criminal justice system

allows for the accurate assessment of risk with two separate populations of offenders: those on

probation and those being released from prison. Doing so will help agencies categorize

offenders on their caseloads by risk level and identify criminogenic needs for case management

for two distinct groups of offenders involved in the Indiana criminal justice system.

Another advantage of implementing a standardized assessment tool across Indiana allows

consistency in the assessment of risk across jurisdictions. Counties in Indiana were using

different methods of assessment, creating a great deal of variation in the practices for assessing

the risk and needs of offenders. Therefore, one of the purposes of IRAS was to promote

consistent and objective assessment of the risk of recidivism for offenders in Indiana.

METHODS

A prospective design was utilized in the validation of the community supervision and

prison reentry tool in Indiana. To accomplish this, offenders who were either on probation or

within 6 months of release from prison in Indiana were assessed with the respective instrument

and subsequently followed for about two years to gather official measures of recidivism. The

validation of the instruments had three phases: planning, data collection, and data analysis. The

10

planning phase involved planning meetings with research and Indiana staff regarding the logistic

obstacles to gaining access to cases and data collection sites. It also involved Institutional

Review Board approval and training of data collectors in the administration of the semi-

structured interview. The planning phase occurred throughout the latter half of 2008.

The data collection phase involved site visits to all pilot sites and the administration of

the risk assessment instruments on two samples of offenders, the community supervision sample

and the prison reentry sample. In all, complete data for 988 cases was gathered from 28

locations. This process occurred from April to July 2009. Outcome measures were gathered

through May 2011, providing an average of a 23.6 month follow-up for recidivism.

The validation and data analysis phase of the project for IRAS began in Fall 2011. It

involved data cleaning and analyses to examine the predictive validity of the instruments on their

respective samples. Validation involved examining the extent to which the instruments were

able to define groups of offenders that were low, moderate, and high risk to reoffend.

Data Collection

Data collection involved the administration of the risk assessment tools using self-report

questionnaires, semi-structured interviews, and file reviews. The self-report questionnaire

consists of approximately 20 questions that offenders fill out on their own. It takes

approximately 5-10 minutes for offenders to complete the self-report questionnaire. The semi-

structured interview is completed by trained personnel and takes approximately 45 to 60 minutes

to complete.

Data collection teams consisted of trained research assistants from the University of

Cincinnati. Depending on the size of the pilot site and the availability of spare rooms, the

research staff size varied from 3 to 13 staff members. Each staff member was trained in the use

11

of the data collection instrument, ethics involving research with human subjects under

correctional control, interview procedures, and interview skills. In addition to training, each

interviewer was supervised for the first 4 interviews and interviews were randomly observed by

team leaders throughout the project.

Table 1 presents the counties and institutions where data was collected. The pilot sites

for the project were selected with the considerations of geographic representation across the

state, recommendations from Indiana staff, and whether the site was available and willing to

participate during the data collection process.

Table 1: Pilot Counties/Institutions that Participated in the Validation of IRAS

Community Supervision Sample: Counties

Allen Jay Grant Posey

Bartholomew Jasper Greene Tippecanoe

Clark Marion Henry Vanderburgh

Elkhart Montgomery Howard Vigo

Floyd Noble

Prison Reentry Sample: Correctional Facilities

Branchville Correctional Facility Miami Correctional Facility

Henryville Correctional Facility New Castle Correctional Facility

Indiana Women’s Prison Plainfield Correctional Facility

Indiana State Prison Outside Rockville Correctional Facility

Madison Correctional Facility Westville Correctional Facility

To facilitate participation from the numerous pilot sites, letters were sent that informed

the selected sites of the project goals. Potential sites were also asked to both facilitate access to

the cases and provide a physical location to conduct the interviews. Although there were some

logistical and scheduling issues that arose at several sites, no site declined to participate in the

project. Eighteen counties participated with the data collection for the Community Supervision

12

Tool, and 10 correctional facilities participated for the Prison Reentry Tool. Overall, data

collection on the initial instrument and then follow-up lasted between April 2009 and June 2011.

Participants

Two independent samples were gathered at different stages in the criminal justice system.

Samples were taken of offenders on community supervision and in prison. Table 2 presents the

number of cases in each sample. There were a total of 988 cases in both samples, 626 in the

community supervision sample, and 362 in the prison reentry sample.

Table 2: Number of Cases in Each Sample

Sample N

Community Supervision 626

Male 390

Female 236

Prison Reentry 362

Male 213

Female 149

Total 988

Interviews for the community supervision data were conducted between May 2009 and

July 2009. To be included into the community supervision sample, individuals had to be an adult

charged with a criminal offense that was recently referred to community supervision services

during the period of data collection at each site. Possible participants were identified at each site

and these individuals were approached by site staff and asked if they would be willing to meet

with the research staff. Once the individual met with the research staff and the project was

explained, individuals were asked to participate in the research process and to sign informed

consent documents.

Interviews were conducted for the prison reentry sample from April through July 2009.

Individuals were selected for the prison reentry sample if they: a) were unrestricted by security

concerns (e.g. solitary), b) agreed to be interviewed, and c) were within six months of release.

13

Due to the restrictive nature of a secure correctional facility, individuals were issued movement

passes prior to the arrival of the research staff. However, since the research was voluntary, the

pass may not have been granted if it interrupted school or job duties, if the inmate declined the

pass, or for security reasons. Once the research staff and inmates met, the project was explained,

participation was requested, and informed consent obtained.

Recidivism

The primary measure of recidivism for this study was arrest for a new crime. Although

data was gathered regarding a variety of other potential outcome measures (e.g. conviction),

arrest was used for two major reasons. First, measures that gather information later in the

criminal justice process, such as convictions, require a longer follow-up period and have lower

base rates than arrest. Second, using arrests in the community as an outcome allows the

assessment tools to identify criminogenic needs that are likely to result in danger to the

community. Although factors that are predictive of rule violations (e.g. probation violations or

institutional violations) are of concern to criminal justice personnel, of most concern is targeting

factors that are related to criminal behavior.

The follow-up time period for all cases was the time between the date of the interview

and June 1, 2011. The follow-up period ranged from 22 to 26 months, with an average of 23.6

months. Follow-up data on arrest was collected using data provided from the Federal Bureau of

Investigation. The FBI gathers a variety of agency level arrest data across the nation. Offenders

in the samples whose names and dates of births matched those in the Indiana arrest data during

the follow-up period were marked as having recidivated.

14

RESULTS

This section describes the samples and validation results for the Community Supervision

Tool and the Prison Reentry Tool. Also presented for each tool is information regarding

priorities in case management by presenting risk levels by domain.

The Community Supervision Tool (CST)

Initial data for the community supervision sample was gathered through site visits to

county probation offices. The CST is designed to assist in both the designation of supervision

level as well as to guide case management for offenders in the community. The community

supervision sample consisted of 626 individuals who were on community supervision in Indiana.

Table 3 presents descriptive statistics for the community supervision sample.

Table 3: Descriptive Statistics for the Community Supervision Sample (n = 626)

Variable N Percent

Sex

Male 390 62

Female 236 38

Race

White 542 87

African American 63 10

Other 16 2.2

Not reported 5 .8

Any New Arrest

Yes 159 25

No 467 75

Average Range

Months at Risk 23.2 22 – 24

(0.7 SD)

Age 33.6 18 – 83

(11.4 SD)

15

Table 3 indicates that 25 percent of offenders in the sample were rearrested and that there

was an average of 23 months at risk. Sixty-two percent of the cases in sample were male and 87

percent were white. The average age of offenders in the sample was 33.6 years.

Figure 1 provides a visual display of the distribution of cases on scores for the CST by

gender. The figure suggests that the distribution of the sample on the CST approaches normality,

with the range from 1 to 37 and the large majority of cases falling near the center of the

distribution. Worth noting, is that the distribution of the sample is slightly skewed right and that

the high end of the scale has a relatively low number of cases, particularly the female sample.

Figure 1: Distribution of Cases on the Community Supervision Tool

(Male n = 390; Female n = 236)

0

5

10

15

20

25

30

1 5 10 15 20 25 30 35 37

Male Female

16

Table 4: Failure Rates by Risk Score for the Community Supervision Tool (n=626)*

Risk Score Total Cases Percent Arrested Risk Score Total Cases Percent Arrested

0 0 – 26 14 36

1 0 – 27 8 25

2 1 0 28 7 43

3 0 – 29 7 43

4 4 0 30 8 50

5 10 10 31 7 57

6 8 0 32 5 40

7 14 14 33 4 50

8 19 21 34 2 50

9 24 13 35 1 100

10 22 5 36 0 –

11 23 9 37 1 100

12 36 20 38 0 – 13 34 24 39 0 – 14 42 19 40 0 – 15 35 31 41 0 – 16 37 21 42 0 – 17 25 24 43 0 – 18 33 24 44 0 –

19 32 31 45 0 –

20 39 21 42 0 –

21 31 26 47 0 –

22 18 61 48 0 –

23 24 29 49 0 –

24 27 41 48 0 –

25 24 33 49 0 –

* r value = .16

Table 4 presents failure rates by CST risk score for the community supervision sample.

The table indicates that as scores on the CST increase, the percentage of cases that were

rearrested increase as well. Further, the strength of the relationship between overall score and

recidivism was .16 indicating a low to moderate relationship between risk score and recidivism.

Table 5 provides the risk levels, cutoffs, and number of cases falling at each level by

gender. For males, cutoffs for risk levels are as follows: low risk = 0-14; moderate risk, 15-23;

high risk = 24-33; and very high risk, 34+. Table 12 also provides the distribution of risk levels

for females. For females the cutoffs are as follows: low risk = 0-14; moderate risk = 15-21; high

risk 22-28; and very high risk = 29+.

17

Table 5: Distribution of Cases by Indiana Risk Cut Offs for the CST

Level N Percent

Males (n = 390)

Low (0-14) 130 33

Moderate (15-23) 166 43

High (24-33) 91 23

Very High (34-49) 3 1

Females (n = 236)

Low (0-14) 107 45

Moderate (15-21) 94 40

High (22-28) 32 14

Very High (29-49) 3 1

Table 6 presents statistics that describe the distribution of cases using the revised risk

levels for males and females. The revised risk levels shift cut-off points for both genders in

order to provide a more even/expected distribution of cases by risk and to more accurately

distinguish between recidivism rates. Worth noting is that the changes in the cut-off involve

decreasing the scores needed to reach each level.

Table 6: Distribution of Cases by Revised Cut Offs for the CST

Level N Percent

Males (n = 390)

Low (0-14) 130 33

Moderate (15-21) 138 35

High (22-29) 97 25

Very High (30+) 25 6

Females (n = 236)

Low (0-13) 90 38

Moderate (14-21) 111 47

High (22+) 35 15

Figure 2 presents the failure rates for each risk level of the CST for male offenders in the

community supervision sample. The first series of bars provides failure rates by the originally

defined risk levels. The table clearly illustrates the incremental increases in the rates of

18

recidivism for each group. Failure rates are 13 percent for low risk males, 31 percent for

moderate risk males, 44 percent for high risk males and 67 percent for very high risk offenders.

Figure 2: Predictive Validity of the Community Supervision Tool for Males (n = 390)

The r value of .27 reveals that the relationship between risk level and recidivism is

relatively strong. The second series of bars presents the percentage of offenders that recidivated

for each risk level using revised cut-off points that were developed using the current data. The

revised risk levels provide slightly more predictive power.

Figure 3 presents the recidivism rates by risk level of the CST for females in the

community supervision sample. The first series of bars presents failure rates by the originally

defined risk levels. The results indicate that the original cut offs display a relatively modest

relationship with recidivism and that although there are increasing rates of recidivism by risk

level, low and moderate risk levels are only somewhat different.

13 13

31 28

44 42

67

56

0

10

20

30

40

50

60

70

80

Original Cut Offs; r = .27 Revised Cut Offs; r = .29

Per

cent A

rres

ted

Low Risk Moderate Risk High Risk Very High Risk

19

Figure 3: Predictive Validity of the Community Supervision Tool for Females

The second series of bars presents the failure rates by the revised risk levels. The revised

risk levels provide a substantively important difference in recidivism between low and moderate

risk levels, but do not provide a large difference in recidivism between moderate and high risk

cases. A likely cause of the limited difference in recidivism between moderate and high risk

cases is the small number of female cases in the sample, especially at these levels. To increase

the sample size, the female CST samples from Indiana and Ohio were combined. The last series

of bars in the figure displays the predictive validity of the CST for female offenders using both

Ohio and Indiana samples. Combining the samples results in a substantial increase in the

recidivism rate of high risk female offenders to 36 percent.

Another goal of the implementation of IRAS was to provide decision makers with the

ability to establish priorities in the management of dynamic risk factors that was based on the

likelihood of recidivism. The priorities essentially disaggregate overall risk level into risk levels

by domain placing each offender at low, moderate or high risk to reoffend for each domain.

Table 7 provides statistics for the priorities in case management for the CST. All but two of the

domains (social support and substance abuse) had correlations above .10.

18 14

12

21 23 22

28 26

36

0 0

5

10

15

20

25

30

35

40

Original Cut Offs

n = 236, r = .10

Revised Cut Offs

n = 236, r = .12

Revised Cut Offs

OH and IN data

n = 401, r = .20

Per

cent A

rres

ted

Low Risk Moderate Risk High Risk Very High Risk

20

Table 7: Priorities in Case Management

Criminal History* Education and Finances Social Support Neighborhood Problems*

# of Items 6 # of Items 6 # of Items 5 # of Items 2

Range 0 – 8 Range 0 – 6 Range 0 – 4 Range 0 – 3

Risk Arrested Risk Arrested Risk Arrested Risk Arrested

Low (0-1) 18% Low (0-1) 16% Low (0-1) 22% Low (0-1) 22%

Mod. (2-5) 26% Mod. (2-4) 24% Mod. (2-3) 26% Mod. (2) 27%

High (6+) 47% High (5+) 40% High (4) 48% High (3) 39%

r = .17 r = .19 r = .09 r = .11

Substance Abuse* Antisocial Associates* Antisocial Attitudes*

# of Items 5 # of Items 4 # of Items 7

Range 0 – 6 Range 0 – 8 Range 0 – 13

Risk Arrested Risk Arrested Risk Arrested

Low (0-1) 19% Low (0-2) 21% Low (0-2) 14%

Mod. (2-3) 26% Mod. (3-4) 30% Mod. (3-5) 26%

High (4+) 31% High (5-8) 40% High (6+) 33%

r = .10 r = .12 r = .15

Although the domains of social support and substance abuse have r values at .10, they

still produce groups with increasing rates of recidivism. For the domain of social support, 22

percent of low risk cases recidivate, while 26 and 48 percent of moderate and high risk

recidivate. On the other hand, domains with larger r values produce groups with larger

differences between groups. For example, the domain of Antisocial Attitudes produces low

moderate and high risk groups that recidivate at 14 percent, 26 percent, and 33 percent

respectively.

The Community Supervision Screening Tools (CSST)

Since the CST was designed to be used as on a potentially large number of offenders

across the state, the Indiana Community Supervision Screening Tool was developed to provide

counties with the ability to quickly identify low risk cases. Those identified as moderate and

high risk cases on the screener would be given the full assessment. Screening out low risk cases

21

avoids spending extra assessment resources on this group which is not likely to need intensive

treatment services.

Initially, Indiana adopted the same screening tool as was designed in Ohio. Screeners are

often designed based on the strength of a few individual items; therefore, it was appropriate to

revalidate an Indiana screener using the data collected for this project. Two versions of the

Indiana CSST are presented here. The first version was developed to minimize the time needed

to conduct the assessment by incorporating items in the criminal history and employment

domains that can be scored from information that is likely to be readily available. Although the

1st version will be faster to administer, the 2

nd version was developed to minimize the number of

false negatives. False negatives are cases that the screener identifies as low risk, but if given the

full assessment would actually be identified as moderate or high risk. False negatives are

important to minimize because these are cases that are treated as low risk and given minimal

treatment and supervision when a full assessment would suggest that they are moderate risk to

recidivate.

The 1st Community Supervision Screening Tool (CSST1). The 8 items included in the

CSST were chosen because of individual item relationship with recidivism and because they rely

on information that was likely to be easily available in the case file (e.g. criminal history) or

verifiable (e.g. employment). Table 8 presents the items that were included in the CSST1. Since

the CSST1 was designed to screen out low risk cases, cut-offs were identified that separated

offenders into two groups: low risk or moderate / high risk. Preliminary analyses revealed that

optimal cut-off scores for the CSST were the same for males and females.

22

Table 8: Items in the 1st Indiana Community Supervision Screening Tool

Item Score

1.1 Most Serious Arrest or Charge Under Age 18

1.2 Number of Prior Adult Felony Convictions

1.3 Prior Sentence as Adult to a Jail or Secure Correctional Facility

1.4 Ever Received Official Misconduct while Incarcerated as Adult

2.1 Highest Education

2.2 Employed at the Time of Arrest

2.3 Currently Employed

2.4 Current Financial Situation

Table 9 presents the distribution of cases by risk level for the CSST1. As the table

indicates, the CSST1 identified 27 percent of males as low risk and 36 percent of females as low

risk.

Table 9: Number of Cases by Risk Level for the 1st Community Supervision Screen Tool

Level N Percent

Males

Low (0-2) 106 27

Moderate – High (3+) 284 73

Total 390 100

Females

Low (0-2) 84 36

Moderate – High (3+) 152 64

Total 165 100

Tables 10 and 11 present information regarding the ability of the screener to accurately

identify cases that would be categorized as low risk by the full CST. Each table contains the

number of cases identified as low risk by the screener cross-classified with the number of cases

identified as low risk by the full assessment instrument. Table 10 presents this information for

males. The statistics in the table indicate that of the cases that the screener identifies as low risk,

34 percent are actually high risk on the full assessment (i.e. false negatives). For cases the screen

identifies as moderate or high risk, only 20 percent of cases are categorized as low risk by the

full assessment instrument (i.e. false positives)

23

Table 10: Revised Risk Level by Screen Risk Level for Males

Male Revised CST Cut Offs

Total Low Mod/High

Screener Low 70 36 106

66% 34%* 27.1%

Mod/High 60 224 284

21%^ 79.6% 72.9%

Total 130 260 390

33% 57% 100%

*percentage of cases that are a false negative

^ percentage of cases that are false positives

Table 11 presents the screener category by full CST category for females. Of the 236

female cases in the sample, the CSST1 identifies 36 percent as low risk. The statistics also

indicate 29 percent of the screened cases are false negatives and 20 percent are false positives.

Table 11: Revised Risk Level by Screen Risk Level for Females

Female Revised CST Cut Offs

Total Low Mod/High

Screener Low 59 25 84

70.2% 29.8%* 35.6%

Mod/High 31 121 152

20.4%^ 79.6% 64.4%

Total 90 146 236

38.1% 61.9% 100%

*percentage of cases that are a false negative

^ percentage of cases that are a false positive

In sum, the 1st Community Supervision Screening Tool provides a means to screen out as

much as a third of all cases from the full assessment (27 percent of males and 36 percent of

females). Further, it does so using only 8 items that require information that is likely to be

readily available in the offender case file. Of concern, the use of this screening tool results in a

relatively high rates of false negatives (34 percent for males and 30 percent for females). The 2nd

24

version of the Community Supervision Screening Instrument is designed to provide a more

accurate means of identifying low risk cases by reducing the number of false negatives.

The 2nd

Community Supervision Screening Tool (CSST2). There are 9 items included in

the CSST2 that are drawn from 4 different domains. In order to increase the accuracy of the

screening instrument in identifying low risk cases, the number of items and the diversity of items

were increased. That is, the CSST2 not only includes items from the criminal history and

employment domains, but also includes items from the domains of substance abuse and

antisocial attitudes. It is worth noting that the increased accuracy comes at the cost of a likely

increased amount of time spent conducting the assessment. Table 12 presents the items that were

included in the CSST2.

Table 12: Items in the 2nd

Indiana Community Supervision Screening Tool

Item Score

1.1 Most Serious Arrest or Charge Under Age 18

1.2 Number of Prior Adult Felony Convictions

1.4 Ever Received Official Misconduct while Incarcerated as Adult

1.5 Prior Sentence to Probation as an Adult

2.1 Highest Education

2.4 Currently Employed

2.6 Current Financial Situation

5.4 Drug Use Caused Legal Problems

7.6 Walks Away from Fights

Since the CSST2 was designed to screen out low risk cases, cut-offs were identified that

separated offenders into two groups: low risk or moderate / high risk. Preliminary analyses

revealed that optimal cut-off scores for the CSST2 were the same for males and females. Table

13 presents the distribution of cases by risk level for the CSST2. As the table indicates, 20

percent of males were identified as low risk cases by the CSST2 and 31 percent of females were

identified as low risk. Compared to the CSST1, the CSST2 thus classifies a smaller percentage

of offenders as low risk for both genders.

25

Table 13: Number of Cases by Risk Level for the 2nd

Community Supervision Screen Tool

Level N Percent

Males

Low (0-2) 76 20

Moderate – High (3+) 314 80

Total 390 100

Females

Low (0-2) 73 31

Moderate – High (3+) 163 69

Total 236 100

Tables 14 and 15 present information regarding the ability of the screener to accurately

identify cases that would be categorized by the full CST. Each table contains the number of

cases identified as low risk by the screener cross-classified with the number of cases identified as

low risk by the full CST assessment instrument. Table 14 presents this information for males.

The statistics in the table 14 indicate that of the cases that the screener identifies as low risk, only

21 percent are actually moderate risk on the full assessment (i.e. false negatives).

26

Table 14: Revised Risk Level by Screen Risk Level for Males

Male Revised CST Cut Offs

Total Low Mod/High

Screener Low 60 16 76

78.9% 21.1%* 19.5%

Mod/High 70 244 314

22.3%^ 81.7% 80.5%

Total 130 260 390

33% 57% 100%

*percentage of cases that are a false negative

^ percentage of cases that are false positives

For cases the screener identifies as moderate or high risk, only 22 percent of cases are

categorized as low risk by the full assessment instrument (i.e. false positives). This is a

substantial reduction in false negatives when compared to the CSST1.

The results for the CSST2 are similar for females. Table 15 presents the screen category

by full CST category for female cases. The statistics in the table indicate 23 percent of the

screened cases are false negatives and 20 percent are false positives. This is also a substantial

reduction when compared to the CSST1.

Table 15: Revised Risk Level by Screen Risk Level for Females

Female Revised CST Cut Offs

Total Low Mod/High

Screener Low 56 17 73

76.7% 23.3%* 30.9%

Mod/High 34 129 163

20.4%^ 79.6% 69.1%

Total 90 146 236

38.1% 61.9% 100%

*percentage of cases that are a false negative

^ percentage of cases that are false positives

27

In summary, the CSST2 provides improvement in the accuracy of identifying low risk

cases that can be screened out of the full assessment. Of the offenders identified to be low risk

by the CSST1, about 30 percent were actually moderate risk, while the CSST2 has a false

negative rate of only about 20 percent. Still, with the increases in accuracy also come decreases

in the number of offenders that are screened out of the full assessment and increases in the

number of items for which information is needed. Further, several of the items in the CSST2 are

also dynamic and will likely require a brief structured interview.

The Prison Reentry Tool (RT)

Data for the prison reentry sample was gathered through site visits to correctional

institutions. The RT is designed be administered as prison inmates approach release in order to

assist in release decisions, aid in determining supervision level, and to guide case management

for offenders as they prepare to reenter the community. The prison reentry sample consists of

362 individuals who were incarcerated in one of the 8 pilot correctional institutions and were

within six months of release from prison. Table 16 presents descriptive statistics for the prison

reentry sample. As indicated in Table 16, 59 percent of the sample is male, 71 percent is white,

and the average age is 34. The average follow-up period was 23 months and 40 percent

experienced a new arrest during the follow-up period.

28

Table 16: Descriptive Statistics for the Prison Reentry Sample (n = 362)

Variable N Percent

Sex

Male 213 59

Female 149 41

Race

White 257 71

African American 88 24

Other 17 5

Any New Arrest

Yes 153 42

No 209 58

Average Range

Months at Risk 23.6 22 - 24

(0.6 SD)

Age 34.6 19 – 70

(9.9 SD)

Figure 4 presents the distribution of cases on the reentry tool by gender. The figure

suggests that the distribution approaches normality for both genders although the distribution of

female cases falls lower on the scale than males.

Figure 4: Distribution of Cases on the Prison Reentry Tool (male n = 213; female n = 149)

0

5

10

15

20

25

1 5 10 15 20 24

Male Female

29

Table 17: Failure Rates by Risk Score for the Prison Reentry Tool (n=362)*

Risk Score Total Cases Percent

Arrested

Risk Score Total Cases Percent

Arrested

0 0 – 21 2 50

1 0 – 22 0 –

2 2 50 23 2 50

3 7 0 24 1 100

4 8 25 25 0 –

5 8 50 26 0 –

6 18 33 27 0 –

7 23 39 28 0 –

8 25 36 29 0 –

9 32 38 30 0 –

10 26 39 31 0 –

11 32 31 32 0 –

12 29 24 33 0 –

13 28 50 34 0 –

14 28 54 35 0 –

15 26 50 36 0 –

16 19 58 37 0 –

17 15 53 38 0 –

18 14 64 39 0 –

19 11 56 40 0 –

20 6 67 * r value = .19

Table 17 presents failure rates by RT risk score for the prison reentry sample. The table

reveals that as scores on the RT increase, the percentage individuals that recidivated also tends to

increase. The correlation between the RT risk score and recidivism is .19. Table 18 presents the

distribution of cases by risk level and gender for the RT using the original cut-offs. Using the

original cut-offs, 29 percent of males were low risk, 45 percent were moderate, and 26 percent

were high. For females, 45 percent were low, 43 were moderate, and 12 were high.

30

Table 18: Distribution of Cases by Indiana Cut Offs for the RT

Level N Percent

Males (n = 213)

Low (0-9) 61 29

Moderate (10-15) 96 45

High (16+) 56 26

Females (n = 149)

Low (0-10) 67 45

Moderate (11-14) 64 43

High (15+) 18 12

Table 19 presents statistics that describe the distribution of cases using the revised risk

levels for males and females. The revised risk levels shift cut points for both genders in order to

provide a more even/expected distribution of cases by risk and to more accurately distinguish

between recidivism rates. Worth noting is that the changes in the cut- off involve decreasing the

scores needed to reach each level.

Table 19: Distribution of Cases by Revised Cut Offs for the RT

Level N Percent

Males (n = 213)

Low (0-7) 31 15

Moderate (8-13) 92 43

High (14-17) 63 30

Very High (18+) 27 12

Females (n = 149)

Low (0-5) 13 9

Moderate (6-12) 91 61

High (13+) 45 30

Figure 5 presents percentage of males that were arrested by risk level on the RT. The

first series of bars provide the risk levels using the original cut-offs. The chart illustrates that

increases in recidivism are seen with increase in risk level. Seventeen percent of low risk cases

recidivated, 32 percent of moderate risk cases recidivated, 58 percent of high risk cases

recidivated and 71 percent of very high risk cases recidivated (r = .21). The second series of bars

31

presents the percentage of offenders that recidivated using the revised cut-offs. The revised cut-

offs provide four levels of risk and similar levels of predictive validity (r = .21).

Figure 5: Predictive Validity of the Prison Reentry Tool for Males

Although there are substantive differences in recidivism by risk level, it is worth noting

that low risk cases recidivated at a relatively high rate (29 percent). A likely cause of the high

rates of recidivism of low risk cases is the small number of low risk male cases in the prison

reentry sample (n = 31). To increase the sample size, the male prison reentry samples from

Indiana and Ohio were combined. The last series of bars in the figure displays the predictive

validity of the RT for male offenders using both Ohio and Indiana samples. Combining the

samples results in a substantial decrease in the recidivism rate of low risk male offenders to 19

percent.

Figure 6 presents the recidivism rates by risk level for females in the prison reentry

sample. The first series using the original risk levels indicates that the levels struggle to

distinguish between low and moderate risk offenders. Thirty-five percent of low risk cases

reoffended while only 31 percent of moderate risk cases recidivated. Fifty percent of those

categorized as high risk recidivated. The revised risk levels perform much better in

distinguishing between rates of recidivism. Fifteen percent of low risk cases recidivated while

34 29

19

47 41 42

61 57 57

63 63

0

10

20

30

40

50

60

70

Original Cut Offs

n = 213, r = .21

Revised Cut Offs

n = 213, r = .21

Revised Cut Offs

OH and IN data

n = 414, r = .25

Per

cent A

rres

ted

32

33 percent of moderate risk cases recidivated. Forty-seven percent of high risk cases recidivated.

The r value for the revised cut-offs is also larger than the original risk level (r = .18 versus .05).

Figure 6: Predictive Validity of the Prison Reentry Tool for Females (n = 156)

Table 20 presents statistics on the distribution and validity of the priorities in case

management. The priorities in cases management essentially disaggregate risk levels of the RT

by domain. Table 20 presents statistics for each of the case management domains. The r values

for the domains range from .10 (social bonds) to .19 (criminal history). The table indicates that

case management domains are individually able to classify offenders into different groups based

on the likelihood to recidivate. For example in the criminal attitudes domain, 33 percent of low

risk cases were arrested, 45 percent of moderate risk cases were arrested, and 55 percent of high

risk cases were rearrested.

35

15

31 33

50 47

0

10

20

30

40

50

60

Original Cut Offs;r = .05 Revised Cut Offs; r = .18

Per

cent A

rres

ted

Low Risk Moderate Risk High Risk

33

Table 20: Priorities in Case Management for the Prison Reentry Tool

Criminal History Social Bonds Criminal Attitudes

# of Items 6 # of Items 4 # of Items 7

Range 0 – 11 Range 0 – 4 Range 0 – 11

Risk Arrested Risk Arrested Risk Arrested

Low (0-3) 32% Low (0-1) 34% Low (0-3) 33%

Mod. (4-6) 44% Mod. (2-3) 42% Mod. (4-7) 45%

High (7+) 56% High (4) 53% High (8+) 55%

r = .19 r = .10 r = .14

SUMMARY AND CONCLUSION

This section of the report provides some conclusions based on the findings of the current

study. It begins with a summary the results for the validation of IRAS. Some limitations of the

current study are also discussed. The report concludes with some recommendations.

Summary of Findings

The Indiana Risk Assessment System was validated using a prospective design that

involved conducting in-depth structured interviews of almost 1,000 offenders that were on

community supervision or had recently been admitted to prison. After interviews were

conducted, offenders were tracked for nearly 2 years to gather follow-up information on

recidivism. Three assessment instruments were validated in this process: the Community

Supervision Tool, the Community Supervision Screening Tool and the Prison Reentry Tool.

Validation involved examining the predictive power of the assessment instruments. The

results reveal that all assessment instruments are able to significantly distinguish between risk

levels. Further, the measures of association (r values) are acceptable for revalidation purposes,

although the instruments had more difficulty distinguishing between moderate and high risk

females. Given the information, the authors provide some suggestions regarding revisions to cut-

off points on the risk assessment instruments.

34

Limitations

There were two primary limitations observed in the current study. The first limitation

revolves around the generalizability of the sample to all offenders in the Indiana criminal justice

system. Although the data collection period gathered information on over 1,000 offenders in

Indiana, it would be imprudent to assume that the findings are representative of all offenders in

Indiana. First, resource constraints limited the inclusion of cases from all counties and

correctional institutions. Second, although the samples were gathered from specific populations,

certain types of cases may be underrepresented in the population (e.g. low risk offenders, sex

offenders, Hispanic offenders, female offenders). The underrepresentation in the population

leads to small numbers of these types of offenders in the sample. For example, the findings from

the RT were based on a sample size of 149 females. Although the results provide evidence that

females have a distribution on the risk levels that is different from men, the findings should be

considered preliminary until data can be collected on a larger sample of women who are released

from prison.

A second limitation to the current study revolves around measurement error. The major

source of data collection for this study was the structured interview, which was undertaken by

trained research staff from the University of Cincinnati. Further, the informed consent process

identified a sample of offenders who were willing to undergo the interview process. In short, the

structured interview process utilized to gather the data will likely be different than the process

used by criminal justice officials to interview cases and assign risk once IRAS is implemented.

Recommendations

Based on the findings and limitations discussed above, several recommendations can be

made. The first major recommendation is that revalidation studies be conducted of IRAS.

35

Revalidation studies will provide further evidence that the risk assessment instruments are able

to predict recidivism across multiple samples from the same population, including the Pre-trial

Tool and Prison Intake Tool. Further, revalidation studies should focus on oversampling

underrepresented groups. Finally, revalidation will also address the issue of measurement error.

That is, data can be gathered on assessments that are given by personnel within the criminal

justice system, examining the predictive validity of IRAS in a real world setting.

Another important recommendation is that Indiana follow the protocol developed by the

University of Cincinnati for training personnel on the assessment instruments. Proper training

cannot be stressed enough because the efficacy of every assessment is heavily dependent upon

the person who conducts the interview and scores the risk level. Training will also help to

minimize the differences in measurement between University research staff conducting the

interviews and criminal justice personnel. Not only is initial training important, but it is

recommended that a system be developed that lays out the process of training, provides

reliability checks for interviewers, and lays out guidelines for retraining.

36

REFERENCES

Andrews, D., Bonta, J., and Hoge R. (1990). Classification for effective rehabilitation:

Rediscovering psychology. Criminal Justice and Behavior 17:19-52.

Andrews, D. A., Zinger, I., Hoge, R., Bonta, J., Gendreau, P. and Cullen F. T. (1990). Does

correctional treatment work? A clinically relevant and psychologically informed meta-

analysis. Criminology, 28: 369-404.

Bonta, J. (1996). Risk-needs assessment and treatment. in A. T. Harland (ed.), Choosing

Correctional Interventions That Work: Defining the Demand and Evaluating the Supply.

Newbury Park, CA: Sage.

Gendreau, P., Little, T. & Goggin C. (1996). A meta-analysis of the predictors of adult offender

recidivism: What works! Criminology, 34: 401-433.

Gottfredson, S. and L. Moriarty (2006). Statistical risk assessment: old problems and new

applications. Crime and Delinquency 52: 178-200.

Jones, D. (1996). Risk prediction in criminal justice. In A.T. Harland (Ed.). Choosing

correctional options that work. London, Sage.

Lowenkamp, C. T. & Latessa E. J. (2002). Evaluation of Ohio’s community based correctional

facilities and halfway house programs: Final report. Unpublished Technical Report:

University of Cincinnati.

Lowenkamp, C. T. & Latessa E. J. (2004). Understanding the risk principle: How and why

correctional interventions can harm low-risk offenders. Topics In Community

Corrections, 2004. Washington, D.C.: U.S. Department of Justice, National Institute of

Corrections.

Lowenkamp, C. T. & Latessa E. J. (2005). Evaluation of Ohio’s CCA funded programs: Final

report. Unpublished Technical Report: University of Cincinnati.

Lowenkamp, C. T. & Latessa E. J. (2005). Increasing the effectiveness of correctional

programming through the risk principle: Identifying offenders for residential placement.

Criminology and Public Policy, 4: 263-290.

Lowenkamp, C.T., Latessa, E. J. and Holsinger, A. (2006). The risk principle in action: What

have we learned from 13,676 offenders and 97 correctional programs? Crime and

Delinquency, 51, 1-17.

Lowenkamp, C. T. Latessa E. J., & Smith, P. (2006). Does correctional program quality really

matter? The impact of adhering to the principles of effective intervention. Criminology

and Public Policy, 5: 575-594.

37

Van Voorhis, P. (2009). An overview of offender classification systems. In P. Van Voorhis, D.

Lester, & M. Brasswell (Eds.) Correctional Counseling and Rehabilitation 7th

Edition.

Cincinnati, OH: Anderson Publishing.

Wright, K., Clear, T. R. & Dickerson, P. (1984). Universal application of probation risk

assessment instruments: A critique. Criminology 22: 113-134.