Promoting Promoting Quality Quality

Prevention Prevention Counseling Counseling Project: Project:

What have we What have we learned?learned?Spring 2005 Texas TourSpring 2005 Texas Tour

Dallas, Fort Worth, Houston, Dallas, Fort Worth, Houston, Midland, AustinMidland, Austin

AgendaAgenda Welcome and IntroductionWelcome and Introduction BackgroundBackground Overview of projectOverview of project Implementation experiences Implementation experiences

by sitesby sites General evaluation findingsGeneral evaluation findings Next StepsNext Steps Q & AQ & A

BackgroundBackgroundRevised HIV Revised HIV Counseling, Counseling, Testing and Testing and Referral Referral (CTR) (CTR) Guidelines, Guidelines, November November 2001 2001

Background Cont’dBackground Cont’d

CDC’s Project RESPECT: CDC’s Project RESPECT: Evidence-based intervention Evidence-based intervention showing significant reduction showing significant reduction of STDs with protocol-based of STDs with protocol-based HIV prevention counseling HIV prevention counseling

RESPECT-2: Refined HIV RESPECT-2: Refined HIV prevention counseling protocol prevention counseling protocol & further developed & further developed “counseling quality assurance” “counseling quality assurance” methodsmethods

RESPECT MethodologyRESPECT Methodology5758 heterosexual, HIV-5758 heterosexual, HIV-

negative patients older than negative patients older than 14 years who came in for 14 years who came in for STD examination STD examination

Five public STD clinics Five public STD clinics (Baltimore, Denver, Long (Baltimore, Denver, Long Beach, Newark and San Beach, Newark and San Francisco)Francisco)

Project RESPECT ResultsProject RESPECT Results*: *: HIV Prevention Counseling HIV Prevention Counseling

EffectiveEffective

107

173

149

211

0255075

100125150175200225250

6 months* 12 months*

New

ST

D d

iag

no

ses

RESPECT Control

Kamb, M.L., et al (1998) Efficacy of risk-reduction counseling to prevent human immunodeficiency virus and sexually transmitted diseases, JAMA, 280 (13):1161-1167

(*p<0.05)

How could we translate How could we translate this intervention into a this intervention into a

real-world setting?real-world setting?

Risk Reduction Risk Reduction Specialist supportSpecialist support

Supervisor supportSupervisor supportPractical toolsPractical tools

Goals of the ProjectGoals of the ProjectDevelop and evaluate tools Develop and evaluate tools

to support protocol-driven to support protocol-driven prevention counseling based prevention counseling based on the RESPECT modelon the RESPECT model

Develop and evaluate QA Develop and evaluate QA proceduresprocedures

Better understand the Better understand the barriers and facilitators of barriers and facilitators of good prevention counselinggood prevention counseling

DefinitionsDefinitions Evidence-based interventionsEvidence-based interventions

Interventions that have demonstrated Interventions that have demonstrated desired outcomes through rigorous desired outcomes through rigorous researchresearch

Core elementsCore elements Components of the intervention that Components of the intervention that

are believed to be essential to achieve are believed to be essential to achieve the desired behavior changethe desired behavior change

ProtocolProtocol A structured approach to achieve core A structured approach to achieve core

elementselements

DefinitionsDefinitions Client-centered prevention Client-centered prevention

counselingcounseling One-on-one interactions with risk-One-on-one interactions with risk-

reduction as its primary goalreduction as its primary goal Risk Reduction SpecialistRisk Reduction Specialist

A trained specialist responsible for A trained specialist responsible for maintaining the focus on a client’s maintaining the focus on a client’s specific risk reduction needsspecific risk reduction needs

ToolsTools Job aides to ensure fidelity to core Job aides to ensure fidelity to core

elements of protocolelements of protocol

cccc

Tarrant CountyHealth Dept.Ft. Worth, TX

Resource CenterOf DallasDallas, TX

Valley AIDS Council,McAllen, TX

Brownsville, TX

City of LaredoHealth Dept.Laredo, TX

Site Locations

What was introduced What was introduced during the project?during the project?

Counseling protocolCounseling protocol Training on the protocolTraining on the protocol Counseling toolsCounseling tools

Spiral book with goals and Spiral book with goals and sample questionssample questions

Laminate “wheel”Laminate “wheel” Documentation form with space Documentation form with space

for RR plan and referralsfor RR plan and referrals Personal review formPersonal review form

Session Documentation Session Documentation FormForm

What QA activities were What QA activities were part of the project?part of the project?

QA protocolQA protocol Emphasizing standardized Emphasizing standardized

preceptorship, observation, routine preceptorship, observation, routine meetings, documentation review, and meetings, documentation review, and feedback on observations and feedback on observations and documentationdocumentation

QA toolsQA tools Supervisor observation toolSupervisor observation tool RRS self assessmentRRS self assessment Chart abstraction and summary toolsChart abstraction and summary tools

Quality Assurance ToolsQuality Assurance Tools

Comparison of Core Comparison of Core ElementsElements

GOAL PCPE RESPECT Protocol

1:1 Counseling Yes Yes Yes

2-Session Model Yes Yes Yes

Follow Protocol No Yes Yes

Focus: Client Risk Behavior Yes Yes Yes

Increase self-perception Risk Yes Yes Yes

Negotiate Realistic RR Step Yes Yes Yes

Provide Referrals Yes Yes Yes

Support Test Decision Yes Yes Yes

Interpret test results Yes Yes Yes

Partner Elicitation/ Referral Yes No Yes

Standardized QA Procedure No Yes Yes

QA Tools Specified No No Yes

Session Tools No Yes Yes

ImplementationImplementation

Training developed for protocol, Training developed for protocol, tools and QAtools and QA

Supervisors and Risk Reduction Supervisors and Risk Reduction Specialists trained in October 2003Specialists trained in October 2003

On-site and off-site TA provided for On-site and off-site TA provided for start-upstart-up

Staggered and tailored Staggered and tailored implementation of protocol, tools, implementation of protocol, tools, and QAand QA

Additional ongoing TA after start-upAdditional ongoing TA after start-up

What Do We Want to What Do We Want to Learn?Learn?

Can you implement protocol-Can you implement protocol-based prevention counseling based prevention counseling with existing resources?with existing resources?

Did the protocols and tools help Did the protocols and tools help them implement the intervention them implement the intervention with fidelity?with fidelity?

What were some of the What were some of the facilitators and barriers of the facilitators and barriers of the implementation of the protocols?implementation of the protocols?

Evaluation DesignEvaluation Design

Two data collection phases: Pre- Two data collection phases: Pre- and Post-interventionand Post-intervention

Evaluation data triangulation: 9 Evaluation data triangulation: 9 complementary quantitative and complementary quantitative and qualitative instrumentsqualitative instruments

Quantitative InstrumentsQuantitative Instruments

Supervisor time logs (pre and Supervisor time logs (pre and post)post)

Client Questionnaires (pre and Client Questionnaires (pre and post)post)

Counseling chart reviews (pre Counseling chart reviews (pre and post)and post)

Observations of counseling by Observations of counseling by evaluators (pre and post)evaluators (pre and post)

Qualitative Instruments Qualitative Instruments

Risk Reduction Specialist (pre Risk Reduction Specialist (pre and post)and post)

Supervisors (pre and post)Supervisors (pre and post) Site Program Managers (post Site Program Managers (post

only)only) WAP (post only)WAP (post only) TDH (post only)TDH (post only)

ResultsResults

Lessons LearnedLessons Learned

Did the protocols and Did the protocols and tools help implement the tools help implement the

intervention with intervention with fidelity? fidelity?

Observations: Initial SessionObservations: Initial SessionGoalsGoals

00.10.20.30.40.50.60.70.80.9

1

Pre Post *P<.05

Observations: Follow-up Observations: Follow-up SessionSessionGoalsGoals

00.10.20.30.40.50.60.70.80.9

1

Pre Post *P<.05

Client survey: Initial sessionClient survey: Initial sessionGoalsGoals

50

60

70

80

90

100

Explain job*

Purpose

Perception of Risk

Life situation

Recent risk**

Past RR

Pre Post

*P<.05

Client survey: Initial sessionClient survey: Initial sessionGoals (cont’d)Goals (cont’d)

50

60

70

80

90

100

Pre Post *P<.05

Client survey: Follow-up sessionClient survey: Follow-up sessionGoalsGoals

50

60

70

80

90

100

Exp. Job

Results early

Tried plan*

Why work

Support*

Referral

RR plan

Pre Post*P<.05

Client Surveys: Client Client Surveys: Client ParticipationParticipation

0%

20%

40%

60%

80%

100%

Talked more(Pre)

Talked more(Post)*

Made plan(Pre)

Made plan(Post)

Me Both RRS

*P<.05

Chart reviewsChart reviewsGoalsGoals

0102030405060708090

100

Percp.Risk*

Recentrisk*

Pattern* Past RR*

PrePost

*P<.05

33

Significant changes seen in initial Significant changes seen in initial sessionssessions

GoalsGoals ObservationObservation Client Client ReportReport

DocumentatDocumentationion

Introduction/Introduction/OrientationOrientation

XX

Enhanced risk Enhanced risk percept.percept.

XX XX

Recent risk Recent risk discusseddiscussed

XX XX XX

Reviewed past RRReviewed past RR XX XX

Sum up pattern of Sum up pattern of riskrisk

XX XX

Risk reduction Risk reduction planplan

XX

Support and Support and referralreferral

XX X (2)X (2) XX

Test decision Test decision counselingcounseling

XX

Appt and Appt and reminderreminder

XX XX

34

Significant changes seen in follow Significant changes seen in follow up sessions up sessions

GoalsGoals ObservationObservation Client Client ReportReport

DocumentaDocumentationtion

Orient and give Orient and give resultsresults

XX

Review RR effortsReview RR efforts XX XX

Risk reduction Risk reduction planplan

XX XX

Support and Support and referralreferral

XX X X XX

Summarize and Summarize and closeclose

XX

““That you had to follow every That you had to follow every single task even though they single task even though they

didn't all apply to everybody [is a didn't all apply to everybody [is a problem]. Protocol doesn't allow problem]. Protocol doesn't allow for individual counseling styles or for individual counseling styles or use of skills RRS's have received use of skills RRS's have received

at prior trainings…it seems at prior trainings…it seems cumbersome and redundant to cumbersome and redundant to

use this protocol with clients with use this protocol with clients with very few risks - although it's easy very few risks - although it's easy

enough to move through the enough to move through the protocol by saying this doesn't protocol by saying this doesn't really apply to you [for certain really apply to you [for certain

tasks].”tasks].”-Risk Reduction Counselor-Risk Reduction Counselor

““[The protocol improved the [The protocol improved the quality of my counseling] quality of my counseling]

because I had a structure to because I had a structure to make sure I wasn’t leaving make sure I wasn’t leaving

anything out.”anything out.”

-Risk Reduction Counselor-Risk Reduction Counselor

Observations: Initial Visit Observations: Initial Visit Use of Counseling SkillsUse of Counseling Skills

0

0.5

1

1.5

2

2.5

3

Pre Post*P<.05

Observations: Follow-up Observations: Follow-up Session Session

Use of Counseling SkillsUse of Counseling Skills

0

0.5

1

1.5

2

2.5

3

Pre Post *P<.05

Client Surveys: Client Client Surveys: Client experienceexperience

50

60

70

80

90

100

Pre Post *P<.05

What were some of the What were some of the facilitators and barriers facilitators and barriers of the implementation of of the implementation of

the protocols?the protocols?

Overall themes and Overall themes and feedbackfeedback

Delivery of protocol-driven Delivery of protocol-driven preventionpreventionProvided structureProvided structureImproves with practiceImproves with practiceAided in identifying risk Aided in identifying risk behaviors and patternsbehaviors and patterns

Protocol questions felt rigidProtocol questions felt rigid More training and TA is More training and TA is

essentialessential

Overall themes and Overall themes and feedback (cont’d)feedback (cont’d)

Spanish version tools are neededSpanish version tools are needed Supervisor buy-in is essentialSupervisor buy-in is essential Supervisor’s other responsibilities Supervisor’s other responsibilities

need to be considered due to time need to be considered due to time constraints of quality assuranceconstraints of quality assurance

Difficult with certain clients (such Difficult with certain clients (such as low risk, outreach, drug as low risk, outreach, drug treatment and jail)treatment and jail)

Using the Counseling Using the Counseling ToolsTools

Most of the RRS found the cards Most of the RRS found the cards to be the most helpful of the to be the most helpful of the tools (72%)tools (72%)

Cards help ensure you cover Cards help ensure you cover everything in order (44%)everything in order (44%)

Wheel was not as helpful (83%)Wheel was not as helpful (83%) Spanish version of the tools is Spanish version of the tools is

neededneeded

Counseling QA by RRSCounseling QA by RRS

Most helpfulMost helpfulObservation by supervisorObservation by supervisorGeneral feedback General feedback Role play Role play Observation by peer/peer Observation by peer/peer observation/document reviewobservation/document review

Case conference Case conference

Counseling QA by RRS Counseling QA by RRS (cont’d)(cont’d)

Least helpfulLeast helpfulDocumentation reviewDocumentation reviewObservation by peerObservation by peerCase conference/self Case conference/self assessmentassessment

Counseling QA by Counseling QA by SupervisorSupervisor

Most helpfulMost helpfulObservation by supervisorObservation by supervisorGeneral feedback General feedback Document reviewDocument reviewCase conferencesCase conferences

Least helpfulLeast helpfulObservation by peerObservation by peerSelf-assessmentSelf-assessment

QA Activities by RRSQA Activities by RRS

0%10%20%30%40%50%60%70%80%90%

100%

PrePost

“[The supervisor observation form] is better because it is less subjective and more structured. …the priorities of the tasks are made clear by the forms and that feedback using these forms makes the whole process self-reinforcing [the process of understanding the expectations of the protocol, using the protocol, and getting feedback—all

have the same language, structure, and expectations]…the new feedback is less stressful

for everybody, including the observer for the stated reasons.”

---Risk Reduction Specialist

I think this new protocol is great—fabulous! Before when they first told us about the program and we went to training, we were all ‘iffy’ and said ‘it’s not gonna work’ ‘no way in heck’ it would be accepted by the people. Now that we are implementing it, we are doing a great job.

When you have to write steps, the clients leave with RR plan in hand, a referral, an appointment card with the date on it in hand. As for review

forms used by the supervisor on documentation, etc. You have the form yourself to be able to

discuss ‘met’ or ‘not met.’---Risk Reduction Specialist

What has been done?What has been done? Changes to trainingChanges to training

Trainers have bought inTrainers have bought in Preceptorship is done first, then attend Preceptorship is done first, then attend

trainingtraining More time for role playMore time for role play Develop their own questions for each stepDevelop their own questions for each step Not a gripe sessionNot a gripe session

Sites learning from each otherSites learning from each other Role plays/Peer observations for practiceRole plays/Peer observations for practice Sign in waiting room for length of Sign in waiting room for length of

sessionssessions Regularly scheduled QA sessionsRegularly scheduled QA sessions

What Now?What Now? State-wide roll-out begins May 2, 2005State-wide roll-out begins May 2, 2005

All DSHS HIV/STD contractors All DSHS HIV/STD contractors

Roll-out completed by July 2006Roll-out completed by July 2006

Protocol Based Counseling Training Protocol Based Counseling Training (PBCT) replaces PCPE by August 2006 as (PBCT) replaces PCPE by August 2006 as the state mandated prevention counseling the state mandated prevention counseling course for risk reduction specialistscourse for risk reduction specialists

HOW??HOW?? 11-week training and technical assistance 11-week training and technical assistance

cyclescycles One month of supervisory training and One month of supervisory training and

developmentdevelopment Two week employee preceptorshipTwo week employee preceptorship Three weeks of employee training and TAThree weeks of employee training and TA Two weeks of independent implementation and Two weeks of independent implementation and

TATA

Ongoing technical assistance and supportOngoing technical assistance and support 7 training staff dedicated to rollout initially7 training staff dedicated to rollout initially Field operations and regional staff supportField operations and regional staff support

When?When? Cycle 1 – May 2Cycle 1 – May 2

SE Texas area agenciesSE Texas area agencies Cycle 2 – July 18Cycle 2 – July 18 Cycle 3 – September 19Cycle 3 – September 19 Cycles 4-6 in 2006Cycles 4-6 in 2006

Agency selection for cycle based on:Agency selection for cycle based on: Epi dataEpi data Field OpsField Ops Agency ReadinessAgency Readiness

Programmatic Programmatic PBC is the prevention counseling model PBC is the prevention counseling model

that must be used if contracted to perform that must be used if contracted to perform PCPE or an ILI as a component of your GLI.PCPE or an ILI as a component of your GLI.

Once committed - no going back to old Once committed - no going back to old PCPE.PCPE.

Changes in the RFP and contract language.Changes in the RFP and contract language.o State 2005/6 State 2005/6 o Federal 2006 Federal 2006Federal 2006 Federal 2006

Big competitive RFP released Spring 2006 Big competitive RFP released Spring 2006 to start funding state 9/1/06 and federal to start funding state 9/1/06 and federal 1/1/071/1/07

Points to Ponder…Points to Ponder…

Structural - your program overall? Buy-Structural - your program overall? Buy-in from your administration, capacity?in from your administration, capacity?

Staffing – Supervision requirements, Staffing – Supervision requirements, hiring, vacancies, current staffhiring, vacancies, current staff

Time for supervision, time for staffing, Time for supervision, time for staffing, time for QA activities, time to perform time for QA activities, time to perform the sessions.the sessions.

Who needs to be trained? PCM, EBI, Who needs to be trained? PCM, EBI, TCADA, Case Managers?TCADA, Case Managers?

BudgetsBudgets

Budgets – how to pay for upcoming Budgets – how to pay for upcoming trainings, how to compensate your staff?trainings, how to compensate your staff? Outreach workers’ conference, OraSure, Outreach workers’ conference, OraSure,

salary savingssalary savings One week of Austin training for supervisor One week of Austin training for supervisor

and any team leaders.and any team leaders. All PCPE staff one week in possibly local All PCPE staff one week in possibly local

area. area. Possible budget amendmentsPossible budget amendments Start thinking about new budgets for 2006Start thinking about new budgets for 2006

WorkplansWorkplans

Where are you doing your PC? What Where are you doing your PC? What populations are you serving? populations are you serving?

What does your PC look like now?What does your PC look like now? How does this change the structure of the How does this change the structure of the

work that you do?work that you do? What type of changes will you need to make in What type of changes will you need to make in

your workplan?your workplan? Look at settings and time and how this will Look at settings and time and how this will

work?work? Partner with your fellow providers to perform Partner with your fellow providers to perform

activities in various settings. activities in various settings.

THINGS YOU CAN DO THINGS YOU CAN DO NOW!NOW!

Look at the quality assurance guidelines Look at the quality assurance guidelines and your contract. Are you doing as and your contract. Are you doing as required now?required now?

Review the PBC tools QA. Please do not Review the PBC tools QA. Please do not use them until you are traineduse them until you are trained

What does your orientation plan look What does your orientation plan look like? Some do a type of preceptorship like? Some do a type of preceptorship already. already.

Are you using the TDH documentation Are you using the TDH documentation guide and sample? Are you using a PCPE guide and sample? Are you using a PCPE review tool? Get them off our web. review tool? Get them off our web.

Things to do now..Things to do now..

Review your objectives, what do you Review your objectives, what do you need to meet your return rates, link need to meet your return rates, link to EI, and PE?to EI, and PE?

What does your counseling look like What does your counseling look like now? now?

Clean RECN dataClean RECN data

TA and MonitoringTA and Monitoring

FO staff will go through Mega-FO staff will go through Mega-training with their programtraining with their program

TA provided with Training staffTA provided with Training staff Monitoring schedule will start six Monitoring schedule will start six

months after all trained datemonths after all trained date

Q & AQ & A