march 2009 performance evaluation of public service delivery (social services)– kenyan experience...
TRANSCRIPT
PERFORMANCE EVALUATION OF PUBLIC SERVICE DELIVERY (Social Services)ndash KENYAN EXPERIENCE
March 2009March 2009
PERSPECTIVES ON IMPACT EVALUATION
Presenter
Paul O OmondiThe Steadman Group
This discussion is outlined as follows
1 Background - This provides the case for current evaluation activity in the Kenya public service
2 Notable downsides with the existing public service evaluation technique
3 Explore potential value for Cost Benefit Analysis as an evaluation technique
Discussion OutlineDiscussion Outline
The Government has recognized the need for evidence based decisions on targets relating to service quality
Thus The New Performance Appraisal System (PAS) has become a key element of the governmentrsquos quality service strategy
From 2004 the PAS has expanded from a pilot of 16 commercial enterprises to cover to cover the entire public service
PAS is based on the need for sustainable efficient effective and quality public service delivery
Background InformationBackground Information
Background Information (Conthellip)Background Information (Conthellip)
PAS requires every public institution to outline its Citizen Service Delivery Charter which empowers the public to demand and to expect specific service standards
The PAS assessments are intended to lead to improvements in service quality particularly among those institutions that failed to meet previous performance targets
Background Information (Conthellip)Background Information (Conthellip)
PAS focuses on 3 parameters
1Performance and quality of service
2 Involvement of service consumers
3 Comparative ranking of institutions by performance
Measurement ApproachesMeasurement Approaches
The government currently uses customers satisfaction information to guide operational decisions to improve service
The range of measurements used include
bullBenchmarking
bullGap analysis
bullRanking based on composite score
Downsides of Current Evaluation TechniqueDownsides of Current Evaluation Technique
The recent PAS ranking demonstrated that the performance appraisal process presented challenges which require review of the process
In a number of some instances there has been a disconnect between comparative rankings and citizensrsquo perceptions of services
Hence evaluation outcomes and rankings have been seen by many as politically contentious
Gaps in the Evaluation ProcessGaps in the Evaluation Process
- In defining survey target group it is evident that the ldquocustomerrdquo concept is largely inapplicable in the public service setting
-Evaluation does not take into consideration resource allocation decisions both in terms of amount and efficiency of disbursement
-Evaluative effort does not adequately examine the underlying need for new services
- Poor coordination has meant limited consistency on tracking of key indicators as well as definition of survey target group
Overarching QuestionsOverarching Questions
The overarching questions are
1048707 Is the national evaluation system as it exists feasible
-Probably a more realistic approach is to focus on key sectors programs and projects
1048707Does the evaluation system have the potential to drive service improvement
- It has potential to drive service improvements if the right techniques are applied
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
This discussion is outlined as follows
1 Background - This provides the case for current evaluation activity in the Kenya public service
2 Notable downsides with the existing public service evaluation technique
3 Explore potential value for Cost Benefit Analysis as an evaluation technique
Discussion OutlineDiscussion Outline
The Government has recognized the need for evidence based decisions on targets relating to service quality
Thus The New Performance Appraisal System (PAS) has become a key element of the governmentrsquos quality service strategy
From 2004 the PAS has expanded from a pilot of 16 commercial enterprises to cover to cover the entire public service
PAS is based on the need for sustainable efficient effective and quality public service delivery
Background InformationBackground Information
Background Information (Conthellip)Background Information (Conthellip)
PAS requires every public institution to outline its Citizen Service Delivery Charter which empowers the public to demand and to expect specific service standards
The PAS assessments are intended to lead to improvements in service quality particularly among those institutions that failed to meet previous performance targets
Background Information (Conthellip)Background Information (Conthellip)
PAS focuses on 3 parameters
1Performance and quality of service
2 Involvement of service consumers
3 Comparative ranking of institutions by performance
Measurement ApproachesMeasurement Approaches
The government currently uses customers satisfaction information to guide operational decisions to improve service
The range of measurements used include
bullBenchmarking
bullGap analysis
bullRanking based on composite score
Downsides of Current Evaluation TechniqueDownsides of Current Evaluation Technique
The recent PAS ranking demonstrated that the performance appraisal process presented challenges which require review of the process
In a number of some instances there has been a disconnect between comparative rankings and citizensrsquo perceptions of services
Hence evaluation outcomes and rankings have been seen by many as politically contentious
Gaps in the Evaluation ProcessGaps in the Evaluation Process
- In defining survey target group it is evident that the ldquocustomerrdquo concept is largely inapplicable in the public service setting
-Evaluation does not take into consideration resource allocation decisions both in terms of amount and efficiency of disbursement
-Evaluative effort does not adequately examine the underlying need for new services
- Poor coordination has meant limited consistency on tracking of key indicators as well as definition of survey target group
Overarching QuestionsOverarching Questions
The overarching questions are
1048707 Is the national evaluation system as it exists feasible
-Probably a more realistic approach is to focus on key sectors programs and projects
1048707Does the evaluation system have the potential to drive service improvement
- It has potential to drive service improvements if the right techniques are applied
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
The Government has recognized the need for evidence based decisions on targets relating to service quality
Thus The New Performance Appraisal System (PAS) has become a key element of the governmentrsquos quality service strategy
From 2004 the PAS has expanded from a pilot of 16 commercial enterprises to cover to cover the entire public service
PAS is based on the need for sustainable efficient effective and quality public service delivery
Background InformationBackground Information
Background Information (Conthellip)Background Information (Conthellip)
PAS requires every public institution to outline its Citizen Service Delivery Charter which empowers the public to demand and to expect specific service standards
The PAS assessments are intended to lead to improvements in service quality particularly among those institutions that failed to meet previous performance targets
Background Information (Conthellip)Background Information (Conthellip)
PAS focuses on 3 parameters
1Performance and quality of service
2 Involvement of service consumers
3 Comparative ranking of institutions by performance
Measurement ApproachesMeasurement Approaches
The government currently uses customers satisfaction information to guide operational decisions to improve service
The range of measurements used include
bullBenchmarking
bullGap analysis
bullRanking based on composite score
Downsides of Current Evaluation TechniqueDownsides of Current Evaluation Technique
The recent PAS ranking demonstrated that the performance appraisal process presented challenges which require review of the process
In a number of some instances there has been a disconnect between comparative rankings and citizensrsquo perceptions of services
Hence evaluation outcomes and rankings have been seen by many as politically contentious
Gaps in the Evaluation ProcessGaps in the Evaluation Process
- In defining survey target group it is evident that the ldquocustomerrdquo concept is largely inapplicable in the public service setting
-Evaluation does not take into consideration resource allocation decisions both in terms of amount and efficiency of disbursement
-Evaluative effort does not adequately examine the underlying need for new services
- Poor coordination has meant limited consistency on tracking of key indicators as well as definition of survey target group
Overarching QuestionsOverarching Questions
The overarching questions are
1048707 Is the national evaluation system as it exists feasible
-Probably a more realistic approach is to focus on key sectors programs and projects
1048707Does the evaluation system have the potential to drive service improvement
- It has potential to drive service improvements if the right techniques are applied
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
Background Information (Conthellip)Background Information (Conthellip)
PAS requires every public institution to outline its Citizen Service Delivery Charter which empowers the public to demand and to expect specific service standards
The PAS assessments are intended to lead to improvements in service quality particularly among those institutions that failed to meet previous performance targets
Background Information (Conthellip)Background Information (Conthellip)
PAS focuses on 3 parameters
1Performance and quality of service
2 Involvement of service consumers
3 Comparative ranking of institutions by performance
Measurement ApproachesMeasurement Approaches
The government currently uses customers satisfaction information to guide operational decisions to improve service
The range of measurements used include
bullBenchmarking
bullGap analysis
bullRanking based on composite score
Downsides of Current Evaluation TechniqueDownsides of Current Evaluation Technique
The recent PAS ranking demonstrated that the performance appraisal process presented challenges which require review of the process
In a number of some instances there has been a disconnect between comparative rankings and citizensrsquo perceptions of services
Hence evaluation outcomes and rankings have been seen by many as politically contentious
Gaps in the Evaluation ProcessGaps in the Evaluation Process
- In defining survey target group it is evident that the ldquocustomerrdquo concept is largely inapplicable in the public service setting
-Evaluation does not take into consideration resource allocation decisions both in terms of amount and efficiency of disbursement
-Evaluative effort does not adequately examine the underlying need for new services
- Poor coordination has meant limited consistency on tracking of key indicators as well as definition of survey target group
Overarching QuestionsOverarching Questions
The overarching questions are
1048707 Is the national evaluation system as it exists feasible
-Probably a more realistic approach is to focus on key sectors programs and projects
1048707Does the evaluation system have the potential to drive service improvement
- It has potential to drive service improvements if the right techniques are applied
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
Background Information (Conthellip)Background Information (Conthellip)
PAS focuses on 3 parameters
1Performance and quality of service
2 Involvement of service consumers
3 Comparative ranking of institutions by performance
Measurement ApproachesMeasurement Approaches
The government currently uses customers satisfaction information to guide operational decisions to improve service
The range of measurements used include
bullBenchmarking
bullGap analysis
bullRanking based on composite score
Downsides of Current Evaluation TechniqueDownsides of Current Evaluation Technique
The recent PAS ranking demonstrated that the performance appraisal process presented challenges which require review of the process
In a number of some instances there has been a disconnect between comparative rankings and citizensrsquo perceptions of services
Hence evaluation outcomes and rankings have been seen by many as politically contentious
Gaps in the Evaluation ProcessGaps in the Evaluation Process
- In defining survey target group it is evident that the ldquocustomerrdquo concept is largely inapplicable in the public service setting
-Evaluation does not take into consideration resource allocation decisions both in terms of amount and efficiency of disbursement
-Evaluative effort does not adequately examine the underlying need for new services
- Poor coordination has meant limited consistency on tracking of key indicators as well as definition of survey target group
Overarching QuestionsOverarching Questions
The overarching questions are
1048707 Is the national evaluation system as it exists feasible
-Probably a more realistic approach is to focus on key sectors programs and projects
1048707Does the evaluation system have the potential to drive service improvement
- It has potential to drive service improvements if the right techniques are applied
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
Measurement ApproachesMeasurement Approaches
The government currently uses customers satisfaction information to guide operational decisions to improve service
The range of measurements used include
bullBenchmarking
bullGap analysis
bullRanking based on composite score
Downsides of Current Evaluation TechniqueDownsides of Current Evaluation Technique
The recent PAS ranking demonstrated that the performance appraisal process presented challenges which require review of the process
In a number of some instances there has been a disconnect between comparative rankings and citizensrsquo perceptions of services
Hence evaluation outcomes and rankings have been seen by many as politically contentious
Gaps in the Evaluation ProcessGaps in the Evaluation Process
- In defining survey target group it is evident that the ldquocustomerrdquo concept is largely inapplicable in the public service setting
-Evaluation does not take into consideration resource allocation decisions both in terms of amount and efficiency of disbursement
-Evaluative effort does not adequately examine the underlying need for new services
- Poor coordination has meant limited consistency on tracking of key indicators as well as definition of survey target group
Overarching QuestionsOverarching Questions
The overarching questions are
1048707 Is the national evaluation system as it exists feasible
-Probably a more realistic approach is to focus on key sectors programs and projects
1048707Does the evaluation system have the potential to drive service improvement
- It has potential to drive service improvements if the right techniques are applied
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
Downsides of Current Evaluation TechniqueDownsides of Current Evaluation Technique
The recent PAS ranking demonstrated that the performance appraisal process presented challenges which require review of the process
In a number of some instances there has been a disconnect between comparative rankings and citizensrsquo perceptions of services
Hence evaluation outcomes and rankings have been seen by many as politically contentious
Gaps in the Evaluation ProcessGaps in the Evaluation Process
- In defining survey target group it is evident that the ldquocustomerrdquo concept is largely inapplicable in the public service setting
-Evaluation does not take into consideration resource allocation decisions both in terms of amount and efficiency of disbursement
-Evaluative effort does not adequately examine the underlying need for new services
- Poor coordination has meant limited consistency on tracking of key indicators as well as definition of survey target group
Overarching QuestionsOverarching Questions
The overarching questions are
1048707 Is the national evaluation system as it exists feasible
-Probably a more realistic approach is to focus on key sectors programs and projects
1048707Does the evaluation system have the potential to drive service improvement
- It has potential to drive service improvements if the right techniques are applied
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
Gaps in the Evaluation ProcessGaps in the Evaluation Process
- In defining survey target group it is evident that the ldquocustomerrdquo concept is largely inapplicable in the public service setting
-Evaluation does not take into consideration resource allocation decisions both in terms of amount and efficiency of disbursement
-Evaluative effort does not adequately examine the underlying need for new services
- Poor coordination has meant limited consistency on tracking of key indicators as well as definition of survey target group
Overarching QuestionsOverarching Questions
The overarching questions are
1048707 Is the national evaluation system as it exists feasible
-Probably a more realistic approach is to focus on key sectors programs and projects
1048707Does the evaluation system have the potential to drive service improvement
- It has potential to drive service improvements if the right techniques are applied
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
Overarching QuestionsOverarching Questions
The overarching questions are
1048707 Is the national evaluation system as it exists feasible
-Probably a more realistic approach is to focus on key sectors programs and projects
1048707Does the evaluation system have the potential to drive service improvement
- It has potential to drive service improvements if the right techniques are applied
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
Overarching QuestionsOverarching Questions
1048707 Are customer satisfaction surveys appropriate techniques in evaluating the public sector
- This very much depends on the sector However it does not adequately demonstrate the competitiveness in terms of the extent to which public expenditure reflect the likely
cost benefits
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions
Discussion QuestionsDiscussion Questions
What would be the advantages and downsides of a cost benefit analysis approach in evaluating public services
-Does it create increased accountability-Does it create ensure better distribution of resources
-Does it allow for more efficient decisions