a point system approach to program ranking

Hilario P. Martinez 1

ISSUES IN AN OBJECTIVE RANKING OF PROGRAMS AND PROJECTS

Difficulties in Finalizing the Annual Workplan


The Selection Dilemma


Which programs and projects do we need to consider for next year?

What criteria do we need to employ to select the most appropriate programs and projects for development and implementation?

Traditional Criteria in Prioritizing Programs and Projects

Financing Requirement

Schedule of Implementation

Sponsoring Institution


What influences the prioritization of social services?


The survey says …

Cashflow and

Funding issues

Concept issues

Collaboration issues



A political promise?

A political

response?

A political gimmick?



Hmmm … Who? What? How?

Why should I?



Can we make

it in time?

Who takes what?

Who gives what?

How do we put

things together?

A DATA-BASED SYSTEM OF PRIORITY RANKING OF PROGRAM/PROJECT IMPLEMENTATION


Criteria for Ranking Programs/Projects

Do-ability

Importance

Magnitude

Urgency

Relevance


The Criteria …

• relates to the capability and capacity of an organization to accomplish its planned targets given the limitation of its resources

D o - a b i l i t y

• refers to the qualitative degree of significance attributed to certain factors to contribute to mission critical

I m p o r t a n c e

• refers to the relative sizes and trends of a particular set of information

M a g n i t u d e

• refers to the degree of exigency of addressing a given situation U r g e n c y

• refers to the degree of congruence to the vision, mission and objectives of a service area

R e l e v a n c e


Weight per Criterion

Do-ability 5 pts

Importance 5 pts

Magnitude 5 pts

Urgency 5 pts

Relevance 5 pts


The Rating Scale …

5 points

Highest

4 points

2nd

Highest

3 points

Median

2 points

2nd

Lowest

1 point

Lowest


The Rating per PPA*


D

O

-

A

B

I

L

I

T

Y

I

M

P

O

R

T

A

N

C

E

M

A

G

N

I

T

U

D

E

U

R

G

E

N

C

Y

R

E

L

E

V

A

N

C

E

S

C

O

R

E

≤5 ≤5 ≤5 ≤5 ≤5 ≤25

* Program/Project/Activity

Agency Program/Project Assessment Form


Page 1 Page 2

Partner Agency Assessment Form

Engaging the active participation of partner agencies in the development and implementation of public sector programs and projects


DETAILS OF THE CRITERIA

The 5 Cs . . .

DOABILITY,

IMPORTANCE,

MAGNITUDE,

URGENCY and

RELEVANCE . . .


Do-ability • PPAs that can be

implemented given the limits of resources and circumstances affecting it

• Must be responded to

by proponent agency

• 90%+ of required resources are available 5 pts

• 80-89% of required resources are available 4 pts



• 50-59% of required resources are available 1 pt


Rating Level:

Doability – Rating Reference


Doability Sample Rating Worksheet (Proponent Agency)


Doability Sample Rating Worksheet (Partner-Agency*)


* - Assuming 1 Partner-Organization per PPA. If more than 1, apply the same process and average rates obtained

Doability Sample Rating Worksheet (Average of Proponent and Partner-Agency)


Importance

The qualitative degree of significance attributed to certain factors to contribute to mission critical programs

Each will determine, according to its mandate and perspective, the degree of importance a joint engagement entails to them

•Highest imperative & value to the agency’s mandate 5 pts

•2nd highest degree of significance 4 pts

•3rd or moderate degree of significance 3 pts

•4th or minor degree of significance 2 pts

•least degree of significance 1 pt


Priority Ranking:

How would you

rate the level of importance of this

PPA to your agency?

Importance – Rating Reference


81% - 100%

61% - 80%

41% - 60%

21% - 40%

01% - 20%

Degree of Importance

How does your Partner-agency rate the level of importance of this PPA to them?

C – Partner-Agency

P – Proponent Agency

Proponent Agency

P5

P4

P3

P2

P1

C5

C4

C3

C2

C1

Partner-Agency

Importance – Rating Matrix

5.0 - C5 & P5

4.5 - C5 & P4, C4 & P5

4.0 - C5 & P3, C4 & P4, C3 & P5

3.5 - C5 & P2, C4 & P3, C3 & P4, C2 & P5

3.0 - C5 & P1, C4 & P2, C3 & P3, C2 & P4, C1 & P5

2.5 - C4 & P1, C3 & P2, C2 & P3, C1 & P4

2.0 - C3 & P1, C2 & P2, C1 & P2

1.5 - C2 & P1, C1 & P2

1.0 - C1 & P1


H I G H

L O W

COMBINATION OF POSSIBLE RESPONSES PER RATE

C – Client-Partner or

Target Sector

P – Proponent Agency

Importance – Sample Rating Worksheet


Magnitude - the relative sizes and trends of

affected beneficiary sector


Determining whether a factor has an increasing or decreasing trend is a function of time series

Determining whether a factor is increasing or decreasing in size compared to another is a function of quantity

The combined ratings for trend and size comprises the rating for “Magnitude"

Priority Ranking:

5 points Very Significant

4 points Significant

3 points Acceptable

2 points Slightly Acceptable

1 point Not Acceptable

What tendency does the analysis of the data disclose? Is it increasing, same as before or decreasing?

Magnitude – Rating Reference


How does the latest data set compare with previous year, bigger, the same or smaller?

• 51% to 100% greater S5=5

• 11% to 50% greater S4=4

• -10% to 10% deviation S3=3

• -11% to 50% lesser S2=2

• - 51% to -100% lesser S1=1

Decreasing T1=1

Increasing T2=2

Size (Quantity) Trend (Time Series) (at least 3 years data set)

Magnitude – Rating Matrix


S – Size

T – Trend

(S x T) / 2

H I G H

L O W

CO

MB

INA

TION

OF P

OSSIB

LE R

ESPO

NSES P

ER P

OIN

T SCO

RE

5 = T2 & S5 4 = T2 & S4 3 = T2 & S3

2.5 = T1 & S5

2 = T2 & S2, T1 & S2

1.5 = T1 & S3

1 = T2 & S1, T1 & S2

0.5 = T1 & S1

Magnitude – Sample Rating Worksheet


Urgency - time consideration or response that must be

made to a given problem or issue over time

5

Most urgent

Within the next

2 months

4

Urgent

Within the next

3 to 4 months

3

Moderately urgent

Within the next

5 to 6 months

2

Least urgent

Within the next

7 to 8 months

1

Not urgent

Within the next 9 to 10 months


This criterion should be responded to by the proponent organization, but preferably in consultation with the client-organization or beneficiary sector

Urgency – Sample Rating Worksheet


Relevance - how a project relates to the vision, mission

and objectives of the service area

5

Most relevant

Beneficiary impact

estimated at 81-100%

4

Relevant

Beneficiary impact

estimated at 61-80%

3

Moderately Relevant

Beneficiary impact

estimated at 41-60%

2

Less Relevant

Beneficiary impact

estimated at 21-40%

1

Not Relevant

Beneficiary impact

estimated at 01-20%


This criterion should be responded to by the proponent organization, but in consultation with the client-organization or beneficiary sector

Relevance – Sample Rating Worksheet


DETERMINING THE PRIORITY RANKING


Putting the Scores together


Priority Rating

Doability

Importance

Magnitude

Urgency

Relevance

Ranking Summary Worksheet


+ + + + +

S O R T

D E S C E N D I N G

Sample Summary Rating Worksheet


Determining the qualified PPAs for prioritization


PPA1 16.5 4

PPA2 14 7

PPA3 19 2

PPA4 11.5 9

PPA5 18 3

PPA6 15 6

PPA7 12.5 8

PPA8 11 10

PPA9 16 5

PPA10 20 1

PPAsPoints

EarnedRanking

PPA10 20 1

PPA3 19 2

PPA5 18 3

PPA1 16.5 4

PPA9 16 5

PPA6 15 6

PPA2 14 7

PPA7 12.5 8

PPA4 11.5 9

PPA8 11 10

PPAsPoints

EarnedRanking

Median = 15.5

Arranged by PPA # Arranged by Points Earned

QU

ALIFIED

PPA

s

What if Magnitude (Size & Trend) could be a factor for additional point/s?


NOTE: The values for Mag(Size) per PPA are for illustrative purposes only, but the ratings for Mag(size) and Mag(trend) are the same with previous data used.

‘

Total Additional Points due to Comparative Implication of Magnitude (Size and Trend)


Values of Magnitude(Size) are ranked, only the top 5 are rated in descending order (1, 0.8, 0.6, 0.4 & 0.2)

Only Magnitude (Trend) with a score of 2 is rated 0.5 for bonus point

Final Total Points Earned per PPA


PPAs’ Final Priority Ranking


Set Median = 15.5

Top

mo

st eligib

le P

PAs

for p

rioritizatio

n

WHAT ADVANTAGES DOES THIS CONCEPT OFFER A PROPONENT?


Advantages of adopting this approach to program/project ranking


The other factor

It discourages, minimizes, if not

totally prevent, the adoption of

“QUESTIONABLE” P.P.As from being considered and

funded for implementation

The

mo

st im

po

rta

nt

ad

van

tag

e o

f u

sin

g t

his

a

pp

lica

tio

n –


a point system approach to program ranking

Government & Nonprofit