design scoring methodology process ii · consultants who express interest in working on nysdot...
TRANSCRIPT
i
Design Scoring Methodology
PROCESS II Effective for Electronic Expressions of Interest due on or after December 1, 2014
Updated on: 06/26/2020
Table of Contents
INTRODUCTION......................................................................................................................... 1
DESIGN SCORING METHODOLOGY ................................................................................... 2
1. General Approach to scoring firm experience. ...............................................................2
2. How is the firm’s experience in their design project inventory scored? .........................2
3. What happens if there is consultant inventory information missing from the DOT
database? ..............................................................................................................................3
4. How are joint ventures handled?.....................................................................................3
5. How does a very high level of subconsultant participation affect scoring of Factors
1A, 1C and 2A?....................................................................................................................3
FACTOR 1A. PRIOR EXPERIENCE FIRM IN GENERAL ................................................. 3
A. Major category of work (except survey and mapping): ..................................................3
B. Survey and Mapping component of Factor 1A: ..............................................................4
C. Project specific environmental work component of Factor 1A: .....................................4
SCORING TABLES ..................................................................................................................... 5
Table A Construction $ Value of Highway Work: .............................................................5
Table A.1 Construction $ Value of Bridge Work: ................................................. 5
Table B Complexity: ...........................................................................................................6
Table C Project Owner:.......................................................................................................6
Table C.1 (for scoring 1C) Project Owner: ............................................................ 6
Table C.2 (apply to the total team score for 1A, 1C and 2A) % Subconsultant
Work: .......................................................................................................... 7
Table C.3 Work class factor for major categories of work except survey and
mapping: ..................................................................................................... 7
Table D Credit for the same or similar task under major categories of work: ....................7
Table E.1 & E. 2 Survey & Mapping component of 1A, 1C & firm experience
component of 6A..................................................................................................................8
Table E.1 Work class = Highway, bridge, highway & bridge, or appurtenance. .. 8
Table E.2 Work class = Other ................................................................................ 8
Table F Project specific environmental work component of 1A, 1C & firm experience
component of 6A..................................................................................................................9
Table F.1 (Project specific environmental work) Credit for same or similar
environmental document. .......................................................................... 10
Table F.1A (Partial credit modifier for claiming only EIS Chapter IV or EA
Chapter IV) ............................................................................................... 10
Table F.2 (Defines when a project is too old to score) ........................................ 10
Table F.3 (Individual experience factor) .............................................................. 10
ii
FACTOR 1B. PRIOR EXPERIENCE STAFF ........................................................................ 10
FACTOR 1C. PRIOR EXPERIENCE WITH DOT ................................................................ 10
FACTOR 2A. PRIOR NYSDOT PERFORMANCE RATINGS ........................................... 10
Table F.4 Weighting used in calculating the team scores for prior NYSDOT performance
evaluations. ........................................................................................................................12
FACTOR 2B1 (Workload with Division) and 2B2 (Workload with Department) ..................... 14
FACTOR 3. LOCATION ........................................................................................................... 15
Table G Outside NYC Metropolitan Area ........................................................................15
Table G.1 NYC Metropolitan Area...................................................................... 16
FACTOR 4. OTHER FACTORS ............................................................................................. 16
1. Standard Factors (except small firm credit): .................................................................16
2. Small firm credit: ..........................................................................................................16
3. Special factors are scored differently than standard factors: ........................................17
FACTOR 5. D/M/WBE PARTICIPATION ............................................................................. 17
FACTOR 6. D/M/WBE SUBCONSULTANTS ........................................................................ 17
6A. Performance And Experience .....................................................................................17
6B. Workload With Division .............................................................................................18
FACTOR 7. TIME SINCE LAST DESIGNATION ................................................................ 19
OPTION TO MODIFY THE SCORING BASE OF THE PROGRAM FOR A SPECIFIC
PROJECT .................................................................................................................................... 21
MORE ABOUT THE SCORING TABLES ............................................................................. 21
1
INTRODUCTION
Consultants who express interest in working on NYSDOT design projects are evaluated in a two-
step process. The first step is evaluation by an automated process which produces a shortlist of
the best-qualified teams. The second step is evaluation by a committee of experts who consider
only the teams which have been shortlisted. From the results of these evaluations, NYSDOT
Executive Management selects the best-qualified team for designation.
The evaluation process considers the following rating factors. Each factor is given a weight as
appropriate for the specific project, and all factors and weights are described in the project
advertisement (there are normally two sets of weights, one for the automated evaluation step and
one for the committee evaluation step). For each step, consultant teams’ individual scores for
each factor are multiplied by the weight for the factor, and then the weighted scores are added to
produce a final aggregate score for the team. The weights for Factors 1 thru 4 will total 100%.
Factors 5 thru 7 are bonus or penalty points that are added to or subtracted from the aggregate
team score.
1) Prior experience
a) Firm in general
b) Staff
c) With NYSDOT
2) Organizational capability for this type of work
a) Prior NYSDOT performance ratings
b) Workload/Capacity
i) Workload with NYSDOT division vs. capacity
ii) Total remaining workload with NYSDOT
3) Logistics & Familiarity
a) Location
b) Familiarity
4) Other Factors (optional)
a) A. Standard
b) B. Special
c) C./D./E. Technical Proposal
d) F. Oral Presentation
5) D/M/WBE Participation
6) D/M/WBE Subconsultants
a) Performance and Experience
b) Workload with NYSDOT division
7) Time since last designation
Rating Factors 1B (staff experience), 3B (familiarity) and, if included, 4C thru 4F are evaluated
only in the committee evaluation step and are not discussed further here. The methodology used
by the automated process to score the remaining factors is presented on the following pages.
2
DESIGN SCORING METHODOLOGY
The tables shown are examples only. The Department may adjust table values for a specific
project. Any changes will be noted in the project advertisement.
1. General Approach to scoring firm experience.
The scoring system is based on the idea that each time a firm performed a service we are looking
for on a project, that firm earns points toward their score. The scoring program has tables which
determine how many points are earned each time a service was provided (up to a maximum
possible score of 10).
The numbers in the scoring tables were based on judgment in estimating how many times a
service needs to be provided, to score a 10. For example, if providing preliminary bridge design
five times was considered a score of 10, then each time the service was provided would earn two
points.
If a project in the consultant’s inventory exceeds a certain age, then that project is not used by
the scoring program. (Refer to table F.2)
2. How is the firm’s experience in their design project inventory scored?
The Department assigns, as appropriate for a specific project, a weight to each task under major
categories of work, project specific environmental work done by the consultant and special
factors.
For each firm doing a task with a weight, the scoring program reviews each project in the
consultant’s inventory to determine if they have done the same or a similar task.
If the consultant has done the same or similar task on a project, the program gives points to the
firm for that task.
After the program reviews and scores all tasks on the projects in the consultant’s inventory, the
ten highest scores for each task are added together (capped at a maximum score of 10). For
specific projects the Department may vary the number of projects to be scored from the
inventory.
The team score for each work task will be a weighted average of the scores for each firm that
will be doing that work task, based on the overall percentage of work that each firm is proposed
to do. If no team member is proposed to do a required task, then the team will be given a “0"
score for that task.
NYSDOT’s Integrated Contract Management System (ICMS) is used by the NYSDOT program
area to indicate what factors will be considered and what weight they will have. The following
categories are each scored differently: Major categories of work (excluding survey and
mapping), Survey and Mapping, Project Specific Environmental Work, Standard Special Factors
and Non-standard Special Factors.
3
3. What happens if there is consultant inventory information missing from the DOT
database?
If a subconsultant has no active design project inventory, then the program will give the
subconsultant a score of “0" for factors 1A and 1C and the standard factor components of
factor 4. For factor 6A the components of the score from 1A and 1C and the standard
factor components of factor 4 will also have a score of “0".
If a prime consultant or joint venture firm has no active design project inventory, their
EOI will not be processed for scoring.
If a firm does not have an active certified salary roster, the scoring program will assume
the firm had 0 staff for calculating factor 2B1 (Workload with division) and factor 7 (Last
designation).
4. How are joint ventures handled?
Since a joint venture (JV) is viewed as one firm, if joint ventures are encouraged, firms making
up the joint venture, are permitted to select from among all of each firm's project inventories, up
to 30 projects, that they want to be considered for scoring, to make up the JV project inventory.
For a joint venture, the JV will indicate on their electronic Expression of Interest (e-EOI) the 30
projects contained in the JV firms' inventory that will be scored.
5. How does a very high level of subconsultant participation affect scoring of Factors 1A,
1C and 2A?
The level of subconsultant participation has no impact on scoring unless specified in the project’s
detailed advertisement. See table C.2.
To account for the added management complexity and performance issues related to high
subconsultant usage, table C.2 factors are applied to the total score for the team for each factor
1A, 1C and 2A.
FACTOR 1A. PRIOR EXPERIENCE FIRM IN GENERAL
A. Major category of work (except survey and mapping):
Develop a raw score for each major category of work (except survey and mapping) for each
project in the consultant’s project inventory.
For Planning Highway Study, Project Scoping Highway, Preliminary Highway Design or Final
Highway Design:
Raw score for each major category of work = (Table A*(cost weighting) +Table B*
(complexity weighting)) *(Table C) *(Table C.3) *(Table D) *(Table F.3).
For Planning Bridge Study, Project Scoping Bridge, Preliminary Bridge Design, Final Bridge Design:
Raw score for each major category = (Table A.1* (cost weighting) +Table B* (complexity
weighting) *(Table C) *(Table C.3) *(Table D) *(Table F.3).
4
Note: For each highway or bridge category of work on each project:
If the value from Table A=0, then the score from Table B becomes 0 for highway work.
If the value from Table A.1 =0, then the score from Table B becomes 0 for bridge work.
Each project in the inventory has a primary work class: highway; bridge; highway and
bridge; appurtenance or other.
If the work class is “other”, a score of 0 is given for each major category (except survey
and mapping)
If the work class is “appurtenance”, their score for each major category (except survey
and mapping) is multiplied by 0.5
If the work class is “appurtenance”, the complexity is routine.
Firms that were subconsultants on individual projects will have the construction $ value
of that project prorated at their percent of responsibility, if they are being proposed as a
prime. Also, the scores for both construction $ value and complexity are multiplied by a
“sub factor” of 0.85 (1.00 if the project is routine and the estimated construction cost is
under $15M).
The cost weighting is 0.50 and the complexity is 0.5 except:
If complexity of the target project is routine and the cost is moderate or large the
cost/complexity weighting is 1.00/0.00.
If complexity of the target project is moderately complex or complex and the cost
large the cost/complexity weighting is 0.70/0.30.
B. Survey and Mapping component of Factor 1A:
Determines a raw score for design or ROW survey and mapping for each project in the
consultant’s project inventory = Table E (No project owner factor is applied) *(Table F.3).
C. Project specific environmental work component of Factor 1A:
Develops a raw score for each environmental task for each project in the consultant’s project
inventory = Table F * Table C *(Table F.3).
5
SCORING TABLES
Note: The “sub factor” is 1.00 if the project is routine and the estimated construction cost is under $15
Million.
Table A Construction $ Value of Highway Work:
Firms that were subconsultants on individual projects will have the construction $ value of that
project prorated at their percent of responsibility, if they are being proposed as a prime. In
addition, the scores for construction $ value will be multiplied by a “sub factor” of 0.85.
Construction $ Value of
Highway Work for
Project where Consultant
Services are Needed
Construction $ Value of Highway Work for Project Submitted
<$2M $2M-$10M $10M-$30M >$30M
<$2M 1.2 1.1 1.0 0.9
$2M-$10M 0.6 1.2 1.1 1.0
>$10M -$30M 0.0 0.6 1.4 1.4
>$30M 0.0 0.1 0.5 2.0
Table A.1 Construction $ Value of Bridge Work:
Firms that were subconsultants on individual projects will have the construction $ value of that
project prorated at their % responsibility, if they are being proposed as a prime. In addition, the
scores for construction $ value will be multiplied by a “sub factor” of 0.85.
Construction $ Value of Bridge
Work for Project where
Consultant Services are Needed
Construction $ Value of Bridge Work for Project Submitted
<$1M $1M-$10M $10M-$20M >$20M
<$1M 1.2 1.1 1.0 0.9
$1M-$10M 0.6 1.2 1.1 1.0
>$10M-$20M 0.0 0.6 1.4 1.4
>$20M 0.0 0.1 0.5 2.0
Note on Table A and A1: When the cost of the target project is at the low end of a range, the
ranges for the "Construction $ Value of Bridge Work for Project Submitted" will be adjusted so
that submitted projects with a cost at the high end of the next range down will be scored as if
they are equivalent.
6
Table B Complexity:
Firms that were subconsultants on individual projects will have the score for complexity
multiplied by a “sub factor” of 0.85, if they are being proposed as a prime. On services for non-
routine projects, if a firm is applying for work as a prime and the firm’s responsibility for a
project in their inventory of experience was a subconsultant, then the complexity is reduced one
level, from that which was appropriate for a prime on a given inventory project.
Complexity of Project where
Consultant Services are Needed
Complexity of Project Submitted
Routine Moderately Complex Complex
Routine 1.2 1.1 1.0
Moderately Complex 0.6 1.4 1.4
Complex 0.1 0.5 2.0
Table C Project Owner:
Project Owner Project Owner Factor
NYSDOT 1.0
NYS Thruway or other Transportation Authority in NYS 1.0
All NYCDOT or other Municipality in NYS if Fed. Aid 1.0
Primary Transportation Agency of Another State 0.9
Municipality Within NYS (Not Fed. Aid) 0.7
Other (Includes permit work on NYSDOT facilities). 0.5
Table C.1 (for scoring 1C) Project Owner:
Project Owner Project Owner Factor
NYSDOT 1.0
NYS Thruway or another Transportation Authority in NYS 1.0
All NYCDOT or other Municipality in NYS if Fed. Aid 1.0
Primary Transportation Authority of Another State 0.0
Municipality Within NYS (Not Fed. Aid) 0.0
Other (Includes permit work on NYSDOT Facilities). 0.0
7
Table C.2 (apply to the total team score for 1A, 1C and 2A) % Subconsultant Work:
Sub % Factor
Estimated. Construction cost <= $6M Estimated Construction cost over $6M
31-35 46-50 1.0
36-40 51-55 1.0
>40 >55 1.0
Table C.3 Work class factor for major categories of work except survey and mapping:
Work Class of Project
where Consultant
Services are Needed
Work class of Project Submitted
Highway Bridge Highway and
Bridge
Appurtenance Other
Highway 1.0 1.0 1.0 0.5 0.0
Bridge 1.0 1.0 1.0 0.5 0.0
Highway and Bridge 1.0 1.0 1.0 0.5 0.0
Appurtenance 1.0 1.0 1.0 1.0 0.0
Table D Credit for the same or similar task under major categories of work:
The following factors will be used for allowing credit for the Consultant Project showing the same or
a similar task under major categories of work. Apply only one score, the score for the task with the
highest applicable factor. The task will be scored and then the score multiplied by the applicable
factor from the table.
Project Where
Consultant
Services are
Required
Consultant Project
Planning
Highway
Planning
Bridge
Scoping
Highway
Scoping
Bridge
Preliminary
Highway
Preliminary
Bridge
Final
Highway
Final
Bridge
Planning
highway
1.0 0.0 0.5 0.0 0.1 0.0 0.0 0.0
Scoping for
Highway
0.5 0.0 1.0 0.0 0.5 0.0 0.1 0.0
Scoping for
Bridge
0.0 0.5 0.0 1.0 0.0 0.5 0.0 0.1
Preliminary
Highway
0.1 0.0 0.5 0.0 1.0 0.0 0.5 0.0
Preliminary
Bridge
0.0 0.1 0.0 0.3 0.0 1.0 0.0 0.5
Final
Highway
0.0 0.0 0.1 0.0 0.5 0.0 1.0 0.0
Final Bridge 0.0 0.0 0.0 0.1 0.0 0.5 0.0 1.0
8
Table E.1 & E. 2 Survey & Mapping component of 1A, 1C & firm experience component of
6A.
For 1A, 1C and 6A, survey and mapping will be scored as follows, each time services were
performed (sum score for 10 highest projects, maximum score of 10). For scoring 1C, apply a
project owner factor from Table C.1
Table E.1 Work class = Highway, Bridge, Highway & Bridge, or Appurtenance.
NYSDOT All NYCDOT
Or Other
Municipality in
NYS If Fed. Aid
NYS
Thruway
Authority
Municipality
Within NYS
(Not Fed. Aid)
Other
In NYS
State
DOT
Outside
NYS
Other
Outside
Of NYS
Design
Survey &
Mapping 1.0 1.0 1.0 0.5 0.5 1.0 0.5
ROW
Survey &
Mapping 1.0 1.0 1.0 0.5 0.0 0.5 0.0
Table E.2 Work class = Other
NYSDOT All NYCDOT
Or Other
Municipality in
NYS If Fed. Aid
NYS
Thruway
Authority
Municipality
Within NYS
(Not Fed. Aid)
Other
In NYS
State
DOT
Outside
NYS
Other
Outside Of
NYS
Design
Survey &
Mapping 0.0 0.0 0.0 0.0 0.0 0.0 0.0
ROW
Survey &
Mapping 0.0 0.0 0.0 0.0 0.0 0.0 0.0
9
Table F Project specific environmental work component of 1A, 1C & firm experience
component of 6A.
Each time a firm performed the specific environmental work task, they will be given a score of
0-1. The maximum score is 10. For scoring 1A, the score will be modified by a factor for Project
Owner from table C. For scoring 1C, the Project Owner factor for will be taken from Table C.1.
Bridge, Highway or
Appurtenance Projects
Other Projects
EIS 1 * Table F.1 *TableF1A 0.5 * Table
F.1*TableF1A EIS Chapter IV
EA 1 * Table F.1*TableF1A 0.5 * Table
F.1*TableF1A EIS Chapter IV
Noise Study 1.0 0.0
Air Quality Study 1.0 0.0
General Ecological Study 1.0 1.0
Endangered Species Study 1.0 1.0
Surface Water Bodies Study 1.0 0.0
Wetlands Study 1.0 0.5
Navigable Waters Study 1.0 0.5
Coastal Zone Management 1.0 0.5
Wild, Scenic and Recreational Rivers Study 1.0 0.5
Storm Water Management 1.0 0.0
Ground Water Study 1.0 0.5
Flood Plain Evaluation 1.0 1.0
Cultural Resources Survey 1.0 0.5
Section 4(f) Evaluation 1.0 0.0
Section 6(f) Evaluation 1.0 0.0
Hazardous Waste Materials Screening 1.0 0.5
Hazardous Waste Materials Assessment 1.0 0.5
Asbestos Preliminary Investigation 1.0 1.0
Asbestos Sampling, Testing and Specifications 1.0 0.5
Farmland Protection Assessment 1.0 0.5
Energy Analysis 1.0 0.0
Visual Impact Assessment 1.0 0.0
Adirondack Park Agency Involvement 1.0 0.5
Landscape Design 1.0 1.0
10
Table F.1 Project specific environmental work.
Credit for same or similar environmental document.
The following factors will be used for allowing credit for doing the same or similar
environmental document. That factor will be used in Table F.
Project Where Consultant
Services are Required
Consultant Project
EIS or EIS
Chapter IV
EA OR EA
Chapter IV
EIS (no EA) 1.0 0.5
EA (no EIS) 1.0 1.0
Table F.1A Partial credit modifier for claiming only EIS Chapter IV or EA Chapter IV
If the firm does not claim EIS or EA credit, but does claim EIS Chapter IV or EA Chapter IV,
then apply Table F.1A to the formula in Table F.1
EIS Chapter IV 0.30
EA Chapter IV 0.30
Table F.2 Defines when a project is too old to score
When no longer score x (in years)
Present year - Year completed > x 12
Table F.3 Individual experience factor
Type of experience Factor
Individual Experience 0.6
FACTOR 1B. PRIOR EXPERIENCE STAFF
This is not a factor for shortlisting purposes.
FACTOR 1C. PRIOR EXPERIENCE WITH DOT
• Uses the same methodology at described under FACTOR 1A except the project owner
factor is taken from Table C1 instead of Table C. A project owner factor from Table C1 is
applied to survey and mapping.
FACTOR 2A. PRIOR NYSDOT PERFORMANCE RATINGS
• Calculate a weighted average of the consultant’s performance evaluations for the past six
(6) years based on the selection factor weights for Factor 1A and 1C subfactors and 4A
factors.
11
The computer always retrieves all highway and bridge evaluations. Based on the project’s
advertised factors the computer also retrieves other applicable evaluations as listed below.
A firm score is calculated for each 1A and 1C subfactor and for each 4A factor. The
scoring for each subfactor or factor only uses the evaluations applicable to that
subfactor/factor. If no applicable evaluations are available, use the highway or bridge
evaluation score, otherwise use the other evaluations that are applicable to the project.
Then the program calculates the weighted average of the firm’s scores for the
subfactors/factors.
• Applicable Evaluations:
Highway work - Highway & Other
Bridge work - Bridge & Other
Survey & Mapping - Highway, TASS, Other
Environmental - Highway, Other
Other Standard Factors-
Traffic Forecasting - Planning, Highway, Other
Soil & Geologic Studies, Foundations - Highway, Bridge, Other
Moveable Bridge Design - Bridge
Bridge Carrying a Railroad- Bridge
Historic Bridge Design- Bridge
Final Design Urban Arterial Interconnected Signal Design - Highway
Final Design Retaining Walls - Over 10 Meters in height - Highway, Bridge
Final Design of Complex MPT: Over 75 sheets- Highway
Highway Lighting Final Design - Highway, Other
Extensive Capacity Analysis of Urban Freeway Mainline and Interchanges - Highway
Community Participation- Highway, Other
Involvement with NYC Agency - Highway, Bridge, Other
ITS/Electronic Hwy Operations- Highway, Other
Intermodal Scoping/Design Analysis- Planning, Highway
Non-standard special factors - Highway, Bridge, TASS, Other
• Only include performance evaluations related to weighted tasks.
• For primes (or JV) only prior performance ratings as a prime will be averaged.
• For JV’s, first calculate the average performance evaluation score for each JV partner.
Then take the score for each partner and average them, giving each JV partner’s score
equal weight.
12
• If firms have no evaluations: Give a prime (or JV partner) the mode of all other primes (or
JV partners) that submit an e-EOI for the project. Give a subconsultant the mode of all
other subconsultants contained in an e-EOI for the project. If a mode cannot be calculated,
give the firm a score of 5. The team score is a weighted average of each firm’s score based
on weightings in table F.4.
Table F.4 Weighting used in calculating the team scores for prior NYSDOT performance
evaluations.
Weighting A
Prime (or Joint Venture) % participation
Each Subconsultant % participation * A 0.5
Calculation Steps for a firm’s performance evaluation scoring for Factor 2A
NOTE:
• Each task will be scored and then a weighted score calculated for each firm on an EOI.
• Treat the Primes and JV partners equally.
1. Get target list of the project’s weighted and unweighted major categories of work
(including survey & mapping components), environmental tasks, and standard other
factors.
2. Get list of firm’s applicable evaluations based on the above-mentioned target list.
For design projects always consider both Highway and Bridge evaluations as applicable.
For firms proposed as Prime (or JV partner) consider only their evaluations as Prime and
JV partner as applicable.
For firms proposed as Sub-consultant consider all their evaluations.
3. If the firm has no applicable evaluations, score as mode:
Firms performance evaluation score = Give the prime (or JV partner) the mode of all
other primes and JV partners that submit an e-EOI for the project. Give a sub-consultant
the mode of all other sub-consultants contained in an e-EOI for the project. If a mode
cannot be calculated, give the firm the current performance score default mode (5).
13
Stop if the firm had no evaluations.
4. If current participant is responsible for weighted task(s) for 2A, or if current participant is
responsible for only un-weighted task(s) for 6A, score the task(s).
A. Sum of the performance evaluation rating totals for each category that applies to the
task.
Example: for task Preliminary Highway, the applicable performance evaluation
categories to sum are Design Highway, Highway Design, and Traffic Systems.
Or, if current participant has no applicable evaluations for the task:
A = The current participant’s performance evaluation ratings total for Highway Design
category is used or, if one does not exist, use their total for Bridge Design category.
Otherwise use the sum of the firm’s evaluations that apply to the project, as
determined under 2 - this includes all required factors for the EOI.
B = Sum of the number of performance evaluation ratings that were totaled to produce the
performance evaluations ratings stored in A.
G = Firm score for this task = (A / B)
If scoring Factor 6A, stop here and go directly to step 6.
C = Sum of the % work proposed in the EOI for each firm that will perform the task.
D = % work of the current participant.
E = Factor weight for standard other factors (4A tasks only) or
Factor weight (1A+1C) * Attribute (sub-factor) weight
F = Adjusted weight = (D/C) * E
5. If scoring Factor 2A, weight the task score: H = Weighted task score (F * G)
6. After computing a (weighted for 2A or un-weighted for 6A) score for each task a firm will
perform, calculate:
Firm’s performance evaluation score =
For Factor 2A calculation: (Sum the value of H for each task / Sum the value of F for each task)
For Factor 6A calculation: (Sum the value of G for each task / Number of G scores summed)
Note: If the firm is not responsible for any weighted factors, the firm’s performance evaluation
will be the average of the unweighted task scores.
14
FACTOR 2B1 (Workload with Division) and 2B2 (Workload with Department)
For rating factor 2B1, use current remaining division workload with NYSDOT submitted by the
prime (or joint venture members) and sub(s). Consistent with the instructions for completing form
CONR 386, “Current Workload and Last Designation Disclosure”, for Biennial Bridge Inspection
and East River Bridge Inspection agreements, once a draft agreement has been accepted by
signature of the prime consultant or JV and received by the NYSDOT's Contract Management
Bureau, the consultant shall not include the non-expendable direct non-salary cost items such as
equipment rental and sub-contracting of Work Zone Traffic Control (WZTC) services.
Non-expendable direct non-salary costs are defined as goods and services rented, leased, or
purchased which are not consumed for project work.
If the design workload for the prime (or summation of the workload for all members of a joint
venture) is >$1 million, it is multiplied by a Capacity Factor which can reduce the prime’s
workload (or summation of the workload for all members of a joint venture) by up to 50%, but
never to less than $1 million. For the capacity factor to apply, the numerator must be less than the
denominator.
*Capacity Factor =
Median number of design personnel in NYS or NYC metro area for all
consultant firms in the current NYSDOT database with >34 employees
Number of design personnel in NYS or NYC metro area for prime (or
total number of design personnel reported by the members of a joint
venture)
*(Exclude from the calculation any personnel on the salary roster that show a termination date.
For upstate projects, exclude from the median calculation firms with only downstate staff. For
downstate projects exclude from the median calculation firms with only upstate staff. The
calculation of the median number of design personnel should not include firms if their total of
highway design personnel plus structural design personnel equals 0.)
Number of design personnel will be as follows:
Only highway personnel if highway/appurtenance project. (Salary roster 01 = Highway Design)
Only structural personnel if bridge project only. (Salary roster 02 = Structural Design.) Both
highway and structural personnel if highway/appurtenance and bridge project. (Salary roster 01 =
Highway Design, Salary roster 02 = Structural Design.). If a firm does not have an active certified
salary roster, the scoring program will assume 0 design personnel in NYS or NYC metro area.
For rating factor 2B2, use current remaining workload with NYSDOT submitted by prime (or
joint venture members) and sub(s). For joint ventures, use the summation of the workload of all
members of a joint venture.
15
For both rating factors 2B1 and 2B2, multiply the workload (or adjusted workload for 2B1 if
applicable) for the prime (or joint venture) and sub(s) by their respective % of work, sum up to
get team workload, then score team based on the following table.
FACTOR 3. LOCATION
Score each firm for distance to project and for distance to Regional Office based on Table G.
Average these scores to calculate a score for each firm. Then calculate the team score by a weighted
average based on the % participation of each firm. Use Table G for projects outside the New York
City metropolitan area. Use Table G.1 for projects in the New York City metropolitan area.
Table G Outside NYC Metropolitan Area
Miles Score
0-65 10
66-95 9
96-125 8
126-155 7
156-190 6
191-220 5
221-250 4
251-280 3
281-315 2
316-345 1
>345 0
Workload Score Workload Score
>$6 million 0 $801 thousand-$1.100 million 6
$4.001 million-$6.000 million 1 $501 thousand - $800 thousand 7
$2.001 million-$4.000 million 2 $201thousand - $500 thousand 8
$1.701 million-$2.000 million 3 $1 - $200 thousand 9
$1.401 million-$1.700 million 4 $0 10
$1.101million - $1.400 million 5
16
Table G.1 NYC Metropolitan Area
Miles Score
0-35 10
36-65 9
66-95 8
96-125 7
126-155 6
156-190 5
191-220 4
221-250 3
251-280 2
281-315 1
>315 0
FACTOR 4. OTHER FACTORS
1. Standard Factors (except small firm credit):
Score points as noted below for each time the service was performed (up to a maximum score of
10). For any inventory project that represents individual experience, this value is multiplied by
Table F.3. The Department has the flexibility to change the points given each time services were
done.
The team score for each standard factor will be a weighted average of the scores of firms that will
be doing that work task based on the overall % work each firm is proposed to do.
1 O1. TRAFFIC FORECASTING
1 O2. SOIL AND GEOLOGIC STUDIES, FOUNDATIONS
2.5 O3. MOVEABLE BRIDGE DESIGN
2.5 O4. BRIDGE CARRYING A RAILROAD
2.5 O5. HISTORIC BRIDGE DESIGN
1 O6. FINAL DESIGN URBAN ARTERIAL INTERCONNECTED SIGNAL DESIGN
1 O7. FINAL DESIGN RETAINING WALLS - OVER 10 METERS IN HEIGHT
1 O8. FINAL DESIGN OF COMPLEX MPT: OVER 75 MPT SHEETS
1 O9. FINAL DESIGN HIGHWAY LIGHTING
1 O10. EXTENSIVE CAPACITY ANALYSIS URBAN FWY MNLINE & INTCHGS
1 O11. COMMUNITY PARTICIPATION
1 O12. INVOLVEMENT WITH NYC AGENCIES
1 O13. ITS/ELECTRONIC HIGHWAY OPERATIONS
1 O14. INTER-MODAL SCOPING/DESIGN ANALYSIS
2. Small firm credit:
If the prime or joint venture answers yes to the Special Factor Small Firm credit question in the e-
EOI and the Department input indicates that the small firm credit applies, the team receives a
credit. The credit = weight of the special factor for small firm credit x base number of the scoring
system (base number default value=10). If the answer is no then the team receives no credit.
17
3. Special factors are scored differently than standard factors:
For special factors, firms (or joint ventures) will be scored based on a specifically requested
numeric response they provide in their e-EOI. Responses will be an integer from 0 to 10. The
project advertisement will provide a description of each special factor and instructions on how to
respond and how it will be scored. Firms (or joint ventures) that do not respond will get a 0 score
for that factor.
NOTE: RATING FACTORS 5-7 ARE BONUSES OR PENALTIES APPLIED TO THE
OVERALL TEAM SCORE CALCULATED BY SCORING AND WEIGHTING
FACTORS 1A THRU 4
FACTOR 5. D/M/WBE PARTICIPATION
Score = ((A x B/10) / 10) x C
A = Total proposed D/M/WBE participation % from the team’s e-EOI (entered as a whole
number). The % participation (currently capped at 20% for DBE, or 30% for M/WBE =
maximum bonus).
B = Consultant’s D/M/WBE performance score from a Contract Management database.
For joint ventures, use the average of the JV’s D/M/WBE performance scores.
C = Standard weight for factor (current standard is 0.10). This value should be in a table
that Contract Management can change.
FACTOR 6. D/M/WBE SUBCONSULTANTS
6A. Performance and Experience
Average for each D/M/WBE subconsultant their scores for 1A, 1C, 2A and 4 as applicable.
For D/M/WBE subconsultants that are proposed to do work under at least one weighted factor:
For each D/M/WBE subconsultant, develop weighted performance and experience score by
averaging the subconsultant’s scores for 1A, 1C, 2A and 4 (consider weighted factors only). If
the firm has no performance evaluations in the system, for purposes of scoring this factor their 2A
score is given a value that is the mode of all subconsultants contained in all Expressions of
Interest for the project. If the subconsultant has no score for a factor, then that factor is not part of
the average.
18
For D/M/WBE subconsultants that are proposed to only do work under factors with no weight:
Factor 1A, 1C and 4 components use the same methodology as for weighted factors, calculate a
raw score for each required non-weighted factor the firm is identified to do.
For a factor 2A score, first calculate a score for each non-weighted factor the firm is proposed to
do. Then average these scores. If the firm has no performance evaluations in the system, for
purposes of scoring this factor their 2A score is given a value equal to the mode of all
subconsultants contained in all Expressions of Interest for the project.
For each D/M/WBE subconsultant, develop weighted performance and experience score by
averaging the subconsultant’s scores for 1A, 1C, 2A and 4. If the subconsultant has no score for a
factor, then that factor is not part of the average.
Note: To calculate the score for 1A, use the firm’s raw score for each task. Then calculate a
weighted average using the weight for each task the firm is proposed to perform. The same
approach is used to calculate the firm’s score for factor 1C and 4.
Score for each D/M/WBE subconsultant = A x (B/20) x C
A= Weighted performance and experience score for that subconsultant.
B= Total proposed D/M/WBE participation of that team’s e-EOI (entered as a whole
number).
C= Weight for factor (current standard is 0.055)
Sum up scores for all D/M/WBE subs and add total to overall team score for factors 1A thru 4.
6B. Workload With Division
Use current remaining design workload with NYSDOT submitted by each D/M/WBE sub. Score
based on the same table as in rating factors 2B1 and 2B2.
Score for each D/M/WBE subconsultant = A x (B/20) x C
A = Workload score for that subconsultant.
B= Total proposed D/M/WBE participation of that subconsultant (entered as a whole
number).
C = Factor 6B weight (currently 0.045)
Sum up scores for all D/M/WBE subs and add total to overall team score for factors 1A thru 4.
19
FACTOR 7. TIME SINCE LAST DESIGNATION
Use the following information submitted for the prime (or each joint venture member) and each
subconsultant: time since last designation, prorated agreement value of last NYSDOT
designation, project type and types of functions performed or proposed to be performed under
the last designation, types of functions proposed to be performed under the subject designation,
and the current remaining workload with NYSDOT.
Score each team member based on the following formula:
Score for each member of the team = (-) (A+B)/2 x C x (D/E) x (F) x (G) x (H)
A = Table A: Value based on months since last designation. (If A = 0, factor 7 = 0)
B = Factor based on value of firm’s share of last designation.
C = Table C: Value based on a comparison of the type of project functions directly
performed or proposed to be directly performed under this designation versus the last
NYSDOT designation.
D = Table D: Value based on remaining workload with NYSDOT. Note: If a firm has no
active certified salary roster, the scoring program will assume they have 0 total
transportation-related staff in NYS or NYC metro area.
E = Capacity Factor, calculated as shown below
F = Total proposed % of work of the firm. (“% work firm will perform” for each Joint
venture partner: It is assumed that each partner of a JV has an equal share of the total
% work that the JV will perform). The % is expressed as a decimal.
G = Factor 7 weight (currently 0.04)
H = Factor applied to a prime or JV partner’s score based on the value of their last
designation.
A. Months since last designation = Date e-EOI was due - date of last designation.
Months since last NYSDOT
designation
<1 1 2 3 4 5 6
Time Factor 50 42 34 25 17 9 0
B. The value of B is determined by the following formula rounded up to an integer and capped at 20:
$ Value (in thousands) of firm’s share of last designation* .004
20
C. Types of Functions information provided as part of the consultant’s EOI:
Comparison of the type of project
functions directly performed or
proposed to be directly performed
under this designation versus the
last NYSDOT designation
All of this
designation’s
functions were also
included under the
last designation
Overlap- At least one
but not all of this
designation’s
functions were also
included under the last
designation
No Overlap -
None of this
designation’s
functions were
included under
the last
designation
Type of Functions factor 1.0 0.7 0.3
1. Primary functions directly performed or proposed to be directly performed by the firm
under the last designation (consultant indicates as many that apply). Each function
indicated must be equal to or greater than 20% of the firm's total direct technical labor
cost.
Highway Design
Structural Design
Bridge Inspection
Construction Inspection
Environmental
Survey (field)
Survey (office)
Planning
Scientist
Architectural
Other
2. Primary functions proposed to be directly performed by the firm under this designation
(consultant indicates as many that apply). Each function indicated must be anticipated to
be equal to or greater than 20% of the firms anticipated total direct technical labor cost.
Highway Design
Structural Design
Bridge Inspection
Construction Inspection
Environmental
Survey (field)
Survey (office)
Planning
Scientist
Architectural
Other
21
• Same functions if all functions under (2-this designation) are included under (1-the
last NYSDOT designation)) = 1.0
• Overlap if at least one function under (2-this designation) is the same as a function
under (1-the last NYSDOT designation), and at least one function under (2-this
designation) is not included under (1-the last NYSDOT designation) = 0.7
• No overlap if all functions under (2-this designation) are not included under (1-the
last NYSDOT designation) = 0.3
D. Value based on remaining workload with NYSDOT:
Remaining workload
with NYSDOT
<$250 thousand $250 thousand
to $1 million
$1-2
million
$2-4
million
>$4
million
Workload Factor 1.0 2.0 3.0 4.0 5.0
E. Value based on transportation-related staff in NYS or NYC metro area:
Total Transportation-related staff
in NYS or NYC metro area
0 1-15 16-50 51-100 101-200 >200
Capacity Factor 5.0 5.0 5.0 5.0 5.0 5.0
H. Factor score based on the value of their last designation:
Role Value of last designation in thousands H
Prime or JV Partner 1-300 1.0
301-999999 1.0
Total the scores of the team members. This score for factor 7 is either 0 or a minus number.
OPTION TO MODIFY THE SCORING BASE OF THE PROGRAM FOR A SPECIFIC
PROJECT
If the base of the proposed scoring system for a specific target project is different than the default
base of 10, the computer program will make the appropriate adjustments by multiplying scores by
the ratio of target base/default base.
MORE ABOUT THE SCORING TABLES
1. Scoring Tables A, A.1 and B (Used for major categories of work scoring, except survey)
• Why are there tables for Construction $ Value and Complexity?
The program uses construction value and complexity as the criteria to compare how the project
service required on our project compares to any services in a consultant’s inventory.
22
The point earned for each are averaged yielding the points a firm earns for the services scored.
• How can I interpret the tables?
Table A determines the value that highway services earn depending on the construction value of
the projects. For example, say our project assigns a weight to preliminary highway design and our
project is valued at $400K. If the consultant provided preliminary highway design on an $8M
project in the consultant’s inventory, then the consultant earns 1.2 points for the service
performed on that project.
Table B decides the value that services earn depending on project complexity. For example, say
our project’s highway portion was moderately complex according to the Consultant Request
information. If the consultant provided services on a project that had complex highway design,
then the consultant earns 1.4 points for the service performed on that project.
2. Scoring Tables C and C.1 (Project Owner)
• Why are there tables for Project Owner?
The project owner is considered a measure of the technical/managerial demands of a project.
Therefore, for factor 1A, prior firm experience, the points given by the tables for construction
value and complexity are multiplied by a number from Table C.
For factor 1C, prior NYSDOT experience, the points given by the tables for construction value
and complexity are multiplied by a number from Table C1 instead of Table C. This table assumes
that only NYSDOT experience should be counted toward the score for factor 1C.
3. Table C.2 (apply to the total team score for 1A, 1C and 2A)
The level of subconsultant participation has no impact on scoring unless specified in the project’s
detailed advertisement. See table C.2.
4. Table D
This allows a firm to score points if they did a similar or the same task under major categories of
work (except survey).
For example, assume that the Department’s project gives a weight for preliminary highway
design. If the firm had performed only final highway design on a project in their inventory,
according to the table the firm would receive a partial credit of 0.5 points.
23
5. Table E (Survey & Mapping component of 1A, 1C and firm experience component of 6A)
• How is Survey and Mapping scored differently than other major categories of
work?
Points are awarded for the type of project that the services were provided on as defined in the
table. The construction value and complexity scoring tables are not used to determine the value of
survey and mapping services.
6. Table F (Project specific environmental component of 1A, 1C and firm experience in 6A).
• How are project specific environmental tasks scored?
Points are awarded for the type of project that the services were provided on as defined in the
table. The construction value and complexity scoring tables are not used to determine the value of
environmental services.
7. Table F.1 (Project Specific Environmental Work)
• What is the reason for Table F.1?
If a firm has prepared a similar but not the same major environmental document that the
Department’ project requires, the firm should receive some credit.
For example, assume that the Department project gives a weight to EIS experience. If the firm
had prepared an EA for a project in their inventory, according to the table the firm would receive
a partial credit of 0.5 points.
8. Table F.2 (Defines when a project is too old to score).
• What is the reason for Table F.2?
An age limit was established so that a project represents experience with current practices,
procedures, technology and requirements.
9. Table F.3 ((Individual experience factor)
• What is the reason for Table F.3?
Projects which represent the experience of an individual employee while with another firm have
their scores for both construction cost and complexity reduced. Although experience of a
management-level individual is a good indication that the firm will be able to manage jobs of a
similar size and complexity, it is not as valuable as firm experience, where it is likely that a larger
number of staff at different levels of responsibility had experience with the project.
24
10. Table G and G.1 (Location Factor)
• What is the purpose of table Tables G and G.1?
This table is used to determine the score for location by giving a score based on distance to the
regional office and the project. The firm’s score for distance to the project and distance to the
regional office is averaged.
Table G is used for projects outside of the New York City metropolitan area.
Table G.1 is used for projects in the New York City metropolitan area and possibly other
situations where traveling is more time consuming.