performance measurement strategy for the canada small ...€¦ · support for small businesses. it...

Performance Measurement Strategy

for the

Canada Small Business Financing Program (CSBFP)

Industry Canada

December 2012

1

2

Table of Contents

Introduction......................................................................................................................... 3

Section 1. CSBFP Profile................................................................................................... 4

1.1 .......................................................................................... 4 Need for the Program

1.2 ........................................................................................................... 5 Objectives

1.3 ................................................................ 6 Alignment with government priorities

1.4 ........................................................................... 7 Stakeholders and Beneficiaries

1.5 ......................................................................................................... 8 Governance

1.6 ............................................................................................................ 8 Resources

Section 2. Logic Model.................................................................................................... 10

2.1 ...................................................................................... 12 Logic Model Narrative

Section 3. Performance Measurement Strategy (PMS) Framework................................ 15

3.1 .......................................................................... 19 Accountabilities and reporting

Section 4. Evaluation Strategy......................................................................................... 21

Appendix A. Summary Table of CSBFP Parameters ...................................................... 27

Introduction

Purpose of the Performance Measurement Strategy (PMS)

In 2008, the Policy on Transfer Payments was revised to include, amongst other requirements,

that a Performance Measurement Strategy be established at the time of program design and be

maintained and updated through a program’s lifecycle. This Performance Measurement Strategy

is an update to the 2006 results-based management and accountability framework (RMAF) for

the Canada Small Business Financing Act (CSBF Act), and has been revised according to current

Treasury Board Secretariat requirements.

Although the Canada Small Business Financing Program (CSBFP) is a statutory program and

not a transfer payment program, the CSBF Act and the Treasury Board Evaluation Policy require

the program to be reviewed every five years. The review draws largely from an in-depth third

party evaluation of the program which will be governed by this strategy. This strategy is

designed to:

$ monitor and evaluate the performance of the CSBFP;

$ assess its success in addressing small business access to financing challenges during the 5

years; and

$ inform the decision, at the end of the 5-year period, whether any program changes are

required.

This PMS will:

$ describe the core program profile, including its origin, rationale, operation, the roles and

responsibilities of the main partners involved in delivering the program, and a description of

the direct beneficiaries, agents and key stakeholders;

$ identify and determine appropriate activity and performance measures, data sources and data

collection methodologies used to track progress, measure outcomes and support subsequent

evaluation activities;

$ describe the evaluation strategy including a draft evaluation framework that outlines the

evaluation issues and questions (and their relative importance) related to achievement of

strategic outcomes;

$ recommend on-going monitoring and evaluation activities and the approach for the five-year

evaluation; and

$ describe the reporting strategy on outcomes during and at the end of the five-year period;

3

Section 1. CSBFP Profile

1.1 Need for the Program

In existence since April 1, 1999, the CSBFP is the continuation of the Small Business Loans Program which began in 1961. It is a loan loss-sharing program, between the government and private sector lenders, to stimulate growth and create jobs among small businesses. It operates by facilitating access to affordable asset-based financing to small businesses for their establishment, expansion, modernization and improvement. The CSBFP is a cornerstone of Industry Canada’s support for small businesses. It plays an important role in providing small businesses with access to financing to grow and start up. Small businesses play a fundamental role in the Canadian economy. In 2010, 98% of enterprises were small businesses, and 48 % of all employees in the private sector worked for small businesses1. To realize their full potential, small businesses need a supportive environment in which to grow and prosper. The Canadian government, like governments in many other countries, including all of Canada’s major trading partners, has long recognized the importance of small business to economic well-being and that access to financing can be a critical issue for small businesses, particularly during their early years. Research has indicated that certain small businesses have difficulty obtaining financing and although the vast majority of small businesses meet their financing needs through market transactions without any government involvement, there are many small businesses in Canada whose financing needs are not being satisfied by private sector providers. This is primarily because loans to start ups and certain types of small businesses are riskier. Lenders respond to this higher degree of risk in a variety of ways that have the effect of limiting the availability of financing for small businesses particularly during periods of economic difficulties. While 87% of all loan applications by start-ups were approved in 20072 , in 2009 approval of rates for start ups dropped to 44%3*. At the same time, there was no significant impact on approvals of loan applications from established SMEs, which remained relatively steady at 81%4 in 2007 and 85%5 in 2009**. This illustrates how vulnerable start ups are as a downturn in the economy can have a drastic effect on access to financing.

1 Statistics Canada, Key Small Business Statistics, July 2011 2 Statistics Canada, Survey on Financing of Small and Medium Enterprises, 2007, March 2009. 3 Industry Canada, Credit Conditions Survey, 2009 4 Statistics Canada, Survey on Financing of Small and Medium Enterprises, 2007, March 2009 5 Industry Canada, Credit Conditions Survey, 2009 * For the 2007 statistics, start-ups are defined as being one year old. For the 2009 statistics, start-ups are defined as being three years old or younger. ** For the 2007 statistics, established SMEs are defined as having 0 to 499 employees and being from five to seven years old. For the 2009 statistics, established SMEs are defined as having 1 to 99 employees and as being four to ten years old.

4

In addition, small businesses with certain characteristics such as little or no credit history, few tangible assets to pledge as collateral and small businesses engaged in developing new products and services also present a higher risk to lenders and thus can have difficulty obtaining financing. At the time of enactment, the CSBFP was considered to be highly relevant to the needs of small businesses and, overall, an efficient, effective and simple mechanism to facilitate debt financing for emerging and established small businesses. It was subsequently confirmed through a 2009 evaluation of the program that there are no attractive alternatives to a federally managed program along the lines of the CSBFP and that there is minimal overlap between it and other federal initiatives to support access to financing by small businesses. In addition to the above, it is estimated that in 2007 between 80% and 85% of small businesses that obtained financing through the CSBFP would not have had access to financing or would only have had access to financing with less favourable terms, if this program had not been available.6 This provides strong evidence that there remains a public policy rationale for the government to help increase the availability of small business financing. Therefore, the government, through the Canada Small Business Financing Program (and its predecessor the Small Business Loans Program) has sought to help small businesses in accessing the financing which is crucial to their start-up, growth and competitiveness.

1.2 Objectives The objective of the CSBFP is to facilitate access to asset-based debt financing for the establishment, expansion, modernization and improvement of small businesses. It does this by sharing the financial risk of lending to small businesses among the borrowers, lenders and the government. The CSBFP operates under two competing objectives: Incrementality: Loans made under the program would not have been made in the absence of

it or would have only been made under less favourable terms (e.g. higher interest rates, greater collateral requirements ) for the small business; and

Cost recovery: The registration and administration fees collected on CSBFP loans will help

offset the cost of claim payments. Cost recovery does not include administrative costs which are absorbed by Industry Canada.

As cost-recovery and incrementality are inversely related (this was pointed out by the Auditor General in 1997 and again in 2002) the focus has shifted towards achieving a reasonable balance between cost recovery and incrementality rather than focusing on each element on its own. ________________________ 6 Canada Works, Canada Small Business Financing Program: Updated Analysis of Incrementality, June 2009

5

As the CSBFP aims to provide small businesses with access to financing, to which they would have not otherwise have access, it also aims to benefit the Canadian economy. This is realized through the financing of small businesses as it enables them to be competitive in the marketplace and gives them opportunities for growth through investments, expansion of their businesses and employment generation. The Economic Impact of the Canada Small Business Financing Program (June 2010) illustrates that participation in the CSBFP increases growth in employment, salaries and revenues.

1.3 Alignment with government priorities Industry Canada’s delivery of the CSBFP is consistent with its mandate and Government directions. The Program’s mandate is derived from the Department of Industry Act and the CSBF Act. The CSBFP is also a separate element of Industry Canada’s Program Activity Architecture (see Figure 1) and contributes to the strategic outcome “Canadian Businesses and Communities are competitive”. Finally, the Program is consistent with the small business priorities of the Government. Program parameters were enhanced in the 2009 Canada Economic Action Plan and improving access to business finance for small businesses was also highlighted as a priority in Budget 2011.

Figure 1-PAA elements linked to CSBFP

INDUSTRY CANADA OUTCOME

Canadian Businesses and Communities are Competitive

PROGRAM ACTIVITY

Small Business Research, Advocacy and Services

SUB-ACTIVITIES

Canada Small Business Financing

6

1.4 Stakeholders and Beneficiaries

Stakeholders

Stakeholders are key in the continuing development and implementation of the program.

Stakeholders with a direct interest in the program include:

The lenders eligible under the program;

Canadian Bankers Association (CBA), representing the major banks and financial

institutions;

Credit Union Central of Canada (CUCC), representing most credit unions in Canada;

Fédération des Caisses Populaires Desjardins, representing the caisses populaires in

Quebec and Ontario;

SME associations, such as the Canadian Federation of Independent Business (CFIB),

Canadian Chamber of Commerce, Canadian Restaurant and Foodservices Association

(CRFA), Retail Council of Canada (RCC) and the Canadian Franchise Association

(CFA), which represent Canadian SMEs;

SME business support organizations such as small business enterprise centres, public

libraries, bookkeepers, financial advisors, colleges and universities, economic

development offices.

Parliamentarians and Canadians.

Beneficiaries

The beneficiaries of the CSBFP are for-profit small businesses operating in Canada with up to $5

million in annual gross revenues. Not-for profit, charitable or religious organizations and farming

businesses are excluded from the program.

The profile of CSBFP borrowers for the period 2004 to 2010 was:

new C small businesses just starting up (less than one year old) received approximately

half of the number and value of loans made;

small C a significant majority of all CSBFP borrowers receiving loans had annual

revenues of less than $300,000;

seeking small amounts C average overall loan size was $103,000; and

key sectors (by value of loans) C accommodation and food services; retail trade;

manufacturing; and transportation and warehousing.

7

8

1.5 Governance

The CSBF loan loss-sharing program is a statutory program and, as such, has very few

equivalents in government. Whereas most government programs see credit decisions being made

by program managers who manage the risk and the size of the program, this is not the case with

the CSBFP. It is delivered by third party lenders including chartered banks, credit unions and

caisses populaires. Lenders provide service to Canadians in all provinces and territories, and are

responsible for all credit decisions, approving the loans and, in the event of default, realizing on

the security and guarantees. Each lender establishes its own lending criteria subject to the

requirements of the CSBFP. Once the loan is approved, the funds that are advanced to the

borrower are the funds of the lender and not the government.

Industry Canada administers the CSBFP by registering loans, collecting fees and paying lenders

eligible portions of losses on defaulted loans. However, the CSBFP does not approve the

borrower loan applications and is not involved in the administration of the loans. By registering

loans with Industry Canada, lenders are entitled to submit claims for eligible losses to Industry

Canada in the event of default.

1.6 Resources

As the CSBF is a statutory program, the cost of claims for loss are sourced directly from the

Consolidated Revenue Fund (CRF) and a portion is offset by collecting revenues from a 2

percent registration fee and a 1.25 percent administration fee paid on the loans. These fees are

submitted to Industry Canada by lenders. These fees do not aim to offset the operating costs of

the Program, which are about $3.1 million annually for a staff of 35 and include costs of

research, IT systems and outreach activities.

On average from 2004-2010 annual claim payments were $90 million and annual fee revenues

received were $54 million. These are the expenses and revenues received each year regardless of

when the loans were made.

There are also a number of parameters under the CSBF Act which limit the Government of

Canada’s exposure to liability. For example, the CSBF Act establishes a $1.5 billion liability

ceiling for each five year lending period. This liability is calculated as the sum of the maximum

potential liability to each lender under the program, for each lending period. The CSBFP has

never come close to reaching its maximum liability limit.

9

Claim payments on defaulted loans are also capped for each lender. For each five-year period

commencing April 1, 1999, the Government of Canada’s obligation to an individual lender is to

pay eligible claims (i.e. 85% of the eligible losses) on defaulted loans up to a maximum of the

aggregate of:

90 percent of the first $250 000 in loans registered; plus

50 percent of the next $250 000; plus

12 percent of all loans in excess of $500 000 for loans made on or after April 1, 2009 (10

percent for loans made before April 1, 2009).

10

Section 2. Logic Model

The logic model provided in Figure 2 serves as the program’s roadmap. It outlines the activities of the program, its corresponding outputs, the immediate outcomes of these outputs, the intermediate or medium term outcomes and the final outcomes of the program.

2.1 Definitions: These definitions are inspired from the definitions listed in Supporting Effective Evaluations: A Guide to Developing Performance Measurement Strategies-Treasury Board Secretariat.

Activities: Activities are what program staff carries out in order to achieve objectives. They are the actions that a departmental organization undertakes to produce one or more outputs under the program.

Outputs: Outputs are the direct products and services generated from the activities of the organization, policy, program or initiative. They are the deliverables, including products and services, resulting from activities (and controlled by activities).

Outcomes:

Outcomes are the changes and differences that result from the program outputs. They are the reason for the existence of the program.

Immediate outcomes: Immediate outcomes are directly attributable to the outputs delivered. In terms of time frame,

these are short term outcomes. Immediate outcomes include the short term effects on those who are directly affected by the program or policy, such as changes in awareness.

Intermediate outcomes: Intermediate outcomes are logically expected to occur once one or more immediate outcomes

have been achieved. They include changes to those beneficiaries (borrowers) directly reached by the program or policy, as well as those influenced by the behavior of the program beneficiaries.

Final outcomes: These are the highest-level outcomes that can be reasonably and causally attributed to a policy,

program, or initiative. Final outcomes are directly linked to the mandate or objectives set for the program.

Communicate with stakeholders

Improvements to Act,Regulations and Guidelines, Consultation Reports, Literature Review/Research, Comprehensive Review Report.

Review/improve and administer Program delivery

OUTPUTS

Presentations, information bulletins, toolkits, how to guide, website improvements, Annual Reports.

Figure 2-CSBF Program Logic Model

IMMEDIATE OUTCOMES

Loans registered, claims processed, fees collected, forms and procedures, reports on administration and on site examination reports.

Regulatory and legislative framework makes the program conducive to use by lenders and borrowers

Lenders share risk of lending to small businesses with Industry Canada

Review/improve Program design ACTIVITIES

Small businesses obtain financing from lenders, with government support, that would have otherwise have been unavailable, or available only under less favourable terms.

Canadian businesses and communities are competitive

INTERMEDIATE OUTCOMES

LONG TERM OUTCOMES

Lenders, small businesses and borrowers have awareness and knowledge of the program

Policy and Communication Administration

11

12

2.2 Logic Model Narrative The CSBFP logic model is separated into two sections, the policy and communication section

and the administrative section. While the administrative side provides the fundamental basis for the daily operation of the program, such as the registering of loans and the processing of claims, the policy and communication side contains the elements that help define the program such as the regulatory and legislative framework.

The CSBFP has two key outcomes. To enable small businesses to obtain financing from lenders

to which they would not otherwise have access, and to contribute to the competitiveness of Canadian businesses and communities. These outcomes as well as immediate outcomes, outputs and activities, and the linkages between these, are explained below.

Activities/Outputs 1) Review/ improve Program Design is an activity that seeks to make the CSBFP more

relevant to the needs of lenders and borrowers. This is achieved by first reviewing the program and then using the feedback received to improve it. For example, evaluations and consultations are important sources of information on the performance of the program, its challenges, its successes and how it addresses the needs of lenders and borrowers. Once the program is reviewed and modifications conducive to its use are identified then changes to the regulatory and legislative framework can be processed. These modifications are essential to maintaining the appeal of the CSBFP with lenders and borrowers alike as needs evolve over time. These activities result in the outputs of Improvements to Act, Regulations and Guidelines, Consultation Reports, Literature Review/Research and Comprehensive Review Report which link directly to the outcome of “Regulatory and legislative framework makes the program conducive to use by lenders and borrowers”.

2) Communicate with stakeholders is an activity that seeks to reach both lenders and

borrowers in order to inform them of the CSBFP, build an initial awareness and refine existing knowledge as well as report to Canadians on program performance. These communications include working with lenders and building relationships with SME business support organizations in an effort to increase awareness and knowledge of the CSBFP. These communication efforts lead to the production of a variety of tools (outputs) such as brochures, bulletins, how to guides, presentations and toolkits, which are essential tools to reach lenders and borrowers and provide information about the CSBFP. Via these outputs, increased knowledge and awareness of the program should follow as an immediate outcome.

3) Review/improve and administer program delivery entails the review, oversight and

improvement of Industry Canada’s roles in program delivery (e.g. loan registration and claims processing) to ensure consistency and uniformity of processes used in the daily delivery of activities of the program The outputs of this activity such as loan registered, claims processed, fees collected, reports on administration and on site examination reports are needed so that lenders are able to use the program to make CSBFP loans, and are able to share in eligible losses with Industry Canada. Once these outputs have been

accomplished, the program is ready to be delivered. However, in order to use the program to make loans, there also needs to be incentives for lenders. One of these incentives is the immediate output of lenders sharing risks with Industry Canada.

Immediate outcomes: 4) Regulatory and legislative framework makes the program conducive to use by lenders

and borrowers. Without a legislative and regulatory framework the CSBFP would not exist. However, in order for the CSBFP to be used by lenders and borrowers it needs to be appealing to them. The CSBFP cannot be looked at on its own but as an entity that plays a role in the financial marketplace. The forces of supply and demand in the debt financing marketplace determine the appeal of the CSBFP for lenders and borrowers and help identify areas for change. As the CSBFP is a government program, its costs must also be considered and reasonably contained. In order to establish a sound program it is important to maintain a coherent regulatory and legislative framework that positions the program as a viable tool in the small business financing market which is appealing to use for lenders and borrowers while incurring costs that are reasonable for the government. Having a program that is designed such that lenders and borrowers find it conducive to use provides access to the program because lenders will offer the program to borrowers that they would have otherwise rejected mostly because of their high risk profile. And in turn, borrowers will be interested in borrowing as CSBFP loan conditions are appealing to them. It follows that a CSBFP that is designed in such a manner that it is conducive to use by lenders and borrowers results in the final outcomes of providing access to financing to small businesses that would not otherwise be able to access financing, and of helping to create competitive businesses.

5) Lenders, small businesses and borrowers have awareness and knowledge of the program

Awareness and knowledge of the program among lenders and small businesses appears to be one of the main factors guiding its use. If small businesses and lenders are aware and knowledgeable about the program, the more small businesses will inquire about the CSBFP and the more lenders may offer the program. Awareness and knowledge of the program help lead to the intermediate outcome of providing small businesses access to financing that they would not otherwise be able to access, and to the long term outcome “Canadian businesses and communities are competitive”.

6) Lenders share risk of lending to small businesses with Industry Canada

The appeal of the CSBFP program to lenders is the fact that losses incurred on CSBFP loans are shared with Industry Canada. By sharing the risk of lending with Industry Canada, lenders are able to make loans to small businesses that they otherwise would not make. Without the CSBFP these small business loans would fall outside of the lender’s risk threshold. Thus, sharing the risk of lending with Industry Canada supports the intermediate outcome of providing small businesses access to financing that they would not otherwise be able to access. It also supports the final outcome of supporting competitive businesses as they use this financing to start-up, modernize and improve their operations.

Intermediate Outcomes:

13

14

.

7) Small businesses obtain financing from lenders, with government support, that would have otherwise have been unavailable, or available only under less favourable terms. This is the main goal of the CSBFP, to help those small businesses that would otherwise not obtain financing or would receive it under less favourable terms, mainly because of the higher level of risk that they present. Businesses that fall into this category are mostly start ups and those in certain sectors such as food services. These small businesses often have difficulties obtaining bank financing as they present a greater credit risk, and in the case of start ups, they often have little collateral and no business history. The CSBFP acts as a financing solution for these businesses that are often turned down by traditional bank financing. All three immediate outcomes mentioned below need to be in place for the program to be used and achieve its objective of providing financing to those small businesses that without the CSBFP would not have access to financing. Firstly the program’s regulatory and legislative framework needs to make the program conducive to use by lenders and borrowers because if the program is not appealing it simply will not be used. Secondly, lenders, small businesses and borrowers need to be aware and have knowledge of the program so they will be more likely to use it. Lastly, lenders must benefit by sharing the risk of lending to small businesses with Industry Canada because mostly high risk small businesses apply to the program and without a shared risk lenders would simply not finance these small businesses.

Long Term Outcome:

8) Canadian businesses and communities are competitive is a priority of Industry Canada. The CSBFP contributes to this long term outcome by encouraging lenders to make more loans to small businesses, which gives an opportunity for new businesses to emerge and existing businesses to grow and be competitive. By improving access to financing for these businesses, and specifically through loans that would otherwise have not been available, the growth and competitiveness of these small businesses is stimulated.

The logic model in Figure 2 outlines only those activities under Industry Canada=s control and which can be used to influence small businesses’ and lenders= decisions to use and participate in the program.

As indicated earlier, Industry Canada is responsible for developing the CSBFP’s design, as well

as implementation and administration of the Program to ensure that it will achieve its objectives

and outcomes. However external factors such as economic fluctuations, market forces and

political influences which are outside Industry Canada=s control or influence may impact on the

extent to which the program will achieve its final outcomes. Significant market forces,

economic influences and industry changes may have an impact on both the demand and supply

of debt financing. As a result, and in addition to monitoring the performance of the program,

Industry Canada will monitor the evolution of the debt financing market to evaluate its impacts

on SME financing products and services and on the ongoing need for the CSBFP. The political

15

climate may have an impact as well on whether any recommended program changes are accepted

and thus on the outcomes of the CSBFP.

The next section, Section 3, deals with performance measurements of the outputs and outcomes

presented in the logic model.

Section 3. Performance Measurement Strategy (PMS) Framework Performance measurement is the regular collection of information for monitoring how a program

is doing at any point in time and on performance trends over time. It can provide reassurance that

outcomes are unfolding as expected, or can serve as an early warning that the planned results are

not occurring (which could lead to a decision for additional research to determine why). In turn,

this information allows making informed decisions and taking appropriate, timely action with

respect to the program.

The Program’s success is measured in terms of progress towards the goals or objectives that can

be attributed to the program.

The results are listed starting with the Program outputs and following the sequence of outcomes

in the logic model, up to the final or key result commitment. The performance indicators and the

data sources then articulate what evidence will be used to demonstrate the Program’s reach and

incremental impacts or effects at each level of outcome.

The mechanisms used to monitor the CSBFP are identified in the following PMS Framework

(Table 3). All outcomes identified in the program logic model are outlined with respect to

associated indicators and frequency of data collection.

The program activity elements that demonstrate the links between the outcomes of the

Government of Canada, Industry Canada and the CSBFP are outlined in Figure 2 to demonstrate

how the program fits in the program activity architecture.

Table 3-Performance Measurement Strategy

Frequency Program outputs and

outcomes Indicator Data source

Annual Every 2 or 3

yrs

Every 5 yrs

Ongoing

Responsibility for indicator

Outputs Feedback from stakeholders integrated into program, Act, Regulations and Guidelines changes

-Recommendations from the evaluation addressed

Improvements to Act, Regulations and Guidelines, Consultation Reports, Literature Review/Research, Comprehensive Review Report.

-Comprehensive Review Report prepared and tabled by legislative deadline

- Consultations - Management response to CSBFP Evaluation

- Policy Unit

-Annual Reports prepared and tabled

Presentation, information

bulletins, toolkits, How To Guide, Web site improvements, Annual Reports. -# and type of outreach activities

Outreach activity tracking database

- Economic and Policy Analysis Unit - Client and Partner Relations Unit

-# of claims processed

-# loans registered

Loans registered, claims processed, fees collected, forms and procedures, reports on administration and on site examinations report -# on site examinations

CSBFP database

- Operations Unit

Immediate Outcomes -Level of satisfaction of lenders with program features and with program parameters.

Regulatory and legislative framework makes program conducive to use by lenders and borrowers

-Level of satisfaction of small businesses and borrowers with program parameters

- Lender and borrower awareness and satisfaction surveys - Consultations - Feedback forms

- Policy Unit - Client and Partner Relations Unit

16

Frequency Program outputs and Responsibility for

Indicator Data source outcomes

Every Annual 2 or 3

yrs

Every 5 indicator Ongoing yrs

-Level of small business, borrower and lender awareness of program

-Level of small business, borrower and lender knowledge of CSBFP and its parameters

-# of visits/visitors to CSBFP website

Lenders, small businesses and borrowers have awareness and knowledge of the program

-# of referring domains to CSBFP website

- lender and borrower awareness and satisfaction surveys -consultations -webinar surveys -web analytics -# lender inquiries -feedback forms

-Client and Partner Relations Unit -Policy Unit

-Maximum liability of program over 5 year period

-Number and value of claims paid by type

Extent towards which lenders share risk of lending to small businesses with Industry Canada

-Trends in program use

-CSBFP database -Annual reports

- Economic and Policy Analysis Unit

Intermediate Outcomes -# of incremental loans registered with program

-Value of incremental loans registered with program

Small businesses obtain financing from lenders, with government support, that would have otherwise have been unavailable, or available only under less favourable terms.

-% incrementality

- CSBFP database -Incrementality Study

- Policy Unit

17

18

Frequency Program outputs and

outcomes Indicator Data source

Annual Every 2 or 3

yrs

Every 5 yrs

Ongoing

Responsibility for indicator

Final Outcomes

Canadian businesses and communities are competitive

Economic performance of CSBFP borrowers including level of impact on employment, survival rates, profits, sales and investments

-Economic Impact Study -Borrower survey

-Policy Unit

3.1 Accountabilities and reporting

ACCOUNTABILITIES:

Industry Canada Industry Canada, through the Small Business Branch’s (SBB) Small Business Financing

Directorate (SBFD), is accountable for the development, implementation and administration of

the CSBFP.

SBFD is accountable for: designing the program and legislation; ensuring awareness of the program; providing interpretation of program requirements to lenders and third parties; ensuring compliance to the program; approving and designating lenders; registering loans and collecting registration and administration fees; reviewing and paying lenders claims for losses on defaulted loans; reviewing and reporting on program performance to Parliament (annual and five year

reviews); collecting performance measurement data; and implementing the PMS.

The Audit and Evaluation Branch (AEB) is accountable for evaluating the program. Participating Lenders (financial institutions) Small businesses apply directly to approved or designated lenders (e.g. banks and credit unions) to access financing under the program, and not to the government. In this context, the lenders are accountable for:

making credit decisions; applying the same due diligence as for conventional loans; registering loans with Industry Canada and remitting the 2% registration fees and 1.25%

annual administration fees; realizing all securities and guarantees to minimize losses (in case of default); and submitting claims for loss to Industry Canada.

19

20

REPORTING

The CSBFP will rely on the following reporting strategy (summarized in Table 4): Table 4- Performance Reporting Strategy

Measurement Activity

Products

2009

2010

2010

2011

2011

2012

2012

2013

2013

2014

2014

2015

2015

2016

Ongoing Performance Measurement

CSBFA Annual Report Main Estimates DPR RPP

T

T

T

T

T

T

T

Mid-term Review

Mid-Term Progress Report

T

Evaluation

Evaluation Report

T

T

Comprehensive Review

Comprehensive Review Report

T

T

The Small Business Financing Directorate (SBFD) will report annually on progress achieved during the previous year in the Annual Report, Departmental Performance Report, Main Estimates and Report on Plans and Priorities. SBFD will also have the responsibility for developing a Mid-Term Progress Report, in 2011-12 which will be submitted to the Assistant Deputy Minister of the Small Business, Tourism and Marketplace Services Sector. This report will focus on the performance of the program to the midpoint of the review cycle and the status of the data collection and evaluation activities necessary to having the required information to feed into the final evaluation. The Evaluation Report will be started in 2013-14 and completed in the summer of 2014 by the Audit and Evaluation Branch, and it will be submitted to the Department=s Audit and Evaluation Committee. The report will be a public document and, as such, available on the Department=s web site and in hard copy to anyone upon request. A summary Comprehensive Review Report, largely based on the Evaluation Report, will be tabled in April 2015 in both Houses of Parliament as per the CSBF Act requirement. The results of ongoing performance measurement will be used to make any necessary adjustments to the initiative. The performance measurement strategy will be reviewed and

21

adjustments will be made, as required, to ensure that the performance information is appropriate and useful for ongoing management requirements.

Section 4. Evaluation Strategy The last evaluation of the CSBFP was completed in June 2009 and evaluated the program’s operations from 2004 to 2008 in accordance with relevant Treasury Board Secretariat policies. In line with this evaluation, AEB will conduct an evaluation in accordance with the Treasury Board Policy on Evaluation (2009), and in consultation with the policy unit of the Small Business Financing Directorate to be completed by summer 2014. This evaluation will focus on similar issues as the 2009 evaluation in order to provide continuity and have points of relevance for assessing program evolution. It will also incorporate new elements from the 2009 Treasury Board Policy on Evaluation. Thus the main issues that will be assessed are the relevance and performance (effectiveness, efficiency and economy) of the program. Attention will be given to the recommendations of the 2009 evaluation. The draft evaluation framework in Table 5 will guide this evaluation. CSBFP Research-Planned

Document Target Completion Year

SME FDI CSBFP Oversampling 2012-13

Economic Impact Study 2012-13

SME FDI CSBFP Borrower Profile 2013-14

Incrementality Study 2013-14

SME Awareness and Satisfaction Survey 2013-14

Lender Awareness and Satisfaction 2013-14

Cost-Benefit Analysis 2013-14

Evaluation Report (2009 to 2014)

Summer 2014

22

Table 5: Draft Evaluation Framework

Evaluation Issues

Evaluation Questions

Indicator

Data Source/ Collection Method

Responsibility for Collection

Timing/ Frequency of Measurement

Relevance

Continued need for the program

To what extent does the program continue to address a demonstrable need?

-Availability and credit conditions of commercial credit for small businesses -Risks associated with small business lending decrease with government intervention

-Document and Literature review including SME FDI Survey data -Interviews with lenders -Interviews with CSBFP officials - Credit conditions survey -OECD SME financing scoreboard -Bank of Canada senior loan officer survey -CFIB Banking survey

Policy Unit Evaluation Directorate

At the time of evaluation

23

Evaluation Issues


Indicator




Alignment with Government Priorities

To what extent is the program aligned with Federal government priorities?

- Alignment with priorities in the Budget

- Alignment with Speech from the Throne.

- Assessment of the linkages between program objectives and departmental strategic outcomes

-Program Activity Architecture -Report on Plans and Priorities -Speech from the Throne -Budget -Economic update

Policy Unit Evaluation directorate


Alignment with Federal Roles and Responsibilities

To what extent is the program aligned with Federal roles and responsibilities?

- Assessment of the role and responsibilities of the federal government in delivering the program

-Program review in 2009 Evaluation -OECD reports -Speech from the Throne -Budget -Economic update -Interviews with CSBFP officials -Document and Literature review

Evaluation Directorate Policy Unit


Performance (effectiveness, efficiency and economy)

24

Evaluation Issues


Indicator




Is the regulatory

framework of the program conducive to use by lenders and borrowers?

-Level of satisfaction of lenders with program features and with program parameters -Level of satisfaction of small businesses and borrowers with program parameters -Trends in the use of the program

- Lender and borrower awareness and satisfaction surveys -Interviews with lenders and borrowers -CSBFP data -Consultations

Policy Unit

Evaluation Directorate

Periodic and at the time of the program evaluation (every five years)

Achievement of Expected Outcomes

To what extent are small businesses (borrowers and non-borrowers) and lenders aware of the CSBFP?

-Level of awareness of CSBFP among small businesses and CSBFP borrowers -Level of awareness among lenders

-Awareness and satisfaction survey of small businesses/borrowers -Awareness and satisfaction survey of lenders -Lyris application -Web analytics -Outreach activity tracking -Requests for information

Policy Unit Client and Partner Relations Unit

At the time of the program evaluation (every five years)

25

Evaluation Issues


Indicator




To what extent do lenders share risk of lending to small businesses with Industry Canada?

-Trends in program use -Maximum liability of program over five years -# and value of claims paid by type

-CSBFP database -Lender interviews

Economic and Policy Analysis Unit


Annually and at the time of the program evaluation (every five years)

What proportion of CSBFP loans would not otherwise have been made, or would only have been made under less favourable terms?

-Level of incrementality -Levels of financial incrementality (fully, partially and not incremental) of CSBFP loans

-Incrementality Study -Lender interviews

Policy Unit


What has been the economic performance of firms in receipt of loans under the CSBFP?

-Impacts on some of the following: employment, business sales, profits, tax remittances, business survival rates

-Economic impact study -Survey with borrowers



Demonstration of efficiency and economy

What is the benefit of the program compared to its cost?

-Costs of the program as compared to the benefits derived

-Cost Benefit Analysis Study



26

Evaluation Issues


Indicator




What is the net cost of the program in relation to the incremental lending it facilitates?

-Cost recovery models

-Levels of financial incrementality (fully, partially and not incremental) of CSBFP loans -# and value of loans

-Cost Recovery Model forecasts -Incrementality Study -Interviews with SBFD officials

Policy Unit

Economic and Policy Analysis Unit



27

Appendix A. Summary Table of CSBFP Parameters

Parameter

Definition Small business eligibility

for-profit Canadian businesses carried on in Canada

annual gross revenues of $5 million or less

excluded are farms and charitable or religious institutions . Assets financed

$ equipment

$ real property or immovables

$ leasehold improvements

$ program registration fees Maximum financing amount

A borrower cannot have more than $500 000 (of which a maximum of $350 000 can be used to finance the purchase or improvement of equipment and leasehold improvements) in total loans outstanding under the CSBFP.

Percentage of asset financed

The maximum amount of financing available is 90 percent of the eligible cost of the assets.

Maximum interest rate

Floating rate: the maximum is the lender’s prime rate plus 3 percent (includes the

1.25 percent administration fee).

Fixed rate: the maximum is the lender’s single family residential mortgage rate plus 3 percent (includes the 1.25 percent administration fee).

Length of term

The maximum term for any loan is 10 years from the date the first principal payment is scheduled to be made.

Fees

A fee of 2 percent of the amount financed is paid at the time of registration. This fee

can be included in the CSBFP loan.

An annual administration fee of 1.25-percent is paid on outstanding loan amounts. This fee may be included as part of the interest rate charged on loans.

Loss-sharing ratio

Eligible losses on loans are shared as follows: 85 percent government and 15 percent lender.

Cap on claims

For each five-year period commencing April 1, 1999, the Government of Canada’s obligation to an individual lender is to pay eligible claims (i.e., 85 percent of the eligible losses) on defaulted loans up to a maximum of the aggregate of: 90 percent of the first $250 000 in loans registered; plus 50 percent of the next $250 000; plus 12 percent of all loans in excess of $500 000 for loans made on or after April 1,

2009 (10 percent for loans made before April 1, 2009).

performance measurement strategy for the canada small ...€¦ · support for small businesses. it...

Documents