event evaluation

Event Evaluation

Learning Objectives

• At the end of this session you should be able to:

• Articulate the nature and importance of evaluation to the events management function

• Describe what event evaluation should focus on

• Explain the purpose of event evaluation, including the needs and expectations of stakeholders

• Describe the cyclical, holistic nature of event evaluation using the three critical Ss of events

• Assess the different forms of evaluation models and methods

• Formulate and prepare an event evaluation report

Event Evaluation

Event Evaluation

Is concerned with assessment, which usually

involves measuring a set of key variables, as

well as monitoring those variables to

determine positive and negative outcomes. It is

a subjective determination that can utilize

objective quantitative measures

(Getz, 1997)

Event Evaluation Data

Numerical and

Descriptive

Numerical

Descriptive

Event Types & Evaluation Tools

• Content Analysis Image Maker

• Visitor Exit Survey Tourism Product

• Economic Impact Assessment Economic Development Catalyst

• Social Impact Scale Analysis Transformer of Socio-cultural Landscape

• Environmental Impact Analysis Sustainable Development

Model

Conducting Event Evaluation (1)

Step 1

• Determine the purpose of the event evaluation exercise, including whether the evaluation is necessary.

Step 2 • Identify what should be the focus of evaluation.

Step 3 • Select the most suitable approach for conducting the

evaluation exercise.

Step 4

• Select model(s) for evaluation as a framework for data collection based on suitability, practicality and relevance.

Conducting Event Evaluation (2)

Step 5

• Develop appropriate instruments and data collection strategies based on the methods selected.

Step 6 • Collect and analyse data and findings.

Step 7

• Prepare and disseminate event evaluation report.

Step 8

• Formulate and implement decisions for improvement based on findings of event evaluation report.

REASONS FOR ALL EVENT EVALUATION Allows for more informed decisions towards greater

efficiency and more positive results to be made

To measure success or failure

INTERNALLY-DRIVEN REASONS

To determine whether goals and objectives have been met

To engender accountability

To identify and address problems and challenges

To determine whether event management functions have achieved expected outcomes

To understand who attends the event to determine who else can be targeted

To determine worth of the event to its workers and volunteers

To determine whether and how the event can remain viable and become sustainable

EXTERNALLY-DRIVEN REASONS

To determine level of awareness of sponsors’ products/services

To satisfy accountability requirements

To determine level of media interest and coverage

To determine event’s level of impact on tourist arrivals

To determine event’s level of impact on business and other related industries

To determine whether event satisfies the expectations of community stakeholders

To determine worth of the event to its patrons

Evaluation Considerations

• Does the event organization/event host have, or have access to

the requisite resources to conduct an event evaluation exercise?

• Have event evaluations been conducted on the event in the

past? And if so, how often and what kind of evaluation?

• What use was made of the evaluation report?

• What is the nature of the environment in which the event

operates?

• Can it facilitate the execution of an event evaluation?

• Does the event have access to a wide range of information it

can use to conduct the evaluation exercise?

What to Evaluate? 1. Event Elements – number of patrons; size of group; demographics

of patrons; attendees' address; source of information on the event;

number of times attending; patron satisfaction; quality and impact

of event programme; quality of merchandise; food and beverage,

etc.

2. Event Organization Elements – nature and quantum of

resources; human resources; volunteers' perceptions working on

the event; income and expenditure; cash flow; level of investment;

sponsorship dollar, etc.

3. Event Context Elements – nature and amount of local suppliers

used; impacts on event; impacts of event; community perceptions;

level of media coverage; media value; interest and presence

Event Evaluation Perspectives

• Event evaluations are focused on measuring and

monitoring the implementation of an event (Allen et

al., 2008; Bowdin et al., 2006)

• Every aspect of the event must be evaluated (Tum et

al., 2006), including factors such as human resource

management and volunteerism, facilities and access and

hospitality (Wendroff, 2004)

• Event evaluation is mostly conducted after the event

(Tum et al., 2006)

Problems with Event Evaluation

• Missed opportunities to correct detected

problems

• Minimal focus on evaluation at the

planning phases, which is vital to

understanding many decisions taken at

the implementation stage

• Key elements of the evaluations may not

be deployed effectively, if at all

Event Evaluation Approaches (1)

• Quantitative – primarily concerned with

counting complex data sets such as attendance

levels, information sources and event activities.

• Challenges and limitations of using quantitative

approaches exclusively:

Low response rates to surveys

Inconsistencies arising out of varied interpretations

of terminology used in surveys leading to over-

calculation and possible misrepresentation of data


• Qualitative – utilizes open-ended to allow for in depth

responses and focuses on capturing opinion and

attitudes on matters such as reasons for attending the

event; benefits sought; level of satisfaction; stakeholder

attitudes, etc.

• Can complement quantitative approaches because

useful details can emerge that may not be derived or

represented numerically

• Challenges and limitations of using qualitative

approaches exclusively:

Cannot reach the numbers of quantitative approaches


• Financial approaches – are concerned with collecting,

assessing and monitoring numerical data on the fiscal

elements of the event such as cash flow, gate receipts,

patron expenditure, profit, loss and debt

• Economic approaches – examine numerical data

related to employment, tax, visitor expenditure and

triggered economic activity in other businesses and

related industries

• Challenge with both approaches can be address through

the use of non-financial and non-economic

approaches


• Non-financial and non-economic approaches – utilize

qualitative approaches to monitor and assess fiscal and

economic matters that cannot be easily quantified but

which are relevant such as:

Assessment of intangible costs and benefits

Assessment of net value

Assessment of stakeholder perceptions

Assessment of economic and market factors that can influence

the event


• Critical three Ss

Event Significance – assessing, measuring and monitoring the

nature, purpose and intent of the event and feasibility to realize the

expected outcome

Event Success – determining whether the event execution and

contributing elements have achieved the expected outcome

Event Sustainability – measuring, assessing and monitoring the

event's impact on the environment and community and; the

potential for the event to be continued in future

Models Used in Event Evaluation

• Impact Assessment – focuses on measuring economic, social,

environmental and cultural impact of events, particularly in instances where

significant justification for the event may be required

• Cost-benefit Analysis – identifies and measures the costs and benefits of

an event and takes into account 'externalities' or spill-over benefits for the

host community (Burgen and Mules, 2000)

• Triple-bottom Line Evaluation – a systematic framework for measuring

and reporting the event's performance against economic, social and

environmental parameters, to determine negative or positive impacts on the

host community (Fredline et al., 2005)

• Ethnographic Profile - Typically used in the evaluation of cultural

festivals/special events, ethnography provides an assessment of these

experience-related events through a process of analysis which

encompasses participant observation, interviewing and documentary

resources

Summary

• Evaluating events provides a measure for performance and

delivery for event organizers, stakeholders and event

consumers

• An event evaluation should focus on, among several other

factors, the three Critical Ss – Event Significance, Event

Success, and Event Sustainability

• Impact Assessment , Cost-benefit Analysis, Triple-bottom Line

Evaluation and Ethnographic Profile are the four models that

can be utilized to evaluate events

• Evaluation methods depend on the nature of the event and

the requirements of stakeholders

event evaluation

Documents