Determining Event Success

Event Evaluation

St. Lucia Jazz

• Was this event a success or failure?

• Who in the country would be more likely to say it was a success?

• Who would be most unhappy with this event?

Festival of Highs and Lows

Learning Objectives

At the end of this session you should be able to:• Articulate the nature and importance of evaluation to

the events management function• Describe what event evaluation should focus on• Explain the purpose of event evaluation, including the

needs and expectations of stakeholders• Describe the cyclical, holistic nature of event

evaluation using the three critical Ss of events• Assess the different forms of evaluation models and

methods• Formulate and prepare an event evaluation report

Event Evaluation

Is concerned with assessment, which usually involves

measuring a set of key variables, as well as

monitoring those variables to determine positive and negative

outcomes. It is a subjective determination that can utilize

objective quantitative measures

(Getz, 1997)

Event Evaluation Perspectives

• Event evaluations are focused on measuring and monitoring the implementation of an event (Allen et al., 2008; Bowdin et al., 2006)

• Every aspect of the event must be evaluated (Tum et al., 2006), including factors such as human resource management and volunteerism, facilities and access and hospitality (Wendroff, 2004)

• Event evaluation is mostly conducted after the event (Tum et al., 2006)

What about Encounters13?

What view of event evaluation should be taken for this event?

REASONS FOR ALL EVENT EVALUATION·Allows for more informed decisions towards greater

efficiency and more positive results to be made

·To measure success or failure

INTERNALLY-DRIVEN REASONS

·To determine whether goals and objectives have been met

·To engender accountability

·To identify and address problems and challenges

·To determine whether event management functions have achieved expected outcomes

·To understand who attends the event to determine who else can be targeted

·To determine worth of the event to its workers and volunteers

·To determine whether and how the event can remain viable and become sustainable

EXTERNALLY-DRIVEN REASONS

·To determine level of awareness of sponsors’ products/services

·To satisfy accountability requirements

·To determine level of media interest and coverage

·To determine event’s level of impact on tourist arrivals

·To determine event’s level of impact on business and other related industries

·To determine whether event satisfies the expectations of community stakeholders

·To determine worth of the event to its patrons

What to Evaluate?

1. Event Elements – number of patrons; size of group; demographics of patrons; attendees' address; source of information on the event; number of times attending; patron satisfaction; quality and impact of event programme; quality of merchandise; food and beverage, etc.

2. Event Organization Elements – nature and quantum of resources; human resources; volunteers' perceptions working on the event; income and expenditure; cash flow; level of investment; sponsorship dollar, etc.

3. Event Context Elements – nature and amount of local suppliers used; impacts on event; impacts of event; community perceptions; level of media coverage; media value; interest and presence

Event Evaluation Data

Numerical and Descriptive

Numerical

Descriptive

Evaluation Considerations

• Does the event organization/event host have, or have access to the requisite resources to conduct an event evaluation exercise?

• Have event evaluations been conducted on the event in the past? And if so, how often and what kind of evaluation?

• What use was made of the evaluation report?

• What is the nature of the environment in which the event operates?

• Can it facilitate the execution of an event evaluation?

• Does the event have access to a wide range of information it can use to conduct the evaluation exercise?

Event Evaluation Approaches (1)

• Quantitative – primarily concerned with counting complex data sets such as attendance levels, information sources and event activities.

• Challenges and limitations of using quantitative approaches exclusively:

Low response rates to surveys

Inconsistencies arising out of varied interpretations of terminology used in surveys leading to over-calculation and possible misrepresentation of data

What about Social Impacts?

• This video highlights some of the negative aspects of the Edinburgh International Festival.

• What types of approaches might be used to evaluate these?

Worries over complacency about festival

Event Evaluation Approaches (2)

• Qualitative – utilizes open-ended to allow for in depth responses and focuses on capturing opinion and attitudes on matters such as reasons for attending the event; benefits sought; level of satisfaction; stakeholder attitudes, etc.

• Can complement quantitative approaches because useful details can emerge that may not be derived or represented numerically

• Challenges and limitations of using qualitative approaches exclusively:

Cannot reach the numbers of quantitative approaches

Event Evaluation Approaches (3)

• Financial approaches – are concerned with collecting, assessing and monitoring numerical data on the fiscal elements of the event such as cash flow, gate receipts, patron expenditure, profit, loss and debt

• Economic approaches – examine numerical data related to employment, tax, visitor expenditure and triggered economic activity in other businesses and related industries

• Challenge with both approaches can be address through the use of non-financial and non-economic approaches

Event Evaluation Approaches (4)

• Non-financial and non-economic approaches – utilize qualitative approaches to monitor and assess fiscal and economic matters that cannot be easily quantified but which are relevant such as:

Assessment of intangible costs and benefits

Assessment of net value

Assessment of stakeholder perceptions

Assessment of economic and market factors that can influence the event

Event Evaluation Approaches (5)

• Critical three Ss

Event Significance – assessing, measuring and monitoring the nature, purpose and intent of the event and feasibility to realize the expected outcome

Event Success – determining whether the event execution and contributing elements have achieved the expected outcome

Event Sustainability – measuring, assessing and monitoring the event's impact on the environment and community and; the potential for the event to be continued in future

Event Evaluation Models• Impact Assessment – focuses on measuring economic, social,

environmental and cultural impact of events, particularly in instances where significant justification for the event may be required

• Cost-benefit Analysis – identifies and measures the costs and benefits of an event and takes into account 'externalities' or spill-over benefits for the host community (Burgen and Mules, 2000)

• Triple-bottom Line Evaluation – a systematic framework for measuring and reporting the event's performance against economic, social and environmental parameters, to determine negative or positive impacts on the host community (Fredline et al., 2005)

• Ethnographic Profile - Typically used in the evaluation of cultural festivals/special events, ethnography provides an assessment of these experience-related events through a process of analysis which encompasses participant observation, interviewing and documentary resources

Event Types & Evaluation Tools

• Content AnalysisImage Maker

• Visitor Exit SurveyTourism Product

• Economic Impact AssessmentEconomic Development Catalyst

• Social Impact Scale AnalysisTransformer of Socio-cultural Landscape

• Environmental Impact AnalysisSustainable DevelopmentModel

Consider Live Earth

• What type of event was it?• Which of the evaluation tools would provide the measures provided in the news report?• Was the event a failure?

Live Earth Concerts Backfire

Problems with Event Evaluation

• Missed opportunities to correct detected problems

• Minimal focus on evaluation at the planning phases, which is vital to understanding many decisions taken at the implementation stage

• Key elements of the evaluations may not be deployed effectively, if at all

Conducting Event Evaluation (1)

Step 1

•Determine the purpose of the event evaluation exercise, including whether the evaluation is necessary.

Step 2

•Identify what should be the focus of evaluation.

Step 3

•Select the most suitable approach for conducting the evaluation exercise.

Step 4

•Select model(s) for evaluation as a framework for data collection based on suitability, practicality and relevance.

Evaluation Exercise

• Complete steps 1 – 4 for the Ecounters13 event being held here at Porvoo Campus

• Display the steps in the form of a poster

Conducting Event Evaluation (2)

Step 5

•Develop appropriate instruments and data collection strategies based on the methods selected.

Step 6

•Collect and analyse data and findings.

Step 7

•Prepare and disseminate event evaluation report.

Step 8

•Formulate and implement decisions for improvement based on findings of event evaluation report.

Summary

• Evaluating events provides a measure for performance and delivery for event organizers, stakeholders and event consumers

• An event evaluation should focus on, among several other factors, the three Critical Ss – Event Significance, Event Success, and Event Sustainability

• Impact Assessment , Cost-benefit Analysis, Triple-bottom Line Evaluation and Ethnographic Profile are the four models that can be utilized to evaluate events

• Evaluation methods depend on the nature of the event and the requirements of stakeholders

References

• Tull, J. (2012) Event evaluation. In N. Ferdinand and P. Kitchin (Eds.) Events Management: An International Approach pp. 173-195. London: Sage Publications

1/4/12