Angela Murphy Amy Antonio

Shirley Reushle

Encouraging Active Participant

Engagement in the Evaluation of Online

Conferencing

Online conferences offer convenient professional

development opportunities without the travel and expense of face to face.

Most conference evaluations focus on participant

satisfaction and reactions and do not evaluate the learning

that actually occurs (Anderson & Anderson, 2010).

This study adapted the new learning methodology (Chapman

et al., 2007) to evaluate the impact of an online

conference holistically and interactively.

The Follow the Sun Online Learning Futures Festival ran non-stop for 48 hours, over six shifts of eight hours each, with consecutive handovers between

Australia (University of Southern Queensland), United Kingdom (Leicester University) and Canada (Athabasca University).

The aim of the conference was to bring together university staff and students from a range of disciplines across the world to share ideas and explore

knowledge development. 750 participants from over 35 countries registered for the conference.

The non-stop nature of the event aimed to

mirror a 24-hour digital society and

the 21st century learner who wants to be engaged with

other learners any time,

anywhere.

Image source: http://www.rgbstock.com/bigphoto/niXkXUu/World+Time+1

1. Identify participation trends during conference sessions.

Objectives & Approach:

Analytics

2. Explore networking and interactions trends. Twitter, chat and social media

3. Real-time insights into participant perceptions and new learnings.

4. Identify and track new learning and intention to action.

5. Presenter experiences and future support requirements.

6. Inform the conduct of future events.

Real time evaluation questions

Summative survey evaluation

Presenters’ survey

Focus group with organisers

“Please tell us about any new learning

you have experienced or any new questions that

have occurred to you as a result of this conference?”

(Chapman et al., 2007)

Image source: http://www.soil-net.com/

Festival analytics

51%

22%

Attended only one session

Attended two sessions

7% 3

Most online conference participants attend only

one or two sessions

5% 4

3% 5

4% 6

1% 7 8 9

Twitter and social media conversations

Conversational Tweets that included an

observation or expressed an opinion

#FTS12 Tweets could be categorised

into four groups

Promotional Tweets that promoted an upcoming presentation

Instructional Tweets with a directive to

perform an activity

“Recordings from #fts12 now available @ http://t.co/smWYn413”

“Interesting conversation about the future of the

world”

“Click latecomers link to join!

http://t.co/5iTLCwN”

“Nominal Group Technique Session @ #fts12. Join us!”

Informative Tweets that pointed to

resources with an accompanying link

Real-time evaluation questions

Please indicate your agreement with the sessions facilitated new ideas and learning : Total Sample (n=60)

Most found that the sessions inspired ideas for daily

practice that they intend to action

Summative survey evaluation for participants

4%

3%

6%

21%

28%

38%

None of these (please specify)

I tried to listen and multitask but didnt manageit very well

I dropped in and out of sessions

I listened occasionally while working or multi-tasking

I listened closely to the sessions but did not askquestions

I was actively listening, asking questions and/orcommunicating

Online conference participants listen and

participate actively during sessions Q8. Please indicate which of the following are the closest to the way in which you

participated in the sessions. (Select one) (n=116)

Summative survey evaluation for presenters

“It's more difficult to keep participants engaged because you can't rely on eye contact, body language or movement around the room to

help maintain interest. You also have to keep talking (most of the time) as the participant isn't sure what is happening during a pause.

You also have to keep faith that the technology will work for both yourself & the participants!”

“It was just hard to keep going with the presentation and keep a tab on what was going on

with the social media at the same time.”

“It was the first time I had talked 'at' participants for 30 min, rather than seeking feedback periodically.

Although I thought it went well, I did find that it was hard to present for that long without seeking some

kind of feedback along the way..”

Presenters found online presentations to

be challenging

Focus group for conference organisers

Time consuming and technical to set up ~ difficult within other

responsibilities Reputational impact and reach much greater than could be achieved through other means

Using the same old tech advertising didn’t reach new audiences within disciplines

Risk

Collaboration

Global reach

XII

III

Limited Responses

Time consuming

Difficult to organise

Immediate Feedback

Identify issues as they

occur

Address issues

promptly Create opportunities

for engagement

What barriers have you experienced in attempting to attend online events in general? Select those that are the most applicable to you) (n=172)

Difficulties in setting aside dedicated time above work commitments is the primary barrier

to attending online conferences

3%

5%

6%

7%

12%

16%

17%

34%

Other preferences

Mini webinars of no longer than half a day that include optionsto post questions and communicate with other delegates

Full online conference over a day

Full online conference over two or more days

Shorter sessions, e g , 30 minutes

Mini webinars of one or two hours that include options to postquestions and communicate with other delegates

Pre-recorded mini webinars that can be accessed at any time

An event with synchronous and asynchronous features, e.g.pre-recorded video accompanied by questions in real time

The combination of synchronous and asynchronous features is considered to be

the ideal online conference format

Learning, technologies and futures are the topics of greatest interest to most participants