(1) makahiki: an extensible open-source platform for energy challenges george lee collaborative...
TRANSCRIPT
(1)
Makahiki: An Extensible Open-Source
Platform for Energy Challenges
George LeeCollaborative Software Development
LaboratoryInformation and Computer Sciences
University of Hawaii
(2)
Energy Competitions
What is an energy competition?• Groups try to reduce energy usage as much as possible• Can involve participation in events
Why have competitions?• Save money on energy costs• Opportunity to educate people about saving energy
Colleges have done this for over 20 years
(3)
Dorm Energy CompetitionsHodge, Chelsea, Dorm Energy Competitions, Presented at the Behavior, Energy, and Climate Change Conference, November 2010
Over 160 colleges in the US planned to have one in 2010-2011 academic year
Reduce energy use during the competition by 9% on average
(4)
Harvard Green Cup
(5)
Bowdoin
(6)
Energy Challenge Competitions focus on energy reduction
Energy literacy and sustained behavior change
Platform for creating energy challenges and researching behavior change
(7)
MakahikiEnergy competitions• Supports near-real time energy feedback• Event and activity tracking• Personalized feedback
Energy challenge• Pedagogical approach• Social network integration• A serious game
(8)
Serious Games Zyda, Michael, From Visual Stimulation to Virtual Reality to Games, IEEE Computer, September 2005
Definition: “a mental contest, played with a computer in accordance with specific rules that uses entertainment to further government or corporate training, education, health, public policy, and strategic communication objectives.”
(9)
Serious Games as MotivationMcGonigal, Jane, Reality is Broken, Penguin Press, January, 2011
Created several “Augmented Reality Games” that involve real world tasks
Games provide a sense of accomplishment that rewards effort
(10)
Serious Games and Productivity
Reeves, Byron, Total Engagement, Harvard Business School Press, 2009
People spend hours on online games
How can we keep people in the workplace engaged?
Harnessing techniques from games can make businesses more competitive
(11)
Serious Games and Learning Murphy, Curtiss, Why Games Work and the Science of Learning, http://www. goodgamesbydesign.com/Files/WhyGamesWork_TheScienceOfLearning_ CMurphy_2011.pdf
Examined how the Navy used a game to train recruits
Saw a 50-80% increase in individual performance after 1 hour of playing
(12)
“Smart Grid” GameWant to educate users on how to be “green”• Present tasks that help them be more aware• Declare commitments• Hold events like movie nights
Reward users in the form of points
A “Serious Game” based around energy literacy
(13)
Integration with WattDepot
Dorm MetersWattDepotSensors
WattDepotServer Makahiki
WattDepotCloud Cache
(14)
Near Real-Time Feedback Peterson et al, Dormitory residents reduce electricity consumption when exposed to real-time visual feedback and incentives, International Journal of Sustainability in Higher Education, 8(1):16–33, 2007.
Experiment at Oberlin College in 2005
Users with near real-time feedback outperformed those with “low-resolution” feedback
(15)
Go Low
(16)
Evaluating Makahiki Evaluation of the alpha version
Three phases of evaluation:• Mockup Evaluation• In-lab Evaluation• Beta evaluation
Actual competition in October 2011
(17)
Goals of the evaluation How can we provide a system that supports researchers interested in users participating in an energy competition?
How can we effectively test the website and the overall design of the competition?
(18)
Mockup Evaluation Created mockups using Balsamiq Mockups
Brought in friends and family to go through scenarios
Scenarios:• Initial set up• Getting points• Viewing energy use• Viewing prizes
(19)
Mockup Landing Page
(20)
Landing Page
(21)
Mockup Get Nutz Page
(22)
Get Nutz Page
(23)
Mockup Evaluation Results Major changes:• Energy Goal Voting removed• Simplification of the “Get Nutz” page• Altruistic raffle prizes
Points were a significant motivator
On the right track
(24)
Discount Usability Engineering
Nielsen, Jakob, Guerrilla HCI: Using discount usability engineering to penetrate the intimidation barrier. Cost-justifying Usability, pages 245–272, 1994.
Think aloud protocol, no need to capture video
Scenarios?
(25)
Hutchworld Cheng et al, Hutchworld: Lessons learned. Virtual Worlds, 1834/2000:12–23, 2000.
Developed by Microsoft to help cancer patients
Three dimensional interface
5 minute test period before the evaluation starts
(26)
Onboarding Users are unlikely to know how the system works beforehand
System has to guide the user and expose them to the various mechanics
The quest bar:• List of things users could do on the site• If completed, the quest does not appear• Not worth any points
(27)
Onboarding Evaluation Representative users
Procedure:• Think aloud protocol• Captured computer screen using software• Let the subject use the site for 45 minutes• Post-evaluation questions• $10 bonus and Gift Certificate quests
Two rounds of onboarding evaluations
(28)
April 2011 Results 5 first-year students from the Hale Lehua tower
Issues found:• Many users missed the quest bar• Rejected admin notifications were missed• Individuals frequently guessed at answers• Initial confusion on Go Low and Get Nutz
Positive feedback:• Found it addictive and would recommend it• One wished they could participate
(29)
July 2011 Results 5 first-year students from the Hale Lehua tower
Many improvements:• Time through setup improved• More questions answered correctly• More quests attempted and completed
Issues:• Quests were not set up properly• Timing of rejection notification
(30)
Beta Evaluation Evaluate the game over a period of time
Evaluation of logging system
Beta evaluation:• Brought in friends and family• Held a 1 week mini competition• Surveyed individuals after the competition• Also analyzed database and logs.
(31)
Beta Evaluation Results 4 teams with 5 subjects each
18 of the subjects logged in at least once
Results:• Averaged 915 points• Completed 16.2 activities
Issues:• Must detect Internet Explorer• Restrict the Daily Energy Goal Game’s points
(32)
2011 Kukui Cup 418 people logged in over the course of the competition
850 hours were spent on the site
42 individuals completed the round 3 survey• 25 found the system easy to use•Understood how to play the game
(33)
2011 Kukui Cup Survey Results
(34)
2011 Kukui Cup Survey Results
(35)
2011 Kukui Cup Survey Results
(36)
ContributionsMakahiki is an open source framework for energy competitions
Makahiki acts as a platform for researchers who are interested in user behavior during energy competitions
To evaluate the system, we created a methodology for testing Serious Games.
(37)
Questions? Kukui Cup Website: http://sites.google.com/site/kukuicup
Makahiki project website: http://github.com/csdl/makahiki/