u s ab i l i ty, a l oh a, an d g l ob al war mi n g: a u s ab i l i ...wix. pre and post test...

1

Usability, Aloha, and Global Warming: A Usability Study on Global Warming in Hawai‘i

University of Hawai‘i at Mānoa Honolulu, HI, USA mgamel@Hawai‘i.edu

Abstract: The Hawaiian community continues to be threatened by increased global temperatures provided through global warming. As coral reefs bleach and die in warm waters, hurricane occurrences increase, coastal erosions and landslides pile up, community members need to be aware of global warming impacts happening in their ‘aina. With awareness, people can make efforts to prepare for impending threats to their livelihoods as well as contribute to efforts to reverse global warming and climate change. In order to reach the greatest audience, a Wix website was created and a usability study was performed in order to best communicate global warming issues in Hawai‘i. The study assessed the website’s ease of use, navigation, and design in order to inform users about the global concern in their community and what can be done to help and protect their livelihoods. Throughout the study various changes were made in order to better develop content, navigation, and design. Materials were well received and new knowledge was formed throughout the usability study. Participants could easily maneuver between web pages and had little difficulty when locating information. Throughout the study participants indicated a strong sense of importance when conversing about global warming, but a lack of effort ultimately was still the ending conclusion to making lasting changes within their control to decrease global warming. Some people though experienced a strengthened desire to make more of an effort within their households and share their knowledge with friends and family members.

Introduction Ten of the warmest years have occurred within the 2000’s making global warming a rising concern (Shaftel, Jackson, Tennenbaum, 2016). This research has been only magnified with articles such as The National Geographic stating “ … our research suggests the warming will be quite rapid” (Jervey, 2014). So why is global warming a problem to Hawai‘i? Rising air and ocean temperatures change the ocean chemistry. With this event, the sustainability of ecosystems on land, in the oceans, and local communities have become threatened (National Climate Assessment, 2014). The need to inform the Hawaiian community about these high potential risks increases with these threats.

USABILITY, ALOHA, AND GLOBAL WARMING 2

According to the Pew Research Center, the Internet assists in informing the public, learning new topics, and increases the capacity to collaborate solutions (Purcell & Lee, 2014). This research thus began the development of a Wix website builder containing information of global warming in Hawai‘i. The purpose of this usability study was to explore the ease of use of an informative global warming website for people living in the islands of Hawai‘i. Hawai‘i is specifically impacted by global warming as rising temperatures increased the melting of the Arctic ice caps, providing more water to the world’s oceans rising sea levels, overwhelming sewer systems, impacting health and sanitation, bleaching coral colonies, and altering marine life and food (Melillo, Richmond, & Yohe, 2014). Literature Review As temperatures rise, health is a growing concern. And heat does not discriminate, everyone is at risk. Those at extra risk include the immune suppressed, young, old, and pregnant women. With higher temperatures, the islands can expect an increase of heat related effects such as heat stroke, dehydration, and heat cramps (Kolivras, 2006). The heat also can contribute to the varying degrees of rainfall between the years such as flash flooding and heightened drought concerns (National Climate Assessment, 2014). Over the past thirty years, Hawai‘i has been in a long standing drought, producing less consistent rainfall throughout the year. This has indicated the lack of freshwater resources for the Hawaiian Islands. Water that is needed in order to prevent the threat of impending dehydration. Warmer weather increases mosquito outbreaks inhibiting Hawai‘i. As mosquitos increase, the risk of transferable diseases such as Dengue fever and the Zika virus spread amongst Hawai‘i (Disease Outbreak Control Division, 2016). Hawai‘i’s bath like waters are actually a reaction due to increased amounts of carbon dioxide in the air and sea water producing carbonic acid. This changes the balance of minerals in the water, affecting the coral, plankton, and other organisms to produce calcium carbonate. Elements essential for marine life in order to create and strengthen their skeletal structures and hard shells for survival (Climate Impacts in the U.S. Islands, 2016). As more organisms and coral begin to die off, the coral becomes a milky white and bleached out. This action’s reaction endangers coral and fish populations in the coral reef ecosystems. With this impact aquatic tourism declines as reefs are not as colorful as they used to be (Climate Impacts in the U.S. Islands, 2016). Fishermen and other services provided by coral reefs also become stressed as their typical 385 million dollar business becomes less desirable and more limited (Climate Impacts in the U.S. Islands, 2016). Droughts in Hawai‘i decrease water for crop irrigation and drinking water. As coral reefs and fisheries also become more condensed, Hawai‘i’s food supply to their own communities become worrisome. More and more people will become dependent on the federal government and/or


substitute healthy foods with cheaper fast foods, high carb/high fat, and higher health risks overall that come with poor diet (National Climate Assessment, 2014).

Hawai‘i is typically described as paradise, and with that beauty comes the throngs of diverse people coming to experience the aloha state. Tourism is the highest earning business the Hawaiian Islands have. Business in Waikiki alone each year is accountable for two billion dollars to the state (Climate Impacts in the U.S. Islands, 2016). As ice caps in the Arctic continue to melt with increasing world temperatures, the risk of losing Waikiki to rising tides is a considerable livelihood threat to Hawaiian residents profiting and living off the tourism industry. With the risks of global warming on the islands, this research worked to inform the Hawaiian community how their lives, health, and land may be at risk. Project Development The informative global warming website is available at http://mgamel4.wixsite.com/Hawai‘iglobalwarming. It was created using research provided in the literature review and references. All the photographs were taken from a still camera or iPhone. Pixabay was also used as a royalty free, Creative Commons website for image downloads. In creating the study, the instructional goals will use similar facets as Dumas and Redish’s Library Website study by understanding that, “... (1) the goal is to improve the usability of the interface, (2) testers should represent real users, (3) testers perform real tasks, (4) user behavior and commentary are observed and recorded, and (5) data are analyzed to recognize problems and suggest solutions” (Becker & Yannotta, 2013). These goals are specifically asked in the post test survey (Appendix N) with questions about design layout, navigation, ease of use, and additional comments. Using content from the interview tasks provided in Appendix G, the usability study conducted a think aloud protocol. This asked participants to verbalize their actions and understand their thought process when trying to locate information. This action alone ensures that the usability study is testing the interface and not the user (Becker & Yannotta, 2013). The theme and look of the website is seen in Appendices O - U. The website has a simple layout containing pictures on every page, along with the website menu. This provided clear information while still maintaining a familiar navigation. Each page had a different layout in order to induce interest and intrigue for users, decrease chances users will leave the site, and use the site as a learning tool. Examples of different layouts are seen in Appendix P - T with varying content materials. All pages contain beach tones and colors to contribute to the beauty and island style of Hawai‘i. Data collection materials consist of Google Forms (Appendix F & N), Interview Tasks (Appendix G), and Observational tools (Appendix H-L).

http://mgamel4.wixsite.com/hawaiiglobalwarming


Methods A usability study was chosen for this research in order to ensure that the topic of global warming in Hawai‘i was easy to access, easy to use, and easy to understand. Through the use of the study’s invaluable data, the website designer and researcher made essential changes in editing, explaining, and exposing website content to Hawaiian citizens.

Research Questions In this usability study, test participants have gone through a series of specific tasks to include these questions: can users determine the site content from the homepage, how easy or difficult is it for participants to navigate the website content, can participants identify ways to contribute to sustainable practices and help decrease the rise of global warming in their communities? Participants Subjects of this usability study worked to include current residents and native citizens on the islands of Hawai‘i. The subjects included people involved in the University of Hawai‘i at Mānoa as students, or staff throughout the UH college system. All participants were at least 18 years of age or older, and have a wide variety of economic, marital, and educational status. Subjects were proficient in English, Internet, and computer use. Nielsen and Tahir report in their various usability studies, that the earlier and more frequent the usability studies occur, the greater the impact will be on the final website design (Becker & Yannotta, 2013). This is incorporated into this usability study with the performance of three rounds of testing, with groups of three people per round, totalling nine participants. Recruitment process started January 15, 2017 with an initial recruitment email sent to possible participants (Appendix A). Upon verbal invitation to join research was agreed upon, a follow up email of research invitation was sent as well. This included fellow Learning Design and Technology Master’s students, University of Hawai‘i faculty, and students who resided in Hawai‘i. Those who expressed interest were sent a follow up email (Appendix B) discussing information discussing meeting times, location, and place. Subjects were asked for 20 minutes to 30 minutes of their time to complete the usability study. The usability study was conducted face-to-face, with a computer device provided at a mutually agreed public location. Those that wished to complete the interview via Google Hangout were sent an instructional email on how to set up Google Hangout (Appendix C). When completed, the participants were sent a confirmation email reiterating date/time and location of interview along with pre-test and consent survey links (Appendix D). The consent form, (Appendix E) described confidentiality, risks, and privacy concerns that are protected. The pre-test link (Appendix F) gathered precursory information. The consent form also outlined that the research is voluntary and provided data will be protected, destroyed, and deleted upon completion of the usability study.


The researcher encouraged participants, as an action to protect their anonymity, to refrain from personal information and experiences. The researcher also informed the subject that a form of coding was to be used to identify answers and enter data.

Instrumentation Instrumentation of this study used the computer screen recording, Screencastify, to analyze screen movement of participants looking at the informational site created by a website builder Wix. Pre and post test questionnaires and surveys used by email and Google Forms evaluated participant perceptions according to the study (Appendix F, M, N). During the interview, an interview task form worked to guide users through a few scenarios users may encounter when looking at the site (Appendix G). While subjects answered the tasks, the researcher used various forms to collect data and observation of the usability study. A preliminary participant data (Appendix H) was gathered prior observation information. An observation log was used to jot notes down that were outside other observation tools (Appendix I). Task data information form was used to clearly identify actions of users during study (Appendix J). The participant difficulties form was used to note continued frustration from users throughout the course of the interview, not categorized within the tasks that were asked. (Appendix K). Data observations were noted in the reflectionary form rating participants experience throughout the interview (Appendix L).

Procedures The study worked to use recruitment emails (Appendix A - D, M), Google Forms (Appendix E, F, N), Screencastify using via Chrome web browser, Wix built website provided at http://mgamel4.wixsite.com/hawaiiglobalwarming, and Google Hangouts. Access to these procedures are available Internet sources from the University of Hawai‘i at Mānoa or another private server available from the researcher. Access to Screencastify, Google Hangout, and website were provided by the researcher during the interview. Participants were asked through email, to complete pre and post test survey questions provided by researcher. Email was sent through Gmail. Subjects also completed a fillable paper form of surveys provided by researcher. Data Analysis There was a total of nine participants and three rounds of the study equating three people per round. Each round of data was collected prior, during, and after the interview. Through careful analysis from Screencastify, Google Forms, and notes during each usability study the analysis worked to clarify the usability of the informational global warming website. This was done through coordinating likert scales to identify familiarity, ease, and change used in Appendices F-N.

http://mgamel4.wixsite.com/hawaiiglobalwarming


Action/edits were made to significant issues and further tried in future studies. Participant observation tools, task data info forms, and participant information difficulties, were used to help the researcher make observations and suggested changes for the website. Overall participants seemed were varied in age as seen in Figure 1.

Majority of participants were current students at the University of Hawai‘i Mānoa and one participant was an alumnus. All participants resided on Oahu, visited websites on a daily basis, and agree that Global Warming affects Hawai‘i. Figure 2 shows responses on comfort level with technology showing a fair amount of computer literacy for this usability study.

As far as global warming, majority of participants believed it is somewhat important to very important. Knowledge of participant background was helpful in order to form realistic tasks during the interview that would better enhance the website. Results and Actions During the interview several changes were made in order to obtain the project goal to make an informative global warming website about its’ impacts in Hawai‘i that was easy to access, easy


to use, and easy to understand. Table 1 shows the various rounds of participants and difficulties they insued.

Table 1. Participant Difficulties

Round one had the greater amount of difficulties with the website while round three had less trouble, but round three’s issues concerned redundant serious concerns. With the information presented in the table above, changes between website pages occured. The home page went through various changes. Figure 3 below showcases some of the changes.


Figure 3. Home page changes.

The next page, impacting livelihoods, also underwent minor changes that included title format for topics and equalizing amount of information available on bullet points.

The health risks page underwent several versions in order for users to effectively derive information from the site. Because this site was originally in a slideshow, it was difficult for users to understand the full site purpose. Adding thumbnails only confused users further as the linking did not work optimally. So eventually the last round went back to traditional scrolling picture and text. Stages of site are seen in Figure 4.


Figure 4. Health risk page changes

The change in land and sea site went through some minor changes more for visual appeal and content presentation. Text was added to columns that had less text originally in order to create a more balanced appearance that flowed with the content more appealing. The, what can we do page, was similar to the health risks page in the sense that it had ultimately changed from a slideshow type view to clear concise scrolling text and images page. To answer the first research question, can users determine the site content from the homepage, task interview question were graded on a one to three point scale. A point of one given for each interview questions indicates the user cannot perform the task asked. A point of two indicates the user can perform the task but has some struggles. And a point of three indicates the user can perform the task quickly with no issues. When the points converted to a percentage, the level of success for users to navigate the tasks are seen in Figure 5. Task information is seen in the below Table 2.


Figure 5. Success rate of completion of tasks


As the majority of the tasks have over a 95% rating this usability study was mostly successful. When identifying task two with a 77.8% rating we look back to the task 2 information. This page was health risks, and as identified above was a page that had several difficulties and versions applied. Also the page changed the title of mosquitoes to dengue fever, zika, and mosquitoes. This had allowed users to better identify task information and navigation on the site. Regarding the first research question, can users determine the site content from the homepage, this was easily understandable. All 9 participants instantly knew the site was about global warming and Hawai‘i. Participant I specifically indicated that the menu bar contributed to the ease of navigation to course contents. This data is supported through Figure 6.

When addressing the research question number two, how easy or difficult is it for participants to navigate the website content, all participants had answered strongly agree as seen in Figure 7.


When discussing layout, design, navigation, and content 77.8% of participants (7 people) expressed strongly agree while 22.2% (2 people). This indicates that the website was user friendly, satisfactory in finding information, and well organized. All points that are essential when using a web page to spread knowledge to greater audiences.

Research question three, can participants identify ways to contribute to sustainable practices and help decrease the rise of global warming in their communities, easily found the ways to the contribute page throughout all three rounds. Thus participants found a variety of contributions they could personally do to prevent global warming. Participant reactions are seen in Table 3. Four responses indicated no lifestyle changes in order to contribute their efforts to global warming or even changes on their opinions on the subject of Global Warming. Two participants talk about how their opinion remains the same, while two other participants respond with positive feedback with efforts to make small changes within their own control. In retrospect, the research question asks if people can identify ways to contribute to the decline of global warming. This was seen with the success in participants when correctly navigating to the what can we do page. Comments made in the interview and received through the post survey from participants were identified in the earlier Table 1.

Conclusion/Discussion In the sense that the usability study was user friendly, easily navigated, and contained informative global warming information in Hawai‘i, the study was a success. The researchers ultimate goal of impacting users with gainful insight and motivate their actions to be better for the environment was moderate. Some people indicated they would further some of their lifestyle


habits, but the majority relayed no change of opinion or habits. This may be because the users were more focused on completing each task, instead of taking more time to read the content available. This may also be because the energy that is necessary to invoke in some of the decline of global warming habits required too much work. Extra transportation to recycling farms, added inconvenience to carrying garbage, or the sacrifice to change their eating habits. The usability study could have encouraged more questions concerning how participants felt about the content, and how it had an impact to their lives. Questions could have also been more focused on participant observations, instead of success of completed tasks through the usability study. If further work would be done with this study, there would need to be a longer amount of dedicated time for open communication about web pages and participant impacts with the content displayed. This would be more of a free flowing conversation and guided questions of “why does that surprise you” or “what strikes you about this page”. This approach potentially could help the researcher understand whether the site made further impactful changes in the users and why. Even so, global warming continues to be an increasingly dangerous global event that affects the entire world as a whole. Thus, for the health of ourselves, our future, and our ‘aina, change is a must. Expanding the knowledge through ease of use in the website, while also broadening the understanding of global warming in Hawai‘i, was one of the first steps to prolonging the benefits of Hawai‘i and it’s citizens.


References Becker, D. A., & Yannotta, L. (2013). Modeling a Library Website Redesign Process:

Developing a User-Centered Website Through Usability Testing. Information Technology & Libraries, 32(1), 6-22.

Bradford, A. (2014, December 15). What Is Global Warming? | Global Warming Facts. Retrieved

from http://www.livescience.com/37003-global-warming.html

Climate Impacts in the U.S. Islands. (2016). Retrieved August 23, 2016, from https://www.epa.gov/climate-impacts/climate-impacts-us-islands

Disease Outbreak Control Division. (2016). Retrieved August 23, 2016, from

http://health.Hawai‘i.gov/docd/dib/disease/zika_virus/ Foundation, G. C. (2016). Should You Eat Less Meat? Retrieved August 26, 2016, from

http://www.sustainabletable.org/794/should-you-eat-less-meat Hawai‘i to experience worst-ever coral bleaching due to high ocean temperatures. (2015,

September 13). Retrieved Agust 25, 2016, from https://www.theguardian.com/us-news/2015/sep/13/Hawai‘i-coral-bleaching-scientists-predict-worst-ever

Jervey, B. (2014, February 11). Where Global Warming Went: Into the Pacific. Retrieved May

01, 2016, from http://news.nationalgeographic.com/news/2014/02/140211-global-warming -pause-trade-winds-pacific-science-climate/

Kolivras, K. N. (2006). Mosquito Habitat and Dengue Risk Potential in Hawai‘i: A Conceptual

Framework and GIS Application. Professional Geographer, 58(2), 139-154. doi:10.1111/j.1467-9272.2006.00521.x

Lallanilla, M. (2015, February 15). Greenhouse Gas Emissions: Causes & Sources. Retrieved

August 22, 2016, from http://www.livescience.com/37821-greenhouse-gases.html Lloyd, R. (2011, February 23). Why Are Americans So Ill-Informed about Climate Change?

Retrieved May 01, 2016, from http://www.scientificamerican.com/article/why-are-americans-so-ill/

Melillo, Jerry M., Terese (T.C.) Richmond, and Gary W. Yohe, Eds., 2014: Highlights of

Climate Change Impacts in the United States: The Third National Climate Assessment. U.S. Global Change Research Program, 148 pp.

National Climate Assessment. (2014). Retrieved May 01, 2016, from

http://nca2014.globalchange.gov/highlights/regions/Hawai‘i#intro-section-2

https://www.theguardian.com/us-news/2015/sep/13/Hawai%E2%80%98i-coral-bleaching-scientists-predict-worst-ever

http://www.livescience.com/37821-greenhouse-gases.html

http://news.nationalgeographic.com/news/2014/02/140211-global-warming-pause-trade-winds-pacific-science-climate/

https://www.theguardian.com/us-news/2015/sep/13/Hawai%E2%80%98i-coral-bleaching-scientists-predict-worst-ever

https://www.epa.gov/climate-impacts/climate-impacts-us-islands

http://news.nationalgeographic.com/news/2014/02/140211-global-warming-pause-trade-winds-pacific-science-climate/

http://www.scientificamerican.com/article/why-are-americans-so-ill/

http://www.livescience.com/37003-global-warming.html

http://www.sustainabletable.org/794/should-you-eat-less-meat

http://nca2014.globalchange.gov/highlights/regions/hawaii%23intro-section-2


Nuwer, R. (2014, April 30). Mosquitoes Kill More Humans Than Human Murderers Do. Retrieved August 23, 2016, from http://www.smithsonianmag.com/smart-news/mosquitoes-kill-more-humans-human-murderers-do-180951272/

Oakley, N. S., & Daudert, B. (2016). Establishing Best Practices to Improve Usefulness and

Usability of Web Interfaces Providing Atmospheric Data. Bulletin Of The American Meteorological Society, 97(2), 263-274. doi:10.1175/BAMS-D-14-00121.1

Purcell, K., & Lee, R. (2014, December 8). "Americans Feel Better Informed Thanks to the

Internet." Pew Research Center: Internet, Science & Tech. 08 Shaftel, H., Jackson, R., & Tennenbaum, L. (2016). Global Climate Change. Retrieved May 01,

2016, from http://climate.nasa.gov/

Shah, A. (2015, February 02). Action on climate change is cheaper than inaction. Retrieved May 01, 2016, from http://www.globalissues.org/article/806/action-cheaper-than-inaction

http://www.smithsonianmag.com/smart-news/mosquitoes-kill-more-humans-human-murderers-do-180951272/

http://climate.nasa.gov/

http://www.smithsonianmag.com/smart-news/mosquitoes-kill-more-humans-human-murderers-do-180951272/


Appendices

Appendix A

Instrumentation: Email Recruits

To: Email Recruits

From: Megan Gamel; mgamel@Hawai‘i.edu

Subject: Assistance for Global Warming Usability Study!

Aloha! Are you feeling the heat in Hawai‘i? Between the weather, hurricanes, warmer waters, and dengue fever, Global Warming has made its appearance on the islands! Come and take a look at exactly how it has changed our ‘Aina! Learn how YOU can be impacted, and how you can personally help to give our community a chill pill. Join me, Megan Gamel, in developing a user friendly Informative Global Warming in Hawai‘i Website by taking part in my usability study! The purpose of this usability study will be to explore the ease of use in the website for people living on the islands of Hawai‘i.

● Usability Study will be conducted face-to-face, with a computer device provided at a mutually agreed public location.

● I can also conduct off site, simultaneous web conferencing through google hangouts at an agreed upon time.

● All information will be kept strictly confidential. ● A report of the study will be available to study volunteers.

If you are interested in participating in this research project and have received this e- mail, please contact Megan Gamel at mgamel@Hawai‘i.edu for more information. Mahalo for your time! Hope to hear from you soon. Sincerely, Megan Gamel

mailto:[email protected]



Appendix B

Instrumentation: Interested Email Recruits

To: Email Recruits



Aloha! If you are receiving this, it means that you expressed an interest for helping me out with my usability study. Thank you! That being said, I’m planning on conducting my next round of studies [place dates/times]. I’m likely going to be conducting the study on-campus at the University of Hawai‘i at Manoa, either in a conference room or in the classroom of Kuykendall Hall, Wist Hall, or the Information Technology Center (ITC building). I can also conduct off site, simultaneous web conferencing through google hangouts at an agreed upon time. Please let me know which day, times, and locations might work best for you! I look forward to your reply, and will also forward you more information about my study when I receive a reply from you. Mahalo for your time, assistance, and interest! Megan Gamel



Appendix C Instrumentation: Google Hangout Email

To: Email Recruits


Subject: Assistance for Global Warming Usability Study! Google Hangout…?

Aloha [name], I’m writing because you have expressed participation in my study via google hangout. To do this I will need your gmail account. If you do not have one please let me know and I can set you up with a temporary one for the study purposes. At the time of our scheduled “hangout” please go to hangouts.google.com. Sign into your gmail, and wait for my call coming from “Megan” at mgamel@Hawai‘i.edu. We can set up a time to test our connection before our meeting to ensure compatibility. Please indicate times/dates you’d be available before our [date/time] meeting. Testing should take up to 15 minutes. Looking forward to “hanging” out, and thank you again for your participation! Mahalo, Megan Gamel


https://hangouts.google.com/



Appendix D Instrumentation: Consent, Pretest Email

To: Email Recruits



Aloha, I look forward to working with you on [date/time/place]. Thank you again for participating in my study. In the meantime if you could please fill out the following surveys to provide consent and answer a few questions before our meeting. Consent form https://docs.google.com/a/Hawai‘i.edu/forms/d/e/1FAIpQLSf0jHjyjsAEpptfdFCtJhUN7jhqUGnhYyiKIE70wY1luSfQzA/viewform Survey https://docs.google.com/a/Hawai‘i.edu/forms/d/e/1FAIpQLSfW45dVnxUjdT9W70YVyLYH68ZcRbN2W1k9mIZBOmjJssYjkQ/viewform Let me know if you have any problems or difficulties accessing the links. Mahalo for your time, assistance, and interest! Megan Gamel


https://docs.google.com/a/hawaii.edu/forms/d/e/1FAIpQLSf0jHjyjsAEpptfdFCtJhUN7jhqUGnhYyiKIE70wY1luSfQzA/viewform

https://docs.google.com/a/hawaii.edu/forms/d/e/1FAIpQLSfW45dVnxUjdT9W70YVyLYH68ZcRbN2W1k9mIZBOmjJssYjkQ/viewform




Appendix E Instrumentation: Google Form Consent

Link: https://docs.google.com/a/Hawai‘i.edu/forms/d/e/1FAIpQLSf0jHjyjsAEpptfdFCtJhUN7jh

qUGnhYyiKIE70wY1luSfQzA/viewform

CONSENT TO PARTICIPATE IN RESEARCH PROJECT Evaluating the usability of global warming in Hawai‘i website from University of Hawai‘i faculty and students. Aloha! My name is Megan Gamel, and I am a student at the University of Hawai`i at Mānoa. I am conducting this research project as a component for a Master’s degree in Learning Design and Technology. The purpose of this usability study will be to explore the ease of use in a Informative Global Warming Website for people living in the islands of Hawai‘i affiliated with the University of Hawai‘i system. You are being asked to participate in this research study because you are affiliated with the University of Hawai‘i system, and live on the islands of Hawai‘i who may or may not have experience with Global Warming and Climate Change. Your participation in this study will help determine the changes to be implemented on the website to improve user experience. What activities will you do and how long will they last? Participation will be either in-person using a provided computer device and internet connection at a mutually agreed location. Participation can also be done through simultaneous web conferencing such as Google Hangout at mutually agreed time with computer devices and internet between both subject and facilitator. If you participate, you will be asked to navigate through the website while being prompted by a series of scenario questions. These questions are intended to evaluate the ease of use and effectiveness of the website. You will be asked share your thoughts out loud as you navigate the website, which will assist the researcher in gaining further insights into the user experience. Participation in the project will consist of an interview with a student investigator. During the interview, you will be asked questions about yourself and your experiences. All interview questions are optional; you are not required to answer every question in order to participate. The interview will be audio and screen recorded for the purpose of




analysis using Screencastify. The recordings may be used as qualitative data in the research. Once all scenarios are completed, the researcher may ask follow up questions as needed. You will be asked to complete an online demographic survey form prior to beginning the usability test, as well as a user satisfaction survey after participating. The user satisfaction survey is intended to gather information on ease of use and navigation throughout the website as a whole. The entire usability study, including both surveys and debriefing interview, will last about 45-60 minutes. The investigator believes there is little or no risk to participating in this research project. Participating in this research may be of no direct benefit to you. It is believed, however, that the results from this project will help the understanding of linguistic patterns and how linguistic information is stored in the mind and accessed during the production of speech. There is no compensation for time spent participating in the research project. If you choose to take part, you do so voluntarily. Confidentiality and Privacy: The data taken from your participation in this study will be used solely for the purpose of this usability study. The data will be stored securely on a password-protected computer. When I report the results of my research project, I will not use your name or any other personal information that would identify you. The recordings from this study will be transcribed to determine commonalities from all participants. Once the research is complete, all recordings will be destroyed. Voluntary Participation: Participation in this research project is strictly voluntary. You are free to choose to participate or not to participate in this project. You may withdraw your permission or discontinue participation at any time without penalty or loss of benefits. Questions: If you have any questions regarding your participation in this research study, please contact me via email at mgamel@Hawai‘i.edu. You may also contact Faculty Advisor


Curtis Ho at curtis@Hawai‘i.edu. If you have any questions about your rights in this project, you can contact the University of Hawai`i, Human Studies Program, by phone at (808) 956-5007 or by e-mail at uhirb@Hawai‘i.edu. Participant: I have read and understand the above information, and agree to participate in this usability study. I understand that I can change my mind about being in the project at any time by notifying the researcher.

❏ Yes ❏ No

I understand that my verbal responses and screen activity will be recorded as I participate in this usability study. These audio recordings will only be accessed by the researcher and will be destroyed once the research is complete.

❏ Yes ❏ No

Please print, date, and sign your name to give consent to participate in the usability study. ___________________________________ _____________ Print name Date ___________________________________ _____________ Signature Date Online google form digital copy available here



Appendix F Instrumentation: Pre-test Questionnaire

Link: https://docs.google.com/a/Hawai‘i.edu/forms/d/e/1FAIpQLSfW45dVnxUjdT9W70YVyLY

H68ZcRbN2W1k9mIZBOmjJssYjkQ/viewform

Participant Pre - Survey for Global Warming Website Usability Study Thank you for your interest in taking part in this study. This pre-survey is designed to gather background information regarding demographics, attitudes, and technology use for the research purposes only, and is not meant to assess your individual performance. This survey should only take you 5-10 minutes at most to complete. Demographics: Demographics

❏ Male ❏ Female ❏ Prefer not to say

What is your age group?

❏ 18-24 ❏ 25-34 ❏ 35-44 ❏ 45-54 ❏ 55-64 ❏ 65+

Are you a Student, Faculty member, or Alumni of University of Hawai‘i Systems?

❏ Student ❏ Faculty ❏ Alumni ❏ Combination of the three

What is your highest level of completed education?

❏ High School Diploma/GED ❏ Associate Degree ❏ Bachelor’s Degree ❏ Master’s Degree ❏ Doctorate Degree

What island do you currently reside on?




❏ Hawai‘i ❏ Maui ❏ Lānaʻi ❏ Molokaʻi ❏ Oʻahu ❏ Kauaʻi

What is your experience? This section of the survey is intended to gather information pertaining to the amount of experience and preconceived knowledge you may have on global warming and technology. How comfortable are you with technology?

❏ Not at all ❏ Somewhat comfortable ❏ Comfortable ❏ Very Comfortable

How often do you visit websites?

❏ Rarely- never ❏ Couple times a month ❏ Couple times a week ❏ Couple times a day

Do you think Global Warming affects Hawai‘i?

❏ Yes ❏ No ❏ Indifferent

Do you believe you can contribute to the decline of Global Warming?

❏ Strongly Disagree ❏ Disagree ❏ Neither Agree or Disagree ❏ Agree ❏ Strongly Agree

What is your opinion of Global Warming?

❏ Very Important ❏ Somewhat Important ❏ Indifferent ❏ Don’t care/Not my problem ❏ Global Warming is not real


Mahalo for your help! Looking forward to meeting with you!

Online document available here



Appendix G Instrumentation: Interview Task Questionnaire

Link: https://docs.google.com/document/d/1__hR0cSFNnMAmCyWcgd1SFEhS5ZJGcji06mYl

rkyHCc/edit?usp=sharing

Interview Tasks

Global Warming in Hawai‘i

Megan Gamel

I. Before the session: 1. Setting up Screencastify extension in Google Chrome browser 2. Getting the website ready 3. Testing the website walkthrough to check the functionality of the mic and

webcam II. During the session:

● Start Screencastify

Facilitator Script

Hi, [insert participant’s name]. My name is Megan Gamel, and I’m going to be walking you through this session today.

Before we begin, I have some information for you, and I’m going to read it to make sure that I cover everything.

We’re asking people to take a look at a website I have designed for a portfolio project in the LTEC Masters program. I would like to see what you think of them and how you think you would complete a few tasks with an interface like this. The session should take about one hour.

The first thing I want to make clear right away is that we’re testing the website, not you. You can’t do anything wrong here. In fact, this is probably the one place today where you don’t have to worry about making mistakes.

As you complete the tasks, I’m going to ask you as much as possible to try to think out loud: to say what you’re looking at, what you’re trying to do, and what you’re thinking. This will be a big help to us.

Also, please don’t worry that you’re going to hurt my feelings. I’m doing this to improve my designs, so I need to hear your honest reactions.

https://docs.google.com/document/d/1__hR0cSFNnMAmCyWcgd1SFEhS5ZJGcji06mYlrkyHCc/edit?usp=sharing

https://docs.google.com/document/d/1__hR0cSFNnMAmCyWcgd1SFEhS5ZJGcji06mYlrkyHCc/edit?usp=sharing


If you have any questions as we go along, just ask them. I may not be able to answer them right away, since I’m interested in how people do when they don’t have someone who can help. But if you still have any questions when we’re done I’ll try to answer them then.

And if you need to take a break at any point, just let me know. Do you have any questions so far?

● Ask participant a few preliminary questions:

OK. Before we look at the site, I’d like to ask you just a few quick questions about your experience working with websites.

1. How often would you say you use websites to learn about new topics or information?

2. How comfortable would you say you are with navigating through a website to find information?

3. Have you ever built or helped edit a website?

4. If so, what program or software did you use?

OK, great. We’re done with the questions, and we can start testing out the site.

● Have participants do a narrative of the website's overall appearance for one or two minutes, at most:

I’m going to ask you to look at this websites of the homepage and tell me what you make of it: what strikes you about it, what you info can you find here, and what it’s for. Just look around and do a little narrative. You can scroll around if you need to.

● Ask participant to complete a few specific tasks (be sure to give the participant a handout of the scenarios):

Thanks for doing that. You did a great job. Now I’m going to ask you to try doing some specific tasks. I’m going to read each one out loud. You should have received a copy of these before this study. Again, as much as possible, it will help me if you can try to think out loud as you go along.

Task One:

You have a friend in town visiting and wanted to take her snorkeling. Upon arrival you find that your favorite spot that used to have vivid coral and marine life, is bleached and not as spectacular as it used to be. Overhearing this may be a cause of global warming,


how would you navigate this website to find out more information about coral bleaching?

Task Two:

You hear on the news about increasing outbreaks of dengue fever on the Big Island due to increased global warming and climate changing conditions. Planning to make a trip next month for vacation, where would you look on this site to determine what preventative measures you should implement and areas to avoid on your vacation?

Task Three:

Your uncle is a fisherman and at the family reunion ends up asking your family for some help financially. He states with the hotter waters he has not been as successful catching the fish he needs to make ends meet affecting his livelihood. Where else would you go to see how global warming and climate change can impact your family's livelihood?

Task Four:

Feeling helpless about the inevitable global warming cases, where would you look to see what you can do to take part in the fight against global warming?

Task Five:

If you are interested in contributing to this page with either an article, question, blog, or experience where would you submit your email?

Task Six:

You have spent an hour looking over the site and now you are interested in the authors and references of the website. Where would you go to find this information?

Task Seven:

How do you return to the home page?

Thanks, that was very helpful.

We are done with the main questions, but I have a few more general questions to ask you.

1. On a scale of 1 to 5, with 1 representing very difficult and 5 representing very easy, how would you rate your experience during today's testing? Please elaborate on your rating.

2. On a scale of 1 to 5, with 1 representing very difficult and 5 representing very easy, how would you rate the ease of use/navigation of the website? Please elaborate on your rating.


3. After participating in this study, would you recommend this website to any of your friends? Why?

4. After participating in this study, would you include any habits or contributions to help decrease global warming? If so what?

That’s the last question, do you have any questions for me now that we’re done?

I want to thank you for your time and willingness to be a participant in this study.

● Stop the screencasting software

After the Session:

1. Save screencast to your local computer 2. Quickly scrub through the video to ensure the integrity of the audio and video 3. After completing both sessions, upload the video to Google Drive


Appendix H Instrumentation: Preliminary Participant Data

Link: https://docs.google.com/a/Hawai‘i.edu/spreadsheets/d/1CH_cbTx1fuUrlzzAYSj-M4-n1zt

EoT-ODB_YRT5y5WE/edit?usp=sharing

https://docs.google.com/a/hawaii.edu/spreadsheets/d/1CH_cbTx1fuUrlzzAYSj-M4-n1ztEoT-ODB_YRT5y5WE/edit?usp=sharing

https://docs.google.com/a/hawaii.edu/spreadsheets/d/1CH_cbTx1fuUrlzzAYSj-M4-n1ztEoT-ODB_YRT5y5WE/edit?usp=sharing


Appendix I Instrumentation: Participant Observation Log

Link: https://docs.google.com/document/d/1923harBPveey0bsRlEW0nYVrHGyl7S2Wly12De

D4TBs/edit?usp=sharing

Participant Observation Log Tester #/ Group:

Date:

Time:

Location:

Video Length: Observations:______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Reflections:__________________________________________________________________________________

https://docs.google.com/document/d/1923harBPveey0bsRlEW0nYVrHGyl7S2Wly12DeD4TBs/edit?usp=sharing

https://docs.google.com/document/d/1923harBPveey0bsRlEW0nYVrHGyl7S2Wly12DeD4TBs/edit?usp=sharing


______________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Suggested Changes: ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________


Appendix J Instrumentation: Task Data Info Log

Link: https://docs.google.com/spreadsheets/d/1oujOTJk7TrDBITdbDkIKMbKSHGwf1C6VST

RpSkDLdtk/edit?usp=sharing

https://docs.google.com/spreadsheets/d/1oujOTJk7TrDBITdbDkIKMbKSHGwf1C6VSTRpSkDLdtk/edit?usp=sharing

https://docs.google.com/spreadsheets/d/1oujOTJk7TrDBITdbDkIKMbKSHGwf1C6VSTRpSkDLdtk/edit?usp=sharing


Appendix K Instrumentation: Participants Difficulties

Link: https://docs.google.com/spreadsheets/d/1R_SlF-OntZOKWvLz1Ilcs9ro31okj8e1mwtzzlc

KxYg/edit?usp=sharing

https://docs.google.com/spreadsheets/d/1R_SlF-OntZOKWvLz1Ilcs9ro31okj8e1mwtzzlcKxYg/edit?usp=sharing

https://docs.google.com/spreadsheets/d/1R_SlF-OntZOKWvLz1Ilcs9ro31okj8e1mwtzzlcKxYg/edit?usp=sharing


Appendix L Instrumentation: Follow Up/ Reflection

Link: https://docs.google.com/spreadsheets/d/1xzB9XkNYUU5rCZlXzzNDRNdG1SL8TrBHmi

iIV8-CaNw/edit?usp=sharing

https://docs.google.com/spreadsheets/d/1xzB9XkNYUU5rCZlXzzNDRNdG1SL8TrBHmiiIV8-CaNw/edit?usp=sharing

https://docs.google.com/spreadsheets/d/1xzB9XkNYUU5rCZlXzzNDRNdG1SL8TrBHmiiIV8-CaNw/edit?usp=sharing


Appendix M Instrumentation: Thank You/ Post-test Link Email

To: Email Recruits



Aloha, Thank you again for taking the time out of your schedule to meet with me [ on day]. Your input is very appreciated and important for me in completing my Masters program. Below is the post-survey that I mentioned. If you can please answer the following questions that will help me round out my research for data purposes. https://docs.google.com/a/Hawai‘i.edu/forms/d/e/1FAIpQLScMXW4xWsA5ZDNM4rVCt-A2E8NPeFcjM1GzCeVgZbXLaupkiw/viewform Do not hesitate to contact me with any questions or concerns! Thank you again for all of your help! Mahalo, Megan Gamel

https://docs.google.com/a/hawaii.edu/forms/d/e/1FAIpQLScMXW4xWsA5ZDNM4rVCt-A2E8NPeFcjM1GzCeVgZbXLaupkiw/viewform




Appendix N Instrumentation: Post-test Questionnaire

Participant Post Survey Form Thank you so much for participating in my usability study! Below are questions I'd like for you to answer about your experience with the website. Your responses will be kept for research purposes only and will not assess your individual performance. This post-survey should take no longer than 10 minutes to complete. Please respond with as much detail in order for me to better enhance and clarify the website. Mahalo! Design Layout: Was the website easy to understand?

❏ Strongly Disagree ❏ Disagree ❏ Neither disagree or agree ❏ Agree ❏ Strongly Agree

Was the text on the website clear and easy to read?


The website is visually appealing?


The organization of the site is logical and easy to follow.


Navigation:


The website was easy to navigate.


Tab labels on website were clear and concise.


I was able to navigate freely through the website without.


Ease of Use: The website was user friendly.


The website has a clean and simple presentation.


The website's layout, design, navigation and content were clearly defined and understandable.

❏ Strongly Disagree ❏ Disagree ❏ Neither disagree or agree ❏ Agree


❏ Strongly Agree Additional Comments What parts of the Website did you enjoy? Are there any parts of the Website that were confusing or did not make sense? If so please explain. Any suggestions to better enhance the website? Has your opinion about Global Warming changed? If so please explain. Mahalo! You are now complete! Thank you for your time and assistance! Online version

available https://docs.google.com/a/Hawai‘i.edu/forms/d/e/1FAIpQLScMXW4xWsA5ZDNM4rVCt-A2

E8NPeFcjM1GzCeVgZbXLaupkiw/viewform




Appendix O Website Prototype: Home


Appendix P Website Prototype: Impacting Livelihoods


Appendix Q Website Prototype: Health Risks


Appendix R Website Prototype: Land and Sea

Appendix S Website Prototype: What Can We Do?


Appendix T Website Prototype: Contact/Contribute


Appendix U Website Prototype: References

u s ab i l i ty, a l oh a, an d g l ob al war mi n g: a u s ab i l i ...wix. pre and post test...

Documents