hasc usability test results conducted 1/7-8, 2013 page 1 hasc usability test results conducted by...

HASC Usability Test ResultsConducted 1/7-8, 2013

HASCUsability Test Results

Conducted by Jayne SchurickUsability [email protected]

(408)353-1293


Objectives• As part of their anti-phishing program, HomeAway is redesigning the homeowner dashboard

so owners can more securely manage their communications with guests. This usability test was intended to explore the new communications workflow, as well as test several redesigned pages. The objectives of the test were to determine, for both basic and power users, whether:

– The communications workflow from guest inquiry to requesting a review makes sense, i.e., receiving and sending messages, next steps, and calls to action are obvious

– The workflow is efficient, i.e., at least as fast and easy as their current processes

Plus,

– Identify new/desired functionality that will improve both acceptance and usability of the new workflow


Test Format and Participants• Remote test using WebEx to allow participants to view and control the computer

displaying the prototype

• Conducted January 7 and 8, 2013

• 6 participants:

– 3 property managers (“power” users)

– 3 homeowners (“basic” users)

– All use HomeAway and/or VRBO and have recently submitted feedback, indicating they are interested in participating in usability testing


Tasks

1. In Gmail, view an inquiry

2. Reply to the inquiry

3. On the HA guest communications page, describe the page

4. Reply using the “available” template

5. Check your rates and include a quote

6. Edit the quote

7. Send the reply and quote

8. Go to the HA inbox and describe the page

9. Go the reservations and describe the page

10. Go back to the inbox and describe the page

11. Book the reservation

12. In Gmail, view the payment

13. Request a review


Results• Many owners don’t use the dashboard to reply to inquiries because the initial reply is all they

can do. There’s not currently a process for managing the back and forth communications with guests, plus they have gotten used to/developed their own processes for tracking emails, and this works for them.

• In the context of this study, participants did what we wanted them to do, which was to click on Reply in the Gmail message to go to the dashboard and then continue from there. But in reality, when the new process is released, many owners might be confused and unhappy because 1) this will force them to rethink their entire guest communications processes, which they may not perceive as a problem (unless they understand the security issues), and 2) it might come across as heavy-handed and controlling on HomeAway’s part. “It’s giving control of my contacts to HomeAway. If I leave HomeAway, I will lose my contacts.”

• Another issue that was expressed by a couple of participants is the number and frequency of changes HomeAway makes to the website. While it’s recognized that these changes might create efficiencies for some users, older and single-property owners often don’t perceive them. “A lot of people who deal with HomeAway are older and the constant changes bother them.” “This might be good for big customers, big rental companies but it’s not user-friendly for mom and pop homeowners.”


HA-formatted Email• Participants recognized this as an inquiry displayed in

their personal email but formatted by HA.

• Missing information includes:

– Thumbnail and/or property name, especially for multiple property owners

– Traveler’s email address and phone number (a PM said she recognizes these more than the person’s name, and this can determine how quickly she follows up)

– Number of adults and children in the party

– Property availability

– Also useful information would be the number of nights requested and whether these are weekday or weekend nights

• Will HA remove the reply commands supplied by Gmail, Outlook, and other email applications, forcing users to reply through the dashboard? Users’ reactions and possible confusion should be further studied prior to release.


Guest Communication• Participants understood that they were on a reply

page in the dashboard and when asked to explore the page, they found the quick links and message templates, and were generally able to describe the purpose of the page and all of the options. (The paper clip and “More” drop-down were a bit confusing because they couldn’t click to explore).

• The quick links need tooltips.

• Rates and saved documents pertain to the property, and the calendar and notes pertain to the guest. The quick tools, therefore, might be better divided into those that are for the property (rates, documents) and those that are for the guest (add to calendar, notes).

• The paperclip, as the sole icon in the list of message options, seems out of place.


Quick Tools• When using one quick tool, the others appear gray

and thus unselectable. This was confusing.

• The quick tools push the reply down and below the page fold. If users are looking at their calendar or rates, they might like to do this while they are composing their message and/or quote. You might therefore consider putting these in a moveable, non-modal dialog or light box instead.


Calendar• Participants expected the calendar quick tool to show

their property calendar, not just be an option to add this guest to the calendar by date.


Notes• There were no issues understanding that these are

optional notes pertaining to the guest.

• However, as mentioned previously, they might be better located next to the guest’s information rather than the property information.


Rates• All participants really liked the addition of their rates

information, often saying that this is one of their biggest issue when sending quotes using the current dashboard.

• One participant wasn’t sure if a dollar sign is the best icon. To her it meant “let’s collect money,’ i.e., book.


Documents• Participants also liked this area for managing

documents they might want to attach to a message.

• Is this where they would also manage their rental agreements? Since several participants mentioned needing to customize their rental agreements with the guest’s name and other information, an extension to the functionality shown might be to allow users to select a document, edit and save it with a new filename in Word, and upload the customized document for attaching to a message or quote.

• Hopefully this is isn’t the only place to manage (upload and delete) documents.


Saved Messages• Participants asked for the ability to create and save

many more templates than “reply available” and “reply unavailable.” (Perhaps this is what “Edit messages” is???)

• One participant (an owner in France) said “use saved message” isn’t clear and its presentation is a bit subtle.


Quote• There were no issues adding a quote, including

checking the property rates and editing the quote. However, it would be advisable to conduct another study where participants have to actually interact with the pages and enter a calculated rental amount.


HA Inbox• When asked to go to their HomeAway inbox, all

participants readily found the link in the blue bar/ breadcrumb. At this point, they began to comprehend that this process offers more than the initial reply to an inquiry, that it manages all of their correspondence. Most participants seemed fine with it but a couple said they don’t need or want another email system, or that their current processes work just fine. (Even the ones who were fine with it might not realize that they will be forced to use it instead of their current ways of managing guest communications, i.e., in email).

• 5 of the 6 participants don’t use online reservations so the “accepted” status and countdown timers didn’t mean anything to them, and actually caused some confusion about which 3 messages are referred to in the left side navigation.

• The difference in background colors between messages that need to be attended to and those that don’t is very subtle and wasn’t noticed by most participants.

• There was confusion about the meaning of the green checkmark and red X next to the dates of 2 messages.


Reservations• When they came to this page, participants weren’t

sure whether the 2 next to Reservations in the left navigation refers to the top 2 requested reservations or the bottom 2 that appear to be reserved. In other words, even though we had talked on the previous page about the 3 inquiries with white backgrounds that still needed to be dealt with, participants didn’t connect the number in the left navigation with the messages they need to do something about (although, to be fair, this could have been due to confusion with the countdown timers and unknown statuses; for example, on this page, they didn’t understand what “blocked” meant.)

• One participant was very insistent that the Reservations page should only show reservations, not reservations pending acceptance, and that it’s confusing to have pending reservations in 2 places.


HA Inbox• Going back to the inbox, most, but not all,

participants recognized that the top inquiry had now replied saying he was interested in the next steps leading to a reservation.

• However, they didn’t know whether the “replied” status referred to their initial reply or that Daniel had now replied back. (It was also unclear whether “accepted” meant they had accepted the guest or the guest had accepted their quote, but this could have been due to confusion about online reservations.)


Guest Communication• Participants continued with the task flow, which was

to book Daniel, but it’s not clear that they actually saw the message from Daniel. Perhaps clicking on the abbreviated message in the inbox should feature the full message on this page, with options to reply, edit quote, and book.

• Owners might not always be “replying,” so adding a “create” or “new message” option might also be warranted, and using simply “send” on the button.

• While the green “book” button stands out, it competes (and creates “cognitive dissonance”) with the orange “send reply” button. Each page should have a single main call to action, with additional options/actions available.


Book• The booking process seemed to work for participants.


Book• This time, participants were confused about whether

to click “book” or “send reply.”


Payment Received• Similar to when they linked from a message in their

HA inbox, it’s not clear whether participants actually saw the message beyond what they saw in Gmail. Again, it might be better to feature the full message and let users choose to reply, edit the reservation, refund, etc.


Guest Communication• The reminder to ask for a review was noticed and

appreciated, although participants had difficulty finding the template.

• One participant suggested putting the link to the template right in the reminder.


RecommendationsHA-formatted Email

1. Include information (or a link to information) about why HA is changing the guest communications process and the benefits of the new process to homeowners. Link to additional FAQs that answer likely questions from users.

2. Include the property name and picture/thumbnail, all information from the inquiry form (traveler’s email address and phone number, number of adults and children), and property availability.

3. Introduce the new process gradually, i.e., initially don’t remove their email reply commands but rather let them read about the new process, explore the UI, and realize its benefits.

Guest Communications Page

4. When users link to this page from their personal email or their HA inbox, show the traveler’s message prominently, at the top, and give options to reply, edit quote, book, etc.

5. Include a “create” or “new message” command for users who aren’t “replying.”

6. Only have 1 main call to action per page, i.e., no green “book” and orange “send reply” button at the same time.

7. Use the word “attach” instead of a paperclip icon in the group of message options.


RecommendationsQuick Links

8. Add tooltips to the quick link icons.

9. Give users access to their property calendar and use the calendar icon for this.

10. Separate quick links into those that pertain to the property (calendar, rates, and documents) and those that pertain to the guest (add to calendar and notes) and locate them with the property vs. guest information.

11. When viewing one quick link, don’t gray out the others. Instead use another method to indicate which quick link is active.

12. Try putting quick link information in floating (moveable and non-modal) dialog or light boxes so they don’t push the owner/guest communications down and below the fold, and allow the user to view them while working on their message or quote.

13. Extend the saved documents area to allow users to select documents, and edit and save them.

Saved Messages

14. Allow users to create and save as many messages as they want. Note, however, that this might make the drop-down impractical.


RecommendationsHA Inbox

15. Use a more obvious method to indicate which messages are new and/or need to be dealt with than a very subtle background change.

16. Somehow indicate that the status column is the owner’s status; in other words, “replied” doesn’t indicate that the traveler replied. (One wonders if “replied” is even a meaningful status; perhaps it should instead be “pending” or “working.”)

hasc usability test results conducted 1/7-8, 2013 page 1 hasc usability test results conducted by...

Documents

hasc usability test

test format

participants remote

usability testing slide

ha guest communications

new communications workflow

new workflow slide

review slide