california dmv heuristic evaluation, 52 pages
DESCRIPTION
The following is an individual project to research, evaluate, and present redesign recommendations for the California DMV Website. Data from real California DMV Website users was collected to represent common challenges faced when navigating through the website and finding essential DMV information.TRANSCRIPT
Heuristic Evaluation:
California DMV Website
Ryan Schabel
11/26/2013
1
Table of Contents 1
Introduction 2
Executive Summary 3
Inquiry Methods 5
- Survey 5
- Interview 7
Global Issues 9
- Error Prevention 9
- Main Menu and Submenus 9
- Navigation & Mapping 12
- Organization and Difficulty Finding the Right Questions 16
Local Issues 18
- Inefficient Use of the Home Page Real Estate 18
- Unnoticed “Important” Alerts 20
- Appointment Feature Needs More Visibility 21
- Making an Appointment 22
- Three Unnoticed Submenus 37
- The “How Do I…” Menu 40
- Finding a DMV Office Location 43
Conclusion 50
Appendix 52
Contact Info 53
2
Introduction
To establish the degree of a website’s usability, a heuristic evaluation should be conducted. A
heuristic evaluation generally requires research of the website, inquiry methods, user surveys,
and feedback reports, and the application of human factors concepts to the design. Thereafter,
suggestions can be put forth to be implemented into the redesign.
The stigma that is widely accepted by the general public is that a trip to the DMV is always
unpleasant. Improving the DMV website can help to make the experience of the DMV as a
whole more enjoyable. It is also important to keep in mind that many users of the California
DMV website do not visit the website frequently or are visiting the site for the first time.
Therefore, many users will not be aware of the location of the information they are searching
for until they become more familiar over multiple uses. With this in mind, it is important to
present information in a way which makes the user feel confident that their first attempts and
decisions are correct for navigating and using the websites features.
3
Executive Summary
DMV.CA.gov is a government website dedicated to providing information regarding the
Department of Motor Vehicles and laws and procedures involving road vehicles. All necessary
documents pertaining to drivers and vehicles should be easily accessible from any location of
the website. It is generally important to assume that the users of the DMV website will have a
very wide age range and language abilities between individuals. Anyone from the age of 16
years and older is eligible to obtain a driver’s license and English language ability is not required
to obtain a driver’s license or to own and operate a motor vehicle.
The heuristic evaluation process began by getting acquainted with the California DMV website
and attempting to explore the site as an average user would. Some of the features of the
website which were observed first were the use of menus, real estate, navigation between
different areas of the website, and finding features relevant to the goals a DMV would normally
allow. Some of these goals include finding information about vehicle registration, drivers
licensing, and finding a DMV office.
To establish a starting point for the evaluation, a survey was distributed to users which included
questions relating to the usability of the California DMV website. Interviews were also
conducted to investigate the usability from the perspective of the users. The survey was
presented after the participants were given four tasks to complete. Based on the tasks, eight
qualitative, multiple choice questions were given. Next, interviews were conducted with users
who had used the website on their own. The questions were open ended about the satisfaction
of the site, and what they thought could be improved about certain aspects of the site.
4
After becoming somewhat more familiar with the website, a comparison between similar
websites like the Illinois DMV (http://www.dmv.org/il-illinois/) and the Washington DMV
(http://www.dol.wa.gov/) were inspected. The major differences between websites, such as
real estate use, menu titles, and overall site interface brought features forward that may
contribute to a better user experience. Both the Illinois and Washington DMV websites had
usability similarities that differed from the California DMV, such as larger links and menu items,
the dedication of different levels of real estate for certain features, better use of organization
and grouping, and navigational tools to accommodate better navigation between pages.
Finally, after all inquiry methods were completed, the California DMV website was evaluated
based on human factors usability principles and heuristics. Some of the issues presented are
local and present in specific pages of the website while others are global and present across
many pages throughout the website.
5
Inquiry Methods
Survey
To gather information and perspective on user opinions of the California DMV website, a survey
was given to five participants. Before beginning the survey questions, four tasks were provided
for the participants to carry out. These four tasks included finding the closest DMV office to a
central location, finding information about registering a vehicle, finding information on how to
obtain a driver’s license, and finding information on how to transfer a car title. The questions
were related to task difficulty, frustrations, and general opinions of their experience.
The following is a brief summary of the survey results:
- 2 out of 5 users felt that the website was somewhat difficult to use
- 2 of the 5 participants reported not fully completing the tasks given
- 2 out of 5 participants responded with the answer stating it took more time than
expected
- 2 out of 5 participants found that the information was somewhat unclear.
- 2 out of 5 participants felt lost while navigating the website.
- 0 out of 5 participants found any difficulty returning to a page they had previously
found.
- How to obtain a driver’s license as a new driver, finding the closest DMV, or finding
information on how to transfer a car title was reported to be a very difficult task to
complete by at least one person.
6
- Finding California registration information was reported to be somewhat difficult.
The results of the survey indicate several usability issues which will be covered by the
heuristic evaluation. The full list of survey questions can be found in the appendix, Figure 1.
7
User Interview
An interview with 6 real-world users of the dmv.ca.org website were conducted to obtain more
expansive answers, compared to the survey feedback. All of the participants of the interviews
were between ages 21 and 24. Two of the six users recently moved to California in the past
three months. The following are important highlights from the responses of the interviews:
- “At first glance, I wasn’t able to find links for what I wanted”
- “It was hard to find where the closest DMV was. There didn’t seem to be an easily
accessible place to search for DMVs near my address”
- When you hover over the categories at the top of the page and the dropdown menu
comes down, it’s hard to keep your curser in the drop down box because it’s so skinny”
- I think the site is pretty s***** in some ways. The different types of registration (e.g.
new driver, from out of state, renewal) are very confusing. As someone coming in from
out of state I feel like the information needed to get a new license and re-register my
car is split into at least 3 different places. They have an "out of state" section but it isn't
really helpful.
- Make the most useful and popular information (how to get a license, nearest DMVs)
more prominent on the home page.
User feedback is very valuable for finding issues within an interface. Although users do not
always know the best solutions for certain problems, it is important to take all feedback into
consideration. Unfortunately, due to the time constraints and timing within the evaluation
process, both the interview and the survey do not cover all issues which will be mentioned in
8
the evaluation report. Ultimately, not all issues can be identified through user feedback and can
instead be brought forward through human factors principles.
9
Global Issues
A global issue is an issue which is present across several locations or aspects of a website and
violates a human factors principle or heuristic.
Main Menu and Submenus
Error Prevention – Eliminate error-prone conditions or check for them and present users
with a confirmation option before they commit to the action.
Above is an image of the home screen of the California DMV website with the cursor hovering
over the “Home” tab. When the tabs on this menu are scrolled over, a submenu appears
underneath all of the tabs horizontally.
10
If the cursor moves outside of the submenu, the submenu disappears and the cursor must
return to the parent “home” tab to restart the process. This aspect violates the Error
Prevention heuristic by forcing the user to be very accurate in their mouse movement.
For example, If someone wants to choose “Seniors” under the “Home” tab, if the cursor slips
out of the grey area, they will need to return to “Home” and hover past “Featured Links,”
“News Room,” “Industry,” and “Teens” in the very thin space given. This task would specifically
be difficult for a senior due to the required fine motor skills required to hover in a small
designated area of the screen.
Similarly, if a different tab is hovered over, the submenu for that tab will immediately replace
the last submenu, as shown by the Offices tab:
11
This issue was also mentioned specifically by a user in an interview: “When you hover over the
categories at the top of the page and the dropdown menu comes down, it’s hard to keep your
cursor in the drop down box because it’s so skinny.” It is clear that this menu design causes
difficulty for some users.
Redesign Recommendation: Resembling the Washington DMV website interface, listing the
submenu items vertically for each tab, instead of horizontally, would give the users a larger
area to move their cursors compared to the California DMV website, as shown:
12
Navigation & Mapping – Give the user a reference point to where they are in the
interface and have consistency between pages to allow for easy navigation.
As reported by the user survey results, 2 out of 5 participants reported difficulties while
navigating the California DMV website. A likely contributor to this issue is the absence of a
navigational tool to show the location of pages within the interface. For example, if a user clicks
on “Vehicle Registration” from the home page:
13
Then clicks “Changing Vehicle Ownership”
And finally clicks on “Requirements for Buying a Vehicle”
They will find themselves at the following page, titled “Buying or Selling a Vehicle Changing
Vehicle Ownership”:
14
The navigation issue surfaces when the users want to navigate to a page previously visited in
the interface. To quickly return to a previous page, the users must either find the page they
have in mind by using the back button or backspace, or they will need to return to the home
page and restart their navigation, both of which rely on short term memory. Additionally, every
new page visited feels like a standalone page, separated from the California DMV website as a
whole.
15
Redesign Recommendation: The Illinois DMV website has implemented a navigational tool for the
website which allows the user to move to one of the parent pages of each site they visit, like so:
In this screenshot, the user is in the “Title Transfers” page, but they can easily move back to the
“Registration & Titling,” “Illinois,” or “Home” page. Without this tool, the user is likely to feel lost
and have to repeat the navigational process if they want to find a page they are already aware of.
Implementing a navigational tool like the one used by the Illinois DMV website would require the
pages of the current California DMV website to be reorganized in a way which includes more
parent and child pages. The inclusion of a navigational tool would give the user a sense of location
within the website while simply providing a useful feature.
16
Organization and Difficulty Finding the Right Questions
Grouping – The grouping principle is used in interface design to position items that are similar
to each other in close proximity.
Throughout the California DMV website, there are pages with links which are intended to help
users find the answers to their questions. However, many of these links are in the form of a
long question, such as, “Where can I locate information regarding the Vehicle License Fee
Increase?” or “What are personalized and special interest license plates?” Both of these
questions may be useful if answered, but due to the poor grouping of these questions, finding a
relevant question becomes a difficult task. Here is one section of the FAQ page:
17
Although all the questions may be relevant to the section title of “Vehicle Registration and Title
Questions,” the links could be grouped into additional subgroups with titles based on the
subcategory they all fit into. For example, there are many questions regarding registration fees,
vehicle license fees, income taxes, and other monetary questions. However, all of the questions
that would fit into this monetary category are disorganized and separated from each other
throughout the list.
Redesign Recommendation: To follow the grouping heuristic, group questions with similar
subjects together while also providing more subheadings to make finding specific questions
easier for the user.
18
Local Issues
A local issue is an issue that is centralized at a location, usually only at one page of a website.
These issues can be the interface of a webpage or a tool or feature within a webpage.
Inefficient Use of the Home Page Real Estate
Real Estate refers to the actual visual space being utilized on the screen by the interface and its
separate features. Because there is a limited amount of space which can be viewed at one time
on a computer screen, the space on the screen must be consolidated appropriately to help
users achieve their goals as quickly as possible. On the home page, there is a large banner
which uses about half of the real estate of the screen.
19
This is very inefficient for providing relevant information to the users. Furthermore, even if the
information on the banner is relevant, the users are probably not going to stall and wait until
they see an image that seems similar enough to their goals.
Redesign Recommendation: An example of efficient real estate use and a better banner
design can be shown with the Illinois DMV homepage:
Pictured above is the first view a user will see of the Illinois DMV website. All of the
information displayed on the homepage is relevant to common user needs, such as the large
search bar, the main menu, and the side bar containing frequently used items. And although
there is still a banner with cycling images, the banner takes up less than half of the home
page real estate. The banner also provides titles for each cycling image, which can be
viewed at the user’s will.
20
Unnoticed “Important” Alerts
Efficient use of real estate is violated again by the “Important California DMV Alerts” section.
This section seems to be a critical part of the home page, but it will only be seen by those who
scroll to the second half of the page:
If these alerts are important, then why isn’t this information shown on a more visible part of
the real estate where the users can see it immediately?
Redesign Recommendation: Put the most important information on the real estate of the
screen which receives the most attention. Also, use a color which stands out from the
background to make the alert more salient, such as red instead of grey.
21
Appointment Feature Needs More Visibility
Because the DMV has an established reputation to be a less-than-enjoyable experience, giving
the appointment-making feature a more prominent location may improve the user experience.
Currently, the link to make an appointment is under “Online Services” outlined in red:
This feature isn’t only difficult to come across because of its location and size, but it is also
located in a submenu of a three part menu system, which is another issue on its own.
Redesign Recommendation: To improve the visibility of the appointment making feature,
there should be more real estate dedicated only to this feature.
22
Making an Appointment
Since many people are going to expect their visit to the DMV to be unpleasant, making any
adjustments to make their visit go smoothly would be beneficial. One of the reasons a visit to
the DMV can be unpleasant, but can be improved upon, is the long waiting time in line.
Therefore, having more people set up appointments for their visit would improve the
experience for all users and possibly the employees, as well. Here are the current steps to
making an appointment:
Begin at the Homepage:
23
Scroll to the middle of the page and click on “Make an Appointment before going to a DMV
Field Office”:
Choose the appointment type (Behind-the-wheel driving test, Office visit, View/Cancel
appointment):
24
Choose or find the DMV office you would like to visit:
If someone is planning on visiting a DMV office for the first time, just being given the town
name, such as San Jose, is not specific enough to know if it is the closest DMV to the user’s
address. In fact, the user probably has no idea where the office is located yet. Therefore, the
users are likely to use the “Map CA DMV Offices” link.
25
When the “Map CA DMV Offices” button is clicked, a new window appears with a map of
northern and southern California:
The town names in red are clickable and used to further narrow down the area of the map.
26
In this case, San Jose was selected and the window was refreshed to the following map:
In this map, only the red towns are clickable while the towns in black are not. However, the
map only shows one DMV office per town instead of all DMV offices within the map area.
27
Once a red town is clicked on, the window automatically closes and the user is left viewing the
appointment-setting screen with the selected town now chosen in the drop-down menu.
28
Although a DMV office has been chosen by the user, the address of the office has not been
provided, which violates the help and documentation heuristic as information should be easy to
search for and should focus on the user’s goals. The user is expected to continue setting the
appointment without knowing the location of the office.
Redesign Recommendation: Following the help and documentation heuristic, the map function
to find a DMV office should use a format similar to the “Find Your Local DMV Field Office” map.
This map provides all DMV locations on a map that can be moved by clicking and dragging. By
using this map, the user will immediately know where the office is located by the map location
and the address provided.
29
Once the location has been selected, the purpose for the appointment must be chosen and the
user must fill out his or her name and phone number.
Redesign Recommendation: The dropdown menu should have the address displayed next to
the town name. There is available real estate to display the full address next to the
dropdown menu once a location has been selected.
30
Then, “submit” at the bottom of the page must be clicked to continue the process.
Redesign Recommendations: A symbol, such as an asterisk, should be used to denote the
mandatory information needed so the form is properly completed on the first attempt. This would
follow the Error Prevention heuristic, preventing problems or errors from occuring in the first place.
31
After clicking “submit” the user will be brought to the next page of the appointment-setting
process.
This page displays relevant information, such as the unavailability of written exams after
4:30pm as well as a reminder stating what this appointment is not being made for. In this case,
the appointment is not for a behind-the-wheel test, but an appointment can be made for one
using the available link. The page also now tells the user the address of the DMV they are
Additionally, the “Submit” button violates the Consistency and Standards heuristic which
states users should not have to wonder whether different words mean the same thing.
Clicking on “submit” does not submit an appointment. Instead, the user is taken to the next
page and therefore, a word like “continue” would be a more appropriate word choice.
32
setting an appointment for and the first available appointment is automatically set as the
default appointment time.
Although this may inform the users of the soonest available time for an appointment, chances
are that they will want to choose specific times and days which are compatible with their
schedules. To pick a date, there is a calendar that can be used to choose a specific day as well as
a specific time.
33
Redesign Recommendation: Although the soonest available time is displayed, there is an
absense of cross-pollination, which is the maintaining of information across aspects of the
same website. To support the cross-pollination principle, the calendar that appears should
not allow the user to click on any dates before the soonest possible appointment time, as it
already has been stated by the website.
34
Once date and time are chosen, the user must check for availability. If the selected time is
available, a message in red states “an appointment for the date and/or time selected is
available” and the user may click on “Schedule This Appointment.”
Redesign Recommendation: To better follow an approprite use of affordances, which are
qualities of objects which suggest certain meanings, choosing an available appointment time
should be shown in green. It would not be surprising to find that many users assumed the time
they have selected is not available due to seeing the color of the text before reading the actual
message.
35
A more efficient way of presenting available dates and times for appointments would be to have
a calendar showing all available times within any given day, similar to a spreadsheet or Google
Calendar to visually aid the user and provide more useful information.
36
Another notable issue within the appointment-setting process is the session time-out. If the
user takes too much time on the appointment page, an error screen will appear stating “Session
timed out, your session has timed out due to inactivity.”
Redesign Recommendation: To help the user return to making an appointment, a link
should be provided to redirect them to a usable page.
37
Three Unnoticed Submenus
While becoming familiar with the website, there were three submenus which went unnoticed
as three separate information sources. It wasn’t until a few weeks into the process that the
three different menus were discovered. These menus include Online Services:
Media Center:
38
And Other Services:
If it took one person several weeks with many visits to realize this menu was here, then it will
likely never be used by many users as they will visit the site only a few times within a short
period. Additionally, displaying so much information in so little space creates real estate which
is too dense with text. Users are very unlikely to focus so much attention onto one small part of
the screen, especially when the users are very goal-driven and the density does not seem to
support their goals.
39
Redesign Recommendation: If the developers of the website feel that all of these features
are necessary to be shown on the first page, the grouping should be improved
dramatically. Grouping organizes objects with similar features into close proximity to one
another. For example, Important Info for Boat Owners, Tell Us How We’re Doing, and
Pedestrian Safety Info, and Office Closures have very little to do with each other and only
one of those links seems to have the importance of an “alert.”
The Latest Press Releases, Major Events Affecting Drivers, DMV Newsroom, and Subscribe
to Email Alerts all have the commonality of new and relevant information for the public.
Instead, all four of the links just mentioned are separated between three different
submenus. However, since most of these links can be found in the main menu, removing
them from this area of the site would decrease the redundancy in design as well as free
more real estate.
40
The “How Do I…” Menu
On the Home page of the California DMV website, there is a “How Do I…” scrolling menu which
is basically a Frequently Asked Questions section, as shown:
Similarly to the three submenus above it, grouping issues are present within this menu. These
items have been listed alphabetically, which makes finding the right question. As an example, if
a user is looking for an item about Teen Drivers, the user would have to begin reading each
item completely before determining whether or not it is a match. Finally, when the user gets to
the 8th item of the “Get information” section, they will find the “get information about teen
drivers” item.
Additionally, this menu also includes a redundancy component since there is also a “Frequently
Asked Questions” Section within the “Other Services” submenu. However, as it has already
been covered, the users are not likely to view the “Other Services” menu to find the link to the
FAQ page because of the density and lack of organization or grouping. Ironically, the FAQ
41
section incorporates some grouping principles, which could have been incorporated into the
“How Do I…” menu, like so:
42
Redesign Recommendation: As the FAQ page has already provided, the “How Do I…”
Section should also incorporate the same organization within the scroll menu, instead of
having the choices listed alphabetically. Furthermore, to provide more flexibility within the
website, while also reducing the redundancy, a link to the full FAQ page could be provided
next to the “How Do I…” menu title. This reduces the redundancy by using no additional
real estate for the same goal, while still allowing the user to choose which view to be
displayed.
43
Finding a DMV Office Location
The following is the process to find a DMV Office Location:
First, the user starts at the Home page, hovers over “Offices” and clicks on “Find Your Local
Office”:
44
Then, the user clicks DMV Field Offices Map, under the Interrupted Services and Closure
Information:
Redesign Recommendation: The salience of the office closures should be improved by using
red instead of blue for the outline. Also, the chat bubble has been changed to an
exclamation mark, which signifies urgency.
45
After clicking on the link for DMV Field Office Map, the user is brought to a page displaying a
map of California. The site is capable of using the internet connection to find the user’s
location. The map is based on Google Maps and displays all California DMV office locations:
The bullet points of information should be shortened to compensate for faster skimming. For
example, instead of, “All DMV Offices will be closed Thursday, November 28, 2013 and the
following day in observance of Thanksgiving Day,” a simpler version such as, “All DMV Offices
will be closed Thursday, November 28 and Friday, November 29 for the Thanksgiving
holiday.” This way, both Thursday and Friday listed, as some users may not notice “the
following day” is also closed while quickly skimming. Also, by changing “in observance of
Thanksgiving Day” to “for the Thanksgiving holiday,” the former suggests a single day while
the latter does not imply only one day and also uses ordinary language, following the
common sense principles.
46
Redesign Recommendation: This is not a critical issue, but it does violate the redundancy
and real estate issues. At the top of the page, “DMV Field Offices” is displayed at the top of
the page while “DMV Office Locations” is displayed again at the top of the map while also
making part of the map unusable, as shown:
Removing the top header on the map would resolve both of these issues.
47
The user can click on one of the locations to view more information about each office. This
information includes the address, wait times with and without appointments, and open hours
of the location, like so:
However, there is no way of making an appointment at an office without using the “make an
appointment” feature, which is completely separated from this page of the website.
48
Redesign Recommendation: The majority of users are going to be using the map feature to
plan a trip to a DMV office. Since wait times for appointments and non-appointments are
already listed in the details of the location, making an appointment should be accessible
from this page. A “make an appointment now” link under the wait times could be created
to follow visibility and grouping principles.
Another helpful feature would include a “Now open” or “closed” icon, which supports
giving the user more useful and relevant information to their goals, as supported by the
help and documentation heuristic.
49
If the “details” tab is clicked, closure alerts, services, and additional services are displayed:
Unfortunately, many users will not choose to click on the “details” tab after getting the
information they were searching for and the alert of the office closure will remain unseen.
Although this alert will likely be on the “Find Your Local Office” page, if the user is quickly
searching for the office map, it is reasonable to assume they may not stop to read the office
closures, especially with the current design.
Although finding a DMV office through the DMV Office Map is fairly straightforward,
redesigning the functionality with these issues in mind will have a significant positive impact on
the overall usability of the features and usability of the task in its entirety.
Redesign Recommendation: To improve the visibility of the office closure alert, the alert
should be displayed at the top of the first tab opened when any office has a planned closure,
change in open hours, or any other status change which could affect the user.
50
Conclusion
The objective of a successful website redesign is to make the experience of the website easy
and intuitive for the users. It is important to assume that differences between people will vary
greatly on dimensions of age, language ability, and experience of using technology in general. It
is widely known that a trip to the DMV is generally unpleasant and finding ways to make the
users of the DMV as enjoyable and stress free as possible. The users of the DMV are not likely
to be strongly familiar with the layout or the included information. Therefore, the website
should be designed with a user centered perspective and the interface should present itself in a
way that can be quickly learned within the first moments of being used.
Although the current design is somewhat aesthetically pleasing, there is much to be improved
upon. The first recommendation would be to rebuild the entire architecture of the website to
integrate better navigation and mapping. Secondly, the real estate of the main menu should be
reorganized to allow users to make more slight errors without consequences. Grouping is also a
top priority for the redesign. Throughout the entire site there are examples of lists which users
must sift through on their own to find any relevant information to help them reach their goals.
Finally, redundancy and clutter seems to be a common theme on the homepage. The main
menu is somewhat usable, but the other side menus are not useful due to their text density and
lack of grouping. The banner also needs to give up some of its real estate for more practical
purposes.
Having a usable appointment system is an excellent feature for a DMV, but the system could
use some improvement. Moreover, the availability of an appointment system should be made
51
much more clear as it is somewhat hidden in the website as it stands. Some of the messages
about appointments as well as office closures may go unnoticed due to low salience or
misinterpretation caused by inappropriate color schemes or location. The map function within
the appointment system was difficult for several users, although the separate map function for
finding an office location was much better. Therefore, if the separate map could replace the
current map within the appointment system, the process could be made much easier.
Although not a crucial issue, the information presented throughout the site could be reworded
to use closer to everyday language. Because about 90 percent of Californians either can speak
English or Spanish, the languages available on the website do not present a critical issue.
However, making the site easier to find the different language DMV handbooks or providing
more languages might benefit a small percent of the population.
52
Appendix
Figure 1