executive summary - dceryesmist.weebly.com file · web viewcalifornia state university. monterey...

Running Head: YACHT CLUB SYSTEM TRAINING 1

CALIFORNIA STATE UNIVERSITY

MONTEREY BAY

Yacht Club System Training

CAPSTONE REPORT

Submitted in partial satisfaction of requirements of the degree of

MASTER OF SCIENCE in

Instructional Science and Technology

Deborah Ceryes

December 11, 2018

Capstone Approvals: (At least one advisor and capstone instructor should approve)

___________________________ _____________

Dr. Sarah Tourtellotte Signature Date

___________________________ _____________

Dr. Miguel Lara Signature Date

YACHT CLUB SYSTEM TRAINING 2

Yacht Club System Training

Deborah Ceryes

California State University, Monterey Bay

December 11, 2018

IST 699 Capstone

Professor Miguel Lara, PhD


Table of Contents

Executive Summary.............................................................................................................5

Yacht Club System Training...............................................................................................6

Problem Description............................................................................................................6

Target Audience...............................................................................................................7

Learner Analysis..............................................................................................................7

Context Analysis..............................................................................................................9

Current State..................................................................................................................10

Environmental Scan.......................................................................................................10

Industry Trends..............................................................................................................10

Solution Description..........................................................................................................11

Objectives......................................................................................................................11

Learning Solution..........................................................................................................12

Task Analysis.................................................................................................................12

Media and Delivery System Decisions..........................................................................13

Learning Theories..........................................................................................................13

Instructional Principles and Strategies...........................................................................15

Multimedia Principles and Strategies............................................................................16

Challenges and Breakthroughs......................................................................................17

Methods/Procedures..........................................................................................................18

Major Deliverables........................................................................................................18

Design Analysis.............................................................................................................18

Agenda.......................................................................................................................19

Activity Descriptions.................................................................................................19

Implementation Plan......................................................................................................20


Resources...........................................................................................................................20

Logistics and Budget.....................................................................................................21

Technical Skills and External Resources.......................................................................21

Timeline/Progress Report..................................................................................................21

Milestones Checklist......................................................................................................21

Evaluation Plans................................................................................................................22

Formative Evaluation.....................................................................................................23

Summative Evaluation...................................................................................................24

Level I —Reaction.....................................................................................................24

Level II—Learning....................................................................................................24

Level III—Behavior...................................................................................................24

Level IV—Results.....................................................................................................25

Conclusion.........................................................................................................................25

References..........................................................................................................................27

Appendices........................................................................................................................28

Appendix A: VYC Membership and ACH Reports......................................................28

Appendix B: Single Subject Testing (SST) Instrument................................................28

Appendix C: Level I – Reaction on Learning Objectives.............................................28

Appendix D: Level II – Learning Analysis using Quiz Results...................................28


Executive Summary The Vallejo Yacht Club (VYC), located in the San Francisco Bay Area, has operated for

over 100 years. (Evans, 2000) Governed by an elected Board of Directors (BOD), a small office

staff manage day to day operations and a host of volunteers oversee club events and programs.

To streamline and improve club operations, the BOD adopted an integrated financial system

from Club Essential, LLC, (CE) a third-party vendor that specializes in club management

systems. Due to the system implementation, customized training modules are needed for: 1)

member online payments, 2) point-of-sale (POS) transactions, and 3) back-office procedures

using the Club Management Application (CMA). Training modules provide opportunities for

members and staff to learn how to use the new system and maximize its investment value.

The learning solution includes a website developed for each subject area. Websites are

embedded with screencasts, software simulations, JavaScript/jQuery interactions, and assessment

tools. Screencasts and simulations include feedback as well as practice options to support

learner engagement and motivation. Self-paced navigation within and across pages embodies

autonomous and self-directed learning. Design components follow a watch/try/assess format and

address the adult learner’s need for active participation. Immediate application to gain

experience with new skills is provided through practice and testing tools. Learners preview

content prior to training on an actual POS device or office workstation. Quiz and survey tools

allow learners to demonstrate and reflect on their learning. Job aids and resources can also be

utilized for synchronous training.

Several forms of evaluation occur to assess usability, learning objectives and knowledge

transfer using formative and summative evaluation tools. Overall feedback on design elements

inform improvements to multimedia as well as the learner experience. Quizzes build confidence

to lift learner readiness. Self-assessments monitor learner readiness to apply new skills or

knowledge. These evaluation sources provide an environment to maximize volitional

competency and commit to changes in behavior. Since the end of formal evaluation period, the

POS modules have been requested for new volunteer onboarding. In addition, membership

onboarding will include the online payment portal site to support. Continued use of the websites

provides additional data to measure learning effectiveness and incorporate updates as needed.


Yacht Club System Training The Vallejo Yacht Club (VYC) established in 1900, sits on the east side of the Napa

River, facing Mare Island to the west. Within its harbor walls, the VYC membership enjoys a

history filled clubhouse, outfitted with a bar, galley and ship’s store as well as a floating harbor,

with berthing for about 195 boats. For the past decade, club operations involve management of a

million-dollar budget resulting in consistent profits to support clubhouse and harbor operations.

Sustaining efficiencies in VYC operations and financial reporting ensures the elected BOD and

flag officers execute their fiduciary responsibility as entrusted by the club membership.

In 2017, the BOD approved implementation of an integrated financial system,

encompassing POS, online payment portal and back-office functions (e.g. accounts

receivables/payables, bank reconciliation, etc.). Several organizational goals influenced the

BOD’s decision to implement a new system, such as: 1) reduce credit card fees totaling over

$17K annually; 2) streamline financial workflows from POS to the back-office; 3) implement an

inventory control process, and 4) digitize paper-based workflows to improve efficiency. The CE

system includes features that address the goals prioritized by the BOD.

Problem Description VYC operations relies primarily on a group of member-elected officers/directors and one

full-time manager. Additional part-time paid staff and rotating volunteers assist with various

office and club functions. Financial transactions are generated across digital (e.g. email,

websites, local area network (LAN)) and non-digital (e.g. paper or verbal) platforms. The CE

system integrates financial transactions (e.g. POS, general ledger, member payments) created

across mobile and wired devices. Due to the system implementation, learning modules need to

sustain volunteer training as well as promote compliant use of the system.

Specific modules provide the necessary orientation and background needed to accurately

use the new integrated financial system. One core workflow includes an interface of POS

transactions into the back-office system. POS transactions occur at the bar, galley and ship’s

store when members and guests purchase beverages, food and retail items. The POS system

allows purchases to occur via cash, charge to member accounts (i.e. “chits”), and credit cards.

Volunteers staffing the POS areas need support to learn the new system and any subsequent

updates to the workflows, as needed.


Another core workflow is online payment of member monthly dues, berthing (if

applicable) and any services charged to the members’ account. The online payment portal

allows members to view their monthly statement and make payments using a credit card or

electronic debit from their bank account. Online payments may require creating an account with

third party vendors to process a payment via credit card or electronic debit. Member feedback

indicates this additional account setup can be confusing and may deter members from using the

online payment portal. New and existing members need support to efficiently navigate the

online payment portal, setup their online payment method and make payments using accurate

financial information (e.g. routing numbers and bank accounts, or credit cards).

The final core workflow encompasses the back-office functions, such as membership

maintenance, accounts receivable (A/R), accounts payable (A/P), bank reconciliation, and

general ledger reporting (e.g. income statements, balance sheets, etc.). These common

bookkeeping activities are fully integrated within the Club Management Application (CMA) of

the CE system. POS and member payments are displayed in CMA as these transactions occur

via the POS devices and the online payment portal. Although the CE system provides very

detailed procedure manuals, the VYC operates differently than larger CE clients. Office staff

and the BOD need support to learn the CE workflows customized for the VYC operational

workflows.

Target AudienceThe target audience within the VYC organizational structure includes: 1) new and

existing members, 2) member volunteers, and 3) back-office staff. The VYC membership roster

includes about 337 memberships with an approximately 500 individuals eligible to use club

services. (See Appendix A) A group of 20-25 volunteers serve as officers, directors and/or staff in

the POS areas. A full-time manager oversees club operations and additional part-time employees

support bookkeeping functions, galley services and other duties as assigned.

Learner AnalysisMember information is initially gathered via a VYC membership application, including

email address. Members’ email addresses serve as a critical component to receive monthly

statements, weekly updates about club events and ongoing communication with the club office

staff. Monthly membership meetings provide feedback from members on various club services


and operational workflows. Ad hoc feedback to VYC officers and office staff provide additional

insight on how members and volunteers utilize and experience the new system.

The vast majority of members use some type of digital device, either wired or mobile,

including email. Members obtain and apply technology skills to support their membership in the

VYC. Door entry codes, member ID numbers, passwords, emails and the new online payment

portal require members to manage several digital components to access the clubhouse as well as

monitor communication with and payments to the club.

Some VYC officers and volunteers are familiar with the devices, software applications,

and digital use expectations needed to support the new CE system. Other

officers/volunteers/staff are either new to the environment or their technical skills require

additional support to learn the new system. The current club manager, with over a decade of

working at the VYC, possesses a broad range of technical skills needed for the various office

systems used within the VYC environment. The current part-time employee has specific

technical skills with basic accounting procedures, web-based systems, relational databases,

productivity suites and local area networks.

All learners possess a range of technology skills and prior knowledge pursuant to their

professional or individual experiences. Some learners are very adept and comfortable using

technology across various digital platforms. Based on their beliefs about technology, other

learners adopt new technology very sparingly and as such, embrace innovative technology very

slowly. Addressing motivation and volition is key to support adoption of the new system within

some learner groups.

Prerequisites specific to each learner group are based on the individual’s roles and

responsibilities at the VYC:

Members : 1) an email address, 2) experience with internet browsers, and 3)

knowledge of personal financial information to create an online payment.

Volunteers in POS areas : 1) knowledge of the VYC services and events, 2)

experience with a touch screen, and 3) ability to reconcile a cash drawer.

Office staff : 1) knowledge of VYC services and bylaws, 2) experience with

financial systems, 3) an intermediate knowledge of common business software

applications, and 4) demonstrated ability to learn and adapt to new technology.


Context AnalysisThe project’s design derives from: 1) membership data, 2) rejected or erroneous online

payments, 3) POS variances and 4) management’s expectation to sustain and improve financial

workflows. Although several members utilize the payment portal successfully, some members

experienced repetitive payments, ACH rejections or insufficient funds. Members onboarding

monthly require guidance on how to make a payment online to prevent errors and avoid

unnecessary merchant service fees.

When visiting the club, members primarily use cash or chits to pay for services. Credit

cards are a smaller portion of financial transactions and are often used by guests that are visiting

the club (as they cannot charge services to their member accounts) or by members for infrequent,

larger purchases. Several members use online banking to pay their current monthly statement.

Other members pay their monthly balances using cash or checks.

The Ship’s store uses the POS system consistently, whereas Bar POS usage is

approximately 2 times per week. The Galley POS system is rarely used. After initial

synchronous training was provided, Galley POS usage declined due to volunteer turnover or lack

of motivation. Quarterly, volunteers or officers onboarding necessitates new training or updated

training for a specific club event.

Related to staff use of the CMA system, entry of “chits” can be delayed which impacts

accuracy of member statements. Missed or late posting of member online payments also erodes

the integrity of member statements. Timely posting of POS batches with accurate cash

accounting reduces variances in financial reporting.

These experiences and incidents impact a broad range of operational areas at the VYC.

POS variances and system downtimes increase when volunteers are not properly trained.

Member satisfaction is negatively impacted due to varying level of volunteer competency with

the POS systems when attending club events. Finally, the need for more financial reporting to

ensure appropriate decision support for management.

Learners are located at the VYC as well as at their residences and businesses. Learning

options occur synchronously or asynchronously. Considerations for synchronous training to

occur at the VYC include: a digital device (either mobile or wired) with remote connection to

the CE system, workflow procedures, note-taking tools and table/chairs, when needed.


Asynchronous training requires an internet connection to access the instructional content on a

website containing various documents, videos and surveys organized by role and responsibility.

Current StateMembership data indicates about 25 members do not have an email address and continue

to request updates via phone or in person from office staff about their accounts. Approximately

216 members with gross deposits of over $370,000 have completed a payment online using the

new online payment portal. Merchant service fees total $1610 with $430 resulting from return

transactions. (See Appendix A)

Manual POS entry occurs primarily with the Galley and Bar POS batches. The Ship’s

Store reconciles exactly to all settlement types and does not experience the same variances as

other areas in the club. Data entry for member charges (i.e. “chits”) requires about four to ten

hours of staff time per week. Two to three POS batches need reconciliation per week. Alternate

staff are not trained in these functions to mitigate delays in financial workflows.

Environmental ScanThe VYC business environment indicates opportunities and challenges in the short- and

long-term. Opportunities to increase operational efficiencies and reduce costs will help address

the harbor management challenges that exist in the current economic climate. Although revenue

and profits continue to sustain from membership and harbor income, a focus on operational

efficiencies is needed to ensure expenses remain flat or decrease when possible, while improving

services to generate new as well as retain existing income.

Industry Trends The recreational marine industry is experiencing several industry pressures ranging from

decreased interest in boat ownership and increased costs for marina management. (MRA, 2013)

In addition, boaters in the Bay Area are experiencing increased costs for berthing (M. Ceryes,

Personal Communication, November 2017) and travel to/from various locations with water

access. Despite these negative trends, an increase in participation in boating related activities

(MRA, 2013) as well as club social events continues to bring new members to join the VYC.

In addition, several clubs no longer accommodate cash purchases or member chits. (M.

Ceryes, personal communication, November 2017) Instead, members provide a credit card, so all


club purchases are automatically charged directly to their credit card on file. Streamlining

payment options decreases accounting workflows by eliminating the need to manage cash and

member chits. An integrated system that manages front-to-back office workflow provides a

foundation for the VYC to implement more efficient accounting workflows in the future.

When researching which system to purchase, the VYC found several Bay Area clubs that

also use the CE system. A site visit to the Corinthian Yacht Club provided insights on the

benefits and risks of implementing the same system at the VYC (M. Ceryes, personal

communication, November 2017). A key feature of the CE system is the ability to integrate

many functional areas - membership, POS, inventory and the back-office. Due to limited staff

resources as well as the turnover that occurs within the volunteer groups, the VYC determined a

fully integrated system, such as CE, would benefit its operational needs. Maximizing operational

efficiencies also addresses the long-term challenges with rising harbor management costs and

intense competition to retain club memberships and sustain berthing income.

Solution DescriptionThis project’s overarching goal is to provide sustainable training resources that allows

members, volunteers and staff to use the new system efficiently and effectively. Outcomes to

measure achievement of this goal include:

Increase in members successfully using the online payment portal

Increase use of the POS system during club events

Decrease in variances when using the POS system

Increase of staff and officers posting POS batches, member payments and chits.

ObjectivesThe following objectives are aligned with the project’s goals:

1) Using written procedures and/or software simulations, members will be able to view

their statements online and make a payment using a credit card or electronic debit

transaction.

2) Using written procedures and/or software simulations, volunteers will be able to:

a. Enter POS transactions for the bar, galley or ship’s store and accept payments

via cash, credit card or member chits.


b. Reconcile the POS shift summary and accurately document any cash

overages/shortages.

3) Using written procedures and/or software simulations, officers and staff will be able

to:

a. Post POS batches with appropriate notes and cash adjustments.

b. Enter and post member payments using cash/check.

c. Enter member chits to charge member accounts and ensure accurate monthly

statements.

Learning SolutionTo address the performance gaps in efficient and effective use of the new CE system, the

learning solution includes:

Written procedures and/or software simulations for online payments, POS

transactions and back-office procedures.

Feedback from practice quizzes, post-tests, self-assessments using online survey tools

and variances identified in financial reports.

A web-based knowledge hub to provide consistent access to procedures, simulations

and feedback tools for continuous, ongoing improvement.

Task AnalysisBased role and responsibilities, learners display mastery of the objectives by accurately

completing the following tasks:

1) Members: View their statement online to find their monthly balance and make a

payment for the balance due via credit card or electronic debit.

2) Volunteers in POS Areas:

a. Create a “quick ticket” for a food, beverage or retail item and process a

payment via cash, member chit or credit card.

b. Print an “end of shift” summary and reconcile cash, chits and credit card

payments to the shift summary.

3) Officers and Office Staff:

a. Reconcile POS batches to “end of shift” summary reports and post to the

General Ledger with appropriate notes and adjustments, as needed.


b. Reconcile member payments to bank deposits and post to the General Ledger.

c. Enter a “chit” as a member charge and select the appropriate items to bill to

the members’ account.

Media and Delivery System DecisionsThe primary delivery formats are instructor led, online and self-study. Email

communication with embedded links to the online payment portal along with written procedures

and a simulation video will be sent as members join the VYC. In addition, monthly membership

events provide an opportunity for synchronous delivery of content to members. For volunteers

in the POS areas, the POS training portal may be provided prior to synchronous training.

Training for officers and office-staff occurs via synchronous and asynchronous sessions as well.

A variety of media was created and deployed to meet adult learners needs as well as the

roles and responsibilities within the VYC. Screencasts in Camtasia allow passive viewing as

well as interactive feedback. Software simulations in Captivate serve as practice to gain

competency with the POS features. Website development occurred in Dreamweaver using

JavaScript and jQuery components. Images and logos were edited in Photoshop incorporate

visual graphics. Videos are stored unlisted within YouTube to mitigate unnecessary access to

member related protocols.

Learning Theories

The primary learning theories applicable to this project are:

Adult Learning Theory (i.e. andragogy), which encompasses self-directed

learning (SDL);

Cognitivism:

o Cognitive Information Processing (CIP), as aligned with Gagne’s nine

events of instruction; and

o Schema Theory

Constructivism


As stated in Merriam’s (2003) article on andragogy and SDL, adult learners require a

different approach to learning. The “five assumptions underlying andragogy” applicable to the

adult learners served by this project are: 1) able to direct their own learning; 2) possess several

life experiences to relate to their learning; 3) learning needs relate to social roles; 4) problem

centered that allows immediate application of the knowledge; and 5) intrinsic motivation.

(Merriam, 2003, p. 5) Learning modules organized by role that are learner paced align well with

andragogy and its SDL components.

Cognitivism, as outlined in Harasim’s Learning Theory and Online Technology (2017),

provided additional guidance on the design of this project. Learners process varying levels of

information for retrieval and application to specific situations as needed. Schema theory

indicates “learning is easier if new subject matter is compared to existing knowledge…”.

(Harasim, Location 1382 of 6204, ebook) Activities such as paying a bill, purchasing retail

services or depositing money into an account are common financial tasks that the learner can

relate to the content included in the modules. Incorporating prerequisites into the evaluation

tools, such as ability to use a touch screen, allows learners to reflect on their current skills or

experiences needed for successful use of the system.

Gagne’s nine events of instruction create a structure for the learning strategies that can

also be utilized during development of the learning modules. Pollock, Jefferson and Wick’s The

Six Disciplines of Breakthrough Learning: How to Turn Training and Development into

Business Results (2015, p.114) cross maps Gagne’s nine events to the corporate learning

environment and ensures development focused on the adult learning audience. Demonstration

videos gain learner attention and are organized into smaller topics, to break down the larger

workflow. Interactive questions and worked examples guide learners with practice options and

knowledge checks while encouraging repetition for increased mastery. Step by step procedures

serve as job aids to support learners transfer skills and change behaviors.

Finally, constructivism’s key principles of: 1) active learning, 2) learning-by-doing, 3)

scaffolded learning, and 4) collaborative learning (Harasim, Location 1879 of 6204, ebook)

provided more direction to design the activities of the learning modules. Learners complete

specific tasks during or immediately after completing the modules, which requires the learner to

“participate and act…rather than learn passively.” (Harasim, Location 1879 of 6204, ebook)

Simulations, case scenarios and problem-based learning (PBL) (Harasim, Location 1950 of 6204,


ebook) incorporated into the modules provides an opportunity to learn by doing the specific tasks

learners will encounter in their roles at the club. Website resources can be access while

proficient members provide synchronous support to other learners during group collaborations,

such as scheduled meetings, as needed.

Instructional Principles and StrategiesMerrill’s “First Principles of Instruction” (2002) offer more guidance to project

development. Problem-centered learning modules demonstrate specific tasks and engage the

learner by practicing or completing the actual task presented in the content. Connecting new

knowledge to existing knowledge supports cognitivism’s schema theory. Software simulations

demonstrate the specific situations the learner will experience. Practice opportunities allow

learners to apply their knowledge. When a learner completes the task in real-time, such as an

online payment or a POS transaction, they are transferring their new skills to their everyday life.

(Merrill, 2002)

The design of the learning modules also aligns with five-step model described in

Stolovitch and Keeps’ Telling Ain’t Training (2011). Rationale, learning objectives, activities

and feedback are key components to support adult learners. In addition, learner readiness can be

measured through a self-assessment or whether the learner meets the prerequisites for each

module. Learner autonomy is a vital part of engaging and motivating the target audience, so

learners may navigate videos and website tabs at their own pace. As part of the constructivist

perspective, applying the new knowledge immediately in practice modules or a quiz also support

learners using feedback opportunities.

Motivational elements are critical to ensure learners participate in and apply the

knowledge gained from the training. Kellers’ ARCs motivational model (Pappas, 2015)

presented many options to address the conditions outlined in the learner analysis. Software

simulations, group interactions and job aids attract learner attention via active participation, use

of a variety of media and demonstration of real-world examples. The modules are relevant

activities that learners can connect to prior experiences and are useful in solving authentic

problems. Prerequisites, feedback and self-paced learning builds learner confidence, which

increases learner motivation.


Multimedia Principles and StrategiesBased on Clark and Mayer’s e-Learning and the Science of Instruction (2016), several

multimedia principles and strategies were applied to the learning modules due to the types of

media and delivery of content. In addition, since learner knowledge varies from novice to

expert, additional considerations influenced the development of the multimedia content.

1) Multimedia Principle: Media with both words and graphics specifically support

novice learners. Learner feedback indicates positive interest in visual elements to

support attention and sustain motivation. Decorative graphics are used primarily

within webpages for visual appeal. Topic organizers, screencasts and software

simulations use graphics to organize content while encouraging learner interaction.

(Clark, Location 2044 of 10199, ebook)

2) Contiguity Principle 1 and 2: In written procedures, words are placed near the

relevant graphic to ensure efficient cognitive processing. In videos with audio,

narration occurs at the same time as signaling occurs. (Clark, Chapter 5)

3) Modality Principle: For software simulations, audio narration helps minimize on-

screen text. (Clark, Location 2776 of 10199, ebook)

4) Redundancy Principle: Procedures and simulations adheres to the redundancy

principle, which avoids presenting words in audio and text, unless “boundary

conditions” exist (e.g. no visuals are present, learner paced, technical/unfamiliar

words and text is shorter than narration) (Clark, Location 3049 of 10199, ebook)

5) Coherence Principle: By applying a “less is more” strategy (Clark, Location 3475 of

10199, ebook), media avoids extraneous words, unnecessary topics and complex

graphics to ensure the learning modules are well organized, uncluttered and effective.

6) Personalization and Embodiment Principle: During software simulations, a human

voice and on-screen agent are employed to support deeper learning. Using

personalized text and polite speech while avoiding passive voice helps sustain a

conversational style during the narration. (Clark, Location 3878 and 3932 of 10199,

ebook)

7) Segmenting and Pretraining Principle: Learning modules are segmented by role and

responsibility. As the complexity of certain tasks increases, lessons are segmented

further to “break a complex lesson into smaller parts”. (Clark, Location 4293 of


10199, ebook) In addition, learners will receive materials before training occurs as

well as an outline on how to navigate the lessons.

8) Learner Engagement: Practice exercises and quizzes provide learners the opportunity

to engage with the lesson materials. As learners become experts in using the CE

system features, they will teach other members to use the system. Signaling

techniques also direct the learner’s attention to relevant content, terms and activities.

9) Worked Examples and Practice: Learners view software simulations of the

segmented lessons, which provides step-by-step examples of each workflow. For

more complex lessons, such as POS transactions, practice elements include

appropriate feedback to leverage deeper learning.

Challenges and Breakthroughs

During the initial launch of the project, human resources were redirected to support

network maintenance due to downtimes occurring with the POS devices. As such, some

multimedia for the CMA module is not as well defined as other media elements and lacks

practice options. To address this deficiency, synchronous training using the CMA website to

reference procedures strengthens the training delivery method with interactive practice directly

into an office workstation.

Accessing the CE system to create the software simulation components of the solution

was resolved using a virtual private network (VPN) license. Successful remote network

connectivity within the CE system allowed software simulation recording to occur outside the

office on a computer with recording software. Syncing narration, signaling and video recordings

required extra time and attention to audio levels, close captioning and timing of elements.

Topics organized into smaller segments along with a watch/try/assess format aligned well

with both Camtasia and Captivate media tools. Adding questions into Camtasia video

demonstrations allowed an additional level of learner interaction. Using Captivate simulation

features allows efficient creation of practice and assessment options with minimal impact to

production timelines. Reuse of website layouts using .css files from prior projects streamlines

coding within Dreamweaver. These development decisions supported meeting the milestones on

the project’s timeline.


Finally, additional technology requests were prioritized to allow completion of the

training modules, which rounds out the implementation for phase one of the CE system. In

addition, department managers of the store and bar already requested additional volunteer

training with the websites. Continued use of the training modules develops opportunities to

improve consistent outcomes and update content.

Methods/Procedures

Major DeliverablesList of major deliverables include website content organized by role/function for: 1) the

online payment portal; 2) POS transactions; 3) the back-office procedures for POS, member

payments and chits. Scripts and handouts were created in word processing software along with

screen shots or other images as needed. Software simulations produced in Camtasia include

signaling and questions with feedback. Captivate allows learners to practice and test workflow

procedures. Images edited in Photoshop provide continuity in graphic design of the website.

Website development in Dreamweaver provides the delivery platform, branching structure as

well as interactive elements using JavaScript and jQuery. Reference materials, such as CE

manuals, are linked for digital distribution. In addition, evaluation tools on usability, pre/post

learner readiness and knowledge transfer obtain feedback on the learner experience.

Design Analysis Characteristics of the Human Performance Technology model for Change Management

(Reiser, page 127, 2018) as well as the ADDIE system design model form the foundation to

analyze the project’s design. Organizational goals cascade with learning objectives to help

achieve business results. Iterative evaluation cycles with the client occurred during each design

phase to confirm topic content, media elements and feedback. VYC management harvested

member experiences to guide which topics are covered in the training as well as the frequently

asked questions. Capstone adviser(s) recommended prioritizing case studies and interactivity

features to enhance active learning. Based on the learner analysis, design elements were

organized by role, so learners can build their knowledge based on their level of involvement at

the VYC.


Agenda.

1) Online payment portal (up to 15 mins – online; at least 15 mins - in person)

a. Send website link and instructions to member for online learning.

b. Open website link during in person training to review with member.

2) POS transactions (up to 25 mins – online; plus 15 mins - in person)

a. Send website link to volunteer for practice before first POS shift

b. Observe volunteer complete POS transaction upon arrival at first shift

3) Back-office procedures (at least 35 mins – preview and in person)

a. Send website link to staff or officer for orientation before first shift.

b. Open website link and CE system to conduct in person training on CMA

modules.

Activity Descriptions.

Agenda Item Topic Description

Online Payment

Portal How to view your

monthly statement

How to make a payment using a bank account

How to make a payment using a credit card

Learners view a software simulation

video, review instructions with

screenshots or navigate through key

concepts.

For synchronous training, learners listen

to an SME and may practice using the

online payment portal.

POS Transaction How to Login

How to create a ticket in a POS system

How to accept a payment

How to end a shift

Learners view demonstration videos

with feedback, interactive simulations to

practice, review instructions with

images, and complete survey/quizzes.


to an SME and demonstrate proficiency

using the POS system.

Back-office

Procedures How to login

How to post POS batches;

Learners review written instructions

embedded with images of key buttons


Agenda Item Topic Description

How to enter member chits;

How to post member payments

and screenshots.


to an SME and practice using the CMA

system.

All Agenda Items Intro and Feedback surveys for learner reflection

Quiz tabs for knowledge transfer

Learners complete Intro, Feedback and

Quiz forms to assess learner readiness

and complete knowledge checks.

Implementation PlanSynchronous training requires an SME, already trained on the CE system, that is

knowledgeable about VYC membership rules, is familiar with the financial workflows, and role

models an early adoption of the new system for their personal membership payments and

position in the VYC organization. SMEs complete training on specific topics as needed to teach

other learners. Office staff manage access to various modules, performance aids and evaluation

tools to support the learning activities of the website.

ResourcesSeveral resources fully supported this project, including: subject matter experts, users,

CE data, digital devices, software and external expertise. Specific roles and resources were:

Subject Matter Experts : Officers, Directors, Manager

Users : Members, Volunteers, Staff, Officers and Directors

Data : Online Payments, POS transactions, Financial Reports from CE and ACH sources

Devices : Desktop/Laptop, Tablets, Microphone, Headphones or speakers, Digital

Camera

Software : CE System, Camtasia, Captivate, Dreamweaver, Photoshop, Microsoft Word,

Google Forms

Space : Within the VYC at POS stations or conference rooms

External Expertise : CE staff, such as the Project Manager, Implementation Specialist, or

Technical Support


Images : Personal photos, VYC logo, VYC burgee

Other : Prior MIST coursework, reading materials and textbooks

Logistics and BudgetSoftware simulations were produced using a VPN account to access the CE system from

a desktop computer installed with screen recording software. Procedure documentation was

drafted with word processing software. Feedback is obtained from embedded practice quizzes or

Google forms. Synchronous training occurs within the VYC and requires meeting space with

tables, chairs and digital devices for demonstrations. Asynchronous training may occur in or

outside the VYC. All training needs an internet connection. No additional expenses are

expected, unless external technical support is needed.

Technical Skills and External ResourcesThe technical skills needed for this project include: 1) knowledge of recording software

with editing features, 2) ability to create websites, and 3) familiarity with Google forms.

Additional technical support related to the CE system will be requested through the CE customer

help desk. Any technical issues (e.g. VPN, LAN) may be requested through the VYC’s

independent contractor for information technology (IT) support.

Timeline/Progress ReportThe project’s timeline spanned four months, from September 2018 through December

2018. To complete the project on time and within scope, the developer, stakeholders and

sponsors needed to sustain priority status for the project and engage member support for the

project. Any budget approvals for external expertise and additional technical support required

timely turnaround.

Milestones ChecklistTimeline Milestone Comments

September 25, 2018 Project Proposal

Storyboards

Outlines design elements to

guide development;

Share with client for

approval of scope and


Timeline Milestone Commentsdeliverables.

October 23, 2018 Develop software simulations and

written procedures

Create draft sample

materials to share with

client for approval

November 18, 2018 Complete media production and

website programming

Adjusted deliverables for

CMA module to written

procedures with

synchronous training.

November 20, 2018 Finish development and usability

tests with learners

Revised and modified

modules based on feedback

and results.

December 4, 2018 Finish summative evaluation with

learners

Organize data and perform

statistical analysis on results

December 11, 2018 Implementation:

Final Materials

Final Report

Share final materials with

client

After December 2018 Continued Evaluation:

Learner surveys Assessments Variances and/or downtime Decrease in fees, variances

and expenses

Share feedback with client

Evaluation PlansBoth formative and summative evaluations occurred to determine the usability of the

learning solution as well as its effectiveness in improving learner knowledge about the new CE

system. Formative evaluation was completed prior to the final release of the learning modules

for pilot testing. To ensure an optimal user experience, remedies were developed related to

erroneous content, navigation challenges or technical/logistical issues when delivering the

instruction. Summative evaluation occurred after implementing the learning modules for


additional testing. Additional metrics related to behaviors and results may also guide future

development and utilization of the learning modules.

Formative Evaluation Usability testing was conducted following a “single subject testing” (SST) methodology

using in person observations. (See Appendix B) One of three users participated in each SST

cycle and revisions were developed for the websites prior to the next test cycle. A final test

concluded the SST and confirmed revisions approved during prior SST cycles were included.

Participants reflected the target audience’s variation in prerequisites and experience with the CE

system.

Additional feedback collected using Google forms indicates instructions and navigations

are easy to follow. (See Figure 1) Narration, graphics and video received high marks, whereas

audio quality and engaging lesson design needs further development. Some learners shared that

the audio was difficult to hear in some sections of the video media.

Only a slim majority of learners “strongly agree” on the module’s overall educational

value, while other feedback encourages another look at the module’s content or delivery

platforms to improve learner engagement and the module’s overall value. Feedback included

embedding additional images and screenshots in the CMA modules to help learners visualize

connections from the procedures to the system features.

The lesson instructions are easy to understand.

The navigation instructions are easy to follow.

Navigating the module was easy.

The lesson design is engaging.

Graphics and video are informative.

Narration adds to the learning experience.

Audio quality is clear.

Overall lesson has educational value.

0 1 2 3 4 5 6

Yacht Club Training Usability Results

Strongly Disagree Disagree NeutralAgree Strongly Agree

Figure 1


Summative Evaluation Evaluating the learning modules effectiveness are based on the four levels outlined in

Kirkpatrick and Kirkpatrick’s Evaluating Training Programs (2006). Twelve evaluation tools

were created to gather data from learners as follows:

1) Introduction and Feedback (2 per website module = 6 total): These surveys collected

pre and post assessments to monitor learner readiness to complete learning objectives.

2) Quizzes (1 for the online payment and CMA modules; 4 for POS module = 6 total):

These quizzes assessed learner knowledge about features and applying system

features to case scenarios.

Level I —Reaction. A self-assessment of learning objectives based on ratings was

completed by six participants, translated into ordinal values and analyzed as the difference of

paired samples. (See Appendix C) The null hypothesis states the difference between the paired

sample measures is negative, which represents no improvement in the learner’s assessment of

skills learned during the modules. Since the Alpha (.05 significance level) is less than the p-

value, the null hypothesis is not rejected and concludes the differences in the pre and post

assessment values are not statistically significant to support learner readiness in completing the

learning objectives.

Level II—Learning. All quiz scores were analyzed together to calculate an average

score for all quizzes. Thirteen (13) quiz results were included in the test of a single sample

mean. (See Appendix D) The established criteria for a “passing” result is to achieve correct

answers on 4 of 5 questions or 80%. The null hypothesis states the average learner score is less

than or equal to 3.99. Since the average mean of the sample is 3.92 and the p-value (0.61) is

greater than the alpha (p<.05), the null hypothesis is not rejected and concludes the sample data

does not provide sufficient evidence that the alternate hypothesis (average score > 3.99) may be

true.

Level III—Behavior. The following metrics as of December 2018 also monitor usage

of the new system:

216 members using the online payment portal. (See Appendix A)

Zero POS device usage for Galley services.


Delayed processing of POS batches, member payments and member chits beyond

3 business days.

Level IV—Results. The BOD may consider additional metrics to assess future business

impacts and ROI such as:

Reduction in annual merchant services fees

Reduction in cash adjustments and variances in financial reports

Club operational expenses remain flat or decrease due to improved efficiencies.

ConclusionDespite positive reactions to usability and test scores, statistical analysis lacks convincing

evidence to determine the learning modules significantly improve learner scores or perceived

readiness to complete learning objectives. Several learners in the sample group possess some

familiarity or actual experience with the new system, which would influence how learner

readiness is impacted within this sample group. In addition, an extreme range of learner

prerequisites exists which may influence the inconclusive analysis of the assessment ratings and

test results. Further, the complexity of applying the appropriate statistical tests also factors into

the analysis and interpretation of the summative results.

The project’s design involves several multimedia options as well as evaluation tools. As

learners navigate the websites, they select the media per their direction. As such, not all learners

completed all evaluation tools as originally expected. Over twelve evaluation tools frame the

feedback and knowledge check options for learner interaction. Well organized, functional

navigation across websites resulted in positive usability and ease of use. Yet, engaging lessons

and audio quality are areas that need further review to continue enhancing the learner experience.

Design and development requirements heightened the need for additional support,

especially in media production. Mastery of media as well as web development, instructional

design, evaluation instruments and project management distract from the creative skills required

for truly engaging media development. A team environment would benefit this project to

address feedback related to engaging lessons as well as audio quality.

After updating additional media elements, additional volunteers will participate in using

the training modules to prepare for POS shifts. Clarifying instructions to navigate the


introduction, feedback and quiz elements of the module may help learners provide more

feedback to analyze effectiveness. Additional talent on the project may help produce improved

media design elements. Multiple, diverse perspectives ensure important facets of instructional

design and technology are intertwined effectively to maximize engagement and improve learner

performance.


ReferencesClark, R., & Mayer, R. (2016). e-Learning and the Science of Instruction: Proven Guidelines for

Consumers and Designers of Multimedia Learning. Hoboken, NJ: John Wiley & Sons,

Inc. ISBN: 9781119239086

Evans, J.M. (2000). Vallejo Yacht Club: The First 100 Years. Vallejo, CA: Vallejo Yacht Club,

Inc.

Harasim, L., (2017) Learning Theory and Online Technologies. 2nd Edition. New York, NY:

Routledge.

Kirkpatrick, D.L. and Kirkpatrick, J.D. (2006). Evaluating Training Programs: The Four Levels.

Oakland, CA: Berrett-Koehler Publishers, Inc.

Marina Recreation Association (MRA) (2013). Annual Conference. Santa Barbara, CA

Merriam, S.B., (2003). Andragogy and Self-Directed Learning: Pillars of Adult Learning

Theory. New Directions of Adult and Continuing Education, No. 89. Jossey-Bass, A

Publishing Unit of John Wiley & Sons, Inc.

Merrill, M.D., (2002). First Principles of Instruction. ETR&D, Vol. 50, No. 3. Retrieved from

firstprinciplesbymerrill.pdf

Pappas, C., (2015). Instructional Design Model and Theories: Keller’s ARCS Model of

Motivation. Retrieved from https://elearningindustry.com/arcs-model-of-motivation

Pollock, R.V.H., Jefferson, A.M., and Wick, C.W., (2015). The Six Disciplines of Breakthrough

Learning: How to Turn Training and Development into Business Results. 3rd Edition.

Hoboken, NJ: John Wiley & Sons, Inc.

Reiser, R.A., & Dewey, J.V., (2018). Trends and Issues in Instructional Design and Technology

4th Edition. New York, NY: Pearson.

Stolovitch, H.D., and Keeps, E.J., (2011). Telling Ain’t Training. Alexandria, VA: American

Society for Training & Development.

https://elearningindustry.com/arcs-model-of-motivation


Appendices

Appendix A: VYC Membership and ACH Reports

Appendix B: Single Subject Testing (SST) Instrument

Appendix C: Level I – Reaction on Learning Objectives

Appendix D: Level II – Learning Analysis using Quiz Results

VYC Membership Data 12062018.pdf

VYC ACH Data 12062018.pdf

VYC ACH Customers 12062018.pdf

SST Test Instrument VYC Portals 111718.pdf

VYC Learner Objectives Stat Test 120918.pdf

VYC Quiz Stat Test 120918.pdf

executive summary - dceryesmist.weebly.com file · web viewcalifornia state university. monterey...

Documents