Recommendations for a Campus Survey Tool

Orlando Leon, Enterprise Architect - Education, ITS

Beth Berrean, Communications/Website Services Manager, ISU

Opinder Bawa, Chief Technology Officer, ITS

Overview

• Charge

• Analysis

• Recommendations

• Next Steps

Our Charge

• Should we purchase and support Qualtrics as an enterprise solution for the Campus?

• Should UCSF decide on a single solution for all survey needs?

• Should UCSF rationalize availability and support of existing survey solutions currently within the Campus?

Analysis: Sample Use Cases Encountered• Education

– Student Learner Survey

– Evaluation

– Feedback

– Registration

• Research– Radiology Orthopedics Study

– mOst Survey Tool

– ATHENA

– Clinical Trial Qualification Surveys

– Research Questionnaires

Analysis: Additional Use Cases Encountered• Administrative

– Polls

– Voting

– Usability Satisfaction/Feedback Surveys

– Interview Candidate Feedback

• Clinical– Nursing PI Surveys

– Patient Satisfaction

Analysis: Existing Survey Tools

Currently supported by

Primary Use Costs

Checkbox SOM ISU, SOM EdTech

Surveys for education and some one-off surveys for SOM

TCO $7600/year for unlimited users

RedCap Cancer Center & ITS ARS

Clinical research or by individual PI’s who need to store ePHI

Unknown, but it is an open-source product

Salesforce SOM ISU Athena, Nursing Performance Improvement Group

Project-based

SurveyMonkey Ad-hoc Simple surveys $780/user/year for highest level of support

Zoomerang Nursing Alumni outreach $599/user/year

Findings on Tools: CheckBox (v4)

• User interface: OK

• No end user (survey creator) training program

• As currently implemented at UCSF,– Not able to collect ePHI

– No support for multi-lingual surveys

– Allows importing of data (Medical Students since 2004)

– Integrated with AD (cannot integrate with MyAccess)

• Key Feature: Actively Used – The SOM MedEd group uses this tool widely and

SOM ISU also uses it. It is stable, and there is a upgrade path.

Findings on Tools: RedCap

• Able to collect ePHI

• User interface: complex

• ARS offers classes

• As currently implemented at UCSF,– No support for multi-lingual surveys

– No native accessibility features

– Not integrated with MyAccess

– Mobile compatible?

• Key Feature: Validated Instruments– Data collection instruments and forms reviewed for

research relevance, accuracy in coding and function by the RedCap Library Oversight Committee.

Findings on Tools: Salesforce - Health Quality Surveys (HQS)• Able to collect ePHI

• Integrated into mobile application development platform

• Open-source survey definition language supported by Harvard, MIT, and the Boston Children's Hospital

• Key Feature: Integration with Salesforce– Integrated with multiple systems (including multiple

EHR systems) and the UCSF IDR

• “Anything is possible.”

Findings on Tools: Qualtrics

• Able to collect ePHI (after signing a BAA)

• SSO integration with MyAccess

• Mobile versions of surveys “out of the box”

• Integration with Salesforce objects

• Supports 48 languages

• ADA compliance-checking tool

• Panels

• Triggers

• Robust user/group/organization/library features

• Programmatic interface to administration tools

• Training and Support

Findings on Tools: Others

• SurveyMonkey– ADA compliant

– Free

• Zoomerang

• SurveyGizmo– Mobile versions built into platform

– Professional services available

– Integration with Salesforce

Considerations

• In conversations with stakeholders (likely survey users—SON, SOM MedEd, Nursing PI, SOM TAC people asked us to consider:– Product Features

– External/Vendor Support

– Internal Product Ownership

– Integration with Existing Systems

– Total Cost of Ownership

– Expertise in Survey Design vs Knowledge of the Technology

Recommendations

• Continue support for RedCap research, Salesforce for research and clinical needs

• Replace Checkbox with Campus license for Qualtrics

• Create a strategic marketing plan to help move all qualified users into Qualtrics

• Create product oversight group for each part of the Campus that intends to use the tool (Schools, Centers, Departments, Administrative) as first line of support for tool

Cost Projection

• 2011-2012 – est $48,000– $20,000 - Annual license cost

– $20,000 - 0.25 FTE @ $80k/year for brand administration, organization configuration and custom template creation

– $5,000 – 0.05 FTE @ $100k/year for MyAccess integration

– $6,000 – 0.10 FTE @ $60k/year for marketing and communications

• 2012-2013+ est $24,000*– $20,000 - Annual license cost

– $4,000 - 0.05 FTE @ $80k/year for brand administration

* The assumption is that each functional unit using Qualtrics will administer their own users and organization configuration, while the overall brand administrator administers the overall Campus/global configuration and setup of new functional units

15

Next Steps

• Endorse recommendation

• Decide where within the enterprise the product should be managed

• Negotiate a BAA and license agreements with Qualtrics

• Create a Product Oversight Committee

• Develop a marketing/communications plan for helping internal constituents identify survey needs and tools