the random sampling/tracking tool: a response to over- surveying steve graves intel corporation
TRANSCRIPT
The Random Sampling/Tracking Tool: A Response to Over-
Surveying
Steve GravesIntel Corporation
Overview Problem Statement Recommended Solution Features User Interface Demo Benefits Concerns Next Steps Possible tool enhancements Beyond Intel Summary
Problem Statement
Perception that too many surveys are deployed to end users from IT – Source: Annual Partnership Excellence Survey with top Intel Execs
Likely causes identified (based on experience as a survey consultant)– Groups tended to survey entire population of interest
instead of a sample (e. g. survey of habits of frequent travelers -- 26,000 people)
– When criteria for a population is only partially available a screening survey is necessary – results in more people surveyed
– Simple random sampling will tend to hit a small percentage of people multiple times
Same Participant
General Population
Study A
Study B
Problem Statement
Effective and efficient process to stop over-surveying does not exist– Previous process relied on incomplete information and
“Guardians” – go between groups– No mechanism to track who has received surveys
– Guardians may be overly cautious
– Meetings used to determine what to do – time consuming and inefficient
– Process is not followed universally
Recommended Solution
Sampling tool containing a repository of Intel employee data– Utilizes data from employee database (non-
confidential)
– Generates random samples of target audience
– Tracks who has been sampled over a given time period – currently 5 months
– Sampled individuals will not be selected again for during this period
Recommended SolutionDatabase perspective
Employee Database Retrieval Database Sampling Database
Sampling DatabaseStorage Database
Tracking Table
Randomly Selected Employees
Recommended SolutionDatabase perspective – previously sampled individuals
removed before Sample is generated
Tracking Table
Generated table based on CriteriaTable that meets criteria and contains no excluded individuals
Recommended Solution
Incorporate into an overall process where researchers are required to utilize the tool and consult with sampling advisors– Advisors can advise researchers on appropriate sampling
techniques – so researchers to survey an entire population (e.g. 3,000 instead of 26,000)
Currently implemented using RAD database programming language
– Tool has been in use for almost 2 years
– Tool is performing reliably and customers are satisfied
– Usage is only through word of mouth
– Approximately 60 samples generated last year
Sampling Tool Features
Current features– Tracking and recording mechanism
– Recording of samples generated by others
– Random Sampling of entire Intel population
– Random sample from group that uses criteria not part of employee database (for example: Employees who use PDAs)
– Random sampling of Sub-Groups (For example: Employees in Folsom California)
– Stratified sampling
User Interface
Demo (Double Click to Start)
Benefits Saves researchers time
–Reduces the need for screening surveys
–No need to re-invent the wheel
–Saves time compared to previous process
Benefits Improvement in sampling process
–Reduce or eliminate “over-sampling”– Delay surveys when target population is not
available– Researchers receive consultation on using
appropriate sample sizes–Distribute sampling more evenly across groups
Concerns May cause sampling bias in some situations
Next Steps Need to develop more sophisticated business rules
around sampling process Currently working to make this an IT wide process
– Early results are promising
Looking at making it an Intel wide process Port to SQL server and have a web based solution
– Current access is through Remote Desktop Connection
Develop strategies to mitigate sampling bias
Possible Future Tool Enhancements
Complex samples through UI – currently requires custom query
Linking to Other database that contains product usage information
Beyond Intel Similar tools could be used in similar situations in other companies or institutions
Summary Problem: Over-surveying by our IT organization
Response: Random Sampling/Tracking Tool in conjunction with business rules to manage survey sampling process
Benefits: Saves researchers time, reduces over-surveying, improves sampling process
Working to make this an IT wide official process and eventually Intel wide
Summary Next steps:
–Link to other databases – Improve business rules –More robust software platform–Strategies to mitigate sampling bias
Similar tools/business rules could be used in other institutions