Cell phone problem and alternative approaches

Chris McCartyUniversity of Florida

PHC 6716 May, 25 2011

2010 UFSRC Surveys by Mode

AASRO Survey Labs 2010

The Cell Phone Problem

Adults versus children

Demographic differences for wireless only households

(Data from NHIS January to June 2010)• More than half of adults aged 25–29 years (51.3%) lived in households with only wireless

telephones

• More than two in three adults living only with unrelated adult roommates (69.4%) were in households with only wireless telephones

• Nearly half of all adults renting their home (47.1%) had only wireless telephones

• Men (26.2%) were more likely than women (23.7%) to be living in households with only wireless telephones

• Adults living in poverty (39.3%) and adults living near poverty (32.9%) were more likely than higher income adults (21.7%) to be living in households with only wireless telephones

• Adults living in the Midwest (26.6%), South (29.3%), and West (23.5%) were more likely than adults living in the Northeast (15.8%) to be living in households with only wireless telephones

• Hispanic adults (34.7%) were more likely than non-Hispanic white adults (22.7%) or non-Hispanic black adults (28.5%) to be living in households with only wireless telephones.

Health differencesHealth Characteristics by Phone Type

0

10

20

30

40

50

60

70

80

Binge

drink

Curre

nt sm

oker

Exerc

ise

Excell

ent/g

ood

healt

h

Influ

enza

shot

Teste

d for

HIV

Uninsu

red

Landline

Cell-phone only

No phone

Issues in calling cell phones – Dual frames

• Typically one land line per household

• Typically one cell phone per person

• Some households have both

• Results must be weighted (somehow)

• Respondent selection for household survey potentially different between cell and landline

Issues in calling cell phones – Getting numbers

• For listed sample there is no problem

• Cell phone companies differ in the way they release numbers

• Numbers may not conform to geography– In Europe they often do – In U.S. they conform to point of purchase, not where they live (you

could easily be calling at inappropriate time)

• Cell phones are truly mobile– People may live outside area code of their cell phone number– This makes overlay of census information virtually impossible

• Turnover of cell numbers is high

Issues in calling cell phones – Legal• Telemarketing Consumer Protection Act of 1991 (TCPA) enforced by the FCC

• Automatic telephone dialing systems cannot be used to contact a wireless telephone number for which the called party is charged without the user's “prior express consent”

• The TCPA defines “automatic telephone dialing system” as equipment that has the capacity to store or produce telephone numbers to be called using a random or sequential number generator, in conjunction with dialing such numbers

• Interpretation - Cell phone calls should be manually dialed (what is manual?)

• May prevent the use of “interviewer initiated dialing” (interviewer pushes a button and the dialer places the call

• Europe does not have this law

• Problems– With landline portability could call cell phone and not know it– Many (if not most) survey centers use Voice over IP (VOIP) technology, so dialing manually

may be impossible

Issues in calling cell phones - Other• Ethical Issues

– Respondent may be in hazardous circumstance (driving)– Respondent may be in public area– Children have cell phones, but may pose as adults to get incentive

• Cost– Typically use incentive to defray cost of call (In Europe you don’t pay for

calls received)– Cell phone require more attempts– Built-in caller id and ringtone identifying when not in memory

• Disposition codes– Types and distribution of call outcome are different than land line– This affects calculation of response rates– Cell phones are often used for business and non-business

Solution 1 –RDD-only weighted using demographics from Census/ACS/CPS

• Conduct RDD landline survey

• Weight for demographic differences

• Some studies suggest that “Wireless mostly” are similar to “Wireless only”– Weight for differences

Solution 2 - Cell phone only weighted

• The reverse of previous slide

• I am not aware of this in practice

• As of 2010 a cell phone only survey would be more biased than a land line only survey

• Over time this may become a more realistic option

• This depends on changes in the trends in land line use

Solution 3 - RDD using Cell phone only for comparison

• BRFSS solution 2007-2009

• Separate RDD and cell phone samples– Cell phone complete sharing between states to

address geography issue

• Unclear how comparison data are used to weight results

Solution 4 - RDD/Cell phone dual frame weighted and combined

• At national level this is being developed

• Use NHIS data to develop model coefficients using 13 demographic variables to assign respondents to one of three groups:

1. Landline only2. Cell only3. Landline and cell

• Apply weights to ACS adults in household data to assign probability to households

• Apply these to BRFSS data

• With national sample cell completes can be “shared” across states

• Not so at sub-national level

Example: Child Trust Design OptionsDual frame RDD

• Miami-Dade survey of households with children

• Marketing Systems group generates probabilistic sample:– Listed landline (probable children)– Landline RDD unlisted and listed

without children– RDD cell phone

Cost per complete by strata. Strata Target Cost/complete Total cost per strata Cell 327 $130 $42,510 Listed with children 700 $45 $31,500 Other 973 $80 $77,840 Total 2,000 $76 $151,850

Trade-off of design effects for cost efficiency.

Stratum Universe Sample Allocation Alternative

Proportional A B C D E F G Cell 168,081 400 377 352 327 302 300 300 300

Listed 171,956 409 500 600 700 800 900 1,000 1,100 Other 500,367 1,191 1,123 1,048 973 898 800 700 600 Total 840,404 2,000 2,000 2,000 2,000 2,000 2,000 2,000 2,000

Design Effect 1.00 1.01 1.04 1.09 1.16 1.25 1.36 1.52 Effective Size 2,000 1,980 1,923 1,835 1,724 1,600 1,471 1,316

Update

June 2010

May 2011

Stratum Universe Sample Allocation Alternative

Proportional A B C D E F G

Cell 168,081 400 377 352 327 302 300 300 300

Listed 171,956 409 500 600 700 800 900 1,000 1,100

Other 500,367 1,191 1,123 1,048 973 898 800 700 600

Total 840,404 2,000 2,000 2,000 2,000 2,000 2,000 2,000 2,000

Design Effect 1.00 1.01 1.04 1.09 1.16 1.25 1.36 1.52

Effective Size 2,000 1,980 1,923 1,835 1,724 1,600 1,471 1,316

Stratum Universe Sample Allocation Alternative

Proportional A B C D E F G

Cell 248,357 454 438 411 400 400 400 400 400

Listed 153,113 343 400 500 600 700 800 900 1,000

Other 492,306 1,204 1,162 1,089 1,000 900 800 700 600

Total 893,776 2,000 2,000 2,000 2,000 2,000 2,000 2,000 2,000

Design Effect 1.00 1.01 1.03 1.08 1.15 1.24 1.36 1.52

Effective Size 2,000 1,980 1,942 1,852 1,739 1,613 1,471 1,316

Solution 5 - Address based sampling with mixed modes

• Likely the solution of the future

• Currently address information relatively comprehensive– U.S. Postal Service– Are those days numbered with the funding

problems of USPS?

Solution 6 - Existing internet panel

• Knowledge Networks or Harris

• How will this work with Medicaid and other disadvantaged populations?

• Expensive?

• Requirement to go through a small number of companies to do surveys– Too much control

Solution 7 – Social Networking Sites• Increasing use of social media as channel of communication

• Who would initiate survey, as users or site manager (e.g. Facebook)?

• Create a new network recruitment mode, similar to respondent driven sampling (RDS)?

• According to Pew Internet & American Life Project, as of 2008:

Typical Sequential Mixed Modes(Pricing Issues)

• Mail – Telephone (NCQA)

• Mail – Telephone – Face-to-face (ACS)

• Telephone – Mail (BEBR – CAHPS)

• Face-to-face - Telephone option (CPS)

• Telephone – Web (Knowledge Networks)

• Web – Telephone (USF Carpool)

Issues with simultaneous mixed modes

• Predicting the mix that comes in– Costs differ dramatically by mode

• AHCA CAHPS rolling sample– Pull records from AHCA Medicaid database– Records with phone number are called– Records without phone number are sent mail– Non-contacts from phone are sent mail