a bayesian approach to measurement bias in networking studies

A Bayesian Approach to Measurement Bias inNetworking Studies∗

Ling Zhu† Scott E. Robinson‡ Rene Torenvlied§

April 11, 2012

∗Paper prepared for the 70th Annual National Conference of the Midwest Political Science Association,April 14, 2011, Chicago, IL.†Department of Political Science, University of Houston, 447 Philip G. Hoffman Hall, Houston, TX,

77204-3011; [email protected].‡Bush School of Government, Texas A&M University, 2010 Allen Building, 4220 TAMU College Station,

TX 77843-4220; [email protected].§Institute of Public Administration, Leiden University, Pieter de la Court Building Wassenaarseweg 52

2333 AK Leiden, Netherlands; [email protected].

1

mailto:[email protected]



Abstract

The study of managerial networking has been growing in the field of public administration.Coincident with this interest in managerial networking is an interest in the use of self-reported survey data to measure managerial behavior in building and maintaining networks.The predominant approach is to use self-reported ordinal scales, aggregated through factoranalytic techniques, to generate a measure of networking activity. However, when publicmanagers use ordinal responses to describe their networking activities, they may answerthese survey questions in ways subject to a variety of response biases. Hence, ordinal scalesmay lead to biased measurements that misrepresent managerial networking. We propose aBayesian item-response theory (IRT) measurement strategy to infer how managers in open-system organizations interact with different external actors and organizations. To quantifythe latent ranking-order of the effort of managerial networking, we use a Bayesian approachto select one-dimensional scales from a bank of multiple survey items. Using 10 ordinalsurvey items in a mail survey of 1026 American hospitals, we demonstrate the advantage ofusing the Bayesian IRT model over factor analytic models.

2

1 Introduction

The study of networking activities is growing in the filed of public administration(Agranoffand McGuire, 2001; Isett et al., 2011; O’Toole and Meier, 2011). The dominant approachto measure managerial networking activities is to apply factor analysis techniques to self-reported ordinal survey data (Agranoff and McGuire, 2003; Meier and O’Toole, 2005; Thom-son, Perry and Miller, 2009). Factor-analytic methods, combined with survey research meth-ods, impose two major challenges to measurement validity: (1) the compatibility issue (i.e.survey respondents may understand the same question in different ways) (Brady, 1985; Kingand Wand, 2007; King et al., 2004) and (2) social desirability bias (i.e. the tendency forindividuals to respond in socially desirable ways – producing a common source bias) in-troduced by perceptual survey items (Donaldson and Grant-Vallone, 2002; Doty and Glick,1998; Graham and Collins, 1991; Podsakoff and Organ, 1986; Schwarz, 1999).

As Meier and O’Toole (2010) note, the field of public administration has rushed intoperformance appraisal and performance management without applying measurement theoryand methodology that deals with measurement bias. The absence of both theoretical andmethodological rigor of measurement, according to Meier and O’Toole (2010), is the Achilles’Heel of public administration research. Indeed, the scholarly search for general theoriesheavily depends on empirical studies that validate analytical propositions. Measurementquality affects our ability of testing causal relationships, making statistical inferences, andmore generally, our understanding of administrative behavior. Improving measurement thatmaps administrative and managerial behavior, thus, is critical to theory-building.

We address this enduring, yet overlooked, problem of measurement bias in the studyof networking activities. Drawing from the Item Response Theory (IRT) (Embretson andReise, 2000; van der Linden and Hambelton, 1996), including non-parametric Mokken Scaleanalysis (Mokken, 1971; van Schuur, 2003, 2011), we propose a Bayesian approach to inferthe latent dimension of managerial networking. The Bayesian approach to measurementconceptualizes the true effort level of managerial networking as a latent variable, which isnot directly observable. Theoretically, managerial networking activities are driven by time-budget constraints (i.e. difficulty in concentrating time-allocation on one networking node)and time-allocation decisions are made based on returns to investment in different networkingdimensions (i.e. the discrimination of one networking node from all the other nodes). ABayesian model, furthermore, is proposed to deal with uncertainty in parameters caused byresponse bias. We argue that the Bayesian approach of measurement is more attractive thanthe factor analytic methods because it relaxes the assumption that respondents truthfullyreport their networking activities – though care must be taken to consider differential itemfunctioning (dif) (Embretson and Reise, 2000). The Bayesian IRT approach, moreover,extend the non-Bayesian IRT models in that it is flexible to handle cross-item heterogeneityand models uncertainty in parameters. The Bayesian IRT approach, additionally, improvesmeasurement reliability via simulation-based inferences (Fox, 2005, 2010; Jackman, 2009).

Using 10 survey items on networking activities in professional health care organizations

3

(American hospitals), we illustrate the statistical procedure of computing the Bayesian IRITmeasure. We also demonstrate that the Bayesian approach to measurement helps to reducemeasurement bias associated with the classic factor-analytic methods and is particularlyuseful when the number of observations is relatively small.

2 Challenges of Measuring Networking Activities

Students of public administration are often interested in studying causes and effects of man-agerial networking (Meier and O’Toole, 2005; Robinson, 2006). Research into networks andpolicy implementation, specifically, build upon assessing latent traits such as the level ofactivity in creating and maintaining networks, the effort in network interactions, etc. Onecommon problem is that such latent traits that network scholars intend to measure do nothave well-established units of measurement. As large-scale research into networking man-agement mushrooms, the issue of lacking clear-cut measurement becomes salient.

Currently, large-scale studies on networking management are sparse (Meier and O’Toole,2005; Robinson, 2006). A few prominent examples are marked by the use of survey methodsto capture self-reported collaborative relationships or managerial activities. For example,Meier and O’Toole (2001) survey top-level administrators in Texas school districts and asksuperintendents to self-report their frequency of contact with different nodes. The articleextracts a single continuous factors from the reported contact frequencies as the basis for theirlong series of studies of the impact of networking among Texas school districts – includingapplications to such areas as disaster response O’Toole et al. (2006).

McGuire (2001) and Agranoff and McGuire (2003) survey city managers about their ver-tical and horizontal collaborative activities. The measurement strategy within these studiesfocuses on the nature of collaborative activities rather than frequency. Respondents reportedcollaboration for information sharing, project-based partnerships, or other specific activities.The shift in focus emphasizes the content of a collaborative relationships and the nature ofthe work conducted in a collaborative manner rather than the frequency of contact.

Finally, one can focus on subjective attitudes related to partnership from the perspectiveof a collaborating organization Thomson, Perry and Miller (2009). Thomson et al. measurecollaboration based on a series of attitudinal questions related to collaborative experiences.These attitudes are groups into five dimensions based on seventeen survey questions relatedto experiences with undifferentiated “partner organizations.” These five dimensions con-tribute to a single dimensional indicator of collaboration. Like with the Meier and O’Toolemeasurement approach, a series of ordinal component measures (this time of attitudes re-lated to collaboration experiences) are combined through factor analysis to create a singlemeasure of collaboration.

Work has also emerged on European administrative networks. Andrew et al. (2011)survey English local government managers and measure external networking based on self-reported interactions with external stakeholders to their organizations. Torenvlied and

4

Akkerman (2012) measure Dutch primary school principals’ managerial networking basedon self-reported survey items measuring how frequent they interact with external network-ing partners.

Although the empirical contexts of these large-scale studies are heterogeneous (i.e. dif-ferent organizations, different policy issue areas and different countries), the similar researchdesign leads to a common set of measurement problems. Multiple survey items, among otherthings, are often scaled based on factor analytic methods and combined into a networkingindex with the assumption that the numerical weights assigned to each survey item are com-parable across all respondents. The potential heterogeneity across respondents is “a typicalsource of variation in response data that needs to be accounted for in a statistical responsemodel (Fox, 2010, 3).” Survey items on managerial networking are extremely vulnerable tothis validity threat in that both managers’ personal traits and their organizational contextsmight induce a specific networking activity and affect its frequency. A factory index withoutaccounting for the cross respondents heterogeneity will be a biased measure, which is notcompatible across all respondents.

Parallel to the compatibility issue is the challenge of scaling different behavioral itemsinto one dimension that maps the latent traits of managerial networking. The theoreticalliterature on managerial networking does not offer much guidance on the dimensionalityof networking activities. Empirical research, moreover, produced mixed findings. Earlierstudies often treat networking as a single-factor measure, with the statistical assumptionthat different survey items share the same underlying distribution (Geletkanycz, Boyd andFinkelstein, 2001; Hicklin, Meier and O’Toole, 2008; Meier and O’Toole, 2001). A few recentstudies, nevertheless, discover the multiple dimensions of managerial networking (Agranoffand McGuire, 1999; Akkerman and Torenvlied, 2011; Torenvlied et al., 2012). Whether man-agerial networking has a single trait dimension or contains multi-dimensional traits remainsto be an unsolved puzzle.

Both the factor-analytic technique and the non-parametric cumulative scaling technique(Mokken Scale), in addition, take the explorative strategy by “letting the survey data speak,”and assume that each behavior item is correctly measured. In other words, measurementscales are produced with the assumption that managers truthfully report their activities ofmanagerial networking. Recent studies show that self-reported survey items are vulnerableto various reporting bias. Meier and O’Toole (2010) demonstrate that managers mightoverreport or underreport particular networking activities due to the social desirability ofthe survey item. As such, common resource bias is likely to occur (Donaldson and Grant-Vallone, 2002). Henry, Lubell and McCoy (2012) find that self-reported survey items arelikely to have recall bias, namely, respondents are unlikely to remember their entire list ofnetwork nodes and report accurately how much they networked with each node. These mostrecent studies, nevertheless, are long on description and short on prescription of how toquantify managerial networking, accounting for measurement bias embedded in the surveydata.

5

3 A Bayesian Item Response Approach to Measurement Bias

We propose a Bayesian Item Response approach to model ordinal survey items of managerialnetworking. Our approach builds on the classic Item Response Theory (IRT) and the cumu-lative scaling technique, but adds explicit conceptualization of errors in the measurement.Bayesian inference is then used to improve measurement reliability.

3.1 Ordinal Response Data and The IRT Approach

The IRT approach of scaling (e.g. Mokken scale) conceptualizes that the probability of aparticular response to a survey item is a function of characteristics of the respondent and thesurvey item (Hemker, Sijtsma and Molenaar, 1995). In the context of analyzing dichotomousitem responses, “the answer a person gives to a question is interpreted as a dominancerelationship between the person and the question (the item). The person dominates the itemif she gives the positive response, and the item dominates the person if she gives the negativeresponse (van Schuur, 2011, 16).” With a large number of survey items and respondents, onecan analyze four types of dyadic dominance relationships: item-to-subject, subject-to-item,item-to-item, and subject-to-subject.

Without measurement errors, this scaling approach can produce a transitive rank-orderamong all items based on the “difficulty” parameter and a transitive rank order among allitems based on the “discrimination” parameter. The latent dimension of ability (or otherlatent traits, such as effort, ideology, attitude, etc.), therefore, is inferred and scaled basedon the cumulative probability of giving a positive response idenfined by the difficulty anddiscrimination parameter.

When applying the IRT approach to ordinal responses, the model essentially becomes acumulative rating scale model or a partial credit model (PCM) (Anderson, 1997; Masters,1988; Samijima, 1969). Ordinal response items in networking studies often generate a scalebased on ordered categories. The survey question may ask a respondent using letter grades,A, B, C, and D ( ordered from low to high), to indicate the effort of his networking activities.A survey item may also ask a respondent the frequency of her networking activities, renderinga scale, ordered as “never”, “yearly”, “monthly”, “weekly”, “daily”, and “more than onceper day”. Attitude items, in addition, normally produce an ordered scale ranked by “stronglydisagree”, “disagree”, “agree”, and “strongly agree”. Compared to the dichotomous surveyitems, ordinal items, or more generally, polytomous items increase the statistical information(Ostini and Nering, 2006) that researchers can use to infer managerial ability or effort ofnetworking.

Let the latent dimension of networking effort for i managers be denoted by θi, one canuse j items to denote different networking activities. The j survey items then will producea stack of item-subject matrixes, illustrated as Figure 1. As Figure 1 shows, for each surveyitem, respondents report their choices among the 6 categories, which will produce a densitydistribution based on all the reported choices for that particular item. For Item 1, the most

6

likely choice is 6, and for Item j, the most likely choice is 3.

[Figure 1 About Here]

One can then use the stack of survey items to estimate the latent effort of managerialnetworking. Specifically, for a survey item j (networking node), the probability of a manageri to report a specific response k in the ordinal k-choice category k = 1, 2, 3.. is defined by thedifference between the probability of responding in (or above) choice k and the probabilityof responding in (or above) the next choice, k + 1 (Fox, 2010, 13).

P (Yi,j = k|θi, aj, bj) = P (Yi,j ≥ k − 1|θi, aj, bj)− P (Yi,j ≥ k|θi, aj, bj) (1)

The cumulative probability of choosing the lowest category and above is assumed to be1 and the probability of choosing above the highest category is 0. As for a k-choice orderedscale, there are (k − 1) cut points to define the ordinal scale. Hence, equation (1) specifiesthe two probability values with respect to cut point k − 1, and k. In equation (1), aj andbj are the discrimination and difficulty parameter, respectively. θi is the latent dimension atinterest. The two probabilities in equation (1), further, can mathematically be representedby the logistic cumulative distribution function (Bradlow and Zaslavsky, 1999; Fox, 2010;Fumiko, 2008).

P (Yi,j = k|θi, aj, bj) = P (Yi,j ≥ k − 1|θi, aj, bj)− P (Yi,j ≥ k|θi, aj, bj)

=exp(aj(θi − bj,k−1))

1 + exp(aj(θi − bj,k−1))− exp(aj(θi − bj,k))

1 + exp(aj(θi − bj,k))

(2)

3.2 Considering Measurement Bias: A Bayesian Approach

Reporting bias and the issue of cross item compatibility can both change the perfect rank-order estimated based on the difficulty and discrimination parameter. The item responsetheory conceptualizes measurement bias as the violation of a transitive relationship betweenitems and subjects. Bias, counted as “reporting errors” (or “wrong answers” ), thus canbe estimated through evaluating the homogeneity of individual items based on pairwiseinformation extracted from each item pairs (van Schuur, 2011). Figure 2 illustrates howthe pairwise association may look like between two ordinal survey items. In Figure 2, bothitems are coded based on a 1-to-6 ordinal scale, and each circle indicates the frequency ofall possible pairwise values, whereby a large circle refers to high frequency and small circlerefers to low frequency. Ideally, if Item I and 2 are created to measure the same latentconstruct, the association between the two items should only be caused by their commonassociations with the latent factor. The pattern shown in Figure 2 obviously demonstrates alot of “noisy” information. Among all 36 possible pairwise values between the two items, 4pairs (16, 25, 32 and 62)are not observed. The concentration of the 16 pairwise values, when

7

both Item 1 and Item 2 take values greater than 2, clearly demonstrate the deterministicrelationship between the two items.


The challenge to map the latent trait based on data variations generated by survey itemsis that we do not know the true measurement score of the latent construct. When observing“noisy” data as shown in Figure 2, it is difficult to handle the uncertainty associated withmeasurement errors. The Bayesian IRT approach becomes particularly appealing to handlesuch measurement issue, because neither the latent traits, nor the item parameters need tobe treated as deterministic.

Building on the likelihood function expressed by equation (2), we add a random error-term εjto represent measurement bias. As such, the probability of manager i reporting aparticular choice kj for survey item j can be expressed as a cumulative logistic density belowthe the latent cut point τk,j associated with choice kj.

P [Yij = kj] = P [ajθi − bj + εj ≤ kj]

= P [ajθi − bj ≤ (kj − εj)]= 1− Z(τk,j − F (θi, aj, bj))

(3)

To simplify the mathematic notation, in equation (3), we use Z(·) to denote the cumulativelogistic density function, and F (θi, aj, bj) to denote the linear link function for Yij definedby latent trait θi, the discrimination parameter aj, the difficulty parameter bj. τk,j is theestimated cut point specific to the “grade” value k, and item j. Reporting bias εj, inadditional, also determines the actual estimated value for τk,j.

In the context of estimating a full Bayesian model, all the parameters in F (·) are es-timated quantities, thus carry measurement uncertainty in parameters. Specifically, thelatent dimension of managerial networking and the the characteristics of survey items areall unknown and we are interested in learning (inferring) their densities jointly based on theassumed likelihood function, the prior information on their distribution, and the observedresponse data. Using π(θ, β) to denote the joint posterior density that we want to infer,whereby θ refers to the latent trait of managerial networking, and β refers to the vector ofdifficulty parameters, and α refers to the vector of discrimination parameters. The marginalposterior density is learned based on using observed data to update prior assumptions abouthow these unknown parameters are distributed. Formally, the posterior density is given as:

π(θ, α, β|Y ) ∝ L(Y |θ, α, β)p(θ)p(α)p(β) (4)

Using the Bayesian approach, one not only can recover the point estimation (e.g. posteriormeans ) of managerial networking (θi), but also can obtain the confidence intervals of themeasurement score (for example, see Treier and Jackman 2008). In the next section, we fully

8

specify each component of the proposed Bayesian IRT model using empirical survey itemson professional health care organization.

4 An Empirical Application: Managerial Networking in American Hospitals

4.1 Data

The empirical data we use to illustrate the Bayesian IRT approach to measurement are col-lected from a mail survey of 6,000+ professional health care organizations, including generalhospitals, specialized hospitals, mental health clinics, childrens’ hospitals, university-ownedmedical centers, rehabilitation centers, and acute long-term care organizations. The surveywas on field in the spring of 2011 and generated 1026 responses, rendering an overall responserate of 16.14%. Although the overall response rate is not high, it is not surprisingly low com-pared to other large-scale surveys on American hospitals.1 We perform a logistic regressionanalysis to check if response rates vary substantially by organizational type, region, state,service type, and organizational size (see Table 1). Our sample does not vary substantiallyalong all these indicators excepting organizational type.2 Table 2 reports survey responserates by different organizational types. In our sample, there are 343 government-ownedhospitals(primarily non-federal hospitals), 483 non-profits, and 195 for-profits.

[Table 1 About Here]


The survey uses a name-roster method, providing top-level hospital administrators alist of 25 networking nodes, whereby 6 are internal to the organization and 19 are externalto the organization. Table 3 reports summary statistics of the full list of 25 networkingnodes. We use the 11 nodes (see Figure3) for external networking patterers to estimate theBayesian measure for managerial networking. Each survey item is measured by a 1-to-6ordered scale ranking the frequency of interactions between a manager and her networkingpartners. The order scale is defined as: 1=never, 2=yearly, 3=monthly, 4=weekly, 5=more

1For example, the Agency for Healthcare Research and Quality (AHRQ under the Department of Healthand Human Services) released the 2012 Hospital Survey on Patient Safety Culture among American Hospitals,and data were reported from 1128 hospitals, which covers a comparable proportion of hospitals to our sample.As for all surveyed hospital staff respondents, the participation rates based on two major areas are relativelylow (surgery-10% and Medicine: 12%) (see, at http://www.ahrq.gov/qual/hospsurvey12/).

2Although service type and organizational size are significantly correlate with the survey response rate,the coefficients are not very large. Organizational type, however, is a strong predictor of the response rate.Because we only use the survey items to demonstrate the computation of our proposed Bayesian measureand then compare the Bayesian measure to the conventional factor analysis, the two measures are baring thesame sample errors. If the survey data were used to analyzing the relationship between managerial variablesand organizational performance, one needs to consider weighting survey responses or controlling for differentorganizational types.

9

than once a week, and 6=daily. We did not use all 19 nodes of external networking primarilydue to the missing data in a few nodes, such as Volunteer Board, Hospital Foundation,Union Representatives, etc. All the selected 11 nodes are important political stake-holdersto hospitals. Substantively, we focus on the 11 external nodes to construct a measure ofexternal organizational networking. This approach is analogous to that used by Meier andO’Toole, Agranoff and McGuire, and others in defining managerial networking as an activitythat crosses organizational boundaries.



4.2 Prior Specification and Estimation

We use the R and JAGS program to implement the Bayesian ordinal IRT model. We specifyuninformative priors for aj, and bj,k. Because the latent dimension of managerial networking,θi and the discrimination parameterαj, and the difficulty parameter betaj,k are simultaneouslyestimated, we set αj to be truncated at 0 and constrain the mean of thetai to be 0 for thepurpose of identification (Bafumi et al., 2005; Curtis, 2010; Fox, 2010). Prior distributionsare specified as following:

α ∼ N(0, 1.4)

β ∼ N(0, 6.25)

θ ∼ N(µ, σ2)

(5)

We do not set θi to be a standard normal distribution, because we want to infer varianceintroduced by reporting errors. Hence, we specify the hyper priors for θi to have a mean of0 and precision of 0.01. The precision parameter is defined as a gamma distribution, withshape=0.5 and rate=0.5. In other words, we fixed the mean value of managerial networkingto be 0, and use the estimated variance to capture uncertainty in the latent dimension. TheBayesian IRT model is estimated by running 100,000 iterations of Markov Chain MonteCarlo (MCMC) simulation.3 To initialize the model, we use JAGS to randomly generateinitial values to all the parameters. Model results are reported based on burning the first10,000 iterations, and restoring every 10th sampled values for αj, βj,k, µ, and σ2. Postestimation diagnostic analysis did not find significant evidence for non-convergence (see thesample trace-plots and density plots in Figure 4).4

3We firstly run the MCMC simulation for 50,000 iterations. Model convergence, however, is not ideal, dueto high serial autocorrelations in parameters related to some networking nodes. Hence, we run an additional50,000 iterations and the model convergence has been substantially improved.

4In total, our model produce ten α parameters indexed by each networking node (j), and (j − 1) × k(9 × 6 = 54) β parameters because we constrain βj,1 to be 0. Figure 4 illustrates the density plots of thediscrimination parameters associated with first 3 networking nodes (State Medicaid, State Medicare, and

10


4.3 Comparing Factor Analysis and the Bayesian IRT Approach

The proposed Bayesian IRT model (Generalized Partial Credit Model) returns a full set ofdiscrimination parameters, difficulty parameters, and recovers the marginal posterior dis-tribution of the latent dimension, θi. Table 4 summarizes the posterior means of the 10discrimination parameters and latent dimension of managerial networking. To compare theBayesian IRT estimation with the factor analytic model and the non-Bayesian IRT model,we perform factor analysis and non-Bayesian item response analysis and report the resultsin Table 5.5

Parameter Reliability. Overall, our IRT model produces a reliable estimation of itemdiscrimination parameters. Comparing the posterior means and medians of the 10 networkingnodes in Table 4, all 8 networking nodes, excepting Insurance Companies and NationalHospital Association have very large values of the discrimination parameter. Although themean α coefficients for Insurance Companies and Local Hospital Association are relativelysmaller than all the other 8 networking nodes, cross-item heterogeneity is reasonably small.

Comparing across the three measurement models, factor analysis seems to be least ap-pealing. Firstly, looking at the factor loading results shown in Table 5, the ten survey itemsreturn 4 factors with ss loading values larger than 1. Checking factor loadings associatedwith each item and their uniqueness parameters, we see a serious problem raised by intra-item heterogeneity. Not surprisingly, the assumption of local independence is violated whenusing factor analysis to predict the networking measure only based on the first factor. Fig-ure 5 further illustrates the problem of local dependence, by plotting all the other 4 factorsagainst factor1 based on the specific factor loading scores associated with each networkingitem. Secondly, the factor-analytic model is very sensitive to the number of estimated fac-tors. Using the 10 survey items, we find that the estimation results of the first factor wouldchange substantially when dropping the number of predicted factors from 5 to 4. Accordingto the results in Table 5, when estimating 5 factors with the 10 networking item, Local Hos-pital Officials, Civic Group and State Hospital Association are not loaded on Factor1. Whenestimating 4 factors, however, State Hospital Association is the only item that does not loadon Factor 1. Factor loading values for each item, furthermore, change substantially fromthe results in Table 5. For example, the factor loading for State Regulatory Agency wouldchange from 0.345 to 0.378, and factor loadings for both StateMedicaid and StateMedicarewould drop below 0.3.


Social Service Agency). The sub-figure on the right-hand side of Figure 4 illustrates the difficulty parameterspecific to item 4 − 6, and choice 2. Because we define αj to be truncated by 0, simulated values for α4−6are always positive.

5In Table 5, factor analysis is performed using the factanal() in the R package stats and the non-BayesianIRT scaling is estimated using the gpcm() in the R package, ltm(Rizopoulos, 2006).

11

The latent dimension of managerial networking, in addition, can be inferred from theposterior distribution of θi (see Figure 6). Our Bayesian analysis returns the values for θiwith a mean of 0.0 94 and a standard deviation of 9.975. The median value of the managerialnetworking scales is 0.037. Comparing the distribution plot of θi estimated based on theBayesian model and the IIC (Item Information Curve ) as well as the Test InformationFunction based on the non-Bayesian IRT analysis, we see that our Bayesian measure reducesthe change in measurement scores dependent to the survey items.


Intra-Item/ Intra-Choice Compatibility. Further comparing the Bayesian IRT and non-Bayesian IRT scaling, we find that the two measurement approaches report comparable re-sults, but the Bayesian IRT approach is superior with respect to reducing the cross item/crosschoice compatibility issue. Our survey data show some cross item/choice compatibility is-sues. A few networking items have very skewed distributions toward the low-frequency end.The last choice category, “Daily”, is somehow problematic, because very a few respondentschecked this category for some networking items. For example, based on our survey sample,less than 1% hospital managers interact with local/state public officials and state regulatoryagency on daily basis.

The issues of intra-item and intra-choice compatibility leads to less reliable parameterestimation when fitting a non-Bayesian generalized particle credit model. Figure 7 and 8illiterates item information curve (IIC) and item characteristic curve (ICC) by each choicecategory. Both plots are generated after fitting the GPCM reported in Table 5. Overall,the non-Bayesian IRT model recovers 86.6% information in the estimated latent dimensionof networking. The estimated scale, however, is skewed toward the high-end (see subfigure(2-2), the Test Information Curve in Figure 7 ). The estimated latent scale recovers 65.81%information for values between 0 and 4 (high level of networking), and only 20.8% informationis recovered for values between -4 and 0 (low-level of networking). Comparing the by-itemand overall IIC with the posterior mean distribution inferred from the Bayesian IRT model,the measurement scale returned by the Bayesian analysis covers to a normal distribution,whereby comparable information is measured for values below and above 0.

The non-Bayesian IRT scale is also affected by cross-item heterogeneity with respectto the discriminative power of each item. Item 1 (Local Political Officials) and Item 9(State Hospital Association) has the highest discriminative power, with limited amount ofinformation, these two items seem to distinguish between managers’ level of networking.Item 3 (State Regulatory Agency), 4(State Medicaid), 5(State Medicare), and 8 (InsuranceCompanies) distinguish less among managers’ level of networking. Comparing the rankingof item-specific discriminative power with the discriminative parameters estimated by theBayesian model, the Bayesian and non-Bayesian IRT produce comparable results on item 1and 8, but produce slightly different results on item 1, 4, 5, and 8. Survey item 1,4,5,and 8are all highly skewed toward low level of networking and the number of observations are quitelimited on the high-networking values.The Bayesian IRT model improves performance of the

12

non-Bayesian IRT model regarding the information learned from these items via MCMCsimulations.

Last, but not least, Figure 8 further illustrates measurement issues caused by cross-category compatibility. The ordinal survey responses are arbitrarily divided into 6 choicecategories. Although the overall ranking is consistent across all items, respondents maynot agree on the relative rankings among the 6 choices. Figure 8 shows a clear pattern oflimited intra-item agreement on the 4 middle choice categories. Item 4 and 5 (State Medicare&Medicaid) seem to stand separately from all the other items and these two items fit witheach other very well. Item 3 (State Regulatory Agency) stands closer to item 4 and 5 andseems to be distinguishable from all the other 7 networking nodes. Also our Bayesian modeldoes not completely clear this problem, the intra-choice heterogeneity is reduced from thenon-Bayesian IRT to a full-Bayesian IRT model if we compare βj,k and the outputs of theIRT model (in Table 5).

5 Discussion

Our primary interest is to provide an alternative to the factor analytic techniques now com-mon in public administration research. Factor analytic techniques provide powerful toolsfor addressing the measurement problems present in key areas of public administration re-search but bring with them a number of constraining assumptions (often ignored or otherwiseconcealed from readers). As an alternative, we propose a Bayesian IRT approach to mea-surement where these assumptions are transparent and chosen by the researcher rather thanimposed by the technique.

The requirement of specifying assumptions is particularly useful where measure theoryand conceptions of dimensionality are limited in the literature. The unsettled nature ofmeasurement in relation to collaborative networking is but one example. One could also finduse for this approach in other areas with unsettled measurement such as red tape, publicservice motivation, and organizational performance.

The Bayesian approach to measurement is not, though, a panacea. The flexibility ofBayesian IRT models allow researchers to develop poorly suited models even as they allowresearchers to use more appropriate models. Furthermore, Bayesian IRT models can notthemselves overcome the limitations of the input data. If the input data includes significantmissing data or a small sample (limiting statistical power), strong skew (reducing variationand again limiting statistical power), or unreliable questions (resulting in either high varianceor systematic bias in the responses), the Bayesian IRT approach will only allow one toincorporate these problems but will not produce simple or useful measures. The advantagein these cases is that Bayesian IRT makes these problems transparent rather than concealingtheir presence behind the assumption that resulting factors will always have a mean zeroand a standard deviation of one.

Of course, this approach is only the beginning of the use of Bayesian IRT in public

13

administration research. We anticipate extending this analysis to including a discussion ofmissing data in measurement models and the intersection of imputation and IRT models.Furthermore, we would like to compare the measurement of external networking with theallocation of time connecting to internal nodes to assess the degree (or existence) of a trade-off between networking and internal management. Finally, we would like to compare variousmeasurement approaches in the context of performance models. Does one measurementapproach create measures that better explain organizational performance? What does thistell us about the nature of organizational performance and the survey reporting of suchperformance.

The field of public administration has not paid a great deal of attention to issues ofmeasurement. With the proliferation of quantitative research and its extension into areasof study where existing, validated measures do not exist in other fields, it is important forpublic administration researchers to take measurement seriously. As this report illustrates,Bayesian IRT approaches provide an alternative to carefully assess measures and create whatmay turn out to be more effective elements for future research.

14

Table 1: Logistic Regression Predicting Survey Responses (Dependent Variable: Re-sponse=1, No-Response=0)

Variable Coefficient (Std. Err.)Service Type -0.005∗ (0.002)State 0.013 (0.017)Region -0.069 (0.164)Public Ownership 0.268∗∗ (0.088)Organizational Size 0.0003∗∗ (0.00005)Intercept -1.773∗∗ (0.128)

N 6351Log-likelihood -2752.537χ2(5) 95.638

Significance levels : † : 10% ∗ : 5% ∗∗ : 1%

15

Table 2: Survey Response Rates by Organizational Type

Owned By AHA Code No Response Response Total Rate(%)State 12 278 57 335 17.01County 13 300 90 390 23.08City 14 75 31 106 29.25City-County 15 28 5 33 15.15Hospital District 16 409 139 548 25.36State/Local Govt. Overall 1090 322 1412 22.80Church 21 480 75 555 13.51Other 23 2182 408 2590 15.76Non-Profit Overall 2661 483 3144 15.36Investor 30 2 0 2 0.00Individual 31 21 5 26 19.23Partnership 32 226 50 276 18.12Corporation 33 1137 140 1277 10.96Private Overall 1386 195 1581 12.33Air Force 41 9 1 10 10.00Army 42 21 2 23 8.70Navy 43 11 2 13 15.38Public Health Service (PHS) 44 9 0 9 0.00Veteran Service 45 120 15 135 11.11PHS Indian Service 46 21 1 22 4.55DOJ 48 2 0 2 0.00Federal Overall 193 21 214 9.81Unidentified NA NA 4 4 NA NATotal 5331 1025 6356 16.13

16

Table 3: Summary Statistics of All Networking Nodes (Listed by the Frequency of Interac-tions)

Networking Node Obs. Mean SD MedianInternal NodeOther Hospital Staff 1006 5.27 1.15 6Hospital Corporate Office 614 4.74 1.18 5Medical Staff 1026 4.69 1.30 5Patients 1009 4.82 1.10 5Hospital Executive Committee 914 4.01 1.20 4Hospital Dept. Management Group 983 4.09 1.20 4External NodeHospital Board of Trustees 959 3.42 0.76 3Volunteer Board 582 2.98 0.88 3Hospital Foundation 578 3.13 0.76 3Local Public Officials 1000 3.05 0.85 3Civic Groups 985 3.16 0.92 3State Hospital Association 992 3.20 0.93 3State Public Officials 994 2.52 0.81 2State Regulatory Agencies 991 2.39 0.73 2State Medicaid 912 2.09 1.03 2State Medicare 913 2.05 1.05 2Accrediting 946 2.17 0.64 2Social Service Agencies 911 2.33 1.08 2Insurance Companies 949 2.40 1.19 2IT vendors 912 2.19 1.12 2Other Vendors 913 2.47 1.12 2National Hospital Association 967 2.43 1.00 2Other Professional Associations 905 2.69 0.92 2Pharmaceutical Companies 880 1.64 1.10 1Union Representatives 586 1.71 1.22 1

17

Table 4: The Discrimination (αj) and Difficulty Parameter (βj,k): Summary of PosteriorDistributions Based on the Generalized Partial Credit Model

αj

Networking Node Mean Median SD Naive SE TS-SeLocal Political Officials 1.064 1.004 0.346 0.003 0.027State Political Officials 0.901 0.833 0.332 0.003 0.023State Regulatory Agency 0.961 0.887 0.351 0.004 0.029State Medicaid 1.001 0.938 0.354 0.004 0.016State Medicare 1.03 0.958 0.335 0.003 0.012Social Service Agency 0.964 0.890 0.339 0.003 0.010Civic Groups 0.864 0.810 0.299 0.003 0.023Insurance Company 0.781 0.720 0.296 0.003 0.010State Hospital Association 1.017 0.956 0.341 0.003 0.028National Hospital Association 0.698 0.635 0.276 0.003 0.012βj,k(Posterior Mean) [j,2] [j,3] [j,4] [j,5] [j,6]Networking Node Yearly Monthly Weekly > Weekly DailyLocal Political Officials -2.328 -3.362 -2.416 -0.733 2.013State Political Officials -2.875 -2.589 0.205 0.800 3.304State Regulatory Agency -3.287 -2.433 0.292 0.485 2.830State Medicaid -0.522 0.622 2.590 3.176 3.078State Medicare -0.228 0.807 2.360 3.458 3.162Social Service Agency -0.431 -0.049 1.457 3.170 3.815Civic Groups -2.137 -3.484 -2.901 -0.801 1.235Insurance Company -0.486 -0.170 1.859 3.000 3.649State Hospital Association -2.230 -3.574 -2.804 -1.478 -0.076National Hospital Association -2.050 -1.220 0.871 2.151 3.781

18

Table 5: Managerial Networking: Factor Analysis and Non-Bayesian IRT Analysis (Gener-alized Partial Credit Model)

Networking Node Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Uniq.Factor AnalysisLocal Political Officials – 0.216 0.114 0.954 – 0.022State Political Officials 0.155 0.621 0.151 0.321 0.118 0.451State Regulatory Agency 0.345 0.776 0.123 0.117 0.102 0.240State Medicaid 0.699 0.393 0.180 – 0.113 0.305State Medicare 0.917 0.168 0.143 – – 0.103Social Service Agency 0.448 0.180 – 0.171 0.547 0.434Civic Groups – 0.125 0.195 0.406 0.277 0.699Insurance Companies 0.599 0.127 0.112 – 0.253 0.544State Hospital Association – – 0.932 0.146 – 0.096National Hospital Association 0.282 0.219 0.526 0.128 0.143 0.559SS Loadings 2.133 1.336 1.304 1.275 0.499Proportion Var 0.213 0.134 0.130 0.128 0.050Cumulative Var 0.213 0.347 0.477 0.605 0.655Ordinal IRTCoefficients Cat.1 Cat.2 Cat.3 Cat.4 Cat.5 Discr.Local Political Officials -4.587 -1.916 1.706 3.417 5.822 0.612State Political Official -3.260 0.252 3.178 1.934 4.681 0.878State Regulatory Agency -2.828 0.628 2.601 1.891 4.099 1.384State Medicaid -0.610 0.892 1.953 2.043 2.364 2.371State Medicare -0.455 0.979 1.938 2.348 2.186 1.730Social Service Agency -0.840 0.547 2.317 3.210 2.780 0.806Civic Groups -4.771 -2.733 1.173 4.352 4.704 0.423Insurance Companies -0.814 0.355 2.314 2.148 2.388 0.828State Hospital Association -5.468 -2.750 1.571 3.072 3.694 0.475National Hospital Association -2.437 0.836 2.564 2.303 3.398 0.623

19

Figure 1: An Illustration of Ordinal Subject-Item Response Matrix

Subject Item1 1 6 2 2 3 6 4 3 5 6 6 6 7 6 … … n 6

Subject Item 2 1 5 2 2 3 5 4 5 5 5 6 5 7 1 … … n 4 … … Subject Item j 1 3 2 2 3 3 4 3 5 3 6 1 7 1 … … n 4

050

100

150

200

250

300

350

400

Freq

uenc

y

1 2 3 4 5 6Item 1

050

100

150

200

250

300

350

400

Freq

uenc

y

1 2 3 4 5 6Item 2

010

020

030

040

050

060

0Fr

eque

ncy

1 2 3 4 5 6Item j

20

Figure 2: An Illustration of the Association between an Item Pair

12

34

56

Item

2

1 2 3 4 5 6Item 1

21

Figure 3: Frequencies of Managerial Networking with Ten External Partners(1=Never...6=Daily)

Loca

l Pub

lic O

fficial

s

Frequency

12

34

56

0100200300

Stat

e Pu

blic O

fficial

s

Frequency

12

34

56

050100150200250300350

Stat

e Re

gulat

ory A

genc

ies

Frequency

12

34

56

0100200300400

Stat

e M

edica

id

Frequency

12

34

56

050100150200250300

Stat

e M

edica

re

Frequency

12

34

56

050100150200250300

Socia

l Ser

vice

Agen

cy

Frequency

12

34

56

050100150200250

Civic

Gro

ups

Frequency

12

34

56

050100150200250300

Insu

ranc

e Co

mpa

nies

Frequency

12

34

56

050100150200

Stat

e Ho

spita

l Ass

ociat

ion

Frequency1

23

45

6050100150200250300350

Natio

nal H

ospit

al As

socia

tion

Frequency

12

34

56

050100150200250300

22

Figure 4: Trace Plots of Selected Parameters

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05

0.5

3.5

Iterations

Trace of alpha[4]

1 2 3 4

0.0

1.2

N = 10000 Bandwidth = 0.05315

Density of alpha[4]

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05

0.5

3.0

Iterations

Trace of alpha[5]

0.5 1.0 1.5 2.0 2.5 3.0

0.0

1.2

N = 10000 Bandwidth = 0.05224

Density of alpha[5]

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05

0.5

3.0

Iterations

Trace of alpha[6]

0.5 1.0 1.5 2.0 2.5 3.0 3.5

0.0

1.5

N = 10000 Bandwidth = 0.0486

Density of alpha[6]

0e+00 2e+04 4e+04 6e+04 8e+04 1e+050.5

Iterations

Trace of beta[4,3]

0.0 0.5 1.0 1.5

0.0

1.5

N = 10000 Bandwidth = 0.03749

Density of beta[4,3]

0e+00 2e+04 4e+04 6e+04 8e+04 1e+05

0.5

2.0

Iterations

Trace of beta[5,3]

0.0 0.5 1.0 1.5 2.0

0.0

1.5

N = 10000 Bandwidth = 0.04376


0e+00 2e+04 4e+04 6e+04 8e+04 1e+05

-0.5

Iterations

Trace of beta[6,3]

-0.5 0.0 0.5

0.0

3.0

N = 10000 Bandwidth = 0.01936


23

Figure 5: Maximum Likelihood Factor Analysis: Comparison of the First Five Factors

0.2 0.4 0.6 0.8

0.1

0.3

0.5

0.7

Factor 1

Fact

or 2

0.2 0.4 0.6 0.8

0.2

0.4

0.6

0.8

Factor 1

Fact

or 3

0.2 0.4 0.6 0.8

0.2

0.4

0.6

0.8

Factor 1

Fact

or 4

0.2 0.4 0.6 0.8

0.0

0.2

0.4

Factor 1

Fact

or 5

24

Figure 6: MCMC Sample of θi: Trace Plot, Autocorrelation Plot, and Density Plot

0e+00 4e+04 8e+04

-40

-20

020

40

Iterations

-40 -20 0 20 40

0.00

0.02

0.04

N = 10000 Bandwidth = 1.677

0 100 200 300 400

-1.0

-0.5

0.00.5

1.0

Lag

Autocorrelation

25

Figure 7: The Item Information Characteristic Curves based on the non-Bayesian PartialCredit Model (V1= Local Political Officials, V2=State Political Officials, V3=State Regulatory Agency,V4=State Medicaid, V5=State Medicare, V6=Social Service Agency, V7=Civic Groups, V8=Insurance Com-panies, V9=State Hospital Association, V10=National Hospital Association)

-4 -2 0 2 4

0.5

1.0

1.5

Item Information Curves

Latent Dimension of Networking

Information

V1V2V3

-4 -2 0 2 4

01

23

45

6



Information

V4V5V6

-4 -2 0 2 4

0.2

0.4

0.6

0.8

1.0



Information

V7V8V9V10

-4 -2 0 2 4

24

68

1014

Test Information Function


Information

26

Figure 8: The Item Category Characteristic Curves based on the non-Bayesian PartialCredit Model (V1= Local Political Officials, V2=State Political Officials, V3=State Regulatory Agency,V4=State Medicaid, V5=State Medicare, V6=Social Service Agency, V7=Civic Groups, V8=Insurance Com-panies, V9=State Hospital Association, V10=National Hospital Association, Category 1= Never, Category2=Yearly, Category 3=Monthly, Category 4= Weekly, Category 5=More than Weekly, Category 6=Daily )

-4 -2 0 2 4

0.0

0.2

0.4

0.6

0.8

1.0

Item Response Category Characteristic Curves Category: 1


Probability

V1V2V3V4V5V6V7V8V9V10

-4 -2 0 2 4

0.0

0.2

0.4

0.6

0.8

1.0



Probability

-4 -2 0 2 40.0

0.2

0.4

0.6

0.8

1.0



Probability

-4 -2 0 2 4

0.0

0.2

0.4

0.6

0.8

1.0



Probability

-4 -2 0 2 4

0.0

0.2

0.4

0.6

0.8

1.0



Probability

-4 -2 0 2 4

0.0

0.2

0.4

0.6

0.8

1.0



Probability

27

References

Agranoff, Robert and Michael McGuire. 1999. “Managing in Network Settings.” PublicAdministration Review 16(1):18–41.

Agranoff, Robert and Michael McGuire. 2001. “Big Questions in Public Management Re-search.” Journal of Public Administration Research and Theory 11(3):295–326.

Agranoff, Robert and Michael McGuire. 2003. Collaborative Public Management: NewStrategies for Local Government. Washington, D.C.: Georgetown University Press.

Akkerman, Agnes and Rene Torenvlied. 2011. “Managing the Environment.” Public Admin-istration Review 13(1):159–74.

Anderson, Erling B. 1997. The Rating Scale Model. In Handbook of Modern Item ResponseTheory, ed. Wim J. van der Linden and Ronald K. Hambleton. New York: Springerpp. 67–84.

Andrew, Rhys, George A. Boyne, Kenneth J. Meier, Laurence J. O’Toole and Richard M.Walker. 2011. “Environmental and Organizational Determinants of External Networking.”The American Review of Public Administration 41(4):355–74.

Bafumi, Joseph, Andrew Gelman, David K. Park and Noah Kaplan. 2005. “Practical Issuesin Implementing and Understanding Bayesian Ideal Point Estimation.” Political Analysis13(2):171–87.

Bradlow, Eric T. and Alan M. Zaslavsky. 1999. “A Hierarchical Latent Variable Model forOrdinal Data from a Customer Satisfaction Survey with “No Answer’ Reponse.” Journalof American Statistical Association 94(445):43–52.

Brady, Henry E. 1985. “The Perils of Survey Research: Inter-Personally IncomparableResponses.” Political Methodology 11(3/4):269–291.

Curtis, S. McKay. 2010. “BUGS Code for Item Response Theory.” Journal of StatisticalSoftware 36(1):Code Snippet 1.

Donaldson, Stewart I. and Elisa J. Grant-Vallone. 2002. “Understanding Self-Report Bias inOrganizational Behavior Research.” Journal of Business and Psychology 17(2):245–260.

Doty, D. Harold and William H. Glick. 1998. “Common Methods Bias: Does CommonMethods Variance Really Bias Results?” Organizational Research Methods 1(4):374–406.

Embretson, Susan E. and Steven P. Reise. 2000. Item Response Theory for Psychoogists.Mahwah, New Jersey: Lawrence Erlbaum Association, Inc.

Fox, Jean-Paul. 2005. “Randomize Item Response Theory Models.” Journal of Educationaland Behavioral Statistics 30(2):189–212.

28

Fox, Jean-Paul. 2010. Bayesian Item Response Modeling: Theory and Applications. NewYork: Springer.

Fumiko, Samejima. 2008. “Graded Response Model Based on the Logistic Positive ExponentFamily of Models for Dichotomous Responses.” Psychometrika 73(4):561–78.

Geletkanycz, Marta A., Brian K. Boyd and Sidney Finkelstein. 2001. “The Strategic Valueof CEO External Directorate Networks: Implications for CEO Compensation.” StrategicManagement Journal 22(9):889–98.

Graham, John W. and Nancy L. Collins. 1991. “Controlling Correlational Bias via Confir-matory Factor Analysis of MTMM.” Multivariate Behavioral Research 26(4):607–629.

Hemker, Bas T., Klass Sijtsma and Ivo W. Molenaar. 1995. “Selection of UnidimensionalScales From a Multidimensional Item Bank in Polytomous Mokken IRT Model.” AppliedPsychological Measurement 19(4):337–352.

Henry, Adam Douglas, Mark Lubell and Michael McCoy. 2012. “Survey-Based Measurementof Public Management and Policy Networks.” Journal of Policy Analysis and Management31(2):433–52.

Hicklin, Alisa K., Kenneth J. Meier and Jr. O’Toole, Laurence J. 2008. “Serpents in theSand: Managerial Networking and Nonlinear Influences on Organizational Performance.”Journal of Public Administration Research and Theory 18(2):253–74.

Isett, Kimberley R., Ines A. Mergel, Kelly LeRoux, Pamela A. Mischen and R. Karl Rethe-meyer. 2011. “Networks in Public Administration Scholarship: Understanding Where WeAre and Where We Need to Go.” Journal of Public Administration Research and Theory21(suppl.):i157–i173.

Jackman, Simon. 2009. Bayesian Analysis for the Social Science. West Sussex, UK: JohnWiley and Sons, Ltd.

King, Gary, Christopher J.L. Murry, Joshua A. Salomon and Ajay Tandon. 2004. “Enhanc-ing the Validity and Cross-Cultural Comparability of Measurement in Survey Research.”American Political Science Review 98(1):191–207.

King, Gary and Jonathan Wand. 2007. “Comparing Incomparable Survey Responses: NewTools for Anchoring Vignettes.” Political Analysis 15(1):46–66.

Masters, Geoggerey N. 1988. “Analysis of Partial Credit Scoring.” Applied Measurement inEducation 1(4):279–297.

McGuire, Michael. 2001. “Managing Networks: Propositions on What Managers Do andWhy They Do It.” Public Adminsitration Review 62(5):599–609.

29

Meier, Kenneth J. and Jr. O’Toole, Laurence J. 2001. “Managerial Stratigies and Bebaviorsin Networks: A Meodel with Evidence from U.S. Public Education.” Journal of PublicAdministration Research and Theory 11(3):271–94.

Meier, Kenneth J. and Jr. O’Toole, Laurence J. 2005. “Managerial Networking: Issues ofMeasurement and Research Design.” Adminsitration and Society 37(4):523–41.

Meier, Kenneth J. and Jr. O’Toole, Laurnece J. 2010. “Organizational Performance: Mea-surement Theory and An Application: Or, Common Source Bias, the Achilles Heel ofPublic Management Research.” APSA 2010 Annual Meeting Paper. Available at SSRN:http://ssrn.com/abstract=1642740.

Mokken, Robert J. 1971. A Theory and Procedure of Scale Analysis with Applications inPolitical Research. New York: De Gruyte.

Ostini, Remo and Michael L. Nering. 2006. Polytomous Item Response Theory Models.Thousand Oaks, CA: Sage Publication.

O’Toole, Laurence J., Jr., Alisa Hicklin, Scott Robinson and Kenneth J. Meier. 2006. “Calm-ing the Storms: Collaborative Public Management, Hurrican Katrina and Rita, and Dis-aster Responses.” Presented at the Research Conference on Collaborative Public Manage-ment, Washington D.C.,September 28-30.

O’Toole, Laurence J., Jr. and Kenneth J. Meier. 2011. Public Management: Organizations,Governance, and Performance. New York: Cambridge University Press.

Podsakoff, Philip M. and Dennis W. Organ. 1986. “Self-Reports in Organizational Research:Problems and Prospects.” Journal of Management 12(4):533–543.

Rizopoulos, Dimitris. 2006. “ltm: An R Package for Latent Variable Modeling and ItemResponse Theory Analysis.” Journal of Statistical Software 17(5):1–25.

Robinson, Scott. 2006. “A Decade of Treating Networks Seriously.” Policy Studies Journal34(4):589–98.

Samijima, Fumiko. 1969. “Estimation of Latent Ability Using a Response Pattern of GradedScores.” Psychometrika Monograph Supplement p. No.17.

Schwarz, Norbert. 1999. “Self-Reports: How the Questions Shapre the Answers.” AmericanPsychologist 54(2):93–105.

Thomson, Ann M., James L. Perry and Theodore K. Miller. 2009. “Conceptualizing and Mea-suring Collaboration.” Journal of Public Administration Research and Theory 19(1):23–56.

Torenvlied, Rene, , Agnes Akkerman, Kenneth J. Meier and Laurence J. O’Toole.2012. “Multiple Dimensions of Managerial Networking.” Advanced access, DOI:10.1177/0275074012440497.

30

Torenvlied, Rene and Agnes Akkerman. 2012. “Effects of Managers’ Working Motivation andNetworking Activity on Their Reported Levels of External Red Tape.” Online AdvancedAccess, February 29.

Treier, Shawn and Simon Jackman. 2008. “Democracy as a Latent Variable.” Americanjournal of Political Science 52(1):201–17.

van der Linden, Wim J. and Ronald K. Hambelton. 1996. “Item Response Theory: Brief His-tory, Common Models, and Extensions”. In Handbook of Modern Item Response Theory,ed. Wim J. van der Linden and Ronald K. Hambelton. New York: Springer pp. 1–28.

van Schuur, Wijbrandt H. 2003. “Mokken Scale Analysis: Between the Guttman Scale andParametric Item Response Theory.” Political Analysis 11(2):139–163.

van Schuur, Wijbrandt H. 2011. Ordinal Item Response Theory: Mokken Scale Analysis.Thousand Oaks, CA: Sage Publication.

31

a bayesian approach to measurement bias in networking studies

Documents