This article was downloaded by: [University of Nebraska, Lincoln]On: 06 November 2014, At: 00:37Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Structural Equation Modeling: AMultidisciplinary JournalPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/hsem20

On Testability of Missing DataMechanisms in Incomplete Data SetsTenko Raykov aa Michigan State UniversityPublished online: 11 Jul 2011.

To cite this article: Tenko Raykov (2011) On Testability of Missing Data Mechanisms in IncompleteData Sets, Structural Equation Modeling: A Multidisciplinary Journal, 18:3, 419-429

To link to this article: http://dx.doi.org/10.1080/10705511.2011.582396

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Structural Equation Modeling, 18:419–429, 2011

Copyright © Taylor & Francis Group, LLC

ISSN: 1070-5511 print/1532-8007 online

DOI: 10.1080/10705511.2011.582396

On Testability of Missing Data Mechanismsin Incomplete Data Sets

Tenko RaykovMichigan State University

This article is concerned with the question of whether the missing data mechanism routinely

referred to as missing completely at random (MCAR) is statistically examinable via a test for

lack of distributional differences between groups with observed and missing data, and related

consequences. A discussion is initially provided, from a formal logic standpoint, of the distinction

between necessary conditions and sufficient conditions. This distinction is used to argue then that

testing for lack of these group distributional differences is not a test for MCAR, and an example is

given. The view is next presented that the desirability of MCAR has been frequently overrated in

empirical research. The article is finalized with a reference to principled, likelihood-based methods

for analyzing incomplete data sets in social and behavioral research.

Keywords: missing at random, missing completely at random, missing data, necessary condition,

observed at random, sufficient condition

Missing data pervade the social, behavioral, educational, and biomedical sciences, as well as

many other scientific fields. Most studies in them lead to incomplete data sets where some

subjects do not provide data on one or more observed variables. Analysis of such data sets

has been always of special interest in these disciplines, and for most of the past century also

a serious challenge. In fact, it would be fair to say that missing data analysis has become a

major area of research over the last several decades in statistics and areas of its application, with

multiple and far-reaching implications for behavioral and social science research in particular.

Statistical methods for dealing with missing data have been attracting the attention of

methodologists and substantive scholars for a number of years. Historically, one of the most

popular procedures has been listwise deletion (LD). As discussed in more recent literature,

LD could be used in situations with: (a) a missing data mechanism (condition) referred to

as missing completely at random (MCAR); (b) a small percentage of missing values (e.g.,

Correspondence should be addressed to Tenko Raykov, 443A Erickson Hall, Michigan State University, East

Lansing, MI 48824, USA. E-mail: raykov@msu.edu

419

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

420 RAYKOV

5% say or less); and (c) a sufficiently large initial sample of studied subjects, to offset the

associated loss in parameter estimation efficiency (cf. e.g., Allison, 2001). Perhaps due to this

fact, for several decades MCAR has been in the center of interest of many behavioral and

social scientists confronted with missing data. Quite naturally then, the question can be raised

as to how to ascertain if an incomplete data set fulfills the MCAR condition.

This article attends to this and related queries. The intention of the following discussion is

to contribute to bridging the gap between the theory of missing data analysis and empirical

research with incomplete data sets in the social and behavioral sciences. As argued herein, a

test for lack of distributional differences for respondents and nonrespondents does not provide

a test of MCAR. To highlight this limitation, a discussion of necessary conditions and sufficient

conditions is provided from a formal logic standpoint. The condition of data being observed

at random (OAR), which has often been indicated incorrectly in the literature as implying

MCAR, is next shown to be only a necessary but not sufficient condition for MCAR, and

thus not guaranteeing the latter. An example is then given where OAR holds but MCAR

is not fulfilled. It is subsequently argued that the desirability of MCAR has been frequently

overrated in social and behavioral research, and that the missing data mechanism that is instead

preferable to be concerned with, possibly on an essentially routine basis, is rather missing at

random (MAR). The article concludes with a reference to modern, state-of-the-art approaches

to the analysis of incomplete data, maximum likelihood and multiple imputation, which are

also readily available within the popular structural equation modeling (SEM) methodology.

These principled approaches allow one to handle missing data in settings complying with the

MAR mechanism as well as in some circumstances with deviations from it.

MISSING DATA MECHANISMS AND THEIR RELEVANCE FOR SOCIAL

AND BEHAVIORAL RESEARCH

As some of the most widely cited literature on analysis of incomplete data sets indicates, it is

useful to discern among three main missing data mechanisms (e.g., Little & Rubin, 2002). The

one that seems to be still most popular among social and behavioral researchers is MCAR.

Data Missing Completely at Random

An incomplete data set complies with the MCAR mechanism when the probability of missing-

ness (e.g., of a datum on a particular variable of interest) does not depend on (a) the actually

missing value, and (b) the observed data (e.g., Allison, 2001). That is, MCAR is by definition

a missing data mechanism whereby the occurrence of a missing value has nothing to do with

the observed data or the actually missing value (e.g., Enders, 2010).

An example when MCAR is (likely to be) fulfilled is when data are missing by design

(MBD; e.g., Schafer & Graham, 2002). This is a relatively recently popularized class of designs,

where one or more variables are observed only on a random subsample from an initial sample.

(More complicated arrangements and designs are also possible, with the common feature that

missing are data only on a random subsample from an originally available sample; e.g., Enders,

2010.) MBDs, when appropriately constructed, have been shown to allow substantial relief in

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

TESTABILITY OF MISSING DATA MECHANISMS 421

respondents’ burden at the expense of a fairly limited loss of statistical power (e.g., Graham,

2009, and references therein).

An alternative example when MCAR is not fulfilled, is when participants withhold their

answer on a particular variable (e.g., a survey question) due to their scores on it being in

some sense unusual (e.g., very large or small), or because of another property of these scores

that sets them apart from the majority of the remaining participants’ values on this measure.

For instance, if participants elect not to provide an answer on a question asking about their

alcohol use habits, due to them taking an inordinately large number of drinks per week, then

the resulting data would not be MCAR.

Data Missing at Random

Aside from possibly the MBD cases, which are not as yet highly popular or widely used, data

sets that are MCAR can be expected to be relatively rare in empirical social and behavioral

research. The reason is that MCAR is a rather stringent and restrictive condition that is unlikely

to be frequently satisfied. Specifically, in many applications, participants give rise to missing

data on certain variables in part because of possessing unusually low or high values on these

or related variables. In such situations, where there is a systematic pattern of missingness, a

less restrictive condition might be fulfilled, which is referred to as MAR. Data are MAR when

the probability of missingness is related to observed data but is not related to the actually

missing values on measures of interest (e.g., Allison, 2001). Two state-of-the-art methods have

been developed over the past several decades that are applicable when data are MAR (e.g.,

Schafer & Graham, 2002). These are maximum likelihood (e.g., Arbuckle, 1996) and multiple

imputation (e.g., Schafer, 1997), which we return to later in this article.

As has been discussed in detail in the literature, MAR is a more general condition than

MCAR (e.g., Enders, 2010). That is, if a data set is MCAR, then it is by definition MAR as

well, but the converse is not necessarily true. When data do not fulfill the MAR condition,

they obviously cannot be MCAR either. Such data sets are called not-MAR (NMAR) and are

characterized by the feature that probability of missingness is related to the actually missing

values.

Another condition of missingness that is related to the MCAR mechanism is the so-called

OAR, which is of particular relevance in the remainder of this article (e.g., Allison, 2001). Data

are OAR with regard to a given variable, say y, when the distribution of all remaining variables

of interest in the group of subjects with data on y (Group 1) is the same as the distribution of

these variables in the group of subjects with missing values on y (Group 2). In some literature

on missing data analysis, OAR has been frequently interpreted incorrectly as implying MCAR,

an issue that we attend to in the remainder of this discussion.

How Is a Missing Data Mechanism Helpful When Analyzing an Incomplete

Data Set?

Knowledge of the missing data mechanism is instrumentally helpful in an empirical setting

associated with an incomplete data set. This is because it is that mechanism that helps a

researcher to decide on an appropriate method of analysis and modeling of the data. Specifically,

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

422 RAYKOV

if data are MAR (or even MCAR), then one does not need to model the missing data mechanism

and will still obtain thereby parameter estimates with desirable properties; however, if data are

not MAR, then to obtain such estimates the missing data mechanism needs to be modeled as

well (e.g., Little & Rubin, 2002). With this in mind, means for ascertaining a missing data

mechanism or its absence are especially attractive. However, as repeatedly discussed in the

literature, the MAR condition is not statistically testable (e.g., Enders, 2010). The reason is

that data needed for such a test to be performed (if it was possible to devise) are actually

missing.1

Given the high popularity of the MCAR condition among empirical social and behavioral

scientists, it is of particular interest to find out how this missing data mechanism could be

ascertained. To address this important issue, it will be helpful to discuss first the important dis-

tinction between necessary conditions and sufficient conditions from a formal logic standpoint.

NECESSARY CONDITIONS, SUFFICIENT CONDITIONS, AND THEIR

IMPORTANCE WHEN STUDYING A MISSING DATA MECHANISM

In the remainder of this discussion, it is essential to differentiate between two types of

conditions (requirements or statements). These are (a) necessary conditions, on the one hand;

and (b) sufficient conditions, on the other hand. Necessary conditions and sufficient conditions

are always considered in tandem with a particular statement that one is interested in examining,

for example, willing to test for, examine, or ascertain.

Sufficient Condition

Let us denote a statement of interest by S in this section. For example, S could be the statement

that an incomplete data set does not exhibit the MCAR feature, or formally S D “data are not

missing completely at random.” A sufficient condition, denoted C, with regard to a given

statement S, is defined as one that implies S. That is, the condition C is sufficient for S, if the

validity of S follows from that of C. In other words, C is sufficient for S, if whenever C is

fulfilled, so also is S. Having a sufficient condition for a statement of interest is particularly

helpful because in that case all that is needed to verify that this statement is correct is checking

whether that condition is fulfilled.

For example, consider the statement S D “An integer number is even.” A sufficient condition

for it would be one, which implies S. For example, the condition C D “An integer number is

divisible by 4” implies that this number is even. That is, from the validity of C follows that

of S. Therefore, C is sufficient for S. In other words, for an integer number, being divisible by

4 is a sufficient condition for being even. Having this sufficient condition C is useful in the

1This article is not using the term statistically testable in any particular relation to Type I and Type II errors

associated with statistical inference. That is, whenever a statement is made that there is no available statistical test

for a given condition (e.g., MAR), or that a test for one missing data mechanism (OAR) is not a test for another

mechanism (MCAR), it is not meant to suggest that this limitation actually results from the possibility of a Type I or

Type II error occurring.

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

TESTABILITY OF MISSING DATA MECHANISMS 423

sense that it helps one claim that a number is even anytime it is divisible by 4, that is, anytime

(or, for any number for which) C is fulfilled.

Necessary Condition

A related but distinct concept from that of sufficient condition is the notion of necessary

condition. Consider again a statement, denoted S1, and a condition denoted C1. C1 is defined

as a necessary condition for S1, if the validity of S1 implies that of C1. Simply put, C1 is

necessary for S1, if S1 cannot hold unless C1 does. In other words, if C1 is necessary for

S1, then one needs C1 to be valid if S1 is to be valid, too. For example, a number cannot be

divisible by 25 unless it is divisible by 5. That is, for an integer number, being divisible by

5 is a necessary condition for that number being divisible by 25. In other words, any number

that is not divisible by 5 cannot be divisible by 25 either.

A Necessary Condition Need Not Be Sufficient

From this discussion, it is readily seen that if a condition, say C2, is necessary for a statement

S2, say, then C2 need not (although it might) be sufficient for S2. In the last number example,

as mentioned, being divisible by 5 .C1/ was not sufficient for being divisible by 25 .S1/,

although the former was necessary for the latter. Similarly, in the context of the first example

considered in this section, being even was necessary but not sufficient for being divisible by 4.

If one wishes to come up with a condition that is both necessary and sufficient for a particular

statement, which is referred to as a necessary and sufficient condition (NSC), then special

care needs to be taken in showing that the condition in question implies the statement and

conversely, that is, demonstrating that when the statement is true then also that condition is

fulfilled, and vice versa.

To give an example of an NSC, let us revisit the last example with number divisibility by 25.

An NSC for an integer number to be divisible by 25 is the following: “The number consisting

only of the last two digits of the one in question is to be divisible by 25.” That is, an NSC for

being divisible by 25 is that the last two digits of a number under consideration form themselves

a number that is divisible by 25. (In this subsection, if a given number consists of a single

digit, place 0 before it when forming the one consisting of its “last two digits.”) Indeed, as can

be readily shown for any number divisible by 25, the one formed from its last two digits will

itself be divisible by 25. Conversely, if for a given number, that formed by its last two digits is

divisible by 25, then the former number is itself divisible by 25. As another example of an NSC,

Raykov and Penev (1999) presented an NCS for covariance structure model equivalence. That

condition consists in the existence of a transformation from the parameter space of one model

onto the parameter space of the other model, which preserves the implied covariance matrix.

We see from these examples that there is a distinction between necessary conditions and

sufficient conditions, and that only some necessary conditions are also sufficient for particular

statements of interest. In other words, not all necessary conditions are also sufficient (and

not all sufficient conditions are necessary as well), with regard to prespecified statements of

interest. For instance, although all crows are black (assuming, of course, no exceptions), it does

not follow that all black birds are crows. That is, simply seeing a black bird, one cannot claim

he or she has seen a crow. This is another simple example that a necessary condition (being a

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

424 RAYKOV

black bird) need not be sufficient for a particular statement of interest (having seen or being a

crow). In fact, often it is quite difficult to come up with sufficient conditions for statements of

concern in certain scientific disciplines, as we also see next.

A NECESSARY BUT NOT SUFFICIENT CONDITION FOR DATA MISSINGCOMPLETELY AT RANDOM

Returning to the context of missing data underlying this article, we find very instructive

applications of the discussion in the last section. As we indicated earlier, a main concern

of this article is with the MCAR condition (missing data mechanism). In the language of the

preceding section, this article is mainly concerned with the issue of whether conditions stated

as sufficient for MCAR in some missing data literature are indeed sufficient, or only necessary.

Observed at Random Is Necessary But Not Sufficient for Missing Completely

at Random

Some previous and recent literature dealing with incomplete data have made claims implying

that a test for MCAR consists of showing no distributional differences on variables of interest

across two groups of subjects: (a) the group of all studied persons with observed data on a main

variable of concern, say y (Group 1); and (b) the group of all subjects with missing data on

y (Group 2). This lack of distributional differences across these two groups amounts however

precisely to the earlier mentioned condition of data OAR (cf. Allison, 2001). Due to MCAR

being defined as lack of relationship between the probability of missingness, on the one hand,

and the observed as well as missing data (on y) on the other hand, it follows that MCAR

implies OAR. That is, OAR is a necessary condition of MCAR, or, in other words, MCAR

cannot hold unless OAR does. The literature mentioned earlier implies, however, that OAR

would also be a sufficient condition for MCAR, without any offered proof of that sufficiency. In

actual fact, such sufficiency does not hold, contrary to statements in some literature on missing

data. Specifically, OAR is not a sufficient condition for MCAR, as I elaborate later. In other

words, even when OAR is fulfilled, there is no guarantee that MCAR will be fulfilled, and in

fact MCAR might well be violated.2

The reason why OAR is not sufficient for MCAR, in spite of claims implying the opposite

in some of the literature on missing data, is readily seen as follows (cf. Allison, 2001). Even

if Groups 1 and 2 of participants with observed and with missing data on y were to have

the same distribution on all remaining variables of interest, it is not known what the actual

values on y are for the participants from Group 2, namely the group with missing data on

y. Hence, it is still possible that these missing values in fact have some relationship to the

probability of missingness. The simplest example for the latter is in an empirical situation

where all missing values are actually very large (or very small), in which case there is a clear

2The situation here is analogous to that with a well-known necessary but not sufficient condition for structural

equation model identification. This is the condition (requirement) for nonnegative degrees of freedom (df); that is, df �

0. Specifically, as discussed at length in the literature (e.g., Bollen, 1989), having nonnegative degrees of freedom does

not guarantee identifiability of a model under consideration, whereas df � 0 is implied when this model is identified.

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

TESTABILITY OF MISSING DATA MECHANISMS 425

relationship between the probability of missingness and the missing values themselves (cf.

Little & Rubin, 2002, ch. 1, and their examples of deterministic missingness; see also the next

section). This relationship invalidates MCAR, which by definition is a mechanism or condition

where the probability of missingness is supposed to be unrelated to the missing values. The

reason for this invalidation of MCAR is that in the indicated empirical situation, the probability

of missingness in fact depends on the magnitude of the actually missing values.

Implications for Past Research Utilizing the MCAR Assumption in ListwiseDeletion When Analyzing Incomplete Data Sets

The preceding discussion in this section shows that empirical research based on MCAR, which

used the lack of distributional differences across the groups of participants with and without

missing data to imply MCAR, did not necessarily have justification to proceed subsequently

as if the MCAR condition was fulfilled. Specifically, research that used LD (even with a small

percentage of missing values and a large initial sample), which effectively assumed MCAR

based on testing for OAR only, might have in fact yielded incorrect conclusions with regard

to the pertinent studied populations. Given the high popularity and widespread use of LD in

social and behavioral research—in part at least due to its implementation as the default option

in major, widely circulated statistical analysis software—it is likely that many publications

using LD (whether explicitly acknowledged or not) have at best proceeded based on testing

only for OAR that is, however, not sufficient for MCAR, as we saw earlier. The reason this

research might have yielded incorrect conclusions is the possibility for actual violations of

MCAR that would not have been sensed then by the only tested condition of OAR (which

does not imply MCAR, as elaborated earlier). These violations of MCAR would have rendered

the LD subsample actually analyzed to be no more representative of the studied population even

if the initial sample was (reasonably) representative of it to begin with. Hence, results from

such LD-based research could well have been associated with biased parameter estimates and

subsequent misleading statistical tests. For this reason, findings from that research could not be

generalizable to their studied populations of actual interest—contrary to possible interpretations

otherwise that were made in that published research.

AN EXAMPLE OF DATA OBSERVED AT RANDOM BUT NOT MISSING

COMPLETELY AT RANDOM

To provide an illustration of a situation where OAR holds but MCAR is violated, which is con-

sistent with the earlier elaboration that OAR is necessary but not sufficient for MCAR, consider

the following empirical setting. Suppose one had generated data for n1 cases on p variables

y1; y2; : : : ; yp.p > 1/, following the p-dimensional multinormal distribution Np.�; †/, with

† being positive definite. Designate the resulting data set by �1. Denote next by �0 and †0

the mean vector and covariance matrix, respectively, of the variables y1; y2; : : : ; yp�1. That is,

�0 is obtained from � by dropping its last component, and †0 is rendered from † by deleting

its last row and column. (Note that †0 is also positive definite; e.g., Johnson & Wichern,

2002.) Due to the nature of the generated data on the n1 cases, each of them is associated with

data independent from that on any other case. Simulate next data on n2 cases following the

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

426 RAYKOV

distribution Np�1.�0; †0/ (using a different seed, to ensure independence from �1), and denote

this data set by �2. Similarly, each of these n2 cases have resulting data that are independent

of that on any other case in �2 as well as in �1, and the data sets �1 and �2 are independent

of one another. Next merge these two data sets �1 and �2 so that �2 has the data assigned to

the variables y1; y2; : : : ; yp�1 (that is, adding the cases in �2 to those in �1 on these p � 1

variables), and denote the resulting merged data set �12. Then, as a following step, extend �12

by defining the values on the variable yp for the n2 last generated cases to equal �p C 5¢p,

where �p and ¢p are the mean and standard deviation on the variable yp . Finally, declare in

the data set �12 as missing on yp all subjects fulfilling the condition yp � �p C 5¢p, and

denote the last resulting data set as �. We emphasize that � is an incomplete data set, where

all subjects have data on all variables except (at least) the last n2 subjects, who have missing

values on the variable yp .3

It is readily realized that the data set � fulfils the OAR condition (when of concern is the

variable yp and its relationships to the remaining variables). Indeed, there are two groups of

participants in �—Group 1 consisting of the first n1 cases who have data on all variables and

thus on yp as well, and Group 2 including (at least) the last n2 cases who have data missing on

yp . (There might be some among the n1 first simulated cases in �1 that fall into Group 2, but

this will be with probability essentially 0 for most sample sizes in current social and behavioral

research. More important, the fact that there might be cases from �1 that belong to Group

2 does not invalidate the argument and illustration in this section, for which the magnitude

of their values on y are of relevance and not their group membership.) Due to the particular

process of data simulation, Groups 1 and 2 consist of cases that follow the same .p � 1/-

dimensional distribution with regard to the fully observed variables y1 through yp�1, namely

Np�1.�0; †0/. (Note that this distribution remains the same even if some cases from �1 fall

into Group 2.) Therefore, there are no distributional differences between Groups 1 and 2 on the

variables y1 through yp�1 (i.e., there are no group differences as far as the p � 1-dimensional

and all lower dimensional distributions of y1 through yp�1 are concerned). Hence, the data in

the set � are OAR; that is, the OAR condition is fulfilled for the incomplete data set �.

As elaborated earlier in this article, although OAR is implied for any incomplete data

set with data being MCAR, the OAR condition is not sufficient for MCAR. The presently

considered data set � in fact provides a clear demonstration and example of this lack of

sufficiency, that is, of the fact that OAR does not imply MCAR. In other words, the data set

� exemplifies that even when OAR holds (as it does in �), MCAR need not hold and in fact

can be violated—as is indeed the case in �. MCAR is violated in the data set � because in

� missing are the data for all participants with very large values on the focal variable yp .

Thus, the probability of missingness in � is related—as opposed to unrelated—to the actually

missing values: Specifically, any value on yp that is large enough, is in fact missing in �.

For this reason, the data in the set � are not MCAR, even though they are OAR (viz. by

construction). This example also shows that a test for OAR is not a test for MCAR, in spite

3The validity of the illustration in this section (viz. of the fact that OAR is not sufficient for MCAR) does not

depend on the magnitude of n1 and n2, and these numbers can be chosen as deemed sufficiently large to avoid triviality

considerations. We stress that the point made in this section of the main text consists in providing an example that

OAR does not imply MCAR. For this point to be valid, it is satisfactory to have two suitably chosen numbers n1 and

n2 with the properties outlined in the current section of the main text.

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

TESTABILITY OF MISSING DATA MECHANISMS 427

of some discussions in the missing data literature stating or implying (or being interpretable

as implying) the opposite (see also Footnote 1).

A SUFFICIENT CONDITION AND TEST FOR DATA NOT MISSING

COMPLETELY AT RANDOM

The discussion in the last and preceding sections of this article demonstrates that the statistical

tests for OAR that were claimed in previous literature for being tests of MCAR are actually not

tests for MCAR, but at the most only for a necessary condition for MCAR—namely for OAR.

The previous discussion in this article is helpful, however, in identifying a sufficient condition

for another statement related to MCAR, to which we next turn.

A Sufficient Condition for Data Being Not-MCAR

The distinction made earlier between a necessary condition and a sufficient condition, from a

formal logic standpoint, allows us to now provide a sufficient condition for a data set being not-

MCAR (i.e., for not being MCAR). This sufficient condition is that the data be not-OAR. That

is, data being not-OAR (e.g., a data set � showing some distributional differences between the

groups with observed data and with missing data on a variable of concern) cannot be MCAR.

The reason is that if that data set � were MCAR, due to OAR being a necessary condition

for MCAR it would follow that � would have to be OAR as well, yet � was assumed in this

argument not-OAR to begin with. The last contradiction shows that data being not-OAR is a

sufficient condition for that data being not-MCAR. In other words, if a data set does not fulfill

the OAR condition (i.e., if the data shows some distributional differences across the preceding

Groups 1 and 2), that data set cannot fulfill the MCAR condition either.

A Test for Data Not Missing Completely at Random

The last subsection indicates also a test for data being not-MCAR. Accordingly, this test consists

of examining whether the data are not-OAR. To accomplish the latter goal, one tests if there

are any distributional differences between the groups with observed data and with missing data

on a variable of concern. How to test for such distributional differences is discussed in detail,

for instance, in Enders (2010), and for space reasons we refer to that source. Along with tests

for mean and variance differences, one of the most complete and recommendable omnibus

tests of distributional differences when multinormality does not hold in a given data set is the

Kolmogorov–Smirnov test.

If the OAR condition is rejected, that is, if for a given data set some distributional differences

are found between these two groups examined (e.g., on means or variances), then one can

conclude that the data set under consideration is not MCAR. However, if the OAR condition

is not rejected—that is, if no distributional differences across these two groups are found—

one cannot claim that the data set is MCAR. The reason is, as discussed in detail earlier in

this article, that OAR is only a necessary condition but not a sufficient condition for MCAR

(although data being not-OAR is a sufficient condition for being not-MCAR, as indicated

earlier).

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

428 RAYKOV

THE RELEVANCE OF THE MISSING AT RANDOM MECHANISM WHEN

ANALYZING AN INCOMPLETE DATA SET

Over the past several decades, the desirability of the MCAR condition has been frequently

overrated in empirical social and behavioral research. For a long period of time, many behavioral

and social scientists have effectively relied on MCAR when using LD, which possibly made

the MCAR condition so attractive and pursued by applied researchers in these and related

disciplines. As the preceding discussion in this article shows, however, usually there is no

reason to be interested in “ensuring” that one’s incomplete data set is characterized by the

MCAR property. The MCAR condition is in actual fact rather restrictive and one that is much

less often fulfilled in practice than earlier thought.

When data are only MAR—which as mentioned is a systematic missing data mechanism

that does not necessarily fulfill the MCAR condition—two state-of-the-art, likelihood-based

methods have been available for some time and implemented in widely circulated software,

including SEM programs like AMOS, EQS, LISREL, and Mplus (Schafer & Graham, 2002).

These two principled methods are maximum likelihood (often referred to as full information

maximum likelihood [FIML]) and multiple imputation (MI; e.g., Arbuckle, 1996; Little &

Rubin, 2002). There is thus no reason to consider a study where MCAR is not fulfilled, as

being uninformative (and even less reason to resort to LD, as might have been done at times

in past applied research). In fact, if the MAR and multinormality assumptions are fulfilled,

both FIML and MI can now be readily employed with popular software, also outside of the

SEM framework. When MAR is deemed—on substantive or related grounds—to be violated,

then an inclusive analytic strategy is readily available within an FIML modeling approach (e.g.,

Graham, 2003). Accordingly, one includes in the models considered auxiliary variables that are

predictive of the missing values (e.g., have notable relationships with the variables on which

data are missing). As discussed, for instance, in Enders (2010), this strategy is easily applicable

in empirical settings with incomplete data sets and such auxiliary variables. The beneficial effect

of this strategy is that it enhances the plausibility of the MAR assumption. When there is a

large number of additional variables that are not considered for inclusion in a particular model

of interest (analytic model), multiple imputation represents a method that can also be employed

in empirical research and is implemented in popular software (e.g., Muthén & Muthén, 2010).

CONCLUSION

This article was concerned with issues of relevance when analyzing incomplete data sets

that might be seen as the rule rather than the exception in contemporary social, behavioral,

educational, and biomedical research. In particular, the question of ascertaining whether the

missing data mechanism routinely referred to as MCAR is fulfilled in a given incomplete data

set, was of main interest. To address this query a discussion was provided, from a formal logic

standpoint, on the important distinction between necessary conditions and sufficient conditions.

This distinction, and especially the fact that a necessary condition need not be sufficient, was

used to argue that a test for data being OAR does not represent a test for MCAR (see also

Footnote 1). At the same time, a relatively straightforward approach was discussed for testing if

a data set is not-MCAR. (It could be argued that some of the mentioned missing data literature

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014

TESTABILITY OF MISSING DATA MECHANISMS 429

in fact meant to imply this relationship rather than testability of MCAR via testing for OAR,

but did not explicate this as would be expected logically.) To this end, it was demonstrated

that all one needs to do is test whether a data set under consideration has the property of

being not-OAR. In an illustration setting, an example was provided where data were OAR

but not-MCAR, thus demonstrating that a test for OAR is not a test for MCAR. It was then

argued that the desirability of MCAR has been frequently overrated in empirical research.

Rather than being interested in examining MCAR or even ensuring it (which might not be

possible in many empirical settings), two state-of-the-art methods for dealing with data MAR

were referred to, FIML and MI. With violations of MAR, one can use an inclusive analytic

strategy that incorporates in a modeling effort auxiliary variables notably related to measures

with missing data, and in this way enhance the plausibility of MAR (e.g., Graham, 2009).

In conclusion, this article shows that a test for OAR is not a test for MCAR in a given

incomplete data set, and there is a relatively simple conclusive approach for ascertaining that

data are not-MCAR. Another main aim of the article was to argue for less attention to be

paid to the MCAR condition when one is faced with an incomplete data set. Instead, wider

use should better be made of principled, likelihood-based approaches to handling missing

data in empirical social and behavioral research, such as FIML and MI. These incomplete data

modeling approaches have been readily available for some time also within the SEM framework

(e.g., Muthén & Muthén, 2010).

ACKNOWLEDGMENTS

Thanks are due to P. D. Allison, B. Muthén, S. Penev, and K.-H. Yuan for valuable discussions

on missing data analysis. I am grateful to the editor and two anonymous referees for comments

on an earlier version of the article.

REFERENCES

Allison, P. D. (2001). Missing data. Thousand Oaks, CA: Sage.

Arbuckle, J. L. (1996). Full information estimation in the presence of incomplete data. In G. A. Marcoulides & R.

E. Schumacker (Eds.), Advanced structural equation modeling (pp. 243–277). Mahwah, NJ: Lawrence Erlbaum

Associates.

Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.

Enders, C. K. (2010). Applied missing data analysis. New York, NY: Guilford.

Graham, J. W. (2003). Adding missing-data relevant variables to FIML-based structural equation models. Structural

Equation Modeling, 10, 80–100.

Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60,

549–576.

Johnson, R. A., & Wichern, D. W. (2002). Applied multivariate statistical analysis. Upper Saddle River, NJ: Prentice

Hall.

Little, R. J., & Rubin, D. B. (2002). Statistical analysis with missing data. New York, NY: Wiley.

Muthén, L. K., & Muthén, B. O. (2010). Mplus user’s guide. Los Angeles, CA: Muthén & Muthén.

Raykov, T., & Penev, S. (1999). On structural equation model equivalence. Multivariate Behavioral Research, 34,

199–244.

Schafer, J. L. (1997). Analysis of incomplete multivariate data. London, UK: Chapman & Hall.

Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the arts. Psychological Methods, 7,

147–177.

Dow

nloa

ded

by [

Uni

vers

ity o

f N

ebra

ska,

Lin

coln

] at

00:

37 0

6 N

ovem

ber

2014