bailyn - research as a cognitive process

7/28/2019 Bailyn - Research as a Cognitive Process

1/21

Qua lity and Qua ntity, 11 (1977) 97-l 17 97@ Elsevier Scienti fic Publ ishing Com pany, Amsterdam - Printed in The Netherlands

RESEARCH AS A COGNITIVE PROCESS: IMPLICATIONSFOR DATA ANALYSIS *

LOTTE BAILYNAssociate Professor of Organizational Psychology and Managem ent, Sloan

School of Management, M.I .T.

My aim in this paper is to consider research as a cognitive process asdistinct from research as a validating process, and to draw some impli-cations of these considerations for the analysis of data. When research isviewed as validation, data [ 1 ] is collected in order to test a prefor-mulated conception, and while that testing can be immensely sophisti-cated it is not cognitive in the present sense. In research that is cogni-tive the researcher develops conceptual grasp as the study progressesand data is integrally involved in the conceptualization process itself[ 21. It is this aspect of research that I wish to examine. In particular, Iwant to discuss the strategies of data analysis that enhance research as acognitive process.

No analyst would deny that these two processes are in some measurepresent in all research efforts [3]; but merely to recognize that fact isfar different from understanding its implications for analysis. Thispaper attempts to probe these implications and to derive some analyticstrategies to guide the optimal use of data in conceptualization. Thetechniques of data collection and analysis used in validation derive fromfairly well-established, formal principles, but such principles have notbeen established for research as a cognitive process. It is peculiarly dif-ficult to establish these principles because the cognitive flow of researchis very personal; researchers do not often talk about how they actuallyarrive at their conclusions [4]. Our espoused theory (Argyris andSchbn, 1974) continues to view research primarily as validating, andpublication norms generally preclude the presentation of results in themanner in which they actually evolved. Only the self-conscious

* An earlier version of this paper wa s presented at a Colloquium of the Organiza-t ion S tudies Group, Sloan School of Managem ent, M.I .T., on May 1, 1975. I amgrateful to Michael Ginzberg, Edgar H. Schein, and John Van Maanen for their care-ful reading of an earlier draft and for their incisive com me nts.


2/21


3/21

99the wives work or career orientations and tried various indicators ofsatisfaction. But every attempt brought further verification of thefinding. I considered a number of factors that might account for therelationship. Income was an obvious candidate: if families with lessincome were likely to have working wives, and low income was accom-panied by family strain - highly plausible and empirically corroboratedassumptions - the association might disappear when income was con-trolled. It did not. I knew, of course, that cross-sectional data could notprovide evidence of the direction of causality. It was certainly possiblethat wives in unhappy couples sought work as an escape from theunsatisfactory family situation, a process that was easier for me toaccept than its opposite. And, indeed, I am sure that this happened insome cases: that there were working couples in which satisfaction wouldhave been even less had the wife decided to stay home. But.if this werethe primary way by which the original association came about, I wouldhave expected the relation between wifes work and unhappiness to beconsiderably less in those couples where the woman was working onlyfor financial reasons - and it was not.

These early analytic steps resulted in a provisional, if reluctant accep-tance of the finding and a search for conditions and contexts in whichit does or does not apply. Such a turn, however, necessitated greaterconceptual clar ity. My interest centered on married women withcareers - women who wanted to combine work with their family roles.I had learned enough in my init ial forays into the data (made particu-larly easy by a highly flexible interactive computer program [5]) torealize that the fact of working during these early years of marriage didnot validly identify such women, since some worked merely for finan-cial reasons and others, though not working at that time, were patientlywaiting for the chance to resume outside employment. It was necessary,therefore, to devise a better indicator for what I had in mind. In theend, women were classified into types of career orientation not on thebasis of their actual work patterns at the time of the survey but by acombination of their attitudes toward married womens careers and thesatisfactions they derived from work.

Thus I arrived at a variable that defined the career orientations of thewives in the sample in a way that seemed to reflect my concerns moreaccurately than had the simple distinction between working and non-working with which I began. But the step from which a new conceptwould eventually evolve had not yet been taken. This only occurredwhen the career orientations of the wives were cross-tabulated withcareer and family orientations of the husbands - when couple patternswere developed involving various combinations of each partners orien-tation to work and family.


4/21

100These patterns were evaluated in terms of the marital satisfaction

associated with them. Gauged by this measure, only one combinationturned out to be bad: that in which husbands with traditional career-orientations were paired with wives who wanted to integrate careersinto their own lives. In other words, when husbands are will ing to sub-ordinate some portion of their career needs to those of family, happymarriages are as likely to occur when wives are career-oriented as whenthey are not. Hence it turned out that it was the husbands attitude -his accommodation of work to family, to use the term I later chose -that was crucial in understanding the original finding.

In my final analysis of the English data I concentrated on this inter-pretation (Bailyn, 1970). It seemed a worthwhile focus for a paperbecause the emphasis on the husbands orientation to his owyl work(not only to his family or to his wifes work) introduced a new elementin the discussion of married womens careers. It arose from thecouples aspect of the data at hand, and also fitted into the moresymmetric view of the relation between work and family for both menand women that was then evolving. This paper also included a com-parison of the traditional couples (wives oriented to family, husbandsto career) with the integrated ones (husbands accommodative, wivesintegrating careers into their l ives). I will not go into the details of theseresults, but only note that they demonstrated that this integratedpattern is not one of role reversal; on the contrary, the accommodativehusbands continued to get a great deal of satisfaction from their workand the integrated wives were still very much involved with theirfamilies.

Thus an analysis that started with a concern with women~ careersended with a concept applicable to rrzens relation to their own work.This evolution made it possible to pursue the research with an entirelydifferent data base, consisting only of male respondents: a survey of thecareer experiences of M.I.T. alumni from the 1950s. These data pro-vided enough information about wives work patterns to allow me tocontinue the investigation of the effect on their families of mens orien-tations to work, (now in a more typically validating phase of theresearch process). This analysis showed that it was the men whose ownlives were integrated and balanced, not those oriented exclusively totheir careers and also not those primarily involved with their families,whose wives were most like ly to be involved in professional careers(Bailyn, 1973). And though the dependent family variable here (careerinvolvement of professional wives) was different from that used inEngland (marital satisfaction), the general conclusion from the Englishdata that mens orientations to their own work have important familyconsequences was corroborated.


5/21

101The example is coming to an end, even though the story is not over.

The M.I.T. data made it possible to extend the work in another direc-tion: to raise the question of the implications of accommodation notonly for mens families but also for their work. Preliminary analysis onthis point indicated that in the technical occupations in which thesemen. were involved, what is good for their families may be problematicfor their work (Bailyn, in press). New research, therefore, must concen-trate on the way family needs and work requirements interact witheach other in the personal and career development of men and women161. New data and another cognitive research phase are needed.

By presenting this example I hope I have been able to convey some-thing of the flexibility in the relation between ideas and data thatmakes research a cognitive rather than simply a validating process. Themore fluid the procedures, the more continuously responsive to sur-prises, nuances, and anomalies they are, the more l ike ly wil l the resultbe a new formulation. I will come back to this example whenever pos-sible in the subsequent pages, where I try to specify the analytic strate-gies that allow one to maximize the cognitive value of empirical data.

Implications for Data AnalysisWhen research is viewed as a cognitive process, analysis is continuous;

it occurs in all phases, from the initial collecting to the final writing.And though, for the sake of clarity, I will subsequently break thisprocess into three stages - collection, compilation, and presentation -the principles of analysis I want to examine apply to the process asswhole. In particular, I concentrate throughout on two general analyticprinciples. One concerns the cognitive complexity of the data and theother relates to the interplay between concepts and data.For data to be maximally useful for research as a cognitive process, i tmust be at the proper level of complexity. If it is too simple and undif-ferentiated it will not provide the researcher with input capable ofaffecting already existent views about the phenomenon under study.On the other hand, it must not be so complex as to overwhelm theanalyst. As will be obvious when we discuss the specific manifestationsof this principle at various stages of the research process, controlling thecognitive complexity of the data is a major analytic task.

Second, the process of analysis proceeds by means of a continuousinterplay between concepts and data. If one envisions concepts as resid-ing in a conceptual plane and data in an empirical one, I would venturethe proposition that the more links, and the more varied the links, that


6/21

102can be established between the two planes, the better the research.Some links, of course, must exist or there would be no empiricalresearch at all. Theorizing occurs all within the conceptual plane, andan analysis that proceeds only at that level is not an empirical investiga-tion. At the other extreme, empiricism refers to manipulations thatstay entirely within the empirical plane. Critics who accuse the empiri-cal social sciences of doing nothing but proving the obvious haveusually limited their vision to the empirical plane and neglected to con-sider the links between it and the conceptual one.

It is the continuous interplay between concepts and data that per-haps more than anything else distinguishes research as a cognitiveprocess from research as a validating process. In the typical validatingstudy there is only one foray from the conceptual to the empiricalplane: to select the potentially validating data and assess ts significance.This is obviously an important step, but it does not describe the totalresearch effort, which needs its cognitive phases as well. And theserequire many more links between the two planes.

Specific strategies of data analysis may be developed from the appli-cation of these two principles. Such strategies vary according to thephases of the research, and thus it is necessary to distinguish the essen-tial task of each research stage.The function of the collection phase is to generate the kind of datathat will permit research to proceed in a cognitive fashion. During datacompilation - which is perhaps the most creative phase - the collecteddata is transformed in such a way that it both guides and reflects theevolving conceptions of the analyst. The personal history given abovewas concentrated in this stage. It is here that conceptually key variables,such as the career orientations of the wives in the example given, aredefined; it is here, also, that standard techniques of data reduction mayfind their place. The stage of data presentation is crucial because thevalidation of the whole process often resides less within one particularstudy than in its fit with the shared conceptual understanding of otherresearchers concerned with similar problems [ 71. Research results mustbe presented in a way that allows them to be assessed in the context ofan evolving consensus.

These steps are obviously not strictly sequential. First drafts, forinstance, are often more valuable for compilation than for presentation,and even data collection can, in certain designs, be intertwined withdata compilation. But their essential functions are distinct, and thus itis fruitful to examine them separately when considering the implica-tions of the principles of research as a cognitive process.


7/21

103SPEC IFIC IMPLICATION S AT VARIOUS STAGES OF ANALYSIS

Table I summarizes, for each phase, some of the specific strategies ofdata analysis that flow from this view of the research process. The cel lsof the table can be identified by the principle of analysis they exem-plify (the row they are in) and by the research stage (column) to whichthey mainly apply. Within each cel l different strategies are identified byletter. Thus 1,2,a locates sequential development as a strategy for con-trolling the cognitive complexity of the data (row 1) in the compilationphase (column 2). The discussion that follows will proceed through thechart cell by cell. Whenever possible I will give specific examples fromthe research described in the beginning of this paper, and draw on ,otherstudies only when a particular strategy was not used in that case.1,l: Controlling cognitive complexity of data in the collection stage

The prescription to provide surplus data (1 ,I ,a) is made in order toinsure that the data collected wil l be sufficiently complex to be cogni-tively useful. In large scale surveys this usually comes about in any casebecause of the low marginal cost of including a few extra questions.And, indeed, secondary analysis, such as my use of the Rapoport data,is possible because most questionnaires contain many more items thanare used in the basic analysis.

But the strategy is equally applicable to more controlled researchdesigns. Let us take as an example a study by Fry (in preparation)which attempts to describe learning environments in terms of theirsituational complexity. Frys hypothesis is that complexity in class-room environments and individual styles of learning can be categorizedon an equivalent set of dimensions, and that the optimum learningsituation occurs when the individual learning style matches the charac-ter of the learning environment. To test this hypothesis it is sufficient(though by no means simple, as is obvious from Frys work) to measurelearning style and to characterize learning environments. But if theresearch is also to serve a cognitive function and to influence theresearchers understanding of the phenomenon he is studying, it willrequire other kinds of data as well. For if the hypothesis of fit is con-firmed, the researcher, though his existing conception of the learningprocess will be strengthened, will gain no new insights; and if not con-firmed, the lack of surplus data to provide possible explanations for thenegative result will, most likely, lead the researcher to conclude thatmeasurement was at fault, and thus also not affect the original concep-tion (cf. McGuire, 1973).

This example provides clues to two kinds of surplus data with poten-


8/21

104


9/21

105tially great cognitive yield: data that will help one understand theprocess by which an expected association comes about; and data thatwill allow one to test possible explanations for negative results. To con-tinue with the Fry example: If the hypothesis is confirmed, one wouldthen want to know why a learning environment that f its a personslearning style is more effective, for there are various possible explana-tions. Is it because one feels.more at ease in such an environment, orfeels freer to experiment? Or does it require less time. and effort toadjust to and hence leaves more for actual learning? Co.nversely, whatmight account for disconfirmation of the hypothesis? Perhaps theenvironment is not accurately perceived; or maybe one had expected anon-congruent learning environment and had already made anticipatoryadjustments which then precluded ones taking advantage of the con-gruence actually found. In both cases, peoples reactions to theirexperiences, though not directly related to the main hypothesis, areobviously relevant. In particular, questions about the subjects percep-tions of the environment in which they find themselves and their evalu-ation of their experiences in it, might provide appropriate surplus data.

Other studies, of course, deal centrally with peoples perceptions andexperiences. In such cases the crucial surplus data might involve nomore than standard demographic information to indicate membershipin certain social groups. Or, if the design allows for greater sophistica-tion, the surplus data might deal directly with environmental character-istics. Every research problem, naturally, will provide its own specificguidelines. But the general principle is to try to collect data on classesof variables that are not central in the init ial conceptualization of theproblem, so that new interpretations (both of expected and of unex-pected findings) can be formulated and subjected to empirical test.

1,2: Controlling cognitive complexity of data in compilation stageControl of the cognitive complexity of data is perhaps most crucialand most difficult to achieve in the compilation phase, as anyone whohas ever been faced with s ix inches of computer output relating every-thing to everything else can testify. An interactive computer program,such as I used in the analysis of the English data, makes it easier for theanalysts thinking to keep up with the flow of data. But even withoutan interactive program, certain strategies can help control the analysis.To be sure, they are not the same as those dictated by the economics ofbatch processing, but their potential cognitive yield should outweighany extra cost or effort.

Sequential development (1,2,a) is, in my mind, the best way toensure enough complexity in the data without producing a cessation of


10/21

106all thought because of overload. One must proceed by looking first atthe distributions of only a few variables and relating them to each otherbefore introducing new ones, and by testing indicators and selectingthose that are conceptually the clearest while rejecting those whose use-fulness has been superseded in the evolving analysis. In my work on theEnglish data, for example, I started by looking at womens currentwork status. But this variable, though crucial in initiating the sequenceof analysis, was completely transformed as the analysis proceeded, andhardly entered at all into the final research report.

The problem is to know where to start and where to stop: whatinitiates an analytic sequence and when does the analyst view it asfinished? In general, my tendency is to start by trying to define a keyclassificatory variable that comes as close as the data will allow to onescentral concern. Once one such key variable is defined, the analysisproceeds by relating that variable to other available information. Whatother information is chosen at this point depends on ones originalunderstanding of the phenomenon under study, and is guided by pastresearch, theories, personal experiences, or the researchers hunches.The sequence stops when a formulation has been reached that is pleas-ing [8] to the analyst and fits the empirical relations that were found.

Sooner or later, of course, all researchers are faced with the situationwhere there is no focal concern to initiate a sequence. This occurs mostoften when analytic results are required by contract (by the need towrite a paper, for instance) rather than by the desire to understand aparticular problem. In such cases it makes sense to start routinely - bylooking at all the marginals, for instance, or relating everything to a fewdemographic characteristics such as sex or age - and move into a cog-nitive research sequence when something surprising catches ones eye[91.

Obviously these are very general prescriptions; each particular casewil l necessarily be different, and inevitably there will be false starts.But the strategy of working slowly and sequentially - taking a fewvariables at a time, looking first at individual distributions, then pair-wise relations, then at more complicated interrelations - should mini-mize the time and effort spent on false starts and maximize the cogni-tive yield of the data. To be sure, a very experienced, selfdisciplinedanalyst could proceed sequentially through six inches of computer out-put. But it is not an easy task and such a procedure severely l imits theavailable sequences.But aside from sequentiality there are various ways of looking at datathat can affect its cognitive potential. In particular, I find the assump-tions of linearity and additivity, which underlie so many standard tech-


11/21

107niques of data reduction, to be very restrictive. When I speak of non-restrict ive assumptions (1,2,b), therefore, as a strategy of data compila-tion, I primarily mean looking at data in such a way that interactionsare clearly visible. The effects the social scientist studies depend on thecontext in which they occur (Cronbach, 1975) and our data analysismust be sensitive to this fact. The example presented above is anchoredin such an interaction: wives career patterns have a different effectwhen their husbands are accommodative than when they are tradi-tionally career oriented; only one specific combination of husbandsand wives patterns stands out as problematic. If data compilationalways proceeds by averaging effects over conditions instead of lookingat them separately for each condition, such results wil l not emerge.1,3: Controlling complexity of data in presentation stage

When research results are written up, the appropriate complexity ofthe data is geared more to the reader than to the researcher - thoughthe discipline imposed on the analyst at this stage often clarfies his orher own understanding of the phenomenon under study. The primetask of researchers in this phase is to present data in such a way thattheir procedures are open to the scrutiny of others. Such scrutiny iscrucial when research is pursued in the manner presented here, sincealternative explanations are often more obvious to the interested readerthan to the original analyst. Hence the prescription of openness (1,3,a).

Perhaps the most closed way to present ones data is merely to indi-cate which results are statistically significant and which are not. Thismode of presentation could allow two relationships which actually arevery similar to be viewed in opposing ways, simply because theirP-values happened to fall on either side of a predetermined value. Andthe unwary reader would have no way of knowing how close to the lineeach one in fact was. Further, presenting only the significance of resultsoften hides their strength. A correlation of 0.66, for instance, may beinsignificant in a small sample. But the reader should be allowed todecide that the design was at fault rather than being forced into theconclusion that there actually is no association between the variablesinvolved. And this is only possible when the correlation is given in thefinal report, even if it has to appear without a * (e.g. Hall, 1963).

But correlations themselves, though more open than P-values, aresti ll relatively closed. A paper, for instance, that reports that the corre-lation between maternal employment and independence in daughters ispositive, while that between maternal employment and independence insons is negative, though intriguing, leaves many questions unanswered(cf. Maccoby and Jacklin, 1974, pp. 3-8). Suppose that independence


12/21

108TABLE II

Mean independe nce scores by maternal employm ent and sexMother

Employed Not employed

(A) Sons 7.0 9.0Daughters 6.0 4.0

(B) Sons 4.0 6.0Daughters 6.0 4.0

is measured on a ten-point scale, then the two hypothetical sets of datashown in Table II - both of which fit the reported correlations -would lead one to very different conclusions. (A) indicates that thoughindependence is more characteristic of boys than of girls under all con-ditions, maternal employment decreases this sex differential. The datain (B), in contrast, show no such androgynous effect of maternal em-ployment. They imply, rather, that a working mother creates moreindependence in her daughter than in her son, whereas a mother who isprimarily at home fosters more independence in her son.

The most open kind of presentation would give the actual distribu-tions of the variables under consideration. But such a format would beoverwhelming to the reader and thus a focus must be introduced. It ishere that the strategy of reduction (1,3,b) comes in (cf. Zeisel, 1968).Its purpose is to reduce the complexity that would result from a fullpresentation by focusing on one particular aspect of a distribution. Thismight be on its central point, for example, or on the weighting at oneor both of its extremes - depending, of course, on the particular con-clusions the presented data support. Such reduction is always necessaryto a certain extent, but is most crucial when more than one variable isinvolved.

TABLE IIIEngineer s alien ation from work by present position and people-o rientationPresent position of initial staff engineers Value placed on working with people

Staff engineerEngineering managerGeneral manager

High

61%a (A= 51)31% (N = 45)17% (N = 64)

Low19% (!v = 58)18% (N = 39)1% (N = 15)

a The figures in the table represent the percentage in each cell who are aliena ted from work: the% who fall at the Low end of an Index of Work Involvement.


13/21

109Table III (adapted from Bailyn and Schein, in prep.) is a relatively

open, yet reduced table. Al l three of the variables represented in it aregiven in reduced form. The stub variable present position (whichdefines the rows of the table) focuses on only those respondents whosefirst jobs were in engineering staff positions and further eliminatesthose of this group whose present jobs do not fall into the three listedcategories. The reduction in the column variable, people-orientation, isnot obvious from this table alone. But readers of the full report wouldknow that the focus here is on the two extremes of the distribution,with a large group of respondents, whose orientation is neither primar-ily to people nor to things, not represented at all. Finally, the presenta-tion of the cel l variable of work involvement focuses entirely on thealienated end of the distribution.

These reductions make it possible to see the main point of the tableclearly. It shows that present position and people-orientation interactwith each other in their effect on work involvement: alienation fromwork is primarily the response of those engineers who very much valueworking with people but who have not moved into managerial posi-tions.

But the table is also sufficiently open and complex to present a fairamount of additional information to an interested reader. It shows, forinstance, that there are main effects as well as interaction effects:respondents who greatly value working with people are more alienatedin technically based careers than are those whose orientation is more tothings; so, too, are those who currently work in staff positions, partic-ularly when compared to those whose work involves general managerialduties. Even further removed from the main point of the table, becauseit ignores the focal variable of work involvement, is the informationimplicit in the cel l frequencies. These show that engineers who .valueworking with people are much more likely to have moved into manage-ment than are those whose values are not centered on people.

To summarize, the presentation of data must be sufficiently focusedso that the reader can easily assimilate the central research results. Atthe same time, it must be open enough to allow interested readers tomake their own inferences and thus to evaluate the cognitive processthe analyst went through to reach the presented conclusions.

Now, as we move to the strategies that flow from the second analyticprinciple - the interplay between concepts and data - we see a shift inemphasis. Where the first set was designed primarily to maximize thepotential cognitive yield of data, this set serves also the function of con-trolling and validating the process of analysis.


14/21

1102,l: Interplay between concepts and data in collection stage

In one sense, the interplay between concepts and data in the stage ofdata collection is obvious. It involves the problems of measurement anddefining operations that are dealt with in every textbook. Though theseaspects are clearly important, I will not deal with them here. Rather, Iwill concentrate on a strategy that is crucial to research viewed in themanner of this paper - defining the structure of the variables (2,l ,a).

By structuring the variables I mean defining conceptually distinctcategories of information and specifying the particular variables that fitinto each. The tripartite partition of variables into those measuring out-comes (dependent variables), determinants (independent variables), andconditions (intervening variables), represents an often used scheme[ 101. Allocating specific variables into such general categories is neces-sary to guide the research process in all phases: even as simple a task asdeciding which way to run percentages depends on some notion ofstructure among the variables involved. Nonetheless, it is a strategy thatis peculiarly relevant to the collection phase because a clear structure toguide that stage greatly eases all analytic tasks. Indeed, one of thereasons that secondary analysis is difficult is that such a structure mustbe evolved at the same time that the data is being compiled.

Generally, the strategy of structuring the variables guides the collec-tion phase by helping the analyst anticipate the kinds of data that mustbe collected and pointing out areas where potentially important surplusdata may be found. More specif ically, such a structure can helpresearchers in the actual development of their instruments. The follow-ing example on the individual level shows how even a very general ini-tial structuring can help one design a questionnaire.

Suppose one wants to distinguish among behavior, attitudes, and per-sonality dispositions, but is dependent for all ones measurement on awritten questionnaire. If, as is most likely, one associates behavior withoutcome and personality dispositions with determinants, one can beguided by this structure in wording ones questions. Thus one couldintroduce questions that deal with anticipated reactions to hypotheticalsituations to measure outcome (see, e.g., Evans, 1974); and one couldword questions in a manner designed to negate, as much as possible, theeffect of specific situations when trying to tap personality dispositions.In some sense, of course, all questionnaire responses are on the samelevel - representative, perhaps, only of respondents attitudes or theattitudes they feel are called forth by the questions. But, by beingaware of the structure of ones variables from the start, and letting itguide the development of ones instrument, the usefulness and probablevalidity of such an approach are maximized.


15/21

1112,2: Interplay between concepts and data in compilation stage

In the compilation phase, the interplay between concepts and data isparticularly important. Without repeated links between the conceptualand empirical planes at this stage there would be no way to control thevalidity of the results emerging from the techniques designed specif-ically to maximize the cognitive potential of the data. Al l the strategiesin this cell fall under the general heading of multiple corroboration -an analytic form of triangulation (Webb et al., 1966) applicable evenwithin a single, if sufficiently complex, measurement instrument.

Theoretical generalization (2,2,a, see Rosenberg, 1968) refers to theuse of alternate indicators for a concept to test the generalizability of aparticular result. In the example given above I used different indicesto represent accommodation in the English and in the M.I.T. samples.Far from undermining the value of such a replication, I feel itstrengthens the case. Within the English study itself, my initial attemptsto disprove the negative relation between womens work and familysatisfaction by switching indicators, inadvertently strenthened the orig-inal finding, by showing that the association was not dependent onchance covariation of the particular measures I had first used.

It is interesting to speculate on the relevance of this strategy for therepeated use of the same instrument. Would students of leadership, forexample, have more confidence in Fiedlers theories (1967, 197 1) if hedid not always rely on the LPC (Least Preferred Coworker) to measureleadership style? But such speculation goes beyond the scope of what ismeant by theoretical generalization in the present context, where itrepresents a way of linking concepts with alternative indicators in orderto test the validity of a particular finding.

The empirical test of an interpretation (2,2,b) is the essential featureof what Rosenberg (1968) has called the logic of survey analysis -though in my mind it is applicable to the analysis of any kind of data -and is one of the reasons that it is important to have surplus data. It is acrucial analytic step because it provides a way to control and validatethe process of data dredging (Selvin and Stuart, 1966). It does thisby prescribing a procedure that guards against the extremes of stayingtoo much in either the conceptual or the empirical plane. First, bymaking one think about ones findings and generate explanations forthem, it forces researchers to link their empirical results to the concep-tual plane. And then it requires analysts to corroborate these post hocinterpretations by deducing testable consequences from them andactually checking these with the data at hand.

The use of income to test the original finding in the example givenabove illustrates the strategy. The interpretation to be tested was that


16/21

112the association between working wives and unhappiness could beexplained by differences in family income. Notice that showing thatincome relates to both womens work and family strain - which couldhave been established by argument or by others research results - isnot a sufficient test of the interpretation. The existence of such rela-tions only makes it possible for income to account for the initial find-ing. Empirical corroboration, however, implies that it actually does so;and this cannot be known unless the association is looked at whenincome is controlled. In the analysis of the English data, it will berecalled, the necessary first order relations did hold but the originalassociation did not disappear when income was held constant. Hencethe hope that income might account for the init ial finding had to berelinquished.

The final strategy in this cell, the use of quantitative and qualitativeindicators (2,2,c), refers to a principle of corroboration based on anotion of opposites: if you have qualitative data, quantify whereverpossible; if your data base is quantitative, dont ignore its qualitativepotential. The former is relatively obvious. It deplores the use ofunspecific quantifiers - e.g. few; many; in almost every case -in the analysis of qualitative data. It is based on the assumption that byforcing the analyst to seek indicators that can actually be counted andto present results in quantitative terms, the reliabil ity of qualitativeanalysis is in creased.

The opposite, the use of qualitative indicators with quantitative data,is perhaps more surprising. The principle is the same: switching thecharacter of indicators is not possible without linking the empirical andconceptual planes, and such links serve as corroboration of findings.Though the application in this case is less obvious, once the principle ofusing qualitative indicators in the analysis of quantitative data isaccepted, the possibil ities are limited only by the creative imaginationof the researcher.

Let us assume, for example, that our quantitative data base consistsof responses to a completely structured questionnaire. There is noqualitative data, and yet certain qualitative strategies are available tothe analyst which can increase the cognitive potential of the data aswell as serve a corroborating function. Here only two will be men-tioned. The first refers to an emphasis on content in addition to fre-quency. Thus one might look at individual items that comprise scales,and select those, for instance, that most differentiate two groups. Con-sideration of the actual wording of these key items might provide anunderstanding that would not have emerged if one had limited oneselfto total scale scores. In the context of a structured questionnaire, the


17/21

113content of such individual items represents qualitative data.

The second possible qualitative strategy refers to a holistic emphasis(cf. Weiss, 1966): analysis by people as well as by variables. With struc-tured questionnaire data from large samples this could take a number offorms. For instance, one could explicate or validate an index by lookingat the questionnaires of the people who fall at its extremes (see, e.g.,Bailyn and Schein, in prep., Chpt. 5). Or, one could try to understandthe meaning of an empirical association by analyzing the people whoseresponses do not show the expected correlation (cf. Kendall and Wolf,1949). In either case, the whole questionnaire of a person represents aricher and more qualitative datum than do the response frequencies toa particular set of questions, and thus provides a potential for corrobo-ration not often utilized in questionnaire studies.

It should be noted, parenthetically, that these strategies have tacticalimplications for the computer processing of questionnaire data. Theyrequire, for example, that identification numbers be entered as retriev-able information. Also, they point to the importance of not deletingresponses to individual items once scale scores have been derived.2,3: Interplay between concepts and data in presentation stage

In the last cell of the chart, the interplay between concepts and dataserves a communication function: it allows readers to follow the pro-cess by which conclusions have been derived from the data, and permitsthem to judge these conclusions in a larger context. Links to the litera-ture (2,3,a) are often very useful at this stage, and have their place inthe results section of research reports as much as in the introduction.

The final strategy that flows from the view of research presentedhere is sometimes viewed with suspicion. To interweave results andinterpretations (2,3,b) in a research report goes against most publica-tion norms, where results and conclusions are neatly separated bysection headings. It should be obvious by now that such a presentationfollows the principles of validating research, where a particular formula-tion is validated by a single empirical datum. Such a datum is then, cor-rectly, presented in a results section which resides between two ver-sions of the conceptual formulation: the unvalidated one (presented ashypotheses) and the validated one given in the conclusions section.But to force results of research undertaken in the cognitive vein intosuch a format does disservice to the process involved as well as to thereader. It is far better to admit what one is doing and lay the processopen to inspection by others, thus maximizing the possibi lity of pro-ceeding from a cognitive phase of research to a validating one. And thispurpose is better served by documenting the way results were inter-


18/21

114preted and how these interpretations were tested empirically, a docu-mentation that cannot follow the traditional format.

These then are some of the strategies that flow from viewing researchas a cognitive process. They would, I believe, if properly used, enrichthe analysis of empirical data.

I would like to conclude with an anecdote. One night I came intothe kitchen to find my husband opening drawer after drawer and mut-tering to himself that he didnt know what he was looking for. Since hehad an empty bowl in his hand, and a container of ice cream was on thetable, it seemed to me obvious that he was looking for the ice creamscoop which, as a matter of fact, was lying on an adjacent counter. Buthaving momentarily forgotten the ice cream, he continued to opendrawers in a seemingly random fashion, This is the obvious part of theprocess: you cant find something unless you know what you are look-ing for. But there is another, equally important part, which relates tothe ability to perceive the relevance of something you find for under-standing the problem at hand. It occurred to me, as I watched my hus-band, that if by accident a bright red plastic spoon had caught his atten-tion, it might have reminded him - perhaps by his reflecting on thefunctions of spoons - of what he was lacking. Unfortunately, there wasnothing in those kitchen drawers that was arresting in this sense, and sothe search continued.

It is the research strategies that aid in identifying and utilizing sucharresting findings - findings that become fruitful, heuristic, generativeas part of a continuous process of conceptualization - that have beenthe subject of this paper.

Notes1 I am using the word data in two se nses: f i rst, as a singular col lective noun to

refer to the totality of available empirical informa tion; and seco nd, in its moreusual sen se, to refer to specific results from a study, in which case i t takes theplural form.

2 Such a process is obviously possible with al l k inds of data. In the social sciencesit has been mo st expl ici tly recognized by advocates of qual i tat ive research (e.g.Glaser and Stra uss, 1967 ; Becker, 1958). But Polyas (1954) distinctionbetween demonstrative and plausible reasoning, which appl ies to mathe matics,is similar; and just recently an appeal for the collection of data for more thanval idation - for close observation of effects the hypothesis under test saysnothing about (p. 122) - has come from within scienti f ic psychology i tself(Cronbach, 1975).


19/21

1153 The fact that empirical research and social theory are mutual ly dependent on

each other w as already documented by Merton (1949) almost thirty years ago.4 A classic exception is Ham mond (1964). More recently, contributors to theFestsc hri f t to Herman Wold (Dalenius et al . , 1970), were asked to report your

personal experience of the research process and how new knowledge is createdin the process (p. 18). In the present context the chapter by Johan Galtung,On the Structure of Cre ativity (1970, pp. 43-61), is particularly relevant.

5 The Conversational Mode Survey Analysis Program developed by L. Haw kinsof Survey A nalysis Ltd., London.

6 Mem bers of the Organization Studies Group at M. I.T., part icularly Edgar H.Schein, John Van Maanen, and mys elf, are currently pursuing some of the theo-retical impl ications of such an interactive view of careers.

7 This view, though not that expressed in standard texts on research m ethods(see, e.g., Kerlinger, 1973), is gaining currenc y in discus sions of the sociology ofscience (e.g. Mitroff and Mason , 1974; Kuhn, 1970).

8 Som etime s wh at is pleasing w ill be emp irical corroboration of an intuitively ortheoretical ly obvious point; at other t ime s one is mo st pleased by a concep-tually clear interpretation of an unexpe cted or contra dictory finding. The crite-ria of closure actually used in any given case will depe nd, of cours e, on the par-ticular problem and researcher involved.

9 The concept of relat ive deprivation, for instance, seem s to have evolved inthis way (Stouffer et al . , 1949).

10 The technique of path analysis forces researchers to provide a very specif icstructure of their variables, one which also includes the causal l inks among thevariables. Path analysis, therefore, would seem to be uniquely suited to effectthis strategy. I t does not, however, f i t some of the other specif icationsdescribed in this paper - particularly not those relating to cogn itive com plexityof data during comp ilation (1,2). It probably is more usefu l, therefore, in avalidating than in a cognitive research phase . Furthe r, it is not alwa ys poss ibleto predetermine causa l links, though if data collection is guided by a clear struc-ture of the variables involved in the research , the likelihood of accura tely speci-fying these l inks is greatly increased.

ReferencesArgyris, C., and Schi jn, D .A. (1974). Theory in practice: Increasing professional

effectiveness. San Francisco: Jossey-B ass.Bai lyn, L. (1965). Notes on the role of choice in the psychology of professional

wom en, in R. J. Li f ton, ed., The wom an in America. Boston: Houghton Miff l in.Bailyn, L. (1970). Career and fam ily orientations of husban ds and wiv es in rela-

t ion to marital happiness, Hum an ReZ utions 23: 97-l 13.Bai lyn, L. (1973). Family constraints on wom ens work, Annals of the New Y ork

Academ y of Sciences 208: 82-90.Bai lyn, L. (In press). Involvement and accomm odation in technical careers: An

inquiry into the relat ion to work at mid-career, in J. Van Maanen, ed., New per-spe ctives in organizational caree rs. London: Wiley International.Bai lyn, L., and Schein, E.H. (In prep.). Work involvement in technical ly based

careers.. A study of M. I.T. alumni at mid-career.


20/21

116Becker, H.S . (1958) Problems of inference and proof in part icipant observation,

American Sociological Review 23: 652-660.Cronbach, L.J. (1975). Beyond the two discipl ines of scienti f ic psychology,

American Psychologist 30: 116-127.Dalenius, T., Karlsson, G., and Malm quist, S., eds. (1970). Scientists at work: Fest-

schri f t in honour of Herman Wold. Stockholm : Almquist and Wiksel l .Evans, P.A.L. (1974) The price of succe ss: Accomm odation to confl icting needs in

managerial careers. Unpubl ished doctoral dissertation, M.I .T.Fiedler, F.E . (1967). A theory of leadership effectiveness. New York: McGraw -Hil l .Fiedler, F.E. (1971). Leadership. Morristown, N. J.: General Learning P ress.Fogarty, M.P ., Rapoport, R., and Rapoport, R.N . (1971). Sex, career, and family.

London: Al len and Unw in.Fry, R. (In prep.). Diagnosing learning environments: An observational scheme for

matching learner styles with levels of si tuational comp lexity. Doctoral disserta-t ion in preparation, M. I.T.Galtung, J. (1970). On the structure of creativi ty, in T. Dalenius, G. Karlsson,

and S. Malmq uist, eds., Scientists at work : Festsc hri f t in honour of HermanWeld. Stockholm : Almquist and Wiksel l .

Glaser, B.G., and Strauss, A. (1967). The discovery of grounded theory, Chicago:Aldine.

Hal l , R.H . (1963). The concept of bureaucracy: An empirical assessm ent, Amer-ican Journal of Sociology 69: 32-40.

Ham mond , P.E., ed. (1964). Sociologist at work: Essa ys on the craft of socialresearch. New York: Basic Books.

Kendal l, P.L., and Wo lf, K .M. (1949). The analysis o f deviant cases in comm unica-t ions research, in P.F . Lazarsfeld and F.N . Stanton, eds., Com municationsresearch 1948-l 949. New York: Harper.

Kerl inger, F.N . (1973). Foundations of behavioral research (2nd ed.). New York:Holt, Rhinehart, and Winston.

Kuhn, T.S . (1970). Postscripts-1969, in The structure of scienti f ic revolutions(2nd, enlarged ed.). Chicago: University of Chicago P ress.

Macc oby, E.E., and Jackl in, C.N . (1974). The nature of the evidence, in Thepsychology of sex differences. Stanford: Stanford U niversity Press.

McGuire, W .J. (1943). The Yin and Yang of progress in social phychology: Sevenkoan, Journal of Person ality and Social Psyc hology 26: 446-456.

Merto n, R. K. (1949). The bearing of emp irical research on sociological theory, inSocial theory and sociaE structure: Toward the codif icafion of theory andresearch. Glencoe, I l l . : Free Press.

Mitroff, I . I . , and Maso n, R.O. (1974). On evaluating the scienti f ic contribution ofthe Apol lo moon mission via information theory : A study of the scientist-scien-tist relationship, Man agem ent Science 20: 1501-l 5 13.

Polya, G. (1954). Induction and analogy in mathe matics. Vol. 1 of Mathe maticsand plausible resoning. Princeton: Princeton Un iversity Pres s.

Rosenberg, M. (1968). The logic of survey analysis. New York: Basic Books.Selvin, H.C ., and Stuart, A. ( 1966). Data-dredging procedures in survey analysis,

Am erican Statistician 20(3): 20-23.Stouffer, S.A., Suchma n, E.A., DeVinney, L.C ., Star, S.A., and Wil l iams, R.M .

(1949) The Am erican soldier: Adjus tmen t during arm y life. Vol. 1 of Studiesin social phychology in World W ar I1. Princeton: Princeton University Press.


21/21

117Webb, E.J., Camp bell , D.T ., Schw artz, R.D ., and Sechrest, L. (1966). Unobtrusive

measures. Nonreactive research in the social sciences. Chicago: Rand McN ally.We iss, R.S . (1966). Alternative approaches in the study of complex si tuations,Hum an Organization 25: 198-206.Zeisel, H. (1968). Say i t with f igures (5th, revised ed.). New York: Harper and

Ro w.

bailyn - research as a cognitive process

Documents