factor analysis:

Timothy D. Kruse, M.S.Ed. Texas A&M University Commerce.

1

Factor Analysis:

A Brief Synopsis of Factor Analytic Methods With an

Emphasis on Nonmathematical Aspects.


2

Factor Analytic Methods

Factor analysis is a set of mathematical techniques used to identify dimensions underlying a set of empirical measurements.


3


Factor analysis is a set of mathematical techniques used to identify dimensions underlying a set of empirical measurements.

It is a data reduction method in which several sets of scores (units) and the correlations between them are mathematically considered.


4


It is an extremely complex procedure that contains numerous, inherent nuances and variety of correlational analyses designed to examine interrelationships among variables; a basic understanding of geometry, algebra, trigonometry and matrix algebra is required.


5

Fundamental Purposes

Factor analytic methods can help scientists to define their variables more precisely and decide what variables they should study and relate to each other in the attempt to develop their science to a higher level (Comrey & Lee, 1992)


6


…the aim is to summarize the interrelationships among the variables in a concise but accurate manner as an aid in conceptualization (Gorsuch, 1983).


7


…a statistical technique applied to a single set of variables when the researcher is interested in discovering which variables in the set form coherent subsets that are relatively independent of one another (Tabachnick & Fidell, 2001).


8


…a statistical technique applied to a single set of variables when the researcher is interested in discovering which variable in the set form coherent subsets that are relatively independent of one another (Tabachnick & Fidell, 2001).

…reducing numerous variables down to a few factors (Tabachnick & Fidell, 2001).


9


All scientists attempt to identify the basic underlying dimensions that can be used to account for the phenomena they study.


10


All scientists attempt to identify the basic underlying dimensions that can be used to account for the phenomena they study.

Scientists analyze the relationships among a set of variables where these relationships are evaluated across a set of individuals under specific conditions.


11


…is to account for the intercorrelations among n variables, by postulating a set of common factors, fewer in number than the number, n, of these variables (Cureton & D’Agostino, 1983).


12


In other words, factor analytic methods assist the researcher in gaining a more comprehensive understanding and conceptualization of complex and poorly defined interrelationships that exist in a large number of imprecisely measured variables.


13

Goals and Objectives

To summarize patterns of correlations (in matrix) among observed variables.


14



To reduce a large number of observed variables to a smaller number of factors.


15




To provide an operational definition (a regression equation) for a process underlying observed variables.


16




To provide an operational definition (a regression equation) for an underlying process of observed variables.

To test a theory of underlying processes.


17

Required Parlance / Lexicon

Variables – the characteristics being measured and can be anything that can be objectively measured or scored.


18


Variables – the characteristics being measured and can be anything that can be objectively measured or scored.

Individuals – the units that provide the data by which the relationships among the variables are evaluated (subjects, cases, etc.)


19


Conditions – that which pertains to all the data collected and sets the study apart from other similar studies (time, space, treatments, scoring variations, etc.).


20


Conditions – that which pertains to all the data collected and sets the study apart from other similar studies (time, space, treatments, scoring variations, etc.).

Observations – a specific variable score of a specific individual under the designated conditions.


21


Factors– Hypothetical constructs or theories that help

interpret the consistency in a data set (Tinsley & Tinsley, 1987).

– A dimension or construct that is a condensed statement of the relationship between a set of variables (Kline, 1994).

– Hypothesized, unmeasured, and underlying variables (Kim & Meuller, 1978).


22


Factors – specific variables that are presumed to influence or explain phenomenon (i.e., test performance); reflect underlying processes or constructs that have created the correlations among variables.– Sometimes referred to as “latent variables.”


23


Common Factors– Represent the dimensions that all the measures

have in common.


24


Common Factors– Represent the dimensions that all the measures

have in common.

Specific Factors– Are related to a specific variables but are not

common to any other variables.


25


Common Factors– Represent the dimensions that all the variables

have in common. Specific Factors

– Are related to a specific variable but are not common to any other variables.

Error Factors– Represent the error of measurement or

unreliability of a variable.


26


Factor Loading – the farther the loading on a factor from zero, the more one can generalize from that factor to the variable; reflects a quantitative relationship.


27


Factor Loading – the farther the loading on a factor from zero, the more on can generalize from that factor to the variable; reflects a quantitative relationship. – The extent to which the variables are related to

the hypothetical factor.


28



the hypothetical factor.– May be thought of as correlations between the

variables and the factor.


29



the hypothetical factor.– May be thought of as correlations between the

variables and the factor.– Sometimes referred to as “saturation.”


30


Observed Correlation Matrix – matrix of observed variables (i.e., standard test score).


31



Reproduced Correlation Matrix – matrix produced by the factor model.


32



Reproduced Correlation Matrix – matrix produced by the factors.

Residual Correlation Matrix – matrix produced by the differences between observed and model matrices.


33


Rotation – is a process by which the solution is made more interpretable without changing its underlying mathematical properties.


34


Rotation – is a process by which the solution is made more interpretable without changing its underlying mathematical properties.– Orthogonal rotation – all factors are

uncorrelated with each other.• Produces loading & factor-score matrices.


35


Rotation – is a process by which the solution is made more interpretable without changing its underlying mathematical properties.– Orthogonal rotation – all factors are

uncorrelated with each other.• Produces loading & factor-score matrices.

– Oblique rotation – factors are correlated.• Produces structure, pattern, & factor-score matrices.


36

Uses of Factor Analysis

Finding underlying factors of ability tests. Identify personality dimensions. Identifying clinical syndromes. Finding dimensions of satisfaction. Finding dimensions of social behaviors.


37

Uses of Factor Analysis

In psychology - the development of objective tests and assessments for the measurement of personality and intelligence.– Explain inter-correlations.– Test theory about factor constructs.– Determine effect of variation / changes.– Verify previous findings.


38

Types of Factor Analysis

Exploratory (EFA) – the researcher attempts to describe and summarize data by grouping together variables that are correlated.


39


Exploratory (EFA) – the researcher attempts to describe and summarize data by grouping together variables that are correlated.– The variables may or may not have been chosen

with potential underlying processes in mind.


40


Exploratory (EFA) – the researcher attempts to describe and summarize data by grouping together variables that are correlated.– The variables may or may not have been chosen

with potential underlying processes in mind. – Used in the early stages of research to

consolidate variables and generate hypotheses about possible underlying processes or constructs.


41


Confirmatory (CFA) – used later in research (advanced stages) to test a theory regarding latent underlying processes / constructs.


42


Confirmatory (CFA) – used later in research (advanced stages) to test a theory regarding latent underlying processes / constructs.– Variables are specifically chosen to reveal

underlying processes / constructs.


43


Confirmatory (CFA) – used later in research (advanced stages) to test a theory regarding latent underlying processes / constructs.– Variables are specifically chosen to reveal or

confirm underlying processes / constructs.– Much more sophisticated technique than EFA.


44

The Fundamental Equation of Factor Analysis

The first step…

zjk = aj1F1k + aj2F2k + … +

ajmFmk + ajsSjk + jeEjk


45

The Fundamental Equation of Factor Analysis

Given the limited time available…let’s don’t and say we did.


46

Factor Analytic Steps and Procedures

1- Select and measure a set of variables.


47


1- Select and measure a set of variables. 2- Compute the matrix of correlations

among the variables.


48



among the variables. 3- Extract a set of unrotated factors.


49



among the variables. 3- Extract a set of unrotated factors. 4- Determine the number of factors.


50



among the variables. 3- Extract a set of unrotated factors. 4- Determine the number of factors. 5- Rotate the factors if needed to increase

interpretability.


51



among the variables. 3- Extract a set of unrotated factors. 4- Determine the number of factors. 5- Rotate the factors if needed to increase

interpretability. 6- Interpret the rotated factor matrix.


52

Psychological Diagnostic Interview

A case example of a formal psychological / psychiatric intake session will be utilized to display the aforementioned factor analytic steps.


53

Step One

Select and measure a set of variables.

The clinician’s observation of the patient’s mood, affect, behavior, cognitions and their description of the presenting problem or chief complaint during the intake session.


54

Step Two

Compute the matrix of correlations among the variables.

The clinician attempts to understand how the themes that were observed “fit” together; what observations were similar to one another (i.e., physiological or motor disturbances, form or content of thought, perception).


55

Step Three

Extract a set of unrotated factors.

The clinician makes decisions based on what everything observed had in common. The clinician begins to assess for processes and underlying dimensions (i.e., anxiety & depression).


56

Step Four & Five

Determine the number of factors.

Rotate the factors if needed to increase interpretability.

These steps involve the clinician making a decision based on the relative weights, predominance, or importance of each of the aforesaid dimensions.


57

Step Six

Interpret the rotated factor matrix.

The clinician establishes a formulation or theoretical conceptualization, based on themes of observations, and develops a tentative treatment plan.


58

Standard FA Example Mathematics & Verbal Ability

Suppose you want to study mathematics and verbal ability.


59


Suppose you want to study mathematics and verbal ability.– Research literature to develop test plan.


60


Suppose you want to study mathematics and verbal ability.– Research literature to develop test plan.– Five items that best measure these abilities.


61



• Vocabulary, Algebra, Word Analogy, Geometry, Algebra Word Problem.


62




• Actual research = a vast and nebulous number of items.


63

Standard FA ExampleMathematics & Verbal Ability



• Actual research = a vast and nebulous number of items.

– Compute matrix of intercorrelations (SPSS).

Cohen, Swerdlik, & Phillips (1996)

64

Mathematics & Verbal AbilityTable 1 The Matrix of Intercorrelations Among the Five Items

1 2 3 4 5

1. Vocab 1.00 .22 .77 .20 .50

2. Algebra .22 .1.00 .21 .65 .48

3. Analogy .77 .21 1.00 .19 .52

4. Geometry .20 .65 .19 1.00 .47

5. Alg-Word .50 .48 .52 .47 1.00


65

Matrix of Intercorrelations Mathematics & Verbal Ability

Each entry = correlation coefficient between 2 items.

1 2 3 4 5

1.Vocab 1.0 .22 .77 .20 .50

2.Albegra .22 .1.0 .21 .65 .48

3.Word Analogy

.77 .21 1.0 .19 .52

4.Geometry .20 .65 .19 1.0 .47

5.Algebra-Word

.50 .48 .52 .47 1.0


66


Each entry = correlation coefficient between 2 items.– Vocab & word analogy

= .77

1 2 3 4 5

1.Vocab 1.0 .22 .77 .20 .50

2.Albegra .22 .1.0 .21 .65 .48

3.Word Analogy

.77 .21 1.0 .19 .52

4.Geometry .20 .65 .19 1.0 .47

5.Algebra-Word

.50 .48 .52 .47 1.0


67



= .77

– Algebra & geometry = .65

1 2 3 4 5

1.Vocab 1.0 .22 .77 .20 .50

2.Albegra .22 .1.0 .21 .65 .48

3.Word Analogy

.77 .21 1.0 .19 .52

4.Geometry .20 .65 .19 1.0 .47

5.Algebra-Word

.50 .48 .52 .47 1.0


68



= .77

– Algebra & geometry = .65

Results suggest…

1 2 3 4 5

1.Vocab 1.0 .22 .77 .20 .50

2.Albegra .22 .1.0 .21 .65 .48

3.Word Analogy

.77 .21 1.0 .19 .52

4.Geometry .20 .65 .19 1.0 .47

5.Algebra-Word

.50 .48 .52 .47 1.0


69



= .77.

– Algebra & geometry = .65.

Results suggest…– 2 underlying factors.

1 2 3 4 5

1.Vocab 1.0 .22 .77 .20 .50

2.Albegra .22 .1.0 .21 .65 .48

3.Word Analogy

.77 .21 1.0 .19 .52

4.Geometry .20 .65 .19 1.0 .47

5.Algebra-Word

.50 .48 .52 .47 1.0


70



= .77.

– Algebra & geometry = .65.

Results suggest…– 2 underlying factors.

– Algebra word may be associate with both.

1 2 3 4 5

1.Vocab 1.0 .22 .77 .20 .50

2.Albegra .22 .1.0 .21 .65 .48

3.Word Analogy

.77 .21 1.0 .19 .52

4.Geometry .20 .65 .19 1.0 .47

5.Algebra-Word

.50 .48 .52 .47 1.0


71

Standard FA Example (Cont.)

Mathematics & Verbal Ability

The next step is to factor the intercorrelations matrix (SPSS).


72



The next step is to factor the intercorrelations matrix (SPSS).– Factor loadings can be treated like correlations

between the measure and the underlying factors.


73





– Determine magnitude of factor loading.


74





– Determine magnitude of factor loading.• Look for salient (significant) factor loadings.


75





– Determine magnitude of factor loading.• Look for salient (significant) factor loadings.

– Cattell (1978) proposed .30 for N > 100, and .40 for N < 100.

Cohen, Swerdlik, & Phillips (1996)

76

Mathematics & Verbal AbilityTable 2 Results of Factor Analysis of Five Items

Salient Factor Loadings, N > 100

Factor I Factor II Communality

Vocabulary .917 .101 .851

Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word .594 .573 .681

Eigenvalue 2.700 1.30

% total Variance

54.00 26.00


77

Mathematics & Verbal Ability Vocabulary, Analogy, Algebra

& Geometry are Factorially Simple because they load on only one factor; reflect one dimension.



Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word

.594 .573 .681


% total Variance

54.00 26.00


78

Mathematics & Verbal Ability Vocabulary, Analogy, Algebra

& Geometry are Factorially Simple because they load on only one factor; reflect one dimension.

Algebra-Word is considered Factorially Complex because it loads on both factors; reflects more than one dimension.



Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word

.594 .573 .681


% total Variance

54.00 26.00


79

Mathematics & Verbal Ability Two factors named.Verbal

AbilityMathematical

AbilityCommunality


Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word

.594 .573 .681


% total Variance

54.00 26.00


80

Mathematics & Verbal Ability Two factors named.

Eigenvalue (or characteristic root) indicates the relative strength of each factor.– Range from 0.0 to # of

measures factored.

Verbal Ability

Mathematical Ability

Communality


Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word

.594 .573 .681


% total Variance

54.00 26.00


81

Mathematics & Verbal Ability Two factors named.

Eigenvalue (or characteristic root) indicates the relative strength of each factor.– Range from 0.0 to # of

measures factored.

% of Total Variance reflects that the Verbal Ability factor (54%) is twice as strong as the Mathematical Ability factor (26%).

Verbal Ability


Communality


Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word

.594 .573 .681


% total Variance

54.00 26.00


82

Mathematics & Verbal Ability Communality…Verbal

AbilityMathematical

AbilityCommunality


Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word

.594 .573 .681


% total Variance

54.00 26.00


83

Mathematics & Verbal Ability Communality…

– assesses how well each measure is explained by the common factors.

Verbal Ability


Communality


Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word

.594 .573 .681


% total Variance

54.00 26.00


84



– indicates the extent to which variables overlap with factors.

Verbal Ability


Communality


Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word

.594 .573 .681


% total Variance

54.00 26.00


85



– indicates the extent to which variables overlap with factors.

– provides proportion of variance in the variables that can be accounted for by the scores in the factors.

Verbal Ability


Communality


Algebra .113 .885 .796

Analogy .925 .094 .864

Geometry .086 .891 .801

Algebra-Word

.594 .573 .681


% total Variance

54.00 26.00


86


The results of this factor analysis permits the estimation of not only how many factors or dimensions there were with our five items but also the relative importance or strength of each of the factors.


87

Problems with Factor Analytic Methods

No criterion variable against which to test the solution.


88



There is an infinite number of rotations available.


89



There is an infinite number of rotations available.

Often used to correct improperly conceptualized (sloppy) research.


90


[Factor analysis should never be used] as a haphazard method to attempt to make order from chaos; it is totally inappropriate to factor-analyze just any set of measure with the hope of finding meaningful common factors (Cohen, Swerdlik, & Phillips, p. 207, 1996).


91

Thank You!

factor analysis:

Documents

university commerce

single set of variables

numerous variables

n variables

set of individuals

set of empirical measurements

set of mathematical

set of common factors