cross-cultural research on the reliability and validity of the mayer-salovey-caruso emotional...

http://ccr.sagepub.com/

Cross-Cultural Research

http://ccr.sagepub.com/content/early/2010/07/27/1069397110377603The online version of this article can be found at:

DOI: 10.1177/1069397110377603 published online 2 August 2010Cross-Cultural Research

Jahanvash Karim and Robert WeiszMayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT)Cross-Cultural Research on the Reliability and Validity of the

Published by:

http://www.sagepublications.com

On behalf of:

Society for Cross-Cultural Research

can be found at:Cross-Cultural ResearchAdditional services and information for

http://ccr.sagepub.com/cgi/alertsEmail Alerts:

http://ccr.sagepub.com/subscriptionsSubscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

by Jahanvash Karim on September 13, 2010ccr.sagepub.comDownloaded from


XX(X) 1 31

2010 SAGE Publications

Reprints and permission: http://www.

sagepub.com/journalsPermissions.nav

DOI: 10.1177/1069397110377603

http://ccr.sagepub.com

377603 CCR

Universit de Paul Czane, France

Corresponding Author:

Jahanvash Karim, CERGAM, IAE dAix en Provence, Universit de Paul Czane, France, Clos

Guiot PuyricardBP 30063, Aix-en-Provence Cedex 2 13089, France

Email: [email protected]


on the Reliability and

Validity of the Mayer-

Salovey-Caruso Emotional

Intelligence Test (MSCEIT)

Jahanvash Karim and Robert Weisz

Abstract

Despite the rather large literature concerning emotional intelligence, the

vast majority of studies concerning development and validation of emotional

intelligence scales have been done in the Western countries. Hence, a major

limitation in this literature is its decidedly Western focus. The aim of this

research was to assess the psychometric properties of the Mayer-Salovey-

Caruso Emotional Intelligence Test (MSCEIT) in a cross-cultural comparative

context involving the collectivist Pakistani (Eastern culture) and the individualist

French (Western culture) students. With the exception of significant mean

differences on the MSCEIT scores between two cultures, the results concerning

the validity of the MSCEIT generalized nicely across both cultures. The results

from multisample analysis revealed that the MSCEIT has the property of factorial

invariance across both cultures, including invariance of factor loadings, unique

variances, and factor variance. For both Pakistani and French students, the

MSCEIT scores were distinguishable from the Big Five personality dimensions,

self-report emotional intelligence measures, and cognitive intelligence.

Furthermore, in both cultures, the MSCEIT scores failed to demonstrate

incremental validity against well-being measures, after controlling for cognitive

intelligence and the Big Five personality dimensions. Finally, within each sample,

females significantly scored higher than males on the MSCEIT total scores.

Keywords

Emotional intelligence; validity

doi:10.1177/1069397110377603Cross-Cultural Research OnlineFirst, published on August 2, 2010 as


2 Cross-Cultural Research XX(X)

Emotional intelligence (EI) exists and has significant impacts on individual

and organizational outcomes, ranging from individual performance, health,

and psychological well-being, to customer satisfaction and organizational

performance (Joseph & Newman, 2010; Schutte, Malouff, Thorsteinsson,

Bhullar, & Rooke, 2007). Indeed, Few fields of psychological investigation

appear to have touched so many disparate areas of human endeavor,

since its inception, as has emotional intelligence (Matthews, Zeidner, &

Roberts, 2004, p. 4). However, despite notable advances in the field, the

psychometric properties of EI instruments have seldom been examined with

demand and rigor across cultures, often leaving open questions of structural

and measurement equivalence. If an EI measure fails to show comparable

psychometric properties across different cultures, then its utility as a

psychological construct is questionable (Ekermans, 2009; Gangopadhyay &

Mandal, 2008; Palmer, Gignac, Ekermans, & Stough, 2008). The current study

sought to address this concern by simultaneously assessing the psychometric

properties of the Mayer-Salovey-Caruso Emotional Intelligence Test

(MSCEIT: Mayer, Salovey, & Caruso, 2002) in two distinct cultural groups:

the collectivist Pakistani and the individualist French. The collectivists tend

to view themselves as members of an extended family (or organization), and

place group interests ahead of individual needs. In contrast, Individualists

tend to believe that personal goals and interests are more important than

group interests (Hofstede, 1980).

A cross-cultural design is an answer to the call made by various research-

ers (e.g., Ekermans, 2009; Gangopadhyay & Mandal, 2008) for more system-

atically investigating cultural differences to determine whether the structure

of EI replicates across distinct cultures and whether correlates of EI are cul-

ture-specific or they cut across cultural boundaries. More specifically, the

current study had five main objectives. First, we compared participants EI

levels across both cultures. Second, we evaluated the structural equivalence

of the MSCEIT across both cultures. Third, we assessed the discriminant

validity of the MSCEIT vis--vis cognitive intelligence (the Ravens

Advanced Progressive Matrices), self-report or mixed model EI measures

(the SREIT and the TEIQue), and the Big Five personality measures. Fourth,

we assessed whether MSCEIT accounts for incremental variance in subjec-

tive well-being (i.e., positive affect, negative affect, and satisfaction with life)

and psychological distress above and beyond Big Five and cognitive intelli-

gence in both cultures. Finally, we examined whether there are gender differ-

ences on the MSCEIT within each culture.


Karim and Weisz 3

Approaches to EI

Two complementary conceptualizations of EIthat is, mixed model frame-

work and ability model frameworkexist side by side in the literature. The

proponents of ability EI framework view EI as a traditional intelligence,

resembling other standard intelligences (e.g., verbal, numerical, figural), com-

prising of a set of skills that combines emotions with cognition measured

through objective tests akin to IQ tests ([MSCEIT: Mayer et al., 2002).

Proponents of mixed models, by contrast, view EI as an eclectic mix (Mayer,

Salovey, &Caruso, 2008) of traits, many dispositional, such as self-esteem,

happiness, impulsiveness, self-management, and optimism, rather than as abil-

ity based. For example, Petrides and Furnham (2003) defined the construct as

a constellation of behavioral dispositions and self-perceptions concerning

ones ability to recognize, process, and utilize emotion-laden information. It

encompasses empathy, impulsivity, and assertiveness as well as elements of

social intelligence and personal intelligence (p. 278). Thus, within these mod-

els, a large number of traits are amassed and mixed in with a few socioemo-

tional abilities (Mayer, Roberts, & Barsade, 2008; Mayer, Salovey et al.,

2008). Researchers in the mixed model framework have typically used self-

report measures to assess EI (e.g., Self-report Emotional Intelligence Test

[SREIT]: Schutte et al., 1998; Trait Emotional Intelligence Questionnaire

[TEIQue]: Petrides, Prez-Gonzalez, & Furnham, 2007). Furthermore, mixed

model measures of EI can typically be organized into one of two complemen-

tary types: self-report ability EI or self-report mixed EI (Joseph & Newman,

2010). The former includes self-report EI measures that are based on ability EI

model (e.g., Self-Rated Emotional Intelligence Scale [SREIS]: Brackett,

Rivers, Shiffman, Lerner, & Salovey, 2006; Self-report Emotional Intelligence

Test [SREIT]: Schutte et al., 1998). The latter includes measures which focus

on noncognitive factors such as social skills, self-esteem, and personality

dimensions (e.g., TEIQue: Petrides et al., 2007). In sum, currently we have

three distinct construct-method pairings of EI: performance-based ability EI,

self-report ability EI, and self-report mixed EI.

Mayer-Salovey-Caruso Emotional Intelligence

Test (MSCEIT: Mayer et al., 2002)

One of the more widely known ability EI models was developed by Mayer

and Salovey (1997), who defined EI as the ability to perceive emotions,

to access and generate emotions so as to assist thought, to understand emo-

tions and emotional knowledge and to reflectively regulate emotions so as to



promote emotional and intellectual growth (p. 10). In line with this opera-

tional definition, the structure of Mayer and Saloveys model is mutifacto-

rial, comprising four conceptually related abilities arranged hierarchically

from the most basic to more psychologically complex. These include (a)

Perceiving Emotions, or the ability to identify emotions in oneself and oth-

ers; (b) Using Emotions, or the ability to use emotions to impact cognitive

processes. This requires the ability to mobilize the appropriate emotions and

feelings to assist in certain cognitive activities such as reasoning, problem-

solving, and decision making; (c) Understanding Emotions, or the ability to

comprehend how emotions combine and how emotions progress by transi-

tioning from one emotion to another; and (d) Managing Emotions, or the

ability to reflectively regulate emotions and emotional relationships.

The Mayer-Salovey-Caruso Emotional Intelligence Test Version 2.0

(MSCEIT; Mayer et al., 2002) is the direct operationalization of Mayer and

Saloveys (1997) ability EI model. In line with ability EI conceptualization,

the MSCEIT measures ones capacity to reason with emotional content and to

use the emotional content to enhance thought. The MSCEIT differs from the

mixed model or trait measures of EI (self-reporting EI measures) as a result of

the nature and style of the assessment. Respondents are asked to solve emo-

tional problems (e.g., how to resolve a conflict with a spouse) rather than

being asked to self-perceive and rate the extent to which their emotional skills

are being used (e.g., rating oneself on 7-point Likert-type scale).

The MSCEIT includes two tasks as measures of each branch: Perceiving

Emotions (faces and pictures); Using Emotions (sensations and facilitation);

Understanding Emotions (blends and changes); and Managing Emotions

(emotion management and emotional relationships). The four branches may

be further grouped into two EI areas: Experiential EI (Perceiving Emotions

and Using Emotions) and Strategic EI (Understanding Emotions and

Managing Emotions).

Scores on the MSCEIT can be obtained through consensus and expert

scoring methods. Consensus scores reflect the proportion of respondents in a

large normative sample who endorsed each MSCEIT response. A score for an

individual is computed by comparing his or her responses to that of the nor-

mative sample. In contrast, expert scores reflect the proportion of 21 emotion

experts who endorsed each response. It is worth to mention that the scoring

methods of the MSCEIT have been the subject of debate and controversy

(e.g., Matthews et al., 2002; Matthews, Roberts, & Zeidner, 2004). Both con-

sensus and expert norms correlate highly (Mayer, Salovey, Caruso, &

Sitarenios, 2003). Mayer et al. (2003) reported acceptable reliabilities for the


Karim and Weisz 5

MSCEIT. The MSCEIT full-test split-half reliability was 0.93 for general and

0.91 for expert consensus scoring. The reliability for four branch scores of

Perceiving, Facilitating, Understanding, and Managing ranged between 0.76

and 0.91. The individual task reliabilities ranged from a low of 0.55 to a high

of 0.88.

Overview of the Current Study

Country Differences on the MSCEIT

Individualism-Collectivism is a major dimension of cultural variable postu-

lated by many theorists (e.g., Hofstede, 1980). This dimension focuses on the

degree a society reinforces individual or collective actions, achievements,

and interpersonal relationships. Collectivism typifies societies of a more col-

lective nature, close ties between individuals, collective goals, and depen-

dence on groups; while individualistic cultures stress individual goals and

independence. For this study, French and Pakistani cultures were selected

because cross-cultural research predominantly involves the comparison of

Eastern and Western cultures. According to Hofstedes (1980) cultural

dimensions, Pakistan is a typical representative of the classical Eastern cul-

ture. France is considered as a prototype of the classical Western culture.

According to Hofstedes rankings (see www.geert-hofstede.com), Pakistan

ranks 14 on individualism which is much lower than the world average of 50,

reflecting an orientation toward a collectivistic culture. France ranks 71,

indicating a society with more individualistic attitudes.

Some research indicates that cultural differences (individualist vs. collec-

tivist) exist across a wide range of emotion-related abilities that essentially

comprise the construct of the ability EI. For example, compared to collectiv-

ists, people from individualistic cultures are better at recognizing and under-

standing emotions (Matsumoto, 1989; 1992), are more likely to express their

emotions (Fernandez, Carrera, Sanchez, Paez, & Candia, 2000), and are bet-

ter able to regulate their emotions ( Gross & John , 2003). Thus the processes

underlying the ability EI factors and their manifestations may differ across

cultures as a consequence of the role culture plays in the development, dis-

play, and interpretation of emotions. Therefore, the first goal was to examine

whether there are cultural differences on the MSCEIT scores across French

(individualists) and Pakistani (collectivists) cultures. Based on literature on

emotions, it was expected that the participants in the French sample would

score higher on the MSCEIT than the participants in the Pakistani sample.



Structural Equivalence

The evidence for structural equivalence can be established by replicating the

factor structure of the MSCEIT and demonstrating that the MSCEIT pos-

sesses robust internal reliability across cultures (Ekermans, 2009).

Regarding the factorial validity of the MSCEIT, Mayer et al. (2003) have

demonstrated that four-factor models provide good fit to the data, suggesting

that this model provides viable representation of the tests underlying factor

structure. However, some have argued that four-factor solution is not prefer-

able due to high correlations between branches perceiving emotions and

using emotions (Fan, Jackson, Yang, Tang, & Zhang, 2010; Roberts, Schulze,

OBrien, MacCann, Reid, & Maul, 2006; Rode et al., 2007; Rossen, Kranzler,

& Algina, 2008) or between using emotions and managing emotions (Palmer,

Gignac, Manocha, & Stough, 2005).

Further to this, an important research question that has yet to be systemati-

cally examined is whether the ability EI construct generalizes across different

cultures. So far, all factor structures of the MSCEIT have evolved only on the

basis of studies done in Western (predominantly individualistic) cultures and

none has assessed the factor structure of the MSCEIT in Eastern (primarily

collectivistic) cultures. It is argued that the construct of EI needs to be vali-

dated in the East (collectivist culture; Gangopadhyay & Mandal, 2008).

When tests are transported from one culture to another, the comparability of

psychological measurements across different cultures should be investigated.

A lack of evidence for measurement invariance across cultures could point

toward bias at the construct level (Ekermans, 2009) and obviates the ability

of the measure to be used in comparisons among different cultural groups.

For instance, one of the major objectives of any cross-cultural study is to

compare the mean level of a certain construct across cultural groups.

Interpretation of the mean differences may be problematic unless the under-

lying constructs are the same or invariant across cultural groups. Therefore,

if the MSCEIT is used to compare mean differences across cultures, the

MSCEIT should have the same meaning across cultural groups. In sum, if

equivalence assumptions remain untested, the practical utility of EI when

utilized across different cultural groups may be questionable (Ekermans,

2009).

The MSCEIT normative sample is based on data collected from more than

5,000 participants, including individuals from both individualistic societies

(e.g., United States, United Kingdom, Canada) as well as from collectivistic

societies (e.g., the Philippines, India, and Slovenia; Papadogiannis, Logan, &


Karim and Weisz 7

Sitarenios, 2009). Thus one can argue that the structure of the MSCEIT will

replicate identical across cultures because of the heterogeneous nature of nor-

mative sample, including collectivists and individualists. It is expected that

ability EI factors are culturally universal and have comparable functions

across cultures.

Discriminant Validity

Discriminant validity is observed when the scores from an EI inventory are

found not to correlate with an inventory that is theoretically postulated to be

unrelated to EI (Gignac, 2009).

MSCEIT and self-report EI measures. Since self-report EI measures assess

emotion-related, self-perceived abilities and traits rather than cognitive abili-

ties per se (as in ability EI), mixed EI and ability EI should be regarded dis-

tinct (Mayer, Roberts et al.,2008; Mayer, Salovey et al., 2008; Petrides, &

Furnham, 2003). Research has consistently supported this distinction by

revealing low correlations between the MSCEIT and various self-report

mixed EI measures (e.g., Brackett & Mayer, 2003; Livingstone, & Day, 2005;

OConnor & Little, 2003) and self-report ability EI measures (Brackett &

Mayer, 2003; Brackett et al., 2006; Joseph & Newman, 2010).

MSCEIT and Cognitive Intelligence. Empirically, there appears to be suffi-

cient discriminant validity between the MSCEIT and various general intelli-

gence measures (Papadogiannis et al., 2009). Various studies have indicated

low to moderate correlations between MSCEIT and measures tapping crys-

tallized intelligence (Gc; e.g., Farrelly & Austin, 2007; Livingstone & Day,

2005; OConnor & Little, 2003; Rode et al., 2007). Interestingly, the MSCEIT

has shown no relation to Ravens Progressive Matrices (Raven, Raven, &

Court, 2003; e.g., Ciarrochi, Chan, & Caputi, 2000; Fabio & Palazzeschi,

2009; Farrelly & Austin, 2007), placing ability EI closer to crystallized

(rather than fluid, i.e., Gf) intelligence within Gf/Gc theory (for details see

Farrelly & Austin, 2007).

MSCEIT and the Big Five. Various studies have well-documented a nonsig-

nificance or low correlations between the MSCEIT and the Big Five person-

ality dimensions, thus providing evidence for the discriminant validity of the

MSCEIT (e.g., Joseph & Newman, 2010; OConnor & Little, 2003; Rode

et al., 2007).

Therefore, the third goal in the present study was to examine the relation-

ship of scores on the MSCEIT with scores on the TEIQue, the SREIT, Ravens



Progressive Advance Matrices, and the Big Five personality dimensions. It

was expected that the scores on the MSCEIT would be unrelated to the scores

on the SREIT, TEIQue, Ravens Progressive Matrices, and the Big Five per-

sonality dimensions.

Incremental Validity

There are many reasons to believe that EI plays an important role in predict-

ing ones subjective sense of well-being and positive mental health. For

example, emotionally intelligent individuals (a) are better able to draw on

positive emotions, which help them to handle anxiety and tolerate distress

even when faced with episodes of negative emotional experiences (Tugade

& Fredrickson, 2001); (b) are more likely to use strategies such as eliciting

social support and disclosure of feelings, in place of maladaptive coping

strategies, such as rumination (Matthews, Emo, Funke, Zeidner, Roberts, &

Costa, 2006); (c) are more likely to retrieve positive memories during mood

induction as an aid to mood regulation (Ciarrochi et al., 2000); (d) have an

advantage in terms of greater social competence, richer social networks, and

more effective coping strategies (Salovey, Bedell, Detweiler, & Mayer,

2000); and (e) are better able to identify and interpret cues that inform self-

regulatory actions to nurture positive affect and avoid negative affect (Mayer

& Salovey, 1997). In a study conducted on undergraduate students, Brackett

et al. (2006) found positive relationship between the MSCEIT and psycho-

logical well-being and life satisfaction.

The fourth goal of this study was to examine whether scores on the

MSCEIT predict scores on measures assessing subjective well-being (posi-

tive affect, negative affect, and life satisfaction) and psychological distress

after controlling for the influence of personality and cognitive intelligence.

As discussed above, MSCEIT is unrelated to fluid intelligence (Gf) and Big

Five personality dimensions; it is expected that MSCEIT will exhibit signifi-

cant incremental validity over Big Five personality traits and cognitive intel-

ligence (i.e., Ravens Advanced Progressive Matrices).

Known Group Validation

Gender differences have been reported consistently in emotions research. For

example, compared with men, women are more accurate in judging the emo-

tional meaning from nonverbal cues (Hall & Matsumoto, 2004), have more


Karim and Weisz 9

complex knowledge (Ciarrochi, Hynes, & Crittenden, 2005), use more emo-

tion regulation strategies (Garnefski, Teerds, Kraaij, Legerstee, & Van den

Kommer, 2004), experience their emotions more intensely (Gross, & John,

1998), show greater emotional awareness (Barrett, Lane, Sechrest, &

Schwartz, 2000), and tend to be more empathic than men (Mehrabian,

Young, & Sato, 1988). More important, the MSCEIT manual (Mayer et al.,

2002) and a recent meta-analytic study by Joseph and Newman (2010) sug-

gest that women score higher on the MSCEIT four factors than men do.

Therefore, the fifth and final research goal was to examine whether there are

gender differences on scores on the MSCEIT. Based on the literature review,

it was expected that women would score higher on the MSCEIT than men

would.

Method

Participants

Participants of this study included 192 students from two nonnative English

speaking national cultures: 111 from a university in Aix-en-Provence, France

(49 males, 62 females), and 81 from a large university in the province of

Balochistan, Pakistan (52 males and 29 females). To attain sample equiva-

lency, participants in both cultures were recruited from the management

sciences subject pool fully conversant with English language and were

enrolled in programs where the medium of instruction was English. As all

students (in both cultures) indicated that they had good command of English

and were able to complete the instruments in the English language, they

completed the English versions of all instruments. Participants received class

credit for their participation. The average age of the participants was

29.46 (SD= 8.46). The French sample included 60% students from regular

master programs and 40% from executive programs. The Pakistani sample

included 73.5% from regular master programs and the rest were from execu-

tive programs.

All participants took the MSCEIT and other tests in two testing sessions,

each lasting 90 min. Eight classes participated in the study. The number of

students for each group was between 20 and 40. The questionnaires were

presented in the same order in all groups. All participants were treated in

accordance with the Ethical principles of Psychologists and Code of

Conduct (American Psychological Association, 2002).



Measures

TEIQue. The TEIQue (Petrides et al., 2007) is predicated on trait EI theory,

which conceptualizes EI as a personality trait, located at the lower levels of

personality hierarchies (Petrides & Furnham, 2003). The sampling domain of

the TEIQue comprises 15 emotion-related behavioral dispositions (e.g., hap-

piness, self-control, self-motivation) thought to affect the ways individuals

cope with demands of the situation. These 15 emotion-related behavioral dis-

positions (traits) are theoretically arranged into four broader or major con-

ceptual components. These include (a) well-being, representing how

successfully one is able to enjoy life and maintains a positive disposition

(e.g., On the whole, Im pleased with my life); (b) self-control, represent-

ing the ability to regulate ones impulsions and emotions as well as managing

emotional pressures (e.g., I usually find it difficult to regulate my emo-

tions); (c) emotionality, representing the ability to identify and express feel-

ings and to use these faculties to maintain close relationships with others

(e.g., Im normally able to get into someones shoes and experience their

emotions); and (d) sociability, representing interpersonal skills and func-

tioning to assert oneself as well as to influence others emotions and decisions

(e.g., I find it difficult to bond well even with those close to me). The

TEIQue is comprised of 153 items with 7-point scale (strongly disagree to

strongly agree).

Affectivity. Affectivity was measured by 20 items Positive and Negative

Affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988). PANAS is

composed of two 10-item mood scales one to measure positive affectivity

and the other to measure negativity affectivity. The higher scores on both

positive affectivity and negative affectivity items indicate the tendency to

experience a positive and negative mood. The 10 positive affective states

were motivated, excited, feel strong, enthusiastic, proud, alert, inspired,

determined, attentive, and active. The 10 negative affective states were dis-

tressed, upset, guilty, scared, hostile, irritable, ashamed, nervous, jittery, and

afraid. Respondents were requested to rate the statement on a 5-point scale

(not at all to extremely) by comparing themselves during the past 2 weeks

with their usual selves.

Psychological distress. Psychological distress was measured by Chans

(2005) 20-item General Health Questionnaire. This scale measures psycho-

logical distress in terms of current nonpsychotic symptoms in the five symp-

tom areas represented by scales of health concerns (Felt exhausted) , sleep


Karim and Weisz 11

problems ( Early awakening), anxiety (Afraid of everything), dysphoria

(Not enjoying activities), and suicidal ideas (Thoughts of ending life).

Respondents were requested to rate each symptom statement on a 5-point

scale (not at all to extremely) by comparing themselves during the past 2

weeks with their usual selves.

Personality. The 50-item version of the International Personality Item Pool

(IPIP; Goldberg et al., 2006), Big-Five Factor markers, was used to assess

personality. The scale contains 10 items for each of the Big-Five personality

factors: Extraversion (E; I am the life of the party), Agreeableness (A;Take

time out for others), Conscientiousness (C; Pay attention to details), Emo-

tional Stability (ES; Seldom feel blue), and Intellect (I; Spend time reflect-

ing on things). Participants were requested to read the 50 items comprising

the IPIP questionnaire and to mark each one according to how much they

believed it described them on a 5-point scale from very inaccurate to very

accurate.

Self-report emotional intelligence test (SREIT). The 33-item emotional intel-

ligence scale (SREIT; Schutte et al. 1998) is a unidimensional self-report

measure of EI based on Salovey and Mayers (1990) ability model of EI. It

has previously demonstrated good reliability and has been shown to be pre-

dictive of various outcomes (Schutte et al. 1998). Respondents indicate their

level of agreement with each of 33 statements on a 7-point scale. Examples

of items are I am aware of the non-verbal messages I send to others, When

I am in a positive mood, I am able to come up with new ideas, and I help

other people feel better when they are down.

Life satisfaction. The Satisfaction with Life Scale (SWLS; Diener, Emmons,

Larsen, & Griffin, 1985) is a subjective self-report measure of life satisfac-

tion. Respondents indicate their level of agreement with each of five state-

ments on a 7-point scale. Examples of items are In most ways my life is

close to my ideal and I am satisfied with my life.

Ability EI. Emotional intelligence ability was measured with the MSCEIT

(Mayer et al., 2002). The MSCEIT is a 141-item test that measures how well

people perform tasks and solve emotional problems on eight tasks that are

divided into four classes or branches of abilities, including (a) perceiving

emotions, (b) facilitating thought, (c) understanding emotions, and (d) man-

aging emotions. For current study, expert scores for the MSCEIT were

requested from the test publisher. Analysis of the data by the test publisher

provides 15 scores, including one for each task, one for each branch, one for

each area, and one for total EI.



Cognitive intelligence. The 48-item Ravens Standard Progressive Matrices

Test (Raven et al., 2003) was used to measure cognitive intelligence. This test

is designed to measure Spearmans g factor and has now been recognized

as one of the purest measures of g available. The test consists of 48 questions

and presents people with a series of patterns, each of which has one part or

piece missing. The task in each case is to select from a set of eight alterna-

tives the piece that will complete the pattern correctly.

Results

Table 1 presents descriptive statistics for the MSCEIT and other variables for

French and Pakistani participants. According to Mayer et al. (2003) sugges-

tions, to accommodate for item heterogeneity, split-half reliabilities were

employed for the total, area, and branch levels. MSCEIT full-test split-half

reliability was r = 0.84 for the French sample and 0.85 for the Pakistani

sample. The two experiential and strategic area score reliabilities were r =

0.86 and 0.63, and r = 0.88 and 0.82 for the French and Pakistani samples,

respectively. The four branch scores of perceiving, using, understanding, and

managing ranged between r = 0.51 and 0.87 and 0.74 and 0.86 for the French

and Pakistani samples, respectively (see Table 1). The individual task reli-

abilities ranged from a low of 0.45 to a high of 0.80 for the French sample

and from a low of 0.50 to a high of 0.82 for the Pakistani sample.

Country Differences on the MSCEIT

To obtain an overall picture of possible cross-cultural differences on the

MSCEIT, we conducted a series of independent sample t tests on MSCEIT

branch, area, and total scores. There were indeed several significant cross-

cultural differences. French participants performed better than their Pakistani

counterparts on perceiving emotions (t = 2.39, p < .05, Cohens d = .35),

using emotions (t = 2.06, p < 0.05, Cohens d = 0.30), understanding emo-

tions (t = 6.24, p < .001, Cohens d = 0.92), managing emotions (t = 5.05,

p < .001, Cohens d = 0.75), experiential EI (t = 2.50, p < 0.05, Cohens d =

0.37), strategic EI (t = 7.15, p < 0.001, Cohens d = 1.06), and total ability EI

(t = 5.38, p < .001, Cohens d = 0.79).

Factorial Invariance

As can be seen in Table 2, each task correlated mostly highly with its sister

subscale with which it combines (e.g., the Faces and Pictures subscales


Table 1. Means, Standard Deviations, Skewness, and Kurtosis for French and Pakistani Samples

France Pakistan

M (SD) S K Reliabilitya M (SD) S K Reliabilitya

Age 30.75 (7.99) .82 0.28 27.85 (8.39) 1.19 2.15

Gender 0.44 (0.49) .23 1.19 0.63 (0.48) 0.57 1.15

RAPM 38.42 (4.80) .87 1.20 27.57 (8.06) 0.93 0.54

Extraversion 3.44 (0.74) .27 0.62 .87 3.29 (0.72) 0.04 0.16 .80

Agreeableness 4.06 (0.51) .62 0.65 .80 4.06 (0.58) 0.69 0.34 .81

Conscientiousness 3.37 (0.72) .57 0.20 .84 3.57 (0.58) 0.07 0.42 .72

Emotional stability 3.19 (0.68) .06 20 .79 2.98 (0.71) 0.22 0.30 .81

Intellect 3.69 (0.57) .47 0.82 .80 3.46 (0.53) 0.03 0.65 .71

Positive affect 3.60 (0.55) .85 1.50 .78 3.54 (0.57) 0.35 0.32 .76

Negative affect 2.11 (0.63) .53 0.08 .81 1.98 (0.69) 0.81 0.66 .82

Psychological distress 1.83 (0.53) .81 0.03 .86 2.04 (0.75) 1.12 1.19 .91

Life satisfaction 4.60 (1.22) .40 0.08 .86 4.82 (1.26) 0.45 0.29 .80

SREIT 5.04 (0.54) .07 0.05 .85 5.32 (0.64) 0.28 0.60 .90

Well-being (TEIQue) 5.22 (0.78) 1.19 1.21 .83 5.05 (0.71) 0.35 0.22 .75

Self-control (TEIQue) 4.13 (0.73) .14 0.19 .70 4.24 (0.61) 0.26 0.91 .68

Emotionality (TEIQue) 4.72 (0.68) .16 0.74 .71 4.64 (0.57) 0.11 0.01 .62

Sociability (TEIQue) 4.71 (0.65) .32 0.12 .70 4.97 (0.68) 0.25 0.05 .76

Total TEIQue 4.69 (0.47) .37 0.68 .82 4.58 (0.48) 0.44 0.20 .86

Faces 99.18 (14.75) .20 0.14 .80 95.49 (15.51) 0.13 0.52 .82

Facilitation 97.81 (15.51) .06 0.69 .51 99.65 (16.42) 0.05 0.32 .61

Changes 89.31 (10.40) .88 1.15 .45 82.92 (11.85) 0.07 0.13 .68

(continued)

13

by Jahanvash Karim on Septem

ber 13, 2010ccr.sagepub.com

Dow

nloaded from

France Pakistan

M (SD) S K Reliabilitya M (SD) S K Reliabilitya

Emotion management 87.35 (8.84) .72 0.68 .53 79.67 (10.16) 0.57 0.60 .62

Pictures 95.50 (11.02) .53 0.95 .75 92.20 (13.86) 0.70 0.86 .85

Sensation 90.24 (14.39) .87 0.64 .50 83.50 (13.92) 0.33 0.11 .66

Blends 90.01 (11.63) .10 0.59 .46 79.88 (11.54) 0.07 0.05 .50

Social management 87.34 (10.16) .10 0.34 .48 81.46 (14.63) 0.62 0.72 .63

Perceiving emotions 97.31 (13.48) .33 0.62 .87 92.51 (13.97) 0.26 0.45 .86

Using emotions 91.45 (15.41) 1.01 1.12 .56 86.65 (16.64) 0.18 0.20 .80

Understanding emotions 87.57 (10.38) .51 0.11 .60 77.62 (11.56) 0.33 0.20 .74

Managing emotions 85.58 (9.35) .23 0.67 .51 77.41 (13.04) 0.33 0.14 .80

Experiential EI 93.69 (12.68) .25 0.07 .86 88.50 (15.46) 0.25 0.14 .88

Strategic EI 84.79 (8.16) .05 0.54 .63 74.88 (11.03) 0.05 0.02 .82

Total ability EI 86.34 (10.70) .07 0.04 .84 76.88 (13.62) 0.01 0.40 .85

Note: N = 111, 81, respectively. S = skewness; K = Kurtosis; SREIT = Self-Report Emotional Intelligence Test; RAPM = Ravens advanced progressive matrices.a.Split-half reliabilities are reported at the total test, area, and branch score levels due to item heterogeneity. Coefficient alpha reliabilities are re-ported at the subtest level due to item homogeneity.*p < .05. **p < .01. ***p < .001.

Table 1 (continued)

14



Dow

nloaded from

Karim and Weisz 15

which measure Perceiving emotions).Table 3 presents goodness-of-fit indi-

ces for the models examined with CFA (N =192). Based on modification

indices, correlated errors were modeled between sensation and emotion man-

agement tasks. As can be seen, the four-factor model indicated satisfactory

levels of fit, 2(13) = 24.33; NFI = 0.92; TLI = 0.91; CFI = 0.96; RMSEA =

0.06 (0.02-0.10). This model was significantly better fitting than the two-

factor, 2(5) = 35.15, p < .001 and both three-factor models, 2(3) = 9.13,

p < 0.05 and 2(3) = 25.89, p < 0.05, respectively. In addition, all factor

loadings for this model were positive and significant (range = 0.50 to 0.92).

Correlations among the four-factors (perceiving, using, understanding, and

managing) ranged from 0.52 to 0.75. Thus a four-factor model served as a

base line model for subsequent multisample analyses.

Next, invariance across cultures was tested on four levels of nested mod-

els. Each model had more constraints than the previous one (Table 3). First, a

multisample analysis with the unconstrained model (Model 1: configural

invariance) showed an acceptable baseline model for both French and

Pakistani samples. This showed that French participants and Pakistani par-

ticipants shared the same MSCEIT underlying factor pattern and that corre-

sponding tasks loaded on the same factors. Then, to test the invariance of the

factor loadings (metric invariance) across cultures, factor loadings were con-

strained to be equal across the two groups (M2). The results revealed that this

constrained model fit the data well. The chi-square difference test between

configural invariant model (M1) and metric invariant model (M2) was not

significant ( 2(4) = 2.84, p > 0.05), suggesting that factor loadings of both

groups were invariant. Next, in addition to the factor loadings, unique vari-

ances of each task were constrained to be equal across the groups (M3).

The chi-square difference test between this model and M2 was significant

(2(9) = 17.92, p < 0.05), suggesting that models are not completely invari-

ant once setting equal error variances. Subsequent analyses revealed that

relaxing constraint on error variance of sensation task yielded a substantial

and significant improvement in model fit. The chi-square difference test

between this M3.1 and Model 2 was not significant (2(8) = 14.08, p > .05).

In sum, except for error variance of sensation task, the error variances in this

four-factor model did not vary with culture. Finally, besides the constrained

mentioned, factor covariances were also constrained to be equal across the

two groups (M4). Multisample analysis revealed that this constrained model

was acceptable. However, the chi-square difference test between this M4 and

M3.1 was significant (2(6) = 13.34, p < .05), suggesting that models are not

completely invariance once constraining the covariances across cultures.

Subsequent analyses revealed that, relaxing constraint on covariance between


16 Table 2. Intercorrelations Among MSCEIT Scales for French and Pakistani Samples

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1. Faces Task .27** .37** .19 .25* .44*** .19 .30** .87*** .45*** .34** .30** .82*** .35** .65***

2. Facilitation Task .23* .39*** .10 .37** .45*** .40*** .30** .38** .72*** .47*** .27* .59*** .45*** .57***

3. Changes Task .15 .16 .41*** .30** .50*** .45*** .50*** .40*** .52*** .84*** .52*** .52*** .78*** .75***

4. Emotion

Management

Task

.01 .22* .08 .16 .39*** .30** .56*** .23* .33** .42*** .81*** .32** .67*** .57***

5. Pictures Task .45*** .25** .22* .02 .21 .02 .41*** .64*** .26* .19 .36** .56*** .32** .49***

6. Sensation Task .01 .23* .33** .23* .20 .40*** .35** .44*** .92*** .54*** .41*** .72*** .56*** .72***

7. Blends Task .09 .17 .35*** .07 .04 .04 .30** .15 .45*** .85*** .35** .32** .73*** .58***

8. Social

Management

Task

.22* .24* .13 .40*** .36*** .21* .20* .42*** .38*** .47*** .92*** .46*** .77*** .71***

9. Perceiving

Emotion

.89*** .29** .22* .03 .78*** .09 .10 .35*** .47*** .33** .39*** .91*** .40*** .73***

10. Using Emotion .11 .66*** .30** .29** .29** .85*** .09 .27** .21* .59*** .42*** .78*** .60*** .77***

11. Understanding

Emotion

.15 .20* .80*** .09 .17 .22* .83*** .21* .20* .23* .51*** .50*** .89*** .78***

12. Managing

Emotion

.15 .28** .13 .75*** .25** .25** .18 .89*** .24* .33** .19* .47*** .83*** .75***

13. Experiential EI .75*** .53*** .32** .18 .72*** .50*** .11 .38*** .86*** .64*** .26** .34** .55*** .87***

14. Strategic EI .20* .29** .64*** .45*** .26** .30** .71*** .65*** .28** .34** .82*** .68*** .37*** .88***

15. Total EI .60*** .51*** .56*** .36*** .61*** .48*** .47*** .60*** .72*** .60*** .63*** .60*** .85*** .80***

Note: N = 111, 81, respectively. Correlations for the French sample are below the diagonal and for the Pakistani sample are above the diagonal.*p < .05. **p < .01. ***p < .001.



Dow

nloaded from

Karim and Weisz 17

Table 3. CFA and Multisample Goodness-of-Fit Indices for the MSCEIT Across Cultures

Model 2 df P NFI TLI CFIRMSEA

[95%C.I]

Confirmatory factor analyses (N = 192)

One-factor 69.39 19 .00 .78 .75 .83 .11 [.08, .14]

Two-factor 59.88 18 .00 .81 .78 .78 .11 [.08, .14]

Three-factora 33.46 16 .00 .89 .89 .94 .07 [.03, .11]

Three-factorb 50.22 16 .00 .84 .80 .88 .10 [.07, .14]

Four-factor 24.33 13 .028 .92 .91 .96 .06 [.02, .10]

Multigroup comparison factor analyses (French sample: N = 111; Pakistani sample: N = 81)

M1 Configural

invariance

40.45 26 .035 .94 .05 [.01, .08]

M2 Metric

invariance

43.29 30 .055 .95 .04 [.00-.07]

2M2 vs. M1 2.84 4 .58 M3 Invariant

uniqueness

61.21 39 .01 .91 .05 [.02-.08]

2 M3 vs. M2 17.92 9 .03 M3.1

sensation, free 57.35 38 .02 .93 .05 [.02-.07]

2 M3.1 vs. M2 14.06 8 .08 M4 Invariant

factor

covariances

70.69 44 .007 .90 .05 [.03-.08]

2 M4 vs. M3.1 13.34 6 .03 M4.1 Covariance

between

understand

and managing

emotions set

free

64.44 43 .01 .92 .05 [.01-.08]

2 M4.1 vs. M3.1 7.09 5 .21

Note: For both groups correlated errors were modeled between sensation and emotion man-agement tasks. NFI = normed fit index; TLI = Tucker-Lewis index; CFI = comparative fit index; RMSEA = root mean square error of approximation. NFI, CFI, and TLI > .90 and RMSEA < .08 are considered acceptable (Bentler & Bonett, 1980; Browne & Cudeck, 1993).a.Three-factor oblique model comprising the perceiving and using allowed to load on a single factor (Fan et al., 2010; Roberts et al., 2006; Rode et al., 2008; Rossen et al., 2008).b.Three-factor oblique model comprising the using and managing allowed to load on a single factor (Palmer et al., 2005).



understanding and managing branches yielded a substantial and significant

improvement in model fit (M4.1). The chi-square difference test between

M4.1 and Model 3.1 was not significant (2(5) = 7.09, p > .05).Therefore,

the hypothesis of partially invariant covariances between cultures was tena-

ble. In sum, multisample CFA analyses revealed that, with few exceptions,

the factor loadings, unique variances, and factor covariances were invariant

across cultures. The parameter estimates of complete invariant model (M4.1)

are presented in Figure 1.

Discriminant Validity

MSCEITs discriminant validity was examined by assessing the correlations

between scores obtained on the MSCEIT and scores obtained for the mea-

sures assessing self-report mixed model measure (TEIQue), self-report abil-

ity EI measure (SREIT), Ravens Advanced Progressive Matrices, and

personality. According to Papadogiannis et al. (2009), those that intercorre-

late between r =0.00 and 0.25 are considered unrelated to minimally related

Figure 1. The parameter estimates of complete invariance model (Model 4.1)Note: Values in parenthesis represent standardized estimates for the Pakistani sample.


Table 4. Zero-Order Correlations Between the MSCEIT and Other Variables in the Study

France Pakistan

Branch scores Area scores Branch scores Area scores

Per Use Und Man Exp Stg Total Per Use Und Man Exp Stg Total

Age .03 .03 .15 .06 .01 .11 .06 .26* .12 .11 .18 .22* .15 .22*

Gender .03 .09 .11 .19* .08 .20* .18 .31** .25* .12 .01 .33** .09 .23*

RAPM .06 .00 .11 .06 .04 .10 .03 .22* .27* .43** .29** .26* .41** .36**

E .01 .01 .11 .14 .00 .00 .00 .04 .01 .01 .11 .03 .05 .01

A .03 .16 .10 .13 .10 .14 .14 .24* .34** .28* .25* .33** .30** .35**

C .10 .13 .00 .08 .12 .05 .10 .11 .08 .08 .06 .11 .09 .11

ES .29** .08 .09 .14 .30** .01 .20* .01 .01 .19 .17 .01 .20 .11

I .17 .11 .04 .04 .08 .07 .07 .14 .40** .31** .31** .28** .37** .38**

PA .01 .08 .17 .20* .03 .02 .01 .02 .07 .17 .06 .02 .15 .11

NA .25** .10 .08 .15 .24** .12 .23* .15 .05 .21 .35** .13 .29** .26*

PD .19* .10 .07 .12 .20* .12 .21* .10 .03 .26* .36** .09 .32** .24*

SWL .17 .14 .06 .22* .23* .19* .25** .09 .03 .02 .01 .08 .015 .07

SREIT .08 .11 .05 .02 .01 -.03 .02 .09 .07 .18 .16 .09 .20 .15

TEIQue

Well-being .17 .23* .00 .29** .26** .17 .25** .13 .25* .40** .28** .21 .38** .33**

Self-control .17 .05 .04 .15 .16 .03 .11 .04 .04 .16 .23* .04 .22** .16

Emotionality .00 .13 .12 .00 .07 .08 .09 .17 .15 .28* .38** .19 .36** .32**

Sociability .04 .08 .10 .02 .05 .07 .01 .17 .21 .16 .34** .21 .28 .27*Global trait EI .15 .17 .01 .19* .20* .08 .17 .18 .22* .34** .40** .22* .41** .36**

Note. Per = perceiving emotions; Use = using emotions; Und = understanding emotions; Man = managing emotions; Exp = experiential EI; Stg = strategic EI; RAPM = Ravens advanced progressive matrices; E = extraversion; A = agreeableness; C = conscientiousness; ES = emotional stability; I = intellect; PA = posi-tive affect; NA = negative affect; PD = psychological distress. SWL = satisfaction with life; TEIQue = Trait emotional intelligence questionnaire.*p < .05. **p < .01.

19



Dow

nloaded from


with one another; r = 0.25 to 0.50 indicate minimal to moderate overlap; r =

0.50 to 0.75 indicate moderate to highly related concepts; and r = 0.75 to

1.00 indicate that instruments share common themes and arguably assess the

same underlying constructs.

MSCEIT and TEIQue. For the participants in the French sample, nonsignifi-

cant to moderate correlations were observed between the MSCEIT branches

and the TEIQue factors, with the greatest correlation found between

MSCEITs managing emotions branch and TEIQues well-being factor (r =

0.29, p < 0.01). Likewise, for the participants in the Pakistani sample, the

greatest of correlation was observed between understanding emotions branch

of the MSCEIT with the well-being factor of the TEIQue (r = 0.40, p < 0.01).

However, the number of low to moderate significant correlations in the Paki-

stani sample (13) was higher than that for the French sample (i.e., only 4). In

sum, the performance-based ability EI in both cultures was found to be inde-

pendent of the TEIQue. (see Table 4).

MSCEIT and SREIT. Nonsignificant correlations were observed between

the MSCEIT factors and the SREIT in both cultures.

MSCEIT and Cognitive Intelligence. For participants in the French sample, all

correlations between the MSCEIT scores and cognitive intelligence revealed

to be nonsignificant. However, for the Pakistani sample, low to moderate

significant correlations were observed between the cognitive intelligence and

the MSCEIT scores, with the greatest correlation observed between Strategic

EI and Ravens Advances Progressive Matrices (r = 0.41, p < .01).

MSCEIT and the Big Five. For participants in the French sample, among the

Big Five personality dimensions, only emotional stability revealed to be a

significant correlate of perceiving emotions (r = 0.29, p < 0.01), experiential

EI (r = 0.30, p < 0.01), and total ability EI (r = 0.20, p < 0.05). In contrast, for

participants in the Pakistani sample, low to moderate correlations were

observed between the Big Five dimensions of agreeableness and intellect and

the MSCEIT scores. Correlations between agreeableness and MSCEIT scores

ranged from 0.24 (perceiving emotions) to 0.35 (total ability EI), whereas

correlations between intellect and the MSCEIT scores ranged from 0.14 (per-

ceiving emotions) to 0.38 (total ability EI).

Incremental Validity

We used the hierarchical multiple regression technique to test for the incre-

mental validity of the MSCEIT. The Big Five personality dimensions and the


Karim and Weisz 21

scores on the Ravens Advanced Progressive Matrices were entered into the

equation first (Step 1). At Step 2, the scores for the MSCEIT four branches

were entered. As can be seen in Tables 5 and 6, none of the MSCEIT

branches was found to be a significant predictor of satisfaction with life, PA,

NA, and psychological distress.

Known Group validation

For the French sample, females scored higher than males on managing emo-

tions, strategic EI, and total MSCEIT, whereas for the Pakistani sample,

females scored higher than males on perceiving emotions, using emotions,

experiential EI, and total MSCEIT (see Table 7).

Table 5. Predicting Satisfaction With Life, PA, NA, and Psychological Distress: Results for French Sample

Life satisfaction PA NA

Psychological distress

Step R2 R2 R2 R2

1 E .11 .21*** .34 .45*** .09 .40*** .14 .30***

A .14 .06 .04 .14

C .19* .42 .10 .03

ES .25** .11 .59*** .47***

I .07 .29 .03 .02

RAPM .25** .01 .05 .13

2 Perceiving .10 .02 .01 .02 .06 .03 .11 .03

Using .04 .04 .03 .09

Understanding .02 .11 .13 .10

Managing .08 .12 .00 .06

Total R2 .24 .47 .43 .33

Final F (df) 3.01** (10, 94)

9.01*** (10, 99)

7.56*** (10, 99)

5.08*** (10, 99)

Note: RAPM = Ravens advanced progressive matrices; E = extraversion; A = agreeableness; C = conscientiousness; ES = emotional stability; I = intellect; PA = positive affect; NA = negative affect.*p < 0.05. **p < 0.01. ***p < 0.001.



Discussion

Cross-cultural validity of EI scales is a constant concern in organizational

behavior research (Gangopadhyay & Mandal, 2008; Palmer et al., 2008). The

present study attempted to evaluate an ability-based measure of EI (the

MSCEIT) cross-culturally. To our knowledge, this study is the first to pro-

vide evidence of the factorial invariance, discriminant and incremental valid-

ity of the MSCEIT across cultures.

The first objective of this study was to assess the mean differences on the

MSCEIT across two cultures. French participants had higher scores than their

Pakistani counterparts on branch, area, and total MSCEIT scores. This

accords well with findings showing that people from individualistic societies

are better at perceiving, understanding, expressing, and regulating emotions

(e.g., Fernandez et al., 2000; Gross & John, 2003; Matsumoto, 1989; 1992).

This finding suggests that care must be taken when selecting people from

Table 6. Predicting Satisfaction With Life, PA, NA, and Psychological Distress: Results for Pakistani Sample

Life satisfaction PA NA

Psychological distress

Step R2 R2 R2 R2

1 E .06 .19* .08 .28** .04 .45*** .01 .31***

A .02 .02 .23 .17

C .20 .44** .15 .07

ES .12 .01 .53*** .46***

I .34 .29* .10 .05

RAPM .25 .05 .01 .06

2 Perceiving .12 .01 .03 .04 .04 .05 .04 .07

Using .04 .20 .18 .14

Understanding .04 .26 .04 .09

Managing .09 .03 .23 .29

Total R2 .20 .22 .50 .38

Final F (df) 1.73 (10, 66)

3.16** (10, 66)

6.70*** (10, 66)

4.20*** (10, 66)

Note: RAPM = Ravens advanced progressive matrices; E = extraversion; A = agreeableness; C = conscientiousness; ES = emotional stability; I = intellect; PA = positive affect; NA = negative affect. = standardized regression weights.*p < .05. **p < .01. ***p < .001.


Table 7. Mean Differences Across Genders Within Each Sample

France Pakistan

Females Males t d Females Males t d

Perceiving 97.73 (13.25) 96.78 (13.89) 0.36 .07 98.31 (12.34) 89.28 (13.89) 2.91** .68

Using 92.80 (14.24) 89.74 (16.74) 1.03 .20 92.34 (16.89) 83.48 (15.78) 2.36* .55

Understanding 88.66 (11.05) 86.19 (9.40) 1.24 .24 79.55 (10.80) 76.55 (11.93) 1.11 .26

Managing 87.16 (9.62) 83.57 (8.67) 2.04* .39 77.72 (13.29) 77.25 (13.03) 0.15 .04

Experiential EI 94.70 (12.53) 92.42 (12.90) 0.93 .18 95.41 (13.98) 84.65 (15.20) 3.16** .73

Strategic EI 86.29 (8.41) 82.89 (7.49) 2.20* .42 76.20 (10.57) 74.15 (11.31) 0.80 .19

Total EI 89.05 (10.57) 83.17 (10.58) 1.99* .56 81.13 (13.40) 74.51 (13.29) 2.14* .50

Note: For the French sample, females = 62 and males = 49 and for the Pakistani sample, females = 29 and males = 52. d = Cohens d.*p < .05. **p < .01. ***p < .001.; Values in parentheses represent standard deviations

23



Dow

nloaded from


pool of individuals from diverse cultures because results may be biased

toward individuals from individualistic societies.

The second objective of this study was to investigate whether the MSCEIT

measures the same construct in both cultures. Multigroup CFA analyses

revealed that the MSCEIT has the same theoretical latent structure, the same

strength of the relationships among factors and tasks, and the same reliability

of tasks regardless of the country. Therefore, the MSCEIT across both cul-

tures can be interpreted in the same way. Significant mean differences and

multigroup CFA analyses across both cultures provide support for the asser-

tion made by Palmer et al. (2008) that, EI factors are culturally universal and

have comparable functions across cultures. However, the processes underly-

ing these factors and their manifestation may differ across cultures as a con-

sequence of the role culture plays in the development, display, and

interpretation of emotions (p. 35). It is worth mentioning that the similarities

in the factor structure across both cultures can be attributed to the shift in

cultural values of Pakistan. Pakistani youth (present study sample-university

students) are much different from previous generations. They have been

raised differently. This youth is the best educated and most culturally diverse

generation. A combination of Western-style of education (mostly American

and U.K.-based curriculum), widespread use of Internet, and currently vibrant

print and electronic media in the country has made this generation exceed-

ingly tolerant and open-minded toward western lifestyles.

The third objective of this study was to assess the discriminant validity of

the MSCEIT vis--vis cognitive intelligence (the Ravens Advanced

Matrices), self-report EI measures (the SREIT and TEIQue), and the Big Five

personality measures. Consistent with past research relating self-report EI

with ability EI measures (e.g., Brackett & Mayer, 2003; Livingstone, & Day,

2005; Joseph & Newman, 2010; OConnor & Little, 2003), in both cultures,

the MSCEIT demonstrated a lack of convergence with the TEIQue and the

SREIT. This important finding suggests that the performance-based ability EI

measure (i.e., MSCEIT) and self-report measures are assessing different con-

structs. These findings support Petrides and Furnhams (2003) assertion that

the tendency to validate ability EI measure against another self-report mea-

sure is problematic given the obvious differences between measurement

methods. Low to moderate correlations were found between the MSCEIT

factors and cognitive intelligence in the Pakistani sample, whereas nonsig-

nificant correlations were observed in the French sample. These findings sup-

port Mayer, Salovey, and Carusos (2004) assertion that EI is different from

other intelligences (p. 203). Finally, as predicted, this study revealed that

MSCEIT was mostly distinguishable from the Big Five personality


Karim and Weisz 25

dimensions in both cultures. The correlations of the MSCEIT branches with

Big Five were mostly nonsignificant or low to moderate in both cultures.

These results were consistent with previous findings (Brackett & Mayer,

2003; OConnor, and Little, 2003; Rode et al., 2007) and provide support for

the assertion that the MSCEIT includes abilities to perceive, integrate, under-

stand, and regulate emotions (Mayer & Salovey, 1997); therefore, it is

unlikely to be related to personality traits (Mayer, Roberts et al.,2008; Mayer,

Salovey et al., 2008).

The fourth objective of this study was to assess the incremental validity of

the MSCEIT. In line with previous studies (Livingstone & Day, 2005; Rode

et al., 2007; Rossen & Kranzler, 2009; Zeidner & Olnick-Shemesh, 2010),

the results indicated that the MSCEIT scales did not add to the prediction of

PA, NA, satisfaction with life, and psychological distress after controlling for

personality and cognitive intelligence in either of the sample. The lack of

incremental validity when explaining NA, PA, satisfaction with life, and psy-

chological distress suggests that the MSCEIT may not increase our under-

standing of subjective well-being, after controlling for cognitive intelligence

and personality variables in the same analyses. Thus the results of this study

further highlight the serious problem associated with the predictive validity

of the MSCEIT (for further review please see Zeidner & Olnick-Shemesh,

2010) .

Finally, the fifth objective of this study was to obtain an overall picture of

possible gender differences on the MSCEIT within each sample. In the

French sample, women outperformed men on managing and total EI, whereas

in the Pakistani sample, women scored higher on perceiving, using, and total

EI. These findings support past research showing that women tend to be bet-

ter at emotion-related abilities than men (Barrett et al., 2000; Ciarrochi et al.,

2005; Hall & Matsumoto, 2004; Garnefski et al., 2004; Gross & John, 1998;

Mayer et al., 2002; Joseph & Newman, 2010; Mehrabian, Young, & Sato,

1988; Palmer et al., 2005). This finding suggests gender bias in the MSCEIT;

therefore, according to some researchers, the MSCEIT should not be used for

personnel selection, unless research demonstrates that gender differences in

test performance reflect gender differences in job performance (Day &

Carroll, 2004).

Limitations and Future Directions

First, there is a possibility that individualism/collectivism along with other

cultural factors will vary within cultures because of demographic, regional,

class, and other differences within cultures. This leads to a limitation found



in much cross-cultural researchthat of generalizing across all cultural

groups or subcultures. The sample at a single university may not reflect the

culture of a heterogeneous nation. Therefore, results collected in big cities

could likely be different from those collected in small cities or villages. This,

of course, points to the need for future research to examine samples across

various subcultures.

Second, and most important, student samples do not represent the culture

as much as do more or less representative samples. Students may experience

different levels of EI, from a general working adult population (Day, Therrien,

& Carroll, 2005). The use of students limits the external validity of the results

(Wintre, North, & Sugar, 2001). Students are more internationally similar

than unselected members of a culture. This may have fostered similarities in

the results across both cultures. Therefore, it is important to examine the

validity of the MSCEIT using a larger sample that is more representative of

the general population.

Third, all participants responded to the MSCEIT in English. However, as

noted by one anonymous reviewer, even for the students taking courses in

English (or fully conversant with English as a second language), the vocabu-

lary associated with emotion concepts may be somewhat obscure and/or such

students do not always possess the necessary emotion terminology. This may

have influenced the pattern of responses in the current study. For example,

participants in both cultures scored low on understanding and managing,

which are language-intensive. This, of course, points to the need for future

research to examine samples across native languages.

Finally, researchers have consistently expressed concerns about the

absence of scientific standards for determining the accuracy of consensus and

expert scores for the MSCEIT (for details see Matthews et al., 2002).

Moreover, American norm group scoring may not work well in other cultures

(Zeidner & Olnick-Shemesh, 2010). Therefore, future cross-cultural research,

instead of using American based scoring, should use proportion consensus-

scores with consensus weights determined from the local samples.

References

American Psychological Association .(2002). Ethical principles of psychologists and

codes of conduct. Washington, DC: APA.

Barrett, L. F., Lane, R. D., Sechrest, L., & Schwartz, G. E. (2000). Sex differences in

emotional awareness. Personality and Social Psychology Bulletin, 26, 1027-1035.

Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness-of-fit in the

analysis of covariance structures. Psychological Bulletin, 88, 588-606.


Karim and Weisz 27

Brackett, M. A., & Mayer, J. D. (2003). Convergent, discriminant, and incremental

validity of competing measures of emotional intelligence. Personality and Social

Psychology Bulletin, 29, 1147-1158.

Brackett, M. A., Rivers, S. E., Shiffman, S., Lerner, N., & Salovey, P. (2006). Relating

emotional intelligence abilities to social functioning: A comparison of self-report

and performance measures of emotional intelligence. Journal of Personality and

Social Psychology, 91 (4), 780-795.

Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K.

A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162).

Newbury Park, CA: Sage Publications.

Chan, D. W. (2005). Emotional Intelligence, social coping, and psychological dis-

tress among Chinese gifted students in Hong Kong. High Ability Studies, 16(2),

163-178.

Ciarrochi, J., Haynes, K., & Crittenden, N. (2005). Can men do better if they try

harder? Sex and motivational effects on emotional awareness. Cognition and

Emotion, 19, 133-141.

Ciarrochi, J. V., Chan, A. Y., & Caputi, P. (2000). A critical evaluation of the emo-

tional intelligence construct. Personality and Individual Differences, 28, 539-561.

Day, A. L., & Carroll, S. A. (2004). Using an ability-based measure of emotional intel-

ligence to predict individual performance, group performance, and group citizen-

ship behaviors. Personality and Individual differences, 36, 1443-1458.

Day, A. L., Therrien, D. L., & Carroll, S. A. (2005). Predicting psychological health:

Assessing the incremental validity of emotional intelligence beyond personality,

Type A behaviour, and daily hassles. European Journal of Personality, 19,

519-536.

Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction with

Life Scale. Journal of Personality Assessment, 49, 71-75.

Ekermans, G. (2009). Emotional intelligence across cultures: Theoretical and meth-

odological considerations. In, C. Stough, D. H. Saklofske, & J. D. A. Parker (Eds),

Assessing emotional intelligence: Theory, research, and applications (pp. 259-

290). New York: Springer.

Fabio, A. Di., & Palazzeschi, L. (2009). An in-depth look at scholastic success: Fluid

intelligence, personality traits or emotional intelligence? Personality and Indi-

vidual Differences, 46, 581-585.

Fan, H., Jackson, T., Yang, X., Tang, W., & Zhang, J. (2010). The factor structure

of the Mayer-Salvoey-Caruso Emotional Intelligence Test V 2.0 (MSCEIT): A

meta-analytic structural equation modeling approach. Personality and Individual

Differences, 48, 781-785.

Farrelly, D., & Austin, E. J. (2007). Ability EI as an intelligence? Associations of

the MSCEIT with performance on emotion processing and social tasks and with

cognitive ability. Cognition and Emotion, 21 (5), 1043-1063.



Fernandez, I., Carrera, P., Sanchez, F., Paez, D., & Candia, L. (2000). Differences

between cultures in emotional verbal and non-verbal reactions. Psicothema, 12,

83-92.

Gangopadhyay, M., & Mandal, M. K. (2008). Emotional Intelligence-A universal or

culture-specific construct? In R. J. Emmerling, V. K. Shanwal, & M. K. Mandal

(Eds.), Emotional intelligence: Theoretical and cultural perspective (pp. 115-

134). New York: Nova Science Publishers, Inc.

Garnefski, N., Teerds, J., Kraaij, V., Legerstee, J., & Van den Kommer, T. (2004).

Cognitive emotion regulation strategies and depressive symptoms: Differences

between males and females. Personality and Individual differences, 36, 267-276.

Gignac, G. E. (2009). Psychometric and the measurement of emotional intelligence.

In, C. Stough, D. H. Saklofske, & J. D. A. Parker (Eds), Assessing emotional

intelligence: Theory, research, and applications (pp. 9-40). New York: Springer.

Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger,

C. R., & Gough, H. C. (2006). The International Personality Item Pool and the

future of public-domain personality measures. Journal of Research in Personality,

40, 84-96.

Gross, J. J., & John, O. P. (2003). Individual differences in two emotion regulation

processes: Implications for affect, relationships, and wellbeing. Journal of Per-

sonality and Social Psychology, 85, 348-362.

Gross, J. J., & John, O. P. (1998). Mapping the domain of expressivity: Multimethod

evidence for a hierarchical model. Journal of Personality and Social Psychology,

74, 170-191.

Hall, J. A. & Matsumoto, D. (2004). Gender differences in judgments of multiple

emotions from facial expressions. Emotion, 4 (2), 201-206.

Hofstede, G. (1980). Cultures consequences: International differences in work-

related values. Newbury Park, CA: Sage.

Joseph, D. L., & Newman, D. A. (2010). Emotional intelligence: An integrative meta-

analysis and cascading model. Journal of Applied Psychology, 95 (1), 54-78.

Livingstone, H. A., & Day, A. L. (2005). Comparing the construct and criterion-

related validity of ability-based and mixed-model measures of emotional intel-

ligence. Educational and Psychological Measurement, 65(5), 757-779).

Matsumoto, D. (1989). Cultural influences on the perception of emotion. Journal of

Cross-Cultural Psychology, 20, 92-105.

Matsumoto, D. (1992). American-Japanese cultural differences in the recognition of

universal facial expressions. Journal of Cross-Cultural Psychology, 23 (1), 72-84.

Matthews, G., Emo, A. K., Funke, G., Zeidner, M., Roberts, R. D., & Costa, P. T.

(2006). Emotional intelligence, personality, and task-induced stress. Journal of

Experimental Psychology, 12 (2), 96-107.

Matthews, G., Roberts, R. D., & Zeidner, M. (2004). Seven myths about emotional

intelligence. Psychological Inquiry, 15, 179-196.


Karim and Weisz 29

Matthews, G., Zeidner, M., & Roberts, R. D. (2002). Emotional intelligence: Science

and myth. Cambridge, MA: MIT Press.

Mayer, J. D., Roberts, R. D., & Barsade, S. G. (2008). Human Abilities: Emotional

Intelligence. Annual Review of Psychology, 59, 507-536.

Mayer, J. D. & Salovey, P. (1997). What is emotional intelligence? In P. Salovey &

D. J. Sluyter (Eds.), Emotional Development and Emotional Intelligence: Educa-

tional Implications. New York: Basic Books, pp. 3-27.

Mayer, J. D., Salovey, P., & Caruso, D. R. (2002). Mayer-Salovey-Caruso Emotional

Intelligence Test (MSCEIT) item booklet, version 2.0. Toronto, Onatario, Canada:

MHS.

Mayer, J. D., Salovey, P., & Caruso, D. R. (2004). Emotional intelligence: Theory,

findings, and implications. Psychological Inquiry, 15, 197-215.

Mayer, J. D., Salovey, P., & Caruso, D. R. (2008). Emotional intelligence: New ability

or eclectic traits? American Psychologist, 63, 503-517.

Mayer, J. D., Salovey, P., Caruso, D., & Sitarenios, G. (2003). Measuring emotional

intelligence with the MSCEIT V2.0. Emotion, 3, 97-105.

Mehrabian, A., Young, A. L., & Sato, S. (1988). Emotional empathy and associated

individual differences. Current Psychology Research and Reviews, 7, 221-240.

OConnor, R. M., & Little, I. S. (2003). Revisiting the predictive validity of emotional

intelligence: Self-report versus ability-based measures. Personality and Individ-

ual Differences, 35, 1893-1902.

Palmer, B. R., Gignac, G., Manocha, R., & Stough, C. (2005). A psychometric evalua-

tion of the MayerSaloveyCaruso Emotional Intelligence Test Version 2.0. Intel-

ligence, 33, 285-305.

Palmer, B. R., Gignac, G., Ekermans, G., & Stough, C. (2008). A comprehensive

framework for emotional intelligence. In R. J. Emmerling, V. K. Shanwal, & M.

K. Mandal (Eds.), Emotional Intelligence: Theoretical and Cultural Perspective

(pp. 17-38), New York: Nova Science.

Papadogiannis, P. K., Logan, D., & Sitarenios, G. (2009). An ability model of emo-

tional intelligence: A rationale, description, and application of the Mayer Salovey

Caruso Emotional Intelligence Test (MSCEIT). In, C. Stough, D. H. Saklofske,

& J. D. A. Parker (Eds), Assessing emotional intelligence: Theory, research, and

applications (pp. 259-290). New York: Springer.

Petrides, K. V., & Furnham, A. (2003). Trait emotional intelligence: Behavioral vali-

dation in two studies of emotion recognition and reactivity to mood induction.

European Journal of Personality, 17, 39-57.

Petrides, K. V., Prez-Gonzalez, J. C., & Furnham, A. (2007). On the criterion and

incremental validity of trait emotional intelligence. Cognition and Emotion, 21,

26-55.



Raven, J., Raven, J.C., & Court, J.H. (2003). Manual for Ravens Progressive Matri-

ces and Vocabulary Scales. Section 1: General Overview. San Antonio, TX: Har-

court Assessment.

Roberts, R. D., Schulze, R., OBrien, K., MacCann, C., Reid, J., & Maul, A. (2006).

Exploring the validity of the MayerSaloveyCaruso Emotional Intelligence Test

(MSCEIT) with established emotions measures. Emotion, 6, 663-669.

Rode, J. C., Mooney, C. H., Arthaud-day, M. L., Near, J. P., Rubin, R. S., Baldwin,

T. T., & Bommer, W. H. (2007). An examination of the structural, discriminant,

nomological, and incremental predictive validity of the MSCEIT V2.0. Intel-

ligence, 36(4), 350-366.

Rossen, E., & Kranzler, J. H. (2009). Incremental validity of Mayer-Salovey-Caruso

Emotional Intelligence Test version 2.0 (MSCEIT) after controlling for personal-

ity and intelligence. Journal of Research in Personality, 43, 60-65.

Rossen, E., Kranzler, J. H., & Algina, J. (2008). Confirmatory factor analysis of the

Mayer-Salovey-Caruso Emotional Intelligence Test V 2.0 (MSCEIT). Personality

and Individual Differences, 44, 1258-1269.

Salovey, P., Bedell, R. T., Detweiler, J., & Mayer, J. (2000). Current directions in emo-

tional intelligence research. In M. Lewis & J. M. Haviland-Jones (Eds.), Hand-

book of emotions (pp. 504-520). New York: Guilford.

Salovey, P. & Mayer, J. D. (1990) Emotional intelligence. Imagination, Cognition and

Personality, 9,185-211.

Schutte, N. S., Malouff, J. M., Hall, L. E., Haggerty, D. J., Cooper, J. T., Golden, C. J.,

et al. (1998). Development and validation of a measure of emotional intelligence.

Personality and Individual Differences, 25, 167-177.

Schutte, N.S., Malouff, J.M., Thorsteinsson, E.B., Bhullar, N., & Rooke, S.E. (2007).

A meta-analytic investigation of the relationship between emotional intelligence

and health. Personality and Individual Differences, 42, 921-933.

Tugade, M. M., & Fredrickson, B. L. (2002). Positive emotions and emotional intel-

ligence. In L. Feldman Barrett & P. Salovey (Eds.), The wisdom of feelings (pp.

319-340). New York: Guilford.

Watson, D., Clark, L.A., & Tellegen, A. (1988). Development and validation of brief

measures of positive and negative affect: The PANAS scales. Journal of Person-

ality and Social Psychology, 54, 1063-1070.

Wintre, M. G., North, C., & Sugar, L. A. (2001). Psychologists response to criticisms

about research based on undergradute participants: A developmental perspective.

Canadian Psychology, 42(3), 216-225.

Zeidner, M., & Olnick-Shemesh, D. (2010). Emotional intelligence and subjective

well-being revisited. Personality and Individual Differences, 48, 431-435.


Karim and Weisz 31

Bios

Jahanvash Karim is a PhD student studying emotional intelligence at the

IAE dAix-en-Provence, Universit de Paul Czane, Aix-en-Provence, France. His

research includes cross-cultural analyses of emotional intelligence measures.

Robert Weisz is full professeur, specialist in organizational behavior & organiza-

tional development, at the IAE dAix-en-Provence, Universit de Paul Czane, Aix-

en-Provence, France. He is adjunct professor at the HEC (Paris) for the international

programs. He also teaches at other well-known business schools such as Monash BS

(Melbourne), WHU (Koblenz), and Steinbeis (Berlin).


cross-cultural research on the reliability and validity of the mayer-salovey-caruso emotional...

Documents

crosscultural research

cognitive intelligence

msceit scores

crosscultural researchxxx

msceit total scores

incremental validity

validity of thepublished

paul czane