Noise and Incongruence: Interpreting Results of the Incongruence

Molecular Phylogenetics and EvolutionVol. 17, No. 3, December, pp. 401–406, 2000doi:10.1006/mpev.2000.0845, available online at http://www.idealibrary.com on

Length Difference TestKonrad Dolphin,* Robert Belshaw,* C. David L. Orme,* and Donald L. J. Quicke* ,†

*Department of Biology, Imperial College at Silwood Park, Ascot, Berkshire SL5 7PY, United Kingdom; and †Department ofEntomology, The Natural History Museum, London SW7 5BD, United Kingdom

Received December 15, 1999; revised August 4, 2000; published online November 3, 2000

volume of Cladistics, which, although published in

Incongruence between data sets is an important

concept in molecular phylogenetics and is commonlymeasured by the incongruence length difference (ILD)test (J. S. Farris et al., Cladistics 10, 315–319). The ILDtest has been used to infer specific evolutionary eventsand to determine whether to combine data sets forphylogenetic analysis. However, the interpretation inthe literature of the test’s results varies because au-thors have conflicting expectations of the effect thatnoise will have. Using simulations we demonstratethat noise can by itself generate highly significant re-sults in the ILD test and demonstrate why this is thecase. To clarify the interpretation of test results, wesuggest an additional procedure in which the result iscompared against a frequency distribution generatedfrom completely shuffled data. As examples, we applythis approach to two previous studies that have re-ported incongruence. © 2000 Academic Press

INTRODUCTION

In a phylogenetic context, congruence is the extent towhich estimates of phylogeny based on different datasets are in mutual agreement (Page and Holmes, 1998;Swofford et al., 1996). The concept plays an importantrole in phylogenetic research, especially with molecu-lar data sets in which several biological processes, e.g.,lineage sorting, gene duplication, and introgression(Page and Holmes, 1998), can lead to the estimation ofdifferent phylogenies of the same taxa from differentgenes.

Congruence may take the form of taxonomic congru-ence, in which tree topologies are compared; charactercongruence, in which the data sets themselves are di-rectly compared in some way; or a combination of thesetwo forms. One of the most intuitively appealing andwidespread of the character congruence measures isthe incongruence length difference (ILD) test of Farriset al. (1994). (Note that this paper is cited in the liter-ature as both Farris et al. (1994) and Farris et al.(1995): this is due to the late publication of the 1994

401

1995, bears the date 1994). As we describe below inmore detail, the ILD test is commonly used to testhypotheses of evolutionary events, and it is used whenconsidering whether to analyze multiple data sets sep-arately or simultaneously. We believe that both ofthese uses are compromised by the current lack of aclear understanding of the relationship between theILD test and noise, defined here as random data (afterWenzel and Siddall, 1999) which by definition cannotreveal any features of shared phyletic history.

The ILD Test

Mickevich and Farris (1981) introduced the ILDmeasure and it was developed (and also named) byFarris et al. (1994).

“For matrices X and Y the incongruence length dif-ference Dxy is given by:

Dxy 5 L(x1y) 2 ~Lx 1 Ly!

Lx, Ly and L(x1y) denote the lengths of most parsimoni-ous trees calculated for each matrix separately and forthe combined matrix, that including all the charac-ters.”

To obtain a measure of the significance of this lengthdifference, a null length distribution is required, and inpractice the test works as follows. (1) Calculate thesum of most-parsimonious tree (MPT) lengths from thetwo matrices (which we will refer to as partitions in thecontext of an ILD test). (2) Create replicate partitionsby pooling all characters and randomly allocating themto two partitions equal in size to that of the originals.(3) Calculate the sum of the MPT lengths from each ofthese replicate partitions to form a tree-length distri-bution. (4) Calculate the probability that the sum oflengths from the original partitions (step 1) lies withinthis distribution: a low probability implies incongru-ence.

The test is implemented in this way in softwareapplications such as arn (Farris, 1991) and PAUP*(Swofford, 1998); in the latter, the test is known as the“partition homogeneity” test.

1055-7903/00 $35.00Copyright © 2000 by Academic PressAll rights of reproduction in any form reserved.

The principle of the test is that if characters are more

tcghtnm

“pisgaVat(tcnzmnn

T

wtre(th(

small data set, or one which lacks the ability to fully

Bptwwat

E

tvbwwdEFrPru

E

pamUg

402 DOLPHIN ET AL.

congruent within than between partitions, then ho-moplasy, and therefore tree lengths, will be minimizedwhen the characters are maintained in their originalpartitions and the ILD measure will be large.

Applications of the ILD Test

Many studies have used the ILD test to infer theprobability of evolutionary events using incongruencebetween data sets. Recent examples include tests forhybridization (Doyle et al., 1999), horizontal generansfer (Cho et al., 1998; Lecointre et al., 1998), re-ombination (Geiser et al., 1998), large-scale conver-ence (Muona, 1995; Quicke and Belshaw, 1999), andybrid origins of parthenogenesis (Normark and Lan-eri, 1998). For these purposes, the generation of sig-ificant results in the test by noise would be positivelyisleading.Another common application of the test is during the

conditional combination” or “prior agreement” ap-roach to phylogeny reconstruction (Bull et al., 1993),n which the analysis of two data sets is carried outeparately if they are found to be significantly incon-ruent (by some measure) and simultaneously if theyre not (see, for example, Johnson and Sorensen, 1998;idal and Lecointre, 1998; Carbone et al., 1999; Hoot etl., 1999; Spangler and Olmstead, 1999). Leaving asidehe debate over separate versus combined analysise.g., Chippindale and Wiens, 1994; Nixon and Carpen-er, 1996), there is currently an unresolved issue con-erning the precise effect that combining noisy and lessoisy data sets has on phylogeny reconstruction (Wen-el and Siddall, 1999; Kallersjo et al., 1999). Thiseans that a test which is used to decide whether orot to combine data sets should not confound bothoise and incongruent signal into a single result.

he Influence of Noise on the ILD Test

There is no consensus in the literature on how noiseill affect the ILD test. The test was published with

he stated expectation that noise would not affect theesult (Farris et al., 1994), and this expectation hasntered standard texts on phylogenetic methodologyKitching et al., 1998). Nonetheless, authors differ inheir opinion as to the relationship between noise oromoplasy and incongruence. For example, Swofford1991), quoted in Farris et al. (1994), states that the

ILD test is not influenced by different levels of ho-moplasy within the matrices but that it should be;Vidal and Lecointre (1998) agree that it is not influ-enced by noise, but they follow Farris et al. (1994) inbelieving that it should not be; Graham et al. (1998)come to the conclusion that it is in fact influenced bynoise and offer advice on how to solve this problem (byexcluding poorly supported branches). Other authorsmake the observation during their work that somesignificant results in the test may be due to a noisy or

resolve trees, without addressing the issue further (forexample Cannatella et al., 1998; Piercey-Normore etal., 1998). This is clearly an unsatisfactory position.

We consider that the problem needs urgent atten-tion, given the inherently noisy nature of moleculardata, combined with the trend toward multiple geneanalyses and the availability of incongruence tests instandard phylogeny reconstruction software. We be-lieve that our study is the first to formally investigatethe relationship between noise and incongruence in theILD test, although several previous studies havetouched upon this. Some studies (e.g., Graham et al.1998) have shown that incongruence can exist betweenreal and shuffled partitions; others (Cunningham,1997; Messenger and McGuire, 1998; Stanger-Hall andCunningham, 1998) show that the reduction or exclu-sion of noise from matrices can reduce levels of incon-gruence. Quicke and Belshaw (1999) showed that theaddition of noise to a small, simulated matrix reducedincongruence with a simulated matrix of incongruentcharacters. We therefore investigated, using simula-tions, how and why noise can affect the ILD test.

MATERIALS AND METHODS

We created two data matrices, called Comb andush, each of 16 taxa with 13 characters forming aerfectly pectinate or symmetrical cladogram, respec-ively. No characters were homoplastic and each nodeas supported by one character (Fig. 1). All analysesere conducted separately for each matrix to revealny effects of topology on the noise–incongruence rela-ionship.

ffect of Noise on ILD Test Results

We assessed the incongruence between each struc-ured matrix and a range of data partitions containingaried amounts of noise. ILD tests were conductedetween each matrix and a duplicate of that matrixhich had been modified by shuffling character states,ithin an increasing number (from 1 to 13) of ran-omly selected characters, using a macro in Microsoftxcel. Examples of shuffled matrices are presented inig. 1. The analyses were carried out on 30 shuffledeplicate partitions for each level of noise. Options onAUP* (version 64) were set to 100 replicates of heu-istic search using simple addition, MULPARS, and annlimited maxtree setting.

xploring the Response of the ILD Test to Noise

To interpret the results of the previous section, weroceeded to investigate the relationship between themount of noise in a matrix and the length of theost-parsimonious tree (MPT) which it would produce.sing replicate matrices with varying amounts of noiseenerated as above, we searched for MPTs using

(

403NOISE AND INCONGRUENCE

PAUP* branch and bound option with an unlimitedmaxtree setting. The analyses were carried out on 30shuffled replicates for each level of noise.

RESULTS

Effect of Noise on ILD Test Results

Both data matrices show increasing incongruencewith their duplicate as the proportion of shuffled char-acters in the duplicated set increases (Fig. 2). Signifi-cant results appear when 50–60% of the charactersrepresent noise. Incongruence can therefore exist be-tween one data matrix and another in which nearlyhalf the characters are perfectly congruent with thefirst matrix and there is no additional systematic sig-nal above that expected by chance alone.

We stress that this result cannot be predicted simplyfrom the test’s design. We would expect, as is suggestedby Farris et al. (1994), that the test should act as the

FIG. 1. The data matrices used in this study, their respective clad(boldface).

ograms, and examples showing replicates with four characters shuffled

FIG. 2. Percentage of ILD tests showing significant incongruenceat the 5% level when comparing a homoplasy-free matrix and aduplicate with an increasing number of characters shuffled. Resultsfrom both the bush matrix (triangles, solid line) and the comb matrixcircles, dotted line) are shown.

noisy matrix. When this difference reaches a certain

((Afcl

404 DOLPHIN ET AL.

equivalent of the Mann–Whitney U test and shouldassess a difference in mean (5 signal) without beingaffected by a difference in variance (5 noise) in the twosamples. Our suggestion as to the cause of this ILDsignificance is described below.

Exploring the Response of the ILD Test to Noise

Figure 3a illustrates how MPT length grows with theincreasing amount of noise in a matrix. This relation-ship is nonlinear: a linear relationship would hold ifeach additional shuffled character contributed thesame number of extra steps to the tree length. In fact,as we progressively increase the number of noisy char-acters, the average length of each shuffled character onthe MPT decreases and the average length of eachnonshuffled character increases until their averagelengths become identical at extremely high noise lev-els. This means that at the noisy end of the spectrum itis less costly in terms of tree length to shuffle eachadditional character, and the resulting length to noiserelationship is curvilinear.

As a consequence of this relationship, matrices withintermediate levels of noise produce longer trees thanpredicted by a linear model. Figure 3b shows the effectsthat this has on the ILD test. Matrix A is 20% noiseand gives a tree length of 22 steps, matrix B is of equalsize, but contains 80% noise and gives a tree length of33 steps: a total initial partition tree length of 55 steps.When the characters are repartitioned, each matrixwill contain on average 50% noise and give a replicatesum of tree lengths totalling (29 3 2) 5 58 steps, whichis 3 steps longer than the original. Although the num-bers are arbitrary in this example, it is generally truethat replicate partitions with an intermediate amountof noise will produce a longer sum of tree lengths thanan original partition comprising one noisy and one less

FIG. 3. (a) Most-parsimonious tree length grows in a nonlinearfashion with an increasing proportion of noise in each matrix. Re-sults from both the bush matrix (triangles) and the comb matrixcircles) are shown. Standard error bars are too small to be visible.b) The effect of this curvilinear relationship on the ILD test. Points

and B represent two equally sized matrices; A is relatively noise-ree and produces a shorter tree than the noisier B. When theharacters (and noise) are repartitioned during the ILD test, theength of the average replicate is closer to B than to A.

level, ILD test results will be significant even in theabsence of systematic incongruence.

DISCUSSION

Our findings that significance in the ILD test can beincreased by a difference in levels of noise in two par-titions have several implications for phylogenetic re-search.

(a) Where the ILD test is used to test for particularevents during molecular evolution (recombination, in-trogression, etc.) by assessing the extent of incongru-ence between two sequences, the unknown amount ofsignificance generated by different levels of homoplasywithin the data sets will render the interpretation of Pvalues generated by the test ambiguous.

(b) The ILD test is increasingly being used to de-termine whether combined or separate analysis of par-titions is favorable. The results of this study show thatthe ILD test could lead to a separate analysis of twomatrices representing a similar or identical underlyingtopology but whose characters had evolved at differentrates and thus displayed different amounts of noise.Whether this consequence is advantageous is open todebate (but see Wenzel and Siddall, 1999 for a discus-sion of combining noisy and less noisy data sets). None-theless, with currently unquantifiable contributions tothe significance of the test coming from both incongru-ent signal and noise, the precise meaning of the ILDtest result will be ambiguous.

(c) On a related topic, some of the recent debateconcerning the utility or otherwise of 3rd codon posi-tions in phylogeny reconstruction has been based onthe interpretation of incongruence inferred from theILD test. The opposing views regarding the treatmentof 3rd codon characters are summarized by Vidal andLecointre (1998). Some researchers believe that 3rdcodon characters can create actual conflict deep withinthe tree and should thus be down-weighted to mini-mize incongruence. Others believe that 3rd codon po-sitions should be included in analyses since they areinformative at subterminal nodes and simply noisy atdeeper levels (and so are not destructive of the signalfrom more conservative characters). A discussion of thepositive consequences of including these characterscan be found in work by Kallersjo et al. (1999). Vidaland Lecointre (1998) demonstrate that there is moreincongruence between codon positions than betweenseparate genes using the ILD test and cite this asevidence that 3rd codon positions are sometimes insignificant conflict with other positions rather thanbeing simply noisy. Thus, they refute the view that 3rdcodon characters should always be included in analy-ses because of their interpretation of the ILD test re-sults.

We propose that an alternative null model for the (branch-and-bound search, 10,000 replicates). In 10 of

C

C

C

C

C

D

F

F

G

G

H

J

K

K

405NOISE AND INCONGRUENCE

ILD test, which makes allowance for the major effect ofnoise demonstrated in our study, would be useful insome cases. To detect incongruence that cannot beattributed to noise alone we recommend the followingprocedure: (1) perform an ILD test on the two originaldata partitions (A and B); (2) shuffle the characterstates within the characters of one matrix (A) repeat-edly to generate a series of pure noise partitions; (3)carry out ILD tests between these shuffled replicatesand the original matrix B; (4) repeat the procedurecomparing shuffled replicates of B against unshuffledA; (5) only if the original level of conflict in step 1 issignificantly higher than the levels observed in bothsteps 3 and 4 can we say that the partitions are signif-icantly incongruent.

Shuffling character states within characters retainsfeatures of a matrix such as frequency and between-character distribution of 0’s and 1’s while replacingsignal with noise. Any difference in the ILD test resultsobtained before and after shuffling one matrix in thisway indicates features of agreement or incongruencethat cannot be attributed to noise in the initial parti-tions (although other effects, such as base compositionbias, cannot be discounted).

Two examples of this approach are the following. (1)We further analyzed the mitochondrial data examinedin Cunningham (1997) in which partitions represent1st, 2nd, and 3rd codon positions with the 3rd codonpositions saturated by multiple substitutions (a goodexample of noise). The initial ILD test using branch-and-bound option finds the 2nd and 3rd codon parti-tions to be highly incongruent (P , 0.001)—a strikingresult, given the putatively identical phylogenetic his-tory of the two partitions. Shuffling the 3rd codon char-acter states tends to reduce the level of incongruence,with the mean difference in tree lengths between orig-inal and replicate partitions being 3.18 S.D. (n 5 30)compared to 6.93 S.D. when the unshuffled data areanalyzed. However, the value of 6.93 S.D. lies withinthe 95% confidence limit calculated for the 30 shuffledpartitions. The importance of distinguishing betweennoise and signal in the case of codon position charac-ters has been mentioned above, and this example high-lights the need for an unambiguous incongruence mea-sure. (2) In their extensive study of leptodactylid frogs,Cannatella et al. (1998) examine phylogenetic matriceswhose characters represent a range of morphological,behavioral, and molecular sources. Multiple pairwiseILD tests between the partitions revealed that the onlyinstance of significant incongruence was when a ma-trix comprising advertisement call characters (whichthey named CALLS) was compared with a morpholog-ical matrix.

We created a series of shuffled replicates of theCALLS matrix and performed ILD tests comparingthese replicates with the original morphological matrix

15 instances, the significance of the shuffled replicateILD test was greater than that of the original partition.Thus, we agree with the authors’ hypothesis that theincongruence is due to the small and noisy nature ofthe CALLS matrix.

ACKNOWLEDGMENTS

David Swofford kindly allowed us to publish results obtained froma test version of his program PAUP*. K.D. was supported by aBBSRC Grant. R.B. and C.D.L.O. were supported by NERC Grants.

REFERENCES

Bull, J. J., Huelsenbeck, J. P., Cunningham, C. W., Swofford, D. L.,and Waddell, P. J. (1993). Partitioning and combining data inphylogenetic analysis. Syst. Biol. 42: 384–397.

annatella, D. C., Hillis, D. M., Chippindale, P. T., Weigt, L., Rand,A. S., and Ryan, M. J. (1998). Phylogeny of frogs of the Physalae-mus pustulosus species group, with an examination of data incon-gruence. Syst. Biol. 47: 311–335.arbone, I., Anderson, J. B., and Kohn, L. M. (1999). Patterns ofdescent in clonal lineages and their multilocus fingerprints areresolved with combined gene genealogies. Evolution 16: 354–362.hippindale, P. T., and Wiens, J. J. (1994). Weighting, partitioning,and combining characters in phylogenetic analysis. Syst. Biol. 43:278–287.ho, Y., Qiu, Y. L., Kuhlman, P., and Palmer, J. D. (1998). Explosiveinvasion of plant mitochondria by a group I intron. Proc. Natl.Acad. Sci. USA 95: 14244–14249.unningham, C. W. (1997) Is congruence between data partitions areliable predictor of phylogenetic accuracy? Empirically testing aniterative procedure for choosing among phylogenetic methods.Syst. Biol. 46: 464–478.oyle, J. J., Doyle, J. L., and Brown, A. H. D. (1999). Incongruence inthe diploid B-genome species complex of Glycine (Leguminosae)revisited: Histone H3-D alleles versus chloroplast haplotypes. Mol.Biol. Evol. 16: 354–362.

arris, J. S. (1991). “Arn”. Program and documentation. Molekular-systematiska laboratoriet, Naturhistoriska riksmuseet, Box 50007S 104–105, Stockholm, Sweden.

arris, J. S., Kallersjo, M., Kluge, A. G., and Bult, C. (1994). Testingsignificance of incongruence. Cladistics 10: 315–319.eiser, D. M., Pitt, J. I., and Taylor, J. W. (1998). Cryptic speciationand recombination in the aflatoxin-producing fungus Aspergillusflavus. Proc. Natl. Acad. Sci. USA 95: 388–393.raham, S. W., Kohn, J. R., Morton, B. R., Eckenwalder, J. E., andBarrett, S. C. H. (1998). Phylogenetic congruence and discordanceamong one morphological and three molecular data sets fromPontederiaceae. Syst. Biol. 47: 545–567.oot, S. B., Magallon, S., and Crane, P. R. (1999). Phylogeny of basaleudicots based on three molecular data sets: atpB, rbcL and 18Snuclear ribosomal DNA sequences. Ann. Missouri Bot. Garden 86:1–32.

ohnson, K. P., and Sorensen, M. D. (1998). Comparing molecularevolution in two mitochondrial protein coding genes (cytochrome band ND2) in the dabbling ducks (Tribe: Anatini). Mol. Phylogenet.Evol. 10: 82–94.allersjo, M., Albert, V. A., and Farris, J. S. (1999). Homoplasyincreases phylogenetic structure. Cladistics 15: 91–93.itching, I. J., Forey, P. L., Humphries, C. J., and Williams, D. M.

(1998). “Cladistics: The Theory and Practice of Parsimony Analy-

M

N

N

P

P

Q

phological data sets: An example from the evolution of endopara-

S

S

S

S

S

V

W

406 DOLPHIN ET AL.

sis,” 2nd ed., Oxford Univ. Press, Oxford.Lecointre, G., Rachdi, L., Darlu, P., and Denamur, E. (1998). Esch-

erichia coli molecular phylogeny using the incongruence lengthdifference test. Mol. Biol. Evol. 15: 1685–1695.

Messenger, S. L., and McGuire, J. A. (1998). Morphology, molecules,and the phylogenetics of cetaceans. Syst. Biol. 47: 90–124.

Mickevich, M. F., and Farris, J. S. (1981). The implications of con-gruence in Menidia. Syst. Zool. 30: 351–370.uona, J. (1995). The phylogeny of Elateroidea (Coleoptera), orwhich tree is best today? Cladistics 11: 317–341.ixon, K. C., and Carpenter, J. M. (1996). On simultaneous analysis.Cladistics 12: 221–241.ormark, B. B., and Lanteri, A. A. (1998). Incongruence betweenmorphological and mitochondrial DNA characters suggests hybridorigins of parthenogenetic weevil lineages (genus Aramigus). Syst.Biol. 47: 475–494.

age, R. D. M., and Holmes, E. C. (1998). “Molecular Evolution: APhylogenetic Approach,” Blackwell Sci., Oxford.

iercey-Normore, M. D., Egger, K. N., and Berube, J. A. (1998).Molecular phylogeny and evolutionary divergence of North Amer-ican species of Armillaria. Mol. Phylogenet. Evol. 10: 49–66.uicke, D.L.J., and Belshaw, R. (1999). Incongruence between mor-

sitism among parasitic wasps (Hymenoptera: Braconidae). Syst.Biol. 48: 436–454.

pangler, R. E., and Olmstead, R. G. (1999). Phylogenetic analysis ofBignoniaceae based on the cpDNA gene sequences rbcL and ndhF.Ann. Missouri Bot. Garden 86: 33–46.

tanger-Hall, I. K., and C. W. Cunningham. (1998). Support for amonophyletic Lemuriformes: Overcoming incongruence betweendata partitions. Mol. Biol. Evol. 15: 1572–1577.

wofford, D. L. (1991). When are phylogeny estimates from molecu-lar and morphological data incongruent? In “Phylogenetic Analysisof DNA Sequences” (M. Miyamoto and J. Cracraft, Eds.), pp. 295–333. Oxford Univ. Press, Oxford.

wofford, D. L. (1998). PAUP*. Phylogenetic Analysis Using Parsi-mony (*and other methods). Version 4. Sinauer, Sunderland, MA.

wofford, D. L., Olsen, G. J., Waddell, P. J., and Hillis, D. M. (1996).Phylogenetic inference. In “Molecular Systematics” (D. M. Hillis,C. Moritz, and B. K. Mable, Eds.), pp. 407–514. Sinauer, Sunder-land, MA.idal, N., and Lecointre, G. (1998). Weighting and congruence: Acase study based on three mitochondrial genes in pitvipers. Mol.Phylogenet. Evol. 9: 366–374.enzel, J. W., and Siddall, M. E. (1999). Noise. Cladistics 15: 51–64.