7/28/2019 Concept Of Statistical Significance On Research.docx

1/2

Research Methods - Mar 2013

Q2. Explain what we mean by the concept of statistical signi f icance and draw on a publ ished

research paper to i ll ustrate an appli cation of the concept

A test of statistical significance is a tool of inference used by researchers to assess the evidence in favour for or

against some claim about a population from which a smaller sample has been drawn. The concept of statistical

significance is intimately linked with the concept of null hypothesis testing where these claims are formulated as

nullH0 and alternativeHa hypotheses which are statements related to certain population parameters (most

commonly the mean).

There are numerous forms of significance testinge.g. Z test , Student t-test, F-Test , etc but the key concept is that

a these tests assume a certain distribution of the test parameter under a null hypotheses and measure the likelihood

(p-value) of the observed results occurring by chance alone ASSUMING the null hypothesis is true. This statistic is

then compared against a level of signifance that is decided prior to the test (typically 0.05 or 0.01) where if the p-

value is less than or equal to the significance level the null hypotheses should be rejected in favour of the alternative

hypotheses. (Thompson (1994) as quoted in Miller & Salkind (2002))

The result of the significance test is always given in terms of the null hypothesis where the options would be to

either rejectH0 in favor ofHa" or "do not rejectH0". (Notethis is not exactly the same as saying accept the

alternative hypotheses). The test of significance is therefore essentially a means for researchers to avoid saying that

there is a relationship where in fact there is none.

Flyvberg et al (2002) conducted a research study focused on making a statistically significant study of cost

escalation in transportation infrastructure projects based on a sample of 258 different project with the aim of

demonstrating that there is a systematic under-estimation of cost at the point of project approval.

The researchers first established a way of defining a metric around which the hypothesis could be developed. This

metric was the cost escalation % of a project which is the completed cost of the project divided by the budgeted cost.

From this metric, a null hypotheses was then proposed that there is equal chance of under-estimation and over-

estimation where the alternative hypotheses is that under-estimation is more common than over-estimation. A two-

sided test using the binomial distribution which is a test of deviations from a theoretically expected distribution of

observations into only two categories (In this example was represented by a binomial of n=258 and p=50% which

was checked against the observed data) Based on this test, the null hypothesis was rejected as the p-value was less

than the significance level of 0.001.

Secondly another null hypotheses was that the size of the under-estimation and over-estimation are similar. A Mann-

Whitney test that is used to assessing whether one of two samples of independent observations tends to have larger

values than the other was applied to check the p-value of the observed data. The null hypotheses was then rejected as

the p-value was less than the significance level of 0.001.

7/28/2019 Concept Of Statistical Significance On Research.docx

2/2