1. Sucheta Tripathy, IICB, November December 2013

2. Chi square test Sucheta Tripathy, Biostatistics coursework IICB,Nov, 2013 3. Definitions Model or Hypothesis Null Hypothesis There is no significant difference between the 2. TRU E FALS EGoodness of fit 4. What you need A probability value Degree of freedom A contingent table Determine if the deviation is due to chanceAccept10% Reject 5. Chi Square Test 6. Example 1 Mendellian law of dominanceAa X AaA aA -> Tall (Dominant) a -> Dwarf (recessive) Aa is .a X AAA Aa Aa aa639 Tall and 281 dwarf Chi square requires that you have numeric values Chi square should not be calculated if the expected value is less than 5 7. Choosing a Test First Check if there is a Hypothesis to check If yes, then decide which one If No, then there is NO statistical test for that. What is there.Parametric tests have data that comes in a standard probability distribution. Non-parametric studies can be used for both normally and not-normally distributed data: Question: Then why not to use them always? Parametric tests make a lot of assumptions: If the assumptions are correct, the results are more accurate. 8. Choosing a Test First Check if there is a Hypothesis to check If yes, then decide which one If No, then there is NO statistical test for that. What is there.Parametric tests have data that comes in a standard probability distribution. Non-parametric studies can be used for both normally and not-normally distributed data: Question: Then why not to use them always? Parametric tests make a lot of assumptions: If the assumptions are correct, the results are more accurate. 9. Example 9:3:3:1 10. Example Number of sixes Rolls 0 1 2 3Number of 48 35 15 03p1 = P(roll 0 sixes) = P(X=0) = 0.58 P(k out of n) = p2 = P(roll 1 six) = P(X=1) = 0.345 p3 = P(roll 2 sixes) = P(X=2) = 0.07 p4 = P(roll 3 sixes) = P(X=3) = 0.005n! k!(n-k)!http://www.mathsisfun.com/data/binomial-distribution.htmlpk(1-p)(n-k) 11. Parametric Two samples compare mean value for some variable of interestNonparametrict-test for independent samplesWald-Wolfowitz runs test Mann-Whitney U test KolmogorovSmirnov two sample test 12. Compare two variables measured in the same sampleParametric t-test for dependent samples If more than two variables Repeated are measured in same measures sample ANOVANonparametric Sign test Wilcoxons matched pairs test Friedmans two way analysis of variance Cochran Q 13. Null Hypothesis Coined by English Geneticist Ronald Fischer in 1935. At a given probability can either be true or falseComparing Populations/datasets: Population A and Population B Null hypothesis is true -> No significant difference between the populations. Null Hypothesis is false -> Significant difference between populationsThere are formula to calculate the value of a population comparison: There are look up tables with values If calculated value is less than look up value -> Null hypothesis is False else True 14. Null Hypothesis Testing You have a doubt that whenever it rains your experiment fails?!!! NULL Hypothesis is True: No significant difference (Your experiment failing and raining) -> Rain has nothing to do with your experiment failingNULL Hypothesis is false: There is a significant damage to your experiments when it rains -> Rain ruins your experiment!!Record when your experiment fails and check if it rains during that time. It may be so that it happens by chance or it may be so that there indeed is a relationship. 15. Lab study vs statistics research http://www.youtube.com/watch?feature=player_embedded&v=PbODigCZqL8 16. T-test The t-statistic was introduced in 1908 by William SealyGosset Used in a normally distributed populationhttp://www.socialresearchmethods.net/kb/stat_t.php 17. T-testSqrt((Sum(D^2) (sum(D))^2/n)/n-1) 18. Why Standardize?? 19. ANOVA: F statistics Analysis of variance One Way Two way22Between Groups Within groupsCancel out between variation with group variation 20. So How big is F? Since F is Mean Square Between / Mean Square Within = MSG / MSEA large value of F indicates relatively more difference between groups than within groups (evidence against H0)To get the P-value, we compare to F(I-1,n-I)-distribution I-1 degrees of freedom in numerator (# groups -1) n - I degrees of freedom in denominator (rest of df) 21. Connections between SST, MST, and standard deviation If ignore the groups for a moment and just compute the standard deviation of the entire data set, we sees2x ij n 1x2SST DFTMSTSo SST = (n -1) s2, and MST = s2. That is, SST and MST measure the TOTAL variation in the data set.SST: Sum of squares of Treatment MST: Mean square of treatment DFT: Degree of freedom of treatment 22. Connections between SSE, MSE, and standard deviation Remember:sixijxini212SS[ Within Group i ] dfiSo SS[Within Group i] = (si2) (df i )This means that we can compute SSE from the standard deviations and sizes (df) of each group:SSESS[Within] 2 is ( ni1)SS[Within Group i ] 2 is (dfi ) 23. Computing ANOVA F statistic data group 5.3 1 6.0 1 6.7 1 5.5 2 6.2 2 6.4 2 5.7 2 7.5 3 7.2 3 7.9 3 TOTAL TOTAL/dfgroup mean 6.00 6.00 6.00 5.95 5.95 5.95 5.95 7.53 7.53 7.53WITHIN difference: data - group mean plain squared -0.70 0.490 0.00 0.000 0.70 0.490 -0.45 0.203 0.25 0.063 0.45 0.203 -0.25 0.063 -0.03 0.001 -0.33 0.109 0.37 0.137 1.757 0.25095714overall mean: 6.44BETWEEN difference group mean - overall mean plain squared -0.4 0.194 -0.4 0.194 -0.4 0.194 -0.5 0.240 -0.5 0.240 -0.5 0.240 -0.5 0.240 1.1 1.188 1.1 1.188 1.1 1.188 5.106 2.55275F = 2.5528/0.25025 = 10.21575 24. Validation The larger the F value ->More variation reject null hypothesis 25. AX-xsqu areBX-xsqu areC162724228149523756331458688056739226487971726406883676945Mea n TMS TO MS MST(between) and MSE (within)df1 and df2X-xsqu are 26. In Summary SST(x ijx)22s (DFT)obsSSE(x ijxi )obsSSG22si (df i ) groups(x i obsSSE SSGx)2n i (x ix)2groupsSST; MSSS ; F DFMSG MSE 27. 2 RStatisticR2 gives the percent of variance due to between group variationR2SS[Between ] SS[Total ]We will see R2 again when we study regression.SSG SST 28. Wheres the Difference? Once ANOVA indicates that the groups do not all appear to have the same means, what do we do?Analysis of Variance for days Source DF SS MS treatmen 2 34.74 17.37 Error 22 59.26 2.69 Total 24 94.00Level A B PN 8 8 9Pooled StDev =Mean 7.250 8.875 10.111 1.641StDev 1.669 1.458 1.764F 6.45P 0.006Individual 95% CIs For Mean Based on Pooled StDev ----------+---------+---------+-----(-------*-------) (-------*-------) (------*-------) ----------+---------+---------+-----7.5 9.0 10.5Clearest difference: P is worse than A (CIs dont overlap) 29. Multiple Comparisons Once ANOVA indicates that the groups do not all have the same means, we can compare them two by two using the 2-sample t test Weneed to adjust our p-value threshold because we are doing multiple tests with the same data. There are several methods for doing this. If we really just want to test the difference between one pair of treatments, we should set the study up that way. 30. Tuckeys Pairwise ComparisonsTukey's pairwise comparisonsFamily error rate = 0.0500 Individual error rate = 0.019995% confidence Use alpha = 0.0199 for each test.Critical value = 3.55Intervals for (column level mean) - (row level mean) A B-4.863 -0.859These give 98.01% CIs for each pairwise difference.-3.685 0.435PB-3.238 0.76698% CI for A-P is (-0.86,-4.86)Only P vs A is significant (both values have same sign) 31. Tukeys Method in R Tukey multiple comparisons of means 95% family-wise confidence leveldiff lwr upr B-A 1.6250 -0.43650 3.6865 P-A 2.8611 0.85769 4.8645 P-B 1.2361 -0.76731 3.2395 32. Independent sample t-test Number of words recalled df = (n1-1) + (n2-1) = 18tx1 x2 s x1 x2t ( 0.05,18) t19 26 1 2.101t ( 0.05,18) Reject H07 33. T test One sample t test Unpaired and paired t test Same set of subjects over a period of time Independent sets of subjects over a period of time.http://www.youtube.com/watch?v=JlfLnx8sh-oOne tailed and two tailed t-test: One tailed: Average height of class A is greater than class B Two tailed: Average height of class A is different from class B 34. Z- test statistics Sample size is large Population variance is knownSample size is small population variance is unknown go for t-test 35. Calculation of z value Z = X - / sqrt (variance/n) Suppose that in a particular geographic region, the mean and standard deviation of scores on a reading test are 100 points, and 12 points, respectively. Our interest is in the scores of 55 students in a particular school who received a mean score of 96. We can ask whether this mean score is significantly lower than the regional mean that is, are the students in this school comparable to a simple random sample of 55 students from the region as a whole, or are their scores surprisingly low? We begin by calculating the standard error of the mean: 36. F-tests / Analysis of Variance (ANOVA) t=obtained difference between sample means difference expected by chance (error)variance (differences) between sample meansF=variance (differences) expected by chance (error)Difference between sample means is easy for 2 samples:(e.g. X1=20, X2=30, difference =10) but if X3=35 the concept of differences between sample means gets tricky 37. F-tests / Analysis of Variance (ANOVA) Simple ANOVA example Total variabilityBetween treatments varianceWithin treatments variance------------------------------------------------------Measures differences due to:Measures differences due to:1.Treatment effects1. Chance2.Chance 38. F-tests / Analysis of Variance (ANOVA)F=MSbetweenWhen treatment has no effect, differences between groups/treatments are entirely due to chance. Numerator and denominator will be similar. F-ratio should have value around 1.00MSwithin When the treatment does have an effect then the between-treatment differences (numerator) should be larger than chance (denominator). F-ratio should be noticeably larger than 1.00 39. F-tests / Analysis of Variance (ANOVA) Simple independent samples ANOVA exampleF(3, 8) = 9.00, p Reject Null Hypothesis 49. Calculating U value For smaller dataset: U is the count of ranks of smallerdataset. For larger dataset:U1 = R1 n1(n1+1)/2 U2 = R2 n2(n2+1)/2 50. Kruskal-Wallis test (H Test) Non-parametric test Equivalent to Anova (F test) in parametric test Does not require the distribution to be normal Distribution need to be independent Used more often when the distribution is un-equal. Data is ordinal Assumes the distribution to have the same shape: 1. If one distribution is skewed to left and other to the right (un-equal variance), this test will give in-accurate result 51. Kruskal Wallis Test Group AGroup BGroup C27203428314143183623721309226 52. Kruskal Wallis Test Define Null or alternative Hypothesis State probability Calculate Degree of Freedom Find critical value Calculate the test hypothesis State resultH0 Accept NULL hypothesis: There is no difference between the samples H1 Reject NULL hypothesis : There is difference between the samples > Critical value reject null hypothesis 53. RankValueGroup AGroup BGroup C1227203423283134414346183623577213068922679814918Group AGroup BGroup C141017106161121382122291813132351115142771241530Total R 3965671631n66173418362 Ri n201H= 12 N(N+1)6-3 (N+1)12/18(19) X (39^2/6 + 65^2/6 + 67^2/6) - 3(18+1) =2.854Critical value Reject NULL hypothesis (5.99) 54. Kolmogorov Smirnov test(KS) Non-parametric Distribution is unknown One way and Two way One way Checks the goodness of fit Two way - compare the distributionGoodness of Fit: A Hypothesis (Mendels law of dominance) NULL Hypothesis: H0 : F(x) = F*(x) for all x H1: F(x) = F*(x) for at least one value of x 55. The K-S statistic Dn is defined as:K-S testDn = max [ | Fn(x) - F(x) | ] whereDn is known as the K-S distance n = total number of data points F(x) = distribution function of the fitted distribution Fn(x) = i/n i = the cumulative rank of the data point 56. Kolmogorov Smirnov test(KS) Group 1Group 2Not confident204Slightly confident3027Somewhat confident1328Confident2018Very confident41471. Take Total 2. Find Frequency 3. Calculate cumulative frequency 4. Find difference 5. Get the largest difference 6. Find critical value (1.36/sqrt(sample size)) 7. Test goodness of fit e.g; Our D > crit D (Distribution is unequal) -> reject NULL Hypothesis 57. Group 1 FreqCumul ative freqGroup 2FreqCumul ative frqDNot confide nt200.16120.161240.03220.03220.129Slightly confide nt300.24190.403270.21770.250.153Somew hat confide nt130.1040.508280.2250.470.032Confide nt200.1610.669180.1450.620.048Very confide nt410.3301470.37910Critical D = 1.36/sqrt(n1+n2/n1*n2) = Test NULL Hypothesis 58. Kolmogorov Smirnov test(KS) Group 1Group 2Not confident204Slightly confident3027Somewhat confident1328Confident2018Very confident41471:2:3:2:11. Take Total 2. Find Frequency 3. Calculate cumulative frequency 4. Find difference 5. Get the largest difference 6. Find critical value (1.36/sqrt(sample size)) 7. Test goodness of fit e.g; Our D > crit D (Distribution is unequal) -> reject NULL Hypothesis 59. Methods of Estimation Methods of moments Maximum likelihood Bayesian Estimators Markov chain monte carlo Why?? Population size is too large Testing a hypothesis on a set of samples 60. Probability density function (pdf ) -> For continuous variables Probability mass function (pmf) -> For discrete variablesParameter space Set of all Family of pdf/pmfEstimator T is unbiased: if the sample parameter is . Population parameter 61. Probability density function 62. Estimation Methods Data gets 2 or multi dimensional.. 63. Method of maximum likelihood The maximum likelihood estimates of a distributiontype are the values of its parameters that produce the maximum joint probability density or mass for the observed data X given the chosen probability model.Maximum likelihood is more general, can be applied on any probability distribution. 64. The MLE Best parameters obtained by maximizing theprobability of the observed samples. Has good convergence properties as sample sizes increase: Estimated value may equal real value with Large N Applications are many: From speech recognition to natural language processing to computational biology. 65. Simple MLE: Coin tossing Toss a coin: Head Tail Flip coin 10 times (n) = H, H, H, T, H, T, T, T, H, T => 1, 1, 1, 0, 1, 0, 0, 0, 1, 0 An appropriate model for getting a head in a single flip is: If P = 0.6 and Xi = 0 and Xi =1 66. The maximum likelihood Example: We want to estimate the probability, p, that individuals are infected with a certain kind of parasite. Ind.:Probability of Infected: observation:11p201-p31p41p501-p61p71p801-p901-p101pThe maximum likelihood method (discrete distribution): 1. Write down the probability of each observation by using the model parameters 2. Write down the probability of all the dataPr( Data | p)p 6 (1 p) 43. Find the value parameter(s) that maximize this probability 67. The maximum likelihood Example: We want to estimate the probability, p, that individuals are infected with a certain kind of parasite.01-p31p41p501-p61p71p801-p901-p101pPr( Data | p)p 6 (1 p) 4- Find the value parameter(s) that maximize this probability 0.00122L( p )0.0008p0.00041L(p, K, N)1Likelihood function:0.0000Ind.:Probability of Infected: observation:0.00.20.40.6 p0.81.0 68. Brute Force 69. Likelihood Function x1n L(P|X1..Xn) = F(xi|P) i=11-x1=P (1-P) x1 x2x21-x2xn1-x1= P P P (1-P) =PxnP (1-P) P (1-P) 1-x2(1-P)..(1-P)1-xnx1+x2+x3.+xn n (x1+x2..+xn) (1-P) n n Xi=Pi=1 Xi(1-P)n - i=11-xn 70. Analytically maximum likelihood can also be foundBy finding the derivative with respect to P and finding where the slope is 0.2 log http://www.ics.uci.edu/~smyth/courses/cs274/papers/MLtutorial.pdf 71. Recap Get the population type set the equation. Write the loglikelihood function Differentiate Set the value of differentiation 0. Solve the equation to estimate the parameter. 72. Methods of moments Oldest method Distribution dependent Geometric Poisson Bernoulii. Depends upon PDF 73. Methods of Moment Population moments can be determined by samplemoments. Can be robust Sample mean can determine population mean andsample variance can determine population variance. Does Not work well when the distribution is exponential.