example statistics v1 basic overview and examples

8/2/2019 Example Statistics v1 Basic Overview and Examples

1/23

1

https://controls.engin.umich.edu/wiki/index.php/Basic_statistics:_mean,_median,_average,_standard_deviation,_z-scores,_and_p-value

Introduction to Statistics:

Statistics is a field of mathematics that pertains to data analysis. Statisticalmethods and equations can be applied to a data set in order to analyze andinterpret results, explain variations in the data, or predict future data. A fewexamples of statistical information we can calculate are:

Measure of Central Tendency

Mean = (aka: Average , A measure of the center of adistribution; sum-up the values, divided by count ofObservations)

Mode = Most frequently occurring number or value in a data set Median = Mid-point between the lowest and highest value of the

set (Better to use Median metric if your data set has extremeoutlier data points)

Measure of spread

Standard Deviation = On average, how much each value deviates from themean (average). Standard deviation is a measure of how spread out thevalues in a data set are from the mean (average). It is a measure of thedispersion of the individual observations from the mean.

If you are comparing Average test scores for different Sub-groups(schools), the standard deviation will tell you how diverse the test scoresare for each school.

o Roughly 68%, or two thirds of the scores will be between 80

and 120 (one standard deviation), on either side of the mean.

o Approximately 95% of the scores will be between 60 and 140(two standard deviations on either side of the mean);
https://controls.engin.umich.edu/wiki/index.php/Basic_statistics:_mean,_median,_average,_standard_deviation,_z-scores,_and_p-valuehttps://controls.engin.umich.edu/wiki/index.php/Basic_statistics:_mean,_median,_average,_standard_deviation,_z-scores,_and_p-valuehttps://controls.engin.umich.edu/wiki/index.php/Basic_statistics:_mean,_median,_average,_standard_deviation,_z-scores,_and_p-valuehttps://controls.engin.umich.edu/wiki/index.php/Basic_statistics:_mean,_median,_average,_standard_deviation,_z-scores,_and_p-valuehttps://controls.engin.umich.edu/wiki/index.php/Basic_statistics:_mean,_median,_average,_standard_deviation,_z-scores,_and_p-value


2/23

2

o Roughly 99% of the scores will be between 40 and 160 (threestandard deviations).

Variance = the average of the sum of the squared deviations (squareddeviations of scores about the mean) The variance is simply the standarddeviation squared. The variance represents the average squared deviationfrom the mean.

Range = The difference between the highest and lowest score in adistribution. (Span of values over which your data set occurs)

Measure of Shape

Skewness = the degree of asymmetry extreme scores in adistribution

Positively skewed - A distribution is positivelyskewed when is has a tail extending out to the right(larger numbers) When a distribution is positivelyskewed, the mean is greater than the median

reflecting the fact that the mean is sensitive to eachscore in the distribution and is subject to large shiftswhen the sample is small and contains extremescores.

Negatively skewed - A negatively skweddistribution has an extended tail pointing to the left(smaller numbers) and reflects bunching of numbersin the upper part of the distribution with fewerscores at the lower end of the measurement scale.

Kurtosis = how scores are concentrated in the center of thedistribution, the upper and lower tails (ends), and the shoulders(between the center and tails) of a distribution.


3/23

3

Test of Association (How two things are correlated)?

CorrelationRegression

SlopeY-Intercept

Test of Inference:

Chi-Square t-test

o Independent sampleso Correlated samples

Analysis-of-Variance

What is a Statistic?

In the mind of a statistician, the world consists of populations and samples.

Population = all 7th graders in the United States.

Parameter = Property or characteristic or description of the Population

Parameter = Avg. weight of all 7 th graders in U.S.

Sample = a sub-group of 7th graders in the United States.

Statistic = Avg. weight of our Sub-group of 7 th graders.

Parameters are to populations as statistics are to samples.

A parameter is a property of a population. As illustrated in the exampleabove, most of the time it is infeasible to directly measure a populationparameter. Instead a sample must be taken and statistic for the sample is


4/23

4

calculated. This statistic can be used to estimate the population parameter.(A branch of statistics know as Inferential Statistics involves using samplesto infer information about a populations.) In the example about thepopulation parameter is the average weight of all 7th graders in the UnitedStates and the sample statistic is the average weight of a group of 7thgraders.

Statistics take on many forms. Examples of statistics can be seen below.

Basic Statistics

When performing statistical analysis on a set of data, the mean, median,mode, and standard deviation are all helpful values to calculate.

Standard deviation is the average distance between the actual data and themean.

Mean and Weighted Average

Mean (also known as average), is obtained by dividing the sum of observedvalues by the number of observations, n . Although data points fall above,below, or on the mean, it can be considered a good estimate for predictingsubsequent data points. The formula for the mean is given below asequation (1). The excel syntax for the mean is AVERAGE(starting cell:ending cell).

(1)

However, equation (1) can only be used when the error associated witheach measurement is the same or unknown. Otherwise, the weightedaverage, which incorporates the standard deviation, should be calculatedusing equation (2) below.


5/23


6/23

6

Figure 4Graphic Display of Flat or Spread-Out Score Distribution

Figure 5Display of a Narrow or Concentrated Distribution

Note that he mean and median of these two quite different distributions arethe same ( = 150, Mdn = 150), so simply calculating and reporting thosetwo measures of central tendency would fail to reveal how different the


7/23

7

dispersion of scores is between the two groups. But we can do this bycalculating the standard deviation.

The standard deviation provides us with a measure of just how spread outthe scores are from the average (mean): a high standard deviation meansthe scores are widely spread; a low standard deviation means they'rebunched up closely on either side of the mean.

We'll now calculate the standard deviation for both these distributions. Theformula for the standard deviation is:

Where:(little sigma) is the standard deviation.

d2 is a score's deviation from the mean squared.

is the number of cases.

The numbers we need to calculate the standard deviation for Figure 4, theflat distribution, are in Table 6.

Table 6Data for Figure 4 the Flat Distribution

A B C D E Test Score

(X)Frequency (f) X Mean (d) fd fd 2

100 8 50 400 20,000110 13 40 520 20,800

120 17 30 510 15,300130 20 20 400 8,000140 21 10 210 2,100150 22 0 0 0160 21 -10 -210 2,100


8/23

8

170 20 -20 -400 8,000180 17 -30 -510 15,300190 13 -40 -520 20,800200 8 -50 -400 20,000

SUM 180 132,400

Column A displays the test scores (X).

Column B shows how many people got each test score (f).

Column C is the test score minus the mean (X minus the mean or d).

Column D is the sum of the deviations in column C (fd).

Column E contains the squares of all the deviations.

Of course, to get the deviation of each score from the mean (column C), wehave to calculate the mean, and you already know how to do that. We nowhave what we need to calculate the standard deviation for the flatdistribution in Figure 4:

or

You can do the last part of this calculation, the square root of 132,400/180(which is 736) by using the square-root button on your little hand calculator.

Now let's compute the standard deviation for the data in Figure 5. The dataare in Table 7, and you follow the same steps we've just completed.

Table 7Example of a Narrow or Concentrated Distribution

A B C D E Test Score

(X) Frequency

(f) X - Mean

(d) fd fd 2

100 0 50 0 0110 0 40 0 0


9/23

9

120 0 30 0 0130 10 20 200 4,000140 45 10 450 4,500150 70 0 0 0160 45 -10 -450 4,500170 10 -20 -200 4,000180 0 -30 0 0190 0 -40 0 0200 0 -50 0 0

SUM 180 17,000

or

The two standard deviations provide a statistical indication of the howdifferent the distributions are: 27 for the spread-out distribution and 10 forthe bunched-up distribution.

So once we know the mean and median, why do we need to know thestandard deviation? What use is it?

The standard deviation is important because, regardless of the mean, itmakes a great deal of difference whether the distribution is spread out overa broad range or bunched up closely around the mean.

For example, suppose you have two classes whose mean reading scoresare the same. With only that information, you would be inclined to teach thetwo classes in the same way. But suppose you discover that the standarddeviation of one of the classes is 27 and the other is 10, as in the exampleswe just finished working with.

In the first class (where Standard Deviation 27), you have manystudents throughout the entire range of performance. You'll need to haveteaching strategies for both the gifted and the challenged.


10/23

10

In the second class (where Standard Deviation = 10), you don't have anygifted or challenged students. They're all average, and your teachingstrategy will be entirely different.

The Normal Curve

Before we leave the standard deviation, it's a good time to learn a littlemore about the normal curve. We'll be coming back to it later.

First, why is it called the normal curve? The reason is that so many thingsin life are distributed in the shape of this curve: IQ, strength, height, weight,musical ability, resistance to disease, and so on. Not everything is normallydistributed, but most things are. Thus the term normal curve.

In Figure 6, we have a set of scores which are normally distributed.

The range is from 0 to 200, the mean and median are 100, and thestandard deviation is 20. In a normal curve, the standard deviationindicates precisely how the scores are distributed. Note that the percentageof scores is marked off by standard deviations on either side of the mean.

In the range between 80 and 20 (thats one standard deviation on either side of the mean), there are 68.26% of the cases. In other words, in anormal distribution,

Roughly 68%, or two thirds of the scores will be between 80 and 120(one standard deviation), on either side of the mean.

Approximately 95% of the scores will be between 60 and 140 (twostandard deviations on either side of the mean);

Roughly 99% of the scores will be between 40 and 160 (threestandard deviations).


11/23


12/23

12

_______________________________________________________

_________________________________________________

A histogram is a graphical representation of a frequency table. Histograms


13/23


14/23

14

To find out if one city really is more dangerous than another, you need todetermine a per capita murder rate . That is, the number of murders foreach person in town.

To find that rate, simply divide the number of murders by the totalpopulation of the city. To keep from using a tiny little decimal, statisticiansusually multiply the result by 100,000 and give the result as the number ofmurders per 100,000 people.

In Springfield's case, 50 murders divided by 800,000 people equals amurder rate of 6.25 per 100,000 people. Capital City's 50 murders dividedby 600,000 people equals a murder rate of 8.33 per 100,000 people.

Five years ago, Springfield's 29 murders divided by 450,000 people

equaled a murder rate of 6.44 per 100,000 people. And Capital City's 42murders divided by 550,000 equaled a murder rate of 7.64 per 100,000people.

In Percent , we found that the number of murders in Springfield increased72 percent over five years, while the number of murders in Capital Citygrew by just 19 percent. But when we now compare per capita murders,Springfield's murder rate decreased by almost 3 percent, while CapitalCity's per capita murder rate increased by more than 9 percent.

_________________________________________________________

Common Error #2: Margin of Error and Poor Sample Sizes:misinterpretations in Political Polls.

The margin of error in a sample = 1 divided by the square root of thenumber of people in the sample

How did someone come up with that formula, you ask? the formula isderived from the standard deviation of the proportion of times that aresearcher gets a sample "right," given a whole bunch of samples.

Which is mathematical jargon for..."Trust me. It works, okay?"
http://www.robertniles.com/stats/percent.shtmlhttp://www.robertniles.com/stats/percent.shtmlhttp://www.robertniles.com/stats/percent.shtmlhttp://www.robertniles.com/stats/percent.shtml


15/23

15

So a sample of 1,600 people gives you a margin of error of 2.5 percent,which is pretty darn good for a poll. (See Margin of Error for more details onthat term, and on polls in general.) Now, remember that the size of theEntire Population does NOT matter here. You could have a nation of250,000 people or 250 million and that won't affect how big your sampleneeds to be to come within your desired margin of error. The Math Gods

just don't care.

Of course, sometimes you'll see polls with anywhere from 600 to 1,800people, all promising the same margin of error. That's because oftenpollsters want to break down their poll results by the gender, age, race orincome of the people in the sample. To do that, the pollster needs to haveenough women, for example, in the overall sample to ensure a reasonablemargin or error among just the women. And the same goes for young

adults, retirees, rich people, poor people, etc. That means that in order tohave a poll with a margin of error of five percent among many differentsubgroups, a survey will need to include many more than the minimum 400people in the overall sample.

Check out the following Statistic error:

WASHINGTON (Reuter) - President Clinton, hit by bad publicity recently

over FBI files and a derogatory book, has slipped against Bob Dole in anew poll released Monday but still maintains a 15 percentage point lead.

The CNN/USA Today/Gallup poll taken June 27-30 of 818 registered votersshowed Clinton would beat his Republican challenger if the election wereheld now, 54 to 39 percent, with seven percent undecided. The poll had amargin of error of plus or minus four percentage points.
http://www.robertniles.com/stats/margin.shtmlhttp://www.robertniles.com/stats/margin.shtmlhttp://www.robertniles.com/stats/margin.shtmlhttp://www.robertniles.com/stats/margin.shtml


16/23


17/23

17

surge by Dole to positive public reaction to his resignation. But the nextweek, Dole's surge was gone.

Perhaps there never was a surge. It very well could be that that week's pollwas the One random chance in 20 where the results lie outside the marginof error. Who knows?

Just remember: never place too much faith in one week's poll or survey.No matter what you are writing about, only by look at many surveys canyou get an accurate look at what is going on.

_____________________________________________________

Common Error #3: Be skeptical of "Regression " a process thatcompares one thing to another to see if they are statistically related.They will call such a relationship a "correlation."

1. Always remember that a correlation DOES NOT mean causation.

A study might find that an increase in the local birth rate wascorrelated with the annual migration of storks over the town. Thisdoes not mean that the storks brought the babies. Or that the babiesbrought the storks.

Statisticians call this sort of thing a "spurious correlation," which is afancy term for "total coincidence."


18/23

18

People who want something from others often use regressionstudies to try to support their cause. They'll say something along thelines of "a study shows that a new police policy that we want led to a20 percent drop in crime over a 10-year period in (some city)."

That might be true, but the drop in crime could be CAUSED (due to)something other than that new policy. What if, say, the average ageof those cities' residents increased significantly over that 10 yearperiod? Since crime is believed to be age-dependent (meaning themore young men you have in an area, the more crime you have), theaging of the population could potentially be the cause of the drop incrime.

The policy change and the drop in crime might be correlated. But that

does NOT mean that one caused the other.Common Error #4: Finally, be aware of numbers taken out ofcontext. Again, data that are "cherry picked" to look interesting mightmean something else entirely once it is placed in a different context.

Consider the following example from Eric Meyer , a professionalreporter now working at the University of Illinois:

My personal favorite was a habit we use to have years ago, when I

was working in Milwaukee. Whenever it snowed heavily, we'd call thesheriff's office, which was responsible for patrolling the freeways, andask how many fender-benders had been reported that day. Inevitably,we'd have a leed Headline that said something like, "A fierce winterstorm dumped 8 inches of snow on Milwaukee, snarled rush-hourtraffic and caused 28 fender-benders on county freeways" -- until oneday I dared to ask the sheriff's department how many fender-benderswere reported on clear, sunny days. The answer -- 48 -- made me

wonder whether in the future we'd run stories saying, "A fierce wintersnowstorm prevented 20 fender-benders on county freeways today."There may or may not have been more accidents per mile traveled inthe snow, but clearly there were fewer accidents when it snowed thanwhen it did not.
http://newslink.org/meyerhttp://newslink.org/meyerhttp://newslink.org/meyerhttp://newslink.org/meyer


19/23

19

It is easy for people to go into brain-lock when they see a stack of papersloaded with numbers, spreadsheets and graphs. (And some sleazy sourcesare counting on it.) But your readers are depending upon you to makesense of that data for them.

__________________________________________________

Good Sources for Statistics Data:

http://www.robertniles.com/data/

Basic Stuff

FedStats

U.S. NavalObservatory .

many meters are in a mile. Convert most anything to anything else atonlineconversion.com .
http://www.robertniles.com/data/http://www.robertniles.com/data/http://www.fedstats.gov/http://www.fedstats.gov/http://tycho.usno.navy.mil/what.htmlhttp://tycho.usno.navy.mil/what.htmlhttp://tycho.usno.navy.mil/what.htmlhttp://www.onlineconversion.com/http://www.onlineconversion.com/http://www.onlineconversion.com/http://tycho.usno.navy.mil/what.htmlhttp://tycho.usno.navy.mil/what.htmlhttp://www.fedstats.gov/http://www.robertniles.com/data/


20/23


21/23

21

d health data from theOrganization for Economic Co-operation and Development

calculator to compare the costs of living in U.S.

and selected Canadian cities.

Australia or Canada .

-stop source for information about Latin Americannations can be found by visiting LANIC at the University of Texas.

ation, housing or economic data for any community inthe U.S. from the Census Bureau .

Bureau of Labor Statistics has made a variety of useful nationaleconomic data available through its site. The "Economy at a Glance"section offers monthly employment, inflation and growth numbers for thepast 14 months. The "Data" section offers access to more-detailed BLStimeseries employment data.

on's taxpayers from the InternalRevenue Service .

Social Security Administration has put data profiling SSI recipientsonline, as well as information on the earnings and employment of SocialSecurity-eligible U.S. workers.

Population Reference Bureau provides population resources andworld population data.

Education

llection of U.S. education data is availablefrom the National Center for Education Statistics .

Energy

Energy Information Administration provides worldwide usagedata and demand forecasts for most any energy source you can think of.
http://www.oecd.org/oecd/pages/home/displaygeneral/0,3380,EN-statistics-20-nodirectorate-no-no-no-20,FF.htmlhttp://www.oecd.org/oecd/pages/home/displaygeneral/0,3380,EN-statistics-20-nodirectorate-no-no-no-20,FF.htmlhttp://www.homefair.com/homefair/cmr/salcalc.htmlhttp://www.homefair.com/homefair/cmr/salcalc.htmlhttp://www.abs.gov.au/http://www.abs.gov.au/http://www.statcan.ca/http://www.statcan.ca/http://www.statcan.ca/http://lanic.utexas.edu/http://lanic.utexas.edu/http://lanic.utexas.edu/http://www.census.gov/http://www.census.gov/http://www.census.gov/http://stats.bls.gov/http://stats.bls.gov/http://www.irs.ustreas.gov/taxstats/index.htmlhttp://www.irs.ustreas.gov/taxstats/index.htmlhttp://www.irs.ustreas.gov/taxstats/index.htmlhttp://www.irs.ustreas.gov/taxstats/index.htmlhttp://www.ssa.gov/policy/pubs/http://www.ssa.gov/policy/pubs/http://www.prb.org/http://www.prb.org/http://nces.ed.gov/http://nces.ed.gov/http://nces.ed.gov/http://www.eia.doe.gov/http://www.eia.doe.gov/http://www.eia.doe.gov/http://nces.ed.gov/http://www.prb.org/http://www.ssa.gov/policy/pubs/http://www.irs.ustreas.gov/taxstats/index.htmlhttp://www.irs.ustreas.gov/taxstats/index.htmlhttp://stats.bls.gov/http://www.census.gov/http://lanic.utexas.edu/http://www.statcan.ca/http://www.abs.gov.au/http://www.homefair.com/homefair/cmr/salcalc.htmlhttp://www.oecd.org/oecd/pages/home/displaygeneral/0,3380,EN-statistics-20-nodirectorate-no-no-no-20,FF.html


22/23

22

Finding People

onlinewhite pages . Look for international phone directories and calling codes

here .

States via anybirthday.com . You can also find individuals' last known zipcode.

guide from the U.S. Centers for Disease Control.

Health

National Center for Health Statistics .

World HealthOrganization .

legal issues in health care.

Investing

-tradedcorporations through the SEC's EDGAR site .

Law and Politics

CIA World Factbook

UnitedStates Code
http://www.whitepages.com/http://www.whitepages.com/http://www.whitepages.com/http://www.whitepages.com/intl_sites.plhttp://www.whitepages.com/intl_sites.plhttp://www.anybirthday.com/http://www.anybirthday.com/http://www.anybirthday.com/http://www.cdc.gov/nchs/about/major/natality/sites.htmhttp://www.cdc.gov/nchs/about/major/natality/sites.htmhttp://www.cdc.gov/nchs/datawh.htmhttp://www.cdc.gov/nchs/datawh.htmhttp://www.who.int/whosis/http://www.who.int/whosis/http://www.who.int/whosis/http://www.netreach.net/~wmanning/index.htmlhttp://www.netreach.net/~wmanning/index.htmlhttp://www.sec.gov/edgar.shtmlhttp://www.sec.gov/edgar.shtmlhttp://www.sec.gov/edgar.shtmlhttps://www.cia.gov/library/publications/the-world-factbook/index.htmlhttps://www.cia.gov/library/publications/the-world-factbook/index.htmlhttp://www4.law.cornell.edu/uscode/http://www4.law.cornell.edu/uscode/http://www4.law.cornell.edu/uscode/http://www4.law.cornell.edu/uscode/http://www4.law.cornell.edu/uscode/https://www.cia.gov/library/publications/the-world-factbook/index.htmlhttp://www.sec.gov/edgar.shtmlhttp://www.netreach.net/~wmanning/index.htmlhttp://www.who.int/whosis/http://www.who.int/whosis/http://www.cdc.gov/nchs/datawh.htmhttp://www.cdc.gov/nchs/about/major/natality/sites.htmhttp://www.anybirthday.com/http://www.whitepages.com/intl_sites.plhttp://www.whitepages.com/http://www.whitepages.com/


23/23

23

Thomas

Check this page for links to the web sites of all 50 U.S. state legislatures

The most important aspect of government and political coverage in theUnited States is tracking campaign contributions. (See my article on thesubject.) Here are some links to help you "follow the money."

FECinfo . This site will show who how much money the U.S. presidentialand Congressional candidates have raised, and who's given it to them. Italso tracks contributions to political parties and candidates' personalpolitical action committees. Be sure to take a look at the tools which trackcontributors by their occupation, and who's giving money to out-of-statecandidates.

OpenSecrets.org . This site offers some nice graphical analysis ofcontribution data, as well as detailed local views.

Federal Election Commission . The government agency in charge ofcollecting campaign contribution data. Here's the "official source" for who'sgiving what to whom. Upside: Data from previous campaigns. Downside:You'll have do the data analysis yourself. No fancy tools here. Just data,and lots of it.

Mail

zip codes for any address, or cities for any zip code.

Canada .
http://thomas.loc.gov/http://thomas.loc.gov/http://www.ncsl.org/public/sitesleg.htmhttp://www.ncsl.org/public/sitesleg.htmhttp://www.ncsl.org/public/sitesleg.htmhttp://www.latimes.com/technology/la-tech-findit4.storyhttp://www.latimes.com/technology/la-tech-findit4.storyhttp://www.latimes.com/technology/la-tech-findit4.storyhttp://www.tray.com/fecinfo/http://www.tray.com/fecinfo/http://www.opensecrets.org/http://www.opensecrets.org/http://www.fec.gov/finance_reports.htmlhttp://www.fec.gov/finance_reports.htmlhttp://www.usps.com/ncsc/lookups/lookup_zip+4.htmlhttp://www.usps.com/ncsc/lookups/lookup_zip+4.htmlhttp://www.canadapost.ca/tools/pcl/bin/advanced-e.asphttp://www.canadapost.ca/tools/pcl/bin/advanced-e.asphttp://www.canadapost.ca/tools/pcl/bin/advanced-e.asphttp://www.usps.com/ncsc/lookups/lookup_zip+4.htmlhttp://www.fec.gov/finance_reports.htmlhttp://www.opensecrets.org/http://www.tray.com/fecinfo/http://www.latimes.com/technology/la-tech-findit4.storyhttp://www.ncsl.org/public/sitesleg.htmhttp://thomas.loc.gov/

example statistics v1 basic overview and examples

Documents