the nature of science design of controlled experiments

The Nature of Science

Design of Controlled Experiments

Lesson Rationale

Based on personal observations while grading AP Free-Response questions and the debrief which ended the conference, most students have a poor understanding of the design of controlled experiments.

Lesson Milestones

Lecture reviews of the:

• essential nature of science

• generally accepted elements of the scientific method

• construct of a controlled experiment

• basic statistical analysis tools from Appendix B: Statistics Primer

Lesson Milestones

Tasks

•Accomplishment and review of the 2013 AP Free Response question, Fruit Fly Choice Chamber

Key Concepts

1. Science as a means of understanding the natural word

2. Scientific method

3. Designing a Controlled Experiment

4. Statistics (Chi Square Test) Appendix B

Learning Objectives

1. Describe what science can do and what it cannot do.

2. Define the terms observation and data.

3. Define and develop a hypothesis.

4. Explain how scientists test a hypothesis using generally accepted steps to a scientific method.

5. Distinguish between manipulated and responding variables.

Learning Objectives

6. Distinguish between the colloquial and scientific meanings of the terms theory and law.

7. Explain the terms falsifiable and reproducibility.

8. Calculate a Chi-square analysis and use the result to justify acceptance or rejection of a null hypothesis.

Materials

1. Matrix for Planning and Assessing Scientific Inquiry

2. AP Free Response, Fruit Fly Behavior

3. AP Handout of Equations

AP Investigation/Lab

• AP Investigations – None

• Chi-Square Lab

Assignments

• Read Appendix B: Statistics Primer

• Focus – Step 5: Inferential Statistics

Discussion

How do these pictures relate to each other?

Discussion

http://en.wikipedia.org/wiki/File:Measles_US_1944-2007_inset.png

Discussion

2011 – 118 cases of measles reported in the United States

89% of those (105) patients had not been vaccinated

What Science Is and Is Not

Science – from the Latin word scientia, meaning “knowledge”

Until the Enlightenment (18th century) science was defined as any recorded knowledge


“We may well discover from science many interesting answers to the question ‘How does life work?’

What we cannot discover, through science alone are the answers to the questions ‘Why is there life anyway?’ and ‘Why am I here?’ “

- Dr. Francis S. Collins, The Language of God


Science – Investigate and understand the natural world, explain events, and use explanations to make useful predictions.

•Process of reaching an understanding of the natural and physical world.

•Requires a combination of open-mindedness to new evidence and skepticism.

•Scientific understanding changes with new discoveries – new evidence.


Three things that set science apart:

1. Deals only with the natural world.

2. Collects and organizes information in an orderly way, looking for patterns and connections between events.

3. Proposes explanations for events that can be tested by experiments.


Faced with a problem or question, scientists seek a solution through testing.

What is the process of finding the solution?• There is no one way; no one method• But there are some generally accepted

guidelines that constitute what is called the scientific method.

Scientific Method

Scientific method• A means of gaining knowledge about the

natural world byo making observations,o posing hypotheses, ando conducting experiments to test those

hypotheses. Glossary, Hillis

Scientific Method

Observation• Information, gathered by the senses, about

structures, processes and events.• Scientific observation should be done in an

orderly way.• What can I see, hear, sense in any way?• What can I measure?

Scientific Method

Observation• Data – information; evidence; recorded

observations and measurements, either in words or numbers.

• Accurate data is the evidence that leads to conclusions and useful predictions.

• Two categories of data:o Qualitative datao Quantitative data

Scientific Method

Qualitative data • Descriptions of appearance, impressions,

etc., no numbers• Involves a judgment made by the observer.

Quantitative data• Expressed as numbers, obtained by

measuring or counting.• Mass, weight, number of individuals, size in

centimeters, etc.

Scientific Method

Qualitative data frequently taken in studies of animal behavior in the wild.

The challenge is to describe the animal behavior in an objective manner.

Unscientific observers frequently attribute human qualities and motivations to animals.• Known as anthropomorphism

Scientific Method

Let’s say you’ve collected some data. How do you…• Interpret the data and develop an

explanation of the event?• Form an idea of how and why something

works?

Scientific Method

Develop the tentative idea into a hypothesis• In science, a hypothesis

– Is a tentative answer to a well-framed question; an explanation on trial

– A predication that can be tested• We all use hypotheses in solving everyday

problems

LE 1-25a

Hypothesis #1:Dead batteries

Hypothesis #2:Burnt-out bulb

Observations

Question

LE 1-25b

Hypothesis #1:Dead batteries

Hypothesis #2:Burnt-out bulb

Test prediction

Test falsifies hypothesis

Prediction:Replacing batterieswill fix problem

Prediction:Replacing bulbwill fix problem

Test prediction

Test does not falsify hypothesis

Scientific Method

About writing a hypothesis…

A valid hypothesis• Contains a prediction• And the predication can be tested in some

way

A valid and correct hypothesis• Is supported by the evidence gathered by

the test• The predication came true!

Scientific Method

This means that you can have a valid, but ultimately incorrect hypothesis.• The hypothesis was not supported by the

evidence gathered by the test.

Scientific Method

Hypotheses are often written in an “If…then” format• If this is done, then the result will be…

“If…then” helps the student construct a valid hypothesis• One that contains a prediction

Scientific Method

Observe, measure, or experiment to test the hypothesis• Did the prediction come true?• Test will help scientists conclude if the

evidence will either support or not support the hypothesis.

Scientific Method

The test of a hypothesis is an experiment.

• Actions or observations taken to test a hypothesis

A more defined and structured experiment is known as a controlled experiment

• All variables except the variable being tested are kept unchanging, or controlled.

Design of a Controlled Experiment

Controlled experiment

An experiment in which a sample is divided into two groups whereby the experimental groups are exposed to manipulations of an independent variable while one group serves are as untreated control.

The data from the various groups are then compared to see if there are changes in a dependent variable as a result of the experimental manipulation.

• Glossary, Hillis


Controlled experiment

All variables, except the variable being tested for the hypothesis,• are kept unchanging, controlled, or

constant• Goal is to make the only difference between

the two groupso the variable whose influence is predicted

in the hypothesis


Variables• Factors in an experiment that can change• Temperature, light, time, materials, etc.• Many variables can be acting at the same

time

Constant• variable which is deliberately maintained at a

predetermined value during the experimento Example: using an incubator to control

temperature while growing bacterial cultures


Two critical variables that apply to the hypothesis:• Manipulated variable, or independent

variable – is deliberately changed.• Responding variable, or dependent

variable – is being observed and changes in response to the manipulated variable.

The hypothesis must connect directly to these two variables!


There are also two “groups” involved in a controlled experiment• Experimental group• Control group


• Experimental Group: Group or sample that is experimented ono sample that is subjected to the

manipulated variable• Control Group: Another group or sample

that is not experimented on, but in every other way exactly alike to the first groupo sample not subjected to manipulated

variable that is used as comparison with experimental group.


Randomization If the design requires sampling of a

population – which is likely – such samples must be randomly selected

Figure B1 Sampling From a Population


About ready to start the controlled experiment…

Will the “If this happens, this should be the result” prediction hold true?

Must be able to compare the results of the experimental and control groups• Must show that a phenomena occurs after a

certain treatment is given, and it does not occur when the treatment is withheld.


Did the data support the predication made by the hypothesis?• If yes, hypothesis is valid

o But are any new questions raised?

• If no, hypothesis is invalido Is it possible to rework the hypothesis or

is an entirely new hypothesis necessary?


Conclusion• The place where you got tired of thinking.• Valid hypothesis may allow scientists to

predict outcomes, or make general statements, for much larger groups.


Multiple trials build confidence in results• Conduct the experimental several times

using the same procedure.• If results are consistent, shows the first time

was not a fluke.


Reproducibility• All scientists must be able to do the

experiment done by one scientist.• Predictions are tested by experiments which

can be reproduced by others to see if the same results occur.

Publishing and Peer Review• Fellow scientists must be able to critique the

experiments and results.


Science is not fair

•Not open to all ideas and explanations.

•Must be driven by the evidence.


Once an explanation is out in the scientific community…

•frequently finds application to related fields and other similar investigations.

A common misconception

Truth be said, impossible to keep everything constant except one variable in field experiments.

Very difficult for highly regulated laboratory experiments.

Control groups also used to cancel the effects of environmental variables that cannot be controlled or eliminated.

When (Controlled) Experiments are Not Possible

At times your situation does not allow you to test a hypothesis:• Technology needed for test does not yet exist• Ethical and legal restrictions on human testing• Laboratory setting may cause bias in results• Too many variables exist; can’t control them• If you want to observe animal behavior in the

wild, then observe them in the wild, not the laboratory

How a Theory Develops

Even though a valid hypothesis exits, the next step is not a brand, new scientific theory.

•Scientific meaning of the word theory is often misunderstood by students and the general public.

•Let’s discuss the word theory for a moment.


Theory

“A well tested explanation that unifies a broad range of observations.”

- Dr. Kenneth Miller


“All scientific theories represent a framework for making sense out of a body of experimental observations.

But the primary utility of a theory is not just to look back, but to look forward. A viable scientific theory predicts other findings and suggests approaches for further experimental verification.”

- Dr. Francis S. Collins


Is there a process to developing a theory?• A hypothesis or set of hypotheses have

been tested – repeatedly and by ideally different investigators – and have not been contradicted by evidence.

• Now an explanation develops which explains of all or much of the data we have and offers valid predictions that can be further tested.


In science, a theory can never be proven true, because we can never assume we know all there is to know.

Instead, theories remain standing until they are disproved by new evidence, at which point they are thrown out altogether or modified to fit the additional data.

http://en.wikipedia.org/wiki/Theory


There are theories which have such a great body of supporting evidence they are essentially “proven” in the scientific sense:• Newton’s theory of universal gravitation• Cell theory• Heliocentric theory• Atomic theory

Can you ever imagine these theories being proved wrong or falsified?


Falsifiability• A theory is scientific if it leaves open the

possibility of being proven wrong or false.o Vital part of science philosophy and

practice.• With the discovery of new evidence, even

theories long held may be discarded and replaced by better theories.


Falsifiability• It must be possible to make an observation

that would show the hypothesis to be false.• If an observation is impossible, then

falsifiability is not possible. Therefore, the theory or hypothesis is not scientific.


Theory: Colloquial Meaning • Speculation; a guess• “Oh, it’s just a theory.”

Oh, it’s just a theory…

"Evolution is a 'theory', just like gravity. If you don't like it, go jump off a bridge."

oAnonymous


"It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong" - Richard Feynman

http://en.wikiquote.org/wiki/Richard_Feynman


"The story of a theory's failure often strikes readers as sad and unsatisfying. Since science thrives on self-correction, we who practice this most challenging of human arts do not share such a feeling. We may be unhappy if a favored hypothesis loses or chagrined if theories that we proposed prove inadequate. But refutation almost always contains positive lessons that overwhelm disappointment, even when [...] no new and comprehensive theory has yet filled the void."

Stephen Jay Gould (1941-2002), "Bully for Brontosaurus", The Face of Miranda (1991)

http://en.wikiquote.org/wiki/Stephen_Jay_Gould


Sometimes you hear of scientific laws

Laws are not:

• the same as theories

• the next rung up on the ladder from theories


Laws are

• Observations that continue to fit the evidenceo Never contradicted by evidenceo Appear everywhere in the universeo Unchanged since first discovered, since

the beginning of the universe and likely will remain so

o Simple in that they are typically expressed in a simple mathematical equation


For example, Newton’s Laws of Motion or Boyle’s Gas Law, or the law of conservation of energy

Why discuss obsolete theories?

History shows that science will discard or change even strongly held theories if new evidence comes to light.

Video Summation

Creative review of the philosophy of science: Symphony of Science – The Poetry of Reality.

http://www.symphonyofscience.com/videos.html

http://www.symphonyofscience.com/videos.html

A final thought on scientific investigation

Occam's Razor (also spelt Ockham's Razor), is a principle attributed to the 14th century English logician and Franciscan friar, William of Ockham.

In its simplest form, Occam's Razor states that one should make no more assumptions than needed. Put into everyday language, it says

The simplest explanation is the best.

A final thought on scientific investigation

Better said:

When multiple explanations are available for a phenomenon, the simplest version is preferred.

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=1l5vMiCi_qRmaM&tbnid=sVcv9wkIPDaqnM:&ved=0CAUQjRw&url=http%3A%2F%2Fundsci.berkeley.edu%2Farticle%2Fhowscienceworks_12&ei=oxLnUZWeE6vFiwKR14BY&bvm=bv.49478099,d.cGE&psig=AFQjCNFOdSuVD2WM2jC0rDBDmiVaS6NneQ&ust=1374184418305731

Summation

The AP Biology ExamThe Reader’s Perspective

AP Biology Exam: Reader Perspective

Distribute handout, AP Reader Experience 2013 2014

Take a moment to read.

Review the memo in class.

AP Biology Exam: Reader Perspective

Distribute the handout, AP Free Response Questions 2013

Take a moment to read quickly through the questions.

Now let’s focus on question #1…

Fruit FlyDrospholia melangaster

Question 1: Fruit Fly behavior


You did not need to know anything about the fruit fly itself…


..except their size and affinity for sugary solutions.

But even without these two facts, it could be answered!


What was essential:

Clear understanding of the nature of science, and

How to set up a controlled experiment!

First a few stories, a lesson on controlled experiments, then you’ll answer this question!


Distribute worksheet, AP Biology Free Response Question #1, Fruit Fly Choice Chamber 2013


Process

•You will write your answer to subparts (a) through (e) within 22 minutes in class

•Then a teacher led review of the answer

•Followed with peer scoring using the actual scoring rubric and class discussion

•Final scoring by teacher after class


But wait…There’s more about the fruit fly!

•Part (d) requires you to perform a Chi-Square test, but you likely do not know what that is…

•Lets walk through an introduction to this important statistical tool, then we’ll get back to answering the fruit fly free response question.

Chi-Square TestAP Biology

Chapter 15

Sources of Error

Every controlled experiment must consider the errors that can be made.

•Scientific instruments are not perfect; there will always be that “margin of error.”

•Random events happen

•Are the samples chosen large enough and/or actually representative of all the subjects?

Statistical tools exist to help take into account the error.

Chi-Square Test

Chi-square test is one of those tools• Should there be a difference between

observed results and predicted or expected results, Chi-square allows investigator to judge the influence of simple, random chance.

• If Chi-square indicates that the difference is likely not due to chance, then the investigator will know to look for other influencing factors.

Chi-Square Test

Example:

Random chance is a major determinant in the inheritance of alleles (different versions of the same gene)

–Which alleles go to which gamete?–Which gamete combines with which

gamete?–Could other factors or errors be influencing

results?

Chi-Square Test

Biologists need a statistical tool which helps judge the probability that random chance influenced an outcome, such as a genetic cross.

We begin with understanding that…

•Hypothesis leads to certain prediction of data.

•Experiment results in observed data.

Chi-Square Test

The following equation calculates the likelihood that the difference is due to chance alone

Observed results

•What you can observe during the course of an experiment; Data you collected

Expected results

•What you expect to see based on your hypothesis (predictions)

Chi-Square Test

Expected results

•Can typically be determined from simple rules of probability.

Plug into the formula the expected and observed result, determining the Chi-Square value X2

Now let’s check our understanding for a moment…

Chi-Square Test

Remember what the Chi-Square test is:

•A statistical test that compares data collected in an experiment (observed) to data that was predicted by the hypothesis and expected as results.

If any difference does exist between the observed and expected results, it could simply be due to random chance.

We call outcome the null hypothesis.

Chi-Square Test

Null hypothesis (H0).

•Any difference between the observed results and the expected results is not significant and due to random chance alone.

•If the Chi-Square test leads us to accept the null hypothesis, then the difference is due to random chance.

•If the Chi-Square test leads us to reject the null hypothesis, then the difference is due to another factor and not chance.

Chi-Square Test

Let’s say we have a calculated Chi-Square value X2

What’s next?

We need to decide an acceptable probability that the observed results, and thereby the Chi-Square value X2, are caused by chance alone.

We need to know two things

•Degrees of Freedom

•Probability value

Chi-Square Test

Chi-Square Distribution Table

The degrees of freedom (df) and probability (p) values are seen in this table.

The Chi-Square values are seen in the boxes representing a cross between df and p.

Chi-Square Test


Degrees of Freedom (left column) is determined by one less than the number of possible outcomes in the experiment (df = n – 1). For example, in a coin toss, there are two possible outcomes – heads or tails. Therefore, the df is one (1).

Chi-Square Test


Notice the grey-shaded column. Biologists usually use a probability (p) value of 0.05., which sets the threshold of significance. In other words, it is the threshold for rejecting or accepting the null hypothesis.

Chi-Square Test


Another way of saying it is this: p=0.05 is setting a hypothesis that the difference between observed data and expected data is entirely due to chance.

Chi-Square Test


If the Chi-Square value falls within a column with a higher probability value than the 0.05 value, we can say the hypothesis could be correct, at least from a statistical standpoint.

Chi-Square Test


If the Chi-Square value falls within a column with an equal or lower probability value than the 0.05 value – the 0.01 probability column, then you cannot accept the null hypothesis – something other than random chance is at work.

Chi-Square Test

Figure B12 The Chi-Square Goodness-of-Fit Test

Chi-Square Test

Now let’s complete part (d) of the fruit fly free-response question

Summation

Following Chapter 8, Inheritance, Genes, and Chromosomes, their will be a lab which examines Chi-Square testing once more.

As you can see, Chi-Square does pop up in the AP Exam.

Summation

Figure B11 The t-test

the nature of science design of controlled experiments

Documents

hillis slide

anthropomorphism slide

statistics primer slide

inferential statistics

recorded knowledge slide

scientific observation

chisquare lab slide

scientific inquiry