Evaluating DataPurpose of Experiments

Evaluating ExperimentsIs to test theory?

Let’s do it!!!

Evaluating DataPurpose of Experiments

Evaluating Experiments

U s e e x p e r i m e n t a l u n c e r t a i n t i e s a n d t h e p r o p e r t i e s o f p r o b a b i l i t y d i s t r i b u t i o n s t o e v a l u a t e t h e e x t e n t t o w h i c h d a t a s u p p o r t s o r r e f u t e s a p h y s ic a l m o d e l .

2006 LEP Constraints

Or is to win Nobel Prizes forTheorists?

July 4 2012 discovery announced

Evaluating Data

Purpose of Experiments

Evaluating Data

Purpose of ExperimentsPeter Higgs2013 Nobel Prize Winner

Austin Ball –Technical Director of CMS (He was in charge of building the CMS experiment)

Pu r p o s e o f Ex p e r im e n ts● Pur p o s e o f e x p e r im e n t :

t e s t e x is t in g m o d e l

“ M o d e l ” ( W e b s te r ) – a s y s t e m o f p o s tu la t e s , d a t a , a n d in f e r e n c e s p re se n t e d a s a m a t h e m a t ic a l d e sc r ip t io n o f a n e n t i t y o r s ta t e o f a f f a i r s

i.e. a f u n c t io n a l r e la t io n s h ip b e t w e e n v a r ia b le s

●

●

5

Standard Model of Particle Physics

https://en.wikipedia.org/wiki/Standard_Model_%28mathematical_formulation%29

CERN Experiments

http://home.web.cern.ch/about/experiments

U.S. HEP Experiments

http://www-sld.slac.stanford.edu/sldwww/sld.html

http://www-cdf.fnal.gov/about/index.html

Standard Model of Particle Physics

Electron-positron cross-section and Z-boson Lineshape

http://rsta.royalsocietypublishing.org/content/370/1961/805

W h a t c a n e x p e r im e n ts a c h i e v e?

● A n e x p e r im e n t c a n n o t p r o v e a m o d e l t o b e t r u e .

A n e x p e r im e n t c a n :– D e m o n s t r a te th a t a

m o d e l is f a ls e w i t h s o m e p r o b a b i l i t y

– Sh o w th a t a m o d e l is v a l id w ith in so m e p re c is ion

●

W h a t c a n e x p e r im e n ts a c h i e v e?

● A n e x p e r im e n t im p r o v e s th e s t r e n g th o f i t s s t a t e m e n ts b y d e c re a s in g u n c e r t a i n t ie s .

H o w d o I q u a n t i f y t h e e x t e n t to w h i c h th e d a t a to t h e r i g h t is in c o n s is t e n t w i t h O h m ' s l a w ?

●

13

E xp e r i m e n ta l E va lu a t i o n

● D e t e r m i n e s ig n i f i c a n c e o f y o u r m e a su r e m e n t .

W h a t d o e s t h e e x p e r im e n t sa y a b o u t t h e m o d e l y o u ' r e t e s t i n g ?

– D o e s i t su p p o r t o r r e f u te th e m o d e l?– Ca n w e q u a n t i f y o u r c o n f id e n c e in t h e

m o d e l?

●

G a u s s ia n D i s t r i b u t i o n

14

G a u s s ia n D i s t r i b u t i o n

A “ 1 s i g m a ” e r r o r in te r v a l is n o t a l l - in c lu s i v e !

15

9

U s e “ 2 /3 R u le ” a s a R o u g h G u id e

● Is d a t a c o n s is t e n t w i t h a Y a n d X h a v i n g a l in e a r r e l a t io n s h ip ?

10

U s e “ 2 /3 R u le ” a s a R o u g h G u id e

● Sp r e a d o f p o i n t s , e r r o r b a rs c o n s is t e n t w i t h n o r m a l e r r o r d is t r ib u t io n .

11

U s e “ 2 /3 R u le ” a s a R o u g h G u id e

● W h a t is t h e p r o b l e m h e re ?

12

U s e “ 2 /3 R u l e ” a s a R o u g h G u i d e

● W h a t is t h e p r o b l e m h e re ?

Ev a lu a t in g M o d e ls : 2

● D e f in i t i o n o f 2

● 2 h a s m e a n in g r e la t i v e t o t h e n u m b e r o f d e g r e e s o f f r e e d o m :

# D O F = ( # d a ta p o in t s ) -( # p a ra m e t e rs d e te r m in e d f ro m d a ta )

Ro u g h ly , 2 / # D O F ~ 1 f o r g o o d d a ta /m o d e l m a t c h

●

●

20

Ev a lu a t in g M o d e ls : 2

● D e f in i t i o n o f 2

● 2 h a s m e a n in g r e la t i v e t o t h e n u m b e r o f d e g r e e so f f r e e d o m :

# D O F = ( # d a ta p o in t s ) -( # p a ra m e t e rs d e te r m in e d f ro m d a ta )

Ro u g h ly , 2 / # D O F ~ 1 f o r g o o d d a ta /m o d e l m a t c h

●

●

“CONSTRAINTS”

21

15

2 is a m o r e q u a n t i t a t i v e m e t h o d :

16

2 is a m o r e q u a n t i t a t i v e m e t h o d :

What is #DOF?

17

2 is a m o r e q u a n t i t a t i v e m e t h o d :

● W h a t is t h e p r o b l e m h e re ?

18

2 is a m o r e q u a n t i t a t i v e m e t h o d :

● W h a t is t h e p r o b l e m h e re ?

Co u n t i n g D e g r e e s o f F r e e d o m● # D e g r e e s o f f r e e d o m i n

th is s t r a ig h t l in e f i t s h o u l d b e e a s y → w h a t is i t? ?

W h a t i f in te r c e p t w e r e a ls o to b e d e t e r m in e d f r o m th e d a t a ?

H o w m a n y c o n s t r a in ts i f I ' m f i t t in g a G a u ss ia n in a c o u n t in g e x p e r im e n t?

. . . i f I ' m f i t t i n g a G a u ss ia n in a n o n -c o u n t in g e x p e r im e n t?

●

●

●

27

p -v a l u e o r “ c h a n c e p r o b a b i l i t y ” : Th e p r o b a b i l i t y o f o b t a i n i n g a r e s u l t a t l e a s t a s e x t r e m e a s a g i v e n d a ta p o in t , a s su m i n g t h e d a t a p o i n t w a s t h e r e s u l t o f c h a n c e a lo n e

● A s t a t e m e n t a b o u t t h e p r o b a b i l i t y t h a t a n “ e f f e c t ” is r e a l ly a s t a t is t i c a l f lu c t u a t io n .

Re p l a c e s d is c re t e T r u e /Fa ls e (o r 1 /0 ) c h o ice w i t h a c o n t i n u u m r a n g i n g f r o m “ l ik e l y t r u e ” t o “ l ik e l y f a l s e ” .

T r u t h → “ Tr u t h in e s s ”

●

●

28

Ex a m p le : Su p p o s e t h e re a re 3 0 , 0 0 0 U n iv e r s it y o f U ta h s tu d e n t s , o f w h ic h 4 0 0 a re p e r m i t t e d t o c a r r y g u n s . If I 'm t e a c h in g a n a s t r o n o m y c la ss o f 1 2 0 s tu d e n t s , w h a t is t h e p ro b a b i l i t y t h a t o n e o r m o re is c a r r y in g a g u n ?

A n s w e r : T h e p r o b a b i l i t y d is t r i b u t i o n f o r t h e n u m b e r o f s tu d e n t s w it h w e a p o n s is

a p p r o x im a t e ly P o iss o n ia n , w it h a m e a n v a lu e o f ( 4 0 0 /3 0 , 0 0 0 ) x 1 2 0 = 1 .6

Th e p ro b a b i l i t y o f o b se r v in g o n e o r m o re in a P o iss o n d is t r ib u t i o n w it h m e a n 1 .6 is :

P ( > = 1 , 1 .6 ) = 1 – P ( 0 ,1 .6 ) = 1 – 0 .2 0 1 = 0 .7 9 9

P P

22

Fol lo w u p : Su p p o s e a m e t a l d e t e c t o r r e v e a ls t h a t th e r e a re 6 p e o p le in John B e l z ' s c la ss c a r r y in g g u n s. W h a t is th e p -v a lu e o f t h is o b se rv a t io n ? Sh o u ld JB b e w o r r ie d t h a t th e y ' re o u t to g e t h im ?

23

Fo l lo w u p : Su p p o s e a m e t a l d e t e c t o r r e v e a ls t h a t t h e r e a re 6 p e o p le in JB ' s c la ss c a r r y in g g u n s. W h a t is t h e p -v a lu e o f t h is o b se r v a t i o n ? Sh o u ld JB b e w o r r ie d t h a t t h e y ' re o u t to g e t h im ?

● A n s w e r : Th e p -v a l u e i s t h e s u m o f p r o b a b i l i t i e s f o r n = 6 , 7 , 8 , 9 . . .

● P-v a lu e P ( > = 6 , 1 . 6 )P

= 0 .0 0 6● Th is is a v e r y s m a l l

c h a n c e p ro b a b i l i t y . Per h a p s JB s h o u ld w e a r a b u l le t p r o o f v e s t !

31

Example: I f l ip a c o in 2 0 t im e s , 1 4 t im e s i t c o m e s u p h e a d s . U s e t h is d a ta a s a t e s t o f t h e h y p o th e s is t h a t t h e c o in is f a i r.

32

Example: I f l ip a c o in 2 0 t im e s , 1 4 t im e s i t c o m e s u p h e a d s . U s e t h is d a ta a s a t e s t o f t h e h y p o th e s is t h a t t h e c o in is f a i r.

p = ( B in o m ia l P r ob a b i l i t y o f > = 1 4 h e a d s ) + ( B i n o m ia l P r ob a b i l i t y o f < = 6 h e a d s )

= 0 .1 1 5

33

Example: I f l ip a c o in 2 0 t im e s , 1 4 t im e s i t c o m e s u p h e a d s . U s e t h is d a ta a s a t e s t o f t h e h y p o th e s is t h a t t h e c o in is f a i r.

p = ( B in o m ia l P r ob a b i l i t y o f > = 1 4 h e a d s ) + ( B i n o m ia l P r ob a b i l i t y o f < = 6 h e a d s )

= 0 .1 1 5

W h ile t h e “ f a ir c o in ” h y p o t h e s is d o e s n o t h a v e a h ig h le v e l o f t r u t h in e ss ( p -v a lu e ) , i t s n o t so lo w t h a t w e c a n sa y f o r su r e t h a t t h e c o in isn ' t f a ir.

Th e 2 a n d n u m b e r o f d e g r e e s o f f r e e d o m

34

c a n b e d i r e c t ly t r a n s la t e d in t o a p -v a l u e :

So u r c e : p d g . lb l . g o v

http://pdg.lbl.gov/2014/reviews/rpp2014-rev-statistics.pdf

http://pdg.lbl.gov/2014/reviews/contents_sports.html

Ta b le f r o m p a g e 2 9 3 o f Ta y lo r

35

Ex a m p le : Taylor 1 2 . 0 4I t h ro w t h r e e d ic e t o g e t h e r a to t a l o f 4 0 0 t im e s , re c o r d t h e n u m b e r o f s ix e s in e a c h t h r o w , a n d o b ta in t h e r e su l t s sh o w n b e lo w . U se t h e b in o m ia l d is t r i b u t io n to f in d t h e e x p e c t e dn u m b e r E f o r e a c h o f t h e t h r e e b in s a n d t h e n c a lc u la te 2 .

k

D o I h a v e re a so n t o s u s p e c t t h e d ic e a r e lo a d e d ?

36

Taylor 1 2 . 0 4

● N = 3 t r ia ls

= 0 , 1 , 2 , 3

su c c e sse s p = 1 /6

●

●

37

Taylor 1 2 . 0 4

38

Taylor 1 2 . 0 4

W h a t is t h e n u m b e r o f d e g r e e s o f f r e e d o m ?39

Taylor 1 2 . 0 4

40

●

2 / d o f =

4 . 1 /2 p -v a lu e

~ 0 .1 4

“Consistent with a statistical fluctuation at the 10% level”

●

●

A d d i t i o n a l R e a d i n g a n d Pr o b l e m s

● Re a d in Ta y l o r :

41

– Ch 5 : T h e N o r m a l D i s t r i b u t i o n

– Ch a p t e r 1 2 : Th e Ch i - Sq u a r e d Te s t f o r a D i s t r i b u t i o n

● Tr y t h e p r o b l e m s :– 5 .1 1 , 5 .1 2 , 5 .2 0 ,

5 .2 1– 1 2 . 4 , 1 2 . 6 , 1 2 .1 1 ,

1 2 .1 4 , 1 2 .1 6

CERN Root Analysis Software Tutorials

• Start RDP session on orion, draco or Cygnus• Open an x-terminal window and issue the following commands

• The following should spew forth….

• Then run the tutorials benchmark by typing the following command

• And now your odyssey begins

CERN Root Tutorials

CERN Root Tutorials

CERN Root Tutorials

CERN Root Tutorials

CERN Root Tutorials

CERN Root Tutorials

CERN Root Tutorials

CERN Root Tutorials• Now let’s play….• From the root command line issue the following command

• The following should spew forth….

• Open another xterm window• Copy and edit the “program” (Macro) tutorials/fit/fit1.C to

~/phys6719/root/sandbox/myfit1.C by typing the following commands.

• Modify the program …

Exercises

1. Use the examples from the tutorials to write a CERN root macro to generate a Histogram containing events sampled from the sum of an exponential probability distribution.

2. Fit this histogram with an exponential function. Verify that the fit returns parameters consistent with the probability distribution that you used to generate your histogram. Report the c2 /D.O.F. goodness of fit parameter.

3. Repeat exercises 1 and 2 for a Gaussian and a Poisson Distribution.4. Document your results including graphs and written descriptions in

a libreoffice document.