chapter 18:image perception and assessment · pdf file18.1 introduction (2 of 2) this chapter...
TRANSCRIPT
![Page 1: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/1.jpg)
IAEAInternational Atomic Energy Agency
Slide set of 110 slides based on the chapter authored by
I. Reiser
of the IAEA publication (ISBN 978-92-0-131010-1):
Diagnostic Radiology Physics:
A Handbook for Teachers and Students
Objective:
To provide the student with an introduction to human visual
perception and to task-based objective assessment of an
imaging system.
Chapter 18: Image Perception and
Assessment
Slide set prepared
by E. Berry (Leeds, UK and
The Open University in
London)
![Page 2: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/2.jpg)
IAEA
CHAPTER 18 TABLE OF CONTENTS
18.1. Introduction
18.2. The Human Visual System
18.3. Specifications of Observer Performance
18.4. Experimental Methodologies
18.5. Observer Models
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18. Slide 1 (02/110)
![Page 3: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/3.jpg)
IAEA
18.1 INTRODUCTION18.1
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.1 Slide 1 (03/110)
![Page 4: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/4.jpg)
IAEA
18.1 INTRODUCTION18.1
Introduction (1 of 2)
� The main purpose of a medical image is to provide
information to a human reader, such as a radiologist, so
that a diagnosis can be reached - rather than to display the
beauty of the human internal workings
� It is important to understand how the human visual system
affects the perception of contrast and spatial resolution of
structures that are present in the image
� If the image is not properly displayed, or the environment
is not appropriate, subtle clinical signs may go unnoticed,
which can potentially lead to a misdiagnosis
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.1 Slide 2 (04/110)
![Page 5: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/5.jpg)
IAEA
18.1 INTRODUCTION18.1
Introduction (2 of 2)
� This chapter provides an introduction to human visual perception and
to task-based objective assessment of an imaging system
� Model for the contrast sensitivity of the human visual system
• used to derive the gray-scale standard display function (GSDF) for
medical displays
� Task-based assessment measures the quality of an imaging system
• requires a measure of the ability of an observer to perform a well-defined
task, based on a set of images
• metrics for observer performance
• experimental methodologies for the measurement of human performance
• estimation of task performance based on mathematical observer models
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.1 Slide 3 (05/110)
![Page 6: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/6.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2 Slide 1 (06/110)
![Page 7: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/7.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2
The human visual system
� 18.2.1 The human eye
� 18.2.2 The Barten model
� 18.2.3 Perceptual linearization
� 18.2.4 Viewing conditions
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2 Slide 2 (07/110)
![Page 8: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/8.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.1 THE HUMAN EYE
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.1 Slide 1 (08/110)
![Page 9: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/9.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2
The human visual system
� 18.2.1 The human eye
� 18.2.2 The Barten model
� 18.2.3 Perceptual linearization
� 18.2.4 Viewing conditions
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.1 Slide 2 (09/110)
![Page 10: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/10.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.1 The Human Eye
Structure of the human eye
� The human eye is a complex organ that processes visual images and
relates information to the brain (Dragoi)
� The retina is the light-sensitive part of the eye
� There are two types of photosensitive cells in the retina
• rods and cones
• light quanta are converted into chemical energy, giving rise to an impulse
that is transferred via neurons to the optic nerve
� Each cone is connected to the optic nerve via a single neuron
• spatial acuity of cones is higher than that for rods
� Several rods are merged into one neuron that connects to the optic
nerve
• light sensitivity is greater for rods than for cones
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.1 Slide 3 (10/110)
![Page 11: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/11.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.1 The Human Eye
Day and night vision
� The highest spatial acuity of the human visual field is in the
fovea, a small region of the retina with a visual field of 2
degrees, where the density of cones is largest
� In scotopic (night) vision, luminance levels are low and
only rods respond to light
• rods are not colour sensitive; therefore objects appear grey at night
� In photopic (day) vision, both rods and cones are activated
• photopic vision occurs at luminance levels between 1 and 106
cd/m2.
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.1 Slide 4 (11/110)
![Page 12: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/12.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 THE BARTEN MODEL
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 1 (12/110)
![Page 13: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/13.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2
The human visual system
� 18.2.1 The human eye
� 18.2.2 The Barten model
� 18.2.3 Perceptual linearization
� 18.2.4 Viewing conditions
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 2 (013/110)
![Page 14: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/14.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Barten model
� A model for the contrast sensitivity of the human visual
system has been developed by Barten
� Contrast sensitivity (S) is inversely proportional to the
threshold modulation (mt)
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 3 (14/110)
![Page 15: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/15.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Threshold modulation
� mt is the ratio of the
minimum luminance
amplitude to the mean
luminance of a sinusoidal
grating such that the
grating has a 50%
probability to be detected
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 4 (15/110)
mean luminance
luminance amplitude
![Page 16: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/16.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Barten model and threshold modulation
� Contrast sensitivity of the human visual system can be
measured by presenting images of a sinusoidal grating to
a human observer
� From repeated trials, the threshold modulation can be
determined
� Barten’s model is based on the observation that in order to
be detected, the threshold modulation of a grating needs
to be larger than that of the internal noise mn by a factor k
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 5 (16/110)
nt kmm =
![Page 17: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/17.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Barten model and MTF
� As the image enters the eye, it is distorted and blurred by
the pupil and optical lens
� This is described by the optical MTF (Mopt) and is a
function of spatial frequency (u) and incident luminance (L)
• the width of Mopt depends on pupil diameter which controls the
impact of lens aberrations (Cab),
• the dependence of pupil diameter (d, in mm) on luminance (L, in
cd/m2) can be approximated by
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 6 (17/110)
( )22
0
2 )()( LdCL ab+=σσ
2))((2),( uL
opt eLuM πσ−=
)log4.0tanh(35)( LLd −=
![Page 18: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/18.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Barten model and photon noise
� When light quanta are detected by the retinal cells, photon noise (Fph)
is incurred with a spectral density given by
where
• η is the quantum detection efficiency of the eye
• p is the luminous flux to photon conversion factor
• E is the retinal illumination
� E(L) includes the Stiles-Crawford effect that accounts for variations in
efficiency as light rays enter the pupil at different locations
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 7 (18/110)
)(
1)(
LpELph η=Φ
})4.12/)(()7.9/)((1{4
)()( 42
2
LdLdLLd
LE +−=π
![Page 19: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/19.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Barten model – lateral inhibition and neural noise
� Lateral inhibition occurs in retinal cells surrounding an
excited cell, and is described as
� Neural noise (F0) further degrades the retinal signals
� These factors are included in threshold and noise
modulation, to give the Barten model modulation
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 8 (19/110)
20 )/(22 1)(
uu
lat euM−−=
![Page 20: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/20.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Barten model modulation
where
• X, Y, T are the spatial and temporal dimensions of the object
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 9 (20/110)
XYT
uMLkuMLuMm
latextph
latoptt
0
2 )())((2)(),(
Φ+Φ+Φ=
![Page 21: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/21.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Complete Barten model for contrast sensitivity of the human
eye
� The complete Barten model is given by
where
• X0 angular extent of the object
• Xmax maximum integration angle of the eye
• Nmax maximum number of cycles over which the eye can integrate
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 10 (21/110)
Φ+
Φ+
++
==
ext
lat
opt
t
uMLpEN
u
XXT
kLuM
LumLuS
)()(
1112
/),(
),(
1),(
2
0
2
max
2
2
max
2
0 η
![Page 22: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/22.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Contrast sensitivity as a function of luminance
Parameters used were k=3, s0=8.33x10-3 deg, Cab=1.33x10-3 deg/mm, T=0.1 s,
Xmax=12 deg, Nmax=15 cycles, h=0.03, F0=3x10-8 sec/deg2, u0=7 cycles/deg,
X0=2 deg, p=1x106 phot/sec/deg2/Td
The values for X0 and p correspond to the standard target used in DICOM14
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 11 (22/110)
![Page 23: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/23.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Contrast sensitivity as a function of spatial frequency
Parameters used were k=3, s0=8.33x10-3 deg, Cab=1.33x10-3 deg/mm, T=0.1 s,
Xmax=12 deg, Nmax=15 cycles, h=0.03, F0=3x10-8 sec/deg2, u0=7 cycles/deg,
X0=2 deg, p=1x106 phot/sec/deg2/Td
The values for X0 and p correspond to the standard target used in DICOM14
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 12 (23/110)
![Page 24: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/24.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Experimental verification
� The Barten model as presented here
• is valid for foveal vision in photopic conditions
• has been verified with a number of experiments covering the
luminance range from 0.0001 to 1000 cd/m2
� Of the parameters in the equation for S(u,L)
• s0, h, k were varied to fit measurements
• s0 varied between 0.45 and 1.1
• h from 0.005 to 0.3
• k from 2.7 to 5.0
• all others were kept fixed
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 13 (24/110)
![Page 25: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/25.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.2 The Barten Model
Extensions to Barten model
� Extensions of the model to parafoveal and temporal
contrast sensitivity have also been described (Barten
1999)
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.2 Slide 14 (25/110)
![Page 26: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/26.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.3 PERCEPTUAL LINEARIZATION
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.3 Slide 1 (26/110)
![Page 27: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/27.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2
The human visual system
� 18.2.1 The human eye
� 18.2.2 The Barten model
� 18.2.3 Perceptual linearization
� 18.2.4 Viewing conditions
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.3 Slide 2 (027/110)
![Page 28: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/28.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.3 Perceptual Linearization
Just Noticeable Difference
� Based on the contrast sensitivity S, a just noticeable
luminance difference (JND) can be defined as twice the
luminance amplitude (at) of a sine-wave grating at
threshold modulation
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.3 Slide 3 (28/110)
),(
2
),(2
2),(
LuS
L
LLum
aLuJND
t
t
=
=
=
![Page 29: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/29.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.3 Perceptual Linearization
Characteristic curve
� In a display device such as an LCD monitor, the
relationship between input gray value ng and output
luminance is non-linear
� The plot of ng versus luminance is the so-called
characteristic curve of the display device
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.3 Slide 4 (29/110)
![Page 30: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/30.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.3 Perceptual Linearization
Grayscale Standard Display Function (GSDF)
� In a perceptually linearized display, a constant difference of input gray
values ∆ng, results in a luminance difference corresponding to a fixed
number j of JND indices, across the entire luminance range of the
display
� To standardize medical devices, medical displays are required to
conform to the Grayscale Standard Display Function (GSDF)
(DICOM14)
� The GSDF defines the relationship between JND and display
luminance for the standard target
• a sinusoidal pattern of 4 cycles/deg over a 2 deg x 2 deg area
• embedded in a background with mean luminance equal to the mean
luminance of the pattern
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.3 Slide 5 (30/110)
JNDjL ×=∆
![Page 31: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/31.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.3 Perceptual Linearization
Grayscale Standard Display Function (GSDF)
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.3 Slide 6 (31/110)
![Page 32: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/32.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.4 VIEWING CONDITIONS
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.4 Slide 1 (32/110)
![Page 33: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/33.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2
The human visual system
� 18.2.1 The human eye
� 18.2.2 The Barten model
� 18.2.3 Perceptual linearization
� 18.2.4 Viewing conditions
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.4 Slide 2 (033/110)
![Page 34: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/34.jpg)
IAEA
18.2 THE HUMAN VISUAL SYSTEM18.2.4 Viewing conditions
Viewing conditions
� Ambient conditions in a reading room can significantly
impact observer performance
� Ambient lighting decreases performance and is
recommended to be kept below 50 lux in a reading room
� It is also recommended that observers allow time for dark
adaptation of their eyes prior to any reading, which takes
about 5 minutes
� Further, there are indications that fatigue deteriorates
performance
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.2.4 Slide 3 (34/110)
![Page 35: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/35.jpg)
IAEA
18.3 SPECIFICATIONS OF
OBSERVER PERFORMANCE18.3
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3 Slide 1 (35/110)
![Page 36: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/36.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3
Specifications of observer performance
� 18.3.1 Decision outcomes
� 18.3.2 Statistical decision theory and ROC
methodology
� 18.3.3 Signal-to-noise ratio
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3 Slide 2 (036/110)
![Page 37: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/37.jpg)
IAEA
18.3 SPECIFICATIONS OF
OBSERVER PERFORMANCE18.3.1 DECISION OUTCOMES
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 1 (37/110)
![Page 38: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/38.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3
Specifications of observer performance
� 18.3.1 Decision outcomes
� 18.3.2 Statistical decision theory and ROC
methodology
� 18.3.3 Signal-to-noise ratio
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 2 (038/110)
![Page 39: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/39.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.1 Decision outcomes
Binary decision
� A binary decision is exemplified by the task of detecting a
signal or abnormality
� A detection task is a binary decision because there are two
truth states
• the “normal” state or the absence of a signal (H0)
• the presence of an abnormality or a signal (H1)
� H0 is often called a negative or disease-free finding
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 3 (39/110)
![Page 40: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/40.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.1 Decision outcomes
Possible detection decision outcomes
� Presented with an image g of unknown truth state, the
observer must decide whether to categorize g as
• normal (D0) or
• as containing a signal (D1)
� There are four possible outcomes of this decision
• for an image that actually contains a signal
• a true positive (TP) occurs when it is assigned to H1
• a false negative (FN) occurs when it is assigned to H0
• for an image that does not contain a signal
• a true negative (TN) occurs when it is assigned to H0
• a false positive (FP) occurs when it is assigned to H1
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 4 (40/110)
![Page 41: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/41.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.1 Decision outcomes
Possible detection decision outcomes
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 5 (41/110)
Actual condition
Decision
H0: signal absent (negative)
H1: signal present (positive)
D0: signal absent (negative) TN FNType II error
D1: signal present (positive) FPType I error TP
![Page 42: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/42.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.1 Decision outcomes
Decision errors
� The observer can make two types of decision errors
� A type I error occurs when the truth state H0 is rejected
when it is actually true (FP)
� A type II error occurs when the truth state H0 is not
rejected when it is not true (FN)
� In medical applications, the cost associated with a
• type I error results in patient anxiety and societal costs because of
additional tests and procedures
• type II error is a missed cancer or misdiagnosis
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 6 (42/110)
![Page 43: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/43.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.1 Decision outcomes
Accuracy
� The accuracy of a medical procedure is given by the correct decisions
per total number of cases (N)
� Accuracy does not account for prevalence and can therefore be
misleading
• in screening mammography, the cancer prevalence in some screening
populations is about 4 per 1000 patients
• the accuracy of a radiologist who classifies all screening cases as normal
is 9996/10000 or 99.96%
• such a radiologist would have missed all cancers!
� More meaningful performance measures are sensitivity, specificity and
associated variables
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 7 (43/110)
N
TNTP )(accuracy
+=
![Page 44: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/44.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.1 Decision outcomes
Sensitivity
� Sensitivity, also known as the true positive fraction (TPF),
measures the proportion of actual true (positive) cases that
are correctly identified
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 8 (44/110)
( )FNTP
TPTPF
+=
![Page 45: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/45.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.1 Decision outcomes
Specificity
� Specificity measures the proportion of negative cases that
are correctly identified
� It is conveniently derived through the false positive fraction
(FPF) where
� and
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 9 (45/110)
( )FPTN
FPFPF
+=
FPF−=1yspecificit
![Page 46: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/46.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.1 Decision outcomes
PPV and NPV
� Often in the literature, positive predictive values (PPV) and
negative predictive values (NPV) are also reported
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.1 Slide 10 (46/110)
( )FPTP
TPPPV
+=
( )FNTN
TNNPV
+=
![Page 47: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/47.jpg)
IAEA
18.3 SPECIFICATIONS OF
OBSERVER PERFORMANCE18.3.2 STATISTICAL DECISION THEORY AND ROC
METHODOLOGY
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 1 (47/110)
![Page 48: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/48.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3
Specifications of observer performance
� 18.3.1 Decision outcomes
� 18.3.2 Statistical decision theory and ROC
methodology
� 18.3.3 Signal-to-noise ratio
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 2 (048/110)
![Page 49: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/49.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Decision theory
� In a probabilistic approach to perception, which is also
referred to as decision theory, the observer derives a
decision variable λ from each image
� The conditional probability density functions (PDFs) of the
decision variable λ under the truth states H0 and H1,
p(λ|H0) and p(λ|H1), are shown on the next slide
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 3 (49/110)
![Page 50: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/50.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Decision outcomes in the probabilistic decision paradigm
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 4 (50/110)
![Page 51: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/51.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Decision outcomes in the
probabilistic decision paradigm
� The operating point of the
observer, λc, is the decision
threshold based on which an
observer will call an image
“normal” (negative) or
“abnormal” (positive)
� Both the TPF and FPF values
depend on λc
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 5 (51/110)
![Page 52: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/52.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Binormal model
� Often it is assumed that both PDFs are normally
distributed so that a binormal model can be used
� If
• p(λ|H0) has zero mean and unit variance
• p(λ|H1) has a mean of a/b and variance 1/b
� The FPF and TPF are given by
where
• Φ(.) is the cumulative normal distribution function
• φ(.) is the normal PDF
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 6 (52/110)
)()( λλ −Φ=FPF )()( λλ baTPF −Φ=
∫ ∞− Φ=Φy
dxxy )()(21
)( xex −=π
φ
![Page 53: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/53.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Location of decision thresholds in binormal model
� Shows the probability density functions associated with H0
and H1
� Different λc describe the vigilance of the reader
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 7 (53/110)
![Page 54: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/54.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Vigilance of the reader
� The choice of operating threshold depends on
• the task
• the costs associated with decision errors
� A low decision threshold (λc = A) corresponds to an “aggressive”
reader
• characterised by a high sensitivity but a low specificity
• appropriate to a diagnostic task
� Higher decision thresholds (operating points B or C)
• when screening for a condition with low prevalence
• the reader typically operates at a high specificity (FPF < 0.1) in order to
reduce the number of callbacks to an acceptable level
• the associated decrease in sensitivity might be offset by the yearly repeat
of the screening procedure
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 8 (54/110)
![Page 55: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/55.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Receiver-operating characteristic (ROC) curve
� The receiver-operating characteristic (ROC) curve reveals
the trade-off between sensitivity and specificity
� This very useful and much-used curve is generated by
plotting the TPF versus the FPF for all values of the
decision threshold λ
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 9 (55/110)
![Page 56: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/56.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Receiver-operating characteristic
(ROC) curve
� The end points of the ROC curve
are (0,0) and (1,1)
� The ROC curve lies on or above
the diagonal
� If an ROC curve is below the
diagonal it implies that the truth
states H0 and H1 have been
interchanged, which can be
caused by an observer misreading
instructions
� If the PDFs have equal variance
under both truth states, the ROC
curve is symmetrical about a line
from (0,1) to (1,0)
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 10 (56/110)
� ROC curves for a = 2, 1.3, 0.4, 0;
b=1 in binormal model
� The underlying PDFs have equal
variance and therefore the ROC
curves are symmetrical
� The operating points A, B and C
correspond to the decision
thresholds shown on earlier slide
![Page 57: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/57.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Area under ROC curve (AUC)
� The area under the ROC curve, AUC, quantifies overall
decision performance and is the integral of TPF over FPF
� The values of AUC vary between 1.0 and 0.5
� A perfect decision maker achieves an AUC of 1.0, while
random guessing results in AUC of 0.5
� Within the binormal model, AUC is given by
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 11 (57/110)
∫=1
0)( dFPFFPFTPFAUC
+Φ=
21 b
aAUC
![Page 58: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/58.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Comparing two procedures
� AUC is an overall measure of decision performance that
does not provide any information about specific regions of
the ROC curve
• which may be important when deciding between two procedures
that are to be used in a specific task
� For example, consider two procedures are to be evaluated
for their performance in a screening task
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 12 (58/110)
![Page 59: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/59.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.2 Statistical decision theory and ROC methodology
Example: Comparing two
procedures for a screening task
� The AUC for both procedures is
0.82
� However, in the region of
specificity < 0.9 the
performance of procedure A
exceeds that of procedure B
� Specificity < 0.9 typically would
be a limit when screening for a
condition with low prevalence
such as breast cancer
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.2 Slide 13 (59/110)
� ROC curves for two procedures
with equal AUC, but different a
and b
![Page 60: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/60.jpg)
IAEA
18.3 SPECIFICATIONS OF
OBSERVER PERFORMANCE18.3.3 SIGNAL-TO-NOISE RATIO
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.3 Slide 1 (60/110)
![Page 61: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/61.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3
Specifications of observer performance
� 18.3.1 Decision outcomes
� 18.3.2 Statistical decision theory and ROC
methodology
� 18.3.3 Signal-to-noise ratio
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.3 Slide 2 (061/110)
![Page 62: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/62.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.3 Signal-to-noise ratio
SNR
� A frequently used measure of performance is the signal-to-noise ratio
(SNR), estimated from the decision variables λ0 and λ1 under the truth
states H0 and H1 through
� This expression is valid only if
• the decision variables are normally distributed
• the distributions of λ are fully characterized by mean and variance
� If the distributions depart from normality, SNR values computed by this
expression can be misleading
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.3 Slide 3 (62/110)
( ))var()var(21 10
01
λλλλ+
><−><=SNR
![Page 63: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/63.jpg)
IAEA
18.3 SPECIFICATIONS OF OBSERVER PERFORMANCE 18.3.3 Signal-to-noise ratio
SNR and AUC
� SNR is related to AUC through
• where Erf is the Gaussian error function
� For a detection task, SNR is often called detectability and
labelled d'
� d' may be estimated directly from the image statistics,
measured experimentally
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.3.3 Slide 4 (63/110)
( ))2/(12
1SNRErfAUC +=
∫ −=z
t dtezErf0
22)(
π
![Page 64: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/64.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES18.4
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4 Slide 1 (64/110)
![Page 65: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/65.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4
Experimental methodologies
� 18.4.1 Contrast-detail methodology
� 18.4.2 Forced choice experiments
� 18.4.3 ROC experiments
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4 Slide 2 (065/110)
![Page 66: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/66.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4
Quantifying image quality
� The preferred metric for quantifying image quality of a
medical imaging system is task performance
• measures the performance of an observer in a well-defined task
based on a set of images
� Image quality is thus a statistical concept
� Three methodologies to measure performance of a human
reader (observer) in the task of detecting a known signal in
a noisy background:
• 18.4.1 Contrast-detail methodology
• 18.4.2 Forced choice experiments
• 18.4.3 ROC experiments
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4 Slide 3 (66/110)
![Page 67: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/67.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES18.4.1 CONTRAST-DETAIL METHODOLOGY
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.1 Slide 1 (67/110)
![Page 68: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/68.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4
Experimental methodologies
� 18.4.1 Contrast-detail methodology
� 18.4.2 Forced choice experiments
� 18.4.3 ROC experiments
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.1 Slide 2 (068/110)
![Page 69: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/69.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.1 Contrast-detail methodology
Contrast-detail methodology
� To provide an objective measure of image quality, imaging
systems have been evaluated using contrast-detail
phantoms
� Consist of disks of increasing diameter and contrast
arranged along columns and rows
� The reader’s task is to identify the lowest contrast where
the signal can be perceived, for each detail size
� The reader’s threshold contrasts, plotted as a function of
disk radius, is called a contrast-detail curve
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.1 Slide 3 (69/110)
![Page 70: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/70.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.1 Contrast-detail methodology
Contrast-detail phantom
� The CDMAM phantom is an example of a contrast-detail phantom
� Consists of gold disks of exponentially increasing diameter (0.06 to 2
mm) and thicknesses (0.002 to 0.03 mm)
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.1 Slide 4 (70/110)
![Page 71: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/71.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.1 Contrast-detail methodology
Drawbacks of contrast-detail methodology
� Threshold contrast is dependent on the internal operating
point of each observer
• inter- and intra-observer variabilities can be quite high
� There may be a memory effect where the observer
anticipates and therefore reports a signal, while the signal
cannot yet be perceived
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.1 Slide 5 (71/110)
![Page 72: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/72.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.1 Contrast-detail methodology
Overcoming drawbacks of contrast-detail methodology
� The CDMAM phantom has been developed to address
some of these shortcomings (Bijkerk et al 2000)
� Places two signals
• one at the centre
• a second randomly in one of the four corners of each cell in the
array
� This arrangement prevents the reader from anticipating the
signal
� An alternative approach is to determine contrast
thresholds using computer reading of the images
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.1 Slide 6 (72/110)
![Page 73: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/73.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES18.4.2 FORCED CHOICE EXPERIMENTS
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.2 Slide 1 (73/110)
![Page 74: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/74.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4
Experimental methodologies
� 18.4.1 Contrast-detail methodology
� 18.4.2 Forced choice experiments
� 18.4.3 ROC experiments
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.2 Slide 2 (074/110)
![Page 75: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/75.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.2 Forced choice experiments
Alternative-forced-choice (AFC) experiments
� A different way of measuring human observer performance
is through alternative-forced-choice (AFC) experiments
� In an AFC experiment, M alternative images are shown to
the observer, whose task is to identify the image that
contains the signal
� The proportion of correct answers (PC) in an M-AFC
experiment is computed through
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.2 Slide 3 (75/110)
trialsofnumber total
trialsscoredcorrectly ofnumber =PC
![Page 76: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/76.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.2 Forced choice experiments
Number of trials required
� The value of PC varies between 1 and the guessing score
1/M
� The number of trials (that is, the number of test images
shown) determines the accuracy of PC
� Typically, more than 100 trials are required
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.2 Slide 4 (76/110)
![Page 77: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/77.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.2 Forced choice experiments
PC in terms of PDF
� PC can be predicted based on the assumption that the observer will
assign the score to the image which evokes the highest internal
response
� If the probability density function of negative responses is normal with
zero mean and unit variance
� Then the probability that the response of the M-1 actually negative
responses exceed that of the actually positive response (normally
distributed with mean d’ and unit variance) becomes
• where φ(.) and Φ(.) are the normal PDF and cumulative normal
distribution function as previously defined
• Eckstein & Whiting, 1996Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.2 Slide 5 (77/110)
dxdxxPC M )()( 1 ′−Φ= ∫∞
∞−
− φ
![Page 78: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/78.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.2 Forced choice experiments
PC and AUC
� Using
� And
� It can be shown that when M=2
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.2 Slide 6 (78/110)
)()( λλ −Φ=FPF
)()( λλ baTPF −Φ=
∫=1
0)( dFPFFPFTPFAUC
dxdxxPC M )()( 1 ′−Φ= ∫∞
∞−
− φ
AUCPC =
![Page 79: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/79.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.2 Forced choice experiments
2-AFC or M-AFC experiments
� Typically 2-AFC experiments are carried out so that the
observers score 90% of all trials correctly
� Depending on the signal, the corresponding amplitude
thresholds may need to be very low, which can strain the
observer
� By increasing the number of alternative locations (M), the
task becomes more difficult and larger threshold
amplitudes are required
� Experiments with M between 2 and 1800 have been
conducted (Burgess & Ghandeharian 1984)
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.2 Slide 7 (79/110)
![Page 80: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/80.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.2 Forced choice experiments
M-AFC experiments
� The alternative signal locations
• can be centres of individual ROIs
• or all possible signal locations can be contained within one single
test image
� When a single test image is used
• must ensure that the distance between locations is larger than the
correlation distance of the background
• otherwise observer responses might become correlated
� If the responses are uncorrelated, d’ can be determined
from PC by inverting
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.2 Slide 8 (80/110)
dxdxxPC M )()( 1 ′−Φ= ∫∞
∞−
− φ
![Page 81: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/81.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES18.4.3 ROC EXPERIMENTS
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.3 Slide 1 (81/110)
![Page 82: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/82.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4
Experimental methodologies
� 18.4.1 Contrast-detail methodology
� 18.4.2 Forced choice experiments
� 18.4.3 ROC experiments
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.3 Slide 2 (082/110)
![Page 83: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/83.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.3 ROC experiments
ROC experiments
� In an ROC experiment, a single image is presented to the
observer whose task is to provide a likelihood rating
� In a detection experiment, this could be the “likelihood of
signal present”
� Researchers have used
• continuous rating scales
• categorical scales
• typically fewer than 10 categories are provided
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.3 Slide 3 (83/110)
![Page 84: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/84.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.3 ROC experiments
ROC vs. 2-AFC experiments
� For a given number of images available, ROC experiments
provide better statistical power because
• one single image is required per trial
• in a 2-AFC experiment, two images are required per trial
� Furthermore, a ROC curve can be generated from rating
data, which provides more detailed information on task
performance in specific regions
� On the other hand, ROC experiments are more demanding
of the observer, which can increase reading time and
observer fatigue
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.3 Slide 4 (84/110)
![Page 85: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/85.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.3 ROC experiments
ROC curve fitting
� Curve fitting problems, such as the presence of hooks, can
be overcome by using ROC curve fitting software provided
by the University of Chicago
• http://www-
radiology.uchicago.edu/krl/KRL_ROC/software_index6.htm
� Their curve fitting model does not allow for the ROC curve
to fall below the diagonal.
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.3 Slide 5 (85/110)
![Page 86: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/86.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.3 ROC experiments
Search experiments
� A drawback of ROC/2-AFC experiments is the lack of
signal localization
� In a clinically realistic task such as screening for cancer,
the radiologist is required to indicate locations of potential
lesions, in addition to his overall rating of the image
� To allow for more clinically realistic laboratory
experiments, several extensions to ROC have been
developed
• LROC
• FROC/AFROC/JAFROC
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.3 Slide 6 (86/110)
![Page 87: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/87.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.3 ROC experiments
Search experiments – LROC
� In the Location ROC (LROC) methodology, the observer is
required
• to indicate the location of a lesion and
• to provide a score
� In an LROC curve, the false positive fraction is plotted
along the x-axis, but on the y-axis, the true positive fraction
with correct signal location is plotted
� The upper right endpoint of the LROC curve is determined
by the proportion of correctly located signals
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.3 Slide 7 (87/110)
![Page 88: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/88.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.3 ROC experiments
Search experiments – FROC
� In the Free-Response ROC (FROC) experiments, the
observer
• indicates the location of a lesion, but
• provides no rating
� The FROC curve is the proportion of correctly detected
signals plotted as a function of the average number of
false positive detections per image
� FROC analysis is often used in the performance
assessment of computer-aided detection schemes
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.3 Slide 8 (88/110)
![Page 89: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/89.jpg)
IAEA
18.4 EXPERIMENTAL METHODOLOGIES 18.4.3 ROC experiments
Search experiments – AFROC and JAFROC
� In Alternative Free-Response ROC (AFROC)
methodology, the proportion of correctly detected signals
is plotted as a function of the probability that at least one
false positive per image is found
� In the AFROC methodology, a summary figure-of-merit
exists, A1J
� This is the probability that lesions are rated higher than
false positive marks in normal images
� JAFROC is a software package developed to estimate A1J
and statistical significance from human observer FROC
data (Chakraborty website)Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.4.3 Slide 9 (89/110)
![Page 90: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/90.jpg)
IAEA
18.5 OBSERVER MODELS18.5
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5 Slide 1 (90/110)
![Page 91: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/91.jpg)
IAEA
18.5 OBSERVER MODELS18.5
� Often it is impractical to determine task performance of
imaging systems from studies involving human observers,
simply because reader time is scarce and expensive,
especially for a trained reader such as a radiologist
� To address this issue, mathematical observer models have
been developed, either to predict human performance or to
estimate ideal observer performance
� The observer models described here assume that images
contain exactly known additive signals
• i.e. signals of known shape and amplitude that have been added to
a noisy background
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5 Slide 2 (91/110)
![Page 92: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/92.jpg)
IAEA
18.5 OBSERVER MODELS18.5.1 THE BAYESIAN IDEAL OBSERVER
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.1 Slide 1 (92/110)
![Page 93: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/93.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.1 The Bayesian ideal observer
The Bayesian ideal observer
� In a detection task, the observer is faced with the decision of assigning
the unknown data g to one of the truth states
� The Ideal Bayesian Observer’s decision strategy is to minimize the
average cost <C> of the decision
� With the observer’s decision outcomes D0 and D1 , the average cost is
where
• C00 and C11 are costs associated with correct decisions
• C01 is the cost of a false negative decision
• C10 is the cost of a false positive decision
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.1 Slide 2 (93/110)
)P(H )H|P(D C + )P(H )H|P(D C +
)P(H )H|P(D C + )P(H )H|P(D C = >C<
0011011001
1111100000
![Page 94: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/94.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.1 The Bayesian ideal observer
The Bayesian ideal observer's decision rule
� Minimizing average cost leads to the decision rule
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.1 Slide 3 (94/110)
0
11010
00100
0
11
choose
else
))]((1[
))((
)|(
)|()( if H choose
H
CCHP
CCHP
Hgp
Hgpg
−−−
≥=Λ
![Page 95: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/95.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.1 The Bayesian ideal observer
The Bayesian criterion
� This decision criterion is the so-called Bayes criterion
� The resulting minimum cost is the Bayes risk
� A test based on the ratio of probabilities is called a
likelihood ratio test
� It can be shown that the likelihood ratio test maximizes
sensitivity for a given false-positive fraction
� Therefore the ideal observer maximizes the AUC and
provides an upper limit on task performance
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.1 Slide 4 (95/110)
![Page 96: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/96.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.1 The Bayesian ideal observer
Log-likelihood ratio
� Knowledge of the probability density functions p(g|Hi) is
required to form the decision variable (also called test
statistic) Λ(g)
� Λ(g) is generally difficult to determine for realistic imaging
tasks
� When the underlying data statistics are normal, it is often
easier to work with the log-likelihood ratio, λ(g) = log(Λ(g))
� Since the logarithmic transformation is monotonic, it does
not affect the decision outcome or AUC
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.1 Slide 5 (96/110)
![Page 97: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/97.jpg)
IAEA
18.5 OBSERVER MODELS18.5.2 OBSERVER PERFORMANCE IN UNCORRELATED
GAUSSIAN NOISE
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.2 Slide 1 (97/110)
![Page 98: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/98.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.2 Observer performance in uncorrelated Gaussian noise
Observer performance in uncorrelated Gaussian noise
� If the image noise is
• zero-mean Gaussian distributed with variance σ2
• independent in each image pixel
� The ideal observer derives the decision variable λ by
convolving each image g with a template w
� The ideal observer template for detection of additive,
exactly known signals is the signal itself, w = s
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.2 Slide 2 (98/110)
gwt ⋅=λ
![Page 99: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/99.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.2 Observer performance in uncorrelated Gaussian noise
Estimation of d' (SNR / detectibility)
� From the test statistic λ, the SNR d' can be estimated
using
� If the image statistics are known, d' can be computed
directly using
where
• the signal energy for a signal containing N pixels is
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.2 Slide 3 (99/110)
SNR =< λ1 > − < λ0 >
1 2 var(λ0) + var(λ1)( )
2
2'σSE
d =
∑ ==
N
i isSE1
2
![Page 100: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/100.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.2 Observer performance in uncorrelated Gaussian noise
Observer model for human performance in uncorrelated
Gaussian noise
� Compared to the ideal observer, a human reader performs
sub-optimally
� To account for the signal degradation by the human visual
system, the template w can be formed by convolving the
signal with an eye filter e
where
• e is defined in the spatial frequency domain as
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.2 Slide 4 (100/110)
cffefE −=)(
sew ⊗=
![Page 101: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/101.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.2 Observer performance in uncorrelated Gaussian noise
Non-prewhitening observer with eye filter (NPWE)
� Typically, c is chosen so that E(f) peaks at 4 cycles/mm
� This model observer model is called a non-prewhitening
observer with eye filter (NPWE)
� This observer model
• can predict the performance of a human reader when the image
background is uniform (Segui & Zhao 2006)
• fails when strong background correlations are present, such as in
anatomic backgrounds (Burgess 1999, Burgess et al 2001)
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.2 Slide 5 (101/110)
![Page 102: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/102.jpg)
IAEA
18.5 OBSERVER MODELS18.5.3 OBSERVER PERFORMANCE IN CORRELATED
GAUSSIAN NOISE
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.3 Slide 1 (102/110)
![Page 103: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/103.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.3 Observer performance in correlated Gaussian noise
Observer performance in correlated Gaussian noise
� When the noise in the image is Gaussian correlated, the
background statistics can be characterized by the
covariance matrix Kg
where
• <.> indicates the expectation value
� In this background type, the ideal observer pre-whitens the
image by using a template that removes the background
correlation
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.3 Slide 2 (103/110)
>><−><−=< t
g ggggK ))((
sKw g
1−=
![Page 104: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/104.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.3 Observer performance in correlated Gaussian noise
Estimation of d' (SNR / detectability)
� For ideal observer
� Estimating the background covariance Kg is often difficult
• but when the image data are stationary, the covariance matrix is
diagonalized by the Fourier transform
• allows computation of d’ using
�
where
• is the Fourier transform of the signal s
• is the background power spectrum
� This equation is the basis of estimation of SNR from noise-equivalent
quanta in chapter 4 (image quality)
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.3 Slide 3 (104/110)
sKsd g
t 12 −=′
fdfW
fSd
2
2
)(
)(∫=′
)( fS
)( fW
![Page 105: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/105.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.3 Observer performance in correlated Gaussian noise
Channel functions
� Another approach to estimating the background
covariance is to expand the image into so-called channels
or basis functions
� The choice of channel functions depends on the objective
of the observer model
� It has been found that
• Laguerre-Gauss are efficient channel functions when
approximating the ideal observer
• whereas Gabor channels provide a good estimate of human
observer performance
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.3 Slide 4 (105/110)
![Page 106: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/106.jpg)
IAEA
18.4 OBSERVER MODELS 18.5.3 Observer performance in correlated Gaussian noise
Channelized Hotelling observers
� Often, 10 channel functions or less are sufficient to
represent g
• they reduce the dimensionality of the problem considerably,
without the assumption of background stationarity
� These observer models are often called channelized
Hotelling observers
� The University of Arizona provides software and tutorials
for assessing image quality by use of observer models
• http://www.radiology.arizona.edu/CGRI/IQ/index.html
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18.5.3 Slide 5 (106/110)
![Page 107: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/107.jpg)
IAEA
18. BIBLIOGRAPHY18.
Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18. Bibliography Slide 1 (107/110)
![Page 108: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/108.jpg)
IAEA Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18. Bibliography Slide 2 (108/110)
18. BIBLIOGRAPHY18.
� BARRETT H. H. and MYERS K., Foundations of Image Science, John Wiley &
Sons, Hoboken, New Jersey (2004)
� BARTEN, P.G., Contrast sensitivity of the human eye and its effects on image
quality, SPIE press, Bellingham, WA (1999)
� BIJKERK, K.R., THIJSSEN, M.A.O., ARNOLDUSSEN, T.J.M., "Modification of
the CDMAM contrast-detail phantom for image quality of Full Field Digital
Mammography systems", IWDM 2000 (Proc. Conf. Toronto, 2000), Medical
Physics Publishing, Madison, WI, 633-640
� BURGESS A. E. and GHANDEHARIAN H., Visual signal detection. II. Signal-
location identification, J. Opt. Soc. Am A 1 (1984) 906-910
� BURGESS A. E., Visual signal detection with two-component noise: low-pass
spectrum effects, J. Opt. Soc. Am. A 16 (1999) 694-704
� BURGESS A. E., JACOBSON F. L, JUDY P. F., Human observer detection
experiments with mammograms and power-law noise, Med. Phys 28 (2001),
419
![Page 109: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/109.jpg)
IAEA Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18. Bibliography Slide 3 (109/110)
18. BIBLIOGRAPHY18.
� CHAKRABORTY, D., FROC web site
http://www.devchakraborty.com/index.php
� DICOM STANDARD, Digital Imaging and Communications in Medicine
(DICOM), Part 14: Grayscale Standard Display Function. National Electrical
Manufacturers Association, http://medical.nema.org
� DRAGOI, V., Chapter 14: Visual Processing: Eye and Retina in Neuroscience
online. Department of Neurobiology and Anatomy, The UT Medical School at
Houston http://neuroscience.uth.tmc.edu/s2/chapter14.html
� ECKSTEIN M. P. and WHITING J. S., Visual signal detection in structured
backgrounds. I. Effect of number of possible locations and spatial contrast, J.
Opt. Soc. A 13 (1996) 1777-1787
![Page 110: Chapter 18:Image Perception and Assessment · PDF file18.1 Introduction (2 of 2) This chapter provides an introduction to human visual perception and to task-based objective assessmentof](https://reader033.vdocuments.site/reader033/viewer/2022052708/5a7273f77f8b9a9d538d8e00/html5/thumbnails/110.jpg)
IAEA Diagnostic Radiology Physics: A Handbook for Teachers and Students – 18. Bibliography Slide 4 (110/110)
18. BIBLIOGRAPHY18.
� INTERNATIONAL COMMISSION ON RADIATION UNITS AND
MEASUREMENTS, Receiver Operating Characteristic Analysis in Medical
Imaging, ICRU Rep. 79 Bethesda, MD (2008)
� SEGUI J. and ZHAO W., Amorphous selenium flat panel detectors for digital
mammography: Validation of a NPWE model observer with CDMAM observer
performance experiments, Med. Phys. 33 (2006), 3711-3722