1
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging is Everywhere
■ Applications of imaging technology are all around us
■ Different applications illustrate many different imaging technologies as well as different links in the imaging chain
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Sample Imaging Systems■Film photography■Digital photography■Television
● VCR● DVD
■Movies■Optical Imaging
● Microscopes● Telescopes● Eyeglasses● Contact Lenses
■“Softcopy” Displays● CRT● LCD● Plasma display
■Human eye■Thermal imagers■Passive IR sensors■Radar■Sonar
■Overhead projector■Slide projector■Holography■Copiers■Scanners■Printers
● inkjet● laser
■Printing ● lithography● screen● intaglio
■CT scan■MRI ■PET scan■Ultrasound Imaging■Laser surgery
■Aerial imaging■Airborne telescopes■Spaceborne telescopes■Satellite imaging■Electronic sensors
● CCD● CMOS
■Color cameras■Color displays■Computer vision■Robotic vision■Nightlight■Image compression
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging Systems
■Why are these all on the same list?
■Are there shared elements across these systems?
● What are they?
■PET scan■Film photography■Digital photography■Television■VCR■DVD■Microscopes■Telescopes■CRT■LCD■Plasma display■Thermal imagers■Passive IR sensors■Radar■Movies■Overhead projector■Slide projector■Holography■Copiers■Scanners■Printers (inkjet. laser)■CT scan
■Printing (lithography)■Printing (screen)■Printing (intaglio)■Aerial imaging■Astronomy ■Airborne telescopes■Orbiting telescopes■Satellite imaging■The eye■CCD■CMOS■Color cameras■Color displays■Computer vision■Robotic vision■Nightlight■Image compression■Human visual system ■Glasses■Ultrasound■Contact lenses■Laser surgery■…
2
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Three Branches of Imaging Science
■ Imaging Chain: study of imaging devices -- how they work and how to control them.
■ Imaging Applications: study and development of useful applications of imaging science.
■ Contributing Sciences: optics, electronics, mathematics, physics, chemistry, computer science, etc.
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
■ Imaging Chain: study of imaging devices -- how they work and how to control them.
■ Imaging Applications: study and development of useful applications of imaging science.
■ Contributing Sciences:
Three Branches of Imaging Science
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Contributing Sciences
■ Imaging is built on many scientific disciplines:●Mathematics● Physics, optics● Electrical Engineering (electronics)●Computer Science● Information Theory●And many others...
An Imaging Scientist must have a working knowledge of many disciplines (even if they don’t realize it!)
3
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
■ Imaging Chain: study of imaging devices -- how they work and how to control them.
■ Imaging Applications: study and development of useful applications of imaging science.
■ Contributing Sciences:
Three Branches of Imaging Science
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
The Imaging Chain
■ 8 Stages of Imaging1. SOURCE: sun, star, incandescent lamp, flashbulb2. OBJECT: flower, car, planet, bones inside the body3. PROPAGATION and COLLECTION: light to lens, mirror 4. DETECTION: film, CCD, thermometer
6. COMPRESSION, STORAGE, TRANSMISSION
8. PERCEPTION: Human Visual SystemImaging systems include at least two links of the imaging chain.
7. DISPLAY: softcopy (monitor), hardcopy (printout)
5. PROCESSING: chemical, electrical, digital
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Three Branches of Imaging Science
■ Imaging Chain: The study of imaging devices -- how they work and how to control them.
■ Imaging Applications: The study and development of useful applications of imaging science.
■ Contributing Sciences:
4
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging is Everywhere!
■ Applications of imaging technology can be seen almost everywhere
■ They illustrate many different imaging technologies and links on the imaging chain
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Some Examples of Images and Applications
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Astronomy: “Zooming in”
constellation of Orion(wide-field)
Orion Nebula – M42(thru Hubble Space Telescope)
5
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Gamma Ray
Astronomy: the whole spectrum
■ Images at different wavelengths (colors) are different!
X-ray
Visible
Infrared
Radio Waves
Milky Way viewed in visible light
■ Milky Way: the galaxy where our solar system is located.
Images from NASA
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Remote Sensing
■ Visible and infrared wavelengths show visual details as well as heat signatures of aircraft and other objects
Image from Digital Globe
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Remote Sensing - Environment
■ Satellites can monitor protected forests for illegal logging activities.
■ Images from space show depletion of ozone from the South Pole.
Images from Digital Globe and NASA
6
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Meteorology
■ Meteorologists use a variety of image data to predict weather patterns, as well as represent data as images
SatelliteSatellite--based visible based visible image showing cloud image showing cloud covercover
GroundGround--based based Doppler radar Doppler radar showing precipitationshowing precipitation
GroundGround--based based temperature data also temperature data also can be shown as an can be shown as an imageimage
Images from The WeatherChann
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Art and Ethnographic Objects: Analysis and Conservation
Christaina Miltor, Paintings Conservator, Winterthur Muesum, Winterthur, DE
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Jennifer L. Mass, Talk2, 36th SRC Users Meeting, 2003(www.chess.cornell.edu/Meetings)
“The Gordon Family”Evidence of previous restorationunderpainting & revisions
X-ray Analysis of Paintings
7
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Visible Light:Why the dark regions?
Near Infrared:Reveals an underpainting!
Jennifer L. Mass, Talk2, 36th SRC Users Meeting, 2003 (www.chess.cornell.edu/Meetings)
X-ray Analysis of Paintings
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Digital Imaging to Recover Writings from Historical Documents
Made legible by applying digital imaging techniques
Image from Chester F. Carlson Center for Imaging Science
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals Column 16 of Temple Scroll
8
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Text Recovered at Different Wavelengths
Visible Infrared (λ ≈ 800 nm)
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Visible IR (λ ≈ 800 nm)
Text Recovered at Different Wavelengths
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Text Recovery: Archimedes Palimpsest
48r © Owner of the Archimedes Palimpsest
9
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
X-ray Fluorescence Imaging of Manuscripts
■ Scan page through narrow beam (≈50μm) of tunable narrowband X rays
■ Measure intensity of energy spectrum of scattered radiation
■ Construct image(s) of different X rays
■ Used to read faded and obscured text
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Archimedes Page Mounted at SSRL
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
XRF Imaging of Unidentified Stub
Verso Side
“OU hOMOION PERI”
10
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Left Edge of Unidentified Stub
Verso Side
Diagram onVerso Side(white)
Characterson Recto(cyan)
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Forensic Imaging
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Medical Imaging
New technologies in medical imaging provide new and more useful data for better patient care
Computed Tomography Computed Tomography ((““CTCT””, , ““CAT scanCAT scan””))
Magnetic Resonance Magnetic Resonance Imaging (Imaging (““MRIMRI”” scan)scan)
Ultrasound ImagingUltrasound Imaging
Images from “The Visible Human” and “Siemens”
11
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Commercial/Personal Imaging
■Photography■Printing■Desktop publishing
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging in the News
500,000 fps high-speed motion analysis: .22 in aluminum, grenade, .22 edge on glass Popular Science July 2003
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging in the News■ ESA SMART-1 Spacecraft■ Functional MRI Scan shows comatose patients can think■ Functional MRI imaging of Economic Decision Making■ Body heat reveals burglar■ Laser Illumination for Handheld MRI Imaging■ New Method for Imaging Cartilage■ CT imaging unreliable for colorectal liver metastases■ NASA and USFS test wildfire imaging systems using
satellite-controlled UAVs■ Search for missing plane and pilot using imaging■ Functional Magnetic Resonance Imaging Guides Brain
Surgery Decisions
12
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging in (old) News
©Space Imaging June 2000
Ikonos Satellite by Space Imaging Launch 9/1999 (2nd attempt)
Altitude: 425 miles (≈50×height of aircraft)Ground speed: 15,000 mphGround resolution: ≈800mm
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging in the News
© Space Imaging June 2000
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging in the News
© Space Imaging Sept. 15, 2001
13
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging in (old) News
■ Satellite Imaging of Weather ●Only a dream until TIROS, 1960
(Television Infrared Observation Satellite Program)
First television picture from space, taken by the TIROS-I Satellite, April 1, 1960
Began continuous weather coverage in 1962
Compare resolution of TIROS image to imagesfrom modern weather satellites
© NASA
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging in the News
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging in the News
14
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Superdome, Before and After
©Digital Globe
8/31/20053/9/2004
©Digital Globe
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
NASA Spirit Rover
This panorama is was taken by NASA's Spirit rover upon reaching the summit of "Husband Hill," located in "Columbia Hills" in Gusev Crater, Mars. It reveals the vast landscape to the east previously hidden behind the Columbia Hills. The rim of "Thira Crater" frames the distant horizon some 15 kilometers away.The summit area is divided by a shallow saddle that slopes north (left) into anarea called "Tennessee Valley." Large amounts of sandy material have been blown up the valley and across the saddle in the left-to-right direction, creating the rippled piles of sand seen in this image. Mosaic was taken by Spirit's panoramic camera, using the blue filter of its right eye
© NASA
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Spirit on Mars
This "postcard" or 2-inch mini-panorama was taken by NASA's Spirit rover on Martian sol 582 (August 23, 2005), just as the rover finally completed its climb up “Husband Hill.” The summit appears to be a windswept plateau of scattered rocks, little sand dunes and small exposures of outcrop. The view is toward the north, looking down into the drifts and outcrops of the "Tennessee Valley," a region that Spirit was not able to visit during its climb to the top of the hill. Image credit: NASA/JPL-Caltech/Cornell
© NASA
15
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
What Has Changed?
Q: Why do images from “new”satellites show so much more than those from TIROS?
A: Imaging Science!● New Sensors● Digital Image Processing (new
computing capabilities)
■ (Also electrical engineering)● New Radio Transmitters/Receivers
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Recall that:
An Image is: a visual representation of some physical property of an object or phenomenon
Imaging Science is: ■ Imaging Chain: The study of imaging
devices -- how they work and how to control them.
■ Imaging Applications: The study and development of useful applications of imaging science.
■ Contributing Sciences:
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
16
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging Science Fundamentals
■ Imaging Chain
■ Contributing Sciences
■ Imaging Applications
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging Chain
1. SOURCE: sun, star, light bulb2. OBJECT: person, flower, car3. COLLECTION: lens, mirror 4. DETECTION: film, CCD
6. STORAGE: film, computer file
8. DISPLAY: printout, monitor7. TRANSMISSION: cable, radio signal, computer
5. PROCESSING: chem, electrical, digital
Group (II): Properties of Imagesand Imaging Devices
Group (I): Image Formation
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging Science FundamentalsCourse Content
Covered as needed within lectures■ Contributing Sciences
■ Imaging Applications
Group (I): Image Formation
Group (II): Properties of Images & Imaging Devices
Human VisionFamiliar Consumer Devices
Medical ApplicationsMuseum ImagingAstronomical Imaging
Color
■ Imaging Chain
17
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Sample Imaging Systems■Film photography■Digital photography■Television
● VCR● DVD
■Movies■Optical Imaging
● Microscopes● Telescopes● Eyeglasses● Contact Lenses
■“Softcopy” Displays● CRT● LCD● Plasma display
■Human eye■Thermal imagers■Passive IR sensors■Radar■Sonar
■Overhead projector■Slide projector■Holography■Copiers■Scanners■Printers
● inkjet● laser
■Printing ● lithography● screen● intaglio
■CT scan■MRI ■PET scan■Ultrasound Imaging■Laser surgery
■Aerial imaging■Airborne telescopes■Spacebornetelescopes■Satellite imaging■Electronic sensors
● CCD● CMOS
■Color cameras■Color displays■Computer vision■Robotic vision■Nightlight■Image compression
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging Systems
■Why are these all on the same list?
■What do they have in common?
■Are there shared elements across these systems?
■PET scan■Film photography■Digital photography■Television■VCR■DVD■Microscopes■Telescopes■CRT■LCD■Plasma display■Thermal imagers■Passive IR sensors■Radar■Movies■Overhead projector■Slide projector■Holography■Copiers■Scanners■Printers (inkjet. laser)■CT scan
■Printing (lithography)■Printing (screen)■Printing (intaglio)■Aerial imaging■Astronomy ■Airborne telescopes■Orbiting telescopes■Satellite imaging■The eye■CCD■CMOS■Color cameras■Color displays■Computer vision■Robotic vision■Nightlight■Image compression■Human visual system ■Glasses■Ultrasound■Contact lenses■Laser surgery■…
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
Imaging Systems: Shared Elements
1. Where did the energy come from?
2. What happens when the energy interacts with object?
3. How is the energy collected after that interaction?
4. How is the collected energy captured?
5. How is the signal manipulated?
6. What do you do with the information?
7. How is the information displayed?
8. How does the brain interpret that information?
■MRI■PET scan■Film photography■Digital photography■Television■VCR■DVD■Microscopes■Telescopes■CRT■LCD■Plasma display■Thermal imagers■Passive IR sensors■Radar■Movies■Overhead projector■Slide projector■Holography■Copiers■Scanners■Printers (inkjet)■Printers (laser)■Printing (lithography)■Printing (screen)■Printing (intaglio)■Aerial imaging■Astronomy ■Airborne telescopes■Orbiting telescopes■Satellite imaging■CCD■CMOS■Color cameras■Color displays■Computer vision■Robotic vision■Nightlight■Image compression■Human visual system ■Glasses■Ultrasound■Contact lenses■Laser surgery■…
18
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
1. Source of Energy
• What is the source of energy used by the imaging system?
• What are the characteristics of that source?
• What is the mechanism by which that form of energy is created?
“Where did the energy come from?”
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
2. Object
“What happens when the energy interacts with objects (matter)?”
• What happens when energy from the source strikes objects?
• What characteristics of an object affect that interaction?
• What information is contained in that interaction?
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
3. Propagation and Collection
“How does the energy get to the sensor?”
• Once the energy has interacted with an object, how do we collect that energy?
• What mechanisms are available to propagate the energy?
• What mechanisms are available to collect the energy
• What are the characteristics of those mechanisms?
• What effect do imperfections in the collection systems have on the overall system?
19
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
4. Detection
“How is the collected energy captured?”
• How is the energy captured or detected?
• What devices are used to transduce the captured energy into a signal that can be displayed and recorded?
• What are the mechanisms by which the energy is transduced?
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
5. Processing
“How is the signal manipulated?”
• Once the affected energy is transduced into a signal, how is it processed?
• What physical mechanisms and computational algorithms are used to process the signals?
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
6. Compression/Storage/Transmission
“What do you do with the information?”
• How do you deal with the huge amount of information in images?
• How can the information be compressed/ decompressed?
• How can the information be stored/recovered?
• How can the information be transmitted/received?
20
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
7. Display
“How is the information displayed?”• How can the data be visualized?
• What kind of displays are used in the system?
• Is the image to be viewed in print (“hardcopy”) or on a dynamic display (“softcopy”)?
• What are the characteristics of the display?
• Does the display limit the information?
© 2006 Carlson Center for Imaging Science / RITImaging Science Fundamentals
8. Perception
“How does the brain interpret that information?”
• How does the human visual system use the information on the display?
• What components and characteristics of the visual system influence perception?
• How does that perception influence the design of imaging systems?
© 2006 Carlson Center for Imaging Science / RIT
A useful structure to study imaging systems
1. Light source2. Object interaction3. Collection (often mirrors and lenses)4. Detector (film, digital sensor, etc.)5. Processing (chemical, digital, neural)6. Compression / storage / transmission7. Display8. Perception
Imaging Chain
Imaging Science Fundamentals
21
© 2006 Carlson Center for Imaging Science / RIT
1. Light source
λ [nm]
Imaging Chain
Imaging Science Fundamentals
© 2006 Carlson Center for Imaging Science / RIT
2. Object interactions
blue sky
sunset
Imaging Chain
Imaging Science Fundamentals
© 2006 Carlson Center for Imaging Science / RIT
3. Propagation & Collection:optics (lenses & mirrors)
Imaging Chain
Imaging Science Fundamentals
22
© 2006 Carlson Center for Imaging Science / RIT
4. Detector or Sensor
Photographic Emulsion Electronic Sensor (CCD)
Imaging Chain
Imaging Science Fundamentals
© 2006 Carlson Center for Imaging Science / RIT
5. Processing
Imaging Chain
Imaging Science Fundamentals
© 2006 Carlson Center for Imaging Science / RIT
6. Compression / Storage / Transmission
Imaging Chain
Imaging Science Fundamentals
23
© 2006 Carlson Center for Imaging Science / RIT
7. Display
Imaging Chain
Imaging Science Fundamentals
© 2006 Carlson Center for Imaging Science / RIT
8. Perception
Imaging Chain
Imaging Science Fundamentals