research and technical studies specialty group...
TRANSCRIPT
Research and Technical Studies Specialty Group Postprints
Volume 5 2014
From the 42nd Annual Meeting San Francisco, CA
May 27 – June 1, 2014
The American Institute for Conservation of Historic and Artistic Works (AIC)
ISSN: 2167-9851
The Research and Technical Studies Specialty Group of the
American Institute for Conservation of Historic and Artistic Works
2013 - 2014 Officers
Chair Silvia Centano
Vice Chair/Program Chair Ainslie Harrison
Secretary/Treasurer Angela Elliott
Communications/Publications Officer Corina Rogge
Chair Emeritus Jason Church
Volume 5 2014
The American Institute for Conservation of Historic and Artistic Works (AIC)
Research and Technical Studies Specialty Group Postprints
PowerPoint™ presentations from the 42nd Annual Meeting of the American Institute for Conservation of Historic and Artistic Works
in San Francisco, CA, May 27 – June 1, 2014
Compiler Julia Sybalsky
This document contains only the Postprint for the third paper from Cultural Heritage Imaging and Simon Fraser University. This
scaled-down version is a smaller file size and supports the RTI audience. The full Volume 5 Postprints with all 9
presentations can be found here:http://www.conservation-us.org/publications-resources/specialty-
group/research-and-technical-studies#.VkQanGAntjo
Volume 5 Online publication copyright © 2014. The Research and Technical Studies Group (RATS) and the American Institute for Conservation of Historic and Artistic Works (AIC).
ISSN: 2167-9851
The manuscripts and presentations in this publication have been edited for grammar yet have not undergone a formal process of peer review. The contents of this volume were originally given as oral presentations at the Annual Meeting of the American Institute for Conservation of Historic and Artistic Works (AIC). The materials, methods, results, and conclusions described herein are the opinion of the contributors and do not reflect the policies or opinions of AIC or RATS. All rights reserved by individual authors.
This online publication is intended for all members of AIC. Copies of this publication are available only in electronic format and may be downloaded.
The American Institute for Conservation of Historic and Artistic Works 1156 15th Street, NW, Suite 320, Washington, DC 20005-1714
www.conservation-us.org
Carla%Schroer1,%Mark%Drew2,%Mark%Mudge3,%Mingjing%Zhang4%%
1%Founder%&%Director,%Cultural%Heritage%Imaging%2%Professor%of%CompuEng%Science,%Simon%Fraser%University%
3%Director,%Cultural%Heritage%Imaging%4%Graduate%Student,%Simon%Fraser%University%
%%
Presented%at%the%RATS%session%42nd%Annual%MeeEng%of%
The%American%InsEtute%for%ConservaEon%of%Historic%and%ArEsEc%Works%(AIC)%San%Francisco,%CA,%May%27QJune%1,%2014%
%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
74
This%paper%presents%new%work%in%the%area%of%Reflectance%TransformaEon%Imaging%(RTI)%and%the%use%of%color%and%surface%normals%data.%It%includes%work%by%Cultural%Heritage%Imaging,%prepared%by%Mark%Mudge%and%Carla%Schroer,%and%research%from%Simon%Fraser%University%by%coQauthors,%Mark%Drew%and%Mingjing%Zhang.%%In%addiEon,%some%collaboraEve%work%from%Sema%Berkiten%and%Szymon%Rusinkiewicz%of%Princeton%University%is%included,%as%well%as%recently%published%work%from%the%Georgia%O’Keeffe%Museum%and%several%collaborators.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
75
This%paper%will%discuss%some%exciEng%new%work%with%Reflectance%TransformaEon%Imaging%(RTI)%and%related%techniques%that%use%the%same%types%of%captured%image%data%sets%as%RTI.%%It%will%start%with%some%background%on%the%technique%and%describe%what%surface%normals%are.%Then%it%will%present%an%RTI%example.%%The%bulk%of%the%paper%will%focus%on%addiEonal%uses%of%RTI%data%sets%starEng%with%examples%of%Algorithmic%Rendering%from%Princeton%University.%%Then%it%will%look%at%ways%in%which%RTI%data%can%be%used%to%track%changes%in%surfaces%over%Eme,%and%also%recent%research%to%improve%the%accuracy%of%the%calculated%surface%normals.%%Finally%it%will%take%a%brief%look%at%the%importance%of%evaluaEng%the%resulEng%images%for%quality,%reliability%and%reuse.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
76
All%of%this%work%takes%place%in%the%context%of%computaEonal%photography.%%What%I%mean%by%this%is%that%instead%of%taking%a%single%image,%you%take%a%sequence%of%images.%Then%using%computer%algorithms%you%extract%informaEon%from%that%set%of%images%to%create%a%new%type%of%image%(or%result)%not%possible%from%a%single%image.%%There%are%many%examples%of%computaEonal%photography,%and%this%is%a%field%that%has%much%acEve%research.%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
77
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
78
Let’s%begin%by%looking%at%an%example%of%an%RTI.%%This%example%is%a%Japanese%Woodblock%Print%from%the%mid%19th%century%in%the%collecEon%of%the%Fine%Arts%Museums%of%San%Francisco.%%%Note:%The%RTI%example%shown%in%the%presentaEon%at%AIC%has%been%replaced%in%this%postprint%due%to%publicaEon%restricEons.%In%addiEon,%a%live%demonstraEon%of%an%RTI%example%was%included.%Here%we%subsEtute%some%sEll%images%taken%from%an%RTI.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
79
With%an%RTI%we%can%dynamically%relight%the%subject%from%any%direcEon.%%We%have%collected%both%color%and%shape%informaEon,%so%we%can%look%at%just%the%surface%shape,%and%we%can%also%apply%mathemaEcal%enhancements%to%the%view.%%In%this%image%the%lower%led%shows%the%“default”%or%“normal”%view%from%the%RTIViewer%while%the%upper%right%shows%the%surface%shape%with%no%color,%and%with%the%surface%rendered%as%specular%(or%shiny)%so%that%we%can%see%the%fine%surface%details.%%Note:%A%live%demonstraEon%was%given%in%the%conference%presentaEon.%Here%we%subsEtute%an%image%created%from%2%views%within%the%RTI%viewer.%A%video%demonstraEng%details%from%this%example%and%showing%the%interacEve%relighEng%and%mathemaEcal%enhancements%in%acEon,%can%be%seen%here:%%hhp://vimeo.com/16942430%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
80
So,%here%are%the%basics%for%creaEng%an%RTI:%%We%have%a%camera%and%a%subject%in%fixed%posiEons.%Then%we%take%a%sequence%of%images%with%light%in%different%posiEons%around%the%subject.%We%have%to%know%where%the%light%is%in%each%image%in%order%to%calculate%the%RTI.%The%result%is%a%2D%image%that%carries%3D%informaEon.%%%This%is%based%on%work%by%Tom%Malzbender%and%Dan%Gelb%from%HP%Labs,%first%published%at%SIGGRAPH%in%2001.%hhp://www.hpl.hp.com/research/ptm/papers/ptm.pdf%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
81
There%are%2%primary%ways%to%collect%RTI%data.%First%I’m%going%to%talk%about%Highlight%RTI.%%In%2006,%Cultural%Heritage%Imaging%and%Tom%Malzbender%coQinvented%Highlight%RTI.%%Highlight%RTI%can%determine%the%light%posiEons%ader%the%capture%session.%%We%place%shiny%black%spheres%in%the%photo%scene.%%The%light%makes%a%sharp%highlight%on%the%sphere’s%surface.%%The%locaEon%of%the%highlight%on%the%sphere’s%surface%indicates%the%posiEon%of%the%light%source,%which%can%be%calculated%in%sodware%ader%the%images%are%taken.%%The%invenEon%of%Highlight%RTI%enabled%broad%adopEon%of%the%technique%because%the%cost%to%capture%RTI%data%was%small,%and%the%setup%is%flexible%for%a%wide%range%of%sizes%and%capture%environments.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
82
This%is%a%simple%Highlight%RTI%configuraEon.%%The%camera%is%poinEng%down%toward%the%subject.%%There%are%two%small%ball%bearings%in%the%field%of%view,%used%to%collect%the%light%posiEon%informaEon.%A%string%is%used%to%keep%the%distance%of%the%light%to%the%subject%constant.%%We%trigger%the%flash%unit%using%radio%triggers%on%the%camera%and%flash.%In%this%case%we%are%also%connected%to%a%computer%so%we%can%fire%the%camera,%download%the%images,%track%file%naming,%and%be%able%to%check%test%images%in%detail.%%A%tethered%computer%isn’t%required,%but%we%recommend%it%whenever%possible.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
83
The%light%posiEon%is%moved%and%another%shot%is%taken.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
84
Here%is%a%third%light%posiEon%for%this%image%set.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
85
We%can%do%this%for%verEcal%subjects%by%mounEng%the%spheres%and%placing%them%in%the%image.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
86
Here%is%another%verEcal%setup,%this%Eme%in%a%gallery.%%We%are%shooEng%a%detail%of%this%large%panel,%and%we%have%placed%the%flash%unit%on%a%longer%pole%so%we%can%get%it%all%the%way%around%the%area%we%are%imaging.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
87
Copy%stands,%found%in%many%museum%and%library%studios,%provide%a%great%environment%for%capturing%RTIs.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
88
RTIs%can%also%be%captured%using%a%light%array%or%dome.%%Prior%to%the%2006%invenEon%of%highlight%RTI,%domes%were%the%primary%method%of%capturing%RTI%data.%They%are%sEll%very%efficient%for%working%with%larger%collecEons%of%smaller%objects.%%This%is%the%most%recent%example%of%a%light%array%built%by%Cultural%Heritage%Imaging.%It%has%45%lights%and%can%image%subjects%up%to%12%inches%in%diameter.%%The%lights%have%a%radius%of%24%inches%from%the%center%of%the%subject.%The%light%array%has%many%features%for%flexibly%placing%the%camera%and%subject%in%the%environment.%%It%takes%2.5%minutes%to%capture%the%45Qimage%sequence.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
89
How%RTIs%work:%%1.%MathemaEcally,%the%direcEon%that%is%perpendicular%to%the%surface%at%any%given%locaEon%is%represented%by%a%vector%(direcEon)%called%a%normal.%%Technically%it%is%a%vector%that%is%perpendicular%to%the%tangent%plane%at%any%point%on%the%surface,%as%real%surfaces%are%in%3%dimensions%and%this%graphic%is%only%2Qdimensional.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
90
2.%Light%bounces%off%of%surfaces%such%that%the%incident%angle%of%the%light%and%the%reflected%angle%of%the%light%are%equal%angles%to%the%surface%normal.%%Since%the%camera%is%in%a%fixed%posiEon,%and%we%know%where%the%light%is%coming%from%in%each%image,%and%because%we%sample%from%a%variety%of%light%posiEons,%we%can%calculate%the%surface%normal%per%pixel%in%the%image.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
91
3.%The%mathemaEcal%descripEon%of%the%normal%is%saved%per%pixel,%along%with%the%RGB%(redQgreenQblue)%color%informaEon%of%a%regular%photograph.%This%ability%to%record%efficiently%the%color%and%true%3%dimensional%shape%informaEon%is%the%source%of%RTI's%documentary%power.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
92
Sodware%to%create%and%work%with%RTIs%is%free%and%open%source.%%Also%available%are%a%free%capture%guide%and%instrucEonal%videos.%%All%these%are%accessible%on%the%Cultural%Heritage%Imaging’s%website.%hhp://culturalheritageimaging.org/downloads%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
93
It%is%also%possible%to%combine%the%RTI%technique%with%UV%and%IR%light%spectrums,%as%well%as%to%do%RTI%under%the%microscope.%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
94
We%have%seen%significant%adopEon%of%RTI%in%museums%in%recent%years,%largely%driven%by%museum%conservaEon.%Here%are%just%a%few%of%the%museums%that%are%acEvely%using%RTI%today.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
95
A%recently%completed%grant%from%the%InsEtute%of%Museum%and%Library%Services%was%an%important%aid%to%RTI%adopEon.%The%grant%included%funding%for%10%4Qday%training%sessions.%%CHI%was%able%to%train%150%parEcipants%represenEng%45%different%insEtuEons.%This%included%all%of%the%ANAGPIC%graduate%programs%in%art%conservaEon%in%North%America%as%well%as%4%regionally%hosted%training%sessions%where%museum%professionals%could%parEcipate%without%a%training%fee.%ParEcipants%were%chosen%through%an%applicaEon%process.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
96
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
97
Let’s%take%a%quick%look%at%a%technique%called%Algorithmic%Rendering%(AR).%%This%uses%the%same%sets%of%images%that%are%collected%to%create%RTIs.%%It%is%based%on%work%from%Szymon%Rusinkiewicz%(and%team)%from%Princeton%University.%%The%approach%allows%you%to%apply%various%signal%processing%algorithms%to%the%color%and%shape%informaEon%collected%in%an%RTI%image%set%and%use%them%to%create%technical%drawings%that%allow%you%to%emphasize%certain%features%while%suppressing%other%features.%%This%paper%describes%much%of%the%early%work%of%Szymon%and%team%on%Algorithmic%Rendering:%%Corey%TolerQFranklin,%Adam%Finkelstein,%and%Szymon%Rusinkiewicz.%IllustraEon%of%Complex%RealQWorld%Objects%using%Images%with%Normals.%InternaEonal%Symposium%on%NonQPhotorealisEc%AnimaEon%and%Rendering%(NPAR),%August%2007.%hhp://gfx.cs.princeton.edu/pubs/TolerQFranklin_2007_IOC/index.php%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
98
There%is%a%range%of%styles%that%can%be%applied%to%the%same%data.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
99
Here%is%a%petroglyph%example%from%Legend%Rock%in%Wyoming.%In%the%lower%right%are%a%series%of%zoomorphs%that%are%very%difficult%to%see%in%a%convenEonal%photograph%(or%with%the%naked%eye).%However,%with%the%collecEon%of%color%and%surface%normal%data%contained%in%an%RTI%image%set…%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
100
…this%representaEon%can%be%produced%using%the%AR%technique,%which%clearly%shows%the%zoomorphs.%%Note%that%this%could%not%be%accomplished%with%a%single%photograph%because%you%only%have%color%data,%and%there%is%no%color%change%that%can%make%these%zoomorphs%visible.%%Because%we%also%have%shape%informaEon%(in%the%form%of%surface%normals),%this%allows%the%creaEon%of%this%representaEon.%%This%data%set%was%collected%by%Cultural%Heritage%Imaging%in%summer%2006,%and%the%AR%result%was%produced%by%Szymon%Rusinkiewicz.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
101
This%example%illustrates%the%idea%of%fusing%informaEon%from%mulEple%images%to%create%a%result%that%supplies%the%most%informaEonQrich%rendering,%not%found%in%any%one%image%by%itself.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
102
One%technique%that%can%be%applied%with%AR%is%exaggerated%shading.%This%is%used%to%bring%out%small%changes%in%the%surface%shape%and%make%them%more%visible.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
103
Here%is%an%example%from%a%petroglyph%in%the%Owens%Valley%near%Bishop%California.%%The%same%data%set%was%processed%as%an%RTI,%and%several%AR%examples%were%also%made.%%This%first%image%is%from%the%RTI%in%the%RTIViewer%default%viewing%mode.%%The%data%was%collected%in%May%2013%by%Cultural%Heritage%Imaging%and%we%appreciate%the%help%of%Greg%Haverstock%of%the%US%Bureau%of%Land%Management%and%rock%art%specialist%David%Lee.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
104
The%next%image%is%also%from%the%RTI,%this%Eme%with%the%mathemaEcal%enhancement,%specular%mode%and%no%color.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
105
The%last%example%from%the%RTI%shows%an%image%rendered%using%the%diffuse%gain%mode.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
106
Now%we%will%look%at%some%examples%where%we%apply%Algorithmic%Rendering%to%the%same%data%set.%%First,%just%color%with%no%shape.%The%color%is%“illuminaEon%independent.”%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
107
This%next%slide%shows%exaggerated%shading%with%very%lihle%color.%The%exaggerated%shading%is%applied%to%the%surface%normal%data%calculated%from%the%image%set.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
108
This%AR%example%shows%exaggerated%shading%applied%to%the%larger%(low%frequency)%details.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
109
This%final%example%shows%exaggerated%shading%which%emphasizes%very%small%surface%changes.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
110
CHI%is%working%with%the%team%at%Princeton%to%create%the%CARE%tool,%which%will%be%open%source.%We%have%a%NaEonal%Science%FoundaEon%Grant%that%supports%this%project.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
111
Now%I’d%like%to%talk%about%recent%work%to%use%surface%normals%to%track%changes%in%a%surface.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
112
This%work%was%presented%in%a%paper%at%the%Digital%Heritage%Conference%in%Marseilles%in%the%fall%of%2013.%%You%can%find%the%full%paper%here:%%hhp://www.gokmconservaEon.org/scienEficQmethods/rE/%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
113
The%graphics%shown%here%are%from%that%paper.%%The%first%step%was%to%look%at%the%repeatability%of%RTI%data.%This%was%done%by%imaging%the%same%subject%several%Emes,%moving%the%camera%slightly%between%each%image%capture,%then%aligning%the%image%sets%and%comparing%the%produced%results.%%What%the%team%found%was%that%the%Polynomial%Texture%Map%(PTM)%algorithm%is%not%very%repeatable,%whereas%the%Hemispherical%Harmonics%(HSH)%algorithm%is%very%repeatable.%Note%that%the%use%of%both%algorithms%is%supported%in%the%free%RTIBuilder%sodware.%%A%discussion%of%the%differences%between%using%the%PTM%and%HSH%algorithms%can%be%found%on%the%CHIForums%here:%hhp://forums.culturalheritageimaging.org/index.php?/topic/190QhshQorQptmQhowQtoQchooseQtheQbestQfiher/%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
114
Once%the%team%understood%the%variability%in%the%results,%they%created%a%test%subject,%a%drawing%they%imaged%with%RTI,%then%damaged,%then%imaged%again.%The%goal%was%to%determine%whether,%by%just%doing%a%mathemaEcal%comparison%of%the%surface%normal%data%produced%by%the%RTI,%the%changes%to%the%drawing%could%be%detected.%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
115
They%only%considered%pixels%that%were%offset%by%more%than%1.9%%to%be%changed.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
116
Next%they%were%able%to%create%a%map%of%the%changed%pixels,%which%clearly%shows%the%modificaEons%to%the%drawing.%%This%work%is%just%the%beginning,%and%more%needs%to%be%done%using%different%types%of%surfaces.%However,%this%is%very%promising%work%and%shows%how%this%kind%of%data%can%be%used%quanEtaEvely%to%detect%change.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
117
Because%we%are%looking%at%addiEonal%uses%of%this%color%and%surface%normal%data,%we%want%to%understand%how%we%might%improve%the%accuracy%of%surface%normals.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
118
In%order%to%understand%the%next%set%of%images%it%is%important%to%understand%how%to%view%a%normal%map.%%A%normal%map%is%a%false%color%visualizaEon%of%the%direcEon%of%the%surface%normals%for%a%surface.%%This%graphic%shows%the%surface%normal%map%of%a%sphere.%%Pixels%shown%as%pink%and%magenta%are%poinEng%to%the%right%and%down,%green%pixels%are%poinEng%up.%%VariaEons%in%the%shading%depict%changes%in%the%direcEon%of%the%surface%normals.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
119
This%is%a%detail%of%a%boomerang%from%Australia.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
120
Here%is%the%normal%map%for%this%same%detail%when%an%RTI%is%created%using%the%Hemispherical%Harmonics%or%HSH%algorithm.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
121
Here%is%the%normal%map%for%this%same%detail%when%calculated%using%the%Polynomial%Texture%Mapping%(PTM)%algorithm.%%You%can%see%that%there%is%a%significant%difference%in%the%result,%especially%where%the%edges%of%the%subject%are%in%the%image.%%This%is%due%to%limitaEons%in%the%PTM%approach%in%areas%that%have%significant%shadows%from%some%illuminaEon%angles.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
122
This%graphic%shows%the%difference%between%the%HSH%and%PTM%normal%maps.%%It%is%clear%from%the%normal%map%comparison%that%the%HSH%algorithm%is%calculaEng%more%accurate%surface%normals%for%the%edges%of%the%subject%when%compared%to%PTM.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
123
In%the%latest%version%of%the%RTiViewer%(1.1)%the%ability%to%look%at%normal%maps,%and%also%save%them%as%image%files,%is%now%available.%%There%are%other%new%features%in%this%update%as%well.%%You%can%download%it,%along%with%documentaEon%and%examples,%here:%%hhp://culturalheritageimaging.org/What_We_Offer/Downloads/View/index.html%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
124
Here%is%a%screen%shot%of%the%latest%version%of%RTIViewer,%which%includes%several%new%features.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
125
As%part%of%the%NaEonal%Science%FoundaEon%grant%project%with%the%Princeton%team,%PhD%student%Sema%Berkiten%and%Professor%Szymon%Rusinkiewicz%have%done%research%on%aligning%image%sets%of%the%type%captured%for%RTI%and%Algorithmic%Rendering.%%They%worked%with%image%sets%of%30Q100%images%where%the%camera%and%the%subject%are%in%a%fixed%posiEon,%and%the%light%is%moved%from%image%to%image.%%Even%though%the%camera%and%subject%are%fixed,%there%can%be%small%movements%due%to%unstable%floors,%wind%in%outdoor%environments,%slight%rolling%or%movements%of%the%subject,%and%so%on.%%%They%began%tesEng%different%algorithms%and%approaches%for%how%to%best%get%subQpixel%alignment%across%the%image%set.%%In%addiEon,%they%tested%the%impact%to%the%resulEng%calculated%normal%maps%before%and%ader%alignment.%%An%open%source%alignment%tool%for%RTI%and%AR%data%sets,%based%on%this%work,%will%be%released%in%the%future.%%A%paper%with%details%on%this%work%is%available:%%Berkiten,%S.%and%Rusinkiewicz,%S.%2014.%Alignment%of%Images%Captured%Under%Different%Light%DirecEons%(TRQ974Q14).%Princeton%University.%%dp://dp.cs.princeton.edu/techreports/2014/974.pdf%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
126
The%next%few%images%show%the%results%of%image%alignment%on%the%surface%normals.%%Here%is%a%surface%normal%map%for%the%petroglyph%from%the%Owens%Valley%in%California,%which%was%discussed%earlier.%%This%shows%the%surface%normals%created%from%images%that%were%not%aligned.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
127
Here%is%the%normal%map%ader%the%images%are%aligned.%%From%this%zoomed%out%view,%we%cannot%see%the%difference.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
128
When%we%zoom%in%and%compare%again,%the%difference%will%become%clear.%%This%is%from%the%unaligned%images.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
129
This%is%the%same%area%ader%alignment.%%You%can%see%that%the%alignment%has%noEceably%%improved%the%surface%normal%map.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
130
AddiEonal%research%is%underway%to%improve%the%accuracy%of%surface%normals,%as%well%as%the%color%and%specular%accuracy%calculated%from%RTI%image%sets.%%This%work%is%by%our%coQauthors,%Professor%Mark%S.%Drew%and%recent%Masters%graduate%Mingjing%Zhang%of%Simon%Fraser%University.%%As%was%parEally%demonstrated%earlier,%the%problem%is%that%the%Polynomial%Texture%Map%(PTM)%algorithm%does%not%properly%handle%highly%shadowed%areas%on%a%surface%nor%really%shiny%or%specular%areas.%This%problem%manifests%itself%in%two%ways.%First,%the%surface%normals%are%not%accurate%in%these%areas;%and%second,%the%appearance%in%an%RTI%viewing%environment%is%incorrect%when%compared%to%“ground%truth.”%%The%ground%truth%used%in%this%case%is%an%input%photograph%from%the%image%set%compared%to%the%view%of%the%subject%within%the%viewing%environment%with%the%light%in%the%same%posiEon%as%the%photograph.%%While%the%Hemispherical%Harmonics%(HSH)%algorithm%does%a%beher%job%than%the%PTM%algorithm,%it%sEll%has%inaccuracies.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
131
This%work%has%been%an%acEve%area%of%research%at%Simon%Fraser%University%since%2009.%%The%team%has%worked%with%mulEple%approaches%and%algorithms,%and%has%been%refining%their%approach%based%on%their%results.%%To%improve%surface%normal%accuracy,%they%are%separaEng%the%shadows%and%specularity%so%that%the%normals%are%calculated%only%on%the%mahe%(or%LamberEan)%surface.%%Then%they%apply%robust%regression%techniques%so%that%the%impact%of%outlier%data%is%reduced.%%The%outlier%data%in%this%case%is%caused%by%highlights%and%shadows.%The%robust%regression%approach%limits%the%impact%of%the%highlights%and%shadows%on%the%overall%calculaEon.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
132
To%improve%the%highlight%and%shadow%appearance%within%the%viewing%environment,%a%different%technique%is%applied.%%The%team%uses%Radial%Basis%FuncEons%to%‘weight’%the%impact%of%lighEng%on%the%rendered%image.%Thus,%the%final%rendered%result%is%more%heavily%influenced%by%the%sampled%light%that%is%closest%in%posiEon%to%the%light%direcEon%being%modeled%in%the%viewing%environment.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
133
There%are%several%published%papers%on%this%work.%%The%next%few%slides%will%include%the%references%for%some%of%them.%%"Robust%Luminance%and%ChromaEcity%for%Mahe%Regression%in%Polynomial%Texture%Mapping”,%Mingjing%Zhang%and%Mark%S.%Drew,%CPCV2012:%European%Conference%on%Computer%Vision%Workshop%on%Color%and%Photometry%in%Computer%Vision,%October%2012.%hhp://www.cs.sfu.ca/~mark/dp/Cpcv2012/cpcv2012a.pdf%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
134
Robust%EsEmaEon%of%Surface%ProperEes%and%InterpolaEon%of%Shadow/Specularity%Components",%Mark%S.%Drew,%Yacov%HelQOr,%Tom%Malzbender,%and%Nasim%Hajari,%Image%and%Vision%CompuEng,%vol.30%(no.%4Q5),%May%2012,%pp.317Q331.%hhp://www.cs.sfu.ca/~mark/dp/Ivc2012/ivc2012.pdf%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
135
“Robust%Surface%Normal%EsEmaEon%via%Greedy%Sparse%Regression”%Mingjing%Zhang,%Msc%Thesis,%School%of%CompuEng%Science,%Simon%Fraser%University,%2014%hhp://summit.sfu.ca/system/files/iritems1/13719/etd8213_MZhang.pdf%%"Efficient%Robust%Image%InterpolaEon%%and%Surface%ProperEes%using%Polynomial%Texture%Mapping”,%Mingjing%Zhang%and%Mark%S.%Drew,%EURASIP%Journal%on%Image%and%Video%Processing,%2014:25,%1%May%2014,.%hhp://www.cs.sfu.ca/~mark/dp/JIVP2014/jivp2014.pdf%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
136
Here%are%some%visual%examples%to%demonstrate%the%results.%%This%image%is%a%photograph%that%was%part%of%the%set%of%images%collected%to%make%an%RTI.%In%this%case%it%is%image%number%34.%%This%data%set%of%a%CelEc%Gold%Stater%was%collected%by%Cultural%Heritage%Imaging.%%All%processing%shown%in%these%slides%was%done%by%Mingjing%Zhang%and%Mark%Drew.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
137
This%is%the%interpolated%image%using%the%above%menEoned%techniques.%%The%light%direcEon%in%the%viewing%environment%is%the%same%as%the%light%direcEon%applied%to%the%photograph.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
138
This%image%shows%the%rendered%image%from%the%PTM%using%the%same%light%posiEon.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
139
This%shows%just%the%calculated%Mahe%surface%of%the%coin.%%The%surface%normals%are%calculated%from%this%data.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
140
This%slide%shows%just%the%calculated%highlights%and%shadows.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
141
Here%is%the%surface%normal%map%produced%by%this%approach.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
142
Here%is%the%normal%%map%produced%by%the%PTM.%%You%can%see%that%the%edges%of%most%of%the%details%are%incorrect%in%that%they%appear%to%show%a%steep%direcEonal%change%in%the%surface.%These%areas%all%have%shadows%in%some%of%the%sampled%images.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
143
This%is%the%normal%map%produced%from%the%RTI%created%using%the%HSH%algorithm.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
144
In%the%AIC%presentaEon,%a%live%demo%of%the%sodware%showing%this%work%was%presented%using%an%addiEonal%example.%%Cultural%Heritage%Imaging%is%currently%working%with%Mark%and%Mingjing%to%incorporate%the%calculaEon%of%more%accurate%surface%normals%into%the%open%source%tools%released%to%the%RTI%community.%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
145
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
146
I’d%like%to%close%the%talk%with%a%brief%discussion%of%the%importance%of%the%scienEfic%method%to%aid%in%the%evaluaEon%of%images%produced%using%these%methods.%%An%essenEal%element%of%tradiEonal%scienEfic%inquiry%is%the%systemaEc%gathering%and%sharing%of%observaEons%about%the%world.%These%observaEons%of%the%senses%are%labeled%‘empirical’.%%Within%scienEfic%discourse,%the%methodology%employed%in%the%process%of%generaEng%scienEfic%informaEon%has%been%tradiEonally%called%the%inquiry’s%‘provenance’.%%%This%provenance%explains%where%the%informaEon%came%from%and%how%it%was%used%in%order%to%allow%evaluaEon%of%the%results,%and%also%to%allow%others%to%replicate%the%work.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
147
CollecEng%and%managing%this%type%of%data%enables%distributed%scholarship%and%also%greatly%improves%the%likelihood%that%data%collected%today%can%be%reused%in%the%future,%including%reprocessing%exisEng%image%sets%using%beher%sodware%and%new%approaches.%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
148
The%Digital%Lab%Notebook%is%intended%to%both%collect%and%manage%all%the%necessary%data%about%a%digital%representaEon.%%It%includes%informaEon%about%how%the%data%was%collected,%why%it%was%collected,%who%was%present,%and%all%subsequent%processing%steps.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
149
The%current%methodology%and%tools%for%RTI%and%AR%%collect%this%informaEon,%but%it%is%not%organized%in%a%way%that%is%truly%useful.%%CHI%is%currently%working%to%manage%this%data,%including%relaEonships%of%the%data,%using%the%CIDOC%Conceptual%Reference%Model%(CRM)%and%producing%Linked%Open%Data.%%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
150
CHI%recently%received%an%NEH%Digital%HumaniEes%StartQup%grant%to%allow%us%to%further%this%work.%A%primary%focus%of%this%project%is%2%case%studies%to%be%done%with%the%Georgia%O’Keeffe%Museum%and%the%Classics%Department%at%University%of%Texas,%AusEn.%%%A%project%page%with%more%informaEon%about%this%project%can%be%found%here:%%hhp://culturalheritageimaging.org/What_We_Do/Projects/nehQstartup/%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
151
So,%please%remember%that%collecEng%RTI%data%is%not%just%for%interacEve%relighEng!%%This%type%of%data%can%be%used%for%quanEtaEve%comparisons.%%Work%is%also%underway%to%improve%surface%normal%accuracy.%%ComputaEonal%Photography%also%lends%itself%to%good%pracEce%for%tracking%all%the%needed%informaEon%to%allow%data%reuse%and%evaluaEon%by%others.%%
RATS Postprints Volume 5, 2014
Schroer, C., et al.
Accurate Measurement and the Quantification of Surface and Material Property Change Using New RTI and AR Techniques
152