video based structural health monitoring using …

VIDEO BASED STRUCTURAL HEALTH MONITORING

USING VIRTUAL VISUAL SENSORS

by

Ali Shariati

A dissertation submitted to the Faculty of the University of Delaware in partial

fulfillment of the requirements for the degree of Doctor of Philosophy in Civil

Engineering

Summer 2016

© 2016 Ali Shariati

All Rights Reserved

All rights reserved

INFORMATION TO ALL USERSThe quality of this reproduction is dependent upon the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscriptand there are missing pages, these will be noted. Also, if material had to be removed,

a note will indicate the deletion.

All rights reserved.This work is protected against unauthorized copying under Title 17, United States Code

Microform Edition © ProQuest LLC.

ProQuest LLC.789 East Eisenhower Parkway

P.O. Box 1346Ann Arbor, MI 48106 - 1346

ProQuest

Published by ProQuest LLC ( ). Copyright of the Dissertation is held by the Author.

ProQuest Number:

10191680

10191680

2016

VIDEO BASED STRUCTURAL HEALTH MONITORING

USING VIRTUAL VISUAL SENSORS

by

Ali Shariati

Approved: __________________________________________________________

Harry W. Shenton III, Ph.D.

Chair of the Department of Civil and Environmental Engineering

Approved: __________________________________________________________

Babatunde A. Ogunnaike, Ph.D.

Dean of the College of Engineering

Approved: __________________________________________________________

Ann L. Ardis, Ph.D.

Senior Vice Provost for Graduate and Professional Education

I certify that I have read this dissertation and that in my opinion it meets

the academic and professional standard required by the University as a

dissertation for the degree of Doctor of Philosophy.

Signed: __________________________________________________________

Thomas Schumacher, Ph.D.

Professor in charge of dissertation




Signed: __________________________________________________________

Harry W. Shenton III, Ph.D.

Member of dissertation committee




Signed: __________________________________________________________

Michael J. Chajes, Ph.D.





Signed: __________________________________________________________

Branko Glisic, Ph.D.


iv

“… the highest appreciation is not to utter words, but to live by them” John F.

Kennedy

First, I thank my advisor Prof. Thomas Schumacher, who always believed in

me and was sometimes even more excited than myself in this research. I also thank my

committee members Profs. Shenton and Chajes for their support and input, without

which the completion of this dissertation would not have been possible. Measurement

of natural frequencies of the Streicker Bridge as presented in Chapter 3 was made

possible by Prof. Glisic, to whom I owe a big “thank you”. He also served as the

fourth committee member and I appreciate his input and critical questions.

Furthermore, I thank my family, who lives far away from me, yet their infinite

love has kept my heart warm and hopeful. Especial thanks go to my mom for all of her

sacrifice and love, the two very words that are synonymous to her precious life.

Finally, my wife. She has known me for the past 5 years, which has been the

most hectic, challenging, and busy phase of my life. I thank her for her unstoppable

care and her love during these years.

ACKNOWLEDGMENTS

v

LIST OF TABLES ...................................................................................................... viii

LIST OF FIGURES ....................................................................................................... ix

ABSTRACT ................................................................................................................ xiii

Chapter

1 INTRODUCTION .............................................................................................. 1

2 MONITORING OF STRUCTURES AND MECHANICAL SYSTEMS

USING VIRTUAL VISUAL SENSORS FOR VIDEO ANALYSIS:

FUNDAMENTAL CONCEPT AND PROOF OF FEASIBILITY .................... 5

2.1 Motivation ................................................................................................. 5

2.2 Proposed Sensing Approach ...................................................................... 6

2.2.1 Background .................................................................................... 6

2.2.2 Methodology .................................................................................. 7

2.2.3 Theoretical Basis ........................................................................... 8

2.3 Experimental Verification ....................................................................... 12

2.3.1 Candidate Virtual Visual Sensors ................................................ 14

2.3.2 Accuracy of Virtual Visual Sensors ............................................ 17

2.3.3 Frequency Analysis over Range of Pixels ................................... 18

2.4 In-Service Monitoring Example .............................................................. 19

2.5 Conclusion and Outlook .......................................................................... 22

3 EULERIAN-BASED VIRTUAL VISUAL SENSORS TO DETECT

NATURAL FREQUENCIES OF STRUCTURES .......................................... 23

3.1 Introduction and Background .................................................................. 24

3.2 Motivation and Objectives ...................................................................... 26

3.3 Description of Theory and Proposed Methods ........................................ 27

TABLE OF CONTENTS

vi

3.3.1 Theoretical Considerations .......................................................... 27

3.3.2 Quantization Error and Signal-to-Noise Ratio ............................ 27

3.3.3 Patch Processing .......................................................................... 30

3.3.4 Linear Gradient Pattern Targets .................................................. 31

3.3.5 Noise Reduction Strategies for LGPTs ....................................... 32

3.4 Laboratory Experiments .......................................................................... 33

3.4.1 Experimental Test Setup .............................................................. 33

3.4.2 Three-Story Structure Tests ......................................................... 33

3.4.2.1 Cameras Used ............................................................... 33

3.4.2.2 Reference Data from Accelerometers .......................... 34

3.4.2.3 Results from GoPro Camera ......................................... 35

3.4.2.4 Results from Photron FASTCAM U100 Camera ......... 38

3.4.2.5 Comparison of Results ................................................. 41

3.4.3 Steel Beam Tests ......................................................................... 42

3.5 Field Test ................................................................................................. 43

3.6 Conclusions and Outlook ........................................................................ 45

4 EULERIAN-BASED VIRTUAL VISUAL SENSORS TO MEASURE

DYNAMIC DISPLACEMENTS OF STRUCTURES ..................................... 47

4.1 Introduction ............................................................................................. 48

4.2 Background and Theory .......................................................................... 49

4.3 Experiments ............................................................................................. 55

4.3.1 Laboratory Setup and Instrumentation ........................................ 55

4.3.2 Data Acquisition and Data Preprocessing ................................... 56

4.4 Results ..................................................................................................... 57

4.4.1 Accuracy of Proposed Approach ................................................. 57

4.4.2 Relationship of Noise and Patch Size .......................................... 63

4.4.3 Dynamic In-Service Load Test on the Streicker Bridge ............. 64

4.5 Discussion and Conclusions .................................................................... 67

5 MEASUREMENT OF TENSION IN CABLES USING VIRTUAL

VISUAL SENSORS ......................................................................................... 69

5.1 Introduction ............................................................................................. 70

5.2 Theory ...................................................................................................... 71

vii

5.3 Laboratory Experiments .......................................................................... 73

5.4 Real World Applications ......................................................................... 79

5.5 Conclusion, Future Work ........................................................................ 81

6 SUMMARY AND OVERALL CONCLUSIONS, FUTURE WORK ............ 83

REFERENCES ............................................................................................................. 85

viii

Table 3.1: Comparison of SNR from three-story structure. ..................................... 41

Table 4.1: Summary table of accuracy evaluation. .................................................. 63

Table 5.1: Measured natural frequencies in Hz using the accelerometer and the

camera, and Δ𝑓/𝑓 as a measure of error. ** ........................................... 77

Table 5.2: Range of load cell values and their average vs. the calculated tension

based on natural frequencies measured by accelerometer and the

camera: .................................................................................................... 78

LIST OF TABLES

ix

Figure 2.1: Proposed methodology of a virtual visual sensor (VVS) to measure

structural vibrations. xp and yp represent fixed coordinates of the

monitored pixel, P. (Figure courtesy of Thomas Schumacher) ................. 8

Figure 2.2: Example of experimental data extracted from a VVS: brightness of (a)

red; (b) green; and (c) blue; (d) computed intensity (used for

subsequent analyses); I, and (e) smoothed intensity (for illustrative

purposes). .................................................................................................. 9

Figure 2.3: Illustration of the relationship between vibration of motion in

direction (one direction only for simplification), amplitude of

vibration A, pixel location xP and pixel size, and shape of the grey-

scale intensity curve I(x). (Figure courtesy of Thomas Schumacher) ..... 10

Figure 2.4: Cantilever test:(a) Snapshot of moving cantilever and (b) candidate

virtual visual sensors (VVS). (Figure courtesy of Thomas

Schumacher) ............................................................................................ 14

Figure 2.5: Time history (left column) and frequency (right column) data for (a)

accelerometer; (b) pixel A; (c) pixel B; and (d) pixel C. Note: the

intensity time histories were centered about 0 vertically to avoid a

large DC component in the frequency domain. ....................................... 16

Figure 2.6: Measurements correlation (a) Table and (b) plot showing correlation

between physical accelerometer and virtual visual sensor (VVS). ............. 18

Figure 2.7: Outline of the moving object (a) Snapshot of cantilever; (b) snapshot

with highlighted pixels of same peak frequency; and (c) snapshot with

normalized highlighted pixels of same peak frequency. ......................... 19

Figure 2.8: Histogram of peak frequencies from all pixels in the bridge video.

Insert: (a) Snapshot of video clip; (b) Colored pixels with same

frequencies. .............................................................................................. 21

Figure 3.1: Experimental test setups for: a) Three-story structure and b) steel

beam. (Figure courtesy of Thomas Schumacher) .................................... 24

LIST OF FIGURES

x

Figure 3.2: Linear gradient pattern targets a) Sample linear gradient pattern

targets (LGPT), b) intensity values captured by the camera and linear

curve-fit, c) calculated noise, and d) the histogram of the noise. ............ 26

Figure 3.3: The effect of non-linearity in the spatial domain. Rows one to four

(top to bottom) show intensity in the spatial domain, I(x),

displacement, d(x), observed intensity in the time domain, I(t), and the

Fourier transform of the former, respectively. Column a) shows the

case of 𝐼𝑥 = 𝑥3, b) the linear case of 𝐼𝑥 = 𝑥, and c) the case of an

impulsive change of intensity modeled by 𝑥 = 𝑥99. .............................. 29

Figure 3.4: Two noised reduction strategies for LGPTs: a) Patch on an LGPT to

average out the noise b) linear regression approach: measured

intensities are mapped onto linear curve-fit. (Figure courtesy of

Thomas Schumacher) .............................................................................. 32

Figure 3.5: Cameras used for the laboratory experiments: a) Photron FASTCAM

UX100, b) Photron FASTCAM SA-X2 and c) GoPro Hero 3. ............... 34

Figure 3.6: Sample data from the accelerometers: a) Second story and b) third

story. The left and right column shows data in the time and frequency

domain, respectively. ............................................................................... 35

Figure 3.7: Data extracted from videos taken by the GoPro camera without

LGPTs: a) One pixel in the middle of first floor, and b) a 50 x 50

patch of pixels in the middle of the first floor. ........................................ 36

Figure 3.8: Data extracted from videos taken by the GoPro camera with LGPTs:

a) One pixel on the gradient target, b) A patch of 5 x 5 pixels on the

target c) Linear regression applied to 10 randomly chosen points on

the gradient. ............................................................................................. 37

Figure 3.9: Data extracted from videos taken by the FASTCAM UX100 camera

with and without LGPTs: a) One pixel at the very bottom, b) a patch

of pixels at the mid-level of the first floor, c) one pixel on the LGPT,

d) a 5 x 5 pixel patch on the LGPT, and e) linear regression applied to

10 random pixels on the LGPT. .............................................................. 40

Figure 3.10: The steel beam test results: (a) Results from the FASTCAM SA_Z

camera, and (b) results from one accelerometer. ..................................... 42

Figure 3.11: Real world application a) View of the Streicker Bridge and b) test

setup, camera position and LGPTs (insert). ............................................ 44

xi

Figure 3.12: Frequency response of the Streicker Bridge from (a) the VVS located

on an LGPT and (b) the fiber-optic sensor system. ................................. 45

Figure 4.1: Illustration of the proposed methodology to measure structural

vibrations using Eulerian-based virtual visual sensors (VVS).

Acronyms: BWT = black-and-white target, FFT = Fast Fourier

transform, IDT = intensity-to-displacement transform. (Courtesy of


Figure 4.2: Illustration of the VVS measurement process using a black-and-white

target (BWT): the target, which is attached to the vibrating structural

element, moves in the x-direction relative to a fixed patch of pixels,

i.e. having Eulerian-coordinates, as a function of time, t. (Figure

courtesy of Thomas Schumacher) ........................................................... 51

Figure 4.3: Illustration for the case where the camera is not oriented perpendicular

to the displacement component of interest, uact. O denotes the camera

location and T the center location of the target mounted to the

vibrating structure. a and b represent horizontal and vertical distance

between the camera and the center of the target. (Figure courtesy of


Figure 4.4: Experimental setup: (a) high-speed data acquisition system, (b)

potentiometer to measure displacements at the first story mass, (c)

digital camera to collect VVS data, and (d) three-degree-of-freedom

laboratory structure. ................................................................................. 56

Figure 4.5: Photo of BWT target with two VVS patch sizes: (a) 40 x 50 pixel

patch and (b) 40 x 100 pixel patch. ......................................................... 58

Figure 4.6: Comparison of results using independent calibration factors, 40 x 50

pixels (left column) and 40 x 100 pixels (right column): (a) Time

history signals of VVS patch data and potentiometer, (b) correlation

between the two measurements, (c) absolute value of the difference

between the two signals (errors), and (d) histogram of the errors. .......... 60

Figure 4.7: Comparison of results using calibration factors based on minimized

difference between measurements, 40 x 50 pixels (left column) and 40

x 100 pixels (right column): (a) Time history signals of camera and

potentiometer, (b) correlation between the two measurements, (c)

absolute value of the difference between the two signals (error), and

(d) histogram of the error. ....................................................................... 62

xii

Figure 4.8: Noise power and the signal-to-noise ratio (SNR): (a) Power of the

noise vs. the number of pixels in the VVS patch (N), (b) histogram of

noise in one pixel, (c) histogram of noise in a 10 x 10 pixel patch, and

(d) the SNR values vs. width over length of the patch. ........................... 64

Figure 4.9: Photograph of the Streicker Bridge showing the measurement setup

and the location of the VVS. The insert shows the location of the 60 x

20 pixel VVS patch (red rectangular). The target was installed by Dr.

Maria Feng’s research team from Columbia University [46]. ................ 66

Figure 4.10: Results from the dynamic load test on the Streicker Bridge: (a)

Computed actual vertical displacement time history and (b) frequency

response of signal (a). .............................................................................. 67

Figure 5.1: Test setup showing: a) load cell: Omegadyne LC101, b)

accelerometer: Entran EGC, c) camera: GoPro Hero 3, and d) cable. .... 74

Figure 5.2: Samples of typical data: a) FFT of acceleration, b) FFT of VVS, c)

FFT of load cell, and d) tension force vs. time. ....................................... 76

Figure 5.3: Lab test results a) The minimum, maximum, and average tension

force in the cable measured by the load cell (Min LC, Max LC and

Mean LC) vs. the average calculated values of tension based on

natural frequencies of accelerometer (Mean Acc) and camera (Mean

Cam), b) The same values divided by the average load cell values. ....... 79

Figure 5.4: Typical pluck test on a cable of the Indian River Inlet Bridge a) The

crane, b) moving the cable to initiate free vibration. ............................... 80

Figure 5.5: Measured natural frequency for the cable 219 E ..................................... 81

xiii

Structural health monitoring (SHM) has become a viable tool to provide

owners of structures and mechanical systems with quantitative and objective data for

maintenance and repair. Traditionally, discrete contact sensors such as strain gages or

accelerometers have been used for SHM. However, distributed remote sensors could

be advantageous since they don’t require cabling and can cover an area rather than just

a few discrete points. Along this line a novel monitoring methodology based on video

analysis is proposed. By employing commercially available digital cameras combined

with efficient signal processing methods, measurement of natural frequencies using a

computationally less demanding algorithm was possible. In this dissertation, the basic

concept of the proposed so-called Eulerian-based virtual visual sensors (VVS) is first

introduced. In order to improve the signal-to-noise ratio (SNR), the application of

oversampling as well as two different targets were considered. The proposed

methodology was evaluated on a set of laboratory experiments to demonstrate the

accuracy of the considered approach. In-service monitoring examples of different

bridges are further provided to show the practical aspects. A discussion of further

work to improve the methodology is also discussed.

ABSTRACT

1

INTRODUCTION

Structural vibrations contain important information about a structural system’s

dynamic characteristics. Changes over time in the vibration response can be caused by

alterations in the loading, boundary conditions, or degradation of the structural system.

Several approaches have been developed to extract information from the measured

vibrational characteristics of structures, which fall in a category well known as

“vibration-based structural health monitoring (SHM)” [1]–[14]. These methods

essentially utilize natural frequencies and mode shapes acquired from different types

of physical sensors such as accelerometers.

A critical step in the process of measuring mode shapes and natural

frequencies, is the gathering of the vibration data using sensors or sensor networks.

The ultimate goal is to have a sensing system that produces objective, quantitative, and

accurate data, inexpensively. Conventional contact-type sensors such as strain gages

or accelerometers that are attached to specific locations of a structure are capable of

measuring the response at that specific point. Accessibility of the member of interest

combined with wiring issues in addition to high local-only sensitivity are some

drawbacks of conventional sensors, which have stimulated innovation to develop non-

contact sensors.

Alternatively, optical devices such as laser Doppler vibrometers, which can be

applied remotely, have been used but are expensive. Video-based methods have

Chapter 1

2

recently been introduced for a variety of applications in structural health monitoring

(SHM). Patsias and Staszewski [15] analyzed digital videos for edge detection and to

approximate the mode shape of a cantilever in a laboratory experiment. By applying a

wavelet transform to the mode shape they were able to detect the location of damage

which was introduced by cutting a groove with decreasing depth into the cross-section.

Lee et al. [16] devised a real-time method to measure in-plane displacements and

rotations using feature tracking techniques based on a Lagrangian approach, and

applied it to a target bridge. Zaurin and Catbas[17] developed a method using digital

video data to locate and measure applied loads on a bridge and devised an index called

unit influence line (UIL) as a measure of the health of bridges. Elgamal et al.[18]

developed a framework to integrate different data types including computer vision data

to create a “decision-support system” for bridges and other lifelines. In a SHM review

on wind turbines by Ciang et al. [19], it is noted that digital image correlation (DIC)

techniques can also be used for these structures, but the 3-D version of these methods

should be investigated in more depth if they are to be applied. Song et al. [20]

modified the Hough Transform to track numerous markers on a beam with a

computationally efficient algorithm and fitted a spline curve to the tracked shape in

order to detect the location of the damage.

The use of digital videos for SHM is only in the beginning stage. With the

availability of inexpensive yet high-quality digital video cameras, there is great

potential that has not been fully explored yet. In this dissertation a methodology to

measure natural frequencies and dynamic displacement is investigated with

commercially available camera technology as well as high speed professional camera.

With an efficient and yet simple methodology to capture and compute structural

3

vibration data from digital videos, this technique has been shown to be effective,

accurate, and have a high signal to noise ratio.

The development of this technique is presented in Chapters 2 to 4. In Chapter 2

the concept of Eulerian virtual visual sensors (VVS) is introduced and based on that

the estimation of the first fundamental natural frequency of vibration of structures is

studied [21]. The methodology is based on the original idea presented in [22] and

further applied in [23]. The essence of the original idea is that single pixels can carry

essential information about minute changes of objects that are not visible to the naked

eye, but can be made visible by a technique called Eulerian motion magnification [22].

It has been shown experimentally, that change of intensity in certain pixels of a digital

video can be related to the natural frequency of a vibrating structure. It should be

noted that this approach is in its very nature is different from what is used in feature

tracking or block matching algorithms such as DIC. It has been shown that in this

technique to achieve the natural frequency the calculation of the displacement is not

necessary.

It is later on shown in Chapter 3 that by employing the basic methodology

introduced in Chapter 2, 2nd and 3rd natural frequencies of a laboratory-scale three-

story structure are not observable. It is discussed that this problem is caused partially

by the low signal-to-quantization noise ratio (SQNR) due to the limited and usually

uncontrollable range of change in intensity. Employment of gradient pattern targets,

which are mounted to the structure at locations of interest, expand the range of

intensity values and thus reduce the quantization noise, is thoroughly investigated to

address this problem. Also another approach to increase the SQNR, which is

oversampling in time is discussed and studied. The idea of using a patch of pixels

4

rather than a single pixel to alleviate challenges related to occlusion due to large

displacements and two efficient ways to reduce the noise are discussed as well.

In Chapter 4, a simple yet effective alternative way to measure dynamic

displacement using Eulerian-based virtual visual sensors (VVS) is proposed, for which

the fundamental basis is developed in earlier Chapters and in the author’s publications

[21], [24]–[27]. It is shown for a black-and-white target (BWT), a linear intensity-

displacement relationship exists for a patch of pixels on the boundary of the target. It

should be noted that this same transform can also be applied without using BWTs, for

example to the edge of a structural member where a distinct boundary between the

member and the background exists.

In Chapter 5, an application of the methodology for the measurement of natural

frequencies of cables in cable stayed bridges is presented. A laboratory study and field

test show that the methodology, as introduced in Chapter 2, can be applied to measure

natural frequencies of cables accurately. Finally, in Chapter 6 overall conclusions are

made and future work is discussed.

5

MONITORING OF STRUCTURES AND MECHANICAL SYSTEMS USING

VIRTUAL VISUAL SENSORS FOR VIDEO ANALYSIS: FUNDAMENTAL

CONCEPT AND PROOF OF FEASIBILITY

In this chapter, the concept of Eulerian virtual visual sensors (VVS), the theory

behind it and application in laboratory and real-world settings are studied. The first

natural frequency of vibration of the structures is studied [21]. As mentioned before

this method genuinely is based on the idea presented in [22]. The original idea is that

minute changes in a single pixel reveals essential information about the dynamic

characteristics of the captured context. It should be emphasized that there is a

fundamental difference between the proposed technique and block matching and

feature tracking algorithms. This chapter is published in Sensors 2013, 13(12), 16551-

16564; doi:10.3390/s131216551 and currently has 7 citations as reported by Google

Scholar.

2.1 Motivation

The objective of this study was to evaluate a novel sensing approach for

structural health monitoring (SHM) purposes which is contactless, inexpensive, and

flexible in its application. Vibration data are important in a number of disciplines such

as mechanical and structural engineering. A comprehensive review on structural health

monitoring (SHM) shows the efforts put forth to estimate damage and damage

Chapter 2

6

location based on observed changes in natural frequencies of vibration [5]. The

literature contains different resources addressing vibration-based SHM as well [8], [9],

[28]–[32]. Finally, natural frequencies from in-service structures are often used to

calibrate finite element (FE) models [33], [34].

2.2 Proposed Sensing Approach

2.2.1 Background

In a recent inspiring paper on Eulerian video magnification, Wu et al. [22]

present an innovative yet beautifully simple approach to magnify subtle motions in

digital videos so that they become visible to the naked eye. This was done using an

Eulerian specification where a pixel with a fixed coordinate is selected and its value

monitored in time. In contrast, in a Lagrangian specification one would attempt to

track a specific feature in a video in time and space. One of the examples presented,

which may have great potential for application in the medical field, measures the pulse

of a person by analyzing a video taken from the person. The inventors found that the

minute change in intensity in the red content, R, of the person’s skin was significant

enough to be analyzed to accurately compute the person’s pulse. Another example was

a video of a person’s wrist where the expansion and contraction of the veins were

amplified to be clearly visible. The advantage is that this approach is contactless and

can be performed continuously without interfering with the person. Motivated by this

article a methodology based on the same fundamental idea for potential use in the field

of structural health monitoring (SHM) for structures and mechanical systems is

proposed.

7

2.2.2 Methodology

It is proposed that every pixel in a digital video taken from a structure

represents a candidate virtual visual sensor (VVS) that may be used for SHM purposes

(first suggested by Patsias and Staszewskiy [15]). The term “VVS” follows the

terminology suggested by Song, Bowen, et al. [35]. Although the approach presented

in the latter paper may appear similar, it is fundamentally different as they were

employing a Lagrangian specification where a target (or feature) is tracked in space

and time.

The proposed methodology uses an Eulerian specification where a specific

pixel is selected and monitored which is illustrated in Figure 2.1; the intensity of the

pixel at location xp and yp is monitored over time and analyzed using the Fast Fourier

Transform (FFT) [36], [37] to reveal the fundamental frequency of vibration. Note that

the pixel value in the time domain represents gray-scale intensity and does not directly

correspond to the amplitude of vibration, i.e., displacement. As a result, at this

point it is not possible to estimate the amplitude but only the frequency of the motion

which represents a limitation.

8

Figure 2.1: Proposed methodology of a virtual visual sensor (VVS) to measure

structural vibrations. xp and yp represent fixed coordinates of the

monitored pixel, P. (Figure courtesy of Thomas Schumacher)

2.2.3 Theoretical Basis

Digital videos are a sequence of digital images captured at a specified frame

rate. Typical frame rates of commercially available cameras are 25, 30, or 60 fps

(=frames per second). In this study a range of cameras are used, including two

inexpensive point-shoot cameras with 25 and 30 fps, respectively, and a new high-

speed camera mainly used in the adventure sports community that can capture videos

up to 240 fps. Video frames are typically stored in RGB (red-green-blue) color mode

as measured by the camera’s image sensor. A single grey-scale value, called intensity,

I, is assigned to each pixel where 0 and 255 represent black and white, respectively.

MATLAB uses a linear combination to calculate I based on RGB values that

eliminates hue and saturation information while retaining the luminance [38]:

0.2989 0.5870 0.1140 I R G B (2.1)

An example of experimental data extracted from a VVS and the resulting

intensity curves are shown in Figure 2.2. If the intensity value is smoothed using a 5-

9

point moving average as shown in Figure 2.2 e, the quantization effects that exist in

the raw brightness values (Figure 2.2c) and the computed grey-scale intensity curve

(Figure 2.2d) can effectively be removed to reveal a relatively harmonic motion. For

this study, only raw intensities (example shown in Figure 2.2d) were used for the

computation of frequencies.

Figure 2.2: Example of experimental data extracted from a VVS: brightness of (a)

red; (b) green; and (c) blue; (d) computed intensity (used for subsequent

analyses); I, and (e) smoothed intensity (for illustrative purposes).

Figure 2.3 illustrates the factors that influence the accuracy and reliability of

the proposed VVS. The dotted line represents the grey-scale intensity curve I(x) along

a path x. For this theoretical example, the background is assumed to be light colored

and the object of interest dark colored. The location and size of the monitored pixel is

depicted by the grey square denoted with P(x,t). L represents the length over which the

10

intensity changes. Figure 2.3 represents a snapshot and as time t progresses the

intensity curve I(x) will vibrate horizontally (in the x-direction) with an amplitude A

causing the pixel to oscillate vertically about xp, following the I(x) curve.

Figure 2.3: Illustration of the relationship between vibration of motion in direction

(one direction only for simplification), amplitude of vibration A, pixel

location xP and pixel size, and shape of the grey-scale intensity curve

I(x). (Figure courtesy of Thomas Schumacher)

If the object, characterized by the intensity I(x), is vibrating at a natural

frequency ώ0 and without losing generality we can write 0sinx A t , hence the

intensity value becomes 0sinI x I A t . If we want to consider the effect of

the function I(x) on our measured peak frequencies we can write:

0sin j tF I x I A t e dt

(2.2)

If I(x) is a linear function and by subtracting the DC term we are able to

compute the exact peak frequency (neglecting any quantization noise)

0 0sin sinI A t C t so that:

11

0 0F I t Cj

(2.3)

where C is a constant. If I x is a nonlinear function, e.g., nx , with the

identity 0sin

2

j t j te et

we can write:

02

0

12 2

n nj t j t nk j n k tj t j t

k

ne e AF I t A e dt e e dt

k

(2.4)

Taking the summation out of the integral we obtain:

0

0

2 1 22

n nk

k

nAF I t n k

k

(2.5)

Equation (2.5) reveals that any nonlinearity of degree n in I(x) produces peak

frequencies at (n – 2k)ώ0 for 0 < k < n. It is important to note that if I(x) can be written

as a power series, the magnitude of the spurious impulses in the frequency domain can

be calculated based on the above equation. From the discussion above, the following

can be concluded: The extreme positions (peak amplitude points) of the VVS with

respect to the intensity curve I(x) should be located on an approximately linear portion

of I(x) and within L. If I(x) is nonlinear, spurious frequency peaks will occur as can be

observed in Figure 2.5b which will be explained in more detail in Section 2.3.1. The

important implications of the above equations are as follows:

The intensity range max minI I I should be maximized, i.e., a small

range will increase quantization noise. This can be achieved by selecting

proper background and lighting conditions.

The number of pixels across A should be maximized which is directly

related to the spatial resolution.

12

The amplitudes of vibration should be small, i.e., the maximum amplitude

of vibration A should lie within L in order to avoid the appearance of

nonlinear system behavior.

The size of the pixel (or VVS) with respect to the length should be small to

avoid averaging of measured intensity values and additional quantization

noise.

Additionally, the following factors influence the accuracy of the VVS:

High sampling rates, i.e., a large number of frames per second, decrease the

quantization noise. Minimum sampling rates as given by the Nyquist-

Shannon sampling theorem [39].

The total signal duration T directly influences the resolution and thus the

uncertainty of the VVS, i.e., the resolution of a signal in the frequency

domain is 1f T .

Finally, moving and shaking of the camera, changing illumination, and

noise in the image sensor influence the signal-to-noise ratio and therefore

the accuracy of the computed peak frequency.

2.3 Experimental Verification

To verify the validity and accuracy of the proposed approach, a cantilever

beam with adjustable length, LC (i.e., variable stiffness) and a concentrated constant

mass on top, equivalent to a single-degree-of-freedom (SDOF) system, was tested as

shown in Figure 2.4 a. The test was initiated by creating an initial displacement (by

hand) and then letting the cantilever vibrate in its natural mode of vibration.

Acceleration was measured using a high-accuracy capacitive accelerometer (Model 2260-

010 by Silicon Design, sampling at 1 kHz) attached to the mass. Additionally, a digital

13

video was taken during the test capturing the motion of the cantilever using two

different cameras: a commercially available digital camcorder (Model ViXIA HFS100

HD by Canon, recording at 30 fps) for frequencies up to 10 Hz and a relatively high-

speed camera (Model Hero 3 by GoPro, recording at 120 fps) for higher frequencies. It

should be noted that, as for any digitally sampled signal, the Nyquist-Shannon [39]

sampling theorem applies, i.e., the sampling rate needs to be set to at least twice the

highest anticipated frequency to be distinguishable in the signal. Anti-aliasing filters

were set to one half of the selected sampling frequency for the accelerometer. For the

cameras, such an option is currently not available, and one of the goals was to

determine whether this represents a problem. The selection of the pixel to be

monitored (candidate VVS) turned out to be critical to obtain meaningful frequency

data from the videos and a thorough discussion is included in the following section.

14

Figure 2.4: Cantilever test:(a) Snapshot of moving cantilever and (b) candidate

virtual visual sensors (VVS). (Figure courtesy of Thomas Schumacher)

2.3.1 Candidate Virtual Visual Sensors

The advantage that every pixel in the video represents a candidate virtual

visual sensor (VVS) is also the challenge. In this section we compare and discuss the

signals computed from a number of different candidate pixels. For this evaluation the

cantilever length, LC was kept constant at 25 in (635 mm). Figure 2.4 b shows three

candidate VVS: Pixels A and B are located near the top and the bottom of the

cantilever where the largest and smallest amplitudes of vibration occur, respectively.

Pixel C is located away from the cantilever but capturing its shadow. Intuitively one

might pick pixel A since it is located where the largest motion takes place which

15

should produce the best data. However, in the case of our proposed approach this does

not work well as it is discussed next.

Figure 2.5a shows example data collected with the accelerometer. Although

pixel B is at the bottom of the cantilever, a place with the smallest motion which can

hardly be observed by the naked eye, the change of intensity (grey scale pixel value) is

represented by a relatively harmonic signal (Figure 2.5c). For pixel A located near the

top of the cantilever, where the amplitude of vibration is largest, the intensity value

experiences periodic impulses due to the sudden occlusion of the mostly grey

background by the beam. As a result, the FFT produces a periodic function as well,

showing pronounced harmonic peaks, as can be observed in Figure 2.5b. This can be

mathematically explained by a highly non-linear function such as I(x) = xn where n is a

large number. Although the peak frequency is present and correct, the upper

harmonics are very strong as well which makes the analysis more difficult.

Alternatively, pixel C is found to produce a relatively harmonic signal as well (Figure

2.5 d). Although it is not located on the structure, it can capture the motion of its

shadow. This represents an opportunity to observe vibrations indirectly, in case the

actual structure is not directly observable. These facts suggest that a reliable candidate

VVS for measuring frequency is a point with small amplitudes of vibration near the

physical boundary (i.e., edge) of the structure where the change in intensity is most

pronounced (a theoretical discussion is presented in Section 2.2.3). For the cantilever

study this can also be a point near the top of the cantilever after damping has reduced

the amplitudes of vibration significantly. For practical purposes, this condition is

usually satisfied considering the camera is relatively far from the structure compared

to the amplitude of vibrations to be captured.

16

Figure 2.5: Time history (left column) and frequency (right column) data for (a)

accelerometer; (b) pixel A; (c) pixel B; and (d) pixel C. Note: the intensity

time histories were centered about 0 vertically to avoid a large DC

component in the frequency domain.

Although the total average amplitude of vibration for the data shown in Figure

2.5a, c was approximately 100 mm and 0.3 mm, respectively, the signal-to-noise ratios

17

for the frequency plots are comparable. This further highlights the potential of this

sensing approach to capture small vibrations.

2.3.2 Accuracy of Virtual Visual Sensors

In order to determine the accuracy of the proposed approach, a pixel close to

the bottom of the cantilever was selected to compute the frequency as described

previously and shown in Figure 2.4a. The length, LC was varied between 50 and 635

mm to produce a range of natural frequencies. Figure 2.6 shows the correlation

between the physical accelerometer and the frequencies computed from the selected

VVS. The computed frequencies listed in Figure 2.6a are given as f ± f/2 to account

for the uncertainty where f = 1/T with T being the duration of the original signal

length in seconds. Although we used zero padding to run the FFT in some cases,

which will provide smoother peaks in the low frequency range, higher accuracy is not

achieved. As can be observed in Figure 2.6b, there is excellent correlation between the

frequencies computed from the two measurements. The squared correlation coefficient

and standard error between accelerometer and the camera’s computed frequency were

found to be 99.993% and 0.0295, respectively. Figure 2.6 a Table and b plot showing

correlation between physical accelerometer and virtual visual sensor (VVS).

18

Figure 2.6: Measurements correlation (a) Table and (b) plot showing correlation

between physical accelerometer and virtual visual sensor (VVS).

2.3.3 Frequency Analysis over Range of Pixels

An extended approach to determine candidate VVS is to analyze a selected

area of pixels in a video around the vibrating structure and then highlighting the pixels

that have the same frequency in the image as illustrated in Figure 2.7. It should be

noted that this only works well for small amplitudes of vibration for reasons

previously discussed. This involves the following steps:

1. Select range of pixels to be analyzed within video (shown as white box in

Figure 2.7b,c).

2. Compute time history of intensity values for each of the selected pixels.

3. Compute the peak frequency for each pixel as described in Section 2.3.2 and

create a histogram.

4. Highlight the pixels with the same peak frequency in the selected range (Figure

2.7b).

5. Normalize the color values with the magnitude of the FFT transform to reduce

noise (optional, Figure 2.7c).

19

6. Repeat steps 1 to 5 if more than one significant frequency is present in the

histogram.

Figure 2.7: Outline of the moving object (a) Snapshot of cantilever; (b) snapshot

with highlighted pixels of same peak frequency; and (c) snapshot with

normalized highlighted pixels of same peak frequency.

As can be observed from Figure 2.7, the result of this analysis is essentially an

image of the outline of the vibrating parts of the structure. Note that this was done for

a period where the cantilever was experiencing small amplitudes of vibration to avoid

problems as discussed in the previous section. This analysis could also be used to

average peak frequencies from several measurements rather than using one

measurement.

2.4 In-Service Monitoring Example

In order to examine the applicability of this method for practical purposes, a

video of an existing major bridge in Oregon was evaluated. The bridge consists of a

20

continuous steel truss and some of the vertical hangers have experienced extensive

torsional vibrations due to the high transverse winds which caused vortex shedding.

Concerns regarding fatigue at the connections have been raised and as a result, the

State Department of Transportation has recently retrofitted some of the susceptible

members. The reason for the vibrations is the low torsional stiffness of the used I-

sections. In a recent research project the problem was investigated in the laboratory to

make predictions on the remaining fatigue life [40].

A video taken by DOT personnel showing torsional oscillations was made

available to us. The movie was taken with an inexpensive point-and-shoot-type

camera recording at 25 fps by hand (no mechanical stabilization) with the intent to

qualitatively document such an occurrence and not necessarily for quantitative

analysis purposes. The video captured severe torsional vibrations of two vertical

hangers labeled (1) and (2) in Figure 2.8a, simultaneously. It should be noted that only

four seconds from the original video were usable which directly influences the

resolution in the frequency domain. For this analysis, all pixels were analyzed and

their peak frequency values computed as described in the Section 2.3.3. Figure 2.8

shows a histogram of all computed peak frequencies. As can be observed, the majority

of computed peak frequencies are close to zero which essentially means that the

majority of the pixel intensity values don’t change. Frequencies between 0 and 1 can

be associated with the fact that the camera was held by hand and therefore probably

not completely fixed. A closer look reveals that there are two distinct frequency peaks

that can be associated with the torsional vibrations of the two hangers.

21

Figure 2.8: Histogram of peak frequencies from all pixels in the bridge video. Insert:

(a) Snapshot of video clip; (b) Colored pixels with same frequencies.

By filtering out pixels that are not within the desired frequency range, the

outline of the oscillating hangers is obtained (Figure 2.8b) as described in Section

2.3.3. The frequencies computed from one selected pixel for members (1) and (2) are

6.1 ± 0.125 and 7.1 ± 0.125 Hz, respectively. A finite element (FE) analysis of a

hanger modeled after one of these two members [40] predicted a torsional vibration

frequency of approximately 6.5 Hz. This result is not the actual measurement but

proves that our computed frequencies obtained from the VVS are plausible. This

demonstrates that, although the movie was never taken with the intent for analysis

with our proposed methodology, it was still possible to deduce useful frequency of

vibration information.

22

2.5 Conclusion and Outlook

The concept of virtual visual sensors (VVS) offers new opportunities for

structural health monitoring of structural and mechanical systems. The following

conclusions can be drawn from this study:

The fundamental frequency of vibration of single-degree-of-freedom

(SDOF) systems can be accurately computed using the proposed

methodology of virtual visual sensors (VVS).

VVS are inexpensive non-contact sensors with great application flexibility.

Multiple independently vibrating elements in one video can be distinguished

and their fundamental frequency of vibration computed.

The accuracy and resolution of the measurements depends on a variety of

factors such as sampling rate, quantization noise (function of pixel size and

location with respect to intensity curve), image sensor quality and size, and

lens type.

By highlighting the pixels with a distinct frequency, the outline of the

vibrating elements in a video can be recovered.

Application of the methodology to multi-degree-of-freedom (MDOF) and

continuous systems will be discussed in Chapter 2. Furthermore, investigations of

approaches that not only give frequency but also amplitude of vibration are presented

in Chapter 3. The sensitivity and reliability of the measurements are discussed and

finally, an application to estimate the tensile force of cables based on this

methodology is proposed in Chapter 5.

23

EULERIAN-BASED VIRTUAL VISUAL SENSORS TO DETECT NATURAL

FREQUENCIES OF STRUCTURES

Benefits of the introduced approach in Chapter 1, are that it allows for

distributed sensing and it is non-contact. However, as intensity does not reflect any

physical quantity, such as displacement, and the range of values is difficult to control,

the signal-to-noise-ratio (SNR) can be relatively low. Furthermore, impulsive changes

of intensity caused by large deformations compared to the pixel size can result in an

impulse train in the frequency domain which leads to ambiguity in determining peak

frequencies as discussed in previous chapter. As a result, it is often only possible to

measure the first fundamental mode of vibration. In this chapter, strategies using

targets mounted to the structure combined with signal processing methods that

significantly improve the signal-to-noise ratio (SNR) and allow for detecting higher

natural frequencies of vibration are presented. The concepts, their mathematical

background, laboratory tests to prove the accuracy and enhancement of SNR, as well

as an example of an in-service pedestrian bridge are presented and discussed. This

chapter is published in the Journal of Civil Structural Health Monitoring 5.4 (2015):

457-468 DOI: 10.1007/s13349-015-0128-5.

Chapter 3

24

3.1 Introduction and Background

By employing the basic methodology introduced in the previous chapter and in

[21] observing all natural frequencies of a laboratory-scale three-story structure was

not possible (see Fig. 3.1 a)). This was partially because of the low signal-to-

quantization noise ratio (SQNR) due to the limited and usually uncontrollable range of

change in intensity. One solution may be to employ gradient pattern targets, which are

mounted to the structure at location of interest, and expand the range of intensity

values and thus reduce the quantization noise. As it is discussed in Section 2.2.3, non-

linear intensity functions produce multiple harmonic frequencies in the frequency

domain. It was decided to evaluate grey-scale linear gradient pattern targets (LGPT)

with a theoretical range of intensities from 0 (= black) to 255 (= white) (see Fig 3.2

a)).

Figure 3.1: Experimental test setups for: a) Three-story structure and b) steel beam.

(Figure courtesy of Thomas Schumacher)

25

The idea is to introduce a linearly varying background avoiding non-linear or

impulse-like behavior. In Section 3.3.3 we discuss that another solution to increase the

SQNR, which is oversampling in time. The idea of using a patch of pixels rather than

a single pixel to alleviate challenges related to occlusion due to large displacements is

discussed in Section 3.3.3. Section 3.3.4 presents the basic idea of LPGTs and in

Section 3.3.5 we discuss two efficient ways to reduce their noise. In Section 3.4 the

laboratory experiments are presented and the results from a number of different digital

cameras used on two different structural systems are discussed. The results from

monitoring of an in-service pedestrian bridge during an impact test are deliberated in

Section 3.5. Finally, in Section 3.6 the conclusions are made and further work are

proposed. It is shown in the previous chapter, experimentally, that change of intensity

in certain pixels of a digital video can be related to the natural frequency of a vibrating

structure. It should be noted that this approach is in its very nature different from what

is used in feature tracking or block matching algorithms such as DIC.

26

Figure 3.2: Linear gradient pattern targets a) Sample linear gradient pattern targets

(LGPT), b) intensity values captured by the camera and linear curve-fit,

c) calculated noise, and d) the histogram of the noise.

3.2 Motivation and Objectives

The objectives of this study were to develop strategies for Eulerian-based

virtual visual sensors (VVS) that will (a) minimize non-linear effects and (b) improve

the signal-to-noise ratio of the recorded data to enable the detection higher natural

frequencies of vibration, which was not previously possible. This is achieved by

introducing linear pattern gradient targets (LGPTs) that are mounted to the structure

and by developing a number of signal processing steps. It is demonstrated that the

strategies work with a set of laboratory experiments as well as a field test on a real

bridge.

27

3.3 Description of Theory and Proposed Methods

3.3.1 Theoretical Considerations

As mentioned previously, non-linearity in the intensity function in the spatial

domain, I(x) will result in higher harmonics in the frequency domain [21]. This is

shown by the following theoretical relationship that is the same as Equation 2.5 which

comes here again for the sake of convenience:

0

0

, 2 1 22

n nk

k

nLI F I x t n k

k (3.1)

where L is the amplitude of vibration, n is the degree of non-linearity, k the

counter from 0 to n, is the radial frequency, 0 is the object’s radial frequency, and

the Delta-Dirac function. For example, if the intensity function in the spatial domain,

I(x) is a third degree polynomial, 3)( xxI as shown in in Figure.3.3 a), and the

displacement follows a sinusoidal function with a frequency of 1 Hz (Figure. 3.3,

second row), the observed intensity response in the time domain, I(t) is not a

sinusoidal function, as illustrated in Figure. 3.3 a), third row. In the absence of noise,

the Fourier transform of the intensity values have two peak frequencies at 1 and 3 Hz,

as shown in the Figure. 3.3 a), fourth row, which verifies Equation. (3.1). On the other

hand, if I(x) is linear, the resulting I(t) is sinusoidal, as shown Figure. 3.3 b). A highly

non-linear case such as I(x) = x99 results in an impulsive response of the intensity, I(t)

in turn leading to multiple peaks in frequency domain as shown in Figure. 3.3 c). This

illustrates the effect of occlusion discussed earlier in this paper and in [21].

3.3.2 Quantization Error and Signal-to-Noise Ratio

Analog-to-digital (A/D) conversion involves two main steps: sampling in time

and quantization. Errors due to quantization and their effect on the signal is a known

28

issue addressed in the literature, e.g. in [41]. Assuming that the quantizer uniformly

covers the limit values and its error is independent of the original signal, it can be

deduced that the error is equivalent to an additive white noise. The white noise model

can also be used with high-resolution quantization, which satisfies the independence

condition. In practical signal processing, in a process called “dithering”, some random

noise within the range of quantization is added to the analog signal prior to digitization

to satisfy the independence of the error from the signal [41].

29

Figure 3.3: The effect of non-linearity in the spatial domain. Rows one to four (top to

bottom) show intensity in the spatial domain, I(x), displacement, d(x),

observed intensity in the time domain, I(t), and the Fourier transform of

the former, respectively. Column a) shows the case of 𝐼(𝑥) = 𝑥3, b) the

linear case of 𝐼(𝑥) = 𝑥, and c) the case of an impulsive change of

intensity modeled by (𝑥) = 𝑥99.

In commercially available cameras, the quantization resolution to reflect the

amount of absorbed energy in CCD sensors is usually 8 bit. However, as discussed

previously in the proposed methodology, this energy (or intensity value) does not

30

correspond to any physical quantity such as displacement or any of its derivatives. In

other words, higher amounts of displacement, velocity, or acceleration do not

necessarily cause higher change of intensity. Assuming that the quantization error can

be modeled as white noise, increasing the sampling frequency will decrease its

amplitude in the frequency domain. It can also be shown that by doubling the

sampling frequency, the power of quantization noise decreases by 3 dB. This means

that by doubling temporal sampling rate, the maximum theoretical increase in the

signal-to-noise ratio (SNR) is 3 dB. The frame rates of commercially available

cameras are in the range of 30, 60 and 120 Hz, which is reasonably sufficient for

measuring frequencies in large structural systems such as bridges but may not be

sufficient to detect all of the natural frequencies due to the high quantization error.

High-speed cameras represent an effective yet expensive solution to this issue, which

is also shown earlier [42]. A discussion of high-speed cameras can be found in Section

3.4.

3.3.3 Patch Processing

As discussed earlier and visualized in Figure. 3.3, choosing one pixel in a

digital video recorded at a comparatively low frame rate and resolution can lead to

ambiguous peaks in the frequency domain, which makes the detection of higher

frequencies difficult or often impossible. To solve the problem of occlusion, which

produces periodic impulses in time and frequency domain, one can choose a patch of

pixels and monitor their average value though time. In other words, by choosing a

patch of pixels, we virtually decrease the ratio of displacement to the pixel size which

makes the change of intensity smoother. Patch processing can be applied to videos

31

where no targets are used or combined with LGPTs as discussed in more detail in

Section 2.5.

3.3.4 Linear Gradient Pattern Targets

In this study the idea of using linear gradient pattern targets (LGPT) mounted

to locations of interest on the structure is proposed. For this case, we do need access to

the structure in order to mount the targets, which may require ladders or lifts,

depending in the situation. Also, the target needs to be oriented in the expected

direction of motion to capture the vibration amplitudes accurately. We employed

LGPTs with different sizes in our experiments as shown in Fig. 3.2 a). The idea of

these targets is to create a well-defined, linearly-varying background to avoid non-

linear behavior as discussed in Section 3.2. The criteria for size is to optimize

maximum amplitude of vibration, A with the length of the target, L. A typical cross

section of an LGPT as it is captured and represented by the camera is shown in Figure.

3.2 b). The intensity value, although designed to be linear, contains noise as is shown

in Figure. 3.2 c). This noise was computed by subtracting the linear curve from the

captured intensity curve. A histogram of the noise is shown in Figure. 3.2 d).

32

Figure 3.4: Two noised reduction strategies for LGPTs: a) Patch on an LGPT to

average out the noise b) linear regression approach: measured intensities

are mapped onto linear curve-fit. (Figure courtesy of Thomas

Schumacher)

3.3.5 Noise Reduction Strategies for LGPTs

A strategy to reduce the noise would be to average the intensities of a patch of

pixels on the LGPT as shown in Figure. 3.4 a). From Figure. 3.2 d) it can be seen that

the average of the noise is close to zero so it can be deduced that by averaging the

pixel values, we essentially reduce the noise. Another strategy for noise reduction is to

choose random pixels on the gradient and fitting a linear function through them

(Figure. 3.4 b)). Tracking the constant part of this linear function through time can

lead to a much less noisy signal, in the case of our laboratory experiments it improved

the SNR by 3 to 6 dB. The requirement for these computationally inexpensive noise

reduction techniques is that during the whole vibration phase, the selected pixels

should never leave the LGPT range, otherwise artificial non-linear behavior is

introduced. A solution to this is to employ LGPTs that consist of several patterns with

different lengths as shown in Figure. 3.2 a). The most appropriate target can then be

33

picked after the digital video has been collected, which is one of the advantages of this

approach.

3.4 Laboratory Experiments

3.4.1 Experimental Test Setup

Two laboratory experiments were performed: Free vibration of a laboratory-

scale three story structure as shown in Figure. 3.1 a) and free vibration of a simply-

supported steel beam (Figure. 3.1 b)). In the first test, an initial displacement was

manually imposed on the structure by hand. Following a sudden release, the system’s

free vibration was recorded until it was damped out. The 3.6 m long simply-support

steel beam with a W15x87 cross-section was struck with an instrumented hammer to

impose structural vibrations. In both experiments, high-precision capacitive

accelerometers were attached to the structures to verify the frequencies estimated from

the VVS data.

3.4.2 Three-Story Structure Tests

3.4.2.1 Cameras Used

Three different cameras were used for the laboratory experiments. For the

three-story structure experiment, a GoPro Hero 3 camera and a Photron UX100

(Figure. 3.5 a) and c)) were used. The resolution of the GoPro camera was 1280 x 720

pixels and the frame rate was 120 fps. The Photron camera was used with 500 fps and

its full resolution of 1280 x 1024 pixels to evaluate the ability of detecting higher-

order frequencies with high-speed cameras. Finally, for the beam experiment, and to

explore the limits of our proposed methodology, a Photron FASTCAM SA-X2 (Figure

34

3.5 b)) with 5000 fps and its full resolution of 1024 x 1024 pixels was employed. It is

important to note that there is a trade-off between resolution and frame rate due to the

bandwidth limit of the camera hardware. Also, based on our own observations, the

spatial noise power in high-speed cameras is relatively high, i.e. higher than in regular

cameras. Three LGPTs with dimensions 8 x 60 mm were attached to the three

different masses of the three-story structure as shown in Figure. 3.1 a). For our

analysis VVS data collected from the first story mass as it was best suited for the size

of our LGPTs is used.

Figure 3.5: Cameras used for the laboratory experiments: a) Photron FASTCAM

UX100, b) Photron FASTCAM SA-X2 and c) GoPro Hero 3.

3.4.2.2 Reference Data from Accelerometers

Two high-fidelity capacitive accelerometers were attached to the side at the

height of masses two and three (see Figure. 3.1 a)) and sampled at 1 kHz using a high-

speed data recorder. The natural frequencies computed from the acceleration data from

the second and third story was essentially the same for all of the experiments, as

shown in Figure. 3.6 a) and b), respectively. The only difference in the frequency

domain was that the magnitude of the peaks slightly varied. This, however, had no

influence on the value of the peak frequency. The fundamental frequencies of

35

vibration were found to be 4.70 Hz, 14.0 Hz and 20.9 Hz for both stories. It can be

observed that the SNR for the second story is higher, which is due to the fact that the

third and second natural frequencies of vibration have a much stronger contribution

than for the third (= top) story.

Figure 3.6: Sample data from the accelerometers: a) Second story and b) third story.

The left and right column shows data in the time and frequency domain,

respectively.

3.4.2.3 Results from GoPro Camera

Fig. 3.7 shows the results from the GoPro Camera without the use of LGPTs.

In Fig. 3.7 a), it can be seen that the first and second peak frequencies are detectable

but at the same time there are several higher harmonics in the frequency domain which

made it difficult to choose the right natural frequency. By using a patch of 50 x 50

pixels, however, it is possible to detect all of the natural frequencies (Figure. 3.7 b))

36

bearing in mind that other peak frequencies are just multiples of the first one.

Although the magnitude of the third mode is not very large, it is still detectable

(Figure. 3.7 b)). The duration of the signals are about 10 s, which produces a

resolution in the frequency domain of 0.10 Hz.

Figure 3.7: Data extracted from videos taken by the GoPro camera without LGPTs:

a) One pixel in the middle of first floor, and b) a 50 x 50 patch of pixels

in the middle of the first floor.

As proposed, linear gradient pattern targets (LGPT) should significantly

improve the results in Figure 3.7. By selecting a pixel on the gradient target of the first

floor, all three natural frequencies could be recovered as is evident in Figure 3.8 a).

Figure 3.8 b) shows the same data processed using a patch of 5 x 5 pixels on the LGPT

which reduced the noise by 2.7 dB. Figure 3.8 c) shows the data when the linear

regression approach is employed. As can be observed, this processing step is capable

37

of reducing the noise even better than the patch if applied for the case when LGPTs

are used.

Figure 3.8: Data extracted from videos taken by the GoPro camera with LGPTs: a)

One pixel on the gradient target, b) A patch of 5 x 5 pixels on the target

c) Linear regression applied to 10 randomly chosen points on the

gradient.

As can be observed from the inserts in Figure 3.8 in the frequency domain,

because of the relatively low SNR in Figure 3.8 a), accurate estimation of the third

peak frequency is difficult. Hence, the small difference in the third peak frequency can

be associated with the low SNR. As can be seen in Figure 3.8 b) and Figure 3.8 c), this

38

problem is resolved using patch processing and linear regression, respectively, which

notably improved the SNR of the signal. It should be noted that the use of LGPTs

improves the contrast and decreases quantization intervals, simultaneously.

Comparing the time history part of Figure 3.8 and Figure 3.7 shows that the range of

change in intensity values is much larger when LGPTs are used. This, as previously

mentioned, helps to reduce the effect of quantization noise, which is partially

responsible for missing peak in the frequency domain (Figure 3.7 a)).

3.4.2.4 Results from Photron FASTCAM U100 Camera

As discussed before, high-speed cameras can help improve the signal-to-

quantization noise ratio. For high-speed cameras, picking a pixel at the bottom of the

three-story structure without any noise reduction strategy can reveal all of the natural

frequencies (Figure 3.9 a)). Selecting a patch of pixels on the other hand will produce

a much less noisy signal and avoid the presence of artificial peaks in the frequency

domain as shown in Figure 3.9 b). As described before, the Photron FASTCAM U100

was used for this experiment. The problems associated with this type of camera are

their limited storage, which leads to shorter recording time, limited bandwidth, which

results in a sacrifice of spatial resolution with higher temporal resolution, and also

higher spatial noise. Also, the higher the frame rate, the better illuminated the medium

should be in order to have high-quality videos. The use of LGPTs for high-speed

cameras can be beneficial as well. As can be seen from Figure 3.9 c), although the

signal is noisy, the peaks are more pronounced. Using a patch of pixels on the LGPT

(Figure 3.9 d)) reduces the noise, increases SNR by 9 dB and shows the peaks even

clearer. Also linear regression can increase SNR by almost 12 dB (Figure 3.9 e). This

39

shows again that linear regression compared to patch processing can result in a better

SNR.

40

Figure 3.9: Data extracted from videos taken by the FASTCAM UX100 camera with

and without LGPTs: a) One pixel at the very bottom, b) a patch of pixels

at the mid-level of the first floor, c) one pixel on the LGPT, d) a 5 x 5

pixel patch on the LGPT, and e) linear regression applied to 10 random

pixels on the LGPT.

41

3.4.2.5 Comparison of Results

Table 3.1 shows the SNR for all experiments on the three-story structure. In

the case of the high-speed camera, the level of SNR of one pixel where no LGPTs are

used is artificially higher due to the exponential decay of the signals where LGPTs are

used. Also, a comparison of cases using low- and high-speed cameras with LGPTs

evidently shows an increase in the SNR from 3.4 to almost 6 dB, which is close to the

theoretical bound due to the improved SQNR. It should be noted that the setup of this

experiment with high speed camera was a little bit different from the previous tests

and that the duration of the signal was 7 s, which gives a resolution around 0.14 Hz.

These factors explain the slight difference in third peak frequency in Figure 3.9, and

the accelerometers’.

Table 3.1: Comparison of SNR from three-story structure.

Capacitive Accelerometer (Reference) with 1 kHz

Second story 31.8

GoPro Camera with 120 fps

Without LGPT With LGPT

One Pixel 14.7 18.0

Patch of pixels 17.1 20.7

Linear regression - 26.0

FastCam UX-100 with 500 fps

One Pixel 21.7 17.4

Patch of pixels 23.4 26.6

Linear regression - 29.4

42

3.4.3 Steel Beam Tests

Finally, and in order to explore the limits of our proposed methodology, we

conducted a test on a steel beam as shown in Figure 3.1 b). The stimulus was provided

by a hammer strike imposed at the mid-span location. In this test, as shown in Figure

3.10 a) and b), several peak frequencies were deducible from the accelerometer data.

These reference data were collected using the same high-precision capacitive

accelerometer as used for the three-story structure. Monitoring a patch of pixels at the

boundary of the steel beam where the gradient of the intensity is maximum (the edge),

it was possible to detect several peak frequencies in congruence with the measured

accelerometers’ peaks.

Figure 3.10: The steel beam test results: (a) Results from the FASTCAM SA_Z

camera, and (b) results from one accelerometer.

Figure 3.10 a) shows the peak frequencies detected by the camera, while

Figure 3.10 b) show the data from the accelerometers for comparison. As can be seen,

even without the LGPT, by using a patch of pixels, several peak frequencies could be

43

detected. The interesting point about this experiment is that it involves a continuous

system where the high-frequency displacements are extremely small, completely

unobservable by the naked eye. However, it was possible to identify frequencies as

high as 764 Hz using our proposed VVS methodology. The resolution in the frequency

domain is approximately 1 Hz.

3.5 Field Test

In order to evaluate the real-world performance of our proposed approach, we

conducted a field test on the Streicker Bridge (Figure 3.11 a)): a prestressed concrete

pedestrian bridge located on Princeton University’s campus in Princeton, NJ. As can

be seen from Figure 3.11 a), the bridge has a unique design with a main span and four

horizontally curved legs. The main span consists of a deck-stiffened arch. The bridge

is equipped with a structural health monitoring (SHM) system consisting of embedded

fiber-optic sensors. The data were made available to us by Prof. Branko Glisic and

allowed for a direct comparison with our measurements. The dynamic stimulus was

provided in form of a group of students jumping in unison for a few seconds at the

location of our LGPTs.

44

Figure 3.11: Real world application a) View of the Streicker Bridge and b) test setup,

camera position and LGPTs (insert).

LGPTs were mounted on the inside of a curved leg to measure vertical

vibrations while the cameras were on the other side of the street, approximately 8 m

away from the targets. The camera used was a Canon T4i with 60 frames per second

and 128 x730 resolution. The test was performed on April 23, 2014, with adequate

lighting conditions and some wind. We verified that the wind did not affect the

measurements by comparing several measurements taken at different instances in

time. Figure 3.12 shows the results in the frequency domain for both measurements.

As can be observed, the two main frequencies of vibration of the leg, namely 3.0 Hz

and 3.6 Hz, were detected by both sensing approaches. The low-frequency content in

the VVS data (Figure 3.12 a)) can possibly be explained by slightly periodically-

varying lighting conditions due to trees rocking with the wind. In this case this is not a

real issue since the motion did not result in an additional peak that could be

45

misinterpreted. Although it was not possible in this case to calculate a SNR, it can be

observed that the two frequency plots are of very comparable quality.

Figure 3.12: Frequency response of the Streicker Bridge from (a) the VVS located on

an LGPT and (b) the fiber-optic sensor system.

3.6 Conclusions and Outlook

In this chapter, we evaluate and discuss a number of strategies to detect higher

frequencies of vibration using the earlier proposed Eulerian-based virtual visual

sensors (VVS) [21]. It can clearly be seen that, based on the experiments, the use of

linear gradient pattern targets (LGPT) and high-speed cameras can improve the signal-

to-noise-ratio (SNR) and help detecting multiple frequencies in multi-degree-of-

freedom (MDOF) structural and mechanical systems. From the study, we conclude the

following:

The introduction of LGPTs increases the SNR and enables detecting higher

natural frequencies which is particularly helpful when standard digital

video cameras are use.

Analyzing a patch of pixels rather than a single pixel can be employed

when no LGPTs are used to smooth the change of intensity, i.e. minimize

impulse-type response in the signal.

46

By analyzing a patch of pixels or apply a linear regression approach, the

SNR of LGPTs can further be improved.

While high-speed camera technology is still expensive and mostly used by

researchers, the use of commercially-available digital video cameras in

conjunction with LGPTs allow for accurate and reliable detection of

multiple natural frequencies.

High-speed cameras benefit from lower noise amplitude due to

oversampling and are able to detect higher frequencies even without

LGPTs.

The methodology also works in the field where we found the same peak

frequencies compared to the existing structural health monitoring (SHM)

system.

A limitation of the approach is that it can only accurately capture vibrations

that are perpendicular to the line of view of the camera and in the direction

of the LGPT.

Correlation between intensity and actual displacement is considered in the next

chapter. Evaluation of signal to noise ratio, the effect and the distribution of noise is

investigated.

47

EULERIAN-BASED VIRTUAL VISUAL SENSORS TO MEASURE DYNAMIC

DISPLACEMENTS OF STRUCTURES

The pixels in the vicinity of the boundary of a vibrating structural element

contain useful frequency information, which we have been able to demonstrate in

earlier chapters. The ultimate goal, however, is to be able to compute dynamic

displacements, i.e. actual displacement amplitudes in the time domain. In order to

achieve that the use of simple black-and-white targets (BWT) that are mounted to

locations of interest on the structure are introduced. By using these targets, intensity

can be directly related to displacement, turning a video camera into a simple,

computationally inexpensive, and accurate displacement sensor with notably low

signal-to-noise ratio (SNR). We show that subpixel accuracy with levels comparable

to computationally-expensive block matching algorithms can be achieved using the

proposed targets. The methodology can be used for laboratory experiments, on real

structures. In this chapter the concept and theory of the proposed methodology is

introduced, laboratory experiments to evaluate the accuracy of the proposed BWT

target are presented, and the results from a field test of an in-service bridge are

discussed. This chapter was submitted as a paper to the Journal of Structural Control

and Health Monitoring and is currently in review.

Chapter 4

48

4.1 Introduction

Digital image correlation (DIC) and other block matching algorithms use

digital video data to measure static displacement fields with high accuracy[43], [44].

However, the computational cost of these methods is relatively high. Efficient yet

accurate non-contact methods are needed that are computationally inexpensive and

work with standard digital video cameras. In this chapter, we propose a simple

alternative way to measure structural vibrations using Eulerian-based virtual visual

sensors (VVS), for which the fundamental basis that have been developed earlier

chapters [21], [24]–[27] It is shown that for a black-and-white target (BWT), a linear

intensity-displacement relationship exists for a patch of pixels on the boundary of the

target. Which is referred to as intensity-to-displacement transform (IDT). It should be

noted that this same transform can also be applied without using BWTs, for example

to the edge of a structural member where a distinct boundary between the member and

the background exists. Figure 4.1 illustrates the concept of our proposed methodology:

a BWT target is attached to a location of interest. From the digital video extracted

from the camera, a VVS located on the BWT is selected. The change of intensity is

recorded over time (green curve) and by employing a linear transformation (= IDT)

converted to actual displacement (blue curve). Also shown is a frequency analysis (red

curve), which can be obtained directly from the intensity time history by using a Fast

Fourier transform (FFT). The focus of this chapter is on how to compute displacement

time histories (blue curve) and the nature of the error in such measurement.

49

Figure 4.1: Illustration of the proposed methodology to measure structural vibrations

using Eulerian-based virtual visual sensors (VVS). Acronyms: BWT =

black-and-white target, FFT = Fast Fourier transform, IDT = intensity-to-

displacement transform. (Courtesy of Thomas Schumacher)

4.2 Background and Theory

As it is discussed in previous chapters [18, 20], change of intensity observed

by the Eulerian-based virtual visual sensor (VVS) did not directly correspond to a

physical quantity such as displacement, velocity, or acceleration. In this chapter, the

use of simple printed black-and-white targets (BWT) that allow the measurement of

actual dynamic displacements, as visualized in Figure 4.2, is evaluated. For an ideal

BWT target observed in a digital video with dimensions Lt x Wt (pixel x pixel), the

pattern colors are represented by the minimum and maximum intensity values (Imin.

and Imax., respectively) corresponding to 0 (= black) and 255 (= white), respectively.

The displacement varies linearly with VVS patch intensity, Ip(t):

50

max min

p p

p

I II t x t n t

L

(4.1)

Where Ip(t) is the average pixel intensity across the patch area, Ap = Wp x Lp

(pixel x pixel) for a frame at time instant, t (sec), x(t) (pixel) measured displacement,

and Lp (pixel) is the length of the patch. It is assumed that intensity across the width of

the patch, Wp (pixel) is constant and it therefore does not appear in Equation. (4.1). It

should be noted that the patch length, Lp (pixel) should be large enough to account for

the maximum displacement amplitude, A (pixel), i.e. Lp > A. At the same time, the

length of the target, Lt (pixel) needs to be able to accommodate for the patch length, Lp

(pixel), i.e. the target cannot leave the patch, otherwise the relationship becomes non-

linear. Finally, there is no perfect BWT target (with perfect black (I = 0) or white (I =

255) intensity values) in a real setting and measurement noise is always present. The

total average noise of the patch, np(t) can be defined as:

1

1 N

p i

i

n t n tN

(4.2)

where N is the total number of pixels in the VVS patch and ni is the noise present in

pixel i.

51

Figure 4.2: Illustration of the VVS measurement process using a black-and-white

target (BWT): the target, which is attached to the vibrating structural

element, moves in the x-direction relative to a fixed patch of pixels, i.e.

having Eulerian-coordinates, as a function of time, t. (Figure courtesy of

Thomas Schumacher)

As an alternative to Equation. (4.1), VVS patch intensity, Ip(t) can also be

computed as the average intensity value of all pixels across the patch area, Ap:

1

1 N

p i

i

I t I tN

(4.3)

where N is the total number of pixels in the VVS patch and Ii is the intensity value of

pixel i.

52

Figure 4.3: Illustration for the case where the camera is not oriented perpendicular to

the displacement component of interest, uact. O denotes the camera

location and T the center location of the target mounted to the vibrating

structure. a and b represent horizontal and vertical distance between the

camera and the center of the target. (Figure courtesy of Thomas

Schumacher)

For the case where the camera is not oriented perpendicular to the

displacement component of interest, a geometric correction factor, C applies. This

factor is calculated based on the location of the camera (O) and the center location of

the target (T), as illustrated in Figure 4.3:

1

1

cos tan

Cb

a

(4.4)

Considering that Ip(t) is known and by introducing a calibration constant, B

(pixel/mm), which can be estimated from a frame extracted from the digital video, and

the geometric correction factor, C, the actual dynamic displacement of the target,

uact(t) can be computed using the following relationship:

act pu t B C I t in (mm) (4.5)

53

The presence of noise may require implementation of a noise reduction

technique. Fortunately, for a BWT the averaging process (expressed by Equations.

(4.1) and (4.3)) by itself helps reducing the noise, i.e. the power of the noise reduces

directly with the number of pixels in the patch, N. Assuming that the noise is

independent of the signal and can be represented by a stationary process, we arrive at:

2 2

max min2 2

2

p

p

p

nI II x

L N

(4.6)

Equation (4.6) relates the size of the VVS patch and the power of the noise and

the signal. As can be seen from the second term of the right hand side of Equation

(4.6), as the number of pixels increases the power of the noise decreases. However,

increasing the length of the patch will have the same effect on the power of the signal,

hence keeping the length as short as possible and the width as large as possible will

maximize the SNR. Substituting N for Wp.Lp in Equation (4.6), which is width x

length of the patch, we can get the following expression for the signal-to-noise ratio

(SNR):

2

2max min

2 22

max min2 2

p p

pp p

p p

I Ix

L W xSNR I I

Ln n

W L

(4.7)

Equation (4.7) shows that in order to reach the maximum SNR one has to

maximize the Wp-to-Lp ratio keeping in mind that Lp should be able to cover the

maximum displacement amplitude, A, as discussed earlier. For a specific camera and

lighting conditions, the pixel noise power can be assumed constant and the appropriate

size of the patch can be specified based on the desired SNR. The other factor that

should be discussed in Equation (4.7) is the second factor, max minI I , which has a

54

more significant effect on the SNR. It can be concluded from Equation (4.7) that the

higher the contrast between black and white in the target, the higher the SNR will be.

Using a confidence level and the noise power in a pixel the patch size for the

desired accuracy can be calculated. Assuming that noise has zero mean and in each

pixel it is independent and identically distributed (iid) RV, based on central limit

theorem we can say that as the patch size become bigger the average noise converge to

normal distribution. Using Equation (4.1) we’ll have:

−1

2𝛼

(𝐼𝑤ℎ𝑖𝑡𝑒 − 𝐼𝑏𝑙𝑎𝑐𝑘)

𝑙≤ 𝑛𝑝 ≤

1

2𝛼


𝑙 (4.8)

In which 𝛼 is the desired accuracy level with 𝛼 = 1 giving the pixel level

accuracy and 𝛼 < 1 the subpixel accuracy. Assuming the normal distribution for the

noise in the patch with zero mean and standard deviation of 𝜎2(𝑛𝑖)

𝑁 and defining the

confidence level 𝛽 with 𝐾𝛽 = Φ−1(1 −𝛽

2) we can write:

𝑃(−𝐾𝛽 < 𝑛𝑝

𝜎(𝑛𝑖)

√𝑁

< 𝐾𝛽) = 1 − 𝛽 (4.9)

So then with the confidence level 𝛽 we can say if:

𝐾𝛽 . 𝜎(𝑛𝑖)

√𝑁=

1

2𝛼


𝑙 (4.10)

Which is equivalent to:

√𝑙

𝑤=

1

2𝛼


𝐾𝛽 . 𝜎(𝑛𝑖) (4.11)

Then we will have:

𝑃 (−1

2𝛼


𝑙≤ 𝑛𝑝 ≤

1

2𝛼


𝑙) = 1 − 𝛽 (4.12)

55

Using the Equation 4.8 one can specify the size of the patch based on the desired

accuracy (𝛼) and the level of confidence 𝛽.

4.3 Experiments

4.3.1 Laboratory Setup and Instrumentation

A laboratory-scale three-degree-of-freedom structural system (Total height =

610 mm (2 ft) as shown in Figure. 4.4 (d) was used to evaluate the accuracy of the

methodology proposed in Section 4.1. The structure was excited by introducing

random initial displacements at two locations on the structure by hand followed by a

sudden release to initiate free vibration. A digital camera (GoPro Hero 3, shown in

Figure 4.4 (c))) capturing the free vibration response was located 305 mm (1 ft) away

from the structure. The displacement of the first floor was also measured using a 12.7

mm (0.5 in) amplitude potentiometer (Figure 4.4 (b)) connected to a high-speed data

acquisition system (Figure 4.4 (a)) using a sampling frequency of 1200 Hz. The frame

rate of the digital camera was 60 frames per seconds (fps).

56

Figure 4.4: Experimental setup: (a) high-speed data acquisition system, (b)

potentiometer to measure displacements at the first story mass, (c) digital

camera to collect VVS data, and (d) three-degree-of-freedom laboratory

structure.

4.3.2 Data Acquisition and Data Preprocessing

In order to compare the two measured signals, two steps have to be taken: (1)

synchronization of the signals in the time domain and (2) multiplication of the

measured signals by their appropriate calibration factors to obtain actual displacement

from the measured data. For step (1), both camera and potentiometer data were

interpolated linearly to two equivalent 3000 Hz signals. Based on the maximum

correlation between the two signals, the time lag between the two signals was

calculated and one of the signals shifted so that they had a common time axis. In order

57

to achieve actual displacement for step (2), the potentiometer was calibrated against a

precision height gage. The mean calibration factor was found to be 1.257 mm/V

(0.0495 in/V). For the camera intensity data, the known target length, Lt was measured

in a video frame in terms of pixels, which produced a mean calibration factor of 0.279

mm/pixel (0.011 in/pixel).

In addition to the independent application of calibration factors as described

above, it was also possible to multiply the VVS intensity data by a factor that

minimizes the second norm of difference, or error, between the two measurements.

This case represents the optimal estimate of the displacement for the VVS, assuming

the potentiometer represents an accurate reference measurement. Obviously, in a real

life scenario only the first approach can be used where a calibration factor has to be

estimated from the video data. It should be noted that the potentiometer does not

necessarily produce the more accurate displacement. This was particularly visible at

the peak displacement points and is discussed in more detail in Section 4.4.1.

4.4 Results

4.4.1 Accuracy of Proposed Approach

Two VVS patch sizes, Wp x Lp = 40 x 50 and 40 x 100 pixels, were selected to

study how the accuracy of the measurements change with the size of the VVS patch,

Ap. Figure 4.5 shows a snapshot of a video frame with the target and the two evaluated

VVS patch sizes.

58

Figure 4.5: Photo of BWT target with two VVS patch sizes: (a) 40 x 50 pixel patch

and (b) 40 x 100 pixel patch.

Figure 4.6 shows a comparison of a sample measurement using independent

calibration factors, as described in Section 4.3.2. As can be observed from Figure.4.6

(a), the displacements measured by the VVS and the potentiometer are, qualitatively,

in close agreement for both patch sizes. However, the inserts in Figure 4.6 (a) reveal

that the end of the signal of the 40 x 100 pixel patch resembles the potentiometer’s

measurement more closely. A direct correlation between the two measurements

(Figure 4.6 (b)) shows approximately a straight line with a slope of 1 (black dashed

line). It is also evident that the patch size with the greater Wp-to-Lp ratio correlates

better with the potentiometer’s measurement. Furthermore, in Figure 4.6 (c), which

shows the absolute value of the difference between the two measurements, less than

2% of the signal difference is greater than a pixel size and roughly 90% of the time the

difference is less than half of a pixel size. It can further be observed that the difference

shows distinct evenly-spaced peaks that are highest at the beginning of the signal.

Also, they appear to coincide with the peak amplitudes of the signal. The difference is

likely due to an error in the potentiometer measurement, when the direction of the

displacement changes. Unfortunately, it was not possible to ascertain this claim

59

completely. Despite this uncertainty, the data shows that subpixel-level accuracy is

achievable with high confidence. The actual difference in terms of noise can be

observed in Figure 4.6 (c), between 4 and 6 s of the difference. In addition, it should

be mentioned that the maximum error can further be reduced by increasing the Wp-to-

Lp ratio of the patch. The distribution of the error with mean, and standard deviation,

is shown in Fig. 6 (d). It can be observed from the distribution of the signal

difference that it appears to follow a Normal distribution, as assumed in Section 4.2.

60

Figure 4.6: Comparison of results using independent calibration factors, 40 x 50

pixels (left column) and 40 x 100 pixels (right column): (a) Time history

signals of VVS patch data and potentiometer, (b) correlation between the

two measurements, (c) absolute value of the difference between the two

signals (errors), and (d) histogram of the errors.

61

Figure 4.7 shows the case where the calibration factor for the VVS was

optimized as discussed in Section 4.3.2. Figure 4.7 (a) compares with Figure 4.6 (a)

while correlation plots shown in Figure 4.7 (b) are even better compared to Figure 4.6

(b). The maximum signal difference (Figure 4.7 (c)) is reduced by almost half of a

pixel size as compared to Figure 4.6 (c). Comparing the patch sizes in Figure 4.7 (d), it

can be observed that the standard deviation of the pixel error has been significantly

decreased from 0.23 pixels to 0.12 pixels for the 40 x 50 pixel patch compared to the

40 x 100 pixel patch, respectively. Also, with a confidence of more than 90%, the

error in the smaller patch is less than one third of a pixel size while in the bigger patch

it is less than one fifth of a pixel size. Again, this approach represents the case where

the calibration factor for the VVS sensor was optimized by minimizing the difference

between the two measurements.

In conclusion from Figures 4.6 and 4.7, we have demonstrated that subpixel

accuracy can be achieved with high confidence even without implementing a

computationally-expensive block matching algorithm.

62

Figure 4.7: Comparison of results using calibration factors based on minimized

difference between measurements, 40 x 50 pixels (left column) and 40 x

100 pixels (right column): (a) Time history signals of camera and

potentiometer, (b) correlation between the two measurements, (c)

absolute value of the difference between the two signals (error), and (d)

histogram of the error.

63

Table 4.1 summarizes the main results of the accuracy evaluation presented in

this section. It can be seen that the larger VVS patch (40 x 100 pixels) was closer to

the potentiometer reading compared to the smaller patch (40 x 50 pixels). For the

displacements computed using the independent factors, the larger sized VVS patch

was closer to the potentiometer measurement. However, the standard deviation of the

difference for the larger patch remained the same because of the calibration issues

explained earlier.

Table 4.1: Summary table of accuracy evaluation.

Independent

calibration factors

(see Fig. 5)

Minimization of signal

difference

(see Fig. 6)

Size of the patch 40 x 50 40 x 100 40 x 50 40 x 100

Correlation Coefficient 0.998 1.000 0.998 1.000

Maximum difference in pixel size 1.5 1.2 1.5 0.6

Mean of the difference 0.105 0.007 0.018 -0.014

Standard deviation of the

difference 0.277 0.300 0.236 0.124

4.4.2 Relationship of Noise and Patch Size

Figure 4.8 (d) shows the relationship between patch noise and number of pixels

in a VVS patch as defined theoretically by Equation (4.7) and observed

experimentally. As can be seen in Figure 4.8 (a), the power of the noise is close to the

theoretical values. Figure 4.8 (b) and (c) show the distribution of the noise for one

pixel and a patch of 10 x 10 pixels. As can be observed, the noise in the patch follows

a Normal distribution and its power is one order of magnitude smaller than that for one

pixel. Also, the SNR values approximately change linearly with the width to length

64

ratio of the patch as predicted from Equation (4.7). This validates the theoretical

framework presented in Section 4.2.

Figure 4.8: Noise power and the signal-to-noise ratio (SNR): (a) Power of the noise

vs. the number of pixels in the VVS patch (N), (b) histogram of noise in

one pixel, (c) histogram of noise in a 10 x 10 pixel patch, and (d) the

SNR values vs. width over length of the patch.

4.4.3 Dynamic In-Service Load Test on the Streicker Bridge

In order to evaluate the applicability of the proposed methodology in a real-

world scenario, the Streicker Bridge was tested dynamically with a black-white target

(BWT) to compute displacement. Located on the Princeton University campus, the

bridge has a unique design with a straight main deck section supported by a steel truss

system underneath and four curved ramps leading up to the straight sections, as shown

65

in Figure 4.9. One of the ramps was instrumented with a fiber-optic measurement

system during construction by Br. Branko Glisic from Princeton University [45]. For

the test, we installed an off-the-shelf Canon EOS Rebel T4i camera with a standard

Canon EF 75-30mm zoom lens aimed at one of the ramps, to take a 60 fps video while

a number of volunteers jumped up and down on it. A VVS patch having 60 x 20 pixels

was chosen to compute displacements. As can be seen in Figure 4.9, the VVS is

located at a black-and-white edge on a target mounted to the edge of the bridge slab.

This target was set up by Dr. Maria Feng’s research team from Columbia University,

who collected data for evaluation of their own video-based monitoring methodology

[46]. The relationships in Equations (4.1) and (4.5) were used to compute the actual

vertical dynamic displacement from the collected VVS patch. The calibration

coefficient, was estimated from the target size as 17.3 (pixel/mm); the geometric

correction factor, C estimated to be 1.02.

66

Figure 4.9: Photograph of the Streicker Bridge showing the measurement setup and

the location of the VVS. The insert shows the location of the 60 x 20

pixel VVS patch (red rectangular). The target was installed by Dr. Maria

Feng’s research team from Columbia University [46].

The computed vertical displacement response of the ramp section due to the

described dynamic forcing for a duration of 15 seconds is presented in Figure 4.10. In

the earlier chapter it is already reported that the natural frequencies for this same test

were found to be the same as those measured by the fiber-optic measurement system.

67

Although there were no other physical measurement available to directly compare and

verify our computed displacement, it is comparable in amplitude to what the Columbia

University team reported[46]. Also, the frequency peak is exactly the same as reported

by the same group.

Figure 4.10: Results from the dynamic load test on the Streicker Bridge: (a) Computed

actual vertical displacement time history and (b) frequency response of

signal (a).

4.5 Discussion and Conclusions

The objective of this study was to evaluate the possibility of computing actual

dynamic displacements using Eulerian-based virtual visual sensors (VVS). This is

based on the idea that either an edge of a vibrating structural element or a black-and-

white target (BWT) can be monitored by a patch of pixels. The noise in the VVS

68

sensor was found to be inversely related to the patch size. The following conclusions

can be made from on our study:

The use of BWT allows for accurate computation of dynamic

displacements of a vibrating structural element.

The laboratory tests demonstrated that sub-pixel accuracy can be achieved

similar to block matching algorithms.

The accuracy in the measurement of the displacement implies that change

of intensity is highly sensitive to even tiny amounts of movement, which

results in the fact that natural frequencies can be measured as proposed in

[21], [25], [26] even if the displacement is much less than a pixel size.

The proposed approach also works in the field, as demonstrated by the

measurements of the Streicker Bridge. A direct validation was not possible

since no other physical displacement data was available, which is typically

the case for field measurements. However, the frequency content of the

signal has already been verified and the displacement amplitude as well as

the frequency peak is comparable to what the team from Columbia

University found [46].

The influence of camera movement and changes in lighting conditions

need to be addressed further in future research.

69

MEASUREMENT OF TENSION IN CABLES USING VIRTUAL VISUAL

SENSORS

A total of 48 cable-stayed bridges and 87 suspension bridges [47], [48] are

currently in service in the United States and as Scientific American [49] points out, we

have to expect the number of the cable-stayed bridges to increase in the future. Having

a critical role in transferring the load from the deck to the pylons, cables need to be

monitored over time fir changes in their tensile force. A way to do this non-

destructively is by measuring the natural frequencies of vibration and estimating from

those the tensile force using the well-known taut cable theory [50]. However, the

implementation and maintenance cost of traditional sensors such as accelerometers for

monitoring bridge cables is relatively high. This cost includes, and is not limited to,

the cost of sensor hardware, its installation and wiring, the data acquisition system,

and data storage, management, and interpretation. As shown in the previous chapters,

standard inexpensive digital video cameras can be used to measure natural frequencies

employing a simple algorithm with high accuracy and low computational cost. In this

chapter, the theories that relate the tensile force to the natural frequencies of a cable

are briefly explained. Controlled laboratory experiments were conducted to investigate

the above-mentioned theory and the capability to measure the natural frequencies of

vibration. Finally, the tensile forces of a cable in a cable-stayed bridge was estimated.

Chapter 5

70

It is demonstrated that digital video cameras can be successfully used to measure

natural frequencies and hence estimate tensile forces of in-service stay cables.

5.1 Introduction

Several structure types including long-span bridges such as stay-cable and

suspension bridges, lift bridges, and communication antenna towers employ cables for

load transfer mechanisms. Relaxation of the tensile force in the cables can cause

critical stresses in other parts of the structure and lead to damage. Playing a critical

role in these types of lifeline structures, cables need to be monitored using non-

destructive approaches to estimate their tension force. Vibration-based methods, which

use natural frequencies of vibration to calculate the tensile force of cables have been

developed. The theory of inextensible cables goes back to the 1730s [51]. Irvine and

Clough refined the linear theory of free vibration in cables by taking into account their

extensibility [51]. Shinke et al. proposed simplified formulae to estimate the tension

force using lower modes of vibration considering the cable’s flexural rigidity [52].

Experimental, numerical, and heuristic methods have been widely investigated to

study various parameters such as flexural rigidity, sag, and the boundary conditions on

the natural frequencies of the cables [53]–[63].

Measurement of natural frequencies in cables are usually done through

acceleration response measurements. This requires that a number of accelerometers

are attached to the cables and connected to a data acquisition system that records the

data. The cost of the instruments and their installation and maintenance cost are

significantly higher compared to the cost of a digital video camera. Also, finding the

right location to get the best signal to noise ratio (SNR) because of the high local

sensitivity of these sensors is another issue. Digital videos have been successfully used

71

for the measurement of displacement fields. Block matching algorithms such as digital

image correlation (DIC) replace the cost of traditional sensors but come with high

computational cost [43], [44]. This research has demonstrated that small changes of

color intensity in each pixel of a video carries essential information about the

vibrations of a structural element. In the previous chapters it has been shown that by

implementing Eulerian-based virtual visual sensors (VVS), measurement of

frequencies even with a subpixel amount of displacement is feasible [21], [24]–[27].

In this chapter, the linear theory of taut cables is reviewed and a practical

formula that is applied to correlate natural frequencies to the tensile force is utilized.

Laboratory experiments that are conducted for this correlation and the basic

observations in these experiments are discussed in Section 5.3. Finally, the Indian

River Inlet Bridge was used as a real-world application example. First, the natural

frequencies for three cables are compared with accelerometer measurements from

previous tests. Based on the discussed formulation in Section 5.2, a tensile force was

estimated, which is compared against the earlier measured values made available by

Prof. Tripp Shenton and his team [64].

5.2 Theory

In the previous chapters it has been shown that the selection of the VVS pixels

should be at locations where the gradient of intensity is high, which is typically along

the edges of the structural element being monitored. The greater the gradient, the

better will be the signal-to-noise ratio (SNR).

Based on the linear-elastic theory of cables, several equations have been

developed to give a better approximation of their dynamic behavior taking into

account the axial and flexural rigidities and the effect of the existing boundary

72

conditions [51]–[61], [65]–[71]. Since the focus of this dissertation is on the

development of Eulerian-based VVS to measure vibrations, and not on advancing the

theory of vibrating cables, the effects of flexural rigidity and boundary conditions

were neglected. The behavior of the cable for such conditions can be approximated

using the classical wave equation:

𝑇𝑑2𝑦

𝑑𝑥2= 𝑚

𝑑2𝑦

𝑑𝑡2 (5.1)

in which T is the tension in the cable, y is the deflection of the cable, x is the length

variable along the cable, m is the mass per unit length of the cable, and t is the time

variable. Separating the variables and enforcing pinned boundary conditions, we

obtain:

𝜔𝑘 =𝜋𝑘

𝑙√

𝑇

𝑚 (

𝑟𝑎𝑑

𝑠) 𝑜𝑟 𝑓𝑘 =

𝑘

2𝑙√

𝑇

𝑚 (𝐻𝑧) (5.2)

where 𝜔𝑘 and 𝑓𝑘 are the kth natural frequencies with l being the length of the cable.

From Equation (5.2) we can now solve for the cable tension force, Tfk:

𝑇𝑓𝑘=

4𝑚𝑙2𝑓𝑘2

𝑘2 (5.3)

Equation (5.3) gives the tension force with the assumption that the flexural rigidity

and the boundary conditions have no effect on the behavior of the cable. It is

important to note that an error in the measured frequency results in an error in the

calculation of the tensile force. Using the notation 𝑇𝑓𝑘+Δ𝑓 =4𝑚𝑙2(𝑓+Δ𝑓)2

𝑘2 and by

neglecting the second order term (Δ𝑓

𝑓)

2 𝑇𝑓𝑘

, it can be seen that:

𝑇𝑓𝑘+Δ𝑓

𝑇𝑓𝑘

= (𝑓 + Δ𝑓

𝑓 )

2

= 1 + 2Δ𝑓

𝑓+ (

Δ𝑓

𝑓)

2

≈ 1 + 2Δ𝑓

𝑓 (5.4)

73

Equation (5.4) shows that an error in the measurement by Δ𝑓 will result in an error of

2Δ𝑓

𝑓𝑇𝑓𝑘

, which for low frequency measurements can be significant. Hence, it is

advised that the duration of the recording is appropriately long and to take measures as

explained in Chapter 3 to increase the SNR so that the error in the tensile force

calculation is minimized.

5.3 Laboratory Experiments

In order to examine the applicability of the proposed methodology to measure

the natural frequencies of vibration of cables and estimate their tensile force,

preliminary laboratory tests were carried out. A seven-strand, 187 inch (4.75 m) long

and 1/4 inch (9.53 mm) thick cable (Figure 5.1 d)) with a unit mass of 0.08 lb/ft (118

gram/m) was connected to a load cell to measure its tensile force. Also, the cable was

monitored using a GoPro Hero3 camera and an accelerometer to measure its natural

frequencies. The instruments used for this purpose and the test setup are shown in

Figure 5.1. The S-shaped load cell (Figure 5.1 a) is an Omegadyne LC101, capable of

measuring up to 1200 lbs. (5.34 kN), the accelerometer is an Entran EGC sensor

located at 78 ½ inch (2.00 m) from the left end of the cable (Figure 5.1 b), while the

same GoPro camera that was introduced in previous chapters was used for these tests.

The sampling frequency of the accelerometer and the load cell was 100 Hz whereas

the camera was recording at 120 frames per second.

The tensile force was adjusted by tightening the screw hook at the one end of

the cable before each of the eight performed pluck tests. Each time data sets from

camera, accelerometer, and load cell were collected during the pluck test. The natural

frequencies of each data set were then plugged into Equation (5.3) and the respective

74

tension forces computed. The estimated tension force was then compared to the

average measured tensile force in the load cell.

Figure 5.1: Test setup showing: a) load cell: Omegadyne LC101, b) accelerometer:

Entran EGC, c) camera: GoPro Hero 3, and d) cable.

Figure 5.2 shows samples of typical accelerometer, camera (or VVS), and load

cell data. As evident from Figure 5.2 a) and b), the VVS measured frequencies are

close to those measured by the accelerometer and, in fact, the camera was able to

measure more natural frequencies. This may be because the location of the

accelerometer is close to mid-span coinciding to a node for the non-symmetrical mode

shapes, which results in a small displacement amplitude, or it may be an error in the

VVS because of the nonlinearity issue discussed in Chapters 2 and 3. Also, it can be

seen from Figure 5.2 c) that the load cell data contains the same first natural

frequency, which implies that the invariability of the tensile force with respect to time

is not a true assumption.

75

All eight measurements are summarized in Table 5.1, which shows in each row

the measured natural frequencies using the accelerometer, camera, and the load cell

data. The Δ𝑓

𝑓=

|𝑓𝑎𝑐𝑐−𝑓𝑐𝑎𝑚|

𝑓𝑎𝑐𝑐 also gives a measure of error between the two

measurements. On the other hand, Table 5.2 shows the estimated tensile force using

the natural frequency data from Table 5.1.

76

Figure 5.2: Samples of typical data: a) FFT of acceleration, b) FFT of VVS, c) FFT

of load cell, and d) tension force vs. time.

77

Table 5.1: Measured natural frequencies in Hz using the accelerometer and the

camera, and Δ𝑓

𝑓 as a measure of error. **

Test Mode 1 2 3 4 5

1

Accelerometer 3.10 6.15

Camera 3.10 6.18

Δ𝑓

𝑓 0.00 0.00

2


Camera 4.10 9.63 14.43

Δ𝑓

𝑓 0.01 0.19*

3


Camera 5.39 10.83 21.67

Δ𝑓

𝑓 0.01 0.05

4


Camera 6.06 11.95 18.24 24.36

Δ𝑓

𝑓 0.00 0.01

5


Camera 7.46 14.90 22.24 29.80 37.12

Δ𝑓

𝑓 0.00 0.01

6


Camera 7.61 15.20 22.87 30.36 37.92

Δ𝑓

𝑓 0.00 0.01

7

Accelerometer 8.10

Camera 8.10 16.13 24.18 32.30

Δ𝑓

𝑓 0.00

8


Camera 8.84 17.63 26.70

Δ𝑓

𝑓 0.00 0.00

*Maximum Δ𝑓

𝑓

** Unobserved natural frequencies are left blank

78

Table 5.2: Range of load cell values and their average vs. the calculated tension

based on natural frequencies measured by accelerometer and the camera:

Load Cell (lb) Accelerometer (lb) Camera (lb)

Test Min Max Average Average Average

1 19.32 26.59 22.94 22.78 22.91

2 35.95 44.26 39.54 39.78 42.22

3 65.04 77.51 69.56 67.05 69.87

4 89.99 98.30 92.83 86.39 87.50

5 144.03 149.22 146.72 132.09 132.24

6 151.30 156.50 153.94 137.65 138.07

7 172.09 177.30 174.85 156.77 155.80

8 209.50 214.70 212.31 186.72 187.22

Figure 5.3 a) shows the average tension force computed from the natural

frequency data of the accelerometer and the VVS vs. the range and average values in

the load cell (the minimum and maximum values of the load cell). As can be seen

from Figure 5.3, the calculated values from the natural frequencies obtained using the

VVS and the accelerometer data match well. Also, as shown in Figure 5.3 b) the

calculated values of tension based on natural frequency data have an error of less than

12 percent.

79

Figure 5.3: Lab test results a) The minimum, maximum, and average tension force in

the cable measured by the load cell (Min LC, Max LC and Mean LC) vs.

the average calculated values of tension based on natural frequencies of

accelerometer (Mean Acc) and camera (Mean Cam), b) The same values

divided by the average load cell values.

5.4 Real World Applications

The Indian River Inlet Bridge located on the Delaware coast is one of the most

densely-instrumented bridges in the country. This bridge has a structural health

80

monitoring (SHM) system consisting of more than 100 sensors, which include strain

gauges, accelerometers, and inclinometers to name only a few types.

The load of the deck of the bridge is transferred using four pylons, which have

19 cables on each side of them. The accelerometers are located at three out of four

pylons, quarter points of each span, and on 11 cables [72]. The purpose of the

accelerometers on the cables was to monitor the tension loss in the cables and the

force time-history during extreme events such as hurricanes. On May 11th 2016, a

pluck test was performed on the cables as shown in Figure 5.4.

Figure 5.4: Typical pluck test on a cable of the Indian River Inlet Bridge a) The

crane, b) moving the cable to initiate free vibration.

Using the GoPro Hero camera that was used in all of our studies, the natural

frequency of a cable was measured and is in accordance with the measurement found

in [64]. For this cable , which is numbered 219 E, the acceleration data was recorded

81

during the test. As can be observed from Figure 5.5 b), the natural frequencies were

approximately the same. From the data in [64], it can be concluded that the tension

force in Cable 219 E is 1356 ± 68.94 kips. The target tension force for this cable by

the end of construction was estimated to be 1438 kips, while the estimated force based

on the pluck test on May 5th 2013 was 1347 kips. Given the uncertainty in our

estimation based on VVS, it can be concluded that the tension force has not changed

between 2013 and our recent measurement.

Figure 5.5: Measured natural frequency for the cable 219 E

5.5 Conclusion, Future Work

The methodology introduced in Chapter 2 has been successfully applied to

measure natural frequencies of cables and estimate their tension force. As previously

mentioned, the advantage of a camera over accelerometers is that it covers a large area

across the element of interest and hence might allow to capture a better range of

natural frequencies since one does not need to be worried about the proximity of the

sensor to a node. On the other hand, the sensitivity of the accelerometer is higher and

independent of weather and lighting conditions. The proposed methodology provides

82

the opportunity for everyone to measure natural frequencies of cables using their

point-and-shoot camera or their cell phones given that the used device has a sufficient

frame rate. It also makes it possible to analyze existing digital videos, as was

demonstrated in Section 2.4.

83

SUMMARY AND OVERALL CONCLUSIONS, FUTURE WORK

In this work the general concept of Eulerian Virtual Visual Sensors (VVS) is

proposed and studied. It is shown in Chapter 2 that measurement of the first natural

frequency is possible by observing the minute change in the intensity values through

time and taking it to frequency domain using the Fast Fourier Transform (FFT). It is

shown however that nonlinear change of intensity with motion will cause higher

harmonics in frequency domain as presented in Equation (2.5). In Chapter 3 the SNR

issue, the quantization noise and the intensity-displacement correlation were

discussed. Methods to tackle each problem to increase the SNR in measurement of

natural frequencies were proposed and compared. In Chapter 4 application of new

black and white target is considered to give a measure of the accuracy and sensitivity.

It is revealed through the laboratory experiments that with a high confidence level,

subpixel displacement accuracy is achievable. The theory in Chapter 4 was developed

to give measures for the patch size based on the desired confidence level of accuracy

and SNR. Through the findings of Chapter 4 it is implied that in order to achieve the

subpixel sensitivity in measurement of natural frequencies it is best practice to choose

places where displacement is in the same direction of maximum spatial gradient of

intensity ∆𝐼 = (𝐼𝑚𝑎𝑥 − 𝐼𝑚𝑖𝑛). Finally, in Chapter 5 an application of the proposed

methodology to monitor tension forces in cables is discussed. It is shown through the

laboratory experiments that VVS can be a feasible alternative to the accelerometers in

Chapter 6

84

natural frequency measurements. The effect of error in measurement of natural

frequency on the calculation of the tensile force is briefly discussed. Also for the pluck

tests on the Indian River Inlet Bridge, for one cable it is shown that the measured

natural frequency is close to the one independently measured by the accelerometers in

the health monitoring system.

The proposed methodology can be simply used for measurement of natural

frequencies or measurement of displacement with subpixel accuracy level using the

discussed targets. In the future, a combination of this method and block matching

algorithms could be used in a multiresolution displacement measurement scenario.

Since the proposed method works well in measurement of pixel-level displacement

with almost no computational cost, in conjunction with a block matching algorithm it

can give highly accurate displacement measurements in a significantly lower amount

of time.

85

[1] A. Alvandi and C. Cremona, “Assessment of vibration-based damage

identification techniques,” J. Sound Vib., vol. 292, no. 1–2, pp. 179–202, Apr.

2006.

[2] A. Bagheri, G. Ghodrati Amiri, and S. A. Seyed Razzaghi, “Vibration-based

damage identification of plate structures via curvelet transform,” J. Sound Vib.,

vol. 327, pp. 593–603, 2009.

[3] E. P. Carden and P. Fanning, “Vibration Based Condition Monitoring: A

Review,” Struct. Heal. Monit., vol. 3, no. 4, pp. 355–377, Dec. 2004.

[4] A. Deraemaeker and K. Worden, New trends in vibration based structural

health monitoring. 2012.

[5] S. W. S. Doebling, C. R. C. Farrar, M. B. M. Prime, and D. W. D. Shevitz,

“Damage identification and health monitoring of structural and mechanical

systems from changes in their vibration characteristics: a literature review,”

1996.

[6] C. R. Farrar and K. Worden, “An introduction to structural health monitoring.,”

Philos. Trans. A. Math. Phys. Eng. Sci., vol. 365, no. 1851, pp. 303–315, 2007.

[7] M. I. Friswell, “Damage identification using inverse methods.,” Philos. Trans.

A. Math. Phys. Eng. Sci., vol. 365, no. 1851, pp. 393–410, 2007.

[8] C. Fritzen, “Vibration-based structural health monitoring–concepts and

applications,” Key Eng. Mater., vol. 294, pp. 3–18, 2005.

[9] D. Montalvao, N. Maia, and A. Ribeiro, “A review of vibration-based structural

health monitoring with special emphasis on composite materials,” Shock Vib.

Dig., vol. 38, no. 4, pp. 295–324, 2006.

[10] W. Ostachowicz and A. Güemes, New trends in structural health monitoring.

2013.

REFERENCES

86

[11] A. Pandey, M. Biswas, and M. Samman, “Damage detection from changes in

curvature mode shapes,” J. Sound Vib., vol. 145, pp. 321–332, 1991.

[12] O. S. Salawu, “Detection of structural damage through changes in frequency: a

review,” Eng. Struct., vol. 19, no. 9, pp. 718–723, Sep. 1997.

[13] Z. Sun and C.-C.-C. Chang, “Vibration based structural health monitoring:

Wavelet packet transform based solution,” Struct. Infrastruct. Eng., vol. 3, no.

4, pp. 313–323, Dec. 2007.

[14] Y. Zou, L. Tong, and G. P. Steven, “Vibration-Based Model-Dependent

Damage (Delamination) Identification and Health Monitoring for Composite

Structures — a Review,” J. Sound Vib., vol. 230, no. 2, pp. 357–378, Feb. 2000.

[15] S. Patsias and W. J. Staszewskiy, “Damage Detection Using Optical

Measurements and Wavelets,” Struct. Heal. Monit., vol. 1, no. 1, pp. 5–22, Jul.

2002.

[16] J. J. Lee and M. Shinozuka, “Real-Time Displacement Measurement of a

Flexible Bridge Using Digital Image Processing Techniques,” Experimental

Mechanics, vol. 46, no. 1. pp. 105–114, 2006.

[17] R. Zaurin and F. Necati Catbas, “Structural health monitoring using video

stream, influence lines, and statistical analysis,” Struct. Heal. Monit., vol. 10,

no. 3, pp. 309–332, Jun. 2010.

[18] M. Fraser, A. Elgamal, X. He, and J. Conte, “Sensor network for structural

health monitoring of a highway bridge,” J. Comput. Civ. Eng., no. February, pp.

11–24, 2010.

[19] C. C. Ciang, J.-R. Lee, and H.-J. Bang, “Structural health monitoring for a wind

turbine system: a review of damage detection methods,” Meas. Sci. Technol.,

vol. 19, no. 12, p. 122001, Dec. 2008.

[20] Y.-Z. Song, C. R. Bowen, A. H. Kim, A. Nassehi, J. Padget, and N. Gathercole,

“Virtual visual sensors and their application in structural health monitoring,”

Struct. Heal. Monit., vol. 13, no. 3, pp. 251–264, Feb. 2014.

[21] T. Schumacher and A. Shariati, “Monitoring of structures and mechanical

systems using virtual visual sensors for video analysis: fundamental concept

and proof of feasibility.,” Sensors (Basel)., vol. 13, no. 12, pp. 16551–64, Jan.

2013.

87

[22] H. Wu, M. Rubinstein, and E. Shih, “Eulerian video magnification for revealing

subtle changes in the world,” 2012.

[23] J. Chen, N. Wadhwa, and Y. Cha, “Structural modal identification through high

speed camera video: Motion magnification,” Top. Modal Anal. I, vol. 7, no.

191–197, 2014.

[24] A. SHARIATI and T. SCHUMACHER, “SHM Using Eulerian-based Virtual

Visual Sensors: Introduction of a New Black-and-white Target for Improved

SNR,” in Structural Health Monitoring 2015, 2015.

[25] A. Shariati and T. Schumacher, “Oversampling in virtual visual sensors as a

means to recover higher modes of vibration,” in AIP Conference Proceedings

(Proceedings of QNDE 2014, July 20-25, Boise, ID.), no. Dic, pp. 1–7.

[26] A. Shariati, T. Schumacher, and N. Ramanna, “Eulerian-based virtual visual

sensors to detect natural frequencies of structures,” J. Civ. Struct. Heal. Monit.,

vol. 5, no. 4, pp. 457– 468, 2015.

[27] A. Shariati, T. Schumacher, and N. Ramanna, “Exploration of Video-Based

Structural Health Monitoring Techniques.”

[28] M. Friswell, “Damage identification using inverse methods,” Phylo Trans. R.

Soc. A …, no. December 2006, pp. 393–410, 2007.

[29] T. Duffey and S. Doebling, “Vibration-based damage identification in

structures exhibiting axial and torsional response,” Trans. …, 2001.

[30] K. D’Souza and B. Epureanu, “Damage detection in nonlinear systems using

system augmentation and generalized minimum rank perturbation theory,”

Smart Mater. Struct., vol. 14, pp. 989–1000, 2005.

[31] C. Farrar and K. Worden, “An introduction to structural health monitoring,”

Philos. Trans. …, vol. 365, no. 1851, pp. 303–315, 2007.

[32] C. Farrar and N. Lieven, “Damage prognosis: the future of structural health

monitoring,” … R. …, vol. 365, no. 1851, pp. 623–632, 2007.

[33] J. E. Mottershead and M. I. Friswell, “Model Updating In Structural Dynamics:

A Survey,” J. Sound Vib., vol. 167, no. 2, pp. 347–375, Oct. 1993.

[34] D. Ribeiro, R. Calçada, R. Delgado, M. Brehm, and V. Zabel, “Finite-element

model calibration of a railway vehicle based on experimental modal

parameters,” Veh. Syst. Dyn., vol. 51, no. 6, pp. 821–856, Jun. 2013.

88

[35] Y. Song, C. Bowen, H. Kim, and A. Nassehi, “Virtual Visual Sensors and Their

Application in Structural Health Monitoring,” urmassn11.iids.org.

[36] G. Strang, Linear Algebra and Its Applications. Harcourth Brace Jovanovich,

1988.

[37] G. Strang, Introduction to Linear Algebra, Second Edition. Wellesley-

Cambridge Press, 1993.

[38] “Convert RGB image or colormap to grayscale - MATLAB rgb2gray.”

[Online]. Available:

http://www.mathworks.com/help/images/ref/rgb2gray.html. [Accessed: 31-Jul-

2013].

[39] C. E. Shannon, “Communication in the presence of noise,” Proc. IRE, vol. 86,

no. 2, pp. 447–457, Feb. 1949.

[40] P. Keller, “Wind induced torsional fatigue behavior of truss bridge verticals,”

Oregon State University, 2012.

[41] B. Widrow and I. Kollár, “Quantization noise,” Cambridge Univ. Press, 2008.

[42] A. Shariati and T. Schumacher, “Oversampling in virtual visual sensors as a

means to recover higher modes of vibration,” in AIP Conference Proceedings

(Proceedings of QNDE 2014, July 20-25, Boise, ID.).

[43] C. A. Murray, W. A. Take, and N. A. Hoult, “Measurement of vertical and

longitudinal rail displacements using digital image correlation,” Can. Geotech.

J., vol. 52, no. 2, pp. 141–155, Feb. 2015.

[44] J. Peddle, A. Goudreau, E. Carson, and E. Santini-Bell, “Bridge displacement

measurement through digital image correlation,” Bridg. Struct. Des. Constr.,

vol. 7, no. 4, 2011.

[45] B. Glisic, J. Chen, and D. Hubbell, “Streicker Bridge: a comparison between

Bragg-grating long-gauge strain and temperature sensors and Brillouin

scattering-based distributed strain and temperature sensors,” in SPIE Smart

Structures and Materials + Nondestructive Evaluation and Health Monitoring,

2011, p. 79812C–79812C–10.

89

[46] D. Feng, M. Q. Feng, E. Ozer, and Y. Fukuda, “A Vision-Based Sensor for

Noncontact Structural Displacement Measurement.,” Sensors (Basel)., vol. 15,

no. 7, pp. 16557–75, Jan. 2015.

[47] “Category:Cable-stayed bridges in the United States.” [Online]. Available:

https://en.wikipedia.org/wiki/Category:Cable-

stayed_bridges_in_the_United_States.

[48] “Category:Suspension bridges in the United States.” .

[49] A. Nordrum, “The Rise of the Cable-Stayed Bridge,” Scientific American, 20-

Jan-2015. [Online]. Available:

http://www.nature.com/doifinder/10.1038/scientificamerican0215-15.

[Accessed: 04-Jul-2016].

[50] A. Chopra, Dynamics of structures. 2012.

[51] H. M. Irvine and T. K. Caughey, “The Linear Theory of Free Vibrations of a

Suspended Cable,” Proc. R. Soc. London. A. Math. Phys. Sci., vol. 341, no.

1626, pp. 299–315, 1974.

[52] T. Shinke, K. Hironaka, H. Zui, and H. Nishimura, “Practical Formulas for

Estimation of Cable Tension By Vibration Method,” J. Struct. Eng., vol. 122,

no. 6, pp. 651–656, 1996.

[53] J. C. Russell and T. J. Lardner, “Experimental Determination of Frequencies

and Tension for Elastic Cables,” J. Eng. Mech., vol. 124, no. 10, pp. 1067–

1072, 1998.

[54] A. B. Mehrabi and H. Tabatabai, “Unified Finite Difference Formulation for

Free Vibration,” J. Struct. Eng., vol. 124, no. November, pp. 1313–1322, Nov.

1998.

[55] Y. Q. Ni, J. M. Ko, and G. Zheng, “Dynamic Analysis of Large-Diameter

Sagged Cables Taking Into Account Flexural Rigidity,” J. Sound Vib., vol. 257,

no. 2, pp. 301–319, 2002.

[56] R. Geier, G. De Roeck, and R. Flesch, “Accurate cable force determination

using ambient vibration measurements,” Struct. Infrastruct. Eng., vol. 2, no.

January, pp. 43–52, 2006.

90

[57] B. H. Kim and T. Park, “Estimation of cable tension force using the frequency-

based system identification method,” J. Sound Vib., vol. 304, no. 3–5, pp. 660–

676, Jul. 2007.

[58] M. A. Ceballos and C. A. Prato, “Determination of the axial force on stay

cables accounting for their bending stiffness and rotational end restraints by free

vibration tests,” J. Sound Vib., vol. 317, no. 1–2, pp. 127–141, 2008.

[59] G. Ricciardi and F. Saitta, “A continuous vibration analysis model for cables

with sag and bending stiffness,” Eng. Struct., vol. 30, no. 5, pp. 1459–1472,

2008.

[60] R. A. Sousa, R. M. Souza, F. P. Figueiredo, and I. F. M. Menezes, “The

influence of bending and shear stiffness and rotational inertia in vibrations of

cables: An analytical approach,” Eng. Struct., vol. 33, no. 3, pp. 689–695, 2011.

[61] H. Nam and N. T. Nghia, “Estimation of cable tension using measured natural

frequencies,” Procedia Eng., vol. 14, pp. 1510–1517, 2011.

[62] E. Nazarian, F. Ansari, and H. Azari, “Recursive optimization method for

monitoring of tension loss in cables of cable-stayed bridges,” J. Intell. Mater.

Syst. Struct., pp. 1–11, 2015.

[63] W. Pakos, Z. Wójcicki, J. Grosel, K. Majcher, and W. Sawicki, “Experimental

research of cable tension tuning of a scaled model of cable stayed bridge,”

Arch. Civ. Mech. Eng., vol. 16, no. 1, pp. 41–52, 2016.

[64] K. Davidson, “Estimation of tension in stay cables at the Indian River Inlet

Bridge using frequency based methods,” University of Delaware, 2013.

[65] H. Irvine, “Studies in the statics and dynamics of simple cable systems,” 1974.

[66] A. S. K. Kwan, “Simple technique for calculating natural frequencies of

geometrically nonlinear prestressed cable structures,” Comput. Struct., vol. 74,

no. 1, pp. 41–50, 2000.

[67] S. Cho, J. P. Lynch, J.-J. Lee, and C.-B. Yun, “Development of an Automated

Wireless Tension Force Estimation System for Cable-stayed Bridges,” J. Intell.

Mater. Syst. Struct., vol. 21, no. 3, pp. 361–376, 2009.

91

[68] T. Park, D. Hyung Lee, and B. Hwa Kim, “Estimation of tension force in

double hangers by a system identification approach,” Inverse Probl. Sci. Eng.,

vol. 18, no. 2, pp. 197–216, 2010.

[69] S. Cho, J. Yim, S. W. Shin, H.-J. Jung, C.-B. Yun, and M. L. Wang,

“Comparative Field Study of Cable Tension Measurement for a Cable-Stayed

Bridge,” J. Bridg. Eng., vol. 18, no. 8, pp. 748–757, Aug. 2013.

[70] Y. Yu, C. Zhang, X. Zhu, W. H. Kang, X. Mao, and B. Uy, “Design and

Experimental Investigations of a Bridge Cable Tension Monitoring,” vol. 17,

no. 11, pp. 1657–1669, 2014.

[71] M. Triantafyllou and L. Grinfogel, “Natural frequencies and modes of inclined

cables,” J. Struct. Eng., 1986.

[72] H. W. Shenton, M. J. Chajes, and G. R. Wenczel, “Indian River Inlet Bridge

Structural Health Monitoring System.” .

video based structural health monitoring using …

Documents