robotic intelligent vision and control for tunnel ...€¦ · interweaving of computer vision and...

Robotic Intelligent Vision and Control for Tunnel Inspection and Evaluation - The ROBO-SPECT EC

Project

Costas Loupos, MEng, MSc, PMP Senior Project Manager Institute of Communication and Computer Systems, Athens, Greece [email protected]

mailto:[email protected]

26 May 2015, Dubrovnik, Croatia Costas Loupos (ICCS)

▪ Smart Integrated Systems Team – Research Fields:

2

• Sensors - signal and image processing

• Communication systems • Computer vision algorithms

(capturing and processing) • Monitoring, control and

automation systems • H/W (digital and analog

electronics)

• S/W engineering and computer technologies

• Control and Robotics • Microwave, MMW and optical

sensors

26 May 2015, Dubrovnik, Croatia Costas Loupos (ICCS)3

▪ Collaborative project (STREP) - FP7 - ICT

▪ THEME 3 - Cognitive Systems and Robotics

▪ Project full title: ROBotic System with Intelligent Vision and Control for Tunnel Structural INSPECTion and Evaluation

ROBO-SPECT


ROBO-SPECT ConsortiumParticipant Organisation Name CountryInstitute of Communications and Computer Systems GreeceCASSIDIAN (AIRBUS) France

University Carlos III, Madrid, Dpt. De Ingeniera de Sistemas Spain

VSH Hagerbach Test Galleries Ltd. SwitzerlandEgnatia Motorway S.A. Greece

Consiglio Nazionale delle Ricerche Italy

RISA GmbH GermanyTECNIC S.p.A. ItalyD. Bairaktaris and Associates Ltd. Greece

Ecole des Ponts-ParisTech France

ROBOTNIK Spain

ROBO-SPECT Associate Partners: LONDON UNDERGROUND PARISSIAN METRO EUROTUNNEL


Need for automated robotic solutions, exploitable in the near to medium term in the field of inspection of the civil infrastructure in general and transportation tunnel infrastructure in particular

The problem (1/2)

INSPECTION

ASSESSMENT

Transportation tunnel

infrastructure is

urgently ageing


Presently, inspection is mostly performed through tunnel wide visual observations by inspectors. This process is:

The problem (2/2)

▪ slow ▪ labor intensive ▪ expensive ▪ subjective ▪ lane shutdown required ▪ safety procedures to be established ▪ human resources needed ▪ empirical and incomplete classification criteria


Robotics

Computer Vision

Sensors

ROBO-

SPECT

7

Proposed solution

Recent and exploitable research

in

ROBO-SPECT is an innovative, integrated, robotic system that, in one pass, will perform inspection and assessment of transportation tunnel linings, minimizing humans’ interaction and has the potential to be commercialized in the short to medium term

scans the intrados for potential defects on the surface

detects and measures r a d i a l d e f o r m a t i o n between parallel cracks


Advanced functionalities of the ROBO-SPECT system will include: ▪ Interweaving of computer vision and sensing techniques with

intelligent control of a multi-degree-of-freedom robot to enable it to provide, speedily, all the required parameters for structural assessment with the required accuracy

▪ Ability to automatically adapt to different use cases ▪ A crack meter sensor for the simultaneous measurement of crack

width and depth and its integration with the robotic arm ▪ A novel quantitative structural assessment tool that, based on the

inspection of the lining intrados, will assess the structural condition of the inspected reinforced concrete tunnel lining

Advanced functionalities


ROBO-SPECT is expected to have a large impact on the robotics market, the tunnel inspection industry and our society:

▪ Increased competitiveness of the tunnel inspection (speed, reliability, working conditions)

▪ Increased passengers safety and reduced tunnels closing time and decreased inspection/assessment costs (also having great financial impact)

▪ Large market for an integrated, automated, robotic system for transportation tunnel inspection and assessment

▪ Components of such a robotic system for the assessment of visible defects will be useful for the inspection of a wealth of additional civil infrastructure systems

Overall objectives


1. To develop an automated robotic inspection and assessment system for tunnels that will include a computer vision system, a sensor system and laser equipment able to measure radial deformation with high accuracies

2. To develop a computer vision system for tunnel inspection and assessment of the structural condition that will detect structural defects and colour changes at the inspected concrete lining intrados

3. To adopt specific methodologies for collecting data. Data collection is an important element in ROBO-SPECT since the quality of this dataset will affect the performance of the developed system.

4. To extend already existing 3D reconstruction tools to precisely detect tunnels’ cracks and use these models at later stages to derive more robust knowledge on tunnels stresses

Scientific & Technical objectives (1/3)


5. To promote a new evaluation framework for tunnels

6. To develop a sensor system suitable for being installed on the robotic inspection system that will measure depth & width of cracks

7. To develop a quantitative structural assessment tool to automatically assess the structural condition and stability of the tunnel at the time of the inspection and at future times

8. To test, validate and benchmark the system at the research infrastructure of the pilots, in terms of its potential take up and operational deployment

Scientific & Technical objectives (2/3)

26 May 2015, Dubrovnik, Croatia

VISUAL DETECTION

•The Curvature Problem: Performance accuracy and computational complexity due to the difficult (i.e., varying) visual conditions in a Tunnel

• The Visibility Problem: There is absence of natural light in tunnels and lighting conditions are very poor

12

ROBOTICS •Attain the expected range with the precision specified •Robotic system able to extend an automated crane at 4 to 7 meter range & an industrial-quality robot manipulator with 1 meter reach approximately, will be selected from the market to be set on the crane

• Achieve harmony between the controllers through the use of the global controller

Project Challenges (1/4)


SENSORS •The Depth Problem: Attainment of the required crack depth measurement accuracy on concrete with high confidence. •Very effective techniques will be needed to compensate for undesired effects in ultrasonic measurements deriving from inhomogeneity of concrete. •Attainment of the required crack width measurement accuracy on concrete with high confidence. Crack width will have to be determined with an accuracy of 0.1 mm. •A two-dimensional positioning system based on piezoelectric actuators that will be setup to this purpose.

13




STRUCTURAL ASSESSMENT • Secure testing that will cover all

the required types of tunnels, damages and operating conditions.

• Produce software that adequately represent the mechanical conditions at the tunnel lining

26 May 2015, Dubrovnik, Croatia 15


INTEGRATION •System integration of the crack measurement system on the robotic platform. The performance achieved in laboratory environment will have to be reproduced on the robotic platform. •Attention in integrating the crack measurement system with to the motion and vision capabilities of the robot. • The solutions developed in the project sufficiently standardized to enable the integration(s) in reasonable conditions, reliability of the integration of the different type of sensors into the Sensing Tip. •To have an integrated, multidisciplinary system function, for the first time, in the real environment that is not regulated and full of lighting and other problems.


ROBO-SPECT Platform


ROBO-SPECT Arm


The automated tunnel inspection system will include:

▪ A robotic platform with navigation and positioning capabilities

▪ An integrated intelligent control component able to: ▪ define the orientation and lighting conditions for the

cameras

▪ command the robotic system so as to receive “good visual features”

▪ Determine the movement step needed to inspect the infrastructure

▪ A computer vision system ▪ state-of-the-art machine learning tools ▪ automatic detection of defective performance of the

lining intrados through visual inspection

Technical Approach - Overview (1/2)


▪ Sensing capability to provide the required measurements in the identified areas of concern (e.g., structural cracks) with the required accuracy

▪ An Expert Module that based on inspection results will automatically assess the structural condition of the tunnel lining through the use of civil engineering algorithms

Technical Approach - Overview (2/2)


Technical Approach - ROBOTICSROBOTIC PLATFORM ▪ Design and assembly of the ROBO-SPECT extended mobile

robotic system ▪ wheeled robotic system able to extend an automated crane

to the lengths commonly found in tunnels (4 to 7 meter range) sustaining a robot manipulator

▪ automated through the use of robotic controllers ▪ mobile robot, automated crane arm, and robot automated

manipulator


▪ Multi View Cameras ▪ 2D and 3D sampling cameras

▪ Deriving of depth and width of a crack as precisely as possible (accuracy of 1mm)

▪ Inverse perspective transformations and 3D reconstruction technologies

▪ Extraction of visual descriptors from the imagery data ▪ Detection of image regions covariant to a class of geometric

distortions which are then used as support regions to compute invariant descriptors

▪ Algorithms: (Scale Invariant Feature Transform (SIFT) , the PCA-SIFT, Histogram of Oriented Gradients (HOG), the Harris affine transform, cross correlation metrics, spin images, etc

Technical Approach – Computer Vision System


▪ Fabrication of Fiber-Optic Ultrasound Sensor Prototypes for the Near-Field Measurements ▪ Silicon micromachining enabling fabrication of fiber-optic

ultrasound sensors (high sensitivity and wide frequency range)

▪ Extremely miniaturized active pixels with diameter around 100 µm, essential for the crack width measurement technique

▪ Crack monitoring system assembled by using commercial piezoelectric ultrasonic transducers (for ultrasound emission and far-field detection) and fiber-optic ultrasound sensors (far and near field detection)

Technical Approach - SENSORS


Use of ultrasounds for crack detection on concrete

Method 1 - ToF Analysis

Method 2 - Rayleigh wave Analysis

Method 3 – Rayleigh wave near-field

Analysis

Technical Approach - SENSORS


System Overview and Operation


▪ Structural Assessment Tool to help tunnel operators establish a planning strategy for lining maintenance ▪ Data Base (DB) to store construction data, information from tunnel

inspections and data about the environment and operation. ▪ User Interface to provide visualization of the results calculated by the

‘Degradation Models,’ ‘Structural Assessment at the Time of the Inspection,’ and ‘Reliability Based Future Structural Assessment’ in a textural and graphical manner

▪ Data will be visualized in a number of ways and through this Tool the user will be able to estimate:

• The internal forces and external loads applied on every point of the lining cross-section

• The structural reliability and probability of local or global failure at the lining cross-section at the time of the inspection.

• The evolution of material damage assessed by algorithms corrected with inspection results

• The probability of structural failure in the future

Technical Approach – STRUCTURAL ASSESSMENT

Thank You!

Costas Loupos, MEng, MSc, PMP Senior Project Manager Institute of Communication and Computer Systems, Athens, Greece [email protected]

mailto:[email protected]

robotic intelligent vision and control for tunnel ...€¦ · interweaving of computer vision and...

Documents