advanced practical course: sensor-enabled intelligent environments barcode-based object recognition...

Advanced Practical Course: Sensor-enabled Intelligent Environments

Barcode-based Object Recognition

Final Presentation

Presented by:Nacer KHALIL

Supervised by:Dejan PANGERCIC

1

Table of content

I- Overall project goal

II- Autofocus

III- Bacode decoding

IV- information retrieval

V- Barcode localization

VI- Conclusion

2

II-AutofocusHow autofocus works

Active vs passive autofocus

Courtesy of howstuffworks.com 3

II-Autofocus(continued)

4

II- AutofocusImplementation in the project

Used camera: Logitech QC PRO 9000Driver used: ROS::uvc_cameraProblem: Autofocus is not supported by the driverSolution:

Autofocus was added to uvc_camera driverAutofocus algorithm was taken from GUVCVIEW

software and integrated within uvc_camera driver

5

II- Autofocus result

6

III-Barcode decodingHow Zbar works

Row 1 Row 2 Row 3 Row 40

2

4

6

8

10

12

Column 1

Column 2

Column 3

Courtesy of Jeff Brown7

IV-Information retrieval

Barcoo is a product information store that has a database composed of 7 million commercial objects.

Access to this database was granted to us.Communication to the database is done through

HTTP protocol.Request: an http link containing the barcodeResponse: XML file containing all information about

the object8http://www.barcoo.com

IV- Information retrievalBarcoo request response example

Request: http://www.barcoo.com/api/get_product_complete? Pi=73705207908

&pins=ean&amp ;format=xml&source=ias-tum

Response: We are parsing for:- Image- product name- category- producer

9

V- Barcode localizationTechniques used

Techniques used to find the barcode region of interest– Blob-based barcode localization– Parallel line-based localization– Adjacent line-based localization

10

V- Barcode localizationBlob-based localization(working example)

11

V- Barcode localizationBlob-based localization (not working example)

12

V- Barcode localizationAdjacent line-based localization

13

V-Barcode localizationHow adjacent line-based localization works

14

V-Barcode localizationAdjacent line-based approach explanation

- Take picture-Convert to grayscale-Parameters: interval size, min/max # of transitions, max Jeffrie’s value, min # of rows per ROI

255 15 56 54 84 165 75 0

250 20 60 84 120 0 240 97

248 18 61 0 13 51 15 85

246 17 55 70 55 52 0 200

1 0 2 2 2 2

1 0 1 2 2 2

1 0 2 1 2 2

1 0 2 1 2 2

Image matrix

Transitions matrix

1 0 -1 -1 -1 -1

1 0 1 -1 -1 -1

1 0 -1 1 -1 -1

1 0 -1 1 -1 -1

Eliminated intervals

15

0,2 1 5,2 8,4 5,3 1,3

1,2 2 2,4 2,4 6,7 1

0,5 1 3,2 0,1 8,4 2,4

Jeffrie ’s distance matrix

1 0 -1 -1 -1 -1

1 0 1 -1 -1 -1

1 0 -1 1 -1 -1

1 0 -1 1 -1 -1

0,2 1 -1 -1 -1 -1

1,2 2 0 -1 -1 -1

0,5 1 -1 0,1 -1 -1

Eliminated intervalsmatrix

Final matrix

V-Barcode localizationAdjacent line-based approach explanation (continued)

16

IV- Barcode localizationAdjacent line-based localization - results

17

18

Open Source Code

Packages list:-zbar_barcode_reader_node-zbar_qt_ros-uvc_camera-barcode_detection

Repositories:-http://code.cs.tum.edu/indefero/index.php//p/seie2011fall/source/tree/HEAD/khalil-http://code.cs.tum.edu/indefero/index.php//p/ias-perception/source/tree/master/

ConclusionProject is composed of three parts:

Barcode localizationImplementation of autofocusInformation retrieval of objects

Future work:Creation of the barcoo ontology and storage on

KnowRobIntegration and testing on PR2Integration with object modeling center

19

20

Demonstrations of the project in the kitchen lab after the presentations end