image complexity and manipulation recognition by human ... · the images are separated according to...

Image Complexity and Manipulation Recognition by Human

Presenter: Xiang GuoSupervisor: Professor Tom Gedeon, Dr. Sabrina Caldwell

19/08/2016u5541646@aun.edu.au

Overview- Aim

- Literature Survey

- Tools

- Database

- Design

- Results

- Conclusion and Future Work 2

Aim

- To what extent that people recognise image manipulation

- How would people identify image complexity in limited time and compared with computer of image complexity identification.

- The correlation between image manipulation and complexity

3

Literature Survey

● Issue of image manipulation- Digital images connect with people closely in modern life

- Manipulating digital images is easy by digital image editing tools which are widely available.

- Issues relating to image manipulation appeared in a wide range of areas, like news reporting, medicine, scientific journal and even legal evidences.

4

Literature Survey

● Types of image manipulations:- Copy

- Remove

- Insertion

- Replacement

5

Literature Survey- Copy

6

Literature Survey- Remove

7

Literature Survey- Insertion

8

Literature Survey- Replacement

9

Literature Survey

● Image Complexity- Image complexity is seen differently by computers and humans

- Image complexity recognised by human in this experiment

- Image complexity have the potential to effect the people’s ability to recognise manipulation.

10

Tools

● Eyetribe

Eye tracking is the process of detecting eye features by sensors and estimate where people is looking at.

Eyetribe is a device use infrared illumination and advanced mathematical models to track the eye gaze.

Eyetribe could be used in device control (hands-free typing, aiming in games) and performance analysis (design, layout).

https://theeyetribe.com/ 11

Tools

● gimp & irfanview

Gimp is a powerful, open-source and free raster graphics editor, which provide image, cropping, resizing, pasting, rotating… The software is easy and convenient for both getting started and deeply learning of image manipulation.

Irfanview is one of the most popular image viewers worldwide. The batch functions of this software help a lot in the experiment in rename and reform images from database.

12

Tools

● Mondrian Question Prototype

Mondrian Question Prototype is the display solution for Mondrian Draw project developed by Haolei Ye. The main function of this prototype in this experiment is to display the image and the attached question with options and store experiments results in database.

To modify the prototype, there are two main steps. The first one is to design and modify the display algorithm, which will be presented in Design section. The second one is to modify eyetribe into prototype to enable tracking participants’ eye gaze during experiment.

13

Database

● CASIA 2.0

CASIA 2.0 is a collection of natural color images and manipulate them with realistic tampering methods. The images are separated according to the image contents into 9 categories, which are animal, architecture, article, character, indoor, nature, plant, scene and texture. There are totally 7491 authentic images and 5123 tampered images.

One problem is that only 1952 out of 7491 authentic images are used to generate the 5123 tampered images.

14

Database

● IMCRD

Image manipulation and complexity recognition dataset (IMCRD) is the dataset I created for this experiment.

- 300 pairs of images- one original image and one corresponding manipulated image- 9 categories- name: index_(O/M)_originalName

15

Design● Image Pool

- Each image is view by 6 time, and complexity/manipulation questions are answered for 3 times respectively.

- Each people view equal numbers of original and manipulated images attached with equal numbers of complexity and manipulation questions 16

Design

● Display Algorithm- for 1 <= P <= 6, the index of orignal images plus 25 for each participant,

for 7 <= P <= 12, the index of manipulated images plus 25 for each participant.

- when the index of participant is odd, the questions begin with manipulation

when the index of participant is even, the questions begin with complexity.

17

Results

● 7 out of 12

18

Results

● Particular Examples

Accuracy: 33% Complexity: 76.00 Accuracy: 100% Complextiy: 0.019

Conclusion and Future Work

● Conclusion- It seems that people have higher possibilities to recognise manipulation than

expected for images from CASIA 2.0

- The higher image complexity could possibly raise the uncertainty for people to recognise manipulation.

- The manipulation methods and sizes also play an important role for people to recognise manipulation

20

Conclusion and Future Work

● Future Work- More people from other areas, ages, gender to participant experiment

- May focus on specific categories, like human face

- Improve the design of the display frame work, like add keyboard operations.

- Try other types of presentation of images

- Better control of variables which may effect the experiment results

- Analysis of eye gaze data

21