1

Matrix mirrors – a very short presentation Renato Roque – FEUP, University of Porto This short text presents very briefly Matrix Mirrors project and some of its results. Matrix Mirrors was driven by a Multimedia Master Thesis in FEUP (2007/2009) and it led as well to a photographic/artistic project with the same name. (please see attached documentation on artistic project). We are looking for ideas in order to continue our work. One main vector for this continuation would be to guarantee that it would cover not only technical/ scientific/ image processing issues but it could be as well the basis for the continuation of the photographic/ artistic project, which was an important result of our previous work. Matrix mirrors – project background Since the 1950’s that a number of psychologists and neurologists believe that Humans have developed specialized mechanisms to learn, memorize and recognize faces. They support that this sophisticated learning and recognition process was an essential achievement for the specie’s survival.

Figure 1 – Areas in the brain involved in face recognition process (red striped)

Concepts and theories from the communication area, such as entropy, sparse coding, redundancy and signal/noise relation were adopted to quantify and try to explain a lot of experimental data, related with visual perception, in particular the perception of human faces. In particular redundancy concept has played a central role in this vision. Horace Barlow was the first to draw the importance of such concept for the economy in representation and for the speed in recognition. He believed that a mechanism to identify regularities or redundancies was crucial to optimize this process. Most information in a new face will correspond to information in other faces. It is redundancy which allows unsupervised learning process. On the other hand, to be able to recognize something new, the system must allow comparing what is being observed with what is usually seen. Related to this vision there has been in the computational areas, since the 1990’s, a lot of investigation to use statistical tools for image-vectors, to create automatic recognition applications. There has been a lot of work to develop valid models and efficient algorithms for the processing of natural images and in particular human faces. There are already today, for example, a few commercial products making use of this work, for example to identify a person from a photo or from a video.

2

Matrix mirrors – a photographic project Photography has been associated since its invention to something involving mystery and magic. Its capability of freezing time and space has still today, in spite the mass vulgarization of photos, an aura of witchcraft. Photography has been clearly associated since its invention in the XIXth century with identity. Photographs have been and are still used in all identity documents, like passports and all kinds of cards. Looking at human portraits trough statistic techniques will allow a reflection around human identity and what it means. Concepts such as entropy will allow quantifying answers to questions such as: What is common and what is different in each human face? What new information do we get when we know a new face? The knowledge that it might be possible to use mathematical/image processing tools to get, from a portrait database, a set of abstract components, which would allow us to reconstruct any portrait, within or without the database used, just adding those components in the right percentage, was something completely magic to add to the magic of photography. Still today, this capability is a wonder to us. Thinking that we might have in our brains an equivalent process is still a bigger wonder. Matrix mirrors – some results All computational work which we referred makes use of sophisticated statistical/ data/ image processing techniques like PCA (Principal Component Analysis), ICA (Independent Component Analysis) or NMF (Non negative Matrix Factorization) which allow, using different criteria, decomposing a portrait into a set of components. We could say that these components correspond to portrait’s global abstract features. If we turn them into images they can be seen as ghostly faces. An example can be seen in Figure 2, which shows some of these components, which we calculated from our portrait Data Base (DB) using PCA.

Figure 2 – PCA components - Eigen faces

In our work we used such techniques but from a photography point of view. We wanted to evaluate and compare the capability of such techniques, not to produce a correct machine identification of a portrait, comparing coefficients, not to improve the recognition rate of existing algorithms, but to reconstruct recognizable human portraits, using generic components. We photographed 439 persons from the Oporto University and built a 400 facial portraits 200x200 pixels database. This resolution was imposed by our computer memory, taking into account the huge size of the matrixes to deal with. The other 39 portraits were used as a test set to simulate the behaviour of the system towards new portraits. We calculated statistical components for our database, using PCA, ICA, NMF and PCA+ICA (a new hybrid statistical system).

3

These components can be added to reconstruct, not only the portraits which were used to calculate them, but other new portraits as well, using simple formula like:

∑+=k

kkDC IAII (1)

First we proved that, with the right coefficients, all 400 portraits from the database can reconstructed with perfection and we need even only a few components to perform identification: with an hybrid PCA+ICA system (the most performing) we only need less than 20 components to succeed in identification. We concluded that even new portraits, like the one in Figure 3, which had not been used to calculate the components, can be reconstructed, although the reconstruction is not perfect this time. In spite of the final error, these reconstructed portraits can be recognized once more making use of only about 30 components. Using a questionnaire we demonstrated that with only a few components (about 20 for portraits in the database and 30 for new portraits) we can reconstruct all portraits, with an error low enough to allow recognition. We observed that an error below 4.5% allows nearly 100% recognition rate. Error in reconstruction was measured using Euclidean distance concept. These values were obtained with PCA+ICA hybrid system and with a normalized database, where portraits were normalized to guarantee that both eyes from all portraits are coincident. Other statistical techniques even with a non normalized database show similar results, they only require a slight bigger number of components.

Figure 3 – The reconstruction of a new portrait, which was not in database step by step

Another apparently very significant result was that we observed that a portrait from a specific group, for example a woman, can be reconstructed without any visible difference, using components calculated from a DB of portraits from a different group, for example men. Women can be reconstructed as well as men from a DB of men's portraits. As well an African face can be reconstructed as well as an European face from an European DB. Matrix mirrors – some statistical results Using our portrait database a few statistical portraits could be calculated and some very interesting results were obtained. We calculated namely average, standard deviation (SD), skew and kurtosis portraits for the whole database and for attribute related groups. Figure 4 shows the DB average portrait.

4

Figure 4 – Average portrait for the 400 portraits in the database

Figure 5 shows average and SD portraits for different groups, which we considered in our work.

Figure 5 – Average and SD portraits for different groups

From left to right: whole DB, men, women, 118 men, men aged over 50, men aged less than 30

We observe clearly that on one hand the whole DB statistic portraits appear to be very similar to men's portraits and that is not due to the fact that we have more men than women in our DB, because we obtain the same result with 118 men, the number of women in our DB, as we can see. On the other hand, according to what one might expect, we can observe that each group appears to lead to specific statistical portraits, where one can observe some group's characteristics. But we decided as well to analyse the evolution of statistical portraits with DB dimension, mixing all kinds of portrits: men, women, old and young people. Figure 6 shows the results for average and SD portraits.

5

Figure 6 – Average and SD portraits for different DB dimensions

From left to right: 6, 25, 50, 100, 200 and 400 portraits

It appears to be very relevant to observe that we need only about 50 persons to get a result which is very, very similar with the statistical portraits for the whole DB, which could lead us to think that this might be the humanity's average and SD portraits! Although each group shows to have its statistical specificities, when we mix different portraits, we come very fast to average and SD generic portraits.

Matrix mirrors – some ideas to continue previous work We would like to enlarge our portrait DB, in order to make it equally representative of men and women and of different ethnical origins, allowing this way to validate some of the results which we intuited in our work. Already after completion of the Master Thesis we have been trying some interesting tools to perform automatic image classification - cluster analysis of portraits - based upon portraits dissimilarity. Matrix mirrors – Main references [1] Aapo Hyvärinen. 1999. Fast and Robust Fixed-Point Algo-rithms for Independent Component Analysis. . IEEE Trans-actions on Neural

Networks 10 (3):626-634. [2] Aapo Hyvärinen, Erkki Oja. 2000. Independent Component Analysis Algorithms and Applications. Neural Networks 13 ((4-5) ):411-430. [3] Barlow Horace. 1989. Unsupervised learning. Neural Compu-tation (1):295–311. [4] Bartlett Marian Stewart. 1998. Face image analysis by unsu-pervised learning and redundancy reduction. Ph.D., Univer-sity of California, San

Diego. [5] Bruce Vicky, Young Andy. 1986. Understanding Face Recognition. British Jounal of Psychology (77):305,327. [6] Chih-Jen Lin. 2007. Projected Gradient Methods for Non-negative Matrix Factorization. Neural Computation 19:2756-2779. [7] Draper B., K. Baek, M.S. Bartlett and R. Beveridge. 2003. Recognizing Faces with PCA and ICA. Computer Vision and Image Understanding

Volume 91 (Issues 1-2):115-137 [8] Edelman, S., B. P. Hiles, H. J. Yang, and N. Intrator. 2001. Probabilistic principles in unsupervised learning of visual structure: human data and a

model. Paper read at 15th An-nual Conference on Neural Information Processing Systems (NIPS), Dec 03-08, at Vancouver, Canada. [9] Edelman, S., N. Intrator, and J. S. Jacobson. 2002. Unsuper-vised learning of visual structure. Paper read at 2nd Interna-tional Workshop on

Biologically Motivated Computer Vi-sion (BMCV 2002), Nov 22-24, at Tubingen, Germany. [10] Ekman Paul. 1999. Handbook of Cognition and Emotions. John Wiley and Sons Ltd ed. Vol. Facial Expressions. [1] Lee Daniel, Seung H. Sebastian. 2001. Algorithms for non-negative matrix factorization. Adv. Neural Info in Process-ing Systems 13 556-562. [11] Ming-Hsuan Yang. 2002. Detecting Faces in Images: a Survey. IEEE Transactions on Pattern Analysis and Machine Intel-ligence 24 (1). [12] Puga André. 2001. A computational allegory for V1. 2nd Inter-national Symposium on Image and Signal Processing and Analysis at PULA,

CROATIA. [13] Wendy S. Yambor, Bruce A. Draper and J. Ross Beveridge. 2002. Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection

and Distance Measures. World Sci-entific Press. [14] Xie Xudong. 2006. Face image analysis and its applications. Ph.D., Hong Kong Polytechnic University (Hong Kong), Hong Kong. [15] Xue Yun. 2007. Non-negative matrix factorization for face recognition. Ph.D., Hong Kong Baptist University (Hong Kong), Hong Kong. [16] Zafeiriou, S., A. Tefas, I. Buciu, and I. Pitas. 2005. Class-specific discriminant non-negative matrix factorization for frontal face verification.

Paper read at 3rd International Conference on Advances in Pattern Recognition, Aug 22-25, at Bath, ENGLAND. [17] Zafeiriou, S., A. Tefas, I. Pitas, and Ieee. 2005. Discriminant NMFFaces for frontal face verification. Paper read at IEEE Workshop on Machine

Learning for Signal Processing (MLSP), Sep 28-30, at Mystic, CT. [18] Zhao W, Chellappa R., Phillips P. J., Rosenfeld A. 2003. Face recognition: A literature survey. ACM Computing Surveys 35 (4):399-458