Research lines:

Social Trait Prediction from Facial Images

Humans constantly evaluate the personalities of other people using their faces. Facial trait judgments have been studied in the psychological field, and have been determined to influence important social outcomes of our lives, such as elections outcomes [1,2] court room decisions [3], and social relationships.
Recent work on textual descriptions of faces has shown that trait judgments are highly correlated. It has been shown from behavioral studies that two orthogonal dimensions, valence and dominance, can describe the basis of the human judgments from faces [4].
In this project, we use the knowledge acquired in the Psychology field to build computational algorithms that can predict social trait judgments from facial images [5,6]. In collaboration with psychologists, we try to build efficient feature extraction algorithms in order to train robust machine learning classifiers that can perform the automatic facial trait classification.
We also focus on finding the correlation between the structural relationships of fiducial facial key points and the facial trait judgments. We are especially interested in which regions/parts of the face lead our facial evaluations.

[1] Ballew CC, Todorov A (2007) Predicting political elections from rapid and unreflective face judg- ments. Proceedings of the National Academy of Sciences of the USA 104: 17948-17953
[2] Little A, Burriss R, Jones B, Roberts S (2007) Facial appearance affects voting decisions. Evolution and Human Behavior 28: 18–27
[3] Blair I, Judd C, Chapleau K (2004) The influence of Afrocentric facial features in criminal sen- tencing. Psychological Science 15: 674–679.
[4] Oosterhof N, Todorov A (2008) The functional basis of face evaluation. Proceedings of the National Academy of Sciences 105: 11087.
[5] M.Rojas, D.Masip, J.Vitrià. “Predicting Dominance Judgments Automatically: A Machine Learning Approach”. IEEE International Workshop On Social Behavior Analysis 2011, (in conjuntion with FG 2011). 21st March 2011. Santa Barbara, CA.
[6] M.Rojas, D.Masip, A.Todorov, J.Vitrià. “ Automatic Point-based Facial Trait Judgments Evaluation”. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2010. 13-18 June 2010. Page(s):1063-6919. ISSN: 1063-6919, ISBN: 978-1-4244-6984-0. San Francisco, CA.
[7] David Masip, Michael S. North, Alexander Todorov, Dan N. Osherson. Automated Prediction of Preferences Using Facial Expressions. PLoS ONE 9(2): e87434. doi:10.1371/journal.pone.0087434. February 2014.
[8] Mario Rojas Quiñones, David Masip, Alexander Todorov, Jordi Vitrià Marca. Automatic Prediction of Facial Trait Judgments: Appearance vs. Structural Models.PLOS One.6 - 8,pp. 1 - 12.PLOS,14/07/2011. ISSN 1932-6203.



Facial Expression Analysis

Facial expression analysis provides an important cue of information about emotional states. The computer vision literature usually focused on the six basic emotions defined by Ekman (Anger, disgust, fear, happiness, sadness, surprise). Nevertheless, there exist multiple facial expressions that provide us useful information on many other human activities. The analysis of these spontaneous expressions could help us to build more user-friendly human-computer Interfaces. In this research line, we propose to automatically weight the activity of facial muscles obtaining the action units score (AUs). Based on this basic action units we could predict spontaneous in a higher level classification stage. In the preliminary work, we focused on the face detection and robust location of fiducial key points from faces.



[1] M. Rojas, D. Masip and J. Vitrià. "Automatic Detection of Facial Feature Points via HOGs and Geometric Prior Models". In proceedings of the fifth Iberian Conference on Pattern Recognition and Image Analysis, IbPRIA 2011 J. Martí et al. (Eds.). Lecture Notes in Computer Science Pattern Recognition and Image Analysis, June 2007, pp. Springer-Verlag Berlin Heidelberg
[2] David Sanchez-Mendoza, David Masip, Agata Lapedriza. Emotion recognition from mid level features. Pattern Recognition Letters. Vol 67 Issue P1, 66-74. Elsevier, North Holland, ISSN 0167-8655. December 2015.



Feature Extraction

Traditionally, linear feature extraction algorithms are focused on finding the projection matrix to a low dimensional space that allows best discrimination among data classes. In the classic FLD (Fisher Linear Discriminant Analysis) Gaussian assumptions are performed in the intra and extra class scatter matrices. In addition, the dimensionality of the reduced space is upper bounded by C-1 (being C the number of classes). Several approaches have been proposed to overcome these drawbacks. In this research line, we propose the use of the Adaboost algorithm to find the embedding to the low dimensional space.
Three different versions of the main technique have been proposed and extended to the multiclass case, and no assumptions on the class distribution of the data have been performed.





[1] David Masip, Jordi Vitrià. "Boosted Discriminant Projections for Nearest Neighbor Classification". Pattern Recognition. Vol 39, n. 2, 164-170. 2006. Elsevier Science.
[2] David Masip, Ludmila I. Kuncheva ,Jordi Vitrià, "An ensemble-based method for linear feature extraction for two class problems". Pattern Análisis and Applications. vol 8, n.3, 227-237, 2005. Springer Ed.
[3] David Masip, Jordi Vitrià. “Shared Feature Extraction for Nearest Neighbor Face Recognition”. IEEE Transactions on Neural Networks. Vol 19, n. 4, 586-595. April 2008.



Machine Learning: Classifier Ensembles

Classification problems are one of the most important research areas in machine learning. We developed a new classification scheme based on an ensemble of separating hiperplanes. The local classifiers are generated using the notion of boundary and its characterizing points. We build locally optimal classifiers and combine them by means of a Tikhonov regularization framework generating a robust classification rule. The algorithm has been validated on 16 UCI baseline problems and 6 different applications.
In this research line we are currently working on extending the algorithm to the multiclass case, improving the training computational cost by building an incremental implementation of the boundary points construction. In addition we are designing a hash function to perform dimensionality reduction.

[1] Oriol Pujol, David Masip. “Geometry-Based Ensembles: Toward a Structural Characterization of the Classification Boundary”. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol 31, Issue 6, 1140-1146. July 2009



Vehicle Routing Problem

Road transportation is nowadays the most extended way of transporting goods along the world. The costs associated to road transportation of products and logistics are increasing each day due to the costs of oil. Moreover, the transportation costs have also a strong impact in our lives in terms of pollution, health and ecology.
In this research line, we propose different meta heuristics to tackle the Capacitated Vehicle Routing Problem (CVRP), mostly based on the classic Clarke and Wright savings algorithm. We are designing robust methods that can yield fast and satisfactory solutions (close to optimal) to the CVRP and the different commercial variants of the problem (time-windows, pick up and delivery, ...).

[1] Juan, AA and Faulin, J. and Jorba, J. and Riera, D. and Masip, D. and Barrios, B. “On the use of Monte Carlo simulation, cache and splitting techniques to improve the Clarke and Wright savings heuristics”. Journal of the Operational Research Society. July 2010
[2] Calvet, L.; Ferrer, A.; Gomes, I.; Juan, A.; Masip, D. : “Combining Statistical Learning with Metaheuristics for the Multi-Depot Vehicle Routing Problem with Market Segmentation”. Computers and Industrial Engineering. April 2016. Vol 94., pp 93-104. ISI JCR IMPACT FACTOR: 1.783 (2014), 2nd quartile; 2014 SJR = 1.583, Q2). ISSN: 0360-8352.

Visual Object Recognition





Visual object recognition might be one of the most challenging problems in the computer vision field. We are developing algorithms applied to object categorization, and automatic detection. Specifically, we participated in the last PASCAL VOC Challenge, in the person layout taster competition.



Face Recognition

In this research line we develop algorithms for face recognition in natural and uncontrolled environments.
The face recognition problem usually involves the use of dimensionality reduction and feature extraction algorithms. In this project we developed methods to extract invariant information of faces along changes in illumination, pose or expressions [2,3,6,7].
In addition, people’s appearance changes as time goes by, online learning features have been explored to: (i) increase the number of samples in the training data, (ii) add new classes to the data base of target subjects, and (iii) evolve and forget the decision rules to update the models representing each person to verify [1].
Usually, only internal information of facial images has been used in state-of-the-art face classification schemes. We also explored the use of the external face information (such as hair, jaw line or ears) as a complementary information cue to improve the face classification done using the internal facial features.



[1] David Masip, Àgata Lapedriza, Jordi Vitrià. “Boosted Online Learning for Face Recognition”. IEEE Transactions on Systems Man and Cybernetics part B. Vol 39, Issue 2, 530-538. April 2009.
[2] Matthias S. Keil, Àgata Lapedriza, David Masip, Jordi Vitrià. “Preferred spatial frequencies for human face processing are associated with optimal class discrimination in the machine”. PLoS ONE, Public Library of Science. PLoS ONE 3(7): e2590 doi:10.1371/journal.pone.0002590.
[3] David Masip, Jordi Vitrià. “Shared Feature Extraction for Nearest Neighbor Face Recognition”. IEEE Transactions on Neural Networks. Vol 19, n. 4, 586-595. April 2008.
[4] Àgata Lapedriza, David Masip, Jordi Vitrià. “On the Use of External Features for Face Verification”. Journal of Multimedia (JMM). Acedemy Publisher. Vol 1, n. 4 July 2006.
[5] Àgata Lapedriza, David Masip, Jordi Vitrià. "The Contribution of External Features to Face Recognition", In Proc. 2nd Iberian Conference on Pattern Recognition and Image Analysis. J.S. Marques et al. (Eds.). Lecture Notes in Computer Science 3523. Pattern Recognition and Image Analysis, June 2005, pp. 537-544. Springer-Verlag Berlin Heidelberg. ISBN 0302-9743.
[6] David Masip, Marco Bressan, Jordi Vitrià. "Feature extraction for real time face detection and classification". EURASIP JASP journal on Applied Signal Processing. Vol 13, 2061-2071. 2005. Hindawi Publishing Corporation.