tensorflow 2.0.0
opencv
hog from skimage.feature
dlib
The age+gender model is trained on Baidu's All-Age-Faces (AAF). link here
The facial expression model is trained on Kaggle dataset FER2013 containing labeled 3589 test images, 28709 train images.
- For Facial Expression model, we use functional API with 4 convolutional layers with ReLU activation and padding 'same' for image input and another separate input layer for HOG (histogram of gradients) & landmarks features. See
FER2013.ipynb
notebook for more details about the model. We don't use transfer learning because our dataset contains greyscale images and doesn't fit in 3 channel pretrained models.
- For Age and Gender We use the two pre-trained models Inception V3 and VGG16 for gender and age detections, respectively.
-
The data from the Kaggle dataset is already split into train and test set. The data is processed from csv file into numpy arrays and fetched into ImageDataGenerator. See jupyter notebook
FER2013.ipynb
for more details. -
Our model achieved ~99% accuracy on train set and ~75% on validation set after 21 epochs
-
We use this paper as benchmark which achieves 75.2% accuracy.
-
The two tables below show training result on 5 expressions done by amineHorseman. It shows that face landmarks and histogram of oriented gradients (HOG) only improves the accuracy by 4.5% at best. On the other hand, batch normalization significantly improves the model performance.
-
Using example code extract face landmarks using dlib shape predictor model
-
Example code get hog features using scikit image hog
-
The Baidu dataset contains 13322 face images (mostly Asian) distributed across all ages (from 2 to 80), including 7381 females and 5941 males. The orignal face images, facial landmarks and aligned face images are stored in folders
original images
,key points
, andaligned faces
. -
We read from the
key points
folder and turn the data into a dataframe consisting of 3 columns:image_name
,gender
andage
. From there we loop through the dataframe and assign the image paths for each correponding row and store them inside sub-folders which serve as the labels. Gender has two labels Male and Female while Age has 5 labels which are the age ranges: 1-15, 16-25, 26-35, 36-45, >46. -
We both go through several pre-trained models such as MobileNet, Inception ResNet V2 and even Levi&Hassner's model, which was specifically trained for age and gender detection, but none seems to fit with the Baidu dataset.
- Our Flask app connects to the webcam which user can use to scan their faces for classification with the output as the user's age, gender and emotion.