Gender determination from periocular images using deep learning based EfficientNet architecture

In the beginning, the images of the eye taken in Visible Spectrum (400nm–700nm) are preprocessed. Preprocessing helps us to better handle the images by bringing them to the same size and format. The images are resized as 75 × 75 size pixel images without affecting the aspect ratio of the images using bilinear interpolation. The images are changed into grayscale (or graylevel) image as minimum information is offered for each pixel. This reduces the computational requirements. To avoid the possibility of exploding gradients and improve the convergence speed, the images are normalized by dividing by 255. Figure 2 shows the general CNN architecture.

Fig. 2

The normalized images are used as the input for the CNN feature extraction module. The CNN model is built from scratch. The training dataset is divided into batch sizes of 10. Batch size is the number of data points taken at a time to train the model. The smaller batch size makes the learning faster and more efficient. It has 2 convolutional-pooling (conv-pooling) layers, 1 flattening layer, and 3 fully connected layers. Multiple kernels (filters) are used to extract features in an image. The first convolutional layer contains 16 kernels of size 3 × 3 × 1 with stride (1,1). The next is with 32 kernels of size 3 × 3 × 1 with stride (1,1). An epoch is the number of times the whole dataset is passed through the entire network. The number of epochs is set to 10 for our model. The hyper-parameters are searched using Bayesian Optimization. The optimum hyperparameters are searched for using 5-fold cross validation. An image can be represented as a cuboid having length, width and height. Now imagine a filter passed through the image vertically. If this filter is passed throughout the image, we get new dimensions for the same image. This process is called convolution. The convolutional layer extracts the attributes from the periocular images, and hence is an integral element of CNN. The kernels generate a kernel map or feature map using the convolution operator (*). The succeeding convolution layer extracts higher-level features, whereas the initial layer extracts low-level features. As an output of the convolutional layers, the pooling layer reduces the dimensions of the feature map obtained. The model uses max-pooling for the same. It is a pooling operation that calculates the maximum value for the patches of the obtained feature map. The output from the convolutional-pooling layer is given as the input to the flattening layer. The 2-D arrays from the pooling layer are converted into a single long continuous linear vector. The combination of convolutions and other inputs is presented as (1) xJL=f(∑i∈Mj(xi(L−1)kijJ+bjL)), x_J^L = f\left( {\sum\limits_{i \in {M_j}} {(x_i^{(L - 1)}{k_i}{j^J} + b_j^L)} } \right), where

xJL−jt x_J^L - {j^t} output of the L^th layer,

The output from the flattening layer is fed into the dense/fully-connected layer. In the dense layer, each neuron receives the input from all the neurons in the previous layer. To avoid the over-fitting of the data, dropout layers with a dropout rate of 0.2 are added in between the dense layers The image is classified by the dense layer based on the output from the previous layers. LeakyReLu is used as the activation for the first two dense layers, while softmax is used for the last dense layer.

In the first step, the images of the eye considered in the Visible Spectrum (400nm–700nm) are pre-processed. The images are resized as (240, 240) size pixel images without affecting the aspect ratio of the images using bilinear interpolation. EfficientNetB1 takes the input data should range [0,255], and input images of shape (240,240,3). Thus, the images are not changed into grayscale.

Transfer learning involves using the previously attained knowledge from other models in the current model. This reduces the time and effort while building a new model thus ensuring better results as the model is already trained on a number of data points from other dataset. An EfficientNet-B1 architecture consists of the stem layer and the custom-made final layers. The stem layer consists of an Input layer, rescaling layers that rescale the images to a desirable size, the normalization layer, the zero padding layer, the Conv2D layer, the Batch normalization layer and the activation layer. The initial layers which form the stem layer are shown in Figure 3.

Fig. 3

After the stem layer, there are seven blocks which are shown in Figure 4.

Fig. 4

The modules used in the architecture are shown in Figure 5.

Fig. 5

The above-mentioned layers have features learnt on images from ImageNet. The feature maps from the EfficientNet-B1 architecture are now passed onto the top layer. The top layer, which is the final layer, is replaced with custom layers which help to classify the images into two classes - 0 (female) and 1 (male). The custom model has an AveragePooling2D layer and a dense layer. In AveragePooling2D, pooling is applied in 2x2 patches. The average is calculated for each patch with a stride of 2. Softmax activation is used in the last layer. The softmax function calculates the relative probabilities and determines if the given image is that of a male or a female eye. (2) softmax(zi)=exp(zi)∑jexp(zi), softmax({z_i}) = {{exp({z_i})} \over {\sum\nolimits_j {exp({z_i})} }}, which z is the value from the neurons in the dense layer.

The accuracy attainment is 94.46% for the CNN model built from scratch. The classification report obtained by using the CNN model is shown in Table 1.

Out of 418 male eyes, 387 are predicted correctly and 31 are predicted incorrectly. Out of 504 female eyes, 484 are predicted correctly and 20 are predicted incorrectly. The confusion matrix obtained by classifying the data with the help of the CNN model built from scratch is as shown in Figure 6.

Fig. 6

The model accuracy changes with each epoch. The accuracy increases with the increasing number of epochs and then reaches a saturation point. The model accuracy obtained by using CNN model built from scratch is as shown in Figure 7.

Fig. 7

The model loss obtained by using the CNN model built from scratch is as shown in Figure 8.

Fig. 8

For the fine-tuned EfficientNetB1, the accuracy obtained is 97.94%. The classification report obtained by using the fine-tuned EfficientNetB1 is as shown in Table 2.

Out of 418 male eyes, 404 are predicted correctly and 14 are predicted incorrectly. Out of 504 female eyes, 499 are predicted correctly and 5 are predicted incorrectly. The accuracy of the model can be determined with the help of the confusion matrix given in Figure 9.

Fig. 9

The model accuracy and loss obtained by using EfficientNetB1 is as shown in Figures 10, 11 and 12 respectively.

Fig. 10

Fig. 11

Fig. 12

EfficientNetB1 gives better accuracy than CNN from scratch. All the other parameters justify that the fine-tuned EfficientNetB1 is a better model than building a CNN model from scratch. The difference in accuracy is due to the different features acquired from each level. The layers in the CNN model built from scratch learn features which are relatively simple to learn. The EfficientNetB1 is already trained on a million images and hence has acquired more advanced features. In the initial layers, the low accuracy is because they only learn simple features and cannot identify the complex features. With an increasing number of epochs, more features are learnt and hence the accuracy increases in both models.