Artificial Neural Network and Regression Models to Evaluate Rheological Properties of Selected Brazilian Honeys

The data were obtained by Silva et al. (2016) and Silva, Torres Filho & Resende (2018), who analyzed twenty-two samples of monofloral honey Assa-peixe (Vernonia polysphaera) (3), Cipó-uva (Serjania lethalis) (5), Eucalyptus (Eucalyptus spp.) (7), and Orange blossom (Citrus sinensis) (7) from the southeast and eighteen samples of multifloral honey from the southeast (8), south (3), northeast (3) and central-west (4) regions of Brazil.

In this study, a supervised multilayer perceptron feedforward artificial neural network (MLP ANN) was used for the development of four independent neural models for the rheological parameter of honeys. Model 1 was developed based on data of η determined in steady shear rheological measurements. Models 2 and 3 were developed from data of G′, G″ and η* measured in oscillatory tests of temperature scans during heating and cooling, respectively. Model 4 was developed from data of G′, G″ and η* measured in oscillatory tests of frequency scans. For models 1, 2 and 3 the input layers were composed of the variables temperature and water content, while for model 4 the input layers were composed of the variables temperature, water content and angular frequency. The input nodes and the neurons in the output layer for each model are shown in Tab. 1 as the statistical indexes of the inputs and outputs of training and testing data in this study.

For the training phase 75% of the data was used and 25% was used in the test phase of the MLP (Faria et al., 2015). The data was normalized to fall within the range of the sigmoidal activation function and the hyperbolic tangent activation function (Khanlari et al., 2012; Xi et al., 2013).

The learning algorithm used was backpropagation with momentum which permits the correction of synaptic weights during training based on the difference (error) between the desired value and that calculated by the network (Abbasi-tarighat, Shahbazi, & Niknam, 2013), and allows that the convergence of weights results in an overall minimum rather than a local minimum of the error (Goktepe et al., 2008).

For analysis of the different network topologies, the activation function (hyperbolic tangent and sigmoidal) and the number of neurons in the hidden layer (3, 6, 9, 12 and 15) were varied, and the learning rate to 0.2 and momentum equal to 0.7 were fixed. After obtaining the best topology, different learning rates (0.2, 0.4, 0.6) and momentum values were tested (0.3, 0.5, 0.7, 0.9) (Llave, Hagiwara, & Sakiyama, 2012).

The study for the determination of the optimal number of iterations for network training was performed through error convergence analysis (RMSE - Equation 1) for a minimum value of the test data (Phimolsiripol, Siripatrawan, & Cleland, 2011). Definition of the optimal number of iterations used during training is an essential criterion to avoid overfitting (Linder & Po, 2003). Selection of the optimal network for each model was determined by a trial and error technique, evaluating different scenarios and configurations and opting for topologies that had lower error values (RMSE). In addition to this statistical criterion, the correlation coefficient (r) was used as a performance measure of the analyzed networks.

The neural network structures were generated using the 2.92 version of Neuroph Neural Network freeware.

In order to accurately evaluate the predicting technique, the combined effect of water content and temperature on the rheological parameters of the honeys, the same data subsets (training set) for the models 1, 2 and 3, shown in Tab. 1, were used to adjust the following exponential model (Oroian 2012) (Eq.1): (1)y=aexp(−bw−cT)y = aex{p^{(- bw - cT)}} where y is the predicted response (η (Pa.s), η* (Pa.s), G′ (Pa) or G″ (Pa)), w is the water content (%), T is temperature (°C), and a, b, and c are empirical constants of the models.

A nonlinear regression technique that incorporates the Levenberg-Marquardt method to solve nonlinear regression was used.

The training data for model 4 was further analyzed using multiple regressions and a second order polynomial model expressed in Eq.2: (2)y=β0+∑i=13βiXi+∑i=13βiiXi2+∑i=12∑j=23βijXiXj+βijkXiXjXk{\rm{y}} = {\beta _0} + \sum\limits_{i = 1}^3 {{\beta _{{\rm{i}}{{\rm{X}}_{\rm{i}}}}}} + \sum\limits_{i = 1}^3 {{\beta _{{\rm{ii}}}}{\rm{X}}_{\rm{i}}^2} + \sum\limits_{i = 1}^2 {\sum\limits_{j = 2}^3 {{\beta _{{\rm{ij}}}}{{\rm{X}}_{\rm{i}}}{{\rm{X}}_{\rm{j}}} + {\beta _{{\rm{ijk}}}}{{\rm{X}}_{\rm{i}}}{{\rm{X}}_{\rm{j}}}{{\rm{X}}_{\rm{k}}}}} where y is the predicted response (η* (Pa.s), G′ (Pa) or G″ (Pa)), β₀ is the intercept coefficient, β_i is the linear coefficient, β_ii is the quadratic coefficient, β_ij and β_ijk are interaction coefficients, X_i is the water content (%), X_j is the temperature (°C) and X_k is the frequency (Hz).

A one-factor analysis of variance (ANOVA) was carried out to assess this model. The significances of the coefficients were examined using the t-test.

Performance of the exponential and multiple regression models was evaluated by means of the RMSE and r statistical parameters.

This statistical analysis was performed using version 9.1 of the SAS (Statistical Analysis System) software.

The optimal number of iterations was obtained by means of the error convergence analysis (Linder & Po, 2003; Phimolsiripol, Siripatrawan, & Cleland, 2011) (Fig. 1). The activation function that presented the best results for all models was sigmoidal. This function remains finite even with x values close to ± ∞, which results in more efficient training of the network (Razmi-Rad et al., 2007).

Fig. 1

The optimal number of iterations is equal to 100 in training the network with 2-12-1 neurons in its layers, referring to model 1 (Fig. 1-a). From this point a considerable increase in the RMSE test was observed, characterizing the overfitting phenomenon. The network with 2-9-3 neurons in its layers, referring to model 2, showed the lowest test RMSE value when completing 1000 iterations (Fig. 1-b), while for the network 2-3-3, referring to model 3, 2500 iterations were required (Fig. 1-c). In both cases a slight increase in the test RMSE was observed after the convergence point. Model 4, whose architecture consisted of 3-9-3 neurons in its layers, required only ten iterations to reach the minimum test RMSE value (Fig. 1-d)

After selecting the best activation function and number of neurons in the hidden layer, all model were evaluated with regards to the influence of the learning rate and momentum on their performance. In all models the learning rate value for the best performance was equal to 0.2. This result is in concordance with that reported in literature, considering that low-learning rate values prevent large fluctuations in error and consequent instability problems (Al-Mahasneh, Rababah, & Ma’Abreh, 2013; Rai, Majumdar, & Dasgupta, 2005; Rossi et al., 2014). In relation to the term momentum, models 1 and 4 performed better for the value of 0.3, while models 2 and 3 performed better at the value of 0.7.

For model 1 (2-12-1 architecture), 500 iterations were required for training. This MLP-ANN produced good results for predicting the viscosity of honey as a function of water content and temperature. The RMSE of the training phase of this model was equal to 0.0359, while the RMSE of the test phase was equal to 0.0430. Comparing the data calculated by the network and that obtained experimentally, they showed high correlation coefficients for both training and test data, equal to 0.9760 and 0.9681, respectively.

The exponential model for assessing the combined effect of water content and temperature on viscosity showed higher accuracy than the MLP ANN of model 1 due to the higher coefficient of determination and lower RMSE for the training data, equal to 0.9981 and 0.0101, respectively (Tab. 2). With respect to the test data, both models showed the same performance (Tab. 2) and indicated a reduction in their prediction quality when unknown data was used.

In relation to models 2 and 3, the error convergence point was reached after 500 and 2500 iterations, respectively. Model 2 (2-9-3 architecture) showed good results in the determination of the rheological properties G′, G″ and η* from the water content and temperature in the temperature sweep curves by means of SAOS measurements during heating. This model presented a RMSE test value equal to 0.0255, while that of training was 0.0310. When comparing each predicted variable separately in this model with the desired experimental values, the G′ property was observed to have the highest RMSE training values, equal to 0.0338, and test values equal to 0.0261 (Tab. 2), as well as the lowest correlation coefficient values, equal to 0.9398 in training and 0.9390 in testing. The properties G″ and η* showed RMSE and training correlation coefficient values equal to 0.0296 and 0.9704, and 0.0295 and 0.9705, respectively, and test values of 0.0252 and 0.9731 (Tab. 2).

Model 3 (2-3-3 architecture) also showed good results in the determination of the rheological properties G″ and η* from the water content and temperature in the temperature sweep curves by means of SAOS measurements during cooling. However, its statistical parameters were inferior to model 2, where the training RMSE is equal to 0.0464 and that of testing is equal to 0.0387. For each variable separately, the RMSE (Tab. 2) and correlation coefficient were, respectively, 0.0675 and 0.6969 in training, and 0.0486 and 0.6629 in testing for G′, were 0.0308 and 0.9758 in training and 0.0326 and 0.9794 in testing for G″, and finally were 0.0309 and 0.9759 in training and 0.0326 and 0.9794 in testing for η*.

Regarding the combined effect of temperature and water content on the viscoelastic properties (G′, G″ and η*) during temperature scanning, in both heating and cooling, the exponential model showed the same performance as models 2 and 3 with the MLP ANNs, due to the similarity of the obtained statistical parameters (Tab. 2).

Model 4 (3-9-3) presented a training RMSE value equal to 0.0306, while that of testing was 0.0270 after fifty iterations. The RMSE values (Tab. 2) and correlation coefficients in training were equal to 0.0260 and 0.7244 for G′, 0.0195 and 0.9604 for G″ and 0.0420 and 0.9636 for η*, respectively. In the test, for G′ they were equal to 0.0158 and 0.7301, for G″ were equal to 0.0176 and 0.9581 and for η* were equal to 0.0407 and 0.9647, respectively. These values of the statistical parameters show a good quality prediction of the properties G″ and η* in function of frequency, water content and temperature by means of the SAOS measurements. Second order multiple regression models were determined to estimate the rheological properties G′, G″ and η* of model 4 (Tab. 3). The prediction quality, based on the correlation coefficient and RMSE parameters, were much lower than the MLP ANN of model 4 (Tab. 2).