Non-destructive, non-invasive, in-line real-time phase-based reflectance for quality monitoring of fruit

The food quality monitoring in the production process is essential. The quality of the food depends on many parameters which are categorized as internal and external parameters. External parameters include the color, shape, surface, hardness, softness, odor, taste, etc., whereas internal parameters include sugar, water, acidity, internal defects, etc. The processing and storage requirements of agro-based products are determined by assessing the initial quality of the fruit or the vegetables. Many industries are finding better ways to assess the quality of the food. The techniques used to monitor the food quality are classified into destructive and non-destructive methods. Industry procedures like chromatography, spectrophotometry, electrophoresis, and titration are examples of destructive techniques. Destructive methods do not allow continuous monitoring besides being expensive, slow, and time consuming. The major limitation of the destructive method is that the sample taken for testing must represent the batch accurately, else it results in a huge wastage of the food. Therefore, it is a cost inefficient solution.

A rapid and affordable method is desired by the food industry to monitor the quality of the food. In non-destructive approach, the information available from the outside of the product is correlated to the quality or freshness of the product. In non-destructive methods, the samples are not destroyed. In the study of Concina et al. (2012), the chemical profile of the fruit is used to monitor the freshness using electronic noses. In the study of Shmulevich et al. (2003), accoustic measurements are used to measure the tissue elasticity which relates to the softening of the apples during storage. In non-destructive optical technique, light spectrum in the range of 400 to 1000 nm is used to determine the response of the fruit to an incident light. The response can be in the form of refraction, reflection, absorption, or scattering. An optical technique allows us to successive measurements of a sample and is very popular due to its ease in implementation.

Among the non-destructive techniques, for monitoring of the internal quality of fruits, near infrared (NIR) spectroscopy is widely used (Nicolaï et al., 2007). The NIR spectroscopy is now commonly used for the assessment of fresh fruits by monitoring the internal quality. NIR radiations cover the electromagnetic spectrum between 780 and 2500 nm. Most agro products behave as semi-transparent or turbid for these wavelengths, and therefore, the light passes through the agro-products and interacts with the internal structures (Askoura et al., 2016). The incident radiation may be either absorbed, transmitted, or reflected. The transmitted and reflected light is collected using a detector and the received information is then processed for quality estimation. The non-invasive and nondestructive nature of the NIR spectroscopy allows its use in industrial application, where continuous quality and process monitoring is desired. Furthermore, NIR spectroscopy allows several constituents to be measured simultaneously. However, NIR spectroscopy is limited to fruits and vegetables with homogeneous pulp (Oliveira et al., 2014). The penetration depth of the NIR spectroscopy is inadequate for internal quality monitoring of various fruits and vegetables. The transmitted or reflected NIR spectrum of a fruit or a vegetable has peaks of absorption band; therefore, statistical processing needs to be done for extracting the information on the internal quality of the fruit. This increases the processing complexity of the NIR spectroscopy.

In the study of Qin and Lu (2008), a novel spatially resolved hyperspectral diffuse reflectance imaging system is proposed to obtain the optical properties of fruits and vegetables. Hyperspectral imaging or multispectral imaging involves image formation using spectral information from a band of wavelengths transmitted or reflected from the same region of an object. Hyperspectral imaging using three-dimensional data cubes is used to analyze the quality of the agro-based products. Hyperspectral imaging uses images from the object, and therefore, it is non-contact and non-destructive. Furthermore, the process of image capture and analysis is relatively simple. However, the computational complexity in creating the data cube is very high and the requirement of specialized camera systems increases the cost. Furthermore, the shape and quality of the skin layer of the fruit affects the scattering, introducing distortion in the measurements. In the study of Cubeddu et al. (2001), a time-resolved approach is used to predict the firmness of fruits and vegetables. Time-resolved spectroscopy relies on the time it takes to penetrate through the fruit. The internal composition of the fruit will affect the scattering, and therefore, the time the incident pulse takes to transmit through the pulp of the fruit can be correlated to its quality. In the study of Cubeddu et al. (2001), a pulsed laser in the wavelength range of 650 to 1000 nm is injected into the fruit. The injected pulse transmits through the pulp of the fruit. The temporal delay and broadening of the pulse is measured at the other end. This technique needs two optical ficube, one for the injection of the laser light and the other for the collection of the transmitted laser light. The collected light is then analyzed and correlated to the internal pulp quality. The experimental set-up for time-resolved spectroscopy is complex. Furthermore, the correlation between the received pulse and the internal pulp quality is weak.

For automatic inspection of fruits and vegetables, artificial vision systems are becoming popular (Du and Sun, 2006; Cubero et al., 2011). The computer vision algorithms are used in grading, quality estimation using external features and also monitoring of fruits during storage. The computer vision-based grading systems allow a large number of fruits to be inspected in short time with minimal human intervention and are more reliable than human inspection. Computer vision-based system relies on the quality of the captured images. The quality of the captured images in turn largely depends on the specifications of the camera being used and the illumination condition on the object. The large variations in the color, size, shape, and features in one type of fruit make the computer vision-based sorting challenging and often results in errors. Furthermore, the color of the skin of a fresh fruit and blemish on the skin of another fruit may be the same. The variations in the performance of camera and illumination conditions used severely affect the quality of the image obtained and, therefore, limit the performance of the computer-based system. For every batch of the fruit, different calibrations on the processing algorithms are required which make computer vision-based systems slow. Image processing-based algorithms always have a trade-off between speed and accuracy.

Optical techniques using reflectance/transmittance suffer from saturation of the detector due to high reflectance, variations in skin color and texture and distortions due to the variation in the shape of the reflecting object (Yousaf and Qin, 2014). In the study of Sarkar et al. (2019), Stokes degree of linear polarization is used to measure the internal quality of the fruits. The Stokes based polarization information is complex in computation. In this paper, a real-time low-complex optical method is proposed for the assessment of the internal quality of fruits. The proposed method relies on the reflection rather than transmission of the incident light. The incident light is linearly polarized and the reflected phase information is monitored which is correlated to the quality of the fruit. The proposed method is non-destructive and is not computationally intensive. To demonstrate the proposed reflectance-based phase spectroscopy, apples are used as test case. As the apples are stored, they degrade, changing the internal chemical composition. The changing chemical composition changes the scattering of the linearly incident light wave. Thus, the reflected phase information is shown to be dependent on the decay rate of the apples. The changes in the reflected phase information can be qualitatively used to predict the quality of the fruit. The use of phase information of the reflected light makes it independent of the color of the skin of the fruit, increasing the accuracy of the quality prediction. The rest of the paper is organized as follows: the second section presents the theory behind phase-based reflection and its use in monitoring the quality of the apples. The third section describes the measurement setup and the correlation between the quality of the apples and the measured reflected phase is derived. The paper is concluded in the fourth section.

The color and spectral reflectance changes are often used to monitor or sort fruits in commercial applications. The fruit color and appearance is primarily due to the proportional content of chlorophylls, cartoneoids, and anthocyanins (Merzlyak et al., 2003; Merzlyak, 2006). The red color of the apples is due to the pigment anthocyanins. The pigment content varies in different types of apples. For example, Red Star has higher anthocyanins as compared to Golden Delicious. Golden Delicious, on the contrary, has higher chlorophyll content compared to Red Star. The anthocyanins and chlorophyll content in the Red Star and Golden Delicious changes as the apples ripen. In the optical technique, the changes in the reflectance due to changing pigment content is monitored to determine the quality of the fruit (Chuma et al., 1977; Merzlyak, 2006). Reflectance spectroscopy depends on the amount of light reflected by the object and, therefore, also depends on the nature of the reflecting surface. The color, texture, and the composition of the object, thus, influence the reflected light. Furthermore, the changes in the internal pigmentation of the fruits change the absorption maxima, and therefore, the reflected light is also wavelength sensitive.

In this section, the reflectance properties of apple are described. Furthermore, the advantages of using phase to monitor the scattered components are elaborated.

The block diagram of the optical setup used to monitor the quality of the fruit is shown in Figure 1. For the experiments, standard CMOS/CCD camera, two polarization filters, a monochromatic light source and apple (Red Star and Golden Delicious) as fruit object are used. One polarization filter is used across the light source (optional), whereas the other is used across the camera. The camera used is a monochromatic Basler camera (acA640 to 90 um, Sony ICX424 mono CCD sensor) with a resolution of 659 × 494 pixels. A DC ambient white light source (make: Holmarc) with maximum intensity of 1300 lux is used. The environmental lighting condition is ambient. The DC white light generated is unpolarized in nature. The unpolarized light is passed through a linear polarization filter (Polarizer 1) to linearly polarize the light which is then incident on the sample apple. The apples reflect the incident light while changing the phase of the incident light. The reflected light from the apple is, therefore, partially polarized. This partially polarized light is transmitted through another linear polarization filter (Polarizer 2) before being detected by the camera. The transmission axis of the polarizer 1 and 2 is varied manually to 0°, 45°, and 90° for different measurement conditions.

Figure 1:

A sample measurement of the apples is shown in Figure 2. The polarized images are obtained by changing the transmission axis of the linear polarizer. Figure 2A is a test image without the linear polarizer filter, which is used a reference image. The images shown in Figure 2B have both polarizer 1 and 2 at 0° transmission angle. The received light intensity when both polarizer 1 and 2 at 0° transmission angle is represented as I(0, 0). In Figure 2C, the transmission axis of polarizer 1 and 2 are at 0° and 45°, respectively. The received image is represented as I(0, 45). Figure 2D represented as I(0, 90) is obtained when the transmission axis of polarizer 1 and 2 is at 0° and 90° respectively. For each measurement, a set of 100 images per observation is taken with and without a linear polarizer. An ROI of 140×140 pixel is used to obtain the average received light intensity to improve the signal-to-noise ratio (SNR). The pixel-to-pixel variations occur due to the variations in the reflected light reaching the camera through the linear polarizer. As per the Malus law, the light reaching the sensor is a function of the transmission axis which is evident from the images shown in Figure 2.

Figure 2:

The reflectance and the polarization state of the reflected light will depend on the pigment content as discussed in the second section. The quality of the apples is expected to decay over time. The decaying or ripening of the apple changes the pigment content which changes the absorption and scattering characteristics of the apples. To monitor the scattering characteristics, the measurements are repeated with the same apples at different time instances over a period of two week. Figure 2 also shows the measurement results of I(0, 0), I(0, 45), and I(0, 90) for Red Star and Golden Delicious apples at day 12. The varying absorption and scattering characteristics influence the diffuse reflection from the apples and, thus, changes the polarization state of the reflected light. The varying polarization state is clearly observed in Figure 2.

Food quality monitoring using white light and single polarizer

Initially, the measurements are performed with single polarizer in the light transmission path. The polarizer is placed in front of the camera in Figure 1. Unpolarized light from the DC light source is incident on the apple. The pigment contents in the apple change the phase information of the reflected light and, therefore, partially polarizes the unpolarized incident light. The reflected partially polarized light is transmitted through a linear polarizer whose transmission axis is varied manually. Figure 3 shows the variation of normalized received polarization intensity with varying observation number when the transmission axis of the linear polarizer is varied by 0°, 45°, and 90° for red light and white light, respectively. Two observations are clearly made from the figure:

The received normalized polarization intensity decreases with increase in the observation number. The drop is very sharp for Red Star in the presence of red light as compared to when white light is used.

The normalized received polarization intensity is higher when white light is used as compared to when red light is used.

Figure 3:

The differential absorption of red light by chlorophyll and anthocyanin influences the spectral reflectance. Therefore, the reflected light from an apple depends on the wavelength of the incident light. For cholorophyll, the absorption coefficient is high for light with wavelength at 675 nm. Therefore, the spectral reflectance of an incident light by an apples depends on the wavelength of light used as well as the storage or ripening period. The stronger reflection of incident white light explains the higher normalized polarization intensity as compared to red light. In case of red light, the higher absorption reduces the scattering, and therefore, the received polarization intensity is lower compared to white light.

For a less mature fruit, the chlorophyll absorption is high. With the ripening or storage, the absorption by the chlorophyll decreases, whereas the scattering increases. The reflected light is, therefore, more unpolarized which decreases the transmittance through a linear polarizer. This decrease is sharp when red light is used as compared to white light. The other wavelength components present in the white light are not scattered much, and therefore, the transmitted intensity is higher though the linear polarizer. The degree of linear polarization (DoLP) as expressed in Equation (3) is shown in Figure 4.

Figure 4:

The degree of linear polarization shows that for both red and white lights, the DoLP increases with increase in an observation number. The variation in the scattering profile with increase in storage time changes the DoLP, and therefore, its variation is directly correlated to the quality of the fruit. For a fresh apple, the DoLP in the presence of both white and red lights is low. As the apples degrades, the DoLP increases over time. The R-squared for red light is 0.2232, whereas for white light it is 0.5458. The change in the DoLP can be used to predict the characteristic changes in the internal quality of the apples. In the study of Sarkar et al. (2019), the DoLP was computed using two polarizers one in front of the light source, whereas the other was used in front of the detector. In the proposed, method, it is demonstrated that the changes in the DoLP for changing pigment content in the apple can be obtained using a single polarizer. The correlation between the changes in the pigment content and changes in the DoLP as measured by the R-squared coefficient is higher when a single polarizer is used as compared to (Sarkar et al., 2019). The use of single polarizer further simplifies the instrumentation and, thus, can be realized for real-time measurements.

Food quality monitoring using white light and two polarizer

In this section, the measurements of the reflected light are repeated using two polarizers as in the study of Sarkar et al. (2019). Polarizer 1 is used to polarize the DC white light. This polarized light is incident on the apple which partially polarizes the light. The partially polarized light is then transmitted through polarizer 2 and detected by the detector. In the study of Sarkar et al. (2019), the DoLP was calculated using Stokes parameters which increases the computational complexity as it needs addition, subtraction to be performed on the received data besides calculating the ratios. The real-time implementation would, therefore, be difficult if the direct algorithm of Sarkar et al., (2019) is used. Thus, the algorithm of Sarkar et al., (2019) is slightly modified here to make it real-time implementable. The DoLP computation in the proposed method uses I(0, 45) information rather than conventional I(0, 90). The incident light is, therefore, 0° polarized, whereas the DoLP is computed based on the received intensity by the detector when the transmission axis of the polarizer in front of it is 0° and 45° . The received intensity profile is as shown in Figure 5.

Figure 5:

The Red Star with low absorption has higher reflectance compared to Golden Delicious which has higher absorption. The larger spectral reflectance helps in retaining the incident polarization phase in Red Star, whereas higher absorption in the case of Golden as compared to Golden Delicious, and therefore, the reflected light is more likely to retain the phase information of the incident light. The 45° filter rejects the reflected 0° polarized light. The multiple scattering in Golden Delicious, unpolarize the reflected light, and therefore, the transmitted light through the 45° filter is higher. I(0, 45) also increases with increase in an observation number. The differential variation in I(0, 45) is much higher than I(0, 90) reported in the study of Sarkar et al. (2019). The increased differential variation would improve the DoLP which is now computed using the modified equation as Delicious causes multiple scattering. The multiple scattering results in diffuse reflection which looses the phase information of the incident light. The lower content of chlorophyll in Red Star causes less absorption, and therefore, for an incident 0° polarized light, the received reflected light at the detector with 0° transmission axis of the linear filter in front of it is higher for Red Star as compared to Golden Delicious. I(0, 0), measured with both polarizer 1 and 2 transmission axis set at 0° for Red Star and Golden Delicious is shown in Figure 5A. It is further observed that I(0, 0) increases with increase in the observation number I(0, 45) measured with the transmission axis of 45° for polarizer 2 in front of the detector, whereas the incident light is still 0° polarized, as shown in Figure 5B. The higher reflectance in Red Star retains the incident phase, and therefore, the measured I(0, 45) is lower for Red Star as compared to Golden Delicious. The scattering in Red Star is weak as compared to Golden Delicious, and therefore, the reflected light is more likely to retain the phase information of the incident light. The 45° filter rejects the reflected 0° polarized light. The multiple scattering in Golden Delicious unpolarizes the reflected light, and therefore, the transmitted light through the 45° filter is higher. I(0, 45) also increases with increase in the observation number. The differential variation in I(0, 45) is much higher than I(0, 90) reported in the study of Sarkar et al. (2019). The increased differential variation would improve the DoLP which is now computed using modified equation as: (4) D o L P = I 45 ° I 0 ° , (4)where I_0° and I_45° are the received light intensity when the transmission axis of the linear polarizer is 0° and 45°, respectively, in front of the camera. The DoLP computed using Equation (4) is shown in Figure 6 for Red Star and Golden Delicious, respectively, when white light source is used. The following observations can be made:

The DoLP for Golden Delicious is lower compared to Red Star.

The DoLP decreases with increase in the observation number.

Figure 6:

In the presence of white light, the absorption of the chlorophyll dominates, reducing the scattering component. The reduced scattering component increases I(0, 0). The Golden Delicious having a higher content of chlorophyll compared to Red Star has higher absorption than scattering. Therefore, the DoLP is lower in Golden Delicious compared to Red Star due to reduced I(0, 45). The DoLP for both Red Star and Golden Delicious apples decreases with increase in time. The increase in the storage time of the apples changes the pigment content. As the apple decays, the scattering is lowered, and therefore, the polarized component in the partially polarized light decreases, decreasing the DoLP. Thus, scattering from the pigments influences the degree of linear polarization (DoLP).

The change in the DoLP for Red Star is lower as compared to change in the DoLP of golden delicious. For the Red Star, the change in the DoLP is 4%, whereas for Golden Delicious it is 12.5% over the 14 day period of storage. Furthermore, the R-squared measures are 0.7087 and 0.3085 for Red Star and Golden Delicious, respectively. The R-squared values obtained are higher than those reported in the study of Sarkar et al. (2019). Therefore, we can conclude that it is better to use I(0, 45) for DoLP computation rather than I(0, 90).

As compared to the degree of linear polarization presented in the study of Sarkar et al. (2019), the DoLP presented here is easy to compute and is shown to reflect the changes in the pigmentation inside an apple due to storage or ripening. This optical method is non-destructive and non-invasive. Compared to other non-invasive and non-destructive optical methods, the proposed method is independent of the color of the skin of an apple or its shininess as the normalized phase information is used for the computation of the DoLP. Furthermore, the reflected phase information is independent on the shape and size of the apple used, and thus, the variations in the shape of an apple do not introduce distortions in the measurements.

To summarize DoLP has the following advantages in food quality monitoring:

In comparison to the DoP calculations proposed in the study of Sarkar et al. (2019), the proposed DoLP calculations are relatively simple and, therefore, are less computationally complex. Furthermore, the instrumentation required is simple.

The DoLP-based correlation to the internal quality of the fruit is independent of the color of the skin of an apple. Furthermore, the size and shape of the apples also does not influence the measurement. The shininess of the apples does not saturate the detector and, thus, also does not influence the measurements. The de-coloration of the fruit skin due to damage do not introduce distortion in the measurements.

The DoLP-based quality monitoring is, therefore, a true non-destructive, noninvasive real-time fruit monitoring technique. Clearly many technical aspects still need to be solved before an industrial application can take place. However, in comparison to the existing food quality monitoring methods, the proposed solution is easy to implement and low cost for real-time non-destructive online monitoring of the quality of the food.

The assessment of internal quality of fruits is desirable for grading and monitoring of internal quality of fruits in food industry. The techniques available for quality monitoring can be broadly classified into destructive and non-destructive methods. In destructive methods, the food sample is destroyed to assess the quality, and therefore, it is not preferred. Non-destructive techniques do not need sample preparation and, therefore, are ideal for in-line real-time monitoring of the quality of the fruits. In this paper, an optical-based technique is proposed which uses the phase of the reflected light from the object to assess the quality of the fruit. In this work, a phase-based nondestructive and non-invasive measurement of optical properties in fruits is proposed. In general, with other optical methods, it is difficult to distinguish the absorption and scattering components. The scattering component is detected using the measurement of the reflected phase information. The degree of linear polarization is shown to vary with decaying of the apples. Monitoring the degree of linear polarization, thus, gives an indication of the quality of the fruit. Furthermore, this technique is independent of the color or shininess of the skin and the shape of the fruit. The developed model is simple and low cost. This novel approach can also be used for diagnostic purpose of other agricultural products.