Plant Parasitic Nematode Identification in Complex Samples with Deep Learning

Nematode samples for imaging were received from UF NDL. Briefly, UF NDL received soil samples from growers to identify and quantify PPN in their field. Nematodes were extracted using sugar floatation and centrifugation method (Gooris and d’Hedre, 1972) using 250 cc of soil. Before imaging, all nematodes were concentrated, briefly heat killed, and placed in a 6 cm diameter Petri plate. Nematode samples were imaged using Keyence digital microscope BZ-X at 20X magnification with Z-stacking function. Z-stacking combines multiple images taken at different focal distances to provide a composite image with a greater depth of field (Johnson 2008; Ray 2002). About 5,000–10,000 images were stitched together to create each composite image, and a total of 100 composite images were collected. Samples encompass multiple hosts and locations, ensuring a range of PPN in each image. A representative image is shown in Figure 1, with colored boxes representing annotations of PPN.

Figure 1:

The acquired composite images were imported to Visual Geometric Group (VGG) image annotator (Dutta, n.d.; annotate.officialstatistics.org). Each image was annotated for all potential plant parasitic nematode heads present in each image by drawing a box around a stylet. Each image contained between 10 and 100 stylet bearing nematodes.

The original resolution of collected images averaged 4,000 × 4,000 pixels. These raw, high-resolution images are not suitable for training CNN-based object detection models due to the large amount of memory needed. To alleviate this issue, a grid cropping strategy was implemented, splitting the original images via a sliding window of size 640 × 640 pixels. Each original image was split into roughly 39 sub-images using the sliding window grid-crop, resulting in 3,503 total crops from the 90 original images.

Data augmentation was used to increase the number of labeled examples for model training, as well as to introduce more variety into the data. Specifically, within the models described this paper, we used translation, rotation, scaling, shearing, color space adjustments, and mosaic augmentations. For translation, each image was translated by a random proportion sampled from a normal distribution with mean 0.1; for rotation, a mean of 0.0 degrees; for scaling, a mean gain of 0.5; for shearing, a mean of 0.0 degrees. For the color space augmentations, the hue, saturation, and value were augmented. The mosaic augmentation samples from the above augmentations and combines four random images into one mosaic image (Wei et al., 2020).

The initial model training stage utilized various popular CNN-based object detection models to determine the feasibility of using these models for the detection task. Namely, CenterNet (Duan et al., 2019), EfficientDet (Tan et al., 2019) and RetinaNet (Lin et al., 2018), and YOLOv5 (Liu et al., 2022) were used to gather initial performance results and select the highest performing model for further experimentation. For the models, the size of the training dataset was increased by a ratio of 4:1 through the data augmentation pipeline described above. Each model was then trained on the data for 1,000 epochs, or until there was no significant change in the loss function. From this the best performing model was selected for further experimentation.

The performance of the detection task was evaluated using the following popular object detection evaluation metrics: mean average precision (mAP) at 0.5 intersection over union (IOU). MAP at 0.5 IOU is the average precision of the predicted bounding boxes where the degree of overlap between the detected bounding boxes and the ground truth bounding boxes is at least 50%. Precision is calculated as the number of true positives divided by the number of true positives and false positives. IOU is calculated by dividing the overlap between the predicted and ground-truth box by the combined area of both the predicted and ground-truth box. For nematode identification, this means that at least half (0.5) of the models predicted box overlaps with the annotated box drawn around the nematode stylet. A higher mAP means a greater number of predicted annotations overlapped by at least 0.5 with the actual annotation.

The baseline YOLOv5 model conducts both region detection and object classification simultaneously. YOLOv5 accomplishes this by first subdividing the input image into an s × s grid. Each box in the grid predicts the probabilities that the object covered in the box belongs to a given class. From these probabilities and boxes, a bounding box is then constructed around the object. The main components of YOLOv5 are the backbone, the neck, and the head. The “backbone” is responsible for extracting features from the image itself, while the “neck” performs feature fusion, the fusing of features from different levels in the backbone hierarchy. Finally, the “head” performs the actual predictions of bounding boxes and classes over the Fused features, which are mapped back to the original image. Similar approaches were used for the other object detection models, CenterNet (Duan et al., 2019), EfficientDet (Tan et al., 2019) and RetinaNet (Lin et al., 2018).

After running the initial experiments, it was found that the baseline YOLOv5 model performed the best in terms of mean average precision. From this conclusion several architectural modifications were made to the YOLOv5 model to increase the accuracy of the model. This includes modifying the “neck,” “backbone,” and depth of the YOLOv5 model. The modified models were trained in the same manner as before, with 1,000 epochs of training preformed.

The Bi-FPN variation of the YOLOv5 model tested in this work involves modifying the neck of the YOLO architecture to allow more flow between features of different levels. The intuition behind its inclusion in our YOLOv5 experiments comes from the idea that allowing a freer fusion of features will better allow the network to place emphasis on lower-level features which may be more important in identifying the small stylets of PPN. The efficacy of the BI-FPN neck in small object detection when used in conjunction with YOLO has also been seen in previous research, where it was shown that a bi-FPN neck outperformed other architectures in small object detection (Benjumea et al., 2021).

The DenseNet variation of the YOLOv5 model involves the modification of the backbone of the YOLO architecture. The default backbone of YOLOv5 was replaced with a DenseNet backbone. DenseNet itself is an image classification CNN architecture (Huang et al., 2017). This work tested the DenseNet backbone as it has been found to be very effective in classifying nematodes (Abade et al., 2021), performing above other classification networks. This led to the hypothesis that DenseNet may better be able to identify useful features within the images required for nematode detection.

To achieve the deeper YOLOv5x model, the YOLO algorithm was simply made larger and deeper by a factor of two. This means that each layer's neuron count was doubled, and the number of repeated layers was also doubled in the backbone and neck of the model. Considering that the larger models showed better results within the released family of YOLOv5 models (YOLOv5n, YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x), this work hypothesized that a larger, deeper model may yet further improve the performance of the algorithm in nematode detection.

Four different CNN object detection models were evaluated using the manually annotated stylets as input (Table 1). YOLOv5 performed the best, with a maximum mAP at 0.5 IOU of 0.787. This means that approximately 78% of the predicted objects overlapped with at least half of the manually annotated stylets. The other CNN object detection models performed similarly, but with slightly lower mAP scores corresponding to approximately 64–69% predicted objects overlapping with at least half of the manually annotated stylets (Table 1).

As the CNN object detection model YOLOv5 performed the best of all tested models, we sought to improve the model by modifying its architecture to increase efficacy in detecting nematode stylets indictive of potential plant parasitic nematodes. From the modified architectures, the best overall performance was seen in the YOLOv5x Bi-FPN model, with a validation mAP score at 0.5 IOU of 0.791. This means that approximately 79% of predicted objects overlapped with at least half of the manual annotations (Table 2). We also examined the maximum precision and maximum recall for modifications to the YOLOv5 model. Here, precision is defined as how often the model correctly predicts objects, indicating how much we can rely on the model's positive predictions; and recall defines as a measure of predicting all possible objects, indicating potential predictions the model may have missed. Together, these can inform the potential for false positive and false negative errors in the model's object detection. The best modification was seen with the Bi-FPN variation of the YOLOv5 model, yielding a maximum precision of approximately 85% and recall of 75% (Table 2). A higher precision compared to recall value indicates that the model is accurate when classifying a nematode stylet, but it may not classify all potential nematode stylets in a sample. Regardless, all models and model variations tested seem to produce relatively similar performance with no large gain being seen from the modifications applied (Table 2).