China is the largest country for cultivating kiwifruits, and Shaanxi Province provides the largest production, which accounts for approximately 70% of the nationwide production and 33% of the global production. Harvesting kiwifruits in this region relies mainly on manual picking which is labor-intensive. Therefore, the introduction of robotic harvesting is highly desirable and suitable. Most researches involved so far in kiwifruit harvesting robots suggest the scenario of daytime harvesting for taking advantage of the sunlight. Robot picking at night can overcome the problem of low work efficiency and will help to minimize fruit damage. In addition, artificial lights can be used to ensure constant illumination instead of the variable natural sunlight for image acquisition. The study object of this paper was a kiwifruit recognition system at night using artificial lighting by identifying the fruit calyx. According to kiwifruits’ growth characteristics, which were grown on sturdy support structures, an RGB (red, green, blue) camera was placed underneath the canopy so that kiwifruits clusters could be included in the images. An image processing algorithm was developed to recognize kiwifruits by identifying the fruit’s calyx. Firstly, it subtracted 1.1R-G gray image, and then segmentation was done using the Otsu method for the thresholding. A morphological operation was applied to remove the noise that adhered to the target fruits (such as branches). Afterwards, an area thresholding method was employed to eliminate the remaining noises. This method is based on finding the biggest area of neighboring white pixels in the image and eliminating all areas which are smaller than 1/5 of the biggest area. Using this image as the mask, a fruit image without background was obtained. After that, V (value) component of HSV (hue, saturation, value) color model was calculated for segmenting the fruit’s calyx from the fruit, also using the Otsu method for thresholding. Black areas were then labeled and sorted by their pixels numbers. The first largest black area was the image background and the second largest black areas was a fruit calyx area that used as the reference area. Since the fruit calyx areas varies in a small range in one image, the fruit calyx areas are judged by comparing with the reference area. If a black area in the image was smaller than the reference area and larger than 1/10 of the reference area, it is a fruit calyx; otherwise, it is not. Finally, the nearest edge pixel for each fruit’s calyx was searched and their distance calculated was as radius, and a circle around the fruit calyx was drawn. Finally, the algorithm was also tested for the robustness under 12 different light illuminations (10, 30, 50, 80, 110, 150, 200, 300, 400, 500, 800 and 1 200 lux). The fruits illumination was estimated by averaging the illumination values, which were measured for 3 times at 3 different locations around the target fruit cluster. Results showed that the image processing algorithm based on the calyx could recognize kiwifruit and reached a success rate of 94.3%. Undetected and wrongly detected fruits appeared mostly at the same cluster where one fruit was adjacent to 3 or more fruits. The calyxes of those fruits sometimes were not in the centers of their fruits’ images, thus, causing undetected fruits. Those fruits also formed dark areas among them, which were wrongly recognized as calyx. On the other hand, most clusters were linearly arranged on the branches, which made them suitable for the proposed algorithm. The algorithm was robust to different illuminations although the success rates were slightly decreased under extremely weak or strong illuminations. It only took 0.5 s in average to recognize a fruit, which is a great step toward filed robotic harvesting of kiwifruit.%根据猕猴桃的棚架式栽培方式，提出了一种适用于猕猴桃采摘机器人夜间识别的方法。采用竖直向上获取果实图像的拍摄方式，以果萼为参考点，进行果实的识别，并测试该方法对光照的鲁棒性。试验结果表明：基于果萼能够有效的识别猕猴桃果实，成功率达94.3%；未识别和误识别的果实一般出现在5果及5果以上的簇中，原因是果实相互挤压导致的果萼部分不在果实图像的中心区域，以及果实之间的三角区形成暗色封闭区域；光照过小或过大会导致成像模糊或过曝，对正确率有细微影响；识别速度达到了0.5 s/个。因此，基于果萼的猕猴桃果实夜间识别方法在正确识别率和速度上都有很大提升，更接近实际应用。
展开▼
