Pitfalls in multispectral in-field strawberry detection: challenges and first results for ripeness classification in robotic harvesting

In agriculture, autonomous robotic harvesting of strawberries in unstructured outdoor fields faces significant challenges from variable lighting, occlusions, and fruit clustering. While RGB-based detection is common, multispectral imaging (MSI) promises enhanced ripeness classification, yet in-field applications remain underexplored due to data scarcity and technical pitfalls. This paper presents a novel MSI dataset from a commercial strawberry farm, comprising 188 recordings with 1690 strawberries, annotated as unripe, ripe, or overripe - extending beyond binary ripe/unripe schemes. We adapt YOLOv8 for MSI input (14 spectral channels from ultraviolet (UV) to short-wave infrared (SWIR)) and compare baselines against RGB counterparts, revealing key challenges like imperfect image registration and high dynamic range (HDR) artifacts, resulting in MSI underperformance (YOLOv8 mAP@50: 0.754 MSI vs. 0.783 RGB). Our findings highlight deployment pitfalls for robotic systems and provide a foundation for future in-field MSI advancements. To our knowledge, this is the first study analyzing high-resolution, aligned MSI data for multi-class strawberry ripeness in realistic field conditions.