Speaker:

Tom Botterill - University of Canterbury

Title:

Model-based 3D reconstruction for a vine pruning robot

Location:

Owheo 106 - 1:00 pm, Friday 3 May

Abstract:

Every winter, grape vines must be pruned to remove old wood and excess new growth, while keeping the healthiest new canes for the following year. Pruning is the most expensive and labour intensive task in the vineyard, so a robot which can prune vines autonomously will be valuable to New Zealand's winegrowers. This project aims to build such a robot.

In order to make decisions on which canes to prune, the robot must estimate a topologically-correct 3D model of the structure of the plant. This model will be estimated from images from the robot's three colour cameras, which image both the vines and a structured light pattern projected onto the vines. Contemporary systems for 3D reconstruction are based on either matching and reconstructing point features, or on establishing a dense stereo correspondence between views, however these methods are generally not well suited for the vine images, which have high levels of occlusion, and complex, non-convex structures. Instead, we propose a model-based approach, where knowledge of the structure of the vines is used to guide each stage of the reconstruction pipeline. 2D features (canes and parts of the trellis) are detected in each frame. These 2D features, together with 3D measurements from the structured light system, are matched together, and are assigned to parts of a 3D model. The 3D model is then optimised using the g2o bundle adjustment framework. This optimisation incorporates knowledge of the structure of the scene, and uses additional observations to extend and refine the model as the robot moves. Given this model, an expert system will decide which canes to prune, and a path planning algorithm will guide a robot arm to prune the vines.

This seminar will describe the reconstruction pipeline in detail, and will describe work on the expert system which makes the decisions on where to cut. Preliminary results on simulated rendered data, and on real vine images will be presented.

Info:

Tom Botterill is a Postdoc in the Department of Computer Science at the University of Canterbury, funded by the MBIE project "Vision-based automated pruning"