Autonomous vehicles are being increasingly adopted in agriculture to improve productivity and efficiency. For an autonomous agricultural vehicle to operate safely, environment perception and interpretation capabilities are fundamental requirements. The Ambient Awareness for Autonomous Agricultural Vehicles (QUAD-AV) project, funded by the ICT-AGRI European Research Program, explores a multisensory approach to provide an autonomous agricultural vehicle with such ambient awareness. The proposed methods and systems will aim at increasing the overall level of safety of an autonomous agricultural vehicle with respect to itself, to people and animals as well as to property. The “obstacle detection” problem is specifically addressed within the QUAD-AV project. The idea is that of using different sensor modalities and multi-algorithm approaches to detect the various kinds of obstacles and to build an obstacle database that can be used for vehicle control.
The QUAD-AV project investigates the potential of four technologies: vision/stereovision, ladar, thermography and microwave radar.
The multi-baseline stereo vision system
Short baseline system: Uses Bumblebee XB3 / 2 baselines: 12-24 cm / 3.8mm focal length lenses
Long baseline system: Custom-built with 3 Flea3 cameras / 2 baselines: 40-80 cm / 12 mm focal length lenses
TYPE AND DATE
STARTING DATE – Feb 15, 2012
ENDING DATE – Feb 14, 2013
COST € 11.873,00
FUNDING € 11.873,00
ISSIA MAIN ACTIVITY
In the context of QUAD-AV, ISSIA was involved in the development of a multi-baseline stereo frame, composed by two two trinocular heads, one featuring a short baseline system and the other one featuring a long baseline system. By employing the narrow baseline to reconstruct nearby points and the wide baseline for more distant points, this system takes the advantage of the small minimum range of the narrow baseline, while preserving the higher accuracy and maximum range of the wide baseline configuration.
The 3D point cloud returned by either trinocular camera provides a rich source of information for the vehicle to perform key navigation tasks, such as terrain identification and scene segmentation. In this investigation, stereo reconstructed points were used as input data to a geometry-based classifier that segments the scene into ground and non-ground regions. This classifier features a self-learning framework, where the ground model is automatically built during an initial bootstrapping stage and is continuously updated to incorporate changes in the ground appearance. During the training stage, the classifier learns to associate the geometric appearance of data with class labels. Then, it makes predictions based on past observations classifying new acquired data.
The system was implemented within the project Ambient Awareness for Autonomous Agricultural Vehicles (QUAD-AV) funded by the ERA-NET ICT-AGRI action, aimed to enable safe autonomous navigation in high-vegetated, off-road terrain.
- makes the point of view closer, thus widening the common field of view of the two cameras.
- improves the reconstruction accuracy and the range resolution at all visible distances, at the cost of a reduction of the field of view;
- needs wider disparity search range, thus leading to an increased possibility of false matches.
- increases the angular field of view;
- induces higher distortion;
- increases (i.e., makes worse) the range resolution;
- produces images that are zoomed in farther, allowing for the detection of distant objects;
- makes the field of view narrower.
Different combinations of baselines and optics should be used in different operational conditions, e.g.:
- a narrow baseline configuration is useful in low-speed operations, where less noisy measurements are needed
- a wide baseline is suitable when the vehicle travels at higher speed, enabling it to perceive far away obstacles. In addition, the wide baseline can improve the quality of the stereo range data for distant terrain mapping
The use of a multi-baseline stereo frame allows one to get good results at a wide range of viewing distances, and to increase the overall flexibility and reliability of the system
Multi-baseline stereo for scene segmentation in natural environments
Fig. 1 Results of 3D reconstruction for two different scenes, obtained by using the XB3 system (a, c) and the Flea3 system (b, d) in the short range and in the long range, respectively.
Fig. 2 Results of classification for the test cases of Fig. 1: (a) results for Fig. 1 a; (b) results for Fig. 1 b ; (c) results for Fig. 1 c; (d) results for Fig. 1 d. Green points denote pixels classified as ground; red points denote pixels classified as non-ground.