Home > Published Issues > 2020 > Volume 9, No. 9, September 2020 >

Automatic Estimation of the Position and Orientation of Stairs to Be Reached and Climbed by a Disaster Response Robot by Analyzing 2D Image and 3D Point Cloud

Kazuya Miyakawa 1, Takuya Kanda 2, Jun Ohya 1, Hiroyuki Ogata 3, Kenji Hashimoto 4, and Atsuo Takanishi 1
1. Department of Modern Mechanical Engineering, Waseda University, Tokyo, Japan
2. Department of Integrative Bioscience and Biomedical Engineering, Waseda University, Tokyo, Japan
3. Department of System Design Engineering, Seikei Univeristy, Tokyo, Japan
4. Department of Mechanical Engineering Informatics, Meiji University, Kanagawa, Japan

Abstract—In order to realize a disaster response robot that can reach and climb straight stairs within a certain range, this paper proposes a method for estimating the position and orientation of the stairs using 2D image and 3D point cloud. In this method, first, an object detection method is applied to an RGB image, and a 3D point cloud including stairs is extracted by combining the detection result and the 3D point cloud. Next, a 3D point cloud of a step candidate is extracted by applying plane estimation and region segmentation to the extracted 3D point cloud. The 3D point cloud of the step candidate is projected on a 2D plane, and the orientation of the stairs is estimated by detecting their contour and lines. In addition, the position of the stairs is estimated by searching for a combination of 3D point clouds of the step candidates located at equal intervals using the structural characteristics of the stairs. As a result of simulation using a disaster response robot WAREC-1, it was confirmed that the orientation of the stairs can be accurately estimated by the proposed method. It was also confirmed that the position could be accurately estimated under specific conditions. 

Index Terms—disaster response robot, reaching stairs, climbing stairs, object detection, 3D point cloud processing

Cite: Kazuya Miyakawa, Takuya Kanda, Jun Ohya, Hiroyuki Ogata, Kenji Hashimoto, and Atsuo Takanishi, "Automatic Estimation of the Position and Orientation of Stairs to Be Reached and Climbed by a Disaster Response Robot by Analyzing 2D Image and 3D Point Cloud," International Journal of Mechanical Engineering and Robotics Research, Vol. 9, No. 9, pp. 1312-1321, September 2020. DOI: 10.18178/ijmerr.9.9.1312-1321