Abstract—We propose in this work an original approach consisting in correcting inertial human hand trajectory with vision-based object tracking in a context of programming by demonstration (PbD) of pick-and-place tasks. One challenge in PbD is to record human demonstrations accurately enough, in an easy way which does not limit human motion. Merging inertial-based and vision-based technologies may take advantage of both and fulfill the requirement of a PbD process. Our method is based on the identification of Positions of Interest (POIs) from object and hand data, corresponding to picking or placing actions. Then objects POIs are paired with hand POIs to modify the human hand trajectory. The method is implemented on a Sawyer robot with Xsens IMU sensors. Pick-and-place tasks with different complexity have been recorded and reproduced by the robot. The robot succeeds to reproduce the demonstrated tasks which validates our method.

Index Terms—programming by demonstration, vision, inertial human motion tracking, merging

Cite: Robin Pellois and Olivier Brüls, "A Vision-based Correction of Inertial Measurement of Human Motion for Robot Programming by Demonstration," International Journal of Mechanical Engineering and Robotics Research, Vol. 11, No. 6, pp. 411-416, June 2022. DOI: 10.18178/ijmerr.11.6.411-416 

Copyright © 2022 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.