
2021–2022 Maeda Lab, Yokohama National University
View-Based Teaching/Playback
We developed a teaching/playback method based on camera images for industrial manipulators
[1][2]. In this method, robot motions and scene images in human demonstrations are recorded to
obtain an image-to-motion mapping, and the mapping is used for playback (Fig. 6). It can achieve
more robustness against changes of task conditions than conventional joint-variable-based teach-
ing/playback. Our method adopts end-to-end learning through view-based image processing and
therefore neither object models nor camera calibration are necessary. We are improving our view-
based teaching/playback by using range images (Fig. 7) and occlusion-aware techniques for more
robustness [3]. For application to force-control tasks, visualization of force information based on
photoelasticity (Fig. 8) is under investigation [4]. We are also trying to integrate reinforcement
learning with the view-based teaching/playback for reduction of human operations for teaching
[5].
References
[1] Y. Maeda and T. Nakamura: View-based teaching/playback for robotic manipulation, ROBOMECH J.,
Vol. 2, 2, 2015.
[2] Y. Maeda and Y. Moriyama: View-Based Teaching/Playback for Industrial Manipulators, Proc. of 2011
IEEE Int. Conf. on Robotics and Automation (ICRA 2011), pp. 4306–4311, 2011.
[3] Y. Maeda and Y. Saito: Lighting- and Occlusion-robust View-based Teaching/Playback for Model-free
Robot Programming, W. Chen et al. eds., Intelligent Autonomous Systems 14, pp. 939–952, Springer,
2017.
[4] Y. Nakagawa, Y. Maeda and S. Ishii: View-Based Teaching/Playback with Photoelasticity for Force-
Control Tasks, W. Chen et al. eds., Intelligent Autonomous Systems 14, pp. 825–837, Springer, 2017.
[5] Y. Maeda and R. Aburata: Teaching and Reinforcement Learning of Robotic View-Based Manipulation,
Proc. of 22nd IEEE Int. Symp. on Robot and Human Interactive Communication (RO-MAN 2013), pp.
87–92, 2013.
camera
robot
object
human instruction
(a) human teaching
(b) image-to-motion mapping
(c) view-based playback
Fig. 6 Outline of View-Based Teaching/Playback
0 100 200 300 400 500
frame number
Grayscale
Images
Light
OFF
Range
Images
Fig. 7 Switching between Grayscale and Range
Images for View-Based Teaching/Playback
Fig. 8 View-based Teaching/
Playback with Photoelasticity
4