Imitation and Reinforcement Learning for Motor Primitives with Perceptual Coupling

2010

Book Chapter

ei

Traditional motor primitive approaches deal largely with open-loop policies which can only deal with small perturbations. In this paper, we present a new type of motor primitive policies which serve as closed-loop policies together with an appropriate learning algorithm. Our new motor primitives are an augmented version version of the dynamical system-based motor primitives [Ijspeert et al(2002)Ijspeert, Nakanishi, and Schaal] that incorporates perceptual coupling to external variables. We show that these motor primitives can perform complex tasks such as Ball-in-a-Cup or Kendama task even with large variances in the initial conditions where a skilled human player would be challenged. We initialize the open-loop policies by imitation learning and the perceptual coupling with a handcrafted solution. We first improve the open-loop policies and subsequently the perceptual coupling using a novel reinforcement learning method which is particularly well-suited for dynamical system-based motor primitives.

@inbook{6234,
  title = {Imitation and Reinforcement Learning for Motor Primitives with Perceptual Coupling},
  author = {Kober, J. and Mohler, B. and Peters, J.},
  booktitle = {From Motor Learning to Interaction Learning in Robots},
  pages = {209-225},
  series = {Studies in Computational Intelligence ; 264},
  editors = {Sigaud, O. and Peters, J.},
  publisher = {Springer},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Berlin, Germany},
  month = jan,
  year = {2010},
  doi = {10.1007/978-3-642-05181-4_10},
  month_numeric = {1}
}