Header logo is ei

Real-time Perception meets Reactive Motion Generation




We address the challenging problem of robotic grasping and manipulation in the presence of uncertainty. This uncertainty is due to noisy sensing, inaccurate models and hard-to-predict environment dynamics. Our approach emphasizes the importance of continuous, real-time perception and its tight integration with reactive motion generation methods. We present a fully integrated system where real-time object and robot tracking as well as ambient world modeling provides the necessary input to feedback controllers and continuous motion optimizers. Specifically, they provide attractive and repulsive potentials based on which the controllers and motion optimizer can online compute movement policies at different time intervals. We extensively evaluate the proposed system on a real robotic platform in four scenarios that exhibit either challenging workspace geometry or a dynamic environment. We compare the proposed integrated system with a more traditional sense-plan-act approach that is still widely used. In 333 experiments, we show the robustness and accuracy of the proposed system.

Award: (Best Systems Paper Finalists - Amazon Robotics Best Paper Awards in Manipulation)
Author(s): Daniel Kappler and Franziska Meier and Jan Issac and Jim Mainprice and Garcia Cifuentes, Cristina and Manuel Wüthrich and Vincent Berenz and Stefan Schaal and Nathan Ratliff and Jeannette Bohg
Journal: IEEE Robotics and Automation Letters
Volume: 3
Number (issue): 3
Pages: 1864-1871
Year: 2018
Month: July

Department(s): Autonomous Motion
Research Project(s): Real-Time Perception meets Reactive Motion Generation
Bibtex Type: Article (article)

Award Paper: Best Systems Paper Finalists - Amazon Robotics Best Paper Awards in Manipulation
DOI: 10.1109/LRA.2018.2795645
State: Published
URL: https://arxiv.org/abs/1703.03512

Links: arxiv


  title = {Real-time Perception meets Reactive Motion Generation},
  author = {Kappler, Daniel and Meier, Franziska and Issac, Jan and Mainprice, Jim and Garcia Cifuentes, Cristina and W{\"u}thrich, Manuel and Berenz, Vincent and Schaal, Stefan and Ratliff, Nathan and Bohg, Jeannette},
  journal = {IEEE Robotics and Automation Letters},
  volume = {3},
  number = {3},
  pages = {1864-1871},
  month = jul,
  year = {2018},
  url = {https://arxiv.org/abs/1703.03512},
  month_numeric = {7}