[ICRA 2019]

Benchmark of Skill Learning from Demonstration: Impact of User Experience, Task Complexity, and Start Configuration on Performance

Abstract - Over the past decade, the robotics community has proposed numerous approaches for skill learning from demonstration. Given the number and diversity of existing techniques, it is critical that comprehensive empirical studies be performed comparing the relative strengths of these learning techniques. In this work, we contribute a large-scale study benchmarking the performance of four algorithms. To perform this evaluation, we collected data from 9 participants, across 4 different manipulation tasks with varying starting conditions. The resulting demonstrations were used to train 180 task models and evaluated on 720 task reproductions on a physical robot. Our results detail how i) complexity of the task, ii) the expertise of the human demonstrator, and iii) the starting configuration of the robot affect task performance.

[coming soon]

[CoRL 2017]

Towards Robust Skill Generalization: Unifying Learning from Demonstration and Motion Planning  

Abstract - In this paper, we present Combined Learning from demonstration And Motion Planning (CLAMP) as an efficient approach to skill learning and generalizable skill reproduction. CLAMP combines the strengths of Learning from Demonstration (LfD) and motion planning into a unifying framework. We carry out probabilistic inference to find trajectories which are optimal with respect to a given skill and also feasible in different scenarios. We use factor graph optimization to speed up inference. To encode optimality, we provide a new probabilistic skill model based on a stochastic dynamical system. This skill model requires minimal parameter tuning to learn, is suitable to encode skill constraints, and allows efficient inference. Preliminary experimental results showing skill generalization over initial robot state and unforeseen obstacles are presented. 

Published in: Conference on Robot Learning (CoRL 2017) and Proceedings of Machine Learning Research (PMLR), Nov, 2017 [PDF][Video]

Generalized Cylinders for Learning, Reproduction, Generalization, and Refinement of Robot Skills

Abstract - This paper presents a novel geometric approach for learning and reproducing trajectory-based skills from human demonstrations. Our approach models a skill as a Generalized Cylinder, a geometric representation composed of an arbitrary space curve called spine and a smoothly varying cross-section. While this model has been utilized to solve other robotics problems, this is the first application of Generalized Cylinders to manipulation. The strengths of our approach are the model’s ability to identify and extract the implicit characteristics of the demonstrated skill, support for multiple reproduction of trajectories that maintain those characteristics, generalization to new situations through nonrigid registration, and interactive human refinement of the resulting model through kinesthetic teaching. We validate our approach through several real-world experiments with a Jaco 6-DOF robotic arm.

Published in: Robotics: Science and Systems (RSS 2017), MIT in Cambridge, MA, USA, Jul. 12-16, 2017 [PDF][video]

Trajectory Learning from Demonstration with Canal Surfaces: A Parameter-free Approach

Abstract— We present a novel geometric framework for intuitively encoding and learning a wide range of trajectory-based skills from human demonstrations. Our approach identifies and extracts the main characteristics of the demonstrated skill, which are spatial correlations across different demonstrations. Using the extracted characteristics, the proposed approach generates a continuous representation of the skill based on the concept of canal surfaces. Canal surfaces are Euclidean surfaces formed as the envelope of a family of regular surfaces (e.g. spheres) whose centers lie on a space curve. The learned skill can be reproduced, as a time-independent trajectory, and generalized to unforeseen situations inside the canal while its main characteristics are preserved. The main advantages of the proposed approach include: (a) requiring no parameter tuning, (b) maintaining the main characteristics and implicit boundaries of the skill, and (c) generalizing the learned skill over the initial condition of the movement, while exploiting the whole demonstration space to reproduce a variety of successful movements. Evaluations using simulated and real-world data exemplify the feasibility and robustness of our approach.

Published in: IEEE-RAS International Conference on Humanoid Robotics (Humanoids 2016), Cancun, Mexico, Nov 15-17, 2016. [PDF]

Learning Symbolic Representations of Actions from Human Demonstrations

Abstract— In this paper, a robot learning approach is proposed which integrates Visuospatial Skill Learning, Imitation Learning, and conventional planning methods. In our approach, the sensorimotor skills (i.e., actions) are learned through a learning from demonstration strategy. The sequence of performed actions is learned through demonstrations using Visuospatial Skill Learning. A standard action-level planner is used to represent a symbolic description of the skill, which allows the system to represent the skill in a discrete, symbolic form. The Visuospatial Skill Learning module identifies the underlying constraints of the task and extracts symbolic predicates (i.e., action preconditions and effects), thereby updating the planner representation while the skills are being learned. Therefore the planner maintains a generalized representation of each skill as a reusable action, which can be planned and performed independently during the learning phase. Preliminary experimental results on the iCub robot are presented.

Published in: IEEE International Conference on Robotics and Automations (ICRA 2015), Seattle, Washington, USA, May 26-30, 2015. [PDF][bibtex][PDF][video]

Learning Reactive Robot Behavior for Autonomous Valve Turning

AbstractA learning approach is proposed for the challenging task of autonomous robotic valve turning in the presence of active disturbances and uncertainties. The valve turning task comprises two phases: reaching and turning. For the reaching phase the manipulator learns how to generate trajectories to reach or retract from the target. The learning is based on a set of trajectories demonstrated in advance by the operator. The turning phase is accomplished using a hybrid force/motion control strategy. Furthermore, a reactive decision making system is devised to react to the disturbances and uncertainties arising during the valve turning process. The reactive controller monitors the changes in force, movement of the arm with respect to the valve, and changes in the distance to the target. Observing the uncertainties, the reactive system modulates the valve turning task by changing the direction and rate of the movement. A real-world experiment with a robot manipulator mounted on a movable base is conducted to show the efficiency and validity of the proposed approach.

Published in: IEEE-RAS International Conference on Humanoid Robots (Humanoids 2014), Madrid, Spain, Nov 18-20. [PDF][bibtex]

Multi-Objective Reinforcement Learning for AUV Thruster Failure Recovery

Abstract—This paper investigates learning approaches for discovering fault-tolerant control policies to overcome thruster failures in Autonomous Underwater Vehicles (AUV). The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the vehicle. When a fault is detected and isolated the model of the AUV is reconfigured according to the new condition. To discover a set of optimal solutions a multi-objective reinforcement learning approach is employed which can deal with multiple conflicting objectives. Each optimal solution can be used to generate a trajectory that is able to navigate the AUV towards a specified target while satisfying multiple objectives. The discovered policies are executed on the robot in a closed-loop using AUV’s state feedback. Unlike most existing methods which disregard the faulty thruster, our approach can also deal with partially broken thrusters to increase the persistent autonomy of the AUV. In addition, the proposed approach is applicable when the AUV either becomes under-actuated or remains redundant in the presence of a fault. We validate the proposed approach on the model of the Girona500.

Published in: IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL 2014), Orlando, FL, Dec 9-12. [PDF][bibtex][PDF][IEEE]

Online Discovery of AUV Control Policies to Overcome Thruster Failures

Abstract— We investigate methods to improve fault-tolerance of Autonomous Underwater Vehicles (AUVs) to increase their reliability and persistent autonomy. We propose a learning-based approach that is able to discover new control policies to overcome thruster failures as they happen. The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the AUV. The model is adapted to a new condition when a fault is detected and isolated. Since the approach generates an optimal trajectory, the learned fault-tolerant policy is able to navigate the AUV towards a specified target with minimum cost. Finally, the learned policy is executed on the real robot in a closed-loop using the state feedback of the AUV. Unlike most existing methods which rely on the redundancy of thrusters, our approach is also applicable when the AUV becomes under-actuated in the presence of a fault. To validate the feasibility and efficiency of the presented approach, we evaluate it with three learning algorithms and three policy representations with increasing complexity. The proposed method is tested on a real AUV, Girona500. 

Published in: IEEE International Conference on Robotics and Automation (ICRA 2014), Hong Kong, China, May 31 - June 7, 2014. [PDF][bibtex][Video][IEEE]

Interactive Robot Learning of Visuospatial Skills

Abstract-- This paper proposes a novel interactive robot learning approach for acquiring visuospatial skills. It allows a robot to acquire new capabilities by observing a demonstration while interacting with a human caregiver. Most existing learning from demonstration approaches focus on the trajectories, whereas in our approach the focus is placed on achieving a desired goal configuration of objects relative to one another. Our approach is based on visual perception which captures the object’s context for each demonstrated action. The context embodies implicitly the visuospatial representation including the relative positioning of the object with respect to multiple other objects simultaneously. The proposed approach is capable of learning and generalizing different skills such as object reconfiguration, classification, and turn-taking interaction. The robot learns to achieve the goal from a single demonstration while requiring minimum a priori knowledge about the environment. We illustrate the capabilities of our approach using four real world experiments with a Barrett WAM robot.

*Visuospatial Skill Learning has been used by Microsoft Robotics [Paper][Video]

Published in Proceedings 16th IEEE International Conference on Advanced Robotics (ICAR 2013), Montevideo, Uruguay, 25-29 Nov 2013.  [PDF][bibtex][video][IEEE]

Online Direct Policy Search for Thruster Failure Recovery in Autonomous Underwater Vehicles

Abstract-- Autonomous underwater vehicles are prone to various factors that may lead a mission to fail and cause unrecoverable damages. Even robust controllers cannot make sure that the robot is able to navigate to a safe location in such situations. In this paper we propose an online learning method for reconfiguring the controller, which tries to recover the robot and survive the mission using the current asset of the system. The proposed method is framed in the reinforcement learning setting, and in particular as a model-based direct policy search approach. Since learning on a damaged vehicle would be impossible owing to time and energy constraints, learning is performed on a model which is identified and kept updated online. We evaluate the applicability of our method with different policy representations and learning algorithms, on the model of the Girona500 autonomous underwater vehicle.

Published in: 6th International workshop on Evolutionary and Reinforcement Learning for Autonomous Robot System (ERLARS 2013), Taormina, Italy, 2-9 Sept 2 2013 [PDF][bibtex]

On-line Learning to Recover from Thruster Failures on Autonomous Underwater Vehicles

Abstract—We propose a method for computing on-line the controller of an Autonomous Underwater Vehicle under thruster
failures. The method is general and can be applied to both redundant and under-actuated AUVs, as it does not rely on
the modification of the thruster control matrix. We define an optimization problem on a specific class of functions, in order to
compute the optimal control law that achieves the target without using the faulty thruster. The method is framed within model-based policy search for reinforcement learning, and we study its applicability on the model of the AUV Girona500. We performed
experiments with policies of increasing complexity, testing the on-line feasibility of the approach as the optimization problem
becomes more complex.

Published in: OCEANS 2013 MTS/IEEE, San Diego, USA, 23-26 Sept 2013. [PDF][bibtex][IEEE]

Visuospatial Skill Learning for Object Reconfiguration Tasks

Abstract-- We present a novel robot learning approach based on visual perception that allows a robot to acquire new skills by observing a demonstration from a tutor. Unlike most existing learning from demonstration approaches, where the focus is placed on the trajectories, in our approach the focus is on achieving a desired goal configuration of objects relative to one another. Our approach is based on visual perception which captures the object's context for each demonstrated action. This context is the basis of the visuospatial representation and encodes implicitly the relative positioning of the object with respect to multiple other objects simultaneously. The proposed approach is capable of learning and generalizing multi-operation skills from a single demonstration, while requiring minimum a priori knowledge about the environment. The learned skills comprise a sequence of operations that aim to achieve the desired goal configuration using the given objects. We illustrate the capabilities of our approach using three object reconfiguration tasks with a Barrett WAM robot.

*Visuospatial Skill Learning has been used by Microsoft Robotics [Paper][Video]

Published in: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), Tokyo, Japan, 3-8 Nov 2013. [PDF][bibtex][video][IEEE]

Autonomous Robotic Valve Turning: A Hierarchical Learning Approach

Abstract— Autonomous valve turning is an extremely challenging task for an Autonomous Underwater Vehicle (AUV). To resolve this challenge, this paper proposes a set of different computational techniques integrated in a three-layer hierarchical scheme. Each layer realizes specific subtasks to improve the persistent autonomy of the system. In the first layer, the robot acquires the motor skills of approaching and grasping the valve by kinesthetic teaching. A Reactive Fuzzy Decision Maker (RFDM) is devised in the second layer which reacts to the relative movement between the valve and the AUV, and alters the robot’s movement accordingly. Apprenticeship learning method, implemented in the third layer, performs tuning of the RFDM based on expert knowledge. Although the long-term goal is to perform the valve turning task on a real AUV, as a first step the proposed approach is tested in a laboratory environment.

Published in: IEEE International Conference on Robotics and Automation (ICRA 2013), Karlsruhe, Germany, May 6-10, 2013.
Autonomous Robotic Valve Turning- A Hierarchical Learning Approach

Modeling of Hyper-Redundant Manipulators Dynamics and Design of Fuzzy Controller for the System

Abstract-- Hyper-redundant manipulators having degrees of freedom much more than required, have many advantages and important capabilities. In this paper, modeling of the manipulator dynamics is done using a special curve called 'backbone' curve, modal method, Lagrangian mechanics and geometric transformation between variables space of backbone curve and manipulator joints. The dependency of the nonlinear and coupled terms of the dynamics model to joint variables makes some difficulties in classical methods to controller design. To overcome this problem, fuzzy controllers that have appropriate efficiency in complex and nonlinear systems are used. For demonstrating this matter, dynamics modeling of 10 degrees of freedom manipulator is done. Then a fuzzy controller is designed with attention to the dynamics behavior of the system. Manipulator behavior through various and noisy inputs are evaluated by simulation of the model including fuzzy controller. The results show very small error in manipulator motions and suitable condition of the designed fuzzy controller based on the dynamics model.

Published in: IEEE International Conference on Integration of Knowledge Intensive Multi-Agent Systems (KIMAS), Waltham, MA, USA, April 18-21, 2005. [PDF][bibtex][IEEE]

Subpages (3): Latest Publication pdfs RL