COMP5500
COMP5500: Robot Learning, UMass Lowell, Fall 2018
When: Tuesdays / Thursdays, 14:00-15:15
Where: 321 Dandeneau Hall, North Campus
Instructor: Reza Ahmadzadeh
Office: 313 Dandeneau Hall
Office hours: To be announced
Course Description
While many classical robotics problems such as path planning and tracking could be solved using existing algorithms that work successfully in controlled environments, in unstructured and dynamic environments with increasing level of uncertainty those approaches fail to produce robust solutions. Recent advances in machine learning have begun to address these challenging robotic problems by allowing robots to learn from their own actions and experiences, and also from interaction with humans.
This course will cover a variety of machine learning approaches that allow robots to learn from their own actions and experiences, and also through interaction with humans largely focusing on applications in manipulation. We will discuss techniques for learning both trajectory-based skills (low-level motor skills) and goal-based skills (high-level task-oriented skills). Topics will include methods from a) imitation learning, b) learning from demonstration (statistical, dynamical, and geometric representations), and c) Reinforcement Learning. We will discuss methods including, but not limited, to data gathering and pre-processing, skill encoding, reproduction, and generalization, skill refinement, obstacle avoidance, symbol grounding, symbolic planning, feature selection and segmentation, active learning, and human factor. The course includes student presentations and a final project where students develop an existing or their own robot learning technique.
Prior knowledge of fundamentals of linear algebra and probability is assumed. Familiarity with kinematics, dynamics, control, and statistical machine learning approaches is plus.
Course Objectives
Upon completion of this course, students will be able to:
describe and explain how robots learn from human teachers
describe and explain how robots learn by exploration
explain trajectory-based vs. goal-based skill learning approaches
describe imitation learning vs. Learning from Demonstration (LfD)
describe mathematically several representations and their reproduction methods
describe a few generalization methods and obstacle avoidance techniques
implement an existing robot learning algorithm
develop a robot learning algorithm from scratch through group projects
construct, program, and test the algorithm and compare it to the state-of-the-art
These goals will be evaluated through written assignments, presentation, discussions, and a final project.
Text Books:
There is no assigned textbook for this course, selected conference and journal articles will be assigned as reading throughout the course. Following references cover the machine learning techniques used in lectures and articles:
[1]. Robot Programming by Demonstration: A Probabilistic Approach, by S. Calinon, CRC Press, 2009.
[2]. Robot Learning from Human Teachers, by S. Chernova and A. Thomaz, M&C, 2014.
[3]. Reinforcement Learning: An Introduction, by R. S. Sutton and A. G. Barto, MIT Press, 2018.
[4]. Machine Learning – A Probabilistic Perspective: K. P. Murphy, MIT Press, 2012.
Targeted Students
This course is designed to be accessible to graduate students of all levels who have basic knowledge of a) linear algebra, b) calculus, and c) statistics. Fair knowledge of Python and/or MATLAB programming is required. This course draws upon techniques in machine learning including unsupervised learning, supervised learning, and reinforcement learning. While completion of a course dedicated to these topics (e.g. COMP 5450) would be helpful, this course will briefly review these topics.
Lectures
Students are expected to attend all classes; unexpected absence will affect your final grade. A significant aspect of the class will be group discussion and participation, thus it is essential that you carefully review the required reading before each class and be prepared to share your perspective. Presentations of the fundamental material for the course will be given by the instructor, the course staff, or invited lecturers. Paper presentations will be led by students. Each student will do one or two presentations depending on class size.
Project
Students are encouraged to propose and conduct a group research project (1-2 students) building upon the topics covered in this course. Students are encouraged to propose projects relevant to their own research to bring their own skill sets, creativity, and unique perspectives to the project. However, the course project must be independent of the student’s thesis, GRA/GTA, and other course projects. The goal of the project is to dive deep into an area of Robot Learning, learn how to evaluate and algorithm, and improve one’s ability to present and disseminate their work.
Assignments and Grading
The course grade breakdown is as follows:
Attendance/Participation: 20%
Report/Homework/Presentation: 40%
Final project: 40%
Extra Credit: up to 5%
No final exam!
The participation grade will be determined based on active participation in class discussions and presentation of reading summaries when assigned. Projects will be determined based on an individual basis in October and will include a written report, in-class presentation, and demo. You may earn extra credit throughout the semester through the following:
Contribute code of an algorithm with interesting, and/or useful simulation or actual robot implementation [2% of the total grade]. This is a contribution in addition to the students’ final project. The code must not be derived or extended from an existing or online source.
Preparing a course-related research proposal based on a new idea and presenting it to the class [2% of the total grade].
Complete the final survey at the end of class. Extra credit of 1% of the total grade if at least 85% of the class completes the survey.
*For the first two options, you are encouraged to coordinate with the instructor in person before implementing your ideas.
Policies
Student Disability Services
if you need course adaptations or accommodations because of a disability, or if you have medical information to share with the instructor, please make an appointment or stop by to speak with Dr. Reza Ahmadzadeh within the first week of classes.
Academic Honesty Policy
Students are expected to honor and follow all CS department and UMass Lowell policies related to academic honesty and integrity. Violators risk failing the course in addition to any actions taken by the university administration. Cheating will not be tolerated, and students who cheat risk failing the course and possible university administrative actions. Please make sure to review the UMass Lowell’s
Academic Dishonesty Policy.
The work on homework assignments must also be the student's own work, with the following exceptions: 1) hints provided by the TA or the instructor may be used, provided that after obtaining such hints the students perform the assignment on their own, and that having obtained hints is acknowledged in writing in the student's work; 2) forming study-groups is allowed (and encouraged) and students may engage in discussions related to homework assignments, provided that following such discussions students complete the homework assignment separately on their own (without referring/copying detailed notes from those discussions) and that occurrence of such discussions is acknowledged in writing on the homework assignment (however, doing a homework assignment together by more than one person is not permitted). Using homework solutions from any source, such as websites or past-year’s solutions obtained from any source, is not permitted.
Project work
The mandatory final-project (see graded work section above) is expected to be done as teamwork performed by self-organized teams of, generally, up to two students enrolled in this course. Accordingly, a collaborative work within these project teams on matters specific to their respective final project is permitted and required (all members of the project team must contribute). Please note, however, that this permission applies to the above final-project work only; it does not apply to any other gradable items (such as written assignments, presentation).
Rule of thumb
Any work you present as your own should represent your own understanding of the material. When external sources were used as significant points of information, the source must be referenced in your submission.
Violations may result in a warning, a possible grade of 0, and/or a report to the university.