State Representation Learning from Demonstration

Image credit: SRLfD

Abstract

Robots could learn their own state and world representation from perception and experience without supervision. This desirable goal is the main focus of our field of interest, state representation learning (SRL). Indeed, a compact representation of such a state is beneficial to help robots grasp onto their environment for interacting. The properties of this representation have a strong impact on the adaptive capability of the agent. In this article we present an approach based on imitation learning. The idea is to train several policies that share the same representation to reproduce various demonstrations. To do so, we use a multi-head neural network with a shared state representation feeding a task-specific agent. If the imitation tasks are diverse, the trained representation will eventually contain the information necessary for all tasks, while discarding irrelevant information. As such, it will potentially become a compact state representation useful for new tasks. We call this approach SRLfD (State Representation Learning from Demonstration). Our experiments confirm that when a controller takes SRLfD-based representations as input, it can achieve better performance than with other representation strategies and promote more efficient reinforcement learning (RL) than with an end-to-end RL strategy.

Publication
In Machine Learning, Optimization, and Data Science (LOD), 2020

Presentation of SRLfD (State Representation Learning from Demonstration)

Astrid Merckling
Astrid Merckling
Doctor in Machine Learning and Robotics

Using machine learning for giving unsupervised exploratory skills to robots.