ISPIL: Interactive Sub-Goal-Planning Imitation Learning for Long-Horizon Tasks With Diverse Goals

Abstract

Imitation Learning (IL) is a promising approach for teaching tasks to robots by human demonstrations, although it faces challenges from long-horizon tasks and diverse goals in real-world settings. These issues stem from (i) a distribution mismatch between demonstrations and real-world execution and (ii) existing policy models that typically focus on prelearned final goals, limiting efficiency with diverse goals. To address this situation, we propose Interactive Sub-Goal-Planning Imitation Learning (ISPIL), an IL framework that learns hierarchical, goal-conditioned policies. Specifically, a high-level policy sets reachable sub-goals for the final goals, and a low-level policy executes the required actions. ISPIL interactively collects two types of demonstration data based on the novelty criteria: meta-sub-goal data, which represent with symbols the causal relationships between sub-goals, and action data, which consist of the physical robotic actions required to achieve these sub-goals. Meta-sub-goal data enable effective planning using a Regression Planning Network (RPN), and a sub-goal switching function helps reduce unnecessary data queries at the high level. We validate ISPIL through simulations and real-robot experiments in a kitchen-like environment and demonstrate improved task execution and generalizability across diverse goals.