Human activity recognition (HAR) systems will be increasingly deployed in real-world environments and for long periods of time. This significantly challenges current approaches to HAR, which have to account for changes in activity routines, the evolution of situations, and sensing technologies. Driven by these challenges, in this paper, we argue the need to move beyond learning to lifelong machine learning—with the ability to incrementally and continuously adapt to changes in the environment being learned. We introduce a conceptual framework for lifelong machine learning to structure various relevant proposals in the area and identify some key research challenges that remain.