Human activity recognition plays a significant role in enabling pervasive applications as it abstracts low-level noisy sensor data into high-level human activities, which applications can respond to. With more and more activity-aware applications deployed in real-world environments, a research challenge emerges - discovering and learning new activities that have not been pre-defined or observed in the training phase. This paper tackles this challenge by proposing a hierarchical mixture of directional statistical models. The model supports incrementally, continuously updating the activity model over time with the reduced annotation effort and without the need for storing historical sensor data. We have validated this solution on four publicly available, third-party smart home datasets, and have demonstrated up to 91.5 % accuracies of detecting and recognising new activities.