AnyDBC: An Efficient Anytime Density-based Clustering Algorithm for Very Large Complex Datasets
Son Mai*, Aarhus University; Ira Assent, ; Martin Storgaard, Aarhus University
The density-based clustering algorithm DBSCAN is a state-of-the-art data clustering technique with numerous applications in many ﬁelds. However, its O(n2) time complexity still remains a severe weakness. In this paper, we propose a novel anytime approach to cope with this problem by reducing both the range query and the label propagation time of DBSCAN. Our algorithm, called AnyDBC, compresses the data into smaller density-connected subsets called primitive clusters and labels objects based on connected components of these primitive clusters for reducing the label propagation time. Moreover, instead of passively performing the range query for all objects like existing techniques, AnyDBC iteratively and actively learns the current cluster structure of the data and selects a few most promising objects for reﬁning clusters at each iteration. Thus, in the end, it performs substantially fewer range queries compared to DBSCAN while still guaranteeing the exact ﬁnal result of DBSCAN. Experiments show speedup factors of orders of magnitude com-pared to DBSCAN and its fastest variants on very large real and synthetic complex datasets.