Time Constrained DL8.5 Using Limited Discrepancy Search

Abstract

Decision trees that minimize the error on the training set with a depth limit have been found to be generally superior to those found by more standard greedy algorithms. However, when the search space to be explored is too large, the depth-first search used by exact algorithms can get trapped in left most branches. Consequently, when the user stops the algorithm, the best tree found so far may be unbalanced and poorly minimize the error. Our work aims to improve the anytime behavior by introducing the limited discrepancy search ingredient in these algorithms. This allows to explore the search space by waves increasingly deviating from standard heuristics such as information gain. Our experimental results show that the anytime behavior of the state-of-the-art exact method DL8.5 is greatly improved.